2024-07-04T07:45:39.740 INFO:root:teuthology version: 0.0.1.dev347+ga4a89a3 2024-07-04T07:45:39.741 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} --archive /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821 -- /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/orig.config.yaml 2024-07-04T07:45:39.802 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-04T07:45:39.891 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821 branch: wip-pdonnell-testing-20240703.143006-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7786821' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240703.143006-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 2e9f68b5506c607b2f7d69d380cb7887cf00dc84 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 2e9f68b5506c607b2f7d69d380cb7887cf00dc84 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240703.143006-debug sha1: 2e9f68b5506c607b2f7d69d380cb7887cf00dc84 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3860 sha1: 2e9f68b5506c607b2f7d69d380cb7887cf00dc84 sleep_before_teardown: 0 subset: 462/512 suite: fs suite_branch: wip-pdonnell-testing-20240703.143006-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2e9f68b5506c607b2f7d69d380cb7887cf00dc84/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2e9f68b5506c607b2f7d69d380cb7887cf00dc84 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCXyyZtJ0Z2L74cdSy9M3YbnbAj6XlyQ1lSYXZOtIQgUyikFigh9FwjvXB3Aqs/LgJwlERMB1KMuFaNl5OsAhsU= smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJO9L+b2oazjfHiy3EgZCOt7h3UKsU/RUDeMN3dDF5QBRQp1tp5MkhFpUWtfe7xfcu2CJ2tExIfr/wkltzTibR0= smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0GdUBoRPosQUFmPzc1ocEze0US5xAAI0cA0nVzk8InjGG0xmWtefYdR7zO/LpOc5uKMy+30LEzuCMZmDh726E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: a4a89a3d72be587ba6bd8509090508e8a19e74c5 timestamp: 2024-07-04_01:31:57 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698093 2024-07-04T07:45:39.892 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2e9f68b5506c607b2f7d69d380cb7887cf00dc84/qa; will attempt to use it 2024-07-04T07:45:39.892 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2e9f68b5506c607b2f7d69d380cb7887cf00dc84/qa/tasks 2024-07-04T07:45:39.892 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-04T07:45:39.894 INFO:teuthology.task.internal:Checking packages... 2024-07-04T07:45:39.915 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2e9f68b5506c607b2f7d69d380cb7887cf00dc84' 2024-07-04T07:45:39.915 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-04T07:45:39.915 INFO:teuthology.packaging:ref: None 2024-07-04T07:45:39.916 INFO:teuthology.packaging:tag: None 2024-07-04T07:45:39.916 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240703.143006-debug 2024-07-04T07:45:39.916 INFO:teuthology.packaging:sha1: 2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:45:39.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240703.143006-debug 2024-07-04T07:45:40.136 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-4737.g2e9f68b5 2024-07-04T07:45:40.138 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-04T07:45:40.144 INFO:teuthology.task.internal:no buildpackages task found 2024-07-04T07:45:40.144 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-04T07:45:40.194 INFO:teuthology.task.internal:Saving configuration 2024-07-04T07:45:40.214 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-04T07:45:40.220 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-04T07:45:40.246 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-04 07:39:57.185979', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCXyyZtJ0Z2L74cdSy9M3YbnbAj6XlyQ1lSYXZOtIQgUyikFigh9FwjvXB3Aqs/LgJwlERMB1KMuFaNl5OsAhsU='} 2024-07-04T07:45:40.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-04 07:39:57.182769', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJO9L+b2oazjfHiy3EgZCOt7h3UKsU/RUDeMN3dDF5QBRQp1tp5MkhFpUWtfe7xfcu2CJ2tExIfr/wkltzTibR0='} 2024-07-04T07:45:40.292 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-04 07:39:57.184206', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0GdUBoRPosQUFmPzc1ocEze0US5xAAI0cA0nVzk8InjGG0xmWtefYdR7zO/LpOc5uKMy+30LEzuCMZmDh726E='} 2024-07-04T07:45:40.292 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-04T07:45:40.298 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-07-04T07:45:40.298 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-07-04T07:45:40.298 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-07-04T07:45:40.298 INFO:teuthology.run_tasks:Running task console_log... 2024-07-04T07:45:40.390 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd0ff5b7ac0>, signals=[15]) 2024-07-04T07:45:40.390 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-04T07:45:40.396 INFO:teuthology.task.internal:Opening connections... 2024-07-04T07:45:40.396 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T07:45:40.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:45:40.473 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-07-04T07:45:40.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:45:40.547 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-07-04T07:45:40.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:45:40.623 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-04T07:45:40.629 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-07-04T07:45:40.648 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-07-04T07:45:40.648 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-07-04T07:45:40.706 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-07-04T07:45:40.706 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-07-04T07:45:40.706 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-07-04T07:45:40.706 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-04T07:45:40.707 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-04T07:45:40.708 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-07-04T07:45:40.743 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-07-04T07:45:40.762 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-07-04T07:45:40.762 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-07-04T07:45:40.820 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-07-04T07:45:40.820 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-07-04T07:45:40.820 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-04T07:45:40.821 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-04T07:45:40.822 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-07-04T07:45:40.843 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-07-04T07:45:40.861 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-07-04T07:45:40.861 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-07-04T07:45:40.918 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-07-04T07:45:40.918 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-07-04T07:45:40.918 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-07-04T07:45:40.918 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-04T07:45:40.919 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-04T07:45:40.919 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-07-04T07:45:40.943 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-04T07:45:40.954 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-04T07:45:40.992 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-04T07:45:40.992 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-07-04T07:45:40.994 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-07-04T07:45:40.997 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-07-04T07:45:41.014 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-04T07:45:41.019 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-04T07:45:41.020 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-07-04T07:45:41.052 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-07-04T07:45:41.055 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-07-04T07:45:41.124 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-04T07:45:41.201 INFO:teuthology.run_tasks:Running task kernel... 2024-07-04T07:45:41.216 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-07-04T07:45:41.216 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-07-04T07:45:41.216 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-07-04T07:45:41.216 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-04T07:45:41.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T07:45:41.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-07-04T07:45:41.402 DEBUG:teuthology.task.kernel:sha1 for client.0 is 0d32a7bba447dd2470f4de43560317375ca9e9c8 2024-07-04T07:45:41.402 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "0d32a7bba447dd2470f4de43560317375ca9e9c8"... 2024-07-04T07:45:41.402 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-07-04T07:45:41.420 INFO:teuthology.orchestra.run.smithi039.stdout:5.14.0-437.el9.x86_64 2024-07-04T07:45:41.420 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 0d32a7bba447dd2470f4de43560317375ca9e9c8 2024-07-04T07:45:41.421 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-07-04T07:45:41.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum upgrade -y linux-firmware 2024-07-04T07:45:44.397 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.2 MB 00:01 2024-07-04T07:45:51.689 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 20 MB 00:06 2024-07-04T07:45:56.214 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.4 MB 00:00 2024-07-04T07:45:57.673 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 17 kB 00:00 2024-07-04T07:45:58.710 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 25 MB/s | 22 MB 00:00 2024-07-04T07:46:04.233 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-07-04T07:46:06.548 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-07-04T07:46:06.553 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-07-04T07:46:06.553 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-07-04T07:46:06.553 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-07-04T07:46:06.553 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-07-04T07:46:06.553 INFO:teuthology.orchestra.run.smithi039.stdout: linux-firmware noarch 20240603-144.el9 baseos 380 M 2024-07-04T07:46:06.554 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:46:06.554 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-07-04T07:46:06.554 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-07-04T07:46:06.554 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 1 Package 2024-07-04T07:46:06.554 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:46:06.555 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 380 M 2024-07-04T07:46:06.555 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-07-04T07:46:15.242 INFO:teuthology.orchestra.run.smithi039.stdout:linux-firmware-20240603-144.el9.noarch.rpm 44 MB/s | 380 MB 00:08 2024-07-04T07:46:15.242 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-07-04T07:46:15.243 INFO:teuthology.orchestra.run.smithi039.stdout:Total 44 MB/s | 380 MB 00:08 2024-07-04T07:46:18.452 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-07-04T07:46:18.524 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-07-04T07:46:18.524 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-07-04T07:46:21.889 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-07-04T07:46:21.890 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-07-04T07:46:30.925 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-07-04T07:46:31.738 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : linux-firmware-20240603-144.el9.noarch 1/2 2024-07-04T07:46:31.826 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-04T07:46:32.561 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-07-04T07:46:32.561 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : linux-firmware-20240603-144.el9.noarch 1/2 2024-07-04T07:46:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-07-04T07:46:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:46:33.306 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-07-04T07:46:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: linux-firmware-20240603-144.el9.noarch 2024-07-04T07:46:33.306 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:46:33.306 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-07-04T07:46:33.602 INFO:teuthology.task.kernel:Downloading kernel 0d32a7bba447dd2470f4de43560317375ca9e9c8 on client.0... 2024-07-04T07:46:33.602 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=0d32a7bba447dd2470f4de43560317375ca9e9c8 2024-07-04T07:46:33.839 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/0d32a7bba447dd2470f4de43560317375ca9e9c8/centos/9/flavors/default/x86_64/ 2024-07-04T07:46:33.840 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_g0d32a7bba447-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/0d32a7bba447dd2470f4de43560317375ca9e9c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-07-04T07:46:33.843 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi039.front.sepia.ceph.com to complete... 2024-07-04T07:46:40.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04 07:46:40 URL:https://1.chacra.ceph.com/r/kernel/testing/0d32a7bba447dd2470f4de43560317375ca9e9c8/centos/9/flavors/default/x86_64/kernel-6.9.0_g0d32a7bba447-1.x86_64.rpm [356888676/356888676] -> "/tmp/kernel.x86_64.rpm" [1] 2024-07-04T07:46:40.199 INFO:teuthology.orchestra.run.smithi039.stderr:FINISHED --2024-07-04 07:46:40-- 2024-07-04T07:46:40.199 INFO:teuthology.orchestra.run.smithi039.stderr:Total wall clock time: 6.3s 2024-07-04T07:46:40.199 INFO:teuthology.orchestra.run.smithi039.stderr:Downloaded: 1 files, 340M in 6.1s (56.2 MB/s) 2024-07-04T07:46:40.202 INFO:teuthology.task.kernel:Installing kernel 0d32a7bba447dd2470f4de43560317375ca9e9c8 on client.0... 2024-07-04T07:46:40.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-07-04T07:46:42.351 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying... ######################################## 2024-07-04T07:46:42.397 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing... ######################################## 2024-07-04T07:46:42.450 INFO:teuthology.orchestra.run.smithi039.stdout:Updating / installing... 2024-07-04T07:46:50.742 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-6.9.0_g0d32a7bba447-1 ######################################## 2024-07-04T07:47:31.293 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi039.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-07-04T07:47:31.293 DEBUG:teuthology.orchestra.run.smithi039:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-07-04T07:47:31.373 INFO:teuthology.orchestra.run.smithi039.stdout:/boot/System.map-6.9.0-g0d32a7bba447 2024-07-04T07:47:31.373 INFO:teuthology.orchestra.run.smithi039.stdout:/boot/config-6.9.0-g0d32a7bba447 2024-07-04T07:47:31.373 INFO:teuthology.orchestra.run.smithi039.stdout:/boot/vmlinuz-6.9.0-g0d32a7bba447 2024-07-04T07:47:31.373 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/System.map 2024-07-04T07:47:31.373 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/config 2024-07-04T07:47:31.373 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel 2024-07-04T07:47:31.373 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/crc32c-intel.ko 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-07-04T07:47:31.374 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-07-04T07:47:31.375 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-07-04T07:47:31.375 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-07-04T07:47:31.375 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/serpent-avx2.ko 2024-07-04T07:47:31.375 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-07-04T07:47:31.375 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-07-04T07:47:31.375 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-07-04T07:47:31.375 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-07-04T07:47:31.376 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-07-04T07:47:31.376 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-07-04T07:47:31.376 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-07-04T07:47:31.376 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kernel 2024-07-04T07:47:31.376 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kernel/cpu 2024-07-04T07:47:31.376 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kernel/cpu/mce 2024-07-04T07:47:31.376 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-07-04T07:47:31.376 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kvm 2024-07-04T07:47:31.376 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kvm/kvm-amd.ko 2024-07-04T07:47:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kvm/kvm-intel.ko 2024-07-04T07:47:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/arch/x86/kvm/kvm.ko 2024-07-04T07:47:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto 2024-07-04T07:47:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/ansi_cprng.ko 2024-07-04T07:47:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/anubis.ko 2024-07-04T07:47:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/arc4.ko 2024-07-04T07:47:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx 2024-07-04T07:47:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_memcpy.ko 2024-07-04T07:47:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_pq.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_raid6_recov.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_tx.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/async_xor.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/async_tx/raid6test.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/authenc.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/authencesn.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/blake2b_generic.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/blowfish_common.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/blowfish_generic.ko 2024-07-04T07:47:31.378 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/camellia_generic.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cast5_generic.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cast6_generic.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cast_common.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/ccm.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cmac.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/crc32_generic.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/crypto_engine.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/crypto_user.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/cts.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/des_generic.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/dh_generic.ko 2024-07-04T07:47:31.379 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/echainiv.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/essiv.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/fcrypt.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/khazad.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/lz4.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/lz4hc.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/md4.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/michael_mic.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/pcbc.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/pcrypt.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/rmd160.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/seed.ko 2024-07-04T07:47:31.380 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/serpent_generic.ko 2024-07-04T07:47:31.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/tcrypt.ko 2024-07-04T07:47:31.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/tea.ko 2024-07-04T07:47:31.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/twofish_common.ko 2024-07-04T07:47:31.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/twofish_generic.ko 2024-07-04T07:47:31.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/vmac.ko 2024-07-04T07:47:31.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/wp512.ko 2024-07-04T07:47:31.381 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/xcbc.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/xor.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/crypto/xxhash_generic.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/acpi_ipmi.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/acpi_pad.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/ec_sys.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/platform_profile.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/sbs.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/sbshc.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/acpi/video.ko 2024-07-04T07:47:31.382 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/acard-ahci.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/ahci_platform.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/ata_generic.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/libahci_platform.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_acpi.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_ali.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_amd.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_artop.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_atiixp.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_atp867x.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_cmd640.ko 2024-07-04T07:47:31.383 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_cmd64x.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_cypress.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_efar.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_hpt366.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_hpt37x.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_hpt3x2n.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_hpt3x3.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_it8213.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_it821x.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_jmicron.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_marvell.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_mpiix.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_netcell.ko 2024-07-04T07:47:31.384 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_ninja32.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_ns87410.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_ns87415.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_oldpiix.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_opti.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_optidma.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_pcmcia.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_pdc2027x.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_piccolo.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_rdc.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_sch.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_serverworks.ko 2024-07-04T07:47:31.385 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_sil680.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_sis.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_sl82c105.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_triflex.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pata_via.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/pdc_adma.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_inic162x.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_mv.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_nv.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_promise.ko 2024-07-04T07:47:31.386 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_qstor.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_sil.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_sil24.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_sis.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_svw.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_sx4.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_uli.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_via.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ata/sata_vsc.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/auxdisplay 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/auxdisplay/ks0108.ko 2024-07-04T07:47:31.387 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/base 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/base/regmap 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/base/regmap/regmap-i2c.ko 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/base/regmap/regmap-mmio.ko 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/bcma 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/bcma/bcma.ko 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/aoe 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/aoe/aoe.ko 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/brd.ko 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/drbd 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/drbd/drbd.ko 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/floppy.ko 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/loop.ko 2024-07-04T07:47:31.388 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/mtip32xx 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/nbd.ko 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/null_blk 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/null_blk/null_blk.ko 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/pktcdvd.ko 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/rbd.ko 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/virtio_blk.ko 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/xen-blkback 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/xen-blkfront.ko 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/zram 2024-07-04T07:47:31.389 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/block/zram/zram.ko 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hangcheck-timer.ko 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/amd-rng.ko 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/intel-rng.ko 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/via-rng.ko 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/hw_random/virtio-rng.ko 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-07-04T07:47:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_si.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/lp.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/mwave 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/mwave/mwave.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/ppdev.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tlclk.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_atmel.ko 2024-07-04T07:47:31.391 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_infineon.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_nsc.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_tis.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/uv_mmtimer.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/char/virtio_console.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/p4-clockmod.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/powernow-k8.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/cpufreq/speedstep-lib.ko 2024-07-04T07:47:31.392 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/ccp 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/ccp/ccp.ko 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat/qat_common 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/padlock-aes.ko 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/padlock-sha.ko 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/virtio 2024-07-04T07:47:31.393 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dca 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dca/dca.ko 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/devfreq 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma/dw 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma/dw/dw_dmac.ko 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma/ioat 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/dma/ioat/ioatdma.ko 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/amd64_edac.ko 2024-07-04T07:47:31.394 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/e752x_edac.ko 2024-07-04T07:47:31.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/edac_mce_amd.ko 2024-07-04T07:47:31.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i3000_edac.ko 2024-07-04T07:47:31.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i3200_edac.ko 2024-07-04T07:47:31.395 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i5100_edac.ko 2024-07-04T07:47:31.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i5400_edac.ko 2024-07-04T07:47:31.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i7300_edac.ko 2024-07-04T07:47:31.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i7core_edac.ko 2024-07-04T07:47:31.397 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/i82975x_edac.ko 2024-07-04T07:47:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/ie31200_edac.ko 2024-07-04T07:47:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/sb_edac.ko 2024-07-04T07:47:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/edac/x38_edac.ko 2024-07-04T07:47:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/firmware 2024-07-04T07:47:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/firmware/edd.ko 2024-07-04T07:47:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/firmware/iscsi_ibft.ko 2024-07-04T07:47:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/gpio 2024-07-04T07:47:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/gpio/gpio-ich.ko 2024-07-04T07:47:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/gpio/gpio-viperboard.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-appleir.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-aureal.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-axff.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-dr.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-elecom.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-elo.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-emsff.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-gaff.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-gt683r.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-gyration.ko 2024-07-04T07:47:31.399 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-holtek-kbd.ko 2024-07-04T07:47:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-holtek-mouse.ko 2024-07-04T07:47:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-holtekff.ko 2024-07-04T07:47:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-hyperv.ko 2024-07-04T07:47:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-icade.ko 2024-07-04T07:47:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-keytouch.ko 2024-07-04T07:47:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-kye.ko 2024-07-04T07:47:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-lcpower.ko 2024-07-04T07:47:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-led.ko 2024-07-04T07:47:31.400 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-lenovo.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-logitech-dj.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-multitouch.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-ortek.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-penmount.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-petalynx.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-picolcd.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-pl.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-plantronics.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-primax.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-rmi.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-arvo.ko 2024-07-04T07:47:31.401 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-common.ko 2024-07-04T07:47:31.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-isku.ko 2024-07-04T07:47:31.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-kone.ko 2024-07-04T07:47:31.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-07-04T07:47:31.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-konepure.ko 2024-07-04T07:47:31.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-07-04T07:47:31.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-lua.ko 2024-07-04T07:47:31.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-pyra.ko 2024-07-04T07:47:31.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-ryos.ko 2024-07-04T07:47:31.402 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat-savu.ko 2024-07-04T07:47:31.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-roccat.ko 2024-07-04T07:47:31.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-saitek.ko 2024-07-04T07:47:31.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-samsung.ko 2024-07-04T07:47:31.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-sensor-hub.ko 2024-07-04T07:47:31.403 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-sjoy.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-sony.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-speedlink.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-steelseries.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-sunplus.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-thrustmaster.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-tivo.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-tmff.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-topseed.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-twinhan.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-uclogic.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-waltop.ko 2024-07-04T07:47:31.404 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-wiimote.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-xinmo.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-zpff.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/hid-zydacron.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/uhid.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hid/wacom.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hv 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hv/hv_balloon.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hv/hv_utils.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hv/hv_vmbus.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/abituguru.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/abituguru3.ko 2024-07-04T07:47:31.405 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/acpi_power_meter.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ad7414.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ad7418.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adc128d818.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm1025.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm1026.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm1029.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm1031.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adm9240.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ads7828.ko 2024-07-04T07:47:31.406 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7410.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7411.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7462.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7470.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7475.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/adt7x10.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/amc6821.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/applesmc.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/asb100.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/asc7621.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/asus_atk0110.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/atxp1.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/coretemp.ko 2024-07-04T07:47:31.407 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/dme1737.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ds1621.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ds620.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/emc1403.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/emc6w201.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/f71805f.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/f71882fg.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/f75375s.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/fam15h_power.ko 2024-07-04T07:47:31.408 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/fschmd.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/g760a.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/g762.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/gl518sm.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/gl520sm.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/hwmon-vid.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/i5500_temp.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/i5k_amb.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ibmaem.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ibmpex.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ina209.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ina2xx.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/it87.ko 2024-07-04T07:47:31.409 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/k10temp.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/k8temp.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lineage-pem.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm63.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm73.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm75.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm77.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm78.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm80.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm83.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm85.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm87.ko 2024-07-04T07:47:31.410 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm90.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm92.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm93.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm95234.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm95241.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/lm95245.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc2945.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4151.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4215.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4222.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4245.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4260.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/ltc4261.ko 2024-07-04T07:47:31.411 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max16065.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max1619.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max1668.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max197.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max6639.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max6650.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/max6697.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/mcp3021.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/nct6683.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/nct6775-core.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/nct6775.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/nct7802.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pc87360.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pc87427.ko 2024-07-04T07:47:31.412 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pcf8591.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/max16064.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/max34440.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/max8688.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-07-04T07:47:31.413 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/powr1220.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sch5627.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sch5636.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sch56xx-common.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sht15.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sht21.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/shtc1.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/sis5595.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/smsc47b397.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/smsc47m1.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/smsc47m192.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/thmc50.ko 2024-07-04T07:47:31.414 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/tmp102.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/tmp103.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/tmp401.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/tmp421.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/via-cputemp.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/via686a.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/vt1211.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/vt8231.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83627ehf.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83627hf.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83781d.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83791d.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83792d.ko 2024-07-04T07:47:31.415 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83793.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83795.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83l785ts.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/hwmon/w83l786ng.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/algos 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-07-04T07:47:31.416 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-i801.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-isch.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-parport.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-07-04T07:47:31.417 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-via.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-dev.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-mux.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-smbus.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/i2c/i2c-stub.ko 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband 2024-07-04T07:47:31.418 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/ib_cm.ko 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/ib_core.ko 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/ib_umad.ko 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/iw_cm.ko 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/rdma_cm.ko 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/cxgb4 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mlx4 2024-07-04T07:47:31.419 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mlx5 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mthca 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/ipoib 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/iser 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/isert 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/srp 2024-07-04T07:47:31.420 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-07-04T07:47:31.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/srpt 2024-07-04T07:47:31.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-07-04T07:47:31.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input 2024-07-04T07:47:31.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/gameport 2024-07-04T07:47:31.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-07-04T07:47:31.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/gameport/fm801-gp.ko 2024-07-04T07:47:31.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/gameport/gameport.ko 2024-07-04T07:47:31.421 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/joydev.ko 2024-07-04T07:47:31.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/keyboard 2024-07-04T07:47:31.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/keyboard/gpio_keys.ko 2024-07-04T07:47:31.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse 2024-07-04T07:47:31.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/appletouch.ko 2024-07-04T07:47:31.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/bcm5974.ko 2024-07-04T07:47:31.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/cyapatp.ko 2024-07-04T07:47:31.422 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/elan_i2c.ko 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/sermouse.ko 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/synaptics_usb.ko 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/mouse/vsxxxaa.ko 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/rmi4 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/rmi4/rmi_core.ko 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio/altera_ps2.ko 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio/arc_ps2.ko 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-07-04T07:47:31.423 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/serio/serio_raw.ko 2024-07-04T07:47:31.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/input/sparse-keymap.ko 2024-07-04T07:47:31.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds 2024-07-04T07:47:31.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/leds-blinkm.ko 2024-07-04T07:47:31.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/leds-lm3530.ko 2024-07-04T07:47:31.424 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/leds-lp3944.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/leds-ss4200.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-07-04T07:47:31.425 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md 2024-07-04T07:47:31.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/bcache 2024-07-04T07:47:31.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/bcache/bcache.ko 2024-07-04T07:47:31.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-bio-prison.ko 2024-07-04T07:47:31.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-cache-smq.ko 2024-07-04T07:47:31.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-cache.ko 2024-07-04T07:47:31.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-crypt.ko 2024-07-04T07:47:31.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-delay.ko 2024-07-04T07:47:31.426 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-flakey.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-log-userspace.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-multipath.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-queue-length.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-raid.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-round-robin.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-service-time.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-switch.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-thin-pool.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/dm-verity.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/persistent-data 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/raid0.ko 2024-07-04T07:47:31.427 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/raid1.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/raid10.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/md/raid456.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptbase.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptctl.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptfc.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptlan.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptsas.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptscsih.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/message/fusion/mptspi.ko 2024-07-04T07:47:31.428 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/lpc_ich.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/lpc_sch.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/mfd-core.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/sm501.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/viperboard.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/vx855.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/mfd/wl1273-core.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/altera-stapl 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/apds9802als.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/apds990x.ko 2024-07-04T07:47:31.429 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/bh1770glc.ko 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/cb710 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/cb710/cb710.ko 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/eeprom 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/eeprom/at24.ko 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/eeprom/max6875.ko 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/enclosure.ko 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/hpilo.ko 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/isl29003.ko 2024-07-04T07:47:31.430 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/isl29020.ko 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/lis3lv02d 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/mei 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/mei/mei-me.ko 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/mei/mei-txe.ko 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/mei/mei.ko 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-gru 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-gru/gru.ko 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-xp 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-xp/xp.ko 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-xp/xpc.ko 2024-07-04T07:47:31.431 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/tifm_7xx1.ko 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/tifm_core.ko 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/tsl2550.ko 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/vmw_balloon.ko 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/vmw_vmci 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/bonding 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/bonding/bonding.ko 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/b53 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/b53/b53_common.ko 2024-07-04T07:47:31.432 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/bcm-sf2.ko 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/mv88e6060.ko 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/mv88e6xxx 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/dummy.ko 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/eql.ko 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-07-04T07:47:31.433 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390/8390.ko 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/adaptec 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/agere 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/agere/et131x.ko 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/alteon 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/altera 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-07-04T07:47:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/amd 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/alx 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atl1c 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atl1e 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atlx 2024-07-04T07:47:31.435 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/genet 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/brocade 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/brocade/bna 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-07-04T07:47:31.436 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-07-04T07:47:31.437 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/cisco 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/cisco/enic 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dlink 2024-07-04T07:47:31.438 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/dnet.ko 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/emulex 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/emulex/benet 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/ethoc.ko 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/fealnx.ko 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e100.ko 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e1000 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-07-04T07:47:31.439 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e1000e 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/fm10k 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/i40e 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/iavf 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/igb 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/igbvf 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/ixgbe 2024-07-04T07:47:31.440 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/ixgbevf 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/jme.ko 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/marvell 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/marvell/skge.ko 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx4 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-07-04T07:47:31.441 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx5 2024-07-04T07:47:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-07-04T07:47:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-07-04T07:47:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/micrel 2024-07-04T07:47:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-07-04T07:47:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/myricom 2024-07-04T07:47:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/myricom/myri10ge 2024-07-04T07:47:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-07-04T07:47:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/natsemi 2024-07-04T07:47:31.442 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/neterion 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/nvidia 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/packetengines 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/netxen 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-07-04T07:47:31.443 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/rdc 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek/atp.ko 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/rocker 2024-07-04T07:47:31.444 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sfc 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/silan 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sis 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sis/sis190.ko 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sis/sis900.ko 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-07-04T07:47:31.445 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/stmicro 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/stmicro/stmmac 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun/cassini.ko 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun/niu.ko 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun/sungem.ko 2024-07-04T07:47:31.446 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-07-04T07:47:31.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/tehuti 2024-07-04T07:47:31.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-07-04T07:47:31.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/ti 2024-07-04T07:47:31.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/ti/tlan.ko 2024-07-04T07:47:31.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/via 2024-07-04T07:47:31.447 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/wiznet 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/xircom 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/geneve.ko 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/hyperv 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ieee802154 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ieee802154/fakelb.ko 2024-07-04T07:47:31.448 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ifb.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ipvlan 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ipvlan/ipvlan.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/macvlan.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/macvtap.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mdio/mdio-regmap.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/mii.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/net_failover.ko 2024-07-04T07:47:31.449 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/netconsole.ko 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/nlmon.ko 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/pcs 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/pcs/pcs-lynx.ko 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/amd.ko 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/bcm7xxx.ko 2024-07-04T07:47:31.450 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/bcm87xx.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/broadcom.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/cicada.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/davicom.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/dp83640.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/et1011c.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/icplus.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/lxt.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/marvell.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/micrel.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/national.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/phylink.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/qsemi.ko 2024-07-04T07:47:31.451 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/realtek.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/smsc.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/ste10Xp.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/phy/vitesse.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/bsd_comp.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_async.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_deflate.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_generic.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_mppe.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/ppp_synctty.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/pppoe.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/pppox.ko 2024-07-04T07:47:31.452 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/ppp/pptp.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/slip 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/slip/slhc.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/slip/slip.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/sungem_phy.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/tap.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_activebackup.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_broadcast.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_random.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/tun.ko 2024-07-04T07:47:31.453 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/veth.ko 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/virtio_net.ko 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/vmxnet3 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/vxlan 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/vxlan/vxlan.ko 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/xen-netback 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/xen-netback/xen-netback.ko 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/net/xen-netfront.ko 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ntb 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ntb/ntb.ko 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/nvme 2024-07-04T07:47:31.454 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/nvme/host 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/nvme/host/nvme-core.ko 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/nvme/host/nvme.ko 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport/parport.ko 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport/parport_cs.ko 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport/parport_pc.ko 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/parport/parport_serial.ko 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/hotplug 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-07-04T07:47:31.455 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/pcie 2024-07-04T07:47:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/pcie/aer_inject.ko 2024-07-04T07:47:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pci/xen-pcifront.ko 2024-07-04T07:47:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pcmcia 2024-07-04T07:47:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pcmcia/i82092.ko 2024-07-04T07:47:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pcmcia/pd6729.ko 2024-07-04T07:47:31.456 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pcmcia/yenta_socket.ko 2024-07-04T07:47:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pinctrl 2024-07-04T07:47:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pinctrl/intel 2024-07-04T07:47:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-07-04T07:47:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform 2024-07-04T07:47:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86 2024-07-04T07:47:31.457 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/acer-wmi.ko 2024-07-04T07:47:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/acerhdf.ko 2024-07-04T07:47:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/apple-gmux.ko 2024-07-04T07:47:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/asus-laptop.ko 2024-07-04T07:47:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-07-04T07:47:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/asus-wmi.ko 2024-07-04T07:47:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/classmate-laptop.ko 2024-07-04T07:47:31.458 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-07-04T07:47:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-07-04T07:47:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-07-04T07:47:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-07-04T07:47:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/hdaps.ko 2024-07-04T07:47:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/intel 2024-07-04T07:47:31.459 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-07-04T07:47:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/intel_ips.ko 2024-07-04T07:47:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/msi-wmi.ko 2024-07-04T07:47:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/mxm-wmi.ko 2024-07-04T07:47:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-07-04T07:47:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/samsung-laptop.ko 2024-07-04T07:47:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/samsung-q10.ko 2024-07-04T07:47:31.460 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/topstar-laptop.ko 2024-07-04T07:47:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-07-04T07:47:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/toshiba_haps.ko 2024-07-04T07:47:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/wmi-bmof.ko 2024-07-04T07:47:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/platform/x86/wmi.ko 2024-07-04T07:47:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/powercap 2024-07-04T07:47:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/powercap/intel_rapl_common.ko 2024-07-04T07:47:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/powercap/intel_rapl_msr.ko 2024-07-04T07:47:31.461 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps 2024-07-04T07:47:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/clients 2024-07-04T07:47:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/clients/pps-gpio.ko 2024-07-04T07:47:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/clients/pps-ldisc.ko 2024-07-04T07:47:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/clients/pps_parport.ko 2024-07-04T07:47:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/pps/pps_core.ko 2024-07-04T07:47:31.462 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ptp 2024-07-04T07:47:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ptp/ptp.ko 2024-07-04T07:47:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ptp/ptp_kvm.ko 2024-07-04T07:47:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras 2024-07-04T07:47:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras/amd 2024-07-04T07:47:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras/amd/atl 2024-07-04T07:47:31.463 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-07-04T07:47:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ras/amd/fmpm.ko 2024-07-04T07:47:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc 2024-07-04T07:47:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-bq32k.ko 2024-07-04T07:47:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1286.ko 2024-07-04T07:47:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1307.ko 2024-07-04T07:47:31.464 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1374.ko 2024-07-04T07:47:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1511.ko 2024-07-04T07:47:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1553.ko 2024-07-04T07:47:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1672.ko 2024-07-04T07:47:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds1742.ko 2024-07-04T07:47:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds2404.ko 2024-07-04T07:47:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-ds3232.ko 2024-07-04T07:47:31.465 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-em3027.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-fm3130.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-isl12022.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-isl1208.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-m41t80.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-m48t35.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-m48t59.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-max6900.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-msm6242.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf2127.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf85063.ko 2024-07-04T07:47:31.466 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf8523.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf8563.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-pcf8583.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rp5c01.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rs5c372.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rx8025.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-rx8581.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/rtc/rtc-x1205.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/3w-9xxx.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/3w-sas.ko 2024-07-04T07:47:31.467 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/3w-xxxx.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/BusLogic.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/a100u2w.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aacraid 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aacraid/aacraid.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/advansys.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aic7xxx 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/am53c974.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/arcmsr 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/atp870u.ko 2024-07-04T07:47:31.468 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/be2iscsi 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bfa 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bfa/bfa.ko 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bnx2fc 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bnx2i 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/ch.ko 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/csiostor 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/csiostor/csiostor.ko 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/cxgb3i 2024-07-04T07:47:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/cxgb4i 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/dc395x.ko 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/dmx3191d.ko 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/esas2r 2024-07-04T07:47:31.470 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/esas2r/esas2r.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/esp_scsi.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fcoe 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fcoe/fcoe.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fnic 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/fnic/fnic.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/hpsa.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/hptiop.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/hv_storvsc.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/initio.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/ipr.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/ips.ko 2024-07-04T07:47:31.471 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/isci 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/isci/isci.ko 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/iscsi_tcp.ko 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libfc 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libfc/libfc.ko 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libiscsi.ko 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libiscsi_tcp.ko 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libsas 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/libsas/libsas.ko 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/lpfc 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/lpfc/lpfc.ko 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid 2024-07-04T07:47:31.472 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mpt3sas 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mvsas 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mvsas/mvsas.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/mvumi.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/pm8001 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/pmcraid.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla1280.ko 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla2xxx 2024-07-04T07:47:31.473 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla4xxx 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/raid_class.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_debug.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_fc.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_sas.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_spi.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/scsi_transport_srp.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/ses.ko 2024-07-04T07:47:31.474 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/st.ko 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/stex.ko 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/sym53c8xx_2 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/virtio_scsi.ko 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/vmw_pvscsi.ko 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/scsi/wd719x.ko 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ssb 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ssb/ssb.ko 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target 2024-07-04T07:47:31.475 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/iscsi 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/loopback 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/loopback/tcm_loop.ko 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_file.ko 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_iblock.ko 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_mod.ko 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_pscsi.ko 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/target_core_user.ko 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/tcm_fc 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal 2024-07-04T07:47:31.476 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-07-04T07:47:31.477 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/ipwireless 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-07-04T07:47:31.478 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/n_gsm.ko 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/n_hdlc.ko 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/nozomi.ko 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/8250 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/arc_uart.ko 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/jsm 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/tty/serial/jsm/jsm.ko 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs/core 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs/core/ufshcd-core.ko 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs/host 2024-07-04T07:47:31.479 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio_aec.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio_cif.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio_pci_generic.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/uio/uio_sercos3.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class/cdc-acm.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class/cdc-wdm.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class/usblp.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/class/usbtmc.ko 2024-07-04T07:47:31.480 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/host 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/host/sl811-hcd.ko 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/image 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/image/mdc800.ko 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/image/microtek.ko 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/adutux.ko 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/appledisplay.ko 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/emi26.ko 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/emi62.ko 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/ezusb.ko 2024-07-04T07:47:31.481 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/idmouse.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/iowarrior.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/isight_firmware.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/ldusb.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/legousbtower.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/sisusbvga 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/trancevibrator.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/usb3503.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/usblcd.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/usbsevseg.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/uss720.ko 2024-07-04T07:47:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/misc/yurex.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/phy 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/phy/phy-generic.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/aircable.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ark3116.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/belkin_sa.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ch341.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/cp210x.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/cyberjack.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/cypress_m8.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/digi_acceleport.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/empeg.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ftdi_sio.ko 2024-07-04T07:47:31.483 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/garmin_gps.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/io_edgeport.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/io_ti.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ipaq.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ipw.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ir-usb.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/keyspan.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/keyspan_pda.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/kl5kusb105.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/kobil_sct.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/mct_u232.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/mos7720.ko 2024-07-04T07:47:31.484 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/mos7840.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/navman.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/omninet.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/opticon.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/option.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/oti6858.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/pl2303.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/qcaux.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/qcserial.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/quatech2.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/safe_serial.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/sierra.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/spcp8x5.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ssu100.ko 2024-07-04T07:47:31.485 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/symbolserial.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/usb_debug.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/usb_wwan.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/visor.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/whiteheat.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/serial/xsens_mt.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/uas.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-alauda.ko 2024-07-04T07:47:31.486 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-cypress.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-datafab.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-freecom.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-isd200.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-karma.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-onetouch.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-realtek.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-sddr09.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-sddr55.ko 2024-07-04T07:47:31.487 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/ums-usbat.ko 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/storage/usb-storage.ko 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/usbip 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/usbip/usbip-core.ko 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/usbip/usbip-host.ko 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/pci 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/pci/vfio-pci.ko 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/vfio.ko 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost 2024-07-04T07:47:31.488 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost/vhost.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost/vhost_iotlb.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost/vhost_net.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/vhost/vhost_scsi.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video/backlight 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video/backlight/apple_bl.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video/backlight/lcd.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/video/backlight/platform_lcd.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_balloon.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_mmio.ko 2024-07-04T07:47:31.489 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_pci.ko 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/virtio/virtio_ring.ko 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/masters 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/masters/ds2482.ko 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/masters/ds2490.ko 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-07-04T07:47:31.490 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_smem.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/slaves/w1_therm.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/w1/wire.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/alim1535_wdt.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/alim7101_wdt.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/f71808e_wdt.ko 2024-07-04T07:47:31.491 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/hpwdt.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/i6300esb.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/iTCO_wdt.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/ib700wdt.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/ibmasr.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/it8712f_wdt.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/it87_wdt.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/machzwd.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/nv_tco.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/pcwd_pci.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/pcwd_usb.ko 2024-07-04T07:47:31.492 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/sch311x_wdt.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/softdog.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/sp5100_tco.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/via_wdt.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/w83877f_wdt.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/w83977f_wdt.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/wdt_pci.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/watchdog/xen_wdt.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-acpi-processor.ko 2024-07-04T07:47:31.493 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-evtchn.ko 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-gntalloc.ko 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-gntdev.ko 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-pciback 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xen-privcmd.ko 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xenfs 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/drivers/xen/xenfs/xenfs.ko 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/9p 2024-07-04T07:47:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/9p/9p.ko 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/affs 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/affs/affs.ko 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/befs 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/befs/befs.ko 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/binfmt_misc.ko 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/btrfs 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/btrfs/btrfs.ko 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/cachefiles 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/cachefiles/cachefiles.ko 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ceph 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ceph/ceph.ko 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/coda 2024-07-04T07:47:31.495 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/coda/coda.ko 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/cramfs 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/cramfs/cramfs.ko 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/dlm 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/dlm/dlm.ko 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ecryptfs 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ecryptfs/ecryptfs.ko 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/f2fs 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/f2fs/f2fs.ko 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fat 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fat/fat.ko 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fat/msdos.ko 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fat/vfat.ko 2024-07-04T07:47:31.496 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fuse 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fuse/cuse.ko 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/fuse/fuse.ko 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/gfs2 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/gfs2/gfs2.ko 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/hfs 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/hfs/hfs.ko 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/hfsplus 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/hfsplus/hfsplus.ko 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/isofs 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/isofs/isofs.ko 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/jfs 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/jfs/jfs.ko 2024-07-04T07:47:31.497 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/lockd 2024-07-04T07:47:31.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/lockd/lockd.ko 2024-07-04T07:47:31.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/minix 2024-07-04T07:47:31.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/minix/minix.ko 2024-07-04T07:47:31.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/netfs 2024-07-04T07:47:31.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/netfs/netfs.ko 2024-07-04T07:47:31.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs 2024-07-04T07:47:31.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/blocklayout 2024-07-04T07:47:31.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-07-04T07:47:31.498 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/filelayout 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/flexfilelayout 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/nfs.ko 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/nfsv3.ko 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs/nfsv4.ko 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs_common 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs_common/grace.ko 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfs_common/nfs_acl.ko 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfsd 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nfsd/nfsd.ko 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nilfs2 2024-07-04T07:47:31.499 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nilfs2/nilfs2.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-celtic.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-centeuro.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-croatian.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-cyrillic.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-gaelic.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-greek.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-iceland.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-inuit.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-roman.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-romanian.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/mac-turkish.ko 2024-07-04T07:47:31.500 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp1250.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp1251.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp1255.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp737.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp775.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp850.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp852.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp855.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp857.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp860.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp861.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp862.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp863.ko 2024-07-04T07:47:31.501 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp864.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp865.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp866.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp869.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp874.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp932.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp936.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp949.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_cp950.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_euc-jp.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-1.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-13.ko 2024-07-04T07:47:31.502 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-14.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-15.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-2.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-3.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-4.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-5.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-6.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-7.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_iso8859-9.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_koi8-r.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_koi8-ru.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_koi8-u.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_ucs2_utils.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/nls/nls_utf8.ko 2024-07-04T07:47:31.503 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/cluster 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/dlm 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/dlmfs 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/ocfs2.ko 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/overlayfs 2024-07-04T07:47:31.504 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/overlayfs/overlay.ko 2024-07-04T07:47:31.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/pstore 2024-07-04T07:47:31.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/pstore/ramoops.ko 2024-07-04T07:47:31.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/reiserfs 2024-07-04T07:47:31.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/reiserfs/reiserfs.ko 2024-07-04T07:47:31.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/romfs 2024-07-04T07:47:31.505 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/romfs/romfs.ko 2024-07-04T07:47:31.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb 2024-07-04T07:47:31.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/client 2024-07-04T07:47:31.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/client/cifs.ko 2024-07-04T07:47:31.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/common 2024-07-04T07:47:31.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/common/cifs_arc4.ko 2024-07-04T07:47:31.506 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/smb/common/cifs_md4.ko 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/squashfs 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/squashfs/squashfs.ko 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/sysv 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/sysv/sysv.ko 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/udf 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/udf/udf.ko 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ufs 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/ufs/ufs.ko 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/xfs 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/fs/xfs/xfs.ko 2024-07-04T07:47:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel 2024-07-04T07:47:31.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/rcu 2024-07-04T07:47:31.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/rcu/rcutorture.ko 2024-07-04T07:47:31.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/torture.ko 2024-07-04T07:47:31.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/trace 2024-07-04T07:47:31.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-07-04T07:47:31.508 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/asn1_encoder.ko 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crc-itu-t.ko 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crc8.ko 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crypto 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crypto/libarc4.ko 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/crypto/libdes.ko 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/libcrc32c.ko 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/lru_cache.ko 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/lz4 2024-07-04T07:47:31.509 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/lz4/lz4_compress.ko 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/lz4/lz4hc_compress.ko 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/math 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/math/cordic.ko 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/raid6 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/raid6/raid6_pq.ko 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/reed_solomon 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/reed_solomon/reed_solomon.ko 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/ts_bm.ko 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/ts_fsm.ko 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/ts_kmp.ko 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/zstd 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/lib/zstd/zstd_compress.ko 2024-07-04T07:47:31.510 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/mm 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/mm/hwpoison-inject.ko 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/6lowpan.ko 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_dest.ko 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_fragment.ko 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_hop.ko 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_ipv6.ko 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_mobility.ko 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_routing.ko 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/6lowpan/nhc_udp.ko 2024-07-04T07:47:31.511 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/garp.ko 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/mrp.ko 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/p8022.ko 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/psnap.ko 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/802/stp.ko 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/8021q 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/8021q/8021q.ko 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p/9pnet.ko 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p/9pnet_fd.ko 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p/9pnet_rdma.ko 2024-07-04T07:47:31.512 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/9p/9pnet_virtio.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/batman-adv 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/batman-adv/batman-adv.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/br_netfilter.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/bridge.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_among.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_arp.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_ip.ko 2024-07-04T07:47:31.513 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_limit.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_log.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_mark.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_snat.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_stp.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-07-04T07:47:31.514 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/ebtables.ko 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ceph 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ceph/libceph.ko 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/core 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/core/failover.ko 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/core/pktgen.ko 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp/dccp.ko 2024-07-04T07:47:31.515 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp/dccp_diag.ko 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp/dccp_ipv4.ko 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dccp/dccp_ipv6.ko 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dns_resolver 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dns_resolver/dns_resolver.ko 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/dsa_core.ko 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/tag_brcm.ko 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/tag_dsa.ko 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/tag_none.ko 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/dsa/tag_trailer.ko 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154/6lowpan 2024-07-04T07:47:31.516 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154/ieee802154.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ieee802154/ieee802154_socket.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ah4.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/esp4.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/fou.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/gre.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/inet_diag.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ip_gre.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ip_tunnel.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ip_vti.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ipcomp.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/ipip.ko 2024-07-04T07:47:31.517 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/arp_tables.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/iptable_security.ko 2024-07-04T07:47:31.518 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_bic.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_dctcp.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_diag.ko 2024-07-04T07:47:31.519 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_highspeed.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_htcp.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_hybla.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_illinois.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_lp.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_scalable.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_vegas.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_veno.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_westwood.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tcp_yeah.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/tunnel4.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/udp_diag.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/udp_tunnel.ko 2024-07-04T07:47:31.520 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv4/xfrm4_tunnel.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ah6.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/esp6.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/fou6.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ip6_tunnel.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ip6_vti.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/ipcomp6.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-07-04T07:47:31.521 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-07-04T07:47:31.522 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-07-04T07:47:31.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-07-04T07:47:31.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-07-04T07:47:31.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-07-04T07:47:31.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-07-04T07:47:31.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-07-04T07:47:31.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-07-04T07:47:31.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-07-04T07:47:31.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-07-04T07:47:31.523 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/sit.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/tunnel6.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/ipv6/xfrm6_tunnel.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/key 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/key/af_key.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_core.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_debugfs.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_eth.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_ip.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_ip6.ko 2024-07-04T07:47:31.524 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_netlink.ko 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/l2tp/l2tp_ppp.ko 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/llc 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/llc/llc.ko 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/mac802154 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/mac802154/mac802154.ko 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/mpls 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/mpls/mpls_gso.ko 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set.ko 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-07-04T07:47:31.525 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-07-04T07:47:31.526 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-07-04T07:47:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conncount.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_h323.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_irc.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-07-04T07:47:31.528 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_sane.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_sip.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_dup_netdev.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_flow_table.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_flow_table_inet.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_log_syslog.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_amanda.ko 2024-07-04T07:47:31.529 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_ftp.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_irc.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_sip.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_nat_tftp.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_synproxy_core.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nf_tables.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink_acct.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink_log.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink_osf.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nfnetlink_queue.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_chain_nat.ko 2024-07-04T07:47:31.530 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_compat.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_ct.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_dup_netdev.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_fib.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_fwd_netdev.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_hash.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_limit.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_log.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_masq.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_nat.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_numgen.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_queue.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_quota.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_redir.ko 2024-07-04T07:47:31.531 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_reject.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_reject_inet.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_socket.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/nft_tunnel.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_AUDIT.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_CHECKSUM.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_CLASSIFY.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_CT.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_DSCP.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_HL.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_HMARK.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_IDLETIMER.ko 2024-07-04T07:47:31.532 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_LED.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_LOG.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_MASQUERADE.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_NETMAP.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_NFLOG.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_NFQUEUE.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_RATEEST.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_REDIRECT.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_SECMARK.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TCPMSS.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TEE.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TPROXY.ko 2024-07-04T07:47:31.533 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_TRACE.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_addrtype.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_bpf.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_cgroup.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_cluster.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_comment.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_connbytes.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_connlabel.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_connlimit.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_connmark.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_conntrack.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_cpu.ko 2024-07-04T07:47:31.534 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_dccp.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_devgroup.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_dscp.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_ecn.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_esp.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_hashlimit.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_helper.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_hl.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_ipcomp.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_iprange.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_ipvs.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_l2tp.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_length.ko 2024-07-04T07:47:31.535 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_limit.ko 2024-07-04T07:47:31.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_mac.ko 2024-07-04T07:47:31.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_mark.ko 2024-07-04T07:47:31.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_multiport.ko 2024-07-04T07:47:31.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_nat.ko 2024-07-04T07:47:31.536 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_nfacct.ko 2024-07-04T07:47:31.537 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_osf.ko 2024-07-04T07:47:31.537 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_owner.ko 2024-07-04T07:47:31.537 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_physdev.ko 2024-07-04T07:47:31.537 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_pkttype.ko 2024-07-04T07:47:31.537 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_policy.ko 2024-07-04T07:47:31.537 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_quota.ko 2024-07-04T07:47:31.537 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_rateest.ko 2024-07-04T07:47:31.537 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_realm.ko 2024-07-04T07:47:31.537 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_recent.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_sctp.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_set.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_socket.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_state.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_statistic.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_string.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_tcpmss.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_time.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netfilter/xt_u32.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netlink 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/netlink/netlink_diag.ko 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/nsh 2024-07-04T07:47:31.538 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/nsh/nsh.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch/openvswitch.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch/vport-geneve.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch/vport-gre.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/openvswitch/vport-vxlan.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/packet 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/packet/af_packet_diag.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/rds 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/rds/rds.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/rds/rds_rdma.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/rds/rds_tcp.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_csum.ko 2024-07-04T07:47:31.539 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_gact.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_mirred.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_nat.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_pedit.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_police.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_simple.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_skbedit.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/act_vlan.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_basic.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_bpf.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_flow.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_fw.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_route.ko 2024-07-04T07:47:31.540 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/cls_u32.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_cmp.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_ipset.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_meta.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_nbyte.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_text.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/em_u32.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_choke.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_codel.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_drr.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_fq.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_gred.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_hfsc.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_hhf.ko 2024-07-04T07:47:31.541 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_htb.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_ingress.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_mqprio.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_mqprio_lib.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_multiq.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_netem.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_pie.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_plug.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_prio.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_qfq.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_red.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_sfb.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_sfq.ko 2024-07-04T07:47:31.542 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_tbf.ko 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sched/sch_teql.ko 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sctp 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sctp/sctp.ko 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sctp/sctp_diag.ko 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/auth_gss 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/sunrpc.ko 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/xprtrdma 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/tipc 2024-07-04T07:47:31.543 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/tipc/tipc.ko 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/tipc/tipc_diag.ko 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/unix 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/unix/unix_diag.ko 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vsock.ko 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vsock_diag.ko 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/vmw_vsock/vsock_loopback.ko 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/xfrm 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/net/xfrm/xfrm_ipcomp.ko 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys 2024-07-04T07:47:31.544 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys/encrypted-keys 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys/trusted-keys 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/kernel/security/keys/trusted-keys/trusted.ko 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.alias 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.alias.bin 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.builtin 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.builtin.alias.bin 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.builtin.bin 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.builtin.modinfo 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.dep 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.dep.bin 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.devname 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.order 2024-07-04T07:47:31.545 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.softdep 2024-07-04T07:47:31.546 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.symbols 2024-07-04T07:47:31.546 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/modules.symbols.bin 2024-07-04T07:47:31.546 INFO:teuthology.orchestra.run.smithi039.stdout:/lib/modules/6.9.0-g0d32a7bba447/vmlinuz 2024-07-04T07:47:31.550 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-g0d32a7bba447 2024-07-04T07:47:31.550 DEBUG:teuthology.orchestra.run.smithi039:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-07-04T07:47:31.573 INFO:teuthology.orchestra.run.smithi039.stdout:postinstall scriptlet (using /bin/sh): 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout: /usr/bin/kernel-install add 6.9.0-g0d32a7bba447 /lib/modules/6.9.0-g0d32a7bba447/vmlinuz 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout:for file in vmlinuz System.map config; do 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout: if ! cmp --silent "/lib/modules/6.9.0-g0d32a7bba447/${file}" "/boot/${file}-6.9.0-g0d32a7bba447"; then 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout: cp "/lib/modules/6.9.0-g0d32a7bba447/${file}" "/boot/${file}-6.9.0-g0d32a7bba447" 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout: fi 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout:done 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -d "/lib/modules/6.9.0-g0d32a7bba447/dtb" ] && \ 2024-07-04T07:47:31.574 INFO:teuthology.orchestra.run.smithi039.stdout: ! diff -rq "/lib/modules/6.9.0-g0d32a7bba447/dtb" "/boot/dtb-6.9.0-g0d32a7bba447" >/dev/null 2>&1; then 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout: rm -rf "/boot/dtb-6.9.0-g0d32a7bba447" 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout: cp -r "/lib/modules/6.9.0-g0d32a7bba447/dtb" "/boot/dtb-6.9.0-g0d32a7bba447" 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout:if [ ! -e "/lib/modules/6.9.0-g0d32a7bba447/modules.dep" ]; then 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout: /usr/sbin/depmod 6.9.0-g0d32a7bba447 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout:preuninstall scriptlet (using /bin/sh): 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-install remove 6.9.0-g0d32a7bba447 2024-07-04T07:47:31.575 INFO:teuthology.orchestra.run.smithi039.stdout:fi 2024-07-04T07:47:31.576 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -qi grub2-tools 2024-07-04T07:47:31.650 INFO:teuthology.orchestra.run.smithi039.stdout:Name : grub2-tools 2024-07-04T07:47:31.650 INFO:teuthology.orchestra.run.smithi039.stdout:Epoch : 1 2024-07-04T07:47:31.650 INFO:teuthology.orchestra.run.smithi039.stdout:Version : 2.06 2024-07-04T07:47:31.650 INFO:teuthology.orchestra.run.smithi039.stdout:Release : 68.el9 2024-07-04T07:47:31.650 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: x86_64 2024-07-04T07:47:31.650 INFO:teuthology.orchestra.run.smithi039.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-04T07:47:31.650 INFO:teuthology.orchestra.run.smithi039.stdout:Group : Unspecified 2024-07-04T07:47:31.650 INFO:teuthology.orchestra.run.smithi039.stdout:Size : 8264254 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:License : GPLv3+ 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:Packager : builder@centos.org 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:Vendor : CentOS 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:Summary : Support tools for GRUB. 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:Description : 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:47:31.651 INFO:teuthology.orchestra.run.smithi039.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-04T07:47:31.652 INFO:teuthology.orchestra.run.smithi039.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-04T07:47:31.652 INFO:teuthology.orchestra.run.smithi039.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-04T07:47:31.652 INFO:teuthology.orchestra.run.smithi039.stdout:hardware devices. 2024-07-04T07:47:31.652 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:47:31.652 INFO:teuthology.orchestra.run.smithi039.stdout:This subpackage provides tools for support of all platforms. 2024-07-04T07:47:31.653 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-04T07:47:31.653 INFO:teuthology.task.kernel:Updating grub on smithi039 to boot 6.9.0-g0d32a7bba447 2024-07-04T07:47:31.653 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-04T07:47:32.343 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2024-07-04T07:47:33.837 INFO:teuthology.orchestra.run.smithi039.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-04T07:47:33.876 INFO:teuthology.orchestra.run.smithi039.stderr:done 2024-07-04T07:47:33.878 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-07-04T07:47:33.893 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.aoe8hfPH2N 2024-07-04T07:47:33.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aoe8hfPH2N 2024-07-04T07:47:33.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.aoe8hfPH2N 2024-07-04T07:47:34.101 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.aoe8hfPH2N is 6KB 2024-07-04T07:47:34.150 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.aoe8hfPH2N 2024-07-04T07:47:34.165 DEBUG:teuthology.orchestra.run.smithi039:> sudo /bin/ls /boot/loader/entries || true 2024-07-04T07:47:34.227 INFO:teuthology.orchestra.run.smithi039.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-04T07:47:34.228 INFO:teuthology.orchestra.run.smithi039.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-04T07:47:34.228 INFO:teuthology.orchestra.run.smithi039.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-04T07:47:34.228 INFO:teuthology.orchestra.run.smithi039.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-g0d32a7bba447.conf 2024-07-04T07:47:34.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-g0d32a7bba447 2024-07-04T07:47:34.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo shutdown -r now 2024-07-04T07:48:04.371 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-07-04T07:48:04.371 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-07-04T07:48:04.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:48:39.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-07-04T07:48:48.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-07-04T07:48:48.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:48:51.134 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-07-04T07:49:03.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-07-04T07:49:03.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:49:21.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-07-04T07:49:36.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-07-04T07:49:36.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:49:37.093 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-07-04T07:49:37.787 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-07-04T07:49:37.787 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-g0d32a7bba447"... 2024-07-04T07:49:37.787 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-07-04T07:49:37.830 INFO:teuthology.orchestra.run.smithi039.stdout:6.9.0-g0d32a7bba447 2024-07-04T07:49:37.830 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-g0d32a7bba447 vs 6.9.0-g0d32a7bba447 2024-07-04T07:49:37.830 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-04T07:49:37.830 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-04T07:49:38.831 DEBUG:teuthology.parallel:result is None 2024-07-04T07:49:38.832 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-04T07:49:38.842 INFO:teuthology.task.internal:Creating test directory... 2024-07-04T07:49:38.843 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-04T07:49:38.849 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-04T07:49:38.852 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-04T07:49:38.892 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-04T07:49:38.898 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-04T07:49:38.905 INFO:teuthology.task.internal:Creating archive directory... 2024-07-04T07:49:38.905 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-04T07:49:38.909 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-04T07:49:38.912 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-04T07:49:38.954 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-04T07:49:38.961 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-04T07:49:38.961 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-04T07:49:38.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T07:49:38.994 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-04T07:49:39.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T07:49:39.010 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-04T07:49:39.017 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-04T07:49:39.036 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-04T07:49:39.072 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-04T07:49:39.085 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-04T07:49:39.092 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-04T07:49:39.099 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-04T07:49:39.133 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-04T07:49:39.177 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-04T07:49:39.182 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-04T07:49:39.188 INFO:teuthology.task.internal:Configuring sudo... 2024-07-04T07:49:39.188 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-04T07:49:39.193 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-04T07:49:39.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-04T07:49:39.255 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-04T07:49:39.265 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-04T07:49:39.265 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-04T07:49:39.268 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-04T07:49:39.271 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-04T07:49:39.308 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-04T07:49:39.425 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-04T07:49:39.502 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-04T07:49:39.573 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-04T07:49:39.636 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T07:49:39.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-04T07:49:39.738 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-04T07:49:39.782 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-04T07:49:39.838 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-04T07:49:39.923 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-04T07:49:39.964 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T07:49:39.965 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-04T07:49:40.031 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-04T07:49:40.076 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-04T07:49:40.142 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-04T07:49:40.185 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-04T07:49:40.227 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T07:49:40.227 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-04T07:49:40.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-07-04T07:49:40.298 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-07-04T07:49:40.301 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-07-04T07:49:40.335 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-04T07:49:40.370 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-04T07:49:40.401 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-04T07:49:40.691 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-04T07:49:40.699 INFO:teuthology.task.internal:Starting timer... 2024-07-04T07:49:40.699 INFO:teuthology.run_tasks:Running task pcp... 2024-07-04T07:49:40.742 INFO:teuthology.run_tasks:Running task selinux... 2024-07-04T07:49:40.753 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-07-04T07:49:40.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2024-07-04T07:49:40.889 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2024-07-04T07:49:40.893 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-07-04T07:49:40.970 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-07-04T07:49:40.972 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-07-04T07:49:41.027 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-07-04T07:49:41.029 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-04T07:49:41.029 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2024-07-04T07:49:41.090 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2024-07-04T07:49:41.091 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-07-04T07:49:41.112 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-07-04T07:49:41.112 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-07-04T07:49:41.146 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-07-04T07:49:41.147 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive'} 2024-07-04T07:49:41.147 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-04T07:49:41.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T07:49:41.217 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-04T07:49:41.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T07:49:41.245 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-04T07:49:41.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T07:49:41.270 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-04T07:49:41.270 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2024-07-04T07:49:41.360 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-07-04T07:49:41.386 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-07-04T07:49:41.411 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-04T07:49:41.420 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-07-04T07:49:41.424 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-07-04T07:49:41.536 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-04T07:49:41.548 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-04T07:49:41.549 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com,smithi050.front.sepia.ceph.com,smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-04T07:56:07.847 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com'), Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2024-07-04T07:56:07.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-07-04T07:56:07.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:56:07.962 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-07-04T07:56:08.123 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-07-04T07:56:08.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-07-04T07:56:08.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:56:08.198 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-07-04T07:56:08.292 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-07-04T07:56:08.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-07-04T07:56:08.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-04T07:56:08.370 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-07-04T07:56:08.459 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-07-04T07:56:08.460 INFO:teuthology.run_tasks:Running task clock... 2024-07-04T07:56:08.471 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-04T07:56:08.472 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-04T07:56:08.472 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-04T07:56:08.480 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-04T07:56:08.480 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-04T07:56:08.482 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-04T07:56:08.482 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-04T07:56:08.514 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-04T07:56:08.530 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-04T07:56:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-04T07:56:08.544 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-04T07:56:08.556 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-07-04T07:56:08.567 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-07-04T07:56:08.568 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-07-04T07:56:08.570 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-04T07:56:08.578 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-07-04T07:56:08.582 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-04T07:56:08.592 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-04T07:56:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-04T07:56:08.606 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-04T07:56:08.616 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-04T07:56:08.660 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-07-04T07:56:08.662 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-07-04T07:56:08.664 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-04T07:56:08.664 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-07-04T07:56:08.664 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.664 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.664 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.664 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.666 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-04T07:56:08.666 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-07-04T07:56:08.666 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.666 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.666 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.666 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.692 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2024-07-04T07:56:08.725 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2024-07-04T07:56:08.766 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-04T07:56:08.811 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-04T07:56:08.967 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-07-04T07:56:08.978 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-04T07:56:08.978 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-07-04T07:56:08.978 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.978 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.978 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.978 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T07:56:08.980 INFO:teuthology.run_tasks:Running task install... 2024-07-04T07:56:08.990 DEBUG:teuthology.task.install:project ceph 2024-07-04T07:56:08.990 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2e9f68b5506c607b2f7d69d380cb7887cf00dc84'}} 2024-07-04T07:56:08.990 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2e9f68b5506c607b2f7d69d380cb7887cf00dc84'} 2024-07-04T07:56:08.991 INFO:teuthology.task.install:Using flavor: default 2024-07-04T07:56:08.996 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-04T07:56:08.996 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-07-04T07:56:08.997 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'tag': None, 'wait_for_package': False} 2024-07-04T07:56:08.997 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:56:09.000 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'tag': None, 'wait_for_package': False} 2024-07-04T07:56:09.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:56:09.001 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'tag': None, 'wait_for_package': False} 2024-07-04T07:56:09.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:56:09.152 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/ 2024-07-04T07:56:09.152 INFO:teuthology.task.install.rpm:Package version is 19.0.0-4737.g2e9f68b5 2024-07-04T07:56:09.239 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/ 2024-07-04T07:56:09.239 INFO:teuthology.task.install.rpm:Package version is 19.0.0-4737.g2e9f68b5 2024-07-04T07:56:09.249 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/ 2024-07-04T07:56:09.249 INFO:teuthology.task.install.rpm:Package version is 19.0.0-4737.g2e9f68b5 2024-07-04T07:56:09.434 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-04T07:56:09.434 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T07:56:09.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-04T07:56:09.445 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-04T07:56:09.445 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T07:56:09.445 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-04T07:56:09.447 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-04T07:56:09.447 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T07:56:09.447 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-04T07:56:09.474 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-04T07:56:09.474 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-04T07:56:09.478 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-04T07:56:09.478 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-04T07:56:09.529 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-04T07:56:09.529 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-04T07:56:09.542 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-04T07:56:09.549 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-04T07:56:09.618 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-04T07:56:09.627 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-04T07:56:09.657 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-04T07:56:09.683 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-07-04T07:56:09.685 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-07-04T07:56:09.693 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-07-04T07:56:09.695 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-07-04T07:56:09.770 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-04T07:56:09.854 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2024-07-04T07:56:09.858 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2024-07-04T07:56:09.913 INFO:teuthology.orchestra.run.smithi133.stdout:45 files removed 2024-07-04T07:56:09.933 INFO:teuthology.orchestra.run.smithi050.stdout:45 files removed 2024-07-04T07:56:09.940 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-04T07:56:09.960 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-04T07:56:10.174 INFO:teuthology.orchestra.run.smithi039.stdout:45 files removed 2024-07-04T07:56:10.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-04T07:56:10.839 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 131 kB/s | 83 kB 00:00 2024-07-04T07:56:10.942 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 117 kB/s | 83 kB 00:00 2024-07-04T07:56:11.508 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 97 kB/s | 83 kB 00:00 2024-07-04T07:56:11.631 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 32 kB/s | 24 kB 00:00 2024-07-04T07:56:11.684 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 34 kB/s | 24 kB 00:00 2024-07-04T07:56:12.210 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-07-04T07:56:12.250 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2024-07-04T07:56:12.306 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-07-04T07:56:13.292 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 2.0 kB/s | 1.9 kB 00:00 2024-07-04T07:56:14.043 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.2 MB 00:01 2024-07-04T07:56:14.232 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-07-04T07:56:15.549 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.2 MB 00:02 2024-07-04T07:56:15.836 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-07-04T07:56:19.174 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 20 MB 00:02 2024-07-04T07:56:21.448 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.4 MB 00:01 2024-07-04T07:56:23.238 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 17 kB 00:00 2024-07-04T07:56:23.548 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 2.4 MB/s | 20 MB 00:08 2024-07-04T07:56:24.126 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 30 MB/s | 22 MB 00:00 2024-07-04T07:56:25.134 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.4 MB 00:01 2024-07-04T07:56:26.868 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 17 kB 00:00 2024-07-04T07:56:27.982 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 24 MB/s | 22 MB 00:00 2024-07-04T07:56:29.631 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-07-04T07:56:29.732 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.4 MB 00:02 2024-07-04T07:56:31.233 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 17 kB 00:00 2024-07-04T07:56:32.127 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-04T07:56:32.127 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-04T07:56:32.139 INFO:teuthology.orchestra.run.smithi050.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-04T07:56:32.139 INFO:teuthology.orchestra.run.smithi050.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-04T07:56:32.140 INFO:teuthology.orchestra.run.smithi050.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-04T07:56:32.141 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-04T07:56:32.143 INFO:teuthology.orchestra.run.smithi050.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-04T07:56:32.145 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-04T07:56:32.146 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-04T07:56:32.214 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-07-04T07:56:32.240 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-07-04T07:56:32.240 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-07-04T07:56:32.240 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-07-04T07:56:32.240 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-07-04T07:56:32.240 INFO:teuthology.orchestra.run.smithi050.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 6.6 k 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 5.4 M 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 872 k 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 148 k 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 1.5 M 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 146 k 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 14 M 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 7.4 M 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 49 k 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 10 M 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 44 M 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 274 k 2024-07-04T07:56:32.241 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 764 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 228 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 27 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 18 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 32 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 766 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 127 k 2024-07-04T07:56:32.242 INFO:teuthology.orchestra.run.smithi050.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 162 k 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 321 k 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 304 k 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 99 k 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 86 k 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 3.1 M 2024-07-04T07:56:32.243 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 170 k 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel x86_64 6.4.0-2.el9 appstream 95 k 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-04T07:56:32.244 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 3.3 M 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 3.1 M 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs x86_64 6.4.0-2.el9 baseos 1.1 M 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 20 M 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 34 k 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 2.2 M 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 278 k 2024-07-04T07:56:32.245 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 4.9 M 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 18 M 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 16 k 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 25 k 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-04T07:56:32.246 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 158 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 505 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 5.2 M 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-04T07:56:32.247 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-04T07:56:32.248 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 45 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 135 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-04T07:56:32.249 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-04T07:56:32.250 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-04T07:56:32.251 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-04T07:56:32.252 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-04T07:56:32.253 INFO:teuthology.orchestra.run.smithi050.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout:Install 148 Packages 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 12 Packages 2024-07-04T07:56:32.254 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:56:32.255 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 231 M 2024-07-04T07:56:32.255 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-07-04T07:56:32.437 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 21 MB/s | 22 MB 00:01 2024-07-04T07:56:32.991 INFO:teuthology.orchestra.run.smithi050.stdout:(1/160): ceph-19.0.0-4737.g2e9f68b5.el9.x86_64. 45 kB/s | 6.6 kB 00:00 2024-07-04T07:56:33.200 INFO:teuthology.orchestra.run.smithi050.stdout:(2/160): ceph-fuse-19.0.0-4737.g2e9f68b5.el9.x8 4.1 MB/s | 872 kB 00:00 2024-07-04T07:56:33.258 INFO:teuthology.orchestra.run.smithi050.stdout:(3/160): ceph-immutable-object-cache-19.0.0-473 2.5 MB/s | 148 kB 00:00 2024-07-04T07:56:33.475 INFO:teuthology.orchestra.run.smithi050.stdout:(4/160): ceph-mds-19.0.0-4737.g2e9f68b5.el9.x86 10 MB/s | 2.2 MB 00:00 2024-07-04T07:56:33.750 INFO:teuthology.orchestra.run.smithi050.stdout:(5/160): ceph-common-19.0.0-4737.g2e9f68b5.el9. 22 MB/s | 20 MB 00:00 2024-07-04T07:56:33.792 INFO:teuthology.orchestra.run.smithi050.stdout:(6/160): ceph-mgr-19.0.0-4737.g2e9f68b5.el9.x86 4.8 MB/s | 1.5 MB 00:00 2024-07-04T07:56:33.934 INFO:teuthology.orchestra.run.smithi050.stdout:(7/160): ceph-mon-19.0.0-4737.g2e9f68b5.el9.x86 27 MB/s | 4.9 MB 00:00 2024-07-04T07:56:34.309 INFO:teuthology.orchestra.run.smithi050.stdout:(8/160): ceph-radosgw-19.0.0-4737.g2e9f68b5.el9 27 MB/s | 10 MB 00:00 2024-07-04T07:56:34.343 INFO:teuthology.orchestra.run.smithi050.stdout:(9/160): ceph-selinux-19.0.0-4737.g2e9f68b5.el9 758 kB/s | 25 kB 00:00 2024-07-04T07:56:34.436 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-07-04T07:56:34.710 INFO:teuthology.orchestra.run.smithi050.stdout:(10/160): ceph-base-19.0.0-4737.g2e9f68b5.el9.x 2.9 MB/s | 5.4 MB 00:01 2024-07-04T07:56:34.794 INFO:teuthology.orchestra.run.smithi050.stdout:(11/160): cephfs-mirror-19.0.0-4737.g2e9f68b5.e 2.7 MB/s | 228 kB 00:00 2024-07-04T07:56:35.036 INFO:teuthology.orchestra.run.smithi050.stdout:(12/160): ceph-osd-19.0.0-4737.g2e9f68b5.el9.x8 14 MB/s | 18 MB 00:01 2024-07-04T07:56:35.053 INFO:teuthology.orchestra.run.smithi050.stdout:(13/160): cephfs-shell-19.0.0-4737.g2e9f68b5.el 104 kB/s | 27 kB 00:00 2024-07-04T07:56:35.069 INFO:teuthology.orchestra.run.smithi050.stdout:(14/160): libcephfs-devel-19.0.0-4737.g2e9f68b5 954 kB/s | 32 kB 00:00 2024-07-04T07:56:35.120 INFO:teuthology.orchestra.run.smithi050.stdout:(15/160): libcephsqlite-19.0.0-4737.g2e9f68b5.e 3.1 MB/s | 158 kB 00:00 2024-07-04T07:56:35.195 INFO:teuthology.orchestra.run.smithi050.stdout:(16/160): libcephfs2-19.0.0-4737.g2e9f68b5.el9. 5.3 MB/s | 766 kB 00:00 2024-07-04T07:56:35.221 INFO:teuthology.orchestra.run.smithi050.stdout:(17/160): librados-devel-19.0.0-4737.g2e9f68b5. 1.2 MB/s | 127 kB 00:00 2024-07-04T07:56:35.338 INFO:teuthology.orchestra.run.smithi050.stdout:(18/160): libradosstriper1-19.0.0-4737.g2e9f68b 3.5 MB/s | 505 kB 00:00 2024-07-04T07:56:35.430 INFO:teuthology.orchestra.run.smithi050.stdout:(19/160): python3-ceph-argparse-19.0.0-4737.g2e 492 kB/s | 45 kB 00:00 2024-07-04T07:56:35.522 INFO:teuthology.orchestra.run.smithi050.stdout:(20/160): python3-ceph-common-19.0.0-4737.g2e9f 1.4 MB/s | 135 kB 00:00 2024-07-04T07:56:35.605 INFO:teuthology.orchestra.run.smithi050.stdout:(21/160): librgw2-19.0.0-4737.g2e9f68b5.el9.x86 14 MB/s | 5.2 MB 00:00 2024-07-04T07:56:35.630 INFO:teuthology.orchestra.run.smithi050.stdout:(22/160): python3-cephfs-19.0.0-4737.g2e9f68b5. 1.5 MB/s | 162 kB 00:00 2024-07-04T07:56:35.656 INFO:teuthology.orchestra.run.smithi050.stdout:(23/160): python3-rados-19.0.0-4737.g2e9f68b5.e 6.3 MB/s | 321 kB 00:00 2024-07-04T07:56:35.698 INFO:teuthology.orchestra.run.smithi050.stdout:(24/160): python3-rbd-19.0.0-4737.g2e9f68b5.el9 4.4 MB/s | 304 kB 00:00 2024-07-04T07:56:35.723 INFO:teuthology.orchestra.run.smithi050.stdout:(25/160): python3-rgw-19.0.0-4737.g2e9f68b5.el9 1.4 MB/s | 99 kB 00:00 2024-07-04T07:56:36.107 INFO:teuthology.orchestra.run.smithi050.stdout:(26/160): ceph-test-19.0.0-4737.g2e9f68b5.el9.x 25 MB/s | 44 MB 00:01 2024-07-04T07:56:36.132 INFO:teuthology.orchestra.run.smithi050.stdout:(27/160): rbd-fuse-19.0.0-4737.g2e9f68b5.el9.x8 199 kB/s | 86 kB 00:00 2024-07-04T07:56:36.174 INFO:teuthology.orchestra.run.smithi050.stdout:(28/160): rbd-nbd-19.0.0-4737.g2e9f68b5.el9.x86 2.5 MB/s | 170 kB 00:00 2024-07-04T07:56:36.199 INFO:teuthology.orchestra.run.smithi050.stdout:(29/160): ceph-grafana-dashboards-19.0.0-4737.g 513 kB/s | 34 kB 00:00 2024-07-04T07:56:36.216 INFO:teuthology.orchestra.run.smithi050.stdout:(30/160): ceph-mgr-cephadm-19.0.0-4737.g2e9f68b 3.4 MB/s | 146 kB 00:00 2024-07-04T07:56:36.292 INFO:teuthology.orchestra.run.smithi050.stdout:(31/160): rbd-mirror-19.0.0-4737.g2e9f68b5.el9. 5.4 MB/s | 3.1 MB 00:00 2024-07-04T07:56:36.342 INFO:teuthology.orchestra.run.smithi050.stdout:(32/160): ceph-mgr-modules-core-19.0.0-4737.g2e 5.4 MB/s | 278 kB 00:00 2024-07-04T07:56:36.409 INFO:teuthology.orchestra.run.smithi050.stdout:(33/160): ceph-mgr-rook-19.0.0-4737.g2e9f68b5.e 741 kB/s | 49 kB 00:00 2024-07-04T07:56:36.526 INFO:teuthology.orchestra.run.smithi050.stdout:(34/160): ceph-mgr-diskprediction-local-19.0.0- 24 MB/s | 7.4 MB 00:00 2024-07-04T07:56:36.543 INFO:teuthology.orchestra.run.smithi050.stdout:(35/160): ceph-prometheus-alerts-19.0.0-4737.g2 123 kB/s | 16 kB 00:00 2024-07-04T07:56:36.576 INFO:teuthology.orchestra.run.smithi050.stdout:(36/160): ceph-volume-19.0.0-4737.g2e9f68b5.el9 5.4 MB/s | 274 kB 00:00 2024-07-04T07:56:36.627 INFO:teuthology.orchestra.run.smithi050.stdout:(37/160): cephadm-19.0.0-4737.g2e9f68b5.el9.noa 8.9 MB/s | 764 kB 00:00 2024-07-04T07:56:36.643 INFO:teuthology.orchestra.run.smithi050.stdout:(38/160): cephfs-top-19.0.0-4737.g2e9f68b5.el9. 273 kB/s | 18 kB 00:00 2024-07-04T07:56:36.752 INFO:teuthology.orchestra.run.smithi050.stdout:(39/160): fuse-2.9.9-15.el9.x86_64.rpm 744 kB/s | 80 kB 00:00 2024-07-04T07:56:36.827 INFO:teuthology.orchestra.run.smithi050.stdout:(40/160): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-07-04T07:56:36.853 INFO:teuthology.orchestra.run.smithi050.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-07-04T07:56:36.886 INFO:teuthology.orchestra.run.smithi050.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-07-04T07:56:36.945 INFO:teuthology.orchestra.run.smithi050.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-07-04T07:56:36.987 INFO:teuthology.orchestra.run.smithi050.stdout:(44/160): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-07-04T07:56:37.013 INFO:teuthology.orchestra.run.smithi050.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-07-04T07:56:37.038 INFO:teuthology.orchestra.run.smithi050.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-07-04T07:56:37.063 INFO:teuthology.orchestra.run.smithi050.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-07-04T07:56:37.097 INFO:teuthology.orchestra.run.smithi050.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-07-04T07:56:37.139 INFO:teuthology.orchestra.run.smithi050.stdout:(49/160): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-07-04T07:56:37.164 INFO:teuthology.orchestra.run.smithi050.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-07-04T07:56:37.189 INFO:teuthology.orchestra.run.smithi050.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-07-04T07:56:37.214 INFO:teuthology.orchestra.run.smithi050.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-07-04T07:56:37.240 INFO:teuthology.orchestra.run.smithi050.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-07-04T07:56:37.256 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-04T07:56:37.257 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-04T07:56:37.279 INFO:teuthology.orchestra.run.smithi039.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-04T07:56:37.280 INFO:teuthology.orchestra.run.smithi039.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-04T07:56:37.282 INFO:teuthology.orchestra.run.smithi050.stdout:(54/160): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-07-04T07:56:37.282 INFO:teuthology.orchestra.run.smithi039.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-04T07:56:37.283 INFO:teuthology.orchestra.run.smithi039.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-04T07:56:37.286 INFO:teuthology.orchestra.run.smithi039.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-04T07:56:37.289 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-04T07:56:37.291 INFO:teuthology.orchestra.run.smithi039.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-04T07:56:37.340 INFO:teuthology.orchestra.run.smithi050.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-07-04T07:56:37.366 INFO:teuthology.orchestra.run.smithi050.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-07-04T07:56:37.370 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-07-04T07:56:37.391 INFO:teuthology.orchestra.run.smithi050.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-07-04T07:56:37.398 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-07-04T07:56:37.398 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-07-04T07:56:37.398 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-07-04T07:56:37.398 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-07-04T07:56:37.398 INFO:teuthology.orchestra.run.smithi039.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 6.6 k 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 5.4 M 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 872 k 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 148 k 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 1.5 M 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 146 k 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 14 M 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 7.4 M 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 49 k 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 10 M 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 44 M 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 274 k 2024-07-04T07:56:37.399 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 764 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 228 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 27 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 18 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 32 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 766 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 127 k 2024-07-04T07:56:37.400 INFO:teuthology.orchestra.run.smithi039.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 162 k 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 321 k 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 304 k 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 99 k 2024-07-04T07:56:37.401 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 86 k 2024-07-04T07:56:37.402 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 3.1 M 2024-07-04T07:56:37.402 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 170 k 2024-07-04T07:56:37.402 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel x86_64 6.4.0-2.el9 appstream 95 k 2024-07-04T07:56:37.402 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-07-04T07:56:37.402 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-04T07:56:37.402 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-04T07:56:37.402 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-04T07:56:37.402 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 3.3 M 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 3.1 M 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs x86_64 6.4.0-2.el9 baseos 1.1 M 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-04T07:56:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 20 M 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 34 k 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 2.2 M 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 278 k 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 4.9 M 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 18 M 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 16 k 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 25 k 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-04T07:56:37.404 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 158 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 505 k 2024-07-04T07:56:37.405 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 5.2 M 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-04T07:56:37.406 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 45 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 135 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-04T07:56:37.407 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-04T07:56:37.408 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-04T07:56:37.409 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-04T07:56:37.410 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-04T07:56:37.411 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout:Installing weak dependencies: 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-07-04T07:56:37.412 INFO:teuthology.orchestra.run.smithi039.stdout:Install 148 Packages 2024-07-04T07:56:37.413 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 12 Packages 2024-07-04T07:56:37.413 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:56:37.413 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 231 M 2024-07-04T07:56:37.413 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-07-04T07:56:37.416 INFO:teuthology.orchestra.run.smithi050.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-07-04T07:56:37.567 INFO:teuthology.orchestra.run.smithi050.stdout:(59/160): ceph-mgr-dashboard-19.0.0-4737.g2e9f6 10 MB/s | 14 MB 00:01 2024-07-04T07:56:37.592 INFO:teuthology.orchestra.run.smithi050.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-07-04T07:56:37.626 INFO:teuthology.orchestra.run.smithi050.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-07-04T07:56:37.651 INFO:teuthology.orchestra.run.smithi050.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-07-04T07:56:37.676 INFO:teuthology.orchestra.run.smithi050.stdout:(63/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-07-04T07:56:37.701 INFO:teuthology.orchestra.run.smithi050.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-07-04T07:56:37.735 INFO:teuthology.orchestra.run.smithi050.stdout:(65/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-07-04T07:56:37.760 INFO:teuthology.orchestra.run.smithi050.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 421 kB/s | 35 kB 00:00 2024-07-04T07:56:37.817 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-07-04T07:56:37.852 INFO:teuthology.orchestra.run.smithi050.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 31 MB/s | 4.7 MB 00:00 2024-07-04T07:56:37.919 INFO:teuthology.orchestra.run.smithi050.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-07-04T07:56:37.953 INFO:teuthology.orchestra.run.smithi050.stdout:(69/160): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-07-04T07:56:37.970 INFO:teuthology.orchestra.run.smithi050.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-07-04T07:56:38.012 INFO:teuthology.orchestra.run.smithi050.stdout:(71/160): postgresql-13.14-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-07-04T07:56:38.045 INFO:teuthology.orchestra.run.smithi050.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-07-04T07:56:38.062 INFO:teuthology.orchestra.run.smithi050.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-07-04T07:56:38.096 INFO:teuthology.orchestra.run.smithi050.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-07-04T07:56:38.171 INFO:teuthology.orchestra.run.smithi050.stdout:(75/160): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-07-04T07:56:38.197 INFO:teuthology.orchestra.run.smithi050.stdout:(76/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-07-04T07:56:38.222 INFO:teuthology.orchestra.run.smithi050.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-07-04T07:56:38.247 INFO:teuthology.orchestra.run.smithi050.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-07-04T07:56:38.273 INFO:teuthology.orchestra.run.smithi050.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-07-04T07:56:38.298 INFO:teuthology.orchestra.run.smithi050.stdout:(80/160): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-07-04T07:56:38.321 INFO:teuthology.orchestra.run.smithi039.stdout:(1/160): ceph-19.0.0-4737.g2e9f68b5.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-07-04T07:56:38.323 INFO:teuthology.orchestra.run.smithi050.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-07-04T07:56:38.407 INFO:teuthology.orchestra.run.smithi050.stdout:(82/160): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-07-04T07:56:38.432 INFO:teuthology.orchestra.run.smithi050.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-07-04T07:56:38.458 INFO:teuthology.orchestra.run.smithi050.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-07-04T07:56:38.483 INFO:teuthology.orchestra.run.smithi050.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-07-04T07:56:38.508 INFO:teuthology.orchestra.run.smithi050.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-07-04T07:56:38.533 INFO:teuthology.orchestra.run.smithi050.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-07-04T07:56:38.550 INFO:teuthology.orchestra.run.smithi039.stdout:(2/160): ceph-fuse-19.0.0-4737.g2e9f68b5.el9.x8 3.7 MB/s | 872 kB 00:00 2024-07-04T07:56:38.558 INFO:teuthology.orchestra.run.smithi050.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-07-04T07:56:38.584 INFO:teuthology.orchestra.run.smithi050.stdout:(89/160): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-07-04T07:56:38.604 INFO:teuthology.orchestra.run.smithi039.stdout:(3/160): ceph-immutable-object-cache-19.0.0-473 2.7 MB/s | 148 kB 00:00 2024-07-04T07:56:38.609 INFO:teuthology.orchestra.run.smithi050.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-07-04T07:56:38.643 INFO:teuthology.orchestra.run.smithi050.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-07-04T07:56:38.727 INFO:teuthology.orchestra.run.smithi050.stdout:(92/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-07-04T07:56:38.746 INFO:teuthology.orchestra.run.smithi039.stdout:(4/160): ceph-base-19.0.0-4737.g2e9f68b5.el9.x8 7.8 MB/s | 5.4 MB 00:00 2024-07-04T07:56:38.752 INFO:teuthology.orchestra.run.smithi050.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-07-04T07:56:38.777 INFO:teuthology.orchestra.run.smithi050.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-07-04T07:56:38.802 INFO:teuthology.orchestra.run.smithi050.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-07-04T07:56:38.827 INFO:teuthology.orchestra.run.smithi050.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-07-04T07:56:38.830 INFO:teuthology.orchestra.run.smithi039.stdout:(5/160): ceph-mds-19.0.0-4737.g2e9f68b5.el9.x86 10 MB/s | 2.2 MB 00:00 2024-07-04T07:56:38.853 INFO:teuthology.orchestra.run.smithi050.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-07-04T07:56:38.878 INFO:teuthology.orchestra.run.smithi050.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-07-04T07:56:38.903 INFO:teuthology.orchestra.run.smithi050.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-07-04T07:56:38.913 INFO:teuthology.orchestra.run.smithi039.stdout:(6/160): ceph-mgr-19.0.0-4737.g2e9f68b5.el9.x86 9.1 MB/s | 1.5 MB 00:00 2024-07-04T07:56:38.929 INFO:teuthology.orchestra.run.smithi050.stdout:(100/160): xfsprogs-devel-6.4.0-2.el9.x86_64.rp 1.9 MB/s | 95 kB 00:00 2024-07-04T07:56:38.963 INFO:teuthology.orchestra.run.smithi050.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-07-04T07:56:38.988 INFO:teuthology.orchestra.run.smithi050.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 760 kB/s | 45 kB 00:00 2024-07-04T07:56:39.064 INFO:teuthology.orchestra.run.smithi039.stdout:(7/160): ceph-mon-19.0.0-4737.g2e9f68b5.el9.x86 21 MB/s | 4.9 MB 00:00 2024-07-04T07:56:39.213 INFO:teuthology.orchestra.run.smithi050.stdout:(103/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-07-04T07:56:39.339 INFO:teuthology.orchestra.run.smithi050.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 7.9 MB/s | 548 kB 00:00 2024-07-04T07:56:39.372 INFO:teuthology.orchestra.run.smithi050.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 9.8 MB/s | 309 kB 00:00 2024-07-04T07:56:39.397 INFO:teuthology.orchestra.run.smithi050.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 859 kB/s | 19 kB 00:00 2024-07-04T07:56:39.506 INFO:teuthology.orchestra.run.smithi050.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-07-04T07:56:39.531 INFO:teuthology.orchestra.run.smithi050.stdout:(108/160): lua-devel-5.4.4-4.el9.x86_64.rpm 39 kB/s | 22 kB 00:00 2024-07-04T07:56:39.556 INFO:teuthology.orchestra.run.smithi039.stdout:(8/160): ceph-radosgw-19.0.0-4737.g2e9f68b5.el9 21 MB/s | 10 MB 00:00 2024-07-04T07:56:39.556 INFO:teuthology.orchestra.run.smithi050.stdout:(109/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-07-04T07:56:39.606 INFO:teuthology.orchestra.run.smithi039.stdout:(9/160): ceph-selinux-19.0.0-4737.g2e9f68b5.el9 515 kB/s | 25 kB 00:00 2024-07-04T07:56:39.607 INFO:teuthology.orchestra.run.smithi050.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.8 MB/s | 67 kB 00:00 2024-07-04T07:56:39.632 INFO:teuthology.orchestra.run.smithi050.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 6.6 MB/s | 151 kB 00:00 2024-07-04T07:56:39.658 INFO:teuthology.orchestra.run.smithi050.stdout:(112/160): liboath-2.6.7-2.el9.x86_64.rpm 386 kB/s | 49 kB 00:00 2024-07-04T07:56:39.699 INFO:teuthology.orchestra.run.smithi050.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-07-04T07:56:39.725 INFO:teuthology.orchestra.run.smithi050.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-04T07:56:39.750 INFO:teuthology.orchestra.run.smithi050.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-04T07:56:39.775 INFO:teuthology.orchestra.run.smithi050.stdout:(116/160): python3-certifi-2021.10.8-2.el9.noar 601 kB/s | 15 kB 00:00 2024-07-04T07:56:39.801 INFO:teuthology.orchestra.run.smithi050.stdout:(117/160): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-07-04T07:56:39.826 INFO:teuthology.orchestra.run.smithi050.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-07-04T07:56:39.860 INFO:teuthology.orchestra.run.smithi050.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 9.0 MB/s | 303 kB 00:00 2024-07-04T07:56:39.873 INFO:teuthology.orchestra.run.smithi039.stdout:(10/160): ceph-osd-19.0.0-4737.g2e9f68b5.el9.x8 19 MB/s | 18 MB 00:00 2024-07-04T07:56:39.893 INFO:teuthology.orchestra.run.smithi050.stdout:(120/160): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-07-04T07:56:39.919 INFO:teuthology.orchestra.run.smithi050.stdout:(121/160): python3-google-auth-2.30.0-1.el9.noa 3.6 MB/s | 213 kB 00:00 2024-07-04T07:56:39.932 INFO:teuthology.orchestra.run.smithi039.stdout:(11/160): cephfs-mirror-19.0.0-4737.g2e9f68b5.e 3.9 MB/s | 228 kB 00:00 2024-07-04T07:56:39.944 INFO:teuthology.orchestra.run.smithi050.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-07-04T07:56:39.978 INFO:teuthology.orchestra.run.smithi050.stdout:(123/160): protobuf-compiler-3.14.0-13.el9.x86_ 873 kB/s | 863 kB 00:00 2024-07-04T07:56:39.982 INFO:teuthology.orchestra.run.smithi039.stdout:(12/160): cephfs-shell-19.0.0-4737.g2e9f68b5.el 549 kB/s | 27 kB 00:00 2024-07-04T07:56:40.032 INFO:teuthology.orchestra.run.smithi039.stdout:(13/160): libcephfs-devel-19.0.0-4737.g2e9f68b5 648 kB/s | 32 kB 00:00 2024-07-04T07:56:40.044 INFO:teuthology.orchestra.run.smithi050.stdout:(124/160): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-07-04T07:56:40.095 INFO:teuthology.orchestra.run.smithi050.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 1.0 MB/s | 23 kB 00:00 2024-07-04T07:56:40.116 INFO:teuthology.orchestra.run.smithi039.stdout:(14/160): libcephfs2-19.0.0-4737.g2e9f68b5.el9. 9.1 MB/s | 766 kB 00:00 2024-07-04T07:56:40.120 INFO:teuthology.orchestra.run.smithi050.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 869 kB/s | 19 kB 00:00 2024-07-04T07:56:40.145 INFO:teuthology.orchestra.run.smithi050.stdout:(127/160): python3-jaraco-classes-3.2.1-5.el9.n 105 kB/s | 18 kB 00:00 2024-07-04T07:56:40.171 INFO:teuthology.orchestra.run.smithi050.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-07-04T07:56:40.174 INFO:teuthology.orchestra.run.smithi039.stdout:(15/160): libcephsqlite-19.0.0-4737.g2e9f68b5.e 2.7 MB/s | 158 kB 00:00 2024-07-04T07:56:40.205 INFO:teuthology.orchestra.run.smithi050.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.3 MB/s | 46 kB 00:00 2024-07-04T07:56:40.218 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-04T07:56:40.219 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-04T07:56:40.229 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-04T07:56:40.230 INFO:teuthology.orchestra.run.smithi050.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-07-04T07:56:40.230 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-04T07:56:40.231 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-04T07:56:40.232 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-04T07:56:40.233 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-04T07:56:40.233 INFO:teuthology.orchestra.run.smithi039.stdout:(16/160): librados-devel-19.0.0-4737.g2e9f68b5. 2.2 MB/s | 127 kB 00:00 2024-07-04T07:56:40.235 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-04T07:56:40.236 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-04T07:56:40.255 INFO:teuthology.orchestra.run.smithi050.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-07-04T07:56:40.289 INFO:teuthology.orchestra.run.smithi050.stdout:(132/160): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-07-04T07:56:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-07-04T07:56:40.300 INFO:teuthology.orchestra.run.smithi039.stdout:(17/160): libradosstriper1-19.0.0-4737.g2e9f68b 7.5 MB/s | 505 kB 00:00 2024-07-04T07:56:40.314 INFO:teuthology.orchestra.run.smithi050.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-07-04T07:56:40.319 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-07-04T07:56:40.319 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 6.6 k 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 5.4 M 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 872 k 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 148 k 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 1.5 M 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 146 k 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 14 M 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 7.4 M 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 49 k 2024-07-04T07:56:40.320 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 10 M 2024-07-04T07:56:40.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 44 M 2024-07-04T07:56:40.321 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 274 k 2024-07-04T07:56:40.321 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 764 k 2024-07-04T07:56:40.321 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 228 k 2024-07-04T07:56:40.321 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 27 k 2024-07-04T07:56:40.321 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 18 k 2024-07-04T07:56:40.321 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-04T07:56:40.321 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-04T07:56:40.321 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 32 k 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 766 k 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 127 k 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-04T07:56:40.322 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 162 k 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 321 k 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 304 k 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 99 k 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 86 k 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 3.1 M 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 170 k 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 6.4.0-2.el9 appstream 95 k 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-04T07:56:40.323 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 3.3 M 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 3.1 M 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs x86_64 6.4.0-2.el9 baseos 1.1 M 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-04T07:56:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 20 M 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 34 k 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 2.2 M 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 278 k 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 4.9 M 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 18 M 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.0.0-4737.g2e9f68b5.el9 ceph-noarch 16 k 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 25 k 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-04T07:56:40.325 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 158 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 505 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 5.2 M 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-04T07:56:40.326 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-04T07:56:40.327 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 45 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.0.0-4737.g2e9f68b5.el9 ceph 135 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-04T07:56:40.328 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.30.0-1.el9 epel 213 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-04T07:56:40.329 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-04T07:56:40.330 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-04T07:56:40.331 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-04T07:56:40.332 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:56:40.333 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-07-04T07:56:40.334 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-07-04T07:56:40.334 INFO:teuthology.orchestra.run.smithi133.stdout:Install 148 Packages 2024-07-04T07:56:40.334 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 12 Packages 2024-07-04T07:56:40.334 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:56:40.334 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 231 M 2024-07-04T07:56:40.334 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-07-04T07:56:40.339 INFO:teuthology.orchestra.run.smithi050.stdout:(134/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-07-04T07:56:40.365 INFO:teuthology.orchestra.run.smithi050.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-07-04T07:56:40.398 INFO:teuthology.orchestra.run.smithi050.stdout:(136/160): python3-kubernetes-26.1.0-2.el9.noar 4.0 MB/s | 1.0 MB 00:00 2024-07-04T07:56:40.424 INFO:teuthology.orchestra.run.smithi050.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-07-04T07:56:40.449 INFO:teuthology.orchestra.run.smithi050.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-07-04T07:56:40.491 INFO:teuthology.orchestra.run.smithi050.stdout:(139/160): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-07-04T07:56:40.525 INFO:teuthology.orchestra.run.smithi050.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-07-04T07:56:40.550 INFO:teuthology.orchestra.run.smithi050.stdout:(141/160): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-07-04T07:56:40.575 INFO:teuthology.orchestra.run.smithi050.stdout:(142/160): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2024-07-04T07:56:40.600 INFO:teuthology.orchestra.run.smithi050.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-07-04T07:56:40.626 INFO:teuthology.orchestra.run.smithi050.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-07-04T07:56:40.651 INFO:teuthology.orchestra.run.smithi050.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-07-04T07:56:40.676 INFO:teuthology.orchestra.run.smithi050.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-04T07:56:40.684 INFO:teuthology.orchestra.run.smithi039.stdout:(18/160): librgw2-19.0.0-4737.g2e9f68b5.el9.x86 14 MB/s | 5.2 MB 00:00 2024-07-04T07:56:40.710 INFO:teuthology.orchestra.run.smithi050.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-07-04T07:56:40.742 INFO:teuthology.orchestra.run.smithi039.stdout:(19/160): python3-ceph-argparse-19.0.0-4737.g2e 787 kB/s | 45 kB 00:00 2024-07-04T07:56:40.760 INFO:teuthology.orchestra.run.smithi050.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-07-04T07:56:40.801 INFO:teuthology.orchestra.run.smithi039.stdout:(20/160): python3-ceph-common-19.0.0-4737.g2e9f 2.3 MB/s | 135 kB 00:00 2024-07-04T07:56:40.876 INFO:teuthology.orchestra.run.smithi039.stdout:(21/160): python3-cephfs-19.0.0-4737.g2e9f68b5. 2.1 MB/s | 162 kB 00:00 2024-07-04T07:56:40.919 INFO:teuthology.orchestra.run.smithi050.stdout:(149/160): NetworkManager-1.48.2-2.el9.x86_64.r 15 MB/s | 2.3 MB 00:00 2024-07-04T07:56:40.943 INFO:teuthology.orchestra.run.smithi039.stdout:(22/160): python3-rados-19.0.0-4737.g2e9f68b5.e 4.8 MB/s | 321 kB 00:00 2024-07-04T07:56:40.978 INFO:teuthology.orchestra.run.smithi050.stdout:(150/160): librados2-19.0.0-4737.g2e9f68b5.el9. 11 MB/s | 3.3 MB 00:00 2024-07-04T07:56:41.028 INFO:teuthology.orchestra.run.smithi050.stdout:(151/160): NetworkManager-libnm-1.48.2-2.el9.x8 17 MB/s | 1.8 MB 00:00 2024-07-04T07:56:41.053 INFO:teuthology.orchestra.run.smithi050.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 542 kB/s | 41 kB 00:00 2024-07-04T07:56:41.079 INFO:teuthology.orchestra.run.smithi050.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 4.8 MB/s | 247 kB 00:00 2024-07-04T07:56:41.104 INFO:teuthology.orchestra.run.smithi050.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 731 kB/s | 37 kB 00:00 2024-07-04T07:56:41.118 INFO:teuthology.orchestra.run.smithi133.stdout:(1/160): ceph-19.0.0-4737.g2e9f68b5.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-07-04T07:56:41.129 INFO:teuthology.orchestra.run.smithi050.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 4.0 MB/s | 207 kB 00:00 2024-07-04T07:56:41.179 INFO:teuthology.orchestra.run.smithi050.stdout:(156/160): openssl-3.2.2-2.el9.x86_64.rpm 27 MB/s | 1.4 MB 00:00 2024-07-04T07:56:41.194 INFO:teuthology.orchestra.run.smithi039.stdout:(23/160): ceph-common-19.0.0-4737.g2e9f68b5.el9 6.2 MB/s | 20 MB 00:03 2024-07-04T07:56:41.205 INFO:teuthology.orchestra.run.smithi050.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.5 MB/s | 260 kB 00:00 2024-07-04T07:56:41.235 INFO:teuthology.orchestra.run.smithi039.stdout:(24/160): python3-rbd-19.0.0-4737.g2e9f68b5.el9 1.0 MB/s | 304 kB 00:00 2024-07-04T07:56:41.263 INFO:teuthology.orchestra.run.smithi050.stdout:(158/160): librbd1-19.0.0-4737.g2e9f68b5.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-07-04T07:56:41.269 INFO:teuthology.orchestra.run.smithi039.stdout:(25/160): python3-rgw-19.0.0-4737.g2e9f68b5.el9 1.3 MB/s | 99 kB 00:00 2024-07-04T07:56:41.294 INFO:teuthology.orchestra.run.smithi039.stdout:(26/160): rbd-fuse-19.0.0-4737.g2e9f68b5.el9.x8 1.4 MB/s | 86 kB 00:00 2024-07-04T07:56:41.314 INFO:teuthology.orchestra.run.smithi050.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-07-04T07:56:41.327 INFO:teuthology.orchestra.run.smithi133.stdout:(2/160): ceph-base-19.0.0-4737.g2e9f68b5.el9.x8 12 MB/s | 5.4 MB 00:00 2024-07-04T07:56:41.345 INFO:teuthology.orchestra.run.smithi039.stdout:(27/160): rbd-nbd-19.0.0-4737.g2e9f68b5.el9.x86 3.4 MB/s | 170 kB 00:00 2024-07-04T07:56:41.369 INFO:teuthology.orchestra.run.smithi133.stdout:(3/160): ceph-immutable-object-cache-19.0.0-473 3.5 MB/s | 148 kB 00:00 2024-07-04T07:56:41.372 INFO:teuthology.orchestra.run.smithi050.stdout:(160/160): xfsprogs-6.4.0-2.el9.x86_64.rpm 6.5 MB/s | 1.1 MB 00:00 2024-07-04T07:56:41.381 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-07-04T07:56:41.381 INFO:teuthology.orchestra.run.smithi050.stdout:Total 25 MB/s | 231 MB 00:09 2024-07-04T07:56:41.386 INFO:teuthology.orchestra.run.smithi039.stdout:(28/160): ceph-grafana-dashboards-19.0.0-4737.g 831 kB/s | 34 kB 00:00 2024-07-04T07:56:41.470 INFO:teuthology.orchestra.run.smithi039.stdout:(29/160): ceph-mgr-cephadm-19.0.0-4737.g2e9f68b 1.7 MB/s | 146 kB 00:00 2024-07-04T07:56:41.503 INFO:teuthology.orchestra.run.smithi133.stdout:(4/160): ceph-mds-19.0.0-4737.g2e9f68b5.el9.x86 17 MB/s | 2.2 MB 00:00 2024-07-04T07:56:41.603 INFO:teuthology.orchestra.run.smithi133.stdout:(5/160): ceph-mgr-19.0.0-4737.g2e9f68b5.el9.x86 15 MB/s | 1.5 MB 00:00 2024-07-04T07:56:41.670 INFO:teuthology.orchestra.run.smithi039.stdout:(30/160): rbd-mirror-19.0.0-4737.g2e9f68b5.el9. 7.7 MB/s | 3.1 MB 00:00 2024-07-04T07:56:41.720 INFO:teuthology.orchestra.run.smithi133.stdout:(6/160): ceph-fuse-19.0.0-4737.g2e9f68b5.el9.x8 1.4 MB/s | 872 kB 00:00 2024-07-04T07:56:41.870 INFO:teuthology.orchestra.run.smithi133.stdout:(7/160): ceph-mon-19.0.0-4737.g2e9f68b5.el9.x86 18 MB/s | 4.9 MB 00:00 2024-07-04T07:56:42.262 INFO:teuthology.orchestra.run.smithi133.stdout:(8/160): ceph-radosgw-19.0.0-4737.g2e9f68b5.el9 26 MB/s | 10 MB 00:00 2024-07-04T07:56:42.304 INFO:teuthology.orchestra.run.smithi133.stdout:(9/160): ceph-selinux-19.0.0-4737.g2e9f68b5.el9 609 kB/s | 25 kB 00:00 2024-07-04T07:56:42.429 INFO:teuthology.orchestra.run.smithi039.stdout:(31/160): ceph-test-19.0.0-4737.g2e9f68b5.el9.x 15 MB/s | 44 MB 00:02 2024-07-04T07:56:42.529 INFO:teuthology.orchestra.run.smithi039.stdout:(32/160): ceph-mgr-modules-core-19.0.0-4737.g2e 2.8 MB/s | 278 kB 00:00 2024-07-04T07:56:42.630 INFO:teuthology.orchestra.run.smithi039.stdout:(33/160): ceph-mgr-rook-19.0.0-4737.g2e9f68b5.e 496 kB/s | 49 kB 00:00 2024-07-04T07:56:42.649 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-07-04T07:56:42.738 INFO:teuthology.orchestra.run.smithi039.stdout:(34/160): ceph-prometheus-alerts-19.0.0-4737.g2 153 kB/s | 16 kB 00:00 2024-07-04T07:56:42.847 INFO:teuthology.orchestra.run.smithi039.stdout:(35/160): ceph-volume-19.0.0-4737.g2e9f68b5.el9 2.5 MB/s | 274 kB 00:00 2024-07-04T07:56:42.864 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-07-04T07:56:42.864 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-07-04T07:56:43.239 INFO:teuthology.orchestra.run.smithi039.stdout:(36/160): ceph-mgr-dashboard-19.0.0-4737.g2e9f6 7.9 MB/s | 14 MB 00:01 2024-07-04T07:56:43.281 INFO:teuthology.orchestra.run.smithi039.stdout:(37/160): cephadm-19.0.0-4737.g2e9f68b5.el9.noa 1.7 MB/s | 764 kB 00:00 2024-07-04T07:56:43.431 INFO:teuthology.orchestra.run.smithi039.stdout:(38/160): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-07-04T07:56:43.456 INFO:teuthology.orchestra.run.smithi039.stdout:(39/160): cephfs-top-19.0.0-4737.g2e9f68b5.el9. 104 kB/s | 18 kB 00:00 2024-07-04T07:56:43.549 INFO:teuthology.orchestra.run.smithi039.stdout:(40/160): fuse-2.9.9-15.el9.x86_64.rpm 302 kB/s | 80 kB 00:00 2024-07-04T07:56:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 939 kB/s | 195 kB 00:00 2024-07-04T07:56:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-07-04T07:56:43.737 INFO:teuthology.orchestra.run.smithi133.stdout:(10/160): ceph-test-19.0.0-4737.g2e9f68b5.el9.x 30 MB/s | 44 MB 00:01 2024-07-04T07:56:43.758 INFO:teuthology.orchestra.run.smithi039.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-07-04T07:56:43.788 INFO:teuthology.orchestra.run.smithi133.stdout:(11/160): cephfs-mirror-19.0.0-4737.g2e9f68b5.e 4.5 MB/s | 228 kB 00:00 2024-07-04T07:56:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:(44/160): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-07-04T07:56:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-07-04T07:56:43.829 INFO:teuthology.orchestra.run.smithi133.stdout:(12/160): cephfs-shell-19.0.0-4737.g2e9f68b5.el 652 kB/s | 27 kB 00:00 2024-07-04T07:56:43.850 INFO:teuthology.orchestra.run.smithi039.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-07-04T07:56:43.880 INFO:teuthology.orchestra.run.smithi133.stdout:(13/160): libcephfs-devel-19.0.0-4737.g2e9f68b5 635 kB/s | 32 kB 00:00 2024-07-04T07:56:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-07-04T07:56:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:(48/160): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-07-04T07:56:43.950 INFO:teuthology.orchestra.run.smithi133.stdout:(14/160): libcephfs2-19.0.0-4737.g2e9f68b5.el9. 11 MB/s | 766 kB 00:00 2024-07-04T07:56:43.976 INFO:teuthology.orchestra.run.smithi039.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-07-04T07:56:43.997 INFO:teuthology.orchestra.run.smithi133.stdout:(15/160): libcephsqlite-19.0.0-4737.g2e9f68b5.e 3.1 MB/s | 158 kB 00:00 2024-07-04T07:56:44.010 INFO:teuthology.orchestra.run.smithi039.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-07-04T07:56:44.035 INFO:teuthology.orchestra.run.smithi039.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 376 kB/s | 22 kB 00:00 2024-07-04T07:56:44.047 INFO:teuthology.orchestra.run.smithi133.stdout:(16/160): librados-devel-19.0.0-4737.g2e9f68b5. 2.5 MB/s | 127 kB 00:00 2024-07-04T07:56:44.085 INFO:teuthology.orchestra.run.smithi039.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-07-04T07:56:44.098 INFO:teuthology.orchestra.run.smithi133.stdout:(17/160): libradosstriper1-19.0.0-4737.g2e9f68b 9.9 MB/s | 505 kB 00:00 2024-07-04T07:56:44.102 INFO:teuthology.orchestra.run.smithi039.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 494 kB/s | 33 kB 00:00 2024-07-04T07:56:44.136 INFO:teuthology.orchestra.run.smithi039.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-07-04T07:56:44.156 INFO:teuthology.orchestra.run.smithi039.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 367 kB/s | 18 kB 00:00 2024-07-04T07:56:44.186 INFO:teuthology.orchestra.run.smithi039.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 225 kB/s | 11 kB 00:00 2024-07-04T07:56:44.271 INFO:teuthology.orchestra.run.smithi039.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-07-04T07:56:44.290 INFO:teuthology.orchestra.run.smithi133.stdout:(18/160): librgw2-19.0.0-4737.g2e9f68b5.el9.x86 27 MB/s | 5.2 MB 00:00 2024-07-04T07:56:44.328 INFO:teuthology.orchestra.run.smithi039.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 321 kB/s | 45 kB 00:00 2024-07-04T07:56:44.368 INFO:teuthology.orchestra.run.smithi133.stdout:(19/160): python3-ceph-argparse-19.0.0-4737.g2e 577 kB/s | 45 kB 00:00 2024-07-04T07:56:44.395 INFO:teuthology.orchestra.run.smithi039.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-07-04T07:56:44.415 INFO:teuthology.orchestra.run.smithi133.stdout:(20/160): python3-ceph-common-19.0.0-4737.g2e9f 2.8 MB/s | 135 kB 00:00 2024-07-04T07:56:44.437 INFO:teuthology.orchestra.run.smithi039.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 5.3 MB/s | 581 kB 00:00 2024-07-04T07:56:44.454 INFO:teuthology.orchestra.run.smithi039.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-07-04T07:56:44.466 INFO:teuthology.orchestra.run.smithi133.stdout:(21/160): python3-cephfs-19.0.0-4737.g2e9f68b5. 3.2 MB/s | 162 kB 00:00 2024-07-04T07:56:44.488 INFO:teuthology.orchestra.run.smithi039.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-07-04T07:56:44.516 INFO:teuthology.orchestra.run.smithi133.stdout:(22/160): python3-rados-19.0.0-4737.g2e9f68b5.e 6.3 MB/s | 321 kB 00:00 2024-07-04T07:56:44.566 INFO:teuthology.orchestra.run.smithi133.stdout:(23/160): python3-rbd-19.0.0-4737.g2e9f68b5.el9 6.0 MB/s | 304 kB 00:00 2024-07-04T07:56:44.605 INFO:teuthology.orchestra.run.smithi039.stdout:(63/160): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-07-04T07:56:44.608 INFO:teuthology.orchestra.run.smithi133.stdout:(24/160): python3-rgw-19.0.0-4737.g2e9f68b5.el9 2.3 MB/s | 99 kB 00:00 2024-07-04T07:56:44.647 INFO:teuthology.orchestra.run.smithi039.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-07-04T07:56:44.658 INFO:teuthology.orchestra.run.smithi133.stdout:(25/160): rbd-fuse-19.0.0-4737.g2e9f68b5.el9.x8 1.7 MB/s | 86 kB 00:00 2024-07-04T07:56:44.672 INFO:teuthology.orchestra.run.smithi039.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 532 kB/s | 35 kB 00:00 2024-07-04T07:56:44.826 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-07-04T07:56:44.826 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-07-04T07:56:44.834 INFO:teuthology.orchestra.run.smithi133.stdout:(26/160): rbd-mirror-19.0.0-4737.g2e9f68b5.el9. 18 MB/s | 3.1 MB 00:00 2024-07-04T07:56:44.884 INFO:teuthology.orchestra.run.smithi133.stdout:(27/160): rbd-nbd-19.0.0-4737.g2e9f68b5.el9.x86 3.3 MB/s | 170 kB 00:00 2024-07-04T07:56:44.943 INFO:teuthology.orchestra.run.smithi133.stdout:(28/160): ceph-grafana-dashboards-19.0.0-4737.g 586 kB/s | 34 kB 00:00 2024-07-04T07:56:45.135 INFO:teuthology.orchestra.run.smithi133.stdout:(29/160): ceph-common-19.0.0-4737.g2e9f68b5.el9 4.6 MB/s | 20 MB 00:04 2024-07-04T07:56:45.160 INFO:teuthology.orchestra.run.smithi133.stdout:(30/160): ceph-mgr-cephadm-19.0.0-4737.g2e9f68b 671 kB/s | 146 kB 00:00 2024-07-04T07:56:45.273 INFO:teuthology.orchestra.run.smithi039.stdout:(66/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-07-04T07:56:45.644 INFO:teuthology.orchestra.run.smithi133.stdout:(31/160): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-07-04T07:56:45.744 INFO:teuthology.orchestra.run.smithi133.stdout:(32/160): ceph-mgr-modules-core-19.0.0-4737.g2e 2.7 MB/s | 278 kB 00:00 2024-07-04T07:56:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:(33/160): ceph-mgr-rook-19.0.0-4737.g2e9f68b5.e 741 kB/s | 49 kB 00:00 2024-07-04T07:56:45.853 INFO:teuthology.orchestra.run.smithi133.stdout:(34/160): ceph-prometheus-alerts-19.0.0-4737.g2 395 kB/s | 16 kB 00:00 2024-07-04T07:56:45.912 INFO:teuthology.orchestra.run.smithi133.stdout:(35/160): ceph-volume-19.0.0-4737.g2e9f68b5.el9 4.6 MB/s | 274 kB 00:00 2024-07-04T07:56:45.970 INFO:teuthology.orchestra.run.smithi133.stdout:(36/160): cephadm-19.0.0-4737.g2e9f68b5.el9.noa 13 MB/s | 764 kB 00:00 2024-07-04T07:56:46.012 INFO:teuthology.orchestra.run.smithi133.stdout:(37/160): cephfs-top-19.0.0-4737.g2e9f68b5.el9. 439 kB/s | 18 kB 00:00 2024-07-04T07:56:46.140 INFO:teuthology.orchestra.run.smithi039.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-07-04T07:56:46.162 INFO:teuthology.orchestra.run.smithi133.stdout:(38/160): fuse-2.9.9-15.el9.x86_64.rpm 536 kB/s | 80 kB 00:00 2024-07-04T07:56:46.396 INFO:teuthology.orchestra.run.smithi133.stdout:(39/160): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-07-04T07:56:46.480 INFO:teuthology.orchestra.run.smithi133.stdout:(40/160): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-07-04T07:56:46.563 INFO:teuthology.orchestra.run.smithi133.stdout:(41/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-07-04T07:56:46.631 INFO:teuthology.orchestra.run.smithi039.stdout:(68/160): postgresql-contrib-13.14-1.el9.x86_64 1.8 MB/s | 911 kB 00:00 2024-07-04T07:56:46.757 INFO:teuthology.orchestra.run.smithi039.stdout:(69/160): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-07-04T07:56:46.855 INFO:teuthology.orchestra.run.smithi133.stdout:(42/160): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-07-04T07:56:46.906 INFO:teuthology.orchestra.run.smithi133.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-07-04T07:56:46.964 INFO:teuthology.orchestra.run.smithi133.stdout:(44/160): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-07-04T07:56:47.014 INFO:teuthology.orchestra.run.smithi133.stdout:(45/160): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-07-04T07:56:47.099 INFO:teuthology.orchestra.run.smithi039.stdout:(70/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 1.8 MB/s | 4.4 MB 00:02 2024-07-04T07:56:47.115 INFO:teuthology.orchestra.run.smithi133.stdout:(46/160): python3-urllib3-1.26.5-6.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-07-04T07:56:47.257 INFO:teuthology.orchestra.run.smithi039.stdout:(71/160): openblas-openmp-0.3.21-2.el9.x86_64.r 1.8 MB/s | 4.7 MB 00:02 2024-07-04T07:56:47.349 INFO:teuthology.orchestra.run.smithi039.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 959 kB/s | 87 kB 00:00 2024-07-04T07:56:47.724 INFO:teuthology.orchestra.run.smithi039.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 1.6 MB/s | 1.0 MB 00:00 2024-07-04T07:56:47.783 INFO:teuthology.orchestra.run.smithi039.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-07-04T07:56:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:(47/160): automake-1.16.2-8.el9.noarch.rpm 950 kB/s | 689 kB 00:00 2024-07-04T07:56:47.933 INFO:teuthology.orchestra.run.smithi039.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-07-04T07:56:48.040 INFO:teuthology.orchestra.run.smithi133.stdout:(48/160): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-07-04T07:56:48.084 INFO:teuthology.orchestra.run.smithi039.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-07-04T07:56:48.140 INFO:teuthology.orchestra.run.smithi133.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-07-04T07:56:48.142 INFO:teuthology.orchestra.run.smithi039.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 444 kB/s | 26 kB 00:00 2024-07-04T07:56:48.201 INFO:teuthology.orchestra.run.smithi039.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-07-04T07:56:48.233 INFO:teuthology.orchestra.run.smithi133.stdout:(50/160): elfutils-libelf-devel-0.191-4.el9.x86 236 kB/s | 22 kB 00:00 2024-07-04T07:56:48.318 INFO:teuthology.orchestra.run.smithi039.stdout:(79/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-07-04T07:56:48.341 INFO:teuthology.orchestra.run.smithi133.stdout:(51/160): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-07-04T07:56:48.377 INFO:teuthology.orchestra.run.smithi039.stdout:(80/160): python3-markupsafe-1.1.1-12.el9.x86_6 601 kB/s | 35 kB 00:00 2024-07-04T07:56:48.441 INFO:teuthology.orchestra.run.smithi133.stdout:(52/160): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-07-04T07:56:48.675 INFO:teuthology.orchestra.run.smithi133.stdout:(53/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-07-04T07:56:48.884 INFO:teuthology.orchestra.run.smithi133.stdout:(54/160): ceph-mgr-dashboard-19.0.0-4737.g2e9f6 3.7 MB/s | 14 MB 00:03 2024-07-04T07:56:48.909 INFO:teuthology.orchestra.run.smithi133.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 78 kB/s | 18 kB 00:00 2024-07-04T07:56:48.985 INFO:teuthology.orchestra.run.smithi133.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-07-04T07:56:49.010 INFO:teuthology.orchestra.run.smithi133.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-07-04T07:56:49.085 INFO:teuthology.orchestra.run.smithi133.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 453 kB/s | 45 kB 00:00 2024-07-04T07:56:49.127 INFO:teuthology.orchestra.run.smithi133.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-07-04T07:56:49.202 INFO:teuthology.orchestra.run.smithi133.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2024-07-04T07:56:49.227 INFO:teuthology.orchestra.run.smithi133.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-07-04T07:56:49.303 INFO:teuthology.orchestra.run.smithi133.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-07-04T07:56:49.336 INFO:teuthology.orchestra.run.smithi133.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-07-04T07:56:49.404 INFO:teuthology.orchestra.run.smithi133.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-07-04T07:56:49.437 INFO:teuthology.orchestra.run.smithi133.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-07-04T07:56:49.654 INFO:teuthology.orchestra.run.smithi133.stdout:(66/160): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-07-04T07:56:49.796 INFO:teuthology.orchestra.run.smithi133.stdout:(67/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-07-04T07:56:49.829 INFO:teuthology.orchestra.run.smithi133.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 9.2 MB/s | 1.6 MB 00:00 2024-07-04T07:56:49.921 INFO:teuthology.orchestra.run.smithi133.stdout:(69/160): postgresql-contrib-13.14-1.el9.x86_64 7.2 MB/s | 911 kB 00:00 2024-07-04T07:56:49.947 INFO:teuthology.orchestra.run.smithi133.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-07-04T07:56:49.968 INFO:teuthology.orchestra.run.smithi039.stdout:(81/160): postgresql-server-13.14-1.el9.x86_64. 1.8 MB/s | 5.8 MB 00:03 2024-07-04T07:56:50.152 INFO:teuthology.orchestra.run.smithi039.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-07-04T07:56:50.197 INFO:teuthology.orchestra.run.smithi133.stdout:(71/160): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-07-04T07:56:50.230 INFO:teuthology.orchestra.run.smithi133.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-07-04T07:56:50.289 INFO:teuthology.orchestra.run.smithi133.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 967 kB/s | 87 kB 00:00 2024-07-04T07:56:50.310 INFO:teuthology.orchestra.run.smithi039.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-07-04T07:56:50.411 INFO:teuthology.orchestra.run.smithi039.stdout:(84/160): python3-packaging-20.9-5.el9.noarch.r 779 kB/s | 77 kB 00:00 2024-07-04T07:56:50.423 INFO:teuthology.orchestra.run.smithi133.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 459 kB/s | 61 kB 00:00 2024-07-04T07:56:50.461 INFO:teuthology.orchestra.run.smithi039.stdout:(85/160): python3-prettytable-0.7.2-27.el9.noar 854 kB/s | 42 kB 00:00 2024-07-04T07:56:50.498 INFO:teuthology.orchestra.run.smithi133.stdout:(75/160): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-07-04T07:56:50.531 INFO:teuthology.orchestra.run.smithi133.stdout:(76/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-07-04T07:56:50.599 INFO:teuthology.orchestra.run.smithi133.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-07-04T07:56:50.624 INFO:teuthology.orchestra.run.smithi133.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 278 kB/s | 26 kB 00:00 2024-07-04T07:56:50.678 INFO:teuthology.orchestra.run.smithi039.stdout:(86/160): python3-babel-2.9.1-2.el9.noarch.rpm 1.8 MB/s | 6.0 MB 00:03 2024-07-04T07:56:50.716 INFO:teuthology.orchestra.run.smithi133.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-07-04T07:56:50.720 INFO:teuthology.orchestra.run.smithi039.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 1.0 MB/s | 269 kB 00:00 2024-07-04T07:56:50.741 INFO:teuthology.orchestra.run.smithi133.stdout:(80/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-07-04T07:56:50.812 INFO:teuthology.orchestra.run.smithi039.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-07-04T07:56:50.916 INFO:teuthology.orchestra.run.smithi133.stdout:(81/160): ceph-osd-19.0.0-4737.g2e9f68b5.el9.x8 1.9 MB/s | 18 MB 00:09 2024-07-04T07:56:50.920 INFO:teuthology.orchestra.run.smithi039.stdout:(89/160): python3-pytz-2021.1-5.el9.noarch.rpm 474 kB/s | 51 kB 00:00 2024-07-04T07:56:50.933 INFO:teuthology.orchestra.run.smithi133.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 160 kB/s | 35 kB 00:00 2024-07-04T07:56:50.987 INFO:teuthology.orchestra.run.smithi039.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 1.0 MB/s | 279 kB 00:00 2024-07-04T07:56:51.012 INFO:teuthology.orchestra.run.smithi039.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 588 kB/s | 54 kB 00:00 2024-07-04T07:56:51.071 INFO:teuthology.orchestra.run.smithi039.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 717 kB/s | 42 kB 00:00 2024-07-04T07:56:51.084 INFO:teuthology.orchestra.run.smithi133.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-07-04T07:56:51.109 INFO:teuthology.orchestra.run.smithi133.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-07-04T07:56:51.130 INFO:teuthology.orchestra.run.smithi039.stdout:(93/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 768 kB/s | 44 kB 00:00 2024-07-04T07:56:51.134 INFO:teuthology.orchestra.run.smithi133.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-07-04T07:56:51.176 INFO:teuthology.orchestra.run.smithi133.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 845 kB/s | 77 kB 00:00 2024-07-04T07:56:51.235 INFO:teuthology.orchestra.run.smithi133.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-07-04T07:56:51.268 INFO:teuthology.orchestra.run.smithi133.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-07-04T07:56:51.294 INFO:teuthology.orchestra.run.smithi133.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-07-04T07:56:51.330 INFO:teuthology.orchestra.run.smithi039.stdout:(94/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.7 MB/s | 5.1 MB 00:02 2024-07-04T07:56:51.344 INFO:teuthology.orchestra.run.smithi133.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-07-04T07:56:51.356 INFO:teuthology.orchestra.run.smithi039.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 983 kB/s | 221 kB 00:00 2024-07-04T07:56:51.386 INFO:teuthology.orchestra.run.smithi133.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-07-04T07:56:51.389 INFO:teuthology.orchestra.run.smithi039.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 622 kB/s | 36 kB 00:00 2024-07-04T07:56:51.411 INFO:teuthology.orchestra.run.smithi133.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 458 kB/s | 54 kB 00:00 2024-07-04T07:56:51.414 INFO:teuthology.orchestra.run.smithi039.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 818 kB/s | 47 kB 00:00 2024-07-04T07:56:51.427 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-07-04T07:56:51.490 INFO:teuthology.orchestra.run.smithi039.stdout:(98/160): uuid-1.6.2-55.el9.x86_64.rpm 778 kB/s | 58 kB 00:00 2024-07-04T07:56:51.510 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-04T07:56:51.536 INFO:teuthology.orchestra.run.smithi133.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 277 kB/s | 42 kB 00:00 2024-07-04T07:56:51.549 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-04T07:56:51.562 INFO:teuthology.orchestra.run.smithi133.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 294 kB/s | 44 kB 00:00 2024-07-04T07:56:51.582 INFO:teuthology.orchestra.run.smithi039.stdout:(99/160): xfsprogs-devel-6.4.0-2.el9.x86_64.rpm 1.0 MB/s | 95 kB 00:00 2024-07-04T07:56:51.603 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-04T07:56:51.615 INFO:teuthology.orchestra.run.smithi039.stdout:(100/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-07-04T07:56:51.636 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-04T07:56:51.653 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-04T07:56:51.674 INFO:teuthology.orchestra.run.smithi039.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 696 kB/s | 64 kB 00:00 2024-07-04T07:56:51.687 INFO:teuthology.orchestra.run.smithi133.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 292 kB/s | 36 kB 00:00 2024-07-04T07:56:51.697 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-04T07:56:51.699 INFO:teuthology.orchestra.run.smithi039.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-07-04T07:56:51.787 INFO:teuthology.orchestra.run.smithi133.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 881 kB/s | 221 kB 00:00 2024-07-04T07:56:51.816 INFO:teuthology.orchestra.run.smithi039.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 158 kB/s | 22 kB 00:00 2024-07-04T07:56:51.837 INFO:teuthology.orchestra.run.smithi133.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 319 kB/s | 47 kB 00:00 2024-07-04T07:56:51.967 INFO:teuthology.orchestra.run.smithi039.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 3.6 MB/s | 548 kB 00:00 2024-07-04T07:56:51.996 INFO:teuthology.orchestra.run.smithi133.stdout:(98/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-07-04T07:56:52.017 INFO:teuthology.orchestra.run.smithi039.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 6.1 MB/s | 309 kB 00:00 2024-07-04T07:56:52.034 INFO:teuthology.orchestra.run.smithi039.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-07-04T07:56:52.046 INFO:teuthology.orchestra.run.smithi133.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-07-04T07:56:52.071 INFO:teuthology.orchestra.run.smithi133.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 249 kB/s | 58 kB 00:00 2024-07-04T07:56:52.091 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-04T07:56:52.096 INFO:teuthology.orchestra.run.smithi133.stdout:(101/160): xfsprogs-devel-6.4.0-2.el9.x86_64.rp 959 kB/s | 95 kB 00:00 2024-07-04T07:56:52.164 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 8/172 2024-07-04T07:56:52.172 INFO:teuthology.orchestra.run.smithi133.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 635 kB/s | 64 kB 00:00 2024-07-04T07:56:52.197 INFO:teuthology.orchestra.run.smithi133.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 357 kB/s | 45 kB 00:00 2024-07-04T07:56:52.201 INFO:teuthology.orchestra.run.smithi039.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-07-04T07:56:52.218 INFO:teuthology.orchestra.run.smithi039.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.6 MB/s | 26 kB 00:00 2024-07-04T07:56:52.222 INFO:teuthology.orchestra.run.smithi133.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-07-04T07:56:52.235 INFO:teuthology.orchestra.run.smithi039.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 3.0 MB/s | 49 kB 00:00 2024-07-04T07:56:52.260 INFO:teuthology.orchestra.run.smithi039.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-07-04T07:56:52.277 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 8/172 2024-07-04T07:56:52.293 INFO:teuthology.orchestra.run.smithi039.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.6 MB/s | 151 kB 00:00 2024-07-04T07:56:52.307 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 9/172 2024-07-04T07:56:52.314 INFO:teuthology.orchestra.run.smithi133.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 5.9 MB/s | 548 kB 00:00 2024-07-04T07:56:52.340 INFO:teuthology.orchestra.run.smithi133.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-07-04T07:56:52.352 INFO:teuthology.orchestra.run.smithi039.stdout:(112/160): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-07-04T07:56:52.373 INFO:teuthology.orchestra.run.smithi133.stdout:(107/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-07-04T07:56:52.400 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 9/172 2024-07-04T07:56:52.407 INFO:teuthology.orchestra.run.smithi133.stdout:(108/160): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-07-04T07:56:52.428 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.0.0-4737.g2e9f68b5.el9.x86_64 10/172 2024-07-04T07:56:52.432 INFO:teuthology.orchestra.run.smithi133.stdout:(109/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 442 kB/s | 26 kB 00:00 2024-07-04T07:56:52.462 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-04T07:56:52.477 INFO:teuthology.orchestra.run.smithi039.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-07-04T07:56:52.482 INFO:teuthology.orchestra.run.smithi133.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-07-04T07:56:52.503 INFO:teuthology.orchestra.run.smithi039.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-04T07:56:52.516 INFO:teuthology.orchestra.run.smithi133.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.8 MB/s | 151 kB 00:00 2024-07-04T07:56:52.528 INFO:teuthology.orchestra.run.smithi039.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-04T07:56:52.541 INFO:teuthology.orchestra.run.smithi133.stdout:(112/160): liboath-2.6.7-2.el9.x86_64.rpm 362 kB/s | 49 kB 00:00 2024-07-04T07:56:52.553 INFO:teuthology.orchestra.run.smithi039.stdout:(116/160): python3-certifi-2021.10.8-2.el9.noar 622 kB/s | 15 kB 00:00 2024-07-04T07:56:52.583 INFO:teuthology.orchestra.run.smithi133.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-07-04T07:56:52.595 INFO:teuthology.orchestra.run.smithi039.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 4.1 MB/s | 172 kB 00:00 2024-07-04T07:56:52.608 INFO:teuthology.orchestra.run.smithi133.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-04T07:56:52.634 INFO:teuthology.orchestra.run.smithi133.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-04T07:56:52.637 INFO:teuthology.orchestra.run.smithi039.stdout:(118/160): python3-asyncssh-2.13.2-5.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-07-04T07:56:52.659 INFO:teuthology.orchestra.run.smithi133.stdout:(116/160): python3-certifi-2021.10.8-2.el9.noar 596 kB/s | 15 kB 00:00 2024-07-04T07:56:52.661 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-04T07:56:52.679 INFO:teuthology.orchestra.run.smithi039.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-07-04T07:56:52.685 INFO:teuthology.orchestra.run.smithi133.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-07-04T07:56:52.710 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-04T07:56:52.713 INFO:teuthology.orchestra.run.smithi039.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-07-04T07:56:52.718 INFO:teuthology.orchestra.run.smithi133.stdout:(118/160): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-07-04T07:56:52.738 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 14/172 2024-07-04T07:56:52.743 INFO:teuthology.orchestra.run.smithi133.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-07-04T07:56:52.746 INFO:teuthology.orchestra.run.smithi039.stdout:(121/160): python3-google-auth-2.30.0-1.el9.noa 3.1 MB/s | 213 kB 00:00 2024-07-04T07:56:52.769 INFO:teuthology.orchestra.run.smithi133.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-07-04T07:56:52.780 INFO:teuthology.orchestra.run.smithi039.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-07-04T07:56:52.794 INFO:teuthology.orchestra.run.smithi133.stdout:(121/160): python3-google-auth-2.30.0-1.el9.noa 4.1 MB/s | 213 kB 00:00 2024-07-04T07:56:52.806 INFO:teuthology.orchestra.run.smithi039.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 440 kB/s | 11 kB 00:00 2024-07-04T07:56:52.831 INFO:teuthology.orchestra.run.smithi039.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 728 kB/s | 18 kB 00:00 2024-07-04T07:56:52.838 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 14/172 2024-07-04T07:56:52.856 INFO:teuthology.orchestra.run.smithi039.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 961 kB/s | 23 kB 00:00 2024-07-04T07:56:52.864 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86 15/172 2024-07-04T07:56:52.869 INFO:teuthology.orchestra.run.smithi133.stdout:(122/160): libarrow-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-07-04T07:56:52.881 INFO:teuthology.orchestra.run.smithi039.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 803 kB/s | 19 kB 00:00 2024-07-04T07:56:52.895 INFO:teuthology.orchestra.run.smithi133.stdout:(123/160): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-07-04T07:56:52.907 INFO:teuthology.orchestra.run.smithi039.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 815 kB/s | 20 kB 00:00 2024-07-04T07:56:52.910 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86 15/172 2024-07-04T07:56:52.912 INFO:teuthology.orchestra.run.smithi133.stdout:(124/160): python3-jaraco-8.2.1-3.el9.noarch.rp 254 kB/s | 11 kB 00:00 2024-07-04T07:56:52.928 INFO:teuthology.orchestra.run.smithi133.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 527 kB/s | 18 kB 00:00 2024-07-04T07:56:52.932 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-04T07:56:52.946 INFO:teuthology.orchestra.run.smithi133.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 691 kB/s | 23 kB 00:00 2024-07-04T07:56:52.953 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-04T07:56:52.962 INFO:teuthology.orchestra.run.smithi133.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 579 kB/s | 19 kB 00:00 2024-07-04T07:56:52.987 INFO:teuthology.orchestra.run.smithi133.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 470 kB/s | 20 kB 00:00 2024-07-04T07:56:52.999 INFO:teuthology.orchestra.run.smithi039.stdout:(128/160): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-07-04T07:56:53.004 INFO:teuthology.orchestra.run.smithi133.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-07-04T07:56:53.024 INFO:teuthology.orchestra.run.smithi039.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.9 MB/s | 46 kB 00:00 2024-07-04T07:56:53.030 INFO:teuthology.orchestra.run.smithi133.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-07-04T07:56:53.049 INFO:teuthology.orchestra.run.smithi039.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.2 MB/s | 79 kB 00:00 2024-07-04T07:56:53.064 INFO:teuthology.orchestra.run.smithi133.stdout:(131/160): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-07-04T07:56:53.083 INFO:teuthology.orchestra.run.smithi039.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 1.8 MB/s | 58 kB 00:00 2024-07-04T07:56:53.089 INFO:teuthology.orchestra.run.smithi133.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 984 kB/s | 58 kB 00:00 2024-07-04T07:56:53.114 INFO:teuthology.orchestra.run.smithi133.stdout:(133/160): python3-portend-3.1.0-2.el9.noarch.r 653 kB/s | 16 kB 00:00 2024-07-04T07:56:53.141 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-04T07:56:53.148 INFO:teuthology.orchestra.run.smithi133.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-07-04T07:56:53.158 INFO:teuthology.orchestra.run.smithi039.stdout:(132/160): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-07-04T07:56:53.173 INFO:teuthology.orchestra.run.smithi133.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-07-04T07:56:53.198 INFO:teuthology.orchestra.run.smithi133.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-07-04T07:56:53.200 INFO:teuthology.orchestra.run.smithi039.stdout:(133/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-07-04T07:56:53.224 INFO:teuthology.orchestra.run.smithi133.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-07-04T07:56:53.225 INFO:teuthology.orchestra.run.smithi039.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-07-04T07:56:53.236 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-04T07:56:53.251 INFO:teuthology.orchestra.run.smithi039.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-07-04T07:56:53.265 INFO:teuthology.orchestra.run.smithi133.stdout:(138/160): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-07-04T07:56:53.276 INFO:teuthology.orchestra.run.smithi039.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-07-04T07:56:53.291 INFO:teuthology.orchestra.run.smithi133.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 887 kB/s | 59 kB 00:00 2024-07-04T07:56:53.301 INFO:teuthology.orchestra.run.smithi039.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 623 kB/s | 31 kB 00:00 2024-07-04T07:56:53.316 INFO:teuthology.orchestra.run.smithi133.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 709 kB/s | 36 kB 00:00 2024-07-04T07:56:53.335 INFO:teuthology.orchestra.run.smithi039.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-07-04T07:56:53.341 INFO:teuthology.orchestra.run.smithi133.stdout:(141/160): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-07-04T07:56:53.360 INFO:teuthology.orchestra.run.smithi039.stdout:(139/160): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-07-04T07:56:53.366 INFO:teuthology.orchestra.run.smithi133.stdout:(142/160): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-07-04T07:56:53.392 INFO:teuthology.orchestra.run.smithi133.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-07-04T07:56:53.394 INFO:teuthology.orchestra.run.smithi039.stdout:(140/160): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-07-04T07:56:53.417 INFO:teuthology.orchestra.run.smithi133.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-07-04T07:56:53.419 INFO:teuthology.orchestra.run.smithi039.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 881 kB/s | 51 kB 00:00 2024-07-04T07:56:53.442 INFO:teuthology.orchestra.run.smithi133.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-07-04T07:56:53.453 INFO:teuthology.orchestra.run.smithi039.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 2.7 MB/s | 90 kB 00:00 2024-07-04T07:56:53.468 INFO:teuthology.orchestra.run.smithi133.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-04T07:56:53.486 INFO:teuthology.orchestra.run.smithi039.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-07-04T07:56:53.493 INFO:teuthology.orchestra.run.smithi133.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-07-04T07:56:53.512 INFO:teuthology.orchestra.run.smithi039.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 827 kB/s | 20 kB 00:00 2024-07-04T07:56:53.535 INFO:teuthology.orchestra.run.smithi133.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-07-04T07:56:53.553 INFO:teuthology.orchestra.run.smithi039.stdout:(145/160): re2-20211101-3.el9.x86_64.rpm 4.6 MB/s | 192 kB 00:00 2024-07-04T07:56:53.555 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : xfsprogs-6.4.0-2.el9.x86_64 18/172 2024-07-04T07:56:53.604 INFO:teuthology.orchestra.run.smithi039.stdout:(146/160): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-07-04T07:56:53.651 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-04T07:56:53.704 INFO:teuthology.orchestra.run.smithi039.stdout:(147/160): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-07-04T07:56:53.712 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-04T07:56:53.802 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T07:56:53.810 INFO:teuthology.orchestra.run.smithi133.stdout:(149/160): librbd1-19.0.0-4737.g2e9f68b5.el9.x8 9.9 MB/s | 3.1 MB 00:00 2024-07-04T07:56:53.871 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T07:56:53.994 INFO:teuthology.orchestra.run.smithi133.stdout:(150/160): NetworkManager-1.48.2-2.el9.x86_64.r 5.0 MB/s | 2.3 MB 00:00 2024-07-04T07:56:54.044 INFO:teuthology.orchestra.run.smithi133.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 818 kB/s | 41 kB 00:00 2024-07-04T07:56:54.070 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T07:56:54.103 INFO:teuthology.orchestra.run.smithi133.stdout:(152/160): NetworkManager-tui-1.48.2-2.el9.x86_ 4.1 MB/s | 247 kB 00:00 2024-07-04T07:56:54.121 INFO:teuthology.orchestra.run.smithi039.stdout:(148/160): librados2-19.0.0-4737.g2e9f68b5.el9. 6.5 MB/s | 3.3 MB 00:00 2024-07-04T07:56:54.145 INFO:teuthology.orchestra.run.smithi133.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 879 kB/s | 37 kB 00:00 2024-07-04T07:56:54.181 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-04T07:56:54.203 INFO:teuthology.orchestra.run.smithi133.stdout:(154/160): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-07-04T07:56:54.209 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi039.stdout:(149/160): librbd1-19.0.0-4737.g2e9f68b5.el9.x8 5.9 MB/s | 3.1 MB 00:00 2024-07-04T07:56:54.245 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-04T07:56:54.262 INFO:teuthology.orchestra.run.smithi133.stdout:(155/160): elfutils-libs-0.191-4.el9.x86_64.rpm 4.3 MB/s | 260 kB 00:00 2024-07-04T07:56:54.320 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-04T07:56:54.371 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-04T07:56:54.411 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-04T07:56:54.437 INFO:teuthology.orchestra.run.smithi133.stdout:(156/160): openssl-3.2.2-2.el9.x86_64.rpm 7.7 MB/s | 1.4 MB 00:00 2024-07-04T07:56:54.455 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-04T07:56:54.479 INFO:teuthology.orchestra.run.smithi133.stdout:(157/160): NetworkManager-libnm-1.48.2-2.el9.x8 2.7 MB/s | 1.8 MB 00:00 2024-07-04T07:56:54.512 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-04T07:56:54.540 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-04T07:56:54.613 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-04T07:56:54.673 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-04T07:56:54.680 INFO:teuthology.orchestra.run.smithi133.stdout:(158/160): librados2-19.0.0-4737.g2e9f68b5.el9. 2.8 MB/s | 3.3 MB 00:01 2024-07-04T07:56:54.701 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-04T07:56:54.730 INFO:teuthology.orchestra.run.smithi133.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 8.2 MB/s | 2.4 MB 00:00 2024-07-04T07:56:54.745 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-04T07:56:54.780 INFO:teuthology.orchestra.run.smithi039.stdout:(150/160): NetworkManager-1.48.2-2.el9.x86_64.r 3.5 MB/s | 2.3 MB 00:00 2024-07-04T07:56:54.824 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-04T07:56:54.856 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.0.0-4737.g2e9f68b5.el9. 35/172 2024-07-04T07:56:54.871 INFO:teuthology.orchestra.run.smithi133.stdout:(160/160): xfsprogs-6.4.0-2.el9.x86_64.rpm 2.8 MB/s | 1.1 MB 00:00 2024-07-04T07:56:54.876 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-07-04T07:56:54.877 INFO:teuthology.orchestra.run.smithi133.stdout:Total 16 MB/s | 231 MB 00:14 2024-07-04T07:56:54.880 INFO:teuthology.orchestra.run.smithi039.stdout:(151/160): NetworkManager-libnm-1.48.2-2.el9.x8 2.8 MB/s | 1.8 MB 00:00 2024-07-04T07:56:54.900 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.0.0-4737.g2e9f68b5.el 36/172 2024-07-04T07:56:54.906 INFO:teuthology.orchestra.run.smithi039.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 328 kB/s | 41 kB 00:00 2024-07-04T07:56:54.936 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.0.0-4737.g2e9f68b5.el9.x86_6 37/172 2024-07-04T07:56:54.964 INFO:teuthology.orchestra.run.smithi039.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 637 kB/s | 37 kB 00:00 2024-07-04T07:56:55.010 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-04T07:56:55.052 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-04T07:56:55.056 INFO:teuthology.orchestra.run.smithi039.stdout:(154/160): NetworkManager-tui-1.48.2-2.el9.x86_ 1.4 MB/s | 247 kB 00:00 2024-07-04T07:56:55.115 INFO:teuthology.orchestra.run.smithi039.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.4 MB/s | 207 kB 00:00 2024-07-04T07:56:55.130 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-04T07:56:55.265 INFO:teuthology.orchestra.run.smithi039.stdout:(156/160): elfutils-libs-0.191-4.el9.x86_64.rpm 1.2 MB/s | 260 kB 00:00 2024-07-04T07:56:55.277 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-04T07:56:55.338 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-04T07:56:55.907 INFO:teuthology.orchestra.run.smithi039.stdout:(157/160): openssl-3.2.2-2.el9.x86_64.rpm 1.7 MB/s | 1.4 MB 00:00 2024-07-04T07:56:55.913 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-04T07:56:55.994 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-04T07:56:56.037 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-04T07:56:56.073 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-04T07:56:56.123 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-07-04T07:56:56.123 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-04T07:56:56.161 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_6 48/172 2024-07-04T07:56:56.204 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-04T07:56:56.317 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-07-04T07:56:56.317 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-07-04T07:56:56.357 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-04T07:56:56.431 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-04T07:56:56.507 INFO:teuthology.orchestra.run.smithi039.stdout:(158/160): xfsprogs-6.4.0-2.el9.x86_64.rpm 1.8 MB/s | 1.1 MB 00:00 2024-07-04T07:56:56.526 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-04T07:56:56.574 INFO:teuthology.orchestra.run.smithi039.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 1.8 MB/s | 2.4 MB 00:01 2024-07-04T07:56:56.589 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-04T07:56:56.663 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-04T07:56:56.699 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-04T07:56:56.736 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-04T07:56:56.783 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-04T07:56:56.818 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-04T07:56:56.862 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-04T07:56:56.958 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-04T07:56:57.166 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-04T07:56:57.214 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-04T07:56:57.256 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 63/172 2024-07-04T07:56:57.358 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-04T07:56:57.419 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 65/172 2024-07-04T07:56:57.584 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-04T07:56:57.755 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-04T07:56:57.779 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-04T07:56:57.859 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-04T07:56:57.987 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-04T07:56:58.020 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-04T07:56:58.040 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-04T07:56:58.074 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-04T07:56:58.114 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-04T07:56:58.154 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-04T07:56:58.165 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-07-04T07:56:58.165 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-07-04T07:56:58.183 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-04T07:56:58.218 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-04T07:56:58.251 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-04T07:56:58.293 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-04T07:56:58.327 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-04T07:56:58.417 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-04T07:56:58.475 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-04T07:56:58.514 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-04T07:56:58.623 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-04T07:56:58.674 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-04T07:56:59.383 INFO:teuthology.orchestra.run.smithi039.stdout:(160/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.9 MB/s | 16 MB 00:08 2024-07-04T07:56:59.394 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-07-04T07:56:59.394 INFO:teuthology.orchestra.run.smithi039.stdout:Total 11 MB/s | 231 MB 00:21 2024-07-04T07:56:59.440 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-04T07:56:59.496 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-04T07:56:59.529 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-04T07:56:59.635 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-04T07:56:59.694 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-04T07:56:59.740 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-04T07:56:59.829 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-04T07:56:59.865 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-04T07:57:00.051 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-04T07:57:00.080 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-04T07:57:00.160 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-04T07:57:00.848 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-04T07:57:00.975 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-04T07:57:02.285 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-04T07:57:02.360 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-04T07:57:02.388 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-04T07:57:02.421 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-04T07:57:02.761 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-04T07:57:02.809 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 104/172 2024-07-04T07:57:02.879 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-07-04T07:57:02.901 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 104/172 2024-07-04T07:57:02.931 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 105/172 2024-07-04T07:57:02.967 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-04T07:57:03.193 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-07-04T07:57:03.193 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-07-04T07:57:03.486 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-04T07:57:03.520 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 108/172 2024-07-04T07:57:03.609 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 108/172 2024-07-04T07:57:03.655 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 109/172 2024-07-04T07:57:04.464 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-07-04T07:57:04.592 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-04T07:57:04.799 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-04T07:57:04.862 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-04T07:57:04.903 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-04T07:57:04.927 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-04T07:57:04.972 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-04T07:57:05.308 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-04T07:57:05.428 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 8/172 2024-07-04T07:57:05.553 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 8/172 2024-07-04T07:57:05.579 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 9/172 2024-07-04T07:57:05.671 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 9/172 2024-07-04T07:57:05.700 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.0.0-4737.g2e9f68b5.el9.x86_64 10/172 2024-07-04T07:57:05.732 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-04T07:57:05.904 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-07-04T07:57:05.904 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-07-04T07:57:05.923 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-04T07:57:05.982 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-04T07:57:06.010 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 14/172 2024-07-04T07:57:06.015 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 110/172 2024-07-04T07:57:06.052 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 110/172 2024-07-04T07:57:06.098 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 14/172 2024-07-04T07:57:06.127 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86 15/172 2024-07-04T07:57:06.189 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86 15/172 2024-07-04T07:57:06.212 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-04T07:57:06.235 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-04T07:57:06.428 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-04T07:57:06.526 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-04T07:57:06.644 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 110/172 2024-07-04T07:57:06.681 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 111/172 2024-07-04T07:57:06.715 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : xfsprogs-6.4.0-2.el9.x86_64 18/172 2024-07-04T07:57:06.779 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 111/172 2024-07-04T07:57:06.779 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-04T07:57:06.779 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-04T07:57:06.779 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:06.807 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 112/172 2024-07-04T07:57:06.869 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-04T07:57:06.930 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-04T07:57:07.034 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T07:57:07.131 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T07:57:07.330 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T07:57:07.441 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-04T07:57:07.477 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-04T07:57:07.520 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-04T07:57:07.595 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-04T07:57:07.648 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-04T07:57:07.688 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-04T07:57:07.732 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-04T07:57:07.764 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-04T07:57:07.793 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-04T07:57:07.863 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-04T07:57:07.916 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-04T07:57:07.955 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-04T07:57:07.999 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-04T07:57:08.075 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-04T07:57:08.109 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.0.0-4737.g2e9f68b5.el9. 35/172 2024-07-04T07:57:08.153 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.0.0-4737.g2e9f68b5.el 36/172 2024-07-04T07:57:08.181 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.0.0-4737.g2e9f68b5.el9.x86_6 37/172 2024-07-04T07:57:08.250 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-04T07:57:08.296 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-04T07:57:08.369 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-04T07:57:08.502 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-04T07:57:08.556 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-04T07:57:09.076 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-04T07:57:09.152 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-04T07:57:09.181 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-04T07:57:09.201 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-04T07:57:09.250 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-04T07:57:09.288 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_6 48/172 2024-07-04T07:57:09.331 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-04T07:57:09.489 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-04T07:57:09.541 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-04T07:57:09.635 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-04T07:57:09.698 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-04T07:57:09.770 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-04T07:57:09.802 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-04T07:57:09.839 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-04T07:57:09.885 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-04T07:57:09.922 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-04T07:57:10.041 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-04T07:57:10.183 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-04T07:57:10.320 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-04T07:57:10.367 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-04T07:57:10.416 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 63/172 2024-07-04T07:57:10.517 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-04T07:57:10.550 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 65/172 2024-07-04T07:57:10.590 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-04T07:57:10.615 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-04T07:57:10.641 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-04T07:57:10.713 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-04T07:57:10.841 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-04T07:57:10.873 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-04T07:57:10.900 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-04T07:57:10.935 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-04T07:57:10.974 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-04T07:57:11.006 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-04T07:57:11.035 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-04T07:57:11.070 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-04T07:57:11.103 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-04T07:57:11.145 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-04T07:57:11.170 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-04T07:57:11.257 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-04T07:57:11.300 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-04T07:57:11.332 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-04T07:57:11.435 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-04T07:57:11.483 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-04T07:57:12.203 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-04T07:57:12.247 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-04T07:57:12.281 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-04T07:57:12.378 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-04T07:57:12.436 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-04T07:57:12.480 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-04T07:57:12.567 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-04T07:57:12.608 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-04T07:57:12.784 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-04T07:57:12.814 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-04T07:57:12.892 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-04T07:57:13.004 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-07-04T07:57:13.098 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-04T07:57:13.185 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-04T07:57:13.293 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-04T07:57:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-04T07:57:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-04T07:57:13.490 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-04T07:57:13.534 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-04T07:57:13.654 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-04T07:57:13.927 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-04T07:57:14.039 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 8/172 2024-07-04T07:57:14.256 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 8/172 2024-07-04T07:57:14.281 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 9/172 2024-07-04T07:57:14.428 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 9/172 2024-07-04T07:57:14.466 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:19.0.0-4737.g2e9f68b5.el9.x86_64 10/172 2024-07-04T07:57:14.543 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-04T07:57:14.865 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-04T07:57:14.876 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-04T07:57:14.929 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-04T07:57:14.959 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-04T07:57:14.987 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-04T07:57:15.020 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-04T07:57:15.161 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 14/172 2024-07-04T07:57:15.341 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-04T07:57:15.362 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 14/172 2024-07-04T07:57:15.387 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86 15/172 2024-07-04T07:57:15.391 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 104/172 2024-07-04T07:57:15.466 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 104/172 2024-07-04T07:57:15.497 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 105/172 2024-07-04T07:57:15.532 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-04T07:57:15.625 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86 15/172 2024-07-04T07:57:15.655 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-04T07:57:15.727 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-04T07:57:16.023 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-04T07:57:16.041 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-04T07:57:16.070 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 108/172 2024-07-04T07:57:16.124 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 108/172 2024-07-04T07:57:16.146 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 109/172 2024-07-04T07:57:16.210 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-04T07:57:16.670 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : xfsprogs-6.4.0-2.el9.x86_64 18/172 2024-07-04T07:57:16.863 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-04T07:57:16.952 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-04T07:57:17.171 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T07:57:17.258 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T07:57:17.618 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-04T07:57:17.816 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-04T07:57:17.855 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-04T07:57:17.912 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-04T07:57:18.033 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-04T07:57:18.103 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-04T07:57:18.163 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-04T07:57:18.240 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-04T07:57:18.294 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-04T07:57:18.330 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-04T07:57:18.375 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 110/172 2024-07-04T07:57:18.401 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 110/172 2024-07-04T07:57:18.453 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-04T07:57:18.531 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-04T07:57:18.573 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-04T07:57:18.634 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-04T07:57:18.753 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-04T07:57:18.779 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 112/172 2024-07-04T07:57:18.779 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-07-04T07:57:18.779 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-07-04T07:57:18.779 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-07-04T07:57:18.780 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-07-04T07:57:18.780 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-07-04T07:57:18.780 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-07-04T07:57:18.780 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-07-04T07:57:18.780 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:18.793 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:19.0.0-4737.g2e9f68b5.el9. 35/172 2024-07-04T07:57:18.849 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:19.0.0-4737.g2e9f68b5.el 36/172 2024-07-04T07:57:18.883 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:19.0.0-4737.g2e9f68b5.el9.x86_6 37/172 2024-07-04T07:57:18.957 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 113/172 2024-07-04T07:57:18.964 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 110/172 2024-07-04T07:57:18.974 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 113/172 2024-07-04T07:57:18.974 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:18.974 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-04T07:57:18.975 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:19.004 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 111/172 2024-07-04T07:57:19.042 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-04T07:57:19.090 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 111/172 2024-07-04T07:57:19.090 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-04T07:57:19.090 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-04T07:57:19.090 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:19.096 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-04T07:57:19.114 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 112/172 2024-07-04T07:57:19.217 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-04T07:57:19.402 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-04T07:57:19.464 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-04T07:57:20.198 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-04T07:57:20.307 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-04T07:57:20.348 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-04T07:57:20.386 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-04T07:57:20.444 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-04T07:57:20.499 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_6 48/172 2024-07-04T07:57:20.573 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-04T07:57:20.610 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 114/172 2024-07-04T07:57:20.637 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 114/172 2024-07-04T07:57:20.637 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:20.638 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-04T07:57:20.638 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-04T07:57:20.638 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-04T07:57:20.638 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:20.809 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-04T07:57:20.869 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 115/172 2024-07-04T07:57:20.884 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-04T07:57:20.893 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 115/172 2024-07-04T07:57:20.893 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:20.893 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-04T07:57:20.893 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-04T07:57:20.893 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-04T07:57:20.894 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:21.020 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-04T07:57:21.099 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-04T07:57:21.195 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-04T07:57:21.268 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-04T07:57:21.308 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-04T07:57:21.378 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 116/172 2024-07-04T07:57:21.380 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-04T07:57:21.402 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 116/172 2024-07-04T07:57:21.403 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:21.403 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-04T07:57:21.403 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-04T07:57:21.403 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-04T07:57:21.403 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:21.425 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-04T07:57:21.491 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-04T07:57:21.509 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-04T07:57:21.556 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-04T07:57:21.568 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-04T07:57:21.631 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-04T07:57:21.771 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-04T07:57:21.820 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-04T07:57:21.830 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-04T07:57:21.871 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-04T07:57:21.875 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 63/172 2024-07-04T07:57:21.952 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-04T07:57:22.012 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-04T07:57:22.050 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-04T07:57:22.052 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:2.30.0-1.el9.noarch 65/172 2024-07-04T07:57:22.110 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-04T07:57:22.151 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-04T07:57:22.184 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-04T07:57:22.191 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-04T07:57:22.229 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-04T07:57:22.261 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-04T07:57:22.288 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-07-04T07:57:22.373 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-04T07:57:22.562 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-04T07:57:22.599 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-04T07:57:22.636 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-04T07:57:22.672 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-04T07:57:22.701 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 128/172 2024-07-04T07:57:22.729 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 128/172 2024-07-04T07:57:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-04T07:57:22.761 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-4737.g2e9f68b5.e 129/172 2024-07-04T07:57:22.786 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-04T07:57:22.825 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-4737.g2e9f68b5. 130/172 2024-07-04T07:57:22.827 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-04T07:57:22.852 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noa 131/172 2024-07-04T07:57:22.873 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-04T07:57:22.924 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-04T07:57:22.978 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-04T07:57:23.020 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-04T07:57:23.158 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-04T07:57:23.238 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-04T07:57:23.281 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-04T07:57:23.424 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-04T07:57:23.502 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-04T07:57:24.541 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-04T07:57:24.626 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-04T07:57:24.661 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-04T07:57:24.825 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-04T07:57:24.887 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-04T07:57:24.953 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-04T07:57:25.069 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-04T07:57:25.121 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-04T07:57:25.385 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-04T07:57:25.434 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-04T07:57:25.564 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-04T07:57:26.176 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noa 131/172 2024-07-04T07:57:26.233 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.n 132/172 2024-07-04T07:57:26.488 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-04T07:57:26.645 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-04T07:57:27.249 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.n 132/172 2024-07-04T07:57:27.291 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9 133/172 2024-07-04T07:57:27.409 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9 133/172 2024-07-04T07:57:27.583 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-4737.g2e9f68b5.el 134/172 2024-07-04T07:57:27.626 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 135/172 2024-07-04T07:57:27.651 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 135/172 2024-07-04T07:57:27.651 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:27.651 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-04T07:57:27.651 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-04T07:57:27.651 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-04T07:57:27.651 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:27.702 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 136/172 2024-07-04T07:57:27.714 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 136/172 2024-07-04T07:57:27.818 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.0.0-4737.g2e9f68b5.el9.x86_64 137/172 2024-07-04T07:57:27.877 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 138/172 2024-07-04T07:57:28.387 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-04T07:57:28.492 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-04T07:57:28.526 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-04T07:57:28.559 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-04T07:57:28.882 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-04T07:57:28.922 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 140/172 2024-07-04T07:57:28.946 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 140/172 2024-07-04T07:57:28.946 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:28.946 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-04T07:57:28.946 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-04T07:57:28.946 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-04T07:57:28.947 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:28.967 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-04T07:57:28.989 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-4737.g2e9f6 141/172 2024-07-04T07:57:29.012 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-4737.g2e9f6 141/172 2024-07-04T07:57:29.012 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:29.012 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-04T07:57:29.012 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:29.012 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 104/172 2024-07-04T07:57:29.064 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 142/172 2024-07-04T07:57:29.088 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 142/172 2024-07-04T07:57:29.088 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:29.088 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-04T07:57:29.089 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-04T07:57:29.089 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-04T07:57:29.089 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:29.137 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 104/172 2024-07-04T07:57:29.170 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 105/172 2024-07-04T07:57:29.220 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-04T07:57:29.424 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 143/172 2024-07-04T07:57:29.447 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 143/172 2024-07-04T07:57:29.448 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:29.448 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-04T07:57:29.448 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-04T07:57:29.448 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-04T07:57:29.448 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:29.804 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-04T07:57:29.849 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 108/172 2024-07-04T07:57:29.934 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 108/172 2024-07-04T07:57:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 109/172 2024-07-04T07:57:30.378 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 112/172 2024-07-04T07:57:30.379 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-07-04T07:57:30.379 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-07-04T07:57:30.379 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-07-04T07:57:30.379 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-07-04T07:57:30.379 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-07-04T07:57:30.379 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-07-04T07:57:30.379 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-07-04T07:57:30.379 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:30.549 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 113/172 2024-07-04T07:57:30.565 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 113/172 2024-07-04T07:57:30.565 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:30.565 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-04T07:57:30.565 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:32.143 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 114/172 2024-07-04T07:57:32.170 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 114/172 2024-07-04T07:57:32.171 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:32.171 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-04T07:57:32.171 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-04T07:57:32.171 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-04T07:57:32.171 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:32.395 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 115/172 2024-07-04T07:57:32.418 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 115/172 2024-07-04T07:57:32.418 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:32.418 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-04T07:57:32.418 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-04T07:57:32.419 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-04T07:57:32.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:32.880 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 116/172 2024-07-04T07:57:32.902 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 116/172 2024-07-04T07:57:32.902 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:32.903 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-04T07:57:32.903 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-04T07:57:32.903 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-04T07:57:32.903 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:33.010 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-04T07:57:33.066 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-04T07:57:33.110 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 110/172 2024-07-04T07:57:33.121 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-04T07:57:33.148 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 110/172 2024-07-04T07:57:33.309 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-04T07:57:33.364 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-04T07:57:33.442 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-04T07:57:33.530 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-04T07:57:33.668 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-04T07:57:33.714 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-04T07:57:33.746 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-04T07:57:33.773 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-07-04T07:57:33.847 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 110/172 2024-07-04T07:57:33.902 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 111/172 2024-07-04T07:57:33.995 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.0.0-4737.g2e9f68b5.el9.x86_64 144/172 2024-07-04T07:57:34.036 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 145/172 2024-07-04T07:57:34.078 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 146/172 2024-07-04T07:57:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 111/172 2024-07-04T07:57:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-04T07:57:34.083 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-04T07:57:34.083 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:57:34.108 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xfsprogs-devel-6.4.0-2.el9.x86_64 147/172 2024-07-04T07:57:34.119 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 112/172 2024-07-04T07:57:34.136 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-04T07:57:34.150 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 128/172 2024-07-04T07:57:34.180 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 128/172 2024-07-04T07:57:34.212 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-4737.g2e9f68b5.e 129/172 2024-07-04T07:57:34.273 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-4737.g2e9f68b5. 130/172 2024-07-04T07:57:34.303 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noa 131/172 2024-07-04T07:57:34.331 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-shell-2:19.0.0-4737.g2e9f68b5.el9.x86_64 149/172 2024-07-04T07:57:34.374 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-04T07:57:34.398 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_ 151/172 2024-07-04T07:57:34.438 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-04T07:57:34.468 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-04T07:57:34.514 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-04T07:57:35.511 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T07:57:35.553 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T07:57:35.574 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T07:57:36.514 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-top-2:19.0.0-4737.g2e9f68b5.el9.noarch 156/172 2024-07-04T07:57:36.600 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-04T07:57:36.634 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-04T07:57:36.727 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-04T07:57:36.770 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-04T07:57:36.795 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-04T07:57:36.842 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-04T07:57:36.880 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-04T07:57:36.880 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-04T07:57:36.964 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-04T07:57:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-04T07:57:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-04T07:57:37.082 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-04T07:57:37.106 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-04T07:57:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T07:57:37.111 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T07:57:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T07:57:37.304 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-04T07:57:37.389 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-04T07:57:37.422 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-04T07:57:37.443 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noa 131/172 2024-07-04T07:57:37.448 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-04T07:57:37.501 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.n 132/172 2024-07-04T07:57:37.523 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T07:57:38.461 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.n 132/172 2024-07-04T07:57:38.484 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9 133/172 2024-07-04T07:57:38.597 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9 133/172 2024-07-04T07:57:38.767 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-4737.g2e9f68b5.el 134/172 2024-07-04T07:57:38.795 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 135/172 2024-07-04T07:57:38.818 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 135/172 2024-07-04T07:57:38.818 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:38.818 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-04T07:57:38.818 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-04T07:57:38.818 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-04T07:57:38.818 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:38.870 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 136/172 2024-07-04T07:57:38.882 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 136/172 2024-07-04T07:57:38.980 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.0.0-4737.g2e9f68b5.el9.x86_64 137/172 2024-07-04T07:57:39.026 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 138/172 2024-07-04T07:57:39.971 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-04T07:57:39.998 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 140/172 2024-07-04T07:57:40.021 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 140/172 2024-07-04T07:57:40.021 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:40.021 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-04T07:57:40.021 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-04T07:57:40.021 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-04T07:57:40.021 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:40.065 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-4737.g2e9f6 141/172 2024-07-04T07:57:40.088 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-4737.g2e9f6 141/172 2024-07-04T07:57:40.088 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:40.088 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-04T07:57:40.088 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:40.141 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 142/172 2024-07-04T07:57:40.163 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 142/172 2024-07-04T07:57:40.163 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:40.163 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-04T07:57:40.163 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-04T07:57:40.163 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-04T07:57:40.163 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:40.483 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 143/172 2024-07-04T07:57:40.506 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 143/172 2024-07-04T07:57:40.506 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:40.506 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-04T07:57:40.506 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-04T07:57:40.506 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-04T07:57:40.506 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:43.601 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T07:57:43.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.0.0-4737.g2e9f68b5.el9.x86_64 1/172 2024-07-04T07:57:43.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2/172 2024-07-04T07:57:43.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 3/172 2024-07-04T07:57:43.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 4/172 2024-07-04T07:57:43.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-4737.g2e9f6 5/172 2024-07-04T07:57:43.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 6/172 2024-07-04T07:57:43.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 7/172 2024-07-04T07:57:43.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 8/172 2024-07-04T07:57:43.602 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 9/172 2024-07-04T07:57:43.603 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 10/172 2024-07-04T07:57:43.603 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 11/172 2024-07-04T07:57:43.603 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.0.0-4737.g2e9f68b5.el9.x86_64 12/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 13/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-shell-2:19.0.0-4737.g2e9f68b5.el9.x86_64 14/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_ 15/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 16/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 17/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_6 18/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86 19/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 20/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.0.0-4737.g2e9f68b5.el 21/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.0.0-4737.g2e9f68b5.el9. 22/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.0.0-4737.g2e9f68b5.el9.x86_6 23/172 2024-07-04T07:57:43.604 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.0.0-4737.g2e9f68b5.el9.x86_64 24/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 25/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 26/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 27/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 28/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 29/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-4737.g2e9f68b5. 30/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noa 31/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.n 32/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9 33/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-4737.g2e9f68b5.el 34/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 35/172 2024-07-04T07:57:43.605 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-4737.g2e9f68b5.e 36/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 37/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 38/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-top-2:19.0.0-4737.g2e9f68b5.el9.noarch 39/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-04T07:57:43.606 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-04T07:57:43.607 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-04T07:57:43.608 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-04T07:57:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-04T07:57:43.610 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-devel-6.4.0-2.el9.x86_64 101/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-04T07:57:43.611 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-04T07:57:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 122/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-04T07:57:43.613 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-04T07:57:43.614 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 149/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 151/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-04T07:57:43.615 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-04T07:57:43.616 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-04T07:57:43.617 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-6.4.0-2.el9.x86_64 171/172 2024-07-04T07:57:44.795 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.0.0-4737.g2e9f68b5.el9.x86_64 144/172 2024-07-04T07:57:44.836 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 145/172 2024-07-04T07:57:44.877 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 146/172 2024-07-04T07:57:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-6.4.0-2.el9.x86_64 147/172 2024-07-04T07:57:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-04T07:57:45.117 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-shell-2:19.0.0-4737.g2e9f68b5.el9.x86_64 149/172 2024-07-04T07:57:45.166 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-04T07:57:45.198 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_ 151/172 2024-07-04T07:57:45.247 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-04T07:57:45.276 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-04T07:57:45.323 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-04T07:57:45.894 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T07:57:45.941 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T07:57:45.958 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T07:57:46.567 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-04T07:57:46.568 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-6.4.0-2.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: automake-1.16.2-8.el9.noarch 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: bison-3.7.4-5.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.569 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.570 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.570 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.570 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.570 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.570 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.570 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.570 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.570 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.570 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.571 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: flex-2.6.4-9.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: fuse-2.9.9-15.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.572 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-04T07:57:46.573 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-04T07:57:46.574 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-13.14-1.el9.x86_64 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-04T07:57:46.575 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-04T07:57:46.576 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-04T07:57:46.577 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-04T07:57:46.577 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-04T07:57:46.577 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-04T07:57:46.577 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-04T07:57:46.577 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-04T07:57:46.577 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-04T07:57:46.577 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-04T07:57:46.578 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-04T07:57:46.579 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-04T07:57:46.579 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-04T07:57:46.579 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-04T07:57:46.579 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-07-04T07:57:46.579 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-04T07:57:46.579 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-04T07:57:46.579 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-04T07:57:46.579 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.580 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-04T07:57:46.581 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-04T07:57:46.581 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-04T07:57:46.581 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-04T07:57:46.581 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-04T07:57:46.581 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-04T07:57:46.581 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-04T07:57:46.581 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-04T07:57:46.581 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-04T07:57:46.581 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2024-07-04T07:57:46.582 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-04T07:57:46.583 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-04T07:57:46.583 INFO:teuthology.orchestra.run.smithi050.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-04T07:57:46.583 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel-6.4.0-2.el9.x86_64 2024-07-04T07:57:46.583 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-04T07:57:46.583 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-04T07:57:46.583 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-07-04T07:57:46.584 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-07-04T07:57:46.798 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:19.0.0-4737.g2e9f68b5.el9.noarch 156/172 2024-07-04T07:57:46.875 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-04T07:57:46.916 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-04T07:57:46.998 DEBUG:teuthology.parallel:result is None 2024-07-04T07:57:46.999 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-04T07:57:47.053 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-04T07:57:47.078 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-04T07:57:47.103 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-04T07:57:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-04T07:57:47.130 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-04T07:57:47.222 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-04T07:57:47.254 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-04T07:57:47.255 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-04T07:57:47.347 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-04T07:57:47.365 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 112/172 2024-07-04T07:57:47.366 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2024-07-04T07:57:47.366 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2024-07-04T07:57:47.366 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2024-07-04T07:57:47.366 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2024-07-04T07:57:47.366 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2024-07-04T07:57:47.366 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /root 2024-07-04T07:57:47.366 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2024-07-04T07:57:47.366 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:57:47.372 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-04T07:57:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T07:57:47.377 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T07:57:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T07:57:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-04T07:57:47.606 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 113/172 2024-07-04T07:57:47.629 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-04T07:57:47.653 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 113/172 2024-07-04T07:57:47.654 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:47.654 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-04T07:57:47.654 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:57:47.663 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-04T07:57:47.689 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-04T07:57:47.761 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T07:57:49.441 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 114/172 2024-07-04T07:57:49.525 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 114/172 2024-07-04T07:57:49.526 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:49.526 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-04T07:57:49.526 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-04T07:57:49.526 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-04T07:57:49.526 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:57:49.783 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 115/172 2024-07-04T07:57:49.854 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 115/172 2024-07-04T07:57:49.854 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:49.854 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-04T07:57:49.854 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-04T07:57:49.854 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-04T07:57:49.854 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:57:50.392 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 116/172 2024-07-04T07:57:50.464 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 116/172 2024-07-04T07:57:50.464 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:50.464 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-04T07:57:50.465 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-04T07:57:50.465 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-04T07:57:50.465 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:57:50.602 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-04T07:57:50.665 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-04T07:57:50.737 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-04T07:57:50.985 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-04T07:57:51.056 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-04T07:57:51.156 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-04T07:57:51.272 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-04T07:57:51.460 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-04T07:57:51.514 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-04T07:57:51.555 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-04T07:57:51.595 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.9-15.el9.x86_64 127/172 2024-07-04T07:57:52.188 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 128/172 2024-07-04T07:57:52.227 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 128/172 2024-07-04T07:57:52.283 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-4737.g2e9f68b5.e 129/172 2024-07-04T07:57:52.367 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-4737.g2e9f68b5. 130/172 2024-07-04T07:57:52.404 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noa 131/172 2024-07-04T07:57:54.002 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T07:57:54.002 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.0.0-4737.g2e9f68b5.el9.x86_64 1/172 2024-07-04T07:57:54.002 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 3/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 4/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-4737.g2e9f6 5/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 6/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 7/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 8/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 9/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 10/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 11/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.0.0-4737.g2e9f68b5.el9.x86_64 12/172 2024-07-04T07:57:54.003 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 13/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-shell-2:19.0.0-4737.g2e9f68b5.el9.x86_64 14/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_ 15/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 16/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 17/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_6 18/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86 19/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 20/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.0.0-4737.g2e9f68b5.el 21/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.0.0-4737.g2e9f68b5.el9. 22/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.0.0-4737.g2e9f68b5.el9.x86_6 23/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.0.0-4737.g2e9f68b5.el9.x86_64 24/172 2024-07-04T07:57:54.004 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 25/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 26/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 27/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 28/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 29/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-4737.g2e9f68b5. 30/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noa 31/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.n 32/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9 33/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-4737.g2e9f68b5.el 34/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 35/172 2024-07-04T07:57:54.005 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-4737.g2e9f68b5.e 36/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 37/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 38/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:19.0.0-4737.g2e9f68b5.el9.noarch 39/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-04T07:57:54.006 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-04T07:57:54.007 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-04T07:57:54.008 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-04T07:57:54.008 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-04T07:57:54.008 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-04T07:57:54.008 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-04T07:57:54.008 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-04T07:57:54.008 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-04T07:57:54.008 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-04T07:57:54.008 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-04T07:57:54.008 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-04T07:57:54.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-04T07:57:54.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-6.4.0-2.el9.x86_64 101/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-04T07:57:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-04T07:57:54.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 122/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-04T07:57:54.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-04T07:57:54.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 149/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 151/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-04T07:57:54.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-04T07:57:54.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-04T07:57:54.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-04T07:57:54.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-04T07:57:54.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-04T07:57:54.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-6.4.0-2.el9.x86_64 171/172 2024-07-04T07:57:56.281 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noa 131/172 2024-07-04T07:57:56.352 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.n 132/172 2024-07-04T07:57:56.624 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T07:57:56.624 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-04T07:57:56.625 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-6.4.0-2.el9.x86_64 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.2-8.el9.noarch 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.7.4-5.el9.x86_64 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.626 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.627 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.4-9.el9.x86_64 2024-07-04T07:57:56.628 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-15.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.629 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-04T07:57:56.630 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-13.14-1.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-04T07:57:56.631 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-04T07:57:56.632 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-04T07:57:56.633 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-04T07:57:56.634 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-04T07:57:56.635 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-04T07:57:56.636 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:57:56.637 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-3.el9.x86_64 2024-07-04T07:57:56.638 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-04T07:57:56.638 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-04T07:57:56.638 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-04T07:57:56.638 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-6.4.0-2.el9.x86_64 2024-07-04T07:57:56.638 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-04T07:57:56.638 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-04T07:57:56.638 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T07:57:56.638 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-07-04T07:57:57.035 DEBUG:teuthology.parallel:result is None 2024-07-04T07:57:57.532 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.n 132/172 2024-07-04T07:57:57.560 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9 133/172 2024-07-04T07:57:57.729 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9 133/172 2024-07-04T07:57:57.931 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-4737.g2e9f68b5.el 134/172 2024-07-04T07:57:57.970 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 135/172 2024-07-04T07:57:58.053 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 135/172 2024-07-04T07:57:58.053 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:58.053 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-04T07:57:58.054 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-04T07:57:58.054 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-04T07:57:58.054 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:57:58.113 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 136/172 2024-07-04T07:57:58.153 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 136/172 2024-07-04T07:57:58.272 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:19.0.0-4737.g2e9f68b5.el9.x86_64 137/172 2024-07-04T07:57:58.351 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 138/172 2024-07-04T07:57:59.465 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-04T07:57:59.499 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 140/172 2024-07-04T07:57:59.572 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 140/172 2024-07-04T07:57:59.572 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:59.572 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-04T07:57:59.572 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-04T07:57:59.572 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-04T07:57:59.572 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:57:59.624 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-4737.g2e9f6 141/172 2024-07-04T07:57:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-4737.g2e9f6 141/172 2024-07-04T07:57:59.696 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:59.696 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-04T07:57:59.696 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:57:59.766 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 142/172 2024-07-04T07:57:59.848 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 142/172 2024-07-04T07:57:59.848 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:57:59.848 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-04T07:57:59.849 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-04T07:57:59.849 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-04T07:57:59.849 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:58:00.275 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 143/172 2024-07-04T07:58:00.359 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 143/172 2024-07-04T07:58:00.360 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-04T07:58:00.360 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-04T07:58:00.360 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-04T07:58:00.360 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-04T07:58:00.360 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:58:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:19.0.0-4737.g2e9f68b5.el9.x86_64 144/172 2024-07-04T07:58:05.408 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 145/172 2024-07-04T07:58:05.465 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 146/172 2024-07-04T07:58:05.502 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xfsprogs-devel-6.4.0-2.el9.x86_64 147/172 2024-07-04T07:58:05.541 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-04T07:58:05.867 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-shell-2:19.0.0-4737.g2e9f68b5.el9.x86_64 149/172 2024-07-04T07:58:06.057 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-04T07:58:06.091 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_ 151/172 2024-07-04T07:58:06.139 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-04T07:58:06.194 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-04T07:58:06.239 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-04T07:58:07.192 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T07:58:07.243 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T07:58:07.303 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-04T07:58:08.759 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephfs-top-2:19.0.0-4737.g2e9f68b5.el9.noarch 156/172 2024-07-04T07:58:08.861 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-04T07:58:08.918 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-04T07:58:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-04T07:58:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-04T07:58:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-04T07:58:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-04T07:58:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-04T07:58:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-04T07:58:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-04T07:58:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-04T07:58:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-04T07:58:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-04T07:58:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-04T07:58:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T07:58:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T07:58:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-04T07:58:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-04T07:58:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-04T07:58:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-04T07:58:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-04T07:58:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T07:58:10.547 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T07:58:10.547 INFO:teuthology.orchestra.run.smithi039.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-07-04T07:58:10.547 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:58:16.899 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:19.0.0-4737.g2e9f68b5.el9.x86_64 1/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 3/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 4/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-4737.g2e9f6 5/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 6/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 7/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 8/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 9/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 10/172 2024-07-04T07:58:16.900 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 11/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:19.0.0-4737.g2e9f68b5.el9.x86_64 12/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 13/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-shell-2:19.0.0-4737.g2e9f68b5.el9.x86_64 14/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_ 15/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 16/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 17/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_6 18/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86 19/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 20/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:19.0.0-4737.g2e9f68b5.el 21/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:19.0.0-4737.g2e9f68b5.el9. 22/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:19.0.0-4737.g2e9f68b5.el9.x86_6 23/172 2024-07-04T07:58:16.901 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:19.0.0-4737.g2e9f68b5.el9.x86_64 24/172 2024-07-04T07:58:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 25/172 2024-07-04T07:58:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 26/172 2024-07-04T07:58:16.902 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 27/172 2024-07-04T07:58:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 28/172 2024-07-04T07:58:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 29/172 2024-07-04T07:58:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-4737.g2e9f68b5. 30/172 2024-07-04T07:58:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noa 31/172 2024-07-04T07:58:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.n 32/172 2024-07-04T07:58:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9 33/172 2024-07-04T07:58:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-4737.g2e9f68b5.el 34/172 2024-07-04T07:58:16.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 35/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-4737.g2e9f68b5.e 36/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 37/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 38/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephfs-top-2:19.0.0-4737.g2e9f68b5.el9.noarch 39/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-04T07:58:16.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-04T07:58:16.916 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-04T07:58:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-04T07:58:16.918 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-04T07:58:16.919 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-devel-6.4.0-2.el9.x86_64 101/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-04T07:58:16.920 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-04T07:58:16.921 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:2.30.0-1.el9.noarch 122/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-04T07:58:16.922 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-04T07:58:16.923 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 149/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 151/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-04T07:58:16.924 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-04T07:58:16.925 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-6.4.0-2.el9.x86_64 171/172 2024-07-04T07:58:19.258 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-04T07:58:19.258 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.259 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-6.4.0-2.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: automake-1.16.2-8.el9.noarch 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: bison-3.7.4-5.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.260 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.261 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-shell-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-top-2:19.0.0-4737.g2e9f68b5.el9.noarch 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: flex-2.6.4-9.el9.x86_64 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-04T07:58:19.262 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: fuse-2.9.9-15.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-04T07:58:19.263 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-04T07:58:19.264 INFO:teuthology.orchestra.run.smithi039.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: lua-5.4.4-4.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-13.14-1.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-04T07:58:19.265 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.266 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-07-04T07:58:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-04T07:58:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-04T07:58:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-04T07:58:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-04T07:58:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-04T07:58:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-04T07:58:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-04T07:58:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:2.30.0-1.el9.noarch 2024-07-04T07:58:19.267 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-04T07:58:19.268 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply-3.11-14.el9.noarch 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-04T07:58:19.269 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-04T07:58:19.270 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-04T07:58:19.270 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-04T07:58:19.270 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-04T07:58:19.270 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-04T07:58:19.270 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-04T07:58:19.270 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-04T07:58:19.270 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-04T07:58:19.270 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.270 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-04T07:58:19.271 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-04T07:58:19.272 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-04T07:58:19.272 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-04T07:58:19.272 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-04T07:58:19.272 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-04T07:58:19.272 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-04T07:58:19.272 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-04T07:58:19.272 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-04T07:58:19.272 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.272 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.273 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:19.0.0-4737.g2e9f68b5.el9.x86_64 2024-07-04T07:58:19.273 INFO:teuthology.orchestra.run.smithi039.stdout: re2-1:20211101-3.el9.x86_64 2024-07-04T07:58:19.273 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-04T07:58:19.273 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-04T07:58:19.273 INFO:teuthology.orchestra.run.smithi039.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-04T07:58:19.273 INFO:teuthology.orchestra.run.smithi039.stdout: xfsprogs-devel-6.4.0-2.el9.x86_64 2024-07-04T07:58:19.273 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-04T07:58:19.274 INFO:teuthology.orchestra.run.smithi039.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-04T07:58:19.274 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:58:19.274 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-07-04T07:58:20.073 DEBUG:teuthology.parallel:result is None 2024-07-04T07:58:20.074 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:58:20.226 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-04T07:58:20.284 INFO:teuthology.orchestra.run.smithi039.stdout:19.0.0-4737.g2e9f68b5.el9 2024-07-04T07:58:20.285 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4737.g2e9f68b5.el9 2024-07-04T07:58:20.285 INFO:teuthology.task.install:The correct ceph version 19.0.0-4737.g2e9f68b5 is installed. 2024-07-04T07:58:20.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:58:20.431 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-04T07:58:20.458 INFO:teuthology.orchestra.run.smithi050.stdout:19.0.0-4737.g2e9f68b5.el9 2024-07-04T07:58:20.459 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4737.g2e9f68b5.el9 2024-07-04T07:58:20.459 INFO:teuthology.task.install:The correct ceph version 19.0.0-4737.g2e9f68b5 is installed. 2024-07-04T07:58:20.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:58:20.604 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-04T07:58:20.630 INFO:teuthology.orchestra.run.smithi133.stdout:19.0.0-4737.g2e9f68b5.el9 2024-07-04T07:58:20.630 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4737.g2e9f68b5.el9 2024-07-04T07:58:20.630 INFO:teuthology.task.install:The correct ceph version 19.0.0-4737.g2e9f68b5 is installed. 2024-07-04T07:58:20.632 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-04T07:58:20.633 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T07:58:20.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-04T07:58:20.712 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T07:58:20.713 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-04T07:58:20.745 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T07:58:20.745 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-04T07:58:20.775 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-04T07:58:20.776 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T07:58:20.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-07-04T07:58:20.840 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-04T07:58:20.944 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T07:58:20.944 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-07-04T07:58:20.973 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-04T07:58:21.043 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T07:58:21.043 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-07-04T07:58:21.072 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-04T07:58:21.139 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-04T07:58:21.139 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T07:58:21.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-04T07:58:21.201 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-04T07:58:21.303 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T07:58:21.303 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-04T07:58:21.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-04T07:58:21.393 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T07:58:21.393 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-04T07:58:21.417 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-04T07:58:21.483 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-04T07:58:21.484 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T07:58:21.484 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-07-04T07:58:21.546 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-04T07:58:21.653 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T07:58:21.653 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-07-04T07:58:21.677 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-04T07:58:21.742 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T07:58:21.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-07-04T07:58:21.765 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-04T07:58:21.828 INFO:teuthology.run_tasks:Running task cephadm... 2024-07-04T07:58:21.917 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-07-04T07:58:21.917 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:58:21.918 INFO:tasks.cephadm:Cluster fsid is 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:58:21.918 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-07-04T07:58:21.918 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.39', 'mon.b': '172.21.15.50', 'mon.c': '172.21.15.133'} 2024-07-04T07:58:21.918 INFO:tasks.cephadm:First mon is mon.a on smithi039 2024-07-04T07:58:21.918 INFO:tasks.cephadm:First mgr is x 2024-07-04T07:58:21.918 INFO:tasks.cephadm:Normalizing hostnames... 2024-07-04T07:58:21.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname $(hostname -s) 2024-07-04T07:58:21.985 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname $(hostname -s) 2024-07-04T07:58:22.010 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2024-07-04T07:58:22.033 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-07-04T07:58:22.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:58:22.253 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/80908/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-4737-g2e9f68b5', 'node_name': '172.21.5.40+adami10', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-4737.g2e9f68b5'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-03 17:08:42.565501', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240703.143006-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/80905/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-4737-g2e9f68b5', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-4737.g2e9f68b5'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-03 16:39:07.383921', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240703.143006-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-07-04T07:58:22.467 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-pdonnell-testing-20240703.143006-debug, sha1 2e9f68b5506c607b2f7d69d380cb7887cf00dc84 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:58:22.468 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/x86_64/flavors/default/cephadm 2024-07-04T07:58:22.469 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/x86_64/flavors/default/cephadm 2024-07-04T07:58:22.469 DEBUG:teuthology.orchestra.run.smithi039:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-04T07:58:23.012 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r--. 1 ubuntu ubuntu 783320 Jul 4 07:58 /home/ubuntu/cephtest/cephadm 2024-07-04T07:58:23.013 DEBUG:teuthology.orchestra.run.smithi050:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-04T07:58:23.390 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r--. 1 ubuntu ubuntu 783320 Jul 4 07:58 /home/ubuntu/cephtest/cephadm 2024-07-04T07:58:23.391 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240703.143006-debug/2e9f68b5506c607b2f7d69d380cb7887cf00dc84/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-04T07:58:23.771 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r--. 1 ubuntu ubuntu 783320 Jul 4 07:58 /home/ubuntu/cephtest/cephadm 2024-07-04T07:58:23.772 DEBUG:teuthology.orchestra.run.smithi039:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-04T07:58:23.818 DEBUG:teuthology.orchestra.run.smithi050:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-04T07:58:23.838 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-04T07:58:23.873 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 on all hosts... 2024-07-04T07:58:23.873 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 pull 2024-07-04T07:58:23.878 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 pull 2024-07-04T07:58:23.881 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 pull 2024-07-04T07:58:24.737 INFO:teuthology.orchestra.run.smithi050.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84... 2024-07-04T07:58:24.748 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84... 2024-07-04T07:58:25.315 INFO:teuthology.orchestra.run.smithi039.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84... 2024-07-04T07:59:00.898 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-07-04T07:59:00.899 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-4737-g2e9f68b5 (2e9f68b5506c607b2f7d69d380cb7887cf00dc84) squid (dev)", 2024-07-04T07:59:00.899 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f", 2024-07-04T07:59:00.899 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2024-07-04T07:59:00.899 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35" 2024-07-04T07:59:00.899 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-07-04T07:59:00.899 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-07-04T07:59:07.474 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-07-04T07:59:07.474 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-4737-g2e9f68b5 (2e9f68b5506c607b2f7d69d380cb7887cf00dc84) squid (dev)", 2024-07-04T07:59:07.474 INFO:teuthology.orchestra.run.smithi050.stdout: "image_id": "ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f", 2024-07-04T07:59:07.474 INFO:teuthology.orchestra.run.smithi050.stdout: "repo_digests": [ 2024-07-04T07:59:07.474 INFO:teuthology.orchestra.run.smithi050.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35" 2024-07-04T07:59:07.474 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-07-04T07:59:07.474 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-07-04T07:59:08.337 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T07:59:08.337 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 19.0.0-4737-g2e9f68b5 (2e9f68b5506c607b2f7d69d380cb7887cf00dc84) squid (dev)", 2024-07-04T07:59:08.337 INFO:teuthology.orchestra.run.smithi039.stdout: "image_id": "ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f", 2024-07-04T07:59:08.337 INFO:teuthology.orchestra.run.smithi039.stdout: "repo_digests": [ 2024-07-04T07:59:08.337 INFO:teuthology.orchestra.run.smithi039.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35" 2024-07-04T07:59:08.337 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T07:59:08.337 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T07:59:08.408 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph 2024-07-04T07:59:08.501 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph 2024-07-04T07:59:08.534 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2024-07-04T07:59:08.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /etc/ceph 2024-07-04T07:59:08.660 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /etc/ceph 2024-07-04T07:59:08.689 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2024-07-04T07:59:08.716 INFO:tasks.cephadm:Writing seed config... 2024-07-04T07:59:08.717 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-07-04T07:59:08.718 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-07-04T07:59:08.719 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-07-04T07:59:08.720 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T07:59:08.720 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-07-04T07:59:08.780 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2ee561ca-39db-11ef-bcab-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-07-04T07:59:08.780 DEBUG:teuthology.orchestra.run.smithi039:mon.a> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.a.service 2024-07-04T07:59:08.828 DEBUG:teuthology.orchestra.run.smithi039:mgr.x> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.x.service 2024-07-04T07:59:08.874 INFO:tasks.cephadm:Bootstrapping... 2024-07-04T07:59:08.874 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 -v bootstrap --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.39 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-07-04T07:59:09.171 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-07-04T07:59:09.171 INFO:teuthology.orchestra.run.smithi039.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', '-v', 'bootstrap', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.39', '--skip-admin-label'] 2024-07-04T07:59:09.171 INFO:teuthology.orchestra.run.smithi039.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-07-04T07:59:09.171 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying podman|docker is present... 2024-07-04T07:59:09.231 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 5.1.1 2024-07-04T07:59:09.232 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying lvm2 is present... 2024-07-04T07:59:09.232 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying time synchronization is in place... 2024-07-04T07:59:09.249 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-04T07:59:09.249 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-04T07:59:09.264 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-04T07:59:09.264 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-07-04T07:59:09.283 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-07-04T07:59:09.302 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-07-04T07:59:09.302 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-07-04T07:59:09.302 INFO:teuthology.orchestra.run.smithi039.stdout:Repeating the final host check... 2024-07-04T07:59:09.367 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout 5.1.1 2024-07-04T07:59:09.367 INFO:teuthology.orchestra.run.smithi039.stdout:podman (/bin/podman) version 5.1.1 is present 2024-07-04T07:59:09.367 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl is present 2024-07-04T07:59:09.367 INFO:teuthology.orchestra.run.smithi039.stdout:lvcreate is present 2024-07-04T07:59:09.384 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-04T07:59:09.384 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-04T07:59:09.407 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-04T07:59:09.407 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-07-04T07:59:09.433 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout enabled 2024-07-04T07:59:09.457 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout active 2024-07-04T07:59:09.457 INFO:teuthology.orchestra.run.smithi039.stdout:Unit chronyd.service is enabled and running 2024-07-04T07:59:09.457 INFO:teuthology.orchestra.run.smithi039.stdout:Host looks OK 2024-07-04T07:59:09.457 INFO:teuthology.orchestra.run.smithi039.stdout:Cluster fsid: 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:09.458 INFO:teuthology.orchestra.run.smithi039.stdout:Acquiring lock 139842686115456 on /run/cephadm/2ee561ca-39db-11ef-bcab-c7b262605968.lock 2024-07-04T07:59:09.458 INFO:teuthology.orchestra.run.smithi039.stdout:Lock 139842686115456 acquired on /run/cephadm/2ee561ca-39db-11ef-bcab-c7b262605968.lock 2024-07-04T07:59:09.459 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 3300 ... 2024-07-04T07:59:09.460 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying IP 172.21.15.39 port 6789 ... 2024-07-04T07:59:09.461 INFO:teuthology.orchestra.run.smithi039.stdout:Base mon IP(s) is [172.21.15.39:3300, 172.21.15.39:6789], mon addrv is [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-07-04T07:59:09.475 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.39 metric 100 2024-07-04T07:59:09.476 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.39 metric 100 2024-07-04T07:59:09.489 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-07-04T07:59:09.489 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-07-04T07:59:09.489 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-07-04T07:59:09.501 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-07-04T07:59:09.502 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-07-04T07:59:09.502 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-04T07:59:09.502 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-07-04T07:59:09.502 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1370/64 scope link noprefixroute 2024-07-04T07:59:09.502 INFO:teuthology.orchestra.run.smithi039.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-04T07:59:09.505 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-07-04T07:59:09.505 INFO:teuthology.orchestra.run.smithi039.stdout:Mon IP `172.21.15.39` is in CIDR network `172.21.0.0/20` 2024-07-04T07:59:09.505 INFO:teuthology.orchestra.run.smithi039.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-07-04T07:59:09.508 INFO:teuthology.orchestra.run.smithi039.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-07-04T07:59:09.510 INFO:teuthology.orchestra.run.smithi039.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84... 2024-07-04T07:59:10.165 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stdout ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f 2024-07-04T07:59:10.165 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84... 2024-07-04T07:59:10.165 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Getting image source signatures 2024-07-04T07:59:10.165 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:0d9665cbb1e45ff458d07612450003bc3c523a9d941d88d73ef519caf59c67ab 2024-07-04T07:59:10.166 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying blob sha256:7d1214c392a3ef380cfb0de40268fb078fc668d8434b1e2a79a3c8f836f5db99 2024-07-04T07:59:10.166 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Copying config sha256:ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f 2024-07-04T07:59:10.166 INFO:teuthology.orchestra.run.smithi039.stdout:/bin/podman: stderr Writing manifest to image destination 2024-07-04T07:59:11.344 INFO:teuthology.orchestra.run.smithi039.stdout:ceph: stdout ceph version 19.0.0-4737-g2e9f68b5 (2e9f68b5506c607b2f7d69d380cb7887cf00dc84) squid (dev) 2024-07-04T07:59:11.344 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph version: ceph version 19.0.0-4737-g2e9f68b5 (2e9f68b5506c607b2f7d69d380cb7887cf00dc84) squid (dev) 2024-07-04T07:59:11.344 INFO:teuthology.orchestra.run.smithi039.stdout:Extracting ceph user uid/gid from container image... 2024-07-04T07:59:12.444 INFO:teuthology.orchestra.run.smithi039.stdout:stat: stdout 167 167 2024-07-04T07:59:12.444 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial keys... 2024-07-04T07:59:13.734 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQBRVoZmoWq1ABAAOwmMphIISuGdxa26YjYeZQ== 2024-07-04T07:59:14.890 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQBSVoZmGRhpExAAmJ1Ok43XBXaCVSjKvEEuFA== 2024-07-04T07:59:16.149 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph-authtool: stdout AQBTVoZmDT38IBAAdGMFgX/T44WjGtU6z1iJIw== 2024-07-04T07:59:16.151 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial monmap... 2024-07-04T07:59:17.387 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-04T07:59:17.387 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-07-04T07:59:17.387 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:17.388 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-04T07:59:17.388 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool for a [v2:172.21.15.39:3300,v1:172.21.15.39:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-04T07:59:17.388 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = quincy 2024-07-04T07:59:17.388 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: set fsid to 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:17.388 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-04T07:59:17.388 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T07:59:17.388 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mon... 2024-07-04T07:59:18.633 INFO:teuthology.orchestra.run.smithi039.stdout:create mon.a on 2024-07-04T07:59:19.037 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-07-04T07:59:19.284 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-07-04T07:59:19.558 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2ee561ca-39db-11ef-bcab-c7b262605968.target → /etc/systemd/system/ceph-2ee561ca-39db-11ef-bcab-c7b262605968.target. 2024-07-04T07:59:19.558 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2ee561ca-39db-11ef-bcab-c7b262605968.target → /etc/systemd/system/ceph-2ee561ca-39db-11ef-bcab-c7b262605968.target. 2024-07-04T07:59:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.a 2024-07-04T07:59:19.921 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.a.service: Unit ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.a.service not loaded. 2024-07-04T07:59:20.200 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2ee561ca-39db-11ef-bcab-c7b262605968.target.wants/ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.a.service → /etc/systemd/system/ceph-2ee561ca-39db-11ef-bcab-c7b262605968@.service. 2024-07-04T07:59:20.958 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:20 smithi039 podman[31674]: 2024-07-04 07:59:20.853612299 +0000 UTC m=+0.378233122 container start 5cc5c84ef2cac610cf7b878a73f3bbbb1fbd8be20f715410fe29430f170246ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True) 2024-07-04T07:59:20.959 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:20 smithi039 bash[31674]: 5cc5c84ef2cac610cf7b878a73f3bbbb1fbd8be20f715410fe29430f170246ce 2024-07-04T07:59:21.004 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-04T07:59:21.004 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-07-04T07:59:21.020 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-04T07:59:21.020 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-07-04T07:59:21.020 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-07-04T07:59:21.020 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-07-04T07:59:21.021 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon to start... 2024-07-04T07:59:21.021 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mon... 2024-07-04T07:59:21.289 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:20 smithi039 systemd[1]: Started Ceph mon.a for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T07:59:21.290 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:21 smithi039 ceph-mon[31693]: mkfs 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:21.290 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:21 smithi039 ceph-mon[31693]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-04T07:59:22.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-04T07:59:22.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: monmap epoch 1 2024-07-04T07:59:22.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:22.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: last_changed 2024-07-04T07:59:16.806394+0000 2024-07-04T07:59:22.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T07:59:22.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: min_mon_release 19 (squid) 2024-07-04T07:59:22.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: election_strategy: 1 2024-07-04T07:59:22.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T07:59:22.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: fsmap 2024-07-04T07:59:22.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: osdmap e1: 0 total, 0 up, 0 in 2024-07-04T07:59:22.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: mgrmap e1: no daemons active 2024-07-04T07:59:22.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:22 smithi039 ceph-mon[31693]: from='client.? 172.21.15.39:0/3755816046' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-07-04T07:59:22.377 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout cluster: 2024-07-04T07:59:22.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout id: 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:22.379 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout services: 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.736758s) 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout data: 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout pgs: 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:22.380 INFO:teuthology.orchestra.run.smithi039.stdout:mon is available 2024-07-04T07:59:22.381 INFO:teuthology.orchestra.run.smithi039.stdout:Assimilating anything we can from ceph.conf... 2024-07-04T07:59:23.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:23 smithi039 ceph-mon[31693]: from='client.? 172.21.15.39:0/966335422' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-04T07:59:23.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:23 smithi039 ceph-mon[31693]: from='client.? 172.21.15.39:0/966335422' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-07-04T07:59:23.748 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:23.748 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-07-04T07:59:23.748 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-04T07:59:23.749 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-04T07:59:23.750 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-04T07:59:23.750 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-04T07:59:23.750 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-04T07:59:23.750 INFO:teuthology.orchestra.run.smithi039.stdout:Generating new minimal ceph.conf... 2024-07-04T07:59:24.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:24 smithi039 ceph-mon[31693]: from='client.? 172.21.15.39:0/2398705127' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T07:59:25.298 INFO:teuthology.orchestra.run.smithi039.stdout:Restarting the monitor... 2024-07-04T07:59:25.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:25 smithi039 systemd[1]: Stopping Ceph mon.a for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T07:59:26.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:25 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a[31689]: 2024-07-04T07:59:25.784+0000 7f3b71c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T07:59:26.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:25 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a[31689]: 2024-07-04T07:59:25.784+0000 7f3b71c00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-07-04T07:59:26.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:25 smithi039 podman[31937]: 2024-07-04 07:59:25.99128752 +0000 UTC m=+0.322604918 container died 5cc5c84ef2cac610cf7b878a73f3bbbb1fbd8be20f715410fe29430f170246ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625) 2024-07-04T07:59:26.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:26 smithi039 podman[31937]: 2024-07-04 07:59:26.395483467 +0000 UTC m=+0.726800912 container cleanup 5cc5c84ef2cac610cf7b878a73f3bbbb1fbd8be20f715410fe29430f170246ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-04T07:59:26.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:26 smithi039 bash[31937]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a 2024-07-04T07:59:26.885 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:26 smithi039 podman[31949]: 2024-07-04 07:59:26.621035822 +0000 UTC m=+0.614321838 container remove 5cc5c84ef2cac610cf7b878a73f3bbbb1fbd8be20f715410fe29430f170246ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-07-04T07:59:27.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 systemd[1]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.a.service: Deactivated successfully. 2024-07-04T07:59:27.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 systemd[1]: Stopped Ceph mon.a for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T07:59:27.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 systemd[1]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.a.service: Consumed 2.045s CPU time. 2024-07-04T07:59:27.137 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 systemd[1]: Starting Ceph mon.a for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T07:59:27.776 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 podman[32044]: 2024-07-04 07:59:27.505238518 +0000 UTC m=+0.117298654 container create c524f94894efdc0689a4fab33c574be8acbcf4855f470d6e185b5872f257159a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0) 2024-07-04T07:59:27.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 podman[32044]: 2024-07-04 07:59:27.419955022 +0000 UTC m=+0.032015121 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:59:27.777 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 podman[32044]: 2024-07-04 07:59:27.77567133 +0000 UTC m=+0.387731473 container init c524f94894efdc0689a4fab33c574be8acbcf4855f470d6e185b5872f257159a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-07-04T07:59:27.918 INFO:teuthology.orchestra.run.smithi039.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-07-04T07:59:28.070 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 podman[32044]: 2024-07-04 07:59:27.787713411 +0000 UTC m=+0.399773577 container start c524f94894efdc0689a4fab33c574be8acbcf4855f470d6e185b5872f257159a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2) 2024-07-04T07:59:28.070 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 bash[32044]: c524f94894efdc0689a4fab33c574be8acbcf4855f470d6e185b5872f257159a 2024-07-04T07:59:28.070 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: set uid:gid to 167:167 (ceph:ceph) 2024-07-04T07:59:28.070 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: ceph version 19.0.0-4737-g2e9f68b5 (2e9f68b5506c607b2f7d69d380cb7887cf00dc84) squid (dev), process ceph-mon, pid 2 2024-07-04T07:59:28.070 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: pidfile_write: ignore empty --pid-file 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 systemd[1]: Started Ceph mon.a for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: load: jerasure load: lrc 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: RocksDB version: 7.9.2 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Git sha 0 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Compile date 2024-07-03 16:11:10 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: DB SUMMARY 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: DB Session ID: SB75LVY9YFWZBWPS8AZ4 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: CURRENT file: CURRENT 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: IDENTITY file: IDENTITY 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-07-04T07:59:28.071 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94409 ; 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.error_if_exists: 0 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.create_if_missing: 0 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.paranoid_checks: 1 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.env: 0x5562702ad200 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.fs: PosixFileSystem 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.info_log: 0x5562a44050e0 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_file_opening_threads: 16 2024-07-04T07:59:28.072 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.statistics: (nil) 2024-07-04T07:59:28.073 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.use_fsync: 0 2024-07-04T07:59:28.073 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_log_file_size: 0 2024-07-04T07:59:28.073 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-04T07:59:28.073 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-04T07:59:28.073 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.keep_log_file_num: 1000 2024-07-04T07:59:28.073 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.recycle_log_file_num: 0 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.allow_fallocate: 1 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.allow_mmap_reads: 0 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.allow_mmap_writes: 0 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.use_direct_reads: 0 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.create_missing_column_families: 0 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.db_log_dir: 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.wal_dir: 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-04T07:59:28.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.advise_random_on_open: 1 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.db_write_buffer_size: 0 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.write_buffer_manager: 0x5562a44095e0 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.rate_limiter: (nil) 2024-07-04T07:59:28.075 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.wal_recovery_mode: 2 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.enable_thread_tracking: 0 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.enable_pipelined_write: 0 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.unordered_write: 0 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.row_cache: None 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.wal_filter: None 2024-07-04T07:59:28.076 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.allow_ingest_behind: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.two_write_queues: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.manual_wal_flush: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.wal_compression: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.atomic_flush: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.log_readahead_size: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.best_efforts_recovery: 0 2024-07-04T07:59:28.077 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.allow_data_in_errors: 0 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.db_host_id: __hostname__ 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_background_jobs: 2 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_background_compactions: -1 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_subcompactions: 1 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_total_wal_size: 0 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-04T07:59:28.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_open_files: -1 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bytes_per_sync: 0 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_readahead_size: 0 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_background_flushes: -1 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Compression algorithms supported: 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: kZSTD supported: 0 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: kXpressCompression supported: 0 2024-07-04T07:59:28.079 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: kBZip2Compression supported: 0 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: kLZ4Compression supported: 1 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: kZlibCompression supported: 1 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: kLZ4HCCompression supported: 1 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: kSnappyCompression supported: 1 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.merge_operator: 2024-07-04T07:59:28.080 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_filter: None 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_filter_factory: None 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.sst_partitioner_factory: None 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5562a4404d00) 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks: 1 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout: pin_top_level_index_and_filter: 1 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_type: 0 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_index_type: 0 2024-07-04T07:59:28.081 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_shortening: 1 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: checksum: 4 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: no_block_cache: 0 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache: 0x5562a03c9610 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_name: BinnedLRUCache 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_options: 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: capacity : 536870912 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_shard_bits : 4 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: strict_capacity_limit : 0 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: high_pri_pool_ratio: 0.000 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_cache_compressed: (nil) 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: persistent_cache: (nil) 2024-07-04T07:59:28.082 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size: 4096 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_size_deviation: 10 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_restart_interval: 16 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: index_block_restart_interval: 1 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: metadata_block_size: 4096 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: partition_filters: 0 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: use_delta_encoding: 1 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: filter_policy: bloomfilter 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: whole_key_filtering: 1 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: verify_compression: 0 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: read_amp_bytes_per_bit: 0 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: format_version: 5 2024-07-04T07:59:28.083 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_index_compression: 1 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout: block_align: 0 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout: max_auto_readahead_size: 262144 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout: prepopulate_block_cache: 0 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout: initial_auto_readahead_size: 8192 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout: num_file_reads_for_auto_readahead: 2 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.write_buffer_size: 33554432 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_write_buffer_number: 2 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression: NoCompression 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression: Disabled 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.prefix_extractor: nullptr 2024-07-04T07:59:28.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-04T07:59:28.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.num_levels: 7 2024-07-04T07:59:28.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-04T07:59:28.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-04T07:59:28.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-04T07:59:28.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-04T07:59:28.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression_opts.level: 32767 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression_opts.strategy: 0 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-04T07:59:28.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression_opts.enabled: false 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.target_file_size_base: 67108864 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-04T07:59:28.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.arena_block_size: 1048576 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-04T07:59:28.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.disable_auto_compactions: 0 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-04T07:59:28.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.inplace_update_support: 0 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.bloom_locality: 0 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.max_successive_merges: 0 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.paranoid_file_checks: 0 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.force_consistency_checks: 1 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.report_bg_io_stats: 0 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.ttl: 2592000 2024-07-04T07:59:28.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.enable_blob_files: false 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.min_blob_size: 0 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.blob_file_size: 268435456 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.blob_file_starting_level: 0 2024-07-04T07:59:28.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0ed167c7-8f7a-4502-abc9-5d3baa2a8fae 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720079967922190, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720079967924915, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90639, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 280, "table_properties": {"data_size": 88680, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11486, "raw_average_key_size": 48, "raw_value_size": 82205, "raw_average_value_size": 343, "num_data_blocks": 12, "num_entries": 239, "num_filter_entries": 239, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1720079967, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0ed167c7-8f7a-4502-abc9-5d3baa2a8fae", "db_session_id": "SB75LVY9YFWZBWPS8AZ4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720079967925126, "job": 1, "event": "recovery_finished"} 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5562a4428a80 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: DB pointer 0x5562a51ca000 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-04T07:59:28.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** DB Stats ** 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-04T07:59:28.093 INFO:journalctl@ceph.mon.a.smithi039.stdout: L0 2/0 90.38 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: Sum 2/0 90.38 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** Compaction Stats [default] ** 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 56.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-04T07:59:28.094 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: Cumulative compaction: 0.00 GB write, 9.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: Interval compaction: 0.00 GB write, 9.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache BinnedLRUCache@0x5562a03c9610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: starting mon.a rank 0 at public addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] at bind addrs [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mon.a@-1(???) e1 preinit fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:28.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mon.a@-1(???).mds e1 new map 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mon.a@-1(???).mds e1 print_map 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout: e1 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout: btime 2024-07-04T07:59:21:009701+0000 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout: legacy client fscid: -1 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout: 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout: No filesystems configured 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-04T07:59:28.096 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: monmap epoch 1 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: last_changed 2024-07-04T07:59:16.806394+0000 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: min_mon_release 19 (squid) 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: election_strategy: 1 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: fsmap 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: osdmap e1: 0 total, 0 up, 0 in 2024-07-04T07:59:28.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:27 smithi039 ceph-mon[32062]: mgrmap e1: no daemons active 2024-07-04T07:59:29.449 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote config to /etc/ceph/ceph.conf 2024-07-04T07:59:29.472 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-07-04T07:59:29.472 INFO:teuthology.orchestra.run.smithi039.stdout:Creating mgr... 2024-07-04T07:59:29.474 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:9283 ... 2024-07-04T07:59:29.475 INFO:teuthology.orchestra.run.smithi039.stdout:Verifying port 0.0.0.0:8765 ... 2024-07-04T07:59:29.890 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.x 2024-07-04T07:59:29.890 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.x.service: Unit ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.x.service not loaded. 2024-07-04T07:59:30.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:29 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2165968747' entity='client.admin' 2024-07-04T07:59:30.150 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2ee561ca-39db-11ef-bcab-c7b262605968.target.wants/ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-2ee561ca-39db-11ef-bcab-c7b262605968@.service. 2024-07-04T07:59:30.543 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:30 smithi039 systemd[1]: Starting Ceph mgr.x for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T07:59:30.852 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:30 smithi039 podman[32272]: 2024-07-04 07:59:30.447491065 +0000 UTC m=+0.034110709 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T07:59:35.102 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:34 smithi039 podman[32272]: 2024-07-04 07:59:34.674924774 +0000 UTC m=+4.261544465 container create fcb3d6dbfbd2b59ef8f87388db058d22b322d6d38d990c4934d1082a0fde5aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-04T07:59:39.594 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-04T07:59:39.594 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-07-04T07:59:39.609 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-04T07:59:39.610 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-07-04T07:59:39.610 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-07-04T07:59:39.610 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to enable service . firewalld.service is not available 2024-07-04T07:59:39.630 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-04T07:59:39.630 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-07-04T07:59:39.647 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-04T07:59:39.647 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-07-04T07:59:39.648 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-07-04T07:59:39.648 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-07-04T07:59:39.648 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr to start... 2024-07-04T07:59:39.648 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr... 2024-07-04T07:59:39.795 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:39 smithi039 podman[32272]: 2024-07-04 07:59:39.452632546 +0000 UTC m=+9.039252224 container init fcb3d6dbfbd2b59ef8f87388db058d22b322d6d38d990c4934d1082a0fde5aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b) 2024-07-04T07:59:39.795 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:39 smithi039 podman[32272]: 2024-07-04 07:59:39.463421221 +0000 UTC m=+9.050040925 container start fcb3d6dbfbd2b59ef8f87388db058d22b322d6d38d990c4934d1082a0fde5aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T07:59:39.795 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:39 smithi039 bash[32272]: fcb3d6dbfbd2b59ef8f87388db058d22b322d6d38d990c4934d1082a0fde5aa6 2024-07-04T07:59:39.795 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:39 smithi039 systemd[1]: Started Ceph mgr.x for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T07:59:40.638 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:40 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1209996961' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-04T07:59:41.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:41.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-07-04T07:59:41.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T07:59:41.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-07-04T07:59:41.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-04T07:59:41.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-04T07:59:41.001 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-04T07:59:41.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:41.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-04T07:59:41.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-04T07:59:41.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-07-04T07:59:41.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-07-04T07:59:41.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-04T07:59:41.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-07-04T07:59:41.002 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-04T07:59:41.004 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-04T07:59:41.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-04T07:59:41.005 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-04T07:59:41.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:41.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-04T07:59:41.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-04T07:59:41.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-04T07:59:41.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-04T07:59:41.008 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "btime": "2024-07-04T07:59:21:009701+0000", 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:41.009 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-07-04T07:59:21.011052+0000", 2024-07-04T07:59:41.010 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-04T07:59:41.011 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:41.011 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-04T07:59:41.011 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-07-04T07:59:41.011 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (1/15)... 2024-07-04T07:59:43.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:43 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2633133400' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-04T07:59:44.501 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-07-04T07:59:44.502 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-04T07:59:44.503 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-04T07:59:44.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-04T07:59:44.504 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-04T07:59:44.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:44.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-04T07:59:44.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-04T07:59:44.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-04T07:59:44.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-04T07:59:44.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-04T07:59:44.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-04T07:59:44.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-04T07:59:44.507 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "btime": "2024-07-04T07:59:21:009701+0000", 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-04T07:59:44.508 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-07-04T07:59:21.011052+0000", 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-07-04T07:59:44.509 INFO:teuthology.orchestra.run.smithi039.stdout:mgr not available, waiting (2/15)... 2024-07-04T07:59:46.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: Activating manager daemon x 2024-07-04T07:59:46.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: mgrmap e2: x(active, starting, since 0.014089s) 2024-07-04T07:59:46.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-04T07:59:46.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-04T07:59:46.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-04T07:59:46.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T07:59:46.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-04T07:59:46.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: Manager daemon x is now available 2024-07-04T07:59:46.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-04T07:59:46.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' 2024-07-04T07:59:46.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-04T07:59:46.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:46 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' 2024-07-04T07:59:47.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:47 smithi039 ceph-mon[32062]: from='mgr.14100 172.21.15.39:0/2162045273' entity='mgr.x' 2024-07-04T07:59:47.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:47 smithi039 ceph-mon[32062]: mgrmap e3: x(active, since 1.02922s) 2024-07-04T07:59:47.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:47 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3530536919' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-04T07:59:47.945 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "health": { 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 0 2024-07-04T07:59:47.946 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "a" 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "quorum_age": 19, 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-04T07:59:47.947 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-04T07:59:47.948 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "btime": "2024-07-04T07:59:21:009701+0000", 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-04T07:59:47.949 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "restful" 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ], 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "modified": "2024-07-04T07:59:21.011052+0000", 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout }, 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-07-04T07:59:47.950 INFO:teuthology.orchestra.run.smithi039.stdout:mgr is available 2024-07-04T07:59:49.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:49 smithi039 ceph-mon[32062]: mgrmap e4: x(active, since 2s) 2024-07-04T07:59:49.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:49 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1604132461' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-04T07:59:49.461 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:49.461 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [global] 2024-07-04T07:59:49.461 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout fsid = 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T07:59:49.462 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-04T07:59:49.462 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.39:3300,v1:172.21.15.39:6789] 2024-07-04T07:59:49.462 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-04T07:59:49.462 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout [osd] 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-04T07:59:49.464 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-04T07:59:49.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-04T07:59:49.465 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-04T07:59:49.465 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling cephadm module... 2024-07-04T07:59:50.848 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:50 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: ignoring --setuser ceph since I am not root 2024-07-04T07:59:50.848 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:50 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: ignoring --setgroup ceph since I am not root 2024-07-04T07:59:50.848 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:50 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1752115398' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-07-04T07:59:51.498 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:51 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1752115398' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-07-04T07:59:51.498 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:51 smithi039 ceph-mon[32062]: mgrmap e5: x(active, since 4s) 2024-07-04T07:59:52.599 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:52 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2529887854' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-04T07:59:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-07-04T07:59:52.608 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-07-04T07:59:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-04T07:59:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-04T07:59:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-04T07:59:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-07-04T07:59:52.609 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-07-04T07:59:52.610 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 5... 2024-07-04T07:59:57.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: Active manager daemon x restarted 2024-07-04T07:59:57.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: Activating manager daemon x 2024-07-04T07:59:57.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: osdmap e2: 0 total, 0 up, 0 in 2024-07-04T07:59:57.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: mgrmap e6: x(active, starting, since 0.0138334s) 2024-07-04T07:59:57.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T07:59:57.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-04T07:59:57.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-04T07:59:57.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-04T07:59:57.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-04T07:59:57.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: Manager daemon x is now available 2024-07-04T07:59:57.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:57 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T07:59:58.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:58 smithi039 ceph-mon[32062]: Found migration_current of "None". Setting to last migration. 2024-07-04T07:59:58.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:58 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T07:59:58.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:58 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T07:59:58.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:58 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T07:59:58.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:58 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-04T07:59:58.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:58 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-04T07:59:58.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:58 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T07:59:58.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:58 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T07:59:58.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:58 smithi039 ceph-mon[32062]: mgrmap e7: x(active, since 1.01776s) 2024-07-04T07:59:58.850 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-07-04T07:59:58.851 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-07-04T07:59:58.851 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-04T07:59:58.851 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-07-04T07:59:58.851 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 5 is available 2024-07-04T07:59:58.851 INFO:teuthology.orchestra.run.smithi039.stdout:Setting orchestrator backend to cephadm... 2024-07-04T07:59:59.493 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-mon[32062]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-04T07:59:59.493 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-mon[32062]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-04T07:59:59.852 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T07:59:59.491+0000 7f6e7ce00640 -1 log_channel(cephadm) log [ERR] : [04/Jul/2024:07:59:59] ENGINE Error in HTTPServer.serve 2024-07-04T07:59:59.852 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: Traceback (most recent call last): 2024-07-04T07:59:59.852 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-04T07:59:59.852 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self._connections.run(self.expiration_interval) 2024-07-04T07:59:59.852 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-04T07:59:59.852 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self._run(expiration_interval) 2024-07-04T07:59:59.852 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: new_conn = self._from_server_socket(self.server.socket) 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: s = self.context.wrap_socket( 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: return self.sslsocket_class._create( 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self.do_handshake() 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self._sslobj.do_handshake() 2024-07-04T07:59:59.853 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-04T07:59:59.854 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 07:59:59 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:00:00.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:00 smithi039 ceph-mon[32062]: mgrmap e8: x(active, since 2s) 2024-07-04T08:00:00.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:00 smithi039 ceph-mon[32062]: [04/Jul/2024:07:59:59] ENGINE Bus STARTING 2024-07-04T08:00:00.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:00 smithi039 ceph-mon[32062]: [04/Jul/2024:07:59:59] ENGINE Serving on http://172.21.15.39:8765 2024-07-04T08:00:00.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:00 smithi039 ceph-mon[32062]: [04/Jul/2024:07:59:59] ENGINE Serving on https://172.21.15.39:7150 2024-07-04T08:00:00.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:00 smithi039 ceph-mon[32062]: [04/Jul/2024:07:59:59] ENGINE Bus STARTED 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:00 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:00 smithi039 ceph-mon[32062]: [04/Jul/2024:07:59:59] ENGINE Error in HTTPServer.serve 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout: Traceback (most recent call last): 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._connections.run(self.expiration_interval) 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._run(expiration_interval) 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-04T08:00:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout: s = self.context.wrap_socket( 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout: return self.sslsocket_class._create( 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout: self.do_handshake() 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._sslobj.do_handshake() 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:00 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:00.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:00 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:00:01.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:01 smithi039 ceph-mon[32062]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:01.909 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout value unchanged 2024-07-04T08:00:01.910 INFO:teuthology.orchestra.run.smithi039.stdout:Generating ssh key... 2024-07-04T08:00:02.446 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:02 smithi039 ceph-mon[32062]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:03.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: Generating public/private rsa key pair. 2024-07-04T08:00:03.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: Your identification has been saved in /tmp/tmpp7x01tm_/key 2024-07-04T08:00:03.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: Your public key has been saved in /tmp/tmpp7x01tm_/key.pub 2024-07-04T08:00:03.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: The key fingerprint is: 2024-07-04T08:00:03.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: SHA256:2dJ7rwJDSYSSR3GA2AdAqpRoiL0Opp8cY/MaPc86/T0 ceph-2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:00:03.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: The key's randomart image is: 2024-07-04T08:00:03.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: +---[RSA 3072]----+ 2024-07-04T08:00:03.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: | .o+.=+=o | 2024-07-04T08:00:03.602 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: |+oo = +.. | 2024-07-04T08:00:03.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: |=+. + . . | 2024-07-04T08:00:03.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: |+ . o+ | 2024-07-04T08:00:03.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: |o.. .S o | 2024-07-04T08:00:03.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: |oo . o. . | 2024-07-04T08:00:03.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: |. B o. o. . | 2024-07-04T08:00:03.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: | + B.+. .E. . | 2024-07-04T08:00:03.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: | =.ooo.. .o... | 2024-07-04T08:00:03.603 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: +----[SHA256]-----+ 2024-07-04T08:00:04.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:04 smithi039 ceph-mon[32062]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:04.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:04 smithi039 ceph-mon[32062]: Generating ssh key... 2024-07-04T08:00:04.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:04 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:04.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:04 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:05.566 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsjZdPZixEdkE3MHdLcPOJfwczme77bEeZ8U3yshDJE0402sA2CYS3gP89wD6tuuawXarIMZkhjUFY9njPwnV5BLODfBNisBnh2XqEn2xrhETPJO5rUEm1OyjKxqOnRfloswVWwaqOxhplbJyIh0l/ywLdI4nVOaQ22VPJ3S0R3cYdgkE25wRfRVWIYhWBEiuYfh65kJ8jeG0WImcdRtt5P+ireSK0NomWbzgEAfWTyc7FvcAIK1O0PBvHJ3EjB+RQdk1E7QNFcU/++jtPZAXaOeEhtu2N5JNYvI1LgIRSB2gVjgVha4jVvxZhfuhEA64bh7tSso+EFqeEjnFEtScTkK/5OljmOv82PcDIf7oQPdKLREXPivEwbcncWDtEB0bmjuSok7FioAxSXgEoFmuZjLY4Mm7TSkVzG7kjlnOxnzAmqeqnd+FVghMfQJNh9M/0MRc0ROCNmN+GT02vHMHP1jKByjYGs4nIaSazQifUwwf/+IObtPVCFDHXiZMLCB8= ceph-2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:00:05.568 INFO:teuthology.orchestra.run.smithi039.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-07-04T08:00:05.568 INFO:teuthology.orchestra.run.smithi039.stdout:Adding key to root@localhost authorized_keys... 2024-07-04T08:00:05.570 INFO:teuthology.orchestra.run.smithi039.stdout:Adding host smithi039... 2024-07-04T08:00:05.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:05 smithi039 ceph-mon[32062]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:06.943 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:06 smithi039 ceph-mon[32062]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi039", "addr": "172.21.15.39", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:08.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:08 smithi039 ceph-mon[32062]: Deploying cephadm binary to smithi039 2024-07-04T08:00:12.905 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Added host 'smithi039' with addr '172.21.15.39' 2024-07-04T08:00:12.906 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mon service... 2024-07-04T08:00:13.478 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:13 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:13.478 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:13 smithi039 ceph-mon[32062]: Added host smithi039 2024-07-04T08:00:13.478 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:13 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:00:14.849 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-07-04T08:00:14.849 INFO:teuthology.orchestra.run.smithi039.stdout:Deploying unmanaged mgr service... 2024-07-04T08:00:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:14 smithi039 ceph-mon[32062]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:14 smithi039 ceph-mon[32062]: Saving service mon spec with placement count:5 2024-07-04T08:00:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:14 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:16.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:16 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:16.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:16 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:16.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:16 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:16.702 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-07-04T08:00:17.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:17 smithi039 ceph-mon[32062]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:17.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:17 smithi039 ceph-mon[32062]: Saving service mgr spec with placement count:2 2024-07-04T08:00:18.756 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:18 smithi039 ceph-mon[32062]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:00:18.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:18 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1868161415' entity='client.admin' 2024-07-04T08:00:18.757 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:18 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:20.258 INFO:teuthology.orchestra.run.smithi039.stdout:Enabling the dashboard module... 2024-07-04T08:00:20.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:20 smithi039 ceph-mon[32062]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:00:20.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:20 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2794163396' entity='client.admin' 2024-07-04T08:00:20.808 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:20 smithi039 ceph-mon[32062]: from='mgr.14118 172.21.15.39:0/2616029546' entity='mgr.x' 2024-07-04T08:00:21.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:21 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/4113187514' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-07-04T08:00:21.851 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:21 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: ignoring --setuser ceph since I am not root 2024-07-04T08:00:21.851 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:21 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: ignoring --setgroup ceph since I am not root 2024-07-04T08:00:22.709 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:22 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/4113187514' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-07-04T08:00:22.709 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:22 smithi039 ceph-mon[32062]: mgrmap e9: x(active, since 24s) 2024-07-04T08:00:23.950 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:23 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/617814638' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-04T08:00:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-07-04T08:00:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-07-04T08:00:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-04T08:00:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-04T08:00:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-04T08:00:24.243 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-07-04T08:00:24.244 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for the mgr to restart... 2024-07-04T08:00:24.244 INFO:teuthology.orchestra.run.smithi039.stdout:Waiting for mgr epoch 9... 2024-07-04T08:00:28.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: Active manager daemon x restarted 2024-07-04T08:00:28.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: Activating manager daemon x 2024-07-04T08:00:28.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: osdmap e3: 0 total, 0 up, 0 in 2024-07-04T08:00:28.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: mgrmap e10: x(active, starting, since 0.0127462s) 2024-07-04T08:00:28.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:00:28.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-04T08:00:28.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-04T08:00:28.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-04T08:00:28.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-04T08:00:28.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:28 smithi039 ceph-mon[32062]: Manager daemon x is now available 2024-07-04T08:00:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:29 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:00:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:29 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-04T08:00:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:29 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-04T08:00:29.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:29 smithi039 ceph-mon[32062]: mgrmap e11: x(active, since 1.01269s) 2024-07-04T08:00:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout { 2024-07-04T08:00:30.093 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-07-04T08:00:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-04T08:00:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout } 2024-07-04T08:00:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:mgr epoch 9 is available 2024-07-04T08:00:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:Generating a dashboard self-signed certificate... 2024-07-04T08:00:30.352 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:00:30.186+0000 7fae70a00640 -1 log_channel(cephadm) log [ERR] : [04/Jul/2024:08:00:30] ENGINE Error in HTTPServer.serve 2024-07-04T08:00:30.352 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: Traceback (most recent call last): 2024-07-04T08:00:30.352 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-04T08:00:30.352 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self._connections.run(self.expiration_interval) 2024-07-04T08:00:30.352 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-04T08:00:30.352 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self._run(expiration_interval) 2024-07-04T08:00:30.352 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: new_conn = self._from_server_socket(self.server.socket) 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: s = self.context.wrap_socket( 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: return self.sslsocket_class._create( 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self.do_handshake() 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self._sslobj.do_handshake() 2024-07-04T08:00:30.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-04T08:00:30.354 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:00:30.746 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-mon[32062]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-04T08:00:30.746 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:30 smithi039 ceph-mon[32062]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-04T08:00:31.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:31 smithi039 ceph-mon[32062]: [04/Jul/2024:08:00:30] ENGINE Bus STARTING 2024-07-04T08:00:31.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:31 smithi039 ceph-mon[32062]: [04/Jul/2024:08:00:30] ENGINE Serving on https://172.21.15.39:7150 2024-07-04T08:00:31.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:31 smithi039 ceph-mon[32062]: [04/Jul/2024:08:00:30] ENGINE Error in HTTPServer.serve 2024-07-04T08:00:31.851 INFO:journalctl@ceph.mon.a.smithi039.stdout: Traceback (most recent call last): 2024-07-04T08:00:31.851 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-04T08:00:31.851 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._connections.run(self.expiration_interval) 2024-07-04T08:00:31.851 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._run(expiration_interval) 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: s = self.context.wrap_socket( 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: return self.sslsocket_class._create( 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: self.do_handshake() 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-04T08:00:31.852 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._sslobj.do_handshake() 2024-07-04T08:00:31.853 INFO:journalctl@ceph.mon.a.smithi039.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-04T08:00:31.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:31 smithi039 ceph-mon[32062]: [04/Jul/2024:08:00:30] ENGINE Serving on http://172.21.15.39:8765 2024-07-04T08:00:31.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:31 smithi039 ceph-mon[32062]: [04/Jul/2024:08:00:30] ENGINE Bus STARTED 2024-07-04T08:00:31.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:31 smithi039 ceph-mon[32062]: mgrmap e12: x(active, since 2s) 2024-07-04T08:00:31.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:31 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:31.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:31 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:31.961 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-07-04T08:00:31.961 INFO:teuthology.orchestra.run.smithi039.stdout:Creating initial admin user... 2024-07-04T08:00:32.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:32 smithi039 ceph-mon[32062]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:32.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:32 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:33.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:33 smithi039 ceph-mon[32062]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:33.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:33 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:33.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:33 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:33.852 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$UMsdq6q1SlxNFaRZmApp7uIP27Wc9F3g7Ot9rV0pjTUDTINOfu8jC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1720080033, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-07-04T08:00:33.853 INFO:teuthology.orchestra.run.smithi039.stdout:Fetching dashboard port number... 2024-07-04T08:00:35.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:35 smithi039 ceph-mon[32062]: mgrmap e13: x(active, since 6s) 2024-07-04T08:00:35.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:35 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2452167740' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-07-04T08:00:35.818 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stdout 8443 2024-07-04T08:00:35.850 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-04T08:00:35.850 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout disabled 2024-07-04T08:00:35.875 INFO:teuthology.orchestra.run.smithi039.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-04T08:00:35.875 INFO:teuthology.orchestra.run.smithi039.stdout:systemctl: stdout inactive 2024-07-04T08:00:35.876 INFO:teuthology.orchestra.run.smithi039.stdout:firewalld.service is not enabled 2024-07-04T08:00:35.876 INFO:teuthology.orchestra.run.smithi039.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-07-04T08:00:35.879 INFO:teuthology.orchestra.run.smithi039.stdout:Ceph Dashboard is now available at: 2024-07-04T08:00:35.879 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: URL: https://smithi039.front.sepia.ceph.com:8443/ 2024-07-04T08:00:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: User: admin 2024-07-04T08:00:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: Password: i83sjqf6hk 2024-07-04T08:00:35.880 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:35.880 INFO:teuthology.orchestra.run.smithi039.stdout:Saving cluster configuration to /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config directory 2024-07-04T08:00:37.785 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-07-04T08:00:37.785 INFO:teuthology.orchestra.run.smithi039.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-07-04T08:00:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-07-04T08:00:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:37.785 INFO:teuthology.orchestra.run.smithi039.stdout:Or, if you are only running a single cluster on this host: 2024-07-04T08:00:37.785 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout:Please consider enabling telemetry to help improve Ceph: 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: ceph telemetry on 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout:For more information see: 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:37.786 INFO:teuthology.orchestra.run.smithi039.stdout:Bootstrap complete. 2024-07-04T08:00:37.901 INFO:tasks.cephadm:Fetching config... 2024-07-04T08:00:37.902 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:00:37.902 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-07-04T08:00:37.950 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-07-04T08:00:37.950 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:00:37.950 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-07-04T08:00:38.035 INFO:tasks.cephadm:Fetching mon keyring... 2024-07-04T08:00:38.035 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:00:38.035 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/keyring of=/dev/stdout 2024-07-04T08:00:38.169 INFO:tasks.cephadm:Fetching pub ssh key... 2024-07-04T08:00:38.169 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:00:38.169 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-07-04T08:00:38.227 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-07-04T08:00:38.228 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsjZdPZixEdkE3MHdLcPOJfwczme77bEeZ8U3yshDJE0402sA2CYS3gP89wD6tuuawXarIMZkhjUFY9njPwnV5BLODfBNisBnh2XqEn2xrhETPJO5rUEm1OyjKxqOnRfloswVWwaqOxhplbJyIh0l/ywLdI4nVOaQ22VPJ3S0R3cYdgkE25wRfRVWIYhWBEiuYfh65kJ8jeG0WImcdRtt5P+ireSK0NomWbzgEAfWTyc7FvcAIK1O0PBvHJ3EjB+RQdk1E7QNFcU/++jtPZAXaOeEhtu2N5JNYvI1LgIRSB2gVjgVha4jVvxZhfuhEA64bh7tSso+EFqeEjnFEtScTkK/5OljmOv82PcDIf7oQPdKLREXPivEwbcncWDtEB0bmjuSok7FioAxSXgEoFmuZjLY4Mm7TSkVzG7kjlnOxnzAmqeqnd+FVghMfQJNh9M/0MRc0ROCNmN+GT02vHMHP1jKByjYGs4nIaSazQifUwwf/+IObtPVCFDHXiZMLCB8= ceph-2ee561ca-39db-11ef-bcab-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-04T08:00:38.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:38 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3788368194' entity='client.admin' 2024-07-04T08:00:38.329 INFO:teuthology.orchestra.run.smithi039.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsjZdPZixEdkE3MHdLcPOJfwczme77bEeZ8U3yshDJE0402sA2CYS3gP89wD6tuuawXarIMZkhjUFY9njPwnV5BLODfBNisBnh2XqEn2xrhETPJO5rUEm1OyjKxqOnRfloswVWwaqOxhplbJyIh0l/ywLdI4nVOaQ22VPJ3S0R3cYdgkE25wRfRVWIYhWBEiuYfh65kJ8jeG0WImcdRtt5P+ireSK0NomWbzgEAfWTyc7FvcAIK1O0PBvHJ3EjB+RQdk1E7QNFcU/++jtPZAXaOeEhtu2N5JNYvI1LgIRSB2gVjgVha4jVvxZhfuhEA64bh7tSso+EFqeEjnFEtScTkK/5OljmOv82PcDIf7oQPdKLREXPivEwbcncWDtEB0bmjuSok7FioAxSXgEoFmuZjLY4Mm7TSkVzG7kjlnOxnzAmqeqnd+FVghMfQJNh9M/0MRc0ROCNmN+GT02vHMHP1jKByjYGs4nIaSazQifUwwf/+IObtPVCFDHXiZMLCB8= ceph-2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:00:38.387 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsjZdPZixEdkE3MHdLcPOJfwczme77bEeZ8U3yshDJE0402sA2CYS3gP89wD6tuuawXarIMZkhjUFY9njPwnV5BLODfBNisBnh2XqEn2xrhETPJO5rUEm1OyjKxqOnRfloswVWwaqOxhplbJyIh0l/ywLdI4nVOaQ22VPJ3S0R3cYdgkE25wRfRVWIYhWBEiuYfh65kJ8jeG0WImcdRtt5P+ireSK0NomWbzgEAfWTyc7FvcAIK1O0PBvHJ3EjB+RQdk1E7QNFcU/++jtPZAXaOeEhtu2N5JNYvI1LgIRSB2gVjgVha4jVvxZhfuhEA64bh7tSso+EFqeEjnFEtScTkK/5OljmOv82PcDIf7oQPdKLREXPivEwbcncWDtEB0bmjuSok7FioAxSXgEoFmuZjLY4Mm7TSkVzG7kjlnOxnzAmqeqnd+FVghMfQJNh9M/0MRc0ROCNmN+GT02vHMHP1jKByjYGs4nIaSazQifUwwf/+IObtPVCFDHXiZMLCB8= ceph-2ee561ca-39db-11ef-bcab-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-04T08:00:38.430 INFO:teuthology.orchestra.run.smithi050.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsjZdPZixEdkE3MHdLcPOJfwczme77bEeZ8U3yshDJE0402sA2CYS3gP89wD6tuuawXarIMZkhjUFY9njPwnV5BLODfBNisBnh2XqEn2xrhETPJO5rUEm1OyjKxqOnRfloswVWwaqOxhplbJyIh0l/ywLdI4nVOaQ22VPJ3S0R3cYdgkE25wRfRVWIYhWBEiuYfh65kJ8jeG0WImcdRtt5P+ireSK0NomWbzgEAfWTyc7FvcAIK1O0PBvHJ3EjB+RQdk1E7QNFcU/++jtPZAXaOeEhtu2N5JNYvI1LgIRSB2gVjgVha4jVvxZhfuhEA64bh7tSso+EFqeEjnFEtScTkK/5OljmOv82PcDIf7oQPdKLREXPivEwbcncWDtEB0bmjuSok7FioAxSXgEoFmuZjLY4Mm7TSkVzG7kjlnOxnzAmqeqnd+FVghMfQJNh9M/0MRc0ROCNmN+GT02vHMHP1jKByjYGs4nIaSazQifUwwf/+IObtPVCFDHXiZMLCB8= ceph-2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:00:38.442 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsjZdPZixEdkE3MHdLcPOJfwczme77bEeZ8U3yshDJE0402sA2CYS3gP89wD6tuuawXarIMZkhjUFY9njPwnV5BLODfBNisBnh2XqEn2xrhETPJO5rUEm1OyjKxqOnRfloswVWwaqOxhplbJyIh0l/ywLdI4nVOaQ22VPJ3S0R3cYdgkE25wRfRVWIYhWBEiuYfh65kJ8jeG0WImcdRtt5P+ireSK0NomWbzgEAfWTyc7FvcAIK1O0PBvHJ3EjB+RQdk1E7QNFcU/++jtPZAXaOeEhtu2N5JNYvI1LgIRSB2gVjgVha4jVvxZhfuhEA64bh7tSso+EFqeEjnFEtScTkK/5OljmOv82PcDIf7oQPdKLREXPivEwbcncWDtEB0bmjuSok7FioAxSXgEoFmuZjLY4Mm7TSkVzG7kjlnOxnzAmqeqnd+FVghMfQJNh9M/0MRc0ROCNmN+GT02vHMHP1jKByjYGs4nIaSazQifUwwf/+IObtPVCFDHXiZMLCB8= ceph-2ee561ca-39db-11ef-bcab-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-04T08:00:38.482 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsjZdPZixEdkE3MHdLcPOJfwczme77bEeZ8U3yshDJE0402sA2CYS3gP89wD6tuuawXarIMZkhjUFY9njPwnV5BLODfBNisBnh2XqEn2xrhETPJO5rUEm1OyjKxqOnRfloswVWwaqOxhplbJyIh0l/ywLdI4nVOaQ22VPJ3S0R3cYdgkE25wRfRVWIYhWBEiuYfh65kJ8jeG0WImcdRtt5P+ireSK0NomWbzgEAfWTyc7FvcAIK1O0PBvHJ3EjB+RQdk1E7QNFcU/++jtPZAXaOeEhtu2N5JNYvI1LgIRSB2gVjgVha4jVvxZhfuhEA64bh7tSso+EFqeEjnFEtScTkK/5OljmOv82PcDIf7oQPdKLREXPivEwbcncWDtEB0bmjuSok7FioAxSXgEoFmuZjLY4Mm7TSkVzG7kjlnOxnzAmqeqnd+FVghMfQJNh9M/0MRc0ROCNmN+GT02vHMHP1jKByjYGs4nIaSazQifUwwf/+IObtPVCFDHXiZMLCB8= ceph-2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:00:38.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-07-04T08:00:38.943 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:00:42.633 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-07-04T08:00:42.634 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-07-04T08:00:43.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:42 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1973563841' entity='client.admin' 2024-07-04T08:00:43.550 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:00:44.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:44 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:44.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:44 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:44.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:44 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:00:44.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:44 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:44.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:44 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:00:44.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:44 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:46.638 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi050 2024-07-04T08:00:46.638 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T08:00:46.638 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.conf 2024-07-04T08:00:46.657 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T08:00:46.658 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:00:46.716 INFO:tasks.cephadm:Adding host smithi050 to orchestrator... 2024-07-04T08:00:46.716 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch host add smithi050 2024-07-04T08:00:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:46 smithi039 ceph-mon[32062]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:46 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:46 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:00:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:46 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:00:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:46 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:00:47.151 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:00:48.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:47 smithi039 ceph-mon[32062]: Updating smithi039:/etc/ceph/ceph.conf 2024-07-04T08:00:48.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:47 smithi039 ceph-mon[32062]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:00:48.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:48 smithi039 ceph-mon[32062]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:00:48.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:48 smithi039 ceph-mon[32062]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:00:48.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:48 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:48.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:48 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:48.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:48 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:50.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:49 smithi039 ceph-mon[32062]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:00:51.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:50 smithi039 ceph-mon[32062]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:00:52.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:51 smithi039 ceph-mon[32062]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:00:52.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:51 smithi039 ceph-mon[32062]: Deploying cephadm binary to smithi050 2024-07-04T08:00:53.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:52 smithi039 ceph-mon[32062]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:00:54.092 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi050' with addr '172.21.15.50' 2024-07-04T08:00:54.869 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch host ls --format=json 2024-07-04T08:00:55.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:55 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:55.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:55 smithi039 ceph-mon[32062]: Added host smithi050 2024-07-04T08:00:55.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:55 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:00:55.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:55 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:55.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:55 smithi039 ceph-mon[32062]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:00:55.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:55 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:00:56.805 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:56 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:57.568 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:00:57.569 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}] 2024-07-04T08:00:57.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:57 smithi039 ceph-mon[32062]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:00:58.319 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi133 2024-07-04T08:00:58.320 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T08:00:58.320 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.conf 2024-07-04T08:00:58.339 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T08:00:58.339 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:00:58.397 INFO:tasks.cephadm:Adding host smithi133 to orchestrator... 2024-07-04T08:00:58.398 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch host add smithi133 2024-07-04T08:00:58.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:58 smithi039 ceph-mon[32062]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:00:58.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:58 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:58.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:58 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:58.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:58 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:58.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:58 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:58.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:58 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:00:58.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:58 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:00:58.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:58 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:00:58.877 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:00:59.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:59 smithi039 ceph-mon[32062]: Updating smithi050:/etc/ceph/ceph.conf 2024-07-04T08:00:59.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:59 smithi039 ceph-mon[32062]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:00:59.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:59 smithi039 ceph-mon[32062]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:00:59.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:59 smithi039 ceph-mon[32062]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:00:59.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:59 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:59.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:59 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:00:59.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:00:59 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:00.666 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:00 smithi039 ceph-mon[32062]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:01.764 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:01 smithi039 ceph-mon[32062]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:02.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:02 smithi039 ceph-mon[32062]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:01:03.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:03 smithi039 ceph-mon[32062]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:03.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:03 smithi039 ceph-mon[32062]: Deploying cephadm binary to smithi133 2024-07-04T08:01:05.489 INFO:teuthology.orchestra.run.smithi039.stdout:Added host 'smithi133' with addr '172.21.15.133' 2024-07-04T08:01:05.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:05 smithi039 ceph-mon[32062]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:05.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:05 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:05.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:05 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:06.234 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch host ls --format=json 2024-07-04T08:01:06.710 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:01:07.044 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:06 smithi039 ceph-mon[32062]: Added host smithi133 2024-07-04T08:01:07.044 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:06 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:07.045 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:06 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:07.923 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:07 smithi039 ceph-mon[32062]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:07.923 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:07 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:01:08.935 INFO:teuthology.orchestra.run.smithi039.stdout:[{"addr": "172.21.15.39", "hostname": "smithi039", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}] 2024-07-04T08:01:09.645 INFO:tasks.cephadm:Setting crush tunables to default 2024-07-04T08:01:09.645 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd crush tunables default 2024-07-04T08:01:09.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:09 smithi039 ceph-mon[32062]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:09.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:09 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:10.113 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:01:10.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:10 smithi039 ceph-mon[32062]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:01:10.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:10 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:10.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:10 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:10.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:10 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:10.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:10 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:10.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:10 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:10.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:10 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:10.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:10 smithi039 ceph-mon[32062]: Updating smithi133:/etc/ceph/ceph.conf 2024-07-04T08:01:11.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:11 smithi039 ceph-mon[32062]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:11.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:11 smithi039 ceph-mon[32062]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:11.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:11 smithi039 ceph-mon[32062]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:11.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:11 smithi039 ceph-mon[32062]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:11.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:11 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:11.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:11 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:11.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:11 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:12.834 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-07-04T08:01:13.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:12 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/712624071' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-04T08:01:13.546 INFO:tasks.cephadm:Adding mon.a on smithi039 2024-07-04T08:01:13.546 INFO:tasks.cephadm:Adding mon.b on smithi050 2024-07-04T08:01:13.547 INFO:tasks.cephadm:Adding mon.c on smithi133 2024-07-04T08:01:13.547 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch apply mon '3;smithi039:172.21.15.39=a;smithi050:172.21.15.50=b;smithi133:172.21.15.133=c' 2024-07-04T08:01:13.798 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:14.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:13 smithi039 ceph-mon[32062]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:14.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:13 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/712624071' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-04T08:01:14.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:13 smithi039 ceph-mon[32062]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:15.394 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mon update... 2024-07-04T08:01:15.908 DEBUG:teuthology.orchestra.run.smithi050:mon.b> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.b.service 2024-07-04T08:01:15.910 DEBUG:teuthology.orchestra.run.smithi133:mon.c> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.c.service 2024-07-04T08:01:15.913 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-04T08:01:15.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-04T08:01:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:15 smithi039 ceph-mon[32062]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:16.124 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:15 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:16.125 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:15 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:16.125 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:15 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:16.125 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:15 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:16.125 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:15 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:16.125 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:15 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:16.125 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:15 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:16.612 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:17.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:16 smithi039 ceph-mon[32062]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi050:172.21.15.50=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:01:17.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:16 smithi039 ceph-mon[32062]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi050:172.21.15.50=b;smithi133:172.21.15.133=c;count:3 2024-07-04T08:01:17.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:16 smithi039 ceph-mon[32062]: Deploying daemon mon.c on smithi133 2024-07-04T08:01:18.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:17 smithi039 ceph-mon[32062]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:19.349 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T08:01:19.349 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","modified":"2024-07-04T07:59:16.806394Z","created":"2024-07-04T07:59:16.806394Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-04T08:01:19.349 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 1 2024-07-04T08:01:19.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:18 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/1782760366' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-04T08:01:19.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:18 smithi039 ceph-mon[32062]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:20.847 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-04T08:01:20.848 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.allow_ingest_behind: 0 2024-07-04T08:01:20.848 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.two_write_queues: 0 2024-07-04T08:01:20.848 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.manual_wal_flush: 0 2024-07-04T08:01:20.848 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.wal_compression: 0 2024-07-04T08:01:20.848 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.atomic_flush: 0 2024-07-04T08:01:20.849 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-04T08:01:20.849 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-04T08:01:20.849 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-04T08:01:20.849 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.log_readahead_size: 0 2024-07-04T08:01:20.849 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-04T08:01:20.849 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.best_efforts_recovery: 0 2024-07-04T08:01:20.849 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-04T08:01:20.849 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-04T08:01:20.849 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.allow_data_in_errors: 0 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.db_host_id: __hostname__ 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_background_jobs: 2 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_background_compactions: -1 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_subcompactions: 1 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_total_wal_size: 0 2024-07-04T08:01:20.850 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-04T08:01:20.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-04T08:01:20.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-04T08:01:20.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-04T08:01:20.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_open_files: -1 2024-07-04T08:01:20.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bytes_per_sync: 0 2024-07-04T08:01:20.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-04T08:01:20.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-04T08:01:20.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_readahead_size: 0 2024-07-04T08:01:20.852 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_background_flushes: -1 2024-07-04T08:01:20.852 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Compression algorithms supported: 2024-07-04T08:01:20.852 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: kZSTD supported: 0 2024-07-04T08:01:20.852 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: kXpressCompression supported: 0 2024-07-04T08:01:20.852 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: kBZip2Compression supported: 0 2024-07-04T08:01:20.852 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-04T08:01:20.852 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: kLZ4Compression supported: 1 2024-07-04T08:01:20.853 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: kZlibCompression supported: 1 2024-07-04T08:01:20.853 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: kLZ4HCCompression supported: 1 2024-07-04T08:01:20.853 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: kSnappyCompression supported: 1 2024-07-04T08:01:20.853 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-04T08:01:20.853 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-04T08:01:20.854 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-07-04T08:01:20.854 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-04T08:01:20.854 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-04T08:01:20.854 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.merge_operator: 2024-07-04T08:01:20.854 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_filter: None 2024-07-04T08:01:20.854 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_filter_factory: None 2024-07-04T08:01:20.855 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.sst_partitioner_factory: None 2024-07-04T08:01:20.855 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-04T08:01:20.855 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-04T08:01:20.855 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c54ba76ce0) 2024-07-04T08:01:20.855 INFO:journalctl@ceph.mon.c.smithi133.stdout: cache_index_and_filter_blocks: 1 2024-07-04T08:01:20.855 INFO:journalctl@ceph.mon.c.smithi133.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-04T08:01:20.856 INFO:journalctl@ceph.mon.c.smithi133.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-04T08:01:20.856 INFO:journalctl@ceph.mon.c.smithi133.stdout: pin_top_level_index_and_filter: 1 2024-07-04T08:01:20.856 INFO:journalctl@ceph.mon.c.smithi133.stdout: index_type: 0 2024-07-04T08:01:20.856 INFO:journalctl@ceph.mon.c.smithi133.stdout: data_block_index_type: 0 2024-07-04T08:01:20.856 INFO:journalctl@ceph.mon.c.smithi133.stdout: index_shortening: 1 2024-07-04T08:01:20.857 INFO:journalctl@ceph.mon.c.smithi133.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-04T08:01:20.857 INFO:journalctl@ceph.mon.c.smithi133.stdout: checksum: 4 2024-07-04T08:01:20.857 INFO:journalctl@ceph.mon.c.smithi133.stdout: no_block_cache: 0 2024-07-04T08:01:20.857 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_cache: 0x55c547a3b610 2024-07-04T08:01:20.857 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_cache_name: BinnedLRUCache 2024-07-04T08:01:20.857 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_cache_options: 2024-07-04T08:01:20.858 INFO:journalctl@ceph.mon.c.smithi133.stdout: capacity : 536870912 2024-07-04T08:01:20.858 INFO:journalctl@ceph.mon.c.smithi133.stdout: num_shard_bits : 4 2024-07-04T08:01:20.858 INFO:journalctl@ceph.mon.c.smithi133.stdout: strict_capacity_limit : 0 2024-07-04T08:01:20.858 INFO:journalctl@ceph.mon.c.smithi133.stdout: high_pri_pool_ratio: 0.000 2024-07-04T08:01:20.858 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_cache_compressed: (nil) 2024-07-04T08:01:20.859 INFO:journalctl@ceph.mon.c.smithi133.stdout: persistent_cache: (nil) 2024-07-04T08:01:20.859 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_size: 4096 2024-07-04T08:01:20.859 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_size_deviation: 10 2024-07-04T08:01:20.859 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_restart_interval: 16 2024-07-04T08:01:20.859 INFO:journalctl@ceph.mon.c.smithi133.stdout: index_block_restart_interval: 1 2024-07-04T08:01:20.859 INFO:journalctl@ceph.mon.c.smithi133.stdout: metadata_block_size: 4096 2024-07-04T08:01:20.859 INFO:journalctl@ceph.mon.c.smithi133.stdout: partition_filters: 0 2024-07-04T08:01:20.860 INFO:journalctl@ceph.mon.c.smithi133.stdout: use_delta_encoding: 1 2024-07-04T08:01:20.860 INFO:journalctl@ceph.mon.c.smithi133.stdout: filter_policy: bloomfilter 2024-07-04T08:01:20.860 INFO:journalctl@ceph.mon.c.smithi133.stdout: whole_key_filtering: 1 2024-07-04T08:01:20.860 INFO:journalctl@ceph.mon.c.smithi133.stdout: verify_compression: 0 2024-07-04T08:01:20.860 INFO:journalctl@ceph.mon.c.smithi133.stdout: read_amp_bytes_per_bit: 0 2024-07-04T08:01:20.861 INFO:journalctl@ceph.mon.c.smithi133.stdout: format_version: 5 2024-07-04T08:01:20.861 INFO:journalctl@ceph.mon.c.smithi133.stdout: enable_index_compression: 1 2024-07-04T08:01:20.861 INFO:journalctl@ceph.mon.c.smithi133.stdout: block_align: 0 2024-07-04T08:01:20.861 INFO:journalctl@ceph.mon.c.smithi133.stdout: max_auto_readahead_size: 262144 2024-07-04T08:01:20.861 INFO:journalctl@ceph.mon.c.smithi133.stdout: prepopulate_block_cache: 0 2024-07-04T08:01:20.861 INFO:journalctl@ceph.mon.c.smithi133.stdout: initial_auto_readahead_size: 8192 2024-07-04T08:01:20.863 INFO:journalctl@ceph.mon.c.smithi133.stdout: num_file_reads_for_auto_readahead: 2 2024-07-04T08:01:20.863 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.write_buffer_size: 33554432 2024-07-04T08:01:20.863 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_write_buffer_number: 2 2024-07-04T08:01:20.863 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression: NoCompression 2024-07-04T08:01:20.864 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression: Disabled 2024-07-04T08:01:20.864 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.prefix_extractor: nullptr 2024-07-04T08:01:20.864 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-04T08:01:20.864 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.num_levels: 7 2024-07-04T08:01:20.864 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-04T08:01:20.865 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-04T08:01:20.865 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-04T08:01:20.865 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-04T08:01:20.865 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-04T08:01:20.865 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-04T08:01:20.865 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-04T08:01:20.865 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-04T08:01:20.866 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-04T08:01:20.866 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-04T08:01:20.866 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-04T08:01:20.866 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-04T08:01:20.866 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-04T08:01:20.866 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression_opts.level: 32767 2024-07-04T08:01:20.867 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression_opts.strategy: 0 2024-07-04T08:01:20.867 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-04T08:01:20.867 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-04T08:01:20.867 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-04T08:01:20.867 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-04T08:01:20.867 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression_opts.enabled: false 2024-07-04T08:01:20.867 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.target_file_size_base: 67108864 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-04T08:01:20.868 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.arena_block_size: 1048576 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-04T08:01:20.869 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.disable_auto_compactions: 0 2024-07-04T08:01:20.870 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-04T08:01:20.870 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-04T08:01:20.870 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-04T08:01:20.870 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-04T08:01:20.870 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-04T08:01:20.870 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-04T08:01:20.870 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-04T08:01:20.870 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.inplace_update_support: 0 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.bloom_locality: 0 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.max_successive_merges: 0 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.paranoid_file_checks: 0 2024-07-04T08:01:20.871 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.force_consistency_checks: 1 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.report_bg_io_stats: 0 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.ttl: 2592000 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.enable_blob_files: false 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.min_blob_size: 0 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.blob_file_size: 268435456 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-04T08:01:20.872 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.blob_file_starting_level: 0 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cfa238b5-87c9-465f-b95b-695e11af444f 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720080080846234, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720080080846807, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1720080080, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cfa238b5-87c9-465f-b95b-695e11af444f", "db_session_id": "RBMM173NW57WTLWSQFE5", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720080080846934, "job": 1, "event": "recovery_finished"} 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-04T08:01:20.873 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c54ba98a80 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: DB pointer 0x55c54c83a000 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** DB Stats ** 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-04T08:01:20.874 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** Compaction Stats [default] ** 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** Compaction Stats [default] ** 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-04T08:01:20.875 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: Block cache BinnedLRUCache@0x55c547a3b610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2024-07-04T08:01:20.876 INFO:journalctl@ceph.mon.c.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-04T08:01:20.877 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-07-04T08:01:20.877 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: using public_addr v2:172.21.15.133:0/0 -> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] 2024-07-04T08:01:20.877 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: starting mon.c rank -1 at public addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] at bind addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:21.059 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-04T08:01:21.059 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-04T08:01:21.303 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:01:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(???) e0 preinit fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:21.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).mds e1 new map 2024-07-04T08:01:21.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).mds e1 print_map 2024-07-04T08:01:21.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: e1 2024-07-04T08:01:21.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: btime 2024-07-04T07:59:21:009701+0000 2024-07-04T08:01:21.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-04T08:01:21.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-04T08:01:21.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: legacy client fscid: -1 2024-07-04T08:01:21.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: 2024-07-04T08:01:21.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: No filesystems configured 2024-07-04T08:01:21.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-07-04T08:01:21.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-07-04T08:01:21.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-07-04T08:01:21.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-07-04T08:01:21.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-07-04T08:01:21.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-07-04T08:01:21.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-04T08:01:21.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-04T08:01:21.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Added host smithi050 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.319 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.320 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.320 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:21.320 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:21.320 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:21.320 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Updating smithi050:/etc/ceph/ceph.conf 2024-07-04T08:01:21.320 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.320 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:21.320 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Deploying cephadm binary to smithi133 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:21.321 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Added host smithi133 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Updating smithi133:/etc/ceph/ceph.conf 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.323 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/712624071' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/712624071' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:21.324 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:21.325 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi050:172.21.15.50=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:01:21.325 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi050:172.21.15.50=b;smithi133:172.21.15.133=c;count:3 2024-07-04T08:01:21.325 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: Deploying daemon mon.c on smithi133 2024-07-04T08:01:21.325 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.325 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/1782760366' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-04T08:01:21.325 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:21.325 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 ceph-mon[37050]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-04T08:01:21.325 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 bash[37031]: 42ab45a0f5c93e5b87a2c690c1eb787cc8b84d249295ca162ac41ad62afa2384 2024-07-04T08:01:21.325 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:20 smithi133 systemd[1]: Started Ceph mon.c for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:01:26.213 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T08:01:26.213 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","modified":"2024-07-04T08:01:20.890492Z","created":"2024-07-04T07:59:16.806394Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-04T08:01:26.213 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 2 2024-07-04T08:01:26.280 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:01:26.280 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: mon.a calling monitor election 2024-07-04T08:01:26.280 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.280 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.280 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:26.280 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.280 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: mon.c calling monitor election 2024-07-04T08:01:26.280 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: monmap epoch 2 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: last_changed 2024-07-04T08:01:20.890492+0000 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: min_mon_release 19 (squid) 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: election_strategy: 1 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-07-04T08:01:26.281 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: fsmap 2024-07-04T08:01:26.282 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:26.282 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: mgrmap e13: x(active, since 57s) 2024-07-04T08:01:26.282 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: overall HEALTH_OK 2024-07-04T08:01:26.282 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:26.282 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:26.282 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:26.282 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:25 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:26.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:01:26.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: mon.a calling monitor election 2024-07-04T08:01:26.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:26.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: mon.c calling monitor election 2024-07-04T08:01:26.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: monmap epoch 2 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: last_changed 2024-07-04T08:01:20.890492+0000 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: min_mon_release 19 (squid) 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: election_strategy: 1 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-07-04T08:01:26.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: fsmap 2024-07-04T08:01:26.354 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:26.354 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: mgrmap e13: x(active, since 57s) 2024-07-04T08:01:26.354 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: overall HEALTH_OK 2024-07-04T08:01:26.354 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:26.354 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:26.354 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:26.354 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:25 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:26 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:26 smithi133 ceph-mon[37050]: Deploying daemon mon.b on smithi050 2024-07-04T08:01:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:26 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/2566186651' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-04T08:01:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:26 smithi133 ceph-mon[37050]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:26 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:27.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:26 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:27.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:26 smithi039 ceph-mon[32062]: Deploying daemon mon.b on smithi050 2024-07-04T08:01:27.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:26 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/2566186651' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-04T08:01:27.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:26 smithi039 ceph-mon[32062]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:27.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:26 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:27.764 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-04T08:01:27.764 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph mon dump -f json 2024-07-04T08:01:28.020 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:01:29.497 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-07-04T08:01:29.497 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dee1288a-db64-4a0e-8cdd-a41260226395 2024-07-04T08:01:29.497 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720080089496888, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-07-04T08:01:29.497 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-07-04T08:01:29.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:29 smithi133 ceph-mon[37050]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.813 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720080089497473, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1720080089, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dee1288a-db64-4a0e-8cdd-a41260226395", "db_session_id": "5R5QDAHTDHB6MLZCLQOZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: EVENT_LOG_v1 {"time_micros": 1720080089497591, "job": 1, "event": "recovery_finished"} 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564195008a80 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: DB pointer 0x564195daa000 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** DB Stats ** 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-04T08:01:29.814 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-04T08:01:29.815 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-04T08:01:29.815 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-04T08:01:29.817 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-04T08:01:29.817 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-07-04T08:01:29.817 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** Compaction Stats [default] ** 2024-07-04T08:01:29.817 INFO:journalctl@ceph.mon.b.smithi050.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-04T08:01:29.817 INFO:journalctl@ceph.mon.b.smithi050.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** Compaction Stats [default] ** 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-07-04T08:01:29.818 INFO:journalctl@ceph.mon.b.smithi050.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative compaction: 0.00 GB write, 0.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval compaction: 0.00 GB write, 0.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: Block cache BinnedLRUCache@0x564190fab610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-07-04T08:01:29.819 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: using public_addr v2:172.21.15.50:0/0 -> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: starting mon.b rank -1 at public addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] at bind addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(???) e0 preinit fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 bash[36558]: 5233190bdf2600f51b27c7101a48b8a5e5419955f9ccaab427eae13a12d5e8f7 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 systemd[1]: Started Ceph mon.b for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).mds e1 new map 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).mds e1 print_map 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout: e1 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout: btime 2024-07-04T07:59:21:009701+0000 2024-07-04T08:01:29.820 INFO:journalctl@ceph.mon.b.smithi050.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout: legacy client fscid: -1 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout: No filesystems configured 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-04T08:01:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Added host smithi050 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Updating smithi050:/etc/ceph/ceph.conf 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:01:29.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Deploying cephadm binary to smithi133 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Added host smithi133 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:01:29.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Updating smithi133:/etc/ceph/ceph.conf 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/712624071' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/712624071' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi039:172.21.15.39=a;smithi050:172.21.15.50=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Saving service mon spec with placement smithi039:172.21.15.39=a;smithi050:172.21.15.50=b;smithi133:172.21.15.133=c;count:3 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Deploying daemon mon.c on smithi133 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/1782760366' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.a calling monitor election 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:29.828 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.c calling monitor election 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: monmap epoch 2 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: last_changed 2024-07-04T08:01:20.890492+0000 2024-07-04T08:01:29.829 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: min_mon_release 19 (squid) 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: election_strategy: 1 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: fsmap 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mgrmap e13: x(active, since 57s) 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: overall HEALTH_OK 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:29.830 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:29.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: Deploying daemon mon.b on smithi050 2024-07-04T08:01:29.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/2566186651' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-04T08:01:29.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:29.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:29.831 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:29 smithi050 ceph-mon[36577]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-04T08:01:29.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:29 smithi039 ceph-mon[32062]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: mon.c calling monitor election 2024-07-04T08:01:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:01:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: mon.a calling monitor election 2024-07-04T08:01:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: monmap epoch 3 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: last_changed 2024-07-04T08:01:29.529450+0000 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: min_mon_release 19 (squid) 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: election_strategy: 1 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-07-04T08:01:35.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: fsmap 2024-07-04T08:01:35.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:35.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: mgrmap e13: x(active, since 66s) 2024-07-04T08:01:35.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: overall HEALTH_OK 2024-07-04T08:01:35.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:35.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:35.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:35.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:34 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: mon.c calling monitor election 2024-07-04T08:01:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:01:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: mon.a calling monitor election 2024-07-04T08:01:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: monmap epoch 3 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: last_changed 2024-07-04T08:01:29.529450+0000 2024-07-04T08:01:35.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: min_mon_release 19 (squid) 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: election_strategy: 1 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: fsmap 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: mgrmap e13: x(active, since 66s) 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: overall HEALTH_OK 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:35.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:34 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:35.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:35 smithi133 ceph-mon[37050]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:35.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:35 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:35.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:35 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:35.996 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-07-04T08:01:35.996 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","modified":"2024-07-04T08:01:29.529450Z","created":"2024-07-04T07:59:16.806394Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:3300","nonce":0},{"type":"v1","addr":"172.21.15.39:6789","nonce":0}]},"addr":"172.21.15.39:6789/0","public_addr":"172.21.15.39:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-04T08:01:35.996 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 3 2024-07-04T08:01:36.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:35 smithi039 ceph-mon[32062]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:36.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:35 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:36.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:35 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:36.676 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-07-04T08:01:36.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config generate-minimal-conf 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: mon.c calling monitor election 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: mon.a calling monitor election 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:36.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: monmap epoch 3 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: last_changed 2024-07-04T08:01:29.529450+0000 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: min_mon_release 19 (squid) 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: election_strategy: 1 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: fsmap 2024-07-04T08:01:36.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:36.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: mgrmap e13: x(active, since 66s) 2024-07-04T08:01:36.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: overall HEALTH_OK 2024-07-04T08:01:36.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:36.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:36.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:36.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:36.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:36.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:36.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:36 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:37.127 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: mon.b calling monitor election 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: Updating smithi039:/etc/ceph/ceph.conf 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: Updating smithi050:/etc/ceph/ceph.conf 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: Updating smithi133:/etc/ceph/ceph.conf 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: mon.b calling monitor election 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: mon.c calling monitor election 2024-07-04T08:01:37.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: mon.a calling monitor election 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: monmap epoch 3 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: last_changed 2024-07-04T08:01:29.529450+0000 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: min_mon_release 19 (squid) 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: election_strategy: 1 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: fsmap 2024-07-04T08:01:37.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: mgrmap e13: x(active, since 68s) 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: overall HEALTH_OK 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.819 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:37.820 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-04T08:01:37.820 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:37.820 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:37 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:37.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: mon.b calling monitor election 2024-07-04T08:01:37.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: Updating smithi039:/etc/ceph/ceph.conf 2024-07-04T08:01:37.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: Updating smithi050:/etc/ceph/ceph.conf 2024-07-04T08:01:37.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: Updating smithi133:/etc/ceph/ceph.conf 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: mon.b calling monitor election 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: mon.c calling monitor election 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: mon.a calling monitor election 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: monmap epoch 3 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: last_changed 2024-07-04T08:01:29.529450+0000 2024-07-04T08:01:37.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: min_mon_release 19 (squid) 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: election_strategy: 1 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: fsmap 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: mgrmap e13: x(active, since 68s) 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: overall HEALTH_OK 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:37.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-04T08:01:37.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:37.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:37 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: mon.b calling monitor election 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: Updating smithi039:/etc/ceph/ceph.conf 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: Updating smithi050:/etc/ceph/ceph.conf 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: Updating smithi133:/etc/ceph/ceph.conf 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: mon.b calling monitor election 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: mon.c calling monitor election 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: mon.a calling monitor election 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: monmap epoch 3 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: last_changed 2024-07-04T08:01:29.529450+0000 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: created 2024-07-04T07:59:16.806394+0000 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: min_mon_release 19 (squid) 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: election_strategy: 1 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: 0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: fsmap 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: osdmap e4: 0 total, 0 up, 0 in 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: mgrmap e13: x(active, since 68s) 2024-07-04T08:01:37.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: overall HEALTH_OK 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-04T08:01:37.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:37.898 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:37 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:38.787 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:38 smithi039 ceph-mon[32062]: Reconfiguring mon.a (unknown last config time)... 2024-07-04T08:01:38.787 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:38 smithi039 ceph-mon[32062]: Reconfiguring daemon mon.a on smithi039 2024-07-04T08:01:38.788 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:38 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:38.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:38 smithi133 ceph-mon[37050]: Reconfiguring mon.a (unknown last config time)... 2024-07-04T08:01:38.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:38 smithi133 ceph-mon[37050]: Reconfiguring daemon mon.a on smithi039 2024-07-04T08:01:38.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:38 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:38.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:38 smithi050 ceph-mon[36577]: Reconfiguring mon.a (unknown last config time)... 2024-07-04T08:01:38.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:38 smithi050 ceph-mon[36577]: Reconfiguring daemon mon.a on smithi039 2024-07-04T08:01:38.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:38 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:39.749 INFO:teuthology.orchestra.run.smithi039.stdout:# minimal ceph.conf for 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:39.749 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-07-04T08:01:39.750 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:01:39.750 INFO:teuthology.orchestra.run.smithi039.stdout: mon_host = [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] 2024-07-04T08:01:39.750 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:39 smithi039 ceph-mon[32062]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:39.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:39 smithi133 ceph-mon[37050]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:39.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:39 smithi050 ceph-mon[36577]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:40.610 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-07-04T08:01:40.611 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:01:40.611 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.conf 2024-07-04T08:01:40.695 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:01:40.696 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:40.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:40 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/619454697' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:40.806 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:40 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:40.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:40 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:40.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:40 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:40.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:40 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-04T08:01:40.807 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:40 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:40.815 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T08:01:40.815 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.conf 2024-07-04T08:01:40.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:40 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/619454697' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:40.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:40 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:40.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:40 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:40.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:40 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:40.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:40 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-04T08:01:40.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:40 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:40.839 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:40 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/619454697' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:40.839 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:40 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:40.839 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:40 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:40.839 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:40 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:40.839 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:40 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-04T08:01:40.839 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:40 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:40.843 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T08:01:40.843 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:40.908 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T08:01:40.908 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2024-07-04T08:01:40.939 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T08:01:40.939 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:41.006 INFO:tasks.cephadm:enabling module snap_schedule 2024-07-04T08:01:41.006 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-07-04T08:01:41.443 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:01:41.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:41 smithi133 ceph-mon[37050]: Reconfiguring mon.b (monmap changed)... 2024-07-04T08:01:41.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:41 smithi133 ceph-mon[37050]: Reconfiguring daemon mon.b on smithi050 2024-07-04T08:01:41.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:41 smithi133 ceph-mon[37050]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:41.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:41 smithi039 ceph-mon[32062]: Reconfiguring mon.b (monmap changed)... 2024-07-04T08:01:41.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:41 smithi039 ceph-mon[32062]: Reconfiguring daemon mon.b on smithi050 2024-07-04T08:01:41.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:41 smithi039 ceph-mon[32062]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:41.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:41 smithi050 ceph-mon[36577]: Reconfiguring mon.b (monmap changed)... 2024-07-04T08:01:41.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:41 smithi050 ceph-mon[36577]: Reconfiguring daemon mon.b on smithi050 2024-07-04T08:01:41.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:41 smithi050 ceph-mon[36577]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:43.030 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:42 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:43.030 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:42 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:43.031 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:42 smithi133 ceph-mon[37050]: Reconfiguring mon.c (monmap changed)... 2024-07-04T08:01:43.031 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:42 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:43.031 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:42 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-04T08:01:43.031 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:42 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:43.031 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:42 smithi133 ceph-mon[37050]: Reconfiguring daemon mon.c on smithi133 2024-07-04T08:01:43.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:42 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:43.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:42 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:43.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:42 smithi039 ceph-mon[32062]: Reconfiguring mon.c (monmap changed)... 2024-07-04T08:01:43.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:42 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:43.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:42 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-04T08:01:43.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:42 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:43.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:42 smithi039 ceph-mon[32062]: Reconfiguring daemon mon.c on smithi133 2024-07-04T08:01:43.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:42 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:43.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:42 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:43.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:42 smithi050 ceph-mon[36577]: Reconfiguring mon.c (monmap changed)... 2024-07-04T08:01:43.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:42 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-04T08:01:43.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:42 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-04T08:01:43.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:42 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:43.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:42 smithi050 ceph-mon[36577]: Reconfiguring daemon mon.c on smithi133 2024-07-04T08:01:44.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:43 smithi133 ceph-mon[37050]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:44.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:43 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:44.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:43 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:44.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:43 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:44.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:43 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:44.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:43 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:44.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:43 smithi133 ceph-mon[37050]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:44.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:43 smithi039 ceph-mon[32062]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:44.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:43 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:44.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:43 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:44.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:43 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:44.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:43 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:44.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:43 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:44.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:43 smithi039 ceph-mon[32062]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:44.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:43 smithi050 ceph-mon[36577]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:01:44.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:43 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:44.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:43 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:44.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:43 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:44.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:43 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:44.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:43 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:44.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:43 smithi050 ceph-mon[36577]: from='mgr.14150 172.21.15.39:0/2651698281' entity='mgr.x' 2024-07-04T08:01:45.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:44 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2502848654' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-04T08:01:45.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:44 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2502848654' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-04T08:01:45.101 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:44 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: ignoring --setuser ceph since I am not root 2024-07-04T08:01:45.101 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:44 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: ignoring --setgroup ceph since I am not root 2024-07-04T08:01:45.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:44 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2502848654' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-04T08:01:45.493 INFO:tasks.cephadm:Adding mgr.x on smithi039 2024-07-04T08:01:45.493 INFO:tasks.cephadm:Adding mgr.y on smithi050 2024-07-04T08:01:45.493 INFO:tasks.cephadm:Adding mgr.z on smithi133 2024-07-04T08:01:45.493 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch apply mgr '3;smithi039=x;smithi050=y;smithi133=z' 2024-07-04T08:01:45.749 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:01:45.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:45 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2502848654' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-04T08:01:45.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:45 smithi133 ceph-mon[37050]: mgrmap e14: x(active, since 76s) 2024-07-04T08:01:46.046 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:45 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2502848654' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-04T08:01:46.046 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:45 smithi050 ceph-mon[36577]: mgrmap e14: x(active, since 76s) 2024-07-04T08:01:46.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:45 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2502848654' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-04T08:01:46.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:45 smithi039 ceph-mon[32062]: mgrmap e14: x(active, since 76s) 2024-07-04T08:01:51.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: Active manager daemon x restarted 2024-07-04T08:01:51.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: Activating manager daemon x 2024-07-04T08:01:51.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: osdmap e5: 0 total, 0 up, 0 in 2024-07-04T08:01:51.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: mgrmap e15: x(active, starting, since 0.013846s) 2024-07-04T08:01:51.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:01:51.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:51.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:51.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-04T08:01:51.711 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-04T08:01:51.711 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-04T08:01:51.711 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-04T08:01:51.711 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:51 smithi050 ceph-mon[36577]: Manager daemon x is now available 2024-07-04T08:01:51.729 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: Active manager daemon x restarted 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: Activating manager daemon x 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: osdmap e5: 0 total, 0 up, 0 in 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: mgrmap e15: x(active, starting, since 0.013846s) 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-04T08:01:51.730 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:51 smithi039 ceph-mon[32062]: Manager daemon x is now available 2024-07-04T08:01:51.738 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: Active manager daemon x restarted 2024-07-04T08:01:51.738 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: Activating manager daemon x 2024-07-04T08:01:51.738 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: osdmap e5: 0 total, 0 up, 0 in 2024-07-04T08:01:51.738 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: mgrmap e15: x(active, starting, since 0.013846s) 2024-07-04T08:01:51.738 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:01:51.738 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:01:51.738 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:01:51.739 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-04T08:01:51.739 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-04T08:01:51.739 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-04T08:01:51.739 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-04T08:01:51.739 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:51 smithi133 ceph-mon[37050]: Manager daemon x is now available 2024-07-04T08:01:52.375 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mgr update... 2024-07-04T08:01:52.660 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:52.660 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-04T08:01:52.661 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-04T08:01:52.661 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:52 smithi050 ceph-mon[36577]: mgrmap e16: x(active, since 1.01659s) 2024-07-04T08:01:52.661 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:52.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:52.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-04T08:01:52.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-04T08:01:52.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:52 smithi133 ceph-mon[37050]: mgrmap e16: x(active, since 1.01659s) 2024-07-04T08:01:52.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:52.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:01:52.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-04T08:01:52.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-04T08:01:52.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:52 smithi039 ceph-mon[32062]: mgrmap e16: x(active, since 1.01659s) 2024-07-04T08:01:52.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:53.593 DEBUG:teuthology.orchestra.run.smithi050:mgr.y> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.y.service 2024-07-04T08:01:53.596 DEBUG:teuthology.orchestra.run.smithi133:mgr.z> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.z.service 2024-07-04T08:01:53.598 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-07-04T08:01:53.598 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-07-04T08:01:53.605 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-07-04T08:01:53.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set client debug_client 20 2024-07-04T08:01:53.605 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:01:53.273+0000 7fdf9cc00640 -1 log_channel(cephadm) log [ERR] : [04/Jul/2024:08:01:53] ENGINE Error in HTTPServer.serve 2024-07-04T08:01:53.605 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: Traceback (most recent call last): 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self._connections.run(self.expiration_interval) 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self._run(expiration_interval) 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: new_conn = self._from_server_socket(self.server.socket) 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: s = self.context.wrap_socket( 2024-07-04T08:01:53.606 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-04T08:01:53.607 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: return self.sslsocket_class._create( 2024-07-04T08:01:53.607 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-04T08:01:53.607 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self.do_handshake() 2024-07-04T08:01:53.607 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-04T08:01:53.607 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: self._sslobj.do_handshake() 2024-07-04T08:01:53.607 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-04T08:01:53.607 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:01:53.651 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-07-04T08:01:53.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set client debug_ms 1 2024-07-04T08:01:53.698 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-07-04T08:01:53.698 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-07-04T08:01:53.744 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-07-04T08:01:53.745 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-07-04T08:01:53.791 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-07-04T08:01:53.792 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds debug_mds 20 2024-07-04T08:01:53.842 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-07-04T08:01:53.843 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-07-04T08:01:53.891 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-07-04T08:01:53.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds debug_ms 2 2024-07-04T08:01:53.942 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-07-04T08:01:53.943 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-07-04T08:01:53.997 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-07-04T08:01:53.998 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-07-04T08:01:54.052 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-07-04T08:01:54.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-07-04T08:01:54.102 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-07-04T08:01:54.103 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-07-04T08:01:54.163 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-07-04T08:01:54.164 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-07-04T08:01:54.213 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-07-04T08:01:54.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-07-04T08:01:54.218 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: Saving service mgr spec with placement smithi039=x;smithi050=y;smithi133=z;count:3 2024-07-04T08:01:54.218 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.218 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.218 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.218 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.218 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.219 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.219 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.219 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.219 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:54.227 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-07-04T08:01:54.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-07-04T08:01:54.291 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-07-04T08:01:54.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-07-04T08:01:54.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: Saving service mgr spec with placement smithi039=x;smithi050=y;smithi133=z;count:3 2024-07-04T08:01:54.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:54.348 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-07-04T08:01:54.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-07-04T08:01:54.392 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:01:54.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: Saving service mgr spec with placement smithi039=x;smithi050=y;smithi133=z;count:3 2024-07-04T08:01:54.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:54.411 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-07-04T08:01:54.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-07-04T08:01:54.470 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-07-04T08:01:54.471 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-07-04T08:01:54.527 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-07-04T08:01:54.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-07-04T08:01:54.587 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-07-04T08:01:54.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-04T08:01:54.646 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-07-04T08:01:54.646 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-04T08:01:54.647 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:01:54.662 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:01:54.670 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-07-04T08:01:54.670 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-07-04T08:01:54.735 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-07-04T08:01:54.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-07-04T08:01:54.793 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-07-04T08:01:54.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-07-04T08:01:54.852 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-07-04T08:01:54.852 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-07-04T08:01:54.906 DEBUG:tasks.cephadm:set 26 configs 2024-07-04T08:01:54.906 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi039') 2024-07-04T08:01:55.066 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:54 smithi039 ceph-mon[32062]: [04/Jul/2024:08:01:53] ENGINE Bus STARTING 2024-07-04T08:01:55.067 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:54 smithi039 ceph-mon[32062]: [04/Jul/2024:08:01:53] ENGINE Serving on http://172.21.15.39:8765 2024-07-04T08:01:55.067 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:54 smithi039 ceph-mon[32062]: [04/Jul/2024:08:01:53] ENGINE Serving on https://172.21.15.39:7150 2024-07-04T08:01:55.067 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:54 smithi039 ceph-mon[32062]: [04/Jul/2024:08:01:53] ENGINE Bus STARTED 2024-07-04T08:01:55.067 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:54 smithi039 ceph-mon[32062]: [04/Jul/2024:08:01:53] ENGINE Error in HTTPServer.serve 2024-07-04T08:01:55.067 INFO:journalctl@ceph.mon.a.smithi039.stdout: Traceback (most recent call last): 2024-07-04T08:01:55.067 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-04T08:01:55.067 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._connections.run(self.expiration_interval) 2024-07-04T08:01:55.067 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-04T08:01:55.067 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._run(expiration_interval) 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: s = self.context.wrap_socket( 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: return self.sslsocket_class._create( 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: self.do_handshake() 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: self._sslobj.do_handshake() 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:54 smithi039 ceph-mon[32062]: mgrmap e17: x(active, since 2s) 2024-07-04T08:01:55.068 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:55.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:55.069 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:55.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:54 smithi133 ceph-mon[37050]: [04/Jul/2024:08:01:53] ENGINE Bus STARTING 2024-07-04T08:01:55.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:54 smithi133 ceph-mon[37050]: [04/Jul/2024:08:01:53] ENGINE Serving on http://172.21.15.39:8765 2024-07-04T08:01:55.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:54 smithi133 ceph-mon[37050]: [04/Jul/2024:08:01:53] ENGINE Serving on https://172.21.15.39:7150 2024-07-04T08:01:55.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:54 smithi133 ceph-mon[37050]: [04/Jul/2024:08:01:53] ENGINE Bus STARTED 2024-07-04T08:01:55.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:54 smithi133 ceph-mon[37050]: [04/Jul/2024:08:01:53] ENGINE Error in HTTPServer.serve 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: Traceback (most recent call last): 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: self._connections.run(self.expiration_interval) 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: self._run(expiration_interval) 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: s = self.context.wrap_socket( 2024-07-04T08:01:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout: return self.sslsocket_class._create( 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout: self.do_handshake() 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout: self._sslobj.do_handshake() 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:54 smithi133 ceph-mon[37050]: mgrmap e17: x(active, since 2s) 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:55.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:55.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:54 smithi050 ceph-mon[36577]: [04/Jul/2024:08:01:53] ENGINE Bus STARTING 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:54 smithi050 ceph-mon[36577]: [04/Jul/2024:08:01:53] ENGINE Serving on http://172.21.15.39:8765 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:54 smithi050 ceph-mon[36577]: [04/Jul/2024:08:01:53] ENGINE Serving on https://172.21.15.39:7150 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:54 smithi050 ceph-mon[36577]: [04/Jul/2024:08:01:53] ENGINE Bus STARTED 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:54 smithi050 ceph-mon[36577]: [04/Jul/2024:08:01:53] ENGINE Error in HTTPServer.serve 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout: Traceback (most recent call last): 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout: self._connections.run(self.expiration_interval) 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout: self._run(expiration_interval) 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-04T08:01:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout: s = self.context.wrap_socket( 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout: return self.sslsocket_class._create( 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout: self.do_handshake() 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout: self._sslobj.do_handshake() 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:54 smithi050 ceph-mon[36577]: mgrmap e17: x(active, since 2s) 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:55.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:55.828 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:01:58.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:58.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:58.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:58.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:58.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:58.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:57 smithi133 ceph-mon[37050]: Updating smithi039:/etc/ceph/ceph.conf 2024-07-04T08:01:58.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:57 smithi133 ceph-mon[37050]: Updating smithi050:/etc/ceph/ceph.conf 2024-07-04T08:01:58.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:57 smithi133 ceph-mon[37050]: Updating smithi133:/etc/ceph/ceph.conf 2024-07-04T08:01:58.082 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:58.082 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:58.082 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:58.082 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:58.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:58.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:57 smithi039 ceph-mon[32062]: Updating smithi039:/etc/ceph/ceph.conf 2024-07-04T08:01:58.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:57 smithi039 ceph-mon[32062]: Updating smithi050:/etc/ceph/ceph.conf 2024-07-04T08:01:58.083 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:57 smithi039 ceph-mon[32062]: Updating smithi133:/etc/ceph/ceph.conf 2024-07-04T08:01:58.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:58.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:58.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi039", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:01:58.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:01:58.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:01:58.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:57 smithi050 ceph-mon[36577]: Updating smithi039:/etc/ceph/ceph.conf 2024-07-04T08:01:58.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:57 smithi050 ceph-mon[36577]: Updating smithi050:/etc/ceph/ceph.conf 2024-07-04T08:01:58.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:57 smithi050 ceph-mon[36577]: Updating smithi133:/etc/ceph/ceph.conf 2024-07-04T08:01:59.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:59.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:59.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:59.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:59.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:59.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:59.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:59.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.070 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.conf 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: Updating smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: Updating smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:01:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:02:00.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:02:00.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-04T08:02:00.051 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-04T08:02:00.051 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-04T08:02:00.051 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:00.051 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:01:59 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3298446445' entity='client.admin' 2024-07-04T08:02:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:02:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:02:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-04T08:02:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-04T08:02:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-04T08:02:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:01:59 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3298446445' entity='client.admin' 2024-07-04T08:02:00.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: Updating smithi039:/etc/ceph/ceph.client.admin.keyring 2024-07-04T08:02:00.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: Updating smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/config/ceph.client.admin.keyring 2024-07-04T08:02:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-04T08:02:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-04T08:02:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-04T08:02:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:00.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:01:59 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3298446445' entity='client.admin' 2024-07-04T08:02:01.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:00 smithi133 ceph-mon[37050]: Deploying daemon mgr.z on smithi133 2024-07-04T08:02:01.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:00 smithi050 ceph-mon[36577]: Deploying daemon mgr.z on smithi133 2024-07-04T08:02:01.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:00 smithi039 ceph-mon[32062]: Deploying daemon mgr.z on smithi133 2024-07-04T08:02:01.816 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jul 04 08:02:01 smithi133 systemd[1]: Started Ceph mgr.z for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:02:02.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 -- 172.21.15.39:0/289006493 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8d3403e710 msgr2=0x7f8d34040c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 --2- 172.21.15.39:0/289006493 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8d3403e710 0x7f8d34040c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009230 tx=0x7f8d3c000bc0 comp rx=0 tx=0).stop 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 -- 172.21.15.39:0/289006493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d50103110 msgr2=0x7f8d50074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 --2- 172.21.15.39:0/289006493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d50103110 0x7f8d50074c60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000e9f0 tx=0x7f8d4000edf0 comp rx=0 tx=0).stop 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 -- 172.21.15.39:0/289006493 shutdown_connections 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 --2- 172.21.15.39:0/289006493 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8d3403e710 0x7f8d34040c20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 --2- 172.21.15.39:0/289006493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d50108a00 0x7f8d500751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 --2- 172.21.15.39:0/289006493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d50109350 0x7f8d500756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 --2- 172.21.15.39:0/289006493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d50103110 0x7f8d50074c60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 -- 172.21.15.39:0/289006493 >> 172.21.15.39:0/289006493 conn(0x7f8d5007ae60 msgr2=0x7f8d50106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:02.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.001+0000 7f8d55e00640 1 -- 172.21.15.39:0/289006493 shutdown_connections 2024-07-04T08:02:02.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:02.002+0000 7f8d55e00640 1 -- 172.21.15.39:0/289006493 wait complete. 2024-07-04T08:02:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:01 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2453974381' entity='client.admin' 2024-07-04T08:02:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:01 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:01 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:01 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:01 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-04T08:02:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:01 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-04T08:02:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:01 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-04T08:02:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:01 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:02.179 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:01 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2453974381' entity='client.admin' 2024-07-04T08:02:02.180 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:01 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:02.180 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:01 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:02.180 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:01 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:02.180 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:01 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-04T08:02:02.180 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:01 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-04T08:02:02.180 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:01 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-04T08:02:02.180 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:01 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:02.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:01 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2453974381' entity='client.admin' 2024-07-04T08:02:02.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:01 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:02.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:01 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:02.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:01 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:02.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:01 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-04T08:02:02.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:01 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-04T08:02:02.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:01 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-04T08:02:02.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:01 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:03.100 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:02 smithi050 ceph-mon[36577]: Deploying daemon mgr.y on smithi050 2024-07-04T08:02:03.100 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:02 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:02 smithi039 ceph-mon[32062]: Deploying daemon mgr.y on smithi050 2024-07-04T08:02:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:02 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:03.169 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi039') 2024-07-04T08:02:03.169 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi039') 2024-07-04T08:02:03.227 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:03.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.234+0000 7f53af200640 1 -- 172.21.15.39:0/721939234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8104630 msgr2=0x7f53a810cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:03.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.234+0000 7f53af200640 1 --2- 172.21.15.39:0/721939234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8104630 0x7f53a810cc70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5398009900 tx=0x7f539802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.234+0000 7f53af200640 1 -- 172.21.15.39:0/721939234 shutdown_connections 2024-07-04T08:02:03.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.234+0000 7f53af200640 1 --2- 172.21.15.39:0/721939234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a810d1b0 0x7f53a810f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.234+0000 7f53af200640 1 --2- 172.21.15.39:0/721939234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8104630 0x7f53a810cc70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.234+0000 7f53af200640 1 --2- 172.21.15.39:0/721939234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8103c00 0x7f53a8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.234+0000 7f53af200640 1 -- 172.21.15.39:0/721939234 >> 172.21.15.39:0/721939234 conn(0x7f53a80fd810 msgr2=0x7f53a80ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.234+0000 7f53af200640 1 -- 172.21.15.39:0/721939234 shutdown_connections 2024-07-04T08:02:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.234+0000 7f53af200640 1 -- 172.21.15.39:0/721939234 wait complete. 2024-07-04T08:02:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.236+0000 7f53af200640 1 Processor -- start 2024-07-04T08:02:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.236+0000 7f53af200640 1 -- start start 2024-07-04T08:02:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.236+0000 7f53af200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8103c00 0x7f53a81974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.236+0000 7f53af200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8104630 0x7f53a8197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.236+0000 7f53af200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a810d1b0 0x7f53a819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.236+0000 7f53af200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8110990 con 0x7f53a810d1b0 2024-07-04T08:02:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.236+0000 7f53af200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8110810 con 0x7f53a8103c00 2024-07-04T08:02:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.236+0000 7f53af200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8110b10 con 0x7f53a8104630 2024-07-04T08:02:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ade00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8103c00 0x7f53a81974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ade00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8103c00 0x7f53a81974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:55818/0 (socket says 172.21.15.39:55818) 2024-07-04T08:02:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ad400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8104630 0x7f53a8197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ade00640 1 -- 172.21.15.39:0/2673594646 learned_addr learned my addr 172.21.15.39:0/2673594646 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ae800640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a810d1b0 0x7f53a819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ade00640 1 -- 172.21.15.39:0/2673594646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8104630 msgr2=0x7f53a8197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ade00640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8104630 0x7f53a8197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ade00640 1 -- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a810d1b0 msgr2=0x7f53a819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ade00640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a810d1b0 0x7f53a819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ade00640 1 -- 172.21.15.39:0/2673594646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a819d430 con 0x7f53a8103c00 2024-07-04T08:02:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.237+0000 7f53ad400640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8104630 0x7f53a8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.238+0000 7f53ade00640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8103c00 0x7f53a81974c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f539c012fc0 tx=0x7f539c010490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.238+0000 7f53a2a00640 1 -- 172.21.15.39:0/2673594646 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c00de50 con 0x7f53a8103c00 2024-07-04T08:02:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.238+0000 7f53a2a00640 1 -- 172.21.15.39:0/2673594646 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f539c01d070 con 0x7f53a8103c00 2024-07-04T08:02:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.238+0000 7f53a2a00640 1 -- 172.21.15.39:0/2673594646 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c018480 con 0x7f53a8103c00 2024-07-04T08:02:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.238+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a819d6c0 con 0x7f53a8103c00 2024-07-04T08:02:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.238+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a81a3e80 con 0x7f53a8103c00 2024-07-04T08:02:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.239+0000 7f53a2a00640 1 -- 172.21.15.39:0/2673594646 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) ==== 50295+0+0 (secure 0 0 0) 0x7f539c016050 con 0x7f53a8103c00 2024-07-04T08:02:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.240+0000 7f53a2a00640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f539003e8b0 0x7f5390040dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.240+0000 7f53a2a00640 1 -- 172.21.15.39:0/2673594646 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f539c056a40 con 0x7f53a8103c00 2024-07-04T08:02:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.240+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a806a9f0 con 0x7f53a8103c00 2024-07-04T08:02:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.240+0000 7f53ad400640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f539003e8b0 0x7f5390040dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.241+0000 7f53ad400640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f539003e8b0 0x7f5390040dc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5398000c00 tx=0x7f5398031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.246+0000 7f53a2a00640 1 -- 172.21.15.39:0/2673594646 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f539c014040 con 0x7f53a8103c00 2024-07-04T08:02:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:02 smithi133 ceph-mon[37050]: Deploying daemon mgr.y on smithi050 2024-07-04T08:02:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:02 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:03.405 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:03.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.588+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f53a81082e0 con 0x7f53a8103c00 2024-07-04T08:02:03.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.595+0000 7f53a2a00640 1 -- 172.21.15.39:0/2673594646 <== mon.2 v2:172.21.15.50:3300/0 7 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f539c016bf0 con 0x7f53a8103c00 2024-07-04T08:02:03.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.596+0000 7f53a2a00640 1 -- 172.21.15.39:0/2673594646 <== mon.2 v2:172.21.15.50:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v13)=0 v13) ==== 123+0+0 (secure 0 0 0) 0x7f539c00b8a0 con 0x7f53a8103c00 2024-07-04T08:02:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.600+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f539003e8b0 msgr2=0x7f5390040dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.600+0000 7f53af200640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f539003e8b0 0x7f5390040dc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5398000c00 tx=0x7f5398031400 comp rx=0 tx=0).stop 2024-07-04T08:02:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.600+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8103c00 msgr2=0x7f53a81974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:03.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.600+0000 7f53af200640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8103c00 0x7f53a81974c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f539c012fc0 tx=0x7f539c010490 comp rx=0 tx=0).stop 2024-07-04T08:02:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.600+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 shutdown_connections 2024-07-04T08:02:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.601+0000 7f53af200640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f539003e8b0 0x7f5390040dc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.601+0000 7f53af200640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a810d1b0 0x7f53a819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.601+0000 7f53af200640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53a8104630 0x7f53a8197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.601+0000 7f53af200640 1 --2- 172.21.15.39:0/2673594646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53a8103c00 0x7f53a81974c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:03.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.601+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 >> 172.21.15.39:0/2673594646 conn(0x7f53a80fd810 msgr2=0x7f53a80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:03.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.602+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 shutdown_connections 2024-07-04T08:02:03.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:03.602+0000 7f53af200640 1 -- 172.21.15.39:0/2673594646 wait complete. 2024-07-04T08:02:03.644 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:02:03 smithi050 systemd[1]: Starting Ceph mgr.y for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:02:03.879 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-07-04T08:02:04.005 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:02:03 smithi050 podman[38940]: 2024-07-04 08:02:03.73580781 +0000 UTC m=+0.097808384 container create 71760403a058fd641abe0d45df05e78fa9fb7e2b173c6de31c4d8597dae25dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-07-04T08:02:04.005 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:02:03 smithi050 podman[38940]: 2024-07-04 08:02:03.655852342 +0000 UTC m=+0.017852908 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:02:04.005 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:02:03 smithi050 podman[38940]: 2024-07-04 08:02:03.9300699 +0000 UTC m=+0.292070456 container init 71760403a058fd641abe0d45df05e78fa9fb7e2b173c6de31c4d8597dae25dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b) 2024-07-04T08:02:04.005 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:02:03 smithi050 podman[38940]: 2024-07-04 08:02:03.932878294 +0000 UTC m=+0.294878850 container start 71760403a058fd641abe0d45df05e78fa9fb7e2b173c6de31c4d8597dae25dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-07-04T08:02:04.005 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:02:03 smithi050 ceph-mgr[38959]: -- 172.21.15.50:0/2754670961 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0ba88f0e0 con 0x55e0bf62c000 2024-07-04T08:02:04.397 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:02:04 smithi050 bash[38940]: 71760403a058fd641abe0d45df05e78fa9fb7e2b173c6de31c4d8597dae25dad 2024-07-04T08:02:04.397 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:02:04 smithi050 systemd[1]: Started Ceph mgr.y for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:02:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:04 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:04.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:04 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:04.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:04.928 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:04 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:04.928 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.929 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.929 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.929 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:04.929 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:05.496 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-07-04T08:02:05.599 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.361 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.362 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.367 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.378 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.390 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.394 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.403 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.405 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.407 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.421 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.421 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.430 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.433 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.581 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.583 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.583 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:06.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:06.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:06.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.598 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: Reconfiguring mgr.x (unknown last config time)... 2024-07-04T08:02:06.599 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-04T08:02:06.599 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-04T08:02:06.599 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:06.599 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:06 smithi050 ceph-mon[36577]: Reconfiguring daemon mgr.x on smithi039 2024-07-04T08:02:06.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.600 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:06.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:06.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: Reconfiguring mgr.x (unknown last config time)... 2024-07-04T08:02:06.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-04T08:02:06.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-04T08:02:06.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:06.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:06 smithi039 ceph-mon[32062]: Reconfiguring daemon mgr.x on smithi039 2024-07-04T08:02:06.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:06.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:06.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:06.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: Reconfiguring mgr.x (unknown last config time)... 2024-07-04T08:02:06.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-04T08:02:06.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-04T08:02:06.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:06.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:06 smithi133 ceph-mon[37050]: Reconfiguring daemon mgr.x on smithi039 2024-07-04T08:02:07.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:07 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:07.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:07 smithi039 ceph-mon[32062]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-04T08:02:07.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:07 smithi039 ceph-mon[32062]: Standby manager daemon z started 2024-07-04T08:02:07.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:07 smithi039 ceph-mon[32062]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-04T08:02:07.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:07 smithi039 ceph-mon[32062]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-04T08:02:07.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:07 smithi039 ceph-mon[32062]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-04T08:02:07.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:07 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:07.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:07 smithi133 ceph-mon[37050]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-04T08:02:07.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:07 smithi133 ceph-mon[37050]: Standby manager daemon z started 2024-07-04T08:02:07.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:07 smithi133 ceph-mon[37050]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-04T08:02:07.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:07 smithi133 ceph-mon[37050]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-04T08:02:07.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:07 smithi133 ceph-mon[37050]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-04T08:02:07.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:07 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:07.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:07 smithi050 ceph-mon[36577]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-04T08:02:07.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:07 smithi050 ceph-mon[36577]: Standby manager daemon z started 2024-07-04T08:02:07.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:07 smithi050 ceph-mon[36577]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-04T08:02:07.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:07 smithi050 ceph-mon[36577]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-04T08:02:07.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:07 smithi050 ceph-mon[36577]: from='mgr.? 172.21.15.133:0/2864202364' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-04T08:02:08.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:08 smithi039 ceph-mon[32062]: mgrmap e18: x(active, since 16s), standbys: z 2024-07-04T08:02:08.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-04T08:02:08.717 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:08 smithi133 ceph-mon[37050]: mgrmap e18: x(active, since 16s), standbys: z 2024-07-04T08:02:08.718 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-04T08:02:08.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:08 smithi050 ceph-mon[36577]: mgrmap e18: x(active, since 16s), standbys: z 2024-07-04T08:02:08.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-04T08:02:09.733 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:09 smithi133 ceph-mon[37050]: Standby manager daemon y started 2024-07-04T08:02:09.733 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:09 smithi133 ceph-mon[37050]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-04T08:02:09.733 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:09 smithi133 ceph-mon[37050]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-04T08:02:09.733 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:09 smithi133 ceph-mon[37050]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-04T08:02:09.733 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:09 smithi133 ceph-mon[37050]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-04T08:02:09.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:09 smithi039 ceph-mon[32062]: Standby manager daemon y started 2024-07-04T08:02:09.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:09 smithi039 ceph-mon[32062]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-04T08:02:09.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:09 smithi039 ceph-mon[32062]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-04T08:02:09.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:09 smithi039 ceph-mon[32062]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-04T08:02:09.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:09 smithi039 ceph-mon[32062]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-04T08:02:09.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:09 smithi050 ceph-mon[36577]: Standby manager daemon y started 2024-07-04T08:02:09.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:09 smithi050 ceph-mon[36577]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-04T08:02:09.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:09 smithi050 ceph-mon[36577]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-04T08:02:09.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:09 smithi050 ceph-mon[36577]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-04T08:02:09.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:09 smithi050 ceph-mon[36577]: from='mgr.? 172.21.15.50:0/2062535525' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-04T08:02:10.552 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:10 smithi039 ceph-mon[32062]: mgrmap e19: x(active, since 18s), standbys: z, y 2024-07-04T08:02:10.552 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-04T08:02:10.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:10 smithi133 ceph-mon[37050]: mgrmap e19: x(active, since 18s), standbys: z, y 2024-07-04T08:02:10.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-04T08:02:10.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:10 smithi050 ceph-mon[36577]: mgrmap e19: x(active, since 18s), standbys: z, y 2024-07-04T08:02:10.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-04T08:02:12.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.358+0000 7fdfc1000640 1 -- 172.21.15.39:0/4102979073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfbc10f990 msgr2=0x7fdfbc1161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.358+0000 7fdfc1000640 1 --2- 172.21.15.39:0/4102979073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfbc10f990 0x7fdfbc1161c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009be0 tx=0x7fdfb402a6c0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.358+0000 7fdfc1000640 1 -- 172.21.15.39:0/4102979073 shutdown_connections 2024-07-04T08:02:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.358+0000 7fdfc1000640 1 --2- 172.21.15.39:0/4102979073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfbc10f990 0x7fdfbc1161c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.358+0000 7fdfc1000640 1 --2- 172.21.15.39:0/4102979073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfbc10f040 0x7fdfbc10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.358+0000 7fdfc1000640 1 --2- 172.21.15.39:0/4102979073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfbc109700 0x7fdfbc109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.358+0000 7fdfc1000640 1 -- 172.21.15.39:0/4102979073 >> 172.21.15.39:0/4102979073 conn(0x7fdfbc077630 msgr2=0x7fdfbc107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.358+0000 7fdfc1000640 1 -- 172.21.15.39:0/4102979073 shutdown_connections 2024-07-04T08:02:12.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.359+0000 7fdfc1000640 1 -- 172.21.15.39:0/4102979073 wait complete. 2024-07-04T08:02:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.360+0000 7fdfc1000640 1 Processor -- start 2024-07-04T08:02:12.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.360+0000 7fdfc1000640 1 -- start start 2024-07-04T08:02:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfc1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfbc109700 0x7fdfbc19ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfc1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfbc10f040 0x7fdfbc1a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfc1000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfbc10f990 0x7fdfbc1a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfc1000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc119520 con 0x7fdfbc10f040 2024-07-04T08:02:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfc1000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1193a0 con 0x7fdfbc109700 2024-07-04T08:02:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfc1000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1196a0 con 0x7fdfbc10f990 2024-07-04T08:02:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfb9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfbc10f040 0x7fdfbc1a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:12.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfb9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfbc109700 0x7fdfbc19ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:12.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfb9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfbc10f040 0x7fdfbc1a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34072/0 (socket says 172.21.15.39:34072) 2024-07-04T08:02:12.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfb9400640 1 -- 172.21.15.39:0/2843179458 learned_addr learned my addr 172.21.15.39:0/2843179458 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:12.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.361+0000 7fdfba800640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfbc10f990 0x7fdfbc1a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:12.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb9e00640 1 -- 172.21.15.39:0/2843179458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfbc10f990 msgr2=0x7fdfbc1a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:12.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb9e00640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfbc10f990 0x7fdfbc1a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb9e00640 1 -- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfbc10f040 msgr2=0x7fdfbc1a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:12.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb9e00640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfbc10f040 0x7fdfbc1a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb9e00640 1 -- 172.21.15.39:0/2843179458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc1a5f00 con 0x7fdfbc109700 2024-07-04T08:02:12.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb9400640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfbc10f040 0x7fdfbc1a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb9e00640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfbc109700 0x7fdfbc19ff90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00e9f0 tx=0x7fdfac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb2a00640 1 -- 172.21.15.39:0/2843179458 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00ccc0 con 0x7fdfbc109700 2024-07-04T08:02:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb2a00640 1 -- 172.21.15.39:0/2843179458 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdfac0035f0 con 0x7fdfbc109700 2024-07-04T08:02:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfb2a00640 1 -- 172.21.15.39:0/2843179458 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac00ce60 con 0x7fdfbc109700 2024-07-04T08:02:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.362+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc1a61f0 con 0x7fdfbc109700 2024-07-04T08:02:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.363+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1ac950 con 0x7fdfbc109700 2024-07-04T08:02:12.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.364+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfbc109b10 con 0x7fdfbc109700 2024-07-04T08:02:12.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.366+0000 7fdfb2a00640 1 -- 172.21.15.39:0/2843179458 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7fdfac003ac0 con 0x7fdfbc109700 2024-07-04T08:02:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.367+0000 7fdfb2a00640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdfa00b27b0 0x7fdfa00b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.368+0000 7fdfb2a00640 1 -- 172.21.15.39:0/2843179458 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdfac0e0890 con 0x7fdfbc109700 2024-07-04T08:02:12.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.368+0000 7fdfb9400640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdfa00b27b0 0x7fdfa00b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:12.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.368+0000 7fdfb9400640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdfa00b27b0 0x7fdfa00b4cc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc06a630 tx=0x7fdfa8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:12.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.372+0000 7fdfb2a00640 1 -- 172.21.15.39:0/2843179458 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fdfac0aa110 con 0x7fdfbc109700 2024-07-04T08:02:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.589+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fdfbc1a10c0 con 0x7fdfbc109700 2024-07-04T08:02:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.600+0000 7fdfb2a00640 1 -- 172.21.15.39:0/2843179458 <== mon.2 v2:172.21.15.50:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfac0a5190 con 0x7fdfbc109700 2024-07-04T08:02:12.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.603+0000 7fdfb2a00640 1 -- 172.21.15.39:0/2843179458 <== mon.2 v2:172.21.15.50:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7fdfac0addc0 con 0x7fdfbc109700 2024-07-04T08:02:12.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdfa00b27b0 msgr2=0x7fdfa00b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:12.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdfa00b27b0 0x7fdfa00b4cc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc06a630 tx=0x7fdfa8009260 comp rx=0 tx=0).stop 2024-07-04T08:02:12.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfbc109700 msgr2=0x7fdfbc19ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:12.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfbc109700 0x7fdfbc19ff90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00e9f0 tx=0x7fdfac00edf0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 shutdown_connections 2024-07-04T08:02:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdfa00b27b0 0x7fdfa00b4cc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdfbc10f040 0x7fdfbc1a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfbc10f990 0x7fdfbc1a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 --2- 172.21.15.39:0/2843179458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfbc109700 0x7fdfbc19ff90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.606+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 >> 172.21.15.39:0/2843179458 conn(0x7fdfbc077630 msgr2=0x7fdfbc107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.607+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 shutdown_connections 2024-07-04T08:02:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:12.607+0000 7fdfc1000640 1 -- 172.21.15.39:0/2843179458 wait complete. 2024-07-04T08:02:12.769 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:12 smithi039 ceph-mon[32062]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:12.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:12 smithi133 ceph-mon[37050]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:12.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:12 smithi050 ceph-mon[36577]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:13.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:13 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:14.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:13 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:14.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:13 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:14.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:14 smithi039 ceph-mon[32062]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:14 smithi050 ceph-mon[36577]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:15.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:14 smithi133 ceph-mon[37050]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:16.889 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:16 smithi039 ceph-mon[32062]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:16 smithi050 ceph-mon[36577]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:17.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:16 smithi133 ceph-mon[37050]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.002+0000 7f9e22600640 1 -- 172.21.15.39:0/3909833587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1c10b380 msgr2=0x7f9e1c111bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.002+0000 7f9e22600640 1 --2- 172.21.15.39:0/3909833587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1c10b380 0x7f9e1c111bb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009ca0 tx=0x7f9e1402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.002+0000 7f9e22600640 1 -- 172.21.15.39:0/3909833587 shutdown_connections 2024-07-04T08:02:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.002+0000 7f9e22600640 1 --2- 172.21.15.39:0/3909833587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1c10b380 0x7f9e1c111bb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.002+0000 7f9e22600640 1 --2- 172.21.15.39:0/3909833587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e1c10aa30 0x7f9e1c10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.002+0000 7f9e22600640 1 --2- 172.21.15.39:0/3909833587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e1c1050f0 0x7f9e1c105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.002+0000 7f9e22600640 1 -- 172.21.15.39:0/3909833587 >> 172.21.15.39:0/3909833587 conn(0x7f9e1c100b60 msgr2=0x7f9e1c102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.002+0000 7f9e22600640 1 -- 172.21.15.39:0/3909833587 shutdown_connections 2024-07-04T08:02:18.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.002+0000 7f9e22600640 1 -- 172.21.15.39:0/3909833587 wait complete. 2024-07-04T08:02:18.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.004+0000 7f9e22600640 1 Processor -- start 2024-07-04T08:02:18.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.004+0000 7f9e22600640 1 -- start start 2024-07-04T08:02:18.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e22600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e1c1050f0 0x7f9e1c19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:18.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e22600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1c10aa30 0x7f9e1c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:18.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e22600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e1c10b380 0x7f9e1c1a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:18.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e22600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1150a0 con 0x7f9e1c1050f0 2024-07-04T08:02:18.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e22600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c114f20 con 0x7f9e1c10b380 2024-07-04T08:02:18.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e22600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c115220 con 0x7f9e1c10aa30 2024-07-04T08:02:18.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e21200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e1c1050f0 0x7f9e1c19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:18.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e21200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e1c1050f0 0x7f9e1c19ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34108/0 (socket says 172.21.15.39:34108) 2024-07-04T08:02:18.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e21200640 1 -- 172.21.15.39:0/3603572607 learned_addr learned my addr 172.21.15.39:0/3603572607 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:18.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e21c00640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e1c10b380 0x7f9e1c1a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:18.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.005+0000 7f9e1be00640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1c10aa30 0x7f9e1c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:18.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.006+0000 7f9e21200640 1 -- 172.21.15.39:0/3603572607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1c10aa30 msgr2=0x7f9e1c19bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.006+0000 7f9e21200640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1c10aa30 0x7f9e1c19bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.006+0000 7f9e21200640 1 -- 172.21.15.39:0/3603572607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e1c10b380 msgr2=0x7f9e1c1a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.006+0000 7f9e21200640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e1c10b380 0x7f9e1c1a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.006+0000 7f9e21200640 1 -- 172.21.15.39:0/3603572607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c1a1980 con 0x7f9e1c1050f0 2024-07-04T08:02:18.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.006+0000 7f9e21200640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e1c1050f0 0x7f9e1c19ba10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000d950 tx=0x7f9e1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:18.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.006+0000 7f9e19600640 1 -- 172.21.15.39:0/3603572607 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10017030 con 0x7f9e1c1050f0 2024-07-04T08:02:18.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.007+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c1a1c70 con 0x7f9e1c1050f0 2024-07-04T08:02:18.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.007+0000 7f9e19600640 1 -- 172.21.15.39:0/3603572607 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e10004020 con 0x7f9e1c1050f0 2024-07-04T08:02:18.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.007+0000 7f9e19600640 1 -- 172.21.15.39:0/3603572607 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e100047a0 con 0x7f9e1c1050f0 2024-07-04T08:02:18.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.007+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c1a83d0 con 0x7f9e1c1050f0 2024-07-04T08:02:18.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.008+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c105500 con 0x7f9e1c1050f0 2024-07-04T08:02:18.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.012+0000 7f9e19600640 1 -- 172.21.15.39:0/3603572607 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f9e10009a30 con 0x7f9e1c1050f0 2024-07-04T08:02:18.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.013+0000 7f9e19600640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e040b2830 0x7f9e040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:18.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.013+0000 7f9e19600640 1 -- 172.21.15.39:0/3603572607 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9e100e00a0 con 0x7f9e1c1050f0 2024-07-04T08:02:18.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.013+0000 7f9e1be00640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e040b2830 0x7f9e040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:18.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.014+0000 7f9e1be00640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e040b2830 0x7f9e040b4d40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c104410 tx=0x7f9e0c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:18.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.018+0000 7f9e19600640 1 -- 172.21.15.39:0/3603572607 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f9e100a9920 con 0x7f9e1c1050f0 2024-07-04T08:02:18.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.238+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f9e1c19cb40 con 0x7f9e1c1050f0 2024-07-04T08:02:18.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.249+0000 7f9e19600640 1 -- 172.21.15.39:0/3603572607 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v16)=0 v16) ==== 127+0+0 (secure 0 0 0) 0x7f9e100a9220 con 0x7f9e1c1050f0 2024-07-04T08:02:18.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e040b2830 msgr2=0x7f9e040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e040b2830 0x7f9e040b4d40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c104410 tx=0x7f9e0c005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:18.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e1c1050f0 msgr2=0x7f9e1c19ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e1c1050f0 0x7f9e1c19ba10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000d950 tx=0x7f9e1000de10 comp rx=0 tx=0).stop 2024-07-04T08:02:18.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 shutdown_connections 2024-07-04T08:02:18.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e040b2830 0x7f9e040b4d40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e1c10b380 0x7f9e1c1a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e1c10aa30 0x7f9e1c19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 --2- 172.21.15.39:0/3603572607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e1c1050f0 0x7f9e1c19ba10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 >> 172.21.15.39:0/3603572607 conn(0x7f9e1c100b60 msgr2=0x7f9e1c10a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:18.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.252+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 shutdown_connections 2024-07-04T08:02:18.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.253+0000 7f9e22600640 1 -- 172.21.15.39:0/3603572607 wait complete. 2024-07-04T08:02:18.772 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:18 smithi039 ceph-mon[32062]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:18.773 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:18 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:18.773 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:18 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3603572607' entity='client.admin' 2024-07-04T08:02:18.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:18 smithi133 ceph-mon[37050]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:18.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:18 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:18.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:18 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3603572607' entity='client.admin' 2024-07-04T08:02:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.254+0000 7f6b67800640 1 -- 172.21.15.39:0/224273802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b60103f00 msgr2=0x7f6b60104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.254+0000 7f6b67800640 1 --2- 172.21.15.39:0/224273802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b60103f00 0x7f6b60104310 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009900 tx=0x7f6b5402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.255+0000 7f6b67800640 1 -- 172.21.15.39:0/224273802 shutdown_connections 2024-07-04T08:02:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.255+0000 7f6b67800640 1 --2- 172.21.15.39:0/224273802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b60104850 0x7f6b6010cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.255+0000 7f6b67800640 1 --2- 172.21.15.39:0/224273802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b6010d360 0x7f6b6010f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.255+0000 7f6b67800640 1 --2- 172.21.15.39:0/224273802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b60103f00 0x7f6b60104310 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.255+0000 7f6b67800640 1 -- 172.21.15.39:0/224273802 >> 172.21.15.39:0/224273802 conn(0x7f6b60078c80 msgr2=0x7f6b60079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:18.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.255+0000 7f6b67800640 1 -- 172.21.15.39:0/224273802 shutdown_connections 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.255+0000 7f6b67800640 1 -- 172.21.15.39:0/224273802 wait complete. 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.256+0000 7f6b67800640 1 Processor -- start 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.256+0000 7f6b67800640 1 -- start start 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.256+0000 7f6b67800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b60103f00 0x7f6b60199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b67800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b60104850 0x7f6b6019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b67800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b6010d360 0x7f6b6019f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b67800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60113040 con 0x7f6b60104850 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b67800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60112ec0 con 0x7f6b60103f00 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b67800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601131c0 con 0x7f6b6010d360 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b5fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b60104850 0x7f6b6019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:18.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b64e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b60103f00 0x7f6b60199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b65800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b6010d360 0x7f6b6019f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b5fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b60104850 0x7f6b6019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34090/0 (socket says 172.21.15.39:34090) 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b64e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b60103f00 0x7f6b60199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34838/0 (socket says 172.21.15.39:34838) 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b65800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b6010d360 0x7f6b6019f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49526/0 (socket says 172.21.15.39:49526) 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b5fe00640 1 -- 172.21.15.39:0/1807409381 learned_addr learned my addr 172.21.15.39:0/1807409381 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b64e00640 1 -- 172.21.15.39:0/1807409381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b6010d360 msgr2=0x7f6b6019f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.257+0000 7f6b64e00640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b6010d360 0x7f6b6019f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b64e00640 1 -- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b60104850 msgr2=0x7f6b6019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b64e00640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b60104850 0x7f6b6019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b64e00640 1 -- 172.21.15.39:0/1807409381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6019fa00 con 0x7f6b60103f00 2024-07-04T08:02:18.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b5fe00640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b60104850 0x7f6b6019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b65800640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b6010d360 0x7f6b6019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b64e00640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b60103f00 0x7f6b60199a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6b5402a910 tx=0x7f6b5403ac10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b5d600640 1 -- 172.21.15.39:0/1807409381 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54035070 con 0x7f6b60103f00 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b5d600640 1 -- 172.21.15.39:0/1807409381 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b54003cb0 con 0x7f6b60103f00 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6019fc90 con 0x7f6b60103f00 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b5d600640 1 -- 172.21.15.39:0/1807409381 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54031ba0 con 0x7f6b60103f00 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.258+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b60101b20 con 0x7f6b60103f00 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.260+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6006aab0 con 0x7f6b60103f00 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.262+0000 7f6b5d600640 1 -- 172.21.15.39:0/1807409381 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f6b54031530 con 0x7f6b60103f00 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.263+0000 7f6b5d600640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6b480b28b0 0x7f6b480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:18.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.264+0000 7f6b5d600640 1 -- 172.21.15.39:0/1807409381 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6b541045c0 con 0x7f6b60103f00 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.264+0000 7f6b5fe00640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6b480b28b0 0x7f6b480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.264+0000 7f6b5fe00640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6b480b28b0 0x7f6b480b4dc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6b6010f0d0 tx=0x7f6b4c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.267+0000 7f6b5d600640 1 -- 172.21.15.39:0/1807409381 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f6b540cde40 con 0x7f6b60103f00 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.493+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f6b60101d80 con 0x7f6b60103f00 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.504+0000 7f6b5d600640 1 -- 172.21.15.39:0/1807409381 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v15)=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f6b540cd740 con 0x7f6b60103f00 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6b480b28b0 msgr2=0x7f6b480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6b480b28b0 0x7f6b480b4dc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6b6010f0d0 tx=0x7f6b4c005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b60103f00 msgr2=0x7f6b60199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b60103f00 0x7f6b60199a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6b5402a910 tx=0x7f6b5403ac10 comp rx=0 tx=0).stop 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 shutdown_connections 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6b480b28b0 0x7f6b480b4dc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b60104850 0x7f6b6019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b6010d360 0x7f6b6019f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 --2- 172.21.15.39:0/1807409381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b60103f00 0x7f6b60199a90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:18.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 >> 172.21.15.39:0/1807409381 conn(0x7f6b60078c80 msgr2=0x7f6b60105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:18.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 shutdown_connections 2024-07-04T08:02:18.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:17.507+0000 7f6b67800640 1 -- 172.21.15.39:0/1807409381 wait complete. 2024-07-04T08:02:18.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:18 smithi050 ceph-mon[36577]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:18.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:18 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:18.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:18 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3603572607' entity='client.admin' 2024-07-04T08:02:19.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.995+0000 7f0a42800640 1 -- 172.21.15.39:0/560142064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a3c10f670 msgr2=0x7f0a3c111a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:19.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.995+0000 7f0a42800640 1 --2- 172.21.15.39:0/560142064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a3c10f670 0x7f0a3c111a90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009ca0 tx=0x7f0a3402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.995+0000 7f0a42800640 1 -- 172.21.15.39:0/560142064 shutdown_connections 2024-07-04T08:02:19.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.995+0000 7f0a42800640 1 --2- 172.21.15.39:0/560142064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a3c10f670 0x7f0a3c111a90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.995+0000 7f0a42800640 1 --2- 172.21.15.39:0/560142064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3c0755f0 0x7f0a3c10f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.995+0000 7f0a42800640 1 --2- 172.21.15.39:0/560142064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c074ca0 0x7f0a3c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.995+0000 7f0a42800640 1 -- 172.21.15.39:0/560142064 >> 172.21.15.39:0/560142064 conn(0x7f0a3c07ada0 msgr2=0x7f0a3c07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.995+0000 7f0a42800640 1 -- 172.21.15.39:0/560142064 shutdown_connections 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.995+0000 7f0a42800640 1 -- 172.21.15.39:0/560142064 wait complete. 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.996+0000 7f0a42800640 1 Processor -- start 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.996+0000 7f0a42800640 1 -- start start 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a42800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3c074ca0 0x7f0a3c19bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a42800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0755f0 0x7f0a3c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a42800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a3c10f670 0x7f0a3c1a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a42800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c1151e0 con 0x7f0a3c10f670 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a42800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c115060 con 0x7f0a3c0755f0 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a42800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c115360 con 0x7f0a3c074ca0 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3c074ca0 0x7f0a3c19bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:19.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a3aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0755f0 0x7f0a3c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a3be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a3c10f670 0x7f0a3c1a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a3aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0755f0 0x7f0a3c19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34878/0 (socket says 172.21.15.39:34878) 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a3b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3c074ca0 0x7f0a3c19bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49558/0 (socket says 172.21.15.39:49558) 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.997+0000 7f0a3aa00640 1 -- 172.21.15.39:0/3075309070 learned_addr learned my addr 172.21.15.39:0/3075309070 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a3aa00640 1 -- 172.21.15.39:0/3075309070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3c074ca0 msgr2=0x7f0a3c19bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a3aa00640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3c074ca0 0x7f0a3c19bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a3aa00640 1 -- 172.21.15.39:0/3075309070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a3c10f670 msgr2=0x7f0a3c1a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a3aa00640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a3c10f670 0x7f0a3c1a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a3aa00640 1 -- 172.21.15.39:0/3075309070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c1a1b80 con 0x7f0a3c0755f0 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a3b400640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3c074ca0 0x7f0a3c19bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a3aa00640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0755f0 0x7f0a3c19c1b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800e9c0 tx=0x7f0a2800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a33e00640 1 -- 172.21.15.39:0/3075309070 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800ccc0 con 0x7f0a3c0755f0 2024-07-04T08:02:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a33e00640 1 -- 172.21.15.39:0/3075309070 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a280054d0 con 0x7f0a3c0755f0 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a33e00640 1 -- 172.21.15.39:0/3075309070 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28010430 con 0x7f0a3c0755f0 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.998+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3c1a1e70 con 0x7f0a3c0755f0 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:18.999+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a3c1a85d0 con 0x7f0a3c0755f0 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.000+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a3c0750b0 con 0x7f0a3c0755f0 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.003+0000 7f0a33e00640 1 -- 172.21.15.39:0/3075309070 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f0a28005020 con 0x7f0a3c0755f0 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.004+0000 7f0a33e00640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0a200b27b0 0x7f0a200b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.004+0000 7f0a33e00640 1 -- 172.21.15.39:0/3075309070 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0a280e0920 con 0x7f0a3c0755f0 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.005+0000 7f0a3b400640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0a200b27b0 0x7f0a200b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.005+0000 7f0a3b400640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0a200b27b0 0x7f0a200b4cc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009fc0 tx=0x7f0a2c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.008+0000 7f0a33e00640 1 -- 172.21.15.39:0/3075309070 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f0a280aa1a0 con 0x7f0a3c0755f0 2024-07-04T08:02:19.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.228+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f0a3c1a88b0 con 0x7f0a3c0755f0 2024-07-04T08:02:19.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.236+0000 7f0a33e00640 1 -- 172.21.15.39:0/3075309070 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f0a280a9aa0 con 0x7f0a3c0755f0 2024-07-04T08:02:19.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0a200b27b0 msgr2=0x7f0a200b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:19.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0a200b27b0 0x7f0a200b4cc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009fc0 tx=0x7f0a2c005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:19.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0755f0 msgr2=0x7f0a3c19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:19.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0755f0 0x7f0a3c19c1b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800e9c0 tx=0x7f0a2800ee80 comp rx=0 tx=0).stop 2024-07-04T08:02:19.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 shutdown_connections 2024-07-04T08:02:19.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0a200b27b0 0x7f0a200b4cc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a3c10f670 0x7f0a3c1a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a3c0755f0 0x7f0a3c19c1b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 --2- 172.21.15.39:0/3075309070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a3c074ca0 0x7f0a3c19bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:19.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 >> 172.21.15.39:0/3075309070 conn(0x7f0a3c07ada0 msgr2=0x7f0a3c1081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:19.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.239+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 shutdown_connections 2024-07-04T08:02:19.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:19.240+0000 7f0a42800640 1 -- 172.21.15.39:0/3075309070 wait complete. 2024-07-04T08:02:20.355 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:20 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:20.355 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:20 smithi039 ceph-mon[32062]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:20.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:20 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:20.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:20 smithi133 ceph-mon[37050]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:20.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:20 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:20.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:20 smithi050 ceph-mon[36577]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:22.729 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:22 smithi039 ceph-mon[32062]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:22.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:22 smithi133 ceph-mon[37050]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:22.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:22 smithi050 ceph-mon[36577]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:24.778 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:24 smithi039 ceph-mon[32062]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:24.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:24 smithi133 ceph-mon[37050]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:24.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:24 smithi050 ceph-mon[36577]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:26.141 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:26.182 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi039') 2024-07-04T08:02:26.706 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:26 smithi050 ceph-mon[36577]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:26.788 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:26 smithi039 ceph-mon[32062]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:26.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:26 smithi133 ceph-mon[37050]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:28.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:28 smithi039 ceph-mon[32062]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:28.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:28 smithi133 ceph-mon[37050]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:28.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:28 smithi050 ceph-mon[36577]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.978+0000 7fdbce800640 1 -- 172.21.15.39:0/2402325648 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc8101fb0 msgr2=0x7fdbc81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.978+0000 7fdbce800640 1 --2- 172.21.15.39:0/2402325648 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc8101fb0 0x7fdbc81023c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009900 tx=0x7fdbb802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.978+0000 7fdbce800640 1 -- 172.21.15.39:0/2402325648 shutdown_connections 2024-07-04T08:02:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.978+0000 7fdbce800640 1 --2- 172.21.15.39:0/2402325648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc81078f0 0x7fdbc8107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.978+0000 7fdbce800640 1 --2- 172.21.15.39:0/2402325648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc8108240 0x7fdbc810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.978+0000 7fdbce800640 1 --2- 172.21.15.39:0/2402325648 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc8101fb0 0x7fdbc81023c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.978+0000 7fdbce800640 1 -- 172.21.15.39:0/2402325648 >> 172.21.15.39:0/2402325648 conn(0x7fdbc8078c80 msgr2=0x7fdbc8079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.978+0000 7fdbce800640 1 -- 172.21.15.39:0/2402325648 shutdown_connections 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.978+0000 7fdbce800640 1 -- 172.21.15.39:0/2402325648 wait complete. 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.980+0000 7fdbce800640 1 Processor -- start 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.980+0000 7fdbce800640 1 -- start start 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.980+0000 7fdbce800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc8101fb0 0x7fdbc81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.980+0000 7fdbce800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc81078f0 0x7fdbc8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.980+0000 7fdbce800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc8108240 0x7fdbc819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbce800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8111eb0 con 0x7fdbc8101fb0 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbce800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8111d30 con 0x7fdbc81078f0 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbce800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc8112030 con 0x7fdbc8108240 2024-07-04T08:02:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc8101fb0 0x7fdbc81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc81078f0 0x7fdbc8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc81078f0 0x7fdbc8199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37604/0 (socket says 172.21.15.39:37604) 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc8101fb0 0x7fdbc81999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44248/0 (socket says 172.21.15.39:44248) 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc6a00640 1 -- 172.21.15.39:0/396740798 learned_addr learned my addr 172.21.15.39:0/396740798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc7e00640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc8108240 0x7fdbc819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc6a00640 1 -- 172.21.15.39:0/396740798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc8108240 msgr2=0x7fdbc819f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc6a00640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc8108240 0x7fdbc819f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc6a00640 1 -- 172.21.15.39:0/396740798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc8101fb0 msgr2=0x7fdbc81999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc6a00640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc8101fb0 0x7fdbc81999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.981+0000 7fdbc6a00640 1 -- 172.21.15.39:0/396740798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc819f920 con 0x7fdbc81078f0 2024-07-04T08:02:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.982+0000 7fdbc6a00640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc81078f0 0x7fdbc8199f50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400e9a0 tx=0x7fdbb400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.982+0000 7fdbbfe00640 1 -- 172.21.15.39:0/396740798 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb400cd80 con 0x7fdbc81078f0 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.982+0000 7fdbbfe00640 1 -- 172.21.15.39:0/396740798 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbb40035f0 con 0x7fdbc81078f0 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.982+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc819fc10 con 0x7fdbc81078f0 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.982+0000 7fdbbfe00640 1 -- 172.21.15.39:0/396740798 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4010690 con 0x7fdbc81078f0 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.982+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc81a6370 con 0x7fdbc81078f0 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.983+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc81023c0 con 0x7fdbc81078f0 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.987+0000 7fdbbfe00640 1 -- 172.21.15.39:0/396740798 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7fdbb4004000 con 0x7fdbc81078f0 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.988+0000 7fdbbfe00640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbac0b27b0 0x7fdbac0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.988+0000 7fdbbfe00640 1 -- 172.21.15.39:0/396740798 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdbb40e0900 con 0x7fdbc81078f0 2024-07-04T08:02:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.988+0000 7fdbc7400640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbac0b27b0 0x7fdbac0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:29.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.989+0000 7fdbc7400640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbac0b27b0 0x7fdbac0b4cc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8031440 tx=0x7fdbb8009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:29.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:28.992+0000 7fdbbfe00640 1 -- 172.21.15.39:0/396740798 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fdbb40042f0 con 0x7fdbc81078f0 2024-07-04T08:02:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.060+0000 7f412ae00640 1 -- 172.21.15.39:0/3560849722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41241045d0 msgr2=0x7f412410cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.060+0000 7f412ae00640 1 --2- 172.21.15.39:0/3560849722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41241045d0 0x7f412410cc10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4110009900 tx=0x7f411002a410 comp rx=0 tx=0).stop 2024-07-04T08:02:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.061+0000 7f412ae00640 1 -- 172.21.15.39:0/3560849722 shutdown_connections 2024-07-04T08:02:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.061+0000 7f412ae00640 1 --2- 172.21.15.39:0/3560849722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41241045d0 0x7f412410cc10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.061+0000 7f412ae00640 1 --2- 172.21.15.39:0/3560849722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f412410d150 0x7f412410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.061+0000 7f412ae00640 1 --2- 172.21.15.39:0/3560849722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4124103ba0 0x7f4124103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.061+0000 7f412ae00640 1 -- 172.21.15.39:0/3560849722 >> 172.21.15.39:0/3560849722 conn(0x7f41240fd810 msgr2=0x7f41240ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:29.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.061+0000 7f412ae00640 1 -- 172.21.15.39:0/3560849722 shutdown_connections 2024-07-04T08:02:29.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.062+0000 7f412ae00640 1 -- 172.21.15.39:0/3560849722 wait complete. 2024-07-04T08:02:29.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.063+0000 7f412ae00640 1 Processor -- start 2024-07-04T08:02:29.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.063+0000 7f412ae00640 1 -- start start 2024-07-04T08:02:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.064+0000 7f412ae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4124103ba0 0x7f412419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.064+0000 7f412ae00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41241045d0 0x7f412419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.064+0000 7f412ae00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f412410d150 0x7f41241a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.064+0000 7f412ae00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4124112dd0 con 0x7f412410d150 2024-07-04T08:02:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.064+0000 7f412ae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4124112c50 con 0x7f4124103ba0 2024-07-04T08:02:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.064+0000 7f412ae00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4124112f50 con 0x7f41241045d0 2024-07-04T08:02:29.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.064+0000 7f4129a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4124103ba0 0x7f412419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:29.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.064+0000 7f4129a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4124103ba0 0x7f412419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37616/0 (socket says 172.21.15.39:37616) 2024-07-04T08:02:29.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.064+0000 7f4129a00640 1 -- 172.21.15.39:0/2080254213 learned_addr learned my addr 172.21.15.39:0/2080254213 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:29.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.065+0000 7f4129a00640 1 -- 172.21.15.39:0/2080254213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41241045d0 msgr2=0x7f412419bf80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-04T08:02:29.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.065+0000 7f4129a00640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41241045d0 0x7f412419bf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.065+0000 7f4129a00640 1 -- 172.21.15.39:0/2080254213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f412410d150 msgr2=0x7f41241a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:29.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.065+0000 7f4129a00640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f412410d150 0x7f41241a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.065+0000 7f4129a00640 1 -- 172.21.15.39:0/2080254213 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41241a1950 con 0x7f4124103ba0 2024-07-04T08:02:29.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.065+0000 7f4129a00640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4124103ba0 0x7f412419b9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f411800ea60 tx=0x7f411800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:29.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.065+0000 7f4122000640 1 -- 172.21.15.39:0/2080254213 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411800cde0 con 0x7f4124103ba0 2024-07-04T08:02:29.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.066+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41241a1c40 con 0x7f4124103ba0 2024-07-04T08:02:29.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.066+0000 7f4122000640 1 -- 172.21.15.39:0/2080254213 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41180035f0 con 0x7f4124103ba0 2024-07-04T08:02:29.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.066+0000 7f4122000640 1 -- 172.21.15.39:0/2080254213 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118010690 con 0x7f4124103ba0 2024-07-04T08:02:29.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.066+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a83a0 con 0x7f4124103ba0 2024-07-04T08:02:29.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.068+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4124103fb0 con 0x7f4124103ba0 2024-07-04T08:02:29.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.071+0000 7f4122000640 1 -- 172.21.15.39:0/2080254213 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f41180037a0 con 0x7f4124103ba0 2024-07-04T08:02:29.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.073+0000 7f4122000640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f410c0b2830 0x7f410c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:29.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.073+0000 7f4129000640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f410c0b2830 0x7f410c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:29.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.074+0000 7f4122000640 1 -- 172.21.15.39:0/2080254213 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f41180e0eb0 con 0x7f4124103ba0 2024-07-04T08:02:29.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.074+0000 7f4122000640 1 -- 172.21.15.39:0/2080254213 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f41180aa730 con 0x7f4124103ba0 2024-07-04T08:02:29.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.075+0000 7f4129000640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f410c0b2830 0x7f410c0b4d40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f411002a930 tx=0x7f4110000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:29.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.227+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fdbc81a66b0 con 0x7fdbc81078f0 2024-07-04T08:02:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.239+0000 7fdbbfe00640 1 -- 172.21.15.39:0/396740798 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v18)=0 v18) ==== 120+0+0 (secure 0 0 0) 0x7fdbb40aa220 con 0x7fdbc81078f0 2024-07-04T08:02:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbac0b27b0 msgr2=0x7fdbac0b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbac0b27b0 0x7fdbac0b4cc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8031440 tx=0x7fdbb8009560 comp rx=0 tx=0).stop 2024-07-04T08:02:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc81078f0 msgr2=0x7fdbc8199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc81078f0 0x7fdbc8199f50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400e9a0 tx=0x7fdbb400ee60 comp rx=0 tx=0).stop 2024-07-04T08:02:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 shutdown_connections 2024-07-04T08:02:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbac0b27b0 0x7fdbac0b4cc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc8108240 0x7fdbc819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc81078f0 0x7fdbc8199f50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 --2- 172.21.15.39:0/396740798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc8101fb0 0x7fdbc81999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 >> 172.21.15.39:0/396740798 conn(0x7fdbc8078c80 msgr2=0x7fdbc8100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.242+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 shutdown_connections 2024-07-04T08:02:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.243+0000 7fdbce800640 1 -- 172.21.15.39:0/396740798 wait complete. 2024-07-04T08:02:29.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.331+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f41241a8630 con 0x7f4124103ba0 2024-07-04T08:02:29.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.343+0000 7f4122000640 1 -- 172.21.15.39:0/2080254213 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v19)=0 v19) ==== 129+0+0 (secure 0 0 0) 0x7f4118078180 con 0x7f4124103ba0 2024-07-04T08:02:29.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.345+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f410c0b2830 msgr2=0x7f410c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:29.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f410c0b2830 0x7f410c0b4d40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f411002a930 tx=0x7f4110000bc0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4124103ba0 msgr2=0x7f412419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:29.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4124103ba0 0x7f412419b9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f411800ea60 tx=0x7f411800ef20 comp rx=0 tx=0).stop 2024-07-04T08:02:29.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 shutdown_connections 2024-07-04T08:02:29.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f410c0b2830 0x7f410c0b4d40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f412410d150 0x7f41241a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41241045d0 0x7f412419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 --2- 172.21.15.39:0/2080254213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4124103ba0 0x7f412419b9e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 >> 172.21.15.39:0/2080254213 conn(0x7f41240fd810 msgr2=0x7f4124101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 shutdown_connections 2024-07-04T08:02:29.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:29.346+0000 7f412ae00640 1 -- 172.21.15.39:0/2080254213 wait complete. 2024-07-04T08:02:30.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:30 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:30.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:30 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:30.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:30 smithi133 ceph-mon[37050]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:30.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:30 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:30.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:30 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:30.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:30 smithi039 ceph-mon[32062]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:30.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:30 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:30.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:30 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:30.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:30 smithi050 ceph-mon[36577]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:32.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.389+0000 7f86ed400640 1 -- 172.21.15.39:0/1977364926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e8074ca0 msgr2=0x7f86e80750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:32.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.389+0000 7f86ed400640 1 --2- 172.21.15.39:0/1977364926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e8074ca0 0x7f86e80750b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009900 tx=0x7f86d802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.389+0000 7f86ed400640 1 -- 172.21.15.39:0/1977364926 shutdown_connections 2024-07-04T08:02:32.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.390+0000 7f86ed400640 1 --2- 172.21.15.39:0/1977364926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86e80755f0 0x7f86e810f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.390+0000 7f86ed400640 1 --2- 172.21.15.39:0/1977364926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86e810f690 0x7f86e8111ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.390+0000 7f86ed400640 1 --2- 172.21.15.39:0/1977364926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e8074ca0 0x7f86e80750b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.390+0000 7f86ed400640 1 -- 172.21.15.39:0/1977364926 >> 172.21.15.39:0/1977364926 conn(0x7f86e80feb20 msgr2=0x7f86e8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:32.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.390+0000 7f86ed400640 1 -- 172.21.15.39:0/1977364926 shutdown_connections 2024-07-04T08:02:32.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.390+0000 7f86ed400640 1 -- 172.21.15.39:0/1977364926 wait complete. 2024-07-04T08:02:32.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.391+0000 7f86ed400640 1 Processor -- start 2024-07-04T08:02:32.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.391+0000 7f86ed400640 1 -- start start 2024-07-04T08:02:32.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.391+0000 7f86ed400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86e8074ca0 0x7f86e819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.391+0000 7f86ed400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e80755f0 0x7f86e819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.391+0000 7f86ed400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86e810f690 0x7f86e81a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.391+0000 7f86ed400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81151f0 con 0x7f86e810f690 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.391+0000 7f86ed400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8115070 con 0x7f86e8074ca0 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.391+0000 7f86ed400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8115370 con 0x7f86e80755f0 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86dfe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e80755f0 0x7f86e819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86dfe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e80755f0 0x7f86e819c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42022/0 (socket says 172.21.15.39:42022) 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86dfe00640 1 -- 172.21.15.39:0/1494067719 learned_addr learned my addr 172.21.15.39:0/1494067719 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86e7200640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86e810f690 0x7f86e81a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:32.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86e6800640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86e8074ca0 0x7f86e819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:32.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86dfe00640 1 -- 172.21.15.39:0/1494067719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86e8074ca0 msgr2=0x7f86e819bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:32.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86dfe00640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86e8074ca0 0x7f86e819bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86dfe00640 1 -- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86e810f690 msgr2=0x7f86e81a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:32.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86dfe00640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86e810f690 0x7f86e81a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.392+0000 7f86dfe00640 1 -- 172.21.15.39:0/1494067719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e81a1bb0 con 0x7f86e80755f0 2024-07-04T08:02:32.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.393+0000 7f86dfe00640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e80755f0 0x7f86e819c1e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f86d000e9f0 tx=0x7f86d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:32.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.393+0000 7f86df400640 1 -- 172.21.15.39:0/1494067719 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d000ccc0 con 0x7f86e80755f0 2024-07-04T08:02:32.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.393+0000 7f86df400640 1 -- 172.21.15.39:0/1494067719 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d00108e0 con 0x7f86e80755f0 2024-07-04T08:02:32.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.393+0000 7f86df400640 1 -- 172.21.15.39:0/1494067719 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0018860 con 0x7f86e80755f0 2024-07-04T08:02:32.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.393+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e81a1ea0 con 0x7f86e80755f0 2024-07-04T08:02:32.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.393+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e81a8600 con 0x7f86e80755f0 2024-07-04T08:02:32.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.394+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e80750b0 con 0x7f86e80755f0 2024-07-04T08:02:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.397+0000 7f86df400640 1 -- 172.21.15.39:0/1494067719 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f86d0010430 con 0x7f86e80755f0 2024-07-04T08:02:32.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.398+0000 7f86df400640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f86cc0b2830 0x7f86cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:32.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.399+0000 7f86df400640 1 -- 172.21.15.39:0/1494067719 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f86d00e08f0 con 0x7f86e80755f0 2024-07-04T08:02:32.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.399+0000 7f86e6800640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f86cc0b2830 0x7f86cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.399+0000 7f86e6800640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f86cc0b2830 0x7f86cc0b4d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86d8031440 tx=0x7f86d8009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:32.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.403+0000 7f86df400640 1 -- 172.21.15.39:0/1494067719 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f86d00a9b20 con 0x7f86e80755f0 2024-07-04T08:02:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.627+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f86e81a8860 con 0x7f86e80755f0 2024-07-04T08:02:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.642+0000 7f86df400640 1 -- 172.21.15.39:0/1494067719 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f86d00ade20 con 0x7f86e80755f0 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f86cc0b2830 msgr2=0x7f86cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f86cc0b2830 0x7f86cc0b4d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f86d8031440 tx=0x7f86d8009560 comp rx=0 tx=0).stop 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e80755f0 msgr2=0x7f86e819c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e80755f0 0x7f86e819c1e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f86d000e9f0 tx=0x7f86d000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 shutdown_connections 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f86cc0b2830 0x7f86cc0b4d40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86e810f690 0x7f86e81a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86e80755f0 0x7f86e819c1e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 --2- 172.21.15.39:0/1494067719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f86e8074ca0 0x7f86e819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 >> 172.21.15.39:0/1494067719 conn(0x7f86e80feb20 msgr2=0x7f86e8109230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 shutdown_connections 2024-07-04T08:02:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:32.645+0000 7f86ed400640 1 -- 172.21.15.39:0/1494067719 wait complete. 2024-07-04T08:02:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:32 smithi133 ceph-mon[37050]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:32 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:33.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:32 smithi039 ceph-mon[32062]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:33.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:33.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:33.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:33.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:33.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:33.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:33.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:32 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:33.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:32 smithi050 ceph-mon[36577]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:33.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:33.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:33.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:33.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:33.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:33.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:33.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:32 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:34.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.917+0000 7f1c57200640 1 -- 172.21.15.39:0/1759516671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50108820 msgr2=0x7f1c50108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:34.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.917+0000 7f1c57200640 1 --2- 172.21.15.39:0/1759516671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50108820 0x7f1c50108c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009900 tx=0x7f1c4002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:34.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.917+0000 7f1c57200640 1 -- 172.21.15.39:0/1759516671 shutdown_connections 2024-07-04T08:02:34.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.917+0000 7f1c57200640 1 --2- 172.21.15.39:0/1759516671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50108820 0x7f1c50108c30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.917+0000 7f1c57200640 1 --2- 172.21.15.39:0/1759516671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c50109170 0x7f1c5010f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.917+0000 7f1c57200640 1 --2- 172.21.15.39:0/1759516671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c50102ee0 0x7f1c501032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.917+0000 7f1c57200640 1 -- 172.21.15.39:0/1759516671 >> 172.21.15.39:0/1759516671 conn(0x7f1c500fe930 msgr2=0x7f1c50100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.917+0000 7f1c57200640 1 -- 172.21.15.39:0/1759516671 shutdown_connections 2024-07-04T08:02:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.917+0000 7f1c57200640 1 -- 172.21.15.39:0/1759516671 wait complete. 2024-07-04T08:02:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.919+0000 7f1c57200640 1 Processor -- start 2024-07-04T08:02:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.919+0000 7f1c57200640 1 -- start start 2024-07-04T08:02:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c57200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50102ee0 0x7f1c50199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c57200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c50108820 0x7f1c50199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c57200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c50109170 0x7f1c5019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c57200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50112e80 con 0x7f1c50102ee0 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c57200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50112d00 con 0x7f1c50109170 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c57200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50113000 con 0x7f1c50108820 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c55e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50102ee0 0x7f1c50199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c55e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50102ee0 0x7f1c50199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58976/0 (socket says 172.21.15.39:58976) 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c55e00640 1 -- 172.21.15.39:0/3885334845 learned_addr learned my addr 172.21.15.39:0/3885334845 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c55400640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c50108820 0x7f1c50199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c56800640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c50109170 0x7f1c5019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c55e00640 1 -- 172.21.15.39:0/3885334845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c50108820 msgr2=0x7f1c50199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.920+0000 7f1c55e00640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c50108820 0x7f1c50199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.921+0000 7f1c55e00640 1 -- 172.21.15.39:0/3885334845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c50109170 msgr2=0x7f1c5019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.921+0000 7f1c55e00640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c50109170 0x7f1c5019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.921+0000 7f1c55e00640 1 -- 172.21.15.39:0/3885334845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5019f7c0 con 0x7f1c50102ee0 2024-07-04T08:02:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.921+0000 7f1c55e00640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50102ee0 0x7f1c50199850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400e9f0 tx=0x7f1c4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.921+0000 7f1c4aa00640 1 -- 172.21.15.39:0/3885334845 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400cd80 con 0x7f1c50102ee0 2024-07-04T08:02:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.921+0000 7f1c4aa00640 1 -- 172.21.15.39:0/3885334845 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c440108e0 con 0x7f1c50102ee0 2024-07-04T08:02:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.921+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5019fab0 con 0x7f1c50102ee0 2024-07-04T08:02:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.921+0000 7f1c4aa00640 1 -- 172.21.15.39:0/3885334845 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44018980 con 0x7f1c50102ee0 2024-07-04T08:02:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.921+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c501a6210 con 0x7f1c50102ee0 2024-07-04T08:02:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.923+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c501032f0 con 0x7f1c50102ee0 2024-07-04T08:02:34.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.925+0000 7f1c4aa00640 1 -- 172.21.15.39:0/3885334845 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f1c44010430 con 0x7f1c50102ee0 2024-07-04T08:02:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.927+0000 7f1c4aa00640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c380b2830 0x7f1c380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.927+0000 7f1c4aa00640 1 -- 172.21.15.39:0/3885334845 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1c440e0e50 con 0x7f1c50102ee0 2024-07-04T08:02:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.927+0000 7f1c55400640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c380b2830 0x7f1c380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:34.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.928+0000 7f1c55400640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c380b2830 0x7f1c380b4d40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c4002a9e0 tx=0x7f1c40005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:34.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.929+0000 7f1c4aa00640 1 -- 172.21.15.39:0/3885334845 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f1c440aa080 con 0x7f1c50102ee0 2024-07-04T08:02:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:34 smithi039 ceph-mon[32062]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:34 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:35.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.001+0000 7fac0c400640 1 -- 172.21.15.39:0/2827545762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac041030b0 msgr2=0x7fac041034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:35.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.001+0000 7fac0c400640 1 --2- 172.21.15.39:0/2827545762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac041030b0 0x7fac041034c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009900 tx=0x7fabf802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.001+0000 7fac0c400640 1 -- 172.21.15.39:0/2827545762 shutdown_connections 2024-07-04T08:02:35.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.001+0000 7fac0c400640 1 --2- 172.21.15.39:0/2827545762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac04109340 0x7fac0410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.001+0000 7fac0c400640 1 --2- 172.21.15.39:0/2827545762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac041089f0 0x7fac04108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.001+0000 7fac0c400640 1 --2- 172.21.15.39:0/2827545762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac041030b0 0x7fac041034c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.001+0000 7fac0c400640 1 -- 172.21.15.39:0/2827545762 >> 172.21.15.39:0/2827545762 conn(0x7fac0407ada0 msgr2=0x7fac0407b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.001+0000 7fac0c400640 1 -- 172.21.15.39:0/2827545762 shutdown_connections 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.002+0000 7fac0c400640 1 -- 172.21.15.39:0/2827545762 wait complete. 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.003+0000 7fac0c400640 1 Processor -- start 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.003+0000 7fac0c400640 1 -- start start 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.003+0000 7fac0c400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac041030b0 0x7fac0419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.003+0000 7fac0c400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac041089f0 0x7fac0419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.003+0000 7fac0c400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac04109340 0x7fac041a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.003+0000 7fac0c400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fac04113010 con 0x7fac041089f0 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.003+0000 7fac0c400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fac04112e90 con 0x7fac04109340 2024-07-04T08:02:35.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.003+0000 7fac0c400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fac04113190 con 0x7fac041030b0 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac09000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac041089f0 0x7fac0419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac09a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac041030b0 0x7fac0419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac09000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac041089f0 0x7fac0419c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58962/0 (socket says 172.21.15.39:58962) 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac09000640 1 -- 172.21.15.39:0/32403441 learned_addr learned my addr 172.21.15.39:0/32403441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac0a400640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac04109340 0x7fac041a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac09a00640 1 -- 172.21.15.39:0/32403441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac04109340 msgr2=0x7fac041a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac09a00640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac04109340 0x7fac041a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac09a00640 1 -- 172.21.15.39:0/32403441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac041089f0 msgr2=0x7fac0419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac09a00640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac041089f0 0x7fac0419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac09a00640 1 -- 172.21.15.39:0/32403441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac041a1bb0 con 0x7fac041030b0 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.004+0000 7fac0a400640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac04109340 0x7fac041a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.005+0000 7fac09a00640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac041030b0 0x7fac0419bc40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fabf802a910 tx=0x7fabf8031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:35.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.005+0000 7fac02000640 1 -- 172.21.15.39:0/32403441 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf803f030 con 0x7fac041030b0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.005+0000 7fac02000640 1 -- 172.21.15.39:0/32403441 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf8004490 con 0x7fac041030b0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.005+0000 7fac02000640 1 -- 172.21.15.39:0/32403441 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8035070 con 0x7fac041030b0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.005+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac041a1e40 con 0x7fac041030b0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.005+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac041a8600 con 0x7fac041030b0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.006+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac04104330 con 0x7fac041030b0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.008+0000 7fac02000640 1 -- 172.21.15.39:0/32403441 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7fabf8042030 con 0x7fac041030b0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.010+0000 7fac02000640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fabec0b2830 0x7fabec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.010+0000 7fac02000640 1 -- 172.21.15.39:0/32403441 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fabf8100940 con 0x7fac041030b0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.010+0000 7fac09000640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fabec0b2830 0x7fabec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.011+0000 7fac09000640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fabec0b2830 0x7fabec0b4d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fac04105af0 tx=0x7fabf4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.014+0000 7fac02000640 1 -- 172.21.15.39:0/32403441 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fabf80ca1c0 con 0x7fac041030b0 2024-07-04T08:02:35.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.236+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fac041a8830 con 0x7fac041030b0 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.248+0000 7fac02000640 1 -- 172.21.15.39:0/32403441 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v21)=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7fabf80d2240 con 0x7fac041030b0 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fabec0b2830 msgr2=0x7fabec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fabec0b2830 0x7fabec0b4d40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fac04105af0 tx=0x7fabf4005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac041030b0 msgr2=0x7fac0419bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac041030b0 0x7fac0419bc40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fabf802a910 tx=0x7fabf8031a20 comp rx=0 tx=0).stop 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 shutdown_connections 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fabec0b2830 0x7fabec0b4d40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac041089f0 0x7fac0419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fac04109340 0x7fac041a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 --2- 172.21.15.39:0/32403441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac041030b0 0x7fac0419bc40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.251+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 >> 172.21.15.39:0/32403441 conn(0x7fac0407ada0 msgr2=0x7fac04106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.252+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 shutdown_connections 2024-07-04T08:02:35.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:34.252+0000 7fac0c400640 1 -- 172.21.15.39:0/32403441 wait complete. 2024-07-04T08:02:35.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:34 smithi050 ceph-mon[36577]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:35.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:34 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:35.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.147+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f1c5002de10 con 0x7f1c50102ee0 2024-07-04T08:02:35.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.153+0000 7f1c4aa00640 1 -- 172.21.15.39:0/3885334845 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f1c440ae380 con 0x7f1c50102ee0 2024-07-04T08:02:35.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c380b2830 msgr2=0x7f1c380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:35.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c380b2830 0x7f1c380b4d40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c4002a9e0 tx=0x7f1c40005a20 comp rx=0 tx=0).stop 2024-07-04T08:02:35.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50102ee0 msgr2=0x7f1c50199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:35.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50102ee0 0x7f1c50199850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400e9f0 tx=0x7f1c4400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 shutdown_connections 2024-07-04T08:02:35.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c380b2830 0x7f1c380b4d40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c50109170 0x7f1c5019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c50108820 0x7f1c50199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 --2- 172.21.15.39:0/3885334845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c50102ee0 0x7f1c50199850 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:35.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 >> 172.21.15.39:0/3885334845 conn(0x7f1c500fe930 msgr2=0x7f1c50100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:35.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 shutdown_connections 2024-07-04T08:02:35.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:35.157+0000 7f1c57200640 1 -- 172.21.15.39:0/3885334845 wait complete. 2024-07-04T08:02:35.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:34 smithi133 ceph-mon[37050]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:35.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:34 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.159+0000 7f2422400640 1 -- 172.21.15.39:0/1616492552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241c109700 msgr2=0x7f241c109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.159+0000 7f2422400640 1 --2- 172.21.15.39:0/1616492552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241c109700 0x7f241c109b10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f240c009900 tx=0x7f240c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.160+0000 7f2422400640 1 -- 172.21.15.39:0/1616492552 shutdown_connections 2024-07-04T08:02:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.160+0000 7f2422400640 1 --2- 172.21.15.39:0/1616492552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241c10f040 0x7f241c10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.160+0000 7f2422400640 1 --2- 172.21.15.39:0/1616492552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241c10f990 0x7f241c1161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.160+0000 7f2422400640 1 --2- 172.21.15.39:0/1616492552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241c109700 0x7f241c109b10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.160+0000 7f2422400640 1 -- 172.21.15.39:0/1616492552 >> 172.21.15.39:0/1616492552 conn(0x7f241c077630 msgr2=0x7f241c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.160+0000 7f2422400640 1 -- 172.21.15.39:0/1616492552 shutdown_connections 2024-07-04T08:02:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.161+0000 7f2422400640 1 -- 172.21.15.39:0/1616492552 wait complete. 2024-07-04T08:02:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.162+0000 7f2422400640 1 Processor -- start 2024-07-04T08:02:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.162+0000 7f2422400640 1 -- start start 2024-07-04T08:02:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.162+0000 7f2422400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241c109700 0x7f241c1a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.162+0000 7f2422400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241c10f040 0x7f241c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f2422400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241c10f990 0x7f241c1a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f2422400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f241c119690 con 0x7f241c109700 2024-07-04T08:02:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f2422400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f241c119510 con 0x7f241c10f990 2024-07-04T08:02:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f2422400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f241c119810 con 0x7f241c10f040 2024-07-04T08:02:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f241b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241c109700 0x7f241c1a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f241b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241c109700 0x7f241c1a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58990/0 (socket says 172.21.15.39:58990) 2024-07-04T08:02:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f241b400640 1 -- 172.21.15.39:0/575897046 learned_addr learned my addr 172.21.15.39:0/575897046 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f241aa00640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241c10f040 0x7f241c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f241be00640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241c10f990 0x7f241c1a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f241b400640 1 -- 172.21.15.39:0/575897046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241c10f040 msgr2=0x7f241c1a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f241b400640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241c10f040 0x7f241c1a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f241b400640 1 -- 172.21.15.39:0/575897046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241c10f990 msgr2=0x7f241c1a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.163+0000 7f241b400640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241c10f990 0x7f241c1a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.164+0000 7f241b400640 1 -- 172.21.15.39:0/575897046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c1a5f70 con 0x7f241c109700 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.164+0000 7f241be00640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241c10f990 0x7f241c1a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.164+0000 7f241b400640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241c109700 0x7f241c1a0000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f240c005ab0 tx=0x7f240c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.165+0000 7f2413e00640 1 -- 172.21.15.39:0/575897046 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c03f070 con 0x7f241c109700 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.165+0000 7f2413e00640 1 -- 172.21.15.39:0/575897046 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c031b60 con 0x7f241c109700 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.165+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c1a6200 con 0x7f241c109700 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.165+0000 7f2413e00640 1 -- 172.21.15.39:0/575897046 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c035030 con 0x7f241c109700 2024-07-04T08:02:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.165+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c1ac9c0 con 0x7f241c109700 2024-07-04T08:02:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.166+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241c109b10 con 0x7f241c109700 2024-07-04T08:02:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.172+0000 7f2413e00640 1 -- 172.21.15.39:0/575897046 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f240c042070 con 0x7f241c109700 2024-07-04T08:02:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.173+0000 7f2413e00640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24000b2830 0x7f24000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:36.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.173+0000 7f2413e00640 1 -- 172.21.15.39:0/575897046 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f240c101420 con 0x7f241c109700 2024-07-04T08:02:36.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.173+0000 7f241aa00640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24000b2830 0x7f24000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:36.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.174+0000 7f241aa00640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24000b2830 0x7f24000b4d40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f241c06a660 tx=0x7f2408005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:36.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.175+0000 7f2413e00640 1 -- 172.21.15.39:0/575897046 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f240c049070 con 0x7f241c109700 2024-07-04T08:02:36.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:36 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3885334845' entity='client.admin' 2024-07-04T08:02:36.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:36 smithi039 ceph-mon[32062]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:36.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.392+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f241c1a1130 con 0x7f241c109700 2024-07-04T08:02:36.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.402+0000 7f2413e00640 1 -- 172.21.15.39:0/575897046 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v23)=0 v23) ==== 120+0+0 (secure 0 0 0) 0x7f240c0caca0 con 0x7f241c109700 2024-07-04T08:02:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24000b2830 msgr2=0x7f24000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24000b2830 0x7f24000b4d40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f241c06a660 tx=0x7f2408005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241c109700 msgr2=0x7f241c1a0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241c109700 0x7f241c1a0000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f240c005ab0 tx=0x7f240c003c10 comp rx=0 tx=0).stop 2024-07-04T08:02:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 shutdown_connections 2024-07-04T08:02:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24000b2830 0x7f24000b4d40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241c10f990 0x7f241c1a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241c10f040 0x7f241c1a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 --2- 172.21.15.39:0/575897046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241c109700 0x7f241c1a0000 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:36.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 >> 172.21.15.39:0/575897046 conn(0x7f241c077630 msgr2=0x7f241c077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:36.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 shutdown_connections 2024-07-04T08:02:36.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:36.405+0000 7f2422400640 1 -- 172.21.15.39:0/575897046 wait complete. 2024-07-04T08:02:36.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:36 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3885334845' entity='client.admin' 2024-07-04T08:02:36.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:36 smithi133 ceph-mon[37050]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:36.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:36 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3885334845' entity='client.admin' 2024-07-04T08:02:36.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:36 smithi050 ceph-mon[36577]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:37.458 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:37 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/575897046' entity='client.admin' 2024-07-04T08:02:37.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:37 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/575897046' entity='client.admin' 2024-07-04T08:02:37.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:37 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/575897046' entity='client.admin' 2024-07-04T08:02:38.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.134+0000 7f0d89400640 1 -- 172.21.15.39:0/1137071493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d84105300 msgr2=0x7f0d84105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.134+0000 7f0d89400640 1 --2- 172.21.15.39:0/1137071493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d84105300 0x7f0d84105710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009900 tx=0x7f0d7402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.134+0000 7f0d89400640 1 -- 172.21.15.39:0/1137071493 shutdown_connections 2024-07-04T08:02:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.134+0000 7f0d89400640 1 --2- 172.21.15.39:0/1137071493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8410b590 0x7f0d84111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.134+0000 7f0d89400640 1 --2- 172.21.15.39:0/1137071493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d8410ac40 0x7f0d8410b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.134+0000 7f0d89400640 1 --2- 172.21.15.39:0/1137071493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d84105300 0x7f0d84105710 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.134+0000 7f0d89400640 1 -- 172.21.15.39:0/1137071493 >> 172.21.15.39:0/1137071493 conn(0x7f0d84100d10 msgr2=0x7f0d84103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.135+0000 7f0d89400640 1 -- 172.21.15.39:0/1137071493 shutdown_connections 2024-07-04T08:02:38.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.135+0000 7f0d89400640 1 -- 172.21.15.39:0/1137071493 wait complete. 2024-07-04T08:02:38.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.136+0000 7f0d89400640 1 Processor -- start 2024-07-04T08:02:38.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.136+0000 7f0d89400640 1 -- start start 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.136+0000 7f0d89400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84105300 0x7f0d8419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.136+0000 7f0d89400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8410ac40 0x7f0d8419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.136+0000 7f0d89400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d8410b590 0x7f0d841a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.136+0000 7f0d89400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84115260 con 0x7f0d8410b590 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.136+0000 7f0d89400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841150e0 con 0x7f0d84105300 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.136+0000 7f0d89400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841153e0 con 0x7f0d8410ac40 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d82800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84105300 0x7f0d8419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d82800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84105300 0x7f0d8419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:49756/0 (socket says 172.21.15.39:49756) 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d82800640 1 -- 172.21.15.39:0/678265159 learned_addr learned my addr 172.21.15.39:0/678265159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d81e00640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8410ac40 0x7f0d8419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d83200640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d8410b590 0x7f0d841a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d82800640 1 -- 172.21.15.39:0/678265159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8410ac40 msgr2=0x7f0d8419c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d82800640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8410ac40 0x7f0d8419c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d82800640 1 -- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d8410b590 msgr2=0x7f0d841a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d82800640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d8410b590 0x7f0d841a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.137+0000 7f0d82800640 1 -- 172.21.15.39:0/678265159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d841a1bc0 con 0x7f0d84105300 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.138+0000 7f0d82800640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84105300 0x7f0d8419bc50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d74005940 tx=0x7f0d740040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.138+0000 7f0d7b400640 1 -- 172.21.15.39:0/678265159 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7403f030 con 0x7f0d84105300 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.138+0000 7f0d7b400640 1 -- 172.21.15.39:0/678265159 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d740044c0 con 0x7f0d84105300 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.138+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d841a1e50 con 0x7f0d84105300 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.138+0000 7f0d7b400640 1 -- 172.21.15.39:0/678265159 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74035070 con 0x7f0d84105300 2024-07-04T08:02:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.138+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841a8610 con 0x7f0d84105300 2024-07-04T08:02:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.140+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d84105710 con 0x7f0d84105300 2024-07-04T08:02:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.142+0000 7f0d7b400640 1 -- 172.21.15.39:0/678265159 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f0d74042030 con 0x7f0d84105300 2024-07-04T08:02:38.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.144+0000 7f0d7b400640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d680b2830 0x7f0d680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:38.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.144+0000 7f0d7b400640 1 -- 172.21.15.39:0/678265159 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0d74100640 con 0x7f0d84105300 2024-07-04T08:02:38.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.144+0000 7f0d81e00640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d680b2830 0x7f0d680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:38.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.145+0000 7f0d81e00640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d680b2830 0x7f0d680b4d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d700095f0 tx=0x7f0d70005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.148+0000 7f0d7b400640 1 -- 172.21.15.39:0/678265159 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f0d740c9870 con 0x7f0d84105300 2024-07-04T08:02:38.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.370+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f0d8410d340 con 0x7f0d84105300 2024-07-04T08:02:38.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.375+0000 7f47c9400640 1 -- 172.21.15.39:0/1290081110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c4105600 msgr2=0x7f47c4105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:38.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.375+0000 7f47c9400640 1 --2- 172.21.15.39:0/1290081110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c4105600 0x7f47c4105a10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009900 tx=0x7f47b002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.375+0000 7f47c9400640 1 -- 172.21.15.39:0/1290081110 shutdown_connections 2024-07-04T08:02:38.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.375+0000 7f47c9400640 1 --2- 172.21.15.39:0/1290081110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c4105f50 0x7f47c410f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.375+0000 7f47c9400640 1 --2- 172.21.15.39:0/1290081110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c4105600 0x7f47c4105a10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.375+0000 7f47c9400640 1 --2- 172.21.15.39:0/1290081110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47c4101320 0x7f47c4101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.375+0000 7f47c9400640 1 -- 172.21.15.39:0/1290081110 >> 172.21.15.39:0/1290081110 conn(0x7f47c4076ff0 msgr2=0x7f47c4077430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:38.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.376+0000 7f47c9400640 1 -- 172.21.15.39:0/1290081110 shutdown_connections 2024-07-04T08:02:38.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.376+0000 7f47c9400640 1 -- 172.21.15.39:0/1290081110 wait complete. 2024-07-04T08:02:38.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.377+0000 7f47c9400640 1 Processor -- start 2024-07-04T08:02:38.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.377+0000 7f47c9400640 1 -- start start 2024-07-04T08:02:38.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.377+0000 7f47c9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47c4101320 0x7f47c419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:38.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c4105600 0x7f47c419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:38.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c4105f50 0x7f47c41a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:38.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c9400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4112e40 con 0x7f47c4101320 2024-07-04T08:02:38.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c9400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4112cc0 con 0x7f47c4105f50 2024-07-04T08:02:38.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c9400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4112fc0 con 0x7f47c4105600 2024-07-04T08:02:38.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47c4101320 0x7f47c419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:38.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c4105600 0x7f47c419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:38.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47c4101320 0x7f47c419ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59020/0 (socket says 172.21.15.39:59020) 2024-07-04T08:02:38.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c3e00640 1 -- 172.21.15.39:0/479367494 learned_addr learned my addr 172.21.15.39:0/479367494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:38.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c4105600 0x7f47c419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42094/0 (socket says 172.21.15.39:42094) 2024-07-04T08:02:38.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c3e00640 1 -- 172.21.15.39:0/479367494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c4105600 msgr2=0x7f47c419bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:38.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c3e00640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c4105600 0x7f47c419bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c3e00640 1 -- 172.21.15.39:0/479367494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c4105f50 msgr2=0x7f47c41a1250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T08:02:38.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.378+0000 7f47c3e00640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c4105f50 0x7f47c41a1250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.379+0000 7f47c3e00640 1 -- 172.21.15.39:0/479367494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c41a1980 con 0x7f47c4101320 2024-07-04T08:02:38.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.379+0000 7f47c3400640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c4105600 0x7f47c419bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:38.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.379+0000 7f47c3e00640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47c4101320 0x7f47c419ba10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f47b400e9f0 tx=0x7f47b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:38.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.381+0000 7f47c0c00640 1 -- 172.21.15.39:0/479367494 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b400cd80 con 0x7f47c4101320 2024-07-04T08:02:38.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.381+0000 7f0d7b400640 1 -- 172.21.15.39:0/678265159 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v24)=0 v24) ==== 116+0+0 (secure 0 0 0) 0x7f0d740cdb70 con 0x7f0d84105300 2024-07-04T08:02:38.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.381+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c41a1c70 con 0x7f47c4101320 2024-07-04T08:02:38.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.381+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c41a83d0 con 0x7f47c4101320 2024-07-04T08:02:38.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.381+0000 7f47c0c00640 1 -- 172.21.15.39:0/479367494 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47b40108e0 con 0x7f47c4101320 2024-07-04T08:02:38.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.381+0000 7f47c0c00640 1 -- 172.21.15.39:0/479367494 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4018980 con 0x7f47c4101320 2024-07-04T08:02:38.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.383+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a4005290 con 0x7f47c4101320 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d680b2830 msgr2=0x7f0d680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d680b2830 0x7f0d680b4d40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d700095f0 tx=0x7f0d70005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84105300 msgr2=0x7f0d8419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84105300 0x7f0d8419bc50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d74005940 tx=0x7f0d740040c0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 shutdown_connections 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d680b2830 0x7f0d680b4d40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d8410b590 0x7f0d841a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d8410ac40 0x7f0d8419c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 --2- 172.21.15.39:0/678265159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d84105300 0x7f0d8419bc50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.384+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 >> 172.21.15.39:0/678265159 conn(0x7f0d84100d10 msgr2=0x7f0d84104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:38.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.385+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 shutdown_connections 2024-07-04T08:02:38.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.385+0000 7f0d89400640 1 -- 172.21.15.39:0/678265159 wait complete. 2024-07-04T08:02:38.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.387+0000 7f47c0c00640 1 -- 172.21.15.39:0/479367494 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f47b4010430 con 0x7f47c4101320 2024-07-04T08:02:38.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.389+0000 7f47c0c00640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f47a80b2830 0x7f47a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:38.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.389+0000 7f47c0c00640 1 -- 172.21.15.39:0/479367494 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f47b40e0910 con 0x7f47c4101320 2024-07-04T08:02:38.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.389+0000 7f47c3400640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f47a80b2830 0x7f47a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:38.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.390+0000 7f47c3400640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f47a80b2830 0x7f47a80b4d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f47b002a9e0 tx=0x7f47b0005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:38.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.391+0000 7f47c0c00640 1 -- 172.21.15.39:0/479367494 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f47b40aa230 con 0x7f47c4101320 2024-07-04T08:02:38.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:38 smithi039 ceph-mon[32062]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:38.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:38 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:38.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.613+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f47a4005580 con 0x7f47c4101320 2024-07-04T08:02:38.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.623+0000 7f47c0c00640 1 -- 172.21.15.39:0/479367494 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v25)=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7f47b40a9b30 con 0x7f47c4101320 2024-07-04T08:02:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f47a80b2830 msgr2=0x7f47a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f47a80b2830 0x7f47a80b4d40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f47b002a9e0 tx=0x7f47b0005a20 comp rx=0 tx=0).stop 2024-07-04T08:02:38.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47c4101320 msgr2=0x7f47c419ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47c4101320 0x7f47c419ba10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f47b400e9f0 tx=0x7f47b400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 shutdown_connections 2024-07-04T08:02:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f47a80b2830 0x7f47a80b4d40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f47c4105f50 0x7f47c41a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47c4105600 0x7f47c419bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 --2- 172.21.15.39:0/479367494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47c4101320 0x7f47c419ba10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:38.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.625+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 >> 172.21.15.39:0/479367494 conn(0x7f47c4076ff0 msgr2=0x7f47c410ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:38.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.626+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 shutdown_connections 2024-07-04T08:02:38.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:38.626+0000 7f47c9400640 1 -- 172.21.15.39:0/479367494 wait complete. 2024-07-04T08:02:38.717 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:38 smithi133 ceph-mon[37050]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:38.718 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:38 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:38.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:38 smithi050 ceph-mon[36577]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:38.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:38 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:39.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:39 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/479367494' entity='client.admin' 2024-07-04T08:02:39.989 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:39 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/479367494' entity='client.admin' 2024-07-04T08:02:40.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:39 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/479367494' entity='client.admin' 2024-07-04T08:02:40.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:40 smithi050 ceph-mon[36577]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:40 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:40.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:40 smithi039 ceph-mon[32062]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:40.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:40 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:40.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:40.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:40.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:40.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:40 smithi133 ceph-mon[37050]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:40 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:41.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:41.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:41.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.268+0000 7f6934200640 1 -- 172.21.15.39:0/947247131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c103f00 msgr2=0x7f692c104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:41.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.268+0000 7f6934200640 1 --2- 172.21.15.39:0/947247131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c103f00 0x7f692c104310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6920009900 tx=0x7f692002a410 comp rx=0 tx=0).stop 2024-07-04T08:02:41.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.269+0000 7f6934200640 1 -- 172.21.15.39:0/947247131 shutdown_connections 2024-07-04T08:02:41.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.269+0000 7f6934200640 1 --2- 172.21.15.39:0/947247131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f692c10d360 0x7f692c10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:41.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.269+0000 7f6934200640 1 --2- 172.21.15.39:0/947247131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f692c104850 0x7f692c10cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:41.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.269+0000 7f6934200640 1 --2- 172.21.15.39:0/947247131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c103f00 0x7f692c104310 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:41.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.269+0000 7f6934200640 1 -- 172.21.15.39:0/947247131 >> 172.21.15.39:0/947247131 conn(0x7f692c078c80 msgr2=0x7f692c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:41.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.269+0000 7f6934200640 1 -- 172.21.15.39:0/947247131 shutdown_connections 2024-07-04T08:02:41.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.269+0000 7f6934200640 1 -- 172.21.15.39:0/947247131 wait complete. 2024-07-04T08:02:41.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.270+0000 7f6934200640 1 Processor -- start 2024-07-04T08:02:41.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6934200640 1 -- start start 2024-07-04T08:02:41.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6934200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f692c103f00 0x7f692c101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:41.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6934200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f692c104850 0x7f692c102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:41.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6934200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c10d360 0x7f692c1026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:41.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6934200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f692c112ff0 con 0x7f692c10d360 2024-07-04T08:02:41.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6934200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f692c112e70 con 0x7f692c103f00 2024-07-04T08:02:41.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6934200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f692c113170 con 0x7f692c104850 2024-07-04T08:02:41.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6931800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f692c103f00 0x7f692c101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:41.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6931800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f692c103f00 0x7f692c101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50038/0 (socket says 172.21.15.39:50038) 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.271+0000 7f6931800640 1 -- 172.21.15.39:0/1440385657 learned_addr learned my addr 172.21.15.39:0/1440385657 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.272+0000 7f6930e00640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f692c104850 0x7f692c102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.272+0000 7f6932200640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c10d360 0x7f692c1026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.272+0000 7f6931800640 1 -- 172.21.15.39:0/1440385657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f692c104850 msgr2=0x7f692c102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.272+0000 7f6931800640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f692c104850 0x7f692c102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.272+0000 7f6931800640 1 -- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c10d360 msgr2=0x7f692c1026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.272+0000 7f6931800640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c10d360 0x7f692c1026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.272+0000 7f6931800640 1 -- 172.21.15.39:0/1440385657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c1a7cc0 con 0x7f692c103f00 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.272+0000 7f6931800640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f692c103f00 0x7f692c101c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6920032fd0 tx=0x7f6920031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:41.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.273+0000 7f692a000640 1 -- 172.21.15.39:0/1440385657 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692003f030 con 0x7f692c103f00 2024-07-04T08:02:41.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.273+0000 7f692a000640 1 -- 172.21.15.39:0/1440385657 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6920004490 con 0x7f692c103f00 2024-07-04T08:02:41.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.273+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c1a7f50 con 0x7f692c103f00 2024-07-04T08:02:41.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.273+0000 7f692a000640 1 -- 172.21.15.39:0/1440385657 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920035070 con 0x7f692c103f00 2024-07-04T08:02:41.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.273+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c1a83a0 con 0x7f692c103f00 2024-07-04T08:02:41.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.274+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692c104310 con 0x7f692c103f00 2024-07-04T08:02:41.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.276+0000 7f692a000640 1 -- 172.21.15.39:0/1440385657 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f6920042030 con 0x7f692c103f00 2024-07-04T08:02:41.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.278+0000 7f692a000640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f69140b27b0 0x7f69140b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:41.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.278+0000 7f692a000640 1 -- 172.21.15.39:0/1440385657 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6920100790 con 0x7f692c103f00 2024-07-04T08:02:41.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.278+0000 7f6930e00640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f69140b27b0 0x7f69140b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:41.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.279+0000 7f6930e00640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f69140b27b0 0x7f69140b4cc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f692c101930 tx=0x7f6918005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:41.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.282+0000 7f692a000640 1 -- 172.21.15.39:0/1440385657 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f69200ca010 con 0x7f692c103f00 2024-07-04T08:02:41.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.525+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f692c02de10 con 0x7f692c103f00 2024-07-04T08:02:41.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.534+0000 7f692a000640 1 -- 172.21.15.39:0/1440385657 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v27)=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7f69200d2090 con 0x7f692c103f00 2024-07-04T08:02:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.537+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f69140b27b0 msgr2=0x7f69140b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.537+0000 7f6934200640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f69140b27b0 0x7f69140b4cc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f692c101930 tx=0x7f6918005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.537+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f692c103f00 msgr2=0x7f692c101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:41.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.537+0000 7f6934200640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f692c103f00 0x7f692c101c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6920032fd0 tx=0x7f6920031a20 comp rx=0 tx=0).stop 2024-07-04T08:02:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.538+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 shutdown_connections 2024-07-04T08:02:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.538+0000 7f6934200640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f69140b27b0 0x7f69140b4cc0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.538+0000 7f6934200640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c10d360 0x7f692c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.538+0000 7f6934200640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f692c104850 0x7f692c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.538+0000 7f6934200640 1 --2- 172.21.15.39:0/1440385657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f692c103f00 0x7f692c101c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:41.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.538+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 >> 172.21.15.39:0/1440385657 conn(0x7f692c078c80 msgr2=0x7f692c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.538+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 shutdown_connections 2024-07-04T08:02:41.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:41.538+0000 7f6934200640 1 -- 172.21.15.39:0/1440385657 wait complete. 2024-07-04T08:02:42.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.759+0000 7f3365800640 1 -- 172.21.15.39:0/1694020678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3360109510 msgr2=0x7f3360109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:42.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.759+0000 7f3365800640 1 --2- 172.21.15.39:0/1694020678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3360109510 0x7f3360109920 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3350009900 tx=0x7f335002a410 comp rx=0 tx=0).stop 2024-07-04T08:02:42.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.759+0000 7f3365800640 1 -- 172.21.15.39:0/1694020678 shutdown_connections 2024-07-04T08:02:42.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.759+0000 7f3365800640 1 --2- 172.21.15.39:0/1694020678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f336010f7a0 0x7f3360115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.759+0000 7f3365800640 1 --2- 172.21.15.39:0/1694020678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f336010ee50 0x7f336010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.759+0000 7f3365800640 1 --2- 172.21.15.39:0/1694020678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3360109510 0x7f3360109920 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.759+0000 7f3365800640 1 -- 172.21.15.39:0/1694020678 >> 172.21.15.39:0/1694020678 conn(0x7f3360078b10 msgr2=0x7f3360107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.759+0000 7f3365800640 1 -- 172.21.15.39:0/1694020678 shutdown_connections 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.759+0000 7f3365800640 1 -- 172.21.15.39:0/1694020678 wait complete. 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.761+0000 7f3365800640 1 Processor -- start 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.761+0000 7f3365800640 1 -- start start 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.761+0000 7f3365800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3360109510 0x7f336019fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f3365800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f336010ee50 0x7f33601a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f3365800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f336010f7a0 0x7f33601a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f3365800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f33601194a0 con 0x7f3360109510 2024-07-04T08:02:42.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f3365800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3360119320 con 0x7f336010f7a0 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f3365800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3360119620 con 0x7f336010ee50 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f335f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f336010ee50 0x7f33601a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f335fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3360109510 0x7f336019fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f335f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f336010ee50 0x7f33601a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42108/0 (socket says 172.21.15.39:42108) 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f335f400640 1 -- 172.21.15.39:0/2015247342 learned_addr learned my addr 172.21.15.39:0/2015247342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f3364e00640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f336010f7a0 0x7f33601a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f335f400640 1 -- 172.21.15.39:0/2015247342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f336010f7a0 msgr2=0x7f33601a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.762+0000 7f335f400640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f336010f7a0 0x7f33601a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f335f400640 1 -- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3360109510 msgr2=0x7f336019fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f335f400640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3360109510 0x7f336019fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f335f400640 1 -- 172.21.15.39:0/2015247342 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33601a5e40 con 0x7f336010ee50 2024-07-04T08:02:42.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f3364e00640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f336010f7a0 0x7f33601a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f335fe00640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3360109510 0x7f336019fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f335f400640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f336010ee50 0x7f33601a0470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f334c00e970 tx=0x7f334c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f335cc00640 1 -- 172.21.15.39:0/2015247342 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c00cd80 con 0x7f336010ee50 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f335cc00640 1 -- 172.21.15.39:0/2015247342 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f334c0035f0 con 0x7f336010ee50 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f335cc00640 1 -- 172.21.15.39:0/2015247342 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c010690 con 0x7f336010ee50 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33601a6130 con 0x7f336010ee50 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.763+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33601ac890 con 0x7f336010ee50 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.765+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3360109920 con 0x7f336010ee50 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.768+0000 7f335cc00640 1 -- 172.21.15.39:0/2015247342 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f334c003aa0 con 0x7f336010ee50 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.769+0000 7f335cc00640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33440b2830 0x7f33440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.769+0000 7f335cc00640 1 -- 172.21.15.39:0/2015247342 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f334c0e0950 con 0x7f336010ee50 2024-07-04T08:02:42.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.769+0000 7f335fe00640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33440b2830 0x7f33440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.770+0000 7f335fe00640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33440b2830 0x7f33440b4d40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3350006010 tx=0x7f3350000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.773+0000 7f335cc00640 1 -- 172.21.15.39:0/2015247342 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f334c0aa1d0 con 0x7f336010ee50 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:39.988+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f3360111580 con 0x7f336010ee50 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.001+0000 7f335cc00640 1 -- 172.21.15.39:0/2015247342 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v26)=0 v26) ==== 133+0+0 (secure 0 0 0) 0x7f334c0a9ad0 con 0x7f336010ee50 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.003+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33440b2830 msgr2=0x7f33440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.003+0000 7f3365800640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33440b2830 0x7f33440b4d40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3350006010 tx=0x7f3350000bc0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.003+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f336010ee50 msgr2=0x7f33601a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.004+0000 7f3365800640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f336010ee50 0x7f33601a0470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f334c00e970 tx=0x7f334c00ee30 comp rx=0 tx=0).stop 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.004+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 shutdown_connections 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.004+0000 7f3365800640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33440b2830 0x7f33440b4d40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.004+0000 7f3365800640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f336010f7a0 0x7f33601a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.004+0000 7f3365800640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f336010ee50 0x7f33601a0470 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.004+0000 7f3365800640 1 --2- 172.21.15.39:0/2015247342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3360109510 0x7f336019fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:42.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.004+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 >> 172.21.15.39:0/2015247342 conn(0x7f3360078b10 msgr2=0x7f3360107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:42.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.004+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 shutdown_connections 2024-07-04T08:02:42.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:40.004+0000 7f3365800640 1 -- 172.21.15.39:0/2015247342 wait complete. 2024-07-04T08:02:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:42 smithi133 ceph-mon[37050]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:42 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:42.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:42 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:42.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:42 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:42.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:42 smithi039 ceph-mon[32062]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:42.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:42 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:42.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:42.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:42.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:42.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:42.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:42 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:42.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:42 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:42.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:42 smithi050 ceph-mon[36577]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:42 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:42 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:42 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:43.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.197+0000 7f7c61a00640 1 -- 172.21.15.39:0/4061370381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c5c1078f0 msgr2=0x7f7c5c107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.197+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4061370381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c5c1078f0 0x7f7c5c107d00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009900 tx=0x7f7c4402a410 comp rx=0 tx=0).stop 2024-07-04T08:02:43.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.197+0000 7f7c61a00640 1 -- 172.21.15.39:0/4061370381 shutdown_connections 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.197+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4061370381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c5c1078f0 0x7f7c5c107d00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.197+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4061370381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c5c108240 0x7f7c5c10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.197+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4061370381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c5c101fb0 0x7f7c5c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.197+0000 7f7c61a00640 1 -- 172.21.15.39:0/4061370381 >> 172.21.15.39:0/4061370381 conn(0x7f7c5c078c80 msgr2=0x7f7c5c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.197+0000 7f7c61a00640 1 -- 172.21.15.39:0/4061370381 shutdown_connections 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.197+0000 7f7c61a00640 1 -- 172.21.15.39:0/4061370381 wait complete. 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.198+0000 7f7c61a00640 1 Processor -- start 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.198+0000 7f7c61a00640 1 -- start start 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c61a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c5c101fb0 0x7f7c5c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c61a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c5c1078f0 0x7f7c5c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c61a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c5c108240 0x7f7c5c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c61a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c111eb0 con 0x7f7c5c1078f0 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c61a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c111d30 con 0x7f7c5c101fb0 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c61a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c112030 con 0x7f7c5c108240 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c52000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c5c1078f0 0x7f7c5c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c5aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c5c101fb0 0x7f7c5c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c5aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c5c101fb0 0x7f7c5c1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50066/0 (socket says 172.21.15.39:50066) 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c5aa00640 1 -- 172.21.15.39:0/4197879611 learned_addr learned my addr 172.21.15.39:0/4197879611 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.199+0000 7f7c52000640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c5c1078f0 0x7f7c5c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55712/0 (socket says 172.21.15.39:55712) 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c5b400640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c5c108240 0x7f7c5c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c5aa00640 1 -- 172.21.15.39:0/4197879611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c5c108240 msgr2=0x7f7c5c19f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c5aa00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c5c108240 0x7f7c5c19f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c5aa00640 1 -- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c5c1078f0 msgr2=0x7f7c5c199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c5aa00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c5c1078f0 0x7f7c5c199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c5aa00640 1 -- 172.21.15.39:0/4197879611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c19f960 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c5aa00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c5c101fb0 0x7f7c5c1999f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00eb90 tx=0x7f7c4c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c53e00640 1 -- 172.21.15.39:0/4197879611 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c004d20 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5c19fc50 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c53e00640 1 -- 172.21.15.39:0/4197879611 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c4c00ee90 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.200+0000 7f7c53e00640 1 -- 172.21.15.39:0/4197879611 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c010670 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.201+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5c1a63b0 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.202+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5c1023c0 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.206+0000 7f7c53e00640 1 -- 172.21.15.39:0/4197879611 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f7c4c010930 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.207+0000 7f7c53e00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7c400b2830 0x7f7c400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.207+0000 7f7c53e00640 1 -- 172.21.15.39:0/4197879611 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7c4c0e0f30 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.208+0000 7f7c53e00640 1 -- 172.21.15.39:0/4197879611 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f7c4c0aa7b0 con 0x7f7c5c101fb0 2024-07-04T08:02:43.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.208+0000 7f7c52000640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7c400b2830 0x7f7c400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.208+0000 7f7c52000640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7c400b2830 0x7f7c400b4d40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7c44000c00 tx=0x7f7c44031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.431+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f7c5c19ab20 con 0x7f7c5c101fb0 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.443+0000 7f7c53e00640 1 -- 172.21.15.39:0/4197879611 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v28)=0 v28) ==== 106+0+0 (secure 0 0 0) 0x7f7c5c19ab20 con 0x7f7c5c101fb0 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7c400b2830 msgr2=0x7f7c400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7c400b2830 0x7f7c400b4d40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7c44000c00 tx=0x7f7c44031400 comp rx=0 tx=0).stop 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c5c101fb0 msgr2=0x7f7c5c1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c5c101fb0 0x7f7c5c1999f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00eb90 tx=0x7f7c4c00c6a0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 shutdown_connections 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7c400b2830 0x7f7c400b4d40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c5c1078f0 0x7f7c5c199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c5c108240 0x7f7c5c19f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 --2- 172.21.15.39:0/4197879611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c5c101fb0 0x7f7c5c1999f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.446+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 >> 172.21.15.39:0/4197879611 conn(0x7f7c5c078c80 msgr2=0x7f7c5c06a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:43.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.447+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 shutdown_connections 2024-07-04T08:02:43.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.447+0000 7f7c61a00640 1 -- 172.21.15.39:0/4197879611 wait complete. 2024-07-04T08:02:43.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.142+0000 7f1cb8e00640 1 -- 172.21.15.39:0/499953470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cb4103dd0 msgr2=0x7f1cb41041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.142+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/499953470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cb4103dd0 0x7f1cb41041e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009900 tx=0x7f1ca402a410 comp rx=0 tx=0).stop 2024-07-04T08:02:43.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.142+0000 7f1cb8e00640 1 -- 172.21.15.39:0/499953470 shutdown_connections 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.142+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/499953470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb410d380 0x7f1cb410f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.142+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/499953470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb4104800 0x7f1cb410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.142+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/499953470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cb4103dd0 0x7f1cb41041e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.142+0000 7f1cb8e00640 1 -- 172.21.15.39:0/499953470 >> 172.21.15.39:0/499953470 conn(0x7f1cb4078d00 msgr2=0x7f1cb4079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.142+0000 7f1cb8e00640 1 -- 172.21.15.39:0/499953470 shutdown_connections 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.142+0000 7f1cb8e00640 1 -- 172.21.15.39:0/499953470 wait complete. 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.143+0000 7f1cb8e00640 1 Processor -- start 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.143+0000 7f1cb8e00640 1 -- start start 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb4103dd0 0x7f1cb4101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cb4104800 0x7f1cb4102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb410d380 0x7f1cb4102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb4113030 con 0x7f1cb4104800 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb8e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb4112eb0 con 0x7f1cb4103dd0 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb8e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb41131b0 con 0x7f1cb410d380 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cb4104800 0x7f1cb4102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb4103dd0 0x7f1cb4101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb4103dd0 0x7f1cb4101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50100/0 (socket says 172.21.15.39:50100) 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb1e00640 1 -- 172.21.15.39:0/56413445 learned_addr learned my addr 172.21.15.39:0/56413445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb2800640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb410d380 0x7f1cb4102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb1400640 1 -- 172.21.15.39:0/56413445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb410d380 msgr2=0x7f1cb4102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb1400640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb410d380 0x7f1cb4102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb1400640 1 -- 172.21.15.39:0/56413445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb4103dd0 msgr2=0x7f1cb4101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.144+0000 7f1cb1400640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb4103dd0 0x7f1cb4101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.145+0000 7f1cb1400640 1 -- 172.21.15.39:0/56413445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb41a7d00 con 0x7f1cb4104800 2024-07-04T08:02:43.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.145+0000 7f1cb1400640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cb4104800 0x7f1cb4102220 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1cb406a280 tx=0x7f1ca000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.145+0000 7f1caaa00640 1 -- 172.21.15.39:0/56413445 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca000ccc0 con 0x7f1cb4104800 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.145+0000 7f1caaa00640 1 -- 172.21.15.39:0/56413445 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca0009e40 con 0x7f1cb4104800 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.145+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb41a7ff0 con 0x7f1cb4104800 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.145+0000 7f1caaa00640 1 -- 172.21.15.39:0/56413445 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0004a00 con 0x7f1cb4104800 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.145+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb41a8440 con 0x7f1cb4104800 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.147+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb41041e0 con 0x7f1cb4104800 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.153+0000 7f1caaa00640 1 -- 172.21.15.39:0/56413445 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f1ca0005020 con 0x7f1cb4104800 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.154+0000 7f1caaa00640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c980b2830 0x7f1c980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.154+0000 7f1caaa00640 1 -- 172.21.15.39:0/56413445 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1ca00e0540 con 0x7f1cb4104800 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.155+0000 7f1cb1e00640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c980b2830 0x7f1c980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.155+0000 7f1cb1e00640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c980b2830 0x7f1c980b4d40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009230 tx=0x7f1ca40058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.159+0000 7f1caaa00640 1 -- 172.21.15.39:0/56413445 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f1ca00a9e60 con 0x7f1cb4104800 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.388+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f1cb41a8910 con 0x7f1cb4104800 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.397+0000 7f1caaa00640 1 -- 172.21.15.39:0/56413445 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f1ca00a9760 con 0x7f1cb4104800 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.400+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c980b2830 msgr2=0x7f1c980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.400+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c980b2830 0x7f1c980b4d40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009230 tx=0x7f1ca40058d0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.400+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cb4104800 msgr2=0x7f1cb4102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.400+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cb4104800 0x7f1cb4102220 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1cb406a280 tx=0x7f1ca000ed90 comp rx=0 tx=0).stop 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.401+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 shutdown_connections 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.401+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c980b2830 0x7f1c980b4d40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.401+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cb4104800 0x7f1cb4102220 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1cb406a280 tx=0x7f1ca000ed90 comp rx=0 tx=0).stop 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.401+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb410d380 0x7f1cb4102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.401+0000 7f1cb8e00640 1 --2- 172.21.15.39:0/56413445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1cb4103dd0 0x7f1cb4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.401+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 >> 172.21.15.39:0/56413445 conn(0x7f1cb4078d00 msgr2=0x7f1cb4106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:43.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.401+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 shutdown_connections 2024-07-04T08:02:43.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.401+0000 7f1cb8e00640 1 -- 172.21.15.39:0/56413445 wait complete. 2024-07-04T08:02:43.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.272+0000 7f76bfa00640 1 -- 172.21.15.39:0/4095072664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 msgr2=0x7f76b81034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.272+0000 7f76bfa00640 1 --2- 172.21.15.39:0/4095072664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 0x7f76b81034e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009900 tx=0x7f76ac02a410 comp rx=0 tx=0).stop 2024-07-04T08:02:43.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.272+0000 7f76bfa00640 1 -- 172.21.15.39:0/4095072664 shutdown_connections 2024-07-04T08:02:43.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.272+0000 7f76bfa00640 1 --2- 172.21.15.39:0/4095072664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8109360 0x7f76b810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.272+0000 7f76bfa00640 1 --2- 172.21.15.39:0/4095072664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b8108a10 0x7f76b8108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.272+0000 7f76bfa00640 1 --2- 172.21.15.39:0/4095072664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 0x7f76b81034e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.272+0000 7f76bfa00640 1 -- 172.21.15.39:0/4095072664 >> 172.21.15.39:0/4095072664 conn(0x7f76b80feb20 msgr2=0x7f76b8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.272+0000 7f76bfa00640 1 -- 172.21.15.39:0/4095072664 shutdown_connections 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.272+0000 7f76bfa00640 1 -- 172.21.15.39:0/4095072664 wait complete. 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.273+0000 7f76bfa00640 1 Processor -- start 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.273+0000 7f76bfa00640 1 -- start start 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bfa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 0x7f76b819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bfa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b8108a10 0x7f76b819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bfa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8109360 0x7f76b81a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bfa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8113010 con 0x7f76b8109360 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bfa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8112e90 con 0x7f76b8108a10 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bfa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76b8113190 con 0x7f76b81030d0 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bd000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 0x7f76b819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76b7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b8108a10 0x7f76b819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bd000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 0x7f76b819bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50830/0 (socket says 172.21.15.39:50830) 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bd000640 1 -- 172.21.15.39:0/916926308 learned_addr learned my addr 172.21.15.39:0/916926308 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.274+0000 7f76bda00640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8109360 0x7f76b81a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76bd000640 1 -- 172.21.15.39:0/916926308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b8108a10 msgr2=0x7f76b819c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76bd000640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b8108a10 0x7f76b819c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76bd000640 1 -- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8109360 msgr2=0x7f76b81a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76bd000640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8109360 0x7f76b81a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76bd000640 1 -- 172.21.15.39:0/916926308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b81a1b90 con 0x7f76b81030d0 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76bd000640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 0x7f76b819bc20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76ac005940 tx=0x7f76ac0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76b5600640 1 -- 172.21.15.39:0/916926308 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac03f030 con 0x7f76b81030d0 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76b5600640 1 -- 172.21.15.39:0/916926308 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76ac0044c0 con 0x7f76b81030d0 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76b5600640 1 -- 172.21.15.39:0/916926308 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac035070 con 0x7f76b81030d0 2024-07-04T08:02:43.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b81a1e20 con 0x7f76b81030d0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.275+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81a85e0 con 0x7f76b81030d0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.276+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b8104350 con 0x7f76b81030d0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.279+0000 7f76b5600640 1 -- 172.21.15.39:0/916926308 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f76ac042030 con 0x7f76b81030d0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.281+0000 7f76b5600640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76a00b27b0 0x7f76a00b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.281+0000 7f76b5600640 1 -- 172.21.15.39:0/916926308 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f76ac100c00 con 0x7f76b81030d0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.281+0000 7f76b7e00640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76a00b27b0 0x7f76a00b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.282+0000 7f76b7e00640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76a00b27b0 0x7f76a00b4cc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f76b8110040 tx=0x7f76a4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.282+0000 7f76b5600640 1 -- 172.21.15.39:0/916926308 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f76ac0c9e30 con 0x7f76b81030d0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.511+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f76b81034e0 con 0x7f76b81030d0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.518+0000 7f76b5600640 1 -- 172.21.15.39:0/916926308 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v29)=0 v29) ==== 116+0+0 (secure 0 0 0) 0x7f76ac0ce130 con 0x7f76b81030d0 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76a00b27b0 msgr2=0x7f76a00b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76a00b27b0 0x7f76a00b4cc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f76b8110040 tx=0x7f76a4005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:43.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 msgr2=0x7f76b819bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 0x7f76b819bc20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f76ac005940 tx=0x7f76ac0040c0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 shutdown_connections 2024-07-04T08:02:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76a00b27b0 0x7f76a00b4cc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b8109360 0x7f76b81a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f76b8108a10 0x7f76b819c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 --2- 172.21.15.39:0/916926308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76b81030d0 0x7f76b819bc20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 >> 172.21.15.39:0/916926308 conn(0x7f76b80feb20 msgr2=0x7f76b810dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 shutdown_connections 2024-07-04T08:02:43.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:42.520+0000 7f76bfa00640 1 -- 172.21.15.39:0/916926308 wait complete. 2024-07-04T08:02:44.470 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:44 smithi039 ceph-mon[32062]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:44.470 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:44 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/56413445' entity='client.admin' 2024-07-04T08:02:44.470 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:44.470 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:44.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:44.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:44.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:44 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2055639935' entity='client.admin' 2024-07-04T08:02:44.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:44 smithi133 ceph-mon[37050]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:44.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:44 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/56413445' entity='client.admin' 2024-07-04T08:02:44.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:44.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:44.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:44.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:44.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:44 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2055639935' entity='client.admin' 2024-07-04T08:02:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.297+0000 7fe557c00640 1 -- 172.21.15.39:0/4147962498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe550101b80 msgr2=0x7fe550101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.297+0000 7fe557c00640 1 --2- 172.21.15.39:0/4147962498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe550101b80 0x7fe550101f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe544009860 tx=0x7fe54402a3d0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.297+0000 7fe557c00640 1 -- 172.21.15.39:0/4147962498 shutdown_connections 2024-07-04T08:02:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.297+0000 7fe557c00640 1 --2- 172.21.15.39:0/4147962498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55010f730 0x7fe550111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.297+0000 7fe557c00640 1 --2- 172.21.15.39:0/4147962498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5501024d0 0x7fe55010f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.297+0000 7fe557c00640 1 --2- 172.21.15.39:0/4147962498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe550101b80 0x7fe550101f90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.297+0000 7fe557c00640 1 -- 172.21.15.39:0/4147962498 >> 172.21.15.39:0/4147962498 conn(0x7fe550078c80 msgr2=0x7fe550079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:44.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.297+0000 7fe557c00640 1 -- 172.21.15.39:0/4147962498 shutdown_connections 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.297+0000 7fe557c00640 1 -- 172.21.15.39:0/4147962498 wait complete. 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.298+0000 7fe557c00640 1 Processor -- start 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.298+0000 7fe557c00640 1 -- start start 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.298+0000 7fe557c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe550101b80 0x7fe55019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe557c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5501024d0 0x7fe55019c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe557c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55010f730 0x7fe5501a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe557c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe5501151d0 con 0x7fe55010f730 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe557c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe550115050 con 0x7fe550101b80 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe557c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe550115350 con 0x7fe5501024d0 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55010f730 0x7fe5501a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:44.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe54fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5501024d0 0x7fe55019c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55010f730 0x7fe5501a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55780/0 (socket says 172.21.15.39:55780) 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555c00640 1 -- 172.21.15.39:0/2055639935 learned_addr learned my addr 172.21.15.39:0/2055639935 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555200640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe550101b80 0x7fe55019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555c00640 1 -- 172.21.15.39:0/2055639935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5501024d0 msgr2=0x7fe55019c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555c00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5501024d0 0x7fe55019c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555c00640 1 -- 172.21.15.39:0/2055639935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe550101b80 msgr2=0x7fe55019bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555c00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe550101b80 0x7fe55019bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555c00640 1 -- 172.21.15.39:0/2055639935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5501a1ad0 con 0x7fe55010f730 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe555200640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe550101b80 0x7fe55019bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.299+0000 7fe54fe00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5501024d0 0x7fe55019c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.300+0000 7fe555c00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55010f730 0x7fe5501a13a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe548009fd0 tx=0x7fe54800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:44.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.300+0000 7fe54d600640 1 -- 172.21.15.39:0/2055639935 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548004b20 con 0x7fe55010f730 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.300+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5501a1dc0 con 0x7fe55010f730 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.300+0000 7fe54d600640 1 -- 172.21.15.39:0/2055639935 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe548003e20 con 0x7fe55010f730 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.300+0000 7fe54d600640 1 -- 172.21.15.39:0/2055639935 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548004820 con 0x7fe55010f730 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.300+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5501a8520 con 0x7fe55010f730 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.302+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe550101f90 con 0x7fe55010f730 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.305+0000 7fe54d600640 1 -- 172.21.15.39:0/2055639935 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7fe54801d020 con 0x7fe55010f730 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.306+0000 7fe54d600640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe5380b27b0 0x7fe5380b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.306+0000 7fe54d600640 1 -- 172.21.15.39:0/2055639935 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe5480e1100 con 0x7fe55010f730 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.306+0000 7fe555200640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe5380b27b0 0x7fe5380b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.307+0000 7fe555200640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe5380b27b0 0x7fe5380b4cc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe5440095f0 tx=0x7fe544005e40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.308+0000 7fe54d600640 1 -- 172.21.15.39:0/2055639935 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe548024030 con 0x7fe55010f730 2024-07-04T08:02:44.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.544+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fe5501a8750 con 0x7fe55010f730 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.549+0000 7fe54d600640 1 -- 172.21.15.39:0/2055639935 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v31)=0 v31) ==== 125+0+0 (secure 0 0 0) 0x7fe5480aa980 con 0x7fe55010f730 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe5380b27b0 msgr2=0x7fe5380b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe5380b27b0 0x7fe5380b4cc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe5440095f0 tx=0x7fe544005e40 comp rx=0 tx=0).stop 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55010f730 msgr2=0x7fe5501a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55010f730 0x7fe5501a13a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe548009fd0 tx=0x7fe54800c800 comp rx=0 tx=0).stop 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 shutdown_connections 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe5380b27b0 0x7fe5380b4cc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe55010f730 0x7fe5501a13a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5501024d0 0x7fe55019c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 --2- 172.21.15.39:0/2055639935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe550101b80 0x7fe55019bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 >> 172.21.15.39:0/2055639935 conn(0x7fe550078c80 msgr2=0x7fe550100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 shutdown_connections 2024-07-04T08:02:44.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:43.552+0000 7fe557c00640 1 -- 172.21.15.39:0/2055639935 wait complete. 2024-07-04T08:02:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:44 smithi050 ceph-mon[36577]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:44 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/56413445' entity='client.admin' 2024-07-04T08:02:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:02:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:02:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:02:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:02:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:44 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2055639935' entity='client.admin' 2024-07-04T08:02:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.713+0000 7f2332800640 1 -- 172.21.15.39:0/4120215961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f232c102f00 msgr2=0x7f232c103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.713+0000 7f2332800640 1 --2- 172.21.15.39:0/4120215961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f232c102f00 0x7f232c103310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2320009900 tx=0x7f232002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.714+0000 7f2332800640 1 -- 172.21.15.39:0/4120215961 shutdown_connections 2024-07-04T08:02:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.714+0000 7f2332800640 1 --2- 172.21.15.39:0/4120215961 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c109190 0x7f232c10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.714+0000 7f2332800640 1 --2- 172.21.15.39:0/4120215961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232c108840 0x7f232c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.714+0000 7f2332800640 1 --2- 172.21.15.39:0/4120215961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f232c102f00 0x7f232c103310 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.714+0000 7f2332800640 1 -- 172.21.15.39:0/4120215961 >> 172.21.15.39:0/4120215961 conn(0x7f232c0fe990 msgr2=0x7f232c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.714+0000 7f2332800640 1 -- 172.21.15.39:0/4120215961 shutdown_connections 2024-07-04T08:02:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.714+0000 7f2332800640 1 -- 172.21.15.39:0/4120215961 wait complete. 2024-07-04T08:02:45.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.715+0000 7f2332800640 1 Processor -- start 2024-07-04T08:02:45.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.715+0000 7f2332800640 1 -- start start 2024-07-04T08:02:45.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2332800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f232c102f00 0x7f232c19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:45.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2332800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232c108840 0x7f232c19c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:45.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2332800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c109190 0x7f232c1a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2332800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112e50 con 0x7f232c102f00 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2332800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112cd0 con 0x7f232c109190 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2332800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f232c112fd0 con 0x7f232c108840 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2331400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f232c102f00 0x7f232c19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2330a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232c108840 0x7f232c19c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2330a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232c108840 0x7f232c19c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50904/0 (socket says 172.21.15.39:50904) 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2330a00640 1 -- 172.21.15.39:0/3082253655 learned_addr learned my addr 172.21.15.39:0/3082253655 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.716+0000 7f2331e00640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c109190 0x7f232c1a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.717+0000 7f2331400640 1 -- 172.21.15.39:0/3082253655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232c108840 msgr2=0x7f232c19c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.717+0000 7f2331400640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232c108840 0x7f232c19c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.717+0000 7f2331400640 1 -- 172.21.15.39:0/3082253655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c109190 msgr2=0x7f232c1a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.717+0000 7f2331400640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c109190 0x7f232c1a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.717+0000 7f2331400640 1 -- 172.21.15.39:0/3082253655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c1a1ad0 con 0x7f232c102f00 2024-07-04T08:02:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.718+0000 7f2331400640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f232c102f00 0x7f232c19bb60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f232002a910 tx=0x7f2320031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.718+0000 7f232a000640 1 -- 172.21.15.39:0/3082253655 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232003f030 con 0x7f232c102f00 2024-07-04T08:02:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.718+0000 7f232a000640 1 -- 172.21.15.39:0/3082253655 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23200043d0 con 0x7f232c102f00 2024-07-04T08:02:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.718+0000 7f232a000640 1 -- 172.21.15.39:0/3082253655 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320035070 con 0x7f232c102f00 2024-07-04T08:02:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.718+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c1a1d60 con 0x7f232c102f00 2024-07-04T08:02:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.718+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c1a8520 con 0x7f232c102f00 2024-07-04T08:02:45.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.720+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c103310 con 0x7f232c102f00 2024-07-04T08:02:45.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.723+0000 7f232a000640 1 -- 172.21.15.39:0/3082253655 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f2320042030 con 0x7f232c102f00 2024-07-04T08:02:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.724+0000 7f232a000640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23140b2830 0x7f23140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.725+0000 7f232a000640 1 -- 172.21.15.39:0/3082253655 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f23201008a0 con 0x7f232c102f00 2024-07-04T08:02:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.725+0000 7f2330a00640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23140b2830 0x7f23140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:45.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.726+0000 7f2330a00640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23140b2830 0x7f23140b4d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f23180095f0 tx=0x7f2318005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:45.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.729+0000 7f232a000640 1 -- 172.21.15.39:0/3082253655 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f23200ca120 con 0x7f232c102f00 2024-07-04T08:02:45.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.947+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f232c1a87c0 con 0x7f232c102f00 2024-07-04T08:02:45.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.954+0000 7f232a000640 1 -- 172.21.15.39:0/3082253655 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v32)=0 v32) ==== 108+0+0 (secure 0 0 0) 0x7f23200d21a0 con 0x7f232c102f00 2024-07-04T08:02:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23140b2830 msgr2=0x7f23140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23140b2830 0x7f23140b4d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f23180095f0 tx=0x7f2318005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f232c102f00 msgr2=0x7f232c19bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:45.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f232c102f00 0x7f232c19bb60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f232002a910 tx=0x7f2320031a20 comp rx=0 tx=0).stop 2024-07-04T08:02:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 shutdown_connections 2024-07-04T08:02:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23140b2830 0x7f23140b4d40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f232c109190 0x7f232c1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f232c108840 0x7f232c19c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 --2- 172.21.15.39:0/3082253655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f232c102f00 0x7f232c19bb60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 >> 172.21.15.39:0/3082253655 conn(0x7f232c0fe990 msgr2=0x7f232c107f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.957+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 shutdown_connections 2024-07-04T08:02:45.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:45.958+0000 7f2332800640 1 -- 172.21.15.39:0/3082253655 wait complete. 2024-07-04T08:02:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:46 smithi039 ceph-mon[32062]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:46 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3082253655' entity='client.admin' 2024-07-04T08:02:47.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:46 smithi133 ceph-mon[37050]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:47.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:46 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3082253655' entity='client.admin' 2024-07-04T08:02:47.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:46 smithi050 ceph-mon[36577]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:47.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:46 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3082253655' entity='client.admin' 2024-07-04T08:02:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.663+0000 7f4ead200640 1 -- 172.21.15.39:0/2848984782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8103bc0 msgr2=0x7f4ea8103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.663+0000 7f4ead200640 1 --2- 172.21.15.39:0/2848984782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8103bc0 0x7f4ea8103fd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009900 tx=0x7f4e9802a410 comp rx=0 tx=0).stop 2024-07-04T08:02:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.663+0000 7f4ead200640 1 -- 172.21.15.39:0/2848984782 shutdown_connections 2024-07-04T08:02:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.663+0000 7f4ead200640 1 --2- 172.21.15.39:0/2848984782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ea81045f0 0x7f4ea810cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.663+0000 7f4ead200640 1 --2- 172.21.15.39:0/2848984782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ea810d170 0x7f4ea810f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:47.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.663+0000 7f4ead200640 1 --2- 172.21.15.39:0/2848984782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8103bc0 0x7f4ea8103fd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.663+0000 7f4ead200640 1 -- 172.21.15.39:0/2848984782 >> 172.21.15.39:0/2848984782 conn(0x7f4ea80fd810 msgr2=0x7f4ea80ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.663+0000 7f4ead200640 1 -- 172.21.15.39:0/2848984782 shutdown_connections 2024-07-04T08:02:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.663+0000 7f4ead200640 1 -- 172.21.15.39:0/2848984782 wait complete. 2024-07-04T08:02:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.665+0000 7f4ead200640 1 Processor -- start 2024-07-04T08:02:47.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.665+0000 7f4ead200640 1 -- start start 2024-07-04T08:02:47.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.665+0000 7f4ead200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8103bc0 0x7f4ea8101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:47.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.665+0000 7f4ead200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ea81045f0 0x7f4ea8102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:47.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.666+0000 7f4ead200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ea810d170 0x7f4ea8102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.666+0000 7f4ead200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8112e20 con 0x7f4ea810d170 2024-07-04T08:02:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.666+0000 7f4ead200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8112ca0 con 0x7f4ea8103bc0 2024-07-04T08:02:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.666+0000 7f4ead200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8112fa0 con 0x7f4ea81045f0 2024-07-04T08:02:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.666+0000 7f4ea7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8103bc0 0x7f4ea8101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.666+0000 7f4ea7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8103bc0 0x7f4ea8101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50162/0 (socket says 172.21.15.39:50162) 2024-07-04T08:02:47.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.666+0000 7f4ea7400640 1 -- 172.21.15.39:0/3860696010 learned_addr learned my addr 172.21.15.39:0/3860696010 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:47.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.666+0000 7f4ea6a00640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ea81045f0 0x7f4ea8102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:47.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.666+0000 7f4ea7e00640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ea810d170 0x7f4ea8102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:47.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4ea7e00640 1 -- 172.21.15.39:0/3860696010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ea81045f0 msgr2=0x7f4ea8102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:47.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4ea7e00640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ea81045f0 0x7f4ea8102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4ea7e00640 1 -- 172.21.15.39:0/3860696010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8103bc0 msgr2=0x7f4ea8101ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4ea7e00640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8103bc0 0x7f4ea8101ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4ea7e00640 1 -- 172.21.15.39:0/3860696010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea81a7ad0 con 0x7f4ea810d170 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4ea6a00640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ea81045f0 0x7f4ea8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4ea7e00640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ea810d170 0x7f4ea8102550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009fd0 tx=0x7f4ea000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4e9fe00640 1 -- 172.21.15.39:0/3860696010 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea000ce20 con 0x7f4ea810d170 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4e9fe00640 1 -- 172.21.15.39:0/3860696010 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea0005020 con 0x7f4ea810d170 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4e9fe00640 1 -- 172.21.15.39:0/3860696010 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0010690 con 0x7f4ea810d170 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.667+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea81a7dc0 con 0x7f4ea810d170 2024-07-04T08:02:47.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.668+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea81a8220 con 0x7f4ea810d170 2024-07-04T08:02:47.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.669+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea8103fd0 con 0x7f4ea810d170 2024-07-04T08:02:47.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.672+0000 7f4e9fe00640 1 -- 172.21.15.39:0/3860696010 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f4ea00054d0 con 0x7f4ea810d170 2024-07-04T08:02:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.674+0000 7f4e9fe00640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e8c0b2830 0x7f4e8c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.675+0000 7f4e9fe00640 1 -- 172.21.15.39:0/3860696010 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4ea00e0d30 con 0x7f4ea810d170 2024-07-04T08:02:47.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.675+0000 7f4ea7400640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e8c0b2830 0x7f4e8c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:47.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.676+0000 7f4ea7400640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e8c0b2830 0x7f4e8c0b4d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e98000c00 tx=0x7f4e98031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:47.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.676+0000 7f4e9fe00640 1 -- 172.21.15.39:0/3860696010 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f4ea00aa5b0 con 0x7f4ea810d170 2024-07-04T08:02:47.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.900+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f4ea81a8740 con 0x7f4ea810d170 2024-07-04T08:02:47.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.909+0000 7f4e9fe00640 1 -- 172.21.15.39:0/3860696010 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v33)=0 v33) ==== 119+0+0 (secure 0 0 0) 0x7f4ea00a9eb0 con 0x7f4ea810d170 2024-07-04T08:02:47.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.912+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e8c0b2830 msgr2=0x7f4e8c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:47.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.912+0000 7f4ead200640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e8c0b2830 0x7f4e8c0b4d40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e98000c00 tx=0x7f4e98031400 comp rx=0 tx=0).stop 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.912+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ea810d170 msgr2=0x7f4ea8102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.912+0000 7f4ead200640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ea810d170 0x7f4ea8102550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009fd0 tx=0x7f4ea000ef10 comp rx=0 tx=0).stop 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.913+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 shutdown_connections 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.913+0000 7f4ead200640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e8c0b2830 0x7f4e8c0b4d40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.913+0000 7f4ead200640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ea810d170 0x7f4ea8102550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.913+0000 7f4ead200640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ea81045f0 0x7f4ea8102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.913+0000 7f4ead200640 1 --2- 172.21.15.39:0/3860696010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4ea8103bc0 0x7f4ea8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.913+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 >> 172.21.15.39:0/3860696010 conn(0x7f4ea80fd810 msgr2=0x7f4ea8106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.913+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 shutdown_connections 2024-07-04T08:02:47.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:47.913+0000 7f4ead200640 1 -- 172.21.15.39:0/3860696010 wait complete. 2024-07-04T08:02:48.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.596+0000 7f0f3a800640 1 -- 172.21.15.39:0/1287891928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34108a00 msgr2=0x7f0f34108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.596+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1287891928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34108a00 0x7f0f34108e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009900 tx=0x7f0f2002a400 comp rx=0 tx=0).stop 2024-07-04T08:02:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.596+0000 7f0f3a800640 1 -- 172.21.15.39:0/1287891928 shutdown_connections 2024-07-04T08:02:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.596+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1287891928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34108a00 0x7f0f34108e10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.596+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1287891928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f34109350 0x7f0f3410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.596+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1287891928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f341030c0 0x7f0f341034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.596+0000 7f0f3a800640 1 -- 172.21.15.39:0/1287891928 >> 172.21.15.39:0/1287891928 conn(0x7f0f340feb30 msgr2=0x7f0f34100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.597+0000 7f0f3a800640 1 -- 172.21.15.39:0/1287891928 shutdown_connections 2024-07-04T08:02:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.597+0000 7f0f3a800640 1 -- 172.21.15.39:0/1287891928 wait complete. 2024-07-04T08:02:48.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.598+0000 7f0f3a800640 1 Processor -- start 2024-07-04T08:02:48.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.598+0000 7f0f3a800640 1 -- start start 2024-07-04T08:02:48.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.598+0000 7f0f3a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f341030c0 0x7f0f34199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:48.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f3a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f34108a00 0x7f0f34199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:48.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f3a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34109350 0x7f0f3419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:48.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f3a800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34112fc0 con 0x7f0f34109350 2024-07-04T08:02:48.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f3a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34112e40 con 0x7f0f341030c0 2024-07-04T08:02:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f3a800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34113140 con 0x7f0f34108a00 2024-07-04T08:02:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f33400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f341030c0 0x7f0f34199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34109350 0x7f0f3419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34109350 0x7f0f3419f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55834/0 (socket says 172.21.15.39:55834) 2024-07-04T08:02:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f33400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f341030c0 0x7f0f34199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50188/0 (socket says 172.21.15.39:50188) 2024-07-04T08:02:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f33e00640 1 -- 172.21.15.39:0/1401811427 learned_addr learned my addr 172.21.15.39:0/1401811427 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f33e00640 1 -- 172.21.15.39:0/1401811427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f34108a00 msgr2=0x7f0f34199fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-04T08:02:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f33e00640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f34108a00 0x7f0f34199fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f33e00640 1 -- 172.21.15.39:0/1401811427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f341030c0 msgr2=0x7f0f34199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.599+0000 7f0f33e00640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f341030c0 0x7f0f34199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.600+0000 7f0f33e00640 1 -- 172.21.15.39:0/1401811427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3419f980 con 0x7f0f34109350 2024-07-04T08:02:48.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.600+0000 7f0f33e00640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34109350 0x7f0f3419f250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00eaa0 tx=0x7f0f2c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:48.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.600+0000 7f0f2be00640 1 -- 172.21.15.39:0/1401811427 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c00cdb0 con 0x7f0f34109350 2024-07-04T08:02:48.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.600+0000 7f0f2be00640 1 -- 172.21.15.39:0/1401811427 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f2c005020 con 0x7f0f34109350 2024-07-04T08:02:48.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.600+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3419fc70 con 0x7f0f34109350 2024-07-04T08:02:48.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.600+0000 7f0f2be00640 1 -- 172.21.15.39:0/1401811427 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0116d0 con 0x7f0f34109350 2024-07-04T08:02:48.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.600+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f341a63d0 con 0x7f0f34109350 2024-07-04T08:02:48.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.602+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f341034d0 con 0x7f0f34109350 2024-07-04T08:02:48.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.605+0000 7f0f2be00640 1 -- 172.21.15.39:0/1401811427 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f0f2c0051d0 con 0x7f0f34109350 2024-07-04T08:02:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.606+0000 7f0f2be00640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f180b27b0 0x7f0f180b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.607+0000 7f0f2be00640 1 -- 172.21.15.39:0/1401811427 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0f2c0e0c30 con 0x7f0f34109350 2024-07-04T08:02:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.607+0000 7f0f33400640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f180b27b0 0x7f0f180b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.608+0000 7f0f33400640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f180b27b0 0x7f0f180b4cc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f240095f0 tx=0x7f0f24005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.612+0000 7f0f2be00640 1 -- 172.21.15.39:0/1401811427 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f0f2c016030 con 0x7f0f34109350 2024-07-04T08:02:48.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.826+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f0f3410b100 con 0x7f0f34109350 2024-07-04T08:02:48.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.834+0000 7f0f2be00640 1 -- 172.21.15.39:0/1401811427 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v34)=0 v34) ==== 121+0+0 (secure 0 0 0) 0x7f0f2c0aa4b0 con 0x7f0f34109350 2024-07-04T08:02:48.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.837+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f180b27b0 msgr2=0x7f0f180b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:48.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.837+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f180b27b0 0x7f0f180b4cc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f240095f0 tx=0x7f0f24005f90 comp rx=0 tx=0).stop 2024-07-04T08:02:48.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.837+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34109350 msgr2=0x7f0f3419f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:48.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.837+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34109350 0x7f0f3419f250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00eaa0 tx=0x7f0f2c00ef60 comp rx=0 tx=0).stop 2024-07-04T08:02:48.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.838+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 shutdown_connections 2024-07-04T08:02:48.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.838+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f180b27b0 0x7f0f180b4cc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:48.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.838+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f34109350 0x7f0f3419f250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:48.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.838+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f34108a00 0x7f0f34199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:48.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.838+0000 7f0f3a800640 1 --2- 172.21.15.39:0/1401811427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f341030c0 0x7f0f34199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:48.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.838+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 >> 172.21.15.39:0/1401811427 conn(0x7f0f340feb30 msgr2=0x7f0f34100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:48.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.838+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 shutdown_connections 2024-07-04T08:02:48.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:48.838+0000 7f0f3a800640 1 -- 172.21.15.39:0/1401811427 wait complete. 2024-07-04T08:02:49.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:48 smithi039 ceph-mon[32062]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:49.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:48 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3860696010' entity='client.admin' 2024-07-04T08:02:49.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:48 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1401811427' entity='client.admin' 2024-07-04T08:02:49.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:48 smithi133 ceph-mon[37050]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:49.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:48 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3860696010' entity='client.admin' 2024-07-04T08:02:49.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:48 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1401811427' entity='client.admin' 2024-07-04T08:02:49.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:48 smithi050 ceph-mon[36577]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:48 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3860696010' entity='client.admin' 2024-07-04T08:02:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:48 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1401811427' entity='client.admin' 2024-07-04T08:02:49.808 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi039') 2024-07-04T08:02:50.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.423+0000 7fdbe3000640 1 -- 172.21.15.39:0/984001075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc10ee50 msgr2=0x7fdbdc10f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:50.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.423+0000 7fdbe3000640 1 --2- 172.21.15.39:0/984001075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc10ee50 0x7fdbdc10f260 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009860 tx=0x7fdbc802a340 comp rx=0 tx=0).stop 2024-07-04T08:02:50.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.424+0000 7fdbe3000640 1 -- 172.21.15.39:0/984001075 shutdown_connections 2024-07-04T08:02:50.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.424+0000 7fdbe3000640 1 --2- 172.21.15.39:0/984001075 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbdc10f7a0 0x7fdbdc115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.424+0000 7fdbe3000640 1 --2- 172.21.15.39:0/984001075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc10ee50 0x7fdbdc10f260 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.424+0000 7fdbe3000640 1 --2- 172.21.15.39:0/984001075 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbdc109510 0x7fdbdc109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.424+0000 7fdbe3000640 1 -- 172.21.15.39:0/984001075 >> 172.21.15.39:0/984001075 conn(0x7fdbdc078b10 msgr2=0x7fdbdc107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:50.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.424+0000 7fdbe3000640 1 -- 172.21.15.39:0/984001075 shutdown_connections 2024-07-04T08:02:50.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.424+0000 7fdbe3000640 1 -- 172.21.15.39:0/984001075 wait complete. 2024-07-04T08:02:50.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.425+0000 7fdbe3000640 1 Processor -- start 2024-07-04T08:02:50.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.425+0000 7fdbe3000640 1 -- start start 2024-07-04T08:02:50.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe3000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc109510 0x7fdbdc19fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:50.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe3000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbdc10ee50 0x7fdbdc1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:50.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe3000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbdc10f7a0 0x7fdbdc1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:50.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe3000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc119410 con 0x7fdbdc10ee50 2024-07-04T08:02:50.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe3000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc119290 con 0x7fdbdc10f7a0 2024-07-04T08:02:50.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe3000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc119590 con 0x7fdbdc109510 2024-07-04T08:02:50.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe1c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc109510 0x7fdbdc19fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:50.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe1c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc109510 0x7fdbdc19fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50952/0 (socket says 172.21.15.39:50952) 2024-07-04T08:02:50.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe1c00640 1 -- 172.21.15.39:0/3826820419 learned_addr learned my addr 172.21.15.39:0/3826820419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:02:50.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.426+0000 7fdbe1200640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbdc10ee50 0x7fdbdc1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:50.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.427+0000 7fdbe2600640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbdc10f7a0 0x7fdbdc1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:50.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.427+0000 7fdbe1c00640 1 -- 172.21.15.39:0/3826820419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbdc10f7a0 msgr2=0x7fdbdc1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.427+0000 7fdbe1c00640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbdc10f7a0 0x7fdbdc1a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.427+0000 7fdbe1c00640 1 -- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbdc10ee50 msgr2=0x7fdbdc1a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.427+0000 7fdbe1c00640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbdc10ee50 0x7fdbdc1a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.427+0000 7fdbe1c00640 1 -- 172.21.15.39:0/3826820419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc1a5df0 con 0x7fdbdc109510 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.427+0000 7fdbe1c00640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc109510 0x7fdbdc19fe80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000e9f0 tx=0x7fdbd000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.428+0000 7fdbda000640 1 -- 172.21.15.39:0/3826820419 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd000cd80 con 0x7fdbdc109510 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.428+0000 7fdbda000640 1 -- 172.21.15.39:0/3826820419 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd00108e0 con 0x7fdbdc109510 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.428+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc1a60e0 con 0x7fdbdc109510 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.428+0000 7fdbda000640 1 -- 172.21.15.39:0/3826820419 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0018980 con 0x7fdbdc109510 2024-07-04T08:02:50.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.428+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc1ac840 con 0x7fdbdc109510 2024-07-04T08:02:50.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.430+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbbc0052c0 con 0x7fdbdc109510 2024-07-04T08:02:50.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.432+0000 7fdbda000640 1 -- 172.21.15.39:0/3826820419 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7fdbd0010430 con 0x7fdbdc109510 2024-07-04T08:02:50.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.433+0000 7fdbda000640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbc40b2830 0x7fdbc40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:02:50.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.434+0000 7fdbda000640 1 -- 172.21.15.39:0/3826820419 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdbd00e0760 con 0x7fdbdc109510 2024-07-04T08:02:50.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.434+0000 7fdbe1200640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbc40b2830 0x7fdbc40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:02:50.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.435+0000 7fdbe1200640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbc40b2830 0x7fdbc40b4d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005940 tx=0x7fdbc80058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:02:50.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.438+0000 7fdbda000640 1 -- 172.21.15.39:0/3826820419 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fdbd00a9990 con 0x7fdbdc109510 2024-07-04T08:02:50.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.656+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fdbbc0055b0 con 0x7fdbdc109510 2024-07-04T08:02:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.666+0000 7fdbda000640 1 -- 172.21.15.39:0/3826820419 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v35)=0 v35) ==== 117+0+0 (secure 0 0 0) 0x7fdbd00adc90 con 0x7fdbdc109510 2024-07-04T08:02:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbc40b2830 msgr2=0x7fdbc40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:50.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbc40b2830 0x7fdbc40b4d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005940 tx=0x7fdbc80058d0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc109510 msgr2=0x7fdbdc19fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc109510 0x7fdbdc19fe80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000e9f0 tx=0x7fdbd000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 shutdown_connections 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdbc40b2830 0x7fdbc40b4d40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbdc10ee50 0x7fdbdc1a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbdc10f7a0 0x7fdbdc1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 --2- 172.21.15.39:0/3826820419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbdc109510 0x7fdbdc19fe80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.669+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 >> 172.21.15.39:0/3826820419 conn(0x7fdbdc078b10 msgr2=0x7fdbdc10d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.670+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 shutdown_connections 2024-07-04T08:02:50.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:02:50.670+0000 7fdbe3000640 1 -- 172.21.15.39:0/3826820419 wait complete. 2024-07-04T08:02:51.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:50 smithi039 ceph-mon[32062]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:51.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:50 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' 2024-07-04T08:02:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:50 smithi050 ceph-mon[36577]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:51.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:50 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' 2024-07-04T08:02:51.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:50 smithi133 ceph-mon[37050]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:51.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:50 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' 2024-07-04T08:02:51.512 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi039') 2024-07-04T08:02:51.512 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi039') 2024-07-04T08:02:51.512 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi039') 2024-07-04T08:02:51.512 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi039') 2024-07-04T08:02:51.513 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi039') 2024-07-04T08:02:51.513 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi039') 2024-07-04T08:02:51.513 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi039') 2024-07-04T08:02:51.513 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi039') 2024-07-04T08:02:51.513 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.059 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.060 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.060 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.060 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.060 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.060 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.060 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '2ee561ca-39db-11ef-bcab-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi039') 2024-07-04T08:02:52.061 INFO:tasks.cephadm:Deploying OSDs... 2024-07-04T08:02:52.061 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:02:52.061 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-07-04T08:02:52.121 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-04T08:02:52.121 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-07-04T08:02:52.177 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-04T08:02:52.177 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:52.177 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-07-04T08:02:52.177 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:52.177 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:52.178 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-07-04 08:00:42.012360329 +0000 2024-07-04T08:02:52.178 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-07-04 07:55:14.874506243 +0000 2024-07-04T08:02:52.178 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-07-04 07:55:14.874506243 +0000 2024-07-04T08:02:52.178 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-07-04 07:55:14.874506243 +0000 2024-07-04T08:02:52.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-04T08:02:52.254 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-07-04T08:02:52.254 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-07-04T08:02:52.254 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000433332 s, 1.2 MB/s 2024-07-04T08:02:52.258 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-04T08:02:52.312 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-07-04T08:02:52.361 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-04T08:02:52.361 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:52.361 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-07-04T08:02:52.362 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:52.362 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:52.362 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-07-04 08:00:42.063359328 +0000 2024-07-04T08:02:52.362 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-07-04 07:55:15.493494222 +0000 2024-07-04T08:02:52.362 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-07-04 07:55:15.493494222 +0000 2024-07-04T08:02:52.362 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-07-04 07:55:15.493494222 +0000 2024-07-04T08:02:52.362 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-04T08:02:52.439 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-07-04T08:02:52.439 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-07-04T08:02:52.440 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000453186 s, 1.1 MB/s 2024-07-04T08:02:52.443 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-04T08:02:52.500 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-07-04T08:02:52.550 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-04T08:02:52.550 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:52.551 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-07-04T08:02:52.551 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:52.551 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:52.551 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-07-04 08:00:42.119358229 +0000 2024-07-04T08:02:52.551 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-07-04 07:55:16.146481540 +0000 2024-07-04T08:02:52.551 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-07-04 07:55:16.147481521 +0000 2024-07-04T08:02:52.551 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-07-04 07:55:16.146481540 +0000 2024-07-04T08:02:52.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-04T08:02:52.630 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-07-04T08:02:52.630 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-07-04T08:02:52.630 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000429746 s, 1.2 MB/s 2024-07-04T08:02:52.634 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-04T08:02:52.693 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-07-04T08:02:52.746 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-04T08:02:52.747 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-07-04T08:02:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-07-04 08:00:42.176357110 +0000 2024-07-04T08:02:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-07-04 07:55:16.786469111 +0000 2024-07-04T08:02:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-07-04 07:55:16.786469111 +0000 2024-07-04T08:02:52.747 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-07-04 07:55:16.786469111 +0000 2024-07-04T08:02:52.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-04T08:02:52.819 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-07-04T08:02:52.820 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-07-04T08:02:52.820 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000401348 s, 1.3 MB/s 2024-07-04T08:02:52.823 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-04T08:02:52.875 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-07-04T08:02:52.876 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-07-04T08:02:52.895 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-04T08:02:52.896 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-07-04T08:02:52.954 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-04T08:02:52.954 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:52.955 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-07-04T08:02:52.955 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:52.955 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:52.955 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-07-04 08:02:03.473122097 +0000 2024-07-04T08:02:52.955 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-07-04 07:53:51.574591186 +0000 2024-07-04T08:02:52.955 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-07-04 07:53:51.574591186 +0000 2024-07-04T08:02:52.955 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-07-04 07:53:51.574591186 +0000 2024-07-04T08:02:52.955 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-04T08:02:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-07-04T08:02:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-07-04T08:02:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000123992 s, 4.1 MB/s 2024-07-04T08:02:53.026 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-04T08:02:53.085 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-07-04T08:02:53.143 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-04T08:02:53.144 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:53.144 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-07-04T08:02:53.144 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:53.144 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:53.144 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-07-04 08:02:03.473122097 +0000 2024-07-04T08:02:53.144 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-07-04 07:53:51.893582671 +0000 2024-07-04T08:02:53.144 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-07-04 07:53:51.893582671 +0000 2024-07-04T08:02:53.144 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-07-04 07:53:51.893582671 +0000 2024-07-04T08:02:53.144 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-04T08:02:53.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:52 smithi050 ceph-mon[36577]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:53.209 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-07-04T08:02:53.209 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-07-04T08:02:53.209 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000126685 s, 4.0 MB/s 2024-07-04T08:02:53.210 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-04T08:02:53.268 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-07-04T08:02:53.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:52 smithi133 ceph-mon[37050]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:53.325 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-04T08:02:53.325 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:53.325 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-07-04T08:02:53.325 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:53.325 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:53.325 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-07-04 08:02:03.473122097 +0000 2024-07-04T08:02:53.325 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-07-04 07:53:52.215574076 +0000 2024-07-04T08:02:53.325 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-07-04 07:53:52.215574076 +0000 2024-07-04T08:02:53.326 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-07-04 07:53:52.215574076 +0000 2024-07-04T08:02:53.326 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-04T08:02:53.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:52 smithi039 ceph-mon[32062]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:53.390 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-07-04T08:02:53.390 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-07-04T08:02:53.390 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000117403 s, 4.4 MB/s 2024-07-04T08:02:53.391 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-04T08:02:53.449 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-07-04T08:02:53.506 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-04T08:02:53.506 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:53.506 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-07-04T08:02:53.506 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:53.506 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:53.506 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-07-04 08:02:03.474122071 +0000 2024-07-04T08:02:53.506 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-07-04 07:53:52.530565668 +0000 2024-07-04T08:02:53.506 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-07-04 07:53:52.530565668 +0000 2024-07-04T08:02:53.506 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-07-04 07:53:52.530565668 +0000 2024-07-04T08:02:53.507 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-04T08:02:53.570 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-07-04T08:02:53.570 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-07-04T08:02:53.570 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.00010872 s, 4.7 MB/s 2024-07-04T08:02:53.571 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-04T08:02:53.629 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-07-04T08:02:53.629 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-07-04T08:02:53.648 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-04T08:02:53.649 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-07-04T08:02:53.706 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-04T08:02:53.706 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:53.706 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-07-04T08:02:53.706 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:53.706 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:53.706 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-07-04 08:02:01.041576583 +0000 2024-07-04T08:02:53.706 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-07-04 07:53:55.102314024 +0000 2024-07-04T08:02:53.706 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-07-04 07:53:55.102314024 +0000 2024-07-04T08:02:53.706 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-07-04 07:53:55.102314024 +0000 2024-07-04T08:02:53.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-04T08:02:53.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-07-04T08:02:53.775 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-07-04T08:02:53.775 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000287116 s, 1.8 MB/s 2024-07-04T08:02:53.776 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-04T08:02:53.834 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-07-04T08:02:53.889 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-04T08:02:53.890 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:53.890 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-07-04T08:02:53.890 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:53.890 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:53.890 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-07-04 08:02:01.041576583 +0000 2024-07-04T08:02:53.890 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-07-04 07:53:55.431307762 +0000 2024-07-04T08:02:53.890 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-07-04 07:53:55.431307762 +0000 2024-07-04T08:02:53.890 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-07-04 07:53:55.431307762 +0000 2024-07-04T08:02:53.890 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-04T08:02:53.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-07-04T08:02:53.953 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-07-04T08:02:53.953 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000178137 s, 2.9 MB/s 2024-07-04T08:02:53.955 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-04T08:02:54.011 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-07-04T08:02:54.067 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-04T08:02:54.067 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:54.067 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-07-04T08:02:54.067 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:54.067 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:54.068 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-07-04 08:02:01.041576583 +0000 2024-07-04T08:02:54.068 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-07-04 07:53:55.765301404 +0000 2024-07-04T08:02:54.068 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-07-04 07:53:55.765301404 +0000 2024-07-04T08:02:54.068 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-07-04 07:53:55.765301404 +0000 2024-07-04T08:02:54.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-04T08:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-07-04T08:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-07-04T08:02:54.131 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000195012 s, 2.6 MB/s 2024-07-04T08:02:54.132 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-04T08:02:54.189 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-07-04T08:02:54.245 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-04T08:02:54.245 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-04T08:02:54.245 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-07-04T08:02:54.245 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:02:54.245 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-07-04T08:02:54.245 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-07-04 08:02:01.042576564 +0000 2024-07-04T08:02:54.245 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-07-04 07:53:56.095295122 +0000 2024-07-04T08:02:54.245 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-07-04 07:53:56.095295122 +0000 2024-07-04T08:02:54.246 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-07-04 07:53:56.095295122 +0000 2024-07-04T08:02:54.246 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-04T08:02:54.308 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-07-04T08:02:54.308 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-07-04T08:02:54.308 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000244062 s, 2.1 MB/s 2024-07-04T08:02:54.310 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-04T08:02:54.366 INFO:tasks.cephadm:Deploying osd.0 on smithi039 with /dev/vg_nvme/lv_4... 2024-07-04T08:02:54.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-04T08:02:54.835 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:55.185 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:54 smithi039 ceph-mon[32062]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:54 smithi133 ceph-mon[37050]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:55.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:54 smithi050 ceph-mon[36577]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:57.167 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-04T08:02:57.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:56 smithi133 ceph-mon[37050]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:57.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:56 smithi039 ceph-mon[32062]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:57.379 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-04T08:02:57.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:56 smithi050 ceph-mon[36577]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:57.423 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-07-04T08:02:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-07-04T08:02:57.423 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0368581 s, 284 MB/s 2024-07-04T08:02:57.423 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-07-04T08:02:58.189 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_4 2024-07-04T08:02:58.674 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:02:59.081 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:02:58 smithi039 ceph-mon[32062]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:59.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:02:58 smithi133 ceph-mon[37050]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:02:59.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:02:58 smithi050 ceph-mon[36577]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:00.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.682+0000 7f670c400640 1 -- 172.21.15.39:0/1488100066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67041030d0 msgr2=0x7f67041034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:00.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.682+0000 7f670c400640 1 --2- 172.21.15.39:0/1488100066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67041030d0 0x7f67041034e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009900 tx=0x7f66f802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:03:00.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.682+0000 7f670c400640 1 -- 172.21.15.39:0/1488100066 shutdown_connections 2024-07-04T08:03:00.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.682+0000 7f670c400640 1 --2- 172.21.15.39:0/1488100066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6704108a10 0x7f6704108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:00.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.682+0000 7f670c400640 1 --2- 172.21.15.39:0/1488100066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6704109360 0x7f670410fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:00.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.682+0000 7f670c400640 1 --2- 172.21.15.39:0/1488100066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67041030d0 0x7f67041034e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:00.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.682+0000 7f670c400640 1 -- 172.21.15.39:0/1488100066 >> 172.21.15.39:0/1488100066 conn(0x7f67040feb20 msgr2=0x7f6704100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:03:00.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.682+0000 7f670c400640 1 -- 172.21.15.39:0/1488100066 shutdown_connections 2024-07-04T08:03:00.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.682+0000 7f670c400640 1 -- 172.21.15.39:0/1488100066 wait complete. 2024-07-04T08:03:00.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.683+0000 7f670c400640 1 Processor -- start 2024-07-04T08:03:00.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.684+0000 7f670c400640 1 -- start start 2024-07-04T08:03:00.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.684+0000 7f670c400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67041030d0 0x7f6704074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:00.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.684+0000 7f670c400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6704108a10 0x7f67040751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:00.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.684+0000 7f670c400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6704109360 0x7f67040756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:00.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.684+0000 7f670c400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6704113010 con 0x7f6704108a10 2024-07-04T08:03:00.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.684+0000 7f670c400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6704112e90 con 0x7f67041030d0 2024-07-04T08:03:00.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.684+0000 7f670c400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6704113190 con 0x7f6704109360 2024-07-04T08:03:00.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f6709a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67041030d0 0x7f6704074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:00.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f670a400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6704109360 0x7f67040756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:00.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f6709a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67041030d0 0x7f6704074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60616/0 (socket says 172.21.15.39:60616) 2024-07-04T08:03:00.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f6709a00640 1 -- 172.21.15.39:0/3005687358 learned_addr learned my addr 172.21.15.39:0/3005687358 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:03:00.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f6709000640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6704108a10 0x7f67040751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:00.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f670a400640 1 -- 172.21.15.39:0/3005687358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67041030d0 msgr2=0x7f6704074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:00.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f670a400640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67041030d0 0x7f6704074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:00.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f670a400640 1 -- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6704108a10 msgr2=0x7f67040751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:00.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f670a400640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6704108a10 0x7f67040751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:00.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f670a400640 1 -- 172.21.15.39:0/3005687358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67041a7d20 con 0x7f6704109360 2024-07-04T08:03:00.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f6709000640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6704108a10 0x7f67040751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:03:00.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.685+0000 7f670a400640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6704109360 0x7f67040756e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009fd0 tx=0x7f66fc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:03:00.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.686+0000 7f6702000640 1 -- 172.21.15.39:0/3005687358 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc00ce20 con 0x7f6704109360 2024-07-04T08:03:00.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.686+0000 7f6702000640 1 -- 172.21.15.39:0/3005687358 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66fc005020 con 0x7f6704109360 2024-07-04T08:03:00.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.686+0000 7f6702000640 1 -- 172.21.15.39:0/3005687358 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc010690 con 0x7f6704109360 2024-07-04T08:03:00.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.686+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67041a7fb0 con 0x7f6704109360 2024-07-04T08:03:00.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.686+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67041a8470 con 0x7f6704109360 2024-07-04T08:03:00.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.687+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67041034e0 con 0x7f6704109360 2024-07-04T08:03:00.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.690+0000 7f6702000640 1 -- 172.21.15.39:0/3005687358 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f66fc0054d0 con 0x7f6704109360 2024-07-04T08:03:00.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.691+0000 7f6702000640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f66ec0b2830 0x7f66ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:00.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.691+0000 7f6702000640 1 -- 172.21.15.39:0/3005687358 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f66fc0e0ce0 con 0x7f6704109360 2024-07-04T08:03:00.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.691+0000 7f6709a00640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f66ec0b2830 0x7f66ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:00.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.692+0000 7f6709a00640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f66ec0b2830 0x7f66ec0b4d40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009230 tx=0x7f66f80058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:03:00.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.695+0000 7f6702000640 1 -- 172.21.15.39:0/3005687358 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f66fc0aa560 con 0x7f6704109360 2024-07-04T08:03:00.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:00.918+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6704062d50 con 0x7f66ec0b2830 2024-07-04T08:03:01.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:00 smithi133 ceph-mon[37050]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:01.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:00 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:03:01.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:00 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:03:01.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:00 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:01.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:00 smithi039 ceph-mon[32062]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:01.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:00 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:03:01.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:00 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:03:01.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:00 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:01.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:00 smithi050 ceph-mon[36577]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:01.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:00 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:03:01.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:00 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:03:01.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:00 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:02.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:01 smithi133 ceph-mon[37050]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:03:02.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:01 smithi039 ceph-mon[32062]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:03:02.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:01 smithi050 ceph-mon[36577]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:03:03.296 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:02 smithi039 ceph-mon[32062]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:02 smithi133 ceph-mon[37050]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:03.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:02 smithi050 ceph-mon[36577]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:05.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:04 smithi039 ceph-mon[32062]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:05.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:04 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1579442788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfc7b735-e66c-4ace-958a-f47b6df89f37"}]: dispatch 2024-07-04T08:03:05.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:04 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfc7b735-e66c-4ace-958a-f47b6df89f37"}]: dispatch 2024-07-04T08:03:05.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:04 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfc7b735-e66c-4ace-958a-f47b6df89f37"}]': finished 2024-07-04T08:03:05.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:04 smithi039 ceph-mon[32062]: osdmap e6: 1 total, 0 up, 1 in 2024-07-04T08:03:05.140 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:05.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:04 smithi133 ceph-mon[37050]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:05.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:04 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1579442788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfc7b735-e66c-4ace-958a-f47b6df89f37"}]: dispatch 2024-07-04T08:03:05.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:04 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfc7b735-e66c-4ace-958a-f47b6df89f37"}]: dispatch 2024-07-04T08:03:05.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:04 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfc7b735-e66c-4ace-958a-f47b6df89f37"}]': finished 2024-07-04T08:03:05.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:04 smithi133 ceph-mon[37050]: osdmap e6: 1 total, 0 up, 1 in 2024-07-04T08:03:05.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:05.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:04 smithi050 ceph-mon[36577]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:04 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1579442788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfc7b735-e66c-4ace-958a-f47b6df89f37"}]: dispatch 2024-07-04T08:03:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:04 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cfc7b735-e66c-4ace-958a-f47b6df89f37"}]: dispatch 2024-07-04T08:03:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:04 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cfc7b735-e66c-4ace-958a-f47b6df89f37"}]': finished 2024-07-04T08:03:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:04 smithi050 ceph-mon[36577]: osdmap e6: 1 total, 0 up, 1 in 2024-07-04T08:03:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:06.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:05 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2645042536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:03:06.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:05 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2645042536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:03:06.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:05 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2645042536' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:03:07.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:06 smithi133 ceph-mon[37050]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:07.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:06 smithi039 ceph-mon[32062]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:07.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:06 smithi050 ceph-mon[36577]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:08.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:07 smithi133 ceph-mon[37050]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:08.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:07 smithi039 ceph-mon[32062]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:08.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:07 smithi050 ceph-mon[36577]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:10.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:10 smithi039 ceph-mon[32062]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:10.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:10 smithi133 ceph-mon[37050]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:10.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:10 smithi050 ceph-mon[36577]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:12.802 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:12 smithi039 ceph-mon[32062]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:12.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:12 smithi133 ceph-mon[37050]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:12.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:12 smithi050 ceph-mon[36577]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:13.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:13 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-04T08:03:13.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:13 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:13.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:13 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-04T08:03:13.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:13 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:13.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:13 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-04T08:03:13.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:13 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:14.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:14 smithi039 ceph-mon[32062]: Deploying daemon osd.0 on smithi039 2024-07-04T08:03:14.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:14 smithi039 ceph-mon[32062]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:14 smithi133 ceph-mon[37050]: Deploying daemon osd.0 on smithi039 2024-07-04T08:03:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:14 smithi133 ceph-mon[37050]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:14 smithi050 ceph-mon[36577]: Deploying daemon osd.0 on smithi039 2024-07-04T08:03:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:14 smithi050 ceph-mon[36577]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:16.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:16 smithi039 ceph-mon[32062]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:16.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:16 smithi133 ceph-mon[37050]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:16 smithi050 ceph-mon[36577]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:18.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:18 smithi039 ceph-mon[32062]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:18.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:18 smithi133 ceph-mon[37050]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:18.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:18 smithi050 ceph-mon[36577]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:20.732 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:20 smithi039 ceph-mon[32062]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:20.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:20 smithi133 ceph-mon[37050]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:20.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:20 smithi050 ceph-mon[36577]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:22.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:22 smithi039 ceph-mon[32062]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:22.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:22 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:22.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:22 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:22.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:22 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:22.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:22 smithi133 ceph-mon[37050]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:22.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:22 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:22.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:22 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:22.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:22 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:22.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:22 smithi050 ceph-mon[36577]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:22.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:22 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:22.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:22 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:22.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:22 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:24.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:24 smithi133 ceph-mon[37050]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:24.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:24.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:24.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:24 smithi039 ceph-mon[32062]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:24.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:24.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:24.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:24 smithi050 ceph-mon[36577]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:24.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:24.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:25.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:25 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:25.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:25 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:25.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:25 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:25.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:25 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:25.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:25 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:25.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:25 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:25.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:25 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:25.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:25 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:25.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:25 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:26.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.464+0000 7f6702000640 1 -- 172.21.15.39:0/3005687358 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6704062d50 con 0x7f66ec0b2830 2024-07-04T08:03:26.467 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 0 on host 'smithi039' 2024-07-04T08:03:26.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.468+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f66ec0b2830 msgr2=0x7f66ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:26.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.468+0000 7f670c400640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f66ec0b2830 0x7f66ec0b4d40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009230 tx=0x7f66f80058d0 comp rx=0 tx=0).stop 2024-07-04T08:03:26.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.468+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6704109360 msgr2=0x7f67040756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:26.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.468+0000 7f670c400640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6704109360 0x7f67040756e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009fd0 tx=0x7f66fc00ef10 comp rx=0 tx=0).stop 2024-07-04T08:03:26.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.469+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 shutdown_connections 2024-07-04T08:03:26.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.469+0000 7f670c400640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f66ec0b2830 0x7f66ec0b4d40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:26.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.469+0000 7f670c400640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6704108a10 0x7f67040751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:26.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.469+0000 7f670c400640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6704109360 0x7f67040756e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:26.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.469+0000 7f670c400640 1 --2- 172.21.15.39:0/3005687358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67041030d0 0x7f6704074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:26.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.469+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 >> 172.21.15.39:0/3005687358 conn(0x7f67040feb20 msgr2=0x7f6704100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:03:26.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.469+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 shutdown_connections 2024-07-04T08:03:26.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:03:26.469+0000 7f670c400640 1 -- 172.21.15.39:0/3005687358 wait complete. 2024-07-04T08:03:26.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:26 smithi133 ceph-mon[37050]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:26.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:26 smithi133 ceph-mon[37050]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-04T08:03:26.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:26 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:26.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:26 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:26.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:26 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:26.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:26 smithi039 ceph-mon[32062]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:26.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:26 smithi039 ceph-mon[32062]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-04T08:03:26.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:26 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:26.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:26 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:26.826 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:26 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:26.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:26 smithi050 ceph-mon[36577]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:26.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:26 smithi050 ceph-mon[36577]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-04T08:03:26.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:26 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:26.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:26 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:26.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:26 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:27.216 DEBUG:teuthology.orchestra.run.smithi039:osd.0> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.0.service 2024-07-04T08:03:27.221 INFO:tasks.cephadm:Deploying osd.1 on smithi050 with /dev/vg_nvme/lv_4... 2024-07-04T08:03:27.222 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-04T08:03:27.489 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:03:27.796 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:27 smithi050 ceph-mon[36577]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-04T08:03:27.797 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:27 smithi050 ceph-mon[36577]: osdmap e7: 1 total, 0 up, 1 in 2024-07-04T08:03:27.797 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:27 smithi050 ceph-mon[36577]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:03:27.797 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:27.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:27 smithi133 ceph-mon[37050]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-04T08:03:27.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:27 smithi133 ceph-mon[37050]: osdmap e7: 1 total, 0 up, 1 in 2024-07-04T08:03:27.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:27 smithi133 ceph-mon[37050]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:03:27.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:27.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:27 smithi039 ceph-mon[32062]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-04T08:03:27.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:27 smithi039 ceph-mon[32062]: osdmap e7: 1 total, 0 up, 1 in 2024-07-04T08:03:27.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:27 smithi039 ceph-mon[32062]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:03:27.833 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:27.833 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:03:27 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0[51606]: 2024-07-04T08:03:27.567+0000 7fb353a00640 -1 osd.0 0 waiting for initial osdmap 2024-07-04T08:03:27.833 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:03:27 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0[51606]: 2024-07-04T08:03:27.578+0000 7fb34ea00640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:03:28.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:28 smithi050 ceph-mon[36577]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:28.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:28 smithi050 ceph-mon[36577]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:03:28.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:28 smithi050 ceph-mon[36577]: osdmap e8: 1 total, 0 up, 1 in 2024-07-04T08:03:28.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:28.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:28.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:28 smithi133 ceph-mon[37050]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:28.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:28 smithi133 ceph-mon[37050]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:03:28.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:28 smithi133 ceph-mon[37050]: osdmap e8: 1 total, 0 up, 1 in 2024-07-04T08:03:28.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:28.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:28.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:28 smithi039 ceph-mon[32062]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:28.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:28 smithi039 ceph-mon[32062]: from='osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:03:28.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:28 smithi039 ceph-mon[32062]: osdmap e8: 1 total, 0 up, 1 in 2024-07-04T08:03:28.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:28.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:29.115 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-04T08:03:29.204 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-04T08:03:29.215 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-07-04T08:03:29.215 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-07-04T08:03:29.215 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00834276 s, 1.3 GB/s 2024-07-04T08:03:29.215 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-07-04T08:03:29.802 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:29 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:03:29.802 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:29 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:03:29.803 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:29 smithi050 ceph-mon[36577]: osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] boot 2024-07-04T08:03:29.803 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:29 smithi050 ceph-mon[36577]: osdmap e9: 1 total, 1 up, 1 in 2024-07-04T08:03:29.803 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:29 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:29.830 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_4 2024-07-04T08:03:29.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:29 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:03:29.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:29 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:03:29.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:29 smithi039 ceph-mon[32062]: osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] boot 2024-07-04T08:03:29.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:29 smithi039 ceph-mon[32062]: osdmap e9: 1 total, 1 up, 1 in 2024-07-04T08:03:29.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:29 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:30.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:29 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:03:30.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:29 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:03:30.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:29 smithi133 ceph-mon[37050]: osd.0 [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] boot 2024-07-04T08:03:30.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:29 smithi133 ceph-mon[37050]: osdmap e9: 1 total, 1 up, 1 in 2024-07-04T08:03:30.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:29 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-04T08:03:30.090 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:03:30.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:30 smithi039 ceph-mon[32062]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:30.828 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:30 smithi039 ceph-mon[32062]: osdmap e10: 1 total, 1 up, 1 in 2024-07-04T08:03:30.857 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:30 smithi050 ceph-mon[36577]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:30.857 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:30 smithi050 ceph-mon[36577]: osdmap e10: 1 total, 1 up, 1 in 2024-07-04T08:03:31.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:30 smithi133 ceph-mon[37050]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-04T08:03:31.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:30 smithi133 ceph-mon[37050]: osdmap e10: 1 total, 1 up, 1 in 2024-07-04T08:03:31.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 -- 172.21.15.50:0/2008571096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c10d3a0 msgr2=0x7f8d4c10f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:31.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 --2- 172.21.15.50:0/2008571096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c10d3a0 0x7f8d4c10f880 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009ca0 tx=0x7f8d4402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:03:31.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 -- 172.21.15.50:0/2008571096 shutdown_connections 2024-07-04T08:03:31.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 --2- 172.21.15.50:0/2008571096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c10d3a0 0x7f8d4c10f880 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:31.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 --2- 172.21.15.50:0/2008571096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d4c104820 0x7f8d4c10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:31.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 --2- 172.21.15.50:0/2008571096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d4c103df0 0x7f8d4c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:31.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 -- 172.21.15.50:0/2008571096 >> 172.21.15.50:0/2008571096 conn(0x7f8d4c078cb0 msgr2=0x7f8d4c0790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:03:31.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 -- 172.21.15.50:0/2008571096 shutdown_connections 2024-07-04T08:03:31.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 -- 172.21.15.50:0/2008571096 wait complete. 2024-07-04T08:03:31.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.630+0000 7f8d52069640 1 Processor -- start 2024-07-04T08:03:31.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d52069640 1 -- start start 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d52069640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c103df0 0x7f8d4c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d52069640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d4c104820 0x7f8d4c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d52069640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d4c10d3a0 0x7f8d4c19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d52069640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c1130c0 con 0x7f8d4c10d3a0 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d52069640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c112f40 con 0x7f8d4c103df0 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d52069640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c113240 con 0x7f8d4c104820 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d4b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c103df0 0x7f8d4c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d4b7fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c103df0 0x7f8d4c199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:45502/0 (socket says 172.21.15.50:45502) 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d4b7fe640 1 -- 172.21.15.50:0/3984599572 learned_addr learned my addr 172.21.15.50:0/3984599572 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-07-04T08:03:31.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d4affd640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d4c104820 0x7f8d4c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d4b7fe640 1 -- 172.21.15.50:0/3984599572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d4c104820 msgr2=0x7f8d4c199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d4b7fe640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d4c104820 0x7f8d4c199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d4b7fe640 1 -- 172.21.15.50:0/3984599572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d4c10d3a0 msgr2=0x7f8d4c19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d4b7fe640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d4c10d3a0 0x7f8d4c19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.631+0000 7f8d4b7fe640 1 -- 172.21.15.50:0/3984599572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c19f980 con 0x7f8d4c103df0 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.632+0000 7f8d4b7fe640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c103df0 0x7f8d4c199a10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00b4a0 tx=0x7f8d3c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.632+0000 7f8d48ff9640 1 -- 172.21.15.50:0/3984599572 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c014070 con 0x7f8d4c103df0 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.632+0000 7f8d48ff9640 1 -- 172.21.15.50:0/3984599572 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d3c010ac0 con 0x7f8d4c103df0 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.632+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4c19fc70 con 0x7f8d4c103df0 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.632+0000 7f8d48ff9640 1 -- 172.21.15.50:0/3984599572 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c017c30 con 0x7f8d4c103df0 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.632+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4c1a63d0 con 0x7f8d4c103df0 2024-07-04T08:03:31.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.633+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4c06b710 con 0x7f8d4c103df0 2024-07-04T08:03:31.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.634+0000 7f8d48ff9640 1 -- 172.21.15.50:0/3984599572 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f8d3c017070 con 0x7f8d4c103df0 2024-07-04T08:03:31.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.635+0000 7f8d48ff9640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8d340b2830 0x7f8d340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:31.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.635+0000 7f8d48ff9640 1 -- 172.21.15.50:0/3984599572 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1982+0+0 (secure 0 0 0) 0x7f8d3c0e0030 con 0x7f8d4c103df0 2024-07-04T08:03:31.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.636+0000 7f8d4affd640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8d340b2830 0x7f8d340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:31.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.637+0000 7f8d4affd640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8d340b2830 0x7f8d340b4d40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c10f0e0 tx=0x7f8d4000b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:03:31.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.639+0000 7f8d48ff9640 1 -- 172.21.15.50:0/3984599572 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f8d3c0a9650 con 0x7f8d4c103df0 2024-07-04T08:03:31.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:31.843+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8d4c1064f0 con 0x7f8d340b2830 2024-07-04T08:03:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi039 2024-07-04T08:03:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:03:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi039 to 17089M 2024-07-04T08:03:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:32.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:32.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:03:32.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:03:32.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:03:32.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi039 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi039 to 17089M 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:03:33.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi039 2024-07-04T08:03:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi039 to 17089M 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:03:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:34.811 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:34 smithi050 ceph-mon[36577]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:34.812 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:34 smithi050 ceph-mon[36577]: from='client.? 172.21.15.50:0/564545476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de735138-774d-4816-b14f-4ad189b8b79d"}]: dispatch 2024-07-04T08:03:34.812 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:34 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de735138-774d-4816-b14f-4ad189b8b79d"}]: dispatch 2024-07-04T08:03:34.812 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:34 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de735138-774d-4816-b14f-4ad189b8b79d"}]': finished 2024-07-04T08:03:34.812 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:34 smithi050 ceph-mon[36577]: osdmap e11: 2 total, 1 up, 2 in 2024-07-04T08:03:34.812 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:34 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:35.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:34 smithi133 ceph-mon[37050]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:35.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:34 smithi133 ceph-mon[37050]: from='client.? 172.21.15.50:0/564545476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de735138-774d-4816-b14f-4ad189b8b79d"}]: dispatch 2024-07-04T08:03:35.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:34 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de735138-774d-4816-b14f-4ad189b8b79d"}]: dispatch 2024-07-04T08:03:35.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:34 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de735138-774d-4816-b14f-4ad189b8b79d"}]': finished 2024-07-04T08:03:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:34 smithi133 ceph-mon[37050]: osdmap e11: 2 total, 1 up, 2 in 2024-07-04T08:03:35.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:34 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:34 smithi039 ceph-mon[32062]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:34 smithi039 ceph-mon[32062]: from='client.? 172.21.15.50:0/564545476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de735138-774d-4816-b14f-4ad189b8b79d"}]: dispatch 2024-07-04T08:03:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:34 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de735138-774d-4816-b14f-4ad189b8b79d"}]: dispatch 2024-07-04T08:03:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:34 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de735138-774d-4816-b14f-4ad189b8b79d"}]': finished 2024-07-04T08:03:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:34 smithi039 ceph-mon[32062]: osdmap e11: 2 total, 1 up, 2 in 2024-07-04T08:03:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:34 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:35.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:35 smithi050 ceph-mon[36577]: from='client.? 172.21.15.50:0/2392307208' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:03:36.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:35 smithi133 ceph-mon[37050]: from='client.? 172.21.15.50:0/2392307208' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:03:36.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:35 smithi039 ceph-mon[32062]: from='client.? 172.21.15.50:0/2392307208' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:03:36.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:36 smithi050 ceph-mon[36577]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:37.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:36 smithi133 ceph-mon[37050]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:37.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:36 smithi039 ceph-mon[32062]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:38.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:38 smithi050 ceph-mon[36577]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:38 smithi133 ceph-mon[37050]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:39.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:38 smithi039 ceph-mon[32062]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:40.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:40 smithi050 ceph-mon[36577]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:40.903 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:40 smithi039 ceph-mon[32062]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:40 smithi133 ceph-mon[37050]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:41.810 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:41 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-04T08:03:41.811 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:41 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:41.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:41 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-04T08:03:41.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:41 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:42.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:41 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-04T08:03:42.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:41 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:42.939 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:42 smithi050 ceph-mon[36577]: Deploying daemon osd.1 on smithi050 2024-07-04T08:03:42.939 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:42 smithi050 ceph-mon[36577]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:42 smithi133 ceph-mon[37050]: Deploying daemon osd.1 on smithi050 2024-07-04T08:03:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:42 smithi133 ceph-mon[37050]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:43.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:42 smithi039 ceph-mon[32062]: Deploying daemon osd.1 on smithi050 2024-07-04T08:03:43.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:42 smithi039 ceph-mon[32062]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:44.696 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:44 smithi050 ceph-mon[36577]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:45.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:44 smithi133 ceph-mon[37050]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:45.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:44 smithi039 ceph-mon[32062]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:46.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:46 smithi050 ceph-mon[36577]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:47.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:46 smithi133 ceph-mon[37050]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:47.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:47.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:47.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:46 smithi039 ceph-mon[32062]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:49.015 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:48 smithi050 ceph-mon[36577]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:49.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:48 smithi133 ceph-mon[37050]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:49.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.072+0000 7f8d48ff9640 1 -- 172.21.15.50:0/3984599572 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8d4c1064f0 con 0x7f8d340b2830 2024-07-04T08:03:49.075 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 1 on host 'smithi050' 2024-07-04T08:03:49.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8d340b2830 msgr2=0x7f8d340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:49.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8d340b2830 0x7f8d340b4d40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c10f0e0 tx=0x7f8d4000b410 comp rx=0 tx=0).stop 2024-07-04T08:03:49.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c103df0 msgr2=0x7f8d4c199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:49.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c103df0 0x7f8d4c199a10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00b4a0 tx=0x7f8d3c00b960 comp rx=0 tx=0).stop 2024-07-04T08:03:49.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 shutdown_connections 2024-07-04T08:03:49.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8d340b2830 0x7f8d340b4d40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:49.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d4c10d3a0 0x7f8d4c19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:49.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d4c104820 0x7f8d4c199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:49.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 --2- 172.21.15.50:0/3984599572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d4c103df0 0x7f8d4c199a10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:49.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 >> 172.21.15.50:0/3984599572 conn(0x7f8d4c078cb0 msgr2=0x7f8d4c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:03:49.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 shutdown_connections 2024-07-04T08:03:49.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:03:49.075+0000 7f8d52069640 1 -- 172.21.15.50:0/3984599572 wait complete. 2024-07-04T08:03:49.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:48 smithi039 ceph-mon[32062]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:49.949 DEBUG:teuthology.orchestra.run.smithi050:osd.1> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.1.service 2024-07-04T08:03:49.952 INFO:tasks.cephadm:Deploying osd.2 on smithi133 with /dev/vg_nvme/lv_4... 2024-07-04T08:03:49.952 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-04T08:03:50.208 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:03:50.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:50 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:50.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:50 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:50.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:50 smithi133 ceph-mon[37050]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:50.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:50 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:50.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:50 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:50.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:50 smithi039 ceph-mon[32062]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:50.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:50 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:50.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:50 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:50.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:50 smithi050 ceph-mon[36577]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:50.395 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:03:50 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1[42438]: 2024-07-04T08:03:50.103+0000 7f859d8af640 -1 osd.1 0 log_to_monitors true 2024-07-04T08:03:51.256 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:51.256 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:51.256 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:51 smithi133 ceph-mon[37050]: from='osd.1 [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-04T08:03:51.256 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:51 smithi133 ceph-mon[37050]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-04T08:03:51.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:51.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:51.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:51 smithi039 ceph-mon[32062]: from='osd.1 [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-04T08:03:51.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:51 smithi039 ceph-mon[32062]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-04T08:03:51.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:51.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:51.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:51 smithi050 ceph-mon[36577]: from='osd.1 [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-04T08:03:51.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:51 smithi050 ceph-mon[36577]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-04T08:03:51.847 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-04T08:03:51.953 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-04T08:03:51.968 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-07-04T08:03:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-07-04T08:03:51.968 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128783 s, 814 MB/s 2024-07-04T08:03:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-07-04T08:03:52.394 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:03:52 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1[42438]: 2024-07-04T08:03:52.098+0000 7f8599831640 -1 osd.1 0 waiting for initial osdmap 2024-07-04T08:03:52.395 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:03:52 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1[42438]: 2024-07-04T08:03:52.101+0000 7f8595630640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:03:52.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:52 smithi050 ceph-mon[36577]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-04T08:03:52.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:52 smithi050 ceph-mon[36577]: from='osd.1 [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:03:52.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:52 smithi050 ceph-mon[36577]: osdmap e12: 2 total, 1 up, 2 in 2024-07-04T08:03:52.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:52.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:52 smithi050 ceph-mon[36577]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:03:52.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:52 smithi050 ceph-mon[36577]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:52.417 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:52 smithi133 ceph-mon[37050]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-04T08:03:52.418 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:52 smithi133 ceph-mon[37050]: from='osd.1 [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:03:52.418 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:52 smithi133 ceph-mon[37050]: osdmap e12: 2 total, 1 up, 2 in 2024-07-04T08:03:52.418 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:52.418 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:52 smithi133 ceph-mon[37050]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:03:52.418 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:52 smithi133 ceph-mon[37050]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:52.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:52 smithi039 ceph-mon[32062]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-04T08:03:52.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:52 smithi039 ceph-mon[32062]: from='osd.1 [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:03:52.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:52 smithi039 ceph-mon[32062]: osdmap e12: 2 total, 1 up, 2 in 2024-07-04T08:03:52.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:52.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:52 smithi039 ceph-mon[32062]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:03:52.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:52 smithi039 ceph-mon[32062]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:52.607 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2024-07-04T08:03:52.867 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:03:53.242 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:53 smithi133 ceph-mon[37050]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:03:53.242 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:53 smithi133 ceph-mon[37050]: osdmap e13: 2 total, 1 up, 2 in 2024-07-04T08:03:53.242 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:53.242 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:53.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:53 smithi039 ceph-mon[32062]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:03:53.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:53 smithi039 ceph-mon[32062]: osdmap e13: 2 total, 1 up, 2 in 2024-07-04T08:03:53.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:53.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:53.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:53 smithi050 ceph-mon[36577]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:03:53.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:53 smithi050 ceph-mon[36577]: osdmap e13: 2 total, 1 up, 2 in 2024-07-04T08:03:53.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:54.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 -- 172.21.15.133:0/4144622696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8101fc0 msgr2=0x7f92b81023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 --2- 172.21.15.133:0/4144622696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8101fc0 0x7f92b81023d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009900 tx=0x7f92a802a410 comp rx=0 tx=0).stop 2024-07-04T08:03:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 -- 172.21.15.133:0/4144622696 shutdown_connections 2024-07-04T08:03:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 --2- 172.21.15.133:0/4144622696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b8107900 0x7f92b8107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 --2- 172.21.15.133:0/4144622696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92b8108250 0x7f92b810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 --2- 172.21.15.133:0/4144622696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8101fc0 0x7f92b81023d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 -- 172.21.15.133:0/4144622696 >> 172.21.15.133:0/4144622696 conn(0x7f92b8078c60 msgr2=0x7f92b8079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:03:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 -- 172.21.15.133:0/4144622696 shutdown_connections 2024-07-04T08:03:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 -- 172.21.15.133:0/4144622696 wait complete. 2024-07-04T08:03:54.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 Processor -- start 2024-07-04T08:03:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.321+0000 7f92bd299640 1 -- start start 2024-07-04T08:03:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92bd299640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92b8101fc0 0x7f92b819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92bd299640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b8107900 0x7f92b819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92bd299640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8108250 0x7f92b819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92bd299640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8111fc0 con 0x7f92b8107900 2024-07-04T08:03:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92bd299640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8111e40 con 0x7f92b8101fc0 2024-07-04T08:03:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92bd299640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8112140 con 0x7f92b8108250 2024-07-04T08:03:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8108250 0x7f92b819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:54.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8108250 0x7f92b819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54540/0 (socket says 172.21.15.133:54540) 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b77fe640 1 -- 172.21.15.133:0/2717415390 learned_addr learned my addr 172.21.15.133:0/2717415390 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b6ffd640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92b8101fc0 0x7f92b819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b77fe640 1 -- 172.21.15.133:0/2717415390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92b8101fc0 msgr2=0x7f92b819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b77fe640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92b8101fc0 0x7f92b819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b77fe640 1 -- 172.21.15.133:0/2717415390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b8107900 msgr2=0x7f92b819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b77fe640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b8107900 0x7f92b819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b77fe640 1 -- 172.21.15.133:0/2717415390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b81a5ac0 con 0x7f92b8108250 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b77fe640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8108250 0x7f92b819b2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f92b000eaa0 tx=0x7f92b000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b4ff9640 1 -- 172.21.15.133:0/2717415390 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b000cdb0 con 0x7f92b8108250 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b4ff9640 1 -- 172.21.15.133:0/2717415390 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b0005020 con 0x7f92b8108250 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92b4ff9640 1 -- 172.21.15.133:0/2717415390 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b00116d0 con 0x7f92b8108250 2024-07-04T08:03:54.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b81a5db0 con 0x7f92b8108250 2024-07-04T08:03:54.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.322+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b81a62e0 con 0x7f92b8108250 2024-07-04T08:03:54.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.323+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b8103240 con 0x7f92b8108250 2024-07-04T08:03:54.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.326+0000 7f92b4ff9640 1 -- 172.21.15.133:0/2717415390 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f92b00051d0 con 0x7f92b8108250 2024-07-04T08:03:54.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.326+0000 7f92b4ff9640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92a40b27e0 0x7f92a40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:03:54.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.326+0000 7f92b4ff9640 1 -- 172.21.15.133:0/2717415390 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2524+0+0 (secure 0 0 0) 0x7f92b00e1030 con 0x7f92b8108250 2024-07-04T08:03:54.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.327+0000 7f92b6ffd640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92a40b27e0 0x7f92a40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:03:54.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.328+0000 7f92b6ffd640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92a40b27e0 0x7f92a40b4cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92a8006010 tx=0x7f92a8000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:03:54.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.330+0000 7f92b4ff9640 1 -- 172.21.15.133:0/2717415390 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f92b0016030 con 0x7f92b8108250 2024-07-04T08:03:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:03:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:03:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: osd.1 [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] boot 2024-07-04T08:03:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: osdmap e14: 2 total, 2 up, 2 in 2024-07-04T08:03:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi050 2024-07-04T08:03:54.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:03:54.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi050 to 17093M 2024-07-04T08:03:54.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:54.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:54.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:54.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:54.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:54.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:54.397 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:03:54.521+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f92b802de70 con 0x7f92a40b27e0 2024-07-04T08:03:54.522 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:03:54.522 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:03:54.522 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: osd.1 [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] boot 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: osdmap e14: 2 total, 2 up, 2 in 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi050 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi050 to 17093M 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:54.523 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:54.524 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:54.524 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:03:54.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:03:54.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: osd.1 [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] boot 2024-07-04T08:03:54.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: osdmap e14: 2 total, 2 up, 2 in 2024-07-04T08:03:54.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-04T08:03:54.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi050 2024-07-04T08:03:54.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:03:54.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi050 to 17093M 2024-07-04T08:03:54.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:54.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:54.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:54.604 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-04T08:03:54.604 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:03:54.604 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:54.604 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:03:54.604 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:03:55.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:55 smithi133 ceph-mon[37050]: osdmap e15: 2 total, 2 up, 2 in 2024-07-04T08:03:55.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:55 smithi133 ceph-mon[37050]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:03:55.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:55 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:03:55.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:55 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:03:55.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:55 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:55.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:55 smithi039 ceph-mon[32062]: osdmap e15: 2 total, 2 up, 2 in 2024-07-04T08:03:55.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:55 smithi039 ceph-mon[32062]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:03:55.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:55 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:03:55.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:55 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:03:55.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:55 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:55.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:55 smithi050 ceph-mon[36577]: osdmap e15: 2 total, 2 up, 2 in 2024-07-04T08:03:55.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:55 smithi050 ceph-mon[36577]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:03:55.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:55 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:03:55.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:55 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:03:55.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:55 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:03:56.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:56 smithi133 ceph-mon[37050]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:03:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:56 smithi039 ceph-mon[32062]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:03:56.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:56 smithi050 ceph-mon[36577]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:03:57.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:57 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/2835186581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad4f9bb6-fc12-476f-a359-b137919ed63d"}]: dispatch 2024-07-04T08:03:57.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:57 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad4f9bb6-fc12-476f-a359-b137919ed63d"}]: dispatch 2024-07-04T08:03:57.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:57 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad4f9bb6-fc12-476f-a359-b137919ed63d"}]': finished 2024-07-04T08:03:57.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:57 smithi133 ceph-mon[37050]: osdmap e16: 3 total, 2 up, 3 in 2024-07-04T08:03:57.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:03:57.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:57 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/2835186581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad4f9bb6-fc12-476f-a359-b137919ed63d"}]: dispatch 2024-07-04T08:03:57.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:57 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad4f9bb6-fc12-476f-a359-b137919ed63d"}]: dispatch 2024-07-04T08:03:57.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:57 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad4f9bb6-fc12-476f-a359-b137919ed63d"}]': finished 2024-07-04T08:03:57.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:57 smithi039 ceph-mon[32062]: osdmap e16: 3 total, 2 up, 3 in 2024-07-04T08:03:57.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:03:57.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:57 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/2835186581' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad4f9bb6-fc12-476f-a359-b137919ed63d"}]: dispatch 2024-07-04T08:03:57.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:57 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad4f9bb6-fc12-476f-a359-b137919ed63d"}]: dispatch 2024-07-04T08:03:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:57 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad4f9bb6-fc12-476f-a359-b137919ed63d"}]': finished 2024-07-04T08:03:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:57 smithi050 ceph-mon[36577]: osdmap e16: 3 total, 2 up, 3 in 2024-07-04T08:03:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:03:58.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:58 smithi133 ceph-mon[37050]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:03:58.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:03:58 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/342921077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:03:58.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:58 smithi039 ceph-mon[32062]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:03:58.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:03:58 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/342921077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:03:58.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:58 smithi050 ceph-mon[36577]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:03:58.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:03:58 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/342921077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:04:00.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:00 smithi133 ceph-mon[37050]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:00.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:00 smithi039 ceph-mon[32062]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:00.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:00 smithi050 ceph-mon[36577]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:02.693 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:02 smithi133 ceph-mon[37050]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:02.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:02 smithi039 ceph-mon[32062]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:02.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:02 smithi050 ceph-mon[36577]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:04.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:04 smithi133 ceph-mon[37050]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:04.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-04T08:04:04.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:04.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:04 smithi133 ceph-mon[37050]: Deploying daemon osd.2 on smithi133 2024-07-04T08:04:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:04 smithi039 ceph-mon[32062]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-04T08:04:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:04 smithi039 ceph-mon[32062]: Deploying daemon osd.2 on smithi133 2024-07-04T08:04:04.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:04 smithi050 ceph-mon[36577]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:04.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-04T08:04:04.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:04.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:04 smithi050 ceph-mon[36577]: Deploying daemon osd.2 on smithi133 2024-07-04T08:04:06.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:06 smithi133 ceph-mon[37050]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:06.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:06 smithi039 ceph-mon[32062]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:06.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:06 smithi050 ceph-mon[36577]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:08.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:08 smithi133 ceph-mon[37050]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:08.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:08 smithi039 ceph-mon[32062]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:08.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:08 smithi050 ceph-mon[36577]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:09.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:09.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:09.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:09.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:09.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:09.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:09.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:09.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:09.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:10.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:10 smithi039 ceph-mon[32062]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:10.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:10 smithi050 ceph-mon[36577]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:10.907 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:10 smithi133 ceph-mon[37050]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:11.885 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 2 on host 'smithi133' 2024-07-04T08:04:11.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.883+0000 7f92b4ff9640 1 -- 172.21.15.133:0/2717415390 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f92b802de70 con 0x7f92a40b27e0 2024-07-04T08:04:11.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92a40b27e0 msgr2=0x7f92a40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:11.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92a40b27e0 0x7f92a40b4cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f92a8006010 tx=0x7f92a8000bc0 comp rx=0 tx=0).stop 2024-07-04T08:04:11.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8108250 msgr2=0x7f92b819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:11.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8108250 0x7f92b819b2b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f92b000eaa0 tx=0x7f92b000ef60 comp rx=0 tx=0).stop 2024-07-04T08:04:11.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 shutdown_connections 2024-07-04T08:04:11.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92a40b27e0 0x7f92a40b4cf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:11.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b8107900 0x7f92b819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:11.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b8108250 0x7f92b819b2b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:11.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 --2- 172.21.15.133:0/2717415390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92b8101fc0 0x7f92b819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:11.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 >> 172.21.15.133:0/2717415390 conn(0x7f92b8078c60 msgr2=0x7f92b81059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:04:11.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 shutdown_connections 2024-07-04T08:04:11.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:04:11.886+0000 7f92bd299640 1 -- 172.21.15.133:0/2717415390 wait complete. 2024-07-04T08:04:12.189 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.189 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.189 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:12.190 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:12.190 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.190 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:12 smithi133 ceph-mon[37050]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:12.190 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:12.190 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.190 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:12 smithi039 ceph-mon[32062]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:12 smithi050 ceph-mon[36577]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:12.502 DEBUG:teuthology.orchestra.run.smithi133:osd.2> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.2.service 2024-07-04T08:04:12.504 INFO:tasks.cephadm:Deploying osd.3 on smithi039 with /dev/vg_nvme/lv_3... 2024-07-04T08:04:12.504 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-04T08:04:12.973 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:04:13.301 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:13 smithi039 ceph-mon[32062]: from='osd.2 [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-04T08:04:13.302 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:13 smithi039 ceph-mon[32062]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-04T08:04:13.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:13 smithi133 ceph-mon[37050]: from='osd.2 [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-04T08:04:13.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:13 smithi133 ceph-mon[37050]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-04T08:04:13.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:13 smithi050 ceph-mon[36577]: from='osd.2 [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-04T08:04:13.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:13 smithi050 ceph-mon[36577]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-04T08:04:14.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:14 smithi133 ceph-mon[37050]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-04T08:04:14.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:14 smithi133 ceph-mon[37050]: osdmap e17: 3 total, 2 up, 3 in 2024-07-04T08:04:14.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:14.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:14 smithi133 ceph-mon[37050]: from='osd.2 [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:04:14.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:14 smithi133 ceph-mon[37050]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:04:14.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:14 smithi133 ceph-mon[37050]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:14.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:14 smithi039 ceph-mon[32062]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-04T08:04:14.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:14 smithi039 ceph-mon[32062]: osdmap e17: 3 total, 2 up, 3 in 2024-07-04T08:04:14.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:14.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:14 smithi039 ceph-mon[32062]: from='osd.2 [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:04:14.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:14 smithi039 ceph-mon[32062]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:04:14.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:14 smithi039 ceph-mon[32062]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:14.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:14 smithi050 ceph-mon[36577]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-04T08:04:14.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:14 smithi050 ceph-mon[36577]: osdmap e17: 3 total, 2 up, 3 in 2024-07-04T08:04:14.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:14.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:14 smithi050 ceph-mon[36577]: from='osd.2 [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:04:14.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:14 smithi050 ceph-mon[36577]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:04:14.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:14 smithi050 ceph-mon[36577]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:14.816 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:14 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2[43380]: 2024-07-04T08:04:14.529+0000 7f646f42d640 -1 osd.2 0 waiting for initial osdmap 2024-07-04T08:04:14.817 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:14 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2[43380]: 2024-07-04T08:04:14.536+0000 7f6467224640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:04:15.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:15 smithi039 ceph-mon[32062]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:04:15.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:15 smithi039 ceph-mon[32062]: osdmap e18: 3 total, 2 up, 3 in 2024-07-04T08:04:15.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:15.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:15.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:15 smithi039 ceph-mon[32062]: from='osd.2 ' entity='osd.2' 2024-07-04T08:04:15.352 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:15 smithi133 ceph-mon[37050]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:04:15.353 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:15 smithi133 ceph-mon[37050]: osdmap e18: 3 total, 2 up, 3 in 2024-07-04T08:04:15.353 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:15.353 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:15.353 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:15 smithi133 ceph-mon[37050]: from='osd.2 ' entity='osd.2' 2024-07-04T08:04:15.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:15 smithi050 ceph-mon[36577]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:04:15.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:15 smithi050 ceph-mon[36577]: osdmap e18: 3 total, 2 up, 3 in 2024-07-04T08:04:15.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:15.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:15.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:15 smithi050 ceph-mon[36577]: from='osd.2 ' entity='osd.2' 2024-07-04T08:04:15.593 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-04T08:04:15.784 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-04T08:04:15.846 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-07-04T08:04:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-07-04T08:04:15.847 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0391732 s, 268 MB/s 2024-07-04T08:04:15.847 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-07-04T08:04:16.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi133 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi133 to 17088M 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:16.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:16.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: osd.2 [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] boot 2024-07-04T08:04:16.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: osdmap e19: 3 total, 3 up, 3 in 2024-07-04T08:04:16.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:16.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi133 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi133 to 17088M 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: osd.2 [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] boot 2024-07-04T08:04:16.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: osdmap e19: 3 total, 3 up, 3 in 2024-07-04T08:04:16.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:16.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi133 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi133 to 17088M 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:16.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:16.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: osd.2 [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] boot 2024-07-04T08:04:16.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: osdmap e19: 3 total, 3 up, 3 in 2024-07-04T08:04:16.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-04T08:04:16.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_3 2024-07-04T08:04:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:04:17.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:17 smithi039 ceph-mon[32062]: osdmap e20: 3 total, 3 up, 3 in 2024-07-04T08:04:17.815 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-04T08:04:17.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:17 smithi133 ceph-mon[37050]: osdmap e20: 3 total, 3 up, 3 in 2024-07-04T08:04:17.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-04T08:04:17.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:17 smithi050 ceph-mon[36577]: osdmap e20: 3 total, 3 up, 3 in 2024-07-04T08:04:17.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-04T08:04:18.794 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:18 smithi039 ceph-mon[32062]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:18.794 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-04T08:04:18.794 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:18 smithi039 ceph-mon[32062]: osdmap e21: 3 total, 3 up, 3 in 2024-07-04T08:04:18.794 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-04T08:04:18.795 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:04:18 smithi039 sudo[56215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-04T08:04:18.795 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:04:18 smithi039 sudo[56215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:18.795 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:04:18 smithi039 sudo[56215]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:18.795 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:04:18 smithi039 sudo[56215]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:18.795 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:04:18 smithi039 sudo[56218]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-04T08:04:18.795 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:04:18 smithi039 sudo[56218]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:18.795 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:04:18 smithi039 sudo[56218]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:18.795 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:04:18 smithi039 sudo[56218]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:18.865 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:18 smithi050 ceph-mon[36577]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:18.865 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-04T08:04:18.865 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:18 smithi050 ceph-mon[36577]: osdmap e21: 3 total, 3 up, 3 in 2024-07-04T08:04:18.865 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-04T08:04:18.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:18 smithi133 ceph-mon[37050]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:18.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-04T08:04:18.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:18 smithi133 ceph-mon[37050]: osdmap e21: 3 total, 3 up, 3 in 2024-07-04T08:04:18.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-04T08:04:19.145 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:04:18 smithi050 sudo[46560]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-04T08:04:19.145 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:04:18 smithi050 sudo[46560]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.145 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:04:18 smithi050 sudo[46560]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.145 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:04:18 smithi050 sudo[46560]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.145 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:04:18 smithi050 sudo[46563]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-04T08:04:19.145 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:04:18 smithi050 sudo[46563]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.145 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:04:18 smithi050 sudo[46563]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.145 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:04:18 smithi050 sudo[46563]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.316 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:18 smithi133 sudo[46987]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-04T08:04:19.316 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:18 smithi133 sudo[46987]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.316 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:18 smithi133 sudo[46987]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.317 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:18 smithi133 sudo[46987]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.317 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:18 smithi133 sudo[46990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-04T08:04:19.317 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:18 smithi133 sudo[46990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.317 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:18 smithi133 sudo[46990]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.317 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:04:18 smithi133 sudo[46990]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 sudo[46993]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-04T08:04:19.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 sudo[46993]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 sudo[46993]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 sudo[46993]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 sudo[46996]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-04T08:04:19.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 sudo[46996]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 sudo[46996]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 sudo[46996]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 sudo[56229]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-04T08:04:19.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 sudo[56229]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 sudo[56229]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 sudo[56229]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 sudo[56244]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-04T08:04:19.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 sudo[56244]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 sudo[56244]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 sudo[56244]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.577 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 sudo[46568]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-04T08:04:19.577 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 sudo[46568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.578 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 sudo[46568]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.578 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 sudo[46568]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.578 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 sudo[46572]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-04T08:04:19.578 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 sudo[46572]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-04T08:04:19.578 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 sudo[46572]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-04T08:04:19.578 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 sudo[46572]: pam_unix(sudo:session): session closed for user root 2024-07-04T08:04:19.621 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: osdmap e22: 3 total, 3 up, 3 in 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:04:19.622 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-04T08:04:19.623 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:04:19.623 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:04:19.623 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:04:19.623 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-04T08:04:19.623 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-04T08:04:19.623 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:04:19.623 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:04:19.623 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:04:19.623 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:19 smithi039 ceph-mon[32062]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-04T08:04:19.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.744+0000 7f9c7f200640 1 -- 172.21.15.39:0/1426148646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c801050f0 msgr2=0x7f9c80105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:19.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.744+0000 7f9c7f200640 1 --2- 172.21.15.39:0/1426148646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c801050f0 0x7f9c80105500 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009900 tx=0x7f9c7002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:04:19.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.744+0000 7f9c7f200640 1 -- 172.21.15.39:0/1426148646 shutdown_connections 2024-07-04T08:04:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.744+0000 7f9c7f200640 1 --2- 172.21.15.39:0/1426148646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c8010b380 0x7f9c80111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.744+0000 7f9c7f200640 1 --2- 172.21.15.39:0/1426148646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c8010aa30 0x7f9c8010ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.744+0000 7f9c7f200640 1 --2- 172.21.15.39:0/1426148646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c801050f0 0x7f9c80105500 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.744+0000 7f9c7f200640 1 -- 172.21.15.39:0/1426148646 >> 172.21.15.39:0/1426148646 conn(0x7f9c80100b60 msgr2=0x7f9c80102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:04:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.744+0000 7f9c7f200640 1 -- 172.21.15.39:0/1426148646 shutdown_connections 2024-07-04T08:04:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.744+0000 7f9c7f200640 1 -- 172.21.15.39:0/1426148646 wait complete. 2024-07-04T08:04:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.746+0000 7f9c7f200640 1 Processor -- start 2024-07-04T08:04:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.746+0000 7f9c7f200640 1 -- start start 2024-07-04T08:04:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.746+0000 7f9c7f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c801050f0 0x7f9c8019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:04:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.746+0000 7f9c7f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c8010aa30 0x7f9c8019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:04:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.746+0000 7f9c7f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8010b380 0x7f9c801a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:04:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.746+0000 7f9c7f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80115080 con 0x7f9c801050f0 2024-07-04T08:04:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.746+0000 7f9c7f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80114f00 con 0x7f9c8010b380 2024-07-04T08:04:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.746+0000 7f9c7f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9c80115200 con 0x7f9c8010aa30 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c7de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c801050f0 0x7f9c8019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c77e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c8010aa30 0x7f9c8019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c77e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c8010aa30 0x7f9c8019bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49668/0 (socket says 172.21.15.39:49668) 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c77e00640 1 -- 172.21.15.39:0/435920295 learned_addr learned my addr 172.21.15.39:0/435920295 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c7e800640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8010b380 0x7f9c801a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c77e00640 1 -- 172.21.15.39:0/435920295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8010b380 msgr2=0x7f9c801a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c77e00640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8010b380 0x7f9c801a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c77e00640 1 -- 172.21.15.39:0/435920295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c801050f0 msgr2=0x7f9c8019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c77e00640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c801050f0 0x7f9c8019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c77e00640 1 -- 172.21.15.39:0/435920295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c801a19c0 con 0x7f9c8010aa30 2024-07-04T08:04:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.747+0000 7f9c77e00640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c8010aa30 0x7f9c8019bff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e9f0 tx=0x7f9c6800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:04:19.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.748+0000 7f9c76a00640 1 -- 172.21.15.39:0/435920295 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800ccc0 con 0x7f9c8010aa30 2024-07-04T08:04:19.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.748+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c801a1cb0 con 0x7f9c8010aa30 2024-07-04T08:04:19.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.748+0000 7f9c76a00640 1 -- 172.21.15.39:0/435920295 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c680108e0 con 0x7f9c8010aa30 2024-07-04T08:04:19.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.748+0000 7f9c76a00640 1 -- 172.21.15.39:0/435920295 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68018860 con 0x7f9c8010aa30 2024-07-04T08:04:19.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.748+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c801a8410 con 0x7f9c8010aa30 2024-07-04T08:04:19.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.749+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5c0052c0 con 0x7f9c8010aa30 2024-07-04T08:04:19.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.751+0000 7f9c76a00640 1 -- 172.21.15.39:0/435920295 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 149613+0+0 (secure 0 0 0) 0x7f9c68010430 con 0x7f9c8010aa30 2024-07-04T08:04:19.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.753+0000 7f9c76a00640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9c640b27b0 0x7f9c640b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:04:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.753+0000 7f9c76a00640 1 -- 172.21.15.39:0/435920295 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3452+0+0 (secure 0 0 0) 0x7f9c680e0f90 con 0x7f9c8010aa30 2024-07-04T08:04:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.753+0000 7f9c7de00640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9c640b27b0 0x7f9c640b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:04:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.754+0000 7f9c7de00640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9c640b27b0 0x7f9c640b4cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c70005ab0 tx=0x7f9c70005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:04:19.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.757+0000 7f9c76a00640 1 -- 172.21.15.39:0/435920295 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f9c680a99a0 con 0x7f9c8010aa30 2024-07-04T08:04:19.836 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-04T08:04:19.836 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: osdmap e22: 3 total, 3 up, 3 in 2024-07-04T08:04:19.836 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:19.836 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:04:19.837 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:04:19.838 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:04:19.838 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:19 smithi133 ceph-mon[37050]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-04T08:04:19.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: osdmap e22: 3 total, 3 up, 3 in 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:04:19.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-04T08:04:19.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:04:19.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:04:19.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:04:19.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-04T08:04:19.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-04T08:04:19.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-04T08:04:19.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-04T08:04:19.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-04T08:04:19.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:19 smithi050 ceph-mon[36577]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-04T08:04:19.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:19.978+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9c5c000bf0 con 0x7f9c640b27b0 2024-07-04T08:04:20.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:20.032+0000 7f9c76a00640 1 -- 172.21.15.39:0/435920295 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f9c680a5c30 con 0x7f9c8010aa30 2024-07-04T08:04:20.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:20 smithi133 ceph-mon[37050]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:20.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:20 smithi133 ceph-mon[37050]: osdmap e23: 3 total, 3 up, 3 in 2024-07-04T08:04:20.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:20 smithi133 ceph-mon[37050]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:04:20.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:04:20.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:04:20.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:20.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:20 smithi133 ceph-mon[37050]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-07-04T08:04:20.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:20 smithi039 ceph-mon[32062]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:20.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:20 smithi039 ceph-mon[32062]: osdmap e23: 3 total, 3 up, 3 in 2024-07-04T08:04:20.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:20 smithi039 ceph-mon[32062]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:04:20.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:04:20.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:04:20.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:20.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:20 smithi039 ceph-mon[32062]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-07-04T08:04:20.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:20 smithi050 ceph-mon[36577]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:20.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:20 smithi050 ceph-mon[36577]: osdmap e23: 3 total, 3 up, 3 in 2024-07-04T08:04:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:20 smithi050 ceph-mon[36577]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:04:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:04:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:04:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:20 smithi050 ceph-mon[36577]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-07-04T08:04:22.741 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:22 smithi039 ceph-mon[32062]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:22.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:22 smithi050 ceph-mon[36577]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:23.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:22 smithi133 ceph-mon[37050]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:24.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:24 smithi039 ceph-mon[32062]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:24.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:24 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/4253305079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7accf662-69bc-4e7d-a7bf-c174c683367a"}]: dispatch 2024-07-04T08:04:24.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:24 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/4253305079' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7accf662-69bc-4e7d-a7bf-c174c683367a"}]': finished 2024-07-04T08:04:24.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:24 smithi039 ceph-mon[32062]: osdmap e24: 4 total, 3 up, 4 in 2024-07-04T08:04:24.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:24.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:24 smithi050 ceph-mon[36577]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:24.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:24 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/4253305079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7accf662-69bc-4e7d-a7bf-c174c683367a"}]: dispatch 2024-07-04T08:04:24.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:24 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/4253305079' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7accf662-69bc-4e7d-a7bf-c174c683367a"}]': finished 2024-07-04T08:04:24.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:24 smithi050 ceph-mon[36577]: osdmap e24: 4 total, 3 up, 4 in 2024-07-04T08:04:24.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:25.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:24 smithi133 ceph-mon[37050]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:25.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:24 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/4253305079' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7accf662-69bc-4e7d-a7bf-c174c683367a"}]: dispatch 2024-07-04T08:04:25.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:24 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/4253305079' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7accf662-69bc-4e7d-a7bf-c174c683367a"}]': finished 2024-07-04T08:04:25.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:24 smithi133 ceph-mon[37050]: osdmap e24: 4 total, 3 up, 4 in 2024-07-04T08:04:25.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:25.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:25 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2552915315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:04:26.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:25 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2552915315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:04:26.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:25 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2552915315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:04:26.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:26 smithi050 ceph-mon[36577]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-07-04T08:04:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:26 smithi133 ceph-mon[37050]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-07-04T08:04:27.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:26 smithi039 ceph-mon[32062]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-07-04T08:04:28.720 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:28 smithi039 ceph-mon[32062]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-07-04T08:04:28.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:28 smithi050 ceph-mon[36577]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-07-04T08:04:29.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:28 smithi133 ceph-mon[37050]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-07-04T08:04:30.707 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:30 smithi039 ceph-mon[32062]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 27 KiB/s wr, 2 op/s 2024-07-04T08:04:30.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:30 smithi050 ceph-mon[36577]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 27 KiB/s wr, 2 op/s 2024-07-04T08:04:31.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:30 smithi133 ceph-mon[37050]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 27 KiB/s wr, 2 op/s 2024-07-04T08:04:32.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:32 smithi039 ceph-mon[32062]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-07-04T08:04:32.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:32 smithi050 ceph-mon[36577]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-07-04T08:04:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:32 smithi133 ceph-mon[37050]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-07-04T08:04:33.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-04T08:04:33.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:34.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-04T08:04:34.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:34.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-04T08:04:34.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:34.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:34 smithi039 ceph-mon[32062]: Deploying daemon osd.3 on smithi039 2024-07-04T08:04:34.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:34 smithi039 ceph-mon[32062]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-07-04T08:04:35.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:34 smithi133 ceph-mon[37050]: Deploying daemon osd.3 on smithi039 2024-07-04T08:04:35.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:34 smithi133 ceph-mon[37050]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-07-04T08:04:35.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:34 smithi050 ceph-mon[36577]: Deploying daemon osd.3 on smithi039 2024-07-04T08:04:35.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:34 smithi050 ceph-mon[36577]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-07-04T08:04:37.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:36 smithi133 ceph-mon[37050]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 900 B/s rd, 23 KiB/s wr, 2 op/s 2024-07-04T08:04:37.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:36 smithi039 ceph-mon[32062]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 900 B/s rd, 23 KiB/s wr, 2 op/s 2024-07-04T08:04:37.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:36 smithi050 ceph-mon[36577]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 900 B/s rd, 23 KiB/s wr, 2 op/s 2024-07-04T08:04:38.936 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:38 smithi039 ceph-mon[32062]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:38 smithi133 ceph-mon[37050]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:39.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:38 smithi050 ceph-mon[36577]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:40.704 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:40 smithi039 ceph-mon[32062]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:40 smithi133 ceph-mon[37050]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:41.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:40 smithi050 ceph-mon[36577]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:42.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:42 smithi039 ceph-mon[32062]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:42.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:42.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:42.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:42 smithi133 ceph-mon[37050]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:43.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:42 smithi050 ceph-mon[36577]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:43.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:43.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:43.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:44.971 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:44 smithi039 ceph-mon[32062]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:45.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:44 smithi133 ceph-mon[37050]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:45.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:44 smithi050 ceph-mon[36577]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:46.518 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:46.653 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:46.653 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:46 smithi039 ceph-mon[32062]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:46.653 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:46.654 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:46.654 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:46.655 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:46.655 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:46 smithi050 ceph-mon[36577]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:46.655 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:46.655 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:46.655 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:46.655 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:46.655 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:46 smithi133 ceph-mon[37050]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:46.655 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:46.655 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:47.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:47 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:47.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:47 smithi039 ceph-mon[32062]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-04T08:04:47.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.490+0000 7f9c76a00640 1 -- 172.21.15.39:0/435920295 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9c5c000bf0 con 0x7f9c640b27b0 2024-07-04T08:04:47.493 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 3 on host 'smithi039' 2024-07-04T08:04:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9c640b27b0 msgr2=0x7f9c640b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9c640b27b0 0x7f9c640b4cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c70005ab0 tx=0x7f9c70005a20 comp rx=0 tx=0).stop 2024-07-04T08:04:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c8010aa30 msgr2=0x7f9c8019bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c8010aa30 0x7f9c8019bff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e9f0 tx=0x7f9c6800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:04:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 shutdown_connections 2024-07-04T08:04:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9c640b27b0 0x7f9c640b4cc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8010b380 0x7f9c801a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c8010aa30 0x7f9c8019bff0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:47.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 --2- 172.21.15.39:0/435920295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c801050f0 0x7f9c8019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:47.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 >> 172.21.15.39:0/435920295 conn(0x7f9c80100b60 msgr2=0x7f9c80102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:04:47.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 shutdown_connections 2024-07-04T08:04:47.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:04:47.494+0000 7f9c7f200640 1 -- 172.21.15.39:0/435920295 wait complete. 2024-07-04T08:04:47.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:47 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:47.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:47 smithi133 ceph-mon[37050]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-04T08:04:47.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:47 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:47.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:47 smithi050 ceph-mon[36577]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-04T08:04:48.415 DEBUG:teuthology.orchestra.run.smithi039:osd.3> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.3.service 2024-07-04T08:04:48.420 INFO:tasks.cephadm:Deploying osd.4 on smithi050 with /dev/vg_nvme/lv_3... 2024-07-04T08:04:48.420 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-04T08:04:48.474 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-mon[32062]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-04T08:04:48.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-mon[32062]: osdmap e25: 4 total, 3 up, 4 in 2024-07-04T08:04:48.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-mon[32062]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:04:48.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:48.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-mon[32062]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:48.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:48.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:48.475 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:48.546 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:48 smithi050 ceph-mon[36577]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-04T08:04:48.546 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:48 smithi050 ceph-mon[36577]: osdmap e25: 4 total, 3 up, 4 in 2024-07-04T08:04:48.546 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:48 smithi050 ceph-mon[36577]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:04:48.546 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:48.546 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:48 smithi050 ceph-mon[36577]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:48.546 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:48.547 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:48.547 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:48.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:48 smithi133 ceph-mon[37050]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-04T08:04:48.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:48 smithi133 ceph-mon[37050]: osdmap e25: 4 total, 3 up, 4 in 2024-07-04T08:04:48.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:48 smithi133 ceph-mon[37050]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:04:48.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:48.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:48 smithi133 ceph-mon[37050]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-07-04T08:04:48.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:04:48.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:48.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:48.690 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:04:49.101 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3[58520]: 2024-07-04T08:04:48.899+0000 7fa64c000640 -1 osd.3 0 waiting for initial osdmap 2024-07-04T08:04:49.101 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:04:48 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3[58520]: 2024-07-04T08:04:48.927+0000 7fa641600640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:04:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:49 smithi050 ceph-mon[36577]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:04:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:49 smithi050 ceph-mon[36577]: osdmap e26: 4 total, 3 up, 4 in 2024-07-04T08:04:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:49 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:49 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:49 smithi050 ceph-mon[36577]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' 2024-07-04T08:04:49.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:49 smithi133 ceph-mon[37050]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:04:49.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:49 smithi133 ceph-mon[37050]: osdmap e26: 4 total, 3 up, 4 in 2024-07-04T08:04:49.563 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:49 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:49.563 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:49 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:49.563 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:49 smithi133 ceph-mon[37050]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' 2024-07-04T08:04:49.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:49 smithi039 ceph-mon[32062]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:04:49.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:49 smithi039 ceph-mon[32062]: osdmap e26: 4 total, 3 up, 4 in 2024-07-04T08:04:49.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:49 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:49.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:49 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:49.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:49 smithi039 ceph-mon[32062]: from='osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793]' entity='osd.3' 2024-07-04T08:04:50.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:50 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:04:50.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:50 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:04:50.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:50 smithi039 ceph-mon[32062]: osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] boot 2024-07-04T08:04:50.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:50 smithi039 ceph-mon[32062]: osdmap e27: 4 total, 4 up, 4 in 2024-07-04T08:04:50.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:50 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:50.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:50 smithi039 ceph-mon[32062]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:04:50.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:50 smithi039 ceph-mon[32062]: osdmap e28: 4 total, 4 up, 4 in 2024-07-04T08:04:50.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:50 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:04:50.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:50 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:04:50.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:50 smithi133 ceph-mon[37050]: osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] boot 2024-07-04T08:04:50.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:50 smithi133 ceph-mon[37050]: osdmap e27: 4 total, 4 up, 4 in 2024-07-04T08:04:50.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:50 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:50.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:50 smithi133 ceph-mon[37050]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:04:50.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:50 smithi133 ceph-mon[37050]: osdmap e28: 4 total, 4 up, 4 in 2024-07-04T08:04:50.582 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-04T08:04:50.582 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:50 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:04:50.582 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:50 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:04:50.582 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:50 smithi050 ceph-mon[36577]: osd.3 [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] boot 2024-07-04T08:04:50.582 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:50 smithi050 ceph-mon[36577]: osdmap e27: 4 total, 4 up, 4 in 2024-07-04T08:04:50.582 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:50 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-04T08:04:50.582 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:50 smithi050 ceph-mon[36577]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:04:50.583 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:50 smithi050 ceph-mon[36577]: osdmap e28: 4 total, 4 up, 4 in 2024-07-04T08:04:50.674 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-04T08:04:50.684 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-07-04T08:04:50.684 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-07-04T08:04:50.685 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00844384 s, 1.2 GB/s 2024-07-04T08:04:50.685 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-07-04T08:04:51.380 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_3 2024-07-04T08:04:51.638 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:04:52.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:52 smithi050 ceph-mon[36577]: osdmap e29: 4 total, 4 up, 4 in 2024-07-04T08:04:52.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:52 smithi050 ceph-mon[36577]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:04:52.510 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:52 smithi039 ceph-mon[32062]: osdmap e29: 4 total, 4 up, 4 in 2024-07-04T08:04:52.510 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:52 smithi039 ceph-mon[32062]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:04:52.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:52 smithi133 ceph-mon[37050]: osdmap e29: 4 total, 4 up, 4 in 2024-07-04T08:04:52.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:52 smithi133 ceph-mon[37050]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:04:53.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.318+0000 7f9763cb4640 1 -- 172.21.15.50:0/4211323850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c1089f0 msgr2=0x7f975c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:53.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.318+0000 7f9763cb4640 1 --2- 172.21.15.50:0/4211323850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c1089f0 0x7f975c108e00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f974c009900 tx=0x7f974c02a410 comp rx=0 tx=0).stop 2024-07-04T08:04:53.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.318+0000 7f9763cb4640 1 -- 172.21.15.50:0/4211323850 shutdown_connections 2024-07-04T08:04:53.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.318+0000 7f9763cb4640 1 --2- 172.21.15.50:0/4211323850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f975c109340 0x7f975c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:53.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.318+0000 7f9763cb4640 1 --2- 172.21.15.50:0/4211323850 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c1089f0 0x7f975c108e00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:53.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.318+0000 7f9763cb4640 1 --2- 172.21.15.50:0/4211323850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f975c1030b0 0x7f975c1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:53.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.318+0000 7f9763cb4640 1 -- 172.21.15.50:0/4211323850 >> 172.21.15.50:0/4211323850 conn(0x7f975c07ada0 msgr2=0x7f975c07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:04:53.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.318+0000 7f9763cb4640 1 -- 172.21.15.50:0/4211323850 shutdown_connections 2024-07-04T08:04:53.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.318+0000 7f9763cb4640 1 -- 172.21.15.50:0/4211323850 wait complete. 2024-07-04T08:04:53.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9763cb4640 1 Processor -- start 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9763cb4640 1 -- start start 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9763cb4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f975c1030b0 0x7f975c118710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9763cb4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f975c1089f0 0x7f975c1129d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9763cb4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c109340 0x7f975c112f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9763cb4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f975c110e60 con 0x7f975c1030b0 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9763cb4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f975c110ce0 con 0x7f975c109340 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9763cb4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f975c110fe0 con 0x7f975c1089f0 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9762251640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c109340 0x7f975c112f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9762251640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c109340 0x7f975c112f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:34276/0 (socket says 172.21.15.50:34276) 2024-07-04T08:04:53.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.319+0000 7f9762251640 1 -- 172.21.15.50:0/1126261744 learned_addr learned my addr 172.21.15.50:0/1126261744 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f976124f640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f975c1089f0 0x7f975c1129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9761a50640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f975c1030b0 0x7f975c118710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9762251640 1 -- 172.21.15.50:0/1126261744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f975c1089f0 msgr2=0x7f975c1129d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9762251640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f975c1089f0 0x7f975c1129d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9762251640 1 -- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f975c1030b0 msgr2=0x7f975c118710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9762251640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f975c1030b0 0x7f975c118710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9762251640 1 -- 172.21.15.50:0/1126261744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c113580 con 0x7f975c109340 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9762251640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c109340 0x7f975c112f10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f975400eaa0 tx=0x7f975400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9752ffd640 1 -- 172.21.15.50:0/1126261744 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975400cdb0 con 0x7f975c109340 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9752ffd640 1 -- 172.21.15.50:0/1126261744 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754005020 con 0x7f975c109340 2024-07-04T08:04:53.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c1b4dd0 con 0x7f975c109340 2024-07-04T08:04:53.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9752ffd640 1 -- 172.21.15.50:0/1126261744 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97540116d0 con 0x7f975c109340 2024-07-04T08:04:53.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.320+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c1b5170 con 0x7f975c109340 2024-07-04T08:04:53.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.321+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975c1034c0 con 0x7f975c109340 2024-07-04T08:04:53.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.323+0000 7f9752ffd640 1 -- 172.21.15.50:0/1126261744 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f97540051d0 con 0x7f975c109340 2024-07-04T08:04:53.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.324+0000 7f9752ffd640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f97480b28d0 0x7f97480b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:04:53.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.324+0000 7f9752ffd640 1 -- 172.21.15.50:0/1126261744 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3884+0+0 (secure 0 0 0) 0x7f97540e15e0 con 0x7f975c109340 2024-07-04T08:04:53.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.324+0000 7f9761a50640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f97480b28d0 0x7f97480b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:04:53.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.325+0000 7f9761a50640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f97480b28d0 0x7f97480b4de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f975c118460 tx=0x7f9758008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:04:53.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.327+0000 7f9752ffd640 1 -- 172.21.15.50:0/1126261744 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f9754016030 con 0x7f975c109340 2024-07-04T08:04:53.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:04:53.532+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f975c062d50 con 0x7f97480b28d0 2024-07-04T08:04:54.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:04:54.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi039 2024-07-04T08:04:54.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:04:54.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:04:54.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi039 to 8544M 2024-07-04T08:04:54.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:54.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:54.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:04:54.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:04:54.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:04:54.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:54.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:04:54.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi039 2024-07-04T08:04:54.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi039 to 8544M 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:04:54.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:04:54.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:04:54.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:54.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:04:54.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi039 2024-07-04T08:04:54.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:04:54.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:04:54.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi039 to 8544M 2024-07-04T08:04:54.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:54.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:04:54.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:04:54.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:04:54.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:04:54.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:04:54.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:04:56.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:56 smithi050 ceph-mon[36577]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-07-04T08:04:56.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:56 smithi050 ceph-mon[36577]: from='client.? 172.21.15.50:0/3573991433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db0ba72-e313-4a94-a5d0-ff30e4e8028f"}]: dispatch 2024-07-04T08:04:56.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:56 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db0ba72-e313-4a94-a5d0-ff30e4e8028f"}]: dispatch 2024-07-04T08:04:56.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:56 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9db0ba72-e313-4a94-a5d0-ff30e4e8028f"}]': finished 2024-07-04T08:04:56.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:56 smithi050 ceph-mon[36577]: osdmap e30: 5 total, 4 up, 5 in 2024-07-04T08:04:56.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:56 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:04:56.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:56 smithi133 ceph-mon[37050]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-07-04T08:04:56.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:56 smithi133 ceph-mon[37050]: from='client.? 172.21.15.50:0/3573991433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db0ba72-e313-4a94-a5d0-ff30e4e8028f"}]: dispatch 2024-07-04T08:04:56.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:56 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db0ba72-e313-4a94-a5d0-ff30e4e8028f"}]: dispatch 2024-07-04T08:04:56.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:56 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9db0ba72-e313-4a94-a5d0-ff30e4e8028f"}]': finished 2024-07-04T08:04:56.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:56 smithi133 ceph-mon[37050]: osdmap e30: 5 total, 4 up, 5 in 2024-07-04T08:04:56.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:56 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:04:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:56 smithi039 ceph-mon[32062]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-07-04T08:04:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:56 smithi039 ceph-mon[32062]: from='client.? 172.21.15.50:0/3573991433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db0ba72-e313-4a94-a5d0-ff30e4e8028f"}]: dispatch 2024-07-04T08:04:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:56 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db0ba72-e313-4a94-a5d0-ff30e4e8028f"}]: dispatch 2024-07-04T08:04:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:56 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9db0ba72-e313-4a94-a5d0-ff30e4e8028f"}]': finished 2024-07-04T08:04:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:56 smithi039 ceph-mon[32062]: osdmap e30: 5 total, 4 up, 5 in 2024-07-04T08:04:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:56 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:04:57.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:57 smithi133 ceph-mon[37050]: from='client.? 172.21.15.50:0/2878528318' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:04:57.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:57 smithi039 ceph-mon[32062]: from='client.? 172.21.15.50:0/2878528318' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:04:57.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:57 smithi050 ceph-mon[36577]: from='client.? 172.21.15.50:0/2878528318' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:04:58.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:04:58 smithi133 ceph-mon[37050]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-07-04T08:04:58.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:04:58 smithi039 ceph-mon[32062]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-07-04T08:04:58.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:04:58 smithi050 ceph-mon[36577]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-07-04T08:05:00.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:00 smithi050 ceph-mon[36577]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-04T08:05:00.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:00 smithi133 ceph-mon[37050]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-04T08:05:00.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:00 smithi039 ceph-mon[32062]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-04T08:05:02.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:02 smithi133 ceph-mon[37050]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:02.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:02 smithi039 ceph-mon[32062]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:02.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:02 smithi050 ceph-mon[36577]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:03.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-04T08:05:03.762 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:03.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-04T08:05:03.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:03.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-04T08:05:03.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:04.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:04 smithi050 ceph-mon[36577]: Deploying daemon osd.4 on smithi050 2024-07-04T08:05:04.776 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:04 smithi050 ceph-mon[36577]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:04.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:04 smithi133 ceph-mon[37050]: Deploying daemon osd.4 on smithi050 2024-07-04T08:05:04.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:04 smithi133 ceph-mon[37050]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:04 smithi039 ceph-mon[32062]: Deploying daemon osd.4 on smithi050 2024-07-04T08:05:04.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:04 smithi039 ceph-mon[32062]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:06.661 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:06 smithi050 ceph-mon[36577]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:06.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:06 smithi133 ceph-mon[37050]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:06.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:06 smithi039 ceph-mon[32062]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:08.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:08 smithi133 ceph-mon[37050]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:08.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:08.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:08.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:08.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:08 smithi039 ceph-mon[32062]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:08.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:08.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:08.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:08.867 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:08 smithi050 ceph-mon[36577]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:08.867 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:08.867 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:08.867 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:10.725 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:10 smithi050 ceph-mon[36577]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:10.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:10 smithi133 ceph-mon[37050]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:10.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:10 smithi039 ceph-mon[32062]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:11.700 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 4 on host 'smithi050' 2024-07-04T08:05:11.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.699+0000 7f9752ffd640 1 -- 172.21.15.50:0/1126261744 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f975c062d50 con 0x7f97480b28d0 2024-07-04T08:05:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f97480b28d0 msgr2=0x7f97480b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f97480b28d0 0x7f97480b4de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f975c118460 tx=0x7f9758008040 comp rx=0 tx=0).stop 2024-07-04T08:05:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c109340 msgr2=0x7f975c112f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c109340 0x7f975c112f10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f975400eaa0 tx=0x7f975400ef60 comp rx=0 tx=0).stop 2024-07-04T08:05:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 shutdown_connections 2024-07-04T08:05:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f97480b28d0 0x7f97480b4de0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f975c109340 0x7f975c112f10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:11.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f975c1089f0 0x7f975c1129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:11.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 --2- 172.21.15.50:0/1126261744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f975c1030b0 0x7f975c118710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:11.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 >> 172.21.15.50:0/1126261744 conn(0x7f975c07ada0 msgr2=0x7f975c106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:05:11.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 shutdown_connections 2024-07-04T08:05:11.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:05:11.701+0000 7f9763cb4640 1 -- 172.21.15.50:0/1126261744 wait complete. 2024-07-04T08:05:12.358 DEBUG:teuthology.orchestra.run.smithi050:osd.4> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.4.service 2024-07-04T08:05:12.361 INFO:tasks.cephadm:Deploying osd.5 on smithi133 with /dev/vg_nvme/lv_3... 2024-07-04T08:05:12.361 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-04T08:05:12.620 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:05:12.967 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:12 smithi133 ceph-mon[37050]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:12.967 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:12.967 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:12.967 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:12 smithi133 ceph-mon[37050]: from='osd.4 [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-04T08:05:12.967 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:12 smithi133 ceph-mon[37050]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-04T08:05:12.967 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:12.968 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:13.034 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:12 smithi050 ceph-mon[36577]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:13.034 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:13.034 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:13.034 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:12 smithi050 ceph-mon[36577]: from='osd.4 [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-04T08:05:13.034 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:12 smithi050 ceph-mon[36577]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-04T08:05:13.034 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:13.034 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:12 smithi039 ceph-mon[32062]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:12 smithi039 ceph-mon[32062]: from='osd.4 [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-04T08:05:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:12 smithi039 ceph-mon[32062]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-04T08:05:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:13.931 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:13 smithi133 ceph-mon[37050]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-04T08:05:13.931 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:13 smithi133 ceph-mon[37050]: from='osd.4 [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:05:13.931 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:13 smithi133 ceph-mon[37050]: osdmap e31: 5 total, 4 up, 5 in 2024-07-04T08:05:13.931 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:13 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:13.931 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:13 smithi133 ceph-mon[37050]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:05:14.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:13 smithi039 ceph-mon[32062]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-04T08:05:14.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:13 smithi039 ceph-mon[32062]: from='osd.4 [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:05:14.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:13 smithi039 ceph-mon[32062]: osdmap e31: 5 total, 4 up, 5 in 2024-07-04T08:05:14.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:13 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:14.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:13 smithi039 ceph-mon[32062]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:05:14.115 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:13 smithi050 ceph-mon[36577]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-04T08:05:14.115 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:13 smithi050 ceph-mon[36577]: from='osd.4 [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:05:14.115 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:13 smithi050 ceph-mon[36577]: osdmap e31: 5 total, 4 up, 5 in 2024-07-04T08:05:14.115 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:13 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:14.115 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:13 smithi050 ceph-mon[36577]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:05:14.116 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:05:13 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4[49345]: 2024-07-04T08:05:13.709+0000 7f1b68520640 -1 osd.4 0 waiting for initial osdmap 2024-07-04T08:05:14.116 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:05:13 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4[49345]: 2024-07-04T08:05:13.717+0000 7f1b6431f640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:05:14.292 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-04T08:05:14.377 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-04T08:05:14.391 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-07-04T08:05:14.392 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-07-04T08:05:14.392 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125762 s, 834 MB/s 2024-07-04T08:05:14.392 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-07-04T08:05:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:14 smithi133 ceph-mon[37050]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:14 smithi133 ceph-mon[37050]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:05:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:14 smithi133 ceph-mon[37050]: osdmap e32: 5 total, 4 up, 5 in 2024-07-04T08:05:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:14.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:14 smithi050 ceph-mon[36577]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:14.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:14 smithi050 ceph-mon[36577]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:05:14.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:14 smithi050 ceph-mon[36577]: osdmap e32: 5 total, 4 up, 5 in 2024-07-04T08:05:14.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:14.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:15.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2024-07-04T08:05:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:14 smithi039 ceph-mon[32062]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:14 smithi039 ceph-mon[32062]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:05:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:14 smithi039 ceph-mon[32062]: osdmap e32: 5 total, 4 up, 5 in 2024-07-04T08:05:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:15.343 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: osd.4 [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] boot 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: osdmap e33: 5 total, 5 up, 5 in 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:15.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:15.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:16.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:05:16.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:05:16.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: osd.4 [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] boot 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: osdmap e33: 5 total, 5 up, 5 in 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:16.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:05:16.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:05:16.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: osd.4 [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] boot 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: osdmap e33: 5 total, 5 up, 5 in 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:16.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:16.796 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:16 smithi133 ceph-mon[37050]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:16.796 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:16 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi050 2024-07-04T08:05:16.797 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:16 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi050 to 8546M 2024-07-04T08:05:16.797 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:16 smithi133 ceph-mon[37050]: osdmap e34: 5 total, 5 up, 5 in 2024-07-04T08:05:16.797 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:16.797 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:16.797 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:16.798 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:16 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:17.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 -- 172.21.15.133:0/3450313035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4101ff0 msgr2=0x7f1ba4102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:17.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 --2- 172.21.15.133:0/3450313035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4101ff0 0x7f1ba4102400 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009900 tx=0x7f1b9402a410 comp rx=0 tx=0).stop 2024-07-04T08:05:17.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 -- 172.21.15.133:0/3450313035 shutdown_connections 2024-07-04T08:05:17.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 --2- 172.21.15.133:0/3450313035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4107930 0x7f1ba4107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:17.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 --2- 172.21.15.133:0/3450313035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba4108280 0x7f1ba410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:17.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 --2- 172.21.15.133:0/3450313035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4101ff0 0x7f1ba4102400 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:17.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 -- 172.21.15.133:0/3450313035 >> 172.21.15.133:0/3450313035 conn(0x7f1ba4078ca0 msgr2=0x7f1ba40790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:05:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 -- 172.21.15.133:0/3450313035 shutdown_connections 2024-07-04T08:05:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 -- 172.21.15.133:0/3450313035 wait complete. 2024-07-04T08:05:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.011+0000 7f1ba8953640 1 Processor -- start 2024-07-04T08:05:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba8953640 1 -- start start 2024-07-04T08:05:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba8953640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba4101ff0 0x7f1ba4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:05:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba8953640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4107930 0x7f1ba4197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:05:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba8953640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4108280 0x7f1ba419d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:05:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba8953640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4110ea0 con 0x7f1ba4107930 2024-07-04T08:05:17.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba8953640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4110d20 con 0x7f1ba4101ff0 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba8953640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4111020 con 0x7f1ba4108280 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4108280 0x7f1ba419d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4108280 0x7f1ba419d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58966/0 (socket says 172.21.15.133:58966) 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba2d9d640 1 -- 172.21.15.133:0/945011743 learned_addr learned my addr 172.21.15.133:0/945011743 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba259c640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba4101ff0 0x7f1ba4197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba2d9d640 1 -- 172.21.15.133:0/945011743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba4101ff0 msgr2=0x7f1ba4197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba2d9d640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba4101ff0 0x7f1ba4197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba2d9d640 1 -- 172.21.15.133:0/945011743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4107930 msgr2=0x7f1ba4197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba2d9d640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4107930 0x7f1ba4197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:17.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba2d9d640 1 -- 172.21.15.133:0/945011743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba419d7a0 con 0x7f1ba4108280 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.012+0000 7f1ba2d9d640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4108280 0x7f1ba419d070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00eaa0 tx=0x7f1b9c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.013+0000 7f1b9b7fe640 1 -- 172.21.15.133:0/945011743 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c00cdb0 con 0x7f1ba4108280 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.013+0000 7f1b9b7fe640 1 -- 172.21.15.133:0/945011743 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b9c005020 con 0x7f1ba4108280 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.013+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba419da90 con 0x7f1ba4108280 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.013+0000 7f1b9b7fe640 1 -- 172.21.15.133:0/945011743 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0116d0 con 0x7f1ba4108280 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.013+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba41a4270 con 0x7f1ba4108280 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.014+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba4102400 con 0x7f1ba4108280 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.015+0000 7f1b9b7fe640 1 -- 172.21.15.133:0/945011743 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f1b9c004b20 con 0x7f1ba4108280 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.016+0000 7f1b9b7fe640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b900b4ad0 0x7f1b900b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.016+0000 7f1b9b7fe640 1 -- 172.21.15.133:0/945011743 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4316+0+0 (secure 0 0 0) 0x7f1b9c0e13f0 con 0x7f1ba4108280 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.016+0000 7f1ba259c640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b900b4ad0 0x7f1b900b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:05:17.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.017+0000 7f1ba259c640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b900b4ad0 0x7f1b900b6fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b94031440 tx=0x7f1b94009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:05:17.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.020+0000 7f1b9b7fe640 1 -- 172.21.15.133:0/945011743 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f1b9c016030 con 0x7f1ba4108280 2024-07-04T08:05:17.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:16 smithi039 ceph-mon[32062]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:17.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:16 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi050 2024-07-04T08:05:17.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:16 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi050 to 8546M 2024-07-04T08:05:17.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:16 smithi039 ceph-mon[32062]: osdmap e34: 5 total, 5 up, 5 in 2024-07-04T08:05:17.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:17.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:17.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:17.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:16 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:17.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:16 smithi050 ceph-mon[36577]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-07-04T08:05:17.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:16 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi050 2024-07-04T08:05:17.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:16 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi050 to 8546M 2024-07-04T08:05:17.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:16 smithi050 ceph-mon[36577]: osdmap e34: 5 total, 5 up, 5 in 2024-07-04T08:05:17.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:17.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:17.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:17.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:16 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:17.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:17.211+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1ba4062d50 con 0x7f1b900b4ad0 2024-07-04T08:05:17.990 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:05:17.990 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:05:17.990 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:18.046 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:05:18.046 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:05:18.046 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:18.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:05:18.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:05:18.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:18.989 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:18 smithi133 ceph-mon[37050]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:05:18.989 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:18 smithi133 ceph-mon[37050]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:19.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:18 smithi039 ceph-mon[32062]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:05:19.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:18 smithi039 ceph-mon[32062]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:19.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:18 smithi050 ceph-mon[36577]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:05:19.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:18 smithi050 ceph-mon[36577]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:20.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:19 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/917082078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a0b6a8f-16d6-4018-a67a-c654f6deb5dc"}]: dispatch 2024-07-04T08:05:20.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:19 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a0b6a8f-16d6-4018-a67a-c654f6deb5dc"}]: dispatch 2024-07-04T08:05:20.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:19 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a0b6a8f-16d6-4018-a67a-c654f6deb5dc"}]': finished 2024-07-04T08:05:20.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:19 smithi133 ceph-mon[37050]: osdmap e35: 6 total, 5 up, 6 in 2024-07-04T08:05:20.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:19 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:20.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:19 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/917082078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a0b6a8f-16d6-4018-a67a-c654f6deb5dc"}]: dispatch 2024-07-04T08:05:20.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:19 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a0b6a8f-16d6-4018-a67a-c654f6deb5dc"}]: dispatch 2024-07-04T08:05:20.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:19 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a0b6a8f-16d6-4018-a67a-c654f6deb5dc"}]': finished 2024-07-04T08:05:20.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:19 smithi039 ceph-mon[32062]: osdmap e35: 6 total, 5 up, 6 in 2024-07-04T08:05:20.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:19 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:20.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:19 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/917082078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a0b6a8f-16d6-4018-a67a-c654f6deb5dc"}]: dispatch 2024-07-04T08:05:20.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:19 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a0b6a8f-16d6-4018-a67a-c654f6deb5dc"}]: dispatch 2024-07-04T08:05:20.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:19 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a0b6a8f-16d6-4018-a67a-c654f6deb5dc"}]': finished 2024-07-04T08:05:20.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:19 smithi050 ceph-mon[36577]: osdmap e35: 6 total, 5 up, 6 in 2024-07-04T08:05:20.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:19 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:21.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:20 smithi133 ceph-mon[37050]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:21.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:20 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/3142802296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:05:21.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:20 smithi039 ceph-mon[32062]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:21.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:20 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/3142802296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:05:21.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:20 smithi050 ceph-mon[36577]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:21.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:20 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/3142802296' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:05:23.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:22 smithi133 ceph-mon[37050]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:23.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:22 smithi039 ceph-mon[32062]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:23.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:22 smithi050 ceph-mon[36577]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:24.990 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:24 smithi133 ceph-mon[37050]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:25.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:24 smithi039 ceph-mon[32062]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:24 smithi050 ceph-mon[36577]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:26 smithi133 ceph-mon[37050]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:26 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-04T08:05:27.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:26 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:27.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:26 smithi039 ceph-mon[32062]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:27.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:26 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-04T08:05:27.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:26 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:27.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:26 smithi050 ceph-mon[36577]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:27.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:26 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-04T08:05:27.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:26 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:28.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:27 smithi133 ceph-mon[37050]: Deploying daemon osd.5 on smithi133 2024-07-04T08:05:28.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:27 smithi039 ceph-mon[32062]: Deploying daemon osd.5 on smithi133 2024-07-04T08:05:28.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:27 smithi050 ceph-mon[36577]: Deploying daemon osd.5 on smithi133 2024-07-04T08:05:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:28 smithi039 ceph-mon[32062]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:29.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:28 smithi050 ceph-mon[36577]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:29.154 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:28 smithi133 ceph-mon[37050]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:31.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:30 smithi039 ceph-mon[32062]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:31.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:30 smithi050 ceph-mon[36577]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:31.219 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:30 smithi133 ceph-mon[37050]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:32 smithi039 ceph-mon[32062]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:33.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:32 smithi050 ceph-mon[36577]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:33.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:33.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:33.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:32 smithi133 ceph-mon[37050]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:34 smithi039 ceph-mon[32062]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:35.102 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:34 smithi133 ceph-mon[37050]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:35.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:34 smithi050 ceph-mon[36577]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:35.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.166+0000 7f1b9b7fe640 1 -- 172.21.15.133:0/945011743 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1ba4062d50 con 0x7f1b900b4ad0 2024-07-04T08:05:35.167 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 5 on host 'smithi133' 2024-07-04T08:05:35.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.168+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b900b4ad0 msgr2=0x7f1b900b6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:35.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.168+0000 7f1ba8953640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b900b4ad0 0x7f1b900b6fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1b94031440 tx=0x7f1b94009560 comp rx=0 tx=0).stop 2024-07-04T08:05:35.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.168+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4108280 msgr2=0x7f1ba419d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:35.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.168+0000 7f1ba8953640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4108280 0x7f1ba419d070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00eaa0 tx=0x7f1b9c00ef60 comp rx=0 tx=0).stop 2024-07-04T08:05:35.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.169+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 shutdown_connections 2024-07-04T08:05:35.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.169+0000 7f1ba8953640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b900b4ad0 0x7f1b900b6fe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:35.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.169+0000 7f1ba8953640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ba4107930 0x7f1ba4197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:35.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.169+0000 7f1ba8953640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ba4108280 0x7f1ba419d070 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:35.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.169+0000 7f1ba8953640 1 --2- 172.21.15.133:0/945011743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ba4101ff0 0x7f1ba4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:35.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.169+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 >> 172.21.15.133:0/945011743 conn(0x7f1ba4078ca0 msgr2=0x7f1ba41059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:05:35.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.169+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 shutdown_connections 2024-07-04T08:05:35.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:05:35.169+0000 7f1ba8953640 1 -- 172.21.15.133:0/945011743 wait complete. 2024-07-04T08:05:35.808 DEBUG:teuthology.orchestra.run.smithi133:osd.5> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.5.service 2024-07-04T08:05:35.810 INFO:tasks.cephadm:Deploying osd.6 on smithi039 with /dev/vg_nvme/lv_2... 2024-07-04T08:05:35.811 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-04T08:05:36.152 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:05:36 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:05:36.025+0000 7f67984f0640 -1 osd.5 0 log_to_monitors true 2024-07-04T08:05:36.288 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:05:36.444 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:36 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:36 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:36 smithi039 ceph-mon[32062]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:36.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:36 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:36 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:36 smithi039 ceph-mon[32062]: from='osd.5 [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-04T08:05:36.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:36 smithi039 ceph-mon[32062]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-04T08:05:36.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:36 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:36 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:36 smithi133 ceph-mon[37050]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:36.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:36 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:36 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:36 smithi133 ceph-mon[37050]: from='osd.5 [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-04T08:05:36.454 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:36 smithi133 ceph-mon[37050]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-04T08:05:36.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:36 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:36 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:36 smithi050 ceph-mon[36577]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:36.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:36 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:36 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:36.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:36 smithi050 ceph-mon[36577]: from='osd.5 [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-04T08:05:36.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:36 smithi050 ceph-mon[36577]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-04T08:05:38.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:37 smithi039 ceph-mon[32062]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-04T08:05:38.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:37 smithi039 ceph-mon[32062]: from='osd.5 [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:05:38.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:37 smithi039 ceph-mon[32062]: osdmap e36: 6 total, 5 up, 6 in 2024-07-04T08:05:38.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:37 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:38.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:37 smithi039 ceph-mon[32062]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:05:38.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:37 smithi133 ceph-mon[37050]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-04T08:05:38.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:37 smithi133 ceph-mon[37050]: from='osd.5 [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:05:38.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:37 smithi133 ceph-mon[37050]: osdmap e36: 6 total, 5 up, 6 in 2024-07-04T08:05:38.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:37 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:38.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:37 smithi133 ceph-mon[37050]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:05:38.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:37 smithi050 ceph-mon[36577]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-04T08:05:38.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:37 smithi050 ceph-mon[36577]: from='osd.5 [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:05:38.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:37 smithi050 ceph-mon[36577]: osdmap e36: 6 total, 5 up, 6 in 2024-07-04T08:05:38.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:37 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:38.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:37 smithi050 ceph-mon[36577]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:05:38.761 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:05:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:05:38.408+0000 7f6794472640 -1 osd.5 0 waiting for initial osdmap 2024-07-04T08:05:38.761 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:05:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:05:38.417+0000 7f678c269640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:05:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:38 smithi133 ceph-mon[37050]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:38 smithi133 ceph-mon[37050]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:05:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:38 smithi133 ceph-mon[37050]: osdmap e37: 6 total, 5 up, 6 in 2024-07-04T08:05:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:38 smithi133 ceph-mon[37050]: osdmap e38: 6 total, 5 up, 6 in 2024-07-04T08:05:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:38 smithi133 ceph-mon[37050]: from='osd.5 ' entity='osd.5' 2024-07-04T08:05:39.143 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-04T08:05:39.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:38 smithi039 ceph-mon[32062]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:39.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:38 smithi039 ceph-mon[32062]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:05:39.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:38 smithi039 ceph-mon[32062]: osdmap e37: 6 total, 5 up, 6 in 2024-07-04T08:05:39.200 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:39.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:38 smithi039 ceph-mon[32062]: osdmap e38: 6 total, 5 up, 6 in 2024-07-04T08:05:39.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:39.201 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:38 smithi039 ceph-mon[32062]: from='osd.5 ' entity='osd.5' 2024-07-04T08:05:39.347 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-04T08:05:39.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:38 smithi050 ceph-mon[36577]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:38 smithi050 ceph-mon[36577]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:05:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:38 smithi050 ceph-mon[36577]: osdmap e37: 6 total, 5 up, 6 in 2024-07-04T08:05:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:38 smithi050 ceph-mon[36577]: osdmap e38: 6 total, 5 up, 6 in 2024-07-04T08:05:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:38 smithi050 ceph-mon[36577]: from='osd.5 ' entity='osd.5' 2024-07-04T08:05:39.395 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-07-04T08:05:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-07-04T08:05:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0377981 s, 277 MB/s 2024-07-04T08:05:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-07-04T08:05:40.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:05:40.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:05:40.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:40.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: osd.5 [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] boot 2024-07-04T08:05:40.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: osdmap e39: 6 total, 6 up, 6 in 2024-07-04T08:05:40.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:40.103 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.301 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_2 2024-07-04T08:05:40.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: osd.5 [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] boot 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: osdmap e39: 6 total, 6 up, 6 in 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:40.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:40.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:40.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:40.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:40.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:05:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:05:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: osd.5 [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] boot 2024-07-04T08:05:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: osdmap e39: 6 total, 6 up, 6 in 2024-07-04T08:05:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-04T08:05:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:05:40.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:05:40.774 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:05:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:40 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi133 2024-07-04T08:05:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:40 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi133 to 8544M 2024-07-04T08:05:41.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:40 smithi133 ceph-mon[37050]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:41.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:40 smithi133 ceph-mon[37050]: osdmap e40: 6 total, 6 up, 6 in 2024-07-04T08:05:41.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:40 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi133 2024-07-04T08:05:41.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:40 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi133 to 8544M 2024-07-04T08:05:41.112 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:40 smithi039 ceph-mon[32062]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:41.113 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:40 smithi039 ceph-mon[32062]: osdmap e40: 6 total, 6 up, 6 in 2024-07-04T08:05:41.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:40 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi133 2024-07-04T08:05:41.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:40 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi133 to 8544M 2024-07-04T08:05:41.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:40 smithi050 ceph-mon[36577]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-07-04T08:05:41.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:40 smithi050 ceph-mon[36577]: osdmap e40: 6 total, 6 up, 6 in 2024-07-04T08:05:42.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:42 smithi039 ceph-mon[32062]: osdmap e41: 6 total, 6 up, 6 in 2024-07-04T08:05:42.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:42 smithi039 ceph-mon[32062]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:42.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:42 smithi050 ceph-mon[36577]: osdmap e41: 6 total, 6 up, 6 in 2024-07-04T08:05:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:42 smithi050 ceph-mon[36577]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:42.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:42 smithi133 ceph-mon[37050]: osdmap e41: 6 total, 6 up, 6 in 2024-07-04T08:05:42.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:42 smithi133 ceph-mon[37050]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:43.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.112+0000 7fb26be00640 1 -- 172.21.15.39:0/1711373728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26c113b50 msgr2=0x7fb26c115f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:43.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.112+0000 7fb26be00640 1 --2- 172.21.15.39:0/1711373728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26c113b50 0x7fb26c115f70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb264009860 tx=0x7fb26402a340 comp rx=0 tx=0).stop 2024-07-04T08:05:43.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.112+0000 7fb26be00640 1 -- 172.21.15.39:0/1711373728 shutdown_connections 2024-07-04T08:05:43.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.112+0000 7fb26be00640 1 --2- 172.21.15.39:0/1711373728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb26c0777e0 0x7fb26c113610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:43.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.112+0000 7fb26be00640 1 --2- 172.21.15.39:0/1711373728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26c113b50 0x7fb26c115f70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:43.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.112+0000 7fb26be00640 1 --2- 172.21.15.39:0/1711373728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb26c076e90 0x7fb26c0772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:43.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.112+0000 7fb26be00640 1 -- 172.21.15.39:0/1711373728 >> 172.21.15.39:0/1711373728 conn(0x7fb26c100be0 msgr2=0x7fb26c103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:05:43.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.112+0000 7fb26be00640 1 -- 172.21.15.39:0/1711373728 shutdown_connections 2024-07-04T08:05:43.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.112+0000 7fb26be00640 1 -- 172.21.15.39:0/1711373728 wait complete. 2024-07-04T08:05:43.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.113+0000 7fb26be00640 1 Processor -- start 2024-07-04T08:05:43.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.113+0000 7fb26be00640 1 -- start start 2024-07-04T08:05:43.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.113+0000 7fb26be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26c076e90 0x7fb26c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.113+0000 7fb26be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb26c0777e0 0x7fb26c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb26c113b50 0x7fb26c1a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c119430 con 0x7fb26c0777e0 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c1192b0 con 0x7fb26c113b50 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c1195b0 con 0x7fb26c076e90 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb262000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb26c0777e0 0x7fb26c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb262000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb26c0777e0 0x7fb26c1a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55492/0 (socket says 172.21.15.39:55492) 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb262000640 1 -- 172.21.15.39:0/3452988844 learned_addr learned my addr 172.21.15.39:0/3452988844 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26b400640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb26c113b50 0x7fb26c1a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26aa00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26c076e90 0x7fb26c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:05:43.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26b400640 1 -- 172.21.15.39:0/3452988844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26c076e90 msgr2=0x7fb26c19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26b400640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26c076e90 0x7fb26c19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26b400640 1 -- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb26c0777e0 msgr2=0x7fb26c1a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:05:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26b400640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb26c0777e0 0x7fb26c1a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:05:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb26b400640 1 -- 172.21.15.39:0/3452988844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c1a5d70 con 0x7fb26c113b50 2024-07-04T08:05:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.114+0000 7fb262000640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb26c0777e0 0x7fb26c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:05:43.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.115+0000 7fb26b400640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb26c113b50 0x7fb26c1a5640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb26402a860 tx=0x7fb264032d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:05:43.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.115+0000 7fb263e00640 1 -- 172.21.15.39:0/3452988844 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264037040 con 0x7fb26c113b50 2024-07-04T08:05:43.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.115+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c1a6000 con 0x7fb26c113b50 2024-07-04T08:05:43.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.115+0000 7fb263e00640 1 -- 172.21.15.39:0/3452988844 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb264041070 con 0x7fb26c113b50 2024-07-04T08:05:43.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.115+0000 7fb263e00640 1 -- 172.21.15.39:0/3452988844 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2640035f0 con 0x7fb26c113b50 2024-07-04T08:05:43.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.115+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c1ac7c0 con 0x7fb26c113b50 2024-07-04T08:05:43.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.116+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c0772a0 con 0x7fb26c113b50 2024-07-04T08:05:43.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.119+0000 7fb263e00640 1 -- 172.21.15.39:0/3452988844 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fb264004050 con 0x7fb26c113b50 2024-07-04T08:05:43.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.121+0000 7fb263e00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2500b2840 0x7fb2500b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:05:43.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.121+0000 7fb263e00640 1 -- 172.21.15.39:0/3452988844 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4748+0+0 (secure 0 0 0) 0x7fb264101970 con 0x7fb26c113b50 2024-07-04T08:05:43.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.121+0000 7fb26aa00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2500b2840 0x7fb2500b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:05:43.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.122+0000 7fb26aa00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2500b2840 0x7fb2500b4d50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005fd0 tx=0x7fb25c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:05:43.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.122+0000 7fb263e00640 1 -- 172.21.15.39:0/3452988844 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb26404b020 con 0x7fb26c113b50 2024-07-04T08:05:43.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:05:43.346+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb26c062d50 con 0x7fb2500b2840 2024-07-04T08:05:43.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:43 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:05:43.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:43 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:05:43.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:43 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:43.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:43 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:05:43.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:43 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:05:43.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:43 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:43.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:43 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:05:43.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:43 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:05:43.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:43 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:44.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:44 smithi133 ceph-mon[37050]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:05:44.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:44 smithi133 ceph-mon[37050]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:44.845 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:44 smithi039 ceph-mon[32062]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:05:44.845 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:44 smithi039 ceph-mon[32062]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:44 smithi050 ceph-mon[36577]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:05:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:44 smithi050 ceph-mon[36577]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:46.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:46 smithi039 ceph-mon[32062]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-07-04T08:05:46.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:46 smithi133 ceph-mon[37050]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-07-04T08:05:46.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:46 smithi050 ceph-mon[36577]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-07-04T08:05:47.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:47 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/661187112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fae5d481-b487-4e26-ae87-395fa655837c"}]: dispatch 2024-07-04T08:05:47.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:47 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fae5d481-b487-4e26-ae87-395fa655837c"}]: dispatch 2024-07-04T08:05:47.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:47 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fae5d481-b487-4e26-ae87-395fa655837c"}]': finished 2024-07-04T08:05:47.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:47 smithi133 ceph-mon[37050]: osdmap e42: 7 total, 6 up, 7 in 2024-07-04T08:05:47.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:47 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:05:47.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:47 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/661187112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fae5d481-b487-4e26-ae87-395fa655837c"}]: dispatch 2024-07-04T08:05:47.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:47 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fae5d481-b487-4e26-ae87-395fa655837c"}]: dispatch 2024-07-04T08:05:47.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:47 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fae5d481-b487-4e26-ae87-395fa655837c"}]': finished 2024-07-04T08:05:47.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:47 smithi039 ceph-mon[32062]: osdmap e42: 7 total, 6 up, 7 in 2024-07-04T08:05:47.832 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:47 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:05:47.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:47 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/661187112' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fae5d481-b487-4e26-ae87-395fa655837c"}]: dispatch 2024-07-04T08:05:47.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:47 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fae5d481-b487-4e26-ae87-395fa655837c"}]: dispatch 2024-07-04T08:05:47.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:47 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fae5d481-b487-4e26-ae87-395fa655837c"}]': finished 2024-07-04T08:05:47.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:47 smithi050 ceph-mon[36577]: osdmap e42: 7 total, 6 up, 7 in 2024-07-04T08:05:47.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:47 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:05:48.796 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:48 smithi050 ceph-mon[36577]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-07-04T08:05:48.796 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:48 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1553626674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:05:48.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:48 smithi133 ceph-mon[37050]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-07-04T08:05:48.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:48 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1553626674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:05:48.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:48 smithi039 ceph-mon[32062]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-07-04T08:05:48.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:48 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1553626674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:05:50.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:50 smithi133 ceph-mon[37050]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-07-04T08:05:50.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:50 smithi039 ceph-mon[32062]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-07-04T08:05:50.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:50 smithi050 ceph-mon[36577]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-07-04T08:05:52.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:52 smithi133 ceph-mon[37050]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:52.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:52 smithi039 ceph-mon[32062]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:52.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:52 smithi050 ceph-mon[36577]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:54.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:54 smithi133 ceph-mon[37050]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:54.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:54 smithi039 ceph-mon[32062]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:54.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:54 smithi050 ceph-mon[36577]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:05:56.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:56 smithi133 ceph-mon[37050]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:56.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:56 smithi039 ceph-mon[32062]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:56.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:56 smithi050 ceph-mon[36577]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:57.800 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-04T08:05:57.800 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:57.800 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:57 smithi039 ceph-mon[32062]: Deploying daemon osd.6 on smithi039 2024-07-04T08:05:57.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-04T08:05:57.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:57.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:57 smithi133 ceph-mon[37050]: Deploying daemon osd.6 on smithi039 2024-07-04T08:05:57.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-04T08:05:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:05:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:57 smithi050 ceph-mon[36577]: Deploying daemon osd.6 on smithi039 2024-07-04T08:05:58.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:05:58 smithi133 ceph-mon[37050]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:58.850 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:05:58 smithi039 ceph-mon[32062]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:05:58.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:05:58 smithi050 ceph-mon[36577]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:00.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:00 smithi133 ceph-mon[37050]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:00.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:00 smithi039 ceph-mon[32062]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:00.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:00 smithi050 ceph-mon[36577]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:02.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:02 smithi133 ceph-mon[37050]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:02.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:02 smithi039 ceph-mon[32062]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:02.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:02 smithi050 ceph-mon[36577]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:04.751 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:04 smithi039 ceph-mon[32062]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:04.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:04 smithi133 ceph-mon[37050]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:04.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:04 smithi050 ceph-mon[36577]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:06.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:06 smithi039 ceph-mon[32062]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:06.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:06.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:06.717 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:06.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:06 smithi133 ceph-mon[37050]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:06.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:06.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:06.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:06.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:06 smithi050 ceph-mon[36577]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:06.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:06.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:06.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:08.785 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:08 smithi039 ceph-mon[32062]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:08.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:08 smithi133 ceph-mon[37050]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:08.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:08 smithi050 ceph-mon[36577]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:10.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:10.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:10.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:10 smithi133 ceph-mon[37050]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:10.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:10 smithi133 ceph-mon[37050]: from='osd.6 [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-04T08:06:10.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:10 smithi133 ceph-mon[37050]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-04T08:06:10.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:10.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:10.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:10.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:10.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:10.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:10 smithi039 ceph-mon[32062]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:10.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:10 smithi039 ceph-mon[32062]: from='osd.6 [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-04T08:06:10.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:10 smithi039 ceph-mon[32062]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-04T08:06:10.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:10.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:10.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:10.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:10.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:10.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:10 smithi050 ceph-mon[36577]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:10.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:10 smithi050 ceph-mon[36577]: from='osd.6 [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-04T08:06:10.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:10 smithi050 ceph-mon[36577]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-04T08:06:10.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:10.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:10.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:11.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:11 smithi039 ceph-mon[32062]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-04T08:06:11.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:11 smithi039 ceph-mon[32062]: from='osd.6 [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:06:11.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:11 smithi039 ceph-mon[32062]: osdmap e43: 7 total, 6 up, 7 in 2024-07-04T08:06:11.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:11.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:11 smithi039 ceph-mon[32062]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:06:11.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.360+0000 7fb263e00640 1 -- 172.21.15.39:0/3452988844 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb26c062d50 con 0x7fb2500b2840 2024-07-04T08:06:11.363 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 6 on host 'smithi039' 2024-07-04T08:06:11.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2500b2840 msgr2=0x7fb2500b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:11.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2500b2840 0x7fb2500b4d50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005fd0 tx=0x7fb25c009500 comp rx=0 tx=0).stop 2024-07-04T08:06:11.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb26c113b50 msgr2=0x7fb26c1a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:11.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb26c113b50 0x7fb26c1a5640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb26402a860 tx=0x7fb264032d40 comp rx=0 tx=0).stop 2024-07-04T08:06:11.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 shutdown_connections 2024-07-04T08:06:11.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2500b2840 0x7fb2500b4d50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:11.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb26c0777e0 0x7fb26c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:11.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb26c113b50 0x7fb26c1a5640 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:11.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 --2- 172.21.15.39:0/3452988844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb26c076e90 0x7fb26c19fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:11.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.364+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 >> 172.21.15.39:0/3452988844 conn(0x7fb26c100be0 msgr2=0x7fb26c115c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:06:11.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.365+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 shutdown_connections 2024-07-04T08:06:11.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:06:11.365+0000 7fb26be00640 1 -- 172.21.15.39:0/3452988844 wait complete. 2024-07-04T08:06:11.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:11 smithi050 ceph-mon[36577]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-04T08:06:11.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:11 smithi050 ceph-mon[36577]: from='osd.6 [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:06:11.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:11 smithi050 ceph-mon[36577]: osdmap e43: 7 total, 6 up, 7 in 2024-07-04T08:06:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:11 smithi050 ceph-mon[36577]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:06:11.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:11 smithi133 ceph-mon[37050]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-04T08:06:11.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:11 smithi133 ceph-mon[37050]: from='osd.6 [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:06:11.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:11 smithi133 ceph-mon[37050]: osdmap e43: 7 total, 6 up, 7 in 2024-07-04T08:06:11.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:11.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:11 smithi133 ceph-mon[37050]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:06:12.141 DEBUG:teuthology.orchestra.run.smithi039:osd.6> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.6.service 2024-07-04T08:06:12.147 INFO:tasks.cephadm:Deploying osd.7 on smithi050 with /dev/vg_nvme/lv_2... 2024-07-04T08:06:12.148 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-04T08:06:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:12 smithi039 ceph-mon[32062]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:06:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:12 smithi039 ceph-mon[32062]: osdmap e44: 7 total, 6 up, 7 in 2024-07-04T08:06:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:12 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:12.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:12 smithi039 ceph-mon[32062]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:12.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:12 smithi050 ceph-mon[36577]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:06:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:12 smithi050 ceph-mon[36577]: osdmap e44: 7 total, 6 up, 7 in 2024-07-04T08:06:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:12 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:12.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:12 smithi050 ceph-mon[36577]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:12.418 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:06:12.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:12 smithi133 ceph-mon[37050]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:06:12.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:12 smithi133 ceph-mon[37050]: osdmap e44: 7 total, 6 up, 7 in 2024-07-04T08:06:12.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:12.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:12.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:12.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:12.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:12 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:12.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:12 smithi133 ceph-mon[37050]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-07-04T08:06:13.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:13 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:06:13.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:13 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:06:13.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:13 smithi050 ceph-mon[36577]: osd.6 [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] boot 2024-07-04T08:06:13.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:13 smithi050 ceph-mon[36577]: osdmap e45: 7 total, 7 up, 7 in 2024-07-04T08:06:13.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:13 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:13.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:13 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:06:13.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:13 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:06:13.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:13 smithi133 ceph-mon[37050]: osd.6 [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] boot 2024-07-04T08:06:13.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:13 smithi133 ceph-mon[37050]: osdmap e45: 7 total, 7 up, 7 in 2024-07-04T08:06:13.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:13 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:13.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:13 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:06:13.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:13 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:06:13.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:13 smithi039 ceph-mon[32062]: osd.6 [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] boot 2024-07-04T08:06:13.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:13 smithi039 ceph-mon[32062]: osdmap e45: 7 total, 7 up, 7 in 2024-07-04T08:06:13.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:13 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-04T08:06:14.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:14 smithi050 ceph-mon[36577]: osdmap e46: 7 total, 7 up, 7 in 2024-07-04T08:06:14.409 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:14 smithi050 ceph-mon[36577]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:14.572 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-04T08:06:14.660 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-04T08:06:14.670 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-07-04T08:06:14.670 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-07-04T08:06:14.670 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00844939 s, 1.2 GB/s 2024-07-04T08:06:14.670 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-07-04T08:06:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:14 smithi133 ceph-mon[37050]: osdmap e46: 7 total, 7 up, 7 in 2024-07-04T08:06:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:14 smithi133 ceph-mon[37050]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:14.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:14 smithi039 ceph-mon[32062]: osdmap e46: 7 total, 7 up, 7 in 2024-07-04T08:06:14.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:14 smithi039 ceph-mon[32062]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:15.350 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_2 2024-07-04T08:06:15.621 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:06:16.692 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:16 smithi039 ceph-mon[32062]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:16.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:16 smithi050 ceph-mon[36577]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:16.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:16 smithi133 ceph-mon[37050]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:17.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.415+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/1932658135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c81030b0 msgr2=0x7fe9c81034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:17.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.415+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/1932658135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c81030b0 0x7fe9c81034c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009900 tx=0x7fe9c402a410 comp rx=0 tx=0).stop 2024-07-04T08:06:17.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.415+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/1932658135 shutdown_connections 2024-07-04T08:06:17.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.415+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/1932658135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c8109340 0x7fe9c810fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:17.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.415+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/1932658135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c81089f0 0x7fe9c8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:17.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.415+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/1932658135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c81030b0 0x7fe9c81034c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:17.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.415+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/1932658135 >> 172.21.15.50:0/1932658135 conn(0x7fe9c807ada0 msgr2=0x7fe9c807b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.416+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/1932658135 shutdown_connections 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.416+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/1932658135 wait complete. 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.416+0000 7fe9cfbdb640 1 Processor -- start 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.416+0000 7fe9cfbdb640 1 -- start start 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9cfbdb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c81030b0 0x7fe9c8199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9cfbdb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c81089f0 0x7fe9c819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9cfbdb640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c8109340 0x7fe9c819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9cfbdb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8113080 con 0x7fe9c81030b0 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9cfbdb640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8112f00 con 0x7fe9c8109340 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9cfbdb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8113200 con 0x7fe9c81089f0 2024-07-04T08:06:17.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9ce178640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c8109340 0x7fe9c819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9ce178640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c8109340 0x7fe9c819f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60340/0 (socket says 172.21.15.50:60340) 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9ce178640 1 -- 172.21.15.50:0/3221847758 learned_addr learned my addr 172.21.15.50:0/3221847758 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9cd176640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c81089f0 0x7fe9c819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9ce178640 1 -- 172.21.15.50:0/3221847758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c81089f0 msgr2=0x7fe9c819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9ce178640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c81089f0 0x7fe9c819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9ce178640 1 -- 172.21.15.50:0/3221847758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c81030b0 msgr2=0x7fe9c8199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9ce178640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c81030b0 0x7fe9c8199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.417+0000 7fe9ce178640 1 -- 172.21.15.50:0/3221847758 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c819fa80 con 0x7fe9c8109340 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.418+0000 7fe9ce178640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c8109340 0x7fe9c819f350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b970 tx=0x7fe9c000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.418+0000 7fe9beffd640 1 -- 172.21.15.50:0/3221847758 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0019070 con 0x7fe9c8109340 2024-07-04T08:06:17.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.418+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c819fd70 con 0x7fe9c8109340 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.418+0000 7fe9beffd640 1 -- 172.21.15.50:0/3221847758 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9c0014440 con 0x7fe9c8109340 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.418+0000 7fe9beffd640 1 -- 172.21.15.50:0/3221847758 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00075d0 con 0x7fe9c8109340 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.418+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c81a64d0 con 0x7fe9c8109340 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.419+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c81034c0 con 0x7fe9c8109340 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.420+0000 7fe9beffd640 1 -- 172.21.15.50:0/3221847758 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fe9c001c070 con 0x7fe9c8109340 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.421+0000 7fe9beffd640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9b40b2840 0x7fe9b40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.421+0000 7fe9beffd640 1 -- 172.21.15.50:0/3221847758 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5180+0+0 (secure 0 0 0) 0x7fe9c00e0bf0 con 0x7fe9c8109340 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.422+0000 7fe9cd977640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9b40b2840 0x7fe9b40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.423+0000 7fe9cd977640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9b40b2840 0x7fe9b40b4d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9c402a930 tx=0x7fe9c4031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:06:17.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.426+0000 7fe9beffd640 1 -- 172.21.15.50:0/3221847758 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe9c00a8f00 con 0x7fe9c8109340 2024-07-04T08:06:17.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:17.628+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe9c8062d50 con 0x7fe9b40b2840 2024-07-04T08:06:18.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi039 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi039 to 5696M 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:06:18.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi039 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi039 to 5696M 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:06:18.084 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:18.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi039 2024-07-04T08:06:18.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi039 to 5696M 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:06:18.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:19.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:18 smithi039 ceph-mon[32062]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:19.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:18 smithi039 ceph-mon[32062]: from='client.24476 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:06:19.111 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:18 smithi050 ceph-mon[36577]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:19.112 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:18 smithi050 ceph-mon[36577]: from='client.24476 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:06:19.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:18 smithi133 ceph-mon[37050]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:19.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:18 smithi133 ceph-mon[37050]: from='client.24476 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:06:21.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:20 smithi039 ceph-mon[32062]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:21.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:20 smithi039 ceph-mon[32062]: from='client.? 172.21.15.50:0/3216652867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ff9a0e2-4610-450d-be51-0726c8bc14ac"}]: dispatch 2024-07-04T08:06:21.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:20 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ff9a0e2-4610-450d-be51-0726c8bc14ac"}]: dispatch 2024-07-04T08:06:21.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:20 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ff9a0e2-4610-450d-be51-0726c8bc14ac"}]': finished 2024-07-04T08:06:21.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:20 smithi039 ceph-mon[32062]: osdmap e47: 8 total, 7 up, 8 in 2024-07-04T08:06:21.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:21.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:20 smithi050 ceph-mon[36577]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:21.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:20 smithi050 ceph-mon[36577]: from='client.? 172.21.15.50:0/3216652867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ff9a0e2-4610-450d-be51-0726c8bc14ac"}]: dispatch 2024-07-04T08:06:21.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:20 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ff9a0e2-4610-450d-be51-0726c8bc14ac"}]: dispatch 2024-07-04T08:06:21.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:20 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ff9a0e2-4610-450d-be51-0726c8bc14ac"}]': finished 2024-07-04T08:06:21.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:20 smithi050 ceph-mon[36577]: osdmap e47: 8 total, 7 up, 8 in 2024-07-04T08:06:21.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:20 smithi133 ceph-mon[37050]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:20 smithi133 ceph-mon[37050]: from='client.? 172.21.15.50:0/3216652867' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ff9a0e2-4610-450d-be51-0726c8bc14ac"}]: dispatch 2024-07-04T08:06:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:20 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ff9a0e2-4610-450d-be51-0726c8bc14ac"}]: dispatch 2024-07-04T08:06:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:20 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ff9a0e2-4610-450d-be51-0726c8bc14ac"}]': finished 2024-07-04T08:06:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:20 smithi133 ceph-mon[37050]: osdmap e47: 8 total, 7 up, 8 in 2024-07-04T08:06:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:22.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:21 smithi050 ceph-mon[36577]: from='client.? 172.21.15.50:0/467549902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:06:22.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:21 smithi133 ceph-mon[37050]: from='client.? 172.21.15.50:0/467549902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:06:22.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:21 smithi039 ceph-mon[32062]: from='client.? 172.21.15.50:0/467549902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:06:23.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:22 smithi050 ceph-mon[36577]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:23.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:22 smithi133 ceph-mon[37050]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:23.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:22 smithi039 ceph-mon[32062]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:25.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:24 smithi050 ceph-mon[36577]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:24 smithi133 ceph-mon[37050]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:25.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:24 smithi039 ceph-mon[32062]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:27.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:26 smithi050 ceph-mon[36577]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:26 smithi133 ceph-mon[37050]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:27.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:26 smithi039 ceph-mon[32062]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:29.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:28 smithi050 ceph-mon[36577]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:29.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-04T08:06:29.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:29.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:28 smithi133 ceph-mon[37050]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:29.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-04T08:06:29.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:29.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:28 smithi039 ceph-mon[32062]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:29.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-04T08:06:29.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:30.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:29 smithi050 ceph-mon[36577]: Deploying daemon osd.7 on smithi050 2024-07-04T08:06:30.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:29 smithi133 ceph-mon[37050]: Deploying daemon osd.7 on smithi050 2024-07-04T08:06:30.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:29 smithi039 ceph-mon[32062]: Deploying daemon osd.7 on smithi050 2024-07-04T08:06:31.290 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:30 smithi050 ceph-mon[36577]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:31.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:30 smithi133 ceph-mon[37050]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:31.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:30 smithi039 ceph-mon[32062]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:33.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:32 smithi050 ceph-mon[36577]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:32 smithi133 ceph-mon[37050]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:33.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:32 smithi039 ceph-mon[32062]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:34.163 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:34.163 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:34.163 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:34.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:34.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:34.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:34.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:34.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:34.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:35.217 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:34 smithi050 ceph-mon[36577]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:35.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:34 smithi133 ceph-mon[37050]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:35.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:34 smithi039 ceph-mon[32062]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:37.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.028+0000 7fe9beffd640 1 -- 172.21.15.50:0/3221847758 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe9c8062d50 con 0x7fe9b40b2840 2024-07-04T08:06:37.031 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 7 on host 'smithi050' 2024-07-04T08:06:37.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9b40b2840 msgr2=0x7fe9b40b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:37.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9b40b2840 0x7fe9b40b4d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9c402a930 tx=0x7fe9c4031400 comp rx=0 tx=0).stop 2024-07-04T08:06:37.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c8109340 msgr2=0x7fe9c819f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:37.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c8109340 0x7fe9c819f350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b970 tx=0x7fe9c000be30 comp rx=0 tx=0).stop 2024-07-04T08:06:37.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 shutdown_connections 2024-07-04T08:06:37.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9b40b2840 0x7fe9b40b4d50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:37.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c8109340 0x7fe9c819f350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:37.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c81089f0 0x7fe9c819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:37.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 --2- 172.21.15.50:0/3221847758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c81030b0 0x7fe9c8199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:37.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 >> 172.21.15.50:0/3221847758 conn(0x7fe9c807ada0 msgr2=0x7fe9c8106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:06:37.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 shutdown_connections 2024-07-04T08:06:37.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:06:37.031+0000 7fe9cfbdb640 1 -- 172.21.15.50:0/3221847758 wait complete. 2024-07-04T08:06:37.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:36 smithi133 ceph-mon[37050]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:37.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:36 smithi039 ceph-mon[32062]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:37.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:36 smithi050 ceph-mon[36577]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:37.756 DEBUG:teuthology.orchestra.run.smithi050:osd.7> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.7.service 2024-07-04T08:06:37.758 INFO:tasks.cephadm:Deploying osd.8 on smithi133 with /dev/vg_nvme/lv_2... 2024-07-04T08:06:37.759 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-04T08:06:38.017 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:06:38.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:38 smithi133 ceph-mon[37050]: from='osd.7 [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-04T08:06:38.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:38 smithi133 ceph-mon[37050]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-04T08:06:38.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.277 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.277 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.277 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:38 smithi050 ceph-mon[36577]: from='osd.7 [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-04T08:06:38.277 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:38 smithi050 ceph-mon[36577]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-04T08:06:38.277 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.277 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:38 smithi039 ceph-mon[32062]: from='osd.7 [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-04T08:06:38.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:38 smithi039 ceph-mon[32062]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-04T08:06:38.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:38.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:39.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:39 smithi133 ceph-mon[37050]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:39.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:39 smithi133 ceph-mon[37050]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-04T08:06:39.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:39 smithi133 ceph-mon[37050]: from='osd.7 [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:06:39.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:39 smithi133 ceph-mon[37050]: osdmap e48: 8 total, 7 up, 8 in 2024-07-04T08:06:39.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:39.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:39 smithi133 ceph-mon[37050]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:06:39.342 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:39 smithi050 ceph-mon[36577]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:39.342 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:39 smithi050 ceph-mon[36577]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-04T08:06:39.342 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:39 smithi050 ceph-mon[36577]: from='osd.7 [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:06:39.342 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:39 smithi050 ceph-mon[36577]: osdmap e48: 8 total, 7 up, 8 in 2024-07-04T08:06:39.342 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:39.342 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:39 smithi050 ceph-mon[36577]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:06:39.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:39 smithi039 ceph-mon[32062]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:39.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:39 smithi039 ceph-mon[32062]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-04T08:06:39.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:39 smithi039 ceph-mon[32062]: from='osd.7 [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:06:39.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:39 smithi039 ceph-mon[32062]: osdmap e48: 8 total, 7 up, 8 in 2024-07-04T08:06:39.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:39.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:39 smithi039 ceph-mon[32062]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:06:39.645 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:06:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:06:39.511+0000 7fc419e47640 -1 osd.7 0 waiting for initial osdmap 2024-07-04T08:06:39.645 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:06:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:06:39.520+0000 7fc411429640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:06:39.875 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-04T08:06:39.954 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-04T08:06:39.968 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-07-04T08:06:39.969 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-07-04T08:06:39.969 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124673 s, 841 MB/s 2024-07-04T08:06:39.969 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-07-04T08:06:40.292 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:40 smithi133 ceph-mon[37050]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:06:40.292 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:40 smithi133 ceph-mon[37050]: osdmap e49: 8 total, 7 up, 8 in 2024-07-04T08:06:40.292 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:40.292 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:40.292 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:40 smithi133 ceph-mon[37050]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:40.292 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:40 smithi133 ceph-mon[37050]: from='osd.7 ' entity='osd.7' 2024-07-04T08:06:40.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:40 smithi039 ceph-mon[32062]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:06:40.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:40 smithi039 ceph-mon[32062]: osdmap e49: 8 total, 7 up, 8 in 2024-07-04T08:06:40.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:40.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:40.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:40 smithi039 ceph-mon[32062]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:40.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:40 smithi039 ceph-mon[32062]: from='osd.7 ' entity='osd.7' 2024-07-04T08:06:40.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:40 smithi050 ceph-mon[36577]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:06:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:40 smithi050 ceph-mon[36577]: osdmap e49: 8 total, 7 up, 8 in 2024-07-04T08:06:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:40 smithi050 ceph-mon[36577]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-07-04T08:06:40.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:40 smithi050 ceph-mon[36577]: from='osd.7 ' entity='osd.7' 2024-07-04T08:06:40.630 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2024-07-04T08:06:40.886 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:06:41.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:41 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:06:41.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:41 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:06:41.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:41 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:41.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:41 smithi050 ceph-mon[36577]: osd.7 [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] boot 2024-07-04T08:06:41.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:41 smithi050 ceph-mon[36577]: osdmap e50: 8 total, 8 up, 8 in 2024-07-04T08:06:41.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:41 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:41.166 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:41 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:06:41.166 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:41 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:06:41.166 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:41 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:41.166 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:41 smithi133 ceph-mon[37050]: osd.7 [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] boot 2024-07-04T08:06:41.166 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:41 smithi133 ceph-mon[37050]: osdmap e50: 8 total, 8 up, 8 in 2024-07-04T08:06:41.167 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:41 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:41.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:41 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:06:41.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:41 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:06:41.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:41 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:41.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:41 smithi039 ceph-mon[32062]: osd.7 [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] boot 2024-07-04T08:06:41.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:41 smithi039 ceph-mon[32062]: osdmap e50: 8 total, 8 up, 8 in 2024-07-04T08:06:41.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:41 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-04T08:06:42.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: osdmap e51: 8 total, 8 up, 8 in 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi050 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi050 to 5697M 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:42.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:42.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:42.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.378 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: osdmap e51: 8 total, 8 up, 8 in 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi050 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi050 to 5697M 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:06:42.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:42.380 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:42.380 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:42.380 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: osdmap e51: 8 total, 8 up, 8 in 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi050 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi050 to 5697M 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:42.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:06:42.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:42.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:42.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:06:42.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.603+0000 7f322cf15640 1 -- 172.21.15.133:0/3798766020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32281048e0 msgr2=0x7f322810ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.603+0000 7f322cf15640 1 --2- 172.21.15.133:0/3798766020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32281048e0 0x7f322810ce30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c02a410 comp rx=0 tx=0).stop 2024-07-04T08:06:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.603+0000 7f322cf15640 1 -- 172.21.15.133:0/3798766020 shutdown_connections 2024-07-04T08:06:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.603+0000 7f322cf15640 1 --2- 172.21.15.133:0/3798766020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322810d3f0 0x7f322810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:42.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.603+0000 7f322cf15640 1 --2- 172.21.15.133:0/3798766020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32281048e0 0x7f322810ce30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:42.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.603+0000 7f322cf15640 1 --2- 172.21.15.133:0/3798766020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3228103f90 0x7f32281043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:42.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.603+0000 7f322cf15640 1 -- 172.21.15.133:0/3798766020 >> 172.21.15.133:0/3798766020 conn(0x7f3228078cc0 msgr2=0x7f32280790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:06:42.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.603+0000 7f322cf15640 1 -- 172.21.15.133:0/3798766020 shutdown_connections 2024-07-04T08:06:42.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.603+0000 7f322cf15640 1 -- 172.21.15.133:0/3798766020 wait complete. 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322cf15640 1 Processor -- start 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322cf15640 1 -- start start 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322cf15640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3228103f90 0x7f3228199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322cf15640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32281048e0 0x7f322819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322cf15640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322810d3f0 0x7f322819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322cf15640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f32281130b0 con 0x7f322810d3f0 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322cf15640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3228112f30 con 0x7f32281048e0 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322cf15640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3228113230 con 0x7f3228103f90 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3228103f90 0x7f3228199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3228103f90 0x7f3228199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37750/0 (socket says 172.21.15.133:37750) 2024-07-04T08:06:42.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f322659c640 1 -- 172.21.15.133:0/689880226 learned_addr learned my addr 172.21.15.133:0/689880226 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.604+0000 7f3225d9b640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32281048e0 0x7f322819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f322659c640 1 -- 172.21.15.133:0/689880226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32281048e0 msgr2=0x7f322819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f322659c640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32281048e0 0x7f322819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f322659c640 1 -- 172.21.15.133:0/689880226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322810d3f0 msgr2=0x7f322819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f322659c640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322810d3f0 0x7f322819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f322659c640 1 -- 172.21.15.133:0/689880226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322819fa90 con 0x7f3228103f90 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f3225d9b640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32281048e0 0x7f322819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f322659c640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3228103f90 0x7f3228199b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f321800d770 tx=0x7f321800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f32177fe640 1 -- 172.21.15.133:0/689880226 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218017070 con 0x7f3228103f90 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f32177fe640 1 -- 172.21.15.133:0/689880226 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32180054d0 con 0x7f3228103f90 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f32177fe640 1 -- 172.21.15.133:0/689880226 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218004c30 con 0x7f3228103f90 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322819fd80 con 0x7f3228103f90 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.605+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32281a64e0 con 0x7f3228103f90 2024-07-04T08:06:42.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.606+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322806b7b0 con 0x7f3228103f90 2024-07-04T08:06:42.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.608+0000 7f32177fe640 1 -- 172.21.15.133:0/689880226 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f3218004dd0 con 0x7f3228103f90 2024-07-04T08:06:42.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.608+0000 7f32177fe640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f32100b28c0 0x7f32100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:06:42.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.608+0000 7f32177fe640 1 -- 172.21.15.133:0/689880226 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5612+0+0 (secure 0 0 0) 0x7f32180e16c0 con 0x7f3228103f90 2024-07-04T08:06:42.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.609+0000 7f3225d9b640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f32100b28c0 0x7f32100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:06:42.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.610+0000 7f3225d9b640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f32100b28c0 0x7f32100b4dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f321c009230 tx=0x7f321c000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:06:42.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.611+0000 7f32177fe640 1 -- 172.21.15.133:0/689880226 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f32180a9ef0 con 0x7f3228103f90 2024-07-04T08:06:42.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:06:42.804+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3228106530 con 0x7f32100b28c0 2024-07-04T08:06:43.344 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:43 smithi133 ceph-mon[37050]: osdmap e52: 8 total, 8 up, 8 in 2024-07-04T08:06:43.344 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:43 smithi133 ceph-mon[37050]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-07-04T08:06:43.344 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:43 smithi133 ceph-mon[37050]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:06:43.344 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:43 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:06:43.345 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:43 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:06:43.345 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:43 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:43.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:43 smithi039 ceph-mon[32062]: osdmap e52: 8 total, 8 up, 8 in 2024-07-04T08:06:43.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:43 smithi039 ceph-mon[32062]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-07-04T08:06:43.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:43 smithi039 ceph-mon[32062]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:06:43.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:43 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:06:43.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:43 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:06:43.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:43 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:43 smithi050 ceph-mon[36577]: osdmap e52: 8 total, 8 up, 8 in 2024-07-04T08:06:43.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:43 smithi050 ceph-mon[36577]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-07-04T08:06:43.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:43 smithi050 ceph-mon[36577]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:06:43.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:43 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:06:43.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:43 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:06:43.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:43 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:44.211 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:44 smithi133 ceph-mon[37050]: pgmap v186: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:06:44.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:44 smithi050 ceph-mon[36577]: pgmap v186: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:06:44.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:44 smithi039 ceph-mon[32062]: pgmap v186: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:06:45.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:45 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/4135393327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1bdae27-2643-4838-ba2b-080777c325e9"}]: dispatch 2024-07-04T08:06:45.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:45 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1bdae27-2643-4838-ba2b-080777c325e9"}]: dispatch 2024-07-04T08:06:45.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:45 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1bdae27-2643-4838-ba2b-080777c325e9"}]': finished 2024-07-04T08:06:45.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:45 smithi039 ceph-mon[32062]: osdmap e53: 9 total, 8 up, 9 in 2024-07-04T08:06:45.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:06:45.604 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:45 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/4135393327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1bdae27-2643-4838-ba2b-080777c325e9"}]: dispatch 2024-07-04T08:06:45.604 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:45 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1bdae27-2643-4838-ba2b-080777c325e9"}]: dispatch 2024-07-04T08:06:45.604 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:45 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1bdae27-2643-4838-ba2b-080777c325e9"}]': finished 2024-07-04T08:06:45.604 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:45 smithi133 ceph-mon[37050]: osdmap e53: 9 total, 8 up, 9 in 2024-07-04T08:06:45.604 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:06:45.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:45 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/4135393327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1bdae27-2643-4838-ba2b-080777c325e9"}]: dispatch 2024-07-04T08:06:45.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:45 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1bdae27-2643-4838-ba2b-080777c325e9"}]: dispatch 2024-07-04T08:06:45.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:45 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1bdae27-2643-4838-ba2b-080777c325e9"}]': finished 2024-07-04T08:06:45.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:45 smithi050 ceph-mon[36577]: osdmap e53: 9 total, 8 up, 9 in 2024-07-04T08:06:45.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:06:46.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:46 smithi039 ceph-mon[32062]: pgmap v188: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:06:46.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:46 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/3999499904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:06:46.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:46 smithi050 ceph-mon[36577]: pgmap v188: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:06:46.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:46 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/3999499904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:06:46.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:46 smithi133 ceph-mon[37050]: pgmap v188: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:06:46.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:46 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/3999499904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:06:47.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:47 smithi133 ceph-mon[37050]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-07-04T08:06:47.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:47 smithi133 ceph-mon[37050]: Cluster is now healthy 2024-07-04T08:06:47.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:47 smithi039 ceph-mon[32062]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-07-04T08:06:47.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:47 smithi039 ceph-mon[32062]: Cluster is now healthy 2024-07-04T08:06:47.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:47 smithi050 ceph-mon[36577]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-07-04T08:06:47.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:47 smithi050 ceph-mon[36577]: Cluster is now healthy 2024-07-04T08:06:48.810 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:48 smithi050 ceph-mon[36577]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 181 KiB/s, 0 objects/s recovering 2024-07-04T08:06:48.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:48 smithi133 ceph-mon[37050]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 181 KiB/s, 0 objects/s recovering 2024-07-04T08:06:48.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:48 smithi039 ceph-mon[32062]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 181 KiB/s, 0 objects/s recovering 2024-07-04T08:06:50.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:50 smithi133 ceph-mon[37050]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-04T08:06:50.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:50 smithi039 ceph-mon[32062]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-04T08:06:50.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:50 smithi050 ceph-mon[36577]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-07-04T08:06:52.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:52 smithi133 ceph-mon[37050]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-07-04T08:06:52.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-04T08:06:52.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:52.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:52 smithi039 ceph-mon[32062]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-07-04T08:06:52.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-04T08:06:52.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:52.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:52 smithi050 ceph-mon[36577]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-07-04T08:06:52.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-04T08:06:52.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:06:53.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:53 smithi039 ceph-mon[32062]: Deploying daemon osd.8 on smithi133 2024-07-04T08:06:53.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:53 smithi050 ceph-mon[36577]: Deploying daemon osd.8 on smithi133 2024-07-04T08:06:53.984 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:53 smithi133 ceph-mon[37050]: Deploying daemon osd.8 on smithi133 2024-07-04T08:06:54.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:54 smithi039 ceph-mon[32062]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-07-04T08:06:54.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:54 smithi050 ceph-mon[36577]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-07-04T08:06:54.941 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:54 smithi133 ceph-mon[37050]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-07-04T08:06:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:56 smithi039 ceph-mon[32062]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-07-04T08:06:56.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:56 smithi050 ceph-mon[36577]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-07-04T08:06:57.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:56 smithi133 ceph-mon[37050]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-07-04T08:06:58.715 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:58 smithi133 ceph-mon[37050]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-04T08:06:58.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:58.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:58.716 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:06:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:58.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:58 smithi039 ceph-mon[32062]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-04T08:06:58.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:58.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:58.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:06:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:58.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:58 smithi050 ceph-mon[36577]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-04T08:06:58.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:06:58.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:06:58.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:06:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:00.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:00 smithi050 ceph-mon[36577]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:07:01.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:00 smithi133 ceph-mon[37050]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:07:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:00 smithi039 ceph-mon[32062]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:07:01.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.180+0000 7f32177fe640 1 -- 172.21.15.133:0/689880226 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3228106530 con 0x7f32100b28c0 2024-07-04T08:07:01.182 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 8 on host 'smithi133' 2024-07-04T08:07:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f32100b28c0 msgr2=0x7f32100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f32100b28c0 0x7f32100b4dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f321c009230 tx=0x7f321c000bc0 comp rx=0 tx=0).stop 2024-07-04T08:07:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3228103f90 msgr2=0x7f3228199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3228103f90 0x7f3228199b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f321800d770 tx=0x7f321800dc30 comp rx=0 tx=0).stop 2024-07-04T08:07:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 shutdown_connections 2024-07-04T08:07:01.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f32100b28c0 0x7f32100b4dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f322810d3f0 0x7f322819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f32281048e0 0x7f322819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 --2- 172.21.15.133:0/689880226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3228103f90 0x7f3228199b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 >> 172.21.15.133:0/689880226 conn(0x7f3228078cc0 msgr2=0x7f3228105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:07:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 shutdown_connections 2024-07-04T08:07:01.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:07:01.183+0000 7f322cf15640 1 -- 172.21.15.133:0/689880226 wait complete. 2024-07-04T08:07:02.121 DEBUG:teuthology.orchestra.run.smithi133:osd.8> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.8.service 2024-07-04T08:07:02.124 INFO:tasks.cephadm:Deploying osd.9 on smithi039 with /dev/vg_nvme/lv_1... 2024-07-04T08:07:02.124 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-04T08:07:02.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:02.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:02.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:02 smithi133 ceph-mon[37050]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:07:02.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:02 smithi133 ceph-mon[37050]: from='osd.8 [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-04T08:07:02.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:02 smithi133 ceph-mon[37050]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-04T08:07:02.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:02.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:02.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:02 smithi039 ceph-mon[32062]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:07:02.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:02 smithi039 ceph-mon[32062]: from='osd.8 [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-04T08:07:02.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:02 smithi039 ceph-mon[32062]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-04T08:07:02.627 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:07:02.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:02.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:02.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:02 smithi050 ceph-mon[36577]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:07:02.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:02 smithi050 ceph-mon[36577]: from='osd.8 [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-04T08:07:02.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:02 smithi050 ceph-mon[36577]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-04T08:07:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:03 smithi133 ceph-mon[37050]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-04T08:07:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:03 smithi133 ceph-mon[37050]: from='osd.8 [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:07:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:03 smithi133 ceph-mon[37050]: osdmap e54: 9 total, 8 up, 9 in 2024-07-04T08:07:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:03 smithi133 ceph-mon[37050]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:07:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:03.375 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:03 smithi039 ceph-mon[32062]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-04T08:07:03.375 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:03 smithi039 ceph-mon[32062]: from='osd.8 [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:07:03.375 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:03 smithi039 ceph-mon[32062]: osdmap e54: 9 total, 8 up, 9 in 2024-07-04T08:07:03.375 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:03.375 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:03 smithi039 ceph-mon[32062]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:07:03.375 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:03.375 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:03.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:03 smithi050 ceph-mon[36577]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-04T08:07:03.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:03 smithi050 ceph-mon[36577]: from='osd.8 [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:07:03.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:03 smithi050 ceph-mon[36577]: osdmap e54: 9 total, 8 up, 9 in 2024-07-04T08:07:03.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:03.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:03 smithi050 ceph-mon[36577]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:07:03.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:03.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:03.972 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:07:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:07:03.639+0000 7f27e3704640 -1 osd.8 0 waiting for initial osdmap 2024-07-04T08:07:03.972 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:07:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:07:03.651+0000 7f27db4fb640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:07:04.246 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:04 smithi133 ceph-mon[37050]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:07:04.246 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:04 smithi133 ceph-mon[37050]: osdmap e55: 9 total, 8 up, 9 in 2024-07-04T08:07:04.246 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:04.246 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:04.246 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:04 smithi133 ceph-mon[37050]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:07:04.246 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:04 smithi133 ceph-mon[37050]: from='osd.8 ' entity='osd.8' 2024-07-04T08:07:04.544 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:04 smithi039 ceph-mon[32062]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:07:04.544 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:04 smithi039 ceph-mon[32062]: osdmap e55: 9 total, 8 up, 9 in 2024-07-04T08:07:04.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:04.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:04.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:04 smithi039 ceph-mon[32062]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:07:04.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:04 smithi039 ceph-mon[32062]: from='osd.8 ' entity='osd.8' 2024-07-04T08:07:04.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:04 smithi050 ceph-mon[36577]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:07:04.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:04 smithi050 ceph-mon[36577]: osdmap e55: 9 total, 8 up, 9 in 2024-07-04T08:07:04.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:04.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:04.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:04 smithi050 ceph-mon[36577]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-07-04T08:07:04.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:04 smithi050 ceph-mon[36577]: from='osd.8 ' entity='osd.8' 2024-07-04T08:07:05.467 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-04T08:07:05.541 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:05 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:07:05.541 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:05 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:07:05.542 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:05 smithi133 ceph-mon[37050]: osd.8 [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] boot 2024-07-04T08:07:05.542 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:05 smithi133 ceph-mon[37050]: osdmap e56: 9 total, 9 up, 9 in 2024-07-04T08:07:05.542 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:05 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:05.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:05 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:07:05.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:05 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:07:05.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:05 smithi039 ceph-mon[32062]: osd.8 [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] boot 2024-07-04T08:07:05.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:05 smithi039 ceph-mon[32062]: osdmap e56: 9 total, 9 up, 9 in 2024-07-04T08:07:05.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:05 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:05.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:05 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:07:05.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:05 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:07:05.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:05 smithi050 ceph-mon[36577]: osd.8 [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] boot 2024-07-04T08:07:05.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:05 smithi050 ceph-mon[36577]: osdmap e56: 9 total, 9 up, 9 in 2024-07-04T08:07:05.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:05 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-04T08:07:05.661 INFO:teuthology.orchestra.run.smithi039.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-04T08:07:05.703 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10+0 records in 2024-07-04T08:07:05.703 INFO:teuthology.orchestra.run.smithi039.stderr:10+0 records out 2024-07-04T08:07:05.703 INFO:teuthology.orchestra.run.smithi039.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0347262 s, 302 MB/s 2024-07-04T08:07:05.704 INFO:teuthology.orchestra.run.smithi039.stderr:--> Zapping successful for: 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: osdmap e57: 9 total, 9 up, 9 in 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi133 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi133 to 5696M 2024-07-04T08:07:06.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:06.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:06.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:06.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:06.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:06.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:06 smithi039 ceph-mon[32062]: osdmap e58: 9 total, 9 up, 9 in 2024-07-04T08:07:06.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: osdmap e57: 9 total, 9 up, 9 in 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi133 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi133 to 5696M 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:06.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:06.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:06.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:06.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:06 smithi133 ceph-mon[37050]: osdmap e58: 9 total, 9 up, 9 in 2024-07-04T08:07:06.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi039:vg_nvme/lv_1 2024-07-04T08:07:06.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: osdmap e57: 9 total, 9 up, 9 in 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi133 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi133 to 5696M 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:06.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:06.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:06.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:06.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:06.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:06 smithi050 ceph-mon[36577]: osdmap e58: 9 total, 9 up, 9 in 2024-07-04T08:07:07.137 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:07:08.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:08 smithi133 ceph-mon[37050]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:08.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:08 smithi039 ceph-mon[32062]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:08.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:08 smithi050 ceph-mon[36577]: pgmap v204: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:09.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.524+0000 7f1056c00640 1 -- 172.21.15.39:0/2685291311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050103110 msgr2=0x7f1050103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:09.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.524+0000 7f1056c00640 1 --2- 172.21.15.39:0/2685291311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050103110 0x7f1050103520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1040009900 tx=0x7f104002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:07:09.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.524+0000 7f1056c00640 1 -- 172.21.15.39:0/2685291311 shutdown_connections 2024-07-04T08:07:09.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.524+0000 7f1056c00640 1 --2- 172.21.15.39:0/2685291311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1050108a00 0x7f1050108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:09.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.524+0000 7f1056c00640 1 --2- 172.21.15.39:0/2685291311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1050109350 0x7f105010fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:09.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.524+0000 7f1056c00640 1 --2- 172.21.15.39:0/2685291311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050103110 0x7f1050103520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:09.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.524+0000 7f1056c00640 1 -- 172.21.15.39:0/2685291311 >> 172.21.15.39:0/2685291311 conn(0x7f105007ae60 msgr2=0x7f105007b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:07:09.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.524+0000 7f1056c00640 1 -- 172.21.15.39:0/2685291311 shutdown_connections 2024-07-04T08:07:09.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.524+0000 7f1056c00640 1 -- 172.21.15.39:0/2685291311 wait complete. 2024-07-04T08:07:09.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.526+0000 7f1056c00640 1 Processor -- start 2024-07-04T08:07:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.526+0000 7f1056c00640 1 -- start start 2024-07-04T08:07:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.526+0000 7f1056c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050103110 0x7f1050074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:07:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.526+0000 7f1056c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1050108a00 0x7f10500751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:07:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.526+0000 7f1056c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1050109350 0x7f10500756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:07:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f1056c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1050113000 con 0x7f1050109350 2024-07-04T08:07:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f1056c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1050112e80 con 0x7f1050103110 2024-07-04T08:07:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f1056c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1050113180 con 0x7f1050108a00 2024-07-04T08:07:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f104f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1050108a00 0x7f10500751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:07:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f104fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050103110 0x7f1050074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:07:09.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f104f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1050108a00 0x7f10500751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59470/0 (socket says 172.21.15.39:59470) 2024-07-04T08:07:09.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f104f400640 1 -- 172.21.15.39:0/4179950043 learned_addr learned my addr 172.21.15.39:0/4179950043 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f104fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050103110 0x7f1050074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41592/0 (socket says 172.21.15.39:41592) 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f1054c00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1050109350 0x7f10500756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f104f400640 1 -- 172.21.15.39:0/4179950043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050103110 msgr2=0x7f1050074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.527+0000 7f104f400640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050103110 0x7f1050074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.528+0000 7f104f400640 1 -- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1050109350 msgr2=0x7f10500756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.528+0000 7f104f400640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1050109350 0x7f10500756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.528+0000 7f104f400640 1 -- 172.21.15.39:0/4179950043 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10501a7cb0 con 0x7f1050108a00 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.528+0000 7f104f400640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1050108a00 0x7f10500751a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f103800ebc0 tx=0x7f103800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.528+0000 7f104cc00640 1 -- 172.21.15.39:0/4179950043 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038010630 con 0x7f1050108a00 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.528+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10501a7fa0 con 0x7f1050108a00 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.528+0000 7f104cc00640 1 -- 172.21.15.39:0/4179950043 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1038010c80 con 0x7f1050108a00 2024-07-04T08:07:09.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.528+0000 7f104cc00640 1 -- 172.21.15.39:0/4179950043 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038017ba0 con 0x7f1050108a00 2024-07-04T08:07:09.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.528+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10501a8450 con 0x7f1050108a00 2024-07-04T08:07:09.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.530+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1050103520 con 0x7f1050108a00 2024-07-04T08:07:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.532+0000 7f104cc00640 1 -- 172.21.15.39:0/4179950043 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f10380107d0 con 0x7f1050108a00 2024-07-04T08:07:09.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.533+0000 7f104cc00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10300b28c0 0x7f10300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:07:09.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.534+0000 7f104cc00640 1 -- 172.21.15.39:0/4179950043 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6044+0+0 (secure 0 0 0) 0x7f10380e2270 con 0x7f1050108a00 2024-07-04T08:07:09.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.534+0000 7f104fe00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10300b28c0 0x7f10300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:07:09.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.535+0000 7f104fe00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10300b28c0 0x7f10300b4dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1040000c00 tx=0x7f1040031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:07:09.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.541+0000 7f104cc00640 1 -- 172.21.15.39:0/4179950043 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f10380aa7d0 con 0x7f1050108a00 2024-07-04T08:07:09.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:09.770+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1050062d50 con 0x7f10300b28c0 2024-07-04T08:07:10.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:10 smithi039 ceph-mon[32062]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:10.754 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:10 smithi039 ceph-mon[32062]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:07:10.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:07:10.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:07:10.755 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:10.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:10 smithi133 ceph-mon[37050]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:10.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:10 smithi133 ceph-mon[37050]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:07:10.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:07:10.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:07:10.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:10.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:10 smithi050 ceph-mon[36577]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:10.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:10 smithi050 ceph-mon[36577]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi039:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:07:10.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:07:10.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:07:10.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:12.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:12 smithi133 ceph-mon[37050]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-07-04T08:07:12.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:12 smithi039 ceph-mon[32062]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-07-04T08:07:12.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:12 smithi050 ceph-mon[36577]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-07-04T08:07:14.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:14 smithi039 ceph-mon[32062]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-04T08:07:14.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:14 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3006763698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "005c7632-7a25-4d09-9d4b-59675ba2cf70"}]: dispatch 2024-07-04T08:07:14.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:14 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3006763698' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "005c7632-7a25-4d09-9d4b-59675ba2cf70"}]': finished 2024-07-04T08:07:14.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:14 smithi039 ceph-mon[32062]: osdmap e59: 10 total, 9 up, 10 in 2024-07-04T08:07:14.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:14 smithi133 ceph-mon[37050]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-04T08:07:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:14 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3006763698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "005c7632-7a25-4d09-9d4b-59675ba2cf70"}]: dispatch 2024-07-04T08:07:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:14 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3006763698' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "005c7632-7a25-4d09-9d4b-59675ba2cf70"}]': finished 2024-07-04T08:07:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:14 smithi133 ceph-mon[37050]: osdmap e59: 10 total, 9 up, 10 in 2024-07-04T08:07:14.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:14.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:14 smithi050 ceph-mon[36577]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-07-04T08:07:14.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:14 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3006763698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "005c7632-7a25-4d09-9d4b-59675ba2cf70"}]: dispatch 2024-07-04T08:07:14.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:14 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3006763698' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "005c7632-7a25-4d09-9d4b-59675ba2cf70"}]': finished 2024-07-04T08:07:14.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:14 smithi050 ceph-mon[36577]: osdmap e59: 10 total, 9 up, 10 in 2024-07-04T08:07:14.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:15.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:15 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2234405438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:07:15.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:15 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2234405438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:07:15.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:15 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2234405438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:07:16.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:16 smithi133 ceph-mon[37050]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-07-04T08:07:16.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:16 smithi039 ceph-mon[32062]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-07-04T08:07:16.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:16 smithi050 ceph-mon[36577]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-07-04T08:07:18.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:18 smithi133 ceph-mon[37050]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:07:18.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:18 smithi039 ceph-mon[32062]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:07:18.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:18 smithi050 ceph-mon[36577]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:07:20.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:20 smithi133 ceph-mon[37050]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:07:20.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:20 smithi039 ceph-mon[32062]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:07:20.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:20 smithi050 ceph-mon[36577]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:07:22.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:22 smithi039 ceph-mon[32062]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:22.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:22 smithi050 ceph-mon[36577]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:23.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:22 smithi133 ceph-mon[37050]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:23.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-04T08:07:23.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:23.897 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-04T08:07:23.898 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:24.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-04T08:07:24.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:24.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:24 smithi039 ceph-mon[32062]: Deploying daemon osd.9 on smithi039 2024-07-04T08:07:24.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:24 smithi039 ceph-mon[32062]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:24.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:24 smithi050 ceph-mon[36577]: Deploying daemon osd.9 on smithi039 2024-07-04T08:07:24.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:24 smithi050 ceph-mon[36577]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:25.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:24 smithi133 ceph-mon[37050]: Deploying daemon osd.9 on smithi039 2024-07-04T08:07:25.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:24 smithi133 ceph-mon[37050]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:26.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:26 smithi039 ceph-mon[32062]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:26.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:26 smithi050 ceph-mon[36577]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:26 smithi133 ceph-mon[37050]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:28.641 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:28 smithi039 ceph-mon[32062]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:28.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:28 smithi050 ceph-mon[36577]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:29.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:28 smithi133 ceph-mon[37050]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:30.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:30 smithi039 ceph-mon[32062]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:30.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:30 smithi050 ceph-mon[36577]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:31.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:30 smithi133 ceph-mon[37050]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:32.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:32 smithi039 ceph-mon[32062]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:32.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:32.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:32.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:32 smithi050 ceph-mon[36577]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:32.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:32 smithi133 ceph-mon[37050]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:34.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:34 smithi039 ceph-mon[32062]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:34.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:34 smithi050 ceph-mon[36577]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:35.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:34 smithi133 ceph-mon[37050]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:36.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:36 smithi039 ceph-mon[32062]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:36.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:36 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:36.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:36 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:36.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:36 smithi039 ceph-mon[32062]: from='osd.9 [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-04T08:07:36.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:36 smithi039 ceph-mon[32062]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-04T08:07:36.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:36 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:36.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:36 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:36.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:36 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:37.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:36 smithi133 ceph-mon[37050]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:37.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:36 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:37.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:36 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:37.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:36 smithi133 ceph-mon[37050]: from='osd.9 [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-04T08:07:37.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:36 smithi133 ceph-mon[37050]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-04T08:07:37.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:36 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:37.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:36 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:37.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:36 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:37.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:36 smithi050 ceph-mon[36577]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:37.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:36 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:37.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:36 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:37.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:36 smithi050 ceph-mon[36577]: from='osd.9 [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-04T08:07:37.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:36 smithi050 ceph-mon[36577]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-04T08:07:37.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:36 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:37.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:36 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:37.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:36 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:37.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.582+0000 7f104cc00640 1 -- 172.21.15.39:0/4179950043 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1050062d50 con 0x7f10300b28c0 2024-07-04T08:07:37.587 INFO:teuthology.orchestra.run.smithi039.stdout:Created osd(s) 9 on host 'smithi039' 2024-07-04T08:07:37.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10300b28c0 msgr2=0x7f10300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10300b28c0 0x7f10300b4dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1040000c00 tx=0x7f1040031400 comp rx=0 tx=0).stop 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1050108a00 msgr2=0x7f10500751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1050108a00 0x7f10500751a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f103800ebc0 tx=0x7f103800c6a0 comp rx=0 tx=0).stop 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 shutdown_connections 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10300b28c0 0x7f10300b4dd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1050109350 0x7f10500756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1050108a00 0x7f10500751a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 --2- 172.21.15.39:0/4179950043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1050103110 0x7f1050074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.585+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 >> 172.21.15.39:0/4179950043 conn(0x7f105007ae60 msgr2=0x7f1050106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.586+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 shutdown_connections 2024-07-04T08:07:37.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:07:37.586+0000 7f1056c00640 1 -- 172.21.15.39:0/4179950043 wait complete. 2024-07-04T08:07:38.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:37 smithi039 ceph-mon[32062]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-04T08:07:38.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:37 smithi039 ceph-mon[32062]: osdmap e60: 10 total, 9 up, 10 in 2024-07-04T08:07:38.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:37 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:38.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:37 smithi039 ceph-mon[32062]: from='osd.9 [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:07:38.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:37 smithi039 ceph-mon[32062]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:07:38.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:37 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:38.004 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:37 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:38.005 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:37 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:38.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:37 smithi133 ceph-mon[37050]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-04T08:07:38.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:37 smithi133 ceph-mon[37050]: osdmap e60: 10 total, 9 up, 10 in 2024-07-04T08:07:38.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:37 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:38.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:37 smithi133 ceph-mon[37050]: from='osd.9 [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:07:38.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:37 smithi133 ceph-mon[37050]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:07:38.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:37 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:38.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:37 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:38.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:37 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:38.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:37 smithi050 ceph-mon[36577]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-04T08:07:38.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:37 smithi050 ceph-mon[36577]: osdmap e60: 10 total, 9 up, 10 in 2024-07-04T08:07:38.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:37 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:38.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:37 smithi050 ceph-mon[36577]: from='osd.9 [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:07:38.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:37 smithi050 ceph-mon[36577]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]: dispatch 2024-07-04T08:07:38.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:37 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:38.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:37 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:38.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:37 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:38.443 DEBUG:teuthology.orchestra.run.smithi039:osd.9> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.9.service 2024-07-04T08:07:38.448 INFO:tasks.cephadm:Deploying osd.10 on smithi050 with /dev/vg_nvme/lv_1... 2024-07-04T08:07:38.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-04T08:07:38.720 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:07:39.002 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:38 smithi039 ceph-mon[32062]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:39.002 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:38 smithi039 ceph-mon[32062]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:07:39.002 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:38 smithi039 ceph-mon[32062]: osdmap e61: 10 total, 9 up, 10 in 2024-07-04T08:07:39.002 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:39.002 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:39.002 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:38 smithi039 ceph-mon[32062]: from='osd.9 ' entity='osd.9' 2024-07-04T08:07:39.062 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:38 smithi050 ceph-mon[36577]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:39.063 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:38 smithi050 ceph-mon[36577]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:07:39.063 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:38 smithi050 ceph-mon[36577]: osdmap e61: 10 total, 9 up, 10 in 2024-07-04T08:07:39.063 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:39.063 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:39.063 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:38 smithi050 ceph-mon[36577]: from='osd.9 ' entity='osd.9' 2024-07-04T08:07:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:38 smithi133 ceph-mon[37050]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:38 smithi133 ceph-mon[37050]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi039", "root=default"]}]': finished 2024-07-04T08:07:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:38 smithi133 ceph-mon[37050]: osdmap e61: 10 total, 9 up, 10 in 2024-07-04T08:07:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:38 smithi133 ceph-mon[37050]: from='osd.9 ' entity='osd.9' 2024-07-04T08:07:39.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:39 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:07:39.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:39 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:07:39.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:39.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:39 smithi039 ceph-mon[32062]: osd.9 [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] boot 2024-07-04T08:07:39.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:39 smithi039 ceph-mon[32062]: osdmap e62: 10 total, 10 up, 10 in 2024-07-04T08:07:39.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:39 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:40.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:39 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:07:40.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:39 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:07:40.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:40.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:39 smithi133 ceph-mon[37050]: osd.9 [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] boot 2024-07-04T08:07:40.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:39 smithi133 ceph-mon[37050]: osdmap e62: 10 total, 10 up, 10 in 2024-07-04T08:07:40.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:39 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:40.090 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:39 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:07:40.090 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:39 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:07:40.090 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:40.091 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:39 smithi050 ceph-mon[36577]: osd.9 [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] boot 2024-07-04T08:07:40.091 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:39 smithi050 ceph-mon[36577]: osdmap e62: 10 total, 10 up, 10 in 2024-07-04T08:07:40.091 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:39 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-04T08:07:40.613 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-04T08:07:40.699 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-04T08:07:40.709 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-07-04T08:07:40.709 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-07-04T08:07:40.709 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00859702 s, 1.2 GB/s 2024-07-04T08:07:40.709 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-07-04T08:07:41.041 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:40 smithi050 ceph-mon[36577]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:41.041 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:40 smithi050 ceph-mon[36577]: osdmap e63: 10 total, 10 up, 10 in 2024-07-04T08:07:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:40 smithi133 ceph-mon[37050]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:40 smithi133 ceph-mon[37050]: osdmap e63: 10 total, 10 up, 10 in 2024-07-04T08:07:41.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:40 smithi039 ceph-mon[32062]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-07-04T08:07:41.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:40 smithi039 ceph-mon[32062]: osdmap e63: 10 total, 10 up, 10 in 2024-07-04T08:07:41.376 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_1 2024-07-04T08:07:41.644 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:07:43.014 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:42 smithi050 ceph-mon[36577]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:43.038 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:42 smithi039 ceph-mon[32062]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:42 smithi133 ceph-mon[37050]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:43.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.265+0000 7fc092a8c640 1 -- 172.21.15.50:0/2475963475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 msgr2=0x7fc08c10ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:43.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.265+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2475963475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 0x7fc08c10ea70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc084009ca0 tx=0x7fc08402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:07:43.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.265+0000 7fc092a8c640 1 -- 172.21.15.50:0/2475963475 shutdown_connections 2024-07-04T08:07:43.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.265+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2475963475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 0x7fc08c10ea70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:43.270 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.265+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2475963475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc08c1078f0 0x7fc08c107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.265+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2475963475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc08c101fb0 0x7fc08c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.265+0000 7fc092a8c640 1 -- 172.21.15.50:0/2475963475 >> 172.21.15.50:0/2475963475 conn(0x7fc08c078c80 msgr2=0x7fc08c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.265+0000 7fc092a8c640 1 -- 172.21.15.50:0/2475963475 shutdown_connections 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.265+0000 7fc092a8c640 1 -- 172.21.15.50:0/2475963475 wait complete. 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.266+0000 7fc092a8c640 1 Processor -- start 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.266+0000 7fc092a8c640 1 -- start start 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.266+0000 7fc092a8c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc08c101fb0 0x7fc08c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.266+0000 7fc092a8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc08c1078f0 0x7fc08c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.266+0000 7fc092a8c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 0x7fc08c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:07:43.271 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.266+0000 7fc092a8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c111eb0 con 0x7fc08c1078f0 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.266+0000 7fc092a8c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c111d30 con 0x7fc08c108240 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.266+0000 7fc092a8c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c112030 con 0x7fc08c101fb0 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.266+0000 7fc091029640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 0x7fc08c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc091029640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 0x7fc08c19b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:57890/0 (socket says 172.21.15.50:57890) 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc091029640 1 -- 172.21.15.50:0/2107265248 learned_addr learned my addr 172.21.15.50:0/2107265248 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc090828640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc08c101fb0 0x7fc08c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc091029640 1 -- 172.21.15.50:0/2107265248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc08c101fb0 msgr2=0x7fc08c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc091029640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc08c101fb0 0x7fc08c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc091029640 1 -- 172.21.15.50:0/2107265248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc08c1078f0 msgr2=0x7fc08c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc091029640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc08c1078f0 0x7fc08c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc091029640 1 -- 172.21.15.50:0/2107265248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c1a5aa0 con 0x7fc08c108240 2024-07-04T08:07:43.272 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc090828640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc08c101fb0 0x7fc08c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:07:43.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc091029640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 0x7fc08c19b260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc084005cc0 tx=0x7fc08402f380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:07:43.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc089ffb640 1 -- 172.21.15.50:0/2107265248 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084003bf0 con 0x7fc08c108240 2024-07-04T08:07:43.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c1a5d30 con 0x7fc08c108240 2024-07-04T08:07:43.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc089ffb640 1 -- 172.21.15.50:0/2107265248 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc084003d90 con 0x7fc08c108240 2024-07-04T08:07:43.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc089ffb640 1 -- 172.21.15.50:0/2107265248 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08402fbc0 con 0x7fc08c108240 2024-07-04T08:07:43.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.267+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a6210 con 0x7fc08c108240 2024-07-04T08:07:43.273 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.269+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c1023c0 con 0x7fc08c108240 2024-07-04T08:07:43.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.270+0000 7fc089ffb640 1 -- 172.21.15.50:0/2107265248 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fc08402fd60 con 0x7fc08c108240 2024-07-04T08:07:43.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.271+0000 7fc089ffb640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc07c0b2840 0x7fc07c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:07:43.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.271+0000 7fc089ffb640 1 -- 172.21.15.50:0/2107265248 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6476+0+0 (secure 0 0 0) 0x7fc084103540 con 0x7fc08c108240 2024-07-04T08:07:43.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.271+0000 7fc090828640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc07c0b2840 0x7fc07c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:07:43.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.272+0000 7fc090828640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc07c0b2840 0x7fc07c0b4d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc080005fd0 tx=0x7fc080005ec0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:07:43.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.274+0000 7fc089ffb640 1 -- 172.21.15.50:0/2107265248 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc0840cb8f0 con 0x7fc08c108240 2024-07-04T08:07:43.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:07:43.479+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc08c02de70 con 0x7fc07c0b2840 2024-07-04T08:07:44.190 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi039 2024-07-04T08:07:44.190 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.190 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.190 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.190 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi039 to 4272M 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:07:44.191 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:44.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi039 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi039 to 4272M 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:44.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:07:44.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:07:44.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:07:44.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi039 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi039 to 4272M 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:44.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:07:44.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:44.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:44.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='client.24551 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:07:44.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:07:44.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:07:44.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:46.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:46 smithi133 ceph-mon[37050]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:46.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:46 smithi133 ceph-mon[37050]: from='client.? 172.21.15.50:0/1730748835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "191d4999-048f-4bd9-b484-e20564492846"}]: dispatch 2024-07-04T08:07:46.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:46 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "191d4999-048f-4bd9-b484-e20564492846"}]: dispatch 2024-07-04T08:07:46.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:46 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "191d4999-048f-4bd9-b484-e20564492846"}]': finished 2024-07-04T08:07:46.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:46 smithi133 ceph-mon[37050]: osdmap e64: 11 total, 10 up, 11 in 2024-07-04T08:07:46.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:07:46.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:46 smithi039 ceph-mon[32062]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:46.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:46 smithi039 ceph-mon[32062]: from='client.? 172.21.15.50:0/1730748835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "191d4999-048f-4bd9-b484-e20564492846"}]: dispatch 2024-07-04T08:07:46.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:46 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "191d4999-048f-4bd9-b484-e20564492846"}]: dispatch 2024-07-04T08:07:46.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:46 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "191d4999-048f-4bd9-b484-e20564492846"}]': finished 2024-07-04T08:07:46.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:46 smithi039 ceph-mon[32062]: osdmap e64: 11 total, 10 up, 11 in 2024-07-04T08:07:46.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:07:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:46 smithi050 ceph-mon[36577]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:46 smithi050 ceph-mon[36577]: from='client.? 172.21.15.50:0/1730748835' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "191d4999-048f-4bd9-b484-e20564492846"}]: dispatch 2024-07-04T08:07:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:46 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "191d4999-048f-4bd9-b484-e20564492846"}]: dispatch 2024-07-04T08:07:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:46 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "191d4999-048f-4bd9-b484-e20564492846"}]': finished 2024-07-04T08:07:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:46 smithi050 ceph-mon[36577]: osdmap e64: 11 total, 10 up, 11 in 2024-07-04T08:07:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:07:47.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:47 smithi133 ceph-mon[37050]: from='client.? 172.21.15.50:0/2738213284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:07:47.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:47 smithi039 ceph-mon[32062]: from='client.? 172.21.15.50:0/2738213284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:07:47.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:47 smithi050 ceph-mon[36577]: from='client.? 172.21.15.50:0/2738213284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:07:48.795 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:48 smithi050 ceph-mon[36577]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:48.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:48 smithi133 ceph-mon[37050]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:48.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:48 smithi039 ceph-mon[32062]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:50.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:50 smithi133 ceph-mon[37050]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:50.817 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:50 smithi050 ceph-mon[36577]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:50.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:50 smithi039 ceph-mon[32062]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:52.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:52 smithi133 ceph-mon[37050]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:52.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:52 smithi039 ceph-mon[32062]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:52.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:52 smithi050 ceph-mon[36577]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:53.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-04T08:07:53.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:53.866 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-04T08:07:53.866 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:54.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-04T08:07:54.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:07:54.747 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:54 smithi050 ceph-mon[36577]: Deploying daemon osd.10 on smithi050 2024-07-04T08:07:54.748 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:54 smithi050 ceph-mon[36577]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:54.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:54 smithi039 ceph-mon[32062]: Deploying daemon osd.10 on smithi050 2024-07-04T08:07:54.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:54 smithi039 ceph-mon[32062]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:55.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:54 smithi133 ceph-mon[37050]: Deploying daemon osd.10 on smithi050 2024-07-04T08:07:55.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:54 smithi133 ceph-mon[37050]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:56.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:56 smithi050 ceph-mon[36577]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:56 smithi039 ceph-mon[32062]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:57.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:56 smithi133 ceph-mon[37050]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:58.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:58 smithi050 ceph-mon[36577]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:58.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:58 smithi039 ceph-mon[32062]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:59.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:58 smithi133 ceph-mon[37050]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:07:59.666 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:59.666 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:59.666 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:07:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:59.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:07:59.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:07:59.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:07:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:00.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:00.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:00.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:07:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:00.779 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:00 smithi050 ceph-mon[36577]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:01.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:00 smithi133 ceph-mon[37050]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:00 smithi039 ceph-mon[32062]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:02.636 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 10 on host 'smithi050' 2024-07-04T08:08:02.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.635+0000 7fc089ffb640 1 -- 172.21.15.50:0/2107265248 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc08c02de70 con 0x7fc07c0b2840 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc07c0b2840 msgr2=0x7fc07c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc07c0b2840 0x7fc07c0b4d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc080005fd0 tx=0x7fc080005ec0 comp rx=0 tx=0).stop 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 msgr2=0x7fc08c19b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 0x7fc08c19b260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc084005cc0 tx=0x7fc08402f380 comp rx=0 tx=0).stop 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 shutdown_connections 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc07c0b2840 0x7fc07c0b4d50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc08c1078f0 0x7fc08c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc08c108240 0x7fc08c19b260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 --2- 172.21.15.50:0/2107265248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc08c101fb0 0x7fc08c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 >> 172.21.15.50:0/2107265248 conn(0x7fc08c078c80 msgr2=0x7fc08c103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:02.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 shutdown_connections 2024-07-04T08:08:02.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-07-04T08:08:02.638+0000 7fc092a8c640 1 -- 172.21.15.50:0/2107265248 wait complete. 2024-07-04T08:08:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:02 smithi039 ceph-mon[32062]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:02 smithi050 ceph-mon[36577]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:02 smithi133 ceph-mon[37050]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:03.349 DEBUG:teuthology.orchestra.run.smithi050:osd.10> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.10.service 2024-07-04T08:08:03.351 INFO:tasks.cephadm:Deploying osd.11 on smithi133 with /dev/vg_nvme/lv_1... 2024-07-04T08:08:03.352 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-04T08:08:03.612 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:08:03.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:03 smithi050 ceph-mon[36577]: from='osd.10 [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-04T08:08:03.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:03 smithi050 ceph-mon[36577]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-04T08:08:04.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:03 smithi133 ceph-mon[37050]: from='osd.10 [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-04T08:08:04.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:03 smithi133 ceph-mon[37050]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-04T08:08:04.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:03 smithi039 ceph-mon[32062]: from='osd.10 [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-04T08:08:04.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:03 smithi039 ceph-mon[32062]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-04T08:08:04.996 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:04 smithi133 ceph-mon[37050]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:04.996 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:04 smithi133 ceph-mon[37050]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-04T08:08:04.996 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:04 smithi133 ceph-mon[37050]: from='osd.10 [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:08:04.996 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:04 smithi133 ceph-mon[37050]: osdmap e65: 11 total, 10 up, 11 in 2024-07-04T08:08:04.996 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:04.996 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:04 smithi133 ceph-mon[37050]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:08:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:04 smithi050 ceph-mon[36577]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:04 smithi050 ceph-mon[36577]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-04T08:08:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:04 smithi050 ceph-mon[36577]: from='osd.10 [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:08:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:04 smithi050 ceph-mon[36577]: osdmap e65: 11 total, 10 up, 11 in 2024-07-04T08:08:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:04 smithi050 ceph-mon[36577]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:08:05.146 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:08:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:08:04.877+0000 7fa20be42640 -1 osd.10 0 waiting for initial osdmap 2024-07-04T08:08:05.146 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:08:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:08:04.891+0000 7fa207c41640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:08:05.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:04 smithi039 ceph-mon[32062]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:05.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:04 smithi039 ceph-mon[32062]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-04T08:08:05.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:04 smithi039 ceph-mon[32062]: from='osd.10 [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:08:05.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:04 smithi039 ceph-mon[32062]: osdmap e65: 11 total, 10 up, 11 in 2024-07-04T08:08:05.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:05.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:04 smithi039 ceph-mon[32062]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-07-04T08:08:05.391 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-04T08:08:05.473 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-04T08:08:05.487 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-07-04T08:08:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-07-04T08:08:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127304 s, 824 MB/s 2024-07-04T08:08:05.487 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 2024-07-04T08:08:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-07-04T08:08:06.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:05 smithi133 ceph-mon[37050]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:08:06.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:05 smithi133 ceph-mon[37050]: osdmap e66: 11 total, 10 up, 11 in 2024-07-04T08:08:06.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:05 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:06.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:05 smithi133 ceph-mon[37050]: osd.10 [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] boot 2024-07-04T08:08:06.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:05 smithi039 ceph-mon[32062]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:08:06.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:05 smithi039 ceph-mon[32062]: osdmap e66: 11 total, 10 up, 11 in 2024-07-04T08:08:06.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:05 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:06.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:05 smithi039 ceph-mon[32062]: osd.10 [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] boot 2024-07-04T08:08:06.102 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2024-07-04T08:08:06.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:05 smithi050 ceph-mon[36577]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-07-04T08:08:06.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:05 smithi050 ceph-mon[36577]: osdmap e66: 11 total, 10 up, 11 in 2024-07-04T08:08:06.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:05 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:06.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:05 smithi050 ceph-mon[36577]: osd.10 [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] boot 2024-07-04T08:08:06.356 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:08:07.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:08:07.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: osdmap e67: 11 total, 11 up, 11 in 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:07.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:07.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:06 smithi133 ceph-mon[37050]: osdmap e68: 11 total, 11 up, 11 in 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: osdmap e67: 11 total, 11 up, 11 in 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:07.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:07.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.353 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:06 smithi039 ceph-mon[32062]: osdmap e68: 11 total, 11 up, 11 in 2024-07-04T08:08:07.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:08:07.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:08:07.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-07-04T08:08:07.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: osdmap e67: 11 total, 11 up, 11 in 2024-07-04T08:08:07.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:07.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-04T08:08:07.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:07.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:07.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:07.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:07.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:06 smithi050 ceph-mon[36577]: osdmap e68: 11 total, 11 up, 11 in 2024-07-04T08:08:08.036 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:07 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi050 2024-07-04T08:08:08.036 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:07 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi050 to 4273M 2024-07-04T08:08:08.037 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:07 smithi133 ceph-mon[37050]: osdmap e69: 11 total, 11 up, 11 in 2024-07-04T08:08:08.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 -- 172.21.15.133:0/3793000192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de01089f0 msgr2=0x7f6de0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:08.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 --2- 172.21.15.133:0/3793000192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de01089f0 0x7f6de0108e00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009900 tx=0x7f6dd402a410 comp rx=0 tx=0).stop 2024-07-04T08:08:08.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 -- 172.21.15.133:0/3793000192 shutdown_connections 2024-07-04T08:08:08.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 --2- 172.21.15.133:0/3793000192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6de0109340 0x7f6de010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:08.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 --2- 172.21.15.133:0/3793000192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de01089f0 0x7f6de0108e00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:08.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 --2- 172.21.15.133:0/3793000192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de01030b0 0x7f6de01034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:08.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 -- 172.21.15.133:0/3793000192 >> 172.21.15.133:0/3793000192 conn(0x7f6de007ada0 msgr2=0x7f6de007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:08.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 -- 172.21.15.133:0/3793000192 shutdown_connections 2024-07-04T08:08:08.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 -- 172.21.15.133:0/3793000192 wait complete. 2024-07-04T08:08:08.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 Processor -- start 2024-07-04T08:08:08.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.027+0000 7f6de636c640 1 -- start start 2024-07-04T08:08:08.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de636c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6de01030b0 0x7f6de0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:08.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de636c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de01089f0 0x7f6de019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:08.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de636c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de0109340 0x7f6de019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:08.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de636c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01130a0 con 0x7f6de01089f0 2024-07-04T08:08:08.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de636c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0112f20 con 0x7f6de01030b0 2024-07-04T08:08:08.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de636c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6de0113220 con 0x7f6de0109340 2024-07-04T08:08:08.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de4909640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de0109340 0x7f6de019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:08.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de4909640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de0109340 0x7f6de019f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44690/0 (socket says 172.21.15.133:44690) 2024-07-04T08:08:08.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de4909640 1 -- 172.21.15.133:0/1874206986 learned_addr learned my addr 172.21.15.133:0/1874206986 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-07-04T08:08:08.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6ddffff640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6de01030b0 0x7f6de0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de4909640 1 -- 172.21.15.133:0/1874206986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6de01030b0 msgr2=0x7f6de0199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de4909640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6de01030b0 0x7f6de0199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de4909640 1 -- 172.21.15.133:0/1874206986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de01089f0 msgr2=0x7f6de019a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de4909640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de01089f0 0x7f6de019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de4909640 1 -- 172.21.15.133:0/1874206986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de019fb40 con 0x7f6de0109340 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.028+0000 7f6de4909640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de0109340 0x7f6de019f410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800b970 tx=0x7f6dd800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.029+0000 7f6ddd7fa640 1 -- 172.21.15.133:0/1874206986 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8019070 con 0x7f6de0109340 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.029+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de019fe30 con 0x7f6de0109340 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.029+0000 7f6ddd7fa640 1 -- 172.21.15.133:0/1874206986 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dd8014440 con 0x7f6de0109340 2024-07-04T08:08:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.029+0000 7f6ddd7fa640 1 -- 172.21.15.133:0/1874206986 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80075d0 con 0x7f6de0109340 2024-07-04T08:08:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.029+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de01a6590 con 0x7f6de0109340 2024-07-04T08:08:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.030+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de0103540 con 0x7f6de0109340 2024-07-04T08:08:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.031+0000 7f6ddd7fa640 1 -- 172.21.15.133:0/1874206986 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f6dd801c070 con 0x7f6de0109340 2024-07-04T08:08:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.032+0000 7f6ddd7fa640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6dc80b2840 0x7f6dc80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.032+0000 7f6ddd7fa640 1 -- 172.21.15.133:0/1874206986 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6909+0+0 (secure 0 0 0) 0x7f6dd80e1710 con 0x7f6de0109340 2024-07-04T08:08:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.032+0000 7f6ddffff640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6dc80b2840 0x7f6dc80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.033+0000 7f6ddffff640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6dc80b2840 0x7f6dc80b4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009990 tx=0x7f6dd0009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.036+0000 7f6ddd7fa640 1 -- 172.21.15.133:0/1874206986 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f6dd80a99b0 con 0x7f6de0109340 2024-07-04T08:08:08.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:08.239+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6de0062d50 con 0x7f6dc80b2840 2024-07-04T08:08:08.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:07 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi050 2024-07-04T08:08:08.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:07 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi050 to 4273M 2024-07-04T08:08:08.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:07 smithi039 ceph-mon[32062]: osdmap e69: 11 total, 11 up, 11 in 2024-07-04T08:08:08.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:07 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi050 2024-07-04T08:08:08.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:07 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi050 to 4273M 2024-07-04T08:08:08.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:07 smithi050 ceph-mon[36577]: osdmap e69: 11 total, 11 up, 11 in 2024-07-04T08:08:09.007 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:08 smithi133 ceph-mon[37050]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:09.007 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:08:09.007 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:08:09.007 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:09.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:08 smithi039 ceph-mon[32062]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:09.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:08:09.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:08:09.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:09.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:08 smithi050 ceph-mon[36577]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:09.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-04T08:08:09.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-04T08:08:09.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:10.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:09 smithi133 ceph-mon[37050]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:08:10.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:09 smithi039 ceph-mon[32062]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:08:10.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:09 smithi050 ceph-mon[36577]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:08:11.180 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:10 smithi133 ceph-mon[37050]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:11.181 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:10 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/3783286713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3dddd70-6e8a-424f-a69d-7c2b41575aae"}]: dispatch 2024-07-04T08:08:11.181 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:10 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3dddd70-6e8a-424f-a69d-7c2b41575aae"}]: dispatch 2024-07-04T08:08:11.181 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:10 smithi133 ceph-mon[37050]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3dddd70-6e8a-424f-a69d-7c2b41575aae"}]': finished 2024-07-04T08:08:11.181 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:10 smithi133 ceph-mon[37050]: osdmap e70: 12 total, 11 up, 12 in 2024-07-04T08:08:11.181 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:11.291 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:10 smithi039 ceph-mon[32062]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:11.291 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:10 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/3783286713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3dddd70-6e8a-424f-a69d-7c2b41575aae"}]: dispatch 2024-07-04T08:08:11.291 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:10 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3dddd70-6e8a-424f-a69d-7c2b41575aae"}]: dispatch 2024-07-04T08:08:11.291 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:10 smithi039 ceph-mon[32062]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3dddd70-6e8a-424f-a69d-7c2b41575aae"}]': finished 2024-07-04T08:08:11.291 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:10 smithi039 ceph-mon[32062]: osdmap e70: 12 total, 11 up, 12 in 2024-07-04T08:08:11.291 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:11.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:10 smithi050 ceph-mon[36577]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:10 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/3783286713' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3dddd70-6e8a-424f-a69d-7c2b41575aae"}]: dispatch 2024-07-04T08:08:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:10 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3dddd70-6e8a-424f-a69d-7c2b41575aae"}]: dispatch 2024-07-04T08:08:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:10 smithi050 ceph-mon[36577]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3dddd70-6e8a-424f-a69d-7c2b41575aae"}]': finished 2024-07-04T08:08:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:10 smithi050 ceph-mon[36577]: osdmap e70: 12 total, 11 up, 12 in 2024-07-04T08:08:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:12.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:11 smithi133 ceph-mon[37050]: from='client.? 172.21.15.133:0/2103218250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:08:12.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:11 smithi039 ceph-mon[32062]: from='client.? 172.21.15.133:0/2103218250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:08:12.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:11 smithi050 ceph-mon[36577]: from='client.? 172.21.15.133:0/2103218250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-04T08:08:13.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:12 smithi133 ceph-mon[37050]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-07-04T08:08:13.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:12 smithi039 ceph-mon[32062]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-07-04T08:08:13.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:12 smithi050 ceph-mon[36577]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-07-04T08:08:15.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:14 smithi133 ceph-mon[37050]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-07-04T08:08:15.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:14 smithi039 ceph-mon[32062]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-07-04T08:08:15.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:14 smithi050 ceph-mon[36577]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-07-04T08:08:17.226 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:16 smithi133 ceph-mon[37050]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-04T08:08:17.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:16 smithi039 ceph-mon[32062]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-04T08:08:17.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:16 smithi050 ceph-mon[36577]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-04T08:08:18.296 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-04T08:08:18.296 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:17 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:18.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-04T08:08:18.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:17 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:18.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-04T08:08:18.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:17 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:19.241 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:18 smithi133 ceph-mon[37050]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-07-04T08:08:19.241 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:18 smithi133 ceph-mon[37050]: Deploying daemon osd.11 on smithi133 2024-07-04T08:08:19.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:18 smithi039 ceph-mon[32062]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-07-04T08:08:19.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:18 smithi039 ceph-mon[32062]: Deploying daemon osd.11 on smithi133 2024-07-04T08:08:19.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:18 smithi050 ceph-mon[36577]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-07-04T08:08:19.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:18 smithi050 ceph-mon[36577]: Deploying daemon osd.11 on smithi133 2024-07-04T08:08:21.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:20 smithi133 ceph-mon[37050]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:08:21.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:20 smithi039 ceph-mon[32062]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:08:21.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:20 smithi050 ceph-mon[36577]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-04T08:08:23.278 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:22 smithi133 ceph-mon[37050]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-07-04T08:08:23.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:22 smithi039 ceph-mon[32062]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-07-04T08:08:23.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:22 smithi050 ceph-mon[36577]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-07-04T08:08:24.273 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:24.273 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:24.273 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:24.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:24.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:24.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:24.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:24.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:24.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:25.290 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:25 smithi133 ceph-mon[37050]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:25.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:25 smithi039 ceph-mon[32062]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:25.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:25 smithi050 ceph-mon[36577]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:27 smithi133 ceph-mon[37050]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:27.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:27.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:27.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:27.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:27.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.200+0000 7f6ddd7fa640 1 -- 172.21.15.133:0/1874206986 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6de0062d50 con 0x7f6dc80b2840 2024-07-04T08:08:27.201 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 11 on host 'smithi133' 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6dc80b2840 msgr2=0x7f6dc80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6dc80b2840 0x7f6dc80b4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009990 tx=0x7f6dd0009110 comp rx=0 tx=0).stop 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de0109340 msgr2=0x7f6de019f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de0109340 0x7f6de019f410 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800b970 tx=0x7f6dd800bd70 comp rx=0 tx=0).stop 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 shutdown_connections 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6dc80b2840 0x7f6dc80b4d50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de01089f0 0x7f6de019a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6de0109340 0x7f6de019f410 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 --2- 172.21.15.133:0/1874206986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6de01030b0 0x7f6de0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.202+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 >> 172.21.15.133:0/1874206986 conn(0x7f6de007ada0 msgr2=0x7f6de0106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:27.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.203+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 shutdown_connections 2024-07-04T08:08:27.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:27.203+0000 7f6de636c640 1 -- 172.21.15.133:0/1874206986 wait complete. 2024-07-04T08:08:27.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:27 smithi039 ceph-mon[32062]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:27.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:27.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:27.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:27.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:27.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:27.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:27 smithi050 ceph-mon[36577]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:27.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:27.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:27.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:27.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:27.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:27.867 DEBUG:teuthology.orchestra.run.smithi133:osd.11> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.11.service 2024-07-04T08:08:27.869 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-07-04T08:08:27.870 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd stat -f json 2024-07-04T08:08:28.216 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:28.216 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:28.216 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:28.216 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:28 smithi133 ceph-mon[37050]: from='osd.11 [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-04T08:08:28.216 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:28 smithi133 ceph-mon[37050]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-04T08:08:28.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:28.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:28.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:28.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:28 smithi039 ceph-mon[32062]: from='osd.11 [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-04T08:08:28.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:28 smithi039 ceph-mon[32062]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-04T08:08:28.358 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:08:28.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:28.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:28.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:28.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:28 smithi050 ceph-mon[36577]: from='osd.11 [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-04T08:08:28.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:28 smithi050 ceph-mon[36577]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-04T08:08:29.045 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:29 smithi133 ceph-mon[37050]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:29.045 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:29 smithi133 ceph-mon[37050]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-04T08:08:29.045 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:29 smithi133 ceph-mon[37050]: osdmap e71: 12 total, 11 up, 12 in 2024-07-04T08:08:29.045 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:29 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:29.046 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:29 smithi133 ceph-mon[37050]: from='osd.11 [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:08:29.046 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:29 smithi133 ceph-mon[37050]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:08:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:29 smithi039 ceph-mon[32062]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:29 smithi039 ceph-mon[32062]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-04T08:08:29.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:29 smithi039 ceph-mon[32062]: osdmap e71: 12 total, 11 up, 12 in 2024-07-04T08:08:29.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:29 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:29.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:29 smithi039 ceph-mon[32062]: from='osd.11 [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:08:29.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:29 smithi039 ceph-mon[32062]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:08:29.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:29 smithi050 ceph-mon[36577]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:29.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:29 smithi050 ceph-mon[36577]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-04T08:08:29.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:29 smithi050 ceph-mon[36577]: osdmap e71: 12 total, 11 up, 12 in 2024-07-04T08:08:29.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:29 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:29.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:29 smithi050 ceph-mon[36577]: from='osd.11 [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:08:29.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:29 smithi050 ceph-mon[36577]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-07-04T08:08:30.067 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:08:29 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:08:29.631+0000 7f7f52a00640 -1 osd.11 0 waiting for initial osdmap 2024-07-04T08:08:30.067 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:08:29 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:08:29.651+0000 7f7f49fe2640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-04T08:08:30.428 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:30 smithi133 ceph-mon[37050]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:08:30.428 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:30 smithi133 ceph-mon[37050]: osdmap e72: 12 total, 11 up, 12 in 2024-07-04T08:08:30.428 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:30 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:30.428 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:30 smithi133 ceph-mon[37050]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:30.429 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:30 smithi133 ceph-mon[37050]: from='osd.11 ' entity='osd.11' 2024-07-04T08:08:30.499 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:30 smithi039 ceph-mon[32062]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:08:30.499 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:30 smithi039 ceph-mon[32062]: osdmap e72: 12 total, 11 up, 12 in 2024-07-04T08:08:30.499 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:30 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:30.499 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:30 smithi039 ceph-mon[32062]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:30.499 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:30 smithi039 ceph-mon[32062]: from='osd.11 ' entity='osd.11' 2024-07-04T08:08:30.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:30 smithi050 ceph-mon[36577]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-07-04T08:08:30.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:30 smithi050 ceph-mon[36577]: osdmap e72: 12 total, 11 up, 12 in 2024-07-04T08:08:30.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:30 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:30.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:30 smithi050 ceph-mon[36577]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-07-04T08:08:30.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:30 smithi050 ceph-mon[36577]: from='osd.11 ' entity='osd.11' 2024-07-04T08:08:30.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.712+0000 7f7e41a00640 1 -- 172.21.15.39:0/533195971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c103bc0 msgr2=0x7f7e3c103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:30.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.712+0000 7f7e41a00640 1 --2- 172.21.15.39:0/533195971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c103bc0 0x7f7e3c103fd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009900 tx=0x7f7e2c02a410 comp rx=0 tx=0).stop 2024-07-04T08:08:30.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.712+0000 7f7e41a00640 1 -- 172.21.15.39:0/533195971 shutdown_connections 2024-07-04T08:08:30.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.712+0000 7f7e41a00640 1 --2- 172.21.15.39:0/533195971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e3c10d170 0x7f7e3c10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.712+0000 7f7e41a00640 1 --2- 172.21.15.39:0/533195971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c1045f0 0x7f7e3c10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.712+0000 7f7e41a00640 1 --2- 172.21.15.39:0/533195971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c103bc0 0x7f7e3c103fd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.712+0000 7f7e41a00640 1 -- 172.21.15.39:0/533195971 >> 172.21.15.39:0/533195971 conn(0x7f7e3c0fd810 msgr2=0x7f7e3c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:30.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.712+0000 7f7e41a00640 1 -- 172.21.15.39:0/533195971 shutdown_connections 2024-07-04T08:08:30.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.712+0000 7f7e41a00640 1 -- 172.21.15.39:0/533195971 wait complete. 2024-07-04T08:08:30.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.713+0000 7f7e41a00640 1 Processor -- start 2024-07-04T08:08:30.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.714+0000 7f7e41a00640 1 -- start start 2024-07-04T08:08:30.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.714+0000 7f7e41a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c103bc0 0x7f7e3c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:30.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.714+0000 7f7e41a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e3c1045f0 0x7f7e3c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.714+0000 7f7e41a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c10d170 0x7f7e3c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.714+0000 7f7e41a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c112dd0 con 0x7f7e3c103bc0 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.714+0000 7f7e41a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c112c50 con 0x7f7e3c10d170 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.714+0000 7f7e41a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c112f50 con 0x7f7e3c1045f0 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.714+0000 7f7e3be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c103bc0 0x7f7e3c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e3be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c103bc0 0x7f7e3c199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48636/0 (socket says 172.21.15.39:48636) 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e3be00640 1 -- 172.21.15.39:0/4138022019 learned_addr learned my addr 172.21.15.39:0/4138022019 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e41000640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c10d170 0x7f7e3c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e33e00640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e3c1045f0 0x7f7e3c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e41000640 1 -- 172.21.15.39:0/4138022019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e3c1045f0 msgr2=0x7f7e3c199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:30.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e41000640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e3c1045f0 0x7f7e3c199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e41000640 1 -- 172.21.15.39:0/4138022019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c103bc0 msgr2=0x7f7e3c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e41000640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c103bc0 0x7f7e3c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e41000640 1 -- 172.21.15.39:0/4138022019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c19f770 con 0x7f7e3c10d170 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.715+0000 7f7e41000640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c10d170 0x7f7e3c19f040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009fd0 tx=0x7f7e3400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.716+0000 7f7e39600640 1 -- 172.21.15.39:0/4138022019 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400ce20 con 0x7f7e3c10d170 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.716+0000 7f7e39600640 1 -- 172.21.15.39:0/4138022019 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e34005020 con 0x7f7e3c10d170 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.716+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c19fa60 con 0x7f7e3c10d170 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.716+0000 7f7e39600640 1 -- 172.21.15.39:0/4138022019 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34010690 con 0x7f7e3c10d170 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.716+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c1a61c0 con 0x7f7e3c10d170 2024-07-04T08:08:30.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.717+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3c103fd0 con 0x7f7e3c10d170 2024-07-04T08:08:30.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.719+0000 7f7e39600640 1 -- 172.21.15.39:0/4138022019 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f7e340054d0 con 0x7f7e3c10d170 2024-07-04T08:08:30.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.721+0000 7f7e39600640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7e200b2840 0x7f7e200b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:30.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.721+0000 7f7e39600640 1 -- 172.21.15.39:0/4138022019 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7342+0+0 (secure 0 0 0) 0x7f7e340e25d0 con 0x7f7e3c10d170 2024-07-04T08:08:30.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.721+0000 7f7e3be00640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7e200b2840 0x7f7e200b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:30.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.722+0000 7f7e3be00640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7e200b2840 0x7f7e200b4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c031440 tx=0x7f7e2c0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:30.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.725+0000 7f7e39600640 1 -- 172.21.15.39:0/4138022019 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f7e340aa620 con 0x7f7e3c10d170 2024-07-04T08:08:30.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.949+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7e3c19a930 con 0x7f7e3c10d170 2024-07-04T08:08:30.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.949+0000 7f7e39600640 1 -- 172.21.15.39:0/4138022019 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f7e340a9f20 con 0x7f7e3c10d170 2024-07-04T08:08:30.952 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:08:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.952+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7e200b2840 msgr2=0x7f7e200b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7e200b2840 0x7f7e200b4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c031440 tx=0x7f7e2c0058d0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c10d170 msgr2=0x7f7e3c19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c10d170 0x7f7e3c19f040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009fd0 tx=0x7f7e3400ef10 comp rx=0 tx=0).stop 2024-07-04T08:08:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 shutdown_connections 2024-07-04T08:08:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7e200b2840 0x7f7e200b4d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7e3c10d170 0x7f7e3c19f040 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e3c1045f0 0x7f7e3c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 --2- 172.21.15.39:0/4138022019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c103bc0 0x7f7e3c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:30.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 >> 172.21.15.39:0/4138022019 conn(0x7f7e3c0fd810 msgr2=0x7f7e3c105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:30.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 shutdown_connections 2024-07-04T08:08:30.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:30.953+0000 7f7e41a00640 1 -- 172.21.15.39:0/4138022019 wait complete. 2024-07-04T08:08:31.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: purged_snaps scrub starts 2024-07-04T08:08:31.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: purged_snaps scrub ok 2024-07-04T08:08:31.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:31.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: osd.11 [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] boot 2024-07-04T08:08:31.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: osdmap e73: 12 total, 12 up, 12 in 2024-07-04T08:08:31.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:31.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: Detected new or changed devices on smithi133 2024-07-04T08:08:31.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi133 to 4272M 2024-07-04T08:08:31.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:31.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:31.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.569 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:31 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/4138022019' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-04T08:08:31.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: purged_snaps scrub starts 2024-07-04T08:08:31.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: purged_snaps scrub ok 2024-07-04T08:08:31.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:31.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: osd.11 [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] boot 2024-07-04T08:08:31.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: osdmap e73: 12 total, 12 up, 12 in 2024-07-04T08:08:31.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:31.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: Detected new or changed devices on smithi133 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi133 to 4272M 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.603 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:31 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/4138022019' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-04T08:08:31.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: purged_snaps scrub starts 2024-07-04T08:08:31.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: purged_snaps scrub ok 2024-07-04T08:08:31.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:31.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: osd.11 [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] boot 2024-07-04T08:08:31.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: osdmap e73: 12 total, 12 up, 12 in 2024-07-04T08:08:31.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-04T08:08:31.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: Detected new or changed devices on smithi133 2024-07-04T08:08:31.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.646 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.647 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:08:31.647 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi133 to 4272M 2024-07-04T08:08:31.649 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.649 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:31.650 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:31.650 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:31.650 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:31 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/4138022019' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-04T08:08:31.778 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1720080510,"num_in_osds":12,"osd_in_since":1720080490,"num_remapped_pgs":0} 2024-07-04T08:08:31.779 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:08:32.264 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:08:32.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:32 smithi133 ceph-mon[37050]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:32.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:32 smithi039 ceph-mon[32062]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:32.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:32 smithi050 ceph-mon[36577]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:33.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:33 smithi133 ceph-mon[37050]: osdmap e74: 12 total, 12 up, 12 in 2024-07-04T08:08:33.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:33 smithi039 ceph-mon[32062]: osdmap e74: 12 total, 12 up, 12 in 2024-07-04T08:08:33.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:33 smithi050 ceph-mon[36577]: osdmap e74: 12 total, 12 up, 12 in 2024-07-04T08:08:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.546+0000 7f2ed3e00640 1 -- 172.21.15.39:0/2301937911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4109190 msgr2=0x7f2ed410f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.546+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/2301937911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4109190 0x7f2ed410f9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009860 tx=0x7f2ecc02a340 comp rx=0 tx=0).stop 2024-07-04T08:08:34.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:34 smithi039 ceph-mon[32062]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.546+0000 7f2ed3e00640 1 -- 172.21.15.39:0/2301937911 shutdown_connections 2024-07-04T08:08:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.546+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/2301937911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4109190 0x7f2ed410f9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.546+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/2301937911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ed4108840 0x7f2ed4108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.546+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/2301937911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ed4102f00 0x7f2ed4103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.546+0000 7f2ed3e00640 1 -- 172.21.15.39:0/2301937911 >> 172.21.15.39:0/2301937911 conn(0x7f2ed40fe950 msgr2=0x7f2ed4100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.546+0000 7f2ed3e00640 1 -- 172.21.15.39:0/2301937911 shutdown_connections 2024-07-04T08:08:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.546+0000 7f2ed3e00640 1 -- 172.21.15.39:0/2301937911 wait complete. 2024-07-04T08:08:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.547+0000 7f2ed3e00640 1 Processor -- start 2024-07-04T08:08:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.548+0000 7f2ed3e00640 1 -- start start 2024-07-04T08:08:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.548+0000 7f2ed3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4102f00 0x7f2ed4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.548+0000 7f2ed3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ed4108840 0x7f2ed40751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.548+0000 7f2ed3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ed4109190 0x7f2ed40756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.548+0000 7f2ed3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4112e50 con 0x7f2ed4109190 2024-07-04T08:08:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.548+0000 7f2ed3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4112cd0 con 0x7f2ed4108840 2024-07-04T08:08:34.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.548+0000 7f2ed3e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4112fd0 con 0x7f2ed4102f00 2024-07-04T08:08:34.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.548+0000 7f2ed2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4102f00 0x7f2ed4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:34.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4102f00 0x7f2ed4074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:46110/0 (socket says 172.21.15.39:46110) 2024-07-04T08:08:34.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed2a00640 1 -- 172.21.15.39:0/1390067797 learned_addr learned my addr 172.21.15.39:0/1390067797 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed3400640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ed4109190 0x7f2ed40756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed2000640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ed4108840 0x7f2ed40751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed2a00640 1 -- 172.21.15.39:0/1390067797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ed4108840 msgr2=0x7f2ed40751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed2a00640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ed4108840 0x7f2ed40751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed2a00640 1 -- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ed4109190 msgr2=0x7f2ed40756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed2a00640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ed4109190 0x7f2ed40756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed2a00640 1 -- 172.21.15.39:0/1390067797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed41a7a60 con 0x7f2ed4102f00 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.549+0000 7f2ed2a00640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4102f00 0x7f2ed4074c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400e9c0 tx=0x7f2ec400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.550+0000 7f2ecb400640 1 -- 172.21.15.39:0/1390067797 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec400cde0 con 0x7f2ed4102f00 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.550+0000 7f2ecb400640 1 -- 172.21.15.39:0/1390067797 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec40035f0 con 0x7f2ed4102f00 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.550+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed41a7d50 con 0x7f2ed4102f00 2024-07-04T08:08:34.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.550+0000 7f2ecb400640 1 -- 172.21.15.39:0/1390067797 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4010690 con 0x7f2ed4102f00 2024-07-04T08:08:34.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.550+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed41a8250 con 0x7f2ed4102f00 2024-07-04T08:08:34.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.551+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed4103310 con 0x7f2ed4102f00 2024-07-04T08:08:34.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.554+0000 7f2ecb400640 1 -- 172.21.15.39:0/1390067797 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f2ec4004000 con 0x7f2ed4102f00 2024-07-04T08:08:34.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.555+0000 7f2ecb400640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2eb80b28c0 0x7f2eb80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:34.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.555+0000 7f2ecb400640 1 -- 172.21.15.39:0/1390067797 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f2ec40e21b0 con 0x7f2ed4102f00 2024-07-04T08:08:34.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.556+0000 7f2ed2000640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2eb80b28c0 0x7f2eb80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:34.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.556+0000 7f2ed2000640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2eb80b28c0 0x7f2eb80b4dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ed41081b0 tx=0x7f2ec0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:34.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.557+0000 7f2ecb400640 1 -- 172.21.15.39:0/1390067797 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f2ec40aa200 con 0x7f2ed4102f00 2024-07-04T08:08:34.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:34 smithi133 ceph-mon[37050]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:34.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:34 smithi050 ceph-mon[36577]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.771+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ed4075f60 con 0x7f2ed4102f00 2024-07-04T08:08:34.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.773+0000 7f2ecb400640 1 -- 172.21.15.39:0/1390067797 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18712 (secure 0 0 0) 0x7f2ed4075f60 con 0x7f2ed4102f00 2024-07-04T08:08:34.775 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:08:34.775 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":74,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:08:32.227985+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:08:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2eb80b28c0 msgr2=0x7f2eb80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2eb80b28c0 0x7f2eb80b4dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ed41081b0 tx=0x7f2ec0009500 comp rx=0 tx=0).stop 2024-07-04T08:08:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4102f00 msgr2=0x7f2ed4074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:34.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4102f00 0x7f2ed4074c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400e9c0 tx=0x7f2ec400ee80 comp rx=0 tx=0).stop 2024-07-04T08:08:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 shutdown_connections 2024-07-04T08:08:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2eb80b28c0 0x7f2eb80b4dd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ed4109190 0x7f2ed40756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ed4108840 0x7f2ed40751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 --2- 172.21.15.39:0/1390067797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed4102f00 0x7f2ed4074c60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 >> 172.21.15.39:0/1390067797 conn(0x7f2ed40fe950 msgr2=0x7f2ed4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.776+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 shutdown_connections 2024-07-04T08:08:34.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:34.777+0000 7f2ed3e00640 1 -- 172.21.15.39:0/1390067797 wait complete. 2024-07-04T08:08:35.543 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:35 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1390067797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:08:35.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:35 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1390067797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:08:35.635 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-04T08:04:17.509668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-04T08:08:35.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-07-04T08:08:35.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:35 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1390067797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:08:36.123 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:08:36.540 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:36 smithi039 ceph-mon[32062]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:36.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:36 smithi133 ceph-mon[37050]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:36.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:36 smithi050 ceph-mon[36577]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:38.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.466+0000 7fce5c600640 1 -- 172.21.15.39:0/3823848215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce54113410 msgr2=0x7fce54113820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.466+0000 7fce5c600640 1 --2- 172.21.15.39:0/3823848215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce54113410 0x7fce54113820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce40009900 tx=0x7fce4002a410 comp rx=0 tx=0).stop 2024-07-04T08:08:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.466+0000 7fce5c600640 1 -- 172.21.15.39:0/3823848215 shutdown_connections 2024-07-04T08:08:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.466+0000 7fce5c600640 1 --2- 172.21.15.39:0/3823848215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce54113410 0x7fce54113820 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.466+0000 7fce5c600640 1 --2- 172.21.15.39:0/3823848215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce54113d60 0x7fce5411a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.466+0000 7fce5c600640 1 --2- 172.21.15.39:0/3823848215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce5410dad0 0x7fce5410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.466+0000 7fce5c600640 1 -- 172.21.15.39:0/3823848215 >> 172.21.15.39:0/3823848215 conn(0x7fce54109910 msgr2=0x7fce5410bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.466+0000 7fce5c600640 1 -- 172.21.15.39:0/3823848215 shutdown_connections 2024-07-04T08:08:38.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.466+0000 7fce5c600640 1 -- 172.21.15.39:0/3823848215 wait complete. 2024-07-04T08:08:38.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.468+0000 7fce5c600640 1 Processor -- start 2024-07-04T08:08:38.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.468+0000 7fce5c600640 1 -- start start 2024-07-04T08:08:38.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce5c600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5410dad0 0x7fce541a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:38.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce5c600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce54113410 0x7fce541a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:38.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce5c600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce54113d60 0x7fce541a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce5c600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce5411da20 con 0x7fce5410dad0 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce5c600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce5411d8a0 con 0x7fce54113410 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce5c600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fce5411dba0 con 0x7fce54113d60 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce59c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5410dad0 0x7fce541a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce59c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5410dad0 0x7fce541a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53496/0 (socket says 172.21.15.39:53496) 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce59200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce54113410 0x7fce541a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce59c00640 1 -- 172.21.15.39:0/1712609483 learned_addr learned my addr 172.21.15.39:0/1712609483 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.469+0000 7fce5a600640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce54113d60 0x7fce541a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce59c00640 1 -- 172.21.15.39:0/1712609483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce54113d60 msgr2=0x7fce541a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce59c00640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce54113d60 0x7fce541a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce59c00640 1 -- 172.21.15.39:0/1712609483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce54113410 msgr2=0x7fce541a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:38.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce59c00640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce54113410 0x7fce541a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:38.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce59c00640 1 -- 172.21.15.39:0/1712609483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce541aa3a0 con 0x7fce5410dad0 2024-07-04T08:08:38.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce59c00640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5410dad0 0x7fce541a4430 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce4800ea60 tx=0x7fce4800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:38.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce52000640 1 -- 172.21.15.39:0/1712609483 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4800cde0 con 0x7fce5410dad0 2024-07-04T08:08:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce52000640 1 -- 172.21.15.39:0/1712609483 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce480035f0 con 0x7fce5410dad0 2024-07-04T08:08:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce541aa690 con 0x7fce5410dad0 2024-07-04T08:08:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce52000640 1 -- 172.21.15.39:0/1712609483 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48010690 con 0x7fce5410dad0 2024-07-04T08:08:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.470+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce541b0df0 con 0x7fce5410dad0 2024-07-04T08:08:38.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.472+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5410dee0 con 0x7fce5410dad0 2024-07-04T08:08:38.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.474+0000 7fce52000640 1 -- 172.21.15.39:0/1712609483 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fce480037a0 con 0x7fce5410dad0 2024-07-04T08:08:38.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.476+0000 7fce52000640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce3c0b2840 0x7fce3c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:38.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.476+0000 7fce52000640 1 -- 172.21.15.39:0/1712609483 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7fce480e23e0 con 0x7fce5410dad0 2024-07-04T08:08:38.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.476+0000 7fce59200640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce3c0b2840 0x7fce3c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:38.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.477+0000 7fce59200640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce3c0b2840 0x7fce3c0b4d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce40009230 tx=0x7fce40031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:38.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.478+0000 7fce52000640 1 -- 172.21.15.39:0/1712609483 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fce48055110 con 0x7fce5410dad0 2024-07-04T08:08:38.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.698+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fce5402de10 con 0x7fce5410dad0 2024-07-04T08:08:38.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.699+0000 7fce52000640 1 -- 172.21.15.39:0/1712609483 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fce480aa430 con 0x7fce5410dad0 2024-07-04T08:08:38.701 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-07-04T08:08:38.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce3c0b2840 msgr2=0x7fce3c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:38.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce3c0b2840 0x7fce3c0b4d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce40009230 tx=0x7fce40031400 comp rx=0 tx=0).stop 2024-07-04T08:08:38.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5410dad0 msgr2=0x7fce541a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:38.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5410dad0 0x7fce541a4430 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce4800ea60 tx=0x7fce4800ef20 comp rx=0 tx=0).stop 2024-07-04T08:08:38.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 shutdown_connections 2024-07-04T08:08:38.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce3c0b2840 0x7fce3c0b4d50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:38.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce54113d60 0x7fce541a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:38.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce54113410 0x7fce541a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:38.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 --2- 172.21.15.39:0/1712609483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5410dad0 0x7fce541a4430 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:38.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 >> 172.21.15.39:0/1712609483 conn(0x7fce54109910 msgr2=0x7fce54112a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:38.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.702+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 shutdown_connections 2024-07-04T08:08:38.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:38.703+0000 7fce5c600640 1 -- 172.21.15.39:0/1712609483 wait complete. 2024-07-04T08:08:38.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:38 smithi133 ceph-mon[37050]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:38.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:38 smithi039 ceph-mon[32062]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:38.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:38 smithi050 ceph-mon[36577]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.a on smithi039 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.d on smithi039 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.g on smithi039 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.j on smithi039 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.b on smithi050 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.e on smithi050 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.h on smithi050 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.k on smithi050 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.c on smithi133 2024-07-04T08:08:39.497 INFO:tasks.cephadm:Adding mds.f on smithi133 2024-07-04T08:08:39.498 INFO:tasks.cephadm:Adding mds.i on smithi133 2024-07-04T08:08:39.498 INFO:tasks.cephadm:Adding mds.l on smithi133 2024-07-04T08:08:39.498 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph orch apply mds all '12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l' 2024-07-04T08:08:39.758 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:08:39.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:39 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1712609483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-04T08:08:39.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:39 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1712609483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-04T08:08:39.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:39 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1712609483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-04T08:08:40.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:40 smithi133 ceph-mon[37050]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:40.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:40 smithi039 ceph-mon[32062]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:40 smithi050 ceph-mon[36577]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.431+0000 7f9af11e6640 1 -- 172.21.15.133:0/66115901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 msgr2=0x7f9aec102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.431+0000 7f9af11e6640 1 --2- 172.21.15.133:0/66115901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 0x7f9aec102450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9adc009900 tx=0x7f9adc02a410 comp rx=0 tx=0).stop 2024-07-04T08:08:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.431+0000 7f9af11e6640 1 -- 172.21.15.133:0/66115901 shutdown_connections 2024-07-04T08:08:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.431+0000 7f9af11e6640 1 --2- 172.21.15.133:0/66115901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aec1082d0 0x7f9aec10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.431+0000 7f9af11e6640 1 --2- 172.21.15.133:0/66115901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec107980 0x7f9aec107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.431+0000 7f9af11e6640 1 --2- 172.21.15.133:0/66115901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 0x7f9aec102450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.431+0000 7f9af11e6640 1 -- 172.21.15.133:0/66115901 >> 172.21.15.133:0/66115901 conn(0x7f9aec078cc0 msgr2=0x7f9aec0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:41.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.431+0000 7f9af11e6640 1 -- 172.21.15.133:0/66115901 shutdown_connections 2024-07-04T08:08:41.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.431+0000 7f9af11e6640 1 -- 172.21.15.133:0/66115901 wait complete. 2024-07-04T08:08:41.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.432+0000 7f9af11e6640 1 Processor -- start 2024-07-04T08:08:41.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.432+0000 7f9af11e6640 1 -- start start 2024-07-04T08:08:41.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.432+0000 7f9af11e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 0x7f9aec199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.432+0000 7f9af11e6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aec107980 0x7f9aec19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.432+0000 7f9af11e6640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec1082d0 0x7f9aec19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.432+0000 7f9af11e6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec112050 con 0x7f9aec107980 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.432+0000 7f9af11e6640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec111ed0 con 0x7f9aec1082d0 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.432+0000 7f9af11e6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec1121d0 con 0x7f9aec102040 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.432+0000 7f9aead9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 0x7f9aec199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9aead9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 0x7f9aec199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38082/0 (socket says 172.21.15.133:38082) 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9aead9d640 1 -- 172.21.15.133:0/1541407502 learned_addr learned my addr 172.21.15.133:0/1541407502 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9aeb59e640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec1082d0 0x7f9aec19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9aead9d640 1 -- 172.21.15.133:0/1541407502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec1082d0 msgr2=0x7f9aec19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9aead9d640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec1082d0 0x7f9aec19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9aead9d640 1 -- 172.21.15.133:0/1541407502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aec107980 msgr2=0x7f9aec19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9aead9d640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aec107980 0x7f9aec19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9aead9d640 1 -- 172.21.15.133:0/1541407502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec19faa0 con 0x7f9aec102040 2024-07-04T08:08:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9aead9d640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 0x7f9aec199b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006010 tx=0x7f9adc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9ad3fff640 1 -- 172.21.15.133:0/1541407502 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc03f030 con 0x7f9aec102040 2024-07-04T08:08:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9ad3fff640 1 -- 172.21.15.133:0/1541407502 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9adc003800 con 0x7f9aec102040 2024-07-04T08:08:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec19fd30 con 0x7f9aec102040 2024-07-04T08:08:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9ad3fff640 1 -- 172.21.15.133:0/1541407502 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc035070 con 0x7f9aec102040 2024-07-04T08:08:41.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.433+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec1a64f0 con 0x7f9aec102040 2024-07-04T08:08:41.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.434+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec1032c0 con 0x7f9aec102040 2024-07-04T08:08:41.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.436+0000 7f9ad3fff640 1 -- 172.21.15.133:0/1541407502 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f9adc0039b0 con 0x7f9aec102040 2024-07-04T08:08:41.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.436+0000 7f9ad3fff640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ad40b2840 0x7f9ad40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:41.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.436+0000 7f9ad3fff640 1 -- 172.21.15.133:0/1541407502 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f9adc102aa0 con 0x7f9aec102040 2024-07-04T08:08:41.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.437+0000 7f9aea59c640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ad40b2840 0x7f9ad40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:41.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.438+0000 7f9aea59c640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ad40b2840 0x7f9ad40b4d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9aec104af0 tx=0x7f9ae0006c90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:41.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.440+0000 7f9ad3fff640 1 -- 172.21.15.133:0/1541407502 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f9adc0cab90 con 0x7f9aec102040 2024-07-04T08:08:41.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.631+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}) -- 0x7f9aec062d50 con 0x7f9ad40b2840 2024-07-04T08:08:41.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.649+0000 7f9ad3fff640 1 -- 172.21.15.133:0/1541407502 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9aec062d50 con 0x7f9ad40b2840 2024-07-04T08:08:41.650 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mds.all update... 2024-07-04T08:08:41.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ad40b2840 msgr2=0x7f9ad40b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:41.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ad40b2840 0x7f9ad40b4d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9aec104af0 tx=0x7f9ae0006c90 comp rx=0 tx=0).stop 2024-07-04T08:08:41.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 msgr2=0x7f9aec199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 0x7f9aec199b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9adc006010 tx=0x7f9adc0042f0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 shutdown_connections 2024-07-04T08:08:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ad40b2840 0x7f9ad40b4d50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9aec107980 0x7f9aec19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9aec1082d0 0x7f9aec19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 --2- 172.21.15.133:0/1541407502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9aec102040 0x7f9aec199b30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.651+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 >> 172.21.15.133:0/1541407502 conn(0x7f9aec078cc0 msgr2=0x7f9aec1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.652+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 shutdown_connections 2024-07-04T08:08:41.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-07-04T08:08:41.652+0000 7f9af11e6640 1 -- 172.21.15.133:0/1541407502 wait complete. 2024-07-04T08:08:42.268 DEBUG:teuthology.orchestra.run.smithi039:mds.a> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.a.service 2024-07-04T08:08:42.275 DEBUG:teuthology.orchestra.run.smithi039:mds.d> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.d.service 2024-07-04T08:08:42.322 DEBUG:teuthology.orchestra.run.smithi039:mds.g> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.g.service 2024-07-04T08:08:42.366 DEBUG:teuthology.orchestra.run.smithi039:mds.j> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.j.service 2024-07-04T08:08:42.413 DEBUG:teuthology.orchestra.run.smithi050:mds.b> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.b.service 2024-07-04T08:08:42.416 DEBUG:teuthology.orchestra.run.smithi050:mds.e> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.e.service 2024-07-04T08:08:42.459 DEBUG:teuthology.orchestra.run.smithi050:mds.h> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.h.service 2024-07-04T08:08:42.502 DEBUG:teuthology.orchestra.run.smithi050:mds.k> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.k.service 2024-07-04T08:08:42.545 DEBUG:teuthology.orchestra.run.smithi133:mds.c> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.c.service 2024-07-04T08:08:42.547 DEBUG:teuthology.orchestra.run.smithi133:mds.f> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.f.service 2024-07-04T08:08:42.591 DEBUG:teuthology.orchestra.run.smithi133:mds.i> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.i.service 2024-07-04T08:08:42.633 DEBUG:teuthology.orchestra.run.smithi133:mds.l> sudo journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.l.service 2024-07-04T08:08:42.677 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-07-04T08:08:42.678 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:42.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:42 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:43.037 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:43.038 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:42 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:43.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:43.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:08:43.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: Saving service mds.all spec with placement smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-07-04T08:08:43.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:43.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:08:43.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:43.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:08:43.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:43.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:43.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:43.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:42 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:43.175 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:08:44.030 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:43 smithi133 ceph-mon[37050]: Deploying daemon mds.l on smithi133 2024-07-04T08:08:44.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:43 smithi039 ceph-mon[32062]: Deploying daemon mds.l on smithi133 2024-07-04T08:08:44.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:43 smithi050 ceph-mon[36577]: Deploying daemon mds.l on smithi133 2024-07-04T08:08:44.567 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:08:44 smithi133 systemd[1]: Starting Ceph mds.l for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:08:44.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:44 smithi039 ceph-mon[32062]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:44.910 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:44 smithi133 ceph-mon[37050]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:44.911 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:08:44 smithi133 podman[68045]: 2024-07-04 08:08:44.707773629 +0000 UTC m=+0.113954804 container create 4a82775ff921d0366e2c4ee6b6ad65bb749baf1ace8ca38ab69a703743f81022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625) 2024-07-04T08:08:44.911 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:08:44 smithi133 podman[68045]: 2024-07-04 08:08:44.611203278 +0000 UTC m=+0.017384460 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:08:44.911 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:08:44 smithi133 podman[68045]: 2024-07-04 08:08:44.910433149 +0000 UTC m=+0.316614317 container init 4a82775ff921d0366e2c4ee6b6ad65bb749baf1ace8ca38ab69a703743f81022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0) 2024-07-04T08:08:45.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:44 smithi050 ceph-mon[36577]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:45.316 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:08:44 smithi133 podman[68045]: 2024-07-04 08:08:44.913060518 +0000 UTC m=+0.319241693 container start 4a82775ff921d0366e2c4ee6b6ad65bb749baf1ace8ca38ab69a703743f81022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:08:45.316 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:08:44 smithi133 ceph-mds[68064]: -- 172.21.15.133:0/1526224338 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555e22277860 con 0x555e26429a80 2024-07-04T08:08:45.316 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:08:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l[68060]: starting mds.l at 2024-07-04T08:08:45.316 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:08:44 smithi133 bash[68045]: 4a82775ff921d0366e2c4ee6b6ad65bb749baf1ace8ca38ab69a703743f81022 2024-07-04T08:08:45.317 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:08:44 smithi133 systemd[1]: Started Ceph mds.l for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:08:45.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.410+0000 7f7363e00640 1 -- 172.21.15.39:0/1381480761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736410ee50 msgr2=0x7f736410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:45.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.410+0000 7f7363e00640 1 --2- 172.21.15.39:0/1381480761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736410ee50 0x7f736410f260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f734c009900 tx=0x7f734c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.411+0000 7f7363e00640 1 -- 172.21.15.39:0/1381480761 shutdown_connections 2024-07-04T08:08:45.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.411+0000 7f7363e00640 1 --2- 172.21.15.39:0/1381480761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f736410f7a0 0x7f7364115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.411+0000 7f7363e00640 1 --2- 172.21.15.39:0/1381480761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f736410ee50 0x7f736410f260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.411+0000 7f7363e00640 1 --2- 172.21.15.39:0/1381480761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7364109510 0x7f7364109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.411+0000 7f7363e00640 1 -- 172.21.15.39:0/1381480761 >> 172.21.15.39:0/1381480761 conn(0x7f7364078b10 msgr2=0x7f7364107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:45.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.411+0000 7f7363e00640 1 -- 172.21.15.39:0/1381480761 shutdown_connections 2024-07-04T08:08:45.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.411+0000 7f7363e00640 1 -- 172.21.15.39:0/1381480761 wait complete. 2024-07-04T08:08:45.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.412+0000 7f7363e00640 1 Processor -- start 2024-07-04T08:08:45.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.412+0000 7f7363e00640 1 -- start start 2024-07-04T08:08:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.413+0000 7f7363e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7364109510 0x7f736419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.413+0000 7f7363e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f736410ee50 0x7f73641a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.413+0000 7f7363e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f736410f7a0 0x7f73641a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.413+0000 7f7363e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7364119410 con 0x7f736410f7a0 2024-07-04T08:08:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.413+0000 7f7363e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7364119290 con 0x7f7364109510 2024-07-04T08:08:45.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.413+0000 7f7363e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7364119590 con 0x7f736410ee50 2024-07-04T08:08:45.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.413+0000 7f7362a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7364109510 0x7f736419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:45.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f735a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f736410ee50 0x7f73641a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f7362a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7364109510 0x7f736419fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53322/0 (socket says 172.21.15.39:53322) 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f7362a00640 1 -- 172.21.15.39:0/4269224907 learned_addr learned my addr 172.21.15.39:0/4269224907 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f7363400640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f736410f7a0 0x7f73641a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f735a000640 1 -- 172.21.15.39:0/4269224907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7364109510 msgr2=0x7f736419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f735a000640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7364109510 0x7f736419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f735a000640 1 -- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f736410f7a0 msgr2=0x7f73641a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f735a000640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f736410f7a0 0x7f73641a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f735a000640 1 -- 172.21.15.39:0/4269224907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73641a5df0 con 0x7f736410ee50 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.414+0000 7f7362a00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7364109510 0x7f736419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:08:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.415+0000 7f735a000640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f736410ee50 0x7f73641a0420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f734c02a910 tx=0x7f734c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:45.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.415+0000 7f735be00640 1 -- 172.21.15.39:0/4269224907 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c03f070 con 0x7f736410ee50 2024-07-04T08:08:45.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.415+0000 7f735be00640 1 -- 172.21.15.39:0/4269224907 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f734c003aa0 con 0x7f736410ee50 2024-07-04T08:08:45.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.415+0000 7f735be00640 1 -- 172.21.15.39:0/4269224907 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c035070 con 0x7f736410ee50 2024-07-04T08:08:45.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.415+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73641a6080 con 0x7f736410ee50 2024-07-04T08:08:45.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.415+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73641ac840 con 0x7f736410ee50 2024-07-04T08:08:45.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.416+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7364109920 con 0x7f736410ee50 2024-07-04T08:08:45.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.419+0000 7f735be00640 1 -- 172.21.15.39:0/4269224907 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f734c003c50 con 0x7f736410ee50 2024-07-04T08:08:45.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.420+0000 7f735be00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73480b28c0 0x7f73480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:45.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.420+0000 7f735be00640 1 -- 172.21.15.39:0/4269224907 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f734c101f30 con 0x7f736410ee50 2024-07-04T08:08:45.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.421+0000 7f7362a00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73480b28c0 0x7f73480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:45.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.421+0000 7f7362a00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73480b28c0 0x7f73480b4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7354011550 tx=0x7f7354005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.424+0000 7f735be00640 1 -- 172.21.15.39:0/4269224907 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f734c0c9f80 con 0x7f736410ee50 2024-07-04T08:08:45.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.679+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73641115b0 con 0x7f736410ee50 2024-07-04T08:08:45.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.680+0000 7f735be00640 1 -- 172.21.15.39:0/4269224907 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f734c0c9880 con 0x7f736410ee50 2024-07-04T08:08:45.681 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:08:45.681 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":2,"btime":"2024-07-04T08:08:44:951363+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-04T08:08:45.682 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 2 2024-07-04T08:08:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.682+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73480b28c0 msgr2=0x7f73480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.682+0000 7f7363e00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73480b28c0 0x7f73480b4dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7354011550 tx=0x7f7354005f90 comp rx=0 tx=0).stop 2024-07-04T08:08:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f736410ee50 msgr2=0x7f73641a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f736410ee50 0x7f73641a0420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f734c02a910 tx=0x7f734c031a20 comp rx=0 tx=0).stop 2024-07-04T08:08:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 shutdown_connections 2024-07-04T08:08:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73480b28c0 0x7f73480b4dd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f736410f7a0 0x7f73641a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f736410ee50 0x7f73641a0420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 --2- 172.21.15.39:0/4269224907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7364109510 0x7f736419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:45.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 >> 172.21.15.39:0/4269224907 conn(0x7f7364078b10 msgr2=0x7f73640793c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:45.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 shutdown_connections 2024-07-04T08:08:45.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:45.683+0000 7f7363e00640 1 -- 172.21.15.39:0/4269224907 wait complete. 2024-07-04T08:08:46.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6832/2411107381,v1:172.21.15.133:6833/2411107381] up:boot 2024-07-04T08:08:46.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: fsmap 1 up:standby 2024-07-04T08:08:46.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-04T08:08:46.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:46.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:46.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:46.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:46.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:46.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:46.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:45 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/4269224907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:08:46.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6832/2411107381,v1:172.21.15.133:6833/2411107381] up:boot 2024-07-04T08:08:46.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: fsmap 1 up:standby 2024-07-04T08:08:46.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-04T08:08:46.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:46.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:46.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:46.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:46.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:46.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:46.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:45 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/4269224907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:08:46.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6832/2411107381,v1:172.21.15.133:6833/2411107381] up:boot 2024-07-04T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: fsmap 1 up:standby 2024-07-04T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-04T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:46.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:45 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/4269224907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:08:47.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:46 smithi133 ceph-mon[37050]: Deploying daemon mds.k on smithi050 2024-07-04T08:08:47.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:46 smithi133 ceph-mon[37050]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:47.349 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:46 smithi050 ceph-mon[36577]: Deploying daemon mds.k on smithi050 2024-07-04T08:08:47.349 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:46 smithi050 ceph-mon[36577]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:47.349 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:08:47 smithi050 systemd[1]: Starting Ceph mds.k for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:08:47.349 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:08:47 smithi050 podman[67831]: 2024-07-04 08:08:47.250653078 +0000 UTC m=+0.017693472 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:08:47.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:46 smithi039 ceph-mon[32062]: Deploying daemon mds.k on smithi050 2024-07-04T08:08:47.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:46 smithi039 ceph-mon[32062]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:47.627 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:08:47 smithi050 podman[67831]: 2024-07-04 08:08:47.365893885 +0000 UTC m=+0.132934264 container create 9628c14682d4b98bdc5a0f648125f29d8142ef4c3494fd35c3291d8d49861c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625) 2024-07-04T08:08:47.895 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:08:47 smithi050 podman[67831]: 2024-07-04 08:08:47.627284658 +0000 UTC m=+0.394325062 container init 9628c14682d4b98bdc5a0f648125f29d8142ef4c3494fd35c3291d8d49861c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b) 2024-07-04T08:08:47.895 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:08:47 smithi050 podman[67831]: 2024-07-04 08:08:47.630452605 +0000 UTC m=+0.397492998 container start 9628c14682d4b98bdc5a0f648125f29d8142ef4c3494fd35c3291d8d49861c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:08:47.895 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:08:47 smithi050 ceph-mds[67850]: -- 172.21.15.50:0/517932652 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf9a99bc20 con 0x55bf9eb4da80 2024-07-04T08:08:47.895 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:08:47 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k[67846]: starting mds.k at 2024-07-04T08:08:47.895 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:08:47 smithi050 bash[67831]: 9628c14682d4b98bdc5a0f648125f29d8142ef4c3494fd35c3291d8d49861c1f 2024-07-04T08:08:47.895 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:08:47 smithi050 systemd[1]: Started Ceph mds.k for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:08:48.491 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:08:48.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:48.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:48.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:48.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:48.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:48.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:48.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:48.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: Deploying daemon mds.j on smithi039 2024-07-04T08:08:48.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6832/264080488,v1:172.21.15.50:6833/264080488] up:boot 2024-07-04T08:08:48.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: fsmap 2 up:standby 2024-07-04T08:08:48.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-04T08:08:49.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: Deploying daemon mds.j on smithi039 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6832/264080488,v1:172.21.15.50:6833/264080488] up:boot 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: fsmap 2 up:standby 2024-07-04T08:08:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: Deploying daemon mds.j on smithi039 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6832/264080488,v1:172.21.15.50:6833/264080488] up:boot 2024-07-04T08:08:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: fsmap 2 up:standby 2024-07-04T08:08:49.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-04T08:08:49.457 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:08:51.046 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:50 smithi050 ceph-mon[36577]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:51.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:50 smithi133 ceph-mon[37050]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:51.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:50 smithi039 ceph-mon[32062]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:51.100 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:08:50 smithi039 systemd[1]: Starting Ceph mds.j for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:08:51.720 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:08:51 smithi039 podman[77229]: 2024-07-04 08:08:51.598830016 +0000 UTC m=+0.050615517 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:08:51.721 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:08:51 smithi039 podman[77229]: 2024-07-04 08:08:51.71882976 +0000 UTC m=+0.170615252 container create 1504f2280418f9d2897594d5e6d63ac9efa38f4128e426c10afd4b9a2aad16ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:08:52.249 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:08:52 smithi039 podman[77229]: 2024-07-04 08:08:52.058086552 +0000 UTC m=+0.509872090 container init 1504f2280418f9d2897594d5e6d63ac9efa38f4128e426c10afd4b9a2aad16ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:08:52.249 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:08:52 smithi039 podman[77229]: 2024-07-04 08:08:52.069774898 +0000 UTC m=+0.521560386 container start 1504f2280418f9d2897594d5e6d63ac9efa38f4128e426c10afd4b9a2aad16ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-07-04T08:08:52.249 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:08:52 smithi039 ceph-mds[77267]: -- 172.21.15.39:0/781800288 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560ec00d860 con 0x5560f01bfa80 2024-07-04T08:08:52.249 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:08:52 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j[77262]: starting mds.j at 2024-07-04T08:08:52.524 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:08:52 smithi039 bash[77229]: 1504f2280418f9d2897594d5e6d63ac9efa38f4128e426c10afd4b9a2aad16ef 2024-07-04T08:08:52.524 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:08:52 smithi039 systemd[1]: Started Ceph mds.j for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:08:52.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.605+0000 7f2bc9200640 1 -- 172.21.15.39:0/3664360375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4102f00 msgr2=0x7f2bc4103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:52.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.605+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3664360375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4102f00 0x7f2bc4103310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009900 tx=0x7f2bb402a410 comp rx=0 tx=0).stop 2024-07-04T08:08:52.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.605+0000 7f2bc9200640 1 -- 172.21.15.39:0/3664360375 shutdown_connections 2024-07-04T08:08:52.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.605+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3664360375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc4109190 0x7f2bc410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:52.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.605+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3664360375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc4108840 0x7f2bc4108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:52.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.605+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3664360375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4102f00 0x7f2bc4103310 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:52.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.605+0000 7f2bc9200640 1 -- 172.21.15.39:0/3664360375 >> 172.21.15.39:0/3664360375 conn(0x7f2bc40fe990 msgr2=0x7f2bc4100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:52.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.606+0000 7f2bc9200640 1 -- 172.21.15.39:0/3664360375 shutdown_connections 2024-07-04T08:08:52.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.606+0000 7f2bc9200640 1 -- 172.21.15.39:0/3664360375 wait complete. 2024-07-04T08:08:52.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.607+0000 7f2bc9200640 1 Processor -- start 2024-07-04T08:08:52.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.607+0000 7f2bc9200640 1 -- start start 2024-07-04T08:08:52.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bc9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc4102f00 0x7f2bc41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:52.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bc9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4108840 0x7f2bc4199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:52.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bc9200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc4109190 0x7f2bc419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:52.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bc9200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4112ea0 con 0x7f2bc4108840 2024-07-04T08:08:52.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bc9200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4112d20 con 0x7f2bc4102f00 2024-07-04T08:08:52.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bc9200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4113020 con 0x7f2bc4109190 2024-07-04T08:08:52.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bbbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4108840 0x7f2bc4199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:52.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bbbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4108840 0x7f2bc4199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56680/0 (socket says 172.21.15.39:56680) 2024-07-04T08:08:52.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bbbe00640 1 -- 172.21.15.39:0/3272275061 learned_addr learned my addr 172.21.15.39:0/3272275061 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:08:52.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.608+0000 7f2bc3400640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc4102f00 0x7f2bc41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:52.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.609+0000 7f2bc3e00640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc4109190 0x7f2bc419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:52.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.609+0000 7f2bbbe00640 1 -- 172.21.15.39:0/3272275061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc4109190 msgr2=0x7f2bc419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:52.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.609+0000 7f2bbbe00640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc4109190 0x7f2bc419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:52.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.609+0000 7f2bbbe00640 1 -- 172.21.15.39:0/3272275061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc4102f00 msgr2=0x7f2bc41998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:52.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.609+0000 7f2bbbe00640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc4102f00 0x7f2bc41998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:52.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.609+0000 7f2bbbe00640 1 -- 172.21.15.39:0/3272275061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc419f840 con 0x7f2bc4108840 2024-07-04T08:08:52.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.609+0000 7f2bbbe00640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4108840 0x7f2bc4199e70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e970 tx=0x7f2bac00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:52.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.610+0000 7f2bc0c00640 1 -- 172.21.15.39:0/3272275061 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00cd80 con 0x7f2bc4108840 2024-07-04T08:08:52.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.610+0000 7f2bc0c00640 1 -- 172.21.15.39:0/3272275061 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bac0035f0 con 0x7f2bc4108840 2024-07-04T08:08:52.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.610+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc419fb30 con 0x7f2bc4108840 2024-07-04T08:08:52.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.610+0000 7f2bc0c00640 1 -- 172.21.15.39:0/3272275061 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac010690 con 0x7f2bc4108840 2024-07-04T08:08:52.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.610+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc41a6290 con 0x7f2bc4108840 2024-07-04T08:08:52.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.611+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc4103310 con 0x7f2bc4108840 2024-07-04T08:08:52.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.616+0000 7f2bc0c00640 1 -- 172.21.15.39:0/3272275061 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f2bac003aa0 con 0x7f2bc4108840 2024-07-04T08:08:52.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.617+0000 7f2bc0c00640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ba80b28c0 0x7f2ba80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:52.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.618+0000 7f2bc0c00640 1 -- 172.21.15.39:0/3272275061 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f2bac0e2320 con 0x7f2bc4108840 2024-07-04T08:08:52.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.618+0000 7f2bc3400640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ba80b28c0 0x7f2ba80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:52.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.619+0000 7f2bc3400640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ba80b28c0 0x7f2ba80b4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005ab0 tx=0x7f2bb4005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:52.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.620+0000 7f2bc0c00640 1 -- 172.21.15.39:0/3272275061 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f2bac055110 con 0x7f2bc4108840 2024-07-04T08:08:52.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.888+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bc419aa00 con 0x7f2bc4108840 2024-07-04T08:08:52.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.888+0000 7f2bc0c00640 1 -- 172.21.15.39:0/3272275061 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f2bc419aa00 con 0x7f2bc4108840 2024-07-04T08:08:52.889 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:08:52.890 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"btime":"2024-07-04T08:08:52:795244+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-07-04T08:08:52.890 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 4 2024-07-04T08:08:52.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ba80b28c0 msgr2=0x7f2ba80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:52.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ba80b28c0 0x7f2ba80b4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005ab0 tx=0x7f2bb4005a20 comp rx=0 tx=0).stop 2024-07-04T08:08:52.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4108840 msgr2=0x7f2bc4199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:52.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4108840 0x7f2bc4199e70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00e970 tx=0x7f2bac00ee30 comp rx=0 tx=0).stop 2024-07-04T08:08:52.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 shutdown_connections 2024-07-04T08:08:52.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ba80b28c0 0x7f2ba80b4dd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:52.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2bc4108840 0x7f2bc4199e70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:52.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc4109190 0x7f2bc419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:52.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 --2- 172.21.15.39:0/3272275061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2bc4102f00 0x7f2bc41998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:52.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 >> 172.21.15.39:0/3272275061 conn(0x7f2bc40fe990 msgr2=0x7f2bc4100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:52.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 shutdown_connections 2024-07-04T08:08:52.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:52.892+0000 7f2bc9200640 1 -- 172.21.15.39:0/3272275061 wait complete. 2024-07-04T08:08:53.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:52 smithi133 ceph-mon[37050]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:53.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:52 smithi039 ceph-mon[32062]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:52 smithi050 ceph-mon[36577]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:53.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:53 smithi039 ceph-mon[32062]: Deploying daemon mds.d on smithi039 2024-07-04T08:08:53.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:53 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.39:6834/1612340282,v1:172.21.15.39:6835/1612340282] up:boot 2024-07-04T08:08:53.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:53 smithi039 ceph-mon[32062]: fsmap 3 up:standby 2024-07-04T08:08:53.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-04T08:08:53.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:53 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3272275061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:08:54.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:53 smithi133 ceph-mon[37050]: Deploying daemon mds.d on smithi039 2024-07-04T08:08:54.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:53 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.39:6834/1612340282,v1:172.21.15.39:6835/1612340282] up:boot 2024-07-04T08:08:54.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:53 smithi133 ceph-mon[37050]: fsmap 3 up:standby 2024-07-04T08:08:54.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-04T08:08:54.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:53 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3272275061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:08:54.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:53 smithi050 ceph-mon[36577]: Deploying daemon mds.d on smithi039 2024-07-04T08:08:54.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:53 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.39:6834/1612340282,v1:172.21.15.39:6835/1612340282] up:boot 2024-07-04T08:08:54.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:53 smithi050 ceph-mon[36577]: fsmap 3 up:standby 2024-07-04T08:08:54.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-04T08:08:54.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:53 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3272275061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:08:55.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:54 smithi039 ceph-mon[32062]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:55.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:54 smithi050 ceph-mon[36577]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:55.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:54 smithi133 ceph-mon[37050]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:55.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:08:56.318 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:08:55 smithi039 systemd[1]: Starting Ceph mds.d for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:08:56.602 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:08:56 smithi039 podman[77618]: 2024-07-04 08:08:56.221045189 +0000 UTC m=+0.034003170 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:08:56.602 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:08:56 smithi039 podman[77618]: 2024-07-04 08:08:56.357665118 +0000 UTC m=+0.170623127 container create 6d577a8c2acde370defa56d20d34a1359c17b9ed991c71c1b48e8a053bddad68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-04T08:08:56.776 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:08:56.938 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:56 smithi039 ceph-mon[32062]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:56.938 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:08:56 smithi039 podman[77618]: 2024-07-04 08:08:56.653633841 +0000 UTC m=+0.466591913 container init 6d577a8c2acde370defa56d20d34a1359c17b9ed991c71c1b48e8a053bddad68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-07-04T08:08:56.938 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:08:56 smithi039 podman[77618]: 2024-07-04 08:08:56.664237691 +0000 UTC m=+0.477195703 container start 6d577a8c2acde370defa56d20d34a1359c17b9ed991c71c1b48e8a053bddad68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True) 2024-07-04T08:08:56.938 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:08:56 smithi039 bash[77618]: 6d577a8c2acde370defa56d20d34a1359c17b9ed991c71c1b48e8a053bddad68 2024-07-04T08:08:56.939 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:08:56 smithi039 systemd[1]: Started Ceph mds.d for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:08:56.939 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:08:56 smithi039 ceph-mds[77663]: -- 172.21.15.39:0/3251562403 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e20c6b1860 con 0x55e210863600 2024-07-04T08:08:56.939 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:08:56 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d[77659]: starting mds.d at 2024-07-04T08:08:57.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:56 smithi050 ceph-mon[36577]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:57.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:56 smithi133 ceph-mon[37050]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:57.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.39:6836/1240380045,v1:172.21.15.39:6837/1240380045] up:boot 2024-07-04T08:08:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: fsmap 4 up:standby 2024-07-04T08:08:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-04T08:08:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:57.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:57.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:57 smithi050 ceph-mon[36577]: Deploying daemon mds.e on smithi050 2024-07-04T08:08:58.062 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.39:6836/1240380045,v1:172.21.15.39:6837/1240380045] up:boot 2024-07-04T08:08:58.062 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: fsmap 4 up:standby 2024-07-04T08:08:58.062 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-04T08:08:58.062 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:58.062 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:58.063 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:58.063 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:58.063 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:58.063 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:58.063 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:57 smithi039 ceph-mon[32062]: Deploying daemon mds.e on smithi050 2024-07-04T08:08:58.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.39:6836/1240380045,v1:172.21.15.39:6837/1240380045] up:boot 2024-07-04T08:08:58.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: fsmap 4 up:standby 2024-07-04T08:08:58.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-04T08:08:58.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:58.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:58.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:08:58.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:08:58.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:08:58.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:08:58.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:57 smithi133 ceph-mon[37050]: Deploying daemon mds.e on smithi050 2024-07-04T08:08:58.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.931+0000 7f79c1400640 1 -- 172.21.15.39:0/1308259449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79bc104660 msgr2=0x7f79bc10cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:58.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.931+0000 7f79c1400640 1 --2- 172.21.15.39:0/1308259449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79bc104660 0x7f79bc10cbb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009ca0 tx=0x7f79a802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:08:58.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.931+0000 7f79c1400640 1 -- 172.21.15.39:0/1308259449 shutdown_connections 2024-07-04T08:08:58.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.931+0000 7f79c1400640 1 --2- 172.21.15.39:0/1308259449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc10d170 0x7f79bc10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:58.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.931+0000 7f79c1400640 1 --2- 172.21.15.39:0/1308259449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79bc104660 0x7f79bc10cbb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:58.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.931+0000 7f79c1400640 1 --2- 172.21.15.39:0/1308259449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79bc103d10 0x7f79bc104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:58.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.932+0000 7f79c1400640 1 -- 172.21.15.39:0/1308259449 >> 172.21.15.39:0/1308259449 conn(0x7f79bc0fd810 msgr2=0x7f79bc0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:58.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.932+0000 7f79c1400640 1 -- 172.21.15.39:0/1308259449 shutdown_connections 2024-07-04T08:08:58.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.932+0000 7f79c1400640 1 -- 172.21.15.39:0/1308259449 wait complete. 2024-07-04T08:08:58.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.933+0000 7f79c1400640 1 Processor -- start 2024-07-04T08:08:58.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.934+0000 7f79c1400640 1 -- start start 2024-07-04T08:08:58.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.934+0000 7f79c1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc103d10 0x7f79bc19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:58.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.934+0000 7f79c1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79bc104660 0x7f79bc19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:58.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79c1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79bc10d170 0x7f79bc1a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:58.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79c1400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc112e20 con 0x7f79bc104660 2024-07-04T08:08:58.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79c1400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc112ca0 con 0x7f79bc10d170 2024-07-04T08:08:58.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79c1400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc112fa0 con 0x7f79bc103d10 2024-07-04T08:08:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79bbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc103d10 0x7f79bc19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79bc104660 0x7f79bc19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79bbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc103d10 0x7f79bc19ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50068/0 (socket says 172.21.15.39:50068) 2024-07-04T08:08:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79bbe00640 1 -- 172.21.15.39:0/2024453695 learned_addr learned my addr 172.21.15.39:0/2024453695 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:08:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79c0a00640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79bc10d170 0x7f79bc1a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79bbe00640 1 -- 172.21.15.39:0/2024453695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79bc10d170 msgr2=0x7f79bc1a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:58.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79bbe00640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79bc10d170 0x7f79bc1a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79bbe00640 1 -- 172.21.15.39:0/2024453695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79bc104660 msgr2=0x7f79bc19bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79bbe00640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79bc104660 0x7f79bc19bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.935+0000 7f79bbe00640 1 -- 172.21.15.39:0/2024453695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79bc1a1980 con 0x7f79bc103d10 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.936+0000 7f79bbe00640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc103d10 0x7f79bc19ba10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00e9f0 tx=0x7f79ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.936+0000 7f79b8c00640 1 -- 172.21.15.39:0/2024453695 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac00cd80 con 0x7f79bc103d10 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.936+0000 7f79b8c00640 1 -- 172.21.15.39:0/2024453695 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79ac0108e0 con 0x7f79bc103d10 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.936+0000 7f79b8c00640 1 -- 172.21.15.39:0/2024453695 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac018980 con 0x7f79bc103d10 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.936+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79bc1a1c70 con 0x7f79bc103d10 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.936+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79bc1a83d0 con 0x7f79bc103d10 2024-07-04T08:08:58.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.937+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79bc104120 con 0x7f79bc103d10 2024-07-04T08:08:58.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.940+0000 7f79b8c00640 1 -- 172.21.15.39:0/2024453695 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f79ac010430 con 0x7f79bc103d10 2024-07-04T08:08:58.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.941+0000 7f79b8c00640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f79a00b2840 0x7f79a00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:08:58.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.942+0000 7f79b8c00640 1 -- 172.21.15.39:0/2024453695 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f79ac0e2500 con 0x7f79bc103d10 2024-07-04T08:08:58.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.942+0000 7f79bb400640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f79a00b2840 0x7f79a00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:08:58.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.942+0000 7f79bb400640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f79a00b2840 0x7f79a00b4d50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f79a8000f80 tx=0x7f79a8005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:08:58.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:58.943+0000 7f79b8c00640 1 -- 172.21.15.39:0/2024453695 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f79ac0a9f00 con 0x7f79bc103d10 2024-07-04T08:08:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:58 smithi039 ceph-mon[32062]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:58 smithi050 ceph-mon[36577]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:59.145 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:08:58 smithi050 systemd[1]: Starting Ceph mds.e for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:08:59.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.198+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79bc1a8670 con 0x7f79bc103d10 2024-07-04T08:08:59.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.199+0000 7f79b8c00640 1 -- 172.21.15.39:0/2024453695 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f79ac0ae200 con 0x7f79bc103d10 2024-07-04T08:08:59.200 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:08:59.201 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"btime":"2024-07-04T08:08:56:832352+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-07-04T08:08:59.201 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 5 2024-07-04T08:08:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f79a00b2840 msgr2=0x7f79a00b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f79a00b2840 0x7f79a00b4d50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f79a8000f80 tx=0x7f79a8005fd0 comp rx=0 tx=0).stop 2024-07-04T08:08:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc103d10 msgr2=0x7f79bc19ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:08:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc103d10 0x7f79bc19ba10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00e9f0 tx=0x7f79ac00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:08:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 shutdown_connections 2024-07-04T08:08:59.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f79a00b2840 0x7f79a00b4d50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79bc104660 0x7f79bc19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f79bc10d170 0x7f79bc1a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 --2- 172.21.15.39:0/2024453695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc103d10 0x7f79bc19ba10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:08:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 >> 172.21.15.39:0/2024453695 conn(0x7f79bc0fd810 msgr2=0x7f79bc101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:08:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 shutdown_connections 2024-07-04T08:08:59.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:08:59.202+0000 7f79c1400640 1 -- 172.21.15.39:0/2024453695 wait complete. 2024-07-04T08:08:59.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:58 smithi133 ceph-mon[37050]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:08:59.435 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:08:59 smithi050 podman[68147]: 2024-07-04 08:08:59.076171648 +0000 UTC m=+0.017756631 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:08:59.435 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:08:59 smithi050 podman[68147]: 2024-07-04 08:08:59.198023448 +0000 UTC m=+0.139608434 container create 9ddac52d00b7eebb49a08361a0ef2d78dd2bde2944ca4bf1fbcdd5b351c309af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-04T08:08:59.435 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:08:59 smithi050 podman[68147]: 2024-07-04 08:08:59.434186427 +0000 UTC m=+0.375771401 container init 9ddac52d00b7eebb49a08361a0ef2d78dd2bde2944ca4bf1fbcdd5b351c309af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-07-04T08:08:59.894 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:08:59 smithi050 podman[68147]: 2024-07-04 08:08:59.437059275 +0000 UTC m=+0.378644259 container start 9ddac52d00b7eebb49a08361a0ef2d78dd2bde2944ca4bf1fbcdd5b351c309af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD) 2024-07-04T08:08:59.895 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:08:59 smithi050 ceph-mds[68166]: -- 172.21.15.50:0/602669382 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56426477d860 con 0x56426892f180 2024-07-04T08:08:59.895 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:08:59 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e[68162]: starting mds.e at 2024-07-04T08:08:59.895 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:08:59 smithi050 bash[68147]: 9ddac52d00b7eebb49a08361a0ef2d78dd2bde2944ca4bf1fbcdd5b351c309af 2024-07-04T08:08:59.895 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:08:59 smithi050 systemd[1]: Started Ceph mds.e for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:09:00.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:59 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2024453695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:00.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:08:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:00.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:59 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2024453695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:00.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:00.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:00.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:00.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:00.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:00.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:08:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:59 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2024453695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:00.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:08:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:01.187 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:00 smithi133 ceph-mon[37050]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:01.187 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:00 smithi133 ceph-mon[37050]: Deploying daemon mds.f on smithi133 2024-07-04T08:09:01.187 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:00 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6834/2710764279,v1:172.21.15.50:6835/2710764279] up:boot 2024-07-04T08:09:01.187 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:00 smithi133 ceph-mon[37050]: fsmap 5 up:standby 2024-07-04T08:09:01.187 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:00 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-04T08:09:01.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:00 smithi039 ceph-mon[32062]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:01.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:00 smithi039 ceph-mon[32062]: Deploying daemon mds.f on smithi133 2024-07-04T08:09:01.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:00 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6834/2710764279,v1:172.21.15.50:6835/2710764279] up:boot 2024-07-04T08:09:01.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:00 smithi039 ceph-mon[32062]: fsmap 5 up:standby 2024-07-04T08:09:01.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:00 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-04T08:09:01.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:00 smithi050 ceph-mon[36577]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:01.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:00 smithi050 ceph-mon[36577]: Deploying daemon mds.f on smithi133 2024-07-04T08:09:01.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:00 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6834/2710764279,v1:172.21.15.50:6835/2710764279] up:boot 2024-07-04T08:09:01.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:00 smithi050 ceph-mon[36577]: fsmap 5 up:standby 2024-07-04T08:09:01.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:00 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-04T08:09:01.817 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:09:01 smithi133 systemd[1]: Starting Ceph mds.f for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:09:02.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:09:02.117 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:09:01 smithi133 podman[68348]: 2024-07-04 08:09:01.766395403 +0000 UTC m=+0.017338671 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:09:02.117 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:09:01 smithi133 podman[68348]: 2024-07-04 08:09:01.880570331 +0000 UTC m=+0.131513583 container create 0422af43ab13d767baa4f15e16c5bc026467bcfccb7d0e52820e6740e28bcfc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625) 2024-07-04T08:09:02.117 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:09:02 smithi133 podman[68348]: 2024-07-04 08:09:02.116896845 +0000 UTC m=+0.367840121 container init 0422af43ab13d767baa4f15e16c5bc026467bcfccb7d0e52820e6740e28bcfc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:09:02.531 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:02.566 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:09:02 smithi133 podman[68348]: 2024-07-04 08:09:02.119654746 +0000 UTC m=+0.370598005 container start 0422af43ab13d767baa4f15e16c5bc026467bcfccb7d0e52820e6740e28bcfc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2) 2024-07-04T08:09:02.566 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:09:02 smithi133 ceph-mds[68367]: -- 172.21.15.133:0/225343206 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588c5117860 con 0x5588c92c9a80 2024-07-04T08:09:02.566 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:09:02 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f[68363]: starting mds.f at 2024-07-04T08:09:02.567 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:09:02 smithi133 bash[68348]: 0422af43ab13d767baa4f15e16c5bc026467bcfccb7d0e52820e6740e28bcfc2 2024-07-04T08:09:02.567 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:09:02 smithi133 systemd[1]: Started Ceph mds.f for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:09:03.005 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:02 smithi133 ceph-mon[37050]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:03.005 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:03.005 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:03.005 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:03.005 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:03.005 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:03.005 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:03.015 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:02 smithi039 ceph-mon[32062]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:03.016 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:03.016 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:03.016 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:03.016 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:03.016 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:03.016 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:03.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:02 smithi050 ceph-mon[36577]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:03.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:03.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:03.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:03.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:03.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:03.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:04.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:03 smithi133 ceph-mon[37050]: Deploying daemon mds.c on smithi133 2024-07-04T08:09:04.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:03 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6834/2187182943,v1:172.21.15.133:6835/2187182943] up:boot 2024-07-04T08:09:04.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:03 smithi133 ceph-mon[37050]: fsmap 6 up:standby 2024-07-04T08:09:04.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-04T08:09:04.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-04T08:09:04.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:03 smithi039 ceph-mon[32062]: Deploying daemon mds.c on smithi133 2024-07-04T08:09:04.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:03 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6834/2187182943,v1:172.21.15.133:6835/2187182943] up:boot 2024-07-04T08:09:04.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:03 smithi039 ceph-mon[32062]: fsmap 6 up:standby 2024-07-04T08:09:04.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-04T08:09:04.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-04T08:09:04.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:03 smithi050 ceph-mon[36577]: Deploying daemon mds.c on smithi133 2024-07-04T08:09:04.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:03 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6834/2187182943,v1:172.21.15.133:6835/2187182943] up:boot 2024-07-04T08:09:04.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:03 smithi050 ceph-mon[36577]: fsmap 6 up:standby 2024-07-04T08:09:04.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-04T08:09:04.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-04T08:09:04.490 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:09:04 smithi133 systemd[1]: Starting Ceph mds.c for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:09:04.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.723+0000 7fcc46800640 1 -- 172.21.15.39:0/1376022072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc401078f0 msgr2=0x7fcc40107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:04.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.723+0000 7fcc46800640 1 --2- 172.21.15.39:0/1376022072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc401078f0 0x7fcc40107d00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009900 tx=0x7fcc2c02a410 comp rx=0 tx=0).stop 2024-07-04T08:09:04.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.723+0000 7fcc46800640 1 -- 172.21.15.39:0/1376022072 shutdown_connections 2024-07-04T08:09:04.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.723+0000 7fcc46800640 1 --2- 172.21.15.39:0/1376022072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc401078f0 0x7fcc40107d00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:04.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.723+0000 7fcc46800640 1 --2- 172.21.15.39:0/1376022072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc40108240 0x7fcc4010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:04.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.723+0000 7fcc46800640 1 --2- 172.21.15.39:0/1376022072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc40101fb0 0x7fcc401023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:04.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.723+0000 7fcc46800640 1 -- 172.21.15.39:0/1376022072 >> 172.21.15.39:0/1376022072 conn(0x7fcc40078c80 msgr2=0x7fcc40079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:04.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.723+0000 7fcc46800640 1 -- 172.21.15.39:0/1376022072 shutdown_connections 2024-07-04T08:09:04.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.723+0000 7fcc46800640 1 -- 172.21.15.39:0/1376022072 wait complete. 2024-07-04T08:09:04.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.724+0000 7fcc46800640 1 Processor -- start 2024-07-04T08:09:04.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc46800640 1 -- start start 2024-07-04T08:09:04.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc46800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc40101fb0 0x7fcc400febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:04.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc46800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc401078f0 0x7fcc400ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:04.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc46800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc40108240 0x7fcc400ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:04.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc46800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40111eb0 con 0x7fcc40101fb0 2024-07-04T08:09:04.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc46800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40111d30 con 0x7fcc401078f0 2024-07-04T08:09:04.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc46800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40112030 con 0x7fcc40108240 2024-07-04T08:09:04.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc40101fb0 0x7fcc400febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:04.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc40101fb0 0x7fcc400febc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49562/0 (socket says 172.21.15.39:49562) 2024-07-04T08:09:04.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.725+0000 7fcc3f400640 1 -- 172.21.15.39:0/1266879282 learned_addr learned my addr 172.21.15.39:0/1266879282 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:04.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.726+0000 7fcc3ea00640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc401078f0 0x7fcc400ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:04.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.726+0000 7fcc3fe00640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc40108240 0x7fcc400ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:04.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.726+0000 7fcc3f400640 1 -- 172.21.15.39:0/1266879282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc40108240 msgr2=0x7fcc400ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:04.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.726+0000 7fcc3f400640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc40108240 0x7fcc400ff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:04.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.726+0000 7fcc3f400640 1 -- 172.21.15.39:0/1266879282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc401078f0 msgr2=0x7fcc400ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:04.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.726+0000 7fcc3f400640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc401078f0 0x7fcc400ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:04.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.726+0000 7fcc3f400640 1 -- 172.21.15.39:0/1266879282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc401a7cb0 con 0x7fcc40101fb0 2024-07-04T08:09:04.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.726+0000 7fcc3f400640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc40101fb0 0x7fcc400febc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000e9f0 tx=0x7fcc3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:04.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.727+0000 7fcc37e00640 1 -- 172.21.15.39:0/1266879282 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3000ccc0 con 0x7fcc40101fb0 2024-07-04T08:09:04.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.727+0000 7fcc37e00640 1 -- 172.21.15.39:0/1266879282 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc300108e0 con 0x7fcc40101fb0 2024-07-04T08:09:04.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.727+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc401a7fa0 con 0x7fcc40101fb0 2024-07-04T08:09:04.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.727+0000 7fcc37e00640 1 -- 172.21.15.39:0/1266879282 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30018880 con 0x7fcc40101fb0 2024-07-04T08:09:04.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.727+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc401a8450 con 0x7fcc40101fb0 2024-07-04T08:09:04.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.728+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc401023c0 con 0x7fcc40101fb0 2024-07-04T08:09:04.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.731+0000 7fcc37e00640 1 -- 172.21.15.39:0/1266879282 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fcc30010430 con 0x7fcc40101fb0 2024-07-04T08:09:04.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.732+0000 7fcc37e00640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc240b28c0 0x7fcc240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:04.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.733+0000 7fcc37e00640 1 -- 172.21.15.39:0/1266879282 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7fcc300e2720 con 0x7fcc40101fb0 2024-07-04T08:09:04.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.733+0000 7fcc3ea00640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc240b28c0 0x7fcc240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:04.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.734+0000 7fcc3ea00640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc240b28c0 0x7fcc240b4dd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c000c00 tx=0x7fcc2c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:04.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.734+0000 7fcc37e00640 1 -- 172.21.15.39:0/1266879282 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fcc300aa770 con 0x7fcc40101fb0 2024-07-04T08:09:04.767 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:09:04 smithi133 podman[68647]: 2024-07-04 08:09:04.392123507 +0000 UTC m=+0.016959677 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:09:04.767 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:09:04 smithi133 podman[68647]: 2024-07-04 08:09:04.495772458 +0000 UTC m=+0.120608621 container create 1cc5e95b4f354cd6a4e7a30382e2478823e54ae67f0bd0c156513a9909635dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-07-04T08:09:04.767 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:09:04 smithi133 podman[68647]: 2024-07-04 08:09:04.731733008 +0000 UTC m=+0.356569164 container init 1cc5e95b4f354cd6a4e7a30382e2478823e54ae67f0bd0c156513a9909635dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:09:04.767 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:09:04 smithi133 podman[68647]: 2024-07-04 08:09:04.734247891 +0000 UTC m=+0.359084063 container start 1cc5e95b4f354cd6a4e7a30382e2478823e54ae67f0bd0c156513a9909635dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-04T08:09:04.767 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:09:04 smithi133 ceph-mds[68666]: -- 172.21.15.133:0/95483483 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cbe1b9b860 con 0x55cbe5d4da80 2024-07-04T08:09:04.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.993+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc4002de10 con 0x7fcc40101fb0 2024-07-04T08:09:04.995 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:04 smithi039 ceph-mon[32062]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:04.995 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:04.995 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:04.995 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:04.995 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:04.995 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:04.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.994+0000 7fcc37e00640 1 -- 172.21.15.39:0/1266879282 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7fcc300aa070 con 0x7fcc40101fb0 2024-07-04T08:09:04.996 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:09:04.996 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":8,"btime":"2024-07-04T08:09:04:931701+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24661,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-07-04T08:09:04.996 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 8 2024-07-04T08:09:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.998+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc240b28c0 msgr2=0x7fcc240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.998+0000 7fcc46800640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc240b28c0 0x7fcc240b4dd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c000c00 tx=0x7fcc2c031400 comp rx=0 tx=0).stop 2024-07-04T08:09:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.998+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc40101fb0 msgr2=0x7fcc400febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.998+0000 7fcc46800640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc40101fb0 0x7fcc400febc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000e9f0 tx=0x7fcc3000edf0 comp rx=0 tx=0).stop 2024-07-04T08:09:05.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.999+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 shutdown_connections 2024-07-04T08:09:05.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.999+0000 7fcc46800640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc240b28c0 0x7fcc240b4dd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:05.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.999+0000 7fcc46800640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc40108240 0x7fcc400ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:05.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.999+0000 7fcc46800640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc401078f0 0x7fcc400ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:05.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.999+0000 7fcc46800640 1 --2- 172.21.15.39:0/1266879282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc40101fb0 0x7fcc400febc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:05.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.999+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 >> 172.21.15.39:0/1266879282 conn(0x7fcc40078c80 msgr2=0x7fcc4006a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:05.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.999+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 shutdown_connections 2024-07-04T08:09:05.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:04.999+0000 7fcc46800640 1 -- 172.21.15.39:0/1266879282 wait complete. 2024-07-04T08:09:05.066 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:09:04 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c[68662]: starting mds.c at 2024-07-04T08:09:05.067 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:09:04 smithi133 bash[68647]: 1cc5e95b4f354cd6a4e7a30382e2478823e54ae67f0bd0c156513a9909635dda 2024-07-04T08:09:05.067 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:09:04 smithi133 systemd[1]: Started Ceph mds.c for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:09:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:04 smithi133 ceph-mon[37050]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:05.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:04 smithi050 ceph-mon[36577]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:06.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:05 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:05 smithi039 ceph-mon[32062]: Deploying daemon mds.h on smithi050 2024-07-04T08:09:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:05 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] up:boot 2024-07-04T08:09:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:05 smithi039 ceph-mon[32062]: fsmap 7 up:standby 2024-07-04T08:09:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:05 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-04T08:09:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:05 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1266879282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:06.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:05 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:06.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:05 smithi133 ceph-mon[37050]: Deploying daemon mds.h on smithi050 2024-07-04T08:09:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:05 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] up:boot 2024-07-04T08:09:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:05 smithi133 ceph-mon[37050]: fsmap 7 up:standby 2024-07-04T08:09:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:05 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-04T08:09:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:05 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1266879282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:06.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:05 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:06.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:05 smithi050 ceph-mon[36577]: Deploying daemon mds.h on smithi050 2024-07-04T08:09:06.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:05 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] up:boot 2024-07-04T08:09:06.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:05 smithi050 ceph-mon[36577]: fsmap 7 up:standby 2024-07-04T08:09:06.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:05 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-04T08:09:06.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:05 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1266879282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:07.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:06 smithi050 ceph-mon[36577]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:07.145 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:09:06 smithi050 systemd[1]: Starting Ceph mds.h for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:09:07.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:06 smithi133 ceph-mon[37050]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:07.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:06 smithi039 ceph-mon[32062]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:07.474 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:09:07 smithi050 podman[68457]: 2024-07-04 08:09:07.102568247 +0000 UTC m=+0.018562313 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:09:07.474 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:09:07 smithi050 podman[68457]: 2024-07-04 08:09:07.212928809 +0000 UTC m=+0.128922884 container create dab9c6006cf41869ccc7e4c2e0ddf6c2b181a1e2f2796952e565fd27c68712df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-04T08:09:07.808 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:09:07.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:09:07 smithi050 podman[68457]: 2024-07-04 08:09:07.474013266 +0000 UTC m=+0.390007323 container init dab9c6006cf41869ccc7e4c2e0ddf6c2b181a1e2f2796952e565fd27c68712df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True) 2024-07-04T08:09:07.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:09:07 smithi050 podman[68457]: 2024-07-04 08:09:07.476951546 +0000 UTC m=+0.392945603 container start dab9c6006cf41869ccc7e4c2e0ddf6c2b181a1e2f2796952e565fd27c68712df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625) 2024-07-04T08:09:07.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:09:07 smithi050 ceph-mds[68476]: -- 172.21.15.50:0/1860972418 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7886e1860 con 0x55f78c893600 2024-07-04T08:09:07.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:09:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h[68472]: starting mds.h at 2024-07-04T08:09:07.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:09:07 smithi050 bash[68457]: dab9c6006cf41869ccc7e4c2e0ddf6c2b181a1e2f2796952e565fd27c68712df 2024-07-04T08:09:07.895 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:09:07 smithi050 systemd[1]: Started Ceph mds.h for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:09:08.281 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:08.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:08.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:08.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:08.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:08.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:08.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:08.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:08.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: Deploying daemon mds.i on smithi133 2024-07-04T08:09:08.765 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] up:boot 2024-07-04T08:09:08.766 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: fsmap 8 up:standby 2024-07-04T08:09:08.766 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-04T08:09:08.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: Deploying daemon mds.i on smithi133 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] up:boot 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: fsmap 8 up:standby 2024-07-04T08:09:08.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-04T08:09:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: Deploying daemon mds.i on smithi133 2024-07-04T08:09:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] up:boot 2024-07-04T08:09:09.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: fsmap 8 up:standby 2024-07-04T08:09:09.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-04T08:09:09.817 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:09:09 smithi133 systemd[1]: Starting Ceph mds.i for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:09:10.147 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:09:09 smithi133 podman[68959]: 2024-07-04 08:09:09.759391867 +0000 UTC m=+0.016761614 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:09:10.147 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:09:09 smithi133 podman[68959]: 2024-07-04 08:09:09.910623237 +0000 UTC m=+0.167992977 container create 88c7fd526c4f164fe73202cdaa15c206a020ca7073c2ddc3a6c43814a4cca60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-04T08:09:10.147 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:09:10 smithi133 podman[68959]: 2024-07-04 08:09:10.146580422 +0000 UTC m=+0.403950181 container init 88c7fd526c4f164fe73202cdaa15c206a020ca7073c2ddc3a6c43814a4cca60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-07-04T08:09:10.566 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:09:10 smithi133 podman[68959]: 2024-07-04 08:09:10.14959415 +0000 UTC m=+0.406963894 container start 88c7fd526c4f164fe73202cdaa15c206a020ca7073c2ddc3a6c43814a4cca60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, ceph=True) 2024-07-04T08:09:10.567 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:09:10 smithi133 ceph-mds[68978]: -- 172.21.15.133:0/28452129 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a780fb860 con 0x559a7c2ad600 2024-07-04T08:09:10.567 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:09:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i[68974]: starting mds.i at 2024-07-04T08:09:10.567 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:09:10 smithi133 bash[68959]: 88c7fd526c4f164fe73202cdaa15c206a020ca7073c2ddc3a6c43814a4cca60b 2024-07-04T08:09:10.567 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:09:10 smithi133 systemd[1]: Started Ceph mds.i for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:09:10.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.607+0000 7f2044e00640 1 -- 172.21.15.39:0/3055232777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2040103d10 msgr2=0x7f2040104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:10.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.607+0000 7f2044e00640 1 --2- 172.21.15.39:0/3055232777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2040103d10 0x7f2040104120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2030009ca0 tx=0x7f203002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.607+0000 7f2044e00640 1 -- 172.21.15.39:0/3055232777 shutdown_connections 2024-07-04T08:09:10.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.607+0000 7f2044e00640 1 --2- 172.21.15.39:0/3055232777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040104660 0x7f204010cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.607+0000 7f2044e00640 1 --2- 172.21.15.39:0/3055232777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f204010d170 0x7f204010f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.607+0000 7f2044e00640 1 --2- 172.21.15.39:0/3055232777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2040103d10 0x7f2040104120 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.607+0000 7f2044e00640 1 -- 172.21.15.39:0/3055232777 >> 172.21.15.39:0/3055232777 conn(0x7f20400fd810 msgr2=0x7f20400ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:10.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.607+0000 7f2044e00640 1 -- 172.21.15.39:0/3055232777 shutdown_connections 2024-07-04T08:09:10.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.607+0000 7f2044e00640 1 -- 172.21.15.39:0/3055232777 wait complete. 2024-07-04T08:09:10.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.609+0000 7f2044e00640 1 Processor -- start 2024-07-04T08:09:10.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.609+0000 7f2044e00640 1 -- start start 2024-07-04T08:09:10.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f2044e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040103d10 0x7f204019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:10.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f2044e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2040104660 0x7f204019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:10.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f2044e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204010d170 0x7f20401a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:10.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f2044e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112e20 con 0x7f2040103d10 2024-07-04T08:09:10.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f2044e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112ca0 con 0x7f2040104660 2024-07-04T08:09:10.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f2044e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112fa0 con 0x7f204010d170 2024-07-04T08:09:10.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f2036a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2040104660 0x7f204019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:10.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f203fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204010d170 0x7f20401a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:10.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f2036a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2040104660 0x7f204019bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53592/0 (socket says 172.21.15.39:53592) 2024-07-04T08:09:10.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f203fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204010d170 0x7f20401a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59150/0 (socket says 172.21.15.39:59150) 2024-07-04T08:09:10.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f2036a00640 1 -- 172.21.15.39:0/1147298634 learned_addr learned my addr 172.21.15.39:0/1147298634 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.610+0000 7f203f400640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040103d10 0x7f204019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.611+0000 7f203fe00640 1 -- 172.21.15.39:0/1147298634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2040104660 msgr2=0x7f204019bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.611+0000 7f203fe00640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2040104660 0x7f204019bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.611+0000 7f203fe00640 1 -- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040103d10 msgr2=0x7f204019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.611+0000 7f203fe00640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040103d10 0x7f204019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.611+0000 7f203fe00640 1 -- 172.21.15.39:0/1147298634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20401a1980 con 0x7f204010d170 2024-07-04T08:09:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.611+0000 7f2036a00640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2040104660 0x7f204019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:09:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.611+0000 7f203fe00640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204010d170 0x7f20401a1250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f203800e970 tx=0x7f203800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:10.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.612+0000 7f203cc00640 1 -- 172.21.15.39:0/1147298634 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203800ccc0 con 0x7f204010d170 2024-07-04T08:09:10.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.612+0000 7f2044e00640 1 -- 172.21.15.39:0/1147298634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20401a1c70 con 0x7f204010d170 2024-07-04T08:09:10.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.612+0000 7f2044e00640 1 -- 172.21.15.39:0/1147298634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20401a83d0 con 0x7f204010d170 2024-07-04T08:09:10.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.612+0000 7f203cc00640 1 -- 172.21.15.39:0/1147298634 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2038005020 con 0x7f204010d170 2024-07-04T08:09:10.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.612+0000 7f203cc00640 1 -- 172.21.15.39:0/1147298634 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038004a00 con 0x7f204010d170 2024-07-04T08:09:10.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.614+0000 7f2035600640 1 -- 172.21.15.39:0/1147298634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2040104120 con 0x7f204010d170 2024-07-04T08:09:10.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.616+0000 7f203cc00640 1 -- 172.21.15.39:0/1147298634 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f2038004ba0 con 0x7f204010d170 2024-07-04T08:09:10.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.618+0000 7f203cc00640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20240b28c0 0x7f20240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:10.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.618+0000 7f203cc00640 1 -- 172.21.15.39:0/1147298634 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f20380e1f10 con 0x7f204010d170 2024-07-04T08:09:10.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.618+0000 7f203f400640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20240b28c0 0x7f20240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:10.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.619+0000 7f203f400640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20240b28c0 0x7f20240b4dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2030006010 tx=0x7f2030000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:10.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.621+0000 7f203cc00640 1 -- 172.21.15.39:0/1147298634 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f20380a9f60 con 0x7f204010d170 2024-07-04T08:09:10.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.888+0000 7f2035600640 1 -- 172.21.15.39:0/1147298634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f204019cb40 con 0x7f204010d170 2024-07-04T08:09:10.889 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:10 smithi039 ceph-mon[32062]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:10.889 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:10.889 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:10.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:10.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:10.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:10.890 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:10.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.889+0000 7f203cc00640 1 -- 172.21.15.39:0/1147298634 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8043 (secure 0 0 0) 0x7f20380a9860 con 0x7f204010d170 2024-07-04T08:09:10.891 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:09:10.891 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":10,"btime":"2024-07-04T08:09:10:655396+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24661,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24679,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-07-04T08:09:10.891 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 10 2024-07-04T08:09:10.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.891+0000 7f2035600640 1 -- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20240b28c0 msgr2=0x7f20240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:10.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.891+0000 7f2035600640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20240b28c0 0x7f20240b4dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2030006010 tx=0x7f2030000f40 comp rx=0 tx=0).stop 2024-07-04T08:09:10.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 -- 172.21.15.39:0/1147298634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204010d170 msgr2=0x7f20401a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:10.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204010d170 0x7f20401a1250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f203800e970 tx=0x7f203800ee30 comp rx=0 tx=0).stop 2024-07-04T08:09:10.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 -- 172.21.15.39:0/1147298634 shutdown_connections 2024-07-04T08:09:10.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20240b28c0 0x7f20240b4dd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f204010d170 0x7f20401a1250 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2040104660 0x7f204019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 --2- 172.21.15.39:0/1147298634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2040103d10 0x7f204019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:10.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 -- 172.21.15.39:0/1147298634 >> 172.21.15.39:0/1147298634 conn(0x7f20400fd810 msgr2=0x7f2040101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:10.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 -- 172.21.15.39:0/1147298634 shutdown_connections 2024-07-04T08:09:10.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:10.892+0000 7f2035600640 1 -- 172.21.15.39:0/1147298634 wait complete. 2024-07-04T08:09:11.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:10 smithi133 ceph-mon[37050]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:11.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:11.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:11.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:11.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:11.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:11.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:10 smithi050 ceph-mon[36577]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:11.885 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:11 smithi039 ceph-mon[32062]: Deploying daemon mds.g on smithi039 2024-07-04T08:09:11.885 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:11 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] up:boot 2024-07-04T08:09:11.885 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:11 smithi039 ceph-mon[32062]: fsmap 9 up:standby 2024-07-04T08:09:11.885 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-04T08:09:11.885 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:11 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1147298634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:12.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:11 smithi133 ceph-mon[37050]: Deploying daemon mds.g on smithi039 2024-07-04T08:09:12.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:11 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] up:boot 2024-07-04T08:09:12.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:11 smithi133 ceph-mon[37050]: fsmap 9 up:standby 2024-07-04T08:09:12.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-04T08:09:12.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:11 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1147298634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:12.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:11 smithi050 ceph-mon[36577]: Deploying daemon mds.g on smithi039 2024-07-04T08:09:12.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:11 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] up:boot 2024-07-04T08:09:12.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:11 smithi050 ceph-mon[36577]: fsmap 9 up:standby 2024-07-04T08:09:12.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-04T08:09:12.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:11 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1147298634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:12.892 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:12 smithi039 ceph-mon[32062]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:13.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:12 smithi133 ceph-mon[37050]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:13.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:12 smithi050 ceph-mon[36577]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:13.700 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:09:14.425 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:09:14 smithi039 podman[78383]: 2024-07-04 08:09:14.053675701 +0000 UTC m=+0.033667700 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:09:14.425 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:09:14 smithi039 podman[78383]: 2024-07-04 08:09:14.422955597 +0000 UTC m=+0.402947646 container init dc3524787faaaa36d46967ad5404726518b3bb604ba9eb0da21c03f1664089d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-g, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:09:14.639 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:14.707 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:09:14 smithi039 podman[78383]: 2024-07-04 08:09:14.434822859 +0000 UTC m=+0.414814935 container start dc3524787faaaa36d46967ad5404726518b3bb604ba9eb0da21c03f1664089d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-04T08:09:14.707 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:09:14 smithi039 bash[78383]: dc3524787faaaa36d46967ad5404726518b3bb604ba9eb0da21c03f1664089d9 2024-07-04T08:09:14.707 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:09:14 smithi039 systemd[1]: Started Ceph mds.g for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:09:14.707 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:09:14 smithi039 ceph-mds[78427]: -- 172.21.15.39:0/2669462811 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556de8891860 con 0x556deca43180 2024-07-04T08:09:14.707 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:09:14 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-g[78423]: starting mds.g at 2024-07-04T08:09:15.001 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:14 smithi039 ceph-mon[32062]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:15.001 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:15.001 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:15.001 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:15.001 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:15.001 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:15.001 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:15.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:14 smithi133 ceph-mon[37050]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:15.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:15.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:15.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:15.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:15.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:15.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:15.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:14 smithi050 ceph-mon[36577]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:16.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:15 smithi133 ceph-mon[37050]: Deploying daemon mds.a on smithi039 2024-07-04T08:09:16.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:15 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.39:6838/2419136306,v1:172.21.15.39:6839/2419136306] up:boot 2024-07-04T08:09:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:15 smithi133 ceph-mon[37050]: fsmap 10 up:standby 2024-07-04T08:09:16.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-04T08:09:16.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:15 smithi039 ceph-mon[32062]: Deploying daemon mds.a on smithi039 2024-07-04T08:09:16.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:15 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.39:6838/2419136306,v1:172.21.15.39:6839/2419136306] up:boot 2024-07-04T08:09:16.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:15 smithi039 ceph-mon[32062]: fsmap 10 up:standby 2024-07-04T08:09:16.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-04T08:09:16.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:15 smithi050 ceph-mon[36577]: Deploying daemon mds.a on smithi039 2024-07-04T08:09:16.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:15 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.39:6838/2419136306,v1:172.21.15.39:6839/2419136306] up:boot 2024-07-04T08:09:16.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:15 smithi050 ceph-mon[36577]: fsmap 10 up:standby 2024-07-04T08:09:16.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-04T08:09:17.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:16 smithi039 ceph-mon[32062]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:17.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:16 smithi133 ceph-mon[37050]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:17.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:16 smithi050 ceph-mon[36577]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:17.977 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:17 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1437368333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:18.046 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:17 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1437368333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:18.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:17 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1437368333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:18.121 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:09:18.121 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":11,"btime":"2024-07-04T08:09:14:698268+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24661,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24679,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-07-04T08:09:18.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.379+0000 7f9e6f200640 1 -- 172.21.15.39:0/1150240032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010aa30 msgr2=0x7f9e7010ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:18.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.380+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1150240032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010aa30 0x7f9e7010ae40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009900 tx=0x7f9e5802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.380+0000 7f9e6f200640 1 -- 172.21.15.39:0/1150240032 shutdown_connections 2024-07-04T08:09:18.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.380+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1150240032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010aa30 0x7f9e7010ae40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.380+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1150240032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e7010b380 0x7f9e70111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.380+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1150240032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e701050f0 0x7f9e70105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.380+0000 7f9e6f200640 1 -- 172.21.15.39:0/1150240032 >> 172.21.15.39:0/1150240032 conn(0x7f9e70100b60 msgr2=0x7f9e70102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:18.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.380+0000 7f9e6f200640 1 -- 172.21.15.39:0/1150240032 shutdown_connections 2024-07-04T08:09:18.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.380+0000 7f9e6f200640 1 -- 172.21.15.39:0/1150240032 wait complete. 2024-07-04T08:09:18.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.381+0000 7f9e6f200640 1 Processor -- start 2024-07-04T08:09:18.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.381+0000 7f9e6f200640 1 -- start start 2024-07-04T08:09:18.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.381+0000 7f9e6f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e701050f0 0x7f9e7019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:18.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7010aa30 0x7f9e7019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:18.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010b380 0x7f9e701a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:18.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e70115040 con 0x7f9e7010b380 2024-07-04T08:09:18.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e70114ec0 con 0x7f9e701050f0 2024-07-04T08:09:18.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e701151c0 con 0x7f9e7010aa30 2024-07-04T08:09:18.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010b380 0x7f9e701a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:18.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010b380 0x7f9e701a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37928/0 (socket says 172.21.15.39:37928) 2024-07-04T08:09:18.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e67e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7010aa30 0x7f9e7019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:18.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6e800640 1 -- 172.21.15.39:0/1437368333 learned_addr learned my addr 172.21.15.39:0/1437368333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:18.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6de00640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e701050f0 0x7f9e7019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6e800640 1 -- 172.21.15.39:0/1437368333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7010aa30 msgr2=0x7f9e7019bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6e800640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7010aa30 0x7f9e7019bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6e800640 1 -- 172.21.15.39:0/1437368333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e701050f0 msgr2=0x7f9e7019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6e800640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e701050f0 0x7f9e7019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.382+0000 7f9e6e800640 1 -- 172.21.15.39:0/1437368333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e701a1980 con 0x7f9e7010b380 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.383+0000 7f9e6e800640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010b380 0x7f9e701a1250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800eb10 tx=0x7f9e6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.383+0000 7f9e66a00640 1 -- 172.21.15.39:0/1437368333 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68004b20 con 0x7f9e7010b380 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.383+0000 7f9e66a00640 1 -- 172.21.15.39:0/1437368333 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e68004cc0 con 0x7f9e7010b380 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.383+0000 7f9e66a00640 1 -- 172.21.15.39:0/1437368333 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68010670 con 0x7f9e7010b380 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.383+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e701a1c70 con 0x7f9e7010b380 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.383+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e701a83d0 con 0x7f9e7010b380 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.384+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e70105500 con 0x7f9e7010b380 2024-07-04T08:09:18.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.388+0000 7f9e66a00640 1 -- 172.21.15.39:0/1437368333 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f9e68005000 con 0x7f9e7010b380 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.390+0000 7f9e66a00640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e540b2840 0x7f9e540b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.390+0000 7f9e66a00640 1 -- 172.21.15.39:0/1437368333 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f9e680e2680 con 0x7f9e7010b380 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.390+0000 7f9e6de00640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e540b2840 0x7f9e540b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.391+0000 7f9e6de00640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e540b2840 0x7f9e540b4d50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e600095f0 tx=0x7f9e60005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.391+0000 7f9e66a00640 1 -- 172.21.15.39:0/1437368333 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f9e68055110 con 0x7f9e7010b380 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.654+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e7019cb40 con 0x7f9e7010b380 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.655+0000 7f9e66a00640 1 -- 172.21.15.39:0/1437368333 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7f9e680aa6d0 con 0x7f9e7010b380 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 11 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e540b2840 msgr2=0x7f9e540b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e540b2840 0x7f9e540b4d50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e600095f0 tx=0x7f9e60005f90 comp rx=0 tx=0).stop 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010b380 msgr2=0x7f9e701a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:18.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010b380 0x7f9e701a1250 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800eb10 tx=0x7f9e6800c2d0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 shutdown_connections 2024-07-04T08:09:18.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e540b2840 0x7f9e540b4d50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e7010b380 0x7f9e701a1250 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e7010aa30 0x7f9e7019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 --2- 172.21.15.39:0/1437368333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e701050f0 0x7f9e7019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:18.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 >> 172.21.15.39:0/1437368333 conn(0x7f9e70100b60 msgr2=0x7f9e70102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:18.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 shutdown_connections 2024-07-04T08:09:18.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:17.659+0000 7f9e6f200640 1 -- 172.21.15.39:0/1437368333 wait complete. 2024-07-04T08:09:18.790 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:18 smithi039 ceph-mon[32062]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:19.062 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:18 smithi050 ceph-mon[36577]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:19.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:18 smithi133 ceph-mon[37050]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:19.564 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:09:19 smithi039 podman[78815]: 2024-07-04 08:09:19.283445709 +0000 UTC m=+0.165531834 container create c8d3c9318c52d513d27443997d6dce74ed25659db5df6e5f5f9396a9bee912cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625) 2024-07-04T08:09:19.565 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:09:19 smithi039 podman[78815]: 2024-07-04 08:09:19.562961483 +0000 UTC m=+0.445047632 container init c8d3c9318c52d513d27443997d6dce74ed25659db5df6e5f5f9396a9bee912cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:09:19.852 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:09:19 smithi039 podman[78815]: 2024-07-04 08:09:19.574251689 +0000 UTC m=+0.456337834 container start c8d3c9318c52d513d27443997d6dce74ed25659db5df6e5f5f9396a9bee912cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-04T08:09:19.852 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:09:19 smithi039 bash[78815]: c8d3c9318c52d513d27443997d6dce74ed25659db5df6e5f5f9396a9bee912cf 2024-07-04T08:09:19.852 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:09:19 smithi039 systemd[1]: Started Ceph mds.a for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:09:19.852 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:09:19 smithi039 ceph-mds[78834]: -- 172.21.15.39:0/3376760807 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55617c40b520 con 0x5561805b3600 2024-07-04T08:09:19.852 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:09:19 smithi039 ceph-mds[78834]: -- 172.21.15.39:0/3376760807 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55617c401860 con 0x5561805b3600 2024-07-04T08:09:19.852 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:09:19 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a[78830]: starting mds.a at 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] up:boot 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: fsmap 11 up:standby 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:20.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:20.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:20 smithi050 ceph-mon[36577]: Deploying daemon mds.b on smithi050 2024-07-04T08:09:20.933 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:09:21.017 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] up:boot 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: fsmap 11 up:standby 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:21.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:20 smithi039 ceph-mon[32062]: Deploying daemon mds.b on smithi050 2024-07-04T08:09:21.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:21.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] up:boot 2024-07-04T08:09:21.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: fsmap 11 up:standby 2024-07-04T08:09:21.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-04T08:09:21.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:21.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:21.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:21.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-04T08:09:21.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-04T08:09:21.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:21.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:20 smithi133 ceph-mon[37050]: Deploying daemon mds.b on smithi050 2024-07-04T08:09:21.416 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:21.895 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:21 smithi050 systemd[1]: Starting Ceph mds.b for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:09:22.264 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:21 smithi050 podman[68772]: 2024-07-04 08:09:21.88880783 +0000 UTC m=+0.017839727 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:09:22.264 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:22 smithi050 podman[68772]: 2024-07-04 08:09:22.00244772 +0000 UTC m=+0.131479614 container create 4c3b3ec82ea0845436c5c8f3ef55571e78e8a5b16b3148ff3757d0e440a99c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625) 2024-07-04T08:09:22.264 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:22 smithi050 podman[68772]: 2024-07-04 08:09:22.263606181 +0000 UTC m=+0.392638060 container init 4c3b3ec82ea0845436c5c8f3ef55571e78e8a5b16b3148ff3757d0e440a99c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-04T08:09:22.645 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:22 smithi050 podman[68772]: 2024-07-04 08:09:22.266606952 +0000 UTC m=+0.395638837 container start 4c3b3ec82ea0845436c5c8f3ef55571e78e8a5b16b3148ff3757d0e440a99c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-04T08:09:22.645 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:22 smithi050 ceph-mds[68791]: -- 172.21.15.50:0/463010647 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556acd34d860 con 0x556ad14ffa80 2024-07-04T08:09:22.645 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:22 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b[68787]: starting mds.b at 2024-07-04T08:09:22.645 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:22 smithi050 bash[68772]: 4c3b3ec82ea0845436c5c8f3ef55571e78e8a5b16b3148ff3757d0e440a99c0f 2024-07-04T08:09:22.645 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:22 smithi050 systemd[1]: Started Ceph mds.b for 2ee561ca-39db-11ef-bcab-c7b262605968. 2024-07-04T08:09:23.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:22 smithi133 ceph-mon[37050]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:23.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:22 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:22 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:22 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:22 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:22 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:22 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:09:23.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:22 smithi039 ceph-mon[32062]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:23.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:22 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:22 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:22 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:22 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:22 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:22 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:09:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:22 smithi050 ceph-mon[36577]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:22 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:22 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:22 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:22 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:22 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:22 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:09:23.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.680+0000 7f2118200640 1 -- 172.21.15.39:0/1244626224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f211010b560 msgr2=0x7f2110111d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:23.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.680+0000 7f2118200640 1 --2- 172.21.15.39:0/1244626224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f211010b560 0x7f2110111d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2108009be0 tx=0x7f210802a6c0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.680+0000 7f2118200640 1 -- 172.21.15.39:0/1244626224 shutdown_connections 2024-07-04T08:09:23.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.680+0000 7f2118200640 1 --2- 172.21.15.39:0/1244626224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f211010b560 0x7f2110111d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.680+0000 7f2118200640 1 --2- 172.21.15.39:0/1244626224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f211010ac10 0x7f211010b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.680+0000 7f2118200640 1 --2- 172.21.15.39:0/1244626224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21101052d0 0x7f21101056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.680+0000 7f2118200640 1 -- 172.21.15.39:0/1244626224 >> 172.21.15.39:0/1244626224 conn(0x7f2110100d60 msgr2=0x7f21101031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:23.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.680+0000 7f2118200640 1 -- 172.21.15.39:0/1244626224 shutdown_connections 2024-07-04T08:09:23.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.681+0000 7f2118200640 1 -- 172.21.15.39:0/1244626224 wait complete. 2024-07-04T08:09:23.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.682+0000 7f2118200640 1 Processor -- start 2024-07-04T08:09:23.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.682+0000 7f2118200640 1 -- start start 2024-07-04T08:09:23.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2118200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21101052d0 0x7f211019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:23.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2118200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f211010ac10 0x7f211019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:23.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2118200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211010b560 0x7f21101a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:23.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2118200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2110115210 con 0x7f21101052d0 2024-07-04T08:09:23.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2118200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2110115090 con 0x7f211010ac10 2024-07-04T08:09:23.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2118200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2110115390 con 0x7f211010b560 2024-07-04T08:09:23.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2115800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21101052d0 0x7f211019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:23.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2115800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21101052d0 0x7f211019bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56526/0 (socket says 172.21.15.39:56526) 2024-07-04T08:09:23.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2115800640 1 -- 172.21.15.39:0/3144508045 learned_addr learned my addr 172.21.15.39:0/3144508045 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:23.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2114e00640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f211010ac10 0x7f211019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:23.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.683+0000 7f2116200640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211010b560 0x7f21101a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:23.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.684+0000 7f2115800640 1 -- 172.21.15.39:0/3144508045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211010b560 msgr2=0x7f21101a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:23.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.684+0000 7f2115800640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211010b560 0x7f21101a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.684+0000 7f2115800640 1 -- 172.21.15.39:0/3144508045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f211010ac10 msgr2=0x7f211019c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:23.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.684+0000 7f2115800640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f211010ac10 0x7f211019c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.684+0000 7f2115800640 1 -- 172.21.15.39:0/3144508045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21101a1b90 con 0x7f21101052d0 2024-07-04T08:09:23.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.684+0000 7f2115800640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21101052d0 0x7f211019bc20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f210400e9f0 tx=0x7f210400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:23.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.684+0000 7f210e000640 1 -- 172.21.15.39:0/3144508045 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210400ccc0 con 0x7f21101052d0 2024-07-04T08:09:23.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.685+0000 7f210e000640 1 -- 172.21.15.39:0/3144508045 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21040108e0 con 0x7f21101052d0 2024-07-04T08:09:23.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.685+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21101a1e80 con 0x7f21101052d0 2024-07-04T08:09:23.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.685+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21101a85e0 con 0x7f21101052d0 2024-07-04T08:09:23.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.685+0000 7f210e000640 1 -- 172.21.15.39:0/3144508045 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104018880 con 0x7f21101052d0 2024-07-04T08:09:23.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.686+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21101056e0 con 0x7f21101052d0 2024-07-04T08:09:23.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.689+0000 7f210e000640 1 -- 172.21.15.39:0/3144508045 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f2104010430 con 0x7f21101052d0 2024-07-04T08:09:23.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.691+0000 7f210e000640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20f80b2840 0x7f20f80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:23.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.692+0000 7f210e000640 1 -- 172.21.15.39:0/3144508045 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f21040e2330 con 0x7f21101052d0 2024-07-04T08:09:23.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.692+0000 7f2114e00640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20f80b2840 0x7f20f80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:23.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.693+0000 7f2114e00640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20f80b2840 0x7f20f80b4d50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2110104460 tx=0x7f20fc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:23.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.693+0000 7f210e000640 1 -- 172.21.15.39:0/3144508045 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f21040aa380 con 0x7f21101052d0 2024-07-04T08:09:23.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.963+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f211019cd50 con 0x7f21101052d0 2024-07-04T08:09:23.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.964+0000 7f210e000640 1 -- 172.21.15.39:0/3144508045 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7f21040b2400 con 0x7f21101052d0 2024-07-04T08:09:23.967 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:09:23.967 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":13,"btime":"2024-07-04T08:09:22:767303+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24661,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24679,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24707,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-07-04T08:09:23.967 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 13 2024-07-04T08:09:23.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.967+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20f80b2840 msgr2=0x7f20f80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:23.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.967+0000 7f2118200640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20f80b2840 0x7f20f80b4d50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2110104460 tx=0x7f20fc009260 comp rx=0 tx=0).stop 2024-07-04T08:09:23.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.967+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21101052d0 msgr2=0x7f211019bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:23.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.968+0000 7f2118200640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21101052d0 0x7f211019bc20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f210400e9f0 tx=0x7f210400edf0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.968+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 shutdown_connections 2024-07-04T08:09:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.968+0000 7f2118200640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f20f80b2840 0x7f20f80b4d50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.968+0000 7f2118200640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211010b560 0x7f21101a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.968+0000 7f2118200640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f211010ac10 0x7f211019c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.968+0000 7f2118200640 1 --2- 172.21.15.39:0/3144508045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f21101052d0 0x7f211019bc20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.968+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 >> 172.21.15.39:0/3144508045 conn(0x7f2110100d60 msgr2=0x7f211010a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.968+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 shutdown_connections 2024-07-04T08:09:23.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:23.968+0000 7f2118200640 1 -- 172.21.15.39:0/3144508045 wait complete. 2024-07-04T08:09:24.048 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:23 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] up:boot 2024-07-04T08:09:24.048 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:23 smithi039 ceph-mon[32062]: fsmap 12 up:standby 2024-07-04T08:09:24.048 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-04T08:09:24.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:23 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] up:boot 2024-07-04T08:09:24.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:23 smithi133 ceph-mon[37050]: fsmap 12 up:standby 2024-07-04T08:09:24.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-04T08:09:24.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:23 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] up:boot 2024-07-04T08:09:24.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:23 smithi050 ceph-mon[36577]: fsmap 12 up:standby 2024-07-04T08:09:24.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-04T08:09:25.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:24 smithi133 ceph-mon[37050]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:25.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:24 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3144508045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:25.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:24 smithi039 ceph-mon[32062]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:25.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:24 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3144508045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:25.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:24 smithi050 ceph-mon[36577]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:25.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:24 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3144508045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:09:25.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:25.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-07-04T08:09:25.705 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:27.033 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:26 smithi039 ceph-mon[32062]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:27.033 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:26 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:26 smithi133 ceph-mon[37050]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:26 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:27.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:26 smithi050 ceph-mon[36577]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:27.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:26 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:27.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.876+0000 7f71ee000640 1 -- 172.21.15.39:0/2435081821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e8108a00 msgr2=0x7f71e8108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.876+0000 7f71ee000640 1 --2- 172.21.15.39:0/2435081821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e8108a00 0x7f71e8108e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009900 tx=0x7f71d402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:09:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.876+0000 7f71ee000640 1 -- 172.21.15.39:0/2435081821 shutdown_connections 2024-07-04T08:09:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.876+0000 7f71ee000640 1 --2- 172.21.15.39:0/2435081821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71e8109350 0x7f71e810fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.876+0000 7f71ee000640 1 --2- 172.21.15.39:0/2435081821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e8108a00 0x7f71e8108e10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.876+0000 7f71ee000640 1 --2- 172.21.15.39:0/2435081821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e81030c0 0x7f71e81034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.876+0000 7f71ee000640 1 -- 172.21.15.39:0/2435081821 >> 172.21.15.39:0/2435081821 conn(0x7f71e80feb30 msgr2=0x7f71e8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.877+0000 7f71ee000640 1 -- 172.21.15.39:0/2435081821 shutdown_connections 2024-07-04T08:09:27.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.877+0000 7f71ee000640 1 -- 172.21.15.39:0/2435081821 wait complete. 2024-07-04T08:09:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.878+0000 7f71ee000640 1 Processor -- start 2024-07-04T08:09:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.878+0000 7f71ee000640 1 -- start start 2024-07-04T08:09:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71ee000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e81030c0 0x7f71e8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71ee000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71e8108a00 0x7f71e80751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71ee000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e8109350 0x7f71e80756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71ee000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8113030 con 0x7f71e8109350 2024-07-04T08:09:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71ee000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8112eb0 con 0x7f71e81030c0 2024-07-04T08:09:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71ee000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81131b0 con 0x7f71e8108a00 2024-07-04T08:09:27.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e81030c0 0x7f71e8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:27.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e81030c0 0x7f71e8074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:55684/0 (socket says 172.21.15.39:55684) 2024-07-04T08:09:27.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e7400640 1 -- 172.21.15.39:0/2946175793 learned_addr learned my addr 172.21.15.39:0/2946175793 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:27.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e7e00640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e8109350 0x7f71e80756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:27.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e6a00640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71e8108a00 0x7f71e80751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e7400640 1 -- 172.21.15.39:0/2946175793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71e8108a00 msgr2=0x7f71e80751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e7400640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71e8108a00 0x7f71e80751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e7400640 1 -- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e8109350 msgr2=0x7f71e80756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e7400640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e8109350 0x7f71e80756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.879+0000 7f71e7400640 1 -- 172.21.15.39:0/2946175793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e81a7c80 con 0x7f71e81030c0 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.880+0000 7f71e7400640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e81030c0 0x7f71e8074c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71d800e9f0 tx=0x7f71d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.880+0000 7f71dfe00640 1 -- 172.21.15.39:0/2946175793 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d800cd80 con 0x7f71e81030c0 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.880+0000 7f71dfe00640 1 -- 172.21.15.39:0/2946175793 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d80108e0 con 0x7f71e81030c0 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.880+0000 7f71dfe00640 1 -- 172.21.15.39:0/2946175793 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8018980 con 0x7f71e81030c0 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.880+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e81a7f70 con 0x7f71e81030c0 2024-07-04T08:09:27.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.880+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e81a8420 con 0x7f71e81030c0 2024-07-04T08:09:27.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.882+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e81034d0 con 0x7f71e81030c0 2024-07-04T08:09:27.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.884+0000 7f71dfe00640 1 -- 172.21.15.39:0/2946175793 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f71d8010430 con 0x7f71e81030c0 2024-07-04T08:09:27.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.886+0000 7f71dfe00640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f71cc0b28c0 0x7f71cc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:27.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.886+0000 7f71dfe00640 1 -- 172.21.15.39:0/2946175793 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7f71d80e21a0 con 0x7f71e81030c0 2024-07-04T08:09:27.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.886+0000 7f71e6a00640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f71cc0b28c0 0x7f71cc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:27.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.887+0000 7f71e6a00640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f71cc0b28c0 0x7f71cc0b4dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f71d402a9e0 tx=0x7f71d4005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:27.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:27.889+0000 7f71dfe00640 1 -- 172.21.15.39:0/2946175793 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f71d80a9ba0 con 0x7f71e81030c0 2024-07-04T08:09:28.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:28.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:28.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:28.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:09:28.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:28.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:27 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:09:28.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:28.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:28.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:28.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:09:28.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:28.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:27 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:09:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:09:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:28.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:27 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:09:28.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.150+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f71e802de10 con 0x7f71e81030c0 2024-07-04T08:09:28.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.151+0000 7f71dfe00640 1 -- 172.21.15.39:0/2946175793 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7f71d80adea0 con 0x7f71e81030c0 2024-07-04T08:09:28.153 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:09:28.153 INFO:teuthology.orchestra.run.smithi039.stdout:[ 2024-07-04T08:09:28.153 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:09:28.153 INFO:teuthology.orchestra.run.smithi039.stdout: "poolnum": 1, 2024-07-04T08:09:28.153 INFO:teuthology.orchestra.run.smithi039.stdout: "poolname": ".mgr" 2024-07-04T08:09:28.153 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:09:28.153 INFO:teuthology.orchestra.run.smithi039.stdout:] 2024-07-04T08:09:28.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.153+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f71cc0b28c0 msgr2=0x7f71cc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:28.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.153+0000 7f71ee000640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f71cc0b28c0 0x7f71cc0b4dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f71d402a9e0 tx=0x7f71d4005a20 comp rx=0 tx=0).stop 2024-07-04T08:09:28.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.153+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e81030c0 msgr2=0x7f71e8074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:28.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.153+0000 7f71ee000640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e81030c0 0x7f71e8074c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f71d800e9f0 tx=0x7f71d800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:09:28.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.154+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 shutdown_connections 2024-07-04T08:09:28.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.154+0000 7f71ee000640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f71cc0b28c0 0x7f71cc0b4dd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:28.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.154+0000 7f71ee000640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e8109350 0x7f71e80756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:28.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.154+0000 7f71ee000640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71e8108a00 0x7f71e80751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:28.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.154+0000 7f71ee000640 1 --2- 172.21.15.39:0/2946175793 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f71e81030c0 0x7f71e8074c60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:28.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.154+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 >> 172.21.15.39:0/2946175793 conn(0x7f71e80feb30 msgr2=0x7f71e8100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:28.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.154+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 shutdown_connections 2024-07-04T08:09:28.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:28.154+0000 7f71ee000640 1 -- 172.21.15.39:0/2946175793 wait complete. 2024-07-04T08:09:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:28 smithi039 ceph-mon[32062]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:28 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2946175793' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-04T08:09:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:28 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:28 smithi050 ceph-mon[36577]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:29.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:28 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2946175793' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-04T08:09:29.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:28 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:28 smithi133 ceph-mon[37050]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:29.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:28 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2946175793' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-04T08:09:29.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:28 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:29.587 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-04T08:09:29.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-04T08:09:30.065 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:31.136 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:30 smithi039 ceph-mon[32062]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:31.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:30 smithi050 ceph-mon[36577]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:31.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:30 smithi133 ceph-mon[37050]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:32.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.394+0000 7ff33da00640 1 -- 172.21.15.39:0/1318732271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33810f7a0 msgr2=0x7ff338115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:32.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.394+0000 7ff33da00640 1 --2- 172.21.15.39:0/1318732271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33810f7a0 0x7ff338115fd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff330009ca0 tx=0x7ff33002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:09:32.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.394+0000 7ff33da00640 1 -- 172.21.15.39:0/1318732271 shutdown_connections 2024-07-04T08:09:32.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.394+0000 7ff33da00640 1 --2- 172.21.15.39:0/1318732271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33810f7a0 0x7ff338115fd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:32.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.394+0000 7ff33da00640 1 --2- 172.21.15.39:0/1318732271 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff33810ee50 0x7ff33810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.394+0000 7ff33da00640 1 --2- 172.21.15.39:0/1318732271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff338109510 0x7ff338109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.394+0000 7ff33da00640 1 -- 172.21.15.39:0/1318732271 >> 172.21.15.39:0/1318732271 conn(0x7ff338078b10 msgr2=0x7ff338107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.395+0000 7ff33da00640 1 -- 172.21.15.39:0/1318732271 shutdown_connections 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.395+0000 7ff33da00640 1 -- 172.21.15.39:0/1318732271 wait complete. 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.396+0000 7ff33da00640 1 Processor -- start 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.396+0000 7ff33da00640 1 -- start start 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.396+0000 7ff33da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff338109510 0x7ff33819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.396+0000 7ff33da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff33810ee50 0x7ff3381a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.396+0000 7ff33da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33810f7a0 0x7ff3381a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.396+0000 7ff33da00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff338119470 con 0x7ff33810f7a0 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.396+0000 7ff33da00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3381192f0 con 0x7ff33810ee50 2024-07-04T08:09:32.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.396+0000 7ff33da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff3381195f0 con 0x7ff338109510 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.396+0000 7ff337e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff338109510 0x7ff33819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff338109510 0x7ff33819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34868/0 (socket says 172.21.15.39:34868) 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337e00640 1 -- 172.21.15.39:0/2065642298 learned_addr learned my addr 172.21.15.39:0/2065642298 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337400640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff33810ee50 0x7ff3381a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff33d000640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33810f7a0 0x7ff3381a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337e00640 1 -- 172.21.15.39:0/2065642298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff33810ee50 msgr2=0x7ff3381a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337e00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff33810ee50 0x7ff3381a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337e00640 1 -- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33810f7a0 msgr2=0x7ff3381a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337e00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33810f7a0 0x7ff3381a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:32.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337e00640 1 -- 172.21.15.39:0/2065642298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3381a5d10 con 0x7ff338109510 2024-07-04T08:09:32.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337400640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff33810ee50 0x7ff3381a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:09:32.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.397+0000 7ff337e00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff338109510 0x7ff33819fda0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9f0 tx=0x7ff32800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:32.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.398+0000 7ff334c00640 1 -- 172.21.15.39:0/2065642298 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32800cd80 con 0x7ff338109510 2024-07-04T08:09:32.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.398+0000 7ff334c00640 1 -- 172.21.15.39:0/2065642298 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3280108e0 con 0x7ff338109510 2024-07-04T08:09:32.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.398+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3381a6000 con 0x7ff338109510 2024-07-04T08:09:32.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.398+0000 7ff334c00640 1 -- 172.21.15.39:0/2065642298 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328018980 con 0x7ff338109510 2024-07-04T08:09:32.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.398+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3381ac760 con 0x7ff338109510 2024-07-04T08:09:32.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.399+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff338109920 con 0x7ff338109510 2024-07-04T08:09:32.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.401+0000 7ff334c00640 1 -- 172.21.15.39:0/2065642298 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7ff328010430 con 0x7ff338109510 2024-07-04T08:09:32.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.403+0000 7ff334c00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff31c0b28c0 0x7ff31c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:32.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.403+0000 7ff334c00640 1 -- 172.21.15.39:0/2065642298 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7342+0+0 (secure 0 0 0) 0x7ff3280e2690 con 0x7ff338109510 2024-07-04T08:09:32.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.403+0000 7ff337400640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff31c0b28c0 0x7ff31c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:32.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.404+0000 7ff337400640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff31c0b28c0 0x7ff31c0b4dd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff33807ad50 tx=0x7ff324005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:32.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.404+0000 7ff334c00640 1 -- 172.21.15.39:0/2065642298 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7ff3280aa090 con 0x7ff338109510 2024-07-04T08:09:32.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:32.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:32.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:32.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:09:32.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:32 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:32.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:32 smithi133 ceph-mon[37050]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:32.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:32.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:32.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:32.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:09:32.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:32 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:32.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:32 smithi039 ceph-mon[32062]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:32.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:32.620+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff33802de10 con 0x7ff338109510 2024-07-04T08:09:32.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:32.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:32.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:09:32.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:09:32.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:32 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:09:32.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:32 smithi050 ceph-mon[36577]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:33.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.269+0000 7ff334c00640 1 -- 172.21.15.39:0/2065642298 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7ff3280ae390 con 0x7ff338109510 2024-07-04T08:09:33.272 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' created 2024-07-04T08:09:33.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.276+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff31c0b28c0 msgr2=0x7ff31c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:33.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.276+0000 7ff33da00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff31c0b28c0 0x7ff31c0b4dd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff33807ad50 tx=0x7ff324005f90 comp rx=0 tx=0).stop 2024-07-04T08:09:33.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.276+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff338109510 msgr2=0x7ff33819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:33.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.276+0000 7ff33da00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff338109510 0x7ff33819fda0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff32800e9f0 tx=0x7ff32800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:09:33.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.277+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 shutdown_connections 2024-07-04T08:09:33.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.277+0000 7ff33da00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff31c0b28c0 0x7ff31c0b4dd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:33.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.277+0000 7ff33da00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff33810f7a0 0x7ff3381a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:33.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.278+0000 7ff33da00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff33810ee50 0x7ff3381a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:33.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.278+0000 7ff33da00640 1 --2- 172.21.15.39:0/2065642298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff338109510 0x7ff33819fda0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:33.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.278+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 >> 172.21.15.39:0/2065642298 conn(0x7ff338078b10 msgr2=0x7ff338079130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:33.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.280+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 shutdown_connections 2024-07-04T08:09:33.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:33.280+0000 7ff33da00640 1 -- 172.21.15.39:0/2065642298 wait complete. 2024-07-04T08:09:33.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:33 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2065642298' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-04T08:09:33.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:33 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-04T08:09:33.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:33 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2065642298' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-04T08:09:33.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:33 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-04T08:09:33.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:33 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2065642298' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-04T08:09:33.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:33 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-04T08:09:34.188 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-07-04T08:09:34.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:34 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-04T08:09:34.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:34 smithi133 ceph-mon[37050]: osdmap e75: 12 total, 12 up, 12 in 2024-07-04T08:09:34.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:34 smithi133 ceph-mon[37050]: pgmap v294: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:34.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:34 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-04T08:09:34.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:34 smithi039 ceph-mon[32062]: osdmap e75: 12 total, 12 up, 12 in 2024-07-04T08:09:34.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:34 smithi039 ceph-mon[32062]: pgmap v294: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:34.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:34 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-04T08:09:34.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:34 smithi050 ceph-mon[36577]: osdmap e75: 12 total, 12 up, 12 in 2024-07-04T08:09:34.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:34 smithi050 ceph-mon[36577]: pgmap v294: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:34.680 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:35.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:35 smithi039 ceph-mon[32062]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-04T08:09:35.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:35 smithi039 ceph-mon[32062]: osdmap e76: 12 total, 12 up, 12 in 2024-07-04T08:09:35.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:35 smithi133 ceph-mon[37050]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-04T08:09:35.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:35 smithi133 ceph-mon[37050]: osdmap e76: 12 total, 12 up, 12 in 2024-07-04T08:09:35.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:35 smithi050 ceph-mon[36577]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-04T08:09:35.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:35 smithi050 ceph-mon[36577]: osdmap e76: 12 total, 12 up, 12 in 2024-07-04T08:09:36.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:36 smithi039 ceph-mon[32062]: osdmap e77: 12 total, 12 up, 12 in 2024-07-04T08:09:36.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:36 smithi039 ceph-mon[32062]: pgmap v297: 33 pgs: 1 creating+activating, 1 creating+peering, 15 unknown, 16 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:36.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:36 smithi133 ceph-mon[37050]: osdmap e77: 12 total, 12 up, 12 in 2024-07-04T08:09:36.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:36 smithi133 ceph-mon[37050]: pgmap v297: 33 pgs: 1 creating+activating, 1 creating+peering, 15 unknown, 16 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:36.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:36 smithi050 ceph-mon[36577]: osdmap e77: 12 total, 12 up, 12 in 2024-07-04T08:09:36.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:36 smithi050 ceph-mon[36577]: pgmap v297: 33 pgs: 1 creating+activating, 1 creating+peering, 15 unknown, 16 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:36.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.996+0000 7f2a24400640 1 -- 172.21.15.39:0/3367294385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a1c103f20 msgr2=0x7f2a1c104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:36.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.996+0000 7f2a24400640 1 --2- 172.21.15.39:0/3367294385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a1c103f20 0x7f2a1c104330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009900 tx=0x7f2a0c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:09:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.997+0000 7f2a24400640 1 -- 172.21.15.39:0/3367294385 shutdown_connections 2024-07-04T08:09:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.997+0000 7f2a24400640 1 --2- 172.21.15.39:0/3367294385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c104870 0x7f2a1c10cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.997+0000 7f2a24400640 1 --2- 172.21.15.39:0/3367294385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a1c10d380 0x7f2a1c10f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.997+0000 7f2a24400640 1 --2- 172.21.15.39:0/3367294385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a1c103f20 0x7f2a1c104330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.997+0000 7f2a24400640 1 -- 172.21.15.39:0/3367294385 >> 172.21.15.39:0/3367294385 conn(0x7f2a1c078c90 msgr2=0x7f2a1c0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.997+0000 7f2a24400640 1 -- 172.21.15.39:0/3367294385 shutdown_connections 2024-07-04T08:09:36.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.997+0000 7f2a24400640 1 -- 172.21.15.39:0/3367294385 wait complete. 2024-07-04T08:09:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.998+0000 7f2a24400640 1 Processor -- start 2024-07-04T08:09:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.998+0000 7f2a24400640 1 -- start start 2024-07-04T08:09:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a24400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c103f20 0x7f2a1c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a24400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a1c104870 0x7f2a1c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a24400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a1c10d380 0x7f2a1c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a24400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c113060 con 0x7f2a1c103f20 2024-07-04T08:09:37.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a24400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c112ee0 con 0x7f2a1c10d380 2024-07-04T08:09:37.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a24400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c1131e0 con 0x7f2a1c104870 2024-07-04T08:09:37.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a21a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c103f20 0x7f2a1c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:37.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a21a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c103f20 0x7f2a1c19bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57372/0 (socket says 172.21.15.39:57372) 2024-07-04T08:09:37.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a21a00640 1 -- 172.21.15.39:0/3037001465 learned_addr learned my addr 172.21.15.39:0/3037001465 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:37.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a21000640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a1c104870 0x7f2a1c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:37.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:36.999+0000 7f2a22400640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a1c10d380 0x7f2a1c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a21000640 1 -- 172.21.15.39:0/3037001465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a1c10d380 msgr2=0x7f2a1c1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a21000640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a1c10d380 0x7f2a1c1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a21000640 1 -- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c103f20 msgr2=0x7f2a1c19bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a21000640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c103f20 0x7f2a1c19bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a21000640 1 -- 172.21.15.39:0/3037001465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c1a1c00 con 0x7f2a1c104870 2024-07-04T08:09:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a21a00640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c103f20 0x7f2a1c19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:09:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a21000640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a1c104870 0x7f2a1c19c230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c06aea0 tx=0x7f2a0800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a1a000640 1 -- 172.21.15.39:0/3037001465 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08010430 con 0x7f2a1c104870 2024-07-04T08:09:37.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a1a000640 1 -- 172.21.15.39:0/3037001465 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a08010a80 con 0x7f2a1c104870 2024-07-04T08:09:37.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a1a000640 1 -- 172.21.15.39:0/3037001465 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08017a20 con 0x7f2a1c104870 2024-07-04T08:09:37.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1c1a1ef0 con 0x7f2a1c104870 2024-07-04T08:09:37.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.000+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1c1a8650 con 0x7f2a1c104870 2024-07-04T08:09:37.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.002+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1c06b790 con 0x7f2a1c104870 2024-07-04T08:09:37.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.004+0000 7f2a1a000640 1 -- 172.21.15.39:0/3037001465 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f2a08017c00 con 0x7f2a1c104870 2024-07-04T08:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.006+0000 7f2a1a000640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2a000b28c0 0x7f2a000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.006+0000 7f2a1a000640 1 -- 172.21.15.39:0/3037001465 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7712+0+0 (secure 0 0 0) 0x7f2a080e2970 con 0x7f2a1c104870 2024-07-04T08:09:37.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.006+0000 7f2a21a00640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2a000b28c0 0x7f2a000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:37.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.007+0000 7f2a21a00640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2a000b28c0 0x7f2a000b4dd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:37.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.010+0000 7f2a1a000640 1 -- 172.21.15.39:0/3037001465 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f2a080aa870 con 0x7f2a1c104870 2024-07-04T08:09:37.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.224+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f2a1c1a8950 con 0x7f2a1c104870 2024-07-04T08:09:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.342+0000 7f2a1a000640 1 -- 172.21.15.39:0/3037001465 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v78) ==== 171+0+0 (secure 0 0 0) 0x7f2a080aa170 con 0x7f2a1c104870 2024-07-04T08:09:37.344 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' created 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.347+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2a000b28c0 msgr2=0x7f2a000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.347+0000 7f2a24400640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2a000b28c0 0x7f2a000b4dd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c005a20 comp rx=0 tx=0).stop 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.347+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a1c104870 msgr2=0x7f2a1c19c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.347+0000 7f2a24400640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a1c104870 0x7f2a1c19c230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c06aea0 tx=0x7f2a0800ef90 comp rx=0 tx=0).stop 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.348+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 shutdown_connections 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.348+0000 7f2a24400640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2a000b28c0 0x7f2a000b4dd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.348+0000 7f2a24400640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a1c10d380 0x7f2a1c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.348+0000 7f2a24400640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a1c104870 0x7f2a1c19c230 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.348+0000 7f2a24400640 1 --2- 172.21.15.39:0/3037001465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a1c103f20 0x7f2a1c19bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:37.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.348+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 >> 172.21.15.39:0/3037001465 conn(0x7f2a1c078c90 msgr2=0x7f2a1c10c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:37.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.349+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 shutdown_connections 2024-07-04T08:09:37.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:37.349+0000 7f2a24400640 1 -- 172.21.15.39:0/3037001465 wait complete. 2024-07-04T08:09:37.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:37 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3037001465' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-04T08:09:37.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:37 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-04T08:09:37.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:37 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3037001465' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-04T08:09:37.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:37 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-04T08:09:37.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:37 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3037001465' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-04T08:09:37.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:37 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-04T08:09:38.233 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-04T08:09:38.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:38 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-07-04T08:09:38.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:38 smithi039 ceph-mon[32062]: osdmap e78: 12 total, 12 up, 12 in 2024-07-04T08:09:38.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:38 smithi039 ceph-mon[32062]: pgmap v299: 97 pgs: 1 creating+activating, 1 creating+peering, 67 unknown, 28 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:38.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:38 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-07-04T08:09:38.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:38 smithi050 ceph-mon[36577]: osdmap e78: 12 total, 12 up, 12 in 2024-07-04T08:09:38.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:38 smithi050 ceph-mon[36577]: pgmap v299: 97 pgs: 1 creating+activating, 1 creating+peering, 67 unknown, 28 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:38.729 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:38.732 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:38 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-07-04T08:09:38.732 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:38 smithi133 ceph-mon[37050]: osdmap e78: 12 total, 12 up, 12 in 2024-07-04T08:09:38.732 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:38 smithi133 ceph-mon[37050]: pgmap v299: 97 pgs: 1 creating+activating, 1 creating+peering, 67 unknown, 28 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:39.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:39 smithi039 ceph-mon[32062]: osdmap e79: 12 total, 12 up, 12 in 2024-07-04T08:09:39.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:39 smithi050 ceph-mon[36577]: osdmap e79: 12 total, 12 up, 12 in 2024-07-04T08:09:39.656 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:39 smithi133 ceph-mon[37050]: osdmap e79: 12 total, 12 up, 12 in 2024-07-04T08:09:40.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:40 smithi050 ceph-mon[36577]: osdmap e80: 12 total, 12 up, 12 in 2024-07-04T08:09:40.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:40 smithi050 ceph-mon[36577]: pgmap v302: 97 pgs: 6 creating+peering, 44 unknown, 47 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:40.654 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:40 smithi039 ceph-mon[32062]: osdmap e80: 12 total, 12 up, 12 in 2024-07-04T08:09:40.654 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:40 smithi039 ceph-mon[32062]: pgmap v302: 97 pgs: 6 creating+peering, 44 unknown, 47 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:40.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:40 smithi133 ceph-mon[37050]: osdmap e80: 12 total, 12 up, 12 in 2024-07-04T08:09:40.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:40 smithi133 ceph-mon[37050]: pgmap v302: 97 pgs: 6 creating+peering, 44 unknown, 47 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:40.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.982+0000 7fdbc6800640 1 -- 172.21.15.39:0/1459597949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc0105100 msgr2=0x7fdbc0105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:40.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.982+0000 7fdbc6800640 1 --2- 172.21.15.39:0/1459597949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc0105100 0x7fdbc0105510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009900 tx=0x7fdbb402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:09:40.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.982+0000 7fdbc6800640 1 -- 172.21.15.39:0/1459597949 shutdown_connections 2024-07-04T08:09:40.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.982+0000 7fdbc6800640 1 --2- 172.21.15.39:0/1459597949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc010b390 0x7fdbc0111bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:40.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.982+0000 7fdbc6800640 1 --2- 172.21.15.39:0/1459597949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc010aa40 0x7fdbc010ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:40.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.982+0000 7fdbc6800640 1 --2- 172.21.15.39:0/1459597949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc0105100 0x7fdbc0105510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:40.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.982+0000 7fdbc6800640 1 -- 172.21.15.39:0/1459597949 >> 172.21.15.39:0/1459597949 conn(0x7fdbc0100bd0 msgr2=0x7fdbc0103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:40.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.983+0000 7fdbc6800640 1 -- 172.21.15.39:0/1459597949 shutdown_connections 2024-07-04T08:09:40.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.983+0000 7fdbc6800640 1 -- 172.21.15.39:0/1459597949 wait complete. 2024-07-04T08:09:40.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.984+0000 7fdbc6800640 1 Processor -- start 2024-07-04T08:09:40.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.984+0000 7fdbc6800640 1 -- start start 2024-07-04T08:09:40.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc6800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc0105100 0x7fdbc019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:40.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc010aa40 0x7fdbc019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:40.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc010b390 0x7fdbc01a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:40.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc6800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc01150b0 con 0x7fdbc010b390 2024-07-04T08:09:40.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc6800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0114f30 con 0x7fdbc010aa40 2024-07-04T08:09:40.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc6800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0115230 con 0x7fdbc0105100 2024-07-04T08:09:40.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc5400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc0105100 0x7fdbc019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc010b390 0x7fdbc01a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc010aa40 0x7fdbc019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc5400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc0105100 0x7fdbc019ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34914/0 (socket says 172.21.15.39:34914) 2024-07-04T08:09:40.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.985+0000 7fdbc5400640 1 -- 172.21.15.39:0/2614982542 learned_addr learned my addr 172.21.15.39:0/2614982542 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:40.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbc5400640 1 -- 172.21.15.39:0/2614982542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc010aa40 msgr2=0x7fdbc019c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:40.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbc5400640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc010aa40 0x7fdbc019c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:40.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbc5400640 1 -- 172.21.15.39:0/2614982542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc010b390 msgr2=0x7fdbc01a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:40.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbc5400640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc010b390 0x7fdbc01a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:40.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbc5400640 1 -- 172.21.15.39:0/2614982542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc01a19f0 con 0x7fdbc0105100 2024-07-04T08:09:40.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbc4a00640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc010aa40 0x7fdbc019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:09:40.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbc5400640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc0105100 0x7fdbc019ba80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4005ab0 tx=0x7fdbb4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:40.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbbe000640 1 -- 172.21.15.39:0/2614982542 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb403f070 con 0x7fdbc0105100 2024-07-04T08:09:40.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbbe000640 1 -- 172.21.15.39:0/2614982542 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbb4031b60 con 0x7fdbc0105100 2024-07-04T08:09:40.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc01a1c80 con 0x7fdbc0105100 2024-07-04T08:09:40.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.986+0000 7fdbbe000640 1 -- 172.21.15.39:0/2614982542 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4035030 con 0x7fdbc0105100 2024-07-04T08:09:40.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.987+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc01a8440 con 0x7fdbc0105100 2024-07-04T08:09:40.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.988+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc0105510 con 0x7fdbc0105100 2024-07-04T08:09:40.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.990+0000 7fdbbe000640 1 -- 172.21.15.39:0/2614982542 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fdbb4042070 con 0x7fdbc0105100 2024-07-04T08:09:40.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.992+0000 7fdbbe000640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdba80b28c0 0x7fdba80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:40.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.992+0000 7fdbbe000640 1 -- 172.21.15.39:0/2614982542 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8094+0+0 (secure 0 0 0) 0x7fdbb4102ed0 con 0x7fdbc0105100 2024-07-04T08:09:40.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.992+0000 7fdbc4a00640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdba80b28c0 0x7fdba80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:40.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.993+0000 7fdbc4a00640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdba80b28c0 0x7fdba80b4dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0095f0 tx=0x7fdbac005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:40.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:40.993+0000 7fdbbe000640 1 -- 172.21.15.39:0/2614982542 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fdbb40ca5e0 con 0x7fdbc0105100 2024-07-04T08:09:41.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.250+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdbc010d170 con 0x7fdbc0105100 2024-07-04T08:09:41.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.402+0000 7fdbbe000640 1 -- 172.21.15.39:0/2614982542 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fdbb40ce8e0 con 0x7fdbc0105100 2024-07-04T08:09:41.404 INFO:teuthology.orchestra.run.smithi039.stderr:new fs with metadata pool 2 and data pool 3 2024-07-04T08:09:41.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.407+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdba80b28c0 msgr2=0x7fdba80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:41.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.407+0000 7fdbc6800640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdba80b28c0 0x7fdba80b4dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0095f0 tx=0x7fdbac005f90 comp rx=0 tx=0).stop 2024-07-04T08:09:41.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.407+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc0105100 msgr2=0x7fdbc019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:41.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.407+0000 7fdbc6800640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc0105100 0x7fdbc019ba80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4005ab0 tx=0x7fdbb4003c10 comp rx=0 tx=0).stop 2024-07-04T08:09:41.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.408+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 shutdown_connections 2024-07-04T08:09:41.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.408+0000 7fdbc6800640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdba80b28c0 0x7fdba80b4dd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:41.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.408+0000 7fdbc6800640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbc010b390 0x7fdbc01a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:41.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.408+0000 7fdbc6800640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbc010aa40 0x7fdbc019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:41.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.408+0000 7fdbc6800640 1 --2- 172.21.15.39:0/2614982542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbc0105100 0x7fdbc019ba80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:41.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.408+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 >> 172.21.15.39:0/2614982542 conn(0x7fdbc0100bd0 msgr2=0x7fdbc0102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:41.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.408+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 shutdown_connections 2024-07-04T08:09:41.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:41.409+0000 7fdbc6800640 1 -- 172.21.15.39:0/2614982542 wait complete. 2024-07-04T08:09:41.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:41 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2614982542' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-04T08:09:41.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:41 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-04T08:09:41.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:41 smithi039 ceph-mon[32062]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-04T08:09:41.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:41 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2614982542' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-04T08:09:41.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:41 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-04T08:09:41.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:41 smithi050 ceph-mon[36577]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-04T08:09:41.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:41 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2614982542' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-04T08:09:41.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:41 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-04T08:09:41.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:41 smithi133 ceph-mon[37050]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-04T08:09:42.330 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-07-04T08:09:42.330 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-07-04T08:09:42.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:42 smithi039 ceph-mon[32062]: osdmap e81: 12 total, 12 up, 12 in 2024-07-04T08:09:42.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:42 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-04T08:09:42.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:42 smithi039 ceph-mon[32062]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-04T08:09:42.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:42 smithi039 ceph-mon[32062]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-04T08:09:42.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:42 smithi039 ceph-mon[32062]: pgmap v304: 97 pgs: 6 creating+peering, 4 unknown, 87 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:42.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:42 smithi050 ceph-mon[36577]: osdmap e81: 12 total, 12 up, 12 in 2024-07-04T08:09:42.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:42 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-04T08:09:42.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:42 smithi050 ceph-mon[36577]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-04T08:09:42.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:42 smithi050 ceph-mon[36577]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-04T08:09:42.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:42 smithi050 ceph-mon[36577]: pgmap v304: 97 pgs: 6 creating+peering, 4 unknown, 87 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:42 smithi133 ceph-mon[37050]: osdmap e81: 12 total, 12 up, 12 in 2024-07-04T08:09:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:42 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-04T08:09:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:42 smithi133 ceph-mon[37050]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-04T08:09:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:42 smithi133 ceph-mon[37050]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-04T08:09:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:42 smithi133 ceph-mon[37050]: pgmap v304: 97 pgs: 6 creating+peering, 4 unknown, 87 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:43.145 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:09:42 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b[68787]: 2024-07-04T08:09:42.786+0000 7f7b0cfab640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-04T08:09:43.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:43 smithi039 ceph-mon[32062]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-04T08:09:43.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:43 smithi039 ceph-mon[32062]: Cluster is now healthy 2024-07-04T08:09:43.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:43 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] up:active 2024-07-04T08:09:43.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:43 smithi039 ceph-mon[32062]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:09:43.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:43 smithi039 ceph-mon[32062]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:09:43.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:43 smithi050 ceph-mon[36577]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-04T08:09:43.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:43 smithi050 ceph-mon[36577]: Cluster is now healthy 2024-07-04T08:09:43.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:43 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] up:active 2024-07-04T08:09:43.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:43 smithi050 ceph-mon[36577]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:09:43.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:43 smithi050 ceph-mon[36577]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:09:43.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:43 smithi133 ceph-mon[37050]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-04T08:09:43.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:43 smithi133 ceph-mon[37050]: Cluster is now healthy 2024-07-04T08:09:43.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:43 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] up:active 2024-07-04T08:09:43.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:43 smithi133 ceph-mon[37050]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:09:43.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:43 smithi133 ceph-mon[37050]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:09:44.690 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:44 smithi039 ceph-mon[32062]: pgmap v305: 97 pgs: 97 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-07-04T08:09:44.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:44 smithi133 ceph-mon[37050]: pgmap v305: 97 pgs: 97 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-07-04T08:09:44.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:44 smithi050 ceph-mon[36577]: pgmap v305: 97 pgs: 97 active+clean; 578 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-07-04T08:09:45.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.079+0000 7f2461200640 1 -- 172.21.15.39:0/2130588323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c108a00 msgr2=0x7f245c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:45.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.079+0000 7f2461200640 1 --2- 172.21.15.39:0/2130588323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c108a00 0x7f245c108e10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2448009900 tx=0x7f244802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.079+0000 7f2461200640 1 -- 172.21.15.39:0/2130588323 shutdown_connections 2024-07-04T08:09:45.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.079+0000 7f2461200640 1 --2- 172.21.15.39:0/2130588323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c108a00 0x7f245c108e10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.079+0000 7f2461200640 1 --2- 172.21.15.39:0/2130588323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f245c109350 0x7f245c10fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.079+0000 7f2461200640 1 --2- 172.21.15.39:0/2130588323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f245c1030c0 0x7f245c1034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.079+0000 7f2461200640 1 -- 172.21.15.39:0/2130588323 >> 172.21.15.39:0/2130588323 conn(0x7f245c0feb30 msgr2=0x7f245c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:45.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.079+0000 7f2461200640 1 -- 172.21.15.39:0/2130588323 shutdown_connections 2024-07-04T08:09:45.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.080+0000 7f2461200640 1 -- 172.21.15.39:0/2130588323 wait complete. 2024-07-04T08:09:45.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.081+0000 7f2461200640 1 Processor -- start 2024-07-04T08:09:45.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.081+0000 7f2461200640 1 -- start start 2024-07-04T08:09:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2461200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c1030c0 0x7f245c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2461200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f245c108a00 0x7f245c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2461200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f245c109350 0x7f245c1a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2461200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f245c113010 con 0x7f245c1030c0 2024-07-04T08:09:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2461200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f245c112e90 con 0x7f245c109350 2024-07-04T08:09:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2461200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f245c113190 con 0x7f245c108a00 2024-07-04T08:09:45.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2459e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c1030c0 0x7f245c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2459e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c1030c0 0x7f245c19bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55050/0 (socket says 172.21.15.39:55050) 2024-07-04T08:09:45.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2459e00640 1 -- 172.21.15.39:0/3767072610 learned_addr learned my addr 172.21.15.39:0/3767072610 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.082+0000 7f2459400640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f245c108a00 0x7f245c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.083+0000 7f245a800640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f245c109350 0x7f245c1a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.083+0000 7f2459e00640 1 -- 172.21.15.39:0/3767072610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f245c108a00 msgr2=0x7f245c19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.083+0000 7f2459e00640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f245c108a00 0x7f245c19c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.083+0000 7f2459e00640 1 -- 172.21.15.39:0/3767072610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f245c109350 msgr2=0x7f245c1a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.083+0000 7f2459e00640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f245c109350 0x7f245c1a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.083+0000 7f2459e00640 1 -- 172.21.15.39:0/3767072610 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c1a1b70 con 0x7f245c1030c0 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.083+0000 7f245a800640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f245c109350 0x7f245c1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.083+0000 7f2459e00640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c1030c0 0x7f245c19bc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e9f0 tx=0x7f244c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:45.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.084+0000 7f2452a00640 1 -- 172.21.15.39:0/3767072610 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c00cd80 con 0x7f245c1030c0 2024-07-04T08:09:45.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.084+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c1a1e60 con 0x7f245c1030c0 2024-07-04T08:09:45.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.084+0000 7f2452a00640 1 -- 172.21.15.39:0/3767072610 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f244c0108e0 con 0x7f245c1030c0 2024-07-04T08:09:45.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.084+0000 7f2452a00640 1 -- 172.21.15.39:0/3767072610 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c018980 con 0x7f245c1030c0 2024-07-04T08:09:45.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.084+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c1a85c0 con 0x7f245c1030c0 2024-07-04T08:09:45.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.085+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c1034d0 con 0x7f245c1030c0 2024-07-04T08:09:45.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.088+0000 7f2452a00640 1 -- 172.21.15.39:0/3767072610 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f244c010430 con 0x7f245c1030c0 2024-07-04T08:09:45.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.090+0000 7f2452a00640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24400b2840 0x7f24400b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:45.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.090+0000 7f2452a00640 1 -- 172.21.15.39:0/3767072610 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8194+0+0 (secure 0 0 0) 0x7f244c0e2b00 con 0x7f245c1030c0 2024-07-04T08:09:45.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.090+0000 7f2459400640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24400b2840 0x7f24400b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:45.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.091+0000 7f2459400640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24400b2840 0x7f24400b4d50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f244802a9e0 tx=0x7f2448005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:45.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.093+0000 7f2452a00640 1 -- 172.21.15.39:0/3767072610 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f244c0aa7e0 con 0x7f245c1030c0 2024-07-04T08:09:45.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.313+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f245c1a88f0 con 0x7f245c1030c0 2024-07-04T08:09:45.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.414+0000 7f2452a00640 1 -- 172.21.15.39:0/3767072610 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v82) ==== 157+0+0 (secure 0 0 0) 0x7f244c0aa0e0 con 0x7f245c1030c0 2024-07-04T08:09:45.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.420+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24400b2840 msgr2=0x7f24400b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:45.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.420+0000 7f2461200640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24400b2840 0x7f24400b4d50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f244802a9e0 tx=0x7f2448005a20 comp rx=0 tx=0).stop 2024-07-04T08:09:45.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.420+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c1030c0 msgr2=0x7f245c19bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:45.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.420+0000 7f2461200640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c1030c0 0x7f245c19bc00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e9f0 tx=0x7f244c00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.421+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 shutdown_connections 2024-07-04T08:09:45.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.421+0000 7f2461200640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24400b2840 0x7f24400b4d50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.421+0000 7f2461200640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f245c109350 0x7f245c1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.421+0000 7f2461200640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f245c108a00 0x7f245c19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.421+0000 7f2461200640 1 --2- 172.21.15.39:0/3767072610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f245c1030c0 0x7f245c19bc00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:45.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.421+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 >> 172.21.15.39:0/3767072610 conn(0x7f245c0feb30 msgr2=0x7f245c10de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:45.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.421+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 shutdown_connections 2024-07-04T08:09:45.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:45.422+0000 7f2461200640 1 -- 172.21.15.39:0/3767072610 wait complete. 2024-07-04T08:09:45.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:45 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3767072610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-07-04T08:09:45.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:45 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3767072610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-07-04T08:09:45.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:45 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3767072610' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-07-04T08:09:46.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-07-04T08:09:46.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:46 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3767072610' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-07-04T08:09:46.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:46 smithi039 ceph-mon[32062]: osdmap e82: 12 total, 12 up, 12 in 2024-07-04T08:09:46.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:46 smithi039 ceph-mon[32062]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-07-04T08:09:46.798 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:46.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:46 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3767072610' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-07-04T08:09:46.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:46 smithi133 ceph-mon[37050]: osdmap e82: 12 total, 12 up, 12 in 2024-07-04T08:09:46.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:46 smithi133 ceph-mon[37050]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-07-04T08:09:46.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:46 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3767072610' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-07-04T08:09:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:46 smithi050 ceph-mon[36577]: osdmap e82: 12 total, 12 up, 12 in 2024-07-04T08:09:46.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:46 smithi050 ceph-mon[36577]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-07-04T08:09:48.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:48 smithi039 ceph-mon[32062]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-04T08:09:48.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:48 smithi050 ceph-mon[36577]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-04T08:09:49.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:48 smithi133 ceph-mon[37050]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-04T08:09:49.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.264+0000 7fc3c1e00640 1 -- 172.21.15.39:0/1058555136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 msgr2=0x7fc3bc10fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:49.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.264+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/1058555136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 0x7fc3bc10fb80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4009be0 tx=0x7fc3b402a6c0 comp rx=0 tx=0).stop 2024-07-04T08:09:49.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.264+0000 7fc3c1e00640 1 -- 172.21.15.39:0/1058555136 shutdown_connections 2024-07-04T08:09:49.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.264+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/1058555136 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 0x7fc3bc10fb80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:49.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.264+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/1058555136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc108a00 0x7fc3bc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:49.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.264+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/1058555136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3bc1030c0 0x7fc3bc1034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:49.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.264+0000 7fc3c1e00640 1 -- 172.21.15.39:0/1058555136 >> 172.21.15.39:0/1058555136 conn(0x7fc3bc0feaf0 msgr2=0x7fc3bc100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:49.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.264+0000 7fc3c1e00640 1 -- 172.21.15.39:0/1058555136 shutdown_connections 2024-07-04T08:09:49.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.264+0000 7fc3c1e00640 1 -- 172.21.15.39:0/1058555136 wait complete. 2024-07-04T08:09:49.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.265+0000 7fc3c1e00640 1 Processor -- start 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.265+0000 7fc3c1e00640 1 -- start start 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc1030c0 0x7fc3bc074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3bc108a00 0x7fc3bc0751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 0x7fc3bc0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3c1e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc113020 con 0x7fc3bc108a00 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3c1e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc112ea0 con 0x7fc3bc109350 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3c1e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc3bc1131a0 con 0x7fc3bc1030c0 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3bb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 0x7fc3bc0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3bb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 0x7fc3bc0756e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50458/0 (socket says 172.21.15.39:50458) 2024-07-04T08:09:49.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3bb400640 1 -- 172.21.15.39:0/91126315 learned_addr learned my addr 172.21.15.39:0/91126315 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3ba000640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3bc108a00 0x7fc3bc0751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.266+0000 7fc3baa00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc1030c0 0x7fc3bc074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.267+0000 7fc3bb400640 1 -- 172.21.15.39:0/91126315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc1030c0 msgr2=0x7fc3bc074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.267+0000 7fc3bb400640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc1030c0 0x7fc3bc074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.267+0000 7fc3bb400640 1 -- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3bc108a00 msgr2=0x7fc3bc0751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.267+0000 7fc3bb400640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3bc108a00 0x7fc3bc0751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.267+0000 7fc3bb400640 1 -- 172.21.15.39:0/91126315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3bc1a7c30 con 0x7fc3bc109350 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.267+0000 7fc3bb400640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 0x7fc3bc0756e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4004af0 tx=0x7fc3b4003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.267+0000 7fc3c0a00640 1 -- 172.21.15.39:0/91126315 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4041020 con 0x7fc3bc109350 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.268+0000 7fc3c0a00640 1 -- 172.21.15.39:0/91126315 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b4003e10 con 0x7fc3bc109350 2024-07-04T08:09:49.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.268+0000 7fc3c0a00640 1 -- 172.21.15.39:0/91126315 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4036040 con 0x7fc3bc109350 2024-07-04T08:09:49.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.268+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3bc1a7ec0 con 0x7fc3bc109350 2024-07-04T08:09:49.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.268+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3bc1a8370 con 0x7fc3bc109350 2024-07-04T08:09:49.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.269+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3bc1034d0 con 0x7fc3bc109350 2024-07-04T08:09:49.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.271+0000 7fc3c0a00640 1 -- 172.21.15.39:0/91126315 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fc3b4044020 con 0x7fc3bc109350 2024-07-04T08:09:49.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.273+0000 7fc3c0a00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc3a00b28c0 0x7fc3a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:49.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.273+0000 7fc3c0a00640 1 -- 172.21.15.39:0/91126315 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8492+0+0 (secure 0 0 0) 0x7fc3b4102ee0 con 0x7fc3bc109350 2024-07-04T08:09:49.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.273+0000 7fc3baa00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc3a00b28c0 0x7fc3a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:49.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.274+0000 7fc3baa00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc3a00b28c0 0x7fc3a00b4dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0096c0 tx=0x7fc3ac009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:49.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.274+0000 7fc3c0a00640 1 -- 172.21.15.39:0/91126315 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc3b404b030 con 0x7fc3bc109350 2024-07-04T08:09:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:49.487+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc3bc02de10 con 0x7fc3bc109350 2024-07-04T08:09:49.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:49 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/91126315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-04T08:09:49.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:49 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-04T08:09:49.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:49 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/91126315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-04T08:09:49.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:49 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-04T08:09:50.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:49 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/91126315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-04T08:09:50.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:49 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-04T08:09:50.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.596+0000 7fc3c0a00640 1 -- 172.21.15.39:0/91126315 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v84) ==== 228+0+0 (secure 0 0 0) 0x7fc3b4030970 con 0x7fc3bc109350 2024-07-04T08:09:50.603 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data_ec' created 2024-07-04T08:09:50.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc3a00b28c0 msgr2=0x7fc3a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:50.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc3a00b28c0 0x7fc3a00b4dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0096c0 tx=0x7fc3ac009260 comp rx=0 tx=0).stop 2024-07-04T08:09:50.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 msgr2=0x7fc3bc0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:50.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 0x7fc3bc0756e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4004af0 tx=0x7fc3b4003c20 comp rx=0 tx=0).stop 2024-07-04T08:09:50.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 shutdown_connections 2024-07-04T08:09:50.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc3a00b28c0 0x7fc3a00b4dd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:50.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc3bc108a00 0x7fc3bc0751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:50.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3bc109350 0x7fc3bc0756e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:50.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 --2- 172.21.15.39:0/91126315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3bc1030c0 0x7fc3bc074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:50.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.604+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 >> 172.21.15.39:0/91126315 conn(0x7fc3bc0feaf0 msgr2=0x7fc3bc107fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:50.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.607+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 shutdown_connections 2024-07-04T08:09:50.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:50.608+0000 7fc3c1e00640 1 -- 172.21.15.39:0/91126315 wait complete. 2024-07-04T08:09:50.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:50 smithi039 ceph-mon[32062]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-04T08:09:50.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:50 smithi039 ceph-mon[32062]: osdmap e83: 12 total, 12 up, 12 in 2024-07-04T08:09:50.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:50 smithi050 ceph-mon[36577]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-04T08:09:50.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:50 smithi050 ceph-mon[36577]: osdmap e83: 12 total, 12 up, 12 in 2024-07-04T08:09:51.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:50 smithi133 ceph-mon[37050]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-04T08:09:51.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:50 smithi133 ceph-mon[37050]: osdmap e83: 12 total, 12 up, 12 in 2024-07-04T08:09:51.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-07-04T08:09:51.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:51 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-04T08:09:51.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:51 smithi039 ceph-mon[32062]: osdmap e84: 12 total, 12 up, 12 in 2024-07-04T08:09:51.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:51 smithi039 ceph-mon[32062]: osdmap e85: 12 total, 12 up, 12 in 2024-07-04T08:09:51.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:51 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-04T08:09:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:51 smithi050 ceph-mon[36577]: osdmap e84: 12 total, 12 up, 12 in 2024-07-04T08:09:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:51 smithi050 ceph-mon[36577]: osdmap e85: 12 total, 12 up, 12 in 2024-07-04T08:09:51.994 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:52.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:51 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-04T08:09:52.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:51 smithi133 ceph-mon[37050]: osdmap e84: 12 total, 12 up, 12 in 2024-07-04T08:09:52.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:51 smithi133 ceph-mon[37050]: osdmap e85: 12 total, 12 up, 12 in 2024-07-04T08:09:52.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:52 smithi039 ceph-mon[32062]: pgmap v312: 129 pgs: 15 creating+peering, 17 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-07-04T08:09:52.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:52 smithi039 ceph-mon[32062]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-04T08:09:52.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:52 smithi039 ceph-mon[32062]: osdmap e86: 12 total, 12 up, 12 in 2024-07-04T08:09:52.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:52 smithi050 ceph-mon[36577]: pgmap v312: 129 pgs: 15 creating+peering, 17 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-07-04T08:09:52.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:52 smithi050 ceph-mon[36577]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-04T08:09:52.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:52 smithi050 ceph-mon[36577]: osdmap e86: 12 total, 12 up, 12 in 2024-07-04T08:09:53.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:52 smithi133 ceph-mon[37050]: pgmap v312: 129 pgs: 15 creating+peering, 17 unknown, 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 0 op/s 2024-07-04T08:09:53.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:52 smithi133 ceph-mon[37050]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-04T08:09:53.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:52 smithi133 ceph-mon[37050]: osdmap e86: 12 total, 12 up, 12 in 2024-07-04T08:09:54.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.341+0000 7fe842800640 1 -- 172.21.15.39:0/1416149185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c113cc0 msgr2=0x7fe83c1160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:54.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.341+0000 7fe842800640 1 --2- 172.21.15.39:0/1416149185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c113cc0 0x7fe83c1160e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe834009ca0 tx=0x7fe83402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:09:54.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.341+0000 7fe842800640 1 -- 172.21.15.39:0/1416149185 shutdown_connections 2024-07-04T08:09:54.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.341+0000 7fe842800640 1 --2- 172.21.15.39:0/1416149185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c113cc0 0x7fe83c1160e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:54.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.341+0000 7fe842800640 1 --2- 172.21.15.39:0/1416149185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c0777e0 0x7fe83c113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:54.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.341+0000 7fe842800640 1 --2- 172.21.15.39:0/1416149185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c076e90 0x7fe83c0772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:54.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.341+0000 7fe842800640 1 -- 172.21.15.39:0/1416149185 >> 172.21.15.39:0/1416149185 conn(0x7fe83c100d10 msgr2=0x7fe83c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:54.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.341+0000 7fe842800640 1 -- 172.21.15.39:0/1416149185 shutdown_connections 2024-07-04T08:09:54.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.341+0000 7fe842800640 1 -- 172.21.15.39:0/1416149185 wait complete. 2024-07-04T08:09:54.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.342+0000 7fe842800640 1 Processor -- start 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.342+0000 7fe842800640 1 -- start start 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe842800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c076e90 0x7fe83c19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe842800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c0777e0 0x7fe83c1a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe842800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c113cc0 0x7fe83c1a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe842800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c119630 con 0x7fe83c076e90 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe842800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1194b0 con 0x7fe83c113cc0 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe842800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1197b0 con 0x7fe83c0777e0 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe83b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c076e90 0x7fe83c19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe83b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c076e90 0x7fe83c19ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52364/0 (socket says 172.21.15.39:52364) 2024-07-04T08:09:54.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe83b400640 1 -- 172.21.15.39:0/1448535238 learned_addr learned my addr 172.21.15.39:0/1448535238 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:54.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe83aa00640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c0777e0 0x7fe83c1a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:54.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.343+0000 7fe83be00640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c113cc0 0x7fe83c1a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:54.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.344+0000 7fe83b400640 1 -- 172.21.15.39:0/1448535238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c0777e0 msgr2=0x7fe83c1a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:54.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.344+0000 7fe83b400640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c0777e0 0x7fe83c1a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:54.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.344+0000 7fe83b400640 1 -- 172.21.15.39:0/1448535238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c113cc0 msgr2=0x7fe83c1a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:54.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.344+0000 7fe83b400640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c113cc0 0x7fe83c1a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:54.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.344+0000 7fe83b400640 1 -- 172.21.15.39:0/1448535238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c1a5f10 con 0x7fe83c076e90 2024-07-04T08:09:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.344+0000 7fe83b400640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c076e90 0x7fe83c19ffa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00ea60 tx=0x7fe82c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.344+0000 7fe833e00640 1 -- 172.21.15.39:0/1448535238 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c00cde0 con 0x7fe83c076e90 2024-07-04T08:09:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.344+0000 7fe833e00640 1 -- 172.21.15.39:0/1448535238 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c0035f0 con 0x7fe83c076e90 2024-07-04T08:09:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.344+0000 7fe833e00640 1 -- 172.21.15.39:0/1448535238 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c010690 con 0x7fe83c076e90 2024-07-04T08:09:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.345+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c1a6200 con 0x7fe83c076e90 2024-07-04T08:09:54.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.345+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c1ac960 con 0x7fe83c076e90 2024-07-04T08:09:54.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.346+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83c0772a0 con 0x7fe83c076e90 2024-07-04T08:09:54.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.349+0000 7fe833e00640 1 -- 172.21.15.39:0/1448535238 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fe82c0037a0 con 0x7fe83c076e90 2024-07-04T08:09:54.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.350+0000 7fe833e00640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8200b28c0 0x7fe8200b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.351+0000 7fe833e00640 1 -- 172.21.15.39:0/1448535238 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8985+0+0 (secure 0 0 0) 0x7fe82c0e29e0 con 0x7fe83c076e90 2024-07-04T08:09:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.351+0000 7fe83aa00640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8200b28c0 0x7fe8200b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:54.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.352+0000 7fe83aa00640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8200b28c0 0x7fe8200b4dd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe83c079430 tx=0x7fe828005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:54.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.353+0000 7fe833e00640 1 -- 172.21.15.39:0/1448535238 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe82c055110 con 0x7fe83c076e90 2024-07-04T08:09:54.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.571+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fe83c1acb30 con 0x7fe83c076e90 2024-07-04T08:09:54.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.641+0000 7fe833e00640 1 -- 172.21.15.39:0/1448535238 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) ==== 169+0+0 (secure 0 0 0) 0x7fe82c0aa3b0 con 0x7fe83c076e90 2024-07-04T08:09:54.643 INFO:teuthology.orchestra.run.smithi039.stderr:set pool 4 allow_ec_overwrites to true 2024-07-04T08:09:54.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.645+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8200b28c0 msgr2=0x7fe8200b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:54.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.645+0000 7fe842800640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8200b28c0 0x7fe8200b4dd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe83c079430 tx=0x7fe828005f90 comp rx=0 tx=0).stop 2024-07-04T08:09:54.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.645+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c076e90 msgr2=0x7fe83c19ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.645+0000 7fe842800640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c076e90 0x7fe83c19ffa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00ea60 tx=0x7fe82c00ef20 comp rx=0 tx=0).stop 2024-07-04T08:09:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.646+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 shutdown_connections 2024-07-04T08:09:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.646+0000 7fe842800640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8200b28c0 0x7fe8200b4dd0 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe83c079430 tx=0x7fe828005f90 comp rx=0 tx=0).stop 2024-07-04T08:09:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.646+0000 7fe842800640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c113cc0 0x7fe83c1a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.646+0000 7fe842800640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c0777e0 0x7fe83c1a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.646+0000 7fe842800640 1 --2- 172.21.15.39:0/1448535238 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c076e90 0x7fe83c19ffa0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00ea60 tx=0x7fe82c00ef20 comp rx=0 tx=0).stop 2024-07-04T08:09:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.646+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 >> 172.21.15.39:0/1448535238 conn(0x7fe83c100d10 msgr2=0x7fe83c103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:54.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.647+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 shutdown_connections 2024-07-04T08:09:54.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:54.647+0000 7fe842800640 1 -- 172.21.15.39:0/1448535238 wait complete. 2024-07-04T08:09:54.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:54 smithi039 ceph-mon[32062]: pgmap v315: 129 pgs: 15 creating+peering, 14 unknown, 100 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:54.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:54 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1448535238' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-07-04T08:09:55.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:54 smithi133 ceph-mon[37050]: pgmap v315: 129 pgs: 15 creating+peering, 14 unknown, 100 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:55.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:54 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1448535238' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-07-04T08:09:55.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:54 smithi050 ceph-mon[36577]: pgmap v315: 129 pgs: 15 creating+peering, 14 unknown, 100 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:55.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:54 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1448535238' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-07-04T08:09:55.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-07-04T08:09:55.996 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:09:56.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:55 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1448535238' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-07-04T08:09:56.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:55 smithi133 ceph-mon[37050]: osdmap e87: 12 total, 12 up, 12 in 2024-07-04T08:09:56.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:55 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1448535238' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-07-04T08:09:56.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:55 smithi039 ceph-mon[32062]: osdmap e87: 12 total, 12 up, 12 in 2024-07-04T08:09:56.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:55 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1448535238' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-07-04T08:09:56.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:55 smithi050 ceph-mon[36577]: osdmap e87: 12 total, 12 up, 12 in 2024-07-04T08:09:56.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:56 smithi039 ceph-mon[32062]: pgmap v317: 129 pgs: 15 creating+peering, 114 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:57.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:56 smithi133 ceph-mon[37050]: pgmap v317: 129 pgs: 15 creating+peering, 114 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:57.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:56 smithi050 ceph-mon[36577]: pgmap v317: 129 pgs: 15 creating+peering, 114 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:58.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.444+0000 7ff3e6c00640 1 -- 172.21.15.39:0/740343746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0104800 msgr2=0x7ff3e010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:58.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.444+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/740343746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0104800 0x7ff3e010ce40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009900 tx=0x7ff3cc02a400 comp rx=0 tx=0).stop 2024-07-04T08:09:58.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.444+0000 7ff3e6c00640 1 -- 172.21.15.39:0/740343746 shutdown_connections 2024-07-04T08:09:58.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.444+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/740343746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e010d380 0x7ff3e010f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:58.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.444+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/740343746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0104800 0x7ff3e010ce40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:58.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.444+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/740343746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3e0103dd0 0x7ff3e01041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:58.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.444+0000 7ff3e6c00640 1 -- 172.21.15.39:0/740343746 >> 172.21.15.39:0/740343746 conn(0x7ff3e0078c70 msgr2=0x7ff3e0079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:58.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.444+0000 7ff3e6c00640 1 -- 172.21.15.39:0/740343746 shutdown_connections 2024-07-04T08:09:58.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.445+0000 7ff3e6c00640 1 -- 172.21.15.39:0/740343746 wait complete. 2024-07-04T08:09:58.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.446+0000 7ff3e6c00640 1 Processor -- start 2024-07-04T08:09:58.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.446+0000 7ff3e6c00640 1 -- start start 2024-07-04T08:09:58.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.446+0000 7ff3e6c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0103dd0 0x7ff3e019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:58.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.446+0000 7ff3e6c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e0104800 0x7ff3e019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:58.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3e6c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3e010d380 0x7ff3e01a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:58.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3e6c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0112fc0 con 0x7ff3e0104800 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3e6c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0112e40 con 0x7ff3e0103dd0 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3e6c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e0113140 con 0x7ff3e010d380 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3dfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0103dd0 0x7ff3e019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3dfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0103dd0 0x7ff3e019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50654/0 (socket says 172.21.15.39:50654) 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3dfe00640 1 -- 172.21.15.39:0/2524054836 learned_addr learned my addr 172.21.15.39:0/2524054836 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3df400640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e0104800 0x7ff3e019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3e4c00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3e010d380 0x7ff3e01a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3dfe00640 1 -- 172.21.15.39:0/2524054836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3e010d380 msgr2=0x7ff3e01a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3dfe00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3e010d380 0x7ff3e01a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:58.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3dfe00640 1 -- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e0104800 msgr2=0x7ff3e019c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:58.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3dfe00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e0104800 0x7ff3e019c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:58.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.447+0000 7ff3dfe00640 1 -- 172.21.15.39:0/2524054836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e01a1b80 con 0x7ff3e0103dd0 2024-07-04T08:09:58.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.448+0000 7ff3dfe00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0103dd0 0x7ff3e019bc10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000e9f0 tx=0x7ff3d000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:58.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.448+0000 7ff3dcc00640 1 -- 172.21.15.39:0/2524054836 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000ccc0 con 0x7ff3e0103dd0 2024-07-04T08:09:58.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.448+0000 7ff3dcc00640 1 -- 172.21.15.39:0/2524054836 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3d000ce60 con 0x7ff3e0103dd0 2024-07-04T08:09:58.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.448+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e01a1e70 con 0x7ff3e0103dd0 2024-07-04T08:09:58.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.448+0000 7ff3dcc00640 1 -- 172.21.15.39:0/2524054836 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0010430 con 0x7ff3e0103dd0 2024-07-04T08:09:58.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.448+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e01a85d0 con 0x7ff3e0103dd0 2024-07-04T08:09:58.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.449+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e006b710 con 0x7ff3e0103dd0 2024-07-04T08:09:58.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.452+0000 7ff3dcc00640 1 -- 172.21.15.39:0/2524054836 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7ff3d00046c0 con 0x7ff3e0103dd0 2024-07-04T08:09:58.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.453+0000 7ff3dcc00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3c40b28c0 0x7ff3c40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:09:58.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.454+0000 7ff3dcc00640 1 -- 172.21.15.39:0/2524054836 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8985+0+0 (secure 0 0 0) 0x7ff3d00e2b10 con 0x7ff3e0103dd0 2024-07-04T08:09:58.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.454+0000 7ff3df400640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3c40b28c0 0x7ff3c40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:09:58.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.454+0000 7ff3df400640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3c40b28c0 0x7ff3c40b4dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc000c00 tx=0x7ff3cc031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:09:58.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.455+0000 7ff3dcc00640 1 -- 172.21.15.39:0/2524054836 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7ff3d0055110 con 0x7ff3e0103dd0 2024-07-04T08:09:58.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:58.718+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7ff3e019cd40 con 0x7ff3e0103dd0 2024-07-04T08:09:59.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:58 smithi133 ceph-mon[37050]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:58 smithi039 ceph-mon[32062]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:58 smithi050 ceph-mon[36577]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:09:59.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.709+0000 7ff3dcc00640 1 -- 172.21.15.39:0/2524054836 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7ff3d00aa4e0 con 0x7ff3e0103dd0 2024-07-04T08:09:59.710 INFO:teuthology.orchestra.run.smithi039.stderr:added data pool 4 to fsmap 2024-07-04T08:09:59.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.712+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3c40b28c0 msgr2=0x7ff3c40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:59.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.712+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3c40b28c0 0x7ff3c40b4dd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc000c00 tx=0x7ff3cc031400 comp rx=0 tx=0).stop 2024-07-04T08:09:59.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.712+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0103dd0 msgr2=0x7ff3e019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:09:59.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.712+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0103dd0 0x7ff3e019bc10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000e9f0 tx=0x7ff3d000edf0 comp rx=0 tx=0).stop 2024-07-04T08:09:59.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.713+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 shutdown_connections 2024-07-04T08:09:59.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.713+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3c40b28c0 0x7ff3c40b4dd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:59.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.713+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3e0104800 0x7ff3e019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:59.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.713+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3e010d380 0x7ff3e01a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:59.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.713+0000 7ff3e6c00640 1 --2- 172.21.15.39:0/2524054836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff3e0103dd0 0x7ff3e019bc10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:09:59.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.713+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 >> 172.21.15.39:0/2524054836 conn(0x7ff3e0078c70 msgr2=0x7ff3e00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:09:59.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.713+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 shutdown_connections 2024-07-04T08:09:59.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:09:59.713+0000 7ff3e6c00640 1 -- 172.21.15.39:0/2524054836 wait complete. 2024-07-04T08:09:59.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:59 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2524054836' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-04T08:09:59.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:09:59 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-04T08:10:00.034 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:59 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2524054836' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-04T08:10:00.034 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:09:59 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-04T08:10:00.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:59 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2524054836' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-04T08:10:00.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:09:59 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-04T08:10:00.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:10:01.029 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:01.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:00 smithi133 ceph-mon[37050]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:01.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:00 smithi133 ceph-mon[37050]: osdmap e88: 12 total, 12 up, 12 in 2024-07-04T08:10:01.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:00 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-07-04T08:10:01.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:00 smithi133 ceph-mon[37050]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:10:01.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:00 smithi133 ceph-mon[37050]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-07-04T08:10:01.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:00 smithi133 ceph-mon[37050]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-07-04T08:10:01.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:00 smithi133 ceph-mon[37050]: application not enabled on pool 'cephfs_data_ec' 2024-07-04T08:10:01.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:00 smithi133 ceph-mon[37050]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-07-04T08:10:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:00 smithi039 ceph-mon[32062]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:00 smithi039 ceph-mon[32062]: osdmap e88: 12 total, 12 up, 12 in 2024-07-04T08:10:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:00 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-07-04T08:10:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:00 smithi039 ceph-mon[32062]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:10:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:00 smithi039 ceph-mon[32062]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-07-04T08:10:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:00 smithi039 ceph-mon[32062]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-07-04T08:10:01.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:00 smithi039 ceph-mon[32062]: application not enabled on pool 'cephfs_data_ec' 2024-07-04T08:10:01.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:00 smithi039 ceph-mon[32062]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-07-04T08:10:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:00 smithi050 ceph-mon[36577]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:00 smithi050 ceph-mon[36577]: osdmap e88: 12 total, 12 up, 12 in 2024-07-04T08:10:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:00 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-07-04T08:10:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:00 smithi050 ceph-mon[36577]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:10:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:00 smithi050 ceph-mon[36577]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-07-04T08:10:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:00 smithi050 ceph-mon[36577]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-07-04T08:10:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:00 smithi050 ceph-mon[36577]: application not enabled on pool 'cephfs_data_ec' 2024-07-04T08:10:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:00 smithi050 ceph-mon[36577]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-07-04T08:10:01.772 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:01 smithi039 ceph-mon[32062]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-04T08:10:01.772 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:01 smithi039 ceph-mon[32062]: Cluster is now healthy 2024-07-04T08:10:02.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:01 smithi133 ceph-mon[37050]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-04T08:10:02.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:01 smithi133 ceph-mon[37050]: Cluster is now healthy 2024-07-04T08:10:02.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:01 smithi050 ceph-mon[36577]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-04T08:10:02.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:01 smithi050 ceph-mon[36577]: Cluster is now healthy 2024-07-04T08:10:02.917 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:02 smithi039 ceph-mon[32062]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:03.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:02 smithi133 ceph-mon[37050]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:02 smithi050 ceph-mon[36577]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.233+0000 7f588fa00640 1 -- 172.21.15.39:0/1369157859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58881089f0 msgr2=0x7f5888108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.233+0000 7f588fa00640 1 --2- 172.21.15.39:0/1369157859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58881089f0 0x7f5888108e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5870009900 tx=0x7f587002a410 comp rx=0 tx=0).stop 2024-07-04T08:10:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.233+0000 7f588fa00640 1 -- 172.21.15.39:0/1369157859 shutdown_connections 2024-07-04T08:10:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.233+0000 7f588fa00640 1 --2- 172.21.15.39:0/1369157859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5888109340 0x7f588810fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.233+0000 7f588fa00640 1 --2- 172.21.15.39:0/1369157859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f58881089f0 0x7f5888108e00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.233+0000 7f588fa00640 1 --2- 172.21.15.39:0/1369157859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58881030b0 0x7f58881034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.233+0000 7f588fa00640 1 -- 172.21.15.39:0/1369157859 >> 172.21.15.39:0/1369157859 conn(0x7f588807ada0 msgr2=0x7f588807b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.233+0000 7f588fa00640 1 -- 172.21.15.39:0/1369157859 shutdown_connections 2024-07-04T08:10:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.233+0000 7f588fa00640 1 -- 172.21.15.39:0/1369157859 wait complete. 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.234+0000 7f588fa00640 1 Processor -- start 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.234+0000 7f588fa00640 1 -- start start 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588fa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58881030b0 0x7f588819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588fa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58881089f0 0x7f588819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588fa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5888109340 0x7f58881a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588fa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5888112fd0 con 0x7f58881030b0 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588fa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5888112e50 con 0x7f5888109340 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588fa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5888113150 con 0x7f58881089f0 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58881030b0 0x7f588819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58881030b0 0x7f588819bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60648/0 (socket says 172.21.15.39:60648) 2024-07-04T08:10:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588d000640 1 -- 172.21.15.39:0/1448147514 learned_addr learned my addr 172.21.15.39:0/1448147514 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f5887e00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58881089f0 0x7f588819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.235+0000 7f588da00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5888109340 0x7f58881a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.236+0000 7f588d000640 1 -- 172.21.15.39:0/1448147514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58881089f0 msgr2=0x7f588819c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.236+0000 7f588d000640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58881089f0 0x7f588819c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.236+0000 7f588d000640 1 -- 172.21.15.39:0/1448147514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5888109340 msgr2=0x7f58881a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.236+0000 7f588d000640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5888109340 0x7f58881a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.236+0000 7f588d000640 1 -- 172.21.15.39:0/1448147514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58881a1b70 con 0x7f58881030b0 2024-07-04T08:10:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.236+0000 7f588da00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5888109340 0x7f58881a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:10:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.236+0000 7f588d000640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58881030b0 0x7f588819bc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f587800e9f0 tx=0x7f587800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.237+0000 7f5885600640 1 -- 172.21.15.39:0/1448147514 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587800cd80 con 0x7f58881030b0 2024-07-04T08:10:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.237+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58881a1e60 con 0x7f58881030b0 2024-07-04T08:10:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.237+0000 7f5885600640 1 -- 172.21.15.39:0/1448147514 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58780054d0 con 0x7f58881030b0 2024-07-04T08:10:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.237+0000 7f5885600640 1 -- 172.21.15.39:0/1448147514 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878010690 con 0x7f58881030b0 2024-07-04T08:10:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.237+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58881a85c0 con 0x7f58881030b0 2024-07-04T08:10:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.238+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58881034c0 con 0x7f58881030b0 2024-07-04T08:10:03.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.241+0000 7f5885600640 1 -- 172.21.15.39:0/1448147514 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f5878004e30 con 0x7f58881030b0 2024-07-04T08:10:03.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.243+0000 7f5885600640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f586c0b2840 0x7f586c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:03.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.244+0000 7f5887e00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f586c0b2840 0x7f586c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.244+0000 7f5885600640 1 -- 172.21.15.39:0/1448147514 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7f58780e2c50 con 0x7f58881030b0 2024-07-04T08:10:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.244+0000 7f5887e00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f586c0b2840 0x7f586c0b4d50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5870009230 tx=0x7f5870031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:03.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.246+0000 7f5885600640 1 -- 172.21.15.39:0/1448147514 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5878055110 con 0x7f58881030b0 2024-07-04T08:10:03.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.505+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f588819cd30 con 0x7f58881030b0 2024-07-04T08:10:03.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.507+0000 7f5885600640 1 -- 172.21.15.39:0/1448147514 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7f588819cd30 con 0x7f58881030b0 2024-07-04T08:10:03.508 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:10:03.509 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"btime":"2024-07-04T08:09:59:681020+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24661,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24679,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:09:58.720875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24707},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24707,"qdb_cluster":[24707]},"id":1}]} 2024-07-04T08:10:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-07-04T08:10:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.509+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f586c0b2840 msgr2=0x7f586c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f586c0b2840 0x7f586c0b4d50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5870009230 tx=0x7f5870031400 comp rx=0 tx=0).stop 2024-07-04T08:10:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58881030b0 msgr2=0x7f588819bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58881030b0 0x7f588819bc00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f587800e9f0 tx=0x7f587800edf0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 shutdown_connections 2024-07-04T08:10:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f586c0b2840 0x7f586c0b4d50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5888109340 0x7f58881a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58881089f0 0x7f588819c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 --2- 172.21.15.39:0/1448147514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58881030b0 0x7f588819bc00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 >> 172.21.15.39:0/1448147514 conn(0x7f588807ada0 msgr2=0x7f58881047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 shutdown_connections 2024-07-04T08:10:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:03.510+0000 7f588fa00640 1 -- 172.21.15.39:0/1448147514 wait complete. 2024-07-04T08:10:04.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:03 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1448147514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:10:04.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:03 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1448147514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:10:04.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:03 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1448147514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:10:04.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:10:04.820 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:05.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:04 smithi133 ceph-mon[37050]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:05.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:04 smithi039 ceph-mon[32062]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:04 smithi050 ceph-mon[36577]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:06.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.961+0000 7f9e45000640 1 -- 172.21.15.39:0/969757608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e40105110 msgr2=0x7f9e40105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.961+0000 7f9e45000640 1 --2- 172.21.15.39:0/969757608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e40105110 0x7f9e40105520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009900 tx=0x7f9e3002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:10:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.961+0000 7f9e45000640 1 -- 172.21.15.39:0/969757608 shutdown_connections 2024-07-04T08:10:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.961+0000 7f9e45000640 1 --2- 172.21.15.39:0/969757608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e4010b3a0 0x7f9e40111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.961+0000 7f9e45000640 1 --2- 172.21.15.39:0/969757608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e4010aa50 0x7f9e4010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.961+0000 7f9e45000640 1 --2- 172.21.15.39:0/969757608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e40105110 0x7f9e40105520 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.961+0000 7f9e45000640 1 -- 172.21.15.39:0/969757608 >> 172.21.15.39:0/969757608 conn(0x7f9e40100b80 msgr2=0x7f9e40102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.961+0000 7f9e45000640 1 -- 172.21.15.39:0/969757608 shutdown_connections 2024-07-04T08:10:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.961+0000 7f9e45000640 1 -- 172.21.15.39:0/969757608 wait complete. 2024-07-04T08:10:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.962+0000 7f9e45000640 1 Processor -- start 2024-07-04T08:10:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e45000640 1 -- start start 2024-07-04T08:10:06.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e45000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e40105110 0x7f9e4019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e45000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e4010aa50 0x7f9e4019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e45000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4010b3a0 0x7f9e401a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e45000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e40115050 con 0x7f9e4010b3a0 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e45000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e40114ed0 con 0x7f9e40105110 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e45000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e401151d0 con 0x7f9e4010aa50 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4010b3a0 0x7f9e401a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e4010aa50 0x7f9e4019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.963+0000 7f9e3f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e40105110 0x7f9e4019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.964+0000 7f9e37e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e4010aa50 0x7f9e4019bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57906/0 (socket says 172.21.15.39:57906) 2024-07-04T08:10:06.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.964+0000 7f9e37e00640 1 -- 172.21.15.39:0/2861981723 learned_addr learned my addr 172.21.15.39:0/2861981723 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.964+0000 7f9e37e00640 1 -- 172.21.15.39:0/2861981723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e40105110 msgr2=0x7f9e4019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.964+0000 7f9e37e00640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e40105110 0x7f9e4019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.964+0000 7f9e37e00640 1 -- 172.21.15.39:0/2861981723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4010b3a0 msgr2=0x7f9e401a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.964+0000 7f9e37e00640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4010b3a0 0x7f9e401a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.964+0000 7f9e37e00640 1 -- 172.21.15.39:0/2861981723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e401a19b0 con 0x7f9e4010aa50 2024-07-04T08:10:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.964+0000 7f9e37e00640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e4010aa50 0x7f9e4019bfe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800e9f0 tx=0x7f9e2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.965+0000 7f9e3cc00640 1 -- 172.21.15.39:0/2861981723 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2800ccc0 con 0x7f9e4010aa50 2024-07-04T08:10:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.965+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e401a1ca0 con 0x7f9e4010aa50 2024-07-04T08:10:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.965+0000 7f9e3cc00640 1 -- 172.21.15.39:0/2861981723 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e280108e0 con 0x7f9e4010aa50 2024-07-04T08:10:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.965+0000 7f9e3cc00640 1 -- 172.21.15.39:0/2861981723 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28018860 con 0x7f9e4010aa50 2024-07-04T08:10:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.965+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e401a8400 con 0x7f9e4010aa50 2024-07-04T08:10:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.966+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e40105520 con 0x7f9e4010aa50 2024-07-04T08:10:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.969+0000 7f9e3cc00640 1 -- 172.21.15.39:0/2861981723 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f9e28010430 con 0x7f9e4010aa50 2024-07-04T08:10:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.971+0000 7f9e3cc00640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e200b2840 0x7f9e200b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.971+0000 7f9e3cc00640 1 -- 172.21.15.39:0/2861981723 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7f9e280e2d60 con 0x7f9e4010aa50 2024-07-04T08:10:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.971+0000 7f9e3f400640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e200b2840 0x7f9e200b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.972+0000 7f9e3f400640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e200b2840 0x7f9e200b4d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e30031440 tx=0x7f9e30009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:06.972+0000 7f9e3cc00640 1 -- 172.21.15.39:0/2861981723 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f9e280aa0c0 con 0x7f9e4010aa50 2024-07-04T08:10:07.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:06 smithi133 ceph-mon[37050]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:07.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:06 smithi039 ceph-mon[32062]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:07.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:06 smithi050 ceph-mon[36577]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:07.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.186+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e4002de10 con 0x7f9e4010aa50 2024-07-04T08:10:07.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.188+0000 7f9e3cc00640 1 -- 172.21.15.39:0/2861981723 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24384 (secure 0 0 0) 0x7f9e280ae3c0 con 0x7f9e4010aa50 2024-07-04T08:10:07.190 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:10:07.190 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":88,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:09:58.720601+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:10:07.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.191+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e200b2840 msgr2=0x7f9e200b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:07.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.191+0000 7f9e45000640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e200b2840 0x7f9e200b4d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e30031440 tx=0x7f9e30009560 comp rx=0 tx=0).stop 2024-07-04T08:10:07.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e4010aa50 msgr2=0x7f9e4019bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:07.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e4010aa50 0x7f9e4019bfe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800e9f0 tx=0x7f9e2800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:10:07.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 shutdown_connections 2024-07-04T08:10:07.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e200b2840 0x7f9e200b4d50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:07.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e4010b3a0 0x7f9e401a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:07.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e4010aa50 0x7f9e4019bfe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:07.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 --2- 172.21.15.39:0/2861981723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e40105110 0x7f9e4019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:07.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 >> 172.21.15.39:0/2861981723 conn(0x7f9e40100b80 msgr2=0x7f9e40102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:07.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 shutdown_connections 2024-07-04T08:10:07.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:07.192+0000 7f9e45000640 1 -- 172.21.15.39:0/2861981723 wait complete. 2024-07-04T08:10:07.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:07 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2861981723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:08.009 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:10:08.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:07 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2861981723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:08.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:07 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2861981723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:08.493 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:08.962 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:08 smithi039 ceph-mon[32062]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:09.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:08 smithi133 ceph-mon[37050]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:08 smithi050 ceph-mon[36577]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:10.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.760+0000 7f63b2400640 1 -- 172.21.15.39:0/1981641656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ac074d00 msgr2=0x7f63ac075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:10.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.760+0000 7f63b2400640 1 --2- 172.21.15.39:0/1981641656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ac074d00 0x7f63ac075110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f639c009900 tx=0x7f639c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.760+0000 7f63b2400640 1 -- 172.21.15.39:0/1981641656 shutdown_connections 2024-07-04T08:10:10.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.760+0000 7f63b2400640 1 --2- 172.21.15.39:0/1981641656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63ac10f460 0x7f63ac1118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.760+0000 7f63b2400640 1 --2- 172.21.15.39:0/1981641656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ac075650 0x7f63ac10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.760+0000 7f63b2400640 1 --2- 172.21.15.39:0/1981641656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ac074d00 0x7f63ac075110 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.760+0000 7f63b2400640 1 -- 172.21.15.39:0/1981641656 >> 172.21.15.39:0/1981641656 conn(0x7f63ac0fe990 msgr2=0x7f63ac100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:10.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.760+0000 7f63b2400640 1 -- 172.21.15.39:0/1981641656 shutdown_connections 2024-07-04T08:10:10.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.760+0000 7f63b2400640 1 -- 172.21.15.39:0/1981641656 wait complete. 2024-07-04T08:10:10.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.761+0000 7f63b2400640 1 Processor -- start 2024-07-04T08:10:10.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.761+0000 7f63b2400640 1 -- start start 2024-07-04T08:10:10.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b2400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63ac074d00 0x7f63ac19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:10.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ac075650 0x7f63ac19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:10.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b2400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ac10f460 0x7f63ac1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b2400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac115060 con 0x7f63ac075650 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b2400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac114ee0 con 0x7f63ac074d00 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b2400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac1151e0 con 0x7f63ac10f460 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63abe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ac075650 0x7f63ac19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63ac074d00 0x7f63ac19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63ac074d00 0x7f63ac19ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47772/0 (socket says 172.21.15.39:47772) 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b1000640 1 -- 172.21.15.39:0/3056614994 learned_addr learned my addr 172.21.15.39:0/3056614994 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b1a00640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ac10f460 0x7f63ac1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b1000640 1 -- 172.21.15.39:0/3056614994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ac10f460 msgr2=0x7f63ac1a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:10.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b1000640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ac10f460 0x7f63ac1a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b1000640 1 -- 172.21.15.39:0/3056614994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ac075650 msgr2=0x7f63ac19bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b1000640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ac075650 0x7f63ac19bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.762+0000 7f63b1000640 1 -- 172.21.15.39:0/3056614994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac1a19a0 con 0x7f63ac074d00 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.763+0000 7f63b1000640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63ac074d00 0x7f63ac19ba30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f639c005ab0 tx=0x7f639c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.763+0000 7f63a9600640 1 -- 172.21.15.39:0/3056614994 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c03f070 con 0x7f63ac074d00 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.763+0000 7f63a9600640 1 -- 172.21.15.39:0/3056614994 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f639c031b60 con 0x7f63ac074d00 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.763+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac1a1c30 con 0x7f63ac074d00 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.763+0000 7f63a9600640 1 -- 172.21.15.39:0/3056614994 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c035030 con 0x7f63ac074d00 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.763+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac1a83f0 con 0x7f63ac074d00 2024-07-04T08:10:10.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.765+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ac075110 con 0x7f63ac074d00 2024-07-04T08:10:10.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.767+0000 7f63a9600640 1 -- 172.21.15.39:0/3056614994 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f639c042070 con 0x7f63ac074d00 2024-07-04T08:10:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.769+0000 7f63a9600640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63900b28c0 0x7f63900b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.769+0000 7f63a9600640 1 -- 172.21.15.39:0/3056614994 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7f639c103280 con 0x7f63ac074d00 2024-07-04T08:10:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.769+0000 7f63abe00640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63900b28c0 0x7f63900b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:10.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.770+0000 7f63abe00640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63900b28c0 0x7f63900b4dd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0ffa30 tx=0x7f6398005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.771+0000 7f63a9600640 1 -- 172.21.15.39:0/3056614994 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f639c0ca5e0 con 0x7f63ac074d00 2024-07-04T08:10:10.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.986+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63ac1a8620 con 0x7f63ac074d00 2024-07-04T08:10:10.988 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:10 smithi039 ceph-mon[32062]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:10.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.988+0000 7f63a9600640 1 -- 172.21.15.39:0/3056614994 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24384 (secure 0 0 0) 0x7f639c0ce8e0 con 0x7f63ac074d00 2024-07-04T08:10:10.990 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:10:10.991 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":88,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:09:58.720601+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:10:10.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63900b28c0 msgr2=0x7f63900b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:10.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63900b28c0 0x7f63900b4dd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0ffa30 tx=0x7f6398005f90 comp rx=0 tx=0).stop 2024-07-04T08:10:10.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63ac074d00 msgr2=0x7f63ac19ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63ac074d00 0x7f63ac19ba30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f639c005ab0 tx=0x7f639c003c10 comp rx=0 tx=0).stop 2024-07-04T08:10:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 shutdown_connections 2024-07-04T08:10:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63900b28c0 0x7f63900b4dd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ac075650 0x7f63ac19bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63ac10f460 0x7f63ac1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 --2- 172.21.15.39:0/3056614994 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f63ac074d00 0x7f63ac19ba30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 >> 172.21.15.39:0/3056614994 conn(0x7f63ac0fe990 msgr2=0x7f63ac109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 shutdown_connections 2024-07-04T08:10:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:10.991+0000 7f63b2400640 1 -- 172.21.15.39:0/3056614994 wait complete. 2024-07-04T08:10:11.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:10 smithi133 ceph-mon[37050]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:10 smithi050 ceph-mon[36577]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:11.824 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:10:12.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:11 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3056614994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:12.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:11 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3056614994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:12.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:11 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3056614994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:12.299 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:12 smithi039 ceph-mon[32062]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:13.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:12 smithi050 ceph-mon[36577]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:13.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:12 smithi133 ceph-mon[37050]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:14.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.474+0000 7f2624a00640 1 -- 172.21.15.39:0/3669329682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010f7a0 msgr2=0x7f2620115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:14.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.474+0000 7f2624a00640 1 --2- 172.21.15.39:0/3669329682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010f7a0 0x7f2620115fd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2618009ca0 tx=0x7f261802a8b0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.474+0000 7f2624a00640 1 -- 172.21.15.39:0/3669329682 shutdown_connections 2024-07-04T08:10:14.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.474+0000 7f2624a00640 1 --2- 172.21.15.39:0/3669329682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f262010ee50 0x7f262010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.474+0000 7f2624a00640 1 --2- 172.21.15.39:0/3669329682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010f7a0 0x7f2620115fd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.474+0000 7f2624a00640 1 --2- 172.21.15.39:0/3669329682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2620109510 0x7f2620109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.474+0000 7f2624a00640 1 -- 172.21.15.39:0/3669329682 >> 172.21.15.39:0/3669329682 conn(0x7f2620078b10 msgr2=0x7f2620107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.474+0000 7f2624a00640 1 -- 172.21.15.39:0/3669329682 shutdown_connections 2024-07-04T08:10:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.474+0000 7f2624a00640 1 -- 172.21.15.39:0/3669329682 wait complete. 2024-07-04T08:10:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.475+0000 7f2624a00640 1 Processor -- start 2024-07-04T08:10:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.476+0000 7f2624a00640 1 -- start start 2024-07-04T08:10:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.476+0000 7f2624a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2620109510 0x7f262019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.476+0000 7f2624a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010ee50 0x7f26201a0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.476+0000 7f2624a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f262010f7a0 0x7f26201a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.476+0000 7f2624a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26201194a0 con 0x7f2620109510 2024-07-04T08:10:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.476+0000 7f2624a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2620119320 con 0x7f262010ee50 2024-07-04T08:10:14.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.476+0000 7f2624a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2620119620 con 0x7f262010f7a0 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f261e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2620109510 0x7f262019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f2617e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010ee50 0x7f26201a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f2617e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010ee50 0x7f26201a0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50676/0 (socket says 172.21.15.39:50676) 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f2617e00640 1 -- 172.21.15.39:0/635118880 learned_addr learned my addr 172.21.15.39:0/635118880 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f261f200640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f262010f7a0 0x7f26201a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f2617e00640 1 -- 172.21.15.39:0/635118880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f262010f7a0 msgr2=0x7f26201a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f2617e00640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f262010f7a0 0x7f26201a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f2617e00640 1 -- 172.21.15.39:0/635118880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2620109510 msgr2=0x7f262019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f2617e00640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2620109510 0x7f262019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f2617e00640 1 -- 172.21.15.39:0/635118880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26201a5d20 con 0x7f262010ee50 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f261f200640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f262010f7a0 0x7f26201a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:10:14.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.477+0000 7f2617e00640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010ee50 0x7f26201a0350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f260800e9f0 tx=0x7f260800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:14.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.478+0000 7f2617400640 1 -- 172.21.15.39:0/635118880 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260800ccc0 con 0x7f262010ee50 2024-07-04T08:10:14.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.478+0000 7f2617400640 1 -- 172.21.15.39:0/635118880 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26080108e0 con 0x7f262010ee50 2024-07-04T08:10:14.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.478+0000 7f2617400640 1 -- 172.21.15.39:0/635118880 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608018860 con 0x7f262010ee50 2024-07-04T08:10:14.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.478+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26201a6010 con 0x7f262010ee50 2024-07-04T08:10:14.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.478+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201ac770 con 0x7f262010ee50 2024-07-04T08:10:14.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.479+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2620109920 con 0x7f262010ee50 2024-07-04T08:10:14.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.482+0000 7f2617400640 1 -- 172.21.15.39:0/635118880 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f2608010430 con 0x7f262010ee50 2024-07-04T08:10:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.483+0000 7f2617400640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26040b28c0 0x7f26040b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.483+0000 7f2617400640 1 -- 172.21.15.39:0/635118880 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7f26080e2d90 con 0x7f262010ee50 2024-07-04T08:10:14.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.484+0000 7f261e800640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26040b28c0 0x7f26040b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:14.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.484+0000 7f261e800640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26040b28c0 0x7f26040b4dd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26100095f0 tx=0x7f2610005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:14.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.485+0000 7f2617400640 1 -- 172.21.15.39:0/635118880 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f26080aa740 con 0x7f262010ee50 2024-07-04T08:10:14.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.743+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26201a0ee0 con 0x7f262010ee50 2024-07-04T08:10:14.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.744+0000 7f2617400640 1 -- 172.21.15.39:0/635118880 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7f26201a0ee0 con 0x7f262010ee50 2024-07-04T08:10:14.747 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:10:14.747 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":17,"btime":"2024-07-04T08:09:59:681020+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24619,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24626,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24661,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24673,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24679,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:09:58.720875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24707},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24707,"qdb_cluster":[24707]},"id":1}]} 2024-07-04T08:10:14.747 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 17 2024-07-04T08:10:14.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26040b28c0 msgr2=0x7f26040b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:14.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26040b28c0 0x7f26040b4dd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26100095f0 tx=0x7f2610005f90 comp rx=0 tx=0).stop 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010ee50 msgr2=0x7f26201a0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010ee50 0x7f26201a0350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f260800e9f0 tx=0x7f260800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 shutdown_connections 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26040b28c0 0x7f26040b4dd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f262010f7a0 0x7f26201a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f262010ee50 0x7f26201a0350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 --2- 172.21.15.39:0/635118880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2620109510 0x7f262019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.747+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 >> 172.21.15.39:0/635118880 conn(0x7f2620078b10 msgr2=0x7f262010c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.748+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 shutdown_connections 2024-07-04T08:10:14.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:14.748+0000 7f2624a00640 1 -- 172.21.15.39:0/635118880 wait complete. 2024-07-04T08:10:15.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:14 smithi039 ceph-mon[32062]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:15.085 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:14 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/635118880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:10:15.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:14 smithi050 ceph-mon[36577]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:14 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/635118880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:10:15.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:14 smithi133 ceph-mon[37050]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:15.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:14 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/635118880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:10:15.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:10:16.021 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:17.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:16 smithi039 ceph-mon[32062]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:17.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:16 smithi050 ceph-mon[36577]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:17.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:16 smithi133 ceph-mon[37050]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:18.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.319+0000 7fe81fe00640 1 -- 172.21.15.39:0/1934389333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe820109510 msgr2=0x7fe820109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:18.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.319+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1934389333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe820109510 0x7fe820109920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe808009900 tx=0x7fe80802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.319+0000 7fe81fe00640 1 -- 172.21.15.39:0/1934389333 shutdown_connections 2024-07-04T08:10:18.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.319+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1934389333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe82010ee50 0x7fe82010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.319+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1934389333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe82010f7a0 0x7fe820115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.319+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1934389333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe820109510 0x7fe820109920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.319+0000 7fe81fe00640 1 -- 172.21.15.39:0/1934389333 >> 172.21.15.39:0/1934389333 conn(0x7fe820078b10 msgr2=0x7fe820107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:18.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.320+0000 7fe81fe00640 1 -- 172.21.15.39:0/1934389333 shutdown_connections 2024-07-04T08:10:18.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.320+0000 7fe81fe00640 1 -- 172.21.15.39:0/1934389333 wait complete. 2024-07-04T08:10:18.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.321+0000 7fe81fe00640 1 Processor -- start 2024-07-04T08:10:18.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.321+0000 7fe81fe00640 1 -- start start 2024-07-04T08:10:18.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.321+0000 7fe81fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe820109510 0x7fe82019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.321+0000 7fe81fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe82010ee50 0x7fe8201a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.321+0000 7fe81fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe82010f7a0 0x7fe8201a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.321+0000 7fe81fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe820119450 con 0x7fe82010ee50 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.321+0000 7fe81fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201192d0 con 0x7fe820109510 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.321+0000 7fe81fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201195d0 con 0x7fe82010f7a0 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe820109510 0x7fe82019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe820109510 0x7fe82019fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50694/0 (socket says 172.21.15.39:50694) 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81ea00640 1 -- 172.21.15.39:0/1981833562 learned_addr learned my addr 172.21.15.39:0/1981833562 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe82010ee50 0x7fe8201a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81f400640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe82010f7a0 0x7fe8201a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81ea00640 1 -- 172.21.15.39:0/1981833562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe82010f7a0 msgr2=0x7fe8201a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:18.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81ea00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe82010f7a0 0x7fe8201a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81ea00640 1 -- 172.21.15.39:0/1981833562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe82010ee50 msgr2=0x7fe8201a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:18.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81ea00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe82010ee50 0x7fe8201a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.322+0000 7fe81ea00640 1 -- 172.21.15.39:0/1981833562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8201a5df0 con 0x7fe820109510 2024-07-04T08:10:18.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.323+0000 7fe81ea00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe820109510 0x7fe82019fe80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe808005940 tx=0x7fe8080040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.323+0000 7fe824a00640 1 -- 172.21.15.39:0/1981833562 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80803f030 con 0x7fe820109510 2024-07-04T08:10:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.323+0000 7fe824a00640 1 -- 172.21.15.39:0/1981833562 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8080044c0 con 0x7fe820109510 2024-07-04T08:10:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.323+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8201a6080 con 0x7fe820109510 2024-07-04T08:10:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.323+0000 7fe824a00640 1 -- 172.21.15.39:0/1981833562 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808035070 con 0x7fe820109510 2024-07-04T08:10:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.323+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8201ac840 con 0x7fe820109510 2024-07-04T08:10:18.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.324+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe820109920 con 0x7fe820109510 2024-07-04T08:10:18.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.327+0000 7fe824a00640 1 -- 172.21.15.39:0/1981833562 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fe808042030 con 0x7fe820109510 2024-07-04T08:10:18.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.329+0000 7fe824a00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8040b28c0 0x7fe8040b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:18.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.329+0000 7fe824a00640 1 -- 172.21.15.39:0/1981833562 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7fe808102b70 con 0x7fe820109510 2024-07-04T08:10:18.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.329+0000 7fe81e000640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8040b28c0 0x7fe8040b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:18.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.330+0000 7fe81e000640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8040b28c0 0x7fe8040b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe82010f710 tx=0x7fe810009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:18.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.330+0000 7fe824a00640 1 -- 172.21.15.39:0/1981833562 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe8080c9ed0 con 0x7fe820109510 2024-07-04T08:10:18.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.549+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe820110f10 con 0x7fe820109510 2024-07-04T08:10:18.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.552+0000 7fe824a00640 1 -- 172.21.15.39:0/1981833562 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24384 (secure 0 0 0) 0x7fe8080ce1d0 con 0x7fe820109510 2024-07-04T08:10:18.555 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:10:18.555 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":88,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:09:58.720601+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:10:18.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8040b28c0 msgr2=0x7fe8040b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8040b28c0 0x7fe8040b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe82010f710 tx=0x7fe810009500 comp rx=0 tx=0).stop 2024-07-04T08:10:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe820109510 msgr2=0x7fe82019fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe820109510 0x7fe82019fe80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe808005940 tx=0x7fe8080040c0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 shutdown_connections 2024-07-04T08:10:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe8040b28c0 0x7fe8040b4dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe82010ee50 0x7fe8201a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe82010f7a0 0x7fe8201a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 --2- 172.21.15.39:0/1981833562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe820109510 0x7fe82019fe80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:18.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.555+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 >> 172.21.15.39:0/1981833562 conn(0x7fe820078b10 msgr2=0x7fe8200794e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.556+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 shutdown_connections 2024-07-04T08:10:18.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:18.556+0000 7fe81fe00640 1 -- 172.21.15.39:0/1981833562 wait complete. 2024-07-04T08:10:19.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:18 smithi039 ceph-mon[32062]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:19.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:18 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1981833562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:19.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:18 smithi050 ceph-mon[36577]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:19.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:18 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1981833562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:19.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:18 smithi133 ceph-mon[37050]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:19.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:18 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1981833562' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:19.412 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-04T08:09:41.252924+0000', 'modified': '2024-07-04T08:09:58.720875+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24707}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24707': {'gid': 24707, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.50:6839/2852157181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 2852157181}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 2852157181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24707, 'qdb_cluster': [24707]} 2024-07-04T08:10:19.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-04T08:10:19.413 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.b status 2024-07-04T08:10:19.909 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:21.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:20 smithi050 ceph-mon[36577]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:21.148 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:20 smithi039 ceph-mon[32062]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:20 smithi133 ceph-mon[37050]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:22.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.103+0000 7fdb91e00640 1 -- 172.21.15.39:0/2657824368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c10f990 msgr2=0x7fdb8c1161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.103+0000 7fdb91e00640 1 --2- 172.21.15.39:0/2657824368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c10f990 0x7fdb8c1161c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009ca0 tx=0x7fdb8402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.103+0000 7fdb91e00640 1 -- 172.21.15.39:0/2657824368 shutdown_connections 2024-07-04T08:10:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.103+0000 7fdb91e00640 1 --2- 172.21.15.39:0/2657824368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c10f990 0x7fdb8c1161c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.103+0000 7fdb91e00640 1 --2- 172.21.15.39:0/2657824368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb8c10f040 0x7fdb8c10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.103+0000 7fdb91e00640 1 --2- 172.21.15.39:0/2657824368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8c109700 0x7fdb8c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.103+0000 7fdb91e00640 1 -- 172.21.15.39:0/2657824368 >> 172.21.15.39:0/2657824368 conn(0x7fdb8c077630 msgr2=0x7fdb8c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.103+0000 7fdb91e00640 1 -- 172.21.15.39:0/2657824368 shutdown_connections 2024-07-04T08:10:22.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.103+0000 7fdb91e00640 1 -- 172.21.15.39:0/2657824368 wait complete. 2024-07-04T08:10:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.104+0000 7fdb91e00640 1 Processor -- start 2024-07-04T08:10:22.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.104+0000 7fdb91e00640 1 -- start start 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.104+0000 7fdb91e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8c109700 0x7fdb8c1a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb91e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb8c10f040 0x7fdb8c1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb91e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c10f990 0x7fdb8c1a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb91e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c119660 con 0x7fdb8c10f990 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb91e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c1194e0 con 0x7fdb8c109700 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb91e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c1197e0 con 0x7fdb8c10f040 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb8b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c10f990 0x7fdb8c1a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb8a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb8c10f040 0x7fdb8c1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb8b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c10f990 0x7fdb8c1a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48956/0 (socket says 172.21.15.39:48956) 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb8aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8c109700 0x7fdb8c1a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb8b400640 1 -- 172.21.15.39:0/3454854553 learned_addr learned my addr 172.21.15.39:0/3454854553 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:22.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.105+0000 7fdb8a000640 1 -- 172.21.15.39:0/3454854553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8c109700 msgr2=0x7fdb8c1a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:22.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8c109700 0x7fdb8c1a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb8a000640 1 -- 172.21.15.39:0/3454854553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c10f990 msgr2=0x7fdb8c1a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:22.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c10f990 0x7fdb8c1a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb8a000640 1 -- 172.21.15.39:0/3454854553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8c1a5fa0 con 0x7fdb8c10f040 2024-07-04T08:10:22.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8c109700 0x7fdb8c1a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:10:22.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb8c10f040 0x7fdb8c1a05d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c06bf70 tx=0x7fdb7800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:22.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb90a00640 1 -- 172.21.15.39:0/3454854553 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800cc40 con 0x7fdb8c10f040 2024-07-04T08:10:22.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb91e00640 1 -- 172.21.15.39:0/3454854553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8c1a6290 con 0x7fdb8c10f040 2024-07-04T08:10:22.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb90a00640 1 -- 172.21.15.39:0/3454854553 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7800cde0 con 0x7fdb8c10f040 2024-07-04T08:10:22.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb90a00640 1 -- 172.21.15.39:0/3454854553 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78004a00 con 0x7fdb8c10f040 2024-07-04T08:10:22.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.106+0000 7fdb91e00640 1 -- 172.21.15.39:0/3454854553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8c1ac9f0 con 0x7fdb8c10f040 2024-07-04T08:10:22.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.110+0000 7fdb98337740 1 Processor -- start 2024-07-04T08:10:22.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.110+0000 7fdb98337740 1 -- start start 2024-07-04T08:10:22.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.110+0000 7fdb90a00640 1 -- 172.21.15.39:0/3454854553 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fdb78004ba0 con 0x7fdb8c10f040 2024-07-04T08:10:22.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.110+0000 7fdb98337740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341054f60 0x5613413f9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.110+0000 7fdb98337740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x5613413f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.110+0000 7fdb98337740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 0x5613413fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.110+0000 7fdb98337740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56134136fca0 con 0x561341054f60 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.110+0000 7fdb98337740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56134135f240 con 0x56134102c0a0 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.110+0000 7fdb98337740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561341363280 con 0x561341034280 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 0x5613413fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341054f60 0x5613413f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 0x5613413fa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:45614/0 (socket says 172.21.15.39:45614) 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8a000640 1 -- 172.21.15.39:0/3732635289 learned_addr learned my addr 172.21.15.39:0/3732635289 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:22.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/3732635289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x5613413f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8a000640 1 -- 172.21.15.39:0/3732635289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 msgr2=0x5613413f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:10:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3732635289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x5613413f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8a000640 1 -- 172.21.15.39:0/3732635289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341054f60 msgr2=0x5613413f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:10:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3732635289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341054f60 0x5613413f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.111+0000 7fdb8a000640 1 -- 172.21.15.39:0/3732635289 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613412f7550 con 0x561341034280 2024-07-04T08:10:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.112+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3732635289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 0x5613413fa0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdb780dfb70 tx=0x7fdb780e8680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7235088026eaef3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.112+0000 7fdb80c00640 1 -- 172.21.15.39:0/3732635289 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780f6070 con 0x561341034280 2024-07-04T08:10:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.112+0000 7fdb80c00640 1 -- 172.21.15.39:0/3732635289 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb780e2d00 con 0x561341034280 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.112+0000 7fdb80c00640 1 -- 172.21.15.39:0/3732635289 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb780ee930 con 0x561341034280 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.112+0000 7fdb90a00640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdb700b2840 0x7fdb700b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.113+0000 7fdb90a00640 1 -- 172.21.15.39:0/3454854553 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7fdb780e2440 con 0x7fdb8c10f040 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.113+0000 7fdb8b400640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdb700b2840 0x7fdb700b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.113+0000 7fdb98337740 1 -- 172.21.15.39:0/3732635289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 msgr2=0x5613413fa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.113+0000 7fdb98337740 1 --2- 172.21.15.39:0/3732635289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 0x5613413fa0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdb780dfb70 tx=0x7fdb780e8680 comp rx=0 tx=0).stop 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.113+0000 7fdb98337740 1 -- 172.21.15.39:0/3732635289 shutdown_connections 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.113+0000 7fdb98337740 1 --2- 172.21.15.39:0/3732635289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 0x5613413fa0f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.113+0000 7fdb98337740 1 --2- 172.21.15.39:0/3732635289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x5613413f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.113+0000 7fdb98337740 1 --2- 172.21.15.39:0/3732635289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341054f60 0x5613413f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.113+0000 7fdb98337740 1 -- 172.21.15.39:0/3732635289 >> 172.21.15.39:0/3732635289 conn(0x561341049350 msgr2=0x5613413f7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:22.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.114+0000 7fdb98337740 1 -- 172.21.15.39:0/3732635289 shutdown_connections 2024-07-04T08:10:22.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.114+0000 7fdb8b400640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdb700b2840 0x7fdb700b4d50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdb84005e30 tx=0x7fdb84005c50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:22.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.114+0000 7fdb98337740 1 -- 172.21.15.39:0/3732635289 wait complete. 2024-07-04T08:10:22.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.114+0000 7fdb98337740 1 -- start start 2024-07-04T08:10:22.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.115+0000 7fdb98337740 1 Processor -- start 2024-07-04T08:10:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.115+0000 7fdb98337740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x561341405270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.115+0000 7fdb98337740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x5613414057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.115+0000 7fdb98337740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341054f60 0x561341411140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.115+0000 7fdb98337740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5613411c35f0 con 0x561341034280 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.115+0000 7fdb98337740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561341306970 con 0x56134102c0a0 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.115+0000 7fdb98337740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561341407e50 con 0x561341054f60 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.115+0000 7fdb98337740 10 client.0 _ll_register_callbacks cb 0x5613413584a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x561341405270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x561341405270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43748/0 (socket says 172.21.15.39:43748) 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8aa00640 1 -- 172.21.15.39:0/3967081185 learned_addr learned my addr 172.21.15.39:0/3967081185 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8b400640 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341054f60 0x561341411140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x5613414057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8aa00640 1 -- 172.21.15.39:0/3967081185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341054f60 msgr2=0x561341411140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341054f60 0x561341411140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8aa00640 1 -- 172.21.15.39:0/3967081185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 msgr2=0x5613414057b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x5613414057b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8aa00640 1 -- 172.21.15.39:0/3967081185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613410c0540 con 0x56134102c0a0 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x5613414057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x561341405270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c012d20 tx=0x7fdb7c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3448cda6609256 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.116+0000 7fdb75600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.117+0000 7fdb75600640 1 -- 172.21.15.39:0/3967081185 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c012110 con 0x56134102c0a0 2024-07-04T08:10:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.117+0000 7fdb98337740 1 -- 172.21.15.39:0/3967081185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb780e2d00 con 0x56134102c0a0 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.117+0000 7fdb75600640 1 -- 172.21.15.39:0/3967081185 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7fdb7c004050 con 0x56134102c0a0 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.117+0000 7fdb75600640 1 -- 172.21.15.39:0/3967081185 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdb7c025050 con 0x56134102c0a0 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.117+0000 7fdb98337740 10 client.24803 fetch_fsmap learned FSMap version 17 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.117+0000 7fdb98337740 1 -- 172.21.15.39:0/3967081185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x561341413f10 con 0x56134102c0a0 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.117+0000 7fdb75600640 1 -- 172.21.15.39:0/3967081185 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb780e2d00 con 0x56134102c0a0 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.118+0000 7fdb75600640 1 -- 172.21.15.39:0/3967081185 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fdb7c010430 con 0x56134102c0a0 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.118+0000 7fdb98337740 10 client.24803 fetch_fsmap finished waiting for FSMap version 17 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.118+0000 7fdb98337740 10 client.24803 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.118+0000 7fdb98337740 20 client.24803 populate_metadata read hostname 'smithi039' 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.118+0000 7fdb98337740 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5613410138e0 0x561341417980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.118+0000 7fdb98337740 4 client.24803 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.118+0000 7fdb98337740 1 -- 172.21.15.39:0/3967081185 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561341371bf0 con 0x5613410138e0 2024-07-04T08:10:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.118+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5613410138e0 0x561341417980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.119+0000 7fdb8a000640 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5613410138e0 0x561341417980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3444ef566298a9a server_cookie=db2bdd3d4ce7dd95 in_seq=0 out_seq=0 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.119+0000 7fdb75600640 10 client.24803 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb75600640 1 -- 172.21.15.39:0/3967081185 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561341371bf0 con 0x5613410138e0 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb75600640 10 client.24803 handle_command_reply: tid=0 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 client.24803 shutdown 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 -- 172.21.15.39:0/3967081185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 msgr2=0x561341405270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x561341405270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c012d20 tx=0x7fdb7c00f790 comp rx=0 tx=0).stop 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 -- 172.21.15.39:0/3967081185 shutdown_connections 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5613410138e0 0x561341417980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x5613414057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341054f60 0x561341411140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb75600640 0 client.24803 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:10:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 --2- 172.21.15.39:0/3967081185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56134102c0a0 0x561341405270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 -- 172.21.15.39:0/3967081185 >> 172.21.15.39:0/3967081185 conn(0x561341049350 msgr2=0x5613413fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 -- 172.21.15.39:0/3967081185 shutdown_connections 2024-07-04T08:10:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.120+0000 7fdb98337740 1 -- 172.21.15.39:0/3967081185 wait complete. 2024-07-04T08:10:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.121+0000 7fdb98337740 20 client.24803 trim_cache size 0 max 16384 2024-07-04T08:10:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.142+0000 7fdb98337740 1 Processor -- start 2024-07-04T08:10:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.142+0000 7fdb98337740 1 -- start start 2024-07-04T08:10:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.142+0000 7fdb98337740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56134102c0a0 0x5613414110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.142+0000 7fdb98337740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 0x561341415380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.142+0000 7fdb98337740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x561341405250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.142+0000 7fdb98337740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561341415cc0 con 0x56134102c0a0 2024-07-04T08:10:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.142+0000 7fdb98337740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561341416390 con 0x561341054f60 2024-07-04T08:10:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.142+0000 7fdb98337740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561341416a60 con 0x561341034280 2024-07-04T08:10:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.143+0000 7fdb8aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56134102c0a0 0x5613414110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.143+0000 7fdb8aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56134102c0a0 0x5613414110b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48992/0 (socket says 172.21.15.39:48992) 2024-07-04T08:10:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.143+0000 7fdb8aa00640 1 -- 172.21.15.39:0/976178588 learned_addr learned my addr 172.21.15.39:0/976178588 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:22.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.143+0000 7fdb8b400640 1 --2- 172.21.15.39:0/976178588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x561341405250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.143+0000 7fdb8aa00640 1 -- 172.21.15.39:0/976178588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 msgr2=0x561341415380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:10:22.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.143+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/976178588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 0x561341415380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.143+0000 7fdb8aa00640 1 -- 172.21.15.39:0/976178588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 msgr2=0x561341405250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:10:22.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.144+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/976178588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x561341405250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.144+0000 7fdb8aa00640 1 -- 172.21.15.39:0/976178588 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613414077b0 con 0x56134102c0a0 2024-07-04T08:10:22.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.144+0000 7fdb8b400640 1 --2- 172.21.15.39:0/976178588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x561341405250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:10:22.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.144+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/976178588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56134102c0a0 0x5613414110b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c012cf0 tx=0x7fdb7c012320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571bf86b7e363186 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:22.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.144+0000 7fdb77e00640 1 -- 172.21.15.39:0/976178588 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c02e070 con 0x56134102c0a0 2024-07-04T08:10:22.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.145+0000 7fdb77e00640 1 -- 172.21.15.39:0/976178588 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c005560 con 0x56134102c0a0 2024-07-04T08:10:22.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.145+0000 7fdb77e00640 1 -- 172.21.15.39:0/976178588 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c0035f0 con 0x56134102c0a0 2024-07-04T08:10:22.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.146+0000 7fdb98337740 1 -- 172.21.15.39:0/976178588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56134102c0a0 msgr2=0x5613414110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:22.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.146+0000 7fdb98337740 1 --2- 172.21.15.39:0/976178588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56134102c0a0 0x5613414110b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c012cf0 tx=0x7fdb7c012320 comp rx=0 tx=0).stop 2024-07-04T08:10:22.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.146+0000 7fdb98337740 1 -- 172.21.15.39:0/976178588 shutdown_connections 2024-07-04T08:10:22.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.146+0000 7fdb98337740 1 --2- 172.21.15.39:0/976178588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x561341405250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.146+0000 7fdb98337740 1 --2- 172.21.15.39:0/976178588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561341034280 0x561341415380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.146+0000 7fdb98337740 1 --2- 172.21.15.39:0/976178588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56134102c0a0 0x5613414110b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.146+0000 7fdb98337740 1 -- 172.21.15.39:0/976178588 >> 172.21.15.39:0/976178588 conn(0x561341049350 msgr2=0x5613414038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:22.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.146+0000 7fdb98337740 1 -- 172.21.15.39:0/976178588 shutdown_connections 2024-07-04T08:10:22.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.146+0000 7fdb98337740 1 -- 172.21.15.39:0/976178588 wait complete. 2024-07-04T08:10:22.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.147+0000 7fdb98337740 1 -- start start 2024-07-04T08:10:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.147+0000 7fdb98337740 1 Processor -- start 2024-07-04T08:10:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb98337740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x56134141a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb98337740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x56134141a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb98337740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561340fe8c90 0x56134141abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb98337740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561341417800 con 0x561341034280 2024-07-04T08:10:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb98337740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561341417130 con 0x561341054f60 2024-07-04T08:10:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb98337740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561341417ed0 con 0x561340fe8c90 2024-07-04T08:10:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb98337740 10 client.0 _ll_register_callbacks cb 0x7fdb7c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:10:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb8aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x56134141a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb8a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x56134141a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb8aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x56134141a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48994/0 (socket says 172.21.15.39:48994) 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb8aa00640 1 -- 172.21.15.39:0/4138540485 learned_addr learned my addr 172.21.15.39:0/4138540485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.148+0000 7fdb8b400640 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561340fe8c90 0x56134141abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb8a000640 1 -- 172.21.15.39:0/4138540485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561340fe8c90 msgr2=0x56134141abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb8a000640 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561340fe8c90 0x56134141abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb8a000640 1 -- 172.21.15.39:0/4138540485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 msgr2=0x56134141a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb8a000640 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x56134141a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb8a000640 1 -- 172.21.15.39:0/4138540485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56134141f1b0 con 0x561341054f60 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x56134141a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb8a000640 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x56134141a6a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb780a69c0 tx=0x7fdb780ee4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d53bfd855616044e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb75600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:10:22.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb75600640 1 -- 172.21.15.39:0/4138540485 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7801c070 con 0x561341054f60 2024-07-04T08:10:22.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb75600640 1 -- 172.21.15.39:0/4138540485 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7fdb780e21f0 con 0x561341054f60 2024-07-04T08:10:22.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.149+0000 7fdb98337740 1 -- 172.21.15.39:0/4138540485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb7c005560 con 0x561341054f60 2024-07-04T08:10:22.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb75600640 1 -- 172.21.15.39:0/4138540485 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fdb780a5610 con 0x561341054f60 2024-07-04T08:10:22.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb98337740 10 client.24809 fetch_fsmap learned FSMap version 17 2024-07-04T08:10:22.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb98337740 1 -- 172.21.15.39:0/4138540485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x561341420430 con 0x561341054f60 2024-07-04T08:10:22.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb75600640 1 -- 172.21.15.39:0/4138540485 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c005560 con 0x561341054f60 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb75600640 1 -- 172.21.15.39:0/4138540485 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fdb780a59e0 con 0x561341054f60 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb98337740 10 client.24809 fetch_fsmap finished waiting for FSMap version 17 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb98337740 10 client.24809 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb98337740 20 client.24809 populate_metadata read hostname 'smithi039' 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb98337740 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561341482430 0x5613414838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb98337740 4 client.24809 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.150+0000 7fdb98337740 1 -- 172.21.15.39:0/4138540485 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "status"}) -- 0x56134136d690 con 0x561341482430 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.151+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561341482430 0x5613414838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.151+0000 7fdb75600640 10 client.24809 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:10:22.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.152+0000 7fdb8aa00640 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561341482430 0x5613414838a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0b1d15e9196e53 server_cookie=dfefd3cc74b854ce in_seq=0 out_seq=0 2024-07-04T08:10:22.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb75600640 1 -- 172.21.15.39:0/4138540485 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56134136d690 con 0x561341482430 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb75600640 10 client.24809 handle_command_reply: tid=0 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 client.24809 shutdown 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 -- 172.21.15.39:0/4138540485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 msgr2=0x56134141a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x56134141a6a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb780a69c0 tx=0x7fdb780ee4d0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 -- 172.21.15.39:0/4138540485 shutdown_connections 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561341482430 0x5613414838a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561340fe8c90 0x56134141abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561341054f60 0x56134141a6a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 --2- 172.21.15.39:0/4138540485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561341034280 0x56134141a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb75600640 0 client.24809 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 -- 172.21.15.39:0/4138540485 >> 172.21.15.39:0/4138540485 conn(0x561341049350 msgr2=0x561341405250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:22.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 -- 172.21.15.39:0/4138540485 shutdown_connections 2024-07-04T08:10:22.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.153+0000 7fdb98337740 1 -- 172.21.15.39:0/4138540485 wait complete. 2024-07-04T08:10:22.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.154+0000 7fdb98337740 20 client.24809 trim_cache size 0 max 16384 2024-07-04T08:10:22.158 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:10:22.158 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:10:22.158 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-07-04T08:10:22.158 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24707, 2024-07-04T08:10:22.158 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:10:22.158 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:10:22.158 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:10:22.159 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 40.752707286000003, 2024-07-04T08:10:22.159 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 17, 2024-07-04T08:10:22.159 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 88, 2024-07-04T08:10:22.159 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 88, 2024-07-04T08:10:22.159 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 59.852457811999997 2024-07-04T08:10:22.159 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:10:22.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.154+0000 7fdb91e00640 1 -- 172.21.15.39:0/3454854553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdb700b2840 msgr2=0x7fdb700b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.154+0000 7fdb91e00640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdb700b2840 0x7fdb700b4d50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdb84005e30 tx=0x7fdb84005c50 comp rx=0 tx=0).stop 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.154+0000 7fdb91e00640 1 -- 172.21.15.39:0/3454854553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb8c10f040 msgr2=0x7fdb8c1a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.154+0000 7fdb91e00640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb8c10f040 0x7fdb8c1a05d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c06bf70 tx=0x7fdb7800ee50 comp rx=0 tx=0).stop 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.155+0000 7fdb91e00640 1 -- 172.21.15.39:0/3454854553 shutdown_connections 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.155+0000 7fdb91e00640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdb700b2840 0x7fdb700b4d50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.155+0000 7fdb91e00640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb8c10f990 0x7fdb8c1a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.155+0000 7fdb91e00640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb8c10f040 0x7fdb8c1a05d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.155+0000 7fdb91e00640 1 --2- 172.21.15.39:0/3454854553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdb8c109700 0x7fdb8c1a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.155+0000 7fdb91e00640 1 -- 172.21.15.39:0/3454854553 >> 172.21.15.39:0/3454854553 conn(0x7fdb8c077630 msgr2=0x7fdb8c077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.155+0000 7fdb91e00640 1 -- 172.21.15.39:0/3454854553 shutdown_connections 2024-07-04T08:10:22.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:22.155+0000 7fdb91e00640 1 -- 172.21.15.39:0/3454854553 wait complete. 2024-07-04T08:10:22.992 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-04T08:10:22.992 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-07-04T08:10:22.992 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs ls 2024-07-04T08:10:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:22 smithi050 ceph-mon[36577]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:23.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:22 smithi133 ceph-mon[37050]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:23.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:22 smithi039 ceph-mon[32062]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:23.465 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:25.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:24 smithi039 ceph-mon[32062]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:25.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:24 smithi050 ceph-mon[36577]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:24 smithi133 ceph-mon[37050]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:25.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.675+0000 7fe453200640 1 -- 172.21.15.39:0/3794582565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe44c10aa30 msgr2=0x7fe44c10ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.675+0000 7fe453200640 1 --2- 172.21.15.39:0/3794582565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe44c10aa30 0x7fe44c10ae40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009900 tx=0x7fe43c02a410 comp rx=0 tx=0).stop 2024-07-04T08:10:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.675+0000 7fe453200640 1 -- 172.21.15.39:0/3794582565 shutdown_connections 2024-07-04T08:10:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.675+0000 7fe453200640 1 --2- 172.21.15.39:0/3794582565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe44c10b380 0x7fe44c111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.675+0000 7fe453200640 1 --2- 172.21.15.39:0/3794582565 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe44c10aa30 0x7fe44c10ae40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.675+0000 7fe453200640 1 --2- 172.21.15.39:0/3794582565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44c1050f0 0x7fe44c105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.675+0000 7fe453200640 1 -- 172.21.15.39:0/3794582565 >> 172.21.15.39:0/3794582565 conn(0x7fe44c100b60 msgr2=0x7fe44c102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.675+0000 7fe453200640 1 -- 172.21.15.39:0/3794582565 shutdown_connections 2024-07-04T08:10:25.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.675+0000 7fe453200640 1 -- 172.21.15.39:0/3794582565 wait complete. 2024-07-04T08:10:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.677+0000 7fe453200640 1 Processor -- start 2024-07-04T08:10:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.677+0000 7fe453200640 1 -- start start 2024-07-04T08:10:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.677+0000 7fe453200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44c1050f0 0x7fe44c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.677+0000 7fe453200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe44c10aa30 0x7fe44c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.677+0000 7fe453200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe44c10b380 0x7fe44c1a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:25.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.677+0000 7fe453200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c114ff0 con 0x7fe44c10b380 2024-07-04T08:10:25.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.677+0000 7fe453200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c114e70 con 0x7fe44c10aa30 2024-07-04T08:10:25.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.677+0000 7fe453200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c115170 con 0x7fe44c1050f0 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe451e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44c1050f0 0x7fe44c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe451400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe44c10aa30 0x7fe44c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe451e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44c1050f0 0x7fe44c19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:45660/0 (socket says 172.21.15.39:45660) 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe451e00640 1 -- 172.21.15.39:0/3387951889 learned_addr learned my addr 172.21.15.39:0/3387951889 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe452800640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe44c10b380 0x7fe44c1a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe451e00640 1 -- 172.21.15.39:0/3387951889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe44c10aa30 msgr2=0x7fe44c19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe451e00640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe44c10aa30 0x7fe44c19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe451e00640 1 -- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe44c10b380 msgr2=0x7fe44c1a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe451e00640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe44c10b380 0x7fe44c1a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.678+0000 7fe451e00640 1 -- 172.21.15.39:0/3387951889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c1a19b0 con 0x7fe44c1050f0 2024-07-04T08:10:25.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.679+0000 7fe451e00640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44c1050f0 0x7fe44c19ba40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e9f0 tx=0x7fe44000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.679+0000 7fe446a00640 1 -- 172.21.15.39:0/3387951889 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44000cd80 con 0x7fe44c1050f0 2024-07-04T08:10:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.679+0000 7fe446a00640 1 -- 172.21.15.39:0/3387951889 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4400108e0 con 0x7fe44c1050f0 2024-07-04T08:10:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.679+0000 7fe446a00640 1 -- 172.21.15.39:0/3387951889 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440018860 con 0x7fe44c1050f0 2024-07-04T08:10:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.679+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c1a1ca0 con 0x7fe44c1050f0 2024-07-04T08:10:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.679+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c1a8400 con 0x7fe44c1050f0 2024-07-04T08:10:25.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.680+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c105500 con 0x7fe44c1050f0 2024-07-04T08:10:25.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.683+0000 7fe446a00640 1 -- 172.21.15.39:0/3387951889 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fe440010430 con 0x7fe44c1050f0 2024-07-04T08:10:25.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.685+0000 7fe446a00640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4340b28c0 0x7fe4340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.685+0000 7fe446a00640 1 -- 172.21.15.39:0/3387951889 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7fe4400e2d30 con 0x7fe44c1050f0 2024-07-04T08:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.685+0000 7fe451400640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4340b28c0 0x7fe4340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:25.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.686+0000 7fe451400640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4340b28c0 0x7fe4340b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009230 tx=0x7fe43c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:25.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.686+0000 7fe446a00640 1 -- 172.21.15.39:0/3387951889 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe4400aa090 con 0x7fe44c1050f0 2024-07-04T08:10:25.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.942+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe44c19cb70 con 0x7fe44c1050f0 2024-07-04T08:10:25.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.943+0000 7fe446a00640 1 -- 172.21.15.39:0/3387951889 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fe4400ae390 con 0x7fe44c1050f0 2024-07-04T08:10:25.944 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-04T08:10:25.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4340b28c0 msgr2=0x7fe4340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:25.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4340b28c0 0x7fe4340b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009230 tx=0x7fe43c031400 comp rx=0 tx=0).stop 2024-07-04T08:10:25.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44c1050f0 msgr2=0x7fe44c19ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:25.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44c1050f0 0x7fe44c19ba40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e9f0 tx=0x7fe44000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 shutdown_connections 2024-07-04T08:10:25.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4340b28c0 0x7fe4340b4dd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe44c10b380 0x7fe44c1a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe44c10aa30 0x7fe44c19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 --2- 172.21.15.39:0/3387951889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44c1050f0 0x7fe44c19ba40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:25.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 >> 172.21.15.39:0/3387951889 conn(0x7fe44c100b60 msgr2=0x7fe44c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:25.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 shutdown_connections 2024-07-04T08:10:25.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:25.947+0000 7fe453200640 1 -- 172.21.15.39:0/3387951889 wait complete. 2024-07-04T08:10:26.753 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-04T08:10:26.753 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T08:10:26.753 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-07-04T08:10:26.754 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-07-04T08:10:26.754 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-07-04T08:10:26.754 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-04T08:10:26.754 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T08:10:26.754 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T08:10:26.754 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-07-04T08:10:26.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:26.754 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-07-04T08:10:26.809 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T08:10:26.810 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41950sec preferred_lft 41950sec 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link noprefixroute 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-07-04T08:10:26.811 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-07-04T08:10:26.813 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-04T08:10:26.813 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:10:26.813 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:10:26.813 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2024-07-04T08:10:26.813 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2024-07-04T08:10:26.813 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2024-07-04T08:10:26.813 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-04T08:10:26.813 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:10:26.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:10:26.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:26 smithi039 ceph-mon[32062]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:26.953 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:26 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3387951889' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-04T08:10:27.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:26 smithi050 ceph-mon[36577]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:27.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:26 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3387951889' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-04T08:10:27.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:10:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:26 smithi133 ceph-mon[37050]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:26 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3387951889' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-04T08:10:27.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:27.319 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T08:10:27.391 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-07-04T08:10:27.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:27.395 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-07-04T08:10:27.449 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-07-04T08:10:27.449 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T08:10:27.449 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T08:10:27.449 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T08:10:27.449 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T08:10:27.450 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:10:27.450 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:10:27.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T08:10:27.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T08:10:27.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T08:10:27.450 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:10:27.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:10:28.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:10:28.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:28.190 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-07-04T08:10:28.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:28.243 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-07-04T08:10:28.295 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:10:28.295 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:10:28.295 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-04T08:10:28.295 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-07-04T08:10:28.295 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:10:28.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:10:28.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:10:28.479 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-07-04T08:10:28.480 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:10:28.480 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:10:28.480 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-07-04T08:10:28.480 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-04T08:10:28.480 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-07-04T08:10:28.480 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-07-04T08:10:28.480 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-07-04T08:10:28.480 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:10:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:10:28.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:10:28.783 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:10:28.783 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:10:28.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-07-04T08:10:28.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2024-07-04T08:10:28.783 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:10:28.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:10:28.958 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:28 smithi039 ceph-mon[32062]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:28.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:10:28.993 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-07-04T08:10:28.993 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T08:10:28.993 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-07-04T08:10:29.040 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-07-04T08:10:29.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T08:10:29.040 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-07-04T08:10:29.091 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-07-04T08:10:29.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:28 smithi050 ceph-mon[36577]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:29.169 DEBUG:teuthology.orchestra.run.smithi039:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T08:10:29.221 INFO:teuthology.orchestra.run.smithi039.stdout:/proc 2024-07-04T08:10:29.221 INFO:teuthology.orchestra.run.smithi039.stdout:/sys 2024-07-04T08:10:29.221 INFO:teuthology.orchestra.run.smithi039.stdout:/dev 2024-07-04T08:10:29.221 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/security 2024-07-04T08:10:29.221 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/shm 2024-07-04T08:10:29.221 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/pts 2024-07-04T08:10:29.221 INFO:teuthology.orchestra.run.smithi039.stdout:/run 2024-07-04T08:10:29.221 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/cgroup 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/pstore 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/bpf 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/ 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/selinux 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/mqueue 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/hugepages 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/debug 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/tracing 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/fuse/connections 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/config 2024-07-04T08:10:29.222 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/1000 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/ceph 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/f44815c12f798171b909d3b700b8a236669c77ec96616fb0282144408af69184/merged 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/a6948ed3f4ab5364908e2e5a6c87cf8c14e5752bcd4f4722078eb9d051f2d984/merged 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/0 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/fb981f71d26ebbd7cd690fe674b44a48703ac2f8ad83ea8b24855923adfc6d29/merged 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/574775a1cea0447ae37b16e9bc47dd1c22e935fb9e244567422d612ae3955e73/merged 2024-07-04T08:10:29.223 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/e8ab2c91470016308cd384fcb0c2847c8f8a316a87f8dc835b1cf4fb9526b285/merged 2024-07-04T08:10:29.224 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/a4f7bef0719e47fd6aa90d179e5c6d451a284b183e49cd5a9ef3b31e74e79301/merged 2024-07-04T08:10:29.224 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/4c378b0b56756333cb19bba271a645e35bf475802e114bb58a38c3612b83fb6c/merged 2024-07-04T08:10:29.224 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/43295e0185d1fa12e837a0f74cc6c14090cb5e5eb47f6456139366a9e92e1180/merged 2024-07-04T08:10:29.224 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/b047b15c9b7ead1797b2595a227da7715fd7ad2b4f0fe1a2209472417c24808a/merged 2024-07-04T08:10:29.224 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/e83f999a87fdc8466192bf64841b46a11af68e273d4e56ba5dacb31da843c7ab/merged 2024-07-04T08:10:29.224 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns 2024-07-04T08:10:29.224 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-04T08:10:29.224 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-04T08:10:29.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:29.225 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-07-04T08:10:29.274 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-04T08:10:29.274 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-07-04T08:10:29.279 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-07-04T08:10:29.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:28 smithi133 ceph-mon[37050]: pgmap v334: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:29.345 DEBUG:teuthology.orchestra.run.smithi039:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T08:10:29.387 INFO:teuthology.orchestra.run.smithi039.stdout:/proc 2024-07-04T08:10:29.387 INFO:teuthology.orchestra.run.smithi039.stdout:/sys 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/dev 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/security 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/shm 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/pts 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/run 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/cgroup 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/pstore 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/bpf 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/ 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/selinux 2024-07-04T08:10:29.388 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/mqueue 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/hugepages 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/debug 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/tracing 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/fuse/connections 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/config 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T08:10:29.389 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/1000 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/ceph 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/f44815c12f798171b909d3b700b8a236669c77ec96616fb0282144408af69184/merged 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/a6948ed3f4ab5364908e2e5a6c87cf8c14e5752bcd4f4722078eb9d051f2d984/merged 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/0 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/fb981f71d26ebbd7cd690fe674b44a48703ac2f8ad83ea8b24855923adfc6d29/merged 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/574775a1cea0447ae37b16e9bc47dd1c22e935fb9e244567422d612ae3955e73/merged 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/e8ab2c91470016308cd384fcb0c2847c8f8a316a87f8dc835b1cf4fb9526b285/merged 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/a4f7bef0719e47fd6aa90d179e5c6d451a284b183e49cd5a9ef3b31e74e79301/merged 2024-07-04T08:10:29.390 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/4c378b0b56756333cb19bba271a645e35bf475802e114bb58a38c3612b83fb6c/merged 2024-07-04T08:10:29.391 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/43295e0185d1fa12e837a0f74cc6c14090cb5e5eb47f6456139366a9e92e1180/merged 2024-07-04T08:10:29.391 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/b047b15c9b7ead1797b2595a227da7715fd7ad2b4f0fe1a2209472417c24808a/merged 2024-07-04T08:10:29.391 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/e83f999a87fdc8466192bf64841b46a11af68e273d4e56ba5dacb31da843c7ab/merged 2024-07-04T08:10:29.391 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns 2024-07-04T08:10:29.391 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-04T08:10:29.391 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-04T08:10:29.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:29.392 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-07-04T08:10:29.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi039.stderr:2024-07-04T08:10:29.550+0000 7f66c660c480 -1 init, newargv = 0x7f66b80046a0 newargc=15 2024-07-04T08:10:29.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi039.stderr:ceph-fuse[82569]: starting ceph client 2024-07-04T08:10:29.562 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi039.stderr:ceph-fuse[82569]: starting fuse 2024-07-04T08:10:30.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo modprobe fuse 2024-07-04T08:10:30.682 DEBUG:teuthology.orchestra.run.smithi039:> cat /proc/self/mounts | awk '{print $2}' 2024-07-04T08:10:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:/proc 2024-07-04T08:10:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:/sys 2024-07-04T08:10:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:/dev 2024-07-04T08:10:30.743 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/security 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/shm 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/pts 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/run 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/cgroup 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/pstore 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/bpf 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/ 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/selinux 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/mqueue 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/hugepages 2024-07-04T08:10:30.744 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/debug 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/tracing 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/fs/fuse/connections 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/sys/kernel/config 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-sysctl.service 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/nfs/rpc_pipefs 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/1000 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/ceph 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/f44815c12f798171b909d3b700b8a236669c77ec96616fb0282144408af69184/merged 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/a6948ed3f4ab5364908e2e5a6c87cf8c14e5752bcd4f4722078eb9d051f2d984/merged 2024-07-04T08:10:30.745 INFO:teuthology.orchestra.run.smithi039.stdout:/run/user/0 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/proc/sys/fs/binfmt_misc 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/fb981f71d26ebbd7cd690fe674b44a48703ac2f8ad83ea8b24855923adfc6d29/merged 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/574775a1cea0447ae37b16e9bc47dd1c22e935fb9e244567422d612ae3955e73/merged 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/e8ab2c91470016308cd384fcb0c2847c8f8a316a87f8dc835b1cf4fb9526b285/merged 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/a4f7bef0719e47fd6aa90d179e5c6d451a284b183e49cd5a9ef3b31e74e79301/merged 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/4c378b0b56756333cb19bba271a645e35bf475802e114bb58a38c3612b83fb6c/merged 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/43295e0185d1fa12e837a0f74cc6c14090cb5e5eb47f6456139366a9e92e1180/merged 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/b047b15c9b7ead1797b2595a227da7715fd7ad2b4f0fe1a2209472417c24808a/merged 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/var/lib/containers/storage/overlay/e83f999a87fdc8466192bf64841b46a11af68e273d4e56ba5dacb31da843c7ab/merged 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-04T08:10:30.746 INFO:teuthology.orchestra.run.smithi039.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-04T08:10:30.747 INFO:teuthology.orchestra.run.smithi039.stdout:/home/ubuntu/cephtest/mnt.admin 2024-07-04T08:10:30.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:30.748 DEBUG:teuthology.orchestra.run.smithi039:> ls /sys/fs/fuse/connections 2024-07-04T08:10:30.801 INFO:teuthology.orchestra.run.smithi039.stdout:119 2024-07-04T08:10:30.802 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-07-04T08:10:30.802 DEBUG:teuthology.orchestra.run.smithi039:> sudo stdin-killer -- python3 -c ' 2024-07-04T08:10:30.802 DEBUG:teuthology.orchestra.run.smithi039:> import glob 2024-07-04T08:10:30.802 DEBUG:teuthology.orchestra.run.smithi039:> import re 2024-07-04T08:10:30.802 DEBUG:teuthology.orchestra.run.smithi039:> import os 2024-07-04T08:10:30.802 DEBUG:teuthology.orchestra.run.smithi039:> import subprocess 2024-07-04T08:10:30.802 DEBUG:teuthology.orchestra.run.smithi039:> 2024-07-04T08:10:30.802 DEBUG:teuthology.orchestra.run.smithi039:> def _find_admin_socket(client_name): 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> files = glob.glob(asok_path) 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> # Given a non-glob path, it better be there 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> if "*" not in asok_path: 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> assert(len(files) == 1) 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> return files[0] 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> for f in files: 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> contents = proc_f.read() 2024-07-04T08:10:30.803 DEBUG:teuthology.orchestra.run.smithi039:> if mountpoint in contents: 2024-07-04T08:10:30.804 DEBUG:teuthology.orchestra.run.smithi039:> return f 2024-07-04T08:10:30.804 DEBUG:teuthology.orchestra.run.smithi039:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-04T08:10:30.804 DEBUG:teuthology.orchestra.run.smithi039:> 2024-07-04T08:10:30.804 DEBUG:teuthology.orchestra.run.smithi039:> print(_find_admin_socket("client.admin")) 2024-07-04T08:10:30.804 DEBUG:teuthology.orchestra.run.smithi039:> ' 2024-07-04T08:10:30.980 INFO:teuthology.orchestra.run.smithi039.stdout:/var/run/ceph/ceph-client.admin.82569.asok 2024-07-04T08:10:30.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:10:30.994 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:30 smithi039 ceph-mon[32062]: pgmap v335: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:30.998 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.82569.asok 2024-07-04T08:10:30.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:30.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.82569.asok status 2024-07-04T08:10:31.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:30 smithi050 ceph-mon[36577]: pgmap v335: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:31.310 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:10:31.310 INFO:teuthology.orchestra.run.smithi039.stdout: "metadata": { 2024-07-04T08:10:31.310 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_sha1": "2e9f68b5506c607b2f7d69d380cb7887cf00dc84", 2024-07-04T08:10:31.310 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_version": "ceph version 19.0.0-4737-g2e9f68b5 (2e9f68b5506c607b2f7d69d380cb7887cf00dc84) squid (dev)", 2024-07-04T08:10:31.310 INFO:teuthology.orchestra.run.smithi039.stdout: "entity_id": "admin", 2024-07-04T08:10:31.310 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": "smithi039", 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "pid": "82569", 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "root": "/" 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_count": 0, 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "dentry_pinned_count": 0, 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 14691, 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "inst": { 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "name": { 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "client", 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: "num": 14691 2024-07-04T08:10:31.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3942515317 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": { 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "192.168.0.1:0", 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3942515317 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:10:31.312 INFO:teuthology.orchestra.run.smithi039.stdout: "inst_str": "client.14691 192.168.0.1:0/3942515317", 2024-07-04T08:10:31.313 INFO:teuthology.orchestra.run.smithi039.stdout: "addr_str": "192.168.0.1:0/3942515317", 2024-07-04T08:10:31.313 INFO:teuthology.orchestra.run.smithi039.stdout: "inode_count": 1, 2024-07-04T08:10:31.313 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_epoch": 17, 2024-07-04T08:10:31.313 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch": 88, 2024-07-04T08:10:31.313 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_epoch_barrier": 0, 2024-07-04T08:10:31.313 INFO:teuthology.orchestra.run.smithi039.stdout: "blocklisted": false, 2024-07-04T08:10:31.313 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs" 2024-07-04T08:10:31.313 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:10:31.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:30 smithi133 ceph-mon[37050]: pgmap v335: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:31.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:31.332 DEBUG:teuthology.orchestra.run.smithi039:> stat --file-system '--printf=%T 2024-07-04T08:10:31.332 DEBUG:teuthology.orchestra.run.smithi039:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-07-04T08:10:31.390 INFO:teuthology.orchestra.run.smithi039.stdout:fuseblk 2024-07-04T08:10:31.390 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-07-04T08:10:31.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:31.390 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-07-04T08:10:31.483 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-07-04T08:10:31.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-04T08:10:31.613 INFO:teuthology.orchestra.run.smithi039.stdout:# file: . 2024-07-04T08:10:31.613 INFO:teuthology.orchestra.run.smithi039.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-07-04T08:10:31.614 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:10:31.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:10:31.624 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:10:31.624 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T08:10:31.672 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd blocklist ls 2024-07-04T08:10:32.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:31 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:10:32.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:31 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:10:32.186 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:32.230 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:31 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:10:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:33 smithi133 ceph-mon[37050]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:33.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:33 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:33 smithi039 ceph-mon[32062]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:33.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:33 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:33 smithi050 ceph-mon[36577]: pgmap v336: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:33 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:34.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.403+0000 7f760b200640 1 -- 172.21.15.39:0/1685780111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f760410d170 msgr2=0x7f760410f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:34.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.403+0000 7f760b200640 1 --2- 172.21.15.39:0/1685780111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f760410d170 0x7f760410f660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7600009ca0 tx=0x7f760002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.403+0000 7f760b200640 1 -- 172.21.15.39:0/1685780111 shutdown_connections 2024-07-04T08:10:34.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.403+0000 7f760b200640 1 --2- 172.21.15.39:0/1685780111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f760410d170 0x7f760410f660 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.403+0000 7f760b200640 1 --2- 172.21.15.39:0/1685780111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7604104660 0x7f760410cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.403+0000 7f760b200640 1 --2- 172.21.15.39:0/1685780111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7604103d10 0x7f7604104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.403+0000 7f760b200640 1 -- 172.21.15.39:0/1685780111 >> 172.21.15.39:0/1685780111 conn(0x7f76040fd810 msgr2=0x7f76040ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:34.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.403+0000 7f760b200640 1 -- 172.21.15.39:0/1685780111 shutdown_connections 2024-07-04T08:10:34.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.403+0000 7f760b200640 1 -- 172.21.15.39:0/1685780111 wait complete. 2024-07-04T08:10:34.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.404+0000 7f760b200640 1 Processor -- start 2024-07-04T08:10:34.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.404+0000 7f760b200640 1 -- start start 2024-07-04T08:10:34.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f760b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7604103d10 0x7f760419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:34.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f760b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7604104660 0x7f760419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:34.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f760b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f760410d170 0x7f76041a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:34.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f760b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7604112df0 con 0x7f7604103d10 2024-07-04T08:10:34.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f760b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7604112c70 con 0x7f7604104660 2024-07-04T08:10:34.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f760b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7604112f70 con 0x7f760410d170 2024-07-04T08:10:34.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f7609e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7604103d10 0x7f760419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:34.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f7609e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7604103d10 0x7f760419b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43012/0 (socket says 172.21.15.39:43012) 2024-07-04T08:10:34.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f7609e00640 1 -- 172.21.15.39:0/2933635875 learned_addr learned my addr 172.21.15.39:0/2933635875 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:34.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f760a800640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f760410d170 0x7f76041a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:34.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.405+0000 7f7609400640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7604104660 0x7f760419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:34.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.406+0000 7f7609e00640 1 -- 172.21.15.39:0/2933635875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f760410d170 msgr2=0x7f76041a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:34.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.406+0000 7f7609e00640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f760410d170 0x7f76041a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.406+0000 7f7609e00640 1 -- 172.21.15.39:0/2933635875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7604104660 msgr2=0x7f760419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:34.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.406+0000 7f7609e00640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7604104660 0x7f760419bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.406+0000 7f7609e00640 1 -- 172.21.15.39:0/2933635875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76041a18d0 con 0x7f7604103d10 2024-07-04T08:10:34.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.406+0000 7f760a800640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f760410d170 0x7f76041a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:10:34.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.406+0000 7f7609400640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7604104660 0x7f760419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:10:34.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.406+0000 7f7609e00640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7604103d10 0x7f760419b960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f75f800e9f0 tx=0x7f75f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:34.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.406+0000 7f75fea00640 1 -- 172.21.15.39:0/2933635875 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f800cd80 con 0x7f7604103d10 2024-07-04T08:10:34.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.407+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76041a1bc0 con 0x7f7604103d10 2024-07-04T08:10:34.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.407+0000 7f75fea00640 1 -- 172.21.15.39:0/2933635875 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75f80108e0 con 0x7f7604103d10 2024-07-04T08:10:34.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.407+0000 7f75fea00640 1 -- 172.21.15.39:0/2933635875 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8018980 con 0x7f7604103d10 2024-07-04T08:10:34.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.407+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76041a8320 con 0x7f7604103d10 2024-07-04T08:10:34.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.408+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7604104120 con 0x7f7604103d10 2024-07-04T08:10:34.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.411+0000 7f75fea00640 1 -- 172.21.15.39:0/2933635875 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f75f8010430 con 0x7f7604103d10 2024-07-04T08:10:34.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.412+0000 7f75fea00640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f75ec0b2840 0x7f75ec0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:34.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.412+0000 7f75fea00640 1 -- 172.21.15.39:0/2933635875 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7f75f80e2d80 con 0x7f7604103d10 2024-07-04T08:10:34.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.412+0000 7f7609400640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f75ec0b2840 0x7f75ec0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:34.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.413+0000 7f7609400640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f75ec0b2840 0x7f75ec0b4d50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7604100f80 tx=0x7f75f4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:34.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.415+0000 7f75fea00640 1 -- 172.21.15.39:0/2933635875 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f75f80aa0e0 con 0x7f7604103d10 2024-07-04T08:10:34.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.642+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f76041a8600 con 0x7f7604103d10 2024-07-04T08:10:34.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.643+0000 7f75fea00640 1 -- 172.21.15.39:0/2933635875 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v88) ==== 81+0+886 (secure 0 0 0) 0x7f75f80ae3e0 con 0x7f7604103d10 2024-07-04T08:10:34.644 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/2939871551 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:34.644 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3207006577 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/2939871551 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1920202184 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/764812681 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3823748890 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1949300646 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2171986895 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1596793936 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3207006577 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1922588202 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1763496185 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/917994444 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:34.645 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1237664844 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:34.646 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1922588202 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:34.646 INFO:teuthology.orchestra.run.smithi039.stderr:listed 15 entries 2024-07-04T08:10:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f75ec0b2840 msgr2=0x7f75ec0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f75ec0b2840 0x7f75ec0b4d50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7604100f80 tx=0x7f75f4005f90 comp rx=0 tx=0).stop 2024-07-04T08:10:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7604103d10 msgr2=0x7f760419b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7604103d10 0x7f760419b960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f75f800e9f0 tx=0x7f75f800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 shutdown_connections 2024-07-04T08:10:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f75ec0b2840 0x7f75ec0b4d50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f760410d170 0x7f76041a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7604104660 0x7f760419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 --2- 172.21.15.39:0/2933635875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7604103d10 0x7f760419b960 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 >> 172.21.15.39:0/2933635875 conn(0x7f76040fd810 msgr2=0x7f7604101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 shutdown_connections 2024-07-04T08:10:34.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:34.646+0000 7f760b200640 1 -- 172.21.15.39:0/2933635875 wait complete. 2024-07-04T08:10:35.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:35 smithi133 ceph-mon[37050]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:35.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:35 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2933635875' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-04T08:10:35.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:35 smithi039 ceph-mon[32062]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:35.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:35 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2933635875' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-04T08:10:35.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:35 smithi050 ceph-mon[36577]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:35.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:35 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2933635875' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-04T08:10:35.452 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:10:35.452 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T08:10:35.497 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd blocklist ls 2024-07-04T08:10:35.998 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:37.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:37 smithi133 ceph-mon[37050]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:37.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:37 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:37.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:37 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:37.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:37 smithi039 ceph-mon[32062]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:37.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:37 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:37.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:37 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:37.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:37 smithi050 ceph-mon[36577]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:37.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:37 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:37.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:37 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:38.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.287+0000 7f3dbe200640 1 -- 172.21.15.39:0/321487768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db810ee50 msgr2=0x7f3db810f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.287+0000 7f3dbe200640 1 --2- 172.21.15.39:0/321487768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db810ee50 0x7f3db810f260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009900 tx=0x7f3da402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.287+0000 7f3dbe200640 1 -- 172.21.15.39:0/321487768 shutdown_connections 2024-07-04T08:10:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.287+0000 7f3dbe200640 1 --2- 172.21.15.39:0/321487768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db810f7a0 0x7f3db8115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.287+0000 7f3dbe200640 1 --2- 172.21.15.39:0/321487768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db810ee50 0x7f3db810f260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.287+0000 7f3dbe200640 1 --2- 172.21.15.39:0/321487768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db8109510 0x7f3db8109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.287+0000 7f3dbe200640 1 -- 172.21.15.39:0/321487768 >> 172.21.15.39:0/321487768 conn(0x7f3db8078b10 msgr2=0x7f3db8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.287+0000 7f3dbe200640 1 -- 172.21.15.39:0/321487768 shutdown_connections 2024-07-04T08:10:38.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.287+0000 7f3dbe200640 1 -- 172.21.15.39:0/321487768 wait complete. 2024-07-04T08:10:38.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.288+0000 7f3dbe200640 1 Processor -- start 2024-07-04T08:10:38.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.288+0000 7f3dbe200640 1 -- start start 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbe200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8109510 0x7f3db819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbe200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db810ee50 0x7f3db81a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbe200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db810f7a0 0x7f3db81a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbe200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8119480 con 0x7f3db810ee50 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbe200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8119300 con 0x7f3db8109510 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbe200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8119600 con 0x7f3db810f7a0 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8109510 0x7f3db819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3db7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db810ee50 0x7f3db81a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db810f7a0 0x7f3db81a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8109510 0x7f3db819fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53914/0 (socket says 172.21.15.39:53914) 2024-07-04T08:10:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbce00640 1 -- 172.21.15.39:0/2121220275 learned_addr learned my addr 172.21.15.39:0/2121220275 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.289+0000 7f3dbd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db810f7a0 0x7f3db81a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37188/0 (socket says 172.21.15.39:37188) 2024-07-04T08:10:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.290+0000 7f3dbce00640 1 -- 172.21.15.39:0/2121220275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db810f7a0 msgr2=0x7f3db81a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.290+0000 7f3dbce00640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db810f7a0 0x7f3db81a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.290+0000 7f3dbce00640 1 -- 172.21.15.39:0/2121220275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db810ee50 msgr2=0x7f3db81a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.290+0000 7f3dbce00640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db810ee50 0x7f3db81a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.290+0000 7f3dbce00640 1 -- 172.21.15.39:0/2121220275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db81a5d80 con 0x7f3db8109510 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.290+0000 7f3dbce00640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8109510 0x7f3db819fe10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3da800e9f0 tx=0x7f3da800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.291+0000 7f3db5600640 1 -- 172.21.15.39:0/2121220275 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da800cd80 con 0x7f3db8109510 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.291+0000 7f3db5600640 1 -- 172.21.15.39:0/2121220275 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da80108e0 con 0x7f3db8109510 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.291+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db81a6070 con 0x7f3db8109510 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.291+0000 7f3db5600640 1 -- 172.21.15.39:0/2121220275 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da8018980 con 0x7f3db8109510 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.291+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db81ac7d0 con 0x7f3db8109510 2024-07-04T08:10:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.292+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db8109920 con 0x7f3db8109510 2024-07-04T08:10:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.295+0000 7f3db5600640 1 -- 172.21.15.39:0/2121220275 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f3da8010430 con 0x7f3db8109510 2024-07-04T08:10:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.297+0000 7f3db5600640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3d9c0b2840 0x7f3d9c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.297+0000 7f3db5600640 1 -- 172.21.15.39:0/2121220275 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7f3da80e2d20 con 0x7f3db8109510 2024-07-04T08:10:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.297+0000 7f3db7e00640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3d9c0b2840 0x7f3d9c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:38.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.298+0000 7f3db7e00640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3d9c0b2840 0x7f3d9c0b4d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3da402a9e0 tx=0x7f3da4005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:38.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.298+0000 7f3db5600640 1 -- 172.21.15.39:0/2121220275 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f3da80aa0e0 con 0x7f3db8109510 2024-07-04T08:10:38.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:10:38.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:10:38.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:38.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:10:38.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:10:38.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:38.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:10:38.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:10:38.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:10:38.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.524+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3db81a0f40 con 0x7f3db8109510 2024-07-04T08:10:38.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.525+0000 7f3db5600640 1 -- 172.21.15.39:0/2121220275 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v88) ==== 81+0+886 (secure 0 0 0) 0x7f3da80ae3e0 con 0x7f3db8109510 2024-07-04T08:10:38.526 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/2939871551 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:38.526 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3207006577 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:38.526 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/2939871551 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:38.526 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1920202184 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:38.526 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/764812681 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3823748890 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1949300646 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2171986895 2024-07-05T07:59:57.104896+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1596793936 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3207006577 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1922588202 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1763496185 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/917994444 2024-07-05T08:00:28.376822+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1237664844 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1922588202 2024-07-05T08:01:51.343953+0000 2024-07-04T08:10:38.527 INFO:teuthology.orchestra.run.smithi039.stderr:listed 15 entries 2024-07-04T08:10:38.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.528+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3d9c0b2840 msgr2=0x7f3d9c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:38.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.528+0000 7f3dbe200640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3d9c0b2840 0x7f3d9c0b4d50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3da402a9e0 tx=0x7f3da4005a20 comp rx=0 tx=0).stop 2024-07-04T08:10:38.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.528+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8109510 msgr2=0x7f3db819fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:38.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.528+0000 7f3dbe200640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8109510 0x7f3db819fe10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3da800e9f0 tx=0x7f3da800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.529+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 shutdown_connections 2024-07-04T08:10:38.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.529+0000 7f3dbe200640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3d9c0b2840 0x7f3d9c0b4d50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.529+0000 7f3dbe200640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3db810ee50 0x7f3db81a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.529+0000 7f3dbe200640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3db810f7a0 0x7f3db81a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.529+0000 7f3dbe200640 1 --2- 172.21.15.39:0/2121220275 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3db8109510 0x7f3db819fe10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:38.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.529+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 >> 172.21.15.39:0/2121220275 conn(0x7f3db8078b10 msgr2=0x7f3db80791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:38.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.529+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 shutdown_connections 2024-07-04T08:10:38.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:38.529+0000 7f3dbe200640 1 -- 172.21.15.39:0/2121220275 wait complete. 2024-07-04T08:10:39.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:39 smithi039 ceph-mon[32062]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:39.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:39 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2121220275' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-04T08:10:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:39 smithi050 ceph-mon[36577]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:39 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2121220275' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-04T08:10:39.406 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:39 smithi133 ceph-mon[37050]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:39.409 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:39 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2121220275' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-04T08:10:39.410 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi039.front.sepia.ceph.com... 2024-07-04T08:10:39.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:39.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-07-04T08:10:39.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi039.stderr:ceph-fuse[82569]: fuse finished with error 0 and tester_r 0 2024-07-04T08:10:39.509 INFO:teuthology.orchestra.run:waiting for 300 2024-07-04T08:10:40.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:40 smithi039 ceph-mon[32062]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:40.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:40 smithi050 ceph-mon[36577]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:40.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:40 smithi133 ceph-mon[37050]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:42.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:42 smithi039 ceph-mon[32062]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:42 smithi050 ceph-mon[36577]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:42 smithi133 ceph-mon[37050]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-04T08:10:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:44 smithi050 ceph-mon[36577]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-04T08:10:45.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:44 smithi133 ceph-mon[37050]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-04T08:10:45.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:44 smithi039 ceph-mon[32062]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-04T08:10:45.610 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T08:10:45.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:10:45.611 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-07-04T08:10:45.680 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:10:46.151 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:46.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:46 smithi050 ceph-mon[36577]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:47.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:46 smithi133 ceph-mon[37050]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:47.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:46 smithi039 ceph-mon[32062]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.381+0000 7fea23200640 1 -- 172.21.15.39:0/852103580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea241055f0 msgr2=0x7fea24105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.381+0000 7fea23200640 1 --2- 172.21.15.39:0/852103580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea241055f0 0x7fea24105a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009900 tx=0x7fea0c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.381+0000 7fea23200640 1 -- 172.21.15.39:0/852103580 shutdown_connections 2024-07-04T08:10:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.381+0000 7fea23200640 1 --2- 172.21.15.39:0/852103580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea241055f0 0x7fea24105a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.381+0000 7fea23200640 1 --2- 172.21.15.39:0/852103580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea24105f40 0x7fea2410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.381+0000 7fea23200640 1 --2- 172.21.15.39:0/852103580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24101310 0x7fea24101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.381+0000 7fea23200640 1 -- 172.21.15.39:0/852103580 >> 172.21.15.39:0/852103580 conn(0x7fea24077000 msgr2=0x7fea24077440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.381+0000 7fea23200640 1 -- 172.21.15.39:0/852103580 shutdown_connections 2024-07-04T08:10:48.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.381+0000 7fea23200640 1 -- 172.21.15.39:0/852103580 wait complete. 2024-07-04T08:10:48.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.382+0000 7fea23200640 1 Processor -- start 2024-07-04T08:10:48.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.383+0000 7fea23200640 1 -- start start 2024-07-04T08:10:48.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.383+0000 7fea23200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24101310 0x7fea2419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:48.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.383+0000 7fea23200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea241055f0 0x7fea2419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:48.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.383+0000 7fea23200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea24105f40 0x7fea241a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:48.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.383+0000 7fea23200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fea24112e90 con 0x7fea241055f0 2024-07-04T08:10:48.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.383+0000 7fea23200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fea24112d10 con 0x7fea24105f40 2024-07-04T08:10:48.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.383+0000 7fea23200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea24113010 con 0x7fea24101310 2024-07-04T08:10:48.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.383+0000 7fea21e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24101310 0x7fea2419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:48.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea21e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24101310 0x7fea2419ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39028/0 (socket says 172.21.15.39:39028) 2024-07-04T08:10:48.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea22800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea24105f40 0x7fea241a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:48.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea1be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea241055f0 0x7fea2419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:48.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea21e00640 1 -- 172.21.15.39:0/1809524073 learned_addr learned my addr 172.21.15.39:0/1809524073 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:48.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea21e00640 1 -- 172.21.15.39:0/1809524073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea24105f40 msgr2=0x7fea241a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:48.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea21e00640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea24105f40 0x7fea241a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea21e00640 1 -- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea241055f0 msgr2=0x7fea2419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:48.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea21e00640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea241055f0 0x7fea2419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea21e00640 1 -- 172.21.15.39:0/1809524073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea241a19d0 con 0x7fea24101310 2024-07-04T08:10:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.384+0000 7fea1be00640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea241055f0 0x7fea2419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:10:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.385+0000 7fea21e00640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24101310 0x7fea2419ba60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea1400e9f0 tx=0x7fea1400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.385+0000 7fea1aa00640 1 -- 172.21.15.39:0/1809524073 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1400cd80 con 0x7fea24101310 2024-07-04T08:10:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.385+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea241a1cc0 con 0x7fea24101310 2024-07-04T08:10:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.385+0000 7fea1aa00640 1 -- 172.21.15.39:0/1809524073 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea140054d0 con 0x7fea24101310 2024-07-04T08:10:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.385+0000 7fea1aa00640 1 -- 172.21.15.39:0/1809524073 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14010690 con 0x7fea24101310 2024-07-04T08:10:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.385+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea241a8420 con 0x7fea24101310 2024-07-04T08:10:48.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.386+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea24101720 con 0x7fea24101310 2024-07-04T08:10:48.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.389+0000 7fea1aa00640 1 -- 172.21.15.39:0/1809524073 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fea14004e30 con 0x7fea24101310 2024-07-04T08:10:48.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.390+0000 7fea1aa00640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea080b2840 0x7fea080b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:48.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.391+0000 7fea1aa00640 1 -- 172.21.15.39:0/1809524073 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9017+0+0 (secure 0 0 0) 0x7fea140e2ba0 con 0x7fea24101310 2024-07-04T08:10:48.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.391+0000 7fea1be00640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea080b2840 0x7fea080b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:48.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.392+0000 7fea1be00640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea080b2840 0x7fea080b4d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea0c02a9e0 tx=0x7fea0c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:48.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.392+0000 7fea1aa00640 1 -- 172.21.15.39:0/1809524073 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fea14055110 con 0x7fea24101310 2024-07-04T08:10:48.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.605+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea2419cb90 con 0x7fea24101310 2024-07-04T08:10:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.607+0000 7fea1aa00640 1 -- 172.21.15.39:0/1809524073 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24384 (secure 0 0 0) 0x7fea2419cb90 con 0x7fea24101310 2024-07-04T08:10:48.608 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:10:48.609 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":88,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:09:58.720601+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:10:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea080b2840 msgr2=0x7fea080b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea080b2840 0x7fea080b4d50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea0c02a9e0 tx=0x7fea0c005a20 comp rx=0 tx=0).stop 2024-07-04T08:10:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24101310 msgr2=0x7fea2419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24101310 0x7fea2419ba60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea1400e9f0 tx=0x7fea1400edf0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 shutdown_connections 2024-07-04T08:10:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea080b2840 0x7fea080b4d50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea241055f0 0x7fea2419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea24105f40 0x7fea241a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 --2- 172.21.15.39:0/1809524073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea24101310 0x7fea2419ba60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 >> 172.21.15.39:0/1809524073 conn(0x7fea24077000 msgr2=0x7fea24106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 shutdown_connections 2024-07-04T08:10:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:48.611+0000 7fea23200640 1 -- 172.21.15.39:0/1809524073 wait complete. 2024-07-04T08:10:48.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:48 smithi050 ceph-mon[36577]: pgmap v344: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:48.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:48 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1809524073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:48.919 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:48 smithi133 ceph-mon[37050]: pgmap v344: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:48.919 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:48 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1809524073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:48.998 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:48 smithi039 ceph-mon[32062]: pgmap v344: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:48.998 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:48 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1809524073' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:49.387 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:10:49.877 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:50.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:50 smithi050 ceph-mon[36577]: pgmap v345: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:50.913 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:50 smithi039 ceph-mon[32062]: pgmap v345: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:51.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:50 smithi133 ceph-mon[37050]: pgmap v345: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-07-04T08:10:51.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-07-04T08:10:51.908 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-07-04T08:10:51.908 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-07-04T08:10:51.908 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-07-04T08:10:51.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-07-04T08:10:51.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-07-04T08:10:51.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-07-04T08:10:51.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-07-04T08:10:51.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-07-04T08:10:51.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-07-04T08:10:51.909 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-07-04T08:10:52.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-07-04T08:10:52.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-07-04T08:10:52.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-07-04T08:10:52.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-07-04T08:10:52.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-07-04T08:10:52.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-07-04T08:10:52.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-07-04T08:10:52.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-07-04T08:10:52.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-07-04T08:10:52.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-07-04T08:10:52.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.268+0000 7fe730e00640 1 -- 172.21.15.39:0/3064099983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c109510 msgr2=0x7fe72c109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:52.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.268+0000 7fe730e00640 1 --2- 172.21.15.39:0/3064099983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c109510 0x7fe72c109920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009900 tx=0x7fe71c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.268+0000 7fe730e00640 1 -- 172.21.15.39:0/3064099983 shutdown_connections 2024-07-04T08:10:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.268+0000 7fe730e00640 1 --2- 172.21.15.39:0/3064099983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72c10f7a0 0x7fe72c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.268+0000 7fe730e00640 1 --2- 172.21.15.39:0/3064099983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c10ee50 0x7fe72c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.268+0000 7fe730e00640 1 --2- 172.21.15.39:0/3064099983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c109510 0x7fe72c109920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.268+0000 7fe730e00640 1 -- 172.21.15.39:0/3064099983 >> 172.21.15.39:0/3064099983 conn(0x7fe72c078b10 msgr2=0x7fe72c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.268+0000 7fe730e00640 1 -- 172.21.15.39:0/3064099983 shutdown_connections 2024-07-04T08:10:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.268+0000 7fe730e00640 1 -- 172.21.15.39:0/3064099983 wait complete. 2024-07-04T08:10:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.269+0000 7fe730e00640 1 Processor -- start 2024-07-04T08:10:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.269+0000 7fe730e00640 1 -- start start 2024-07-04T08:10:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe730e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c109510 0x7fe72c19ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe730e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c10ee50 0x7fe72c1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe730e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72c10f7a0 0x7fe72c1a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe730e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1194a0 con 0x7fe72c10ee50 2024-07-04T08:10:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe730e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c119320 con 0x7fe72c109510 2024-07-04T08:10:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe730e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c119620 con 0x7fe72c10f7a0 2024-07-04T08:10:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe723e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c10ee50 0x7fe72c1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe72b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c109510 0x7fe72c19ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe723e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c10ee50 0x7fe72c1a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53036/0 (socket says 172.21.15.39:53036) 2024-07-04T08:10:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe723e00640 1 -- 172.21.15.39:0/1868503081 learned_addr learned my addr 172.21.15.39:0/1868503081 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.270+0000 7fe72be00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72c10f7a0 0x7fe72c1a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe723e00640 1 -- 172.21.15.39:0/1868503081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72c10f7a0 msgr2=0x7fe72c1a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe723e00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72c10f7a0 0x7fe72c1a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe723e00640 1 -- 172.21.15.39:0/1868503081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c109510 msgr2=0x7fe72c19ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe723e00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c109510 0x7fe72c19ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe723e00640 1 -- 172.21.15.39:0/1868503081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c1a5e80 con 0x7fe72c10ee50 2024-07-04T08:10:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe72be00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72c10f7a0 0x7fe72c1a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:10:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe723e00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c10ee50 0x7fe72c1a04b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e970 tx=0x7fe71400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe728c00640 1 -- 172.21.15.39:0/1868503081 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71400cd80 con 0x7fe72c10ee50 2024-07-04T08:10:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c1a6170 con 0x7fe72c10ee50 2024-07-04T08:10:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.271+0000 7fe728c00640 1 -- 172.21.15.39:0/1868503081 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7140035f0 con 0x7fe72c10ee50 2024-07-04T08:10:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.272+0000 7fe728c00640 1 -- 172.21.15.39:0/1868503081 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714010690 con 0x7fe72c10ee50 2024-07-04T08:10:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.272+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c1ac8d0 con 0x7fe72c10ee50 2024-07-04T08:10:52.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.273+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72c109920 con 0x7fe72c10ee50 2024-07-04T08:10:52.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.277+0000 7fe728c00640 1 -- 172.21.15.39:0/1868503081 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fe714003aa0 con 0x7fe72c10ee50 2024-07-04T08:10:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.279+0000 7fe728c00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7100b28c0 0x7fe7100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.279+0000 7fe728c00640 1 -- 172.21.15.39:0/1868503081 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9307+0+0 (secure 0 0 0) 0x7fe7140e2a90 con 0x7fe72c10ee50 2024-07-04T08:10:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.279+0000 7fe72b400640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7100b28c0 0x7fe7100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.280+0000 7fe72b400640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7100b28c0 0x7fe7100b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005ab0 tx=0x7fe71c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.281+0000 7fe728c00640 1 -- 172.21.15.39:0/1868503081 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe7140aa3b0 con 0x7fe72c10ee50 2024-07-04T08:10:52.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.493+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe72c02de10 con 0x7fe72c10ee50 2024-07-04T08:10:52.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.495+0000 7fe728c00640 1 -- 172.21.15.39:0/1868503081 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24909 (secure 0 0 0) 0x7fe7140a9cb0 con 0x7fe72c10ee50 2024-07-04T08:10:52.497 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:10:52.498 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":89,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:10:51.631276+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:10:52.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7100b28c0 msgr2=0x7fe7100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:52.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7100b28c0 0x7fe7100b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005ab0 tx=0x7fe71c005a20 comp rx=0 tx=0).stop 2024-07-04T08:10:52.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c10ee50 msgr2=0x7fe72c1a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:52.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c10ee50 0x7fe72c1a04b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e970 tx=0x7fe71400ee30 comp rx=0 tx=0).stop 2024-07-04T08:10:52.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 shutdown_connections 2024-07-04T08:10:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7100b28c0 0x7fe7100b4dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe72c10ee50 0x7fe72c1a04b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe72c10f7a0 0x7fe72c1a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 --2- 172.21.15.39:0/1868503081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c109510 0x7fe72c19ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 >> 172.21.15.39:0/1868503081 conn(0x7fe72c078b10 msgr2=0x7fe72c0792e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 shutdown_connections 2024-07-04T08:10:52.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:52.499+0000 7fe730e00640 1 -- 172.21.15.39:0/1868503081 wait complete. 2024-07-04T08:10:52.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: pgmap v346: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:52.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-07-04T08:10:52.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-07-04T08:10:52.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-07-04T08:10:52.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-07-04T08:10:52.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-07-04T08:10:52.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-07-04T08:10:52.853 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-07-04T08:10:52.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-07-04T08:10:52.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-07-04T08:10:52.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-07-04T08:10:52.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: osdmap e89: 12 total, 12 up, 12 in 2024-07-04T08:10:52.854 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:52 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1868503081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:53.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: pgmap v346: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-07-04T08:10:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-07-04T08:10:53.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: osdmap e89: 12 total, 12 up, 12 in 2024-07-04T08:10:53.068 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:52 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1868503081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: pgmap v346: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-04T08:10:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-07-04T08:10:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-07-04T08:10:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-07-04T08:10:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-07-04T08:10:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-07-04T08:10:53.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-07-04T08:10:53.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-07-04T08:10:53.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-07-04T08:10:53.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-07-04T08:10:53.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-07-04T08:10:53.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: osdmap e89: 12 total, 12 up, 12 in 2024-07-04T08:10:53.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:52 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1868503081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:10:53.353 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-07-04T08:10:53.832 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:53.978 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:53 smithi039 ceph-mon[32062]: osdmap e90: 12 total, 12 up, 12 in 2024-07-04T08:10:54.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:53 smithi133 ceph-mon[37050]: osdmap e90: 12 total, 12 up, 12 in 2024-07-04T08:10:54.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:53 smithi050 ceph-mon[36577]: osdmap e90: 12 total, 12 up, 12 in 2024-07-04T08:10:55.030 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:54 smithi039 ceph-mon[32062]: pgmap v349: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:55.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:54 smithi133 ceph-mon[37050]: pgmap v349: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:55.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:54 smithi050 ceph-mon[36577]: pgmap v349: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:10:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.126+0000 7f6a6a600640 1 -- 172.21.15.39:0/236445562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641050f0 msgr2=0x7f6a64105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:56.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.126+0000 7f6a6a600640 1 --2- 172.21.15.39:0/236445562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641050f0 0x7f6a64105500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009900 tx=0x7f6a5802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.127+0000 7f6a6a600640 1 -- 172.21.15.39:0/236445562 shutdown_connections 2024-07-04T08:10:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.127+0000 7f6a6a600640 1 --2- 172.21.15.39:0/236445562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a6410b380 0x7f6a64111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.127+0000 7f6a6a600640 1 --2- 172.21.15.39:0/236445562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a6410aa30 0x7f6a6410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.127+0000 7f6a6a600640 1 --2- 172.21.15.39:0/236445562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a641050f0 0x7f6a64105500 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.127+0000 7f6a6a600640 1 -- 172.21.15.39:0/236445562 >> 172.21.15.39:0/236445562 conn(0x7f6a64100b60 msgr2=0x7f6a64102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.127+0000 7f6a6a600640 1 -- 172.21.15.39:0/236445562 shutdown_connections 2024-07-04T08:10:56.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.127+0000 7f6a6a600640 1 -- 172.21.15.39:0/236445562 wait complete. 2024-07-04T08:10:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.128+0000 7f6a6a600640 1 Processor -- start 2024-07-04T08:10:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.128+0000 7f6a6a600640 1 -- start start 2024-07-04T08:10:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.128+0000 7f6a6a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a641050f0 0x7f6a6419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a6a600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a6410aa30 0x7f6a6419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a6a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6410b380 0x7f6a641a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a6a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6a641150a0 con 0x7f6a6410b380 2024-07-04T08:10:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a6a600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64114f20 con 0x7f6a6410aa30 2024-07-04T08:10:56.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a6a600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64115220 con 0x7f6a641050f0 2024-07-04T08:10:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a63e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a6410aa30 0x7f6a6419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a69c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6410b380 0x7f6a641a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a63e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a6410aa30 0x7f6a6419c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56490/0 (socket says 172.21.15.39:56490) 2024-07-04T08:10:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a63e00640 1 -- 172.21.15.39:0/2793855579 learned_addr learned my addr 172.21.15.39:0/2793855579 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:10:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.129+0000 7f6a69200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a641050f0 0x7f6a6419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.130+0000 7f6a63e00640 1 -- 172.21.15.39:0/2793855579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a641050f0 msgr2=0x7f6a6419ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.130+0000 7f6a63e00640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a641050f0 0x7f6a6419ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.130+0000 7f6a63e00640 1 -- 172.21.15.39:0/2793855579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6410b380 msgr2=0x7f6a641a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.130+0000 7f6a63e00640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6410b380 0x7f6a641a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.130+0000 7f6a63e00640 1 -- 172.21.15.39:0/2793855579 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a641a1a00 con 0x7f6a6410aa30 2024-07-04T08:10:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.130+0000 7f6a63e00640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a6410aa30 0x7f6a6419c030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000e9c0 tx=0x7f6a5000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.130+0000 7f6a61600640 1 -- 172.21.15.39:0/2793855579 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000ccc0 con 0x7f6a6410aa30 2024-07-04T08:10:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.131+0000 7f6a61600640 1 -- 172.21.15.39:0/2793855579 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a500054d0 con 0x7f6a6410aa30 2024-07-04T08:10:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.131+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a641a1cf0 con 0x7f6a6410aa30 2024-07-04T08:10:56.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.131+0000 7f6a61600640 1 -- 172.21.15.39:0/2793855579 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50010430 con 0x7f6a6410aa30 2024-07-04T08:10:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.131+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a641a8450 con 0x7f6a6410aa30 2024-07-04T08:10:56.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.132+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a64105500 con 0x7f6a6410aa30 2024-07-04T08:10:56.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.135+0000 7f6a61600640 1 -- 172.21.15.39:0/2793855579 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f6a50005020 con 0x7f6a6410aa30 2024-07-04T08:10:56.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.137+0000 7f6a61600640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6a4c0b2840 0x7f6a4c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:10:56.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.137+0000 7f6a61600640 1 -- 172.21.15.39:0/2793855579 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f6a500e29f0 con 0x7f6a6410aa30 2024-07-04T08:10:56.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.137+0000 7f6a69200640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6a4c0b2840 0x7f6a4c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:10:56.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.138+0000 7f6a61600640 1 -- 172.21.15.39:0/2793855579 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f6a500aa280 con 0x7f6a6410aa30 2024-07-04T08:10:56.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.138+0000 7f6a69200640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6a4c0b2840 0x7f6a4c0b4d50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005ab0 tx=0x7f6a58005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:10:56.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.397+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6a6402de10 con 0x7f6a6410aa30 2024-07-04T08:10:56.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.703+0000 7f6a61600640 1 -- 172.21.15.39:0/2793855579 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f6a500a9b80 con 0x7f6a6410aa30 2024-07-04T08:10:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6a4c0b2840 msgr2=0x7f6a4c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6a4c0b2840 0x7f6a4c0b4d50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005ab0 tx=0x7f6a58005a20 comp rx=0 tx=0).stop 2024-07-04T08:10:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a6410aa30 msgr2=0x7f6a6419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:10:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a6410aa30 0x7f6a6419c030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000e9c0 tx=0x7f6a5000ee80 comp rx=0 tx=0).stop 2024-07-04T08:10:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 shutdown_connections 2024-07-04T08:10:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6a4c0b2840 0x7f6a4c0b4d50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6410b380 0x7f6a641a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6a6410aa30 0x7f6a6419c030 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 --2- 172.21.15.39:0/2793855579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a641050f0 0x7f6a6419ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:10:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.707+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 >> 172.21.15.39:0/2793855579 conn(0x7f6a64100b60 msgr2=0x7f6a64102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:10:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.708+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 shutdown_connections 2024-07-04T08:10:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:10:56.708+0000 7f6a6a600640 1 -- 172.21.15.39:0/2793855579 wait complete. 2024-07-04T08:10:57.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:56 smithi133 ceph-mon[37050]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:10:57.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:56 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2793855579' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-04T08:10:57.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:56 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-04T08:10:57.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:56 smithi039 ceph-mon[32062]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:10:57.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:56 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2793855579' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-04T08:10:57.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:56 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-04T08:10:57.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:56 smithi050 ceph-mon[36577]: pgmap v350: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:10:57.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:56 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2793855579' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-04T08:10:57.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:56 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-04T08:10:57.664 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-07-04T08:10:57.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-07-04T08:10:57.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:57 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-04T08:10:57.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:57 smithi039 ceph-mon[32062]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:10:58.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:57 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-04T08:10:58.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:57 smithi133 ceph-mon[37050]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:10:58.141 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:10:58.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:57 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-04T08:10:58.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:57 smithi050 ceph-mon[36577]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:10:59.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:10:58 smithi133 ceph-mon[37050]: pgmap v351: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:10:59.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:10:58 smithi039 ceph-mon[32062]: pgmap v351: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:10:59.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:10:58 smithi050 ceph-mon[36577]: pgmap v351: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:00.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.506+0000 7fb6fdc00640 1 -- 172.21.15.39:0/207111259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f810ac40 msgr2=0x7fb6f810b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:00.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.506+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/207111259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f810ac40 0x7fb6f810b050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009900 tx=0x7fb6e402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:11:00.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.507+0000 7fb6fdc00640 1 -- 172.21.15.39:0/207111259 shutdown_connections 2024-07-04T08:11:00.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.507+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/207111259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f810b590 0x7fb6f8111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:00.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.507+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/207111259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f810ac40 0x7fb6f810b050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:00.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.507+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/207111259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f8105300 0x7fb6f8105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:00.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.507+0000 7fb6fdc00640 1 -- 172.21.15.39:0/207111259 >> 172.21.15.39:0/207111259 conn(0x7fb6f8100d10 msgr2=0x7fb6f8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:00.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.507+0000 7fb6fdc00640 1 -- 172.21.15.39:0/207111259 shutdown_connections 2024-07-04T08:11:00.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.507+0000 7fb6fdc00640 1 -- 172.21.15.39:0/207111259 wait complete. 2024-07-04T08:11:00.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.508+0000 7fb6fdc00640 1 Processor -- start 2024-07-04T08:11:00.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.508+0000 7fb6fdc00640 1 -- start start 2024-07-04T08:11:00.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6fdc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f8105300 0x7fb6f819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:00.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6fdc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f810ac40 0x7fb6f819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:00.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6fdc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f810b590 0x7fb6f81a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:00.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6fdc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8115250 con 0x7fb6f810b590 2024-07-04T08:11:00.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6fdc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f81150d0 con 0x7fb6f8105300 2024-07-04T08:11:00.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6fdc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f81153d0 con 0x7fb6f810ac40 2024-07-04T08:11:00.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f8105300 0x7fb6f819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:00.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f8105300 0x7fb6f819bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56512/0 (socket says 172.21.15.39:56512) 2024-07-04T08:11:00.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6f6a00640 1 -- 172.21.15.39:0/1737072389 learned_addr learned my addr 172.21.15.39:0/1737072389 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:00.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6f6000640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f810ac40 0x7fb6f819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:00.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.509+0000 7fb6f7400640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f810b590 0x7fb6f81a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:00.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.510+0000 7fb6f6a00640 1 -- 172.21.15.39:0/1737072389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f810ac40 msgr2=0x7fb6f819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:00.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.510+0000 7fb6f6a00640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f810ac40 0x7fb6f819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:00.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.510+0000 7fb6f6a00640 1 -- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f810b590 msgr2=0x7fb6f81a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:00.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.510+0000 7fb6f6a00640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f810b590 0x7fb6f81a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:00.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.510+0000 7fb6f6a00640 1 -- 172.21.15.39:0/1737072389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f81a1bb0 con 0x7fb6f8105300 2024-07-04T08:11:00.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.510+0000 7fb6f6a00640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f8105300 0x7fb6f819bc40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9f0 tx=0x7fb6e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:00.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.511+0000 7fb6ef400640 1 -- 172.21.15.39:0/1737072389 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800cd80 con 0x7fb6f8105300 2024-07-04T08:11:00.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.511+0000 7fb6ef400640 1 -- 172.21.15.39:0/1737072389 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e80108e0 con 0x7fb6f8105300 2024-07-04T08:11:00.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.511+0000 7fb6ef400640 1 -- 172.21.15.39:0/1737072389 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8018980 con 0x7fb6f8105300 2024-07-04T08:11:00.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.511+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f81a1ea0 con 0x7fb6f8105300 2024-07-04T08:11:00.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.511+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f81a8600 con 0x7fb6f8105300 2024-07-04T08:11:00.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.512+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f8105710 con 0x7fb6f8105300 2024-07-04T08:11:00.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.515+0000 7fb6ef400640 1 -- 172.21.15.39:0/1737072389 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fb6e8010430 con 0x7fb6f8105300 2024-07-04T08:11:00.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.516+0000 7fb6ef400640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6dc0b28c0 0x7fb6dc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:00.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.516+0000 7fb6ef400640 1 -- 172.21.15.39:0/1737072389 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fb6e80e2e80 con 0x7fb6f8105300 2024-07-04T08:11:00.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.516+0000 7fb6f6000640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6dc0b28c0 0x7fb6dc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:00.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.517+0000 7fb6f6000640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6dc0b28c0 0x7fb6dc0b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb6e402a9e0 tx=0x7fb6e4005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:00.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.518+0000 7fb6ef400640 1 -- 172.21.15.39:0/1737072389 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb6e80aa0c0 con 0x7fb6f8105300 2024-07-04T08:11:00.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:00.782+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fb6f802de10 con 0x7fb6f8105300 2024-07-04T08:11:01.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:00 smithi133 ceph-mon[37050]: pgmap v352: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:00 smithi039 ceph-mon[32062]: pgmap v352: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:00 smithi050 ceph-mon[36577]: pgmap v352: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:01.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.752+0000 7fb6ef400640 1 -- 172.21.15.39:0/1737072389 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fb6e80ae3c0 con 0x7fb6f8105300 2024-07-04T08:11:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6dc0b28c0 msgr2=0x7fb6dc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6dc0b28c0 0x7fb6dc0b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb6e402a9e0 tx=0x7fb6e4005a20 comp rx=0 tx=0).stop 2024-07-04T08:11:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f8105300 msgr2=0x7fb6f819bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f8105300 0x7fb6f819bc40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9f0 tx=0x7fb6e800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:11:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 shutdown_connections 2024-07-04T08:11:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6dc0b28c0 0x7fb6dc0b4dd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f810b590 0x7fb6f81a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f810ac40 0x7fb6f819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 --2- 172.21.15.39:0/1737072389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6f8105300 0x7fb6f819bc40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 >> 172.21.15.39:0/1737072389 conn(0x7fb6f8100d10 msgr2=0x7fb6f81041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 shutdown_connections 2024-07-04T08:11:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:01.757+0000 7fb6fdc00640 1 -- 172.21.15.39:0/1737072389 wait complete. 2024-07-04T08:11:02.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:01 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1737072389' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-04T08:11:02.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:01 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-04T08:11:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:01 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1737072389' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-04T08:11:02.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:01 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-04T08:11:02.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:01 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1737072389' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-04T08:11:02.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:01 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-04T08:11:02.508 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-07-04T08:11:02.994 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:03.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:02 smithi133 ceph-mon[37050]: pgmap v353: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:03.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:02 smithi133 ceph-mon[37050]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-04T08:11:03.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:02 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-04T08:11:03.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:02 smithi133 ceph-mon[37050]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-04T08:11:03.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:02 smithi133 ceph-mon[37050]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:11:03.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:02 smithi133 ceph-mon[37050]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-07-04T08:11:03.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:02 smithi133 ceph-mon[37050]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-07-04T08:11:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:02 smithi039 ceph-mon[32062]: pgmap v353: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:02 smithi039 ceph-mon[32062]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-04T08:11:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:02 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-04T08:11:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:02 smithi039 ceph-mon[32062]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-04T08:11:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:02 smithi039 ceph-mon[32062]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:11:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:02 smithi039 ceph-mon[32062]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-07-04T08:11:03.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:02 smithi039 ceph-mon[32062]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-07-04T08:11:03.144 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:11:02 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b[68787]: 2024-07-04T08:11:02.793+0000 7f7b0cfab640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-07-04T08:11:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:02 smithi050 ceph-mon[36577]: pgmap v353: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:02 smithi050 ceph-mon[36577]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-04T08:11:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:02 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-04T08:11:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:02 smithi050 ceph-mon[36577]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-04T08:11:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:02 smithi050 ceph-mon[36577]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-04T08:11:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:02 smithi050 ceph-mon[36577]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-07-04T08:11:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:02 smithi050 ceph-mon[36577]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-07-04T08:11:04.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:03 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] up:active 2024-07-04T08:11:04.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:03 smithi133 ceph-mon[37050]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-04T08:11:04.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:03 smithi133 ceph-mon[37050]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-07-04T08:11:04.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:03 smithi133 ceph-mon[37050]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-07-04T08:11:04.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:03 smithi133 ceph-mon[37050]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-07-04T08:11:04.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:03 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] up:active 2024-07-04T08:11:04.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:03 smithi039 ceph-mon[32062]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-04T08:11:04.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:03 smithi039 ceph-mon[32062]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-07-04T08:11:04.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:03 smithi039 ceph-mon[32062]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-07-04T08:11:04.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:03 smithi039 ceph-mon[32062]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-07-04T08:11:04.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:03 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] up:active 2024-07-04T08:11:04.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:03 smithi050 ceph-mon[36577]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-04T08:11:04.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:03 smithi050 ceph-mon[36577]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-07-04T08:11:04.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:03 smithi050 ceph-mon[36577]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-07-04T08:11:04.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:03 smithi050 ceph-mon[36577]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-07-04T08:11:05.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:04 smithi039 ceph-mon[32062]: pgmap v354: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 188 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:05.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:04 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] up:active 2024-07-04T08:11:05.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:04 smithi039 ceph-mon[32062]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-04T08:11:05.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:04 smithi039 ceph-mon[32062]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-07-04T08:11:05.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:04 smithi039 ceph-mon[32062]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-07-04T08:11:05.031 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:04 smithi039 ceph-mon[32062]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-07-04T08:11:05.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:04 smithi133 ceph-mon[37050]: pgmap v354: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 188 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:04 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] up:active 2024-07-04T08:11:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:04 smithi133 ceph-mon[37050]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-04T08:11:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:04 smithi133 ceph-mon[37050]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-07-04T08:11:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:04 smithi133 ceph-mon[37050]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-07-04T08:11:05.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:04 smithi133 ceph-mon[37050]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-07-04T08:11:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:04 smithi050 ceph-mon[36577]: pgmap v354: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 188 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:04 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] up:active 2024-07-04T08:11:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:04 smithi050 ceph-mon[36577]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-04T08:11:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:04 smithi050 ceph-mon[36577]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-07-04T08:11:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:04 smithi050 ceph-mon[36577]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-07-04T08:11:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:04 smithi050 ceph-mon[36577]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-07-04T08:11:05.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.252+0000 7f6ffe000640 1 -- 172.21.15.39:0/522139818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff8109340 msgr2=0x7f6ff810fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.252+0000 7f6ffe000640 1 --2- 172.21.15.39:0/522139818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff8109340 0x7f6ff810fb70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009900 tx=0x7f6ff002a400 comp rx=0 tx=0).stop 2024-07-04T08:11:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.252+0000 7f6ffe000640 1 -- 172.21.15.39:0/522139818 shutdown_connections 2024-07-04T08:11:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.252+0000 7f6ffe000640 1 --2- 172.21.15.39:0/522139818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff8109340 0x7f6ff810fb70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.252+0000 7f6ffe000640 1 --2- 172.21.15.39:0/522139818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff81089f0 0x7f6ff8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.252+0000 7f6ffe000640 1 --2- 172.21.15.39:0/522139818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff81030b0 0x7f6ff81034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.252+0000 7f6ffe000640 1 -- 172.21.15.39:0/522139818 >> 172.21.15.39:0/522139818 conn(0x7f6ff807ada0 msgr2=0x7f6ff807b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.252+0000 7f6ffe000640 1 -- 172.21.15.39:0/522139818 shutdown_connections 2024-07-04T08:11:05.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.252+0000 7f6ffe000640 1 -- 172.21.15.39:0/522139818 wait complete. 2024-07-04T08:11:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.253+0000 7f6ffe000640 1 Processor -- start 2024-07-04T08:11:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.254+0000 7f6ffe000640 1 -- start start 2024-07-04T08:11:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.254+0000 7f6ffe000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff81030b0 0x7f6ff81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.254+0000 7f6ffe000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff81089f0 0x7f6ff8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.254+0000 7f6ffe000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff8109340 0x7f6ff819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:05.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.254+0000 7f6ffe000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8112ff0 con 0x7f6ff81089f0 2024-07-04T08:11:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.254+0000 7f6ffe000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8112e70 con 0x7f6ff81030b0 2024-07-04T08:11:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.254+0000 7f6ffe000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8113170 con 0x7f6ff8109340 2024-07-04T08:11:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff81089f0 0x7f6ff8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff8109340 0x7f6ff819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:05.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff8109340 0x7f6ff819f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53254/0 (socket says 172.21.15.39:53254) 2024-07-04T08:11:05.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff81030b0 0x7f6ff81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:05.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff7e00640 1 -- 172.21.15.39:0/3458088872 learned_addr learned my addr 172.21.15.39:0/3458088872 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:05.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff7e00640 1 -- 172.21.15.39:0/3458088872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff81030b0 msgr2=0x7f6ff81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:05.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff7e00640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff81030b0 0x7f6ff81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:05.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff7e00640 1 -- 172.21.15.39:0/3458088872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff81089f0 msgr2=0x7f6ff8199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:05.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff7e00640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff81089f0 0x7f6ff8199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:05.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.255+0000 7f6ff7e00640 1 -- 172.21.15.39:0/3458088872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff819f910 con 0x7f6ff8109340 2024-07-04T08:11:05.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.256+0000 7f6ff7e00640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff8109340 0x7f6ff819f1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0004770 tx=0x7f6ff0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:05.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.256+0000 7f6fefe00640 1 -- 172.21.15.39:0/3458088872 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff003f030 con 0x7f6ff8109340 2024-07-04T08:11:05.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.256+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff819fba0 con 0x7f6ff8109340 2024-07-04T08:11:05.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.256+0000 7f6fefe00640 1 -- 172.21.15.39:0/3458088872 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff0004440 con 0x7f6ff8109340 2024-07-04T08:11:05.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.256+0000 7f6fefe00640 1 -- 172.21.15.39:0/3458088872 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0035070 con 0x7f6ff8109340 2024-07-04T08:11:05.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.256+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff81a6360 con 0x7f6ff8109340 2024-07-04T08:11:05.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.257+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff8104330 con 0x7f6ff8109340 2024-07-04T08:11:05.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.260+0000 7f6fefe00640 1 -- 172.21.15.39:0/3458088872 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f6ff0042030 con 0x7f6ff8109340 2024-07-04T08:11:05.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.262+0000 7f6fefe00640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fdc0b2840 0x7f6fdc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:05.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.262+0000 7f6fefe00640 1 -- 172.21.15.39:0/3458088872 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f6ff0102ca0 con 0x7f6ff8109340 2024-07-04T08:11:05.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.262+0000 7f6ff7400640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fdc0b2840 0x7f6fdc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:05.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.263+0000 7f6ff7400640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fdc0b2840 0x7f6fdc0b4d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8005fd0 tx=0x7f6fe8009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:05.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.264+0000 7f6fefe00640 1 -- 172.21.15.39:0/3458088872 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f6ff00c9ee0 con 0x7f6ff8109340 2024-07-04T08:11:05.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.519+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6ff810b190 con 0x7f6ff8109340 2024-07-04T08:11:05.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.853+0000 7f6fefe00640 1 -- 172.21.15.39:0/3458088872 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f6ff00ce1e0 con 0x7f6ff8109340 2024-07-04T08:11:05.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.857+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fdc0b2840 msgr2=0x7f6fdc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:05.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.857+0000 7f6ffe000640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fdc0b2840 0x7f6fdc0b4d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8005fd0 tx=0x7f6fe8009500 comp rx=0 tx=0).stop 2024-07-04T08:11:05.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.858+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff8109340 msgr2=0x7f6ff819f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:05.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.858+0000 7f6ffe000640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff8109340 0x7f6ff819f1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0004770 tx=0x7f6ff0004040 comp rx=0 tx=0).stop 2024-07-04T08:11:05.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.858+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 shutdown_connections 2024-07-04T08:11:05.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.858+0000 7f6ffe000640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fdc0b2840 0x7f6fdc0b4d50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:05.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.858+0000 7f6ffe000640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff81089f0 0x7f6ff8199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:05.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.858+0000 7f6ffe000640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff8109340 0x7f6ff819f1e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:05.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.858+0000 7f6ffe000640 1 --2- 172.21.15.39:0/3458088872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff81030b0 0x7f6ff81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:05.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.858+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 >> 172.21.15.39:0/3458088872 conn(0x7f6ff807ada0 msgr2=0x7f6ff8106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:05.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.858+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 shutdown_connections 2024-07-04T08:11:05.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:05.859+0000 7f6ffe000640 1 -- 172.21.15.39:0/3458088872 wait complete. 2024-07-04T08:11:06.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:05 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] up:active 2024-07-04T08:11:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:05 smithi039 ceph-mon[32062]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-04T08:11:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:05 smithi039 ceph-mon[32062]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-04T08:11:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:05 smithi039 ceph-mon[32062]: Cluster is now healthy 2024-07-04T08:11:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:05 smithi039 ceph-mon[32062]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-07-04T08:11:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:05 smithi039 ceph-mon[32062]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-04T08:11:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:05 smithi039 ceph-mon[32062]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-07-04T08:11:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:05 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3458088872' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-04T08:11:06.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:05 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-04T08:11:06.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:05 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] up:active 2024-07-04T08:11:06.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:05 smithi050 ceph-mon[36577]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-04T08:11:06.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:05 smithi050 ceph-mon[36577]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-04T08:11:06.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:05 smithi050 ceph-mon[36577]: Cluster is now healthy 2024-07-04T08:11:06.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:05 smithi050 ceph-mon[36577]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-07-04T08:11:06.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:05 smithi050 ceph-mon[36577]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-04T08:11:06.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:05 smithi050 ceph-mon[36577]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-07-04T08:11:06.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:05 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3458088872' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-04T08:11:06.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:05 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-04T08:11:06.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:05 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] up:active 2024-07-04T08:11:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:05 smithi133 ceph-mon[37050]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-04T08:11:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:05 smithi133 ceph-mon[37050]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-04T08:11:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:05 smithi133 ceph-mon[37050]: Cluster is now healthy 2024-07-04T08:11:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:05 smithi133 ceph-mon[37050]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-07-04T08:11:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:05 smithi133 ceph-mon[37050]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-04T08:11:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:05 smithi133 ceph-mon[37050]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-07-04T08:11:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:05 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3458088872' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-04T08:11:06.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:05 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-04T08:11:06.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-07-04T08:11:07.120 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:07.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:06 smithi050 ceph-mon[36577]: pgmap v355: 129 pgs: 129 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:07.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:06 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-04T08:11:07.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:06 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] up:active 2024-07-04T08:11:07.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:06 smithi050 ceph-mon[36577]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-04T08:11:07.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:06 smithi050 ceph-mon[36577]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-04T08:11:07.271 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:06 smithi039 ceph-mon[32062]: pgmap v355: 129 pgs: 129 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:07.271 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:06 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-04T08:11:07.271 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:06 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] up:active 2024-07-04T08:11:07.272 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:06 smithi039 ceph-mon[32062]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-04T08:11:07.272 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:06 smithi039 ceph-mon[32062]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-04T08:11:07.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:06 smithi133 ceph-mon[37050]: pgmap v355: 129 pgs: 129 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:07.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:06 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-04T08:11:07.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:06 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] up:active 2024-07-04T08:11:07.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:06 smithi133 ceph-mon[37050]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-04T08:11:07.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:06 smithi133 ceph-mon[37050]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-04T08:11:09.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:08 smithi039 ceph-mon[32062]: pgmap v356: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.3 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:09.118 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:08 smithi133 ceph-mon[37050]: pgmap v356: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.3 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:08 smithi050 ceph-mon[36577]: pgmap v356: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.3 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2024-07-04T08:11:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.396+0000 7f5c49000640 1 -- 172.21.15.39:0/2889199031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c440777e0 msgr2=0x7f5c44113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.396+0000 7f5c49000640 1 --2- 172.21.15.39:0/2889199031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c440777e0 0x7f5c44113780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009900 tx=0x7f5c2c02a410 comp rx=0 tx=0).stop 2024-07-04T08:11:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.396+0000 7f5c49000640 1 -- 172.21.15.39:0/2889199031 shutdown_connections 2024-07-04T08:11:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.396+0000 7f5c49000640 1 --2- 172.21.15.39:0/2889199031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c44113cc0 0x7f5c441160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.396+0000 7f5c49000640 1 --2- 172.21.15.39:0/2889199031 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c440777e0 0x7f5c44113780 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.396+0000 7f5c49000640 1 --2- 172.21.15.39:0/2889199031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c44076e90 0x7f5c440772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.396+0000 7f5c49000640 1 -- 172.21.15.39:0/2889199031 >> 172.21.15.39:0/2889199031 conn(0x7f5c44100d10 msgr2=0x7f5c44103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:09.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.396+0000 7f5c49000640 1 -- 172.21.15.39:0/2889199031 shutdown_connections 2024-07-04T08:11:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.396+0000 7f5c49000640 1 -- 172.21.15.39:0/2889199031 wait complete. 2024-07-04T08:11:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.398+0000 7f5c49000640 1 Processor -- start 2024-07-04T08:11:09.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.398+0000 7f5c49000640 1 -- start start 2024-07-04T08:11:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.398+0000 7f5c49000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c44076e90 0x7f5c441a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.398+0000 7f5c49000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c440777e0 0x7f5c441a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.398+0000 7f5c49000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c44113cc0 0x7f5c441a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.398+0000 7f5c49000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5c44119610 con 0x7f5c44113cc0 2024-07-04T08:11:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.398+0000 7f5c49000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5c44119490 con 0x7f5c44076e90 2024-07-04T08:11:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.398+0000 7f5c49000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5c44119790 con 0x7f5c440777e0 2024-07-04T08:11:09.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c39400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c440777e0 0x7f5c441a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:09.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c42800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c44113cc0 0x7f5c441a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:09.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c39400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c440777e0 0x7f5c441a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53272/0 (socket says 172.21.15.39:53272) 2024-07-04T08:11:09.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c39400640 1 -- 172.21.15.39:0/843261530 learned_addr learned my addr 172.21.15.39:0/843261530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:09.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c41e00640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c44076e90 0x7f5c441a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c42800640 1 -- 172.21.15.39:0/843261530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c440777e0 msgr2=0x7f5c441a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c42800640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c440777e0 0x7f5c441a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c42800640 1 -- 172.21.15.39:0/843261530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c44076e90 msgr2=0x7f5c441a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c42800640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c44076e90 0x7f5c441a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c42800640 1 -- 172.21.15.39:0/843261530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c441a5f70 con 0x7f5c44113cc0 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c41e00640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c44076e90 0x7f5c441a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.399+0000 7f5c39400640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c440777e0 0x7f5c441a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.400+0000 7f5c42800640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c44113cc0 0x7f5c441a5840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e970 tx=0x7f5c3c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.400+0000 7f5c3b400640 1 -- 172.21.15.39:0/843261530 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00ccc0 con 0x7f5c44113cc0 2024-07-04T08:11:09.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.400+0000 7f5c3b400640 1 -- 172.21.15.39:0/843261530 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c005020 con 0x7f5c44113cc0 2024-07-04T08:11:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.400+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c441a6260 con 0x7f5c44113cc0 2024-07-04T08:11:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.400+0000 7f5c3b400640 1 -- 172.21.15.39:0/843261530 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c004a00 con 0x7f5c44113cc0 2024-07-04T08:11:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.400+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c441ac9c0 con 0x7f5c44113cc0 2024-07-04T08:11:09.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.402+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c440772a0 con 0x7f5c44113cc0 2024-07-04T08:11:09.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.406+0000 7f5c3b400640 1 -- 172.21.15.39:0/843261530 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f5c3c004ba0 con 0x7f5c44113cc0 2024-07-04T08:11:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.407+0000 7f5c3b400640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5c280b2840 0x7f5c280b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.408+0000 7f5c3b400640 1 -- 172.21.15.39:0/843261530 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f5c3c0e26d0 con 0x7f5c44113cc0 2024-07-04T08:11:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.408+0000 7f5c41e00640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5c280b2840 0x7f5c280b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:09.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.409+0000 7f5c41e00640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5c280b2840 0x7f5c280b4d50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c34005fd0 tx=0x7f5c34009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:09.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.409+0000 7f5c3b400640 1 -- 172.21.15.39:0/843261530 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5c3c0a9ff0 con 0x7f5c44113cc0 2024-07-04T08:11:09.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.666+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5c4402de10 con 0x7f5c44113cc0 2024-07-04T08:11:09.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.880+0000 7f5c3b400640 1 -- 172.21.15.39:0/843261530 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f5c3c0a98f0 con 0x7f5c44113cc0 2024-07-04T08:11:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5c280b2840 msgr2=0x7f5c280b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5c280b2840 0x7f5c280b4d50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c34005fd0 tx=0x7f5c34009500 comp rx=0 tx=0).stop 2024-07-04T08:11:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c44113cc0 msgr2=0x7f5c441a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:09.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c44113cc0 0x7f5c441a5840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e970 tx=0x7f5c3c00ee30 comp rx=0 tx=0).stop 2024-07-04T08:11:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 shutdown_connections 2024-07-04T08:11:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5c280b2840 0x7f5c280b4d50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c44113cc0 0x7f5c441a5840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c440777e0 0x7f5c441a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 --2- 172.21.15.39:0/843261530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5c44076e90 0x7f5c441a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.885+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 >> 172.21.15.39:0/843261530 conn(0x7f5c44100d10 msgr2=0x7f5c44103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.886+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 shutdown_connections 2024-07-04T08:11:09.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:09.886+0000 7f5c49000640 1 -- 172.21.15.39:0/843261530 wait complete. 2024-07-04T08:11:10.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:09 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/843261530' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-04T08:11:10.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:09 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/843261530' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-04T08:11:10.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:09 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/843261530' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-04T08:11:10.830 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-07-04T08:11:10.831 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-07-04T08:11:11.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:10 smithi039 ceph-mon[32062]: pgmap v357: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-07-04T08:11:11.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:10 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/843261530' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-04T08:11:11.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:10 smithi039 ceph-mon[32062]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-04T08:11:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:10 smithi050 ceph-mon[36577]: pgmap v357: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-07-04T08:11:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:10 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/843261530' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-04T08:11:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:10 smithi050 ceph-mon[36577]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-04T08:11:11.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:10 smithi133 ceph-mon[37050]: pgmap v357: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-07-04T08:11:11.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:10 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/843261530' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-04T08:11:11.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:10 smithi133 ceph-mon[37050]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-04T08:11:11.319 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:13.052 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:12 smithi039 ceph-mon[32062]: pgmap v358: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-07-04T08:11:13.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:12 smithi133 ceph-mon[37050]: pgmap v358: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-07-04T08:11:13.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:12 smithi050 ceph-mon[36577]: pgmap v358: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-07-04T08:11:13.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.667+0000 7f571e800640 1 -- 172.21.15.39:0/153747614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571810b590 msgr2=0x7f5718111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:13.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.667+0000 7f571e800640 1 --2- 172.21.15.39:0/153747614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571810b590 0x7f5718111dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5710009ca0 tx=0x7f571002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.667+0000 7f571e800640 1 -- 172.21.15.39:0/153747614 shutdown_connections 2024-07-04T08:11:13.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.667+0000 7f571e800640 1 --2- 172.21.15.39:0/153747614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f571810b590 0x7f5718111dc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.667+0000 7f571e800640 1 --2- 172.21.15.39:0/153747614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f571810ac40 0x7f571810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.667+0000 7f571e800640 1 --2- 172.21.15.39:0/153747614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5718105300 0x7f5718105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.667+0000 7f571e800640 1 -- 172.21.15.39:0/153747614 >> 172.21.15.39:0/153747614 conn(0x7f5718100d10 msgr2=0x7f5718103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:13.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.668+0000 7f571e800640 1 -- 172.21.15.39:0/153747614 shutdown_connections 2024-07-04T08:11:13.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.668+0000 7f571e800640 1 -- 172.21.15.39:0/153747614 wait complete. 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.669+0000 7f571e800640 1 Processor -- start 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.669+0000 7f571e800640 1 -- start start 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.669+0000 7f571e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5718105300 0x7f571819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.669+0000 7f571e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f571810ac40 0x7f571819c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.669+0000 7f571e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f571810b590 0x7f57181a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.669+0000 7f571e800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5718115260 con 0x7f5718105300 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.669+0000 7f571e800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57181150e0 con 0x7f571810b590 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.669+0000 7f571e800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57181153e0 con 0x7f571810ac40 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5716a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f571810ac40 0x7f571819c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5717400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5718105300 0x7f571819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:13.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5717e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f571810b590 0x7f57181a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5716a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f571810ac40 0x7f571819c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36354/0 (socket says 172.21.15.39:36354) 2024-07-04T08:11:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5716a00640 1 -- 172.21.15.39:0/1315848503 learned_addr learned my addr 172.21.15.39:0/1315848503 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:13.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5717400640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5718105300 0x7f571819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55738/0 (socket says 172.21.15.39:55738) 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5716a00640 1 -- 172.21.15.39:0/1315848503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f571810b590 msgr2=0x7f57181a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5716a00640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f571810b590 0x7f57181a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5716a00640 1 -- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5718105300 msgr2=0x7f571819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5716a00640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5718105300 0x7f571819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.670+0000 7f5716a00640 1 -- 172.21.15.39:0/1315848503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57181a1b40 con 0x7f571810ac40 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.671+0000 7f5716a00640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f571810ac40 0x7f571819c170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f570400e9c0 tx=0x7f570400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.671+0000 7f570fe00640 1 -- 172.21.15.39:0/1315848503 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570400ccc0 con 0x7f571810ac40 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.671+0000 7f570fe00640 1 -- 172.21.15.39:0/1315848503 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57040054d0 con 0x7f571810ac40 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.671+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57181a1e30 con 0x7f571810ac40 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.671+0000 7f570fe00640 1 -- 172.21.15.39:0/1315848503 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704010430 con 0x7f571810ac40 2024-07-04T08:11:13.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.671+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57181a8590 con 0x7f571810ac40 2024-07-04T08:11:13.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.672+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5718105710 con 0x7f571810ac40 2024-07-04T08:11:13.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.676+0000 7f570fe00640 1 -- 172.21.15.39:0/1315848503 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f5704005000 con 0x7f571810ac40 2024-07-04T08:11:13.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.677+0000 7f570fe00640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56fc0b28c0 0x7f56fc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:13.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.678+0000 7f570fe00640 1 -- 172.21.15.39:0/1315848503 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f57040e2ba0 con 0x7f571810ac40 2024-07-04T08:11:13.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.678+0000 7f5717400640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56fc0b28c0 0x7f56fc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:13.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.678+0000 7f570fe00640 1 -- 172.21.15.39:0/1315848503 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5704055110 con 0x7f571810ac40 2024-07-04T08:11:13.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.679+0000 7f5717400640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56fc0b28c0 0x7f56fc0b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57080095f0 tx=0x7f5708005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:13.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.925+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5718062d50 con 0x7f56fc0b28c0 2024-07-04T08:11:13.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.972+0000 7f570fe00640 1 -- 172.21.15.39:0/1315848503 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5718062d50 con 0x7f56fc0b28c0 2024-07-04T08:11:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56fc0b28c0 msgr2=0x7f56fc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56fc0b28c0 0x7f56fc0b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57080095f0 tx=0x7f5708005f90 comp rx=0 tx=0).stop 2024-07-04T08:11:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f571810ac40 msgr2=0x7f571819c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f571810ac40 0x7f571819c170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f570400e9c0 tx=0x7f570400ee80 comp rx=0 tx=0).stop 2024-07-04T08:11:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 shutdown_connections 2024-07-04T08:11:13.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56fc0b28c0 0x7f56fc0b4dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f571810b590 0x7f57181a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f571810ac40 0x7f571819c170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 --2- 172.21.15.39:0/1315848503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5718105300 0x7f571819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 >> 172.21.15.39:0/1315848503 conn(0x7f5718100d10 msgr2=0x7f571810a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 shutdown_connections 2024-07-04T08:11:13.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:13.975+0000 7f571e800640 1 -- 172.21.15.39:0/1315848503 wait complete. 2024-07-04T08:11:14.352 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:11:13 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:11:13.937+0000 7fdfac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:11:14.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:11:13 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:11:13.937+0000 7fdfac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:11:14.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:11:13 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:11:13.937+0000 7fdfac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:11:14.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:11:13 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:11:13.937+0000 7fdfac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:11:14.353 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:11:13 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:11:13.937+0000 7fdfac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:11:14.820 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-07-04T08:11:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:14 smithi039 ceph-mon[32062]: pgmap v359: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-07-04T08:11:15.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:14 smithi039 ceph-mon[32062]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:15.296 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:15.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:14 smithi133 ceph-mon[37050]: pgmap v359: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-07-04T08:11:15.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:14 smithi133 ceph-mon[37050]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:15.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:14 smithi050 ceph-mon[36577]: pgmap v359: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-07-04T08:11:15.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:14 smithi050 ceph-mon[36577]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:17.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:16 smithi039 ceph-mon[32062]: pgmap v360: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.9 KiB/s wr, 21 op/s 2024-07-04T08:11:17.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:16 smithi039 ceph-mon[32062]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-07-04T08:11:17.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:16 smithi133 ceph-mon[37050]: pgmap v360: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.9 KiB/s wr, 21 op/s 2024-07-04T08:11:17.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:16 smithi133 ceph-mon[37050]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-07-04T08:11:17.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:16 smithi050 ceph-mon[36577]: pgmap v360: 129 pgs: 129 active+clean; 587 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.9 KiB/s wr, 21 op/s 2024-07-04T08:11:17.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:16 smithi050 ceph-mon[36577]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-07-04T08:11:17.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.716+0000 7fe026600640 1 -- 172.21.15.39:0/3856234693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 msgr2=0x7fe020109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.716+0000 7fe026600640 1 --2- 172.21.15.39:0/3856234693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 0x7fe020109920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe014009900 tx=0x7fe01402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:11:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.716+0000 7fe026600640 1 -- 172.21.15.39:0/3856234693 shutdown_connections 2024-07-04T08:11:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.716+0000 7fe026600640 1 --2- 172.21.15.39:0/3856234693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe02010ee50 0x7fe02010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.716+0000 7fe026600640 1 --2- 172.21.15.39:0/3856234693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe02010f7a0 0x7fe020115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.716+0000 7fe026600640 1 --2- 172.21.15.39:0/3856234693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 0x7fe020109920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.716+0000 7fe026600640 1 -- 172.21.15.39:0/3856234693 >> 172.21.15.39:0/3856234693 conn(0x7fe020078b10 msgr2=0x7fe020107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.716+0000 7fe026600640 1 -- 172.21.15.39:0/3856234693 shutdown_connections 2024-07-04T08:11:17.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.716+0000 7fe026600640 1 -- 172.21.15.39:0/3856234693 wait complete. 2024-07-04T08:11:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.717+0000 7fe026600640 1 Processor -- start 2024-07-04T08:11:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.717+0000 7fe026600640 1 -- start start 2024-07-04T08:11:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe026600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 0x7fe02019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe026600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe02010ee50 0x7fe0201a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe026600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe02010f7a0 0x7fe0201a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:17.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe026600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe0201194a0 con 0x7fe02010ee50 2024-07-04T08:11:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe026600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe020119320 con 0x7fe02010f7a0 2024-07-04T08:11:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe026600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe020119620 con 0x7fe020109510 2024-07-04T08:11:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe025200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 0x7fe02019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe01fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe02010ee50 0x7fe0201a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe025200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 0x7fe02019ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36382/0 (socket says 172.21.15.39:36382) 2024-07-04T08:11:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe025200640 1 -- 172.21.15.39:0/1456635352 learned_addr learned my addr 172.21.15.39:0/1456635352 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:17.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.718+0000 7fe025c00640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe02010f7a0 0x7fe0201a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.719+0000 7fe01fe00640 1 -- 172.21.15.39:0/1456635352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 msgr2=0x7fe02019ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.719+0000 7fe01fe00640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 0x7fe02019ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.719+0000 7fe01fe00640 1 -- 172.21.15.39:0/1456635352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe02010f7a0 msgr2=0x7fe0201a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.719+0000 7fe01fe00640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe02010f7a0 0x7fe0201a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.719+0000 7fe01fe00640 1 -- 172.21.15.39:0/1456635352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0201a5e80 con 0x7fe02010ee50 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.719+0000 7fe025200640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 0x7fe02019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.719+0000 7fe01fe00640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe02010ee50 0x7fe0201a04b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe02006bd90 tx=0x7fe01000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.720+0000 7fe01d600640 1 -- 172.21.15.39:0/1456635352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01000cc40 con 0x7fe02010ee50 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.720+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0201a6170 con 0x7fe02010ee50 2024-07-04T08:11:17.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.720+0000 7fe01d600640 1 -- 172.21.15.39:0/1456635352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe01000cde0 con 0x7fe02010ee50 2024-07-04T08:11:17.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.720+0000 7fe01d600640 1 -- 172.21.15.39:0/1456635352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010018760 con 0x7fe02010ee50 2024-07-04T08:11:17.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.720+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0201ac8d0 con 0x7fe02010ee50 2024-07-04T08:11:17.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.721+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe020109920 con 0x7fe02010ee50 2024-07-04T08:11:17.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.724+0000 7fe01d600640 1 -- 172.21.15.39:0/1456635352 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7fe010010a90 con 0x7fe02010ee50 2024-07-04T08:11:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.725+0000 7fe01d600640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0080b2950 0x7fe0080b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:17.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.725+0000 7fe01d600640 1 -- 172.21.15.39:0/1456635352 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fe0100e2a70 con 0x7fe02010ee50 2024-07-04T08:11:17.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.725+0000 7fe025200640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0080b2950 0x7fe0080b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:17.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.726+0000 7fe025200640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0080b2950 0x7fe0080b4e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe0140096c0 tx=0x7fe014005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:17.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.729+0000 7fe01d600640 1 -- 172.21.15.39:0/1456635352 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe0100a9bb0 con 0x7fe02010ee50 2024-07-04T08:11:17.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:17.980+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe020062d50 con 0x7fe0080b2950 2024-07-04T08:11:18.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.015+0000 7fe01d600640 1 -- 172.21.15.39:0/1456635352 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe020062d50 con 0x7fe0080b2950 2024-07-04T08:11:18.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0080b2950 msgr2=0x7fe0080b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:18.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0080b2950 0x7fe0080b4e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe0140096c0 tx=0x7fe014005a20 comp rx=0 tx=0).stop 2024-07-04T08:11:18.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe02010ee50 msgr2=0x7fe0201a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:18.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe02010ee50 0x7fe0201a04b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe02006bd90 tx=0x7fe01000ee50 comp rx=0 tx=0).stop 2024-07-04T08:11:18.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 shutdown_connections 2024-07-04T08:11:18.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0080b2950 0x7fe0080b4e60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:18.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe02010ee50 0x7fe0201a04b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:18.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe02010f7a0 0x7fe0201a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:18.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 --2- 172.21.15.39:0/1456635352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe020109510 0x7fe02019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:18.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 >> 172.21.15.39:0/1456635352 conn(0x7fe020078b10 msgr2=0x7fe0200794e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:18.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 shutdown_connections 2024-07-04T08:11:18.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:18.018+0000 7fe026600640 1 -- 172.21.15.39:0/1456635352 wait complete. 2024-07-04T08:11:18.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-07-04T08:11:19.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:18 smithi039 ceph-mon[32062]: pgmap v361: 129 pgs: 129 active+clean; 594 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-07-04T08:11:19.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:18 smithi039 ceph-mon[32062]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:19.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:18 smithi050 ceph-mon[36577]: pgmap v361: 129 pgs: 129 active+clean; 594 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-07-04T08:11:19.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:18 smithi050 ceph-mon[36577]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:19.277 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:19.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:18 smithi133 ceph-mon[37050]: pgmap v361: 129 pgs: 129 active+clean; 594 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 KiB/s wr, 18 op/s 2024-07-04T08:11:19.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:18 smithi133 ceph-mon[37050]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:21.266 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:20 smithi039 ceph-mon[32062]: pgmap v362: 129 pgs: 129 active+clean; 594 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-07-04T08:11:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:20 smithi133 ceph-mon[37050]: pgmap v362: 129 pgs: 129 active+clean; 594 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-07-04T08:11:21.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:20 smithi050 ceph-mon[36577]: pgmap v362: 129 pgs: 129 active+clean; 594 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-07-04T08:11:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.486+0000 7f7987200640 1 -- 172.21.15.39:0/3542221040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f798810f7a0 msgr2=0x7f7988115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.486+0000 7f7987200640 1 --2- 172.21.15.39:0/3542221040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f798810f7a0 0x7f7988115fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7980009ca0 tx=0x7f798002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.486+0000 7f7987200640 1 -- 172.21.15.39:0/3542221040 shutdown_connections 2024-07-04T08:11:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.486+0000 7f7987200640 1 --2- 172.21.15.39:0/3542221040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f798810f7a0 0x7f7988115fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.486+0000 7f7987200640 1 --2- 172.21.15.39:0/3542221040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798810ee50 0x7f798810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.486+0000 7f7987200640 1 --2- 172.21.15.39:0/3542221040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7988109510 0x7f7988109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.486+0000 7f7987200640 1 -- 172.21.15.39:0/3542221040 >> 172.21.15.39:0/3542221040 conn(0x7f7988078b10 msgr2=0x7f7988107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.486+0000 7f7987200640 1 -- 172.21.15.39:0/3542221040 shutdown_connections 2024-07-04T08:11:21.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.486+0000 7f7987200640 1 -- 172.21.15.39:0/3542221040 wait complete. 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.487+0000 7f7987200640 1 Processor -- start 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.488+0000 7f7987200640 1 -- start start 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.488+0000 7f7987200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7988109510 0x7f798819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.488+0000 7f7987200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798810ee50 0x7f79881a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.488+0000 7f7987200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f798810f7a0 0x7f79881a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.488+0000 7f7987200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7988119450 con 0x7f7988109510 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.488+0000 7f7987200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f79881192d0 con 0x7f798810ee50 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.488+0000 7f7987200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79881195d0 con 0x7f798810f7a0 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7988109510 0x7f798819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798810ee50 0x7f79881a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:21.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7988109510 0x7f798819fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41470/0 (socket says 172.21.15.39:41470) 2024-07-04T08:11:21.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798810ee50 0x7f79881a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:36402/0 (socket says 172.21.15.39:36402) 2024-07-04T08:11:21.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985e00640 1 -- 172.21.15.39:0/365894931 learned_addr learned my addr 172.21.15.39:0/365894931 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:21.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7986800640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f798810f7a0 0x7f79881a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:21.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985400640 1 -- 172.21.15.39:0/365894931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f798810f7a0 msgr2=0x7f79881a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:21.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985400640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f798810f7a0 0x7f79881a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985400640 1 -- 172.21.15.39:0/365894931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7988109510 msgr2=0x7f798819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985400640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7988109510 0x7f798819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985400640 1 -- 172.21.15.39:0/365894931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79881a5d90 con 0x7f798810ee50 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.489+0000 7f7985400640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798810ee50 0x7f79881a03c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f798806bd90 tx=0x7f797000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.490+0000 7f797ea00640 1 -- 172.21.15.39:0/365894931 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797000cc40 con 0x7f798810ee50 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.490+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79881a6080 con 0x7f798810ee50 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.490+0000 7f797ea00640 1 -- 172.21.15.39:0/365894931 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f797000cde0 con 0x7f798810ee50 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.490+0000 7f797ea00640 1 -- 172.21.15.39:0/365894931 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970018760 con 0x7f798810ee50 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.490+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79881ac7e0 con 0x7f798810ee50 2024-07-04T08:11:21.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.491+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7988109920 con 0x7f798810ee50 2024-07-04T08:11:21.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.493+0000 7f797ea00640 1 -- 172.21.15.39:0/365894931 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f7970010a90 con 0x7f798810ee50 2024-07-04T08:11:21.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.495+0000 7f797ea00640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f796c0b28d0 0x7f796c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:21.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.495+0000 7f797ea00640 1 -- 172.21.15.39:0/365894931 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f79700e2ae0 con 0x7f798810ee50 2024-07-04T08:11:21.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.495+0000 7f7985e00640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f796c0b28d0 0x7f796c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:21.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.496+0000 7f7985e00640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f796c0b28d0 0x7f796c0b4de0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7978009c90 tx=0x7f7978005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:21.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.499+0000 7f797ea00640 1 -- 172.21.15.39:0/365894931 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f79700a9bc0 con 0x7f798810ee50 2024-07-04T08:11:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.744+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7988062d50 con 0x7f796c0b28d0 2024-07-04T08:11:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.751+0000 7f797ea00640 1 -- 172.21.15.39:0/365894931 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7988062d50 con 0x7f796c0b28d0 2024-07-04T08:11:21.752 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/qa/sv_0/981a0ff4-5507-4523-a95b-610ad6fdcc73 2024-07-04T08:11:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f796c0b28d0 msgr2=0x7f796c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f796c0b28d0 0x7f796c0b4de0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7978009c90 tx=0x7f7978005f90 comp rx=0 tx=0).stop 2024-07-04T08:11:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798810ee50 msgr2=0x7f79881a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798810ee50 0x7f79881a03c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f798806bd90 tx=0x7f797000ee50 comp rx=0 tx=0).stop 2024-07-04T08:11:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 shutdown_connections 2024-07-04T08:11:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f796c0b28d0 0x7f796c0b4de0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f798810f7a0 0x7f79881a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f798810ee50 0x7f79881a03c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 --2- 172.21.15.39:0/365894931 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7988109510 0x7f798819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 >> 172.21.15.39:0/365894931 conn(0x7f7988078b10 msgr2=0x7f79881078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 shutdown_connections 2024-07-04T08:11:21.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:21.754+0000 7f7987200640 1 -- 172.21.15.39:0/365894931 wait complete. 2024-07-04T08:11:22.511 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-07-04T08:11:22.990 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:23.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:22 smithi133 ceph-mon[37050]: pgmap v363: 129 pgs: 129 active+clean; 594 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-07-04T08:11:23.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:22 smithi133 ceph-mon[37050]: from='client.24893 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:22 smithi039 ceph-mon[32062]: pgmap v363: 129 pgs: 129 active+clean; 594 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-07-04T08:11:23.334 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:22 smithi039 ceph-mon[32062]: from='client.24893 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:23.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:22 smithi050 ceph-mon[36577]: pgmap v363: 129 pgs: 129 active+clean; 594 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-07-04T08:11:23.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:22 smithi050 ceph-mon[36577]: from='client.24893 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:24.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:24 smithi133 ceph-mon[37050]: pgmap v364: 129 pgs: 129 active+clean; 608 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-07-04T08:11:24.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:24 smithi039 ceph-mon[32062]: pgmap v364: 129 pgs: 129 active+clean; 608 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-07-04T08:11:24.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:24 smithi050 ceph-mon[36577]: pgmap v364: 129 pgs: 129 active+clean; 608 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-07-04T08:11:25.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.185+0000 7fbac7c00640 1 -- 172.21.15.39:0/3384993506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 msgr2=0x7fbac01034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:25.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.185+0000 7fbac7c00640 1 --2- 172.21.15.39:0/3384993506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 0x7fbac01034c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009900 tx=0x7fbab002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.185+0000 7fbac7c00640 1 -- 172.21.15.39:0/3384993506 shutdown_connections 2024-07-04T08:11:25.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.185+0000 7fbac7c00640 1 --2- 172.21.15.39:0/3384993506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbac01089f0 0x7fbac0108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.185+0000 7fbac7c00640 1 --2- 172.21.15.39:0/3384993506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac0109340 0x7fbac010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.185+0000 7fbac7c00640 1 --2- 172.21.15.39:0/3384993506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 0x7fbac01034c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.185+0000 7fbac7c00640 1 -- 172.21.15.39:0/3384993506 >> 172.21.15.39:0/3384993506 conn(0x7fbac007ada0 msgr2=0x7fbac007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.186+0000 7fbac7c00640 1 -- 172.21.15.39:0/3384993506 shutdown_connections 2024-07-04T08:11:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.186+0000 7fbac7c00640 1 -- 172.21.15.39:0/3384993506 wait complete. 2024-07-04T08:11:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.187+0000 7fbac7c00640 1 Processor -- start 2024-07-04T08:11:25.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.188+0000 7fbac7c00640 1 -- start start 2024-07-04T08:11:25.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.188+0000 7fbac7c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 0x7fbac019bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:25.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.188+0000 7fbac7c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac01089f0 0x7fbac019c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:25.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.188+0000 7fbac7c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbac0109340 0x7fbac01a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:25.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.188+0000 7fbac7c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0112ff0 con 0x7fbac0109340 2024-07-04T08:11:25.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.188+0000 7fbac7c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0112e70 con 0x7fbac01030b0 2024-07-04T08:11:25.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.188+0000 7fbac7c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0113170 con 0x7fbac01089f0 2024-07-04T08:11:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbac5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbac0109340 0x7fbac01a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbab7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac01089f0 0x7fbac019c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:25.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbac5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 0x7fbac019bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbab7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac01089f0 0x7fbac019c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42712/0 (socket says 172.21.15.39:42712) 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbac5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 0x7fbac019bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:36420/0 (socket says 172.21.15.39:36420) 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbab7e00640 1 -- 172.21.15.39:0/2687471312 learned_addr learned my addr 172.21.15.39:0/2687471312 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbac5c00640 1 -- 172.21.15.39:0/2687471312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac01089f0 msgr2=0x7fbac019c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbac5c00640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac01089f0 0x7fbac019c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbac5c00640 1 -- 172.21.15.39:0/2687471312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 msgr2=0x7fbac019bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbac5c00640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 0x7fbac019bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbac5c00640 1 -- 172.21.15.39:0/2687471312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac01a1bb0 con 0x7fbac0109340 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.189+0000 7fbac5200640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 0x7fbac019bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:11:25.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.190+0000 7fbac5c00640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbac0109340 0x7fbac01a1480 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009fd0 tx=0x7fbab800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:25.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.190+0000 7fbabe000640 1 -- 172.21.15.39:0/2687471312 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab800ce20 con 0x7fbac0109340 2024-07-04T08:11:25.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.190+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac01a1ea0 con 0x7fbac0109340 2024-07-04T08:11:25.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.190+0000 7fbabe000640 1 -- 172.21.15.39:0/2687471312 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbab8005020 con 0x7fbac0109340 2024-07-04T08:11:25.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.190+0000 7fbabe000640 1 -- 172.21.15.39:0/2687471312 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab8010690 con 0x7fbac0109340 2024-07-04T08:11:25.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.190+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac01a8600 con 0x7fbac0109340 2024-07-04T08:11:25.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.191+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac01034c0 con 0x7fbac0109340 2024-07-04T08:11:25.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.194+0000 7fbabe000640 1 -- 172.21.15.39:0/2687471312 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7fbab80054d0 con 0x7fbac0109340 2024-07-04T08:11:25.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.195+0000 7fbabe000640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbaa40b2950 0x7fbaa40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:25.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.196+0000 7fbabe000640 1 -- 172.21.15.39:0/2687471312 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fbab80e2dd0 con 0x7fbac0109340 2024-07-04T08:11:25.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.196+0000 7fbac5200640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbaa40b2950 0x7fbaa40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:25.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.197+0000 7fbac5200640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbaa40b2950 0x7fbaa40b4e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbab0031440 tx=0x7fbab0032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:25.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.197+0000 7fbabe000640 1 -- 172.21.15.39:0/2687471312 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fbab80aa500 con 0x7fbac0109340 2024-07-04T08:11:25.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.440+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbac0106f70 con 0x7fbaa40b2950 2024-07-04T08:11:25.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.478+0000 7fbabe000640 1 -- 172.21.15.39:0/2687471312 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbac0106f70 con 0x7fbaa40b2950 2024-07-04T08:11:25.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbaa40b2950 msgr2=0x7fbaa40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:25.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbaa40b2950 0x7fbaa40b4e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbab0031440 tx=0x7fbab0032ed0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbac0109340 msgr2=0x7fbac01a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:25.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbac0109340 0x7fbac01a1480 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009fd0 tx=0x7fbab800ef10 comp rx=0 tx=0).stop 2024-07-04T08:11:25.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 shutdown_connections 2024-07-04T08:11:25.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbaa40b2950 0x7fbaa40b4e60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbac0109340 0x7fbac01a1480 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbac01089f0 0x7fbac019c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 --2- 172.21.15.39:0/2687471312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbac01030b0 0x7fbac019bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:25.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 >> 172.21.15.39:0/2687471312 conn(0x7fbac007ada0 msgr2=0x7fbac0106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:25.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 shutdown_connections 2024-07-04T08:11:25.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:25.481+0000 7fbac7c00640 1 -- 172.21.15.39:0/2687471312 wait complete. 2024-07-04T08:11:26.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-07-04T08:11:26.863 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:26.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:26 smithi050 ceph-mon[36577]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:26.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:26 smithi050 ceph-mon[36577]: pgmap v365: 129 pgs: 129 active+clean; 608 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-07-04T08:11:27.015 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:26 smithi039 ceph-mon[32062]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:27.015 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:26 smithi039 ceph-mon[32062]: pgmap v365: 129 pgs: 129 active+clean; 608 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-07-04T08:11:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:26 smithi133 ceph-mon[37050]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:27.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:26 smithi133 ceph-mon[37050]: pgmap v365: 129 pgs: 129 active+clean; 608 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-07-04T08:11:28.803 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:28 smithi039 ceph-mon[32062]: pgmap v366: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-07-04T08:11:28.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:28 smithi050 ceph-mon[36577]: pgmap v366: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-07-04T08:11:29.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.026+0000 7fb6b0e00640 1 -- 172.21.15.39:0/3674583221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac10aa30 msgr2=0x7fb6ac10ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.026+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/3674583221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac10aa30 0x7fb6ac10ae40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb698009900 tx=0x7fb69802a410 comp rx=0 tx=0).stop 2024-07-04T08:11:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.026+0000 7fb6b0e00640 1 -- 172.21.15.39:0/3674583221 shutdown_connections 2024-07-04T08:11:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.026+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/3674583221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ac10b380 0x7fb6ac111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.026+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/3674583221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac10aa30 0x7fb6ac10ae40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.026+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/3674583221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ac1050f0 0x7fb6ac105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.026+0000 7fb6b0e00640 1 -- 172.21.15.39:0/3674583221 >> 172.21.15.39:0/3674583221 conn(0x7fb6ac100b60 msgr2=0x7fb6ac102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.026+0000 7fb6b0e00640 1 -- 172.21.15.39:0/3674583221 shutdown_connections 2024-07-04T08:11:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.026+0000 7fb6b0e00640 1 -- 172.21.15.39:0/3674583221 wait complete. 2024-07-04T08:11:29.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.027+0000 7fb6b0e00640 1 Processor -- start 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.027+0000 7fb6b0e00640 1 -- start start 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6b0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ac1050f0 0x7fb6ac19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6b0e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac10aa30 0x7fb6ac19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6b0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ac10b380 0x7fb6ac1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6b0e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac115060 con 0x7fb6ac1050f0 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6b0e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac114ee0 con 0x7fb6ac10b380 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6b0e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac1151e0 con 0x7fb6ac10aa30 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6ab400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ac1050f0 0x7fb6ac19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6ab400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ac1050f0 0x7fb6ac19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41502/0 (socket says 172.21.15.39:41502) 2024-07-04T08:11:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6ab400640 1 -- 172.21.15.39:0/1221712305 learned_addr learned my addr 172.21.15.39:0/1221712305 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:29.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.028+0000 7fb6aaa00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac10aa30 0x7fb6ac19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:29.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.029+0000 7fb6abe00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ac10b380 0x7fb6ac1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.029+0000 7fb6ab400640 1 -- 172.21.15.39:0/1221712305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac10aa30 msgr2=0x7fb6ac19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.029+0000 7fb6ab400640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac10aa30 0x7fb6ac19bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.029+0000 7fb6ab400640 1 -- 172.21.15.39:0/1221712305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ac10b380 msgr2=0x7fb6ac1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.029+0000 7fb6ab400640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ac10b380 0x7fb6ac1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.029+0000 7fb6ab400640 1 -- 172.21.15.39:0/1221712305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac1a19c0 con 0x7fb6ac1050f0 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.030+0000 7fb6ab400640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ac1050f0 0x7fb6ac19ba50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00e9f0 tx=0x7fb69c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.030+0000 7fb6a3e00640 1 -- 172.21.15.39:0/1221712305 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c00cd80 con 0x7fb6ac1050f0 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.030+0000 7fb6a3e00640 1 -- 172.21.15.39:0/1221712305 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb69c0054d0 con 0x7fb6ac1050f0 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.030+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ac1a1cb0 con 0x7fb6ac1050f0 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.030+0000 7fb6a3e00640 1 -- 172.21.15.39:0/1221712305 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c010690 con 0x7fb6ac1050f0 2024-07-04T08:11:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.030+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ac1a8410 con 0x7fb6ac1050f0 2024-07-04T08:11:29.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.032+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ac106370 con 0x7fb6ac1050f0 2024-07-04T08:11:29.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.036+0000 7fb6a3e00640 1 -- 172.21.15.39:0/1221712305 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7fb69c004e50 con 0x7fb6ac1050f0 2024-07-04T08:11:29.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.037+0000 7fb6a3e00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6900b2950 0x7fb6900b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:29.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.037+0000 7fb6a3e00640 1 -- 172.21.15.39:0/1221712305 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fb69c0e2dc0 con 0x7fb6ac1050f0 2024-07-04T08:11:29.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.038+0000 7fb6aaa00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6900b2950 0x7fb6900b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:29.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.038+0000 7fb6a3e00640 1 -- 172.21.15.39:0/1221712305 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb69c055110 con 0x7fb6ac1050f0 2024-07-04T08:11:29.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.039+0000 7fb6aaa00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6900b2950 0x7fb6900b4e60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb69802aa00 tx=0x7fb698005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:29.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:28 smithi133 ceph-mon[37050]: pgmap v366: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-07-04T08:11:29.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.283+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb6ac062d50 con 0x7fb6900b2950 2024-07-04T08:11:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.295+0000 7fb6a3e00640 1 -- 172.21.15.39:0/1221712305 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb6ac062d50 con 0x7fb6900b2950 2024-07-04T08:11:29.296 INFO:teuthology.orchestra.run.smithi039.stdout:/volumes/qa/sv_1/ea788566-52d9-4ba0-9ed3-9e28950c3dc4 2024-07-04T08:11:29.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6900b2950 msgr2=0x7fb6900b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:29.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6900b2950 0x7fb6900b4e60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb69802aa00 tx=0x7fb698005a20 comp rx=0 tx=0).stop 2024-07-04T08:11:29.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ac1050f0 msgr2=0x7fb6ac19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:29.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ac1050f0 0x7fb6ac19ba50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb69c00e9f0 tx=0x7fb69c00edf0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 shutdown_connections 2024-07-04T08:11:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb6900b2950 0x7fb6900b4e60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb6ac10b380 0x7fb6ac1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6ac10aa30 0x7fb6ac19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 --2- 172.21.15.39:0/1221712305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6ac1050f0 0x7fb6ac19ba50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 >> 172.21.15.39:0/1221712305 conn(0x7fb6ac100b60 msgr2=0x7fb6ac102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 shutdown_connections 2024-07-04T08:11:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:29.298+0000 7fb6b0e00640 1 -- 172.21.15.39:0/1221712305 wait complete. 2024-07-04T08:11:30.076 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:11:30.594 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:30.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:30 smithi050 ceph-mon[36577]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:30.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:30 smithi050 ceph-mon[36577]: pgmap v367: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-07-04T08:11:30.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:30 smithi039 ceph-mon[32062]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:30.934 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:30 smithi039 ceph-mon[32062]: pgmap v367: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-07-04T08:11:31.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:30 smithi133 ceph-mon[37050]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:11:31.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:30 smithi133 ceph-mon[37050]: pgmap v367: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-07-04T08:11:32.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.871+0000 7fd482800640 1 -- 172.21.15.39:0/4011539779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c10d360 msgr2=0x7fd47c10f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:32.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.871+0000 7fd482800640 1 --2- 172.21.15.39:0/4011539779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c10d360 0x7fd47c10f850 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd474009900 tx=0x7fd47402a400 comp rx=0 tx=0).stop 2024-07-04T08:11:32.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.871+0000 7fd482800640 1 -- 172.21.15.39:0/4011539779 shutdown_connections 2024-07-04T08:11:32.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.871+0000 7fd482800640 1 --2- 172.21.15.39:0/4011539779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c10d360 0x7fd47c10f850 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:32.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.871+0000 7fd482800640 1 --2- 172.21.15.39:0/4011539779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd47c104850 0x7fd47c10cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:32.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.871+0000 7fd482800640 1 --2- 172.21.15.39:0/4011539779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd47c103f00 0x7fd47c104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:32.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.871+0000 7fd482800640 1 -- 172.21.15.39:0/4011539779 >> 172.21.15.39:0/4011539779 conn(0x7fd47c078c80 msgr2=0x7fd47c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:32.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.871+0000 7fd482800640 1 -- 172.21.15.39:0/4011539779 shutdown_connections 2024-07-04T08:11:32.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.871+0000 7fd482800640 1 -- 172.21.15.39:0/4011539779 wait complete. 2024-07-04T08:11:32.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.872+0000 7fd482800640 1 Processor -- start 2024-07-04T08:11:32.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.872+0000 7fd482800640 1 -- start start 2024-07-04T08:11:32.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd482800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd47c103f00 0x7fd47c101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:32.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd482800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd47c104850 0x7fd47c102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:32.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd482800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c10d360 0x7fd47c1026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:32.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd482800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c113030 con 0x7fd47c104850 2024-07-04T08:11:32.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd482800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c112eb0 con 0x7fd47c103f00 2024-07-04T08:11:32.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd482800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c1131b0 con 0x7fd47c10d360 2024-07-04T08:11:32.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd47aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd47c104850 0x7fd47c102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:32.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd47b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd47c103f00 0x7fd47c101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:32.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd47aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd47c104850 0x7fd47c102180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57738/0 (socket says 172.21.15.39:57738) 2024-07-04T08:11:32.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd47aa00640 1 -- 172.21.15.39:0/649781421 learned_addr learned my addr 172.21.15.39:0/649781421 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:32.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.873+0000 7fd47be00640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c10d360 0x7fd47c1026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:32.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.874+0000 7fd47aa00640 1 -- 172.21.15.39:0/649781421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c10d360 msgr2=0x7fd47c1026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:32.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.874+0000 7fd47aa00640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c10d360 0x7fd47c1026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:32.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.874+0000 7fd47aa00640 1 -- 172.21.15.39:0/649781421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd47c103f00 msgr2=0x7fd47c101c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:32.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.874+0000 7fd47aa00640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd47c103f00 0x7fd47c101c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:32.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.874+0000 7fd47aa00640 1 -- 172.21.15.39:0/649781421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c1a7cc0 con 0x7fd47c104850 2024-07-04T08:11:32.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.874+0000 7fd47b400640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd47c103f00 0x7fd47c101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:11:32.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.874+0000 7fd47aa00640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd47c104850 0x7fd47c102180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd46400eb90 tx=0x7fd46400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:32.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.874+0000 7fd473e00640 1 -- 172.21.15.39:0/649781421 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464004d20 con 0x7fd47c104850 2024-07-04T08:11:32.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.875+0000 7fd473e00640 1 -- 172.21.15.39:0/649781421 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd46400ee90 con 0x7fd47c104850 2024-07-04T08:11:32.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.875+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47c1a7fb0 con 0x7fd47c104850 2024-07-04T08:11:32.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.875+0000 7fd473e00640 1 -- 172.21.15.39:0/649781421 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4640107f0 con 0x7fd47c104850 2024-07-04T08:11:32.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.875+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47c1a8400 con 0x7fd47c104850 2024-07-04T08:11:32.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.876+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd47c104310 con 0x7fd47c104850 2024-07-04T08:11:32.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.879+0000 7fd473e00640 1 -- 172.21.15.39:0/649781421 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7fd464010990 con 0x7fd47c104850 2024-07-04T08:11:32.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.880+0000 7fd473e00640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd45c0b28d0 0x7fd45c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:32.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.881+0000 7fd473e00640 1 -- 172.21.15.39:0/649781421 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fd4640e2ff0 con 0x7fd47c104850 2024-07-04T08:11:32.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.881+0000 7fd47b400640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd45c0b28d0 0x7fd45c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:32.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.882+0000 7fd47b400640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd45c0b28d0 0x7fd45c0b4de0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd47c103690 tx=0x7fd46c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:32.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:32.882+0000 7fd473e00640 1 -- 172.21.15.39:0/649781421 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fd4640aa760 con 0x7fd47c104850 2024-07-04T08:11:32.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:32 smithi050 ceph-mon[36577]: pgmap v368: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2024-07-04T08:11:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:32 smithi133 ceph-mon[37050]: pgmap v368: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2024-07-04T08:11:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:32 smithi039 ceph-mon[32062]: pgmap v368: 129 pgs: 129 active+clean; 627 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.6 KiB/s wr, 11 op/s 2024-07-04T08:11:33.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.141+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd47c1a86c0 con 0x7fd47c104850 2024-07-04T08:11:33.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.143+0000 7fd473e00640 1 -- 172.21.15.39:0/649781421 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11975 (secure 0 0 0) 0x7fd4640aa060 con 0x7fd47c104850 2024-07-04T08:11:33.144 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:11:33.144 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-07-04T08:11:09:862205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:11:09.668485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24673,24674,24679,24701,24707]},"id":1}]} 2024-07-04T08:11:33.145 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-07-04T08:11:33.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd45c0b28d0 msgr2=0x7fd45c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:33.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd45c0b28d0 0x7fd45c0b4de0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd47c103690 tx=0x7fd46c005f90 comp rx=0 tx=0).stop 2024-07-04T08:11:33.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd47c104850 msgr2=0x7fd47c102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:33.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd47c104850 0x7fd47c102180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd46400eb90 tx=0x7fd46400c6a0 comp rx=0 tx=0).stop 2024-07-04T08:11:33.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 shutdown_connections 2024-07-04T08:11:33.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd45c0b28d0 0x7fd45c0b4de0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:33.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd47c104850 0x7fd47c102180 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:33.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd47c10d360 0x7fd47c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:33.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 --2- 172.21.15.39:0/649781421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd47c103f00 0x7fd47c101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:33.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 >> 172.21.15.39:0/649781421 conn(0x7fd47c078c80 msgr2=0x7fd47c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:33.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 shutdown_connections 2024-07-04T08:11:33.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:33.146+0000 7fd482800640 1 -- 172.21.15.39:0/649781421 wait complete. 2024-07-04T08:11:33.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:33 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/649781421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:11:33.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:33 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/649781421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:11:33.956 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:11:34.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:33 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/649781421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:11:34.444 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:34.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:34 smithi039 ceph-mon[32062]: pgmap v369: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-07-04T08:11:35.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:34 smithi133 ceph-mon[37050]: pgmap v369: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-07-04T08:11:35.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:34 smithi050 ceph-mon[36577]: pgmap v369: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-07-04T08:11:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.683+0000 7f29b5a00640 1 -- 172.21.15.39:0/4209749902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b010f040 msgr2=0x7f29b010f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.683+0000 7f29b5a00640 1 --2- 172.21.15.39:0/4209749902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b010f040 0x7f29b010f450 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2998009900 tx=0x7f299802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.683+0000 7f29b5a00640 1 -- 172.21.15.39:0/4209749902 shutdown_connections 2024-07-04T08:11:36.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.683+0000 7f29b5a00640 1 --2- 172.21.15.39:0/4209749902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b010f040 0x7f29b010f450 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.683+0000 7f29b5a00640 1 --2- 172.21.15.39:0/4209749902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29b010f990 0x7f29b01161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.683+0000 7f29b5a00640 1 --2- 172.21.15.39:0/4209749902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29b0109700 0x7f29b0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.683+0000 7f29b5a00640 1 -- 172.21.15.39:0/4209749902 >> 172.21.15.39:0/4209749902 conn(0x7f29b0077630 msgr2=0x7f29b0107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:36.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.683+0000 7f29b5a00640 1 -- 172.21.15.39:0/4209749902 shutdown_connections 2024-07-04T08:11:36.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.683+0000 7f29b5a00640 1 -- 172.21.15.39:0/4209749902 wait complete. 2024-07-04T08:11:36.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.685+0000 7f29b5a00640 1 Processor -- start 2024-07-04T08:11:36.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.685+0000 7f29b5a00640 1 -- start start 2024-07-04T08:11:36.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29b5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b0109700 0x7f29b01a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:36.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29b5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29b010f040 0x7f29b01a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:36.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29b5a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29b010f990 0x7f29b01a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:36.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29b5a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0119670 con 0x7f29b0109700 2024-07-04T08:11:36.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29b5a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01194f0 con 0x7f29b010f040 2024-07-04T08:11:36.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29b5a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29b01197f0 con 0x7f29b010f990 2024-07-04T08:11:36.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29ae000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29b010f040 0x7f29b01a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:36.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29ae000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29b010f040 0x7f29b01a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38030/0 (socket says 172.21.15.39:38030) 2024-07-04T08:11:36.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29ae000640 1 -- 172.21.15.39:0/3805303014 learned_addr learned my addr 172.21.15.39:0/3805303014 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:36.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29aea00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b0109700 0x7f29b01a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:36.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.686+0000 7f29af400640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29b010f990 0x7f29b01a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:36.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.687+0000 7f29ae000640 1 -- 172.21.15.39:0/3805303014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29b010f990 msgr2=0x7f29b01a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:36.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.687+0000 7f29ae000640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29b010f990 0x7f29b01a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.687+0000 7f29ae000640 1 -- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b0109700 msgr2=0x7f29b01a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:36.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.687+0000 7f29ae000640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b0109700 0x7f29b01a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.687+0000 7f29ae000640 1 -- 172.21.15.39:0/3805303014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b01a5fb0 con 0x7f29b010f040 2024-07-04T08:11:36.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.687+0000 7f29aea00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b0109700 0x7f29b01a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:11:36.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.687+0000 7f29ae000640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29b010f040 0x7f29b01a05e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2998005ab0 tx=0x7f2998003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:36.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.687+0000 7f29a7400640 1 -- 172.21.15.39:0/3805303014 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299803f070 con 0x7f29b010f040 2024-07-04T08:11:36.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.688+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b01a6240 con 0x7f29b010f040 2024-07-04T08:11:36.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.688+0000 7f29a7400640 1 -- 172.21.15.39:0/3805303014 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2998031b50 con 0x7f29b010f040 2024-07-04T08:11:36.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.688+0000 7f29a7400640 1 -- 172.21.15.39:0/3805303014 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998035030 con 0x7f29b010f040 2024-07-04T08:11:36.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.688+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b01aca00 con 0x7f29b010f040 2024-07-04T08:11:36.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.689+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b010a980 con 0x7f29b010f040 2024-07-04T08:11:36.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.691+0000 7f29a7400640 1 -- 172.21.15.39:0/3805303014 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f2998042070 con 0x7f29b010f040 2024-07-04T08:11:36.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.693+0000 7f29a7400640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29940b28d0 0x7f29940b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:36.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.693+0000 7f29a7400640 1 -- 172.21.15.39:0/3805303014 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f2998102de0 con 0x7f29b010f040 2024-07-04T08:11:36.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.694+0000 7f29aea00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29940b28d0 0x7f29940b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:36.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.694+0000 7f29aea00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29940b28d0 0x7f29940b4de0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009fc0 tx=0x7f29a0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:36.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.697+0000 7f29a7400640 1 -- 172.21.15.39:0/3805303014 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f29980ca510 con 0x7f29b010f040 2024-07-04T08:11:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.915+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29b01117a0 con 0x7f29b010f040 2024-07-04T08:11:36.917 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:36 smithi039 ceph-mon[32062]: pgmap v370: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-07-04T08:11:36.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.917+0000 7f29a7400640 1 -- 172.21.15.39:0/3805303014 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24909 (secure 0 0 0) 0x7f29980c9e10 con 0x7f29b010f040 2024-07-04T08:11:36.919 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:11:36.920 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":90,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:10:52.644883+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:11:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29940b28d0 msgr2=0x7f29940b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29940b28d0 0x7f29940b4de0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009fc0 tx=0x7f29a0005f90 comp rx=0 tx=0).stop 2024-07-04T08:11:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29b010f040 msgr2=0x7f29b01a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29b010f040 0x7f29b01a05e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2998005ab0 tx=0x7f2998003b90 comp rx=0 tx=0).stop 2024-07-04T08:11:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 shutdown_connections 2024-07-04T08:11:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29940b28d0 0x7f29940b4de0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29b010f990 0x7f29b01a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29b010f040 0x7f29b01a05e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 --2- 172.21.15.39:0/3805303014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29b0109700 0x7f29b01a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 >> 172.21.15.39:0/3805303014 conn(0x7f29b0077630 msgr2=0x7f29b0077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 shutdown_connections 2024-07-04T08:11:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:36.922+0000 7f29b5a00640 1 -- 172.21.15.39:0/3805303014 wait complete. 2024-07-04T08:11:37.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:36 smithi133 ceph-mon[37050]: pgmap v370: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-07-04T08:11:37.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:36 smithi050 ceph-mon[36577]: pgmap v370: 129 pgs: 129 active+clean; 639 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-07-04T08:11:37.745 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-04T08:11:37.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph pg dump --format=json 2024-07-04T08:11:38.066 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:37 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3805303014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:11:38.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:37 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3805303014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:11:38.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:37 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3805303014' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:11:38.205 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:38 smithi133 ceph-mon[37050]: pgmap v371: 129 pgs: 129 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-07-04T08:11:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:11:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:38 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:38 smithi039 ceph-mon[32062]: pgmap v371: 129 pgs: 129 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-07-04T08:11:39.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:11:39.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:38 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:38 smithi050 ceph-mon[36577]: pgmap v371: 129 pgs: 129 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-07-04T08:11:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:11:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:38 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.563+0000 7f8177000640 1 -- 172.21.15.39:0/2782597260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8170103d10 msgr2=0x7f8170104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:40.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.563+0000 7f8177000640 1 --2- 172.21.15.39:0/2782597260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8170103d10 0x7f8170104120 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8164009900 tx=0x7f816402a410 comp rx=0 tx=0).stop 2024-07-04T08:11:40.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.564+0000 7f8177000640 1 -- 172.21.15.39:0/2782597260 shutdown_connections 2024-07-04T08:11:40.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.564+0000 7f8177000640 1 --2- 172.21.15.39:0/2782597260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817010d170 0x7f817010f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:40.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.564+0000 7f8177000640 1 --2- 172.21.15.39:0/2782597260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8170104660 0x7f817010cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.564+0000 7f8177000640 1 --2- 172.21.15.39:0/2782597260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8170103d10 0x7f8170104120 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.564+0000 7f8177000640 1 -- 172.21.15.39:0/2782597260 >> 172.21.15.39:0/2782597260 conn(0x7f81700fd810 msgr2=0x7f81700ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.564+0000 7f8177000640 1 -- 172.21.15.39:0/2782597260 shutdown_connections 2024-07-04T08:11:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.564+0000 7f8177000640 1 -- 172.21.15.39:0/2782597260 wait complete. 2024-07-04T08:11:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.565+0000 7f8177000640 1 Processor -- start 2024-07-04T08:11:40.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.565+0000 7f8177000640 1 -- start start 2024-07-04T08:11:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.566+0000 7f8177000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8170103d10 0x7f817019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.566+0000 7f8177000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8170104660 0x7f817019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.566+0000 7f8177000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817010d170 0x7f81701a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.566+0000 7f8177000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8170112e40 con 0x7f8170103d10 2024-07-04T08:11:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.566+0000 7f8177000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8170112cc0 con 0x7f8170104660 2024-07-04T08:11:40.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.566+0000 7f8177000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8170112fc0 con 0x7f817010d170 2024-07-04T08:11:40.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8170103d10 0x7f817019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:40.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8170104660 0x7f817019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:40.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8170104660 0x7f817019bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38054/0 (socket says 172.21.15.39:38054) 2024-07-04T08:11:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175200640 1 -- 172.21.15.39:0/3966910858 learned_addr learned my addr 172.21.15.39:0/3966910858 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8176600640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817010d170 0x7f81701a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175200640 1 -- 172.21.15.39:0/3966910858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817010d170 msgr2=0x7f81701a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175200640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817010d170 0x7f81701a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175200640 1 -- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8170103d10 msgr2=0x7f817019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175200640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8170103d10 0x7f817019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175200640 1 -- 172.21.15.39:0/3966910858 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81701a19a0 con 0x7f8170104660 2024-07-04T08:11:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.567+0000 7f8175c00640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8170103d10 0x7f817019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:11:40.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.568+0000 7f8175200640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8170104660 0x7f817019bfd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f815c00e9c0 tx=0x7f815c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.568+0000 7f816e000640 1 -- 172.21.15.39:0/3966910858 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c00ccc0 con 0x7f8170104660 2024-07-04T08:11:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.568+0000 7f816e000640 1 -- 172.21.15.39:0/3966910858 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f815c0054d0 con 0x7f8170104660 2024-07-04T08:11:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.568+0000 7f816e000640 1 -- 172.21.15.39:0/3966910858 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c010430 con 0x7f8170104660 2024-07-04T08:11:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.568+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81701a1c90 con 0x7f8170104660 2024-07-04T08:11:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.568+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81701a83f0 con 0x7f8170104660 2024-07-04T08:11:40.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.569+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8170104120 con 0x7f8170104660 2024-07-04T08:11:40.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.571+0000 7f816e000640 1 -- 172.21.15.39:0/3966910858 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f815c005020 con 0x7f8170104660 2024-07-04T08:11:40.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.574+0000 7f816e000640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81580b2950 0x7f81580b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:40.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.574+0000 7f816e000640 1 -- 172.21.15.39:0/3966910858 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f815c0e2bd0 con 0x7f8170104660 2024-07-04T08:11:40.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.574+0000 7f8175c00640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81580b2950 0x7f81580b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:40.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.575+0000 7f8175c00640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81580b2950 0x7f81580b4e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8164000c00 tx=0x7f8164031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:40.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.577+0000 7f816e000640 1 -- 172.21.15.39:0/3966910858 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f815c0aa300 con 0x7f8170104660 2024-07-04T08:11:40.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.797+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8170102040 con 0x7f81580b2950 2024-07-04T08:11:40.799 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.799 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.799 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:40 smithi039 ceph-mon[32062]: pgmap v372: 129 pgs: 129 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-04T08:11:40.799 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.800 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:40 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.810+0000 7f816e000640 1 -- 172.21.15.39:0/3966910858 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340707 (secure 0 0 0) 0x7f8170102040 con 0x7f81580b2950 2024-07-04T08:11:40.816 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:11:40.818 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:11:40.821 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.821 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.821 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:40 smithi133 ceph-mon[37050]: pgmap v372: 129 pgs: 129 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-04T08:11:40.821 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.821 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:40 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81580b2950 msgr2=0x7f81580b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:40.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81580b2950 0x7f81580b4e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8164000c00 tx=0x7f8164031400 comp rx=0 tx=0).stop 2024-07-04T08:11:40.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8170104660 msgr2=0x7f817019bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:40.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8170104660 0x7f817019bfd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f815c00e9c0 tx=0x7f815c00ee80 comp rx=0 tx=0).stop 2024-07-04T08:11:40.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 shutdown_connections 2024-07-04T08:11:40.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81580b2950 0x7f81580b4e60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:40.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f817010d170 0x7f81701a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:40.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8170104660 0x7f817019bfd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:40.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 --2- 172.21.15.39:0/3966910858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8170103d10 0x7f817019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:40.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 >> 172.21.15.39:0/3966910858 conn(0x7f81700fd810 msgr2=0x7f8170101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:40.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 shutdown_connections 2024-07-04T08:11:40.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:40.817+0000 7f8177000640 1 -- 172.21.15.39:0/3966910858 wait complete. 2024-07-04T08:11:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:40 smithi050 ceph-mon[36577]: pgmap v372: 129 pgs: 129 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-07-04T08:11:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:40.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:40 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:41.553 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-07-04T08:11:39.570509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655689,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":432,"num_read_kb":316,"num_write":370,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373876,"kb_used_data":12244,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567948,"statfs":{"total":1151940427776,"available":1151557578752,"internally_reserved":0,"allocated":12537856,"data_stored":8235390,"data_compressed":428013,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13404,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":67,"num_write":12,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007645"},"pg_stats":[{"pgid":"2.1f","version":"90'13","reported_seq":83,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:32.937176+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:11:32.937176+0000","last_peered":"2024-07-04T08:11:32.937176+0000","last_clean":"2024-07-04T08:11:32.937176+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:11:32.937176+0000","last_undegraded":"2024-07-04T08:11:32.937176+0000","last_fullsized":"2024-07-04T08:11:32.937176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679679+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:10:52.679679+0000","last_peered":"2024-07-04T08:10:52.679679+0000","last_clean":"2024-07-04T08:10:52.679679+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:10:52.679679+0000","last_undegraded":"2024-07-04T08:10:52.679679+0000","last_fullsized":"2024-07-04T08:10:52.679679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654299+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:10:52.654299+0000","last_peered":"2024-07-04T08:10:52.654299+0000","last_clean":"2024-07-04T08:10:52.654299+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:10:52.654299+0000","last_undegraded":"2024-07-04T08:10:52.654299+0000","last_fullsized":"2024-07-04T08:10:52.654299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:02.798150+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:02.798150+0000","last_peered":"2024-07-04T08:11:02.798150+0000","last_clean":"2024-07-04T08:11:02.798150+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:02.798150+0000","last_undegraded":"2024-07-04T08:11:02.798150+0000","last_fullsized":"2024-07-04T08:11:02.798150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.671323+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:10:52.671323+0000","last_peered":"2024-07-04T08:10:52.671323+0000","last_clean":"2024-07-04T08:10:52.671323+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:10:52.671323+0000","last_undegraded":"2024-07-04T08:10:52.671323+0000","last_fullsized":"2024-07-04T08:10:52.671323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.125619+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:10:53.125619+0000","last_peered":"2024-07-04T08:10:53.125619+0000","last_clean":"2024-07-04T08:10:53.125619+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:10:53.125619+0000","last_undegraded":"2024-07-04T08:10:53.125619+0000","last_fullsized":"2024-07-04T08:10:53.125619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:05.866831+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:11:05.866831+0000","last_peered":"2024-07-04T08:11:05.866831+0000","last_clean":"2024-07-04T08:11:05.866831+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:11:05.866831+0000","last_undegraded":"2024-07-04T08:11:05.866831+0000","last_fullsized":"2024-07-04T08:11:05.866831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657420+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:10:52.657420+0000","last_peered":"2024-07-04T08:10:52.657420+0000","last_clean":"2024-07-04T08:10:52.657420+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:10:52.657420+0000","last_undegraded":"2024-07-04T08:10:52.657420+0000","last_fullsized":"2024-07-04T08:10:52.657420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126090+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:10:53.126090+0000","last_peered":"2024-07-04T08:10:53.126090+0000","last_clean":"2024-07-04T08:10:53.126090+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:10:53.126090+0000","last_undegraded":"2024-07-04T08:10:53.126090+0000","last_fullsized":"2024-07-04T08:10:53.126090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.859030+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:11:04.859030+0000","last_peered":"2024-07-04T08:11:04.859030+0000","last_clean":"2024-07-04T08:11:04.859030+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:11:04.859030+0000","last_undegraded":"2024-07-04T08:11:04.859030+0000","last_fullsized":"2024-07-04T08:11:04.859030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667481+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:10:52.667481+0000","last_peered":"2024-07-04T08:10:52.667481+0000","last_clean":"2024-07-04T08:10:52.667481+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:10:52.667481+0000","last_undegraded":"2024-07-04T08:10:52.667481+0000","last_fullsized":"2024-07-04T08:10:52.667481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.125946+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:10:53.125946+0000","last_peered":"2024-07-04T08:10:53.125946+0000","last_clean":"2024-07-04T08:10:53.125946+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:10:53.125946+0000","last_undegraded":"2024-07-04T08:10:53.125946+0000","last_fullsized":"2024-07-04T08:10:53.125946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654646+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:10:52.654646+0000","last_peered":"2024-07-04T08:10:52.654646+0000","last_clean":"2024-07-04T08:10:52.654646+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:10:52.654646+0000","last_undegraded":"2024-07-04T08:10:52.654646+0000","last_fullsized":"2024-07-04T08:10:52.654646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:01.788372+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:01.788372+0000","last_peered":"2024-07-04T08:11:01.788372+0000","last_clean":"2024-07-04T08:11:01.788372+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:01.788372+0000","last_undegraded":"2024-07-04T08:11:01.788372+0000","last_fullsized":"2024-07-04T08:11:01.788372+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126268+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:10:53.126268+0000","last_peered":"2024-07-04T08:10:53.126268+0000","last_clean":"2024-07-04T08:10:53.126268+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:10:53.126268+0000","last_undegraded":"2024-07-04T08:10:53.126268+0000","last_fullsized":"2024-07-04T08:10:53.126268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.677266+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:10:52.677266+0000","last_peered":"2024-07-04T08:10:52.677266+0000","last_clean":"2024-07-04T08:10:52.677266+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:10:52.677266+0000","last_undegraded":"2024-07-04T08:10:52.677266+0000","last_fullsized":"2024-07-04T08:10:52.677266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:03.827675+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:03.827675+0000","last_peered":"2024-07-04T08:11:03.827675+0000","last_clean":"2024-07-04T08:11:03.827675+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:03.827675+0000","last_undegraded":"2024-07-04T08:11:03.827675+0000","last_fullsized":"2024-07-04T08:11:03.827675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126260+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:10:53.126260+0000","last_peered":"2024-07-04T08:10:53.126260+0000","last_clean":"2024-07-04T08:10:53.126260+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:10:53.126260+0000","last_undegraded":"2024-07-04T08:10:53.126260+0000","last_fullsized":"2024-07-04T08:10:53.126260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658428+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:10:52.658428+0000","last_peered":"2024-07-04T08:10:52.658428+0000","last_clean":"2024-07-04T08:10:52.658428+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:10:52.658428+0000","last_undegraded":"2024-07-04T08:10:52.658428+0000","last_fullsized":"2024-07-04T08:10:52.658428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.863662+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:04.863662+0000","last_peered":"2024-07-04T08:11:04.863662+0000","last_clean":"2024-07-04T08:11:04.863662+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:04.863662+0000","last_undegraded":"2024-07-04T08:11:04.863662+0000","last_fullsized":"2024-07-04T08:11:04.863662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.682686+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:10:52.682686+0000","last_peered":"2024-07-04T08:10:52.682686+0000","last_clean":"2024-07-04T08:10:52.682686+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:10:52.682686+0000","last_undegraded":"2024-07-04T08:10:52.682686+0000","last_fullsized":"2024-07-04T08:10:52.682686+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.656952+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:10:52.656952+0000","last_peered":"2024-07-04T08:10:52.656952+0000","last_clean":"2024-07-04T08:10:52.656952+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:10:52.656952+0000","last_undegraded":"2024-07-04T08:10:52.656952+0000","last_fullsized":"2024-07-04T08:10:52.656952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:01.787685+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:01.787685+0000","last_peered":"2024-07-04T08:11:01.787685+0000","last_clean":"2024-07-04T08:11:01.787685+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:01.787685+0000","last_undegraded":"2024-07-04T08:11:01.787685+0000","last_fullsized":"2024-07-04T08:11:01.787685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126202+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:10:53.126202+0000","last_peered":"2024-07-04T08:10:53.126202+0000","last_clean":"2024-07-04T08:10:53.126202+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:10:53.126202+0000","last_undegraded":"2024-07-04T08:10:53.126202+0000","last_fullsized":"2024-07-04T08:10:53.126202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666589+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:10:52.666589+0000","last_peered":"2024-07-04T08:10:52.666589+0000","last_clean":"2024-07-04T08:10:52.666589+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:10:52.666589+0000","last_undegraded":"2024-07-04T08:10:52.666589+0000","last_fullsized":"2024-07-04T08:10:52.666589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.859642+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:04.859642+0000","last_peered":"2024-07-04T08:11:04.859642+0000","last_clean":"2024-07-04T08:11:04.859642+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:04.859642+0000","last_undegraded":"2024-07-04T08:11:04.859642+0000","last_fullsized":"2024-07-04T08:11:04.859642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654445+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:10:52.654445+0000","last_peered":"2024-07-04T08:10:52.654445+0000","last_clean":"2024-07-04T08:10:52.654445+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:10:52.654445+0000","last_undegraded":"2024-07-04T08:10:52.654445+0000","last_fullsized":"2024-07-04T08:10:52.654445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679271+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:10:52.679271+0000","last_peered":"2024-07-04T08:10:52.679271+0000","last_clean":"2024-07-04T08:10:52.679271+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:10:52.679271+0000","last_undegraded":"2024-07-04T08:10:52.679271+0000","last_fullsized":"2024-07-04T08:10:52.679271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:27.426249+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:11:27.426249+0000","last_peered":"2024-07-04T08:11:27.426249+0000","last_clean":"2024-07-04T08:11:27.426249+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:11:27.426249+0000","last_undegraded":"2024-07-04T08:11:27.426249+0000","last_fullsized":"2024-07-04T08:11:27.426249+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.674576+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:10:52.674576+0000","last_peered":"2024-07-04T08:10:52.674576+0000","last_clean":"2024-07-04T08:10:52.674576+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:10:52.674576+0000","last_undegraded":"2024-07-04T08:10:52.674576+0000","last_fullsized":"2024-07-04T08:10:52.674576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.681557+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:10:52.681557+0000","last_peered":"2024-07-04T08:10:52.681557+0000","last_clean":"2024-07-04T08:10:52.681557+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:10:52.681557+0000","last_undegraded":"2024-07-04T08:10:52.681557+0000","last_fullsized":"2024-07-04T08:10:52.681557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.864213+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:04.864213+0000","last_peered":"2024-07-04T08:11:04.864213+0000","last_clean":"2024-07-04T08:11:04.864213+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:04.864213+0000","last_undegraded":"2024-07-04T08:11:04.864213+0000","last_fullsized":"2024-07-04T08:11:04.864213+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658117+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:10:52.658117+0000","last_peered":"2024-07-04T08:10:52.658117+0000","last_clean":"2024-07-04T08:10:52.658117+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:10:52.658117+0000","last_undegraded":"2024-07-04T08:10:52.658117+0000","last_fullsized":"2024-07-04T08:10:52.658117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667859+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:10:52.667859+0000","last_peered":"2024-07-04T08:10:52.667859+0000","last_clean":"2024-07-04T08:10:52.667859+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:10:52.667859+0000","last_undegraded":"2024-07-04T08:10:52.667859+0000","last_fullsized":"2024-07-04T08:10:52.667859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.665221+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:10:52.665221+0000","last_peered":"2024-07-04T08:10:52.665221+0000","last_clean":"2024-07-04T08:10:52.665221+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:10:52.665221+0000","last_undegraded":"2024-07-04T08:10:52.665221+0000","last_fullsized":"2024-07-04T08:10:52.665221+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657044+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:10:52.657044+0000","last_peered":"2024-07-04T08:10:52.657044+0000","last_clean":"2024-07-04T08:10:52.657044+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:10:52.657044+0000","last_undegraded":"2024-07-04T08:10:52.657044+0000","last_fullsized":"2024-07-04T08:10:52.657044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.676059+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:10:52.676059+0000","last_peered":"2024-07-04T08:10:52.676059+0000","last_clean":"2024-07-04T08:10:52.676059+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:10:52.676059+0000","last_undegraded":"2024-07-04T08:10:52.676059+0000","last_fullsized":"2024-07-04T08:10:52.676059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:34.943778+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:11:34.943778+0000","last_peered":"2024-07-04T08:11:34.943778+0000","last_clean":"2024-07-04T08:11:34.943778+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:11:34.943778+0000","last_undegraded":"2024-07-04T08:11:34.943778+0000","last_fullsized":"2024-07-04T08:11:34.943778+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.681326+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:10:52.681326+0000","last_peered":"2024-07-04T08:10:52.681326+0000","last_clean":"2024-07-04T08:10:52.681326+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:10:52.681326+0000","last_undegraded":"2024-07-04T08:10:52.681326+0000","last_fullsized":"2024-07-04T08:10:52.681326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667029+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:10:52.667029+0000","last_peered":"2024-07-04T08:10:52.667029+0000","last_clean":"2024-07-04T08:10:52.667029+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:10:52.667029+0000","last_undegraded":"2024-07-04T08:10:52.667029+0000","last_fullsized":"2024-07-04T08:10:52.667029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:36.947685+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:11:36.947685+0000","last_peered":"2024-07-04T08:11:36.947685+0000","last_clean":"2024-07-04T08:11:36.947685+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:11:36.947685+0000","last_undegraded":"2024-07-04T08:11:36.947685+0000","last_fullsized":"2024-07-04T08:11:36.947685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.681978+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:10:52.681978+0000","last_peered":"2024-07-04T08:10:52.681978+0000","last_clean":"2024-07-04T08:10:52.681978+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:10:52.681978+0000","last_undegraded":"2024-07-04T08:10:52.681978+0000","last_fullsized":"2024-07-04T08:10:52.681978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679083+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:10:52.679083+0000","last_peered":"2024-07-04T08:10:52.679083+0000","last_clean":"2024-07-04T08:10:52.679083+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:10:52.679083+0000","last_undegraded":"2024-07-04T08:10:52.679083+0000","last_fullsized":"2024-07-04T08:10:52.679083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.857965+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:04.857965+0000","last_peered":"2024-07-04T08:11:04.857965+0000","last_clean":"2024-07-04T08:11:04.857965+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:04.857965+0000","last_undegraded":"2024-07-04T08:11:04.857965+0000","last_fullsized":"2024-07-04T08:11:04.857965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.684905+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:10:52.684905+0000","last_peered":"2024-07-04T08:10:52.684905+0000","last_clean":"2024-07-04T08:10:52.684905+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:10:52.684905+0000","last_undegraded":"2024-07-04T08:10:52.684905+0000","last_fullsized":"2024-07-04T08:10:52.684905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.678368+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:10:52.678368+0000","last_peered":"2024-07-04T08:10:52.678368+0000","last_clean":"2024-07-04T08:10:52.678368+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:10:52.678368+0000","last_undegraded":"2024-07-04T08:10:52.678368+0000","last_fullsized":"2024-07-04T08:10:52.678368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:05.866947+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:05.866947+0000","last_peered":"2024-07-04T08:11:05.866947+0000","last_clean":"2024-07-04T08:11:05.866947+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:05.866947+0000","last_undegraded":"2024-07-04T08:11:05.866947+0000","last_fullsized":"2024-07-04T08:11:05.866947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":114,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655606+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:10:52.655606+0000","last_peered":"2024-07-04T08:10:52.655606+0000","last_clean":"2024-07-04T08:10:52.655606+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:10:52.655606+0000","last_undegraded":"2024-07-04T08:10:52.655606+0000","last_fullsized":"2024-07-04T08:10:52.655606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655236+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:10:52.655236+0000","last_peered":"2024-07-04T08:10:52.655236+0000","last_clean":"2024-07-04T08:10:52.655236+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:10:52.655236+0000","last_undegraded":"2024-07-04T08:10:52.655236+0000","last_fullsized":"2024-07-04T08:10:52.655236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655336+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:10:52.655336+0000","last_peered":"2024-07-04T08:10:52.655336+0000","last_clean":"2024-07-04T08:10:52.655336+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:10:52.655336+0000","last_undegraded":"2024-07-04T08:10:52.655336+0000","last_fullsized":"2024-07-04T08:10:52.655336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:37.915140+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:11:37.915140+0000","last_peered":"2024-07-04T08:11:37.915140+0000","last_clean":"2024-07-04T08:11:37.915140+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:11:37.915140+0000","last_undegraded":"2024-07-04T08:11:37.915140+0000","last_fullsized":"2024-07-04T08:11:37.915140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.656979+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:10:52.656979+0000","last_peered":"2024-07-04T08:10:52.656979+0000","last_clean":"2024-07-04T08:10:52.656979+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:10:52.656979+0000","last_undegraded":"2024-07-04T08:10:52.656979+0000","last_fullsized":"2024-07-04T08:10:52.656979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654965+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:10:52.654965+0000","last_peered":"2024-07-04T08:10:52.654965+0000","last_clean":"2024-07-04T08:10:52.654965+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:10:52.654965+0000","last_undegraded":"2024-07-04T08:10:52.654965+0000","last_fullsized":"2024-07-04T08:10:52.654965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.863668+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:04.863668+0000","last_peered":"2024-07-04T08:11:04.863668+0000","last_clean":"2024-07-04T08:11:04.863668+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:11:04.863668+0000","last_undegraded":"2024-07-04T08:11:04.863668+0000","last_fullsized":"2024-07-04T08:11:04.863668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.663648+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:10:52.663648+0000","last_peered":"2024-07-04T08:10:52.663648+0000","last_clean":"2024-07-04T08:10:52.663648+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:10:52.663648+0000","last_undegraded":"2024-07-04T08:10:52.663648+0000","last_fullsized":"2024-07-04T08:10:52.663648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654983+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:10:52.654983+0000","last_peered":"2024-07-04T08:10:52.654983+0000","last_clean":"2024-07-04T08:10:52.654983+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:10:52.654983+0000","last_undegraded":"2024-07-04T08:10:52.654983+0000","last_fullsized":"2024-07-04T08:10:52.654983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.864527+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:04.864527+0000","last_peered":"2024-07-04T08:11:04.864527+0000","last_clean":"2024-07-04T08:11:04.864527+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:04.864527+0000","last_undegraded":"2024-07-04T08:11:04.864527+0000","last_fullsized":"2024-07-04T08:11:04.864527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666623+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:10:52.666623+0000","last_peered":"2024-07-04T08:10:52.666623+0000","last_clean":"2024-07-04T08:10:52.666623+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:10:52.666623+0000","last_undegraded":"2024-07-04T08:10:52.666623+0000","last_fullsized":"2024-07-04T08:10:52.666623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658305+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:10:52.658305+0000","last_peered":"2024-07-04T08:10:52.658305+0000","last_clean":"2024-07-04T08:10:52.658305+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:10:52.658305+0000","last_undegraded":"2024-07-04T08:10:52.658305+0000","last_fullsized":"2024-07-04T08:10:52.658305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.785269+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:04.785269+0000","last_peered":"2024-07-04T08:11:04.785269+0000","last_clean":"2024-07-04T08:11:04.785269+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:04.785269+0000","last_undegraded":"2024-07-04T08:11:04.785269+0000","last_fullsized":"2024-07-04T08:11:04.785269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666830+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:10:52.666830+0000","last_peered":"2024-07-04T08:10:52.666830+0000","last_clean":"2024-07-04T08:10:52.666830+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:10:52.666830+0000","last_undegraded":"2024-07-04T08:10:52.666830+0000","last_fullsized":"2024-07-04T08:10:52.666830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.128942+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:10:53.128942+0000","last_peered":"2024-07-04T08:10:53.128942+0000","last_clean":"2024-07-04T08:10:53.128942+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:10:53.128942+0000","last_undegraded":"2024-07-04T08:10:53.128942+0000","last_fullsized":"2024-07-04T08:10:53.128942+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.864722+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:04.864722+0000","last_peered":"2024-07-04T08:11:04.864722+0000","last_clean":"2024-07-04T08:11:04.864722+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:04.864722+0000","last_undegraded":"2024-07-04T08:11:04.864722+0000","last_fullsized":"2024-07-04T08:11:04.864722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.681787+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:10:52.681787+0000","last_peered":"2024-07-04T08:10:52.681787+0000","last_clean":"2024-07-04T08:10:52.681787+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:10:52.681787+0000","last_undegraded":"2024-07-04T08:10:52.681787+0000","last_fullsized":"2024-07-04T08:10:52.681787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654382+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:10:52.654382+0000","last_peered":"2024-07-04T08:10:52.654382+0000","last_clean":"2024-07-04T08:10:52.654382+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:10:52.654382+0000","last_undegraded":"2024-07-04T08:10:52.654382+0000","last_fullsized":"2024-07-04T08:10:52.654382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:33.941109+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:11:33.941109+0000","last_peered":"2024-07-04T08:11:33.941109+0000","last_clean":"2024-07-04T08:11:33.941109+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:11:33.941109+0000","last_undegraded":"2024-07-04T08:11:33.941109+0000","last_fullsized":"2024-07-04T08:11:33.941109+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.664560+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:10:52.664560+0000","last_peered":"2024-07-04T08:10:52.664560+0000","last_clean":"2024-07-04T08:10:52.664560+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:10:52.664560+0000","last_undegraded":"2024-07-04T08:10:52.664560+0000","last_fullsized":"2024-07-04T08:10:52.664560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654582+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:10:52.654582+0000","last_peered":"2024-07-04T08:10:52.654582+0000","last_clean":"2024-07-04T08:10:52.654582+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:10:52.654582+0000","last_undegraded":"2024-07-04T08:10:52.654582+0000","last_fullsized":"2024-07-04T08:10:52.654582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.863668+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:04.863668+0000","last_peered":"2024-07-04T08:11:04.863668+0000","last_clean":"2024-07-04T08:11:04.863668+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:04.863668+0000","last_undegraded":"2024-07-04T08:11:04.863668+0000","last_fullsized":"2024-07-04T08:11:04.863668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.683951+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:10:52.683951+0000","last_peered":"2024-07-04T08:10:52.683951+0000","last_clean":"2024-07-04T08:10:52.683951+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:10:52.683951+0000","last_undegraded":"2024-07-04T08:10:52.683951+0000","last_fullsized":"2024-07-04T08:10:52.683951+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.677112+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:10:52.677112+0000","last_peered":"2024-07-04T08:10:52.677112+0000","last_clean":"2024-07-04T08:10:52.677112+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:10:52.677112+0000","last_undegraded":"2024-07-04T08:10:52.677112+0000","last_fullsized":"2024-07-04T08:10:52.677112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:05.867006+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:05.867006+0000","last_peered":"2024-07-04T08:11:05.867006+0000","last_clean":"2024-07-04T08:11:05.867006+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:05.867006+0000","last_undegraded":"2024-07-04T08:11:05.867006+0000","last_fullsized":"2024-07-04T08:11:05.867006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667447+0000","last_change":"2024-07-04T08:09:51.640257+0000","last_active":"2024-07-04T08:10:52.667447+0000","last_peered":"2024-07-04T08:10:52.667447+0000","last_clean":"2024-07-04T08:10:52.667447+0000","last_became_active":"2024-07-04T08:09:51.639733+0000","last_became_peered":"2024-07-04T08:09:51.639733+0000","last_unstale":"2024-07-04T08:10:52.667447+0000","last_undegraded":"2024-07-04T08:10:52.667447+0000","last_fullsized":"2024-07-04T08:10:52.667447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:53:40.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657958+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:10:52.657958+0000","last_peered":"2024-07-04T08:10:52.657958+0000","last_clean":"2024-07-04T08:10:52.657958+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:10:52.657958+0000","last_undegraded":"2024-07-04T08:10:52.657958+0000","last_fullsized":"2024-07-04T08:10:52.657958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:35.920933+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:11:35.920933+0000","last_peered":"2024-07-04T08:11:35.920933+0000","last_clean":"2024-07-04T08:11:35.920933+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:11:35.920933+0000","last_undegraded":"2024-07-04T08:11:35.920933+0000","last_fullsized":"2024-07-04T08:11:35.920933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57883,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":91,"num_write":35,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.662287+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:10:52.662287+0000","last_peered":"2024-07-04T08:10:52.662287+0000","last_clean":"2024-07-04T08:10:52.662287+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:10:52.662287+0000","last_undegraded":"2024-07-04T08:10:52.662287+0000","last_fullsized":"2024-07-04T08:10:52.662287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657501+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:10:52.657501+0000","last_peered":"2024-07-04T08:10:52.657501+0000","last_clean":"2024-07-04T08:10:52.657501+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:10:52.657501+0000","last_undegraded":"2024-07-04T08:10:52.657501+0000","last_fullsized":"2024-07-04T08:10:52.657501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:03.822478+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:03.822478+0000","last_peered":"2024-07-04T08:11:03.822478+0000","last_clean":"2024-07-04T08:11:03.822478+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:03.822478+0000","last_undegraded":"2024-07-04T08:11:03.822478+0000","last_fullsized":"2024-07-04T08:11:03.822478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126373+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:10:53.126373+0000","last_peered":"2024-07-04T08:10:53.126373+0000","last_clean":"2024-07-04T08:10:53.126373+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:10:53.126373+0000","last_undegraded":"2024-07-04T08:10:53.126373+0000","last_fullsized":"2024-07-04T08:10:53.126373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.671698+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:10:52.671698+0000","last_peered":"2024-07-04T08:10:52.671698+0000","last_clean":"2024-07-04T08:10:52.671698+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:10:52.671698+0000","last_undegraded":"2024-07-04T08:10:52.671698+0000","last_fullsized":"2024-07-04T08:10:52.671698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.857299+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:11:04.857299+0000","last_peered":"2024-07-04T08:11:04.857299+0000","last_clean":"2024-07-04T08:11:04.857299+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:11:04.857299+0000","last_undegraded":"2024-07-04T08:11:04.857299+0000","last_fullsized":"2024-07-04T08:11:04.857299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.661322+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:10:52.661322+0000","last_peered":"2024-07-04T08:10:52.661322+0000","last_clean":"2024-07-04T08:10:52.661322+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:10:52.661322+0000","last_undegraded":"2024-07-04T08:10:52.661322+0000","last_fullsized":"2024-07-04T08:10:52.661322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657635+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:10:52.657635+0000","last_peered":"2024-07-04T08:10:52.657635+0000","last_clean":"2024-07-04T08:10:52.657635+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:10:52.657635+0000","last_undegraded":"2024-07-04T08:10:52.657635+0000","last_fullsized":"2024-07-04T08:10:52.657635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":106,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:35.912078+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:11:35.912078+0000","last_peered":"2024-07-04T08:11:35.912078+0000","last_clean":"2024-07-04T08:11:35.912078+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:11:35.912078+0000","last_undegraded":"2024-07-04T08:11:35.912078+0000","last_fullsized":"2024-07-04T08:11:35.912078+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.677936+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:10:52.677936+0000","last_peered":"2024-07-04T08:10:52.677936+0000","last_clean":"2024-07-04T08:10:52.677936+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:10:52.677936+0000","last_undegraded":"2024-07-04T08:10:52.677936+0000","last_fullsized":"2024-07-04T08:10:52.677936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.665672+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:10:52.665672+0000","last_peered":"2024-07-04T08:10:52.665672+0000","last_clean":"2024-07-04T08:10:52.665672+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:10:52.665672+0000","last_undegraded":"2024-07-04T08:10:52.665672+0000","last_fullsized":"2024-07-04T08:10:52.665672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.665987+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:10:52.665987+0000","last_peered":"2024-07-04T08:10:52.665987+0000","last_clean":"2024-07-04T08:10:52.665987+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:10:52.665987+0000","last_undegraded":"2024-07-04T08:10:52.665987+0000","last_fullsized":"2024-07-04T08:10:52.665987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.857911+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:04.857911+0000","last_peered":"2024-07-04T08:11:04.857911+0000","last_clean":"2024-07-04T08:11:04.857911+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:04.857911+0000","last_undegraded":"2024-07-04T08:11:04.857911+0000","last_fullsized":"2024-07-04T08:11:04.857911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657875+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:10:52.657875+0000","last_peered":"2024-07-04T08:10:52.657875+0000","last_clean":"2024-07-04T08:10:52.657875+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:10:52.657875+0000","last_undegraded":"2024-07-04T08:10:52.657875+0000","last_fullsized":"2024-07-04T08:10:52.657875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.674274+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:10:52.674274+0000","last_peered":"2024-07-04T08:10:52.674274+0000","last_clean":"2024-07-04T08:10:52.674274+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:10:52.674274+0000","last_undegraded":"2024-07-04T08:10:52.674274+0000","last_fullsized":"2024-07-04T08:10:52.674274+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:02.798151+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:11:02.798151+0000","last_peered":"2024-07-04T08:11:02.798151+0000","last_clean":"2024-07-04T08:11:02.798151+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:11:02.798151+0000","last_undegraded":"2024-07-04T08:11:02.798151+0000","last_fullsized":"2024-07-04T08:11:02.798151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:03.833435+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:11:03.833435+0000","last_peered":"2024-07-04T08:11:03.833435+0000","last_clean":"2024-07-04T08:11:03.833435+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:11:03.833435+0000","last_undegraded":"2024-07-04T08:11:03.833435+0000","last_fullsized":"2024-07-04T08:11:03.833435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667051+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:10:52.667051+0000","last_peered":"2024-07-04T08:10:52.667051+0000","last_clean":"2024-07-04T08:10:52.667051+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:10:52.667051+0000","last_undegraded":"2024-07-04T08:10:52.667051+0000","last_fullsized":"2024-07-04T08:10:52.667051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654630+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:10:52.654630+0000","last_peered":"2024-07-04T08:10:52.654630+0000","last_clean":"2024-07-04T08:10:52.654630+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:10:52.654630+0000","last_undegraded":"2024-07-04T08:10:52.654630+0000","last_fullsized":"2024-07-04T08:10:52.654630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":77,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:33.911011+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:11:33.911011+0000","last_peered":"2024-07-04T08:11:33.911011+0000","last_clean":"2024-07-04T08:11:33.911011+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:11:33.911011+0000","last_undegraded":"2024-07-04T08:11:33.911011+0000","last_fullsized":"2024-07-04T08:11:33.911011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657655+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:10:52.657655+0000","last_peered":"2024-07-04T08:10:52.657655+0000","last_clean":"2024-07-04T08:10:52.657655+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:10:52.657655+0000","last_undegraded":"2024-07-04T08:10:52.657655+0000","last_fullsized":"2024-07-04T08:10:52.657655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654821+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:10:52.654821+0000","last_peered":"2024-07-04T08:10:52.654821+0000","last_clean":"2024-07-04T08:10:52.654821+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:10:52.654821+0000","last_undegraded":"2024-07-04T08:10:52.654821+0000","last_fullsized":"2024-07-04T08:10:52.654821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.656924+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:10:52.656924+0000","last_peered":"2024-07-04T08:10:52.656924+0000","last_clean":"2024-07-04T08:10:52.656924+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:10:52.656924+0000","last_undegraded":"2024-07-04T08:10:52.656924+0000","last_fullsized":"2024-07-04T08:10:52.656924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654767+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:10:52.654767+0000","last_peered":"2024-07-04T08:10:52.654767+0000","last_clean":"2024-07-04T08:10:52.654767+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:10:52.654767+0000","last_undegraded":"2024-07-04T08:10:52.654767+0000","last_fullsized":"2024-07-04T08:10:52.654767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654388+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:10:52.654388+0000","last_peered":"2024-07-04T08:10:52.654388+0000","last_clean":"2024-07-04T08:10:52.654388+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:10:52.654388+0000","last_undegraded":"2024-07-04T08:10:52.654388+0000","last_fullsized":"2024-07-04T08:10:52.654388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.664171+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:10:52.664171+0000","last_peered":"2024-07-04T08:10:52.664171+0000","last_clean":"2024-07-04T08:10:52.664171+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:10:52.664171+0000","last_undegraded":"2024-07-04T08:10:52.664171+0000","last_fullsized":"2024-07-04T08:10:52.664171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.685603+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:10:52.685603+0000","last_peered":"2024-07-04T08:10:52.685603+0000","last_clean":"2024-07-04T08:10:52.685603+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:10:52.685603+0000","last_undegraded":"2024-07-04T08:10:52.685603+0000","last_fullsized":"2024-07-04T08:10:52.685603+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657589+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:10:52.657589+0000","last_peered":"2024-07-04T08:10:52.657589+0000","last_clean":"2024-07-04T08:10:52.657589+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:10:52.657589+0000","last_undegraded":"2024-07-04T08:10:52.657589+0000","last_fullsized":"2024-07-04T08:10:52.657589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655145+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:10:52.655145+0000","last_peered":"2024-07-04T08:10:52.655145+0000","last_clean":"2024-07-04T08:10:52.655145+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:10:52.655145+0000","last_undegraded":"2024-07-04T08:10:52.655145+0000","last_fullsized":"2024-07-04T08:10:52.655145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679089+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:10:52.679089+0000","last_peered":"2024-07-04T08:10:52.679089+0000","last_clean":"2024-07-04T08:10:52.679089+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:10:52.679089+0000","last_undegraded":"2024-07-04T08:10:52.679089+0000","last_fullsized":"2024-07-04T08:10:52.679089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666957+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:10:52.666957+0000","last_peered":"2024-07-04T08:10:52.666957+0000","last_clean":"2024-07-04T08:10:52.666957+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:10:52.666957+0000","last_undegraded":"2024-07-04T08:10:52.666957+0000","last_fullsized":"2024-07-04T08:10:52.666957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.678494+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:10:52.678494+0000","last_peered":"2024-07-04T08:10:52.678494+0000","last_clean":"2024-07-04T08:10:52.678494+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:10:52.678494+0000","last_undegraded":"2024-07-04T08:10:52.678494+0000","last_fullsized":"2024-07-04T08:10:52.678494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657262+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:10:52.657262+0000","last_peered":"2024-07-04T08:10:52.657262+0000","last_clean":"2024-07-04T08:10:52.657262+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:10:52.657262+0000","last_undegraded":"2024-07-04T08:10:52.657262+0000","last_fullsized":"2024-07-04T08:10:52.657262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679568+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:10:52.679568+0000","last_peered":"2024-07-04T08:10:52.679568+0000","last_clean":"2024-07-04T08:10:52.679568+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:10:52.679568+0000","last_undegraded":"2024-07-04T08:10:52.679568+0000","last_fullsized":"2024-07-04T08:10:52.679568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657856+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:10:52.657856+0000","last_peered":"2024-07-04T08:10:52.657856+0000","last_clean":"2024-07-04T08:10:52.657856+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:10:52.657856+0000","last_undegraded":"2024-07-04T08:10:52.657856+0000","last_fullsized":"2024-07-04T08:10:52.657856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654973+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:10:52.654973+0000","last_peered":"2024-07-04T08:10:52.654973+0000","last_clean":"2024-07-04T08:10:52.654973+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:10:52.654973+0000","last_undegraded":"2024-07-04T08:10:52.654973+0000","last_fullsized":"2024-07-04T08:10:52.654973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.664377+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:10:52.664377+0000","last_peered":"2024-07-04T08:10:52.664377+0000","last_clean":"2024-07-04T08:10:52.664377+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:10:52.664377+0000","last_undegraded":"2024-07-04T08:10:52.664377+0000","last_fullsized":"2024-07-04T08:10:52.664377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658461+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:10:52.658461+0000","last_peered":"2024-07-04T08:10:52.658461+0000","last_clean":"2024-07-04T08:10:52.658461+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:10:52.658461+0000","last_undegraded":"2024-07-04T08:10:52.658461+0000","last_fullsized":"2024-07-04T08:10:52.658461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.656760+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:10:52.656760+0000","last_peered":"2024-07-04T08:10:52.656760+0000","last_clean":"2024-07-04T08:10:52.656760+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:10:52.656760+0000","last_undegraded":"2024-07-04T08:10:52.656760+0000","last_fullsized":"2024-07-04T08:10:52.656760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.662263+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:10:52.662263+0000","last_peered":"2024-07-04T08:10:52.662263+0000","last_clean":"2024-07-04T08:10:52.662263+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:10:52.662263+0000","last_undegraded":"2024-07-04T08:10:52.662263+0000","last_fullsized":"2024-07-04T08:10:52.662263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.663082+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:10:52.663082+0000","last_peered":"2024-07-04T08:10:52.663082+0000","last_clean":"2024-07-04T08:10:52.663082+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:10:52.663082+0000","last_undegraded":"2024-07-04T08:10:52.663082+0000","last_fullsized":"2024-07-04T08:10:52.663082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679412+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:10:52.679412+0000","last_peered":"2024-07-04T08:10:52.679412+0000","last_clean":"2024-07-04T08:10:52.679412+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:10:52.679412+0000","last_undegraded":"2024-07-04T08:10:52.679412+0000","last_fullsized":"2024-07-04T08:10:52.679412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.127162+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:10:53.127162+0000","last_peered":"2024-07-04T08:10:53.127162+0000","last_clean":"2024-07-04T08:10:53.127162+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:10:53.127162+0000","last_undegraded":"2024-07-04T08:10:53.127162+0000","last_fullsized":"2024-07-04T08:10:53.127162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655313+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:10:52.655313+0000","last_peered":"2024-07-04T08:10:52.655313+0000","last_clean":"2024-07-04T08:10:52.655313+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:10:52.655313+0000","last_undegraded":"2024-07-04T08:10:52.655313+0000","last_fullsized":"2024-07-04T08:10:52.655313+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.668312+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:10:52.668312+0000","last_peered":"2024-07-04T08:10:52.668312+0000","last_clean":"2024-07-04T08:10:52.668312+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:10:52.668312+0000","last_undegraded":"2024-07-04T08:10:52.668312+0000","last_fullsized":"2024-07-04T08:10:52.668312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657796+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:10:52.657796+0000","last_peered":"2024-07-04T08:10:52.657796+0000","last_clean":"2024-07-04T08:10:52.657796+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:10:52.657796+0000","last_undegraded":"2024-07-04T08:10:52.657796+0000","last_fullsized":"2024-07-04T08:10:52.657796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.673947+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:10:52.673947+0000","last_peered":"2024-07-04T08:10:52.673947+0000","last_clean":"2024-07-04T08:10:52.673947+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:10:52.673947+0000","last_undegraded":"2024-07-04T08:10:52.673947+0000","last_fullsized":"2024-07-04T08:10:52.673947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654560+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:10:52.654560+0000","last_peered":"2024-07-04T08:10:52.654560+0000","last_clean":"2024-07-04T08:10:52.654560+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:10:52.654560+0000","last_undegraded":"2024-07-04T08:10:52.654560+0000","last_fullsized":"2024-07-04T08:10:52.654560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667468+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:10:52.667468+0000","last_peered":"2024-07-04T08:10:52.667468+0000","last_clean":"2024-07-04T08:10:52.667468+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:10:52.667468+0000","last_undegraded":"2024-07-04T08:10:52.667468+0000","last_fullsized":"2024-07-04T08:10:52.667468+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657051+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:10:52.657051+0000","last_peered":"2024-07-04T08:10:52.657051+0000","last_clean":"2024-07-04T08:10:52.657051+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:10:52.657051+0000","last_undegraded":"2024-07-04T08:10:52.657051+0000","last_fullsized":"2024-07-04T08:10:52.657051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.127184+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:10:53.127184+0000","last_peered":"2024-07-04T08:10:53.127184+0000","last_clean":"2024-07-04T08:10:53.127184+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:10:53.127184+0000","last_undegraded":"2024-07-04T08:10:53.127184+0000","last_fullsized":"2024-07-04T08:10:53.127184+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.665735+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:10:52.665735+0000","last_peered":"2024-07-04T08:10:52.665735+0000","last_clean":"2024-07-04T08:10:52.665735+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:10:52.665735+0000","last_undegraded":"2024-07-04T08:10:52.665735+0000","last_fullsized":"2024-07-04T08:10:52.665735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666254+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:10:52.666254+0000","last_peered":"2024-07-04T08:10:52.666254+0000","last_clean":"2024-07-04T08:10:52.666254+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:10:52.666254+0000","last_undegraded":"2024-07-04T08:10:52.666254+0000","last_fullsized":"2024-07-04T08:10:52.666254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658279+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:10:52.658279+0000","last_peered":"2024-07-04T08:10:52.658279+0000","last_clean":"2024-07-04T08:10:52.658279+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:10:52.658279+0000","last_undegraded":"2024-07-04T08:10:52.658279+0000","last_fullsized":"2024-07-04T08:10:52.658279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65087,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":351,"num_read_kb":247,"num_write":245,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":201514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612648,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":524455,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1169545,"data_compressed":63402,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972400,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":528481,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168632,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":524763,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364862,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":1121651,"data_compressed":63402,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528387,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":520280,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1116451,"data_compressed":63402,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":524354,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117075,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":519597,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378714,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":577852,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":586616,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705763,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":521345,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":60010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:11:41.560 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph pg dump --format=json 2024-07-04T08:11:41.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:41 smithi133 ceph-mon[37050]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:11:41.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:41 smithi039 ceph-mon[32062]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:11:41.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:41 smithi050 ceph-mon[36577]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:11:42.090 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:42.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:42 smithi133 ceph-mon[37050]: pgmap v373: 129 pgs: 129 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-07-04T08:11:42.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:42 smithi039 ceph-mon[32062]: pgmap v373: 129 pgs: 129 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-07-04T08:11:42.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:42 smithi050 ceph-mon[36577]: pgmap v373: 129 pgs: 129 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-07-04T08:11:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.186+0000 7fd9cf200640 1 -- 172.21.15.39:0/271512015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 msgr2=0x7fd9c8101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.186+0000 7fd9cf200640 1 --2- 172.21.15.39:0/271512015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 0x7fd9c8101da0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009900 tx=0x7fd9bc02a480 comp rx=0 tx=0).stop 2024-07-04T08:11:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.186+0000 7fd9cf200640 1 -- 172.21.15.39:0/271512015 shutdown_connections 2024-07-04T08:11:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.186+0000 7fd9cf200640 1 --2- 172.21.15.39:0/271512015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9c810f540 0x7fd9c8111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.186+0000 7fd9cf200640 1 --2- 172.21.15.39:0/271512015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9c81022e0 0x7fd9c810f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.186+0000 7fd9cf200640 1 --2- 172.21.15.39:0/271512015 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 0x7fd9c8101da0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.186+0000 7fd9cf200640 1 -- 172.21.15.39:0/271512015 >> 172.21.15.39:0/271512015 conn(0x7fd9c80fd810 msgr2=0x7fd9c80ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.186+0000 7fd9cf200640 1 -- 172.21.15.39:0/271512015 shutdown_connections 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.186+0000 7fd9cf200640 1 -- 172.21.15.39:0/271512015 wait complete. 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.187+0000 7fd9cf200640 1 Processor -- start 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.187+0000 7fd9cf200640 1 -- start start 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.187+0000 7fd9cf200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 0x7fd9c819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cf200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9c81022e0 0x7fd9c819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cf200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9c810f540 0x7fd9c81a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cf200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8114fb0 con 0x7fd9c8101990 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cf200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8114e30 con 0x7fd9c810f540 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cf200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8115130 con 0x7fd9c81022e0 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 0x7fd9c819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:44.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 0x7fd9c819b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54062/0 (socket says 172.21.15.39:54062) 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cde00640 1 -- 172.21.15.39:0/134619420 learned_addr learned my addr 172.21.15.39:0/134619420 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cd400640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9c81022e0 0x7fd9c819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9ce800640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9c810f540 0x7fd9c81a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cde00640 1 -- 172.21.15.39:0/134619420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9c81022e0 msgr2=0x7fd9c819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cde00640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9c81022e0 0x7fd9c819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cde00640 1 -- 172.21.15.39:0/134619420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9c810f540 msgr2=0x7fd9c81a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.188+0000 7fd9cde00640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9c810f540 0x7fd9c81a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.189+0000 7fd9cde00640 1 -- 172.21.15.39:0/134619420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c81a1950 con 0x7fd9c8101990 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.189+0000 7fd9cde00640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 0x7fd9c819b9e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005940 tx=0x7fd9bc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.189+0000 7fd9c2a00640 1 -- 172.21.15.39:0/134619420 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc03f030 con 0x7fd9c8101990 2024-07-04T08:11:44.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.189+0000 7fd9c2a00640 1 -- 172.21.15.39:0/134619420 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9bc0044c0 con 0x7fd9c8101990 2024-07-04T08:11:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.189+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c81a1be0 con 0x7fd9c8101990 2024-07-04T08:11:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.189+0000 7fd9c2a00640 1 -- 172.21.15.39:0/134619420 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc035070 con 0x7fd9c8101990 2024-07-04T08:11:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.189+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81a83a0 con 0x7fd9c8101990 2024-07-04T08:11:44.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.191+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c8101da0 con 0x7fd9c8101990 2024-07-04T08:11:44.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.194+0000 7fd9c2a00640 1 -- 172.21.15.39:0/134619420 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7fd9bc042030 con 0x7fd9c8101990 2024-07-04T08:11:44.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.196+0000 7fd9c2a00640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd9b00b2950 0x7fd9b00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:44.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.196+0000 7fd9c2a00640 1 -- 172.21.15.39:0/134619420 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fd9bc102da0 con 0x7fd9c8101990 2024-07-04T08:11:44.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.196+0000 7fd9cd400640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd9b00b2950 0x7fd9b00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.197+0000 7fd9c2a00640 1 -- 172.21.15.39:0/134619420 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fd9bc0c9e80 con 0x7fd9c8101990 2024-07-04T08:11:44.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.197+0000 7fd9cd400640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd9b00b2950 0x7fd9b00b4e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8110820 tx=0x7fd9b8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:44.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.429+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd9c8062d50 con 0x7fd9b00b2950 2024-07-04T08:11:44.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.442+0000 7fd9c2a00640 1 -- 172.21.15.39:0/134619420 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340696 (secure 0 0 0) 0x7fd9c8062d50 con 0x7fd9b00b2950 2024-07-04T08:11:44.448 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:11:44.450 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:11:44.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.447+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd9b00b2950 msgr2=0x7fd9b00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:44.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.447+0000 7fd9cf200640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd9b00b2950 0x7fd9b00b4e60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8110820 tx=0x7fd9b8005f90 comp rx=0 tx=0).stop 2024-07-04T08:11:44.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.447+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 msgr2=0x7fd9c819b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:44.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.447+0000 7fd9cf200640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 0x7fd9c819b9e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005940 tx=0x7fd9bc0040c0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.448+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 shutdown_connections 2024-07-04T08:11:44.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.448+0000 7fd9cf200640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd9b00b2950 0x7fd9b00b4e60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.448+0000 7fd9cf200640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd9c810f540 0x7fd9c81a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.448+0000 7fd9cf200640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9c81022e0 0x7fd9c819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.448+0000 7fd9cf200640 1 --2- 172.21.15.39:0/134619420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9c8101990 0x7fd9c819b9e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:44.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.448+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 >> 172.21.15.39:0/134619420 conn(0x7fd9c80fd810 msgr2=0x7fd9c81116b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:44.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.448+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 shutdown_connections 2024-07-04T08:11:44.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:44.448+0000 7fd9cf200640 1 -- 172.21.15.39:0/134619420 wait complete. 2024-07-04T08:11:44.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:44.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:44.817 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:44 smithi133 ceph-mon[37050]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-07-04T08:11:44.829 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:44.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:44.830 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:44 smithi039 ceph-mon[32062]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-07-04T08:11:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:44.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:44 smithi050 ceph-mon[36577]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-07-04T08:11:45.188 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-07-04T08:11:43.572117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661942,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":478,"num_read_kb":348,"num_write":374,"num_write_kb":2295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373892,"kb_used_data":12260,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567932,"statfs":{"total":1151940427776,"available":1151557562368,"internally_reserved":0,"allocated":12554240,"data_stored":8247896,"data_compressed":428013,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19657,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":126,"num_read_kb":89,"num_write":11,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006886"},"pg_stats":[{"pgid":"2.1f","version":"90'13","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:38.951859+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:11:38.951859+0000","last_peered":"2024-07-04T08:11:38.951859+0000","last_clean":"2024-07-04T08:11:38.951859+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:11:38.951859+0000","last_undegraded":"2024-07-04T08:11:38.951859+0000","last_fullsized":"2024-07-04T08:11:38.951859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679679+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:10:52.679679+0000","last_peered":"2024-07-04T08:10:52.679679+0000","last_clean":"2024-07-04T08:10:52.679679+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:10:52.679679+0000","last_undegraded":"2024-07-04T08:10:52.679679+0000","last_fullsized":"2024-07-04T08:10:52.679679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654299+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:10:52.654299+0000","last_peered":"2024-07-04T08:10:52.654299+0000","last_clean":"2024-07-04T08:10:52.654299+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:10:52.654299+0000","last_undegraded":"2024-07-04T08:10:52.654299+0000","last_fullsized":"2024-07-04T08:10:52.654299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:02.798150+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:02.798150+0000","last_peered":"2024-07-04T08:11:02.798150+0000","last_clean":"2024-07-04T08:11:02.798150+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:02.798150+0000","last_undegraded":"2024-07-04T08:11:02.798150+0000","last_fullsized":"2024-07-04T08:11:02.798150+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.671323+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:10:52.671323+0000","last_peered":"2024-07-04T08:10:52.671323+0000","last_clean":"2024-07-04T08:10:52.671323+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:10:52.671323+0000","last_undegraded":"2024-07-04T08:10:52.671323+0000","last_fullsized":"2024-07-04T08:10:52.671323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.125619+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:10:53.125619+0000","last_peered":"2024-07-04T08:10:53.125619+0000","last_clean":"2024-07-04T08:10:53.125619+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:10:53.125619+0000","last_undegraded":"2024-07-04T08:10:53.125619+0000","last_fullsized":"2024-07-04T08:10:53.125619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:05.866831+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:11:05.866831+0000","last_peered":"2024-07-04T08:11:05.866831+0000","last_clean":"2024-07-04T08:11:05.866831+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:11:05.866831+0000","last_undegraded":"2024-07-04T08:11:05.866831+0000","last_fullsized":"2024-07-04T08:11:05.866831+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657420+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:10:52.657420+0000","last_peered":"2024-07-04T08:10:52.657420+0000","last_clean":"2024-07-04T08:10:52.657420+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:10:52.657420+0000","last_undegraded":"2024-07-04T08:10:52.657420+0000","last_fullsized":"2024-07-04T08:10:52.657420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126090+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:10:53.126090+0000","last_peered":"2024-07-04T08:10:53.126090+0000","last_clean":"2024-07-04T08:10:53.126090+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:10:53.126090+0000","last_undegraded":"2024-07-04T08:10:53.126090+0000","last_fullsized":"2024-07-04T08:10:53.126090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.859030+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:11:04.859030+0000","last_peered":"2024-07-04T08:11:04.859030+0000","last_clean":"2024-07-04T08:11:04.859030+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:11:04.859030+0000","last_undegraded":"2024-07-04T08:11:04.859030+0000","last_fullsized":"2024-07-04T08:11:04.859030+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667481+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:10:52.667481+0000","last_peered":"2024-07-04T08:10:52.667481+0000","last_clean":"2024-07-04T08:10:52.667481+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:10:52.667481+0000","last_undegraded":"2024-07-04T08:10:52.667481+0000","last_fullsized":"2024-07-04T08:10:52.667481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.125946+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:10:53.125946+0000","last_peered":"2024-07-04T08:10:53.125946+0000","last_clean":"2024-07-04T08:10:53.125946+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:10:53.125946+0000","last_undegraded":"2024-07-04T08:10:53.125946+0000","last_fullsized":"2024-07-04T08:10:53.125946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654646+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:10:52.654646+0000","last_peered":"2024-07-04T08:10:52.654646+0000","last_clean":"2024-07-04T08:10:52.654646+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:10:52.654646+0000","last_undegraded":"2024-07-04T08:10:52.654646+0000","last_fullsized":"2024-07-04T08:10:52.654646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:01.788372+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:01.788372+0000","last_peered":"2024-07-04T08:11:01.788372+0000","last_clean":"2024-07-04T08:11:01.788372+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:01.788372+0000","last_undegraded":"2024-07-04T08:11:01.788372+0000","last_fullsized":"2024-07-04T08:11:01.788372+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126268+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:10:53.126268+0000","last_peered":"2024-07-04T08:10:53.126268+0000","last_clean":"2024-07-04T08:10:53.126268+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:10:53.126268+0000","last_undegraded":"2024-07-04T08:10:53.126268+0000","last_fullsized":"2024-07-04T08:10:53.126268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.677266+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:10:52.677266+0000","last_peered":"2024-07-04T08:10:52.677266+0000","last_clean":"2024-07-04T08:10:52.677266+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:10:52.677266+0000","last_undegraded":"2024-07-04T08:10:52.677266+0000","last_fullsized":"2024-07-04T08:10:52.677266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:03.827675+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:03.827675+0000","last_peered":"2024-07-04T08:11:03.827675+0000","last_clean":"2024-07-04T08:11:03.827675+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:03.827675+0000","last_undegraded":"2024-07-04T08:11:03.827675+0000","last_fullsized":"2024-07-04T08:11:03.827675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126260+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:10:53.126260+0000","last_peered":"2024-07-04T08:10:53.126260+0000","last_clean":"2024-07-04T08:10:53.126260+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:10:53.126260+0000","last_undegraded":"2024-07-04T08:10:53.126260+0000","last_fullsized":"2024-07-04T08:10:53.126260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658428+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:10:52.658428+0000","last_peered":"2024-07-04T08:10:52.658428+0000","last_clean":"2024-07-04T08:10:52.658428+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:10:52.658428+0000","last_undegraded":"2024-07-04T08:10:52.658428+0000","last_fullsized":"2024-07-04T08:10:52.658428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.863662+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:04.863662+0000","last_peered":"2024-07-04T08:11:04.863662+0000","last_clean":"2024-07-04T08:11:04.863662+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:04.863662+0000","last_undegraded":"2024-07-04T08:11:04.863662+0000","last_fullsized":"2024-07-04T08:11:04.863662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.682686+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:10:52.682686+0000","last_peered":"2024-07-04T08:10:52.682686+0000","last_clean":"2024-07-04T08:10:52.682686+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:10:52.682686+0000","last_undegraded":"2024-07-04T08:10:52.682686+0000","last_fullsized":"2024-07-04T08:10:52.682686+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.656952+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:10:52.656952+0000","last_peered":"2024-07-04T08:10:52.656952+0000","last_clean":"2024-07-04T08:10:52.656952+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:10:52.656952+0000","last_undegraded":"2024-07-04T08:10:52.656952+0000","last_fullsized":"2024-07-04T08:10:52.656952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:01.787685+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:01.787685+0000","last_peered":"2024-07-04T08:11:01.787685+0000","last_clean":"2024-07-04T08:11:01.787685+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:01.787685+0000","last_undegraded":"2024-07-04T08:11:01.787685+0000","last_fullsized":"2024-07-04T08:11:01.787685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126202+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:10:53.126202+0000","last_peered":"2024-07-04T08:10:53.126202+0000","last_clean":"2024-07-04T08:10:53.126202+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:10:53.126202+0000","last_undegraded":"2024-07-04T08:10:53.126202+0000","last_fullsized":"2024-07-04T08:10:53.126202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666589+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:10:52.666589+0000","last_peered":"2024-07-04T08:10:52.666589+0000","last_clean":"2024-07-04T08:10:52.666589+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:10:52.666589+0000","last_undegraded":"2024-07-04T08:10:52.666589+0000","last_fullsized":"2024-07-04T08:10:52.666589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.859642+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:04.859642+0000","last_peered":"2024-07-04T08:11:04.859642+0000","last_clean":"2024-07-04T08:11:04.859642+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:04.859642+0000","last_undegraded":"2024-07-04T08:11:04.859642+0000","last_fullsized":"2024-07-04T08:11:04.859642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654445+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:10:52.654445+0000","last_peered":"2024-07-04T08:10:52.654445+0000","last_clean":"2024-07-04T08:10:52.654445+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:10:52.654445+0000","last_undegraded":"2024-07-04T08:10:52.654445+0000","last_fullsized":"2024-07-04T08:10:52.654445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679271+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:10:52.679271+0000","last_peered":"2024-07-04T08:10:52.679271+0000","last_clean":"2024-07-04T08:10:52.679271+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:10:52.679271+0000","last_undegraded":"2024-07-04T08:10:52.679271+0000","last_fullsized":"2024-07-04T08:10:52.679271+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:31.427247+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:11:31.427247+0000","last_peered":"2024-07-04T08:11:31.427247+0000","last_clean":"2024-07-04T08:11:31.427247+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:11:31.427247+0000","last_undegraded":"2024-07-04T08:11:31.427247+0000","last_fullsized":"2024-07-04T08:11:31.427247+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.674576+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:10:52.674576+0000","last_peered":"2024-07-04T08:10:52.674576+0000","last_clean":"2024-07-04T08:10:52.674576+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:10:52.674576+0000","last_undegraded":"2024-07-04T08:10:52.674576+0000","last_fullsized":"2024-07-04T08:10:52.674576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.681613+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:10:52.681613+0000","last_peered":"2024-07-04T08:10:52.681613+0000","last_clean":"2024-07-04T08:10:52.681613+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:10:52.681613+0000","last_undegraded":"2024-07-04T08:10:52.681613+0000","last_fullsized":"2024-07-04T08:10:52.681613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:05.866881+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:05.866881+0000","last_peered":"2024-07-04T08:11:05.866881+0000","last_clean":"2024-07-04T08:11:05.866881+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:05.866881+0000","last_undegraded":"2024-07-04T08:11:05.866881+0000","last_fullsized":"2024-07-04T08:11:05.866881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658117+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:10:52.658117+0000","last_peered":"2024-07-04T08:10:52.658117+0000","last_clean":"2024-07-04T08:10:52.658117+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:10:52.658117+0000","last_undegraded":"2024-07-04T08:10:52.658117+0000","last_fullsized":"2024-07-04T08:10:52.658117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667859+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:10:52.667859+0000","last_peered":"2024-07-04T08:10:52.667859+0000","last_clean":"2024-07-04T08:10:52.667859+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:10:52.667859+0000","last_undegraded":"2024-07-04T08:10:52.667859+0000","last_fullsized":"2024-07-04T08:10:52.667859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:03.833262+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:11:03.833262+0000","last_peered":"2024-07-04T08:11:03.833262+0000","last_clean":"2024-07-04T08:11:03.833262+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:11:03.833262+0000","last_undegraded":"2024-07-04T08:11:03.833262+0000","last_fullsized":"2024-07-04T08:11:03.833262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657044+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:10:52.657044+0000","last_peered":"2024-07-04T08:10:52.657044+0000","last_clean":"2024-07-04T08:10:52.657044+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:10:52.657044+0000","last_undegraded":"2024-07-04T08:10:52.657044+0000","last_fullsized":"2024-07-04T08:10:52.657044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.676059+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:10:52.676059+0000","last_peered":"2024-07-04T08:10:52.676059+0000","last_clean":"2024-07-04T08:10:52.676059+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:10:52.676059+0000","last_undegraded":"2024-07-04T08:10:52.676059+0000","last_fullsized":"2024-07-04T08:10:52.676059+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":85,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:40.957096+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:11:40.957096+0000","last_peered":"2024-07-04T08:11:40.957096+0000","last_clean":"2024-07-04T08:11:40.957096+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:11:40.957096+0000","last_undegraded":"2024-07-04T08:11:40.957096+0000","last_fullsized":"2024-07-04T08:11:40.957096+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.681326+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:10:52.681326+0000","last_peered":"2024-07-04T08:10:52.681326+0000","last_clean":"2024-07-04T08:10:52.681326+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:10:52.681326+0000","last_undegraded":"2024-07-04T08:10:52.681326+0000","last_fullsized":"2024-07-04T08:10:52.681326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667098+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:10:52.667098+0000","last_peered":"2024-07-04T08:10:52.667098+0000","last_clean":"2024-07-04T08:10:52.667098+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:10:52.667098+0000","last_undegraded":"2024-07-04T08:10:52.667098+0000","last_fullsized":"2024-07-04T08:10:52.667098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":85,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:42.961219+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:11:42.961219+0000","last_peered":"2024-07-04T08:11:42.961219+0000","last_clean":"2024-07-04T08:11:42.961219+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:11:42.961219+0000","last_undegraded":"2024-07-04T08:11:42.961219+0000","last_fullsized":"2024-07-04T08:11:42.961219+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.681978+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:10:52.681978+0000","last_peered":"2024-07-04T08:10:52.681978+0000","last_clean":"2024-07-04T08:10:52.681978+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:10:52.681978+0000","last_undegraded":"2024-07-04T08:10:52.681978+0000","last_fullsized":"2024-07-04T08:10:52.681978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679249+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:10:52.679249+0000","last_peered":"2024-07-04T08:10:52.679249+0000","last_clean":"2024-07-04T08:10:52.679249+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:10:52.679249+0000","last_undegraded":"2024-07-04T08:10:52.679249+0000","last_fullsized":"2024-07-04T08:10:52.679249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.857965+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:04.857965+0000","last_peered":"2024-07-04T08:11:04.857965+0000","last_clean":"2024-07-04T08:11:04.857965+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:04.857965+0000","last_undegraded":"2024-07-04T08:11:04.857965+0000","last_fullsized":"2024-07-04T08:11:04.857965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.684905+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:10:52.684905+0000","last_peered":"2024-07-04T08:10:52.684905+0000","last_clean":"2024-07-04T08:10:52.684905+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:10:52.684905+0000","last_undegraded":"2024-07-04T08:10:52.684905+0000","last_fullsized":"2024-07-04T08:10:52.684905+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.678368+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:10:52.678368+0000","last_peered":"2024-07-04T08:10:52.678368+0000","last_clean":"2024-07-04T08:10:52.678368+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:10:52.678368+0000","last_undegraded":"2024-07-04T08:10:52.678368+0000","last_fullsized":"2024-07-04T08:10:52.678368+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:05.867278+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:05.867278+0000","last_peered":"2024-07-04T08:11:05.867278+0000","last_clean":"2024-07-04T08:11:05.867278+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:05.867278+0000","last_undegraded":"2024-07-04T08:11:05.867278+0000","last_fullsized":"2024-07-04T08:11:05.867278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":114,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655606+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:10:52.655606+0000","last_peered":"2024-07-04T08:10:52.655606+0000","last_clean":"2024-07-04T08:10:52.655606+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:10:52.655606+0000","last_undegraded":"2024-07-04T08:10:52.655606+0000","last_fullsized":"2024-07-04T08:10:52.655606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655236+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:10:52.655236+0000","last_peered":"2024-07-04T08:10:52.655236+0000","last_clean":"2024-07-04T08:10:52.655236+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:10:52.655236+0000","last_undegraded":"2024-07-04T08:10:52.655236+0000","last_fullsized":"2024-07-04T08:10:52.655236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655336+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:10:52.655336+0000","last_peered":"2024-07-04T08:10:52.655336+0000","last_clean":"2024-07-04T08:10:52.655336+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:10:52.655336+0000","last_undegraded":"2024-07-04T08:10:52.655336+0000","last_fullsized":"2024-07-04T08:10:52.655336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:37.915140+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:11:37.915140+0000","last_peered":"2024-07-04T08:11:37.915140+0000","last_clean":"2024-07-04T08:11:37.915140+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:11:37.915140+0000","last_undegraded":"2024-07-04T08:11:37.915140+0000","last_fullsized":"2024-07-04T08:11:37.915140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.656979+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:10:52.656979+0000","last_peered":"2024-07-04T08:10:52.656979+0000","last_clean":"2024-07-04T08:10:52.656979+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:10:52.656979+0000","last_undegraded":"2024-07-04T08:10:52.656979+0000","last_fullsized":"2024-07-04T08:10:52.656979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654965+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:10:52.654965+0000","last_peered":"2024-07-04T08:10:52.654965+0000","last_clean":"2024-07-04T08:10:52.654965+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:10:52.654965+0000","last_undegraded":"2024-07-04T08:10:52.654965+0000","last_fullsized":"2024-07-04T08:10:52.654965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.863668+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:04.863668+0000","last_peered":"2024-07-04T08:11:04.863668+0000","last_clean":"2024-07-04T08:11:04.863668+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:11:04.863668+0000","last_undegraded":"2024-07-04T08:11:04.863668+0000","last_fullsized":"2024-07-04T08:11:04.863668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.663648+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:10:52.663648+0000","last_peered":"2024-07-04T08:10:52.663648+0000","last_clean":"2024-07-04T08:10:52.663648+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:10:52.663648+0000","last_undegraded":"2024-07-04T08:10:52.663648+0000","last_fullsized":"2024-07-04T08:10:52.663648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654983+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:10:52.654983+0000","last_peered":"2024-07-04T08:10:52.654983+0000","last_clean":"2024-07-04T08:10:52.654983+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:10:52.654983+0000","last_undegraded":"2024-07-04T08:10:52.654983+0000","last_fullsized":"2024-07-04T08:10:52.654983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.864527+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:04.864527+0000","last_peered":"2024-07-04T08:11:04.864527+0000","last_clean":"2024-07-04T08:11:04.864527+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:04.864527+0000","last_undegraded":"2024-07-04T08:11:04.864527+0000","last_fullsized":"2024-07-04T08:11:04.864527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666623+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:10:52.666623+0000","last_peered":"2024-07-04T08:10:52.666623+0000","last_clean":"2024-07-04T08:10:52.666623+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:10:52.666623+0000","last_undegraded":"2024-07-04T08:10:52.666623+0000","last_fullsized":"2024-07-04T08:10:52.666623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658305+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:10:52.658305+0000","last_peered":"2024-07-04T08:10:52.658305+0000","last_clean":"2024-07-04T08:10:52.658305+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:10:52.658305+0000","last_undegraded":"2024-07-04T08:10:52.658305+0000","last_fullsized":"2024-07-04T08:10:52.658305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.785269+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:04.785269+0000","last_peered":"2024-07-04T08:11:04.785269+0000","last_clean":"2024-07-04T08:11:04.785269+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:04.785269+0000","last_undegraded":"2024-07-04T08:11:04.785269+0000","last_fullsized":"2024-07-04T08:11:04.785269+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666830+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:10:52.666830+0000","last_peered":"2024-07-04T08:10:52.666830+0000","last_clean":"2024-07-04T08:10:52.666830+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:10:52.666830+0000","last_undegraded":"2024-07-04T08:10:52.666830+0000","last_fullsized":"2024-07-04T08:10:52.666830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.128942+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:10:53.128942+0000","last_peered":"2024-07-04T08:10:53.128942+0000","last_clean":"2024-07-04T08:10:53.128942+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:10:53.128942+0000","last_undegraded":"2024-07-04T08:10:53.128942+0000","last_fullsized":"2024-07-04T08:10:53.128942+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.864722+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:04.864722+0000","last_peered":"2024-07-04T08:11:04.864722+0000","last_clean":"2024-07-04T08:11:04.864722+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:04.864722+0000","last_undegraded":"2024-07-04T08:11:04.864722+0000","last_fullsized":"2024-07-04T08:11:04.864722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.681787+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:10:52.681787+0000","last_peered":"2024-07-04T08:10:52.681787+0000","last_clean":"2024-07-04T08:10:52.681787+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:10:52.681787+0000","last_undegraded":"2024-07-04T08:10:52.681787+0000","last_fullsized":"2024-07-04T08:10:52.681787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654382+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:10:52.654382+0000","last_peered":"2024-07-04T08:10:52.654382+0000","last_clean":"2024-07-04T08:10:52.654382+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:10:52.654382+0000","last_undegraded":"2024-07-04T08:10:52.654382+0000","last_fullsized":"2024-07-04T08:10:52.654382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:38.953290+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:11:38.953290+0000","last_peered":"2024-07-04T08:11:38.953290+0000","last_clean":"2024-07-04T08:11:38.953290+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:11:38.953290+0000","last_undegraded":"2024-07-04T08:11:38.953290+0000","last_fullsized":"2024-07-04T08:11:38.953290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.664560+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:10:52.664560+0000","last_peered":"2024-07-04T08:10:52.664560+0000","last_clean":"2024-07-04T08:10:52.664560+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:10:52.664560+0000","last_undegraded":"2024-07-04T08:10:52.664560+0000","last_fullsized":"2024-07-04T08:10:52.664560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654582+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:10:52.654582+0000","last_peered":"2024-07-04T08:10:52.654582+0000","last_clean":"2024-07-04T08:10:52.654582+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:10:52.654582+0000","last_undegraded":"2024-07-04T08:10:52.654582+0000","last_fullsized":"2024-07-04T08:10:52.654582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.863668+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:04.863668+0000","last_peered":"2024-07-04T08:11:04.863668+0000","last_clean":"2024-07-04T08:11:04.863668+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:04.863668+0000","last_undegraded":"2024-07-04T08:11:04.863668+0000","last_fullsized":"2024-07-04T08:11:04.863668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.683951+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:10:52.683951+0000","last_peered":"2024-07-04T08:10:52.683951+0000","last_clean":"2024-07-04T08:10:52.683951+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:10:52.683951+0000","last_undegraded":"2024-07-04T08:10:52.683951+0000","last_fullsized":"2024-07-04T08:10:52.683951+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.677112+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:10:52.677112+0000","last_peered":"2024-07-04T08:10:52.677112+0000","last_clean":"2024-07-04T08:10:52.677112+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:10:52.677112+0000","last_undegraded":"2024-07-04T08:10:52.677112+0000","last_fullsized":"2024-07-04T08:10:52.677112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:05.867006+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:05.867006+0000","last_peered":"2024-07-04T08:11:05.867006+0000","last_clean":"2024-07-04T08:11:05.867006+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:05.867006+0000","last_undegraded":"2024-07-04T08:11:05.867006+0000","last_fullsized":"2024-07-04T08:11:05.867006+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667447+0000","last_change":"2024-07-04T08:09:51.640257+0000","last_active":"2024-07-04T08:10:52.667447+0000","last_peered":"2024-07-04T08:10:52.667447+0000","last_clean":"2024-07-04T08:10:52.667447+0000","last_became_active":"2024-07-04T08:09:51.639733+0000","last_became_peered":"2024-07-04T08:09:51.639733+0000","last_unstale":"2024-07-04T08:10:52.667447+0000","last_undegraded":"2024-07-04T08:10:52.667447+0000","last_fullsized":"2024-07-04T08:10:52.667447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:53:40.655524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657958+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:10:52.657958+0000","last_peered":"2024-07-04T08:10:52.657958+0000","last_clean":"2024-07-04T08:10:52.657958+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:10:52.657958+0000","last_undegraded":"2024-07-04T08:10:52.657958+0000","last_fullsized":"2024-07-04T08:10:52.657958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'35","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:40.927865+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:11:40.927865+0000","last_peered":"2024-07-04T08:11:40.927865+0000","last_clean":"2024-07-04T08:11:40.927865+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:11:40.927865+0000","last_undegraded":"2024-07-04T08:11:40.927865+0000","last_fullsized":"2024-07-04T08:11:40.927865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64136,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":81,"num_read_kb":103,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.662287+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:10:52.662287+0000","last_peered":"2024-07-04T08:10:52.662287+0000","last_clean":"2024-07-04T08:10:52.662287+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:10:52.662287+0000","last_undegraded":"2024-07-04T08:10:52.662287+0000","last_fullsized":"2024-07-04T08:10:52.662287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657501+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:10:52.657501+0000","last_peered":"2024-07-04T08:10:52.657501+0000","last_clean":"2024-07-04T08:10:52.657501+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:10:52.657501+0000","last_undegraded":"2024-07-04T08:10:52.657501+0000","last_fullsized":"2024-07-04T08:10:52.657501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:03.822478+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:03.822478+0000","last_peered":"2024-07-04T08:11:03.822478+0000","last_clean":"2024-07-04T08:11:03.822478+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:03.822478+0000","last_undegraded":"2024-07-04T08:11:03.822478+0000","last_fullsized":"2024-07-04T08:11:03.822478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.126373+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:10:53.126373+0000","last_peered":"2024-07-04T08:10:53.126373+0000","last_clean":"2024-07-04T08:10:53.126373+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:10:53.126373+0000","last_undegraded":"2024-07-04T08:10:53.126373+0000","last_fullsized":"2024-07-04T08:10:53.126373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.671698+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:10:52.671698+0000","last_peered":"2024-07-04T08:10:52.671698+0000","last_clean":"2024-07-04T08:10:52.671698+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:10:52.671698+0000","last_undegraded":"2024-07-04T08:10:52.671698+0000","last_fullsized":"2024-07-04T08:10:52.671698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.857299+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:11:04.857299+0000","last_peered":"2024-07-04T08:11:04.857299+0000","last_clean":"2024-07-04T08:11:04.857299+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:11:04.857299+0000","last_undegraded":"2024-07-04T08:11:04.857299+0000","last_fullsized":"2024-07-04T08:11:04.857299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.661322+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:10:52.661322+0000","last_peered":"2024-07-04T08:10:52.661322+0000","last_clean":"2024-07-04T08:10:52.661322+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:10:52.661322+0000","last_undegraded":"2024-07-04T08:10:52.661322+0000","last_fullsized":"2024-07-04T08:10:52.661322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657635+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:10:52.657635+0000","last_peered":"2024-07-04T08:10:52.657635+0000","last_clean":"2024-07-04T08:10:52.657635+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:10:52.657635+0000","last_undegraded":"2024-07-04T08:10:52.657635+0000","last_fullsized":"2024-07-04T08:10:52.657635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:40.917926+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:11:40.917926+0000","last_peered":"2024-07-04T08:11:40.917926+0000","last_clean":"2024-07-04T08:11:40.917926+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:11:40.917926+0000","last_undegraded":"2024-07-04T08:11:40.917926+0000","last_fullsized":"2024-07-04T08:11:40.917926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.677936+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:10:52.677936+0000","last_peered":"2024-07-04T08:10:52.677936+0000","last_clean":"2024-07-04T08:10:52.677936+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:10:52.677936+0000","last_undegraded":"2024-07-04T08:10:52.677936+0000","last_fullsized":"2024-07-04T08:10:52.677936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.665672+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:10:52.665672+0000","last_peered":"2024-07-04T08:10:52.665672+0000","last_clean":"2024-07-04T08:10:52.665672+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:10:52.665672+0000","last_undegraded":"2024-07-04T08:10:52.665672+0000","last_fullsized":"2024-07-04T08:10:52.665672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.665987+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:10:52.665987+0000","last_peered":"2024-07-04T08:10:52.665987+0000","last_clean":"2024-07-04T08:10:52.665987+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:10:52.665987+0000","last_undegraded":"2024-07-04T08:10:52.665987+0000","last_fullsized":"2024-07-04T08:10:52.665987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:04.857911+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:04.857911+0000","last_peered":"2024-07-04T08:11:04.857911+0000","last_clean":"2024-07-04T08:11:04.857911+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:04.857911+0000","last_undegraded":"2024-07-04T08:11:04.857911+0000","last_fullsized":"2024-07-04T08:11:04.857911+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657875+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:10:52.657875+0000","last_peered":"2024-07-04T08:10:52.657875+0000","last_clean":"2024-07-04T08:10:52.657875+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:10:52.657875+0000","last_undegraded":"2024-07-04T08:10:52.657875+0000","last_fullsized":"2024-07-04T08:10:52.657875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.674274+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:10:52.674274+0000","last_peered":"2024-07-04T08:10:52.674274+0000","last_clean":"2024-07-04T08:10:52.674274+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:10:52.674274+0000","last_undegraded":"2024-07-04T08:10:52.674274+0000","last_fullsized":"2024-07-04T08:10:52.674274+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:02.798151+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:11:02.798151+0000","last_peered":"2024-07-04T08:11:02.798151+0000","last_clean":"2024-07-04T08:11:02.798151+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:11:02.798151+0000","last_undegraded":"2024-07-04T08:11:02.798151+0000","last_fullsized":"2024-07-04T08:11:02.798151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:03.833435+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:11:03.833435+0000","last_peered":"2024-07-04T08:11:03.833435+0000","last_clean":"2024-07-04T08:11:03.833435+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:11:03.833435+0000","last_undegraded":"2024-07-04T08:11:03.833435+0000","last_fullsized":"2024-07-04T08:11:03.833435+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667051+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:10:52.667051+0000","last_peered":"2024-07-04T08:10:52.667051+0000","last_clean":"2024-07-04T08:10:52.667051+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:10:52.667051+0000","last_undegraded":"2024-07-04T08:10:52.667051+0000","last_fullsized":"2024-07-04T08:10:52.667051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654630+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:10:52.654630+0000","last_peered":"2024-07-04T08:10:52.654630+0000","last_clean":"2024-07-04T08:10:52.654630+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:10:52.654630+0000","last_undegraded":"2024-07-04T08:10:52.654630+0000","last_fullsized":"2024-07-04T08:10:52.654630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":82,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:11:38.917197+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:11:38.917197+0000","last_peered":"2024-07-04T08:11:38.917197+0000","last_clean":"2024-07-04T08:11:38.917197+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:11:38.917197+0000","last_undegraded":"2024-07-04T08:11:38.917197+0000","last_fullsized":"2024-07-04T08:11:38.917197+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657655+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:10:52.657655+0000","last_peered":"2024-07-04T08:10:52.657655+0000","last_clean":"2024-07-04T08:10:52.657655+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:10:52.657655+0000","last_undegraded":"2024-07-04T08:10:52.657655+0000","last_fullsized":"2024-07-04T08:10:52.657655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654821+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:10:52.654821+0000","last_peered":"2024-07-04T08:10:52.654821+0000","last_clean":"2024-07-04T08:10:52.654821+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:10:52.654821+0000","last_undegraded":"2024-07-04T08:10:52.654821+0000","last_fullsized":"2024-07-04T08:10:52.654821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.656924+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:10:52.656924+0000","last_peered":"2024-07-04T08:10:52.656924+0000","last_clean":"2024-07-04T08:10:52.656924+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:10:52.656924+0000","last_undegraded":"2024-07-04T08:10:52.656924+0000","last_fullsized":"2024-07-04T08:10:52.656924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654767+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:10:52.654767+0000","last_peered":"2024-07-04T08:10:52.654767+0000","last_clean":"2024-07-04T08:10:52.654767+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:10:52.654767+0000","last_undegraded":"2024-07-04T08:10:52.654767+0000","last_fullsized":"2024-07-04T08:10:52.654767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654388+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:10:52.654388+0000","last_peered":"2024-07-04T08:10:52.654388+0000","last_clean":"2024-07-04T08:10:52.654388+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:10:52.654388+0000","last_undegraded":"2024-07-04T08:10:52.654388+0000","last_fullsized":"2024-07-04T08:10:52.654388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.664171+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:10:52.664171+0000","last_peered":"2024-07-04T08:10:52.664171+0000","last_clean":"2024-07-04T08:10:52.664171+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:10:52.664171+0000","last_undegraded":"2024-07-04T08:10:52.664171+0000","last_fullsized":"2024-07-04T08:10:52.664171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.685603+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:10:52.685603+0000","last_peered":"2024-07-04T08:10:52.685603+0000","last_clean":"2024-07-04T08:10:52.685603+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:10:52.685603+0000","last_undegraded":"2024-07-04T08:10:52.685603+0000","last_fullsized":"2024-07-04T08:10:52.685603+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657589+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:10:52.657589+0000","last_peered":"2024-07-04T08:10:52.657589+0000","last_clean":"2024-07-04T08:10:52.657589+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:10:52.657589+0000","last_undegraded":"2024-07-04T08:10:52.657589+0000","last_fullsized":"2024-07-04T08:10:52.657589+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655145+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:10:52.655145+0000","last_peered":"2024-07-04T08:10:52.655145+0000","last_clean":"2024-07-04T08:10:52.655145+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:10:52.655145+0000","last_undegraded":"2024-07-04T08:10:52.655145+0000","last_fullsized":"2024-07-04T08:10:52.655145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679089+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:10:52.679089+0000","last_peered":"2024-07-04T08:10:52.679089+0000","last_clean":"2024-07-04T08:10:52.679089+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:10:52.679089+0000","last_undegraded":"2024-07-04T08:10:52.679089+0000","last_fullsized":"2024-07-04T08:10:52.679089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666957+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:10:52.666957+0000","last_peered":"2024-07-04T08:10:52.666957+0000","last_clean":"2024-07-04T08:10:52.666957+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:10:52.666957+0000","last_undegraded":"2024-07-04T08:10:52.666957+0000","last_fullsized":"2024-07-04T08:10:52.666957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.678494+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:10:52.678494+0000","last_peered":"2024-07-04T08:10:52.678494+0000","last_clean":"2024-07-04T08:10:52.678494+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:10:52.678494+0000","last_undegraded":"2024-07-04T08:10:52.678494+0000","last_fullsized":"2024-07-04T08:10:52.678494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657262+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:10:52.657262+0000","last_peered":"2024-07-04T08:10:52.657262+0000","last_clean":"2024-07-04T08:10:52.657262+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:10:52.657262+0000","last_undegraded":"2024-07-04T08:10:52.657262+0000","last_fullsized":"2024-07-04T08:10:52.657262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679568+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:10:52.679568+0000","last_peered":"2024-07-04T08:10:52.679568+0000","last_clean":"2024-07-04T08:10:52.679568+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:10:52.679568+0000","last_undegraded":"2024-07-04T08:10:52.679568+0000","last_fullsized":"2024-07-04T08:10:52.679568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657856+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:10:52.657856+0000","last_peered":"2024-07-04T08:10:52.657856+0000","last_clean":"2024-07-04T08:10:52.657856+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:10:52.657856+0000","last_undegraded":"2024-07-04T08:10:52.657856+0000","last_fullsized":"2024-07-04T08:10:52.657856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654973+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:10:52.654973+0000","last_peered":"2024-07-04T08:10:52.654973+0000","last_clean":"2024-07-04T08:10:52.654973+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:10:52.654973+0000","last_undegraded":"2024-07-04T08:10:52.654973+0000","last_fullsized":"2024-07-04T08:10:52.654973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.664377+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:10:52.664377+0000","last_peered":"2024-07-04T08:10:52.664377+0000","last_clean":"2024-07-04T08:10:52.664377+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:10:52.664377+0000","last_undegraded":"2024-07-04T08:10:52.664377+0000","last_fullsized":"2024-07-04T08:10:52.664377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658461+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:10:52.658461+0000","last_peered":"2024-07-04T08:10:52.658461+0000","last_clean":"2024-07-04T08:10:52.658461+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:10:52.658461+0000","last_undegraded":"2024-07-04T08:10:52.658461+0000","last_fullsized":"2024-07-04T08:10:52.658461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.656760+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:10:52.656760+0000","last_peered":"2024-07-04T08:10:52.656760+0000","last_clean":"2024-07-04T08:10:52.656760+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:10:52.656760+0000","last_undegraded":"2024-07-04T08:10:52.656760+0000","last_fullsized":"2024-07-04T08:10:52.656760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.662263+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:10:52.662263+0000","last_peered":"2024-07-04T08:10:52.662263+0000","last_clean":"2024-07-04T08:10:52.662263+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:10:52.662263+0000","last_undegraded":"2024-07-04T08:10:52.662263+0000","last_fullsized":"2024-07-04T08:10:52.662263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.663082+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:10:52.663082+0000","last_peered":"2024-07-04T08:10:52.663082+0000","last_clean":"2024-07-04T08:10:52.663082+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:10:52.663082+0000","last_undegraded":"2024-07-04T08:10:52.663082+0000","last_fullsized":"2024-07-04T08:10:52.663082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.679412+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:10:52.679412+0000","last_peered":"2024-07-04T08:10:52.679412+0000","last_clean":"2024-07-04T08:10:52.679412+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:10:52.679412+0000","last_undegraded":"2024-07-04T08:10:52.679412+0000","last_fullsized":"2024-07-04T08:10:52.679412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.127162+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:10:53.127162+0000","last_peered":"2024-07-04T08:10:53.127162+0000","last_clean":"2024-07-04T08:10:53.127162+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:10:53.127162+0000","last_undegraded":"2024-07-04T08:10:53.127162+0000","last_fullsized":"2024-07-04T08:10:53.127162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.655313+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:10:52.655313+0000","last_peered":"2024-07-04T08:10:52.655313+0000","last_clean":"2024-07-04T08:10:52.655313+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:10:52.655313+0000","last_undegraded":"2024-07-04T08:10:52.655313+0000","last_fullsized":"2024-07-04T08:10:52.655313+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.668312+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:10:52.668312+0000","last_peered":"2024-07-04T08:10:52.668312+0000","last_clean":"2024-07-04T08:10:52.668312+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:10:52.668312+0000","last_undegraded":"2024-07-04T08:10:52.668312+0000","last_fullsized":"2024-07-04T08:10:52.668312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657796+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:10:52.657796+0000","last_peered":"2024-07-04T08:10:52.657796+0000","last_clean":"2024-07-04T08:10:52.657796+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:10:52.657796+0000","last_undegraded":"2024-07-04T08:10:52.657796+0000","last_fullsized":"2024-07-04T08:10:52.657796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.673947+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:10:52.673947+0000","last_peered":"2024-07-04T08:10:52.673947+0000","last_clean":"2024-07-04T08:10:52.673947+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:10:52.673947+0000","last_undegraded":"2024-07-04T08:10:52.673947+0000","last_fullsized":"2024-07-04T08:10:52.673947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.654560+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:10:52.654560+0000","last_peered":"2024-07-04T08:10:52.654560+0000","last_clean":"2024-07-04T08:10:52.654560+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:10:52.654560+0000","last_undegraded":"2024-07-04T08:10:52.654560+0000","last_fullsized":"2024-07-04T08:10:52.654560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.667468+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:10:52.667468+0000","last_peered":"2024-07-04T08:10:52.667468+0000","last_clean":"2024-07-04T08:10:52.667468+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:10:52.667468+0000","last_undegraded":"2024-07-04T08:10:52.667468+0000","last_fullsized":"2024-07-04T08:10:52.667468+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.657051+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:10:52.657051+0000","last_peered":"2024-07-04T08:10:52.657051+0000","last_clean":"2024-07-04T08:10:52.657051+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:10:52.657051+0000","last_undegraded":"2024-07-04T08:10:52.657051+0000","last_fullsized":"2024-07-04T08:10:52.657051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:53.127252+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:10:53.127252+0000","last_peered":"2024-07-04T08:10:53.127252+0000","last_clean":"2024-07-04T08:10:53.127252+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:10:53.127252+0000","last_undegraded":"2024-07-04T08:10:53.127252+0000","last_fullsized":"2024-07-04T08:10:53.127252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.665735+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:10:52.665735+0000","last_peered":"2024-07-04T08:10:52.665735+0000","last_clean":"2024-07-04T08:10:52.665735+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:10:52.665735+0000","last_undegraded":"2024-07-04T08:10:52.665735+0000","last_fullsized":"2024-07-04T08:10:52.665735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.666254+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:10:52.666254+0000","last_peered":"2024-07-04T08:10:52.666254+0000","last_clean":"2024-07-04T08:10:52.666254+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:10:52.666254+0000","last_undegraded":"2024-07-04T08:10:52.666254+0000","last_fullsized":"2024-07-04T08:10:52.666254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-07-04T08:10:52.658279+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:10:52.658279+0000","last_peered":"2024-07-04T08:10:52.658279+0000","last_clean":"2024-07-04T08:10:52.658279+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:10:52.658279+0000","last_undegraded":"2024-07-04T08:10:52.658279+0000","last_fullsized":"2024-07-04T08:10:52.658279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71340,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":397,"num_read_kb":279,"num_write":249,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612649,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":524455,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808876,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1175798,"data_compressed":63402,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":528481,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168633,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":524763,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364863,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":1121651,"data_compressed":63402,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528388,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":520280,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1116451,"data_compressed":63402,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":524354,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117075,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":519597,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378715,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":584105,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542239,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":586616,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705764,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":521345,"data_compressed":26423,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:11:45.193 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-04T08:11:45.194 INFO:tasks.cephadm:Setting up client nodes... 2024-07-04T08:11:45.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-04T08:11:45.687 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:45.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:45 smithi133 ceph-mon[37050]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:11:45.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:11:45.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:11:45.816 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:45 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:45.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:45 smithi039 ceph-mon[32062]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:11:45.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:11:45.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:11:45.842 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:45 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:45.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:45 smithi050 ceph-mon[36577]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:11:45.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:11:45.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:11:45.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:45 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:11:46.764 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:46 smithi039 ceph-mon[32062]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 852 B/s wr, 10 op/s 2024-07-04T08:11:46.818 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:46 smithi133 ceph-mon[37050]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 852 B/s wr, 10 op/s 2024-07-04T08:11:46.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:46 smithi050 ceph-mon[36577]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 852 B/s wr, 10 op/s 2024-07-04T08:11:48.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.176+0000 7fbe27a00640 1 -- 172.21.15.39:0/1885247246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe20102510 msgr2=0x7fbe2010f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:48.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.176+0000 7fbe27a00640 1 --2- 172.21.15.39:0/1885247246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe20102510 0x7fbe2010f230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009860 tx=0x7fbe0c02a3d0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.176+0000 7fbe27a00640 1 -- 172.21.15.39:0/1885247246 shutdown_connections 2024-07-04T08:11:48.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.176+0000 7fbe27a00640 1 --2- 172.21.15.39:0/1885247246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2010f770 0x7fbe20111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.176+0000 7fbe27a00640 1 --2- 172.21.15.39:0/1885247246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe20102510 0x7fbe2010f230 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.176+0000 7fbe27a00640 1 --2- 172.21.15.39:0/1885247246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe20101bc0 0x7fbe20101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.176+0000 7fbe27a00640 1 -- 172.21.15.39:0/1885247246 >> 172.21.15.39:0/1885247246 conn(0x7fbe20078cb0 msgr2=0x7fbe200790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:48.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.176+0000 7fbe27a00640 1 -- 172.21.15.39:0/1885247246 shutdown_connections 2024-07-04T08:11:48.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.176+0000 7fbe27a00640 1 -- 172.21.15.39:0/1885247246 wait complete. 2024-07-04T08:11:48.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.178+0000 7fbe27a00640 1 Processor -- start 2024-07-04T08:11:48.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.178+0000 7fbe27a00640 1 -- start start 2024-07-04T08:11:48.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.179+0000 7fbe27a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe20101bc0 0x7fbe2019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:48.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.179+0000 7fbe27a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe20102510 0x7fbe2019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:48.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.179+0000 7fbe27a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2010f770 0x7fbe201a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:48.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.179+0000 7fbe27a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201151e0 con 0x7fbe2010f770 2024-07-04T08:11:48.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.179+0000 7fbe27a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20115060 con 0x7fbe20102510 2024-07-04T08:11:48.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.179+0000 7fbe27a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20115360 con 0x7fbe20101bc0 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.179+0000 7fbe25a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2010f770 0x7fbe201a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.180+0000 7fbe25a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2010f770 0x7fbe201a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54078/0 (socket says 172.21.15.39:54078) 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.180+0000 7fbe25a00640 1 -- 172.21.15.39:0/900806089 learned_addr learned my addr 172.21.15.39:0/900806089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.179+0000 7fbe1fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe20102510 0x7fbe2019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.180+0000 7fbe25000640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe20101bc0 0x7fbe2019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.180+0000 7fbe25000640 1 -- 172.21.15.39:0/900806089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe20102510 msgr2=0x7fbe2019c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.180+0000 7fbe25000640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe20102510 0x7fbe2019c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.180+0000 7fbe25000640 1 -- 172.21.15.39:0/900806089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2010f770 msgr2=0x7fbe201a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.180+0000 7fbe25000640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2010f770 0x7fbe201a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.180+0000 7fbe25000640 1 -- 172.21.15.39:0/900806089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe201a1b00 con 0x7fbe20101bc0 2024-07-04T08:11:48.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.181+0000 7fbe25000640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe20101bc0 0x7fbe2019bb90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e9f0 tx=0x7fbe1400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:48.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.181+0000 7fbe1d600640 1 -- 172.21.15.39:0/900806089 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1400cd80 con 0x7fbe20101bc0 2024-07-04T08:11:48.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.181+0000 7fbe1d600640 1 -- 172.21.15.39:0/900806089 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe140035f0 con 0x7fbe20101bc0 2024-07-04T08:11:48.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.181+0000 7fbe1d600640 1 -- 172.21.15.39:0/900806089 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14010670 con 0x7fbe20101bc0 2024-07-04T08:11:48.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.181+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe201a1df0 con 0x7fbe20101bc0 2024-07-04T08:11:48.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.181+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe201a8550 con 0x7fbe20101bc0 2024-07-04T08:11:48.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.182+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe20101fd0 con 0x7fbe20101bc0 2024-07-04T08:11:48.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.185+0000 7fbe1d600640 1 -- 172.21.15.39:0/900806089 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7fbe140037a0 con 0x7fbe20101bc0 2024-07-04T08:11:48.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.187+0000 7fbe1d600640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe080b28d0 0x7fbe080b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:48.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.187+0000 7fbe1fe00640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe080b28d0 0x7fbe080b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:48.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.188+0000 7fbe1d600640 1 -- 172.21.15.39:0/900806089 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7fbe140e2ce0 con 0x7fbe20101bc0 2024-07-04T08:11:48.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.188+0000 7fbe1d600640 1 -- 172.21.15.39:0/900806089 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fbe14055110 con 0x7fbe20101bc0 2024-07-04T08:11:48.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.189+0000 7fbe1fe00640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe080b28d0 0x7fbe080b4de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02a9c0 tx=0x7fbe0c0058b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:48.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.480+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fbe201a8890 con 0x7fbe20101bc0 2024-07-04T08:11:48.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.493+0000 7fbe1d600640 1 -- 172.21.15.39:0/900806089 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7fbe140aa410 con 0x7fbe20101bc0 2024-07-04T08:11:48.494 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2024-07-04T08:11:48.494 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBEWYZmh7zSHBAAiiwovsi2rZM75U0z/Jmh1Q== 2024-07-04T08:11:48.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe080b28d0 msgr2=0x7fbe080b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:48.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe080b28d0 0x7fbe080b4de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02a9c0 tx=0x7fbe0c0058b0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe20101bc0 msgr2=0x7fbe2019bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe20101bc0 0x7fbe2019bb90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400e9f0 tx=0x7fbe1400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 shutdown_connections 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe080b28d0 0x7fbe080b4de0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe2010f770 0x7fbe201a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbe20102510 0x7fbe2019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 --2- 172.21.15.39:0/900806089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe20101bc0 0x7fbe2019bb90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 >> 172.21.15.39:0/900806089 conn(0x7fbe20078cb0 msgr2=0x7fbe20109280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 shutdown_connections 2024-07-04T08:11:48.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:48.496+0000 7fbe27a00640 1 -- 172.21.15.39:0/900806089 wait complete. 2024-07-04T08:11:48.845 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:48 smithi039 ceph-mon[32062]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 852 B/s wr, 11 op/s 2024-07-04T08:11:48.845 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:48 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/900806089' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-04T08:11:48.845 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:48 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-04T08:11:48.845 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:48 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-04T08:11:48.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:48 smithi050 ceph-mon[36577]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 852 B/s wr, 11 op/s 2024-07-04T08:11:48.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:48 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/900806089' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-04T08:11:48.895 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:48 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-04T08:11:48.896 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:48 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-04T08:11:49.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:48 smithi133 ceph-mon[37050]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 852 B/s wr, 11 op/s 2024-07-04T08:11:49.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:48 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/900806089' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-04T08:11:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:48 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-04T08:11:49.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:48 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-04T08:11:49.222 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:11:49.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-07-04T08:11:49.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-04T08:11:49.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph config log 1 --format=json 2024-07-04T08:11:49.894 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:50.894 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:50 smithi050 ceph-mon[36577]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-07-04T08:11:51.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:50 smithi133 ceph-mon[37050]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-07-04T08:11:51.082 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:50 smithi039 ceph-mon[32062]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-07-04T08:11:52.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.022+0000 7f0cb9200640 1 -- 172.21.15.39:0/1844264531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb4074ca0 msgr2=0x7f0cb40750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:52.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.022+0000 7f0cb9200640 1 --2- 172.21.15.39:0/1844264531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb4074ca0 0x7f0cb40750b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009900 tx=0x7f0c9c02a400 comp rx=0 tx=0).stop 2024-07-04T08:11:52.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.023+0000 7f0cb9200640 1 -- 172.21.15.39:0/1844264531 shutdown_connections 2024-07-04T08:11:52.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.023+0000 7f0cb9200640 1 --2- 172.21.15.39:0/1844264531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cb410f690 0x7f0cb4111ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.023+0000 7f0cb9200640 1 --2- 172.21.15.39:0/1844264531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cb40755f0 0x7f0cb410f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.023+0000 7f0cb9200640 1 --2- 172.21.15.39:0/1844264531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb4074ca0 0x7f0cb40750b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.023+0000 7f0cb9200640 1 -- 172.21.15.39:0/1844264531 >> 172.21.15.39:0/1844264531 conn(0x7f0cb40feb60 msgr2=0x7f0cb4100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:52.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.023+0000 7f0cb9200640 1 -- 172.21.15.39:0/1844264531 shutdown_connections 2024-07-04T08:11:52.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.023+0000 7f0cb9200640 1 -- 172.21.15.39:0/1844264531 wait complete. 2024-07-04T08:11:52.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.024+0000 7f0cb9200640 1 Processor -- start 2024-07-04T08:11:52.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.025+0000 7f0cb9200640 1 -- start start 2024-07-04T08:11:52.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.025+0000 7f0cb9200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cb4074ca0 0x7f0cb419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:52.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.025+0000 7f0cb9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cb40755f0 0x7f0cb419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:52.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.025+0000 7f0cb9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb410f690 0x7f0cb41a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:52.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.025+0000 7f0cb9200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb41151f0 con 0x7f0cb40755f0 2024-07-04T08:11:52.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.025+0000 7f0cb9200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb4115070 con 0x7f0cb410f690 2024-07-04T08:11:52.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.025+0000 7f0cb9200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb4115370 con 0x7f0cb4074ca0 2024-07-04T08:11:52.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cb4074ca0 0x7f0cb419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:52.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb410f690 0x7f0cb41a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cb4074ca0 0x7f0cb419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:55766/0 (socket says 172.21.15.39:55766) 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb410f690 0x7f0cb41a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38568/0 (socket says 172.21.15.39:38568) 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb1e00640 1 -- 172.21.15.39:0/3821186455 learned_addr learned my addr 172.21.15.39:0/3821186455 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb1400640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cb40755f0 0x7f0cb419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb2800640 1 -- 172.21.15.39:0/3821186455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cb4074ca0 msgr2=0x7f0cb419bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb2800640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cb4074ca0 0x7f0cb419bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb2800640 1 -- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cb40755f0 msgr2=0x7f0cb419c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb2800640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cb40755f0 0x7f0cb419c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb2800640 1 -- 172.21.15.39:0/3821186455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb41a1c30 con 0x7f0cb410f690 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.026+0000 7f0cb2800640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb410f690 0x7f0cb41a1500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d6b0 tx=0x7f0cac00db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.027+0000 7f0caaa00640 1 -- 172.21.15.39:0/3821186455 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac017070 con 0x7f0cb410f690 2024-07-04T08:11:52.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.027+0000 7f0caaa00640 1 -- 172.21.15.39:0/3821186455 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cac009cd0 con 0x7f0cb410f690 2024-07-04T08:11:52.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.027+0000 7f0caaa00640 1 -- 172.21.15.39:0/3821186455 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac003cc0 con 0x7f0cb410f690 2024-07-04T08:11:52.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.027+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb41a1f20 con 0x7f0cb410f690 2024-07-04T08:11:52.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.027+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb41a8680 con 0x7f0cb410f690 2024-07-04T08:11:52.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.028+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb40761b0 con 0x7f0cb410f690 2024-07-04T08:11:52.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.030+0000 7f0caaa00640 1 -- 172.21.15.39:0/3821186455 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f0cac009820 con 0x7f0cb410f690 2024-07-04T08:11:52.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.032+0000 7f0caaa00640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c980b2950 0x7f0c980b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:52.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.032+0000 7f0caaa00640 1 -- 172.21.15.39:0/3821186455 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9307+0+0 (secure 0 0 0) 0x7f0cac0e2070 con 0x7f0cb410f690 2024-07-04T08:11:52.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.032+0000 7f0cb1e00640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c980b2950 0x7f0c980b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:52.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.033+0000 7f0cb1e00640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c980b2950 0x7f0c980b4e60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c031440 tx=0x7f0c9c0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:52.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.036+0000 7f0caaa00640 1 -- 172.21.15.39:0/3821186455 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f0cac0a97a0 con 0x7f0cb410f690 2024-07-04T08:11:52.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.251+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f0cb419cdf0 con 0x7f0cb410f690 2024-07-04T08:11:52.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.252+0000 7f0caaa00640 1 -- 172.21.15.39:0/3821186455 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v55) ==== 86+0+133 (secure 0 0 0) 0x7f0cac0a90a0 con 0x7f0cb410f690 2024-07-04T08:11:52.255 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:11:52.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.255+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c980b2950 msgr2=0x7f0c980b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:52.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.255+0000 7f0cb9200640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c980b2950 0x7f0c980b4e60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c031440 tx=0x7f0c9c0058d0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.255+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb410f690 msgr2=0x7f0cb41a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:52.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.255+0000 7f0cb9200640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb410f690 0x7f0cb41a1500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d6b0 tx=0x7f0cac00db70 comp rx=0 tx=0).stop 2024-07-04T08:11:52.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.255+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 shutdown_connections 2024-07-04T08:11:52.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.255+0000 7f0cb9200640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c980b2950 0x7f0c980b4e60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.255+0000 7f0cb9200640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cb40755f0 0x7f0cb419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.255+0000 7f0cb9200640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0cb410f690 0x7f0cb41a1500 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.255+0000 7f0cb9200640 1 --2- 172.21.15.39:0/3821186455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0cb4074ca0 0x7f0cb419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:52.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.256+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 >> 172.21.15.39:0/3821186455 conn(0x7f0cb40feb60 msgr2=0x7f0cb4109230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:52.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.256+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 shutdown_connections 2024-07-04T08:11:52.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:52.256+0000 7f0cb9200640 1 -- 172.21.15.39:0/3821186455 wait complete. 2024-07-04T08:11:52.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:52 smithi039 ceph-mon[32062]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-04T08:11:52.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-07-04T08:11:52.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:52 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3821186455' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-04T08:11:53.040 INFO:teuthology.orchestra.run.smithi039.stdout:[{"version":55,"timestamp":"2024-07-04T08:09:22.492770+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-07-04T08:11:53.046 INFO:tasks.ceph_manager:config epoch is 55 2024-07-04T08:11:53.047 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-04T08:11:53.047 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-07-04T08:11:53.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph mgr dump --format=json 2024-07-04T08:11:53.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:52 smithi133 ceph-mon[37050]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-04T08:11:53.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-07-04T08:11:53.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:52 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3821186455' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-04T08:11:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:52 smithi050 ceph-mon[36577]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-04T08:11:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]: dispatch 2024-07-04T08:11:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:52 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3821186455' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-04T08:11:53.530 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:53.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-07-04T08:11:53.954 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:53 smithi039 ceph-mon[32062]: osdmap e91: 12 total, 12 up, 12 in 2024-07-04T08:11:54.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-07-04T08:11:54.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:53 smithi133 ceph-mon[37050]: osdmap e91: 12 total, 12 up, 12 in 2024-07-04T08:11:54.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.15", "id": [5, 10]}]': finished 2024-07-04T08:11:54.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:53 smithi050 ceph-mon[36577]: osdmap e91: 12 total, 12 up, 12 in 2024-07-04T08:11:54.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:54 smithi039 ceph-mon[32062]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:11:54.825 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:54 smithi039 ceph-mon[32062]: osdmap e92: 12 total, 12 up, 12 in 2024-07-04T08:11:55.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:54 smithi133 ceph-mon[37050]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:11:55.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:54 smithi133 ceph-mon[37050]: osdmap e92: 12 total, 12 up, 12 in 2024-07-04T08:11:55.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:54 smithi050 ceph-mon[36577]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:11:55.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:54 smithi050 ceph-mon[36577]: osdmap e92: 12 total, 12 up, 12 in 2024-07-04T08:11:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.754+0000 7f123b200640 1 -- 172.21.15.39:0/2398887787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 msgr2=0x7f123c103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.754+0000 7f123b200640 1 --2- 172.21.15.39:0/2398887787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 0x7f123c103310 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f122c009900 tx=0x7f122c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:11:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.754+0000 7f123b200640 1 -- 172.21.15.39:0/2398887787 shutdown_connections 2024-07-04T08:11:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.754+0000 7f123b200640 1 --2- 172.21.15.39:0/2398887787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f123c109190 0x7f123c10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.754+0000 7f123b200640 1 --2- 172.21.15.39:0/2398887787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c108840 0x7f123c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:55.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.754+0000 7f123b200640 1 --2- 172.21.15.39:0/2398887787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 0x7f123c103310 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:55.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.754+0000 7f123b200640 1 -- 172.21.15.39:0/2398887787 >> 172.21.15.39:0/2398887787 conn(0x7f123c0fe990 msgr2=0x7f123c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:55.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.754+0000 7f123b200640 1 -- 172.21.15.39:0/2398887787 shutdown_connections 2024-07-04T08:11:55.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.754+0000 7f123b200640 1 -- 172.21.15.39:0/2398887787 wait complete. 2024-07-04T08:11:55.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.756+0000 7f123b200640 1 Processor -- start 2024-07-04T08:11:55.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.756+0000 7f123b200640 1 -- start start 2024-07-04T08:11:55.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.756+0000 7f123b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 0x7f123c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:55.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.756+0000 7f123b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f123c108840 0x7f123c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:55.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.756+0000 7f123b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c109190 0x7f123c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:55.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.756+0000 7f123b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f123c112ea0 con 0x7f123c102f00 2024-07-04T08:11:55.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.756+0000 7f123b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f123c112d20 con 0x7f123c108840 2024-07-04T08:11:55.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.756+0000 7f123b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f123c113020 con 0x7f123c109190 2024-07-04T08:11:55.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f1239e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 0x7f123c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:55.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f123a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c109190 0x7f123c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:55.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f1233e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f123c108840 0x7f123c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:55.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f123a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c109190 0x7f123c1a1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:55790/0 (socket says 172.21.15.39:55790) 2024-07-04T08:11:55.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f123a800640 1 -- 172.21.15.39:0/2751662330 learned_addr learned my addr 172.21.15.39:0/2751662330 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:55.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f1239e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 0x7f123c19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52646/0 (socket says 172.21.15.39:52646) 2024-07-04T08:11:55.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f1233e00640 1 -- 172.21.15.39:0/2751662330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c109190 msgr2=0x7f123c1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:55.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f1233e00640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c109190 0x7f123c1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:55.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f1233e00640 1 -- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 msgr2=0x7f123c19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:55.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f1233e00640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 0x7f123c19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:55.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f1233e00640 1 -- 172.21.15.39:0/2751662330 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c1a19c0 con 0x7f123c108840 2024-07-04T08:11:55.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.757+0000 7f1239e00640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 0x7f123c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:11:55.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.758+0000 7f1233e00640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f123c108840 0x7f123c19bff0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f123c06aee0 tx=0x7f122400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:55.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.758+0000 7f1232a00640 1 -- 172.21.15.39:0/2751662330 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122400cd00 con 0x7f123c108840 2024-07-04T08:11:55.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.758+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123c1a1cb0 con 0x7f123c108840 2024-07-04T08:11:55.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.758+0000 7f1232a00640 1 -- 172.21.15.39:0/2751662330 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12240035f0 con 0x7f123c108840 2024-07-04T08:11:55.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.758+0000 7f1232a00640 1 -- 172.21.15.39:0/2751662330 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224010690 con 0x7f123c108840 2024-07-04T08:11:55.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.758+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123c1a8410 con 0x7f123c108840 2024-07-04T08:11:55.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.760+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123c103310 con 0x7f123c108840 2024-07-04T08:11:55.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.762+0000 7f1232a00640 1 -- 172.21.15.39:0/2751662330 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f12240037a0 con 0x7f123c108840 2024-07-04T08:11:55.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.764+0000 7f1232a00640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f12200b28d0 0x7f12200b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:55.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.764+0000 7f1232a00640 1 -- 172.21.15.39:0/2751662330 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f12240e2af0 con 0x7f123c108840 2024-07-04T08:11:55.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.764+0000 7f1239e00640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f12200b28d0 0x7f12200b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:55.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.765+0000 7f1239e00640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f12200b28d0 0x7f12200b4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f122c005ab0 tx=0x7f122c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:55.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:55.767+0000 7f1232a00640 1 -- 172.21.15.39:0/2751662330 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f12240aa200 con 0x7f123c108840 2024-07-04T08:11:56.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.044+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f123c1a8710 con 0x7f123c108840 2024-07-04T08:11:56.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.051+0000 7f1232a00640 1 -- 172.21.15.39:0/2751662330 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+288662 (secure 0 0 0) 0x7f12240a9b00 con 0x7f123c108840 2024-07-04T08:11:56.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.058+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f12200b28d0 msgr2=0x7f12200b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.058+0000 7f123b200640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f12200b28d0 0x7f12200b4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f122c005ab0 tx=0x7f122c005a20 comp rx=0 tx=0).stop 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.058+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f123c108840 msgr2=0x7f123c19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.058+0000 7f123b200640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f123c108840 0x7f123c19bff0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f123c06aee0 tx=0x7f122400ee50 comp rx=0 tx=0).stop 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.058+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 shutdown_connections 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.058+0000 7f123b200640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f12200b28d0 0x7f12200b4de0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.059+0000 7f123b200640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c109190 0x7f123c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.059+0000 7f123b200640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f123c108840 0x7f123c19bff0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.059+0000 7f123b200640 1 --2- 172.21.15.39:0/2751662330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f123c102f00 0x7f123c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.059+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 >> 172.21.15.39:0/2751662330 conn(0x7f123c0fe990 msgr2=0x7f123c107ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.059+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 shutdown_connections 2024-07-04T08:11:56.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:56.059+0000 7f123b200640 1 -- 172.21.15.39:0/2751662330 wait complete. 2024-07-04T08:11:56.826 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":21,"flags":0,"active_gid":14214,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":3120140278},{"type":"v1","addr":"172.21.15.39:6801","nonce":3120140278}]},"active_addr":"172.21.15.39:6801/3120140278","active_change":"2024-07-04T08:01:51.344333+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24148,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24161,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.39:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":2623988781}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":1811411189}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":4259770207}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":757586727}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3257545225}]}]} 2024-07-04T08:11:56.833 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-07-04T08:11:56.833 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-07-04T08:11:56.834 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:11:56.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:56 smithi039 ceph-mon[32062]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-07-04T08:11:56.955 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:56 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2751662330' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-04T08:11:57.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:56 smithi133 ceph-mon[37050]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-07-04T08:11:57.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:56 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2751662330' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-04T08:11:57.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:56 smithi050 ceph-mon[36577]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-07-04T08:11:57.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:56 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2751662330' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-04T08:11:57.359 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:11:59.026 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:11:58 smithi039 ceph-mon[32062]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-07-04T08:11:59.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:11:58 smithi133 ceph-mon[37050]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-07-04T08:11:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:11:58 smithi050 ceph-mon[36577]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-07-04T08:11:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.600+0000 7f0666600640 1 -- 172.21.15.39:0/2991741343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f066010aa50 msgr2=0x7f066010ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.600+0000 7f0666600640 1 --2- 172.21.15.39:0/2991741343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f066010aa50 0x7f066010ae60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f064c009ca0 tx=0x7f064c02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.600+0000 7f0666600640 1 -- 172.21.15.39:0/2991741343 shutdown_connections 2024-07-04T08:11:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.600+0000 7f0666600640 1 --2- 172.21.15.39:0/2991741343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f066010aa50 0x7f066010ae60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.600+0000 7f0666600640 1 --2- 172.21.15.39:0/2991741343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066010b3a0 0x7f0660111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.600+0000 7f0666600640 1 --2- 172.21.15.39:0/2991741343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0660105110 0x7f0660105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.600+0000 7f0666600640 1 -- 172.21.15.39:0/2991741343 >> 172.21.15.39:0/2991741343 conn(0x7f0660100b80 msgr2=0x7f0660102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.600+0000 7f0666600640 1 -- 172.21.15.39:0/2991741343 shutdown_connections 2024-07-04T08:11:59.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.600+0000 7f0666600640 1 -- 172.21.15.39:0/2991741343 wait complete. 2024-07-04T08:11:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.601+0000 7f0666600640 1 Processor -- start 2024-07-04T08:11:59.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0666600640 1 -- start start 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0666600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0660105110 0x7f066019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0666600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f066010aa50 0x7f066019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0666600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066010b3a0 0x7f06601a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0666600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0660115010 con 0x7f0660105110 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0666600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0660114e90 con 0x7f066010aa50 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0666600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0660115190 con 0x7f066010b3a0 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0665200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0660105110 0x7f066019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0665200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0660105110 0x7f066019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52672/0 (socket says 172.21.15.39:52672) 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.602+0000 7f0665200640 1 -- 172.21.15.39:0/2643153023 learned_addr learned my addr 172.21.15.39:0/2643153023 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.603+0000 7f065fe00640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f066010aa50 0x7f066019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:59.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.603+0000 7f0665c00640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066010b3a0 0x7f06601a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.603+0000 7f0665200640 1 -- 172.21.15.39:0/2643153023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066010b3a0 msgr2=0x7f06601a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.603+0000 7f0665200640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066010b3a0 0x7f06601a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.603+0000 7f0665200640 1 -- 172.21.15.39:0/2643153023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f066010aa50 msgr2=0x7f066019c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.603+0000 7f0665200640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f066010aa50 0x7f066019c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.603+0000 7f0665200640 1 -- 172.21.15.39:0/2643153023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06601a19d0 con 0x7f0660105110 2024-07-04T08:11:59.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.603+0000 7f0665200640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0660105110 0x7f066019ba60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f065000e9f0 tx=0x7f065000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:59.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.604+0000 7f065d600640 1 -- 172.21.15.39:0/2643153023 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065000cd80 con 0x7f0660105110 2024-07-04T08:11:59.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.604+0000 7f065d600640 1 -- 172.21.15.39:0/2643153023 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06500108e0 con 0x7f0660105110 2024-07-04T08:11:59.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.604+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06601a1cc0 con 0x7f0660105110 2024-07-04T08:11:59.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.604+0000 7f065d600640 1 -- 172.21.15.39:0/2643153023 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650018980 con 0x7f0660105110 2024-07-04T08:11:59.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.604+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06601a8420 con 0x7f0660105110 2024-07-04T08:11:59.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.605+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0660105520 con 0x7f0660105110 2024-07-04T08:11:59.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.608+0000 7f065d600640 1 -- 172.21.15.39:0/2643153023 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f0650010430 con 0x7f0660105110 2024-07-04T08:11:59.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.609+0000 7f065d600640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f06440b28d0 0x7f06440b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:11:59.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.610+0000 7f065d600640 1 -- 172.21.15.39:0/2643153023 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f06500e2bd0 con 0x7f0660105110 2024-07-04T08:11:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.610+0000 7f065fe00640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f06440b28d0 0x7f06440b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:11:59.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.610+0000 7f065fe00640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f06440b28d0 0x7f06440b4de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f064c000f80 tx=0x7f064c009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:11:59.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.614+0000 7f065d600640 1 -- 172.21.15.39:0/2643153023 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f06500a9c90 con 0x7f0660105110 2024-07-04T08:11:59.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.834+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f066010cbc0 con 0x7f0660105110 2024-07-04T08:11:59.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.840+0000 7f065d600640 1 -- 172.21.15.39:0/2643153023 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24957 (secure 0 0 0) 0x7f06500adf90 con 0x7f0660105110 2024-07-04T08:11:59.843 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:11:59.843 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:11:53.672125+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:11:59.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f06440b28d0 msgr2=0x7f06440b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:59.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f06440b28d0 0x7f06440b4de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f064c000f80 tx=0x7f064c009360 comp rx=0 tx=0).stop 2024-07-04T08:11:59.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0660105110 msgr2=0x7f066019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:11:59.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0660105110 0x7f066019ba60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f065000e9f0 tx=0x7f065000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 shutdown_connections 2024-07-04T08:11:59.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f06440b28d0 0x7f06440b4de0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f066010b3a0 0x7f06601a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f066010aa50 0x7f066019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 --2- 172.21.15.39:0/2643153023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0660105110 0x7f066019ba60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:11:59.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 >> 172.21.15.39:0/2643153023 conn(0x7f0660100b80 msgr2=0x7f0660102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:11:59.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 shutdown_connections 2024-07-04T08:11:59.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:11:59.844+0000 7f0666600640 1 -- 172.21.15.39:0/2643153023 wait complete. 2024-07-04T08:12:00.649 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-07-04T08:12:00.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:12:00.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:00 smithi039 ceph-mon[32062]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-07-04T08:12:00.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:00 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2643153023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:12:01.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:00 smithi133 ceph-mon[37050]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-07-04T08:12:01.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:00 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2643153023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:12:01.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:00 smithi050 ceph-mon[36577]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-07-04T08:12:01.146 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:00 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2643153023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:12:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:02.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:02 smithi039 ceph-mon[32062]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-07-04T08:12:03.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:02 smithi133 ceph-mon[37050]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-07-04T08:12:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:02 smithi050 ceph-mon[36577]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-07-04T08:12:03.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.333+0000 7f986da00640 1 -- 172.21.15.39:0/1884655750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f986810ac40 msgr2=0x7f986810b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:03.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.333+0000 7f986da00640 1 --2- 172.21.15.39:0/1884655750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f986810ac40 0x7f986810b050 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9850009900 tx=0x7f985002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.334+0000 7f986da00640 1 -- 172.21.15.39:0/1884655750 shutdown_connections 2024-07-04T08:12:03.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.334+0000 7f986da00640 1 --2- 172.21.15.39:0/1884655750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f986810b590 0x7f9868111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.334+0000 7f986da00640 1 --2- 172.21.15.39:0/1884655750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f986810ac40 0x7f986810b050 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.334+0000 7f986da00640 1 --2- 172.21.15.39:0/1884655750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9868105300 0x7f9868105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.334+0000 7f986da00640 1 -- 172.21.15.39:0/1884655750 >> 172.21.15.39:0/1884655750 conn(0x7f9868100d10 msgr2=0x7f9868103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:03.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.334+0000 7f986da00640 1 -- 172.21.15.39:0/1884655750 shutdown_connections 2024-07-04T08:12:03.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.334+0000 7f986da00640 1 -- 172.21.15.39:0/1884655750 wait complete. 2024-07-04T08:12:03.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.335+0000 7f986da00640 1 Processor -- start 2024-07-04T08:12:03.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.335+0000 7f986da00640 1 -- start start 2024-07-04T08:12:03.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f986da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9868105300 0x7f986819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:03.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f986da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f986810ac40 0x7f986819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:03.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f986da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986810b590 0x7f98681a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:03.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f986da00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9868115200 con 0x7f986810b590 2024-07-04T08:12:03.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f986da00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9868115080 con 0x7f986810ac40 2024-07-04T08:12:03.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f986da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9868115380 con 0x7f9868105300 2024-07-04T08:12:03.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f9867400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986810b590 0x7f98681a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:03.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f9866a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9868105300 0x7f986819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:03.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f9866a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9868105300 0x7f986819bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39686/0 (socket says 172.21.15.39:39686) 2024-07-04T08:12:03.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f9866a00640 1 -- 172.21.15.39:0/2585261975 learned_addr learned my addr 172.21.15.39:0/2585261975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:03.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.336+0000 7f9866000640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f986810ac40 0x7f986819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:03.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.337+0000 7f9866a00640 1 -- 172.21.15.39:0/2585261975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f986810ac40 msgr2=0x7f986819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:03.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.337+0000 7f9866a00640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f986810ac40 0x7f986819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.337+0000 7f9866a00640 1 -- 172.21.15.39:0/2585261975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986810b590 msgr2=0x7f98681a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:03.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.337+0000 7f9866a00640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986810b590 0x7f98681a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.337+0000 7f9866a00640 1 -- 172.21.15.39:0/2585261975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98681a1c40 con 0x7f9868105300 2024-07-04T08:12:03.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.337+0000 7f9866a00640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9868105300 0x7f986819bcd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f985800ea60 tx=0x7f985800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:03.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.338+0000 7f985f400640 1 -- 172.21.15.39:0/2585261975 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985800cde0 con 0x7f9868105300 2024-07-04T08:12:03.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.338+0000 7f985f400640 1 -- 172.21.15.39:0/2585261975 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98580035f0 con 0x7f9868105300 2024-07-04T08:12:03.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.338+0000 7f985f400640 1 -- 172.21.15.39:0/2585261975 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858010690 con 0x7f9868105300 2024-07-04T08:12:03.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.338+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98681a1f30 con 0x7f9868105300 2024-07-04T08:12:03.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.338+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a8690 con 0x7f9868105300 2024-07-04T08:12:03.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.339+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9868105710 con 0x7f9868105300 2024-07-04T08:12:03.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.342+0000 7f985f400640 1 -- 172.21.15.39:0/2585261975 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f98580037a0 con 0x7f9868105300 2024-07-04T08:12:03.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.343+0000 7f985f400640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f984c0b2950 0x7f984c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:03.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.344+0000 7f985f400640 1 -- 172.21.15.39:0/2585261975 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f98580e2ae0 con 0x7f9868105300 2024-07-04T08:12:03.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.344+0000 7f9866000640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f984c0b2950 0x7f984c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:03.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.345+0000 7f9866000640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f984c0b2950 0x7f984c0b4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9850009230 tx=0x7f9850031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:03.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.345+0000 7f985f400640 1 -- 172.21.15.39:0/2585261975 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f98580aa1f0 con 0x7f9868105300 2024-07-04T08:12:03.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.561+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f986819ce00 con 0x7f9868105300 2024-07-04T08:12:03.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.562+0000 7f985f400640 1 -- 172.21.15.39:0/2585261975 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24957 (secure 0 0 0) 0x7f98580a9af0 con 0x7f9868105300 2024-07-04T08:12:03.565 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:12:03.566 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:11:53.672125+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.565+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f984c0b2950 msgr2=0x7f984c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.565+0000 7f986da00640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f984c0b2950 0x7f984c0b4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9850009230 tx=0x7f9850031400 comp rx=0 tx=0).stop 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.565+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9868105300 msgr2=0x7f986819bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.565+0000 7f986da00640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9868105300 0x7f986819bcd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f985800ea60 tx=0x7f985800ef20 comp rx=0 tx=0).stop 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.566+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 shutdown_connections 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.566+0000 7f986da00640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f984c0b2950 0x7f984c0b4e60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.566+0000 7f986da00640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f986810b590 0x7f98681a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.566+0000 7f986da00640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f986810ac40 0x7f986819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.566+0000 7f986da00640 1 --2- 172.21.15.39:0/2585261975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9868105300 0x7f986819bcd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:03.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.566+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 >> 172.21.15.39:0/2585261975 conn(0x7f9868100d10 msgr2=0x7f9868104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:03.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.566+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 shutdown_connections 2024-07-04T08:12:03.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:03.566+0000 7f986da00640 1 -- 172.21.15.39:0/2585261975 wait complete. 2024-07-04T08:12:03.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:03 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2585261975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:12:04.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:03 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2585261975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:12:04.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:03 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2585261975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:12:04.461 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-07-04T08:12:04.462 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-07-04T08:12:04.462 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-07-04T08:12:04.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-07-04T08:12:04.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-07-04T08:12:04.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-07-04T08:12:04.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-07-04T08:12:04.465 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-07-04T08:12:04.465 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-07-04T08:12:04.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-07-04T08:12:04.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-07-04T08:12:04.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-07-04T08:12:05.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:04 smithi133 ceph-mon[37050]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:12:05.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:04 smithi039 ceph-mon[32062]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:12:05.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:04 smithi050 ceph-mon[36577]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:12:05.599 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:05.603 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:05.604 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:05.615 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:06.282 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:06.284 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:06.284 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:06.285 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:06.286 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:06.293 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:07.012 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:06 smithi039 ceph-mon[32062]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:07.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:06 smithi133 ceph-mon[37050]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:07.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:06 smithi050 ceph-mon[36577]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:07.339 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:08.396 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:09.037 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:08 smithi039 ceph-mon[32062]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:09.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:08 smithi133 ceph-mon[37050]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:08 smithi050 ceph-mon[36577]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:11.027 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:10 smithi039 ceph-mon[32062]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:11.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:10 smithi133 ceph-mon[37050]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:10 smithi050 ceph-mon[36577]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:12.923 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:12 smithi039 ceph-mon[32062]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:13.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:12 smithi133 ceph-mon[37050]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:13.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:12 smithi050 ceph-mon[36577]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:15.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:14 smithi133 ceph-mon[37050]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:12:15.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:14 smithi039 ceph-mon[32062]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:12:15.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:14 smithi050 ceph-mon[36577]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:12:16.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.375+0000 7f947e800640 1 -- 172.21.15.39:0/542489530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9478105600 msgr2=0x7f9478105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.375+0000 7f947e800640 1 --2- 172.21.15.39:0/542489530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9478105600 0x7f9478105a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9464009900 tx=0x7f946402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.376+0000 7f947e800640 1 -- 172.21.15.39:0/542489530 shutdown_connections 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.376+0000 7f947e800640 1 --2- 172.21.15.39:0/542489530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9478105f50 0x7f947810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.376+0000 7f947e800640 1 --2- 172.21.15.39:0/542489530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9478105600 0x7f9478105a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.376+0000 7f947e800640 1 --2- 172.21.15.39:0/542489530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9478101320 0x7f9478101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.376+0000 7f947e800640 1 -- 172.21.15.39:0/542489530 >> 172.21.15.39:0/542489530 conn(0x7f9478076ff0 msgr2=0x7f9478077430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.376+0000 7f947e800640 1 -- 172.21.15.39:0/542489530 shutdown_connections 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.376+0000 7f947e800640 1 -- 172.21.15.39:0/542489530 wait complete. 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.377+0000 7f947e800640 1 Processor -- start 2024-07-04T08:12:16.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.377+0000 7f947e800640 1 -- start start 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9478101320 0x7f947819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9478105600 0x7f947819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9478105f50 0x7f94781a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947e800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9478112e10 con 0x7f9478101320 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947e800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9478112c90 con 0x7f9478105600 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947e800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9478112f90 con 0x7f9478105f50 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9478101320 0x7f947819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9478101320 0x7f947819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40358/0 (socket says 172.21.15.39:40358) 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947d400640 1 -- 172.21.15.39:0/3847052899 learned_addr learned my addr 172.21.15.39:0/3847052899 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947de00640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9478105f50 0x7f94781a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947ca00640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9478105600 0x7f947819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947d400640 1 -- 172.21.15.39:0/3847052899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9478105f50 msgr2=0x7f94781a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947d400640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9478105f50 0x7f94781a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947d400640 1 -- 172.21.15.39:0/3847052899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9478105600 msgr2=0x7f947819c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947d400640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9478105600 0x7f947819c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.378+0000 7f947d400640 1 -- 172.21.15.39:0/3847052899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94781a1a50 con 0x7f9478101320 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.379+0000 7f947d400640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9478101320 0x7f947819bae0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f946c00e9f0 tx=0x7f946c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.379+0000 7f9476000640 1 -- 172.21.15.39:0/3847052899 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00cd80 con 0x7f9478101320 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.379+0000 7f9476000640 1 -- 172.21.15.39:0/3847052899 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f946c0108e0 con 0x7f9478101320 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.379+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94781a1d40 con 0x7f9478101320 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.379+0000 7f9476000640 1 -- 172.21.15.39:0/3847052899 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c018980 con 0x7f9478101320 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.379+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781a84a0 con 0x7f9478101320 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.380+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9478107010 con 0x7f9478101320 2024-07-04T08:12:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.383+0000 7f9476000640 1 -- 172.21.15.39:0/3847052899 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f946c010430 con 0x7f9478101320 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.384+0000 7f9476000640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94600b28d0 0x7f94600b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.385+0000 7f9476000640 1 -- 172.21.15.39:0/3847052899 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f946c0e32e0 con 0x7f9478101320 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.385+0000 7f947ca00640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94600b28d0 0x7f94600b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.385+0000 7f9476000640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] conn(0x7f94600bddf0 0x7f94600c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.385+0000 7f9476000640 1 -- 172.21.15.39:0/3847052899 --> [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94600c09b0 con 0x7f94600bddf0 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.385+0000 7f9476000640 1 -- 172.21.15.39:0/3847052899 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f946c0e3690 con 0x7f9478101320 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.385+0000 7f947de00640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] conn(0x7f94600bddf0 0x7f94600c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.386+0000 7f947ca00640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94600b28d0 0x7f94600b4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f946402a9e0 tx=0x7f9464005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.386+0000 7f947de00640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] conn(0x7f94600bddf0 0x7f94600c02e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.387+0000 7f9476000640 1 -- 172.21.15.39:0/3847052899 <== osd.2 v2:172.21.15.133:6800/3737341609 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94600c09b0 con 0x7f94600bddf0 2024-07-04T08:12:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.403+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 --> [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f947806a160 con 0x7f94600bddf0 2024-07-04T08:12:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.404+0000 7f9476000640 1 -- 172.21.15.39:0/3847052899 <== osd.2 v2:172.21.15.133:6800/3737341609 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f947806a160 con 0x7f94600bddf0 2024-07-04T08:12:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.404+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 >> [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] conn(0x7f94600bddf0 msgr2=0x7f94600c02e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.404+0000 7f947e800640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] conn(0x7f94600bddf0 0x7f94600c02e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.404+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94600b28d0 msgr2=0x7f94600b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.404+0000 7f947e800640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94600b28d0 0x7f94600b4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f946402a9e0 tx=0x7f9464005a20 comp rx=0 tx=0).stop 2024-07-04T08:12:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.405+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9478101320 msgr2=0x7f947819bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.405+0000 7f947e800640 1 --2- 172.21.15.39:0/3847052899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9478101320 0x7f947819bae0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f946c00e9f0 tx=0x7f946c00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:12:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.405+0000 7f947de00640 1 -- 172.21.15.39:0/3847052899 reap_dead start 2024-07-04T08:12:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.405+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 shutdown_connections 2024-07-04T08:12:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.405+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 >> 172.21.15.39:0/3847052899 conn(0x7f9478076ff0 msgr2=0x7f9478106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.405+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 shutdown_connections 2024-07-04T08:12:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:16.405+0000 7f947e800640 1 -- 172.21.15.39:0/3847052899 wait complete. 2024-07-04T08:12:17.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:16 smithi133 ceph-mon[37050]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:17.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:16 smithi039 ceph-mon[32062]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:17.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:16 smithi050 ceph-mon[36577]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:18.951 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:18 smithi039 ceph-mon[32062]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:19.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.636+0000 7f9ec9a00640 1 -- 172.21.15.39:0/1543424897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec410d360 msgr2=0x7f9ec410f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.636+0000 7f9ec9a00640 1 --2- 172.21.15.39:0/1543424897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec410d360 0x7f9ec410f850 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009be0 tx=0x7f9ebc02a6c0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.636+0000 7f9ec9a00640 1 -- 172.21.15.39:0/1543424897 shutdown_connections 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.636+0000 7f9ec9a00640 1 --2- 172.21.15.39:0/1543424897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec410d360 0x7f9ec410f850 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.636+0000 7f9ec9a00640 1 --2- 172.21.15.39:0/1543424897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec4104850 0x7f9ec410cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.636+0000 7f9ec9a00640 1 --2- 172.21.15.39:0/1543424897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec4103f00 0x7f9ec4104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.636+0000 7f9ec9a00640 1 -- 172.21.15.39:0/1543424897 >> 172.21.15.39:0/1543424897 conn(0x7f9ec4078c80 msgr2=0x7f9ec4079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.636+0000 7f9ec9a00640 1 -- 172.21.15.39:0/1543424897 shutdown_connections 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.636+0000 7f9ec9a00640 1 -- 172.21.15.39:0/1543424897 wait complete. 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.637+0000 7f9ec9a00640 1 Processor -- start 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.638+0000 7f9ec9a00640 1 -- start start 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.638+0000 7f9ec9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec4103f00 0x7f9ec4101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.638+0000 7f9ec9a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec4104850 0x7f9ec4102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.638+0000 7f9ec9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec410d360 0x7f9ec41026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.638+0000 7f9ec9a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4112fc0 con 0x7f9ec410d360 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.638+0000 7f9ec9a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4112e40 con 0x7f9ec4103f00 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.638+0000 7f9ec9a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4113140 con 0x7f9ec4104850 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec410d360 0x7f9ec41026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec4103f00 0x7f9ec4101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec4103f00 0x7f9ec4101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44656/0 (socket says 172.21.15.39:44656) 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2a00640 1 -- 172.21.15.39:0/2202175239 learned_addr learned my addr 172.21.15.39:0/2202175239 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2000640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec4104850 0x7f9ec4102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2a00640 1 -- 172.21.15.39:0/2202175239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec4104850 msgr2=0x7f9ec4102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2a00640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec4104850 0x7f9ec4102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2a00640 1 -- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec410d360 msgr2=0x7f9ec41026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2a00640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ec410d360 0x7f9ec41026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2a00640 1 -- 172.21.15.39:0/2202175239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec41a7c50 con 0x7f9ec4103f00 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.639+0000 7f9ec2000640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec4104850 0x7f9ec4102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.640+0000 7f9ec2a00640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec4103f00 0x7f9ec4101c40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400e9a0 tx=0x7f9eb400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.640+0000 7f9ebb400640 1 -- 172.21.15.39:0/2202175239 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400cd80 con 0x7f9ec4103f00 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.640+0000 7f9ebb400640 1 -- 172.21.15.39:0/2202175239 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9eb40108e0 con 0x7f9ec4103f00 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.640+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec41a7f40 con 0x7f9ec4103f00 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.640+0000 7f9ebb400640 1 -- 172.21.15.39:0/2202175239 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4018960 con 0x7f9ec4103f00 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.640+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec41a83f0 con 0x7f9ec4103f00 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.641+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ec4100510 con 0x7f9ec4103f00 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.644+0000 7f9ebb400640 1 -- 172.21.15.39:0/2202175239 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f9eb4010a90 con 0x7f9ec4103f00 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.645+0000 7f9ebb400640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ea80b28d0 0x7f9ea80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.646+0000 7f9ebb400640 1 -- 172.21.15.39:0/2202175239 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f9eb40e3600 con 0x7f9ec4103f00 2024-07-04T08:12:19.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.646+0000 7f9ec2000640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ea80b28d0 0x7f9ea80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.646+0000 7f9ebb400640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] conn(0x7f9ea80bddf0 0x7f9ea80c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.646+0000 7f9ebb400640 1 -- 172.21.15.39:0/2202175239 --> [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9eb4027e50 con 0x7f9ea80bddf0 2024-07-04T08:12:19.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.646+0000 7f9ebb400640 1 -- 172.21.15.39:0/2202175239 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f9eb40e39b0 con 0x7f9ec4103f00 2024-07-04T08:12:19.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.646+0000 7f9ec3400640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] conn(0x7f9ea80bddf0 0x7f9ea80c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.647+0000 7f9ec2000640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ea80b28d0 0x7f9ea80b4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ec410efb0 tx=0x7f9eb0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:19.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.647+0000 7f9ec3400640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] conn(0x7f9ea80bddf0 0x7f9ea80c02e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.649+0000 7f9ebb400640 1 -- 172.21.15.39:0/2202175239 <== osd.9 v2:172.21.15.39:6826/2701785798 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9eb4027e50 con 0x7f9ea80bddf0 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.665+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 --> [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ec406aab0 con 0x7f9ea80bddf0 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.665+0000 7f9ebb400640 1 -- 172.21.15.39:0/2202175239 <== osd.9 v2:172.21.15.39:6826/2701785798 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9ec406aab0 con 0x7f9ea80bddf0 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] conn(0x7f9ea80bddf0 msgr2=0x7f9ea80c02e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec9a00640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] conn(0x7f9ea80bddf0 0x7f9ea80c02e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ea80b28d0 msgr2=0x7f9ea80b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec9a00640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9ea80b28d0 0x7f9ea80b4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ec410efb0 tx=0x7f9eb0005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec4103f00 msgr2=0x7f9ec4101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec9a00640 1 --2- 172.21.15.39:0/2202175239 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ec4103f00 0x7f9ec4101c40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400e9a0 tx=0x7f9eb400ee60 comp rx=0 tx=0).stop 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec3400640 1 -- 172.21.15.39:0/2202175239 reap_dead start 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 shutdown_connections 2024-07-04T08:12:19.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 >> 172.21.15.39:0/2202175239 conn(0x7f9ec4078c80 msgr2=0x7f9ec40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:19.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.666+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 shutdown_connections 2024-07-04T08:12:19.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:18.667+0000 7f9ec9a00640 1 -- 172.21.15.39:0/2202175239 wait complete. 2024-07-04T08:12:19.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:18 smithi133 ceph-mon[37050]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:19.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:18 smithi050 ceph-mon[36577]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.696+0000 7ff8d9200640 1 -- 172.21.15.39:0/2529393503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4108840 msgr2=0x7ff8d4108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.696+0000 7ff8d9200640 1 --2- 172.21.15.39:0/2529393503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4108840 0x7ff8d4108c50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009900 tx=0x7ff8c002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.696+0000 7ff8d9200640 1 -- 172.21.15.39:0/2529393503 shutdown_connections 2024-07-04T08:12:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.696+0000 7ff8d9200640 1 --2- 172.21.15.39:0/2529393503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8d4109190 0x7ff8d410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.696+0000 7ff8d9200640 1 --2- 172.21.15.39:0/2529393503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4108840 0x7ff8d4108c50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.696+0000 7ff8d9200640 1 --2- 172.21.15.39:0/2529393503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4102f00 0x7ff8d4103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.696+0000 7ff8d9200640 1 -- 172.21.15.39:0/2529393503 >> 172.21.15.39:0/2529393503 conn(0x7ff8d40fe950 msgr2=0x7ff8d4100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.696+0000 7ff8d9200640 1 -- 172.21.15.39:0/2529393503 shutdown_connections 2024-07-04T08:12:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.696+0000 7ff8d9200640 1 -- 172.21.15.39:0/2529393503 wait complete. 2024-07-04T08:12:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.698+0000 7ff8d9200640 1 Processor -- start 2024-07-04T08:12:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.698+0000 7ff8d9200640 1 -- start start 2024-07-04T08:12:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.698+0000 7ff8d9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4102f00 0x7ff8d419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.698+0000 7ff8d9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4108840 0x7ff8d419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.698+0000 7ff8d9200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8d4109190 0x7ff8d41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.698+0000 7ff8d9200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d4112e50 con 0x7ff8d4102f00 2024-07-04T08:12:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.698+0000 7ff8d9200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d4112cd0 con 0x7ff8d4108840 2024-07-04T08:12:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.698+0000 7ff8d9200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d4112fd0 con 0x7ff8d4109190 2024-07-04T08:12:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4108840 0x7ff8d419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4108840 0x7ff8d419bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44680/0 (socket says 172.21.15.39:44680) 2024-07-04T08:12:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4102f00 0x7ff8d419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d2a00640 1 -- 172.21.15.39:0/3131587178 learned_addr learned my addr 172.21.15.39:0/3131587178 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d3e00640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8d4109190 0x7ff8d41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d2a00640 1 -- 172.21.15.39:0/3131587178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8d4109190 msgr2=0x7ff8d41a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d2a00640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8d4109190 0x7ff8d41a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d2a00640 1 -- 172.21.15.39:0/3131587178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4102f00 msgr2=0x7ff8d419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d2a00640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d4102f00 0x7ff8d419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.699+0000 7ff8d2a00640 1 -- 172.21.15.39:0/3131587178 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d41a1990 con 0x7ff8d4108840 2024-07-04T08:12:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.700+0000 7ff8d2a00640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4108840 0x7ff8d419bfc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8c002a910 tx=0x7ff8c0031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.700+0000 7ff8cbe00640 1 -- 172.21.15.39:0/3131587178 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c003f030 con 0x7ff8d4108840 2024-07-04T08:12:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.700+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d41a1c20 con 0x7ff8d4108840 2024-07-04T08:12:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.700+0000 7ff8cbe00640 1 -- 172.21.15.39:0/3131587178 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c0004490 con 0x7ff8d4108840 2024-07-04T08:12:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.700+0000 7ff8cbe00640 1 -- 172.21.15.39:0/3131587178 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0035070 con 0x7ff8d4108840 2024-07-04T08:12:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.700+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d41a83e0 con 0x7ff8d4108840 2024-07-04T08:12:19.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.701+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8d410efe0 con 0x7ff8d4108840 2024-07-04T08:12:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.704+0000 7ff8cbe00640 1 -- 172.21.15.39:0/3131587178 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7ff8c0042030 con 0x7ff8d4108840 2024-07-04T08:12:19.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.706+0000 7ff8cbe00640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8b80b28d0 0x7ff8b80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.706+0000 7ff8cbe00640 1 -- 172.21.15.39:0/3131587178 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff8c0102ff0 con 0x7ff8d4108840 2024-07-04T08:12:19.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.706+0000 7ff8d3400640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8b80b28d0 0x7ff8b80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.706+0000 7ff8cbe00640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] conn(0x7ff8b80bddf0 0x7ff8b80c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.706+0000 7ff8cbe00640 1 -- 172.21.15.39:0/3131587178 --> [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8c0046e50 con 0x7ff8b80bddf0 2024-07-04T08:12:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.706+0000 7ff8cbe00640 1 -- 172.21.15.39:0/3131587178 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff8c01033a0 con 0x7ff8d4108840 2024-07-04T08:12:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.707+0000 7ff8d3e00640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] conn(0x7ff8b80bddf0 0x7ff8b80c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.707+0000 7ff8d3400640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8b80b28d0 0x7ff8b80b4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009bd0 tx=0x7ff8c4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.707+0000 7ff8d3e00640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] conn(0x7ff8b80bddf0 0x7ff8b80c02e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.708+0000 7ff8cbe00640 1 -- 172.21.15.39:0/3131587178 <== osd.11 v2:172.21.15.133:6824/3104157254 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8c0046e50 con 0x7ff8b80bddf0 2024-07-04T08:12:19.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.725+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 --> [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff8d4104180 con 0x7ff8b80bddf0 2024-07-04T08:12:19.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.726+0000 7ff8cbe00640 1 -- 172.21.15.39:0/3131587178 <== osd.11 v2:172.21.15.133:6824/3104157254 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff8d4104180 con 0x7ff8b80bddf0 2024-07-04T08:12:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.726+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 >> [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] conn(0x7ff8b80bddf0 msgr2=0x7ff8b80c02e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.726+0000 7ff8d9200640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] conn(0x7ff8b80bddf0 0x7ff8b80c02e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.726+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8b80b28d0 msgr2=0x7ff8b80b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.727+0000 7ff8d9200640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8b80b28d0 0x7ff8b80b4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009bd0 tx=0x7ff8c4005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.727+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4108840 msgr2=0x7ff8d419bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.727+0000 7ff8d9200640 1 --2- 172.21.15.39:0/3131587178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8d4108840 0x7ff8d419bfc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8c002a910 tx=0x7ff8c0031a20 comp rx=0 tx=0).stop 2024-07-04T08:12:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.727+0000 7ff8d3e00640 1 -- 172.21.15.39:0/3131587178 reap_dead start 2024-07-04T08:12:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.727+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 shutdown_connections 2024-07-04T08:12:19.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.727+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 >> 172.21.15.39:0/3131587178 conn(0x7ff8d40fe950 msgr2=0x7ff8d4107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:19.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.727+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 shutdown_connections 2024-07-04T08:12:19.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:19.727+0000 7ff8d9200640 1 -- 172.21.15.39:0/3131587178 wait complete. 2024-07-04T08:12:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.366+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2535235489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc074ca0 msgr2=0x7f2cbc0750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.366+0000 7f2cc0a00640 1 --2- 172.21.15.39:0/2535235489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc074ca0 0x7f2cbc0750b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009900 tx=0x7f2cac02a410 comp rx=0 tx=0).stop 2024-07-04T08:12:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.366+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2535235489 shutdown_connections 2024-07-04T08:12:20.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.366+0000 7f2cc0a00640 1 --2- 172.21.15.39:0/2535235489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f4c0 0x7f2cbc1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.366+0000 7f2cc0a00640 1 --2- 172.21.15.39:0/2535235489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc0755f0 0x7f2cbc10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.366+0000 7f2cc0a00640 1 --2- 172.21.15.39:0/2535235489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc074ca0 0x7f2cbc0750b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.366+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2535235489 >> 172.21.15.39:0/2535235489 conn(0x7f2cbc0fe950 msgr2=0x7f2cbc100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.366+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2535235489 shutdown_connections 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.366+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2535235489 wait complete. 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.368+0000 7f2cc0a00640 1 Processor -- start 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.368+0000 7f2cc0a00640 1 -- start start 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cc0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc074ca0 0x7f2cbc19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cc0a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc0755f0 0x7f2cbc19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cc0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f4c0 0x7f2cbc1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cc0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc115020 con 0x7f2cbc074ca0 2024-07-04T08:12:20.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cc0a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc114ea0 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cc0a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc1151a0 con 0x7f2cbc0755f0 2024-07-04T08:12:20.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cbb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc074ca0 0x7f2cbc19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:20.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cbbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f4c0 0x7f2cbc1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:20.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cbb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc074ca0 0x7f2cbc19b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40396/0 (socket says 172.21.15.39:40396) 2024-07-04T08:12:20.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cbb400640 1 -- 172.21.15.39:0/2869355034 learned_addr learned my addr 172.21.15.39:0/2869355034 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:20.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.369+0000 7f2cbaa00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc0755f0 0x7f2cbc19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:20.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2869355034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc0755f0 msgr2=0x7f2cbc19bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc0755f0 0x7f2cbc19bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2869355034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc074ca0 msgr2=0x7f2cbc19b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc074ca0 0x7f2cbc19b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2869355034 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc1a1960 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f4c0 0x7f2cbc1a1230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009fd0 tx=0x7f2cb400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2869355034 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb400ce20 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2869355034 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cb4005020 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc1a1c50 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2869355034 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4010690 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.370+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1a83b0 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.371+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2cbc109a80 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.374+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2869355034 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f2cb40054d0 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.376+0000 7f2cb3e00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b2950 0x7f2ca00b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.376+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2869355034 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2cb40e2ff0 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.376+0000 7f2cbb400640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b2950 0x7f2ca00b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.376+0000 7f2cb3e00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] conn(0x7f2ca00bde70 0x7f2ca00c0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.376+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2869355034 --> [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2cb4019ec0 con 0x7f2ca00bde70 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.376+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2869355034 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f2cb40e3440 con 0x7f2cbc10f4c0 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.377+0000 7f2cbaa00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] conn(0x7f2ca00bde70 0x7f2ca00c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.377+0000 7f2cbb400640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b2950 0x7f2ca00b4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2cac031440 tx=0x7f2cac0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.377+0000 7f2cbaa00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] conn(0x7f2ca00bde70 0x7f2ca00c0360 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.378+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2869355034 <== osd.7 v2:172.21.15.50:6816/1720421182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cb4019ec0 con 0x7f2ca00bde70 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.395+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 --> [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2cbc0761b0 con 0x7f2ca00bde70 2024-07-04T08:12:20.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.395+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2869355034 <== osd.7 v2:172.21.15.50:6816/1720421182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2cbc0761b0 con 0x7f2ca00bde70 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 >> [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] conn(0x7f2ca00bde70 msgr2=0x7f2ca00c0360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] conn(0x7f2ca00bde70 0x7f2ca00c0360 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b2950 msgr2=0x7f2ca00b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b2950 0x7f2ca00b4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2cac031440 tx=0x7f2cac0058d0 comp rx=0 tx=0).stop 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f4c0 msgr2=0x7f2cbc1a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 --2- 172.21.15.39:0/2869355034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f4c0 0x7f2cbc1a1230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009fd0 tx=0x7f2cb400ef10 comp rx=0 tx=0).stop 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2869355034 reap_dead start 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 shutdown_connections 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 >> 172.21.15.39:0/2869355034 conn(0x7f2cbc0fe950 msgr2=0x7f2cbc109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 shutdown_connections 2024-07-04T08:12:20.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:20.396+0000 7f2cc0a00640 1 -- 172.21.15.39:0/2869355034 wait complete. 2024-07-04T08:12:21.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:20 smithi133 ceph-mon[37050]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-07-04T08:12:21.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.080+0000 7f7392c00640 1 -- 172.21.15.39:0/247524966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f738c103d10 msgr2=0x7f738c104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:21.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.080+0000 7f7392c00640 1 --2- 172.21.15.39:0/247524966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f738c103d10 0x7f738c104120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7384009ca0 tx=0x7f738402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:12:21.082 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:20 smithi039 ceph-mon[32062]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-07-04T08:12:21.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.080+0000 7f7392c00640 1 -- 172.21.15.39:0/247524966 shutdown_connections 2024-07-04T08:12:21.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.080+0000 7f7392c00640 1 --2- 172.21.15.39:0/247524966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f738c10d170 0x7f738c10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:21.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.080+0000 7f7392c00640 1 --2- 172.21.15.39:0/247524966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f738c104660 0x7f738c10cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:21.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.080+0000 7f7392c00640 1 --2- 172.21.15.39:0/247524966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f738c103d10 0x7f738c104120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:21.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.080+0000 7f7392c00640 1 -- 172.21.15.39:0/247524966 >> 172.21.15.39:0/247524966 conn(0x7f738c0fd810 msgr2=0x7f738c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:21.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.080+0000 7f7392c00640 1 -- 172.21.15.39:0/247524966 shutdown_connections 2024-07-04T08:12:21.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.080+0000 7f7392c00640 1 -- 172.21.15.39:0/247524966 wait complete. 2024-07-04T08:12:21.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.082+0000 7f7392c00640 1 Processor -- start 2024-07-04T08:12:21.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.082+0000 7f7392c00640 1 -- start start 2024-07-04T08:12:21.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.082+0000 7f7392c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f738c103d10 0x7f738c19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:21.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.082+0000 7f7392c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f738c104660 0x7f738c19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:21.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.083+0000 7f7392c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f738c10d170 0x7f738c1a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:21.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.083+0000 7f7392c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f738c112e20 con 0x7f738c104660 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.083+0000 7f7392c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f738c112ca0 con 0x7f738c103d10 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.083+0000 7f7392c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f738c112fa0 con 0x7f738c10d170 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.083+0000 7f7391800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f738c103d10 0x7f738c19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.083+0000 7f7390e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f738c104660 0x7f738c19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.083+0000 7f7391800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f738c103d10 0x7f738c19ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34122/0 (socket says 172.21.15.39:34122) 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.083+0000 7f7391800640 1 -- 172.21.15.39:0/4010682967 learned_addr learned my addr 172.21.15.39:0/4010682967 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.083+0000 7f7392200640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f738c10d170 0x7f738c1a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.084+0000 7f7391800640 1 -- 172.21.15.39:0/4010682967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f738c10d170 msgr2=0x7f738c1a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.084+0000 7f7391800640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f738c10d170 0x7f738c1a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.084+0000 7f7391800640 1 -- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f738c104660 msgr2=0x7f738c19c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:21.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.084+0000 7f7391800640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f738c104660 0x7f738c19c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:21.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.084+0000 7f7391800640 1 -- 172.21.15.39:0/4010682967 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c1a1a00 con 0x7f738c103d10 2024-07-04T08:12:21.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.084+0000 7f7391800640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f738c103d10 0x7f738c19ba90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7384005e30 tx=0x7f7384031840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:21.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.085+0000 7f738a000640 1 -- 172.21.15.39:0/4010682967 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384003b70 con 0x7f738c103d10 2024-07-04T08:12:21.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.085+0000 7f738a000640 1 -- 172.21.15.39:0/4010682967 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7384003d10 con 0x7f738c103d10 2024-07-04T08:12:21.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.085+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738c1a1c90 con 0x7f738c103d10 2024-07-04T08:12:21.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.085+0000 7f738a000640 1 -- 172.21.15.39:0/4010682967 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738402f900 con 0x7f738c103d10 2024-07-04T08:12:21.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.085+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738c1a8450 con 0x7f738c103d10 2024-07-04T08:12:21.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.086+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f738c10cc50 con 0x7f738c103d10 2024-07-04T08:12:21.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.089+0000 7f738a000640 1 -- 172.21.15.39:0/4010682967 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f738402faa0 con 0x7f738c103d10 2024-07-04T08:12:21.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.090+0000 7f738a000640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73700b28d0 0x7f73700b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:21.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.090+0000 7f738a000640 1 -- 172.21.15.39:0/4010682967 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7384103d10 con 0x7f738c103d10 2024-07-04T08:12:21.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.091+0000 7f7390e00640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73700b28d0 0x7f73700b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.091+0000 7f738a000640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] conn(0x7f73700bddf0 0x7f73700c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.091+0000 7f738a000640 1 -- 172.21.15.39:0/4010682967 --> [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f738404fe50 con 0x7f73700bddf0 2024-07-04T08:12:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.091+0000 7f738a000640 1 -- 172.21.15.39:0/4010682967 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7384104160 con 0x7f738c103d10 2024-07-04T08:12:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.091+0000 7f7392200640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] conn(0x7f73700bddf0 0x7f73700c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.092+0000 7f7390e00640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73700b28d0 0x7f73700b4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f738c10f650 tx=0x7f7378005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:21.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.093+0000 7f7392200640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] conn(0x7f73700bddf0 0x7f73700c02e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.094+0000 7f738a000640 1 -- 172.21.15.39:0/4010682967 <== osd.3 v2:172.21.15.39:6810/4257278793 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f738404fe50 con 0x7f73700bddf0 2024-07-04T08:12:21.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.111+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 --> [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f738c06aab0 con 0x7f73700bddf0 2024-07-04T08:12:21.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.112+0000 7f738a000640 1 -- 172.21.15.39:0/4010682967 <== osd.3 v2:172.21.15.39:6810/4257278793 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f738c06aab0 con 0x7f73700bddf0 2024-07-04T08:12:21.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.112+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] conn(0x7f73700bddf0 msgr2=0x7f73700c02e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:21.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.112+0000 7f7392c00640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] conn(0x7f73700bddf0 0x7f73700c02e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:21.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.113+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73700b28d0 msgr2=0x7f73700b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:21.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.113+0000 7f7392c00640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73700b28d0 0x7f73700b4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f738c10f650 tx=0x7f7378005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:21.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.113+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f738c103d10 msgr2=0x7f738c19ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:21.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.113+0000 7f7392c00640 1 --2- 172.21.15.39:0/4010682967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f738c103d10 0x7f738c19ba90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7384005e30 tx=0x7f7384031840 comp rx=0 tx=0).stop 2024-07-04T08:12:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.113+0000 7f7392200640 1 -- 172.21.15.39:0/4010682967 reap_dead start 2024-07-04T08:12:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.113+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 shutdown_connections 2024-07-04T08:12:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.113+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 >> 172.21.15.39:0/4010682967 conn(0x7f738c0fd810 msgr2=0x7f738c10c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.114+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 shutdown_connections 2024-07-04T08:12:21.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:21.114+0000 7f7392c00640 1 -- 172.21.15.39:0/4010682967 wait complete. 2024-07-04T08:12:21.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:20 smithi050 ceph-mon[36577]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-07-04T08:12:22.326 INFO:teuthology.orchestra.run.smithi039.stdout:81604378723 2024-07-04T08:12:22.327 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-07-04T08:12:22.980 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:22 smithi039 ceph-mon[32062]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:23.036 INFO:teuthology.orchestra.run.smithi039.stdout:214748364871 2024-07-04T08:12:23.036 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-07-04T08:12:23.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:22 smithi133 ceph-mon[37050]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:22 smithi050 ceph-mon[36577]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:24.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.005+0000 7f5662200640 1 -- 172.21.15.39:0/1824209083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f565c10b560 msgr2=0x7f565c111d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.005+0000 7f5662200640 1 --2- 172.21.15.39:0/1824209083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f565c10b560 0x7f565c111d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f564c009ca0 tx=0x7f564c02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.006+0000 7f5662200640 1 -- 172.21.15.39:0/1824209083 shutdown_connections 2024-07-04T08:12:24.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.006+0000 7f5662200640 1 --2- 172.21.15.39:0/1824209083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f565c10b560 0x7f565c111d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.006+0000 7f5662200640 1 --2- 172.21.15.39:0/1824209083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c10ac10 0x7f565c10b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.006+0000 7f5662200640 1 --2- 172.21.15.39:0/1824209083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c1052d0 0x7f565c1056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.006+0000 7f5662200640 1 -- 172.21.15.39:0/1824209083 >> 172.21.15.39:0/1824209083 conn(0x7f565c100d60 msgr2=0x7f565c1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:24.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.006+0000 7f5662200640 1 -- 172.21.15.39:0/1824209083 shutdown_connections 2024-07-04T08:12:24.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.006+0000 7f5662200640 1 -- 172.21.15.39:0/1824209083 wait complete. 2024-07-04T08:12:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.007+0000 7f5662200640 1 Processor -- start 2024-07-04T08:12:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.007+0000 7f5662200640 1 -- start start 2024-07-04T08:12:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f5662200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c1052d0 0x7f565c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f5662200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c10ac10 0x7f565c19c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f5662200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f565c10b560 0x7f565c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f5662200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f565c115230 con 0x7f565c10ac10 2024-07-04T08:12:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f5662200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1150b0 con 0x7f565c10b560 2024-07-04T08:12:24.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f5662200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1153b0 con 0x7f565c1052d0 2024-07-04T08:12:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f565b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c1052d0 0x7f565c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f565b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c1052d0 0x7f565c19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57132/0 (socket says 172.21.15.39:57132) 2024-07-04T08:12:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f565aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c10ac10 0x7f565c19c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f565b400640 1 -- 172.21.15.39:0/3435249213 learned_addr learned my addr 172.21.15.39:0/3435249213 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.008+0000 7f565be00640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f565c10b560 0x7f565c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f565b400640 1 -- 172.21.15.39:0/3435249213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f565c10b560 msgr2=0x7f565c1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f565b400640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f565c10b560 0x7f565c1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f565b400640 1 -- 172.21.15.39:0/3435249213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c10ac10 msgr2=0x7f565c19c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f565b400640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f565c10ac10 0x7f565c19c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f565b400640 1 -- 172.21.15.39:0/3435249213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c1a1b10 con 0x7f565c1052d0 2024-07-04T08:12:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f565b400640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c1052d0 0x7f565c19bba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f565400d950 tx=0x7f565400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f5653e00640 1 -- 172.21.15.39:0/3435249213 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654017030 con 0x7f565c1052d0 2024-07-04T08:12:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f5653e00640 1 -- 172.21.15.39:0/3435249213 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56540099b0 con 0x7f565c1052d0 2024-07-04T08:12:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f5653e00640 1 -- 172.21.15.39:0/3435249213 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654003bc0 con 0x7f565c1052d0 2024-07-04T08:12:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.009+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565c1a1e00 con 0x7f565c1052d0 2024-07-04T08:12:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.010+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565c1a8560 con 0x7f565c1052d0 2024-07-04T08:12:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.011+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f565c0630a0 con 0x7f565c1052d0 2024-07-04T08:12:24.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.014+0000 7f5653e00640 1 -- 172.21.15.39:0/3435249213 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f565401a070 con 0x7f565c1052d0 2024-07-04T08:12:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.015+0000 7f5653e00640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56400b2980 0x7f56400b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.016+0000 7f5653e00640 1 -- 172.21.15.39:0/3435249213 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f56540e23d0 con 0x7f565c1052d0 2024-07-04T08:12:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.016+0000 7f565aa00640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56400b2980 0x7f56400b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.016+0000 7f5653e00640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] conn(0x7f56400bdea0 0x7f56400c0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.016+0000 7f5653e00640 1 -- 172.21.15.39:0/3435249213 --> [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5654010e50 con 0x7f56400bdea0 2024-07-04T08:12:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.016+0000 7f5653e00640 1 -- 172.21.15.39:0/3435249213 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f56540e2780 con 0x7f565c1052d0 2024-07-04T08:12:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.016+0000 7f565be00640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] conn(0x7f56400bdea0 0x7f56400c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.017+0000 7f565aa00640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56400b2980 0x7f56400b4e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f565c1045b0 tx=0x7f5648005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:24.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.017+0000 7f565be00640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] conn(0x7f56400bdea0 0x7f56400c0390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:24.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.018+0000 7f5653e00640 1 -- 172.21.15.39:0/3435249213 <== osd.1 v2:172.21.15.50:6800/735117817 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5654010e50 con 0x7f56400bdea0 2024-07-04T08:12:24.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.034+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 --> [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f565c0634a0 con 0x7f56400bdea0 2024-07-04T08:12:24.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f5653e00640 1 -- 172.21.15.39:0/3435249213 <== osd.1 v2:172.21.15.50:6800/735117817 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f565c0634a0 con 0x7f56400bdea0 2024-07-04T08:12:24.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 >> [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] conn(0x7f56400bdea0 msgr2=0x7f56400c0390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f5662200640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] conn(0x7f56400bdea0 0x7f56400c0390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56400b2980 msgr2=0x7f56400b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f5662200640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56400b2980 0x7f56400b4e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f565c1045b0 tx=0x7f5648005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:24.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c1052d0 msgr2=0x7f565c19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f5662200640 1 --2- 172.21.15.39:0/3435249213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c1052d0 0x7f565c19bba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f565400d950 tx=0x7f565400de10 comp rx=0 tx=0).stop 2024-07-04T08:12:24.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f565be00640 1 -- 172.21.15.39:0/3435249213 reap_dead start 2024-07-04T08:12:24.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 shutdown_connections 2024-07-04T08:12:24.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.035+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 >> 172.21.15.39:0/3435249213 conn(0x7f565c100d60 msgr2=0x7f565c10a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:24.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.036+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 shutdown_connections 2024-07-04T08:12:24.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.036+0000 7f5662200640 1 -- 172.21.15.39:0/3435249213 wait complete. 2024-07-04T08:12:24.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.907+0000 7f1dcb400640 1 -- 172.21.15.39:0/2589719207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dc4104820 msgr2=0x7f1dc410ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.907+0000 7f1dcb400640 1 --2- 172.21.15.39:0/2589719207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dc4104820 0x7f1dc410ce60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009900 tx=0x7f1db002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.908+0000 7f1dcb400640 1 -- 172.21.15.39:0/2589719207 shutdown_connections 2024-07-04T08:12:24.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.908+0000 7f1dcb400640 1 --2- 172.21.15.39:0/2589719207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc410d3a0 0x7f1dc410f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.908+0000 7f1dcb400640 1 --2- 172.21.15.39:0/2589719207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dc4104820 0x7f1dc410ce60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.908+0000 7f1dcb400640 1 --2- 172.21.15.39:0/2589719207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dc4103df0 0x7f1dc4104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.908+0000 7f1dcb400640 1 -- 172.21.15.39:0/2589719207 >> 172.21.15.39:0/2589719207 conn(0x7f1dc4078c90 msgr2=0x7f1dc40790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:24.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.908+0000 7f1dcb400640 1 -- 172.21.15.39:0/2589719207 shutdown_connections 2024-07-04T08:12:24.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.908+0000 7f1dcb400640 1 -- 172.21.15.39:0/2589719207 wait complete. 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.911+0000 7f1dcb400640 1 Processor -- start 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.911+0000 7f1dcb400640 1 -- start start 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.911+0000 7f1dcb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dc4103df0 0x7f1dc419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.911+0000 7f1dcb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dc410d3a0 0x7f1dc419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.911+0000 7f1dcb400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc41a14f0 0x7f1dc41a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.912+0000 7f1dcb400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4112fc0 con 0x7f1dc4103df0 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.912+0000 7f1dcb400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4112e40 con 0x7f1dc410d3a0 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.912+0000 7f1dcb400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4113140 con 0x7f1dc41a14f0 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.912+0000 7f1dc3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dc410d3a0 0x7f1dc419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.912+0000 7f1dc8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dc4103df0 0x7f1dc419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.912+0000 7f1dc3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dc410d3a0 0x7f1dc419c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34134/0 (socket says 172.21.15.39:34134) 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.912+0000 7f1dc3e00640 1 -- 172.21.15.39:0/3147826886 learned_addr learned my addr 172.21.15.39:0/3147826886 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:24.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.912+0000 7f1dc9400640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc41a14f0 0x7f1dc41a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.913+0000 7f1dc3e00640 1 -- 172.21.15.39:0/3147826886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc41a14f0 msgr2=0x7f1dc41a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.913+0000 7f1dc3e00640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1dc41a14f0 0x7f1dc41a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.913+0000 7f1dc3e00640 1 -- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dc4103df0 msgr2=0x7f1dc419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.913+0000 7f1dc3e00640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1dc4103df0 0x7f1dc419bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.913+0000 7f1dc3e00640 1 -- 172.21.15.39:0/3147826886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc41a20d0 con 0x7f1dc410d3a0 2024-07-04T08:12:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.913+0000 7f1dc3e00640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dc410d3a0 0x7f1dc419c310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1db0006010 tx=0x7f1db0031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.914+0000 7f1dc1600640 1 -- 172.21.15.39:0/3147826886 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db003f030 con 0x7f1dc410d3a0 2024-07-04T08:12:24.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.914+0000 7f1dc1600640 1 -- 172.21.15.39:0/3147826886 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db00043d0 con 0x7f1dc410d3a0 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.914+0000 7f1dc1600640 1 -- 172.21.15.39:0/3147826886 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0035070 con 0x7f1dc410d3a0 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.914+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc41a8670 con 0x7f1dc410d3a0 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.914+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a8a10 con 0x7f1dc410d3a0 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.915+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1dc40630a0 con 0x7f1dc410d3a0 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.918+0000 7f1dc1600640 1 -- 172.21.15.39:0/3147826886 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f1db0042030 con 0x7f1dc410d3a0 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.921+0000 7f1dc1600640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1dac0b2950 0x7f1dac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.921+0000 7f1dc1600640 1 -- 172.21.15.39:0/3147826886 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1db0102ff0 con 0x7f1dc410d3a0 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.921+0000 7f1dc8a00640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1dac0b2950 0x7f1dac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.921+0000 7f1dc1600640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] conn(0x7f1dac0bde70 0x7f1dac0c0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.921+0000 7f1dc1600640 1 -- 172.21.15.39:0/3147826886 --> [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1db003ce50 con 0x7f1dac0bde70 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.921+0000 7f1dc1600640 1 -- 172.21.15.39:0/3147826886 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f1db0103440 con 0x7f1dc410d3a0 2024-07-04T08:12:24.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.922+0000 7f1dc9400640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] conn(0x7f1dac0bde70 0x7f1dac0c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.922+0000 7f1dc9400640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] conn(0x7f1dac0bde70 0x7f1dac0c0360 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.922+0000 7f1dc8a00640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1dac0b2950 0x7f1dac0b4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1dc406bdf0 tx=0x7f1db8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.924+0000 7f1dc1600640 1 -- 172.21.15.39:0/3147826886 <== osd.6 v2:172.21.15.39:6818/636897432 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1db003ce50 con 0x7f1dac0bde70 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.940+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 --> [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1dc40634a0 con 0x7f1dac0bde70 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.941+0000 7f1dc1600640 1 -- 172.21.15.39:0/3147826886 <== osd.6 v2:172.21.15.39:6818/636897432 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1dc40634a0 con 0x7f1dac0bde70 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] conn(0x7f1dac0bde70 msgr2=0x7f1dac0c0360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dcb400640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] conn(0x7f1dac0bde70 0x7f1dac0c0360 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1dac0b2950 msgr2=0x7f1dac0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dcb400640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1dac0b2950 0x7f1dac0b4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1dc406bdf0 tx=0x7f1db8005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dc410d3a0 msgr2=0x7f1dc419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dcb400640 1 --2- 172.21.15.39:0/3147826886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1dc410d3a0 0x7f1dc419c310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1db0006010 tx=0x7f1db0031a50 comp rx=0 tx=0).stop 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dc9400640 1 -- 172.21.15.39:0/3147826886 reap_dead start 2024-07-04T08:12:24.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 shutdown_connections 2024-07-04T08:12:24.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 >> 172.21.15.39:0/3147826886 conn(0x7f1dc4078c90 msgr2=0x7f1dc41093a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:24.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.942+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 shutdown_connections 2024-07-04T08:12:24.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:23.943+0000 7f1dcb400640 1 -- 172.21.15.39:0/3147826886 wait complete. 2024-07-04T08:12:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.654+0000 7f65c6000640 1 -- 172.21.15.39:0/793296755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65c010f460 msgr2=0x7f65c01118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.654+0000 7f65c6000640 1 --2- 172.21.15.39:0/793296755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65c010f460 0x7f65c01118a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009ca0 tx=0x7f65b802a8b0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.655+0000 7f65c6000640 1 -- 172.21.15.39:0/793296755 shutdown_connections 2024-07-04T08:12:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.655+0000 7f65c6000640 1 --2- 172.21.15.39:0/793296755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65c010f460 0x7f65c01118a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.655+0000 7f65c6000640 1 --2- 172.21.15.39:0/793296755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0075650 0x7f65c010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.655+0000 7f65c6000640 1 --2- 172.21.15.39:0/793296755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c0074d00 0x7f65c0075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.655+0000 7f65c6000640 1 -- 172.21.15.39:0/793296755 >> 172.21.15.39:0/793296755 conn(0x7f65c00fe990 msgr2=0x7f65c0100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.655+0000 7f65c6000640 1 -- 172.21.15.39:0/793296755 shutdown_connections 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.655+0000 7f65c6000640 1 -- 172.21.15.39:0/793296755 wait complete. 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.656+0000 7f65c6000640 1 Processor -- start 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.656+0000 7f65c6000640 1 -- start start 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65c6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0074d00 0x7f65c019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65c6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65c0075650 0x7f65c019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65c6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c010f460 0x7f65c01a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65c6000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0114ff0 con 0x7f65c010f460 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65c6000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0114e70 con 0x7f65c0075650 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65c6000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0115170 con 0x7f65c0074d00 2024-07-04T08:12:24.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65c4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0074d00 0x7f65c019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65c0075650 0x7f65c019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65c0075650 0x7f65c019bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34184/0 (socket says 172.21.15.39:34184) 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65bfe00640 1 -- 172.21.15.39:0/1340926177 learned_addr learned my addr 172.21.15.39:0/1340926177 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65c4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0074d00 0x7f65c019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57150/0 (socket says 172.21.15.39:57150) 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65c5600640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c010f460 0x7f65c01a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65bfe00640 1 -- 172.21.15.39:0/1340926177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0074d00 msgr2=0x7f65c019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65bfe00640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0074d00 0x7f65c019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65bfe00640 1 -- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c010f460 msgr2=0x7f65c01a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65bfe00640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c010f460 0x7f65c01a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.657+0000 7f65bfe00640 1 -- 172.21.15.39:0/1340926177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c01a19b0 con 0x7f65c0075650 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.658+0000 7f65c4c00640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0074d00 0x7f65c019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:24.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.658+0000 7f65bfe00640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65c0075650 0x7f65c019bfe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00e9c0 tx=0x7f65ac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.658+0000 7f65bd600640 1 -- 172.21.15.39:0/1340926177 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac00cd80 con 0x7f65c0075650 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.658+0000 7f65bd600640 1 -- 172.21.15.39:0/1340926177 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65ac0035f0 con 0x7f65c0075650 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.658+0000 7f65bd600640 1 -- 172.21.15.39:0/1340926177 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac010690 con 0x7f65c0075650 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.658+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c01a1ca0 con 0x7f65c0075650 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.658+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c01a8400 con 0x7f65c0075650 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.659+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65c0109970 con 0x7f65c0075650 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.662+0000 7f65bd600640 1 -- 172.21.15.39:0/1340926177 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f65ac0037a0 con 0x7f65c0075650 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.664+0000 7f65bd600640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f65a80b28d0 0x7f65a80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.664+0000 7f65bd600640 1 -- 172.21.15.39:0/1340926177 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f65ac0e2da0 con 0x7f65c0075650 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.664+0000 7f65c4c00640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f65a80b28d0 0x7f65a80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.664+0000 7f65bd600640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] conn(0x7f65a80bddf0 0x7f65a80c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:24.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.664+0000 7f65bd600640 1 -- 172.21.15.39:0/1340926177 --> [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65a80c09b0 con 0x7f65a80bddf0 2024-07-04T08:12:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.664+0000 7f65bd600640 1 -- 172.21.15.39:0/1340926177 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f65ac0e3150 con 0x7f65c0075650 2024-07-04T08:12:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.665+0000 7f65c5600640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] conn(0x7f65a80bddf0 0x7f65a80c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.665+0000 7f65c4c00640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f65a80b28d0 0x7f65a80b4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009fc0 tx=0x7f65b4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.665+0000 7f65c5600640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] conn(0x7f65a80bddf0 0x7f65a80c02e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.666+0000 7f65bd600640 1 -- 172.21.15.39:0/1340926177 <== osd.0 v2:172.21.15.39:6802/1924112753 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65a80c09b0 con 0x7f65a80bddf0 2024-07-04T08:12:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.684+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 --> [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65c0076210 con 0x7f65a80bddf0 2024-07-04T08:12:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.685+0000 7f65bd600640 1 -- 172.21.15.39:0/1340926177 <== osd.0 v2:172.21.15.39:6802/1924112753 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f65c0076210 con 0x7f65a80bddf0 2024-07-04T08:12:24.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] conn(0x7f65a80bddf0 msgr2=0x7f65a80c02e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] conn(0x7f65a80bddf0 0x7f65a80c02e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:24.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f65a80b28d0 msgr2=0x7f65a80b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f65a80b28d0 0x7f65a80b4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009fc0 tx=0x7f65b4005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:24.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65c0075650 msgr2=0x7f65c019bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:24.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 --2- 172.21.15.39:0/1340926177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65c0075650 0x7f65c019bfe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00e9c0 tx=0x7f65ac00ee80 comp rx=0 tx=0).stop 2024-07-04T08:12:24.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c5600640 1 -- 172.21.15.39:0/1340926177 reap_dead start 2024-07-04T08:12:24.978 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:24 smithi039 ceph-mon[32062]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:12:24.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 shutdown_connections 2024-07-04T08:12:24.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 >> 172.21.15.39:0/1340926177 conn(0x7f65c00fe990 msgr2=0x7f65c0108f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:24.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 shutdown_connections 2024-07-04T08:12:24.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:24.686+0000 7f65c6000640 1 -- 172.21.15.39:0/1340926177 wait complete. 2024-07-04T08:12:25.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:24 smithi050 ceph-mon[36577]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:12:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:24 smithi133 ceph-mon[37050]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:12:25.924 INFO:teuthology.orchestra.run.smithi039.stdout:193273528397 2024-07-04T08:12:25.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-07-04T08:12:26.045 INFO:teuthology.orchestra.run.smithi039.stdout:115964117084 2024-07-04T08:12:26.045 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-07-04T08:12:27.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:26 smithi039 ceph-mon[32062]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:27.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:26 smithi050 ceph-mon[36577]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:26 smithi133 ceph-mon[37050]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:27.420 INFO:teuthology.orchestra.run.smithi039.stdout:266287972409 2024-07-04T08:12:27.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-07-04T08:12:27.601 INFO:teuthology.orchestra.run.smithi039.stdout:60129542248 2024-07-04T08:12:27.601 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-07-04T08:12:28.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.080+0000 7f995fe00640 1 -- 172.21.15.39:0/3261382423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9960108840 msgr2=0x7f9960108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.080+0000 7f995fe00640 1 --2- 172.21.15.39:0/3261382423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9960108840 0x7f9960108c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f994c009900 tx=0x7f994c02a410 comp rx=0 tx=0).stop 2024-07-04T08:12:28.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.081+0000 7f995fe00640 1 -- 172.21.15.39:0/3261382423 shutdown_connections 2024-07-04T08:12:28.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.081+0000 7f995fe00640 1 --2- 172.21.15.39:0/3261382423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9960108840 0x7f9960108c50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.081+0000 7f995fe00640 1 --2- 172.21.15.39:0/3261382423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9960109190 0x7f996010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.081+0000 7f995fe00640 1 --2- 172.21.15.39:0/3261382423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9960102f00 0x7f9960103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.081+0000 7f995fe00640 1 -- 172.21.15.39:0/3261382423 >> 172.21.15.39:0/3261382423 conn(0x7f99600fe950 msgr2=0x7f9960100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:28.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.081+0000 7f995fe00640 1 -- 172.21.15.39:0/3261382423 shutdown_connections 2024-07-04T08:12:28.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.081+0000 7f995fe00640 1 -- 172.21.15.39:0/3261382423 wait complete. 2024-07-04T08:12:28.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.082+0000 7f995fe00640 1 Processor -- start 2024-07-04T08:12:28.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.082+0000 7f995fe00640 1 -- start start 2024-07-04T08:12:28.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9960102f00 0x7f996019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9960108840 0x7f996019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9960109190 0x7f99601a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9960112ea0 con 0x7f9960102f00 2024-07-04T08:12:28.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9960112d20 con 0x7f9960108840 2024-07-04T08:12:28.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9960113020 con 0x7f9960109190 2024-07-04T08:12:28.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9960108840 0x7f996019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9960108840 0x7f996019bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34204/0 (socket says 172.21.15.39:34204) 2024-07-04T08:12:28.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995e000640 1 -- 172.21.15.39:0/2620949261 learned_addr learned my addr 172.21.15.39:0/2620949261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:28.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995ea00640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9960102f00 0x7f996019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.083+0000 7f995f400640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9960109190 0x7f99601a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.084+0000 7f995e000640 1 -- 172.21.15.39:0/2620949261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9960109190 msgr2=0x7f99601a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.084+0000 7f995e000640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9960109190 0x7f99601a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.084+0000 7f995e000640 1 -- 172.21.15.39:0/2620949261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9960102f00 msgr2=0x7f996019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.084+0000 7f995e000640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9960102f00 0x7f996019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.084+0000 7f995e000640 1 -- 172.21.15.39:0/2620949261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99601a19c0 con 0x7f9960108840 2024-07-04T08:12:28.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.084+0000 7f995f400640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9960109190 0x7f99601a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:12:28.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.084+0000 7f995e000640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9960108840 0x7f996019bff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f994c005ab0 tx=0x7f994c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:28.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.085+0000 7f9957400640 1 -- 172.21.15.39:0/2620949261 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c03f070 con 0x7f9960108840 2024-07-04T08:12:28.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.085+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99601a1c50 con 0x7f9960108840 2024-07-04T08:12:28.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.085+0000 7f9957400640 1 -- 172.21.15.39:0/2620949261 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994c031b50 con 0x7f9960108840 2024-07-04T08:12:28.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.085+0000 7f9957400640 1 -- 172.21.15.39:0/2620949261 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c035030 con 0x7f9960108840 2024-07-04T08:12:28.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.085+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99601a8410 con 0x7f9960108840 2024-07-04T08:12:28.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.086+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f996010f6f0 con 0x7f9960108840 2024-07-04T08:12:28.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.088+0000 7f9957400640 1 -- 172.21.15.39:0/2620949261 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f994c042070 con 0x7f9960108840 2024-07-04T08:12:28.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.090+0000 7f9957400640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f99440b28d0 0x7f99440b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.090+0000 7f9957400640 1 -- 172.21.15.39:0/2620949261 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f994c1032c0 con 0x7f9960108840 2024-07-04T08:12:28.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.090+0000 7f995ea00640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f99440b28d0 0x7f99440b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.091+0000 7f9957400640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] conn(0x7f99440bddf0 0x7f99440c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.091+0000 7f9957400640 1 -- 172.21.15.39:0/2620949261 --> [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f994c038e90 con 0x7f99440bddf0 2024-07-04T08:12:28.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.091+0000 7f9957400640 1 -- 172.21.15.39:0/2620949261 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f994c103670 con 0x7f9960108840 2024-07-04T08:12:28.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.091+0000 7f995f400640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] conn(0x7f99440bddf0 0x7f99440c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.091+0000 7f995ea00640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f99440b28d0 0x7f99440b4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9950009bd0 tx=0x7f9950005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:28.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.092+0000 7f995f400640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] conn(0x7f99440bddf0 0x7f99440c02e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:28.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.093+0000 7f9957400640 1 -- 172.21.15.39:0/2620949261 <== osd.10 v2:172.21.15.50:6824/2876845435 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f994c038e90 con 0x7f99440bddf0 2024-07-04T08:12:28.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.109+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 --> [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9960104180 con 0x7f99440bddf0 2024-07-04T08:12:28.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.110+0000 7f9957400640 1 -- 172.21.15.39:0/2620949261 <== osd.10 v2:172.21.15.50:6824/2876845435 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9960104180 con 0x7f99440bddf0 2024-07-04T08:12:28.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.110+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 >> [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] conn(0x7f99440bddf0 msgr2=0x7f99440c02e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.110+0000 7f995fe00640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] conn(0x7f99440bddf0 0x7f99440c02e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.111+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f99440b28d0 msgr2=0x7f99440b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.111+0000 7f995fe00640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f99440b28d0 0x7f99440b4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9950009bd0 tx=0x7f9950005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:28.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.111+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9960108840 msgr2=0x7f996019bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.111+0000 7f995fe00640 1 --2- 172.21.15.39:0/2620949261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9960108840 0x7f996019bff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f994c005ab0 tx=0x7f994c003b90 comp rx=0 tx=0).stop 2024-07-04T08:12:28.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.111+0000 7f995f400640 1 -- 172.21.15.39:0/2620949261 reap_dead start 2024-07-04T08:12:28.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.111+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 shutdown_connections 2024-07-04T08:12:28.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.111+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 >> 172.21.15.39:0/2620949261 conn(0x7f99600fe950 msgr2=0x7f996010dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:28.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.111+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 shutdown_connections 2024-07-04T08:12:28.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.111+0000 7f995fe00640 1 -- 172.21.15.39:0/2620949261 wait complete. 2024-07-04T08:12:28.649 INFO:teuthology.orchestra.run.smithi039.stdout:38654705774 2024-07-04T08:12:28.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-07-04T08:12:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.605+0000 7f915fe00640 1 -- 172.21.15.39:0/3091472891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160108840 msgr2=0x7f9160108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.605+0000 7f915fe00640 1 --2- 172.21.15.39:0/3091472891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160108840 0x7f9160108c50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f914c009900 tx=0x7f914c02a410 comp rx=0 tx=0).stop 2024-07-04T08:12:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.606+0000 7f915fe00640 1 -- 172.21.15.39:0/3091472891 shutdown_connections 2024-07-04T08:12:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.606+0000 7f915fe00640 1 --2- 172.21.15.39:0/3091472891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9160109190 0x7f916010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.606+0000 7f915fe00640 1 --2- 172.21.15.39:0/3091472891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160108840 0x7f9160108c50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.606+0000 7f915fe00640 1 --2- 172.21.15.39:0/3091472891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160102f00 0x7f9160103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.606+0000 7f915fe00640 1 -- 172.21.15.39:0/3091472891 >> 172.21.15.39:0/3091472891 conn(0x7f91600fe990 msgr2=0x7f9160100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.606+0000 7f915fe00640 1 -- 172.21.15.39:0/3091472891 shutdown_connections 2024-07-04T08:12:28.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.606+0000 7f915fe00640 1 -- 172.21.15.39:0/3091472891 wait complete. 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.607+0000 7f915fe00640 1 Processor -- start 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.608+0000 7f915fe00640 1 -- start start 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.608+0000 7f915fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9160102f00 0x7f9160074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.608+0000 7f915fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160108840 0x7f91600751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.608+0000 7f915fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160109190 0x7f91600756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.608+0000 7f915fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9160112ea0 con 0x7f9160102f00 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.608+0000 7f915fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9160112d20 con 0x7f9160109190 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.608+0000 7f915fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9160113020 con 0x7f9160108840 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9160102f00 0x7f9160074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160108840 0x7f91600751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160108840 0x7f91600751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57188/0 (socket says 172.21.15.39:57188) 2024-07-04T08:12:28.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9160102f00 0x7f9160074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41876/0 (socket says 172.21.15.39:41876) 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915e000640 1 -- 172.21.15.39:0/3424055037 learned_addr learned my addr 172.21.15.39:0/3424055037 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915f400640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160109190 0x7f91600756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915ea00640 1 -- 172.21.15.39:0/3424055037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160108840 msgr2=0x7f91600751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915ea00640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160108840 0x7f91600751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915ea00640 1 -- 172.21.15.39:0/3424055037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160109190 msgr2=0x7f91600756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915ea00640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9160109190 0x7f91600756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915ea00640 1 -- 172.21.15.39:0/3424055037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91601a7b30 con 0x7f9160102f00 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.609+0000 7f915e000640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160108840 0x7f91600751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.610+0000 7f915ea00640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9160102f00 0x7f9160074c60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f915000e9f0 tx=0x7f915000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.610+0000 7f9164a00640 1 -- 172.21.15.39:0/3424055037 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915000cd80 con 0x7f9160102f00 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.610+0000 7f9164a00640 1 -- 172.21.15.39:0/3424055037 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91500035f0 con 0x7f9160102f00 2024-07-04T08:12:28.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.610+0000 7f9164a00640 1 -- 172.21.15.39:0/3424055037 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150010670 con 0x7f9160102f00 2024-07-04T08:12:28.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.610+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91601a7e20 con 0x7f9160102f00 2024-07-04T08:12:28.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.611+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91601a8280 con 0x7f9160102f00 2024-07-04T08:12:28.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.612+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91600630a0 con 0x7f9160102f00 2024-07-04T08:12:28.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.617+0000 7f9164a00640 1 -- 172.21.15.39:0/3424055037 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f91500037a0 con 0x7f9160102f00 2024-07-04T08:12:28.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.620+0000 7f9164a00640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91440b2950 0x7f91440b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.620+0000 7f9164a00640 1 -- 172.21.15.39:0/3424055037 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f91500e2e20 con 0x7f9160102f00 2024-07-04T08:12:28.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.621+0000 7f915e000640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91440b2950 0x7f91440b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.621+0000 7f9164a00640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] conn(0x7f91440bde70 0x7f91440c0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.621+0000 7f9164a00640 1 -- 172.21.15.39:0/3424055037 --> [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f915001be50 con 0x7f91440bde70 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.621+0000 7f9164a00640 1 -- 172.21.15.39:0/3424055037 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f91500e3270 con 0x7f9160102f00 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.621+0000 7f915f400640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] conn(0x7f91440bde70 0x7f91440c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.622+0000 7f915e000640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91440b2950 0x7f91440b4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f914c006010 tx=0x7f914c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.622+0000 7f915f400640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] conn(0x7f91440bde70 0x7f91440c0360 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.623+0000 7f9164a00640 1 -- 172.21.15.39:0/3424055037 <== osd.8 v2:172.21.15.133:6816/341977020 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f915001be50 con 0x7f91440bde70 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.639+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 --> [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9160063520 con 0x7f91440bde70 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.640+0000 7f9164a00640 1 -- 172.21.15.39:0/3424055037 <== osd.8 v2:172.21.15.133:6816/341977020 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9160063520 con 0x7f91440bde70 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.641+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 >> [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] conn(0x7f91440bde70 msgr2=0x7f91440c0360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.641+0000 7f915fe00640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] conn(0x7f91440bde70 0x7f91440c0360 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.641+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91440b2950 msgr2=0x7f91440b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.641+0000 7f915fe00640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91440b2950 0x7f91440b4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f914c006010 tx=0x7f914c005a20 comp rx=0 tx=0).stop 2024-07-04T08:12:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.641+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9160102f00 msgr2=0x7f9160074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.641+0000 7f915fe00640 1 --2- 172.21.15.39:0/3424055037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9160102f00 0x7f9160074c60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f915000e9f0 tx=0x7f915000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:12:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.641+0000 7f915f400640 1 -- 172.21.15.39:0/3424055037 reap_dead start 2024-07-04T08:12:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.641+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 shutdown_connections 2024-07-04T08:12:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.641+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 >> 172.21.15.39:0/3424055037 conn(0x7f91600fe990 msgr2=0x7f9160100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.642+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 shutdown_connections 2024-07-04T08:12:28.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:28.642+0000 7f915fe00640 1 -- 172.21.15.39:0/3424055037 wait complete. 2024-07-04T08:12:29.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:28 smithi039 ceph-mon[32062]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:29.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:28 smithi050 ceph-mon[36577]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:29.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:28 smithi133 ceph-mon[37050]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:29.451 INFO:teuthology.orchestra.run.smithi039.stdout:313532612657 2024-07-04T08:12:29.451 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-07-04T08:12:29.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.801+0000 7fe167200640 1 -- 172.21.15.39:0/960206986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe16010f6b0 msgr2=0x7fe160111ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:29.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.801+0000 7fe167200640 1 --2- 172.21.15.39:0/960206986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe16010f6b0 0x7fe160111ad0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe158009ca0 tx=0x7fe15802a8b0 comp rx=0 tx=0).stop 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.802+0000 7fe167200640 1 -- 172.21.15.39:0/960206986 shutdown_connections 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.802+0000 7fe167200640 1 --2- 172.21.15.39:0/960206986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe16010f6b0 0x7fe160111ad0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.802+0000 7fe167200640 1 --2- 172.21.15.39:0/960206986 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe1600755d0 0x7fe16010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.802+0000 7fe167200640 1 --2- 172.21.15.39:0/960206986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160074c80 0x7fe160075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.802+0000 7fe167200640 1 -- 172.21.15.39:0/960206986 >> 172.21.15.39:0/960206986 conn(0x7fe1600feae0 msgr2=0x7fe160100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.802+0000 7fe167200640 1 -- 172.21.15.39:0/960206986 shutdown_connections 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.802+0000 7fe167200640 1 -- 172.21.15.39:0/960206986 wait complete. 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.803+0000 7fe167200640 1 Processor -- start 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.803+0000 7fe167200640 1 -- start start 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe167200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160074c80 0x7fe16019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:29.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe167200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1600755d0 0x7fe16019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe167200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe16010f6b0 0x7fe1601a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe167200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe160115260 con 0x7fe1600755d0 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe167200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601150e0 con 0x7fe16010f6b0 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe167200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601153e0 con 0x7fe160074c80 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe165200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe16010f6b0 0x7fe1601a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe15fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160074c80 0x7fe16019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe157e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1600755d0 0x7fe16019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe165200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe16010f6b0 0x7fe1601a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34230/0 (socket says 172.21.15.39:34230) 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.804+0000 7fe165200640 1 -- 172.21.15.39:0/2897194596 learned_addr learned my addr 172.21.15.39:0/2897194596 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe165200640 1 -- 172.21.15.39:0/2897194596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160074c80 msgr2=0x7fe16019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe165200640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160074c80 0x7fe16019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe165200640 1 -- 172.21.15.39:0/2897194596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1600755d0 msgr2=0x7fe16019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:29.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe165200640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1600755d0 0x7fe16019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:29.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe165200640 1 -- 172.21.15.39:0/2897194596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1601a1b60 con 0x7fe16010f6b0 2024-07-04T08:12:29.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe15fe00640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe160074c80 0x7fe16019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:29.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe165200640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe16010f6b0 0x7fe1601a1430 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe158005e30 tx=0x7fe158030dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:29.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe15d600640 1 -- 172.21.15.39:0/2897194596 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158004040 con 0x7fe16010f6b0 2024-07-04T08:12:29.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe15d600640 1 -- 172.21.15.39:0/2897194596 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe158040070 con 0x7fe16010f6b0 2024-07-04T08:12:29.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1601a1df0 con 0x7fe16010f6b0 2024-07-04T08:12:29.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.805+0000 7fe15d600640 1 -- 172.21.15.39:0/2897194596 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158045a80 con 0x7fe16010f6b0 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.806+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1601a85b0 con 0x7fe16010f6b0 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.806+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe160109c20 con 0x7fe16010f6b0 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.809+0000 7fe15d600640 1 -- 172.21.15.39:0/2897194596 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7fe1580041e0 con 0x7fe16010f6b0 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.811+0000 7fe15d600640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe1440b2950 0x7fe1440b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.811+0000 7fe15d600640 1 -- 172.21.15.39:0/2897194596 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fe158103d80 con 0x7fe16010f6b0 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.811+0000 7fe15fe00640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe1440b2950 0x7fe1440b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.811+0000 7fe15d600640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] conn(0x7fe1440bde70 0x7fe1440c0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.811+0000 7fe15d600640 1 -- 172.21.15.39:0/2897194596 --> [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1440c0a30 con 0x7fe1440bde70 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.811+0000 7fe15d600640 1 -- 172.21.15.39:0/2897194596 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fe1581041d0 con 0x7fe16010f6b0 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.811+0000 7fe157e00640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] conn(0x7fe1440bde70 0x7fe1440c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.812+0000 7fe15fe00640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe1440b2950 0x7fe1440b4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe150009fc0 tx=0x7fe150005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:29.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.812+0000 7fe157e00640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] conn(0x7fe1440bde70 0x7fe1440c0360 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.813+0000 7fe15d600640 1 -- 172.21.15.39:0/2897194596 <== osd.4 v2:172.21.15.50:6808/1784161771 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1440c0a30 con 0x7fe1440bde70 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.829+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 --> [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe160076190 con 0x7fe1440bde70 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.830+0000 7fe15d600640 1 -- 172.21.15.39:0/2897194596 <== osd.4 v2:172.21.15.50:6808/1784161771 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe160076190 con 0x7fe1440bde70 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 >> [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] conn(0x7fe1440bde70 msgr2=0x7fe1440c0360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] conn(0x7fe1440bde70 0x7fe1440c0360 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe1440b2950 msgr2=0x7fe1440b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe1440b2950 0x7fe1440b4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe150009fc0 tx=0x7fe150005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe16010f6b0 msgr2=0x7fe1601a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 --2- 172.21.15.39:0/2897194596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe16010f6b0 0x7fe1601a1430 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe158005e30 tx=0x7fe158030dc0 comp rx=0 tx=0).stop 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe165200640 1 -- 172.21.15.39:0/2897194596 reap_dead start 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 shutdown_connections 2024-07-04T08:12:29.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 >> 172.21.15.39:0/2897194596 conn(0x7fe1600feae0 msgr2=0x7fe1601091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 shutdown_connections 2024-07-04T08:12:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:29.831+0000 7fe167200640 1 -- 172.21.15.39:0/2897194596 wait complete. 2024-07-04T08:12:30.902 INFO:teuthology.orchestra.run.smithi039.stdout:287762808886 2024-07-04T08:12:30.902 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-07-04T08:12:30.908 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:31.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:30 smithi039 ceph-mon[32062]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-07-04T08:12:31.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:30 smithi050 ceph-mon[36577]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-07-04T08:12:31.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:30 smithi133 ceph-mon[37050]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-07-04T08:12:31.482 INFO:teuthology.orchestra.run.smithi039.stdout:240518168643 2024-07-04T08:12:31.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-07-04T08:12:32.235 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:33.025 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:32 smithi039 ceph-mon[32062]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-07-04T08:12:33.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:32 smithi050 ceph-mon[36577]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-07-04T08:12:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:32 smithi133 ceph-mon[37050]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-07-04T08:12:33.355 INFO:teuthology.orchestra.run.smithi039.stdout:141733920856 2024-07-04T08:12:33.356 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-07-04T08:12:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.973+0000 7f4bf1e00640 1 -- 172.21.15.39:0/919339910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bec103d10 msgr2=0x7f4bec104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.973+0000 7f4bf1e00640 1 --2- 172.21.15.39:0/919339910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bec103d10 0x7f4bec104120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009900 tx=0x7f4be002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.973+0000 7f4bf1e00640 1 -- 172.21.15.39:0/919339910 shutdown_connections 2024-07-04T08:12:33.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.973+0000 7f4bf1e00640 1 --2- 172.21.15.39:0/919339910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bec10d170 0x7f4bec10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.973+0000 7f4bf1e00640 1 --2- 172.21.15.39:0/919339910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec104660 0x7f4bec10cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.973+0000 7f4bf1e00640 1 --2- 172.21.15.39:0/919339910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bec103d10 0x7f4bec104120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.973+0000 7f4bf1e00640 1 -- 172.21.15.39:0/919339910 >> 172.21.15.39:0/919339910 conn(0x7f4bec0fd810 msgr2=0x7f4bec0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.973+0000 7f4bf1e00640 1 -- 172.21.15.39:0/919339910 shutdown_connections 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.973+0000 7f4bf1e00640 1 -- 172.21.15.39:0/919339910 wait complete. 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.975+0000 7f4bf1e00640 1 Processor -- start 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.975+0000 7f4bf1e00640 1 -- start start 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.975+0000 7f4bf1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bec103d10 0x7f4bec19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.975+0000 7f4bf1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec104660 0x7f4bec19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.975+0000 7f4bf1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bec10d170 0x7f4bec1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:33.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.975+0000 7f4bf1e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec112df0 con 0x7f4bec10d170 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.975+0000 7f4bf1e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec112c70 con 0x7f4bec103d10 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.975+0000 7f4bf1e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4bec112f70 con 0x7f4bec104660 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bec103d10 0x7f4bec19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bec10d170 0x7f4bec1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bec103d10 0x7f4bec19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51720/0 (socket says 172.21.15.39:51720) 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bebe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec104660 0x7f4bec19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf0a00640 1 -- 172.21.15.39:0/218731848 learned_addr learned my addr 172.21.15.39:0/218731848 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf1400640 1 -- 172.21.15.39:0/218731848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec104660 msgr2=0x7f4bec19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf1400640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec104660 0x7f4bec19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf1400640 1 -- 172.21.15.39:0/218731848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bec103d10 msgr2=0x7f4bec19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf1400640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bec103d10 0x7f4bec19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:33.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf1400640 1 -- 172.21.15.39:0/218731848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bec1a1990 con 0x7f4bec10d170 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bebe00640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bec104660 0x7f4bec19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.976+0000 7f4bf1400640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bec10d170 0x7f4bec1a1260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009fd0 tx=0x7f4be400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.977+0000 7f4be9600640 1 -- 172.21.15.39:0/218731848 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be400ce20 con 0x7f4bec10d170 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.977+0000 7f4be9600640 1 -- 172.21.15.39:0/218731848 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be4005020 con 0x7f4bec10d170 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.977+0000 7f4be9600640 1 -- 172.21.15.39:0/218731848 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4010690 con 0x7f4bec10d170 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.977+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bec1a1c80 con 0x7f4bec10d170 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.977+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bec1a83e0 con 0x7f4bec10d170 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.978+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bec102400 con 0x7f4bec10d170 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.981+0000 7f4be9600640 1 -- 172.21.15.39:0/218731848 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f4be40054d0 con 0x7f4bec10d170 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.982+0000 7f4be9600640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4bd40b2950 0x7f4bd40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.982+0000 7f4be9600640 1 -- 172.21.15.39:0/218731848 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4be40e2ff0 con 0x7f4bec10d170 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.983+0000 7f4bf0a00640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4bd40b2950 0x7f4bd40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.983+0000 7f4be9600640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] conn(0x7f4bd40bde70 0x7f4bd40c0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.983+0000 7f4be9600640 1 -- 172.21.15.39:0/218731848 --> [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4be4019ec0 con 0x7f4bd40bde70 2024-07-04T08:12:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.983+0000 7f4be9600640 1 -- 172.21.15.39:0/218731848 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f4be40e3440 con 0x7f4bec10d170 2024-07-04T08:12:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.983+0000 7f4bebe00640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] conn(0x7f4bd40bde70 0x7f4bd40c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.983+0000 7f4bf0a00640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4bd40b2950 0x7f4bd40b4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4be0031440 tx=0x7f4be00058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.984+0000 7f4bebe00640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] conn(0x7f4bd40bde70 0x7f4bd40c0360 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:32.985+0000 7f4be9600640 1 -- 172.21.15.39:0/218731848 <== osd.5 v2:172.21.15.133:6808/787209680 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4be4019ec0 con 0x7f4bd40bde70 2024-07-04T08:12:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.001+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 --> [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bec06aab0 con 0x7f4bd40bde70 2024-07-04T08:12:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.002+0000 7f4be9600640 1 -- 172.21.15.39:0/218731848 <== osd.5 v2:172.21.15.133:6808/787209680 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4bec06aab0 con 0x7f4bd40bde70 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.002+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 >> [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] conn(0x7f4bd40bde70 msgr2=0x7f4bd40c0360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.002+0000 7f4bf1e00640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] conn(0x7f4bd40bde70 0x7f4bd40c0360 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.003+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4bd40b2950 msgr2=0x7f4bd40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.003+0000 7f4bf1e00640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4bd40b2950 0x7f4bd40b4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4be0031440 tx=0x7f4be00058d0 comp rx=0 tx=0).stop 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.003+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bec10d170 msgr2=0x7f4bec1a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.003+0000 7f4bf1e00640 1 --2- 172.21.15.39:0/218731848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bec10d170 0x7f4bec1a1260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009fd0 tx=0x7f4be400ef10 comp rx=0 tx=0).stop 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.003+0000 7f4bf1400640 1 -- 172.21.15.39:0/218731848 reap_dead start 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.003+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 shutdown_connections 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.003+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 >> 172.21.15.39:0/218731848 conn(0x7f4bec0fd810 msgr2=0x7f4bec1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.003+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 shutdown_connections 2024-07-04T08:12:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:33.003+0000 7f4bf1e00640 1 -- 172.21.15.39:0/218731848 wait complete. 2024-07-04T08:12:34.568 INFO:teuthology.orchestra.run.smithi039.stdout:167503724628 2024-07-04T08:12:34.568 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-07-04T08:12:34.685 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:34.689 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:34.692 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:34.696 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:35.015 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:34 smithi039 ceph-mon[32062]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:12:35.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:34 smithi050 ceph-mon[36577]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:12:35.160 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:35.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:34 smithi133 ceph-mon[37050]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:12:35.451 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:37.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:36 smithi050 ceph-mon[36577]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:37.261 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:36 smithi039 ceph-mon[32062]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:37.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:36 smithi133 ceph-mon[37050]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.986+0000 7f4f8b000640 1 -- 172.21.15.39:0/194654274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f84101fc0 msgr2=0x7f4f841023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.986+0000 7f4f8b000640 1 --2- 172.21.15.39:0/194654274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f84101fc0 0x7f4f841023d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009900 tx=0x7f4f7402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.986+0000 7f4f8b000640 1 -- 172.21.15.39:0/194654274 shutdown_connections 2024-07-04T08:12:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.986+0000 7f4f8b000640 1 --2- 172.21.15.39:0/194654274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84107900 0x7f4f84107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.986+0000 7f4f8b000640 1 --2- 172.21.15.39:0/194654274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f84108250 0x7f4f8410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.986+0000 7f4f8b000640 1 --2- 172.21.15.39:0/194654274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f84101fc0 0x7f4f841023d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.986+0000 7f4f8b000640 1 -- 172.21.15.39:0/194654274 >> 172.21.15.39:0/194654274 conn(0x7f4f84078c80 msgr2=0x7f4f84079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.987+0000 7f4f8b000640 1 -- 172.21.15.39:0/194654274 shutdown_connections 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.987+0000 7f4f8b000640 1 -- 172.21.15.39:0/194654274 wait complete. 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.988+0000 7f4f8b000640 1 Processor -- start 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.988+0000 7f4f8b000640 1 -- start start 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f8b000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84101fc0 0x7f4f8419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f8b000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f84107900 0x7f4f8419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f8b000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f84108250 0x7f4f841a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f8b000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84111f10 con 0x7f4f84101fc0 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f8b000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84111d90 con 0x7f4f84107900 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f8b000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84112090 con 0x7f4f84108250 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f83e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84101fc0 0x7f4f8419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:38.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f83e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84101fc0 0x7f4f8419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55764/0 (socket says 172.21.15.39:55764) 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f83e00640 1 -- 172.21.15.39:0/1316081091 learned_addr learned my addr 172.21.15.39:0/1316081091 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f83400640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f84107900 0x7f4f8419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.989+0000 7f4f89000640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f84108250 0x7f4f841a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.990+0000 7f4f83400640 1 -- 172.21.15.39:0/1316081091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f84108250 msgr2=0x7f4f841a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.990+0000 7f4f83400640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f84108250 0x7f4f841a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.990+0000 7f4f83400640 1 -- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84101fc0 msgr2=0x7f4f8419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.990+0000 7f4f83400640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84101fc0 0x7f4f8419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.990+0000 7f4f83400640 1 -- 172.21.15.39:0/1316081091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f841a1af0 con 0x7f4f84107900 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.990+0000 7f4f83e00640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84101fc0 0x7f4f8419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.990+0000 7f4f83400640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f84107900 0x7f4f8419c120 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000e9c0 tx=0x7f4f7000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:38.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.990+0000 7f4f80c00640 1 -- 172.21.15.39:0/1316081091 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7000ccc0 con 0x7f4f84107900 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.991+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f841a1de0 con 0x7f4f84107900 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.991+0000 7f4f80c00640 1 -- 172.21.15.39:0/1316081091 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f700108e0 con 0x7f4f84107900 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.991+0000 7f4f80c00640 1 -- 172.21.15.39:0/1316081091 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f700187a0 con 0x7f4f84107900 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.991+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f841a8540 con 0x7f4f84107900 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.992+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f84103240 con 0x7f4f84107900 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.995+0000 7f4f80c00640 1 -- 172.21.15.39:0/1316081091 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f4f70010430 con 0x7f4f84107900 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.997+0000 7f4f80c00640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f680b2950 0x7f4f680b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.997+0000 7f4f80c00640 1 -- 172.21.15.39:0/1316081091 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4f700e2ae0 con 0x7f4f84107900 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.997+0000 7f4f83e00640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f680b2950 0x7f4f680b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.998+0000 7f4f80c00640 1 -- 172.21.15.39:0/1316081091 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f4f700a9ba0 con 0x7f4f84107900 2024-07-04T08:12:38.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:37.998+0000 7f4f83e00640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f680b2950 0x7f4f680b4e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f740096c0 tx=0x7f4f74005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:38.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.218+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4f841023d0 con 0x7f4f84107900 2024-07-04T08:12:38.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.219+0000 7f4f80c00640 1 -- 172.21.15.39:0/1316081091 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4f700b2270 con 0x7f4f84107900 2024-07-04T08:12:38.220 INFO:teuthology.orchestra.run.smithi039.stdout:81604378727 2024-07-04T08:12:38.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f680b2950 msgr2=0x7f4f680b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:38.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f680b2950 0x7f4f680b4e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f740096c0 tx=0x7f4f74005a20 comp rx=0 tx=0).stop 2024-07-04T08:12:38.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f84107900 msgr2=0x7f4f8419c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:38.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f84107900 0x7f4f8419c120 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000e9c0 tx=0x7f4f7000ee80 comp rx=0 tx=0).stop 2024-07-04T08:12:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 shutdown_connections 2024-07-04T08:12:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f680b2950 0x7f4f680b4e60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f84108250 0x7f4f841a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f84107900 0x7f4f8419c120 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 --2- 172.21.15.39:0/1316081091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f84101fc0 0x7f4f8419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.222+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 >> 172.21.15.39:0/1316081091 conn(0x7f4f84078c80 msgr2=0x7f4f8406b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.223+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 shutdown_connections 2024-07-04T08:12:38.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.223+0000 7f4f8b000640 1 -- 172.21.15.39:0/1316081091 wait complete. 2024-07-04T08:12:39.114 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:38 smithi039 ceph-mon[32062]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:39.114 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:38 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1316081091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-04T08:12:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:38 smithi050 ceph-mon[36577]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:38 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1316081091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-04T08:12:39.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:38 smithi133 ceph-mon[37050]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:39.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:38 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1316081091' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-04T08:12:40.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:39 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1431871503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-04T08:12:40.126 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:40.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:39 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1431871503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-04T08:12:40.218 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:39 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1431871503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-04T08:12:40.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.888+0000 7f2e8f200640 1 -- 172.21.15.39:0/2081578291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e90109510 msgr2=0x7f2e90109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:40.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.888+0000 7f2e8f200640 1 --2- 172.21.15.39:0/2081578291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e90109510 0x7f2e90109920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009ca0 tx=0x7f2e8002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.888+0000 7f2e8f200640 1 -- 172.21.15.39:0/2081578291 shutdown_connections 2024-07-04T08:12:40.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.888+0000 7f2e8f200640 1 --2- 172.21.15.39:0/2081578291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010f7a0 0x7f2e90115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.888+0000 7f2e8f200640 1 --2- 172.21.15.39:0/2081578291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9010ee50 0x7f2e9010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.888+0000 7f2e8f200640 1 --2- 172.21.15.39:0/2081578291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e90109510 0x7f2e90109920 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.888+0000 7f2e8f200640 1 -- 172.21.15.39:0/2081578291 >> 172.21.15.39:0/2081578291 conn(0x7f2e90078b10 msgr2=0x7f2e90107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:40.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.888+0000 7f2e8f200640 1 -- 172.21.15.39:0/2081578291 shutdown_connections 2024-07-04T08:12:40.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.888+0000 7f2e8f200640 1 -- 172.21.15.39:0/2081578291 wait complete. 2024-07-04T08:12:40.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.889+0000 7f2e8f200640 1 Processor -- start 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.889+0000 7f2e8f200640 1 -- start start 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e90109510 0x7f2e9019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9010ee50 0x7f2e901a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010f7a0 0x7f2e901a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901194a0 con 0x7f2e90109510 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2e90119320 con 0x7f2e9010ee50 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e90119620 con 0x7f2e9010f7a0 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e90109510 0x7f2e9019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010f7a0 0x7f2e901a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010f7a0 0x7f2e901a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51882/0 (socket says 172.21.15.39:51882) 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8e800640 1 -- 172.21.15.39:0/1431871503 learned_addr learned my addr 172.21.15.39:0/1431871503 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:40.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.890+0000 7f2e8d400640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9010ee50 0x7f2e901a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.891+0000 7f2e8e800640 1 -- 172.21.15.39:0/1431871503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9010ee50 msgr2=0x7f2e901a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.891+0000 7f2e8e800640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9010ee50 0x7f2e901a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.891+0000 7f2e8e800640 1 -- 172.21.15.39:0/1431871503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e90109510 msgr2=0x7f2e9019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.891+0000 7f2e8e800640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e90109510 0x7f2e9019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.891+0000 7f2e8e800640 1 -- 172.21.15.39:0/1431871503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e901a5dc0 con 0x7f2e9010f7a0 2024-07-04T08:12:40.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.891+0000 7f2e8e800640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010f7a0 0x7f2e901a5690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800e970 tx=0x7f2e8800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:40.483 INFO:teuthology.orchestra.run.smithi039.stdout:214748364875 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.891+0000 7f2e86a00640 1 -- 172.21.15.39:0/1431871503 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800ccc0 con 0x7f2e9010f7a0 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.891+0000 7f2e86a00640 1 -- 172.21.15.39:0/1431871503 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e8800ce60 con 0x7f2e9010f7a0 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.891+0000 7f2e86a00640 1 -- 172.21.15.39:0/1431871503 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88010430 con 0x7f2e9010f7a0 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.892+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e901a60b0 con 0x7f2e9010f7a0 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.892+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e901ac810 con 0x7f2e9010f7a0 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.893+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e90109920 con 0x7f2e9010f7a0 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.895+0000 7f2e86a00640 1 -- 172.21.15.39:0/1431871503 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f2e88004ff0 con 0x7f2e9010f7a0 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.897+0000 7f2e86a00640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2e740b2950 0x7f2e740b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.897+0000 7f2e86a00640 1 -- 172.21.15.39:0/1431871503 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2e880e2c10 con 0x7f2e9010f7a0 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.897+0000 7f2e8de00640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2e740b2950 0x7f2e740b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:40.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.898+0000 7f2e8de00640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2e740b2950 0x7f2e740b4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e80006010 tx=0x7f2e80009820 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:38.901+0000 7f2e86a00640 1 -- 172.21.15.39:0/1431871503 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f2e880aa320 con 0x7f2e9010f7a0 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.128+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2e9002de10 con 0x7f2e9010f7a0 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.129+0000 7f2e86a00640 1 -- 172.21.15.39:0/1431871503 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2e880a9c20 con 0x7f2e9010f7a0 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2e740b2950 msgr2=0x7f2e740b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2e740b2950 0x7f2e740b4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e80006010 tx=0x7f2e80009820 comp rx=0 tx=0).stop 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010f7a0 msgr2=0x7f2e901a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010f7a0 0x7f2e901a5690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800e970 tx=0x7f2e8800ed70 comp rx=0 tx=0).stop 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 shutdown_connections 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2e740b2950 0x7f2e740b4e60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e9010f7a0 0x7f2e901a5690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e9010ee50 0x7f2e901a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 --2- 172.21.15.39:0/1431871503 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e90109510 0x7f2e9019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.132+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 >> 172.21.15.39:0/1431871503 conn(0x7f2e90078b10 msgr2=0x7f2e900792e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.133+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 shutdown_connections 2024-07-04T08:12:40.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:39.133+0000 7f2e8f200640 1 -- 172.21.15.39:0/1431871503 wait complete. 2024-07-04T08:12:41.037 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:40 smithi039 ceph-mon[32062]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:41.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:40 smithi050 ceph-mon[36577]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:41.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:40 smithi133 ceph-mon[37050]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:42.137 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364871 got 214748364875 for osd.7 2024-07-04T08:12:42.138 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:43.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:42 smithi133 ceph-mon[37050]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-07-04T08:12:43.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:42 smithi039 ceph-mon[32062]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-07-04T08:12:43.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:42 smithi050 ceph-mon[36577]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-07-04T08:12:44.885 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:44.928 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378723 got 81604378727 for osd.2 2024-07-04T08:12:44.928 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:45.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:44 smithi039 ceph-mon[32062]: pgmap v406: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-07-04T08:12:45.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:44 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:12:45.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:44 smithi133 ceph-mon[37050]: pgmap v406: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-07-04T08:12:45.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:44 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:12:45.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:44 smithi050 ceph-mon[36577]: pgmap v406: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-07-04T08:12:45.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:44 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:12:46.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.302+0000 7ff452000640 1 -- 172.21.15.39:0/656227108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff44c075320 msgr2=0x7ff44c075730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.302+0000 7ff452000640 1 --2- 172.21.15.39:0/656227108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff44c075320 0x7ff44c075730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff438009900 tx=0x7ff43802a400 comp rx=0 tx=0).stop 2024-07-04T08:12:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.302+0000 7ff452000640 1 -- 172.21.15.39:0/656227108 shutdown_connections 2024-07-04T08:12:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.302+0000 7ff452000640 1 --2- 172.21.15.39:0/656227108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c1137b0 0x7ff44c115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.302+0000 7ff452000640 1 --2- 172.21.15.39:0/656227108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff44c075320 0x7ff44c075730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.302+0000 7ff452000640 1 --2- 172.21.15.39:0/656227108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff44c0768d0 0x7ff44c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.302+0000 7ff452000640 1 -- 172.21.15.39:0/656227108 >> 172.21.15.39:0/656227108 conn(0x7ff44c0fe930 msgr2=0x7ff44c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.303+0000 7ff452000640 1 -- 172.21.15.39:0/656227108 shutdown_connections 2024-07-04T08:12:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.303+0000 7ff452000640 1 -- 172.21.15.39:0/656227108 wait complete. 2024-07-04T08:12:46.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.304+0000 7ff452000640 1 Processor -- start 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.304+0000 7ff452000640 1 -- start start 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff452000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c075320 0x7ff44c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff452000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff44c0768d0 0x7ff44c1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff452000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff44c1137b0 0x7ff44c1a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff452000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c119410 con 0x7ff44c0768d0 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff452000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c119290 con 0x7ff44c075320 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff452000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff44c119590 con 0x7ff44c1137b0 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff44be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff44c0768d0 0x7ff44c1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff451600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff44c1137b0 0x7ff44c1a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff44be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff44c0768d0 0x7ff44c1a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55958/0 (socket says 172.21.15.39:55958) 2024-07-04T08:12:46.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff450c00640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c075320 0x7ff44c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.305+0000 7ff44be00640 1 -- 172.21.15.39:0/1112746841 learned_addr learned my addr 172.21.15.39:0/1112746841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff451600640 1 -- 172.21.15.39:0/1112746841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c075320 msgr2=0x7ff44c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff451600640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c075320 0x7ff44c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff451600640 1 -- 172.21.15.39:0/1112746841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff44c0768d0 msgr2=0x7ff44c1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff451600640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff44c0768d0 0x7ff44c1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff451600640 1 -- 172.21.15.39:0/1112746841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44c1a5d90 con 0x7ff44c1137b0 2024-07-04T08:12:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff450c00640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c075320 0x7ff44c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff451600640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff44c1137b0 0x7ff44c1a5660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff444009fd0 tx=0x7ff44400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:46.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff449600640 1 -- 172.21.15.39:0/1112746841 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44400ce20 con 0x7ff44c1137b0 2024-07-04T08:12:46.829 INFO:teuthology.orchestra.run.smithi039.stdout:115964117089 2024-07-04T08:12:46.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff449600640 1 -- 172.21.15.39:0/1112746841 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff444005020 con 0x7ff44c1137b0 2024-07-04T08:12:46.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff449600640 1 -- 172.21.15.39:0/1112746841 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444010690 con 0x7ff44c1137b0 2024-07-04T08:12:46.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.306+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44c1a6080 con 0x7ff44c1137b0 2024-07-04T08:12:46.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.307+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44c1ac7e0 con 0x7ff44c1137b0 2024-07-04T08:12:46.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.308+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff44c075730 con 0x7ff44c1137b0 2024-07-04T08:12:46.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.310+0000 7ff449600640 1 -- 172.21.15.39:0/1112746841 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7ff4440054d0 con 0x7ff44c1137b0 2024-07-04T08:12:46.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.312+0000 7ff449600640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4340b2950 0x7ff4340b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:46.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.312+0000 7ff449600640 1 -- 172.21.15.39:0/1112746841 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff4440e2f00 con 0x7ff44c1137b0 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.312+0000 7ff450c00640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4340b2950 0x7ff4340b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.313+0000 7ff450c00640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4340b2950 0x7ff4340b4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff440009fc0 tx=0x7ff440005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.316+0000 7ff449600640 1 -- 172.21.15.39:0/1112746841 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7ff4440aa610 con 0x7ff44c1137b0 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.538+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff44c111f50 con 0x7ff44c1137b0 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.539+0000 7ff449600640 1 -- 172.21.15.39:0/1112746841 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff4440a9f10 con 0x7ff44c1137b0 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4340b2950 msgr2=0x7ff4340b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4340b2950 0x7ff4340b4e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff440009fc0 tx=0x7ff440005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff44c1137b0 msgr2=0x7ff44c1a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff44c1137b0 0x7ff44c1a5660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff444009fd0 tx=0x7ff44400ef10 comp rx=0 tx=0).stop 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 shutdown_connections 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4340b2950 0x7ff4340b4e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:46.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff44c0768d0 0x7ff44c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff44c1137b0 0x7ff44c1a5660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 --2- 172.21.15.39:0/1112746841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff44c075320 0x7ff44c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.542+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 >> 172.21.15.39:0/1112746841 conn(0x7ff44c0fe930 msgr2=0x7ff44c10f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.543+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 shutdown_connections 2024-07-04T08:12:46.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:46.543+0000 7ff452000640 1 -- 172.21.15.39:0/1112746841 wait complete. 2024-07-04T08:12:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:46 smithi039 ceph-mon[32062]: pgmap v407: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:47.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:12:47.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:12:47.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:12:47.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:46 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1112746841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-04T08:12:47.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:46 smithi133 ceph-mon[37050]: pgmap v407: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:47.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:12:47.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:12:47.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:12:47.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:46 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1112746841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-04T08:12:47.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.321+0000 7f6ca8000640 1 -- 172.21.15.39:0/121779241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca010a1c0 msgr2=0x7f6ca010c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:47.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.321+0000 7f6ca8000640 1 --2- 172.21.15.39:0/121779241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca010a1c0 0x7f6ca010c220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009930 tx=0x7f6c8c02a420 comp rx=0 tx=0).stop 2024-07-04T08:12:47.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.322+0000 7f6ca8000640 1 -- 172.21.15.39:0/121779241 shutdown_connections 2024-07-04T08:12:47.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.322+0000 7f6ca8000640 1 --2- 172.21.15.39:0/121779241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ca010c760 0x7f6ca010cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.322+0000 7f6ca8000640 1 --2- 172.21.15.39:0/121779241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca010a1c0 0x7f6ca010c220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.322+0000 7f6ca8000640 1 --2- 172.21.15.39:0/121779241 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca006a4e0 0x7f6ca0109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.322+0000 7f6ca8000640 1 -- 172.21.15.39:0/121779241 >> 172.21.15.39:0/121779241 conn(0x7f6ca0070840 msgr2=0x7f6ca00733a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:47.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.322+0000 7f6ca8000640 1 -- 172.21.15.39:0/121779241 shutdown_connections 2024-07-04T08:12:47.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.322+0000 7f6ca8000640 1 -- 172.21.15.39:0/121779241 wait complete. 2024-07-04T08:12:47.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.323+0000 7f6ca8000640 1 Processor -- start 2024-07-04T08:12:47.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.324+0000 7f6ca8000640 1 -- start start 2024-07-04T08:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.324+0000 7f6ca8000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ca006a4e0 0x7f6ca0071490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.324+0000 7f6ca8000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca010a1c0 0x7f6ca00719d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.324+0000 7f6ca8000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca010c760 0x7f6ca0071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:47.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.324+0000 7f6ca8000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca011a740 con 0x7f6ca006a4e0 2024-07-04T08:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.324+0000 7f6ca8000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca011a5c0 con 0x7f6ca010a1c0 2024-07-04T08:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.324+0000 7f6ca8000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca011a8c0 con 0x7f6ca010c760 2024-07-04T08:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.324+0000 7f6ca5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ca006a4e0 0x7f6ca0071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ca006a4e0 0x7f6ca0071490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55982/0 (socket says 172.21.15.39:55982) 2024-07-04T08:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca5600640 1 -- 172.21.15.39:0/2407889581 learned_addr learned my addr 172.21.15.39:0/2407889581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca6000640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca010c760 0x7f6ca0071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:47.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca4c00640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca010a1c0 0x7f6ca00719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca5600640 1 -- 172.21.15.39:0/2407889581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca010c760 msgr2=0x7f6ca0071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca5600640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca010c760 0x7f6ca0071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca5600640 1 -- 172.21.15.39:0/2407889581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca010a1c0 msgr2=0x7f6ca00719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca5600640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca010a1c0 0x7f6ca00719d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca5600640 1 -- 172.21.15.39:0/2407889581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca01b0520 con 0x7f6ca006a4e0 2024-07-04T08:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.325+0000 7f6ca6000640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca010c760 0x7f6ca0071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:47.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.326+0000 7f6ca5600640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ca006a4e0 0x7f6ca0071490 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e9f0 tx=0x7f6c9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.326+0000 7f6c9e000640 1 -- 172.21.15.39:0/2407889581 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400cd80 con 0x7f6ca006a4e0 2024-07-04T08:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.326+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca01b0810 con 0x7f6ca006a4e0 2024-07-04T08:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.326+0000 7f6c9e000640 1 -- 172.21.15.39:0/2407889581 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c940054d0 con 0x7f6ca006a4e0 2024-07-04T08:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.326+0000 7f6c9e000640 1 -- 172.21.15.39:0/2407889581 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94010690 con 0x7f6ca006a4e0 2024-07-04T08:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.326+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca01b0d20 con 0x7f6ca006a4e0 2024-07-04T08:12:47.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.327+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca010c220 con 0x7f6ca006a4e0 2024-07-04T08:12:47.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.330+0000 7f6c9e000640 1 -- 172.21.15.39:0/2407889581 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f6c94004e30 con 0x7f6ca006a4e0 2024-07-04T08:12:47.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.331+0000 7f6c9e000640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c880b28d0 0x7f6c880b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.332+0000 7f6c9e000640 1 -- 172.21.15.39:0/2407889581 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f6c940e2ec0 con 0x7f6ca006a4e0 2024-07-04T08:12:47.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.332+0000 7f6ca4c00640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c880b28d0 0x7f6c880b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.333+0000 7f6ca4c00640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c880b28d0 0x7f6c880b4de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009900 tx=0x7f6c8c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:47.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.334+0000 7f6c9e000640 1 -- 172.21.15.39:0/2407889581 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f6c94055110 con 0x7f6ca006a4e0 2024-07-04T08:12:47.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:46 smithi050 ceph-mon[36577]: pgmap v407: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:47.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:12:47.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:12:47.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:12:47.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:46 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1112746841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-04T08:12:47.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.554+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6ca0072790 con 0x7f6ca006a4e0 2024-07-04T08:12:47.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.556+0000 7f6c9e000640 1 -- 172.21.15.39:0/2407889581 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6ca0072790 con 0x7f6ca006a4e0 2024-07-04T08:12:47.563 INFO:teuthology.orchestra.run.smithi039.stdout:193273528401 2024-07-04T08:12:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.559+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c880b28d0 msgr2=0x7f6c880b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.559+0000 7f6ca8000640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c880b28d0 0x7f6c880b4de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009900 tx=0x7f6c8c005a20 comp rx=0 tx=0).stop 2024-07-04T08:12:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.559+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ca006a4e0 msgr2=0x7f6ca0071490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:47.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.559+0000 7f6ca8000640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ca006a4e0 0x7f6ca0071490 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e9f0 tx=0x7f6c9400edf0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.561+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 shutdown_connections 2024-07-04T08:12:47.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.561+0000 7f6ca8000640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c880b28d0 0x7f6c880b4de0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.561+0000 7f6ca8000640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ca010c760 0x7f6ca0071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.561+0000 7f6ca8000640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ca010a1c0 0x7f6ca00719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.561+0000 7f6ca8000640 1 --2- 172.21.15.39:0/2407889581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ca006a4e0 0x7f6ca0071490 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.561+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 >> 172.21.15.39:0/2407889581 conn(0x7f6ca0070840 msgr2=0x7f6ca0073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:47.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.561+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 shutdown_connections 2024-07-04T08:12:47.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.564+0000 7f6ca8000640 1 -- 172.21.15.39:0/2407889581 wait complete. 2024-07-04T08:12:47.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.791+0000 7fb361800640 1 -- 172.21.15.39:0/3837461833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb35c10b6f0 msgr2=0x7fb35c10bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:47.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.791+0000 7fb361800640 1 --2- 172.21.15.39:0/3837461833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb35c10b6f0 0x7fb35c10bb00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009900 tx=0x7fb34c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.791+0000 7fb361800640 1 -- 172.21.15.39:0/3837461833 shutdown_connections 2024-07-04T08:12:47.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.791+0000 7fb361800640 1 --2- 172.21.15.39:0/3837461833 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c111980 0x7fb35c1181b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.791+0000 7fb361800640 1 --2- 172.21.15.39:0/3837461833 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35c111030 0x7fb35c111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.791+0000 7fb361800640 1 --2- 172.21.15.39:0/3837461833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb35c10b6f0 0x7fb35c10bb00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.791+0000 7fb361800640 1 -- 172.21.15.39:0/3837461833 >> 172.21.15.39:0/3837461833 conn(0x7fb35c0724b0 msgr2=0x7fb35c109910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.791+0000 7fb361800640 1 -- 172.21.15.39:0/3837461833 shutdown_connections 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.791+0000 7fb361800640 1 -- 172.21.15.39:0/3837461833 wait complete. 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.792+0000 7fb361800640 1 Processor -- start 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.793+0000 7fb361800640 1 -- start start 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.793+0000 7fb361800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35c10b6f0 0x7fb35c1a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.793+0000 7fb361800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c111030 0x7fb35c1a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:47.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.793+0000 7fb361800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb35c111980 0x7fb35c1a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.793+0000 7fb361800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c11b650 con 0x7fb35c111030 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.793+0000 7fb361800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c11b4d0 con 0x7fb35c10b6f0 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.793+0000 7fb361800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c11b7d0 con 0x7fb35c111980 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35c10b6f0 0x7fb35c1a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35c10b6f0 0x7fb35c1a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34800/0 (socket says 172.21.15.39:34800) 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c111030 0x7fb35c1a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35be00640 1 -- 172.21.15.39:0/1530482619 learned_addr learned my addr 172.21.15.39:0/1530482619 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35be00640 1 -- 172.21.15.39:0/1530482619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb35c111980 msgr2=0x7fb35c1a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35be00640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb35c111980 0x7fb35c1a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35be00640 1 -- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c111030 msgr2=0x7fb35c1a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35be00640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c111030 0x7fb35c1a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:47.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35be00640 1 -- 172.21.15.39:0/1530482619 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c1aa270 con 0x7fb35c10b6f0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.794+0000 7fb35b400640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c111030 0x7fb35c1a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.795+0000 7fb35be00640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35c10b6f0 0x7fb35c1a4300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb34c005940 tx=0x7fb34c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.795+0000 7fb358c00640 1 -- 172.21.15.39:0/1530482619 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c03f030 con 0x7fb35c10b6f0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.795+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c1aa500 con 0x7fb35c10b6f0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.795+0000 7fb358c00640 1 -- 172.21.15.39:0/1530482619 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb34c0044c0 con 0x7fb35c10b6f0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.795+0000 7fb358c00640 1 -- 172.21.15.39:0/1530482619 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c035070 con 0x7fb35c10b6f0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.795+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c1b0cc0 con 0x7fb35c10b6f0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.797+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c10bb00 con 0x7fb35c10b6f0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.800+0000 7fb358c00640 1 -- 172.21.15.39:0/1530482619 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7fb34c042030 con 0x7fb35c10b6f0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.801+0000 7fb358c00640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3400b2950 0x7fb3400b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.801+0000 7fb358c00640 1 -- 172.21.15.39:0/1530482619 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb34c102e20 con 0x7fb35c10b6f0 2024-07-04T08:12:47.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.801+0000 7fb35b400640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3400b2950 0x7fb3400b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:47.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.802+0000 7fb35b400640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3400b2950 0x7fb3400b4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb35c1118c0 tx=0x7fb348005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:47.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:47.803+0000 7fb358c00640 1 -- 172.21.15.39:0/1530482619 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb34c0ca530 con 0x7fb35c10b6f0 2024-07-04T08:12:48.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.022+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb35c1a5430 con 0x7fb35c10b6f0 2024-07-04T08:12:48.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.023+0000 7fb358c00640 1 -- 172.21.15.39:0/1530482619 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb34c0c9e30 con 0x7fb35c10b6f0 2024-07-04T08:12:48.026 INFO:teuthology.orchestra.run.smithi039.stdout:60129542253 2024-07-04T08:12:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.026+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3400b2950 msgr2=0x7fb3400b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.026+0000 7fb361800640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3400b2950 0x7fb3400b4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb35c1118c0 tx=0x7fb348005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.026+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35c10b6f0 msgr2=0x7fb35c1a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.026+0000 7fb361800640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35c10b6f0 0x7fb35c1a4300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb34c005940 tx=0x7fb34c0040c0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.026+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 shutdown_connections 2024-07-04T08:12:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.026+0000 7fb361800640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3400b2950 0x7fb3400b4e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.026+0000 7fb361800640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb35c111030 0x7fb35c1a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.026+0000 7fb361800640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb35c111980 0x7fb35c1a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.027+0000 7fb361800640 1 --2- 172.21.15.39:0/1530482619 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb35c10b6f0 0x7fb35c1a4300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.027+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 >> 172.21.15.39:0/1530482619 conn(0x7fb35c0724b0 msgr2=0x7fb35c109c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.027+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 shutdown_connections 2024-07-04T08:12:48.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.027+0000 7fb361800640 1 -- 172.21.15.39:0/1530482619 wait complete. 2024-07-04T08:12:48.193 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:47 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2407889581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-04T08:12:48.222 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:48.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.295+0000 7f73c5a00640 1 -- 172.21.15.39:0/1805166524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c0076e90 msgr2=0x7f73c00772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.295+0000 7f73c5a00640 1 --2- 172.21.15.39:0/1805166524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c0076e90 0x7f73c00772a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009900 tx=0x7f73b002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.295+0000 7f73c5a00640 1 -- 172.21.15.39:0/1805166524 shutdown_connections 2024-07-04T08:12:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.295+0000 7f73c5a00640 1 --2- 172.21.15.39:0/1805166524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73c0113c90 0x7f73c01160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.295+0000 7f73c5a00640 1 --2- 172.21.15.39:0/1805166524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c00777e0 0x7f73c0113750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.295+0000 7f73c5a00640 1 --2- 172.21.15.39:0/1805166524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c0076e90 0x7f73c00772a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.295+0000 7f73c5a00640 1 -- 172.21.15.39:0/1805166524 >> 172.21.15.39:0/1805166524 conn(0x7f73c0100d60 msgr2=0x7f73c01031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.295+0000 7f73c5a00640 1 -- 172.21.15.39:0/1805166524 shutdown_connections 2024-07-04T08:12:48.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.295+0000 7f73c5a00640 1 -- 172.21.15.39:0/1805166524 wait complete. 2024-07-04T08:12:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.297+0000 7f73c5a00640 1 Processor -- start 2024-07-04T08:12:48.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.297+0000 7f73c5a00640 1 -- start start 2024-07-04T08:12:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73c5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73c0076e90 0x7f73c019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73c5a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c00777e0 0x7f73c01a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73c5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c0113c90 0x7f73c01a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73c5a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0119600 con 0x7f73c0113c90 2024-07-04T08:12:48.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73c5a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0119480 con 0x7f73c0076e90 2024-07-04T08:12:48.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73c5a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0119780 con 0x7f73c00777e0 2024-07-04T08:12:48.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73bea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73c0076e90 0x7f73c019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:48.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73bea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73c0076e90 0x7f73c019fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34816/0 (socket says 172.21.15.39:34816) 2024-07-04T08:12:48.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73bea00640 1 -- 172.21.15.39:0/3063718205 learned_addr learned my addr 172.21.15.39:0/3063718205 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:48.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73bf400640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c0113c90 0x7f73c01a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:48.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.298+0000 7f73b7e00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c00777e0 0x7f73c01a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.299+0000 7f73bea00640 1 -- 172.21.15.39:0/3063718205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c00777e0 msgr2=0x7f73c01a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.299+0000 7f73bea00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c00777e0 0x7f73c01a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.299+0000 7f73bea00640 1 -- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c0113c90 msgr2=0x7f73c01a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:48.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.299+0000 7f73bea00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c0113c90 0x7f73c01a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.299+0000 7f73bea00640 1 -- 172.21.15.39:0/3063718205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c01a5f60 con 0x7f73c0076e90 2024-07-04T08:12:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.299+0000 7f73b7e00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c00777e0 0x7f73c01a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.299+0000 7f73bf400640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c0113c90 0x7f73c01a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.299+0000 7f73bea00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73c0076e90 0x7f73c019fff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73b0005940 tx=0x7f73b00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.300+0000 7f73b7400640 1 -- 172.21.15.39:0/3063718205 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b003f030 con 0x7f73c0076e90 2024-07-04T08:12:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.300+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c01a61f0 con 0x7f73c0076e90 2024-07-04T08:12:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.300+0000 7f73b7400640 1 -- 172.21.15.39:0/3063718205 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73b0003cb0 con 0x7f73c0076e90 2024-07-04T08:12:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.300+0000 7f73b7400640 1 -- 172.21.15.39:0/3063718205 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0035070 con 0x7f73c0076e90 2024-07-04T08:12:48.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.300+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c01ac9b0 con 0x7f73c0076e90 2024-07-04T08:12:48.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.301+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c00772a0 con 0x7f73c0076e90 2024-07-04T08:12:48.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.303+0000 7f73b7400640 1 -- 172.21.15.39:0/3063718205 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f73b0042030 con 0x7f73c0076e90 2024-07-04T08:12:48.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.305+0000 7f73b7400640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73a40b28d0 0x7f73a40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:48.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.305+0000 7f73b7400640 1 -- 172.21.15.39:0/3063718205 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f73b0103340 con 0x7f73c0076e90 2024-07-04T08:12:48.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.305+0000 7f73b7e00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73a40b28d0 0x7f73a40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:48.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.306+0000 7f73b7e00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73a40b28d0 0x7f73a40b4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f73c0079790 tx=0x7f73a800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:48.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.307+0000 7f73b7400640 1 -- 172.21.15.39:0/3063718205 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f73b00ca400 con 0x7f73c0076e90 2024-07-04T08:12:48.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:47 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2407889581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-04T08:12:48.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:47 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2407889581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-04T08:12:48.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.511+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f73c010ee50 con 0x7f73c0076e90 2024-07-04T08:12:48.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.512+0000 7f73b7400640 1 -- 172.21.15.39:0/3063718205 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f73b00ce700 con 0x7f73c0076e90 2024-07-04T08:12:48.515 INFO:teuthology.orchestra.run.smithi039.stdout:38654705778 2024-07-04T08:12:48.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73a40b28d0 msgr2=0x7f73a40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:48.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73a40b28d0 0x7f73a40b4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f73c0079790 tx=0x7f73a800b390 comp rx=0 tx=0).stop 2024-07-04T08:12:48.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73c0076e90 msgr2=0x7f73c019fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:48.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73c0076e90 0x7f73c019fff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73b0005940 tx=0x7f73b00042f0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 shutdown_connections 2024-07-04T08:12:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f73a40b28d0 0x7f73a40b4de0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73c0113c90 0x7f73c01a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c00777e0 0x7f73c01a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 --2- 172.21.15.39:0/3063718205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f73c0076e90 0x7f73c019fff0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 >> 172.21.15.39:0/3063718205 conn(0x7f73c0100d60 msgr2=0x7f73c0115d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 shutdown_connections 2024-07-04T08:12:48.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:48.515+0000 7f73c5a00640 1 -- 172.21.15.39:0/3063718205 wait complete. 2024-07-04T08:12:49.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:48 smithi133 ceph-mon[37050]: pgmap v408: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:49.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:48 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1530482619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-04T08:12:49.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:48 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3063718205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-04T08:12:49.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:48 smithi039 ceph-mon[32062]: pgmap v408: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:49.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:48 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1530482619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-04T08:12:49.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:48 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3063718205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-04T08:12:49.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:48 smithi050 ceph-mon[36577]: pgmap v408: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:12:49.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:48 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1530482619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-04T08:12:49.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:48 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3063718205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-04T08:12:49.606 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:12:50.137 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528397 got 193273528401 for osd.6 2024-07-04T08:12:50.138 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:50.573 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705774 got 38654705778 for osd.0 2024-07-04T08:12:50.573 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:50.890 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542248 got 60129542253 for osd.1 2024-07-04T08:12:50.890 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:51.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:50 smithi039 ceph-mon[32062]: pgmap v409: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:51.197 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:50 smithi050 ceph-mon[36577]: pgmap v409: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:51.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:50 smithi133 ceph-mon[37050]: pgmap v409: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:52.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:51 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1197998251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-04T08:12:52.276 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117084 got 115964117089 for osd.3 2024-07-04T08:12:52.277 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:52.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:51 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1197998251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-04T08:12:52.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:51 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1197998251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-04T08:12:52.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.514+0000 7f922d400640 1 -- 172.21.15.39:0/3141614841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 msgr2=0x7f9228111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.514+0000 7f922d400640 1 --2- 172.21.15.39:0/3141614841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 0x7f9228111630 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9210009900 tx=0x7f921002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.514+0000 7f922d400640 1 -- 172.21.15.39:0/3141614841 shutdown_connections 2024-07-04T08:12:52.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.514+0000 7f922d400640 1 --2- 172.21.15.39:0/3141614841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 0x7f9228111630 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.514+0000 7f922d400640 1 --2- 172.21.15.39:0/3141614841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9228111b70 0x7f9228118430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.514+0000 7f922d400640 1 --2- 172.21.15.39:0/3141614841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9228109d70 0x7f922810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.514+0000 7f922d400640 1 -- 172.21.15.39:0/3141614841 >> 172.21.15.39:0/3141614841 conn(0x7f9228073c30 msgr2=0x7f9228074070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:52.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.514+0000 7f922d400640 1 -- 172.21.15.39:0/3141614841 shutdown_connections 2024-07-04T08:12:52.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.514+0000 7f922d400640 1 -- 172.21.15.39:0/3141614841 wait complete. 2024-07-04T08:12:52.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.516+0000 7f922d400640 1 Processor -- start 2024-07-04T08:12:52.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.516+0000 7f922d400640 1 -- start start 2024-07-04T08:12:52.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.516+0000 7f922d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9228109d70 0x7f92281a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:52.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.516+0000 7f922d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 0x7f92281a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:52.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.516+0000 7f922d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9228111b70 0x7f92281a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:52.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.516+0000 7f922d400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f922811b820 con 0x7f9228111220 2024-07-04T08:12:52.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.516+0000 7f922d400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f922811b6a0 con 0x7f9228111b70 2024-07-04T08:12:52.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.516+0000 7f922d400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f922811b9a0 con 0x7f9228109d70 2024-07-04T08:12:52.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f921fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 0x7f92281a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f921fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 0x7f92281a49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44876/0 (socket says 172.21.15.39:44876) 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f921fe00640 1 -- 172.21.15.39:0/3940298650 learned_addr learned my addr 172.21.15.39:0/3940298650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f9226800640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9228109d70 0x7f92281a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f9227200640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9228111b70 0x7f92281a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f921fe00640 1 -- 172.21.15.39:0/3940298650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9228109d70 msgr2=0x7f92281a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f921fe00640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9228109d70 0x7f92281a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f921fe00640 1 -- 172.21.15.39:0/3940298650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9228111b70 msgr2=0x7f92281a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f921fe00640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9228111b70 0x7f92281a9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f921fe00640 1 -- 172.21.15.39:0/3940298650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92281aa380 con 0x7f9228111220 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.517+0000 7f9227200640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9228111b70 0x7f92281a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:12:52.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.518+0000 7f921fe00640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 0x7f92281a49b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f921002a910 tx=0x7f9210031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:52.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.518+0000 7f921f400640 1 -- 172.21.15.39:0/3940298650 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921003f030 con 0x7f9228111220 2024-07-04T08:12:52.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.518+0000 7f921f400640 1 -- 172.21.15.39:0/3940298650 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9210004490 con 0x7f9228111220 2024-07-04T08:12:52.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.518+0000 7f921f400640 1 -- 172.21.15.39:0/3940298650 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210035070 con 0x7f9228111220 2024-07-04T08:12:52.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.518+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92281aa610 con 0x7f9228111220 2024-07-04T08:12:52.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.518+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92281b0dd0 con 0x7f9228111220 2024-07-04T08:12:52.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.520+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922810a180 con 0x7f9228111220 2024-07-04T08:12:52.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.522+0000 7f921f400640 1 -- 172.21.15.39:0/3940298650 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f9210042030 con 0x7f9228111220 2024-07-04T08:12:52.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.523+0000 7f921f400640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f920c0b2840 0x7f920c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:52.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.524+0000 7f921f400640 1 -- 172.21.15.39:0/3940298650 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f9210102740 con 0x7f9228111220 2024-07-04T08:12:52.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.524+0000 7f9226800640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f920c0b2840 0x7f920c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:52.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.525+0000 7f9226800640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f920c0b2840 0x7f920c0b4d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9218009bd0 tx=0x7f9218005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:52.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.528+0000 7f921f400640 1 -- 172.21.15.39:0/3940298650 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f92100c9fb0 con 0x7f9228111220 2024-07-04T08:12:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.758+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f92281b0fa0 con 0x7f9228111220 2024-07-04T08:12:52.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.759+0000 7f921f400640 1 -- 172.21.15.39:0/3940298650 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f92100c98b0 con 0x7f9228111220 2024-07-04T08:12:52.761 INFO:teuthology.orchestra.run.smithi039.stdout:266287972416 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f920c0b2840 msgr2=0x7f920c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f920c0b2840 0x7f920c0b4d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9218009bd0 tx=0x7f9218005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 msgr2=0x7f92281a49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 0x7f92281a49b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f921002a910 tx=0x7f9210031a20 comp rx=0 tx=0).stop 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 shutdown_connections 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f920c0b2840 0x7f920c0b4d50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9228111220 0x7f92281a49b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9228111b70 0x7f92281a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 --2- 172.21.15.39:0/3940298650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9228109d70 0x7f92281a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.761+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 >> 172.21.15.39:0/3940298650 conn(0x7f9228073c30 msgr2=0x7f922810f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:52.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.762+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 shutdown_connections 2024-07-04T08:12:52.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:52.762+0000 7f922d400640 1 -- 172.21.15.39:0/3940298650 wait complete. 2024-07-04T08:12:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.350+0000 7f05f7e00640 1 -- 172.21.15.39:0/3394933371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 msgr2=0x7f05f8075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.350+0000 7f05f7e00640 1 --2- 172.21.15.39:0/3394933371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 0x7f05f8075110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009900 tx=0x7f05e802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.350+0000 7f05f7e00640 1 -- 172.21.15.39:0/3394933371 shutdown_connections 2024-07-04T08:12:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.350+0000 7f05f7e00640 1 --2- 172.21.15.39:0/3394933371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f8075650 0x7f05f810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.350+0000 7f05f7e00640 1 --2- 172.21.15.39:0/3394933371 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05f810f440 0x7f05f8111880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.350+0000 7f05f7e00640 1 --2- 172.21.15.39:0/3394933371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 0x7f05f8075110 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.350+0000 7f05f7e00640 1 -- 172.21.15.39:0/3394933371 >> 172.21.15.39:0/3394933371 conn(0x7f05f80fe930 msgr2=0x7f05f8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:52.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.350+0000 7f05f7e00640 1 -- 172.21.15.39:0/3394933371 shutdown_connections 2024-07-04T08:12:52.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.350+0000 7f05f7e00640 1 -- 172.21.15.39:0/3394933371 wait complete. 2024-07-04T08:12:52.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.352+0000 7f05f7e00640 1 Processor -- start 2024-07-04T08:12:52.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.352+0000 7f05f7e00640 1 -- start start 2024-07-04T08:12:52.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.352+0000 7f05f7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 0x7f05f819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:52.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.352+0000 7f05f7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f8075650 0x7f05f819c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:52.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.352+0000 7f05f7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05f810f440 0x7f05f81a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:52.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.352+0000 7f05f7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8114fd0 con 0x7f05f8075650 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.352+0000 7f05f7e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8114e50 con 0x7f05f810f440 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.352+0000 7f05f7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8115150 con 0x7f05f8074d00 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 0x7f05f819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f8075650 0x7f05f819c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 0x7f05f819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59968/0 (socket says 172.21.15.39:59968) 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6a00640 1 -- 172.21.15.39:0/1197998251 learned_addr learned my addr 172.21.15.39:0/1197998251 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f7400640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05f810f440 0x7f05f81a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6a00640 1 -- 172.21.15.39:0/1197998251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05f810f440 msgr2=0x7f05f81a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6a00640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05f810f440 0x7f05f81a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6a00640 1 -- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f8075650 msgr2=0x7f05f819c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6a00640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f8075650 0x7f05f819c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6a00640 1 -- 172.21.15.39:0/1197998251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f81a1a30 con 0x7f05f8074d00 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f7400640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05f810f440 0x7f05f81a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.353+0000 7f05f6000640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f8075650 0x7f05f819c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.354+0000 7f05f6a00640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 0x7f05f819bac0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005940 tx=0x7f05e80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.354+0000 7f05ef400640 1 -- 172.21.15.39:0/1197998251 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e803f030 con 0x7f05f8074d00 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.354+0000 7f05ef400640 1 -- 172.21.15.39:0/1197998251 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e80044c0 con 0x7f05f8074d00 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.354+0000 7f05ef400640 1 -- 172.21.15.39:0/1197998251 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8035070 con 0x7f05f8074d00 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.354+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f81a1cc0 con 0x7f05f8074d00 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.354+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f81a8480 con 0x7f05f8074d00 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.355+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f8075110 con 0x7f05f8074d00 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.358+0000 7f05ef400640 1 -- 172.21.15.39:0/1197998251 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 149720+0+0 (secure 0 0 0) 0x7f05e8042030 con 0x7f05f8074d00 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.361+0000 7f05ef400640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f05dc0b28d0 0x7f05dc0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.361+0000 7f05ef400640 1 -- 172.21.15.39:0/1197998251 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f05e8102920 con 0x7f05f8074d00 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.361+0000 7f05f6000640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f05dc0b28d0 0x7f05dc0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.362+0000 7f05f6000640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f05dc0b28d0 0x7f05dc0b4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05f81011c0 tx=0x7f05e4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:52.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.364+0000 7f05ef400640 1 -- 172.21.15.39:0/1197998251 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f05e80ca030 con 0x7f05f8074d00 2024-07-04T08:12:52.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.631+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f05f81a8720 con 0x7f05f8074d00 2024-07-04T08:12:52.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.632+0000 7f05ef400640 1 -- 172.21.15.39:0/1197998251 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f05e80c9930 con 0x7f05f8074d00 2024-07-04T08:12:52.781 INFO:teuthology.orchestra.run.smithi039.stdout:313532612663 2024-07-04T08:12:52.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f05dc0b28d0 msgr2=0x7f05dc0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f05dc0b28d0 0x7f05dc0b4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05f81011c0 tx=0x7f05e4005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 msgr2=0x7f05f819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 0x7f05f819bac0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005940 tx=0x7f05e80040c0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 shutdown_connections 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f05dc0b28d0 0x7f05dc0b4de0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05f8075650 0x7f05f819c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f05f810f440 0x7f05f81a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 --2- 172.21.15.39:0/1197998251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05f8074d00 0x7f05f819bac0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 >> 172.21.15.39:0/1197998251 conn(0x7f05f80fe930 msgr2=0x7f05f8108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.634+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 shutdown_connections 2024-07-04T08:12:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:51.635+0000 7f05f7e00640 1 -- 172.21.15.39:0/1197998251 wait complete. 2024-07-04T08:12:53.206 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:52 smithi039 ceph-mon[32062]: pgmap v410: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:53.206 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:52 smithi039 ceph-mon[32062]: mgrmap e22: x(active, since 11m), standbys: z, y 2024-07-04T08:12:53.206 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:52 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3940298650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-04T08:12:53.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:52 smithi133 ceph-mon[37050]: pgmap v410: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:53.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:52 smithi133 ceph-mon[37050]: mgrmap e22: x(active, since 11m), standbys: z, y 2024-07-04T08:12:53.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:52 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3940298650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-04T08:12:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:52 smithi050 ceph-mon[36577]: pgmap v410: 129 pgs: 129 active+clean; 646 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:12:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:52 smithi050 ceph-mon[36577]: mgrmap e22: x(active, since 11m), standbys: z, y 2024-07-04T08:12:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:52 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3940298650' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-04T08:12:53.595 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612657 got 313532612663 for osd.11 2024-07-04T08:12:53.596 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.061+0000 7fb007200640 1 -- 172.21.15.39:0/385511884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb008102f00 msgr2=0x7fb008103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.061+0000 7fb007200640 1 --2- 172.21.15.39:0/385511884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb008102f00 0x7fb008103310 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faff8009900 tx=0x7faff802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.061+0000 7fb007200640 1 -- 172.21.15.39:0/385511884 shutdown_connections 2024-07-04T08:12:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.061+0000 7fb007200640 1 --2- 172.21.15.39:0/385511884 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb008109190 0x7fb00810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.061+0000 7fb007200640 1 --2- 172.21.15.39:0/385511884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb008108840 0x7fb008108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.061+0000 7fb007200640 1 --2- 172.21.15.39:0/385511884 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb008102f00 0x7fb008103310 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.061+0000 7fb007200640 1 -- 172.21.15.39:0/385511884 >> 172.21.15.39:0/385511884 conn(0x7fb0080fe990 msgr2=0x7fb008100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.061+0000 7fb007200640 1 -- 172.21.15.39:0/385511884 shutdown_connections 2024-07-04T08:12:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.061+0000 7fb007200640 1 -- 172.21.15.39:0/385511884 wait complete. 2024-07-04T08:12:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.062+0000 7fb007200640 1 Processor -- start 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.062+0000 7fb007200640 1 -- start start 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb007200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb008102f00 0x7fb008198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb007200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb008108840 0x7fb008198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb007200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb008109190 0x7fb00819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb007200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb008112e50 con 0x7fb008109190 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb007200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb008112cd0 con 0x7fb008108840 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb007200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb008112fd0 con 0x7fb008102f00 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb005e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb008102f00 0x7fb008198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb006800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb008109190 0x7fb00819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb005e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb008102f00 0x7fb008198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60012/0 (socket says 172.21.15.39:60012) 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb005e00640 1 -- 172.21.15.39:0/1163523109 learned_addr learned my addr 172.21.15.39:0/1163523109 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.063+0000 7fb005400640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb008108840 0x7fb008198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.064+0000 7fb006800640 1 -- 172.21.15.39:0/1163523109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb008102f00 msgr2=0x7fb008198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.064+0000 7fb006800640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb008102f00 0x7fb008198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.064+0000 7fb006800640 1 -- 172.21.15.39:0/1163523109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb008108840 msgr2=0x7fb008198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.064+0000 7fb006800640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb008108840 0x7fb008198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.064+0000 7fb006800640 1 -- 172.21.15.39:0/1163523109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00819f8a0 con 0x7fb008109190 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.064+0000 7fb006800640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb008109190 0x7fb00819f360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb00000da10 tx=0x7fb00000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.064+0000 7faffea00640 1 -- 172.21.15.39:0/1163523109 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00000bd70 con 0x7fb008109190 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.065+0000 7faffea00640 1 -- 172.21.15.39:0/1163523109 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb000003e20 con 0x7fb008109190 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.065+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00819fb90 con 0x7fb008109190 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.065+0000 7faffea00640 1 -- 172.21.15.39:0/1163523109 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0000106c0 con 0x7fb008109190 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.065+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0081a0080 con 0x7fb008109190 2024-07-04T08:12:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.066+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb008103310 con 0x7fb008109190 2024-07-04T08:12:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.069+0000 7faffea00640 1 -- 172.21.15.39:0/1163523109 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fb000004270 con 0x7fb008109190 2024-07-04T08:12:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.070+0000 7faffea00640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fafec0b2840 0x7fafec0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.070+0000 7faffea00640 1 -- 172.21.15.39:0/1163523109 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb0000e2a30 con 0x7fb008109190 2024-07-04T08:12:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.071+0000 7fb005e00640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fafec0b2840 0x7fafec0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:54.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.071+0000 7fb005e00640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fafec0b2840 0x7fafec0b4d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faff8009230 tx=0x7faff80058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.075+0000 7faffea00640 1 -- 172.21.15.39:0/1163523109 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb0000aa2a0 con 0x7fb008109190 2024-07-04T08:12:54.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.294+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb008101670 con 0x7fb008109190 2024-07-04T08:12:54.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.295+0000 7faffea00640 1 -- 172.21.15.39:0/1163523109 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb0000a9ba0 con 0x7fb008109190 2024-07-04T08:12:54.297 INFO:teuthology.orchestra.run.smithi039.stdout:141733920861 2024-07-04T08:12:54.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fafec0b2840 msgr2=0x7fafec0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:54.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fafec0b2840 0x7fafec0b4d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faff8009230 tx=0x7faff80058d0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb008109190 msgr2=0x7fb00819f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:54.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb008109190 0x7fb00819f360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb00000da10 tx=0x7fb00000ded0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 shutdown_connections 2024-07-04T08:12:54.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fafec0b2840 0x7fafec0b4d50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb008109190 0x7fb00819f360 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb008108840 0x7fb008198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 --2- 172.21.15.39:0/1163523109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb008102f00 0x7fb008198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:54.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 >> 172.21.15.39:0/1163523109 conn(0x7fb0080fe990 msgr2=0x7fb0080feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:54.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 shutdown_connections 2024-07-04T08:12:54.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:54.298+0000 7fb007200640 1 -- 172.21.15.39:0/1163523109 wait complete. 2024-07-04T08:12:55.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:54 smithi133 ceph-mon[37050]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-04T08:12:55.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:54 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1163523109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-04T08:12:55.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:54 smithi039 ceph-mon[32062]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-04T08:12:55.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:54 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1163523109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-04T08:12:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:54 smithi050 ceph-mon[36577]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-04T08:12:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:54 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1163523109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-04T08:12:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.159+0000 7f5f56600640 1 -- 172.21.15.39:0/657921840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5010ee50 msgr2=0x7f5f5010f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.159+0000 7f5f56600640 1 --2- 172.21.15.39:0/657921840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5010ee50 0x7f5f5010f260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009900 tx=0x7f5f3c02a410 comp rx=0 tx=0).stop 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.159+0000 7f5f56600640 1 -- 172.21.15.39:0/657921840 shutdown_connections 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.159+0000 7f5f56600640 1 --2- 172.21.15.39:0/657921840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5010ee50 0x7f5f5010f260 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.159+0000 7f5f56600640 1 --2- 172.21.15.39:0/657921840 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f5010f7a0 0x7f5f50115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.159+0000 7f5f56600640 1 --2- 172.21.15.39:0/657921840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f50109510 0x7f5f50109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.159+0000 7f5f56600640 1 -- 172.21.15.39:0/657921840 >> 172.21.15.39:0/657921840 conn(0x7f5f50078b10 msgr2=0x7f5f50107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.159+0000 7f5f56600640 1 -- 172.21.15.39:0/657921840 shutdown_connections 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.159+0000 7f5f56600640 1 -- 172.21.15.39:0/657921840 wait complete. 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.160+0000 7f5f56600640 1 Processor -- start 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.160+0000 7f5f56600640 1 -- start start 2024-07-04T08:12:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f56600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f50109510 0x7f5f5019fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f56600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5010ee50 0x7f5f501a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f56600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f5010f7a0 0x7f5f501a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f56600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50119410 con 0x7f5f5010ee50 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f56600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50119290 con 0x7f5f5010f7a0 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f56600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50119590 con 0x7f5f50109510 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f55200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f50109510 0x7f5f5019fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f4fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5010ee50 0x7f5f501a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f55200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f50109510 0x7f5f5019fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60030/0 (socket says 172.21.15.39:60030) 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f55200640 1 -- 172.21.15.39:0/722189903 learned_addr learned my addr 172.21.15.39:0/722189903 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.161+0000 7f5f55c00640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f5010f7a0 0x7f5f501a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.162+0000 7f5f55200640 1 -- 172.21.15.39:0/722189903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f5010f7a0 msgr2=0x7f5f501a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.162+0000 7f5f55200640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f5010f7a0 0x7f5f501a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.162+0000 7f5f55200640 1 -- 172.21.15.39:0/722189903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5010ee50 msgr2=0x7f5f501a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.162+0000 7f5f55200640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5010ee50 0x7f5f501a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.162+0000 7f5f55200640 1 -- 172.21.15.39:0/722189903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f501a5e10 con 0x7f5f50109510 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.162+0000 7f5f55200640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f50109510 0x7f5f5019fea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400ea60 tx=0x7f5f4400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.162+0000 7f5f4d600640 1 -- 172.21.15.39:0/722189903 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400cde0 con 0x7f5f50109510 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.162+0000 7f5f4d600640 1 -- 172.21.15.39:0/722189903 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f440035f0 con 0x7f5f50109510 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.162+0000 7f5f4d600640 1 -- 172.21.15.39:0/722189903 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44010690 con 0x7f5f50109510 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.163+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f501a6100 con 0x7f5f50109510 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.163+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f501ac860 con 0x7f5f50109510 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.164+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f50109920 con 0x7f5f50109510 2024-07-04T08:12:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.166+0000 7f5f4d600640 1 -- 172.21.15.39:0/722189903 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f5f440037a0 con 0x7f5f50109510 2024-07-04T08:12:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.168+0000 7f5f4d600640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5f380b2840 0x7f5f380b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.168+0000 7f5f4d600640 1 -- 172.21.15.39:0/722189903 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f5f440e2bc0 con 0x7f5f50109510 2024-07-04T08:12:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.168+0000 7f5f4fe00640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5f380b2840 0x7f5f380b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.169+0000 7f5f4fe00640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5f380b2840 0x7f5f380b4d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009230 tx=0x7f5f3c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.169+0000 7f5f4d600640 1 -- 172.21.15.39:0/722189903 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5f44055110 con 0x7f5f50109510 2024-07-04T08:12:55.611 INFO:teuthology.orchestra.run.smithi039.stdout:240518168648 2024-07-04T08:12:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.391+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f5f501a0fd0 con 0x7f5f50109510 2024-07-04T08:12:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.392+0000 7f5f4d600640 1 -- 172.21.15.39:0/722189903 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5f440aa430 con 0x7f5f50109510 2024-07-04T08:12:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5f380b2840 msgr2=0x7f5f380b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5f380b2840 0x7f5f380b4d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009230 tx=0x7f5f3c031400 comp rx=0 tx=0).stop 2024-07-04T08:12:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f50109510 msgr2=0x7f5f5019fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f50109510 0x7f5f5019fea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400ea60 tx=0x7f5f4400ef20 comp rx=0 tx=0).stop 2024-07-04T08:12:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 shutdown_connections 2024-07-04T08:12:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5f380b2840 0x7f5f380b4d50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f5010ee50 0x7f5f501a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5f5010f7a0 0x7f5f501a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 --2- 172.21.15.39:0/722189903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f50109510 0x7f5f5019fea0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 >> 172.21.15.39:0/722189903 conn(0x7f5f50078b10 msgr2=0x7f5f50079410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 shutdown_connections 2024-07-04T08:12:55.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:55.395+0000 7f5f56600640 1 -- 172.21.15.39:0/722189903 wait complete. 2024-07-04T08:12:56.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:55 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/722189903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-04T08:12:56.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:55 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/722189903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-04T08:12:56.390 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972409 got 266287972416 for osd.9 2024-07-04T08:12:56.390 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:56.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:55 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/722189903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-04T08:12:57.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:56 smithi039 ceph-mon[32062]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-04T08:12:57.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:56 smithi133 ceph-mon[37050]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-04T08:12:57.368 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920856 got 141733920861 for osd.4 2024-07-04T08:12:57.369 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:57.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:56 smithi050 ceph-mon[36577]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-04T08:12:57.618 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168643 got 240518168648 for osd.8 2024-07-04T08:12:57.618 DEBUG:teuthology.parallel:result is None 2024-07-04T08:12:58.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.481+0000 7f7a32a00640 1 -- 172.21.15.39:0/2676391717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a2c108050 msgr2=0x7f7a2c10e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.481+0000 7f7a32a00640 1 --2- 172.21.15.39:0/2676391717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a2c108050 0x7f7a2c10e880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009ca0 tx=0x7f7a2402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.481+0000 7f7a32a00640 1 -- 172.21.15.39:0/2676391717 shutdown_connections 2024-07-04T08:12:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.481+0000 7f7a32a00640 1 --2- 172.21.15.39:0/2676391717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a2c108050 0x7f7a2c10e880 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.481+0000 7f7a32a00640 1 --2- 172.21.15.39:0/2676391717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c107700 0x7f7a2c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.481+0000 7f7a32a00640 1 --2- 172.21.15.39:0/2676391717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a2c101dc0 0x7f7a2c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.481+0000 7f7a32a00640 1 -- 172.21.15.39:0/2676391717 >> 172.21.15.39:0/2676391717 conn(0x7f7a2c0fd810 msgr2=0x7f7a2c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.482+0000 7f7a32a00640 1 -- 172.21.15.39:0/2676391717 shutdown_connections 2024-07-04T08:12:58.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.482+0000 7f7a32a00640 1 -- 172.21.15.39:0/2676391717 wait complete. 2024-07-04T08:12:58.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.483+0000 7f7a32a00640 1 Processor -- start 2024-07-04T08:12:58.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.483+0000 7f7a32a00640 1 -- start start 2024-07-04T08:12:58.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a32a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a2c101dc0 0x7f7a2c19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:58.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a32a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a2c107700 0x7f7a2c19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:58.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a32a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c108050 0x7f7a2c1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:58.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a32a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c111d10 con 0x7f7a2c101dc0 2024-07-04T08:12:58.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a32a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c111b90 con 0x7f7a2c108050 2024-07-04T08:12:58.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a32a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c111e90 con 0x7f7a2c107700 2024-07-04T08:12:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a31600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a2c101dc0 0x7f7a2c19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:58.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a31600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a2c101dc0 0x7f7a2c19b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44928/0 (socket says 172.21.15.39:44928) 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a31600640 1 -- 172.21.15.39:0/665340590 learned_addr learned my addr 172.21.15.39:0/665340590 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a30c00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a2c107700 0x7f7a2c19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.484+0000 7f7a32000640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c108050 0x7f7a2c1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a30c00640 1 -- 172.21.15.39:0/665340590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c108050 msgr2=0x7f7a2c1a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a30c00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c108050 0x7f7a2c1a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a30c00640 1 -- 172.21.15.39:0/665340590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a2c101dc0 msgr2=0x7f7a2c19b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a30c00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a2c101dc0 0x7f7a2c19b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a30c00640 1 -- 172.21.15.39:0/665340590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c1a1920 con 0x7f7a2c107700 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a30c00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a2c107700 0x7f7a2c19bf50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c069fe0 tx=0x7f7a1800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a2a000640 1 -- 172.21.15.39:0/665340590 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800cc40 con 0x7f7a2c107700 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a2a000640 1 -- 172.21.15.39:0/665340590 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a1800cde0 con 0x7f7a2c107700 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a2a000640 1 -- 172.21.15.39:0/665340590 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18018780 con 0x7f7a2c107700 2024-07-04T08:12:58.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.485+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c1a1c10 con 0x7f7a2c107700 2024-07-04T08:12:58.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.486+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c1a8370 con 0x7f7a2c107700 2024-07-04T08:12:58.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.487+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c1021d0 con 0x7f7a2c107700 2024-07-04T08:12:58.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.490+0000 7f7a2a000640 1 -- 172.21.15.39:0/665340590 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f7a18010a90 con 0x7f7a2c107700 2024-07-04T08:12:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.491+0000 7f7a2a000640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a140b2840 0x7f7a140b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.491+0000 7f7a2a000640 1 -- 172.21.15.39:0/665340590 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7a180e2e80 con 0x7f7a2c107700 2024-07-04T08:12:58.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.492+0000 7f7a31600640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a140b2840 0x7f7a140b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:58.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.492+0000 7f7a31600640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a140b2840 0x7f7a140b4d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009c90 tx=0x7f7a20005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:58.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.493+0000 7f7a2a000640 1 -- 172.21.15.39:0/665340590 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f7a180aa0a0 con 0x7f7a2c107700 2024-07-04T08:12:58.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.715+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7a2c1a8610 con 0x7f7a2c107700 2024-07-04T08:12:58.717 INFO:teuthology.orchestra.run.smithi039.stdout:287762808892 2024-07-04T08:12:58.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.716+0000 7f7a2a000640 1 -- 172.21.15.39:0/665340590 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7a180ae3a0 con 0x7f7a2c107700 2024-07-04T08:12:58.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.718+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a140b2840 msgr2=0x7f7a140b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:58.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.718+0000 7f7a32a00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a140b2840 0x7f7a140b4d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009c90 tx=0x7f7a20005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:58.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.718+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a2c107700 msgr2=0x7f7a2c19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:58.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.718+0000 7f7a32a00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a2c107700 0x7f7a2c19bf50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c069fe0 tx=0x7f7a1800ee50 comp rx=0 tx=0).stop 2024-07-04T08:12:58.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.719+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 shutdown_connections 2024-07-04T08:12:58.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.719+0000 7f7a32a00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a140b2840 0x7f7a140b4d50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.719+0000 7f7a32a00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a2c108050 0x7f7a2c1a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.719+0000 7f7a32a00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a2c107700 0x7f7a2c19bf50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.719+0000 7f7a32a00640 1 --2- 172.21.15.39:0/665340590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a2c101dc0 0x7f7a2c19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:58.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.719+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 >> 172.21.15.39:0/665340590 conn(0x7f7a2c0fd810 msgr2=0x7f7a2c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:58.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.720+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 shutdown_connections 2024-07-04T08:12:58.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:58.720+0000 7f7a32a00640 1 -- 172.21.15.39:0/665340590 wait complete. 2024-07-04T08:12:59.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:58 smithi039 ceph-mon[32062]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-07-04T08:12:59.293 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:58 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/665340590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-04T08:12:59.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:58 smithi133 ceph-mon[37050]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-07-04T08:12:59.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:58 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/665340590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-04T08:12:59.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:58 smithi050 ceph-mon[36577]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-07-04T08:12:59.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:58 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/665340590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-04T08:12:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.395+0000 7f5443e00640 1 -- 172.21.15.39:0/985970745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 msgr2=0x7f5444103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.395+0000 7f5443e00640 1 --2- 172.21.15.39:0/985970745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 0x7f5444103310 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5434009ca0 tx=0x7f543402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.395+0000 7f5443e00640 1 -- 172.21.15.39:0/985970745 shutdown_connections 2024-07-04T08:12:59.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.395+0000 7f5443e00640 1 --2- 172.21.15.39:0/985970745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5444109190 0x7f544410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.395+0000 7f5443e00640 1 --2- 172.21.15.39:0/985970745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5444108840 0x7f5444108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.395+0000 7f5443e00640 1 --2- 172.21.15.39:0/985970745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 0x7f5444103310 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.395+0000 7f5443e00640 1 -- 172.21.15.39:0/985970745 >> 172.21.15.39:0/985970745 conn(0x7f54440fe990 msgr2=0x7f5444100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:59.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.395+0000 7f5443e00640 1 -- 172.21.15.39:0/985970745 shutdown_connections 2024-07-04T08:12:59.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.395+0000 7f5443e00640 1 -- 172.21.15.39:0/985970745 wait complete. 2024-07-04T08:12:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.396+0000 7f5443e00640 1 Processor -- start 2024-07-04T08:12:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.396+0000 7f5443e00640 1 -- start start 2024-07-04T08:12:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5443e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 0x7f544419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5443e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5444108840 0x7f544419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5443e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5444109190 0x7f54441a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5443e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5444112ea0 con 0x7f5444102f00 2024-07-04T08:12:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5443e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5444112d20 con 0x7f5444109190 2024-07-04T08:12:59.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5443e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5444113020 con 0x7f5444108840 2024-07-04T08:12:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5442a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 0x7f544419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5442a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 0x7f544419ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44942/0 (socket says 172.21.15.39:44942) 2024-07-04T08:12:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5442a00640 1 -- 172.21.15.39:0/2494011684 learned_addr learned my addr 172.21.15.39:0/2494011684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:12:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f543a000640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5444108840 0x7f544419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:59.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.397+0000 7f5443400640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5444109190 0x7f54441a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.398+0000 7f5442a00640 1 -- 172.21.15.39:0/2494011684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5444108840 msgr2=0x7f544419c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.398+0000 7f5442a00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5444108840 0x7f544419c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.398+0000 7f5442a00640 1 -- 172.21.15.39:0/2494011684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5444109190 msgr2=0x7f54441a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.398+0000 7f5442a00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5444109190 0x7f54441a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.398+0000 7f5442a00640 1 -- 172.21.15.39:0/2494011684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54441a1a00 con 0x7f5444102f00 2024-07-04T08:12:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.398+0000 7f543a000640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5444108840 0x7f544419c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.398+0000 7f5443400640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5444109190 0x7f54441a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:12:59.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.398+0000 7f5442a00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 0x7f544419ba90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5434005e30 tx=0x7f5434031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:59.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.399+0000 7f543be00640 1 -- 172.21.15.39:0/2494011684 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5434003b70 con 0x7f5444102f00 2024-07-04T08:12:59.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.399+0000 7f543be00640 1 -- 172.21.15.39:0/2494011684 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5434003d10 con 0x7f5444102f00 2024-07-04T08:12:59.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.399+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54441a1c90 con 0x7f5444102f00 2024-07-04T08:12:59.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.399+0000 7f543be00640 1 -- 172.21.15.39:0/2494011684 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543402fa30 con 0x7f5444102f00 2024-07-04T08:12:59.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.399+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54441a8450 con 0x7f5444102f00 2024-07-04T08:12:59.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.400+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5444103310 con 0x7f5444102f00 2024-07-04T08:12:59.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.403+0000 7f543be00640 1 -- 172.21.15.39:0/2494011684 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f543402fbd0 con 0x7f5444102f00 2024-07-04T08:12:59.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.404+0000 7f543be00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54280b28c0 0x7f54280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:12:59.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.404+0000 7f543be00640 1 -- 172.21.15.39:0/2494011684 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f5434103ab0 con 0x7f5444102f00 2024-07-04T08:12:59.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.405+0000 7f543a000640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54280b28c0 0x7f54280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:12:59.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.405+0000 7f543a000640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54280b28c0 0x7f54280b4dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5444104f40 tx=0x7f542c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:12:59.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.411+0000 7f543be00640 1 -- 172.21.15.39:0/2494011684 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f54340cb320 con 0x7f5444102f00 2024-07-04T08:12:59.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.632+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f544419cbc0 con 0x7f5444102f00 2024-07-04T08:12:59.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.633+0000 7f543be00640 1 -- 172.21.15.39:0/2494011684 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f544419cbc0 con 0x7f5444102f00 2024-07-04T08:12:59.634 INFO:teuthology.orchestra.run.smithi039.stdout:167503724633 2024-07-04T08:12:59.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54280b28c0 msgr2=0x7f54280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54280b28c0 0x7f54280b4dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5444104f40 tx=0x7f542c005f90 comp rx=0 tx=0).stop 2024-07-04T08:12:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 msgr2=0x7f544419ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:12:59.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 0x7f544419ba90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5434005e30 tx=0x7f5434031840 comp rx=0 tx=0).stop 2024-07-04T08:12:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 shutdown_connections 2024-07-04T08:12:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54280b28c0 0x7f54280b4dd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5444109190 0x7f54441a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5444108840 0x7f544419c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 --2- 172.21.15.39:0/2494011684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5444102f00 0x7f544419ba90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:12:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 >> 172.21.15.39:0/2494011684 conn(0x7f54440fe990 msgr2=0x7f5444107ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:12:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 shutdown_connections 2024-07-04T08:12:59.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:12:59.636+0000 7f5443e00640 1 -- 172.21.15.39:0/2494011684 wait complete. 2024-07-04T08:13:00.149 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808886 got 287762808892 for osd.10 2024-07-04T08:13:00.149 DEBUG:teuthology.parallel:result is None 2024-07-04T08:13:00.280 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:12:59 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2494011684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-04T08:13:00.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:12:59 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2494011684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-04T08:13:00.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:12:59 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2494011684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-04T08:13:00.698 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724628 got 167503724633 for osd.5 2024-07-04T08:13:00.698 DEBUG:teuthology.parallel:result is None 2024-07-04T08:13:00.698 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-04T08:13:00.698 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph pg dump --format=json 2024-07-04T08:13:01.203 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:01.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:00 smithi133 ceph-mon[37050]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-04T08:13:01.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:00 smithi039 ceph-mon[32062]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-04T08:13:01.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:00 smithi050 ceph-mon[36577]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-04T08:13:03.238 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:03 smithi039 ceph-mon[32062]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-04T08:13:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:03 smithi133 ceph-mon[37050]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-04T08:13:03.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:03 smithi050 ceph-mon[36577]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-04T08:13:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.551+0000 7f2e1c800640 1 -- 172.21.15.39:0/3518333697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1410ac40 msgr2=0x7f2e1410b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.551+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3518333697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1410ac40 0x7f2e1410b050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009900 tx=0x7f2e0402a410 comp rx=0 tx=0).stop 2024-07-04T08:13:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.551+0000 7f2e1c800640 1 -- 172.21.15.39:0/3518333697 shutdown_connections 2024-07-04T08:13:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.551+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3518333697 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e1410ac40 0x7f2e1410b050 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.551+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3518333697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e1410b590 0x7f2e14111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.551+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3518333697 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e14105300 0x7f2e14105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.551+0000 7f2e1c800640 1 -- 172.21.15.39:0/3518333697 >> 172.21.15.39:0/3518333697 conn(0x7f2e14100d10 msgr2=0x7f2e14103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.551+0000 7f2e1c800640 1 -- 172.21.15.39:0/3518333697 shutdown_connections 2024-07-04T08:13:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.551+0000 7f2e1c800640 1 -- 172.21.15.39:0/3518333697 wait complete. 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.552+0000 7f2e1c800640 1 Processor -- start 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e1c800640 1 -- start start 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e1c800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e14105300 0x7f2e1419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e1c800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e1410ac40 0x7f2e1419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e1c800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e1410b590 0x7f2e141a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e1c800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2e14115270 con 0x7f2e14105300 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e1c800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141150f0 con 0x7f2e1410ac40 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e1c800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e141153f0 con 0x7f2e1410b590 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e19e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e14105300 0x7f2e1419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e19e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e14105300 0x7f2e1419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52424/0 (socket says 172.21.15.39:52424) 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e19e00640 1 -- 172.21.15.39:0/3005766013 learned_addr learned my addr 172.21.15.39:0/3005766013 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.553+0000 7f2e19400640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e1410ac40 0x7f2e1419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:03.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.554+0000 7f2e1a800640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e1410b590 0x7f2e141a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:03.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.554+0000 7f2e19e00640 1 -- 172.21.15.39:0/3005766013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e1410b590 msgr2=0x7f2e141a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.554+0000 7f2e19e00640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e1410b590 0x7f2e141a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.554+0000 7f2e19e00640 1 -- 172.21.15.39:0/3005766013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e1410ac40 msgr2=0x7f2e1419c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.554+0000 7f2e19e00640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e1410ac40 0x7f2e1419c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.554+0000 7f2e19e00640 1 -- 172.21.15.39:0/3005766013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e141a1bd0 con 0x7f2e14105300 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.554+0000 7f2e19400640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e1410ac40 0x7f2e1419c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.554+0000 7f2e19e00640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e14105300 0x7f2e1419bc60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800e9f0 tx=0x7f2e0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.555+0000 7f2e0ea00640 1 -- 172.21.15.39:0/3005766013 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800cd80 con 0x7f2e14105300 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.555+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e141a1ec0 con 0x7f2e14105300 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.555+0000 7f2e0ea00640 1 -- 172.21.15.39:0/3005766013 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e080108e0 con 0x7f2e14105300 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.555+0000 7f2e0ea00640 1 -- 172.21.15.39:0/3005766013 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08018980 con 0x7f2e14105300 2024-07-04T08:13:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.555+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e141a8620 con 0x7f2e14105300 2024-07-04T08:13:03.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.556+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e14105710 con 0x7f2e14105300 2024-07-04T08:13:03.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.559+0000 7f2e0ea00640 1 -- 172.21.15.39:0/3005766013 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f2e08010430 con 0x7f2e14105300 2024-07-04T08:13:03.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.560+0000 7f2e0ea00640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2dfc0b2840 0x7f2dfc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:03.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.561+0000 7f2e0ea00640 1 -- 172.21.15.39:0/3005766013 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2e080e2ec0 con 0x7f2e14105300 2024-07-04T08:13:03.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.561+0000 7f2e19400640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2dfc0b2840 0x7f2dfc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:03.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.562+0000 7f2e19400640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2dfc0b2840 0x7f2dfc0b4d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e0402aa00 tx=0x7f2e04005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:03.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.562+0000 7f2e0ea00640 1 -- 172.21.15.39:0/3005766013 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f2e080aa0e0 con 0x7f2e14105300 2024-07-04T08:13:03.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.776+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e14062d50 con 0x7f2dfc0b2840 2024-07-04T08:13:03.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.787+0000 7f2e0ea00640 1 -- 172.21.15.39:0/3005766013 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340805 (secure 0 0 0) 0x7f2e14062d50 con 0x7f2dfc0b2840 2024-07-04T08:13:03.790 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:13:03.796 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:13:03.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2dfc0b2840 msgr2=0x7f2dfc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:03.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2dfc0b2840 0x7f2dfc0b4d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e0402aa00 tx=0x7f2e04005a20 comp rx=0 tx=0).stop 2024-07-04T08:13:03.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e14105300 msgr2=0x7f2e1419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:03.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e14105300 0x7f2e1419bc60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800e9f0 tx=0x7f2e0800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 shutdown_connections 2024-07-04T08:13:03.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2dfc0b2840 0x7f2dfc0b4d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e1410b590 0x7f2e141a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e1410ac40 0x7f2e1419c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 --2- 172.21.15.39:0/3005766013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2e14105300 0x7f2e1419bc60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:03.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.796+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 >> 172.21.15.39:0/3005766013 conn(0x7f2e14100d10 msgr2=0x7f2e141041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:03.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.797+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 shutdown_connections 2024-07-04T08:13:03.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:03.797+0000 7f2e1c800640 1 -- 172.21.15.39:0/3005766013 wait complete. 2024-07-04T08:13:04.613 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-07-04T08:13:03.613949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662473,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1278,"num_read_kb":750,"num_write":383,"num_write_kb":2302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374180,"kb_used_data":12548,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567644,"statfs":{"total":1151940427776,"available":1151557267456,"internally_reserved":0,"allocated":12849152,"data_stored":8478797,"data_compressed":498873,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":72,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006136"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":174,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:58.149998+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:12:58.149998+0000","last_peered":"2024-07-04T08:12:58.149998+0000","last_clean":"2024-07-04T08:12:58.149998+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:12:58.149998+0000","last_undegraded":"2024-07-04T08:12:58.149998+0000","last_fullsized":"2024-07-04T08:12:58.149998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542203+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:12:51.542203+0000","last_peered":"2024-07-04T08:12:51.542203+0000","last_clean":"2024-07-04T08:12:51.542203+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:12:51.542203+0000","last_undegraded":"2024-07-04T08:12:51.542203+0000","last_fullsized":"2024-07-04T08:12:51.542203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691313+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:11:53.691313+0000","last_peered":"2024-07-04T08:11:53.691313+0000","last_clean":"2024-07-04T08:11:53.691313+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:11:53.691313+0000","last_undegraded":"2024-07-04T08:11:53.691313+0000","last_fullsized":"2024-07-04T08:11:53.691313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.698423+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:53.698423+0000","last_peered":"2024-07-04T08:11:53.698423+0000","last_clean":"2024-07-04T08:11:53.698423+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:53.698423+0000","last_undegraded":"2024-07-04T08:11:53.698423+0000","last_fullsized":"2024-07-04T08:11:53.698423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569847+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:12:51.569847+0000","last_peered":"2024-07-04T08:12:51.569847+0000","last_clean":"2024-07-04T08:12:51.569847+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:12:51.569847+0000","last_undegraded":"2024-07-04T08:12:51.569847+0000","last_fullsized":"2024-07-04T08:12:51.569847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691301+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:11:53.691301+0000","last_peered":"2024-07-04T08:11:53.691301+0000","last_clean":"2024-07-04T08:11:53.691301+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:11:53.691301+0000","last_undegraded":"2024-07-04T08:11:53.691301+0000","last_fullsized":"2024-07-04T08:11:53.691301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":68,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771431+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:11:53.771431+0000","last_peered":"2024-07-04T08:11:53.771431+0000","last_clean":"2024-07-04T08:11:53.771431+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:11:53.771431+0000","last_undegraded":"2024-07-04T08:11:53.771431+0000","last_fullsized":"2024-07-04T08:11:53.771431+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.549686+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:12:51.549686+0000","last_peered":"2024-07-04T08:12:51.549686+0000","last_clean":"2024-07-04T08:12:51.549686+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:12:51.549686+0000","last_undegraded":"2024-07-04T08:12:51.549686+0000","last_fullsized":"2024-07-04T08:12:51.549686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691526+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:11:53.691526+0000","last_peered":"2024-07-04T08:11:53.691526+0000","last_clean":"2024-07-04T08:11:53.691526+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:11:53.691526+0000","last_undegraded":"2024-07-04T08:11:53.691526+0000","last_fullsized":"2024-07-04T08:11:53.691526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771226+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:11:53.771226+0000","last_peered":"2024-07-04T08:11:53.771226+0000","last_clean":"2024-07-04T08:11:53.771226+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:11:53.771226+0000","last_undegraded":"2024-07-04T08:11:53.771226+0000","last_fullsized":"2024-07-04T08:11:53.771226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.535777+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:12:51.535777+0000","last_peered":"2024-07-04T08:12:51.535777+0000","last_clean":"2024-07-04T08:12:51.535777+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:12:51.535777+0000","last_undegraded":"2024-07-04T08:12:51.535777+0000","last_fullsized":"2024-07-04T08:12:51.535777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691485+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:11:53.691485+0000","last_peered":"2024-07-04T08:11:53.691485+0000","last_clean":"2024-07-04T08:11:53.691485+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:11:53.691485+0000","last_undegraded":"2024-07-04T08:11:53.691485+0000","last_fullsized":"2024-07-04T08:11:53.691485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562435+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:12:51.562435+0000","last_peered":"2024-07-04T08:12:51.562435+0000","last_clean":"2024-07-04T08:12:51.562435+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:12:51.562435+0000","last_undegraded":"2024-07-04T08:12:51.562435+0000","last_fullsized":"2024-07-04T08:12:51.562435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844487+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:53.844487+0000","last_peered":"2024-07-04T08:11:53.844487+0000","last_clean":"2024-07-04T08:11:53.844487+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:53.844487+0000","last_undegraded":"2024-07-04T08:11:53.844487+0000","last_fullsized":"2024-07-04T08:11:53.844487+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692064+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:11:53.692064+0000","last_peered":"2024-07-04T08:11:53.692064+0000","last_clean":"2024-07-04T08:11:53.692064+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:11:53.692064+0000","last_undegraded":"2024-07-04T08:11:53.692064+0000","last_fullsized":"2024-07-04T08:11:53.692064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569552+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:12:51.569552+0000","last_peered":"2024-07-04T08:12:51.569552+0000","last_clean":"2024-07-04T08:12:51.569552+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:12:51.569552+0000","last_undegraded":"2024-07-04T08:12:51.569552+0000","last_fullsized":"2024-07-04T08:12:51.569552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691459+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:53.691459+0000","last_peered":"2024-07-04T08:11:53.691459+0000","last_clean":"2024-07-04T08:11:53.691459+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:53.691459+0000","last_undegraded":"2024-07-04T08:11:53.691459+0000","last_fullsized":"2024-07-04T08:11:53.691459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691735+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:11:53.691735+0000","last_peered":"2024-07-04T08:11:53.691735+0000","last_clean":"2024-07-04T08:11:53.691735+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:11:53.691735+0000","last_undegraded":"2024-07-04T08:11:53.691735+0000","last_fullsized":"2024-07-04T08:11:53.691735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566516+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:12:51.566516+0000","last_peered":"2024-07-04T08:12:51.566516+0000","last_clean":"2024-07-04T08:12:51.566516+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:12:51.566516+0000","last_undegraded":"2024-07-04T08:12:51.566516+0000","last_fullsized":"2024-07-04T08:12:51.566516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.696923+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:53.696923+0000","last_peered":"2024-07-04T08:11:53.696923+0000","last_clean":"2024-07-04T08:11:53.696923+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:53.696923+0000","last_undegraded":"2024-07-04T08:11:53.696923+0000","last_fullsized":"2024-07-04T08:11:53.696923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771192+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:11:53.771192+0000","last_peered":"2024-07-04T08:11:53.771192+0000","last_clean":"2024-07-04T08:11:53.771192+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:11:53.771192+0000","last_undegraded":"2024-07-04T08:11:53.771192+0000","last_fullsized":"2024-07-04T08:11:53.771192+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.493100+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:12:51.493100+0000","last_peered":"2024-07-04T08:12:51.493100+0000","last_clean":"2024-07-04T08:12:51.493100+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:12:51.493100+0000","last_undegraded":"2024-07-04T08:12:51.493100+0000","last_fullsized":"2024-07-04T08:12:51.493100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936252+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:53.936252+0000","last_peered":"2024-07-04T08:11:53.936252+0000","last_clean":"2024-07-04T08:11:53.936252+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:53.936252+0000","last_undegraded":"2024-07-04T08:11:53.936252+0000","last_fullsized":"2024-07-04T08:11:53.936252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691994+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:11:53.691994+0000","last_peered":"2024-07-04T08:11:53.691994+0000","last_clean":"2024-07-04T08:11:53.691994+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:11:53.691994+0000","last_undegraded":"2024-07-04T08:11:53.691994+0000","last_fullsized":"2024-07-04T08:11:53.691994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567815+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:12:51.567815+0000","last_peered":"2024-07-04T08:12:51.567815+0000","last_clean":"2024-07-04T08:12:51.567815+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:12:51.567815+0000","last_undegraded":"2024-07-04T08:12:51.567815+0000","last_fullsized":"2024-07-04T08:12:51.567815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692114+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:53.692114+0000","last_peered":"2024-07-04T08:11:53.692114+0000","last_clean":"2024-07-04T08:11:53.692114+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:53.692114+0000","last_undegraded":"2024-07-04T08:11:53.692114+0000","last_fullsized":"2024-07-04T08:11:53.692114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691612+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:11:53.691612+0000","last_peered":"2024-07-04T08:11:53.691612+0000","last_clean":"2024-07-04T08:11:53.691612+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:11:53.691612+0000","last_undegraded":"2024-07-04T08:11:53.691612+0000","last_fullsized":"2024-07-04T08:11:53.691612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.556440+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:12:51.556440+0000","last_peered":"2024-07-04T08:12:51.556440+0000","last_clean":"2024-07-04T08:12:51.556440+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:12:51.556440+0000","last_undegraded":"2024-07-04T08:12:51.556440+0000","last_fullsized":"2024-07-04T08:12:51.556440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'16","reported_seq":71,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993720+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:11:53.993720+0000","last_peered":"2024-07-04T08:11:53.993720+0000","last_clean":"2024-07-04T08:11:53.993720+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:11:53.993720+0000","last_undegraded":"2024-07-04T08:11:53.993720+0000","last_fullsized":"2024-07-04T08:11:53.993720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.697159+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:11:53.697159+0000","last_peered":"2024-07-04T08:11:53.697159+0000","last_clean":"2024-07-04T08:11:53.697159+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:11:53.697159+0000","last_undegraded":"2024-07-04T08:11:53.697159+0000","last_fullsized":"2024-07-04T08:11:53.697159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537810+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:12:51.537810+0000","last_peered":"2024-07-04T08:12:51.537810+0000","last_clean":"2024-07-04T08:12:51.537810+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:12:51.537810+0000","last_undegraded":"2024-07-04T08:12:51.537810+0000","last_fullsized":"2024-07-04T08:12:51.537810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771362+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:53.771362+0000","last_peered":"2024-07-04T08:11:53.771362+0000","last_clean":"2024-07-04T08:11:53.771362+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:53.771362+0000","last_undegraded":"2024-07-04T08:11:53.771362+0000","last_fullsized":"2024-07-04T08:11:53.771362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844703+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:11:53.844703+0000","last_peered":"2024-07-04T08:11:53.844703+0000","last_clean":"2024-07-04T08:11:53.844703+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:11:53.844703+0000","last_undegraded":"2024-07-04T08:11:53.844703+0000","last_fullsized":"2024-07-04T08:11:53.844703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542427+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:12:51.542427+0000","last_peered":"2024-07-04T08:12:51.542427+0000","last_clean":"2024-07-04T08:12:51.542427+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:12:51.542427+0000","last_undegraded":"2024-07-04T08:12:51.542427+0000","last_fullsized":"2024-07-04T08:12:51.542427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993344+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:11:53.993344+0000","last_peered":"2024-07-04T08:11:53.993344+0000","last_clean":"2024-07-04T08:11:53.993344+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:11:53.993344+0000","last_undegraded":"2024-07-04T08:11:53.993344+0000","last_fullsized":"2024-07-04T08:11:53.993344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.770822+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:11:53.770822+0000","last_peered":"2024-07-04T08:11:53.770822+0000","last_clean":"2024-07-04T08:11:53.770822+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:11:53.770822+0000","last_undegraded":"2024-07-04T08:11:53.770822+0000","last_fullsized":"2024-07-04T08:11:53.770822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.543556+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:12:51.543556+0000","last_peered":"2024-07-04T08:12:51.543556+0000","last_clean":"2024-07-04T08:12:51.543556+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:12:51.543556+0000","last_undegraded":"2024-07-04T08:12:51.543556+0000","last_fullsized":"2024-07-04T08:12:51.543556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":170,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:59.153777+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:12:59.153777+0000","last_peered":"2024-07-04T08:12:59.153777+0000","last_clean":"2024-07-04T08:12:59.153777+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:12:59.153777+0000","last_undegraded":"2024-07-04T08:12:59.153777+0000","last_fullsized":"2024-07-04T08:12:59.153777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995173+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:11:53.995173+0000","last_peered":"2024-07-04T08:11:53.995173+0000","last_clean":"2024-07-04T08:11:53.995173+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:11:53.995173+0000","last_undegraded":"2024-07-04T08:11:53.995173+0000","last_fullsized":"2024-07-04T08:11:53.995173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.516818+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:12:51.516818+0000","last_peered":"2024-07-04T08:12:51.516818+0000","last_clean":"2024-07-04T08:12:51.516818+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:12:51.516818+0000","last_undegraded":"2024-07-04T08:12:51.516818+0000","last_fullsized":"2024-07-04T08:12:51.516818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":169,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:01.166190+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:13:01.166190+0000","last_peered":"2024-07-04T08:13:01.166190+0000","last_clean":"2024-07-04T08:13:01.166190+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:13:01.166190+0000","last_undegraded":"2024-07-04T08:13:01.166190+0000","last_fullsized":"2024-07-04T08:13:01.166190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995252+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:11:53.995252+0000","last_peered":"2024-07-04T08:11:53.995252+0000","last_clean":"2024-07-04T08:11:53.995252+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:11:53.995252+0000","last_undegraded":"2024-07-04T08:11:53.995252+0000","last_fullsized":"2024-07-04T08:11:53.995252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534643+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:12:51.534643+0000","last_peered":"2024-07-04T08:12:51.534643+0000","last_clean":"2024-07-04T08:12:51.534643+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:12:51.534643+0000","last_undegraded":"2024-07-04T08:12:51.534643+0000","last_fullsized":"2024-07-04T08:12:51.534643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851356+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:53.851356+0000","last_peered":"2024-07-04T08:11:53.851356+0000","last_clean":"2024-07-04T08:11:53.851356+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:53.851356+0000","last_undegraded":"2024-07-04T08:11:53.851356+0000","last_fullsized":"2024-07-04T08:11:53.851356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939614+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:11:53.939614+0000","last_peered":"2024-07-04T08:11:53.939614+0000","last_clean":"2024-07-04T08:11:53.939614+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:11:53.939614+0000","last_undegraded":"2024-07-04T08:11:53.939614+0000","last_fullsized":"2024-07-04T08:11:53.939614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.558211+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:12:51.558211+0000","last_peered":"2024-07-04T08:12:51.558211+0000","last_clean":"2024-07-04T08:12:51.558211+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:12:51.558211+0000","last_undegraded":"2024-07-04T08:12:51.558211+0000","last_fullsized":"2024-07-04T08:12:51.558211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691801+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:53.691801+0000","last_peered":"2024-07-04T08:11:53.691801+0000","last_clean":"2024-07-04T08:11:53.691801+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:53.691801+0000","last_undegraded":"2024-07-04T08:11:53.691801+0000","last_fullsized":"2024-07-04T08:11:53.691801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":119,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851572+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:11:53.851572+0000","last_peered":"2024-07-04T08:11:53.851572+0000","last_clean":"2024-07-04T08:11:53.851572+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:11:53.851572+0000","last_undegraded":"2024-07-04T08:11:53.851572+0000","last_fullsized":"2024-07-04T08:11:53.851572+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851869+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:11:53.851869+0000","last_peered":"2024-07-04T08:11:53.851869+0000","last_clean":"2024-07-04T08:11:53.851869+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:11:53.851869+0000","last_undegraded":"2024-07-04T08:11:53.851869+0000","last_fullsized":"2024-07-04T08:11:53.851869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553706+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:12:51.553706+0000","last_peered":"2024-07-04T08:12:51.553706+0000","last_clean":"2024-07-04T08:12:51.553706+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:12:51.553706+0000","last_undegraded":"2024-07-04T08:12:51.553706+0000","last_fullsized":"2024-07-04T08:12:51.553706+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":136,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:58.024133+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:12:58.024133+0000","last_peered":"2024-07-04T08:12:58.024133+0000","last_clean":"2024-07-04T08:12:58.024133+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:12:58.024133+0000","last_undegraded":"2024-07-04T08:12:58.024133+0000","last_fullsized":"2024-07-04T08:12:58.024133+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936202+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:11:53.936202+0000","last_peered":"2024-07-04T08:11:53.936202+0000","last_clean":"2024-07-04T08:11:53.936202+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:11:53.936202+0000","last_undegraded":"2024-07-04T08:11:53.936202+0000","last_fullsized":"2024-07-04T08:11:53.936202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.561382+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:12:51.561382+0000","last_peered":"2024-07-04T08:12:51.561382+0000","last_clean":"2024-07-04T08:12:51.561382+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:12:51.561382+0000","last_undegraded":"2024-07-04T08:12:51.561382+0000","last_fullsized":"2024-07-04T08:12:51.561382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939874+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:53.939874+0000","last_peered":"2024-07-04T08:11:53.939874+0000","last_clean":"2024-07-04T08:11:53.939874+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:11:53.939874+0000","last_undegraded":"2024-07-04T08:11:53.939874+0000","last_fullsized":"2024-07-04T08:11:53.939874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.105658+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:11:54.105658+0000","last_peered":"2024-07-04T08:11:54.105658+0000","last_clean":"2024-07-04T08:11:54.105658+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:11:54.105658+0000","last_undegraded":"2024-07-04T08:11:54.105658+0000","last_fullsized":"2024-07-04T08:11:54.105658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552444+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:12:51.552444+0000","last_peered":"2024-07-04T08:12:51.552444+0000","last_clean":"2024-07-04T08:12:51.552444+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:12:51.552444+0000","last_undegraded":"2024-07-04T08:12:51.552444+0000","last_fullsized":"2024-07-04T08:12:51.552444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.994981+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:53.994981+0000","last_peered":"2024-07-04T08:11:53.994981+0000","last_clean":"2024-07-04T08:11:53.994981+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:53.994981+0000","last_undegraded":"2024-07-04T08:11:53.994981+0000","last_fullsized":"2024-07-04T08:11:53.994981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.938978+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:11:53.938978+0000","last_peered":"2024-07-04T08:11:53.938978+0000","last_clean":"2024-07-04T08:11:53.938978+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:11:53.938978+0000","last_undegraded":"2024-07-04T08:11:53.938978+0000","last_fullsized":"2024-07-04T08:11:53.938978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531754+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:12:51.531754+0000","last_peered":"2024-07-04T08:12:51.531754+0000","last_clean":"2024-07-04T08:12:51.531754+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:12:51.531754+0000","last_undegraded":"2024-07-04T08:12:51.531754+0000","last_fullsized":"2024-07-04T08:12:51.531754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771313+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:53.771313+0000","last_peered":"2024-07-04T08:11:53.771313+0000","last_clean":"2024-07-04T08:11:53.771313+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:53.771313+0000","last_undegraded":"2024-07-04T08:11:53.771313+0000","last_fullsized":"2024-07-04T08:11:53.771313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939064+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:11:53.939064+0000","last_peered":"2024-07-04T08:11:53.939064+0000","last_clean":"2024-07-04T08:11:53.939064+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:11:53.939064+0000","last_undegraded":"2024-07-04T08:11:53.939064+0000","last_fullsized":"2024-07-04T08:11:53.939064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542560+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:12:51.542560+0000","last_peered":"2024-07-04T08:12:51.542560+0000","last_clean":"2024-07-04T08:12:51.542560+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:12:51.542560+0000","last_undegraded":"2024-07-04T08:12:51.542560+0000","last_fullsized":"2024-07-04T08:12:51.542560+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":63,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.770763+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:53.770763+0000","last_peered":"2024-07-04T08:11:53.770763+0000","last_clean":"2024-07-04T08:11:53.770763+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:53.770763+0000","last_undegraded":"2024-07-04T08:11:53.770763+0000","last_fullsized":"2024-07-04T08:11:53.770763+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995018+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:11:53.995018+0000","last_peered":"2024-07-04T08:11:53.995018+0000","last_clean":"2024-07-04T08:11:53.995018+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:11:53.995018+0000","last_undegraded":"2024-07-04T08:11:53.995018+0000","last_fullsized":"2024-07-04T08:11:53.995018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551660+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:12:51.551660+0000","last_peered":"2024-07-04T08:12:51.551660+0000","last_clean":"2024-07-04T08:12:51.551660+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:12:51.551660+0000","last_undegraded":"2024-07-04T08:12:51.551660+0000","last_fullsized":"2024-07-04T08:12:51.551660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:59.162750+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:12:59.162750+0000","last_peered":"2024-07-04T08:12:59.162750+0000","last_clean":"2024-07-04T08:12:59.162750+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:12:59.162750+0000","last_undegraded":"2024-07-04T08:12:59.162750+0000","last_fullsized":"2024-07-04T08:12:59.162750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993663+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:11:53.993663+0000","last_peered":"2024-07-04T08:11:53.993663+0000","last_clean":"2024-07-04T08:11:53.993663+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:11:53.993663+0000","last_undegraded":"2024-07-04T08:11:53.993663+0000","last_fullsized":"2024-07-04T08:11:53.993663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564845+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:12:51.564845+0000","last_peered":"2024-07-04T08:12:51.564845+0000","last_clean":"2024-07-04T08:12:51.564845+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:12:51.564845+0000","last_undegraded":"2024-07-04T08:12:51.564845+0000","last_fullsized":"2024-07-04T08:12:51.564845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939360+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:53.939360+0000","last_peered":"2024-07-04T08:11:53.939360+0000","last_clean":"2024-07-04T08:11:53.939360+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:53.939360+0000","last_undegraded":"2024-07-04T08:11:53.939360+0000","last_fullsized":"2024-07-04T08:11:53.939360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844593+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:11:53.844593+0000","last_peered":"2024-07-04T08:11:53.844593+0000","last_clean":"2024-07-04T08:11:53.844593+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:11:53.844593+0000","last_undegraded":"2024-07-04T08:11:53.844593+0000","last_fullsized":"2024-07-04T08:11:53.844593+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562054+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:12:51.562054+0000","last_peered":"2024-07-04T08:12:51.562054+0000","last_clean":"2024-07-04T08:12:51.562054+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:12:51.562054+0000","last_undegraded":"2024-07-04T08:12:51.562054+0000","last_fullsized":"2024-07-04T08:12:51.562054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691602+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:53.691602+0000","last_peered":"2024-07-04T08:11:53.691602+0000","last_clean":"2024-07-04T08:11:53.691602+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:53.691602+0000","last_undegraded":"2024-07-04T08:11:53.691602+0000","last_fullsized":"2024-07-04T08:11:53.691602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.997536+0000","last_change":"2024-07-04T08:11:53.997034+0000","last_active":"2024-07-04T08:11:53.997536+0000","last_peered":"2024-07-04T08:11:53.997536+0000","last_clean":"2024-07-04T08:11:53.997536+0000","last_became_active":"2024-07-04T08:11:53.996456+0000","last_became_peered":"2024-07-04T08:11:53.996456+0000","last_unstale":"2024-07-04T08:11:53.997536+0000","last_undegraded":"2024-07-04T08:11:53.997536+0000","last_fullsized":"2024-07-04T08:11:53.997536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.140549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.521670+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:12:51.521670+0000","last_peered":"2024-07-04T08:12:51.521670+0000","last_clean":"2024-07-04T08:12:51.521670+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:12:51.521670+0000","last_undegraded":"2024-07-04T08:12:51.521670+0000","last_fullsized":"2024-07-04T08:12:51.521670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":325,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:01.062466+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:13:01.062466+0000","last_peered":"2024-07-04T08:13:01.062466+0000","last_clean":"2024-07-04T08:13:01.062466+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:13:01.062466+0000","last_undegraded":"2024-07-04T08:13:01.062466+0000","last_fullsized":"2024-07-04T08:13:01.062466+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":184,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.992479+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:11:53.992479+0000","last_peered":"2024-07-04T08:11:53.992479+0000","last_clean":"2024-07-04T08:11:53.992479+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:11:53.992479+0000","last_undegraded":"2024-07-04T08:11:53.992479+0000","last_fullsized":"2024-07-04T08:11:53.992479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541776+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:12:51.541776+0000","last_peered":"2024-07-04T08:12:51.541776+0000","last_clean":"2024-07-04T08:12:51.541776+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:12:51.541776+0000","last_undegraded":"2024-07-04T08:12:51.541776+0000","last_fullsized":"2024-07-04T08:12:51.541776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939902+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:53.939902+0000","last_peered":"2024-07-04T08:11:53.939902+0000","last_clean":"2024-07-04T08:11:53.939902+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:53.939902+0000","last_undegraded":"2024-07-04T08:11:53.939902+0000","last_fullsized":"2024-07-04T08:11:53.939902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691764+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:11:53.691764+0000","last_peered":"2024-07-04T08:11:53.691764+0000","last_clean":"2024-07-04T08:11:53.691764+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:11:53.691764+0000","last_undegraded":"2024-07-04T08:11:53.691764+0000","last_fullsized":"2024-07-04T08:11:53.691764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569726+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:12:51.569726+0000","last_peered":"2024-07-04T08:12:51.569726+0000","last_clean":"2024-07-04T08:12:51.569726+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:12:51.569726+0000","last_undegraded":"2024-07-04T08:12:51.569726+0000","last_fullsized":"2024-07-04T08:12:51.569726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.852004+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:11:53.852004+0000","last_peered":"2024-07-04T08:11:53.852004+0000","last_clean":"2024-07-04T08:11:53.852004+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:11:53.852004+0000","last_undegraded":"2024-07-04T08:11:53.852004+0000","last_fullsized":"2024-07-04T08:11:53.852004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.991447+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:11:53.991447+0000","last_peered":"2024-07-04T08:11:53.991447+0000","last_clean":"2024-07-04T08:11:53.991447+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:11:53.991447+0000","last_undegraded":"2024-07-04T08:11:53.991447+0000","last_fullsized":"2024-07-04T08:11:53.991447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.546983+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:12:51.546983+0000","last_peered":"2024-07-04T08:12:51.546983+0000","last_clean":"2024-07-04T08:12:51.546983+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:12:51.546983+0000","last_undegraded":"2024-07-04T08:12:51.546983+0000","last_fullsized":"2024-07-04T08:12:51.546983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":281,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:01.027719+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:13:01.027719+0000","last_peered":"2024-07-04T08:13:01.027719+0000","last_clean":"2024-07-04T08:13:01.027719+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:13:01.027719+0000","last_undegraded":"2024-07-04T08:13:01.027719+0000","last_fullsized":"2024-07-04T08:13:01.027719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.696582+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:11:53.696582+0000","last_peered":"2024-07-04T08:11:53.696582+0000","last_clean":"2024-07-04T08:11:53.696582+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:11:53.696582+0000","last_undegraded":"2024-07-04T08:11:53.696582+0000","last_fullsized":"2024-07-04T08:11:53.696582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.106088+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:11:54.106088+0000","last_peered":"2024-07-04T08:11:54.106088+0000","last_clean":"2024-07-04T08:11:54.106088+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:11:54.106088+0000","last_undegraded":"2024-07-04T08:11:54.106088+0000","last_fullsized":"2024-07-04T08:11:54.106088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564360+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:12:51.564360+0000","last_peered":"2024-07-04T08:12:51.564360+0000","last_clean":"2024-07-04T08:12:51.564360+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:12:51.564360+0000","last_undegraded":"2024-07-04T08:12:51.564360+0000","last_fullsized":"2024-07-04T08:12:51.564360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939445+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:53.939445+0000","last_peered":"2024-07-04T08:11:53.939445+0000","last_clean":"2024-07-04T08:11:53.939445+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:53.939445+0000","last_undegraded":"2024-07-04T08:11:53.939445+0000","last_fullsized":"2024-07-04T08:11:53.939445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844766+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:11:53.844766+0000","last_peered":"2024-07-04T08:11:53.844766+0000","last_clean":"2024-07-04T08:11:53.844766+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:11:53.844766+0000","last_undegraded":"2024-07-04T08:11:53.844766+0000","last_fullsized":"2024-07-04T08:11:53.844766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565483+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:12:51.565483+0000","last_peered":"2024-07-04T08:12:51.565483+0000","last_clean":"2024-07-04T08:12:51.565483+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:12:51.565483+0000","last_undegraded":"2024-07-04T08:12:51.565483+0000","last_fullsized":"2024-07-04T08:12:51.565483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.994878+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:11:53.994878+0000","last_peered":"2024-07-04T08:11:53.994878+0000","last_clean":"2024-07-04T08:11:53.994878+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:11:53.994878+0000","last_undegraded":"2024-07-04T08:11:53.994878+0000","last_fullsized":"2024-07-04T08:11:53.994878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.699922+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:11:53.699922+0000","last_peered":"2024-07-04T08:11:53.699922+0000","last_clean":"2024-07-04T08:11:53.699922+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:11:53.699922+0000","last_undegraded":"2024-07-04T08:11:53.699922+0000","last_fullsized":"2024-07-04T08:11:53.699922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.533724+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:12:51.533724+0000","last_peered":"2024-07-04T08:12:51.533724+0000","last_clean":"2024-07-04T08:12:51.533724+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:12:51.533724+0000","last_undegraded":"2024-07-04T08:12:51.533724+0000","last_fullsized":"2024-07-04T08:12:51.533724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691660+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:11:53.691660+0000","last_peered":"2024-07-04T08:11:53.691660+0000","last_clean":"2024-07-04T08:11:53.691660+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:11:53.691660+0000","last_undegraded":"2024-07-04T08:11:53.691660+0000","last_fullsized":"2024-07-04T08:11:53.691660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:58.040961+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:12:58.040961+0000","last_peered":"2024-07-04T08:12:58.040961+0000","last_clean":"2024-07-04T08:12:58.040961+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:12:58.040961+0000","last_undegraded":"2024-07-04T08:12:58.040961+0000","last_fullsized":"2024-07-04T08:12:58.040961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.518982+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:12:51.518982+0000","last_peered":"2024-07-04T08:12:51.518982+0000","last_clean":"2024-07-04T08:12:51.518982+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:12:51.518982+0000","last_undegraded":"2024-07-04T08:12:51.518982+0000","last_fullsized":"2024-07-04T08:12:51.518982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691843+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:11:53.691843+0000","last_peered":"2024-07-04T08:11:53.691843+0000","last_clean":"2024-07-04T08:11:53.691843+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:11:53.691843+0000","last_undegraded":"2024-07-04T08:11:53.691843+0000","last_fullsized":"2024-07-04T08:11:53.691843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534082+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:12:51.534082+0000","last_peered":"2024-07-04T08:12:51.534082+0000","last_clean":"2024-07-04T08:12:51.534082+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:12:51.534082+0000","last_undegraded":"2024-07-04T08:12:51.534082+0000","last_fullsized":"2024-07-04T08:12:51.534082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.539557+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:12:51.539557+0000","last_peered":"2024-07-04T08:12:51.539557+0000","last_clean":"2024-07-04T08:12:51.539557+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:12:51.539557+0000","last_undegraded":"2024-07-04T08:12:51.539557+0000","last_fullsized":"2024-07-04T08:12:51.539557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566225+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:12:51.566225+0000","last_peered":"2024-07-04T08:12:51.566225+0000","last_clean":"2024-07-04T08:12:51.566225+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:12:51.566225+0000","last_undegraded":"2024-07-04T08:12:51.566225+0000","last_fullsized":"2024-07-04T08:12:51.566225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567598+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:12:51.567598+0000","last_peered":"2024-07-04T08:12:51.567598+0000","last_clean":"2024-07-04T08:12:51.567598+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:12:51.567598+0000","last_undegraded":"2024-07-04T08:12:51.567598+0000","last_fullsized":"2024-07-04T08:12:51.567598+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569096+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:12:51.569096+0000","last_peered":"2024-07-04T08:12:51.569096+0000","last_clean":"2024-07-04T08:12:51.569096+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:12:51.569096+0000","last_undegraded":"2024-07-04T08:12:51.569096+0000","last_fullsized":"2024-07-04T08:12:51.569096+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532204+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:12:51.532204+0000","last_peered":"2024-07-04T08:12:51.532204+0000","last_clean":"2024-07-04T08:12:51.532204+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:12:51.532204+0000","last_undegraded":"2024-07-04T08:12:51.532204+0000","last_fullsized":"2024-07-04T08:12:51.532204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540456+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:12:51.540456+0000","last_peered":"2024-07-04T08:12:51.540456+0000","last_clean":"2024-07-04T08:12:51.540456+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:12:51.540456+0000","last_undegraded":"2024-07-04T08:12:51.540456+0000","last_fullsized":"2024-07-04T08:12:51.540456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552090+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:12:51.552090+0000","last_peered":"2024-07-04T08:12:51.552090+0000","last_clean":"2024-07-04T08:12:51.552090+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:12:51.552090+0000","last_undegraded":"2024-07-04T08:12:51.552090+0000","last_fullsized":"2024-07-04T08:12:51.552090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531386+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:12:51.531386+0000","last_peered":"2024-07-04T08:12:51.531386+0000","last_clean":"2024-07-04T08:12:51.531386+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:12:51.531386+0000","last_undegraded":"2024-07-04T08:12:51.531386+0000","last_fullsized":"2024-07-04T08:12:51.531386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.555323+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:12:51.555323+0000","last_peered":"2024-07-04T08:12:51.555323+0000","last_clean":"2024-07-04T08:12:51.555323+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:12:51.555323+0000","last_undegraded":"2024-07-04T08:12:51.555323+0000","last_fullsized":"2024-07-04T08:12:51.555323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560682+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:12:51.560682+0000","last_peered":"2024-07-04T08:12:51.560682+0000","last_clean":"2024-07-04T08:12:51.560682+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:12:51.560682+0000","last_undegraded":"2024-07-04T08:12:51.560682+0000","last_fullsized":"2024-07-04T08:12:51.560682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570606+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:12:51.570606+0000","last_peered":"2024-07-04T08:12:51.570606+0000","last_clean":"2024-07-04T08:12:51.570606+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:12:51.570606+0000","last_undegraded":"2024-07-04T08:12:51.570606+0000","last_fullsized":"2024-07-04T08:12:51.570606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532600+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:12:51.532600+0000","last_peered":"2024-07-04T08:12:51.532600+0000","last_clean":"2024-07-04T08:12:51.532600+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:12:51.532600+0000","last_undegraded":"2024-07-04T08:12:51.532600+0000","last_fullsized":"2024-07-04T08:12:51.532600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.538492+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:12:51.538492+0000","last_peered":"2024-07-04T08:12:51.538492+0000","last_clean":"2024-07-04T08:12:51.538492+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:12:51.538492+0000","last_undegraded":"2024-07-04T08:12:51.538492+0000","last_fullsized":"2024-07-04T08:12:51.538492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540580+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:12:51.540580+0000","last_peered":"2024-07-04T08:12:51.540580+0000","last_clean":"2024-07-04T08:12:51.540580+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:12:51.540580+0000","last_undegraded":"2024-07-04T08:12:51.540580+0000","last_fullsized":"2024-07-04T08:12:51.540580+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569630+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:12:51.569630+0000","last_peered":"2024-07-04T08:12:51.569630+0000","last_clean":"2024-07-04T08:12:51.569630+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:12:51.569630+0000","last_undegraded":"2024-07-04T08:12:51.569630+0000","last_fullsized":"2024-07-04T08:12:51.569630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.571003+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:12:51.571003+0000","last_peered":"2024-07-04T08:12:51.571003+0000","last_clean":"2024-07-04T08:12:51.571003+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:12:51.571003+0000","last_undegraded":"2024-07-04T08:12:51.571003+0000","last_fullsized":"2024-07-04T08:12:51.571003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552014+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:12:51.552014+0000","last_peered":"2024-07-04T08:12:51.552014+0000","last_clean":"2024-07-04T08:12:51.552014+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:12:51.552014+0000","last_undegraded":"2024-07-04T08:12:51.552014+0000","last_fullsized":"2024-07-04T08:12:51.552014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567116+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:12:51.567116+0000","last_peered":"2024-07-04T08:12:51.567116+0000","last_clean":"2024-07-04T08:12:51.567116+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:12:51.567116+0000","last_undegraded":"2024-07-04T08:12:51.567116+0000","last_fullsized":"2024-07-04T08:12:51.567116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.550951+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:12:51.550951+0000","last_peered":"2024-07-04T08:12:51.550951+0000","last_clean":"2024-07-04T08:12:51.550951+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:12:51.550951+0000","last_undegraded":"2024-07-04T08:12:51.550951+0000","last_fullsized":"2024-07-04T08:12:51.550951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.533735+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:12:51.533735+0000","last_peered":"2024-07-04T08:12:51.533735+0000","last_clean":"2024-07-04T08:12:51.533735+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:12:51.533735+0000","last_undegraded":"2024-07-04T08:12:51.533735+0000","last_fullsized":"2024-07-04T08:12:51.533735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553530+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:12:51.553530+0000","last_peered":"2024-07-04T08:12:51.553530+0000","last_clean":"2024-07-04T08:12:51.553530+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:12:51.553530+0000","last_undegraded":"2024-07-04T08:12:51.553530+0000","last_fullsized":"2024-07-04T08:12:51.553530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540232+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:12:51.540232+0000","last_peered":"2024-07-04T08:12:51.540232+0000","last_clean":"2024-07-04T08:12:51.540232+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:12:51.540232+0000","last_undegraded":"2024-07-04T08:12:51.540232+0000","last_fullsized":"2024-07-04T08:12:51.540232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540940+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:12:51.540940+0000","last_peered":"2024-07-04T08:12:51.540940+0000","last_clean":"2024-07-04T08:12:51.540940+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:12:51.540940+0000","last_undegraded":"2024-07-04T08:12:51.540940+0000","last_fullsized":"2024-07-04T08:12:51.540940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560242+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:12:51.560242+0000","last_peered":"2024-07-04T08:12:51.560242+0000","last_clean":"2024-07-04T08:12:51.560242+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:12:51.560242+0000","last_undegraded":"2024-07-04T08:12:51.560242+0000","last_fullsized":"2024-07-04T08:12:51.560242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.550113+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:12:51.550113+0000","last_peered":"2024-07-04T08:12:51.550113+0000","last_clean":"2024-07-04T08:12:51.550113+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:12:51.550113+0000","last_undegraded":"2024-07-04T08:12:51.550113+0000","last_fullsized":"2024-07-04T08:12:51.550113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541407+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:12:51.541407+0000","last_peered":"2024-07-04T08:12:51.541407+0000","last_clean":"2024-07-04T08:12:51.541407+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:12:51.541407+0000","last_undegraded":"2024-07-04T08:12:51.541407+0000","last_fullsized":"2024-07-04T08:12:51.541407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.536688+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:12:51.536688+0000","last_peered":"2024-07-04T08:12:51.536688+0000","last_clean":"2024-07-04T08:12:51.536688+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:12:51.536688+0000","last_undegraded":"2024-07-04T08:12:51.536688+0000","last_fullsized":"2024-07-04T08:12:51.536688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537059+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:12:51.537059+0000","last_peered":"2024-07-04T08:12:51.537059+0000","last_clean":"2024-07-04T08:12:51.537059+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:12:51.537059+0000","last_undegraded":"2024-07-04T08:12:51.537059+0000","last_fullsized":"2024-07-04T08:12:51.537059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565082+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:12:51.565082+0000","last_peered":"2024-07-04T08:12:51.565082+0000","last_clean":"2024-07-04T08:12:51.565082+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:12:51.565082+0000","last_undegraded":"2024-07-04T08:12:51.565082+0000","last_fullsized":"2024-07-04T08:12:51.565082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567958+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:12:51.567958+0000","last_peered":"2024-07-04T08:12:51.567958+0000","last_clean":"2024-07-04T08:12:51.567958+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:12:51.567958+0000","last_undegraded":"2024-07-04T08:12:51.567958+0000","last_fullsized":"2024-07-04T08:12:51.567958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569411+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:12:51.569411+0000","last_peered":"2024-07-04T08:12:51.569411+0000","last_clean":"2024-07-04T08:12:51.569411+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:12:51.569411+0000","last_undegraded":"2024-07-04T08:12:51.569411+0000","last_fullsized":"2024-07-04T08:12:51.569411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71871,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1197,"num_read_kb":681,"num_write":258,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612666,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":543564,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808893,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1195297,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":547590,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168650,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":543872,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364880,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1141042,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528405,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":539530,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724634,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1135560,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920863,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":543604,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117092,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":538706,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378732,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":603463,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542256,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":606115,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705781,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":540454,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:13:04.619 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph pg dump --format=json 2024-07-04T08:13:05.155 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:05.307 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:05 smithi039 ceph-mon[32062]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-07-04T08:13:05.307 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:05 smithi039 ceph-mon[32062]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:13:05.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:05 smithi133 ceph-mon[37050]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-07-04T08:13:05.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:05 smithi133 ceph-mon[37050]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:13:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:05 smithi050 ceph-mon[36577]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-07-04T08:13:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:05 smithi050 ceph-mon[36577]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:13:07.230 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:07 smithi039 ceph-mon[32062]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-04T08:13:07.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:07 smithi133 ceph-mon[37050]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-04T08:13:07.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:07 smithi050 ceph-mon[36577]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-04T08:13:07.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.448+0000 7f827dc00640 1 -- 172.21.15.39:0/923845104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8278102f00 msgr2=0x7f8278103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.448+0000 7f827dc00640 1 --2- 172.21.15.39:0/923845104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8278102f00 0x7f8278103310 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8268009900 tx=0x7f826802a410 comp rx=0 tx=0).stop 2024-07-04T08:13:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.448+0000 7f827dc00640 1 -- 172.21.15.39:0/923845104 shutdown_connections 2024-07-04T08:13:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.448+0000 7f827dc00640 1 --2- 172.21.15.39:0/923845104 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8278109190 0x7f827810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.448+0000 7f827dc00640 1 --2- 172.21.15.39:0/923845104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8278108840 0x7f8278108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.448+0000 7f827dc00640 1 --2- 172.21.15.39:0/923845104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8278102f00 0x7f8278103310 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.448+0000 7f827dc00640 1 -- 172.21.15.39:0/923845104 >> 172.21.15.39:0/923845104 conn(0x7f82780fe990 msgr2=0x7f8278100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.448+0000 7f827dc00640 1 -- 172.21.15.39:0/923845104 shutdown_connections 2024-07-04T08:13:07.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.448+0000 7f827dc00640 1 -- 172.21.15.39:0/923845104 wait complete. 2024-07-04T08:13:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.449+0000 7f827dc00640 1 Processor -- start 2024-07-04T08:13:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.450+0000 7f827dc00640 1 -- start start 2024-07-04T08:13:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.450+0000 7f827dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8278102f00 0x7f8278074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.450+0000 7f827dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8278108840 0x7f82780751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.450+0000 7f827dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8278109190 0x7f82780756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.450+0000 7f827dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8278112ea0 con 0x7f8278108840 2024-07-04T08:13:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.450+0000 7f827dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8278112d20 con 0x7f8278109190 2024-07-04T08:13:07.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.450+0000 7f827dc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8278113020 con 0x7f8278102f00 2024-07-04T08:13:07.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.450+0000 7f8277e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8278102f00 0x7f8278074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:07.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.450+0000 7f826fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8278108840 0x7f82780751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f827d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8278109190 0x7f82780756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f8277e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8278102f00 0x7f8278074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:52056/0 (socket says 172.21.15.39:52056) 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f826fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8278108840 0x7f82780751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52454/0 (socket says 172.21.15.39:52454) 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f8277e00640 1 -- 172.21.15.39:0/4150209821 learned_addr learned my addr 172.21.15.39:0/4150209821 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f827d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8278109190 0x7f82780756e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:40658/0 (socket says 172.21.15.39:40658) 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f8277e00640 1 -- 172.21.15.39:0/4150209821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8278109190 msgr2=0x7f82780756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f8277e00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8278109190 0x7f82780756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f8277e00640 1 -- 172.21.15.39:0/4150209821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8278108840 msgr2=0x7f82780751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f8277e00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8278108840 0x7f82780751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f8277e00640 1 -- 172.21.15.39:0/4150209821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82781a7b30 con 0x7f8278102f00 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f827d200640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8278109190 0x7f82780756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:13:07.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.451+0000 7f8277e00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8278102f00 0x7f8278074c60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8268005ab0 tx=0x7f8268003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.452+0000 7f8275600640 1 -- 172.21.15.39:0/4150209821 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826803f070 con 0x7f8278102f00 2024-07-04T08:13:07.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.452+0000 7f8275600640 1 -- 172.21.15.39:0/4150209821 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8268031b60 con 0x7f8278102f00 2024-07-04T08:13:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.452+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82781a7dc0 con 0x7f8278102f00 2024-07-04T08:13:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.452+0000 7f8275600640 1 -- 172.21.15.39:0/4150209821 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268035030 con 0x7f8278102f00 2024-07-04T08:13:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.452+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82781a8260 con 0x7f8278102f00 2024-07-04T08:13:07.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.453+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8278103310 con 0x7f8278102f00 2024-07-04T08:13:07.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.456+0000 7f8275600640 1 -- 172.21.15.39:0/4150209821 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f8268042070 con 0x7f8278102f00 2024-07-04T08:13:07.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.458+0000 7f8275600640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f825c0b28c0 0x7f825c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:07.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.458+0000 7f8275600640 1 -- 172.21.15.39:0/4150209821 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f82681033e0 con 0x7f8278102f00 2024-07-04T08:13:07.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.458+0000 7f826fe00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f825c0b28c0 0x7f825c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:07.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.459+0000 7f826fe00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f825c0b28c0 0x7f825c0b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8278108420 tx=0x7f8260005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:07.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.460+0000 7f8275600640 1 -- 172.21.15.39:0/4150209821 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f82680ca600 con 0x7f8278102f00 2024-07-04T08:13:07.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.670+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8278062d50 con 0x7f825c0b28c0 2024-07-04T08:13:07.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.689+0000 7f8275600640 1 -- 172.21.15.39:0/4150209821 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340805 (secure 0 0 0) 0x7f8278062d50 con 0x7f825c0b28c0 2024-07-04T08:13:07.696 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:13:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f825c0b28c0 msgr2=0x7f825c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f825c0b28c0 0x7f825c0b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8278108420 tx=0x7f8260005f90 comp rx=0 tx=0).stop 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8278102f00 msgr2=0x7f8278074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8278102f00 0x7f8278074c60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8268005ab0 tx=0x7f8268003c10 comp rx=0 tx=0).stop 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 shutdown_connections 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f825c0b28c0 0x7f825c0b4dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8278108840 0x7f82780751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8278109190 0x7f82780756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 --2- 172.21.15.39:0/4150209821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8278102f00 0x7f8278074c60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.696+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 >> 172.21.15.39:0/4150209821 conn(0x7f82780fe990 msgr2=0x7f8278100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.697+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 shutdown_connections 2024-07-04T08:13:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:07.697+0000 7f827dc00640 1 -- 172.21.15.39:0/4150209821 wait complete. 2024-07-04T08:13:08.565 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-07-04T08:13:07.616083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662473,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1323,"num_read_kb":770,"num_write":383,"num_write_kb":2302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374180,"kb_used_data":12548,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567644,"statfs":{"total":1151940427776,"available":1151557267456,"internally_reserved":0,"allocated":12849152,"data_stored":8478797,"data_compressed":498873,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":67,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006488"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":179,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:03.162014+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:13:03.162014+0000","last_peered":"2024-07-04T08:13:03.162014+0000","last_clean":"2024-07-04T08:13:03.162014+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:13:03.162014+0000","last_undegraded":"2024-07-04T08:13:03.162014+0000","last_fullsized":"2024-07-04T08:13:03.162014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542203+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:12:51.542203+0000","last_peered":"2024-07-04T08:12:51.542203+0000","last_clean":"2024-07-04T08:12:51.542203+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:12:51.542203+0000","last_undegraded":"2024-07-04T08:12:51.542203+0000","last_fullsized":"2024-07-04T08:12:51.542203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691313+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:11:53.691313+0000","last_peered":"2024-07-04T08:11:53.691313+0000","last_clean":"2024-07-04T08:11:53.691313+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:11:53.691313+0000","last_undegraded":"2024-07-04T08:11:53.691313+0000","last_fullsized":"2024-07-04T08:11:53.691313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.698423+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:53.698423+0000","last_peered":"2024-07-04T08:11:53.698423+0000","last_clean":"2024-07-04T08:11:53.698423+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:53.698423+0000","last_undegraded":"2024-07-04T08:11:53.698423+0000","last_fullsized":"2024-07-04T08:11:53.698423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569847+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:12:51.569847+0000","last_peered":"2024-07-04T08:12:51.569847+0000","last_clean":"2024-07-04T08:12:51.569847+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:12:51.569847+0000","last_undegraded":"2024-07-04T08:12:51.569847+0000","last_fullsized":"2024-07-04T08:12:51.569847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691301+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:11:53.691301+0000","last_peered":"2024-07-04T08:11:53.691301+0000","last_clean":"2024-07-04T08:11:53.691301+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:11:53.691301+0000","last_undegraded":"2024-07-04T08:11:53.691301+0000","last_fullsized":"2024-07-04T08:11:53.691301+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":68,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771431+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:11:53.771431+0000","last_peered":"2024-07-04T08:11:53.771431+0000","last_clean":"2024-07-04T08:11:53.771431+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:11:53.771431+0000","last_undegraded":"2024-07-04T08:11:53.771431+0000","last_fullsized":"2024-07-04T08:11:53.771431+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.549686+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:12:51.549686+0000","last_peered":"2024-07-04T08:12:51.549686+0000","last_clean":"2024-07-04T08:12:51.549686+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:12:51.549686+0000","last_undegraded":"2024-07-04T08:12:51.549686+0000","last_fullsized":"2024-07-04T08:12:51.549686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691526+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:11:53.691526+0000","last_peered":"2024-07-04T08:11:53.691526+0000","last_clean":"2024-07-04T08:11:53.691526+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:11:53.691526+0000","last_undegraded":"2024-07-04T08:11:53.691526+0000","last_fullsized":"2024-07-04T08:11:53.691526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771226+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:11:53.771226+0000","last_peered":"2024-07-04T08:11:53.771226+0000","last_clean":"2024-07-04T08:11:53.771226+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:11:53.771226+0000","last_undegraded":"2024-07-04T08:11:53.771226+0000","last_fullsized":"2024-07-04T08:11:53.771226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.535777+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:12:51.535777+0000","last_peered":"2024-07-04T08:12:51.535777+0000","last_clean":"2024-07-04T08:12:51.535777+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:12:51.535777+0000","last_undegraded":"2024-07-04T08:12:51.535777+0000","last_fullsized":"2024-07-04T08:12:51.535777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691485+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:11:53.691485+0000","last_peered":"2024-07-04T08:11:53.691485+0000","last_clean":"2024-07-04T08:11:53.691485+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:11:53.691485+0000","last_undegraded":"2024-07-04T08:11:53.691485+0000","last_fullsized":"2024-07-04T08:11:53.691485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562435+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:12:51.562435+0000","last_peered":"2024-07-04T08:12:51.562435+0000","last_clean":"2024-07-04T08:12:51.562435+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:12:51.562435+0000","last_undegraded":"2024-07-04T08:12:51.562435+0000","last_fullsized":"2024-07-04T08:12:51.562435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844487+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:53.844487+0000","last_peered":"2024-07-04T08:11:53.844487+0000","last_clean":"2024-07-04T08:11:53.844487+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:53.844487+0000","last_undegraded":"2024-07-04T08:11:53.844487+0000","last_fullsized":"2024-07-04T08:11:53.844487+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692064+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:11:53.692064+0000","last_peered":"2024-07-04T08:11:53.692064+0000","last_clean":"2024-07-04T08:11:53.692064+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:11:53.692064+0000","last_undegraded":"2024-07-04T08:11:53.692064+0000","last_fullsized":"2024-07-04T08:11:53.692064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569552+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:12:51.569552+0000","last_peered":"2024-07-04T08:12:51.569552+0000","last_clean":"2024-07-04T08:12:51.569552+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:12:51.569552+0000","last_undegraded":"2024-07-04T08:12:51.569552+0000","last_fullsized":"2024-07-04T08:12:51.569552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691459+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:53.691459+0000","last_peered":"2024-07-04T08:11:53.691459+0000","last_clean":"2024-07-04T08:11:53.691459+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:53.691459+0000","last_undegraded":"2024-07-04T08:11:53.691459+0000","last_fullsized":"2024-07-04T08:11:53.691459+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691735+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:11:53.691735+0000","last_peered":"2024-07-04T08:11:53.691735+0000","last_clean":"2024-07-04T08:11:53.691735+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:11:53.691735+0000","last_undegraded":"2024-07-04T08:11:53.691735+0000","last_fullsized":"2024-07-04T08:11:53.691735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566516+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:12:51.566516+0000","last_peered":"2024-07-04T08:12:51.566516+0000","last_clean":"2024-07-04T08:12:51.566516+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:12:51.566516+0000","last_undegraded":"2024-07-04T08:12:51.566516+0000","last_fullsized":"2024-07-04T08:12:51.566516+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.696923+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:53.696923+0000","last_peered":"2024-07-04T08:11:53.696923+0000","last_clean":"2024-07-04T08:11:53.696923+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:53.696923+0000","last_undegraded":"2024-07-04T08:11:53.696923+0000","last_fullsized":"2024-07-04T08:11:53.696923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771192+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:11:53.771192+0000","last_peered":"2024-07-04T08:11:53.771192+0000","last_clean":"2024-07-04T08:11:53.771192+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:11:53.771192+0000","last_undegraded":"2024-07-04T08:11:53.771192+0000","last_fullsized":"2024-07-04T08:11:53.771192+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.493100+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:12:51.493100+0000","last_peered":"2024-07-04T08:12:51.493100+0000","last_clean":"2024-07-04T08:12:51.493100+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:12:51.493100+0000","last_undegraded":"2024-07-04T08:12:51.493100+0000","last_fullsized":"2024-07-04T08:12:51.493100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936252+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:53.936252+0000","last_peered":"2024-07-04T08:11:53.936252+0000","last_clean":"2024-07-04T08:11:53.936252+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:53.936252+0000","last_undegraded":"2024-07-04T08:11:53.936252+0000","last_fullsized":"2024-07-04T08:11:53.936252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691994+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:11:53.691994+0000","last_peered":"2024-07-04T08:11:53.691994+0000","last_clean":"2024-07-04T08:11:53.691994+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:11:53.691994+0000","last_undegraded":"2024-07-04T08:11:53.691994+0000","last_fullsized":"2024-07-04T08:11:53.691994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567815+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:12:51.567815+0000","last_peered":"2024-07-04T08:12:51.567815+0000","last_clean":"2024-07-04T08:12:51.567815+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:12:51.567815+0000","last_undegraded":"2024-07-04T08:12:51.567815+0000","last_fullsized":"2024-07-04T08:12:51.567815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692114+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:53.692114+0000","last_peered":"2024-07-04T08:11:53.692114+0000","last_clean":"2024-07-04T08:11:53.692114+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:53.692114+0000","last_undegraded":"2024-07-04T08:11:53.692114+0000","last_fullsized":"2024-07-04T08:11:53.692114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691612+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:11:53.691612+0000","last_peered":"2024-07-04T08:11:53.691612+0000","last_clean":"2024-07-04T08:11:53.691612+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:11:53.691612+0000","last_undegraded":"2024-07-04T08:11:53.691612+0000","last_fullsized":"2024-07-04T08:11:53.691612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.556440+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:12:51.556440+0000","last_peered":"2024-07-04T08:12:51.556440+0000","last_clean":"2024-07-04T08:12:51.556440+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:12:51.556440+0000","last_undegraded":"2024-07-04T08:12:51.556440+0000","last_fullsized":"2024-07-04T08:12:51.556440+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'16","reported_seq":71,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993720+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:11:53.993720+0000","last_peered":"2024-07-04T08:11:53.993720+0000","last_clean":"2024-07-04T08:11:53.993720+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:11:53.993720+0000","last_undegraded":"2024-07-04T08:11:53.993720+0000","last_fullsized":"2024-07-04T08:11:53.993720+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.697159+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:11:53.697159+0000","last_peered":"2024-07-04T08:11:53.697159+0000","last_clean":"2024-07-04T08:11:53.697159+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:11:53.697159+0000","last_undegraded":"2024-07-04T08:11:53.697159+0000","last_fullsized":"2024-07-04T08:11:53.697159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537810+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:12:51.537810+0000","last_peered":"2024-07-04T08:12:51.537810+0000","last_clean":"2024-07-04T08:12:51.537810+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:12:51.537810+0000","last_undegraded":"2024-07-04T08:12:51.537810+0000","last_fullsized":"2024-07-04T08:12:51.537810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771362+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:53.771362+0000","last_peered":"2024-07-04T08:11:53.771362+0000","last_clean":"2024-07-04T08:11:53.771362+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:53.771362+0000","last_undegraded":"2024-07-04T08:11:53.771362+0000","last_fullsized":"2024-07-04T08:11:53.771362+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844703+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:11:53.844703+0000","last_peered":"2024-07-04T08:11:53.844703+0000","last_clean":"2024-07-04T08:11:53.844703+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:11:53.844703+0000","last_undegraded":"2024-07-04T08:11:53.844703+0000","last_fullsized":"2024-07-04T08:11:53.844703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542427+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:12:51.542427+0000","last_peered":"2024-07-04T08:12:51.542427+0000","last_clean":"2024-07-04T08:12:51.542427+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:12:51.542427+0000","last_undegraded":"2024-07-04T08:12:51.542427+0000","last_fullsized":"2024-07-04T08:12:51.542427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993344+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:11:53.993344+0000","last_peered":"2024-07-04T08:11:53.993344+0000","last_clean":"2024-07-04T08:11:53.993344+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:11:53.993344+0000","last_undegraded":"2024-07-04T08:11:53.993344+0000","last_fullsized":"2024-07-04T08:11:53.993344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.770822+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:11:53.770822+0000","last_peered":"2024-07-04T08:11:53.770822+0000","last_clean":"2024-07-04T08:11:53.770822+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:11:53.770822+0000","last_undegraded":"2024-07-04T08:11:53.770822+0000","last_fullsized":"2024-07-04T08:11:53.770822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.543556+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:12:51.543556+0000","last_peered":"2024-07-04T08:12:51.543556+0000","last_clean":"2024-07-04T08:12:51.543556+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:12:51.543556+0000","last_undegraded":"2024-07-04T08:12:51.543556+0000","last_fullsized":"2024-07-04T08:12:51.543556+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":175,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:04.165595+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:13:04.165595+0000","last_peered":"2024-07-04T08:13:04.165595+0000","last_clean":"2024-07-04T08:13:04.165595+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:13:04.165595+0000","last_undegraded":"2024-07-04T08:13:04.165595+0000","last_fullsized":"2024-07-04T08:13:04.165595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995173+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:11:53.995173+0000","last_peered":"2024-07-04T08:11:53.995173+0000","last_clean":"2024-07-04T08:11:53.995173+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:11:53.995173+0000","last_undegraded":"2024-07-04T08:11:53.995173+0000","last_fullsized":"2024-07-04T08:11:53.995173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.516818+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:12:51.516818+0000","last_peered":"2024-07-04T08:12:51.516818+0000","last_clean":"2024-07-04T08:12:51.516818+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:12:51.516818+0000","last_undegraded":"2024-07-04T08:12:51.516818+0000","last_fullsized":"2024-07-04T08:12:51.516818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":169,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:01.166190+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:13:01.166190+0000","last_peered":"2024-07-04T08:13:01.166190+0000","last_clean":"2024-07-04T08:13:01.166190+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:13:01.166190+0000","last_undegraded":"2024-07-04T08:13:01.166190+0000","last_fullsized":"2024-07-04T08:13:01.166190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995252+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:11:53.995252+0000","last_peered":"2024-07-04T08:11:53.995252+0000","last_clean":"2024-07-04T08:11:53.995252+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:11:53.995252+0000","last_undegraded":"2024-07-04T08:11:53.995252+0000","last_fullsized":"2024-07-04T08:11:53.995252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534643+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:12:51.534643+0000","last_peered":"2024-07-04T08:12:51.534643+0000","last_clean":"2024-07-04T08:12:51.534643+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:12:51.534643+0000","last_undegraded":"2024-07-04T08:12:51.534643+0000","last_fullsized":"2024-07-04T08:12:51.534643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851356+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:53.851356+0000","last_peered":"2024-07-04T08:11:53.851356+0000","last_clean":"2024-07-04T08:11:53.851356+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:53.851356+0000","last_undegraded":"2024-07-04T08:11:53.851356+0000","last_fullsized":"2024-07-04T08:11:53.851356+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939614+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:11:53.939614+0000","last_peered":"2024-07-04T08:11:53.939614+0000","last_clean":"2024-07-04T08:11:53.939614+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:11:53.939614+0000","last_undegraded":"2024-07-04T08:11:53.939614+0000","last_fullsized":"2024-07-04T08:11:53.939614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.558211+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:12:51.558211+0000","last_peered":"2024-07-04T08:12:51.558211+0000","last_clean":"2024-07-04T08:12:51.558211+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:12:51.558211+0000","last_undegraded":"2024-07-04T08:12:51.558211+0000","last_fullsized":"2024-07-04T08:12:51.558211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691801+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:53.691801+0000","last_peered":"2024-07-04T08:11:53.691801+0000","last_clean":"2024-07-04T08:11:53.691801+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:53.691801+0000","last_undegraded":"2024-07-04T08:11:53.691801+0000","last_fullsized":"2024-07-04T08:11:53.691801+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":119,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851572+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:11:53.851572+0000","last_peered":"2024-07-04T08:11:53.851572+0000","last_clean":"2024-07-04T08:11:53.851572+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:11:53.851572+0000","last_undegraded":"2024-07-04T08:11:53.851572+0000","last_fullsized":"2024-07-04T08:11:53.851572+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851869+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:11:53.851869+0000","last_peered":"2024-07-04T08:11:53.851869+0000","last_clean":"2024-07-04T08:11:53.851869+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:11:53.851869+0000","last_undegraded":"2024-07-04T08:11:53.851869+0000","last_fullsized":"2024-07-04T08:11:53.851869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553706+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:12:51.553706+0000","last_peered":"2024-07-04T08:12:51.553706+0000","last_clean":"2024-07-04T08:12:51.553706+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:12:51.553706+0000","last_undegraded":"2024-07-04T08:12:51.553706+0000","last_fullsized":"2024-07-04T08:12:51.553706+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":141,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:03.031090+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:13:03.031090+0000","last_peered":"2024-07-04T08:13:03.031090+0000","last_clean":"2024-07-04T08:13:03.031090+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:13:03.031090+0000","last_undegraded":"2024-07-04T08:13:03.031090+0000","last_fullsized":"2024-07-04T08:13:03.031090+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936202+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:11:53.936202+0000","last_peered":"2024-07-04T08:11:53.936202+0000","last_clean":"2024-07-04T08:11:53.936202+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:11:53.936202+0000","last_undegraded":"2024-07-04T08:11:53.936202+0000","last_fullsized":"2024-07-04T08:11:53.936202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.561382+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:12:51.561382+0000","last_peered":"2024-07-04T08:12:51.561382+0000","last_clean":"2024-07-04T08:12:51.561382+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:12:51.561382+0000","last_undegraded":"2024-07-04T08:12:51.561382+0000","last_fullsized":"2024-07-04T08:12:51.561382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939874+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:53.939874+0000","last_peered":"2024-07-04T08:11:53.939874+0000","last_clean":"2024-07-04T08:11:53.939874+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:11:53.939874+0000","last_undegraded":"2024-07-04T08:11:53.939874+0000","last_fullsized":"2024-07-04T08:11:53.939874+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.105658+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:11:54.105658+0000","last_peered":"2024-07-04T08:11:54.105658+0000","last_clean":"2024-07-04T08:11:54.105658+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:11:54.105658+0000","last_undegraded":"2024-07-04T08:11:54.105658+0000","last_fullsized":"2024-07-04T08:11:54.105658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552444+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:12:51.552444+0000","last_peered":"2024-07-04T08:12:51.552444+0000","last_clean":"2024-07-04T08:12:51.552444+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:12:51.552444+0000","last_undegraded":"2024-07-04T08:12:51.552444+0000","last_fullsized":"2024-07-04T08:12:51.552444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.994981+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:53.994981+0000","last_peered":"2024-07-04T08:11:53.994981+0000","last_clean":"2024-07-04T08:11:53.994981+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:53.994981+0000","last_undegraded":"2024-07-04T08:11:53.994981+0000","last_fullsized":"2024-07-04T08:11:53.994981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.938978+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:11:53.938978+0000","last_peered":"2024-07-04T08:11:53.938978+0000","last_clean":"2024-07-04T08:11:53.938978+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:11:53.938978+0000","last_undegraded":"2024-07-04T08:11:53.938978+0000","last_fullsized":"2024-07-04T08:11:53.938978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531754+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:12:51.531754+0000","last_peered":"2024-07-04T08:12:51.531754+0000","last_clean":"2024-07-04T08:12:51.531754+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:12:51.531754+0000","last_undegraded":"2024-07-04T08:12:51.531754+0000","last_fullsized":"2024-07-04T08:12:51.531754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771313+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:53.771313+0000","last_peered":"2024-07-04T08:11:53.771313+0000","last_clean":"2024-07-04T08:11:53.771313+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:53.771313+0000","last_undegraded":"2024-07-04T08:11:53.771313+0000","last_fullsized":"2024-07-04T08:11:53.771313+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939064+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:11:53.939064+0000","last_peered":"2024-07-04T08:11:53.939064+0000","last_clean":"2024-07-04T08:11:53.939064+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:11:53.939064+0000","last_undegraded":"2024-07-04T08:11:53.939064+0000","last_fullsized":"2024-07-04T08:11:53.939064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542560+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:12:51.542560+0000","last_peered":"2024-07-04T08:12:51.542560+0000","last_clean":"2024-07-04T08:12:51.542560+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:12:51.542560+0000","last_undegraded":"2024-07-04T08:12:51.542560+0000","last_fullsized":"2024-07-04T08:12:51.542560+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":63,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.770763+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:53.770763+0000","last_peered":"2024-07-04T08:11:53.770763+0000","last_clean":"2024-07-04T08:11:53.770763+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:53.770763+0000","last_undegraded":"2024-07-04T08:11:53.770763+0000","last_fullsized":"2024-07-04T08:11:53.770763+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995018+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:11:53.995018+0000","last_peered":"2024-07-04T08:11:53.995018+0000","last_clean":"2024-07-04T08:11:53.995018+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:11:53.995018+0000","last_undegraded":"2024-07-04T08:11:53.995018+0000","last_fullsized":"2024-07-04T08:11:53.995018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551660+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:12:51.551660+0000","last_peered":"2024-07-04T08:12:51.551660+0000","last_clean":"2024-07-04T08:12:51.551660+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:12:51.551660+0000","last_undegraded":"2024-07-04T08:12:51.551660+0000","last_fullsized":"2024-07-04T08:12:51.551660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":176,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:04.174735+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:13:04.174735+0000","last_peered":"2024-07-04T08:13:04.174735+0000","last_clean":"2024-07-04T08:13:04.174735+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:13:04.174735+0000","last_undegraded":"2024-07-04T08:13:04.174735+0000","last_fullsized":"2024-07-04T08:13:04.174735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993663+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:11:53.993663+0000","last_peered":"2024-07-04T08:11:53.993663+0000","last_clean":"2024-07-04T08:11:53.993663+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:11:53.993663+0000","last_undegraded":"2024-07-04T08:11:53.993663+0000","last_fullsized":"2024-07-04T08:11:53.993663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564845+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:12:51.564845+0000","last_peered":"2024-07-04T08:12:51.564845+0000","last_clean":"2024-07-04T08:12:51.564845+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:12:51.564845+0000","last_undegraded":"2024-07-04T08:12:51.564845+0000","last_fullsized":"2024-07-04T08:12:51.564845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939360+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:53.939360+0000","last_peered":"2024-07-04T08:11:53.939360+0000","last_clean":"2024-07-04T08:11:53.939360+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:53.939360+0000","last_undegraded":"2024-07-04T08:11:53.939360+0000","last_fullsized":"2024-07-04T08:11:53.939360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844593+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:11:53.844593+0000","last_peered":"2024-07-04T08:11:53.844593+0000","last_clean":"2024-07-04T08:11:53.844593+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:11:53.844593+0000","last_undegraded":"2024-07-04T08:11:53.844593+0000","last_fullsized":"2024-07-04T08:11:53.844593+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562054+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:12:51.562054+0000","last_peered":"2024-07-04T08:12:51.562054+0000","last_clean":"2024-07-04T08:12:51.562054+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:12:51.562054+0000","last_undegraded":"2024-07-04T08:12:51.562054+0000","last_fullsized":"2024-07-04T08:12:51.562054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691602+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:53.691602+0000","last_peered":"2024-07-04T08:11:53.691602+0000","last_clean":"2024-07-04T08:11:53.691602+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:53.691602+0000","last_undegraded":"2024-07-04T08:11:53.691602+0000","last_fullsized":"2024-07-04T08:11:53.691602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.997536+0000","last_change":"2024-07-04T08:11:53.997034+0000","last_active":"2024-07-04T08:11:53.997536+0000","last_peered":"2024-07-04T08:11:53.997536+0000","last_clean":"2024-07-04T08:11:53.997536+0000","last_became_active":"2024-07-04T08:11:53.996456+0000","last_became_peered":"2024-07-04T08:11:53.996456+0000","last_unstale":"2024-07-04T08:11:53.997536+0000","last_undegraded":"2024-07-04T08:11:53.997536+0000","last_fullsized":"2024-07-04T08:11:53.997536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.140549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.521670+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:12:51.521670+0000","last_peered":"2024-07-04T08:12:51.521670+0000","last_clean":"2024-07-04T08:12:51.521670+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:12:51.521670+0000","last_undegraded":"2024-07-04T08:12:51.521670+0000","last_fullsized":"2024-07-04T08:12:51.521670+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":335,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:06.070929+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:13:06.070929+0000","last_peered":"2024-07-04T08:13:06.070929+0000","last_clean":"2024-07-04T08:13:06.070929+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:13:06.070929+0000","last_undegraded":"2024-07-04T08:13:06.070929+0000","last_fullsized":"2024-07-04T08:13:06.070929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":189,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.992479+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:11:53.992479+0000","last_peered":"2024-07-04T08:11:53.992479+0000","last_clean":"2024-07-04T08:11:53.992479+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:11:53.992479+0000","last_undegraded":"2024-07-04T08:11:53.992479+0000","last_fullsized":"2024-07-04T08:11:53.992479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541776+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:12:51.541776+0000","last_peered":"2024-07-04T08:12:51.541776+0000","last_clean":"2024-07-04T08:12:51.541776+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:12:51.541776+0000","last_undegraded":"2024-07-04T08:12:51.541776+0000","last_fullsized":"2024-07-04T08:12:51.541776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939902+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:53.939902+0000","last_peered":"2024-07-04T08:11:53.939902+0000","last_clean":"2024-07-04T08:11:53.939902+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:53.939902+0000","last_undegraded":"2024-07-04T08:11:53.939902+0000","last_fullsized":"2024-07-04T08:11:53.939902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691764+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:11:53.691764+0000","last_peered":"2024-07-04T08:11:53.691764+0000","last_clean":"2024-07-04T08:11:53.691764+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:11:53.691764+0000","last_undegraded":"2024-07-04T08:11:53.691764+0000","last_fullsized":"2024-07-04T08:11:53.691764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569726+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:12:51.569726+0000","last_peered":"2024-07-04T08:12:51.569726+0000","last_clean":"2024-07-04T08:12:51.569726+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:12:51.569726+0000","last_undegraded":"2024-07-04T08:12:51.569726+0000","last_fullsized":"2024-07-04T08:12:51.569726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.852004+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:11:53.852004+0000","last_peered":"2024-07-04T08:11:53.852004+0000","last_clean":"2024-07-04T08:11:53.852004+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:11:53.852004+0000","last_undegraded":"2024-07-04T08:11:53.852004+0000","last_fullsized":"2024-07-04T08:11:53.852004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.991447+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:11:53.991447+0000","last_peered":"2024-07-04T08:11:53.991447+0000","last_clean":"2024-07-04T08:11:53.991447+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:11:53.991447+0000","last_undegraded":"2024-07-04T08:11:53.991447+0000","last_fullsized":"2024-07-04T08:11:53.991447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.546983+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:12:51.546983+0000","last_peered":"2024-07-04T08:12:51.546983+0000","last_clean":"2024-07-04T08:12:51.546983+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:12:51.546983+0000","last_undegraded":"2024-07-04T08:12:51.546983+0000","last_fullsized":"2024-07-04T08:12:51.546983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":291,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:06.034661+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:13:06.034661+0000","last_peered":"2024-07-04T08:13:06.034661+0000","last_clean":"2024-07-04T08:13:06.034661+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:13:06.034661+0000","last_undegraded":"2024-07-04T08:13:06.034661+0000","last_fullsized":"2024-07-04T08:13:06.034661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":242,"num_read_kb":242,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.696582+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:11:53.696582+0000","last_peered":"2024-07-04T08:11:53.696582+0000","last_clean":"2024-07-04T08:11:53.696582+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:11:53.696582+0000","last_undegraded":"2024-07-04T08:11:53.696582+0000","last_fullsized":"2024-07-04T08:11:53.696582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.106088+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:11:54.106088+0000","last_peered":"2024-07-04T08:11:54.106088+0000","last_clean":"2024-07-04T08:11:54.106088+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:11:54.106088+0000","last_undegraded":"2024-07-04T08:11:54.106088+0000","last_fullsized":"2024-07-04T08:11:54.106088+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564360+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:12:51.564360+0000","last_peered":"2024-07-04T08:12:51.564360+0000","last_clean":"2024-07-04T08:12:51.564360+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:12:51.564360+0000","last_undegraded":"2024-07-04T08:12:51.564360+0000","last_fullsized":"2024-07-04T08:12:51.564360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939445+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:53.939445+0000","last_peered":"2024-07-04T08:11:53.939445+0000","last_clean":"2024-07-04T08:11:53.939445+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:53.939445+0000","last_undegraded":"2024-07-04T08:11:53.939445+0000","last_fullsized":"2024-07-04T08:11:53.939445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844766+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:11:53.844766+0000","last_peered":"2024-07-04T08:11:53.844766+0000","last_clean":"2024-07-04T08:11:53.844766+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:11:53.844766+0000","last_undegraded":"2024-07-04T08:11:53.844766+0000","last_fullsized":"2024-07-04T08:11:53.844766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565483+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:12:51.565483+0000","last_peered":"2024-07-04T08:12:51.565483+0000","last_clean":"2024-07-04T08:12:51.565483+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:12:51.565483+0000","last_undegraded":"2024-07-04T08:12:51.565483+0000","last_fullsized":"2024-07-04T08:12:51.565483+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.994878+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:11:53.994878+0000","last_peered":"2024-07-04T08:11:53.994878+0000","last_clean":"2024-07-04T08:11:53.994878+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:11:53.994878+0000","last_undegraded":"2024-07-04T08:11:53.994878+0000","last_fullsized":"2024-07-04T08:11:53.994878+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.699922+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:11:53.699922+0000","last_peered":"2024-07-04T08:11:53.699922+0000","last_clean":"2024-07-04T08:11:53.699922+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:11:53.699922+0000","last_undegraded":"2024-07-04T08:11:53.699922+0000","last_fullsized":"2024-07-04T08:11:53.699922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.533724+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:12:51.533724+0000","last_peered":"2024-07-04T08:12:51.533724+0000","last_clean":"2024-07-04T08:12:51.533724+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:12:51.533724+0000","last_undegraded":"2024-07-04T08:12:51.533724+0000","last_fullsized":"2024-07-04T08:12:51.533724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691660+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:11:53.691660+0000","last_peered":"2024-07-04T08:11:53.691660+0000","last_clean":"2024-07-04T08:11:53.691660+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:11:53.691660+0000","last_undegraded":"2024-07-04T08:11:53.691660+0000","last_fullsized":"2024-07-04T08:11:53.691660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":171,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:03.049579+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:13:03.049579+0000","last_peered":"2024-07-04T08:13:03.049579+0000","last_clean":"2024-07-04T08:13:03.049579+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:13:03.049579+0000","last_undegraded":"2024-07-04T08:13:03.049579+0000","last_fullsized":"2024-07-04T08:13:03.049579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.518982+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:12:51.518982+0000","last_peered":"2024-07-04T08:12:51.518982+0000","last_clean":"2024-07-04T08:12:51.518982+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:12:51.518982+0000","last_undegraded":"2024-07-04T08:12:51.518982+0000","last_fullsized":"2024-07-04T08:12:51.518982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691843+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:11:53.691843+0000","last_peered":"2024-07-04T08:11:53.691843+0000","last_clean":"2024-07-04T08:11:53.691843+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:11:53.691843+0000","last_undegraded":"2024-07-04T08:11:53.691843+0000","last_fullsized":"2024-07-04T08:11:53.691843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534082+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:12:51.534082+0000","last_peered":"2024-07-04T08:12:51.534082+0000","last_clean":"2024-07-04T08:12:51.534082+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:12:51.534082+0000","last_undegraded":"2024-07-04T08:12:51.534082+0000","last_fullsized":"2024-07-04T08:12:51.534082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.539557+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:12:51.539557+0000","last_peered":"2024-07-04T08:12:51.539557+0000","last_clean":"2024-07-04T08:12:51.539557+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:12:51.539557+0000","last_undegraded":"2024-07-04T08:12:51.539557+0000","last_fullsized":"2024-07-04T08:12:51.539557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566225+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:12:51.566225+0000","last_peered":"2024-07-04T08:12:51.566225+0000","last_clean":"2024-07-04T08:12:51.566225+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:12:51.566225+0000","last_undegraded":"2024-07-04T08:12:51.566225+0000","last_fullsized":"2024-07-04T08:12:51.566225+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567598+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:12:51.567598+0000","last_peered":"2024-07-04T08:12:51.567598+0000","last_clean":"2024-07-04T08:12:51.567598+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:12:51.567598+0000","last_undegraded":"2024-07-04T08:12:51.567598+0000","last_fullsized":"2024-07-04T08:12:51.567598+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569096+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:12:51.569096+0000","last_peered":"2024-07-04T08:12:51.569096+0000","last_clean":"2024-07-04T08:12:51.569096+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:12:51.569096+0000","last_undegraded":"2024-07-04T08:12:51.569096+0000","last_fullsized":"2024-07-04T08:12:51.569096+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532204+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:12:51.532204+0000","last_peered":"2024-07-04T08:12:51.532204+0000","last_clean":"2024-07-04T08:12:51.532204+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:12:51.532204+0000","last_undegraded":"2024-07-04T08:12:51.532204+0000","last_fullsized":"2024-07-04T08:12:51.532204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540456+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:12:51.540456+0000","last_peered":"2024-07-04T08:12:51.540456+0000","last_clean":"2024-07-04T08:12:51.540456+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:12:51.540456+0000","last_undegraded":"2024-07-04T08:12:51.540456+0000","last_fullsized":"2024-07-04T08:12:51.540456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552090+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:12:51.552090+0000","last_peered":"2024-07-04T08:12:51.552090+0000","last_clean":"2024-07-04T08:12:51.552090+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:12:51.552090+0000","last_undegraded":"2024-07-04T08:12:51.552090+0000","last_fullsized":"2024-07-04T08:12:51.552090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531386+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:12:51.531386+0000","last_peered":"2024-07-04T08:12:51.531386+0000","last_clean":"2024-07-04T08:12:51.531386+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:12:51.531386+0000","last_undegraded":"2024-07-04T08:12:51.531386+0000","last_fullsized":"2024-07-04T08:12:51.531386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.555323+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:12:51.555323+0000","last_peered":"2024-07-04T08:12:51.555323+0000","last_clean":"2024-07-04T08:12:51.555323+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:12:51.555323+0000","last_undegraded":"2024-07-04T08:12:51.555323+0000","last_fullsized":"2024-07-04T08:12:51.555323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560682+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:12:51.560682+0000","last_peered":"2024-07-04T08:12:51.560682+0000","last_clean":"2024-07-04T08:12:51.560682+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:12:51.560682+0000","last_undegraded":"2024-07-04T08:12:51.560682+0000","last_fullsized":"2024-07-04T08:12:51.560682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570606+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:12:51.570606+0000","last_peered":"2024-07-04T08:12:51.570606+0000","last_clean":"2024-07-04T08:12:51.570606+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:12:51.570606+0000","last_undegraded":"2024-07-04T08:12:51.570606+0000","last_fullsized":"2024-07-04T08:12:51.570606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532600+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:12:51.532600+0000","last_peered":"2024-07-04T08:12:51.532600+0000","last_clean":"2024-07-04T08:12:51.532600+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:12:51.532600+0000","last_undegraded":"2024-07-04T08:12:51.532600+0000","last_fullsized":"2024-07-04T08:12:51.532600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.538492+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:12:51.538492+0000","last_peered":"2024-07-04T08:12:51.538492+0000","last_clean":"2024-07-04T08:12:51.538492+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:12:51.538492+0000","last_undegraded":"2024-07-04T08:12:51.538492+0000","last_fullsized":"2024-07-04T08:12:51.538492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540580+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:12:51.540580+0000","last_peered":"2024-07-04T08:12:51.540580+0000","last_clean":"2024-07-04T08:12:51.540580+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:12:51.540580+0000","last_undegraded":"2024-07-04T08:12:51.540580+0000","last_fullsized":"2024-07-04T08:12:51.540580+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569630+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:12:51.569630+0000","last_peered":"2024-07-04T08:12:51.569630+0000","last_clean":"2024-07-04T08:12:51.569630+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:12:51.569630+0000","last_undegraded":"2024-07-04T08:12:51.569630+0000","last_fullsized":"2024-07-04T08:12:51.569630+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.571003+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:12:51.571003+0000","last_peered":"2024-07-04T08:12:51.571003+0000","last_clean":"2024-07-04T08:12:51.571003+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:12:51.571003+0000","last_undegraded":"2024-07-04T08:12:51.571003+0000","last_fullsized":"2024-07-04T08:12:51.571003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552014+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:12:51.552014+0000","last_peered":"2024-07-04T08:12:51.552014+0000","last_clean":"2024-07-04T08:12:51.552014+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:12:51.552014+0000","last_undegraded":"2024-07-04T08:12:51.552014+0000","last_fullsized":"2024-07-04T08:12:51.552014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567116+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:12:51.567116+0000","last_peered":"2024-07-04T08:12:51.567116+0000","last_clean":"2024-07-04T08:12:51.567116+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:12:51.567116+0000","last_undegraded":"2024-07-04T08:12:51.567116+0000","last_fullsized":"2024-07-04T08:12:51.567116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.550951+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:12:51.550951+0000","last_peered":"2024-07-04T08:12:51.550951+0000","last_clean":"2024-07-04T08:12:51.550951+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:12:51.550951+0000","last_undegraded":"2024-07-04T08:12:51.550951+0000","last_fullsized":"2024-07-04T08:12:51.550951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.533735+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:12:51.533735+0000","last_peered":"2024-07-04T08:12:51.533735+0000","last_clean":"2024-07-04T08:12:51.533735+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:12:51.533735+0000","last_undegraded":"2024-07-04T08:12:51.533735+0000","last_fullsized":"2024-07-04T08:12:51.533735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553530+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:12:51.553530+0000","last_peered":"2024-07-04T08:12:51.553530+0000","last_clean":"2024-07-04T08:12:51.553530+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:12:51.553530+0000","last_undegraded":"2024-07-04T08:12:51.553530+0000","last_fullsized":"2024-07-04T08:12:51.553530+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540232+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:12:51.540232+0000","last_peered":"2024-07-04T08:12:51.540232+0000","last_clean":"2024-07-04T08:12:51.540232+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:12:51.540232+0000","last_undegraded":"2024-07-04T08:12:51.540232+0000","last_fullsized":"2024-07-04T08:12:51.540232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540940+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:12:51.540940+0000","last_peered":"2024-07-04T08:12:51.540940+0000","last_clean":"2024-07-04T08:12:51.540940+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:12:51.540940+0000","last_undegraded":"2024-07-04T08:12:51.540940+0000","last_fullsized":"2024-07-04T08:12:51.540940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560242+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:12:51.560242+0000","last_peered":"2024-07-04T08:12:51.560242+0000","last_clean":"2024-07-04T08:12:51.560242+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:12:51.560242+0000","last_undegraded":"2024-07-04T08:12:51.560242+0000","last_fullsized":"2024-07-04T08:12:51.560242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.550113+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:12:51.550113+0000","last_peered":"2024-07-04T08:12:51.550113+0000","last_clean":"2024-07-04T08:12:51.550113+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:12:51.550113+0000","last_undegraded":"2024-07-04T08:12:51.550113+0000","last_fullsized":"2024-07-04T08:12:51.550113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541407+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:12:51.541407+0000","last_peered":"2024-07-04T08:12:51.541407+0000","last_clean":"2024-07-04T08:12:51.541407+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:12:51.541407+0000","last_undegraded":"2024-07-04T08:12:51.541407+0000","last_fullsized":"2024-07-04T08:12:51.541407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.536688+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:12:51.536688+0000","last_peered":"2024-07-04T08:12:51.536688+0000","last_clean":"2024-07-04T08:12:51.536688+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:12:51.536688+0000","last_undegraded":"2024-07-04T08:12:51.536688+0000","last_fullsized":"2024-07-04T08:12:51.536688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537059+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:12:51.537059+0000","last_peered":"2024-07-04T08:12:51.537059+0000","last_clean":"2024-07-04T08:12:51.537059+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:12:51.537059+0000","last_undegraded":"2024-07-04T08:12:51.537059+0000","last_fullsized":"2024-07-04T08:12:51.537059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565082+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:12:51.565082+0000","last_peered":"2024-07-04T08:12:51.565082+0000","last_clean":"2024-07-04T08:12:51.565082+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:12:51.565082+0000","last_undegraded":"2024-07-04T08:12:51.565082+0000","last_fullsized":"2024-07-04T08:12:51.565082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567958+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:12:51.567958+0000","last_peered":"2024-07-04T08:12:51.567958+0000","last_clean":"2024-07-04T08:12:51.567958+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:12:51.567958+0000","last_undegraded":"2024-07-04T08:12:51.567958+0000","last_fullsized":"2024-07-04T08:12:51.567958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569411+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:12:51.569411+0000","last_peered":"2024-07-04T08:12:51.569411+0000","last_clean":"2024-07-04T08:12:51.569411+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:12:51.569411+0000","last_undegraded":"2024-07-04T08:12:51.569411+0000","last_fullsized":"2024-07-04T08:12:51.569411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71871,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1242,"num_read_kb":701,"num_write":258,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":215613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612667,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":543564,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808894,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1195297,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":547590,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168651,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":543872,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364881,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1141042,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528405,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":539530,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724635,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1135560,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920864,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":543604,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117093,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":538706,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378733,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":603463,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542257,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":606115,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705782,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":540454,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:13:08.571 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-04T08:13:08.571 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-04T08:13:08.571 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-07-04T08:13:08.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph health --format=json 2024-07-04T08:13:09.094 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:09.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:09 smithi133 ceph-mon[37050]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-07-04T08:13:09.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:09 smithi133 ceph-mon[37050]: from='client.25105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:13:09.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:09 smithi039 ceph-mon[32062]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-07-04T08:13:09.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:09 smithi039 ceph-mon[32062]: from='client.25105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:13:09.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:09 smithi050 ceph-mon[36577]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-07-04T08:13:09.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:09 smithi050 ceph-mon[36577]: from='client.25105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.224+0000 7fc703e00640 1 -- 172.21.15.39:0/2696728818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc704074ca0 msgr2=0x7fc7040750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.224+0000 7fc703e00640 1 --2- 172.21.15.39:0/2696728818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc704074ca0 0x7fc7040750b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009900 tx=0x7fc6f402a410 comp rx=0 tx=0).stop 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.224+0000 7fc703e00640 1 -- 172.21.15.39:0/2696728818 shutdown_connections 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.224+0000 7fc703e00640 1 --2- 172.21.15.39:0/2696728818 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc70410f4c0 0x7fc7041118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.224+0000 7fc703e00640 1 --2- 172.21.15.39:0/2696728818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7040755f0 0x7fc70410ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.224+0000 7fc703e00640 1 --2- 172.21.15.39:0/2696728818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc704074ca0 0x7fc7040750b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.224+0000 7fc703e00640 1 -- 172.21.15.39:0/2696728818 >> 172.21.15.39:0/2696728818 conn(0x7fc7040fe950 msgr2=0x7fc704100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.224+0000 7fc703e00640 1 -- 172.21.15.39:0/2696728818 shutdown_connections 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.224+0000 7fc703e00640 1 -- 172.21.15.39:0/2696728818 wait complete. 2024-07-04T08:13:11.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.226+0000 7fc703e00640 1 Processor -- start 2024-07-04T08:13:11.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.226+0000 7fc703e00640 1 -- start start 2024-07-04T08:13:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.226+0000 7fc703e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704074ca0 0x7fc70419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.226+0000 7fc703e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7040755f0 0x7fc70419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.226+0000 7fc703e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc70410f4c0 0x7fc7041a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.226+0000 7fc703e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc704115030 con 0x7fc70410f4c0 2024-07-04T08:13:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.226+0000 7fc703e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc704114eb0 con 0x7fc704074ca0 2024-07-04T08:13:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.226+0000 7fc703e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc7041151b0 con 0x7fc7040755f0 2024-07-04T08:13:11.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704074ca0 0x7fc70419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7040755f0 0x7fc70419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7040755f0 0x7fc70419bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53394/0 (socket says 172.21.15.39:53394) 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704074ca0 0x7fc70419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60406/0 (socket says 172.21.15.39:60406) 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc703400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc70410f4c0 0x7fc7041a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702000640 1 -- 172.21.15.39:0/2380021482 learned_addr learned my addr 172.21.15.39:0/2380021482 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702000640 1 -- 172.21.15.39:0/2380021482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704074ca0 msgr2=0x7fc70419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702000640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704074ca0 0x7fc70419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702000640 1 -- 172.21.15.39:0/2380021482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc70410f4c0 msgr2=0x7fc7041a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702000640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc70410f4c0 0x7fc7041a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.227+0000 7fc702000640 1 -- 172.21.15.39:0/2380021482 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7041a1990 con 0x7fc7040755f0 2024-07-04T08:13:11.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.228+0000 7fc702000640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7040755f0 0x7fc70419bfc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000e9c0 tx=0x7fc6f000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.228+0000 7fc6fb400640 1 -- 172.21.15.39:0/2380021482 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f000ccc0 con 0x7fc7040755f0 2024-07-04T08:13:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.228+0000 7fc6fb400640 1 -- 172.21.15.39:0/2380021482 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6f00054d0 con 0x7fc7040755f0 2024-07-04T08:13:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.228+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7041a1c80 con 0x7fc7040755f0 2024-07-04T08:13:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.228+0000 7fc6fb400640 1 -- 172.21.15.39:0/2380021482 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0010430 con 0x7fc7040755f0 2024-07-04T08:13:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.228+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7041a83e0 con 0x7fc7040755f0 2024-07-04T08:13:11.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.229+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7040750b0 con 0x7fc7040755f0 2024-07-04T08:13:11.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.232+0000 7fc6fb400640 1 -- 172.21.15.39:0/2380021482 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fc6f0005000 con 0x7fc7040755f0 2024-07-04T08:13:11.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.233+0000 7fc6fb400640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6e80b2840 0x7fc6e80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:11.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.234+0000 7fc6fb400640 1 -- 172.21.15.39:0/2380021482 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc6f00e2c30 con 0x7fc7040755f0 2024-07-04T08:13:11.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.234+0000 7fc702a00640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6e80b2840 0x7fc6e80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:11.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.234+0000 7fc702a00640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6e80b2840 0x7fc6e80b4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4031440 tx=0x7fc6f4009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:11.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.235+0000 7fc6fb400640 1 -- 172.21.15.39:0/2380021482 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc6f0055110 con 0x7fc7040755f0 2024-07-04T08:13:11.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:11 smithi133 ceph-mon[37050]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-04T08:13:11.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:11 smithi039 ceph-mon[32062]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-04T08:13:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:11 smithi050 ceph-mon[36577]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-04T08:13:11.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.477+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc70419cb50 con 0x7fc7040755f0 2024-07-04T08:13:11.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.477+0000 7fc6fb400640 1 -- 172.21.15.39:0/2380021482 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc6f00052f0 con 0x7fc7040755f0 2024-07-04T08:13:11.479 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:13:11.479 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-04T08:13:11.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6e80b2840 msgr2=0x7fc6e80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6e80b2840 0x7fc6e80b4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4031440 tx=0x7fc6f4009560 comp rx=0 tx=0).stop 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7040755f0 msgr2=0x7fc70419bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7040755f0 0x7fc70419bfc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000e9c0 tx=0x7fc6f000ee80 comp rx=0 tx=0).stop 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 shutdown_connections 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6e80b2840 0x7fc6e80b4d50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc70410f4c0 0x7fc7041a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc7040755f0 0x7fc70419bfc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 --2- 172.21.15.39:0/2380021482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc704074ca0 0x7fc70419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 >> 172.21.15.39:0/2380021482 conn(0x7fc7040fe950 msgr2=0x7fc704109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:11.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.480+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 shutdown_connections 2024-07-04T08:13:11.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:11.481+0000 7fc703e00640 1 -- 172.21.15.39:0/2380021482 wait complete. 2024-07-04T08:13:12.234 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:12 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2380021482' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-04T08:13:12.295 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-07-04T08:13:12.296 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:13:12.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:12 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2380021482' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-04T08:13:12.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:12 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2380021482' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-04T08:13:12.782 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:13.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:13 smithi039 ceph-mon[32062]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:13.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:13 smithi133 ceph-mon[37050]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:13.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:13 smithi050 ceph-mon[36577]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.932+0000 7f224b200640 1 -- 172.21.15.39:0/2394779708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f224c108820 msgr2=0x7f224c108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.932+0000 7f224b200640 1 --2- 172.21.15.39:0/2394779708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f224c108820 0x7f224c108c30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2234009900 tx=0x7f223402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:13:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.932+0000 7f224b200640 1 -- 172.21.15.39:0/2394779708 shutdown_connections 2024-07-04T08:13:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.932+0000 7f224b200640 1 --2- 172.21.15.39:0/2394779708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c109170 0x7f224c10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.932+0000 7f224b200640 1 --2- 172.21.15.39:0/2394779708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f224c108820 0x7f224c108c30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.932+0000 7f224b200640 1 --2- 172.21.15.39:0/2394779708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f224c102ee0 0x7f224c1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.932+0000 7f224b200640 1 -- 172.21.15.39:0/2394779708 >> 172.21.15.39:0/2394779708 conn(0x7f224c0fe930 msgr2=0x7f224c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.932+0000 7f224b200640 1 -- 172.21.15.39:0/2394779708 shutdown_connections 2024-07-04T08:13:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.932+0000 7f224b200640 1 -- 172.21.15.39:0/2394779708 wait complete. 2024-07-04T08:13:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.934+0000 7f224b200640 1 Processor -- start 2024-07-04T08:13:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.934+0000 7f224b200640 1 -- start start 2024-07-04T08:13:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.934+0000 7f224b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f224c102ee0 0x7f224c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f224b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f224c108820 0x7f224c0751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f224b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c109170 0x7f224c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f224b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f224c112e50 con 0x7f224c109170 2024-07-04T08:13:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f224b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f224c112cd0 con 0x7f224c108820 2024-07-04T08:13:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f224b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f224c112fd0 con 0x7f224c102ee0 2024-07-04T08:13:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f224a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c109170 0x7f224c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f224a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c109170 0x7f224c0756e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33462/0 (socket says 172.21.15.39:33462) 2024-07-04T08:13:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f224a800640 1 -- 172.21.15.39:0/2873884914 learned_addr learned my addr 172.21.15.39:0/2873884914 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f2241400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f224c108820 0x7f224c0751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.935+0000 7f2249e00640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f224c102ee0 0x7f224c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f224a800640 1 -- 172.21.15.39:0/2873884914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f224c102ee0 msgr2=0x7f224c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f224a800640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f224c102ee0 0x7f224c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f224a800640 1 -- 172.21.15.39:0/2873884914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f224c108820 msgr2=0x7f224c0751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f224a800640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f224c108820 0x7f224c0751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f224a800640 1 -- 172.21.15.39:0/2873884914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c1a7ba0 con 0x7f224c109170 2024-07-04T08:13:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f224a800640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c109170 0x7f224c0756e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f224400efd0 tx=0x7f224400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f2243400640 1 -- 172.21.15.39:0/2873884914 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244003b70 con 0x7f224c109170 2024-07-04T08:13:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f2243400640 1 -- 172.21.15.39:0/2873884914 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2244003d10 con 0x7f224c109170 2024-07-04T08:13:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c1a7e90 con 0x7f224c109170 2024-07-04T08:13:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.936+0000 7f2243400640 1 -- 172.21.15.39:0/2873884914 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244010810 con 0x7f224c109170 2024-07-04T08:13:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.937+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c1a8370 con 0x7f224c109170 2024-07-04T08:13:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.938+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c1032f0 con 0x7f224c109170 2024-07-04T08:13:14.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.940+0000 7f2243400640 1 -- 172.21.15.39:0/2873884914 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f2244004020 con 0x7f224c109170 2024-07-04T08:13:14.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.942+0000 7f2243400640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22300b2840 0x7f22300b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:14.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.942+0000 7f2243400640 1 -- 172.21.15.39:0/2873884914 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f22440e2bb0 con 0x7f224c109170 2024-07-04T08:13:14.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.942+0000 7f2249e00640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22300b2840 0x7f22300b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:14.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.943+0000 7f2249e00640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22300b2840 0x7f22300b4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f224c0766b0 tx=0x7f223c009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:14.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:14.946+0000 7f2243400640 1 -- 172.21.15.39:0/2873884914 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f22440aa420 con 0x7f224c109170 2024-07-04T08:13:15.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.202+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f224c02de10 con 0x7f224c109170 2024-07-04T08:13:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.203+0000 7f2243400640 1 -- 172.21.15.39:0/2873884914 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11975 (secure 0 0 0) 0x7f22440a9d20 con 0x7f224c109170 2024-07-04T08:13:15.205 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:13:15.205 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-07-04T08:11:09:862205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:11:09.668485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24673,24674,24679,24701,24707]},"id":1}]} 2024-07-04T08:13:15.205 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-07-04T08:13:15.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22300b2840 msgr2=0x7f22300b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22300b2840 0x7f22300b4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f224c0766b0 tx=0x7f223c009110 comp rx=0 tx=0).stop 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c109170 msgr2=0x7f224c0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c109170 0x7f224c0756e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f224400efd0 tx=0x7f224400c6a0 comp rx=0 tx=0).stop 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 shutdown_connections 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22300b2840 0x7f22300b4d50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f224c109170 0x7f224c0756e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f224c108820 0x7f224c0751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 --2- 172.21.15.39:0/2873884914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f224c102ee0 0x7f224c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.206+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 >> 172.21.15.39:0/2873884914 conn(0x7f224c0fe930 msgr2=0x7f224c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:15.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.207+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 shutdown_connections 2024-07-04T08:13:15.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:15.207+0000 7f224b200640 1 -- 172.21.15.39:0/2873884914 wait complete. 2024-07-04T08:13:15.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:15 smithi133 ceph-mon[37050]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:15.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:15 smithi039 ceph-mon[32062]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:15.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:15 smithi050 ceph-mon[36577]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:16.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:13:16.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:16 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2873884914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:13:16.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:16 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2873884914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:13:16.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:16 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2873884914' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:13:16.483 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:17.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:17 smithi039 ceph-mon[32062]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:17.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:17 smithi050 ceph-mon[36577]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:17.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:17 smithi133 ceph-mon[37050]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.720+0000 7f352d000640 1 -- 172.21.15.39:0/1087359508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352810f990 msgr2=0x7f35281161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.720+0000 7f352d000640 1 --2- 172.21.15.39:0/1087359508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352810f990 0x7f35281161c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3520009ca0 tx=0x7f352002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.720+0000 7f352d000640 1 -- 172.21.15.39:0/1087359508 shutdown_connections 2024-07-04T08:13:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.720+0000 7f352d000640 1 --2- 172.21.15.39:0/1087359508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352810f990 0x7f35281161c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.720+0000 7f352d000640 1 --2- 172.21.15.39:0/1087359508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f352810f040 0x7f352810f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.720+0000 7f352d000640 1 --2- 172.21.15.39:0/1087359508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3528109700 0x7f3528109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.720+0000 7f352d000640 1 -- 172.21.15.39:0/1087359508 >> 172.21.15.39:0/1087359508 conn(0x7f3528077630 msgr2=0x7f3528107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:18.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.720+0000 7f352d000640 1 -- 172.21.15.39:0/1087359508 shutdown_connections 2024-07-04T08:13:18.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.720+0000 7f352d000640 1 -- 172.21.15.39:0/1087359508 wait complete. 2024-07-04T08:13:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.722+0000 7f352d000640 1 Processor -- start 2024-07-04T08:13:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.722+0000 7f352d000640 1 -- start start 2024-07-04T08:13:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.722+0000 7f352d000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3528109700 0x7f352819ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.722+0000 7f352d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352810f040 0x7f35281a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.722+0000 7f352d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f352810f990 0x7f35281a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.722+0000 7f352d000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3528119660 con 0x7f352810f040 2024-07-04T08:13:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.722+0000 7f352d000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f35281194e0 con 0x7f352810f990 2024-07-04T08:13:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.722+0000 7f352d000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35281197e0 con 0x7f3528109700 2024-07-04T08:13:18.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3525e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3528109700 0x7f352819ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:18.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f351d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352810f040 0x7f35281a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:18.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3525e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3528109700 0x7f352819ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53416/0 (socket says 172.21.15.39:53416) 2024-07-04T08:13:18.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3525e00640 1 -- 172.21.15.39:0/3637811188 learned_addr learned my addr 172.21.15.39:0/3637811188 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:18.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3526800640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f352810f990 0x7f35281a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:18.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3525e00640 1 -- 172.21.15.39:0/3637811188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f352810f990 msgr2=0x7f35281a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3525e00640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f352810f990 0x7f35281a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3525e00640 1 -- 172.21.15.39:0/3637811188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352810f040 msgr2=0x7f35281a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3525e00640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352810f040 0x7f35281a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3525e00640 1 -- 172.21.15.39:0/3637811188 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35281a5f40 con 0x7f3528109700 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.723+0000 7f3525e00640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3528109700 0x7f352819ffd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f351800e9f0 tx=0x7f351800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.724+0000 7f351f400640 1 -- 172.21.15.39:0/3637811188 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351800cd80 con 0x7f3528109700 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.724+0000 7f351f400640 1 -- 172.21.15.39:0/3637811188 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35180108e0 con 0x7f3528109700 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.724+0000 7f351f400640 1 -- 172.21.15.39:0/3637811188 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518018980 con 0x7f3528109700 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.724+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35281a6230 con 0x7f3528109700 2024-07-04T08:13:18.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.724+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35281ac990 con 0x7f3528109700 2024-07-04T08:13:18.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.725+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3528109b10 con 0x7f3528109700 2024-07-04T08:13:18.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.728+0000 7f351f400640 1 -- 172.21.15.39:0/3637811188 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f3518010430 con 0x7f3528109700 2024-07-04T08:13:18.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.729+0000 7f351f400640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f350c0b2840 0x7f350c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:18.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.729+0000 7f351f400640 1 -- 172.21.15.39:0/3637811188 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f35180e2ec0 con 0x7f3528109700 2024-07-04T08:13:18.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.730+0000 7f351d400640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f350c0b2840 0x7f350c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:18.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.730+0000 7f351d400640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f350c0b2840 0x7f350c0b4d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f352806a590 tx=0x7f3510005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:18.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.731+0000 7f351f400640 1 -- 172.21.15.39:0/3637811188 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f35180aa0e0 con 0x7f3528109700 2024-07-04T08:13:18.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.985+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35281a1100 con 0x7f3528109700 2024-07-04T08:13:18.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.986+0000 7f351f400640 1 -- 172.21.15.39:0/3637811188 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11975 (secure 0 0 0) 0x7f35180ae3e0 con 0x7f3528109700 2024-07-04T08:13:18.988 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:13:18.988 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-07-04T08:11:09:862205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:11:09.668485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24673,24674,24679,24701,24707]},"id":1}]} 2024-07-04T08:13:18.988 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-07-04T08:13:18.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f350c0b2840 msgr2=0x7f350c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:18.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f350c0b2840 0x7f350c0b4d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f352806a590 tx=0x7f3510005f90 comp rx=0 tx=0).stop 2024-07-04T08:13:18.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3528109700 msgr2=0x7f352819ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:18.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3528109700 0x7f352819ffd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f351800e9f0 tx=0x7f351800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 shutdown_connections 2024-07-04T08:13:18.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f350c0b2840 0x7f350c0b4d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352810f040 0x7f35281a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f352810f990 0x7f35281a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 --2- 172.21.15.39:0/3637811188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3528109700 0x7f352819ffd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 >> 172.21.15.39:0/3637811188 conn(0x7f3528077630 msgr2=0x7f3528077cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 shutdown_connections 2024-07-04T08:13:18.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:18.990+0000 7f352d000640 1 -- 172.21.15.39:0/3637811188 wait complete. 2024-07-04T08:13:19.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:19 smithi039 ceph-mon[32062]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:19.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:19 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3637811188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:13:19.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:19 smithi050 ceph-mon[36577]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:19.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:19 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3637811188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:13:19.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:19 smithi133 ceph-mon[37050]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:19.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:19 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3637811188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:13:19.903 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:13:20.405 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:21.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:21 smithi039 ceph-mon[32062]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:21.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:21 smithi050 ceph-mon[36577]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:21.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:21 smithi133 ceph-mon[37050]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:22.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.658+0000 7fc101a00640 1 -- 172.21.15.39:0/2518987429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0fc108820 msgr2=0x7fc0fc108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:22.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.658+0000 7fc101a00640 1 --2- 172.21.15.39:0/2518987429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0fc108820 0x7fc0fc108c30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009900 tx=0x7fc0e802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.658+0000 7fc101a00640 1 -- 172.21.15.39:0/2518987429 shutdown_connections 2024-07-04T08:13:22.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.658+0000 7fc101a00640 1 --2- 172.21.15.39:0/2518987429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0fc108820 0x7fc0fc108c30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.658+0000 7fc101a00640 1 --2- 172.21.15.39:0/2518987429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc109170 0x7fc0fc10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.658+0000 7fc101a00640 1 --2- 172.21.15.39:0/2518987429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc102ee0 0x7fc0fc1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.658+0000 7fc101a00640 1 -- 172.21.15.39:0/2518987429 >> 172.21.15.39:0/2518987429 conn(0x7fc0fc0fe930 msgr2=0x7fc0fc100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:22.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.658+0000 7fc101a00640 1 -- 172.21.15.39:0/2518987429 shutdown_connections 2024-07-04T08:13:22.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.658+0000 7fc101a00640 1 -- 172.21.15.39:0/2518987429 wait complete. 2024-07-04T08:13:22.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.659+0000 7fc101a00640 1 Processor -- start 2024-07-04T08:13:22.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.660+0000 7fc101a00640 1 -- start start 2024-07-04T08:13:22.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.660+0000 7fc101a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc102ee0 0x7fc0fc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:22.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.660+0000 7fc101a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0fc108820 0x7fc0fc19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:22.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.660+0000 7fc101a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc109170 0x7fc0fc1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:22.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.660+0000 7fc101a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc112e30 con 0x7fc0fc108820 2024-07-04T08:13:22.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.660+0000 7fc101a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc112cb0 con 0x7fc0fc109170 2024-07-04T08:13:22.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.660+0000 7fc101a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc112fb0 con 0x7fc0fc102ee0 2024-07-04T08:13:22.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.660+0000 7fc0fb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0fc108820 0x7fc0fc19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:22.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.660+0000 7fc0fb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0fc108820 0x7fc0fc19c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:37984/0 (socket says 172.21.15.39:37984) 2024-07-04T08:13:22.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.661+0000 7fc0fb400640 1 -- 172.21.15.39:0/3763101939 learned_addr learned my addr 172.21.15.39:0/3763101939 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:22.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.661+0000 7fc0fbe00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc102ee0 0x7fc0fc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:22.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.661+0000 7fc101000640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc109170 0x7fc0fc1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.661+0000 7fc0fbe00640 1 -- 172.21.15.39:0/3763101939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc109170 msgr2=0x7fc0fc1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.661+0000 7fc0fbe00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc109170 0x7fc0fc1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.661+0000 7fc0fbe00640 1 -- 172.21.15.39:0/3763101939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0fc108820 msgr2=0x7fc0fc19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.661+0000 7fc0fbe00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0fc108820 0x7fc0fc19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.661+0000 7fc0fbe00640 1 -- 172.21.15.39:0/3763101939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc1a19d0 con 0x7fc0fc102ee0 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.662+0000 7fc0fbe00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc102ee0 0x7fc0fc19ba60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00e9f0 tx=0x7fc0ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.662+0000 7fc0f8c00640 1 -- 172.21.15.39:0/3763101939 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00cd80 con 0x7fc0fc102ee0 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.662+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0fc1a1cc0 con 0x7fc0fc102ee0 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.662+0000 7fc0f8c00640 1 -- 172.21.15.39:0/3763101939 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0ec0035f0 con 0x7fc0fc102ee0 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.662+0000 7fc0f8c00640 1 -- 172.21.15.39:0/3763101939 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec010690 con 0x7fc0fc102ee0 2024-07-04T08:13:22.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.662+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0fc1a8420 con 0x7fc0fc102ee0 2024-07-04T08:13:22.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.663+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0fc1032f0 con 0x7fc0fc102ee0 2024-07-04T08:13:22.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.666+0000 7fc0f8c00640 1 -- 172.21.15.39:0/3763101939 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fc0ec0037a0 con 0x7fc0fc102ee0 2024-07-04T08:13:22.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.667+0000 7fc0f8c00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e00b28c0 0x7fc0e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:22.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.668+0000 7fc0f8c00640 1 -- 172.21.15.39:0/3763101939 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc0ec0e2af0 con 0x7fc0fc102ee0 2024-07-04T08:13:22.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.668+0000 7fc0fb400640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e00b28c0 0x7fc0e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.669+0000 7fc0fb400640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e00b28c0 0x7fc0e00b4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc105030 tx=0x7fc0e8009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:22.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.669+0000 7fc0f8c00640 1 -- 172.21.15.39:0/3763101939 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc0ec055110 con 0x7fc0fc102ee0 2024-07-04T08:13:22.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.882+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0fc1a8800 con 0x7fc0fc102ee0 2024-07-04T08:13:22.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.884+0000 7fc0f8c00640 1 -- 172.21.15.39:0/3763101939 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24957 (secure 0 0 0) 0x7fc0ec0aa360 con 0x7fc0fc102ee0 2024-07-04T08:13:22.886 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:13:22.886 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:11:53.672125+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e00b28c0 msgr2=0x7fc0e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e00b28c0 0x7fc0e00b4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc105030 tx=0x7fc0e8009560 comp rx=0 tx=0).stop 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc102ee0 msgr2=0x7fc0fc19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc102ee0 0x7fc0fc19ba60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00e9f0 tx=0x7fc0ec00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 shutdown_connections 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e00b28c0 0x7fc0e00b4dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0fc108820 0x7fc0fc19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0fc109170 0x7fc0fc1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 --2- 172.21.15.39:0/3763101939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0fc102ee0 0x7fc0fc19ba60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.888+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 >> 172.21.15.39:0/3763101939 conn(0x7fc0fc0fe930 msgr2=0x7fc0fc107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:22.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.889+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 shutdown_connections 2024-07-04T08:13:22.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:22.889+0000 7fc101a00640 1 -- 172.21.15.39:0/3763101939 wait complete. 2024-07-04T08:13:23.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:23 smithi039 ceph-mon[32062]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:23.320 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:23 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3763101939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:13:23.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:23 smithi050 ceph-mon[36577]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:23.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:23 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3763101939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:13:23.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:23 smithi133 ceph-mon[37050]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:23.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:23 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3763101939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:13:23.748 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:13:24.249 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:25.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:25 smithi050 ceph-mon[36577]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:25.451 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:25 smithi039 ceph-mon[32062]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:25.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:25 smithi133 ceph-mon[37050]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:26.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.505+0000 7fb551200640 1 -- 172.21.15.39:0/315371622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c109190 msgr2=0x7fb54c10f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:26.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.505+0000 7fb551200640 1 --2- 172.21.15.39:0/315371622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c109190 0x7fb54c10f9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb544009ca0 tx=0x7fb54402a8b0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.505+0000 7fb551200640 1 -- 172.21.15.39:0/315371622 shutdown_connections 2024-07-04T08:13:26.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.505+0000 7fb551200640 1 --2- 172.21.15.39:0/315371622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c109190 0x7fb54c10f9c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.505+0000 7fb551200640 1 --2- 172.21.15.39:0/315371622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c108840 0x7fb54c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.505+0000 7fb551200640 1 --2- 172.21.15.39:0/315371622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb54c102f00 0x7fb54c103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.505+0000 7fb551200640 1 -- 172.21.15.39:0/315371622 >> 172.21.15.39:0/315371622 conn(0x7fb54c0fe950 msgr2=0x7fb54c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:26.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.506+0000 7fb551200640 1 -- 172.21.15.39:0/315371622 shutdown_connections 2024-07-04T08:13:26.508 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:26 smithi039 ceph-mon[32062]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:26.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.506+0000 7fb551200640 1 -- 172.21.15.39:0/315371622 wait complete. 2024-07-04T08:13:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.508+0000 7fb551200640 1 Processor -- start 2024-07-04T08:13:26.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.508+0000 7fb551200640 1 -- start start 2024-07-04T08:13:26.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb551200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c102f00 0x7fb54c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:26.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb551200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c108840 0x7fb54c0751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:26.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb551200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb54c109190 0x7fb54c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:26.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb551200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c112ea0 con 0x7fb54c102f00 2024-07-04T08:13:26.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb551200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c112d20 con 0x7fb54c109190 2024-07-04T08:13:26.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb551200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c113020 con 0x7fb54c108840 2024-07-04T08:13:26.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb54b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c102f00 0x7fb54c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:26.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb54b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c102f00 0x7fb54c074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38000/0 (socket says 172.21.15.39:38000) 2024-07-04T08:13:26.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb54b400640 1 -- 172.21.15.39:0/2688718956 learned_addr learned my addr 172.21.15.39:0/2688718956 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:26.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb54be00640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb54c109190 0x7fb54c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:26.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.509+0000 7fb54aa00640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c108840 0x7fb54c0751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb54be00640 1 -- 172.21.15.39:0/2688718956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c108840 msgr2=0x7fb54c0751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb54be00640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c108840 0x7fb54c0751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb54be00640 1 -- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c102f00 msgr2=0x7fb54c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb54be00640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c102f00 0x7fb54c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb54be00640 1 -- 172.21.15.39:0/2688718956 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c1a7a70 con 0x7fb54c109190 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb54b400640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c102f00 0x7fb54c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb54be00640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb54c109190 0x7fb54c0756e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb544005e30 tx=0x7fb544030dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb543e00640 1 -- 172.21.15.39:0/2688718956 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544003e00 con 0x7fb54c109190 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb543e00640 1 -- 172.21.15.39:0/2688718956 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb544040070 con 0x7fb54c109190 2024-07-04T08:13:26.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c1a7d00 con 0x7fb54c109190 2024-07-04T08:13:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.510+0000 7fb543e00640 1 -- 172.21.15.39:0/2688718956 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544045a20 con 0x7fb54c109190 2024-07-04T08:13:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.511+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c1a81c0 con 0x7fb54c109190 2024-07-04T08:13:26.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.512+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54c103310 con 0x7fb54c109190 2024-07-04T08:13:26.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.514+0000 7fb543e00640 1 -- 172.21.15.39:0/2688718956 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fb544045bc0 con 0x7fb54c109190 2024-07-04T08:13:26.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.516+0000 7fb543e00640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5300b2840 0x7fb5300b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.516+0000 7fb543e00640 1 -- 172.21.15.39:0/2688718956 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb544103860 con 0x7fb54c109190 2024-07-04T08:13:26.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.516+0000 7fb54b400640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5300b2840 0x7fb5300b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:26.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.517+0000 7fb54b400640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5300b2840 0x7fb5300b4d50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0766b0 tx=0x7fb53c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:26.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.518+0000 7fb543e00640 1 -- 172.21.15.39:0/2688718956 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb5440cb0d0 con 0x7fb54c109190 2024-07-04T08:13:26.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:26 smithi133 ceph-mon[37050]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:26.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:26 smithi050 ceph-mon[36577]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:26.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.773+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb54c075f60 con 0x7fb54c109190 2024-07-04T08:13:26.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.774+0000 7fb543e00640 1 -- 172.21.15.39:0/2688718956 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11975 (secure 0 0 0) 0x7fb5440dfe20 con 0x7fb54c109190 2024-07-04T08:13:26.776 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:13:26.776 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-07-04T08:11:09:862205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:11:09.668485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24673,24674,24679,24701,24707]},"id":1}]} 2024-07-04T08:13:26.776 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-07-04T08:13:26.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.777+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5300b2840 msgr2=0x7fb5300b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:26.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.777+0000 7fb551200640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5300b2840 0x7fb5300b4d50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0766b0 tx=0x7fb53c005f90 comp rx=0 tx=0).stop 2024-07-04T08:13:26.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.777+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb54c109190 msgr2=0x7fb54c0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:26.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.777+0000 7fb551200640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb54c109190 0x7fb54c0756e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb544005e30 tx=0x7fb544030dc0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.778+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 shutdown_connections 2024-07-04T08:13:26.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.778+0000 7fb551200640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5300b2840 0x7fb5300b4d50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.778+0000 7fb551200640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb54c109190 0x7fb54c0756e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.778+0000 7fb551200640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c108840 0x7fb54c0751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.778+0000 7fb551200640 1 --2- 172.21.15.39:0/2688718956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb54c102f00 0x7fb54c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:26.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.778+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 >> 172.21.15.39:0/2688718956 conn(0x7fb54c0fe950 msgr2=0x7fb54c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:26.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.778+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 shutdown_connections 2024-07-04T08:13:26.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:26.778+0000 7fb551200640 1 -- 172.21.15.39:0/2688718956 wait complete. 2024-07-04T08:13:27.524 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:27 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2688718956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:13:27.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:27 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2688718956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:13:27.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-04T08:09:41.252924+0000', 'modified': '2024-07-04T08:11:09.668485+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24707, 'mds_1': 24701, 'mds_2': 24679, 'mds_3': 24674, 'mds_4': 24673}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24619': {'gid': 24619, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6833/2411107381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6832', 'nonce': 2411107381}, {'type': 'v1', 'addr': '172.21.15.133:6833', 'nonce': 2411107381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24626': {'gid': 24626, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6833/264080488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6832', 'nonce': 264080488}, {'type': 'v1', 'addr': '172.21.15.50:6833', 'nonce': 264080488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6837/1240380045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 1240380045}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 1240380045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24656': {'gid': 24656, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6835/2710764279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 2710764279}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 2710764279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24661': {'gid': 24661, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/2187182943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2187182943}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 2187182943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24673': {'gid': 24673, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.133:6837/431595690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 431595690}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 431595690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.50:6837/4268659351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6836', 'nonce': 4268659351}, {'type': 'v1', 'addr': '172.21.15.50:6837', 'nonce': 4268659351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24679': {'gid': 24679, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.133:6839/1264801316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 1264801316}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 1264801316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24701': {'gid': 24701, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.39:6841/2753029904', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 2753029904}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 2753029904}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24707': {'gid': 24707, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.50:6839/2852157181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 2852157181}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 2852157181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24707, 'qdb_cluster': [24673, 24674, 24679, 24701, 24707]} 2024-07-04T08:13:27.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-04T08:13:27.598 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.c status 2024-07-04T08:13:27.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:27 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2688718956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:13:28.082 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:28.449 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:28 smithi039 ceph-mon[32062]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:28.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:28 smithi133 ceph-mon[37050]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:28.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:28 smithi050 ceph-mon[36577]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:30.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.423+0000 7fa997600640 1 -- 172.21.15.39:0/2458026437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa990076e90 msgr2=0x7fa9900772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:30.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.423+0000 7fa997600640 1 --2- 172.21.15.39:0/2458026437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa990076e90 0x7fa9900772a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa980009900 tx=0x7fa98002a410 comp rx=0 tx=0).stop 2024-07-04T08:13:30.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.423+0000 7fa997600640 1 -- 172.21.15.39:0/2458026437 shutdown_connections 2024-07-04T08:13:30.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.423+0000 7fa997600640 1 --2- 172.21.15.39:0/2458026437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa990113cc0 0x7fa9901160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.423+0000 7fa997600640 1 --2- 172.21.15.39:0/2458026437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9900777e0 0x7fa990113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.423+0000 7fa997600640 1 --2- 172.21.15.39:0/2458026437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa990076e90 0x7fa9900772a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.423+0000 7fa997600640 1 -- 172.21.15.39:0/2458026437 >> 172.21.15.39:0/2458026437 conn(0x7fa990100d10 msgr2=0x7fa990103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.423+0000 7fa997600640 1 -- 172.21.15.39:0/2458026437 shutdown_connections 2024-07-04T08:13:30.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.423+0000 7fa997600640 1 -- 172.21.15.39:0/2458026437 wait complete. 2024-07-04T08:13:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.425+0000 7fa997600640 1 Processor -- start 2024-07-04T08:13:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.425+0000 7fa997600640 1 -- start start 2024-07-04T08:13:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.425+0000 7fa997600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa990076e90 0x7fa9901a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.425+0000 7fa997600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900777e0 0x7fa9901a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.425+0000 7fa997600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa990113cc0 0x7fa9901a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.425+0000 7fa997600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901196a0 con 0x7fa990076e90 2024-07-04T08:13:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.425+0000 7fa997600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa990119520 con 0x7fa990113cc0 2024-07-04T08:13:30.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.425+0000 7fa997600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa990119820 con 0x7fa9900777e0 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa994c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa990076e90 0x7fa9901a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa994c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa990076e90 0x7fa9901a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38010/0 (socket says 172.21.15.39:38010) 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa994c00640 1 -- 172.21.15.39:0/1345352499 learned_addr learned my addr 172.21.15.39:0/1345352499 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa987e00640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900777e0 0x7fa9901a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa995600640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa990113cc0 0x7fa9901a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa987e00640 1 -- 172.21.15.39:0/1345352499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa990113cc0 msgr2=0x7fa9901a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa987e00640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa990113cc0 0x7fa9901a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa987e00640 1 -- 172.21.15.39:0/1345352499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa990076e90 msgr2=0x7fa9901a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa987e00640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa990076e90 0x7fa9901a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.426+0000 7fa987e00640 1 -- 172.21.15.39:0/1345352499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9901a5f80 con 0x7fa9900777e0 2024-07-04T08:13:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.427+0000 7fa987e00640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900777e0 0x7fa9901a05b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa99006bec0 tx=0x7fa97800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.427+0000 7fa98e000640 1 -- 172.21.15.39:0/1345352499 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97800cc40 con 0x7fa9900777e0 2024-07-04T08:13:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.427+0000 7fa98e000640 1 -- 172.21.15.39:0/1345352499 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa97800cde0 con 0x7fa9900777e0 2024-07-04T08:13:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.427+0000 7fa997600640 1 -- 172.21.15.39:0/1345352499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9901a6270 con 0x7fa9900777e0 2024-07-04T08:13:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.427+0000 7fa98e000640 1 -- 172.21.15.39:0/1345352499 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978018780 con 0x7fa9900777e0 2024-07-04T08:13:30.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.427+0000 7fa997600640 1 -- 172.21.15.39:0/1345352499 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9901ac9d0 con 0x7fa9900777e0 2024-07-04T08:13:30.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.432+0000 7fa99dc7b740 1 Processor -- start 2024-07-04T08:13:30.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.432+0000 7fa99dc7b740 1 -- start start 2024-07-04T08:13:30.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f50650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f50b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b696ec6c80 con 0x55b696babf60 2024-07-04T08:13:30.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b696eb6220 con 0x55b696b8b280 2024-07-04T08:13:30.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b696eba260 con 0x55b696b830a0 2024-07-04T08:13:30.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa994c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f50b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa994c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f50b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34074/0 (socket says 172.21.15.39:34074) 2024-07-04T08:13:30.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa994c00640 1 -- 172.21.15.39:0/1056292716 learned_addr learned my addr 172.21.15.39:0/1056292716 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:30.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa995600640 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f50650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.433+0000 7fa98e000640 1 -- 172.21.15.39:0/1345352499 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fa978010a90 con 0x7fa9900777e0 2024-07-04T08:13:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.434+0000 7fa987e00640 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.434+0000 7fa994c00640 1 -- 172.21.15.39:0/1056292716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 msgr2=0x55b696f510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.434+0000 7fa994c00640 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.434+0000 7fa994c00640 1 -- 172.21.15.39:0/1056292716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 msgr2=0x55b696f50650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.434+0000 7fa994c00640 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f50650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.434+0000 7fa994c00640 1 -- 172.21.15.39:0/1056292716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b696e4e530 con 0x55b696b830a0 2024-07-04T08:13:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.434+0000 7fa987e00640 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:13:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.435+0000 7fa994c00640 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f50b90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa980006010 tx=0x7fa980004100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e327974eec016a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:30.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.435+0000 7fa986000640 1 -- 172.21.15.39:0/1056292716 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980040070 con 0x55b696b830a0 2024-07-04T08:13:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.435+0000 7fa986000640 1 -- 172.21.15.39:0/1056292716 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9800054d0 con 0x55b696b830a0 2024-07-04T08:13:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.435+0000 7fa986000640 1 -- 172.21.15.39:0/1056292716 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980035040 con 0x55b696b830a0 2024-07-04T08:13:30.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa98e000640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9740b2940 0x7fa9740b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1056292716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 msgr2=0x55b696f50b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f50b90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa980006010 tx=0x7fa980004100 comp rx=0 tx=0).stop 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa98e000640 1 -- 172.21.15.39:0/1345352499 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa9780e2960 con 0x7fa9900777e0 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1056292716 shutdown_connections 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f50b90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1056292716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f50650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1056292716 >> 172.21.15.39:0/1056292716 conn(0x55b696ba0350 msgr2=0x55b696f4e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa995600640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9740b2940 0x7fa9740b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1056292716 shutdown_connections 2024-07-04T08:13:30.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.436+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1056292716 wait complete. 2024-07-04T08:13:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.437+0000 7fa995600640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9740b2940 0x7fa9740b4e50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa988005fd0 tx=0x7fa988009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:30.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.437+0000 7fa99dc7b740 1 -- start start 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.437+0000 7fa99dc7b740 1 Processor -- start 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.438+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b830a0 0x55b696f5c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.438+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b8b280 0x55b696f5c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.438+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f68120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.438+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b696d1a5f0 con 0x55b696babf60 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.438+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b696e5d950 con 0x55b696b830a0 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.438+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b696f5edd0 con 0x55b696b8b280 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.438+0000 7fa99dc7b740 10 client.0 _ll_register_callbacks cb 0x55b696eaf480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.438+0000 7fa995600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f68120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa995600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f68120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38022/0 (socket says 172.21.15.39:38022) 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa995600640 1 -- 172.21.15.39:0/1538048146 learned_addr learned my addr 172.21.15.39:0/1538048146 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:30.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa987e00640 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b8b280 0x55b696f5c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa994c00640 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b830a0 0x55b696f5c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa994c00640 1 -- 172.21.15.39:0/1538048146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b8b280 msgr2=0x55b696f5c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa994c00640 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b8b280 0x55b696f5c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa994c00640 1 -- 172.21.15.39:0/1538048146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 msgr2=0x55b696f68120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa994c00640 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f68120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa994c00640 1 -- 172.21.15.39:0/1538048146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b696c17540 con 0x55b696b830a0 2024-07-04T08:13:30.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa994c00640 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b830a0 0x55b696f5c240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa980005ab0 tx=0x7fa980009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6353a17e01d1d68d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.439+0000 7fa97ea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.440+0000 7fa97ea00640 1 -- 172.21.15.39:0/1538048146 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980040070 con 0x55b696b830a0 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.440+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1538048146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9800054d0 con 0x55b696b830a0 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.440+0000 7fa97ea00640 1 -- 172.21.15.39:0/1538048146 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa980004fb0 con 0x55b696b830a0 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.440+0000 7fa97ea00640 1 -- 172.21.15.39:0/1538048146 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa980035040 con 0x55b696b830a0 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.440+0000 7fa99dc7b740 10 client.25151 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.440+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1538048146 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b696f6ad80 con 0x55b696b830a0 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.441+0000 7fa97ea00640 1 -- 172.21.15.39:0/1538048146 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9800054d0 con 0x55b696b830a0 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.441+0000 7fa97ea00640 1 -- 172.21.15.39:0/1538048146 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa9800314a0 con 0x55b696b830a0 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.441+0000 7fa99dc7b740 10 client.25151 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.441+0000 7fa99dc7b740 10 client.25151 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.441+0000 7fa99dc7b740 20 client.25151 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:30.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.441+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55b696b6a8e0 0x55b696f6e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.441+0000 7fa99dc7b740 4 client.25151 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:13:30.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.441+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1538048146 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b696ec8bd0 con 0x55b696b6a8e0 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.442+0000 7fa987e00640 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55b696b6a8e0 0x55b696f6e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.442+0000 7fa987e00640 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55b696b6a8e0 0x55b696f6e380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5e262ababee7723a server_cookie=78171c6d8a3c0394 in_seq=0 out_seq=0 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.442+0000 7fa97ea00640 10 client.25151 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.443+0000 7fa97ea00640 1 -- 172.21.15.39:0/1538048146 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b696ec8bd0 con 0x55b696b6a8e0 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.443+0000 7fa97ea00640 10 client.25151 handle_command_reply: tid=0 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.443+0000 7fa99dc7b740 1 client.25151 shutdown 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.443+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1538048146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b830a0 msgr2=0x55b696f5c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.443+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b830a0 0x55b696f5c240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa980005ab0 tx=0x7fa980009260 comp rx=0 tx=0).stop 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.443+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1538048146 shutdown_connections 2024-07-04T08:13:30.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.443+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55b696b6a8e0 0x55b696f6e380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.444+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f68120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.444+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b8b280 0x55b696f5c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.444+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1538048146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b830a0 0x55b696f5c240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.444+0000 7fa97ea00640 0 client.25151 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:13:30.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.444+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1538048146 >> 172.21.15.39:0/1538048146 conn(0x55b696ba0350 msgr2=0x55b696f53830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:30.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.444+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1538048146 shutdown_connections 2024-07-04T08:13:30.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.444+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1538048146 wait complete. 2024-07-04T08:13:30.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.444+0000 7fa99dc7b740 20 client.25151 trim_cache size 0 max 16384 2024-07-04T08:13:30.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.465+0000 7fa99dc7b740 1 Processor -- start 2024-07-04T08:13:30.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.465+0000 7fa99dc7b740 1 -- start start 2024-07-04T08:13:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.465+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696b830a0 0x55b696f68090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.465+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f571f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.465+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696babf60 0x55b696f4e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.465+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b696f6c6c0 con 0x55b696b830a0 2024-07-04T08:13:30.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.465+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b696f6cd90 con 0x55b696b8b280 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.465+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b696f6d460 con 0x55b696babf60 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa994c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696b830a0 0x55b696f68090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa994c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696b830a0 0x55b696f68090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38034/0 (socket says 172.21.15.39:38034) 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa994c00640 1 -- 172.21.15.39:0/292899414 learned_addr learned my addr 172.21.15.39:0/292899414 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa987e00640 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa995600640 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696babf60 0x55b696f4e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa994c00640 1 -- 172.21.15.39:0/292899414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696babf60 msgr2=0x55b696f4e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa994c00640 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696babf60 0x55b696f4e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa994c00640 1 -- 172.21.15.39:0/292899414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 msgr2=0x55b696f571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa994c00640 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa994c00640 1 -- 172.21.15.39:0/292899414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b696f5d300 con 0x55b696b830a0 2024-07-04T08:13:30.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.466+0000 7fa987e00640 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:13:30.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.467+0000 7fa994c00640 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696b830a0 0x55b696f68090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa980000c00 tx=0x7fa980005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4082ddc99d7ccaf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:30.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.467+0000 7fa985600640 1 -- 172.21.15.39:0/292899414 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980040070 con 0x55b696b830a0 2024-07-04T08:13:30.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.467+0000 7fa985600640 1 -- 172.21.15.39:0/292899414 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9800049b0 con 0x55b696b830a0 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.467+0000 7fa985600640 1 -- 172.21.15.39:0/292899414 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9800053e0 con 0x55b696b830a0 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.468+0000 7fa99dc7b740 1 -- 172.21.15.39:0/292899414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696b830a0 msgr2=0x55b696f68090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.468+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696b830a0 0x55b696f68090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa980000c00 tx=0x7fa980005a20 comp rx=0 tx=0).stop 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.468+0000 7fa99dc7b740 1 -- 172.21.15.39:0/292899414 shutdown_connections 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.468+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696babf60 0x55b696f4e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.468+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.468+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/292899414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696b830a0 0x55b696f68090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.468+0000 7fa99dc7b740 1 -- 172.21.15.39:0/292899414 >> 172.21.15.39:0/292899414 conn(0x55b696ba0350 msgr2=0x55b696f4ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.468+0000 7fa99dc7b740 1 -- 172.21.15.39:0/292899414 shutdown_connections 2024-07-04T08:13:30.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.468+0000 7fa99dc7b740 1 -- 172.21.15.39:0/292899414 wait complete. 2024-07-04T08:13:30.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.469+0000 7fa99dc7b740 1 -- start start 2024-07-04T08:13:30.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.469+0000 7fa99dc7b740 1 Processor -- start 2024-07-04T08:13:30.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f76aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f67370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa99dc7b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f678b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b696f6e200 con 0x55b696babf60 2024-07-04T08:13:30.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b696f6db30 con 0x55b696b8b280 2024-07-04T08:13:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa99dc7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b696f6e8d0 con 0x55b696b830a0 2024-07-04T08:13:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa99dc7b740 10 client.0 _ll_register_callbacks cb 0x7fa988009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa995600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa994c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f76aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa987e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f67370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa995600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f678b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38040/0 (socket says 172.21.15.39:38040) 2024-07-04T08:13:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.470+0000 7fa995600640 1 -- 172.21.15.39:0/1138495342 learned_addr learned my addr 172.21.15.39:0/1138495342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.471+0000 7fa994c00640 1 -- 172.21.15.39:0/1138495342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 msgr2=0x55b696f67370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.471+0000 7fa994c00640 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f67370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.471+0000 7fa994c00640 1 -- 172.21.15.39:0/1138495342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 msgr2=0x55b696f678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.471+0000 7fa994c00640 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f678b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.471+0000 7fa994c00640 1 -- 172.21.15.39:0/1138495342 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b696f67df0 con 0x55b696b830a0 2024-07-04T08:13:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.472+0000 7fa994c00640 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f76aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa980031440 tx=0x7fa9800095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ade7eb7d728e0a5a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:30.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.472+0000 7fa97ea00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:13:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.473+0000 7fa97ea00640 1 -- 172.21.15.39:0/1138495342 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980069060 con 0x55b696b830a0 2024-07-04T08:13:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.473+0000 7fa97ea00640 1 -- 172.21.15.39:0/1138495342 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa980005130 con 0x55b696b830a0 2024-07-04T08:13:30.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.473+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1138495342 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9800049b0 con 0x55b696b830a0 2024-07-04T08:13:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.473+0000 7fa97ea00640 1 -- 172.21.15.39:0/1138495342 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa980070050 con 0x55b696b830a0 2024-07-04T08:13:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.473+0000 7fa99dc7b740 10 client.25150 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.473+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1138495342 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b696fd7620 con 0x55b696b830a0 2024-07-04T08:13:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.474+0000 7fa97ea00640 1 -- 172.21.15.39:0/1138495342 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9800049b0 con 0x55b696b830a0 2024-07-04T08:13:30.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.474+0000 7fa97ea00640 1 -- 172.21.15.39:0/1138495342 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa980003c20 con 0x55b696b830a0 2024-07-04T08:13:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.474+0000 7fa99dc7b740 10 client.25150 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.474+0000 7fa99dc7b740 10 client.25150 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T08:13:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.474+0000 7fa99dc7b740 20 client.25150 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.474+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55b696b3fc90 0x55b696fdafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.474+0000 7fa99dc7b740 4 client.25150 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:13:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.474+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1138495342 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "status"}) -- 0x55b696ec4670 con 0x55b696b3fc90 2024-07-04T08:13:30.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.475+0000 7fa987e00640 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55b696b3fc90 0x55b696fdafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:30.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.475+0000 7fa987e00640 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55b696b3fc90 0x55b696fdafe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42c5ba1dfb6671c3 server_cookie=81800375f0c870d7 in_seq=0 out_seq=0 2024-07-04T08:13:30.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.475+0000 7fa97ea00640 10 client.25150 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:13:30.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa97ea00640 1 -- 172.21.15.39:0/1138495342 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b696ec4670 con 0x55b696b3fc90 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa97ea00640 10 client.25150 handle_command_reply: tid=0 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 client.25150 shutdown 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1138495342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 msgr2=0x55b696f76aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f76aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa980031440 tx=0x7fa9800095e0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1138495342 shutdown_connections 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55b696b3fc90 0x55b696fdafe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b696babf60 0x55b696f678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b696b8b280 0x55b696f67370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 --2- 172.21.15.39:0/1138495342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b696b830a0 0x55b696f76aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa97ea00640 0 client.25150 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1138495342 >> 172.21.15.39:0/1138495342 conn(0x55b696ba0350 msgr2=0x55b696f53750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:30.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1138495342 shutdown_connections 2024-07-04T08:13:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.476+0000 7fa99dc7b740 1 -- 172.21.15.39:0/1138495342 wait complete. 2024-07-04T08:13:30.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa99dc7b740 20 client.25150 trim_cache size 0 max 16384 2024-07-04T08:13:30.479 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:13:30.479 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:13:30.479 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-07-04T08:13:30.479 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24673, 2024-07-04T08:13:30.479 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:13:30.479 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:13:30.480 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:13:30.480 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 145.628629762, 2024-07-04T08:13:30.480 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:13:30.480 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:13:30.480 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:13:30.480 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 265.70761492999998 2024-07-04T08:13:30.480 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:13:30.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 -- 172.21.15.39:0/1345352499 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9740b2940 msgr2=0x7fa9740b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9740b2940 0x7fa9740b4e50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa988005fd0 tx=0x7fa988009500 comp rx=0 tx=0).stop 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 -- 172.21.15.39:0/1345352499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900777e0 msgr2=0x7fa9901a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900777e0 0x7fa9901a05b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa99006bec0 tx=0x7fa97800ee50 comp rx=0 tx=0).stop 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 -- 172.21.15.39:0/1345352499 shutdown_connections 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9740b2940 0x7fa9740b4e50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa990113cc0 0x7fa9901a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9900777e0 0x7fa9901a05b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 --2- 172.21.15.39:0/1345352499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa990076e90 0x7fa9901a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.477+0000 7fa997600640 1 -- 172.21.15.39:0/1345352499 >> 172.21.15.39:0/1345352499 conn(0x7fa990100d10 msgr2=0x7fa990103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.478+0000 7fa997600640 1 -- 172.21.15.39:0/1345352499 shutdown_connections 2024-07-04T08:13:30.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:30.478+0000 7fa997600640 1 -- 172.21.15.39:0/1345352499 wait complete. 2024-07-04T08:13:30.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:30 smithi039 ceph-mon[32062]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:31.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:30 smithi133 ceph-mon[37050]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:31.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:30 smithi050 ceph-mon[36577]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:31.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.h status 2024-07-04T08:13:31.930 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:33.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:32 smithi133 ceph-mon[37050]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:33.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:32 smithi039 ceph-mon[32062]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:33.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:32 smithi050 ceph-mon[36577]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:34.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.203+0000 7fdd7f000640 1 -- 172.21.15.39:0/2632063678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd7810b3a0 msgr2=0x7fdd78111bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:34.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.203+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2632063678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd7810b3a0 0x7fdd78111bd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009ca0 tx=0x7fdd7002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.203+0000 7fdd7f000640 1 -- 172.21.15.39:0/2632063678 shutdown_connections 2024-07-04T08:13:34.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.203+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2632063678 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd7810b3a0 0x7fdd78111bd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.203+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2632063678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd7810aa50 0x7fdd7810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.203+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2632063678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78105110 0x7fdd78105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.203+0000 7fdd7f000640 1 -- 172.21.15.39:0/2632063678 >> 172.21.15.39:0/2632063678 conn(0x7fdd78100b80 msgr2=0x7fdd78102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:34.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.203+0000 7fdd7f000640 1 -- 172.21.15.39:0/2632063678 shutdown_connections 2024-07-04T08:13:34.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.203+0000 7fdd7f000640 1 -- 172.21.15.39:0/2632063678 wait complete. 2024-07-04T08:13:34.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.205+0000 7fdd7f000640 1 Processor -- start 2024-07-04T08:13:34.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.205+0000 7fdd7f000640 1 -- start start 2024-07-04T08:13:34.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7f000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78105110 0x7fdd7819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7f000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd7810aa50 0x7fdd7819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7f000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd7810b3a0 0x7fdd781a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7f000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781150c0 con 0x7fdd7810b3a0 2024-07-04T08:13:34.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7f000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78114f40 con 0x7fdd78105110 2024-07-04T08:13:34.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7f000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78115240 con 0x7fdd7810aa50 2024-07-04T08:13:34.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78105110 0x7fdd7819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd7810b3a0 0x7fdd781a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78105110 0x7fdd7819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:40238/0 (socket says 172.21.15.39:40238) 2024-07-04T08:13:34.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd7810b3a0 0x7fdd781a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36616/0 (socket says 172.21.15.39:36616) 2024-07-04T08:13:34.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.206+0000 7fdd7dc00640 1 -- 172.21.15.39:0/2451085535 learned_addr learned my addr 172.21.15.39:0/2451085535 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:34.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd7d200640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd7810aa50 0x7fdd7819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd7dc00640 1 -- 172.21.15.39:0/2451085535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd7810aa50 msgr2=0x7fdd7819bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:34.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd7dc00640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd7810aa50 0x7fdd7819bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd7dc00640 1 -- 172.21.15.39:0/2451085535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd7810b3a0 msgr2=0x7fdd781a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:34.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd7dc00640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd7810b3a0 0x7fdd781a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd7dc00640 1 -- 172.21.15.39:0/2451085535 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd781a1980 con 0x7fdd78105110 2024-07-04T08:13:34.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd7dc00640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78105110 0x7fdd7819ba10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00d950 tx=0x7fdd6c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:34.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd76000640 1 -- 172.21.15.39:0/2451085535 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c017030 con 0x7fdd78105110 2024-07-04T08:13:34.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd76000640 1 -- 172.21.15.39:0/2451085535 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6c004020 con 0x7fdd78105110 2024-07-04T08:13:34.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.207+0000 7fdd7f000640 1 -- 172.21.15.39:0/2451085535 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd781a1c70 con 0x7fdd78105110 2024-07-04T08:13:34.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.208+0000 7fdd76000640 1 -- 172.21.15.39:0/2451085535 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0047a0 con 0x7fdd78105110 2024-07-04T08:13:34.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.208+0000 7fdd7f000640 1 -- 172.21.15.39:0/2451085535 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781a83d0 con 0x7fdd78105110 2024-07-04T08:13:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.211+0000 7fdd86c0e740 1 Processor -- start 2024-07-04T08:13:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.211+0000 7fdd86c0e740 1 -- start start 2024-07-04T08:13:34.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.211+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710bc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.211+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.211+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710bd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.211+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564071032c80 con 0x564070cf7280 2024-07-04T08:13:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.211+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564071022220 con 0x564070d17f60 2024-07-04T08:13:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.211+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564071026260 con 0x564070cef0a0 2024-07-04T08:13:34.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd76000640 1 -- 172.21.15.39:0/2451085535 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fdd6c009a30 con 0x7fdd78105110 2024-07-04T08:13:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710bd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710bc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710bd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36622/0 (socket says 172.21.15.39:36622) 2024-07-04T08:13:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7d200640 1 -- 172.21.15.39:0/2569235243 learned_addr learned my addr 172.21.15.39:0/2569235243 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7e600640 1 -- 172.21.15.39:0/2569235243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 msgr2=0x5640710bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:34.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7dc00640 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7e600640 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710bcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7e600640 1 -- 172.21.15.39:0/2569235243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 msgr2=0x5640710bd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7e600640 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710bd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7e600640 1 -- 172.21.15.39:0/2569235243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564070fba530 con 0x564070d17f60 2024-07-04T08:13:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7dc00640 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:13:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.212+0000 7fdd7d200640 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710bd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:13:34.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.213+0000 7fdd7e600640 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710bc830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005e30 tx=0x7fdd7002f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f823667fd7cbd19 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:34.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.213+0000 7fdd66a00640 1 -- 172.21.15.39:0/2569235243 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70031e30 con 0x564070d17f60 2024-07-04T08:13:34.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.214+0000 7fdd66a00640 1 -- 172.21.15.39:0/2569235243 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd70036030 con 0x564070d17f60 2024-07-04T08:13:34.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.214+0000 7fdd66a00640 1 -- 172.21.15.39:0/2569235243 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70004330 con 0x564070d17f60 2024-07-04T08:13:34.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.214+0000 7fdd76000640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd600b2840 0x7fdd600b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.214+0000 7fdd76000640 1 -- 172.21.15.39:0/2451085535 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdd6c0e21f0 con 0x7fdd78105110 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.214+0000 7fdd7dc00640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd600b2840 0x7fdd600b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2569235243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 msgr2=0x5640710bc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710bc830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005e30 tx=0x7fdd7002f3c0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2569235243 shutdown_connections 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710bd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2569235243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710bc830 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2569235243 >> 172.21.15.39:0/2569235243 conn(0x564070d0c350 msgr2=0x5640710ba720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2569235243 shutdown_connections 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2569235243 wait complete. 2024-07-04T08:13:34.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.215+0000 7fdd7dc00640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd600b2840 0x7fdd600b4d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0e2490 tx=0x7fdd6c000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:34.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.216+0000 7fdd86c0e740 1 -- start start 2024-07-04T08:13:34.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.216+0000 7fdd86c0e740 1 Processor -- start 2024-07-04T08:13:34.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.216+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.216+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710d4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564070e865f0 con 0x564070cf7280 2024-07-04T08:13:34.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564070fc9950 con 0x564070d17f60 2024-07-04T08:13:34.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5640710cae30 con 0x564070cef0a0 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd86c0e740 10 client.0 _ll_register_callbacks cb 0x56407101b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710d4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36630/0 (socket says 172.21.15.39:36630) 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710d4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:40250/0 (socket says 172.21.15.39:40250) 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7d200640 1 -- 172.21.15.39:0/2727714294 learned_addr learned my addr 172.21.15.39:0/2727714294 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7e600640 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7d200640 1 -- 172.21.15.39:0/2727714294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 msgr2=0x5640710c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7d200640 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7d200640 1 -- 172.21.15.39:0/2727714294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 msgr2=0x5640710d4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7d200640 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710d4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.217+0000 7fdd7d200640 1 -- 172.21.15.39:0/2727714294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564070d83540 con 0x564070cf7280 2024-07-04T08:13:34.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.218+0000 7fdd7e600640 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710d4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:13:34.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.218+0000 7fdd7d200640 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710c89f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800bf40 tx=0x7fdd680128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2ef0dd14d36887 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:34.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.218+0000 7fdd5b400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:13:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.218+0000 7fdd5b400640 1 -- 172.21.15.39:0/2727714294 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6801f060 con 0x564070cf7280 2024-07-04T08:13:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.218+0000 7fdd5b400640 1 -- 172.21.15.39:0/2727714294 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdd680108a0 con 0x564070cf7280 2024-07-04T08:13:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.218+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2727714294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd70036030 con 0x564070cf7280 2024-07-04T08:13:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.219+0000 7fdd5b400640 1 -- 172.21.15.39:0/2727714294 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd68026050 con 0x564070cf7280 2024-07-04T08:13:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.219+0000 7fdd86c0e740 10 client.15045 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.219+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2727714294 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640710d6ee0 con 0x564070cf7280 2024-07-04T08:13:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.219+0000 7fdd5b400640 1 -- 172.21.15.39:0/2727714294 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd70036030 con 0x564070cf7280 2024-07-04T08:13:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.219+0000 7fdd5b400640 1 -- 172.21.15.39:0/2727714294 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdd68019030 con 0x564070cf7280 2024-07-04T08:13:34.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.219+0000 7fdd86c0e740 10 client.15045 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:34.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.220+0000 7fdd86c0e740 10 client.15045 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-04T08:13:34.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.220+0000 7fdd86c0e740 20 client.15045 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:34.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.220+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x564070cd68e0 0x5640710da960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.220+0000 7fdd86c0e740 4 client.15045 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:13:34.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.220+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2727714294 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564071034bd0 con 0x564070cd68e0 2024-07-04T08:13:34.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.220+0000 7fdd7e600640 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x564070cd68e0 0x5640710da960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.220+0000 7fdd7e600640 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x564070cd68e0 0x5640710da960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a1efe19f8fce89e7 server_cookie=1a45fe7ae974dd28 in_seq=0 out_seq=0 2024-07-04T08:13:34.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.220+0000 7fdd5b400640 10 client.15045 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:13:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.221+0000 7fdd5b400640 1 -- 172.21.15.39:0/2727714294 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564071034bd0 con 0x564070cd68e0 2024-07-04T08:13:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.221+0000 7fdd5b400640 10 client.15045 handle_command_reply: tid=0 2024-07-04T08:13:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.221+0000 7fdd86c0e740 1 client.15045 shutdown 2024-07-04T08:13:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2727714294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 msgr2=0x5640710c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:34.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710c89f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800bf40 tx=0x7fdd680128e0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2727714294 shutdown_connections 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x564070cd68e0 0x5640710da960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710c89f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070d17f60 0x5640710d4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/2727714294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2727714294 >> 172.21.15.39:0/2727714294 conn(0x564070d0c350 msgr2=0x5640710bfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd5b400640 0 client.15045 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2727714294 shutdown_connections 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 1 -- 172.21.15.39:0/2727714294 wait complete. 2024-07-04T08:13:34.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.222+0000 7fdd86c0e740 20 client.15045 trim_cache size 0 max 16384 2024-07-04T08:13:34.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.243+0000 7fdd86c0e740 1 Processor -- start 2024-07-04T08:13:34.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.243+0000 7fdd86c0e740 1 -- start start 2024-07-04T08:13:34.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cf7280 0x5640710d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070d17f60 0x5640710c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5640710d8ca0 con 0x564070d17f60 2024-07-04T08:13:34.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5640710d9370 con 0x564070cf7280 2024-07-04T08:13:34.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5640710d9a40 con 0x564070cef0a0 2024-07-04T08:13:34.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cf7280 0x5640710d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cf7280 0x5640710d4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:40254/0 (socket says 172.21.15.39:40254) 2024-07-04T08:13:34.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd7d200640 1 -- 172.21.15.39:0/1898767333 learned_addr learned my addr 172.21.15.39:0/1898767333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:34.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.244+0000 7fdd7e600640 1 --2- 172.21.15.39:0/1898767333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070d17f60 0x5640710c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.245+0000 7fdd7d200640 1 -- 172.21.15.39:0/1898767333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 msgr2=0x5640710b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:34.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.245+0000 7fdd7d200640 1 --2- 172.21.15.39:0/1898767333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.245+0000 7fdd7d200640 1 -- 172.21.15.39:0/1898767333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070d17f60 msgr2=0x5640710c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:34.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.245+0000 7fdd7d200640 1 --2- 172.21.15.39:0/1898767333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070d17f60 0x5640710c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.245+0000 7fdd7d200640 1 -- 172.21.15.39:0/1898767333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640710beb20 con 0x564070cf7280 2024-07-04T08:13:34.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.245+0000 7fdd7e600640 1 --2- 172.21.15.39:0/1898767333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070d17f60 0x5640710c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:13:34.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.245+0000 7fdd7d200640 1 --2- 172.21.15.39:0/1898767333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cf7280 0x5640710d4090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd68005bc0 tx=0x7fdd68013f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce1e986b3d16b8e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:34.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.246+0000 7fdd66000640 1 -- 172.21.15.39:0/1898767333 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68005230 con 0x564070cf7280 2024-07-04T08:13:34.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.246+0000 7fdd66000640 1 -- 172.21.15.39:0/1898767333 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6802f070 con 0x564070cf7280 2024-07-04T08:13:34.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.246+0000 7fdd66000640 1 -- 172.21.15.39:0/1898767333 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68010990 con 0x564070cf7280 2024-07-04T08:13:34.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.247+0000 7fdd86c0e740 1 -- 172.21.15.39:0/1898767333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cf7280 msgr2=0x5640710d4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:34.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.247+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/1898767333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cf7280 0x5640710d4090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdd68005bc0 tx=0x7fdd68013f80 comp rx=0 tx=0).stop 2024-07-04T08:13:34.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.247+0000 7fdd86c0e740 1 -- 172.21.15.39:0/1898767333 shutdown_connections 2024-07-04T08:13:34.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.247+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/1898767333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070d17f60 0x5640710c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.247+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/1898767333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cf7280 0x5640710d4090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.247+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/1898767333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070cef0a0 0x5640710b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.247+0000 7fdd86c0e740 1 -- 172.21.15.39:0/1898767333 >> 172.21.15.39:0/1898767333 conn(0x564070d0c350 msgr2=0x5640710c3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:34.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.247+0000 7fdd86c0e740 1 -- 172.21.15.39:0/1898767333 shutdown_connections 2024-07-04T08:13:34.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.247+0000 7fdd86c0e740 1 -- 172.21.15.39:0/1898767333 wait complete. 2024-07-04T08:13:34.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.248+0000 7fdd86c0e740 1 -- start start 2024-07-04T08:13:34.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.248+0000 7fdd86c0e740 1 Processor -- start 2024-07-04T08:13:34.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cef0a0 0x5640710dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710dd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd86c0e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070d17f60 0x5640710e13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5640710da7e0 con 0x564070cf7280 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5640710da110 con 0x564070cef0a0 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd86c0e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5640710daeb0 con 0x564070d17f60 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd86c0e740 10 client.0 _ll_register_callbacks cb 0x7fdd6800b7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710dd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cef0a0 0x5640710dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd7e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cef0a0 0x5640710dd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:40264/0 (socket says 172.21.15.39:40264) 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd7e600640 1 -- 172.21.15.39:0/3965101868 learned_addr learned my addr 172.21.15.39:0/3965101868 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd7d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710dd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36656/0 (socket says 172.21.15.39:36656) 2024-07-04T08:13:34.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.249+0000 7fdd7e600640 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070d17f60 0x5640710e13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.250+0000 7fdd7d200640 1 -- 172.21.15.39:0/3965101868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070d17f60 msgr2=0x5640710e13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.250+0000 7fdd7d200640 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070d17f60 0x5640710e13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.250+0000 7fdd7d200640 1 -- 172.21.15.39:0/3965101868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cef0a0 msgr2=0x5640710dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.250+0000 7fdd7d200640 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cef0a0 0x5640710dd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.250+0000 7fdd7d200640 1 -- 172.21.15.39:0/3965101868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640710e1a30 con 0x564070cf7280 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.250+0000 7fdd7e600640 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cef0a0 0x5640710dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.250+0000 7fdd7d200640 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710dd910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800bf40 tx=0x7fdd6801a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b672b9657381f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.250+0000 7fdd5b400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.251+0000 7fdd5b400640 1 -- 172.21.15.39:0/3965101868 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6801f040 con 0x564070cf7280 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.251+0000 7fdd5b400640 1 -- 172.21.15.39:0/3965101868 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdd680106a0 con 0x564070cf7280 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.251+0000 7fdd86c0e740 1 -- 172.21.15.39:0/3965101868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd6802f070 con 0x564070cf7280 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.251+0000 7fdd5b400640 1 -- 172.21.15.39:0/3965101868 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd68044050 con 0x564070cf7280 2024-07-04T08:13:34.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.251+0000 7fdd86c0e740 10 client.15051 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:34.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.251+0000 7fdd86c0e740 1 -- 172.21.15.39:0/3965101868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640710e2df0 con 0x564070cf7280 2024-07-04T08:13:34.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.251+0000 7fdd5b400640 1 -- 172.21.15.39:0/3965101868 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6802f070 con 0x564070cf7280 2024-07-04T08:13:34.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.252+0000 7fdd5b400640 1 -- 172.21.15.39:0/3965101868 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdd68019040 con 0x564070cf7280 2024-07-04T08:13:34.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.252+0000 7fdd86c0e740 10 client.15051 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.252+0000 7fdd86c0e740 10 client.15051 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-04T08:13:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.252+0000 7fdd86c0e740 20 client.15051 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.252+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x564070cabc90 0x564071146660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.252+0000 7fdd86c0e740 4 client.15051 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:13:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.252+0000 7fdd86c0e740 1 -- 172.21.15.39:0/3965101868 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "status"}) -- 0x564071030670 con 0x564070cabc90 2024-07-04T08:13:34.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.252+0000 7fdd7e600640 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x564070cabc90 0x564071146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.253+0000 7fdd7e600640 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x564070cabc90 0x564071146660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c769a1eab2672a52 server_cookie=5a726315adc649bc in_seq=0 out_seq=0 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.253+0000 7fdd5b400640 10 client.15051 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.253+0000 7fdd5b400640 1 -- 172.21.15.39:0/3965101868 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564071030670 con 0x564070cabc90 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.253+0000 7fdd5b400640 10 client.15051 handle_command_reply: tid=0 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.253+0000 7fdd86c0e740 1 client.15051 shutdown 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 -- 172.21.15.39:0/3965101868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 msgr2=0x5640710dd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710dd910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800bf40 tx=0x7fdd6801a620 comp rx=0 tx=0).stop 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 -- 172.21.15.39:0/3965101868 shutdown_connections 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x564070cabc90 0x564071146660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564070cf7280 0x5640710dd910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564070d17f60 0x5640710e13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 --2- 172.21.15.39:0/3965101868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564070cef0a0 0x5640710dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 -- 172.21.15.39:0/3965101868 >> 172.21.15.39:0/3965101868 conn(0x564070d0c350 msgr2=0x5640710c69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd5b400640 0 client.15051 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:13:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 -- 172.21.15.39:0/3965101868 shutdown_connections 2024-07-04T08:13:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 1 -- 172.21.15.39:0/3965101868 wait complete. 2024-07-04T08:13:34.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.254+0000 7fdd86c0e740 20 client.15051 trim_cache size 0 max 16384 2024-07-04T08:13:34.258 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:13:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:13:34.258 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24674, 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 150.445415903, 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 266.74229811499998 2024-07-04T08:13:34.259 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 -- 172.21.15.39:0/2451085535 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd600b2840 msgr2=0x7fdd600b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd600b2840 0x7fdd600b4d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0e2490 tx=0x7fdd6c000bc0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 -- 172.21.15.39:0/2451085535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78105110 msgr2=0x7fdd7819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78105110 0x7fdd7819ba10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00d950 tx=0x7fdd6c00de10 comp rx=0 tx=0).stop 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 -- 172.21.15.39:0/2451085535 shutdown_connections 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd600b2840 0x7fdd600b4d50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd7810b3a0 0x7fdd781a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd7810aa50 0x7fdd7819bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 --2- 172.21.15.39:0/2451085535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd78105110 0x7fdd7819ba10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:34.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 -- 172.21.15.39:0/2451085535 >> 172.21.15.39:0/2451085535 conn(0x7fdd78100b80 msgr2=0x7fdd7810a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:34.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.255+0000 7fdd7f000640 1 -- 172.21.15.39:0/2451085535 shutdown_connections 2024-07-04T08:13:34.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:34.256+0000 7fdd7f000640 1 -- 172.21.15.39:0/2451085535 wait complete. 2024-07-04T08:13:35.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:34 smithi133 ceph-mon[37050]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:35.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:34 smithi039 ceph-mon[32062]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:35.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:34 smithi050 ceph-mon[36577]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:35.182 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.i status 2024-07-04T08:13:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:37.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:36 smithi133 ceph-mon[37050]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:37.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:36 smithi039 ceph-mon[32062]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:37.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:36 smithi050 ceph-mon[36577]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:38.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.096+0000 7f6ceb200640 1 -- 172.21.15.39:0/760032211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec1137d0 msgr2=0x7f6cec115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.096+0000 7f6ceb200640 1 --2- 172.21.15.39:0/760032211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec1137d0 0x7f6cec115ca0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009ca0 tx=0x7f6ce402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.096+0000 7f6ceb200640 1 -- 172.21.15.39:0/760032211 shutdown_connections 2024-07-04T08:13:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.096+0000 7f6ceb200640 1 --2- 172.21.15.39:0/760032211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec1137d0 0x7f6cec115ca0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.096+0000 7f6ceb200640 1 --2- 172.21.15.39:0/760032211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cec075320 0x7f6cec075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.096+0000 7f6ceb200640 1 --2- 172.21.15.39:0/760032211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cec0768d0 0x7f6cec074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.096+0000 7f6ceb200640 1 -- 172.21.15.39:0/760032211 >> 172.21.15.39:0/760032211 conn(0x7f6cec0fe990 msgr2=0x7f6cec100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.096+0000 7f6ceb200640 1 -- 172.21.15.39:0/760032211 shutdown_connections 2024-07-04T08:13:38.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.096+0000 7f6ceb200640 1 -- 172.21.15.39:0/760032211 wait complete. 2024-07-04T08:13:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.097+0000 7f6ceb200640 1 Processor -- start 2024-07-04T08:13:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.098+0000 7f6ceb200640 1 -- start start 2024-07-04T08:13:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.098+0000 7f6ceb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cec075320 0x7f6cec19fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.098+0000 7f6ceb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0768d0 0x7f6cec1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.098+0000 7f6ceb200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cec1137d0 0x7f6cec1a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.098+0000 7f6ceb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec119400 con 0x7f6cec0768d0 2024-07-04T08:13:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.098+0000 7f6ceb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec119280 con 0x7f6cec075320 2024-07-04T08:13:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.098+0000 7f6ceb200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6cec119580 con 0x7f6cec1137d0 2024-07-04T08:13:38.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.098+0000 7f6ce1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0768d0 0x7f6cec1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.098+0000 7f6ce1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0768d0 0x7f6cec1a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36672/0 (socket says 172.21.15.39:36672) 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.099+0000 7f6ce1400640 1 -- 172.21.15.39:0/247401414 learned_addr learned my addr 172.21.15.39:0/247401414 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.099+0000 7f6ce9e00640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cec075320 0x7f6cec19fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.099+0000 7f6ce1400640 1 -- 172.21.15.39:0/247401414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cec1137d0 msgr2=0x7f6cec1a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.099+0000 7f6ce1400640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cec1137d0 0x7f6cec1a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.099+0000 7f6ce1400640 1 -- 172.21.15.39:0/247401414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cec075320 msgr2=0x7f6cec19fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.099+0000 7f6ce1400640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cec075320 0x7f6cec19fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.099+0000 7f6ce1400640 1 -- 172.21.15.39:0/247401414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cec1a5de0 con 0x7f6cec0768d0 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.099+0000 7f6ce9e00640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cec075320 0x7f6cec19fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:13:38.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.100+0000 7f6ce1400640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0768d0 0x7f6cec1a0410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4005fd0 tx=0x7f6cd400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:38.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.100+0000 7f6ce3400640 1 -- 172.21.15.39:0/247401414 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4017070 con 0x7f6cec0768d0 2024-07-04T08:13:38.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.100+0000 7f6ceb200640 1 -- 172.21.15.39:0/247401414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cec1a60d0 con 0x7f6cec0768d0 2024-07-04T08:13:38.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.100+0000 7f6ce3400640 1 -- 172.21.15.39:0/247401414 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cd4009c30 con 0x7f6cec0768d0 2024-07-04T08:13:38.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.100+0000 7f6ce3400640 1 -- 172.21.15.39:0/247401414 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4003d30 con 0x7f6cec0768d0 2024-07-04T08:13:38.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.100+0000 7f6ceb200640 1 -- 172.21.15.39:0/247401414 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cec1ac830 con 0x7f6cec0768d0 2024-07-04T08:13:38.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.104+0000 7f6cf7321740 1 Processor -- start 2024-07-04T08:13:38.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.105+0000 7f6cf7321740 1 -- start start 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.105+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3ca4f60 0x5592c4049830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.105+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c7c0a0 0x5592c4049d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.105+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c404a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.105+0000 7f6cf7321740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5592c3fbfc80 con 0x5592c3ca4f60 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.105+0000 7f6cf7321740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5592c3faf220 con 0x5592c3c84280 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.105+0000 7f6cf7321740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592c3fb3260 con 0x5592c3c7c0a0 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6cea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3ca4f60 0x5592c4049830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c7c0a0 0x5592c4049d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6ce3400640 1 -- 172.21.15.39:0/247401414 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f6cd4003ed0 con 0x7f6cec0768d0 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6cea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3ca4f60 0x5592c4049830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36676/0 (socket says 172.21.15.39:36676) 2024-07-04T08:13:38.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6cea800640 1 -- 172.21.15.39:0/3953591457 learned_addr learned my addr 172.21.15.39:0/3953591457 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:38.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6ce9e00640 1 -- 172.21.15.39:0/3953591457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 msgr2=0x5592c404a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:38.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6ce9e00640 1 --2- 172.21.15.39:0/3953591457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c404a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6ce9e00640 1 -- 172.21.15.39:0/3953591457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3ca4f60 msgr2=0x5592c4049830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:38.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6ce9e00640 1 --2- 172.21.15.39:0/3953591457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3ca4f60 0x5592c4049830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.106+0000 7f6ce9e00640 1 -- 172.21.15.39:0/3953591457 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c3f47530 con 0x5592c3c7c0a0 2024-07-04T08:13:38.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.107+0000 7f6ce9e00640 1 --2- 172.21.15.39:0/3953591457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c7c0a0 0x5592c4049d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009f50 tx=0x7f6cdc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34ebbdbfee3ed315 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:38.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.107+0000 7f6cdb400640 1 -- 172.21.15.39:0/3953591457 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc004030 con 0x5592c3c7c0a0 2024-07-04T08:13:38.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.107+0000 7f6cdb400640 1 -- 172.21.15.39:0/3953591457 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cdc005020 con 0x5592c3c7c0a0 2024-07-04T08:13:38.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.107+0000 7f6cdb400640 1 -- 172.21.15.39:0/3953591457 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc004c40 con 0x5592c3c7c0a0 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6ce3400640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6ccc0b2840 0x7f6ccc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6ce3400640 1 -- 172.21.15.39:0/247401414 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f6cd40e23a0 con 0x7f6cec0768d0 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6cea800640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6ccc0b2840 0x7f6ccc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6cf7321740 1 -- 172.21.15.39:0/3953591457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c7c0a0 msgr2=0x5592c4049d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3953591457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c7c0a0 0x5592c4049d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009f50 tx=0x7f6cdc0118d0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6cf7321740 1 -- 172.21.15.39:0/3953591457 shutdown_connections 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3953591457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c404a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3953591457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c7c0a0 0x5592c4049d70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3953591457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3ca4f60 0x5592c4049830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.108+0000 7f6cf7321740 1 -- 172.21.15.39:0/3953591457 >> 172.21.15.39:0/3953591457 conn(0x5592c3c99350 msgr2=0x5592c4047720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.109+0000 7f6cf7321740 1 -- 172.21.15.39:0/3953591457 shutdown_connections 2024-07-04T08:13:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.109+0000 7f6cf7321740 1 -- 172.21.15.39:0/3953591457 wait complete. 2024-07-04T08:13:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.109+0000 7f6cea800640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6ccc0b2840 0x7f6ccc0b4d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ce402f500 tx=0x7f6ce402f350 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.109+0000 7f6cf7321740 1 -- start start 2024-07-04T08:13:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.110+0000 7f6cf7321740 1 Processor -- start 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.110+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c40553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.110+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c84280 0x5592c40558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.110+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3ca4f60 0x5592c4061330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.110+0000 7f6cf7321740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5592c3e135f0 con 0x5592c3c7c0a0 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.110+0000 7f6cf7321740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5592c3f56950 con 0x5592c3ca4f60 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.110+0000 7f6cf7321740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592c4058010 con 0x5592c3c84280 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.110+0000 7f6cf7321740 10 client.0 _ll_register_callbacks cb 0x5592c3fa8480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c40553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c40553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36690/0 (socket says 172.21.15.39:36690) 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce9e00640 1 -- 172.21.15.39:0/3016739084 learned_addr learned my addr 172.21.15.39:0/3016739084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce1400640 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c84280 0x5592c40558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6cea800640 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3ca4f60 0x5592c4061330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce1400640 1 -- 172.21.15.39:0/3016739084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3ca4f60 msgr2=0x5592c4061330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce1400640 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3ca4f60 0x5592c4061330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce1400640 1 -- 172.21.15.39:0/3016739084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 msgr2=0x5592c40553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce1400640 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c40553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce1400640 1 -- 172.21.15.39:0/3016739084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c3d10540 con 0x5592c3c84280 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6ce1400640 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c84280 0x5592c40558e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009610 tx=0x7f6cd40a9160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483e56b6c865c4f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.111+0000 7f6cd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.112+0000 7f6cd3e00640 1 -- 172.21.15.39:0/3016739084 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd40910d0 con 0x5592c3c84280 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.112+0000 7f6cf7321740 1 -- 172.21.15.39:0/3016739084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cdc005020 con 0x5592c3c84280 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.112+0000 7f6cd3e00640 1 -- 172.21.15.39:0/3016739084 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f6cd40e2120 con 0x5592c3c84280 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.112+0000 7f6cd3e00640 1 -- 172.21.15.39:0/3016739084 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cd40ad8c0 con 0x5592c3c84280 2024-07-04T08:13:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.112+0000 7f6cf7321740 10 client.25174 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.112+0000 7f6cf7321740 1 -- 172.21.15.39:0/3016739084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592c4063fa0 con 0x5592c3c84280 2024-07-04T08:13:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.112+0000 7f6cd3e00640 1 -- 172.21.15.39:0/3016739084 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cdc005020 con 0x5592c3c84280 2024-07-04T08:13:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.113+0000 7f6cd3e00640 1 -- 172.21.15.39:0/3016739084 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6cd40a4e80 con 0x5592c3c84280 2024-07-04T08:13:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.113+0000 7f6cf7321740 10 client.25174 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.113+0000 7f6cf7321740 10 client.25174 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-04T08:13:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.113+0000 7f6cf7321740 20 client.25174 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.113+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5592c3c638e0 0x5592c40679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.113+0000 7f6cf7321740 4 client.25174 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:13:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.113+0000 7f6cf7321740 1 -- 172.21.15.39:0/3016739084 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c3fc1bd0 con 0x5592c3c638e0 2024-07-04T08:13:38.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.113+0000 7f6ce9e00640 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5592c3c638e0 0x5592c40679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.114+0000 7f6ce9e00640 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5592c3c638e0 0x5592c40679a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71deffc8075b04dc server_cookie=da3ff4fcd0ddb87e in_seq=0 out_seq=0 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.114+0000 7f6cd3e00640 10 client.25174 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cd3e00640 1 -- 172.21.15.39:0/3016739084 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5592c3fc1bd0 con 0x5592c3c638e0 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cd3e00640 10 client.25174 handle_command_reply: tid=0 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cf7321740 1 client.25174 shutdown 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cf7321740 1 -- 172.21.15.39:0/3016739084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c84280 msgr2=0x5592c40558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c84280 0x5592c40558e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009610 tx=0x7f6cd40a9160 comp rx=0 tx=0).stop 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cf7321740 1 -- 172.21.15.39:0/3016739084 shutdown_connections 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5592c3c638e0 0x5592c40679a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3ca4f60 0x5592c4061330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3c84280 0x5592c40558e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cf7321740 1 --2- 172.21.15.39:0/3016739084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c40553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cd3e00640 0 client.25174 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:13:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.115+0000 7f6cf7321740 1 -- 172.21.15.39:0/3016739084 >> 172.21.15.39:0/3016739084 conn(0x5592c3c99350 msgr2=0x5592c404ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.116+0000 7f6cf7321740 1 -- 172.21.15.39:0/3016739084 shutdown_connections 2024-07-04T08:13:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.116+0000 7f6cf7321740 1 -- 172.21.15.39:0/3016739084 wait complete. 2024-07-04T08:13:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.116+0000 7f6cf7321740 20 client.25174 trim_cache size 0 max 16384 2024-07-04T08:13:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.137+0000 7f6cf7321740 1 Processor -- start 2024-07-04T08:13:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.137+0000 7f6cf7321740 1 -- start start 2024-07-04T08:13:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.137+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c404d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.137+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c40612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.137+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 0x5592c40617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.137+0000 7f6cf7321740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5592c4065ce0 con 0x5592c3c7c0a0 2024-07-04T08:13:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.137+0000 7f6cf7321740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5592c40663b0 con 0x5592c3c84280 2024-07-04T08:13:38.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.137+0000 7f6cf7321740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592c4066a80 con 0x5592c3ca4f60 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c404d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c40612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c40612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:40292/0 (socket says 172.21.15.39:40292) 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c404d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36696/0 (socket says 172.21.15.39:36696) 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce1400640 1 -- 172.21.15.39:0/4221297652 learned_addr learned my addr 172.21.15.39:0/4221297652 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6cea800640 1 --2- 172.21.15.39:0/4221297652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 0x5592c40617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce1400640 1 -- 172.21.15.39:0/4221297652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 msgr2=0x5592c40617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce1400640 1 --2- 172.21.15.39:0/4221297652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 0x5592c40617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce1400640 1 -- 172.21.15.39:0/4221297652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 msgr2=0x5592c404d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:38.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce1400640 1 --2- 172.21.15.39:0/4221297652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c404d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.138+0000 7f6ce1400640 1 -- 172.21.15.39:0/4221297652 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c404c0c0 con 0x5592c3c84280 2024-07-04T08:13:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.139+0000 7f6ce1400640 1 --2- 172.21.15.39:0/4221297652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c40612a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40ada80 tx=0x7f6cd40a9690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb15e82eea2a98a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.139+0000 7f6cdaa00640 1 -- 172.21.15.39:0/4221297652 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd40a9c10 con 0x5592c3c84280 2024-07-04T08:13:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.139+0000 7f6cdaa00640 1 -- 172.21.15.39:0/4221297652 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cd40b2370 con 0x5592c3c84280 2024-07-04T08:13:38.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.139+0000 7f6cdaa00640 1 -- 172.21.15.39:0/4221297652 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd40a83f0 con 0x5592c3c84280 2024-07-04T08:13:38.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.140+0000 7f6cf7321740 1 -- 172.21.15.39:0/4221297652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 msgr2=0x5592c40612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:38.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.140+0000 7f6cf7321740 1 --2- 172.21.15.39:0/4221297652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c40612a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40ada80 tx=0x7f6cd40a9690 comp rx=0 tx=0).stop 2024-07-04T08:13:38.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.140+0000 7f6cf7321740 1 -- 172.21.15.39:0/4221297652 shutdown_connections 2024-07-04T08:13:38.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.140+0000 7f6cf7321740 1 --2- 172.21.15.39:0/4221297652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 0x5592c40617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.140+0000 7f6cf7321740 1 --2- 172.21.15.39:0/4221297652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c40612a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.140+0000 7f6cf7321740 1 --2- 172.21.15.39:0/4221297652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c404d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.140+0000 7f6cf7321740 1 -- 172.21.15.39:0/4221297652 >> 172.21.15.39:0/4221297652 conn(0x5592c3c99350 msgr2=0x5592c4050350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:38.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.140+0000 7f6cf7321740 1 -- 172.21.15.39:0/4221297652 shutdown_connections 2024-07-04T08:13:38.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.140+0000 7f6cf7321740 1 -- 172.21.15.39:0/4221297652 wait complete. 2024-07-04T08:13:38.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.141+0000 7f6cf7321740 1 -- start start 2024-07-04T08:13:38.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.141+0000 7f6cf7321740 1 Processor -- start 2024-07-04T08:13:38.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.141+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c40703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.141+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c4069880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.141+0000 7f6cf7321740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 0x5592c4069e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6cf7321740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5592c4067820 con 0x5592c3c7c0a0 2024-07-04T08:13:38.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6cf7321740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5592c4067150 con 0x5592c3c84280 2024-07-04T08:13:38.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6cf7321740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592c4067ef0 con 0x5592c3ca4f60 2024-07-04T08:13:38.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6cf7321740 10 client.0 _ll_register_callbacks cb 0x7f6cd400df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:38.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6ce1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6ce9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c40703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6ce1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c4069880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:40308/0 (socket says 172.21.15.39:40308) 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6ce1400640 1 -- 172.21.15.39:0/2429577098 learned_addr learned my addr 172.21.15.39:0/2429577098 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6cea800640 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 0x5592c4069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6ce1400640 1 -- 172.21.15.39:0/2429577098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 msgr2=0x5592c4069e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6ce1400640 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 0x5592c4069e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6ce1400640 1 -- 172.21.15.39:0/2429577098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 msgr2=0x5592c40703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6ce1400640 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c40703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.142+0000 7f6ce1400640 1 -- 172.21.15.39:0/2429577098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c406a660 con 0x5592c3c84280 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.143+0000 7f6ce1400640 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c4069880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40e2690 tx=0x7f6cd40a5450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10ea4c6a48d4e94a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.143+0000 7f6cd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.143+0000 7f6cd3e00640 1 -- 172.21.15.39:0/2429577098 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd40a81e0 con 0x5592c3c84280 2024-07-04T08:13:38.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.143+0000 7f6cd3e00640 1 -- 172.21.15.39:0/2429577098 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f6cd40c3e00 con 0x5592c3c84280 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.143+0000 7f6cf7321740 1 -- 172.21.15.39:0/2429577098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd40b2370 con 0x5592c3c84280 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.143+0000 7f6cd3e00640 1 -- 172.21.15.39:0/2429577098 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6cd40a9c10 con 0x5592c3c84280 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.143+0000 7f6cf7321740 10 client.25190 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.143+0000 7f6cf7321740 1 -- 172.21.15.39:0/2429577098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592c40d0c40 con 0x5592c3c84280 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.144+0000 7f6cd3e00640 1 -- 172.21.15.39:0/2429577098 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cd40b2370 con 0x5592c3c84280 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.144+0000 7f6cd3e00640 1 -- 172.21.15.39:0/2429577098 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6cd40041c0 con 0x5592c3c84280 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.144+0000 7f6cf7321740 10 client.25190 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.144+0000 7f6cf7321740 10 client.25190 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.144+0000 7f6cf7321740 20 client.25190 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.144+0000 7f6cf7321740 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5592c3c38c90 0x5592c40d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.144+0000 7f6cf7321740 4 client.25190 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:13:38.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.144+0000 7f6cf7321740 1 -- 172.21.15.39:0/2429577098 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "status"}) -- 0x5592c3fbd670 con 0x5592c3c38c90 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.144+0000 7f6ce9e00640 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5592c3c38c90 0x5592c40d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.145+0000 7f6ce9e00640 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5592c3c38c90 0x5592c40d3f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adda94369b097ac6 server_cookie=22b2c9cb84f3e9d in_seq=0 out_seq=0 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.145+0000 7f6cd3e00640 10 client.25190 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.145+0000 7f6cd3e00640 1 -- 172.21.15.39:0/2429577098 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5592c3fbd670 con 0x5592c3c38c90 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.145+0000 7f6cd3e00640 10 client.25190 handle_command_reply: tid=0 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.145+0000 7f6cf7321740 1 client.25190 shutdown 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.145+0000 7f6cf7321740 1 -- 172.21.15.39:0/2429577098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 msgr2=0x5592c4069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.145+0000 7f6cf7321740 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c4069880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40e2690 tx=0x7f6cd40a5450 comp rx=0 tx=0).stop 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cf7321740 1 -- 172.21.15.39:0/2429577098 shutdown_connections 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cf7321740 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5592c3c38c90 0x5592c40d3f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cf7321740 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592c3ca4f60 0x5592c4069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cf7321740 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592c3c84280 0x5592c4069880 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cf7321740 1 --2- 172.21.15.39:0/2429577098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592c3c7c0a0 0x5592c40703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cf7321740 1 -- 172.21.15.39:0/2429577098 >> 172.21.15.39:0/2429577098 conn(0x5592c3c99350 msgr2=0x5592c4050350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cd3e00640 0 client.25190 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cf7321740 1 -- 172.21.15.39:0/2429577098 shutdown_connections 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cf7321740 1 -- 172.21.15.39:0/2429577098 wait complete. 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.146+0000 7f6cf7321740 20 client.25190 trim_cache size 0 max 16384 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:13:38.151 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24679, 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 155.36805295400001, 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 267.96196212400002 2024-07-04T08:13:38.152 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:13:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 -- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6ccc0b2840 msgr2=0x7f6ccc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6ccc0b2840 0x7f6ccc0b4d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ce402f500 tx=0x7f6ce402f350 comp rx=0 tx=0).stop 2024-07-04T08:13:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 -- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0768d0 msgr2=0x7f6cec1a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0768d0 0x7f6cec1a0410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4005fd0 tx=0x7f6cd400dde0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 -- 172.21.15.39:0/247401414 shutdown_connections 2024-07-04T08:13:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6ccc0b2840 0x7f6ccc0b4d50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0768d0 0x7f6cec1a0410 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6cec1137d0 0x7f6cec1a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 --2- 172.21.15.39:0/247401414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cec075320 0x7f6cec19fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:38.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 -- 172.21.15.39:0/247401414 >> 172.21.15.39:0/247401414 conn(0x7f6cec0fe990 msgr2=0x7f6cec10f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:38.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 -- 172.21.15.39:0/247401414 shutdown_connections 2024-07-04T08:13:38.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:38.147+0000 7f6ceb200640 1 -- 172.21.15.39:0/247401414 wait complete. 2024-07-04T08:13:38.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:38 smithi039 ceph-mon[32062]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:39.048 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.a status 2024-07-04T08:13:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:38 smithi133 ceph-mon[37050]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:38 smithi050 ceph-mon[36577]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:39.542 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:40 smithi133 ceph-mon[37050]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:41.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:40 smithi039 ceph-mon[32062]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:41.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:40 smithi050 ceph-mon[36577]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:41.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.804+0000 7fb027e00640 1 -- 172.21.15.39:0/2270497713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb028108050 msgr2=0x7fb02810e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:41.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.804+0000 7fb027e00640 1 --2- 172.21.15.39:0/2270497713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb028108050 0x7fb02810e880 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb020009ca0 tx=0x7fb02002a8b0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.805+0000 7fb027e00640 1 -- 172.21.15.39:0/2270497713 shutdown_connections 2024-07-04T08:13:41.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.805+0000 7fb027e00640 1 --2- 172.21.15.39:0/2270497713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb028108050 0x7fb02810e880 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.805+0000 7fb027e00640 1 --2- 172.21.15.39:0/2270497713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb028107700 0x7fb028107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.805+0000 7fb027e00640 1 --2- 172.21.15.39:0/2270497713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb028101dc0 0x7fb0281021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.805+0000 7fb027e00640 1 -- 172.21.15.39:0/2270497713 >> 172.21.15.39:0/2270497713 conn(0x7fb0280fd810 msgr2=0x7fb0280ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:41.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.805+0000 7fb027e00640 1 -- 172.21.15.39:0/2270497713 shutdown_connections 2024-07-04T08:13:41.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.805+0000 7fb027e00640 1 -- 172.21.15.39:0/2270497713 wait complete. 2024-07-04T08:13:41.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.806+0000 7fb027e00640 1 Processor -- start 2024-07-04T08:13:41.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.806+0000 7fb027e00640 1 -- start start 2024-07-04T08:13:41.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.806+0000 7fb027e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb028101dc0 0x7fb0280ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb027e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb028107700 0x7fb0280fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb027e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb028108050 0x7fb0280fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb027e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb028111d10 con 0x7fb028107700 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb027e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb028111b90 con 0x7fb028101dc0 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb027e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb028111e90 con 0x7fb028108050 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb026000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb028107700 0x7fb0280fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb026a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb028101dc0 0x7fb0280ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb026a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb028101dc0 0x7fb0280ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57750/0 (socket says 172.21.15.39:57750) 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb027400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb028108050 0x7fb0280fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb026a00640 1 -- 172.21.15.39:0/1375186800 learned_addr learned my addr 172.21.15.39:0/1375186800 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:41.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb026a00640 1 -- 172.21.15.39:0/1375186800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb028108050 msgr2=0x7fb0280fe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:41.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb026a00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb028108050 0x7fb0280fe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb026a00640 1 -- 172.21.15.39:0/1375186800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb028107700 msgr2=0x7fb0280fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:41.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb026a00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb028107700 0x7fb0280fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.807+0000 7fb026a00640 1 -- 172.21.15.39:0/1375186800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0280feaa0 con 0x7fb028101dc0 2024-07-04T08:13:41.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.808+0000 7fb026a00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb028101dc0 0x7fb0280ff750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb01800e9f0 tx=0x7fb01800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:41.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.808+0000 7fb01f400640 1 -- 172.21.15.39:0/1375186800 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01800cd80 con 0x7fb028101dc0 2024-07-04T08:13:41.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.808+0000 7fb01f400640 1 -- 172.21.15.39:0/1375186800 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0180108e0 con 0x7fb028101dc0 2024-07-04T08:13:41.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.808+0000 7fb027e00640 1 -- 172.21.15.39:0/1375186800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0281ac170 con 0x7fb028101dc0 2024-07-04T08:13:41.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.808+0000 7fb01f400640 1 -- 172.21.15.39:0/1375186800 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018018980 con 0x7fb028101dc0 2024-07-04T08:13:41.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.808+0000 7fb027e00640 1 -- 172.21.15.39:0/1375186800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0281ac6b0 con 0x7fb028101dc0 2024-07-04T08:13:41.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.812+0000 7fb033be4740 1 Processor -- start 2024-07-04T08:13:41.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.812+0000 7fb033be4740 1 -- start start 2024-07-04T08:13:41.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.812+0000 7fb01f400640 1 -- 172.21.15.39:0/1375186800 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fb018010430 con 0x7fb028101dc0 2024-07-04T08:13:41.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.812+0000 7fb033be4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f5a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.812+0000 7fb033be4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b8d0a0 0x559b50f5ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.812+0000 7fb033be4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f5b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.812+0000 7fb033be4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559b50ed0ca0 con 0x559b50bb5f60 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.812+0000 7fb033be4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559b50ec0240 con 0x559b50b8d0a0 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.812+0000 7fb033be4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559b50ec4280 con 0x559b50b95280 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb027400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f5a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb027400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f5a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58884/0 (socket says 172.21.15.39:58884) 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb027400640 1 -- 172.21.15.39:0/247263946 learned_addr learned my addr 172.21.15.39:0/247263946 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb026a00640 1 --2- 172.21.15.39:0/247263946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b8d0a0 0x559b50f5ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb026000640 1 --2- 172.21.15.39:0/247263946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f5b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb027400640 1 -- 172.21.15.39:0/247263946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 msgr2=0x559b50f5b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:41.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb027400640 1 --2- 172.21.15.39:0/247263946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f5b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb027400640 1 -- 172.21.15.39:0/247263946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b8d0a0 msgr2=0x559b50f5ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb027400640 1 --2- 172.21.15.39:0/247263946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b8d0a0 0x559b50f5ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb027400640 1 -- 172.21.15.39:0/247263946 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b50e58550 con 0x559b50bb5f60 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.813+0000 7fb027400640 1 --2- 172.21.15.39:0/247263946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f5a850 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb020035000 tx=0x7fb020009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93d4147e4218f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.814+0000 7fb01f400640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb00c0b2840 0x7fb00c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.814+0000 7fb013e00640 1 -- 172.21.15.39:0/247263946 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200305a0 con 0x559b50bb5f60 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.814+0000 7fb013e00640 1 -- 172.21.15.39:0/247263946 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb020036030 con 0x559b50bb5f60 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.814+0000 7fb01f400640 1 -- 172.21.15.39:0/1375186800 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb0180e2a40 con 0x7fb028101dc0 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.814+0000 7fb013e00640 1 -- 172.21.15.39:0/247263946 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200042a0 con 0x559b50bb5f60 2024-07-04T08:13:41.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.814+0000 7fb026a00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb00c0b2840 0x7fb00c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb033be4740 1 -- 172.21.15.39:0/247263946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 msgr2=0x559b50f5a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:41.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb033be4740 1 --2- 172.21.15.39:0/247263946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f5a850 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb020035000 tx=0x7fb020009800 comp rx=0 tx=0).stop 2024-07-04T08:13:41.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb026a00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb00c0b2840 0x7fb00c0b4d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0180e0090 tx=0x7fb0180e8530 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:41.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb033be4740 1 -- 172.21.15.39:0/247263946 shutdown_connections 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb033be4740 1 --2- 172.21.15.39:0/247263946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f5b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb033be4740 1 --2- 172.21.15.39:0/247263946 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b8d0a0 0x559b50f5ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb033be4740 1 --2- 172.21.15.39:0/247263946 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f5a850 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb033be4740 1 -- 172.21.15.39:0/247263946 >> 172.21.15.39:0/247263946 conn(0x559b50baa350 msgr2=0x559b50f58740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb033be4740 1 -- 172.21.15.39:0/247263946 shutdown_connections 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.815+0000 7fb033be4740 1 -- 172.21.15.39:0/247263946 wait complete. 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.816+0000 7fb033be4740 1 -- start start 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.816+0000 7fb033be4740 1 Processor -- start 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.816+0000 7fb033be4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b8d0a0 0x559b50f663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb033be4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f66930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb033be4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb033be4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559b50d245f0 con 0x559b50bb5f60 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb033be4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559b50e67970 con 0x559b50b95280 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb033be4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559b50f68fd0 con 0x559b50b8d0a0 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb033be4740 10 client.0 _ll_register_callbacks cb 0x559b50eb94a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb027400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b8d0a0 0x559b50f663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb027400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb026000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f66930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb026000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f66930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57770/0 (socket says 172.21.15.39:57770) 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb027400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58898/0 (socket says 172.21.15.39:58898) 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb026000640 1 -- 172.21.15.39:0/157484977 learned_addr learned my addr 172.21.15.39:0/157484977 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb026000640 1 -- 172.21.15.39:0/157484977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b8d0a0 msgr2=0x559b50f663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb026000640 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b8d0a0 0x559b50f663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb026000640 1 -- 172.21.15.39:0/157484977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 msgr2=0x559b50f722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:41.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb026000640 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.817+0000 7fb026000640 1 -- 172.21.15.39:0/157484977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b50c21540 con 0x559b50b95280 2024-07-04T08:13:41.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.818+0000 7fb026000640 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f66930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb0140126e0 tx=0x7fb014012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f09e5e00982d05 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:41.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.818+0000 7fb010c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:13:41.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.818+0000 7fb010c00640 1 -- 172.21.15.39:0/157484977 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014012190 con 0x559b50b95280 2024-07-04T08:13:41.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.818+0000 7fb033be4740 1 -- 172.21.15.39:0/157484977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb020036030 con 0x559b50b95280 2024-07-04T08:13:41.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.818+0000 7fb010c00640 1 -- 172.21.15.39:0/157484977 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb014004020 con 0x559b50b95280 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.818+0000 7fb010c00640 1 -- 172.21.15.39:0/157484977 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb014025050 con 0x559b50b95280 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.818+0000 7fb033be4740 10 client.25202 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.818+0000 7fb033be4740 1 -- 172.21.15.39:0/157484977 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b50f74ff0 con 0x559b50b95280 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.819+0000 7fb010c00640 1 -- 172.21.15.39:0/157484977 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb020036030 con 0x559b50b95280 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.819+0000 7fb010c00640 1 -- 172.21.15.39:0/157484977 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb014003680 con 0x559b50b95280 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.820+0000 7fb033be4740 10 client.25202 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.820+0000 7fb033be4740 10 client.25202 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.820+0000 7fb033be4740 20 client.25202 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.820+0000 7fb033be4740 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559b50b748e0 0x559b50f78a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.820+0000 7fb033be4740 4 client.25202 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.820+0000 7fb033be4740 1 -- 172.21.15.39:0/157484977 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b50ed2bf0 con 0x559b50b748e0 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.820+0000 7fb027400640 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559b50b748e0 0x559b50f78a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.821+0000 7fb027400640 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559b50b748e0 0x559b50f78a60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1014c2cf45e20d3 server_cookie=e095021eab2bf0d1 in_seq=0 out_seq=0 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.821+0000 7fb010c00640 10 client.25202 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.821+0000 7fb010c00640 1 -- 172.21.15.39:0/157484977 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559b50ed2bf0 con 0x559b50b748e0 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.821+0000 7fb010c00640 10 client.25202 handle_command_reply: tid=0 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 client.25202 shutdown 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 -- 172.21.15.39:0/157484977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 msgr2=0x559b50f66930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f66930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb0140126e0 tx=0x7fb014012ba0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 -- 172.21.15.39:0/157484977 shutdown_connections 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559b50b748e0 0x559b50f78a60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50bb5f60 0x559b50f722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f66930 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb010c00640 0 client.25202 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:13:41.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 --2- 172.21.15.39:0/157484977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b8d0a0 0x559b50f663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 -- 172.21.15.39:0/157484977 >> 172.21.15.39:0/157484977 conn(0x559b50baa350 msgr2=0x559b50f5da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:41.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 -- 172.21.15.39:0/157484977 shutdown_connections 2024-07-04T08:13:41.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 1 -- 172.21.15.39:0/157484977 wait complete. 2024-07-04T08:13:41.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.822+0000 7fb033be4740 20 client.25202 trim_cache size 0 max 16384 2024-07-04T08:13:41.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.842+0000 7fb033be4740 1 Processor -- start 2024-07-04T08:13:41.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.842+0000 7fb033be4740 1 -- start start 2024-07-04T08:13:41.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb033be4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f53880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb033be4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f72260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb033be4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50bb5f60 0x559b50f727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb033be4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559b50f76da0 con 0x559b50b8d0a0 2024-07-04T08:13:41.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb033be4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559b50f77470 con 0x559b50b95280 2024-07-04T08:13:41.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb033be4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559b50f77b40 con 0x559b50bb5f60 2024-07-04T08:13:41.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb026000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f72260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb027400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f53880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb026000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f72260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57786/0 (socket says 172.21.15.39:57786) 2024-07-04T08:13:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb026000640 1 -- 172.21.15.39:0/4266007406 learned_addr learned my addr 172.21.15.39:0/4266007406 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:41.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb026000640 1 -- 172.21.15.39:0/4266007406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50bb5f60 msgr2=0x559b50f727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb026000640 1 --2- 172.21.15.39:0/4266007406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50bb5f60 0x559b50f727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb026000640 1 -- 172.21.15.39:0/4266007406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 msgr2=0x559b50f53880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb026000640 1 --2- 172.21.15.39:0/4266007406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f53880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb026000640 1 -- 172.21.15.39:0/4266007406 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b50f646f0 con 0x559b50b95280 2024-07-04T08:13:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.843+0000 7fb027400640 1 --2- 172.21.15.39:0/4266007406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f53880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:13:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.844+0000 7fb026000640 1 --2- 172.21.15.39:0/4266007406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f72260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb01400a010 tx=0x7fb014005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c29667e636a81292 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.844+0000 7fb013400640 1 -- 172.21.15.39:0/4266007406 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014022040 con 0x559b50b95280 2024-07-04T08:13:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.844+0000 7fb013400640 1 -- 172.21.15.39:0/4266007406 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb01400fdd0 con 0x559b50b95280 2024-07-04T08:13:41.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.844+0000 7fb013400640 1 -- 172.21.15.39:0/4266007406 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014005550 con 0x559b50b95280 2024-07-04T08:13:41.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.845+0000 7fb033be4740 1 -- 172.21.15.39:0/4266007406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 msgr2=0x559b50f72260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:41.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.845+0000 7fb033be4740 1 --2- 172.21.15.39:0/4266007406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f72260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb01400a010 tx=0x7fb014005e70 comp rx=0 tx=0).stop 2024-07-04T08:13:41.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.845+0000 7fb033be4740 1 -- 172.21.15.39:0/4266007406 shutdown_connections 2024-07-04T08:13:41.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.845+0000 7fb033be4740 1 --2- 172.21.15.39:0/4266007406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50bb5f60 0x559b50f727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.845+0000 7fb033be4740 1 --2- 172.21.15.39:0/4266007406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50b95280 0x559b50f72260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.845+0000 7fb033be4740 1 --2- 172.21.15.39:0/4266007406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f53880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.845+0000 7fb033be4740 1 -- 172.21.15.39:0/4266007406 >> 172.21.15.39:0/4266007406 conn(0x559b50baa350 msgr2=0x559b50f64a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.845+0000 7fb033be4740 1 -- 172.21.15.39:0/4266007406 shutdown_connections 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.845+0000 7fb033be4740 1 -- 172.21.15.39:0/4266007406 wait complete. 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.846+0000 7fb033be4740 1 -- start start 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.846+0000 7fb033be4740 1 Processor -- start 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.846+0000 7fb033be4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f7a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.846+0000 7fb033be4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f7a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb033be4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50bb5f60 0x559b50f7f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb033be4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559b50f788e0 con 0x559b50b8d0a0 2024-07-04T08:13:41.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb033be4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559b50f78210 con 0x559b50bb5f60 2024-07-04T08:13:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb033be4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559b50f78fb0 con 0x559b50b95280 2024-07-04T08:13:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb033be4740 10 client.0 _ll_register_callbacks cb 0x7fb014009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb027400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f7a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb026000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f7a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb027400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f7a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58920/0 (socket says 172.21.15.39:58920) 2024-07-04T08:13:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb027400640 1 -- 172.21.15.39:0/3118544286 learned_addr learned my addr 172.21.15.39:0/3118544286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb026000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f7a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:35498/0 (socket says 172.21.15.39:35498) 2024-07-04T08:13:41.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb027400640 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50bb5f60 0x559b50f7f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.847+0000 7fb026000640 1 -- 172.21.15.39:0/3118544286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50bb5f60 msgr2=0x559b50f7f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb026000640 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50bb5f60 0x559b50f7f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb026000640 1 -- 172.21.15.39:0/3118544286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 msgr2=0x559b50f7a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb026000640 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f7a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb026000640 1 -- 172.21.15.39:0/3118544286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b50f7fd70 con 0x559b50b95280 2024-07-04T08:13:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb027400640 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50bb5f60 0x559b50f7f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:13:41.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb026000640 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f7a830 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x559b50f79ae0 tx=0x7fb014003880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=281c9ff7be22907c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb010c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb010c00640 1 -- 172.21.15.39:0/3118544286 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014005a70 con 0x559b50b95280 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb010c00640 1 -- 172.21.15.39:0/3118544286 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb01400fb30 con 0x559b50b95280 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.848+0000 7fb033be4740 1 -- 172.21.15.39:0/3118544286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb01400fdd0 con 0x559b50b95280 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.849+0000 7fb010c00640 1 -- 172.21.15.39:0/3118544286 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb014005250 con 0x559b50b95280 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.849+0000 7fb033be4740 10 client.25195 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.849+0000 7fb033be4740 1 -- 172.21.15.39:0/3118544286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b50f81130 con 0x559b50b95280 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.849+0000 7fb010c00640 1 -- 172.21.15.39:0/3118544286 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb01400fdd0 con 0x559b50b95280 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.849+0000 7fb010c00640 1 -- 172.21.15.39:0/3118544286 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb01401ca40 con 0x559b50b95280 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.849+0000 7fb033be4740 10 client.25195 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.849+0000 7fb033be4740 10 client.25195 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T08:13:41.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.849+0000 7fb033be4740 20 client.25195 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.850+0000 7fb033be4740 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559b50b49c90 0x559b50fe4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.850+0000 7fb033be4740 4 client.25195 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.850+0000 7fb033be4740 1 -- 172.21.15.39:0/3118544286 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "status"}) -- 0x559b50ece690 con 0x559b50b49c90 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.850+0000 7fb027400640 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559b50b49c90 0x559b50fe4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.850+0000 7fb027400640 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559b50b49c90 0x559b50fe4750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41514c47fb8a2e69 server_cookie=6d2d066b194e5429 in_seq=0 out_seq=0 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.850+0000 7fb010c00640 10 client.25195 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb010c00640 1 -- 172.21.15.39:0/3118544286 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559b50ece690 con 0x559b50b49c90 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb010c00640 10 client.25195 handle_command_reply: tid=0 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 client.25195 shutdown 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 -- 172.21.15.39:0/3118544286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 msgr2=0x559b50f7a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f7a830 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x559b50f79ae0 tx=0x7fb014003880 comp rx=0 tx=0).stop 2024-07-04T08:13:41.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 -- 172.21.15.39:0/3118544286 shutdown_connections 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559b50b49c90 0x559b50fe4750 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559b50bb5f60 0x559b50f7f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b50b95280 0x559b50f7a830 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 --2- 172.21.15.39:0/3118544286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559b50b8d0a0 0x559b50f7a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb010c00640 0 client.25195 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 -- 172.21.15.39:0/3118544286 >> 172.21.15.39:0/3118544286 conn(0x559b50baa350 msgr2=0x559b50f5b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.851+0000 7fb033be4740 1 -- 172.21.15.39:0/3118544286 shutdown_connections 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.852+0000 7fb033be4740 1 -- 172.21.15.39:0/3118544286 wait complete. 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.852+0000 7fb033be4740 20 client.25195 trim_cache size 0 max 16384 2024-07-04T08:13:41.857 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24701, 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 160.08302032700001, 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:13:41.858 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:13:41.859 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 262.14527803099998 2024-07-04T08:13:41.859 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:13:41.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.852+0000 7fb027e00640 1 -- 172.21.15.39:0/1375186800 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb00c0b2840 msgr2=0x7fb00c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:41.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.852+0000 7fb027e00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb00c0b2840 0x7fb00c0b4d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0180e0090 tx=0x7fb0180e8530 comp rx=0 tx=0).stop 2024-07-04T08:13:41.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.852+0000 7fb027e00640 1 -- 172.21.15.39:0/1375186800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb028101dc0 msgr2=0x7fb0280ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:41.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.852+0000 7fb027e00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb028101dc0 0x7fb0280ff750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb01800e9f0 tx=0x7fb01800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.853+0000 7fb027e00640 1 -- 172.21.15.39:0/1375186800 shutdown_connections 2024-07-04T08:13:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.853+0000 7fb027e00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb00c0b2840 0x7fb00c0b4d50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.853+0000 7fb027e00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb028107700 0x7fb0280fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.853+0000 7fb027e00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb028108050 0x7fb0280fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.853+0000 7fb027e00640 1 --2- 172.21.15.39:0/1375186800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb028101dc0 0x7fb0280ff750 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.853+0000 7fb027e00640 1 -- 172.21.15.39:0/1375186800 >> 172.21.15.39:0/1375186800 conn(0x7fb0280fd810 msgr2=0x7fb02806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.853+0000 7fb027e00640 1 -- 172.21.15.39:0/1375186800 shutdown_connections 2024-07-04T08:13:41.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:41.853+0000 7fb027e00640 1 -- 172.21.15.39:0/1375186800 wait complete. 2024-07-04T08:13:42.604 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.b status 2024-07-04T08:13:42.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:42 smithi039 ceph-mon[32062]: pgmap v435: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:42 smithi133 ceph-mon[37050]: pgmap v435: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:43.086 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:43.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:42 smithi050 ceph-mon[36577]: pgmap v435: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:44.852 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:44 smithi039 ceph-mon[32062]: pgmap v436: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:45.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:44 smithi133 ceph-mon[37050]: pgmap v436: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:45.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:44 smithi050 ceph-mon[36577]: pgmap v436: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:45.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.423+0000 7fddbea00640 1 -- 172.21.15.39:0/567656375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 msgr2=0x7fddb8105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.423+0000 7fddbea00640 1 --2- 172.21.15.39:0/567656375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 0x7fddb8105710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009900 tx=0x7fdda802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.423+0000 7fddbea00640 1 -- 172.21.15.39:0/567656375 shutdown_connections 2024-07-04T08:13:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.423+0000 7fddbea00640 1 --2- 172.21.15.39:0/567656375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb810ac40 0x7fddb810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.423+0000 7fddbea00640 1 --2- 172.21.15.39:0/567656375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb810b590 0x7fddb8111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.423+0000 7fddbea00640 1 --2- 172.21.15.39:0/567656375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 0x7fddb8105710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.423+0000 7fddbea00640 1 -- 172.21.15.39:0/567656375 >> 172.21.15.39:0/567656375 conn(0x7fddb8100d30 msgr2=0x7fddb8103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.423+0000 7fddbea00640 1 -- 172.21.15.39:0/567656375 shutdown_connections 2024-07-04T08:13:45.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.423+0000 7fddbea00640 1 -- 172.21.15.39:0/567656375 wait complete. 2024-07-04T08:13:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.425+0000 7fddbea00640 1 Processor -- start 2024-07-04T08:13:45.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.426+0000 7fddbea00640 1 -- start start 2024-07-04T08:13:45.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.426+0000 7fddbea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 0x7fddb819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.426+0000 7fddbea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb810ac40 0x7fddb819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.426+0000 7fddbea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb810b590 0x7fddb81a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.426+0000 7fddbea00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8115240 con 0x7fddb810ac40 2024-07-04T08:13:45.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.426+0000 7fddbea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81150c0 con 0x7fddb8105300 2024-07-04T08:13:45.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.426+0000 7fddbea00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fddb81153c0 con 0x7fddb810b590 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddb7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 0x7fddb819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddb7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 0x7fddb819bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57826/0 (socket says 172.21.15.39:57826) 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddb7e00640 1 -- 172.21.15.39:0/1276266008 learned_addr learned my addr 172.21.15.39:0/1276266008 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddafe00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb810ac40 0x7fddb819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddbca00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb810b590 0x7fddb81a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddb7e00640 1 -- 172.21.15.39:0/1276266008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb810b590 msgr2=0x7fddb81a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddb7e00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb810b590 0x7fddb81a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddb7e00640 1 -- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb810ac40 msgr2=0x7fddb819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddb7e00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb810ac40 0x7fddb819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.427+0000 7fddb7e00640 1 -- 172.21.15.39:0/1276266008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb81a1be0 con 0x7fddb8105300 2024-07-04T08:13:45.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.428+0000 7fddb7e00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 0x7fddb819bc70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdda8005940 tx=0x7fdda8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:45.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.428+0000 7fddb5600640 1 -- 172.21.15.39:0/1276266008 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda803f030 con 0x7fddb8105300 2024-07-04T08:13:45.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.428+0000 7fddbea00640 1 -- 172.21.15.39:0/1276266008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb81a1e70 con 0x7fddb8105300 2024-07-04T08:13:45.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.428+0000 7fddb5600640 1 -- 172.21.15.39:0/1276266008 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda8003aa0 con 0x7fddb8105300 2024-07-04T08:13:45.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.428+0000 7fddb5600640 1 -- 172.21.15.39:0/1276266008 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8035070 con 0x7fddb8105300 2024-07-04T08:13:45.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.428+0000 7fddbea00640 1 -- 172.21.15.39:0/1276266008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb81a8630 con 0x7fddb8105300 2024-07-04T08:13:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.432+0000 7fddc506b740 1 Processor -- start 2024-07-04T08:13:45.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.432+0000 7fddc506b740 1 -- start start 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.432+0000 7fddc506b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa72b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.432+0000 7fddc506b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa35e0a0 0x560bfa72bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.432+0000 7fddc506b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 0x560bfa72c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.432+0000 7fddc506b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560bfa6a1ca0 con 0x560bfa35e0a0 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.432+0000 7fddc506b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560bfa691240 con 0x560bfa386f60 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.432+0000 7fddc506b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560bfa695280 con 0x560bfa366280 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddb5600640 1 -- 172.21.15.39:0/1276266008 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fdda8042030 con 0x7fddb8105300 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddbca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa72b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddbca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa72b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57830/0 (socket says 172.21.15.39:57830) 2024-07-04T08:13:45.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddbca00640 1 -- 172.21.15.39:0/533794683 learned_addr learned my addr 172.21.15.39:0/533794683 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:45.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddafe00640 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 0x560bfa72c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddb7e00640 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa35e0a0 0x560bfa72bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddbca00640 1 -- 172.21.15.39:0/533794683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 msgr2=0x560bfa72c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:45.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddbca00640 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 0x560bfa72c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddbca00640 1 -- 172.21.15.39:0/533794683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa35e0a0 msgr2=0x560bfa72bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:45.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddbca00640 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa35e0a0 0x560bfa72bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddbca00640 1 -- 172.21.15.39:0/533794683 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfa629550 con 0x560bfa386f60 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.433+0000 7fddb7e00640 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa35e0a0 0x560bfa72bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.434+0000 7fddbca00640 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa72b670 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fddb0011440 tx=0x7fddb0011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42fd819f16c7b1f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.434+0000 7fddad600640 1 -- 172.21.15.39:0/533794683 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0003c30 con 0x560bfa386f60 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.434+0000 7fddad600640 1 -- 172.21.15.39:0/533794683 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb0003dd0 con 0x560bfa386f60 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.434+0000 7fddad600640 1 -- 172.21.15.39:0/533794683 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb000fa80 con 0x560bfa386f60 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddb5600640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd9c0b28c0 0x7fdd9c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddb5600640 1 -- 172.21.15.39:0/1276266008 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdda8102c80 con 0x7fddb8105300 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddb7e00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd9c0b28c0 0x7fdd9c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddc506b740 1 -- 172.21.15.39:0/533794683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 msgr2=0x560bfa72b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:45.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddc506b740 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa72b670 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fddb0011440 tx=0x7fddb0011900 comp rx=0 tx=0).stop 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddc506b740 1 -- 172.21.15.39:0/533794683 shutdown_connections 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddc506b740 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa35e0a0 0x560bfa72bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddc506b740 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 0x560bfa72c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddc506b740 1 --2- 172.21.15.39:0/533794683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa72b670 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.435+0000 7fddc506b740 1 -- 172.21.15.39:0/533794683 >> 172.21.15.39:0/533794683 conn(0x560bfa37b350 msgr2=0x560bfa729560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.436+0000 7fddc506b740 1 -- 172.21.15.39:0/533794683 shutdown_connections 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.436+0000 7fddc506b740 1 -- 172.21.15.39:0/533794683 wait complete. 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.436+0000 7fddb7e00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd9c0b28c0 0x7fdd9c0b4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560bfa72a2f0 tx=0x7fdda802a3f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.436+0000 7fddc506b740 1 -- start start 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.437+0000 7fddc506b740 1 Processor -- start 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.437+0000 7fddc506b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa35e0a0 0x560bfa7372a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.437+0000 7fddc506b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa7377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.437+0000 7fddc506b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa7431a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.437+0000 7fddc506b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560bfa4f55f0 con 0x560bfa366280 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.437+0000 7fddc506b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560bfa638970 con 0x560bfa386f60 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.437+0000 7fddc506b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560bfa739e50 con 0x560bfa35e0a0 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.437+0000 7fddc506b740 10 client.0 _ll_register_callbacks cb 0x560bfa68a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa7377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa7377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58958/0 (socket says 172.21.15.39:58958) 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddafe00640 1 -- 172.21.15.39:0/1409144455 learned_addr learned my addr 172.21.15.39:0/1409144455 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddbca00640 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa35e0a0 0x560bfa7372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddbca00640 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa7431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddafe00640 1 -- 172.21.15.39:0/1409144455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa35e0a0 msgr2=0x560bfa7372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddafe00640 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa35e0a0 0x560bfa7372a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddafe00640 1 -- 172.21.15.39:0/1409144455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 msgr2=0x560bfa7431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:45.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddafe00640 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa7431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddafe00640 1 -- 172.21.15.39:0/1409144455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfa3f2540 con 0x560bfa366280 2024-07-04T08:13:45.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.438+0000 7fddbca00640 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa7431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:13:45.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.439+0000 7fddafe00640 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa7377e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdda000b7e0 tx=0x7fdda000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90102755e8cec02d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:45.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.439+0000 7fdda6000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:13:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.439+0000 7fdda6000640 1 -- 172.21.15.39:0/1409144455 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda001b440 con 0x560bfa366280 2024-07-04T08:13:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.439+0000 7fdda6000640 1 -- 172.21.15.39:0/1409144455 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdda001b5e0 con 0x560bfa366280 2024-07-04T08:13:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.439+0000 7fddc506b740 1 -- 172.21.15.39:0/1409144455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb0003dd0 con 0x560bfa366280 2024-07-04T08:13:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.439+0000 7fdda6000640 1 -- 172.21.15.39:0/1409144455 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdda0024050 con 0x560bfa366280 2024-07-04T08:13:45.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.439+0000 7fddc506b740 10 client.15096 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.439+0000 7fddc506b740 1 -- 172.21.15.39:0/1409144455 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bfa745ea0 con 0x560bfa366280 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.440+0000 7fdda6000640 1 -- 172.21.15.39:0/1409144455 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb0003dd0 con 0x560bfa366280 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.440+0000 7fdda6000640 1 -- 172.21.15.39:0/1409144455 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdda0019020 con 0x560bfa366280 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.440+0000 7fddc506b740 10 client.15096 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.440+0000 7fddc506b740 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.440+0000 7fddc506b740 20 client.15096 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.440+0000 7fddc506b740 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560bfa3458e0 0x560bfa749990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.440+0000 7fddc506b740 4 client.15096 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.440+0000 7fddc506b740 1 -- 172.21.15.39:0/1409144455 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bfa6a3bf0 con 0x560bfa3458e0 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.441+0000 7fddbca00640 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560bfa3458e0 0x560bfa749990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.441+0000 7fddbca00640 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560bfa3458e0 0x560bfa749990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11a5968cd622ba server_cookie=205d368c8ddc4e9c in_seq=0 out_seq=0 2024-07-04T08:13:45.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.441+0000 7fdda6000640 10 client.15096 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.442+0000 7fdda6000640 1 -- 172.21.15.39:0/1409144455 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560bfa6a3bf0 con 0x560bfa3458e0 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.442+0000 7fdda6000640 10 client.15096 handle_command_reply: tid=0 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.442+0000 7fddc506b740 1 client.15096 shutdown 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 -- 172.21.15.39:0/1409144455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 msgr2=0x560bfa7377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa7377e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdda000b7e0 tx=0x7fdda000bca0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 -- 172.21.15.39:0/1409144455 shutdown_connections 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560bfa3458e0 0x560bfa749990 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa7377e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa386f60 0x560bfa7431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 --2- 172.21.15.39:0/1409144455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa35e0a0 0x560bfa7372a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 -- 172.21.15.39:0/1409144455 >> 172.21.15.39:0/1409144455 conn(0x560bfa37b350 msgr2=0x560bfa72e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fdda6000640 0 client.15096 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:13:45.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 -- 172.21.15.39:0/1409144455 shutdown_connections 2024-07-04T08:13:45.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 1 -- 172.21.15.39:0/1409144455 wait complete. 2024-07-04T08:13:45.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.443+0000 7fddc506b740 20 client.15096 trim_cache size 0 max 16384 2024-07-04T08:13:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.464+0000 7fddc506b740 1 Processor -- start 2024-07-04T08:13:45.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.464+0000 7fddc506b740 1 -- start start 2024-07-04T08:13:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddc506b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa743110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddc506b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa735920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddc506b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa386f60 0x560bfa735e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddc506b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560bfa747cd0 con 0x560bfa366280 2024-07-04T08:13:45.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddc506b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560bfa7483a0 con 0x560bfa35e0a0 2024-07-04T08:13:45.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddc506b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560bfa748a70 con 0x560bfa386f60 2024-07-04T08:13:45.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddafe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa735920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddbca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa743110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddbca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa743110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57852/0 (socket says 172.21.15.39:57852) 2024-07-04T08:13:45.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddbca00640 1 -- 172.21.15.39:0/3044319193 learned_addr learned my addr 172.21.15.39:0/3044319193 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.465+0000 7fddbca00640 1 --2- 172.21.15.39:0/3044319193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa386f60 0x560bfa735e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.466+0000 7fddbca00640 1 -- 172.21.15.39:0/3044319193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa386f60 msgr2=0x560bfa735e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.466+0000 7fddbca00640 1 --2- 172.21.15.39:0/3044319193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa386f60 0x560bfa735e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.466+0000 7fddbca00640 1 -- 172.21.15.39:0/3044319193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 msgr2=0x560bfa735920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.466+0000 7fddbca00640 1 --2- 172.21.15.39:0/3044319193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa735920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.466+0000 7fddbca00640 1 -- 172.21.15.39:0/3044319193 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfa738360 con 0x560bfa35e0a0 2024-07-04T08:13:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.467+0000 7fddbca00640 1 --2- 172.21.15.39:0/3044319193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa743110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fddb0014040 tx=0x7fddb000f5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a7395a2aafdc208 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:45.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.467+0000 7fddacc00640 1 -- 172.21.15.39:0/3044319193 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb000f6f0 con 0x560bfa35e0a0 2024-07-04T08:13:45.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.467+0000 7fddacc00640 1 -- 172.21.15.39:0/3044319193 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb000fd40 con 0x560bfa35e0a0 2024-07-04T08:13:45.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.467+0000 7fddacc00640 1 -- 172.21.15.39:0/3044319193 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0028450 con 0x560bfa35e0a0 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.468+0000 7fddc506b740 1 -- 172.21.15.39:0/3044319193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 msgr2=0x560bfa743110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.468+0000 7fddc506b740 1 --2- 172.21.15.39:0/3044319193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa743110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fddb0014040 tx=0x7fddb000f5e0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.468+0000 7fddc506b740 1 -- 172.21.15.39:0/3044319193 shutdown_connections 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.468+0000 7fddc506b740 1 --2- 172.21.15.39:0/3044319193 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa366280 0x560bfa735920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.468+0000 7fddc506b740 1 --2- 172.21.15.39:0/3044319193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa386f60 0x560bfa735e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.468+0000 7fddc506b740 1 --2- 172.21.15.39:0/3044319193 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa743110 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.468+0000 7fddc506b740 1 -- 172.21.15.39:0/3044319193 >> 172.21.15.39:0/3044319193 conn(0x560bfa37b350 msgr2=0x560bfa732250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.468+0000 7fddc506b740 1 -- 172.21.15.39:0/3044319193 shutdown_connections 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.468+0000 7fddc506b740 1 -- 172.21.15.39:0/3044319193 wait complete. 2024-07-04T08:13:45.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.469+0000 7fddc506b740 1 -- start start 2024-07-04T08:13:45.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.469+0000 7fddc506b740 1 Processor -- start 2024-07-04T08:13:45.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddc506b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa74bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddc506b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 0x560bfa74c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddc506b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa386f60 0x560bfa74c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddc506b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560bfa749810 con 0x560bfa386f60 2024-07-04T08:13:45.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddc506b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560bfa749140 con 0x560bfa35e0a0 2024-07-04T08:13:45.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddc506b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560bfa749ee0 con 0x560bfa366280 2024-07-04T08:13:45.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddc506b740 10 client.0 _ll_register_callbacks cb 0x7fdda0005c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:13:45.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddbca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa74bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddbca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa74bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57858/0 (socket says 172.21.15.39:57858) 2024-07-04T08:13:45.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddafe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 0x560bfa74c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddbca00640 1 -- 172.21.15.39:0/767099877 learned_addr learned my addr 172.21.15.39:0/767099877 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:45.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.470+0000 7fddbca00640 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa386f60 0x560bfa74c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fddafe00640 1 -- 172.21.15.39:0/767099877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 msgr2=0x560bfa74bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fddafe00640 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa74bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fddafe00640 1 -- 172.21.15.39:0/767099877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa386f60 msgr2=0x560bfa74c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:13:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fddafe00640 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa386f60 0x560bfa74c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fddafe00640 1 -- 172.21.15.39:0/767099877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfa751350 con 0x560bfa366280 2024-07-04T08:13:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fddbca00640 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa386f60 0x560bfa74c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:13:45.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fddafe00640 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 0x560bfa74c080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdda00118b0 tx=0x7fdda0005940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f844c931f6da8f40 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fdda6000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:13:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fdda6000640 1 -- 172.21.15.39:0/767099877 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0011650 con 0x560bfa366280 2024-07-04T08:13:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fdda6000640 1 -- 172.21.15.39:0/767099877 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdda0011b20 con 0x560bfa366280 2024-07-04T08:13:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.471+0000 7fddc506b740 1 -- 172.21.15.39:0/767099877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb000fd40 con 0x560bfa366280 2024-07-04T08:13:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.472+0000 7fdda6000640 1 -- 172.21.15.39:0/767099877 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdda003a050 con 0x560bfa366280 2024-07-04T08:13:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.472+0000 7fddc506b740 10 client.25213 fetch_fsmap learned FSMap version 29 2024-07-04T08:13:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.472+0000 7fddc506b740 1 -- 172.21.15.39:0/767099877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bfa7bb6f0 con 0x560bfa366280 2024-07-04T08:13:45.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.472+0000 7fdda6000640 1 -- 172.21.15.39:0/767099877 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb000fd40 con 0x560bfa366280 2024-07-04T08:13:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.472+0000 7fdda6000640 1 -- 172.21.15.39:0/767099877 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdda0019040 con 0x560bfa366280 2024-07-04T08:13:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.473+0000 7fddc506b740 10 client.25213 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:13:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.473+0000 7fddc506b740 10 client.25213 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T08:13:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.473+0000 7fddc506b740 20 client.25213 populate_metadata read hostname 'smithi039' 2024-07-04T08:13:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.473+0000 7fddc506b740 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560bfa31ac90 0x560bfa7bf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:45.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.473+0000 7fddc506b740 4 client.25213 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.473+0000 7fddc506b740 1 -- 172.21.15.39:0/767099877 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "status"}) -- 0x560bfa69f690 con 0x560bfa31ac90 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.473+0000 7fddbca00640 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560bfa31ac90 0x560bfa7bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddbca00640 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560bfa31ac90 0x560bfa7bf010 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e6cb6d78cfdd3b6 server_cookie=374529d25e658a57 in_seq=0 out_seq=0 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fdda6000640 10 client.25213 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fdda6000640 1 -- 172.21.15.39:0/767099877 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560bfa69f690 con 0x560bfa31ac90 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fdda6000640 10 client.25213 handle_command_reply: tid=0 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddc506b740 1 client.25213 shutdown 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddc506b740 1 -- 172.21.15.39:0/767099877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 msgr2=0x560bfa74c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddc506b740 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 0x560bfa74c080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdda00118b0 tx=0x7fdda0005940 comp rx=0 tx=0).stop 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddc506b740 1 -- 172.21.15.39:0/767099877 shutdown_connections 2024-07-04T08:13:45.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddc506b740 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560bfa31ac90 0x560bfa7bf010 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddc506b740 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560bfa386f60 0x560bfa74c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddc506b740 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560bfa366280 0x560bfa74c080 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddc506b740 1 --2- 172.21.15.39:0/767099877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560bfa35e0a0 0x560bfa74bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fdda6000640 0 client.25213 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:13:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.474+0000 7fddc506b740 1 -- 172.21.15.39:0/767099877 >> 172.21.15.39:0/767099877 conn(0x560bfa37b350 msgr2=0x560bfa732250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.475+0000 7fddc506b740 1 -- 172.21.15.39:0/767099877 shutdown_connections 2024-07-04T08:13:45.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.475+0000 7fddc506b740 1 -- 172.21.15.39:0/767099877 wait complete. 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.475+0000 7fddc506b740 20 client.25213 trim_cache size 0 max 16384 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24707, 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 244.073152816, 2024-07-04T08:13:45.478 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:13:45.479 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:13:45.479 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:13:45.479 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 263.17289613100002 2024-07-04T08:13:45.479 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:13:45.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 -- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd9c0b28c0 msgr2=0x7fdd9c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:45.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd9c0b28c0 0x7fdd9c0b4dd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560bfa72a2f0 tx=0x7fdda802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 -- 172.21.15.39:0/1276266008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 msgr2=0x7fddb819bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 0x7fddb819bc70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdda8005940 tx=0x7fdda8004270 comp rx=0 tx=0).stop 2024-07-04T08:13:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 -- 172.21.15.39:0/1276266008 shutdown_connections 2024-07-04T08:13:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd9c0b28c0 0x7fdd9c0b4dd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb810ac40 0x7fddb819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb810b590 0x7fddb81a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 --2- 172.21.15.39:0/1276266008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fddb8105300 0x7fddb819bc70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 -- 172.21.15.39:0/1276266008 >> 172.21.15.39:0/1276266008 conn(0x7fddb8100d30 msgr2=0x7fddb810a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 -- 172.21.15.39:0/1276266008 shutdown_connections 2024-07-04T08:13:45.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:45.476+0000 7fddbea00640 1 -- 172.21.15.39:0/1276266008 wait complete. 2024-07-04T08:13:46.413 INFO:tasks.cephadm:Setup complete, yielding 2024-07-04T08:13:46.414 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-04T08:13:46.424 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T08:13:46.424 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- bash -c 'ceph orch status' 2024-07-04T08:13:46.908 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:47.061 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:46 smithi039 ceph-mon[32062]: pgmap v437: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:47.061 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:46 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:13:47.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:46 smithi133 ceph-mon[37050]: pgmap v437: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:47.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:46 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:13:47.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:46 smithi050 ceph-mon[36577]: pgmap v437: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:47.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:46 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:13:48.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:47 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:13:48.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:47 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:13:48.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:47 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:13:48.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:47 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:13:48.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:47 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:13:48.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:47 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:13:48.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:47 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:13:48.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:47 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:13:48.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:47 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:13:48.822 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:48 smithi039 ceph-mon[32062]: pgmap v438: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:49.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:48 smithi133 ceph-mon[37050]: pgmap v438: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:48 smithi050 ceph-mon[36577]: pgmap v438: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:49.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.148+0000 7efe07200640 1 -- 172.21.15.39:0/46464373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe08076e90 msgr2=0x7efe080772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:49.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.148+0000 7efe07200640 1 --2- 172.21.15.39:0/46464373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe08076e90 0x7efe080772a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009900 tx=0x7efdf802a400 comp rx=0 tx=0).stop 2024-07-04T08:13:49.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.149+0000 7efe07200640 1 -- 172.21.15.39:0/46464373 shutdown_connections 2024-07-04T08:13:49.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.149+0000 7efe07200640 1 --2- 172.21.15.39:0/46464373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe08113ac0 0x7efe08115ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:49.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.149+0000 7efe07200640 1 --2- 172.21.15.39:0/46464373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe080777e0 0x7efe08113580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:49.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.149+0000 7efe07200640 1 --2- 172.21.15.39:0/46464373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe08076e90 0x7efe080772a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:49.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.149+0000 7efe07200640 1 -- 172.21.15.39:0/46464373 >> 172.21.15.39:0/46464373 conn(0x7efe08100b90 msgr2=0x7efe08102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:49.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.149+0000 7efe07200640 1 -- 172.21.15.39:0/46464373 shutdown_connections 2024-07-04T08:13:49.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.149+0000 7efe07200640 1 -- 172.21.15.39:0/46464373 wait complete. 2024-07-04T08:13:49.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.150+0000 7efe07200640 1 Processor -- start 2024-07-04T08:13:49.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.150+0000 7efe07200640 1 -- start start 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.150+0000 7efe07200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe08076e90 0x7efe0819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.150+0000 7efe07200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe080777e0 0x7efe081a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.150+0000 7efe07200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe08113ac0 0x7efe081a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.150+0000 7efe07200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efe081194a0 con 0x7efe08113ac0 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.150+0000 7efe07200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efe08119320 con 0x7efe08076e90 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.150+0000 7efe07200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efe08119620 con 0x7efe080777e0 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe05e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe08076e90 0x7efe0819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe06800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe08113ac0 0x7efe081a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe05e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe08076e90 0x7efe0819fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57870/0 (socket says 172.21.15.39:57870) 2024-07-04T08:13:49.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe05e00640 1 -- 172.21.15.39:0/4241713957 learned_addr learned my addr 172.21.15.39:0/4241713957 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:49.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe05400640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe080777e0 0x7efe081a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:49.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe05e00640 1 -- 172.21.15.39:0/4241713957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe080777e0 msgr2=0x7efe081a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe05e00640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe080777e0 0x7efe081a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe05e00640 1 -- 172.21.15.39:0/4241713957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe08113ac0 msgr2=0x7efe081a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe05e00640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe08113ac0 0x7efe081a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.151+0000 7efe05e00640 1 -- 172.21.15.39:0/4241713957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe081a5e60 con 0x7efe08076e90 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.152+0000 7efe05e00640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe08076e90 0x7efe0819fef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efdf8005ab0 tx=0x7efdf8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.152+0000 7efdfea00640 1 -- 172.21.15.39:0/4241713957 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf803f070 con 0x7efe08076e90 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.152+0000 7efdfea00640 1 -- 172.21.15.39:0/4241713957 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf8031b60 con 0x7efe08076e90 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.152+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe081a60f0 con 0x7efe08076e90 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.152+0000 7efdfea00640 1 -- 172.21.15.39:0/4241713957 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8035030 con 0x7efe08076e90 2024-07-04T08:13:49.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.152+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe081ac8b0 con 0x7efe08076e90 2024-07-04T08:13:49.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.153+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe080772a0 con 0x7efe08076e90 2024-07-04T08:13:49.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.156+0000 7efdfea00640 1 -- 172.21.15.39:0/4241713957 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7efdf8042070 con 0x7efe08076e90 2024-07-04T08:13:49.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.157+0000 7efdfea00640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdec0b28c0 0x7efdec0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:49.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.158+0000 7efdfea00640 1 -- 172.21.15.39:0/4241713957 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7efdf81034c0 con 0x7efe08076e90 2024-07-04T08:13:49.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.158+0000 7efe05400640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdec0b28c0 0x7efdec0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:49.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.159+0000 7efe05400640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdec0b28c0 0x7efdec0b4dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe08079770 tx=0x7efdf4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:49.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.159+0000 7efdfea00640 1 -- 172.21.15.39:0/4241713957 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7efdf80ca6e0 con 0x7efe08076e90 2024-07-04T08:13:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.389+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7efe0810ecb0 con 0x7efdec0b28c0 2024-07-04T08:13:49.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.390+0000 7efdfea00640 1 -- 172.21.15.39:0/4241713957 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7efe0810ecb0 con 0x7efdec0b28c0 2024-07-04T08:13:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:Backend: cephadm 2024-07-04T08:13:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:Available: Yes 2024-07-04T08:13:49.392 INFO:teuthology.orchestra.run.smithi039.stdout:Paused: No 2024-07-04T08:13:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.393+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdec0b28c0 msgr2=0x7efdec0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.393+0000 7efe07200640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdec0b28c0 0x7efdec0b4dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe08079770 tx=0x7efdf4005f90 comp rx=0 tx=0).stop 2024-07-04T08:13:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.393+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe08076e90 msgr2=0x7efe0819fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.393+0000 7efe07200640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe08076e90 0x7efe0819fef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efdf8005ab0 tx=0x7efdf8003c10 comp rx=0 tx=0).stop 2024-07-04T08:13:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.394+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 shutdown_connections 2024-07-04T08:13:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.394+0000 7efe07200640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdec0b28c0 0x7efdec0b4dd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.394+0000 7efe07200640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe08113ac0 0x7efe081a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.394+0000 7efe07200640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe080777e0 0x7efe081a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.394+0000 7efe07200640 1 --2- 172.21.15.39:0/4241713957 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efe08076e90 0x7efe0819fef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.394+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 >> 172.21.15.39:0/4241713957 conn(0x7efe08100b90 msgr2=0x7efe08115ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.394+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 shutdown_connections 2024-07-04T08:13:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:49.394+0000 7efe07200640 1 -- 172.21.15.39:0/4241713957 wait complete. 2024-07-04T08:13:50.440 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- bash -c 'ceph orch ps' 2024-07-04T08:13:50.938 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:51.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:50 smithi133 ceph-mon[37050]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:13:51.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:50 smithi133 ceph-mon[37050]: pgmap v439: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:51.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:50 smithi039 ceph-mon[32062]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:13:51.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:50 smithi039 ceph-mon[32062]: pgmap v439: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:51.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:50 smithi050 ceph-mon[36577]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:13:51.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:50 smithi050 ceph-mon[36577]: pgmap v439: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:53.069 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:52 smithi133 ceph-mon[37050]: pgmap v440: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:53.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:52 smithi039 ceph-mon[32062]: pgmap v440: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:52 smithi050 ceph-mon[36577]: pgmap v440: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.318+0000 7f3580400640 1 -- 172.21.15.39:0/727224037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f357810b560 msgr2=0x7f3578111d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.318+0000 7f3580400640 1 --2- 172.21.15.39:0/727224037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f357810b560 0x7f3578111d90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3570009ca0 tx=0x7f357002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.318+0000 7f3580400640 1 -- 172.21.15.39:0/727224037 shutdown_connections 2024-07-04T08:13:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.318+0000 7f3580400640 1 --2- 172.21.15.39:0/727224037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f357810b560 0x7f3578111d90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.318+0000 7f3580400640 1 --2- 172.21.15.39:0/727224037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f357810ac10 0x7f357810b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.318+0000 7f3580400640 1 --2- 172.21.15.39:0/727224037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35781052d0 0x7f35781056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.318+0000 7f3580400640 1 -- 172.21.15.39:0/727224037 >> 172.21.15.39:0/727224037 conn(0x7f3578100d60 msgr2=0x7f35781031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.318+0000 7f3580400640 1 -- 172.21.15.39:0/727224037 shutdown_connections 2024-07-04T08:13:53.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.318+0000 7f3580400640 1 -- 172.21.15.39:0/727224037 wait complete. 2024-07-04T08:13:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.319+0000 7f3580400640 1 Processor -- start 2024-07-04T08:13:53.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.319+0000 7f3580400640 1 -- start start 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f3580400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35781052d0 0x7f357819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f3580400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f357810ac10 0x7f357819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f3580400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f357810b560 0x7f35781a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f3580400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3578115230 con 0x7f35781052d0 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f3580400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f35781150b0 con 0x7f357810b560 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f3580400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35781153b0 con 0x7f357810ac10 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f357da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35781052d0 0x7f357819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f357da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35781052d0 0x7f357819bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46470/0 (socket says 172.21.15.39:46470) 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f357da00640 1 -- 172.21.15.39:0/1065410696 learned_addr learned my addr 172.21.15.39:0/1065410696 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.320+0000 7f357d000640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f357810ac10 0x7f357819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:53.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.321+0000 7f357e400640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f357810b560 0x7f35781a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:53.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.321+0000 7f357da00640 1 -- 172.21.15.39:0/1065410696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f357810ac10 msgr2=0x7f357819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:53.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.321+0000 7f357da00640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f357810ac10 0x7f357819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.321+0000 7f357da00640 1 -- 172.21.15.39:0/1065410696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f357810b560 msgr2=0x7f35781a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.321+0000 7f357da00640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f357810b560 0x7f35781a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.321+0000 7f357da00640 1 -- 172.21.15.39:0/1065410696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35781a1bb0 con 0x7f35781052d0 2024-07-04T08:13:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.321+0000 7f357da00640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35781052d0 0x7f357819bc40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f356c00e9f0 tx=0x7f356c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.322+0000 7f3576000640 1 -- 172.21.15.39:0/1065410696 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c00cd80 con 0x7f35781052d0 2024-07-04T08:13:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.322+0000 7f3576000640 1 -- 172.21.15.39:0/1065410696 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356c0054d0 con 0x7f35781052d0 2024-07-04T08:13:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.322+0000 7f3576000640 1 -- 172.21.15.39:0/1065410696 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c010690 con 0x7f35781052d0 2024-07-04T08:13:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.322+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35781a1ea0 con 0x7f35781052d0 2024-07-04T08:13:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.322+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35781a8600 con 0x7f35781052d0 2024-07-04T08:13:53.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.323+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35781056e0 con 0x7f35781052d0 2024-07-04T08:13:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.326+0000 7f3576000640 1 -- 172.21.15.39:0/1065410696 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f356c004e30 con 0x7f35781052d0 2024-07-04T08:13:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.327+0000 7f3576000640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f35600b2840 0x7f35600b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.327+0000 7f3576000640 1 -- 172.21.15.39:0/1065410696 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f356c0e2d30 con 0x7f35781052d0 2024-07-04T08:13:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.328+0000 7f357d000640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f35600b2840 0x7f35600b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:53.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.328+0000 7f357d000640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f35600b2840 0x7f35600b4d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f35640095f0 tx=0x7f3564005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.330+0000 7f3576000640 1 -- 172.21.15.39:0/1065410696 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f356c055110 con 0x7f35781052d0 2024-07-04T08:13:53.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.557+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3578062d50 con 0x7f35600b2840 2024-07-04T08:13:53.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.595+0000 7f3576000640 1 -- 172.21.15.39:0/1065410696 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f3578062d50 con 0x7f35600b2840 2024-07-04T08:13:53.597 INFO:teuthology.orchestra.run.smithi039.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-04T08:13:53.597 INFO:teuthology.orchestra.run.smithi039.stdout:mds.a smithi039 running (4m) 2m ago 4m 86.3M - 19.0.0-4737-g2e9f68b5 ef295368c834 c8d3c9318c52 2024-07-04T08:13:53.597 INFO:teuthology.orchestra.run.smithi039.stdout:mds.b smithi050 running (4m) 2m ago 4m 94.5M - 19.0.0-4737-g2e9f68b5 ef295368c834 4c3b3ec82ea0 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.c smithi133 running (4m) 2m ago 4m 85.7M - 19.0.0-4737-g2e9f68b5 ef295368c834 1cc5e95b4f35 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.d smithi039 running (4m) 2m ago 4m 83.0M - 19.0.0-4737-g2e9f68b5 ef295368c834 6d577a8c2acd 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.e smithi050 running (4m) 2m ago 4m 84.6M - 19.0.0-4737-g2e9f68b5 ef295368c834 9ddac52d00b7 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.f smithi133 running (4m) 2m ago 4m 84.7M - 19.0.0-4737-g2e9f68b5 ef295368c834 0422af43ab13 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.g smithi039 running (4m) 2m ago 4m 80.7M - 19.0.0-4737-g2e9f68b5 ef295368c834 dc3524787faa 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.h smithi050 running (4m) 2m ago 4m 84.8M - 19.0.0-4737-g2e9f68b5 ef295368c834 dab9c6006cf4 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.i smithi133 running (4m) 2m ago 4m 85.1M - 19.0.0-4737-g2e9f68b5 ef295368c834 88c7fd526c4f 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.j smithi039 running (5m) 2m ago 5m 80.8M - 19.0.0-4737-g2e9f68b5 ef295368c834 1504f2280418 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.k smithi050 running (5m) 2m ago 5m 84.7M - 19.0.0-4737-g2e9f68b5 ef295368c834 9628c14682d4 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mds.l smithi133 running (5m) 2m ago 5m 84.3M - 19.0.0-4737-g2e9f68b5 ef295368c834 4a82775ff921 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.x smithi039 *:9283,8765 running (14m) 2m ago 14m 583M - 19.0.0-4737-g2e9f68b5 ef295368c834 fcb3d6dbfbd2 2024-07-04T08:13:53.598 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.y smithi050 *:8443,8765 running (11m) 2m ago 11m 522M - 19.0.0-4737-g2e9f68b5 ef295368c834 71760403a058 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:mgr.z smithi133 *:8443,8765 running (11m) 2m ago 11m 523M - 19.0.0-4737-g2e9f68b5 ef295368c834 4bc09aefe454 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:mon.a smithi039 running (14m) 2m ago 14m 118M 2048M 19.0.0-4737-g2e9f68b5 ef295368c834 c524f94894ef 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:mon.b smithi050 running (12m) 2m ago 12m 112M 2048M 19.0.0-4737-g2e9f68b5 ef295368c834 5233190bdf26 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:mon.c smithi133 running (12m) 2m ago 12m 114M 2048M 19.0.0-4737-g2e9f68b5 ef295368c834 42ab45a0f5c9 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 smithi039 running (10m) 2m ago 10m 112M 4272M 19.0.0-4737-g2e9f68b5 ef295368c834 1be44477711f 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 smithi050 running (10m) 2m ago 10m 116M 4273M 19.0.0-4737-g2e9f68b5 ef295368c834 587b8a7b2ff1 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 smithi133 running (9m) 2m ago 9m 128M 4272M 19.0.0-4737-g2e9f68b5 ef295368c834 87154d8b7e7b 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 smithi039 running (9m) 2m ago 9m 122M 4272M 19.0.0-4737-g2e9f68b5 ef295368c834 f7dc0efe5f50 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 smithi050 running (8m) 2m ago 8m 111M 4273M 19.0.0-4737-g2e9f68b5 ef295368c834 effa0a9709ab 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 smithi133 running (8m) 2m ago 8m 122M 4272M 19.0.0-4737-g2e9f68b5 ef295368c834 69b990624810 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 smithi039 running (7m) 2m ago 7m 110M 4272M 19.0.0-4737-g2e9f68b5 ef295368c834 ba3d86240c22 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 smithi050 running (7m) 2m ago 7m 123M 4273M 19.0.0-4737-g2e9f68b5 ef295368c834 6ead830bfc3d 2024-07-04T08:13:53.599 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 smithi133 running (6m) 2m ago 6m 124M 4272M 19.0.0-4737-g2e9f68b5 ef295368c834 1b4187dc5088 2024-07-04T08:13:53.600 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 smithi039 running (6m) 2m ago 6m 120M 4272M 19.0.0-4737-g2e9f68b5 ef295368c834 e3d35e52eb8e 2024-07-04T08:13:53.600 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 smithi050 running (5m) 2m ago 5m 111M 4273M 19.0.0-4737-g2e9f68b5 ef295368c834 a7adfbf174d7 2024-07-04T08:13:53.600 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 smithi133 running (5m) 2m ago 5m 120M 4272M 19.0.0-4737-g2e9f68b5 ef295368c834 2d09fe9c181a 2024-07-04T08:13:53.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f35600b2840 msgr2=0x7f35600b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f35600b2840 0x7f35600b4d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f35640095f0 tx=0x7f3564005f90 comp rx=0 tx=0).stop 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35781052d0 msgr2=0x7f357819bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35781052d0 0x7f357819bc40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f356c00e9f0 tx=0x7f356c00edf0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 shutdown_connections 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f35600b2840 0x7f35600b4d50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f357810b560 0x7f35781a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f357810ac10 0x7f357819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 --2- 172.21.15.39:0/1065410696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35781052d0 0x7f357819bc40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.598+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 >> 172.21.15.39:0/1065410696 conn(0x7f3578100d60 msgr2=0x7f35781081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.599+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 shutdown_connections 2024-07-04T08:13:53.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:53.599+0000 7f3580400640 1 -- 172.21.15.39:0/1065410696 wait complete. 2024-07-04T08:13:54.488 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- bash -c 'ceph orch ls' 2024-07-04T08:13:54.972 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:55.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:54 smithi133 ceph-mon[37050]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:13:55.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:54 smithi133 ceph-mon[37050]: pgmap v441: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:55.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:54 smithi039 ceph-mon[32062]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:13:55.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:54 smithi039 ceph-mon[32062]: pgmap v441: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:55.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:54 smithi050 ceph-mon[36577]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:13:55.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:54 smithi050 ceph-mon[36577]: pgmap v441: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:13:56.889 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:56 smithi039 ceph-mon[32062]: pgmap v442: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:57.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:56 smithi133 ceph-mon[37050]: pgmap v442: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:57.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:56 smithi050 ceph-mon[36577]: pgmap v442: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:13:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.221+0000 7fc715400640 1 -- 172.21.15.39:0/4224489975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71010f7a0 msgr2=0x7fc710115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.222+0000 7fc715400640 1 --2- 172.21.15.39:0/4224489975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71010f7a0 0x7fc710115fd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc708009be0 tx=0x7fc70802a6c0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.222+0000 7fc715400640 1 -- 172.21.15.39:0/4224489975 shutdown_connections 2024-07-04T08:13:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.222+0000 7fc715400640 1 --2- 172.21.15.39:0/4224489975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc71010ee50 0x7fc71010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.222+0000 7fc715400640 1 --2- 172.21.15.39:0/4224489975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc71010f7a0 0x7fc710115fd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.222+0000 7fc715400640 1 --2- 172.21.15.39:0/4224489975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc710109510 0x7fc710109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.222+0000 7fc715400640 1 -- 172.21.15.39:0/4224489975 >> 172.21.15.39:0/4224489975 conn(0x7fc710078b10 msgr2=0x7fc710107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.222+0000 7fc715400640 1 -- 172.21.15.39:0/4224489975 shutdown_connections 2024-07-04T08:13:57.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.223+0000 7fc715400640 1 -- 172.21.15.39:0/4224489975 wait complete. 2024-07-04T08:13:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.224+0000 7fc715400640 1 Processor -- start 2024-07-04T08:13:57.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.224+0000 7fc715400640 1 -- start start 2024-07-04T08:13:57.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc715400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc710109510 0x7fc71019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:57.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc715400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71010ee50 0x7fc7101a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:57.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc715400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc71010f7a0 0x7fc7101a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:57.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc715400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc710119450 con 0x7fc71010f7a0 2024-07-04T08:13:57.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc715400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101192d0 con 0x7fc71010ee50 2024-07-04T08:13:57.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc715400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101195d0 con 0x7fc710109510 2024-07-04T08:13:57.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc707e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71010ee50 0x7fc7101a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:57.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc707e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71010ee50 0x7fc7101a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58488/0 (socket says 172.21.15.39:58488) 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc707e00640 1 -- 172.21.15.39:0/1560754951 learned_addr learned my addr 172.21.15.39:0/1560754951 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc70fe00640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc710109510 0x7fc71019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc714a00640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc71010f7a0 0x7fc7101a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc707e00640 1 -- 172.21.15.39:0/1560754951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc710109510 msgr2=0x7fc71019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.225+0000 7fc707e00640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc710109510 0x7fc71019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.226+0000 7fc707e00640 1 -- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc71010f7a0 msgr2=0x7fc7101a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.226+0000 7fc707e00640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc71010f7a0 0x7fc7101a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.226+0000 7fc707e00640 1 -- 172.21.15.39:0/1560754951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7101a5d30 con 0x7fc71010ee50 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.226+0000 7fc707e00640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71010ee50 0x7fc7101a0360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800e9c0 tx=0x7fc6f800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.226+0000 7fc70d600640 1 -- 172.21.15.39:0/1560754951 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800ccc0 con 0x7fc71010ee50 2024-07-04T08:13:57.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.226+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7101a6020 con 0x7fc71010ee50 2024-07-04T08:13:57.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.226+0000 7fc70d600640 1 -- 172.21.15.39:0/1560754951 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6f80108e0 con 0x7fc71010ee50 2024-07-04T08:13:57.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.226+0000 7fc70d600640 1 -- 172.21.15.39:0/1560754951 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8018730 con 0x7fc71010ee50 2024-07-04T08:13:57.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.226+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101ac780 con 0x7fc71010ee50 2024-07-04T08:13:57.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.227+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc710109920 con 0x7fc71010ee50 2024-07-04T08:13:57.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.230+0000 7fc70d600640 1 -- 172.21.15.39:0/1560754951 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fc6f8010430 con 0x7fc71010ee50 2024-07-04T08:13:57.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.232+0000 7fc70d600640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6f40b28c0 0x7fc6f40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:13:57.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.232+0000 7fc70d600640 1 -- 172.21.15.39:0/1560754951 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc6f80e2da0 con 0x7fc71010ee50 2024-07-04T08:13:57.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.232+0000 7fc70fe00640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6f40b28c0 0x7fc6f40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:13:57.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.233+0000 7fc70fe00640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6f40b28c0 0x7fc6f40b4dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7000095f0 tx=0x7fc700005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:13:57.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.233+0000 7fc70d600640 1 -- 172.21.15.39:0/1560754951 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc6f80a9fc0 con 0x7fc71010ee50 2024-07-04T08:13:57.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.457+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fc710062d50 con 0x7fc6f40b28c0 2024-07-04T08:13:57.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.466+0000 7fc70d600640 1 -- 172.21.15.39:0/1560754951 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fc710062d50 con 0x7fc6f40b28c0 2024-07-04T08:13:57.467 INFO:teuthology.orchestra.run.smithi039.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-07-04T08:13:57.467 INFO:teuthology.orchestra.run.smithi039.stdout:mds.all 12/12 2m ago 5m smithi039=a;smithi039=d;smithi039=g;smithi039=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-07-04T08:13:57.467 INFO:teuthology.orchestra.run.smithi039.stdout:mgr 3/3 2m ago 12m smithi039=x;smithi050=y;smithi133=z;count:3 2024-07-04T08:13:57.467 INFO:teuthology.orchestra.run.smithi039.stdout:mon 3/3 2m ago 12m smithi039:172.21.15.39=a;smithi050:172.21.15.50=b;smithi133:172.21.15.133=c;count:3 2024-07-04T08:13:57.467 INFO:teuthology.orchestra.run.smithi039.stdout:osd 12 2m ago - 2024-07-04T08:13:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6f40b28c0 msgr2=0x7fc6f40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6f40b28c0 0x7fc6f40b4dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7000095f0 tx=0x7fc700005f90 comp rx=0 tx=0).stop 2024-07-04T08:13:57.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71010ee50 msgr2=0x7fc7101a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:13:57.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71010ee50 0x7fc7101a0360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800e9c0 tx=0x7fc6f800ee80 comp rx=0 tx=0).stop 2024-07-04T08:13:57.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 shutdown_connections 2024-07-04T08:13:57.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6f40b28c0 0x7fc6f40b4dd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc71010f7a0 0x7fc7101a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc71010ee50 0x7fc7101a0360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 --2- 172.21.15.39:0/1560754951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc710109510 0x7fc71019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:13:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 >> 172.21.15.39:0/1560754951 conn(0x7fc710078b10 msgr2=0x7fc71010c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:13:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.469+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 shutdown_connections 2024-07-04T08:13:57.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:13:57.470+0000 7fc715400640 1 -- 172.21.15.39:0/1560754951 wait complete. 2024-07-04T08:13:58.370 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- bash -c 'ceph orch host ls' 2024-07-04T08:13:58.866 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:13:59.052 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:58 smithi039 ceph-mon[32062]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:13:59.052 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:13:58 smithi039 ceph-mon[32062]: pgmap v443: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:59.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:58 smithi133 ceph-mon[37050]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:13:59.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:13:58 smithi133 ceph-mon[37050]: pgmap v443: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:13:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:58 smithi050 ceph-mon[36577]: from='client.25244 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:13:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:13:58 smithi050 ceph-mon[36577]: pgmap v443: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:00.939 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:00 smithi039 ceph-mon[32062]: pgmap v444: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:01.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:00 smithi133 ceph-mon[37050]: pgmap v444: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:01.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:00 smithi050 ceph-mon[36577]: pgmap v444: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.156+0000 7f382f200640 1 -- 172.21.15.39:0/1477269753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38281094e0 msgr2=0x7f38281098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.156+0000 7f382f200640 1 --2- 172.21.15.39:0/1477269753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38281094e0 0x7f38281098f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f381c009900 tx=0x7f381c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.156+0000 7f382f200640 1 -- 172.21.15.39:0/1477269753 shutdown_connections 2024-07-04T08:14:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.156+0000 7f382f200640 1 --2- 172.21.15.39:0/1477269753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f382810f770 0x7f3828115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.156+0000 7f382f200640 1 --2- 172.21.15.39:0/1477269753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f382810ee20 0x7f382810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.156+0000 7f382f200640 1 --2- 172.21.15.39:0/1477269753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38281094e0 0x7f38281098f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.156+0000 7f382f200640 1 -- 172.21.15.39:0/1477269753 >> 172.21.15.39:0/1477269753 conn(0x7f3828078ad0 msgr2=0x7f3828107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.157+0000 7f382f200640 1 -- 172.21.15.39:0/1477269753 shutdown_connections 2024-07-04T08:14:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.157+0000 7f382f200640 1 -- 172.21.15.39:0/1477269753 wait complete. 2024-07-04T08:14:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.158+0000 7f382f200640 1 Processor -- start 2024-07-04T08:14:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.158+0000 7f382f200640 1 -- start start 2024-07-04T08:14:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.158+0000 7f382f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38281094e0 0x7f382819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.158+0000 7f382f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f382810ee20 0x7f38281a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.158+0000 7f382f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f382810f770 0x7f38281a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.158+0000 7f382f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3828119440 con 0x7f382810f770 2024-07-04T08:14:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.158+0000 7f382f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f38281192c0 con 0x7f38281094e0 2024-07-04T08:14:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.158+0000 7f382f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38281195c0 con 0x7f382810ee20 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f382810f770 0x7f38281a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f382810ee20 0x7f38281a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38281094e0 0x7f382819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f382810f770 0x7f38281a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46542/0 (socket says 172.21.15.39:46542) 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382e800640 1 -- 172.21.15.39:0/2423411536 learned_addr learned my addr 172.21.15.39:0/2423411536 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f382810ee20 0x7f38281a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50392/0 (socket says 172.21.15.39:50392) 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382d400640 1 -- 172.21.15.39:0/2423411536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38281094e0 msgr2=0x7f382819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382d400640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38281094e0 0x7f382819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382d400640 1 -- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f382810f770 msgr2=0x7f38281a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382d400640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f382810f770 0x7f38281a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382d400640 1 -- 172.21.15.39:0/2423411536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38281a5da0 con 0x7f382810ee20 2024-07-04T08:14:01.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.159+0000 7f382e800640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f382810f770 0x7f38281a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:01.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.160+0000 7f382d400640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f382810ee20 0x7f38281a03d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f382806ae70 tx=0x7f381800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:01.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.160+0000 7f3822a00640 1 -- 172.21.15.39:0/2423411536 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381800cc40 con 0x7f382810ee20 2024-07-04T08:14:01.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.160+0000 7f3822a00640 1 -- 172.21.15.39:0/2423411536 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f381800cde0 con 0x7f382810ee20 2024-07-04T08:14:01.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.160+0000 7f3822a00640 1 -- 172.21.15.39:0/2423411536 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818004a00 con 0x7f382810ee20 2024-07-04T08:14:01.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.160+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38281a6090 con 0x7f382810ee20 2024-07-04T08:14:01.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.160+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38281ac7f0 con 0x7f382810ee20 2024-07-04T08:14:01.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.161+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38281098f0 con 0x7f382810ee20 2024-07-04T08:14:01.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.164+0000 7f3822a00640 1 -- 172.21.15.39:0/2423411536 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f3818004ba0 con 0x7f382810ee20 2024-07-04T08:14:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.166+0000 7f3822a00640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f38100b28c0 0x7f38100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:01.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.166+0000 7f3822a00640 1 -- 172.21.15.39:0/2423411536 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f38180e26c0 con 0x7f382810ee20 2024-07-04T08:14:01.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.166+0000 7f382de00640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f38100b28c0 0x7f38100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:01.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.167+0000 7f382de00640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f38100b28c0 0x7f38100b4dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f381c031440 tx=0x7f381c009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:01.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.167+0000 7f3822a00640 1 -- 172.21.15.39:0/2423411536 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f38180a9fc0 con 0x7f382810ee20 2024-07-04T08:14:01.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.387+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3828062d50 con 0x7f38100b28c0 2024-07-04T08:14:01.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.390+0000 7f3822a00640 1 -- 172.21.15.39:0/2423411536 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3828062d50 con 0x7f38100b28c0 2024-07-04T08:14:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:HOST ADDR LABELS STATUS 2024-07-04T08:14:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 172.21.15.39 2024-07-04T08:14:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:smithi050 172.21.15.50 2024-07-04T08:14:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:smithi133 172.21.15.133 2024-07-04T08:14:01.392 INFO:teuthology.orchestra.run.smithi039.stdout:3 hosts in cluster 2024-07-04T08:14:01.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f38100b28c0 msgr2=0x7f38100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f38100b28c0 0x7f38100b4dd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f381c031440 tx=0x7f381c009560 comp rx=0 tx=0).stop 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f382810ee20 msgr2=0x7f38281a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f382810ee20 0x7f38281a03d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f382806ae70 tx=0x7f381800ee50 comp rx=0 tx=0).stop 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 shutdown_connections 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f38100b28c0 0x7f38100b4dd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f382810f770 0x7f38281a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f382810ee20 0x7f38281a03d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 --2- 172.21.15.39:0/2423411536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38281094e0 0x7f382819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.393+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 >> 172.21.15.39:0/2423411536 conn(0x7f3828078ad0 msgr2=0x7f38280793f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.394+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 shutdown_connections 2024-07-04T08:14:01.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:01.394+0000 7f382f200640 1 -- 172.21.15.39:0/2423411536 wait complete. 2024-07-04T08:14:02.166 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- bash -c 'ceph orch device ls' 2024-07-04T08:14:02.660 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:03.043 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:02 smithi039 ceph-mon[32062]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:14:03.044 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:02 smithi039 ceph-mon[32062]: pgmap v445: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:03.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:02 smithi133 ceph-mon[37050]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:14:03.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:02 smithi133 ceph-mon[37050]: pgmap v445: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:02 smithi050 ceph-mon[36577]: from='client.25237 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:14:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:02 smithi050 ceph-mon[36577]: pgmap v445: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:04.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.964+0000 7f3cab800640 1 -- 172.21.15.39:0/1212710023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca4105300 msgr2=0x7f3ca4105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:04.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.964+0000 7f3cab800640 1 --2- 172.21.15.39:0/1212710023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca4105300 0x7f3ca4105710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009ca0 tx=0x7f3c9402a8b0 comp rx=0 tx=0).stop 2024-07-04T08:14:04.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.965+0000 7f3cab800640 1 -- 172.21.15.39:0/1212710023 shutdown_connections 2024-07-04T08:14:04.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.965+0000 7f3cab800640 1 --2- 172.21.15.39:0/1212710023 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410ac40 0x7f3ca410b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:04.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.965+0000 7f3cab800640 1 --2- 172.21.15.39:0/1212710023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca410b590 0x7f3ca4111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:04.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.965+0000 7f3cab800640 1 --2- 172.21.15.39:0/1212710023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca4105300 0x7f3ca4105710 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:04.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.965+0000 7f3cab800640 1 -- 172.21.15.39:0/1212710023 >> 172.21.15.39:0/1212710023 conn(0x7f3ca4100d10 msgr2=0x7f3ca4103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:04.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.965+0000 7f3cab800640 1 -- 172.21.15.39:0/1212710023 shutdown_connections 2024-07-04T08:14:04.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.965+0000 7f3cab800640 1 -- 172.21.15.39:0/1212710023 wait complete. 2024-07-04T08:14:04.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.966+0000 7f3cab800640 1 Processor -- start 2024-07-04T08:14:04.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.966+0000 7f3cab800640 1 -- start start 2024-07-04T08:14:04.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.966+0000 7f3cab800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca4105300 0x7f3ca419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:04.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.966+0000 7f3cab800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410ac40 0x7f3ca419c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:04.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.966+0000 7f3cab800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b590 0x7f3ca41a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:04.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3cab800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4115290 con 0x7f3ca410ac40 2024-07-04T08:14:04.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3cab800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4115110 con 0x7f3ca410b590 2024-07-04T08:14:04.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3cab800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4115410 con 0x7f3ca4105300 2024-07-04T08:14:04.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca4105300 0x7f3ca419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:04.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca4105300 0x7f3ca419bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50418/0 (socket says 172.21.15.39:50418) 2024-07-04T08:14:04.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410ac40 0x7f3ca419c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:04.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca8e00640 1 -- 172.21.15.39:0/267209725 learned_addr learned my addr 172.21.15.39:0/267209725 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:04.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca9800640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b590 0x7f3ca41a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca8e00640 1 -- 172.21.15.39:0/267209725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b590 msgr2=0x7f3ca41a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca8e00640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b590 0x7f3ca41a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca8e00640 1 -- 172.21.15.39:0/267209725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410ac40 msgr2=0x7f3ca419c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca8e00640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410ac40 0x7f3ca419c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.967+0000 7f3ca8e00640 1 -- 172.21.15.39:0/267209725 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca41a1c10 con 0x7f3ca4105300 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.968+0000 7f3ca8e00640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca4105300 0x7f3ca419bca0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3c94005e30 tx=0x7f3c94030f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.968+0000 7f3ca1600640 1 -- 172.21.15.39:0/267209725 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94004040 con 0x7f3ca4105300 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.968+0000 7f3ca1600640 1 -- 172.21.15.39:0/267209725 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c94040070 con 0x7f3ca4105300 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.968+0000 7f3ca1600640 1 -- 172.21.15.39:0/267209725 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94031a70 con 0x7f3ca4105300 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.968+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca41a1ea0 con 0x7f3ca4105300 2024-07-04T08:14:04.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.968+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca41a8660 con 0x7f3ca4105300 2024-07-04T08:14:04.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.969+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca4105710 con 0x7f3ca4105300 2024-07-04T08:14:04.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.972+0000 7f3ca1600640 1 -- 172.21.15.39:0/267209725 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f3c94004b40 con 0x7f3ca4105300 2024-07-04T08:14:04.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.974+0000 7f3ca1600640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b28c0 0x7f3c880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:04.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.974+0000 7f3ca1600640 1 -- 172.21.15.39:0/267209725 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f3c941038c0 con 0x7f3ca4105300 2024-07-04T08:14:04.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.974+0000 7f3ca3e00640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b28c0 0x7f3c880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:04.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.975+0000 7f3ca3e00640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b28c0 0x7f3c880b4dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4103580 tx=0x7f3c90005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:04.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:04.975+0000 7f3ca1600640 1 -- 172.21.15.39:0/267209725 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f3c940cb1c0 con 0x7f3ca4105300 2024-07-04T08:14:05.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:04 smithi133 ceph-mon[37050]: pgmap v446: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:05.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:04 smithi039 ceph-mon[32062]: pgmap v446: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:05.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:04 smithi050 ceph-mon[36577]: pgmap v446: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.202+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f3ca4109b20 con 0x7f3c880b28c0 2024-07-04T08:14:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.206+0000 7f3ca1600640 1 -- 172.21.15.39:0/267209725 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f3ca4109b20 con 0x7f3c880b28c0 2024-07-04T08:14:05.207 INFO:teuthology.orchestra.run.smithi039.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-07-04T08:14:05.207 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102QX4P0KGN 3726G No 6m ago Has a FileSystem, LVM detected 2024-07-04T08:14:05.207 INFO:teuthology.orchestra.run.smithi039.stdout:smithi050 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001F480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-07-04T08:14:05.207 INFO:teuthology.orchestra.run.smithi039.stdout:smithi133 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401AQ400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-07-04T08:14:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.208+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b28c0 msgr2=0x7f3c880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.208+0000 7f3cab800640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b28c0 0x7f3c880b4dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4103580 tx=0x7f3c90005f90 comp rx=0 tx=0).stop 2024-07-04T08:14:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.208+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca4105300 msgr2=0x7f3ca419bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.208+0000 7f3cab800640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca4105300 0x7f3ca419bca0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3c94005e30 tx=0x7f3c94030f70 comp rx=0 tx=0).stop 2024-07-04T08:14:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.209+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 shutdown_connections 2024-07-04T08:14:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.209+0000 7f3cab800640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b28c0 0x7f3c880b4dd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.209+0000 7f3cab800640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410ac40 0x7f3ca419c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.209+0000 7f3cab800640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b590 0x7f3ca41a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.209+0000 7f3cab800640 1 --2- 172.21.15.39:0/267209725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca4105300 0x7f3ca419bca0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.209+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 >> 172.21.15.39:0/267209725 conn(0x7f3ca4100d10 msgr2=0x7f3ca4109410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.209+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 shutdown_connections 2024-07-04T08:14:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:05.209+0000 7f3cab800640 1 -- 172.21.15.39:0/267209725 wait complete. 2024-07-04T08:14:06.132 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-04T08:14:06.141 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T08:14:06.141 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-04T08:14:06.652 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:07.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:06 smithi133 ceph-mon[37050]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:14:07.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:06 smithi133 ceph-mon[37050]: pgmap v447: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:07.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:06 smithi039 ceph-mon[32062]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:14:07.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:06 smithi039 ceph-mon[32062]: pgmap v447: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:07.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:06 smithi050 ceph-mon[36577]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:14:07.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:06 smithi050 ceph-mon[36577]: pgmap v447: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:08.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.827+0000 7f523a600640 1 -- 172.21.15.39:0/1717928306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523410f7a0 msgr2=0x7f5234115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:08.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.827+0000 7f523a600640 1 --2- 172.21.15.39:0/1717928306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523410f7a0 0x7f5234115fd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f522c009ca0 tx=0x7f522c02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:14:08.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.828+0000 7f523a600640 1 -- 172.21.15.39:0/1717928306 shutdown_connections 2024-07-04T08:14:08.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.828+0000 7f523a600640 1 --2- 172.21.15.39:0/1717928306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523410f7a0 0x7f5234115fd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:08.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.828+0000 7f523a600640 1 --2- 172.21.15.39:0/1717928306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523410ee50 0x7f523410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:08.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.828+0000 7f523a600640 1 --2- 172.21.15.39:0/1717928306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5234109510 0x7f5234109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:08.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.828+0000 7f523a600640 1 -- 172.21.15.39:0/1717928306 >> 172.21.15.39:0/1717928306 conn(0x7f5234078b10 msgr2=0x7f5234107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:08.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.828+0000 7f523a600640 1 -- 172.21.15.39:0/1717928306 shutdown_connections 2024-07-04T08:14:08.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.828+0000 7f523a600640 1 -- 172.21.15.39:0/1717928306 wait complete. 2024-07-04T08:14:08.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.829+0000 7f523a600640 1 Processor -- start 2024-07-04T08:14:08.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.829+0000 7f523a600640 1 -- start start 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.829+0000 7f523a600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5234109510 0x7f523419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f523a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523410ee50 0x7f52341a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f523a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523410f7a0 0x7f52341a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f523a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5234119470 con 0x7f523410f7a0 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f523a600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52341192f0 con 0x7f5234109510 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f523a600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52341195f0 con 0x7f523410ee50 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5234109510 0x7f523419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5234109510 0x7f523419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51598/0 (socket says 172.21.15.39:51598) 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239200640 1 -- 172.21.15.39:0/1435118517 learned_addr learned my addr 172.21.15.39:0/1435118517 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5233e00640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523410ee50 0x7f52341a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:08.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239c00640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523410f7a0 0x7f52341a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:08.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239200640 1 -- 172.21.15.39:0/1435118517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523410ee50 msgr2=0x7f52341a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:08.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239200640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523410ee50 0x7f52341a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:08.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239200640 1 -- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523410f7a0 msgr2=0x7f52341a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:08.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239200640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523410f7a0 0x7f52341a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:08.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239200640 1 -- 172.21.15.39:0/1435118517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52341a5d10 con 0x7f5234109510 2024-07-04T08:14:08.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.830+0000 7f5239c00640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523410f7a0 0x7f52341a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:08.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.831+0000 7f5239200640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5234109510 0x7f523419fda0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f522400e9f0 tx=0x7f522400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:08.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.831+0000 7f5231600640 1 -- 172.21.15.39:0/1435118517 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522400cd80 con 0x7f5234109510 2024-07-04T08:14:08.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.831+0000 7f5231600640 1 -- 172.21.15.39:0/1435118517 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52240108e0 con 0x7f5234109510 2024-07-04T08:14:08.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.831+0000 7f5231600640 1 -- 172.21.15.39:0/1435118517 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224018980 con 0x7f5234109510 2024-07-04T08:14:08.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.831+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52341a6000 con 0x7f5234109510 2024-07-04T08:14:08.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.831+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341ac760 con 0x7f5234109510 2024-07-04T08:14:08.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.833+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5234109920 con 0x7f5234109510 2024-07-04T08:14:08.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.835+0000 7f5231600640 1 -- 172.21.15.39:0/1435118517 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f5224010430 con 0x7f5234109510 2024-07-04T08:14:08.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.836+0000 7f5231600640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f52180b2840 0x7f52180b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:08.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.837+0000 7f5231600640 1 -- 172.21.15.39:0/1435118517 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f52240e2af0 con 0x7f5234109510 2024-07-04T08:14:08.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.837+0000 7f5233e00640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f52180b2840 0x7f52180b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:08.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.837+0000 7f5233e00640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f52180b2840 0x7f52180b4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f523407ad50 tx=0x7f5220005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:08.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:08.841+0000 7f5231600640 1 -- 172.21.15.39:0/1435118517 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f52240aa360 con 0x7f5234109510 2024-07-04T08:14:09.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:08 smithi133 ceph-mon[37050]: pgmap v448: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:09.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.099+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f523402de10 con 0x7f5234109510 2024-07-04T08:14:09.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:08 smithi039 ceph-mon[32062]: pgmap v448: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:09.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.100+0000 7f5231600640 1 -- 172.21.15.39:0/1435118517 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3130 (secure 0 0 0) 0x7f52240a9c60 con 0x7f5234109510 2024-07-04T08:14:09.103 INFO:teuthology.orchestra.run.smithi039.stdout:e29 2024-07-04T08:14:09.103 INFO:teuthology.orchestra.run.smithi039.stdout:btime 2024-07-04T08:11:09:862205+0000 2024-07-04T08:14:09.103 INFO:teuthology.orchestra.run.smithi039.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-04T08:14:09.103 INFO:teuthology.orchestra.run.smithi039.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-04T08:14:09.103 INFO:teuthology.orchestra.run.smithi039.stdout:legacy client fscid: 1 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:Filesystem 'cephfs' (1) 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:fs_name cephfs 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 29 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-07-04T08:09:41.252924+0000 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-07-04T08:11:09.668485+0000 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:tableserver 0 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:root 0 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:session_timeout 300 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:session_autoclose 300 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:max_file_size 1099511627776 2024-07-04T08:14:09.104 INFO:teuthology.orchestra.run.smithi039.stdout:max_xattr_size 65536 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:required_client_features {} 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure 0 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:last_failure_osd_epoch 0 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:max_mds 5 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:in 0,1,2,3,4 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:up {0=24707,1=24701,2=24679,3=24674,4=24673} 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:failed 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:damaged 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:stopped 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:data_pools [3,4] 2024-07-04T08:14:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:metadata_pool 2 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:inline_data disabled 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:balancer 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:bal_rank_mask -1 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:standby_count_wanted 0 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:qdb_cluster leader: 24707 members: 24673,24674,24679,24701,24707 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b{0:24707} state up:active seq 6 addr [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.f{0:24661} state up:standby-replay seq 1 addr [v2:172.21.15.133:6834/2187182943,v1:172.21.15.133:6835/2187182943] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a{1:24701} state up:active seq 27 addr [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.e{1:24656} state up:standby-replay seq 1 addr [v2:172.21.15.50:6834/2710764279,v1:172.21.15.50:6835/2710764279] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.i{2:24679} state up:active seq 30 addr [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d{2:24644} state up:standby-replay seq 1 addr [v2:172.21.15.39:6836/1240380045,v1:172.21.15.39:6837/1240380045] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.h{3:24674} state up:active seq 31 addr [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.106 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.k{3:24626} state up:standby-replay seq 1 addr [v2:172.21.15.50:6832/264080488,v1:172.21.15.50:6833/264080488] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.107 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c{4:24673} state up:active seq 32 addr [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.107 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.l{4:24619} state up:standby-replay seq 1 addr [v2:172.21.15.133:6832/2411107381,v1:172.21.15.133:6833/2411107381] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:14:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:14:09.107 INFO:teuthology.orchestra.run.smithi039.stdout:Standby daemons: 2024-07-04T08:14:09.107 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:14:09.107 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.j{-1:14529} state up:standby seq 1 addr [v2:172.21.15.39:6834/1612340282,v1:172.21.15.39:6835/1612340282] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.107 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.g{-1:14565} state up:standby seq 1 addr [v2:172.21.15.39:6838/2419136306,v1:172.21.15.39:6839/2419136306] compat {c=[1],r=[1],i=[1fff]}] 2024-07-04T08:14:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-07-04T08:14:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.104+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f52180b2840 msgr2=0x7f52180b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.104+0000 7f523a600640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f52180b2840 0x7f52180b4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f523407ad50 tx=0x7f5220005f90 comp rx=0 tx=0).stop 2024-07-04T08:14:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.104+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5234109510 msgr2=0x7f523419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.104+0000 7f523a600640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5234109510 0x7f523419fda0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f522400e9f0 tx=0x7f522400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:14:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.105+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 shutdown_connections 2024-07-04T08:14:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.105+0000 7f523a600640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f52180b2840 0x7f52180b4d50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.105+0000 7f523a600640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f523410f7a0 0x7f52341a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.105+0000 7f523a600640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f523410ee50 0x7f52341a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.105+0000 7f523a600640 1 --2- 172.21.15.39:0/1435118517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5234109510 0x7f523419fda0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.105+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 >> 172.21.15.39:0/1435118517 conn(0x7f5234078b10 msgr2=0x7f5234079130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.105+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 shutdown_connections 2024-07-04T08:14:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:09.105+0000 7f523a600640 1 -- 172.21.15.39:0/1435118517 wait complete. 2024-07-04T08:14:09.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:08 smithi050 ceph-mon[36577]: pgmap v448: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:10.065 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- bash -c 'ceph osd dump' 2024-07-04T08:14:10.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:09 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1435118517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-04T08:14:10.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:09 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1435118517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-04T08:14:10.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:09 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1435118517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-04T08:14:10.551 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:11.059 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:10 smithi039 ceph-mon[32062]: pgmap v449: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:11.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:10 smithi050 ceph-mon[36577]: pgmap v449: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:11.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:10 smithi133 ceph-mon[37050]: pgmap v449: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:12.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.906+0000 7fbd89400640 1 -- 172.21.15.39:0/3783088423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd8410ee50 msgr2=0x7fbd8410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:12.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.906+0000 7fbd89400640 1 --2- 172.21.15.39:0/3783088423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd8410ee50 0x7fbd8410f260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009860 tx=0x7fbd7002a340 comp rx=0 tx=0).stop 2024-07-04T08:14:12.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.906+0000 7fbd89400640 1 -- 172.21.15.39:0/3783088423 shutdown_connections 2024-07-04T08:14:12.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.906+0000 7fbd89400640 1 --2- 172.21.15.39:0/3783088423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd8410f7a0 0x7fbd84115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:12.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.906+0000 7fbd89400640 1 --2- 172.21.15.39:0/3783088423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd8410ee50 0x7fbd8410f260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:12.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.906+0000 7fbd89400640 1 --2- 172.21.15.39:0/3783088423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd84109510 0x7fbd84109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:12.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.906+0000 7fbd89400640 1 -- 172.21.15.39:0/3783088423 >> 172.21.15.39:0/3783088423 conn(0x7fbd84078b10 msgr2=0x7fbd84107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:12.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.906+0000 7fbd89400640 1 -- 172.21.15.39:0/3783088423 shutdown_connections 2024-07-04T08:14:12.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.906+0000 7fbd89400640 1 -- 172.21.15.39:0/3783088423 wait complete. 2024-07-04T08:14:12.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.907+0000 7fbd89400640 1 Processor -- start 2024-07-04T08:14:12.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.907+0000 7fbd89400640 1 -- start start 2024-07-04T08:14:12.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd89400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd84109510 0x7fbd8419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:12.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd89400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd8410ee50 0x7fbd841a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:12.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd89400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd8410f7a0 0x7fbd841a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:12.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd89400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbd84119410 con 0x7fbd8410f7a0 2024-07-04T08:14:12.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd89400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd84119290 con 0x7fbd8410ee50 2024-07-04T08:14:12.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd89400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbd84119590 con 0x7fbd84109510 2024-07-04T08:14:12.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd88a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd8410f7a0 0x7fbd841a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:12.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd88a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd8410f7a0 0x7fbd841a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56090/0 (socket says 172.21.15.39:56090) 2024-07-04T08:14:12.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd88a00640 1 -- 172.21.15.39:0/900086242 learned_addr learned my addr 172.21.15.39:0/900086242 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:12.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.908+0000 7fbd83400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd8410ee50 0x7fbd841a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:12.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd83e00640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd84109510 0x7fbd8419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:12.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd88a00640 1 -- 172.21.15.39:0/900086242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd84109510 msgr2=0x7fbd8419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:12.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd88a00640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd84109510 0x7fbd8419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:12.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd88a00640 1 -- 172.21.15.39:0/900086242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd8410ee50 msgr2=0x7fbd841a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:12.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd88a00640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd8410ee50 0x7fbd841a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:12.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd88a00640 1 -- 172.21.15.39:0/900086242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd841a5db0 con 0x7fbd8410f7a0 2024-07-04T08:14:12.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd83e00640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd84109510 0x7fbd8419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:14:12.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd88a00640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd8410f7a0 0x7fbd841a5680 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00efc0 tx=0x7fbd7c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:12.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd80c00640 1 -- 172.21.15.39:0/900086242 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c003970 con 0x7fbd8410f7a0 2024-07-04T08:14:12.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd841a60a0 con 0x7fbd8410f7a0 2024-07-04T08:14:12.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd80c00640 1 -- 172.21.15.39:0/900086242 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd7c004020 con 0x7fbd8410f7a0 2024-07-04T08:14:12.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd80c00640 1 -- 172.21.15.39:0/900086242 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c00b940 con 0x7fbd8410f7a0 2024-07-04T08:14:12.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.909+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd841ac800 con 0x7fbd8410f7a0 2024-07-04T08:14:12.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.911+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd84109920 con 0x7fbd8410f7a0 2024-07-04T08:14:12.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.913+0000 7fbd80c00640 1 -- 172.21.15.39:0/900086242 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fbd7c01d020 con 0x7fbd8410f7a0 2024-07-04T08:14:12.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.915+0000 7fbd80c00640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd680b2840 0x7fbd680b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:12.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.915+0000 7fbd80c00640 1 -- 172.21.15.39:0/900086242 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fbd7c0e2690 con 0x7fbd8410f7a0 2024-07-04T08:14:12.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.915+0000 7fbd83e00640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd680b2840 0x7fbd680b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:12.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.916+0000 7fbd83e00640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd680b2840 0x7fbd680b4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009b50 tx=0x7fbd74005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:12.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:12.919+0000 7fbd80c00640 1 -- 172.21.15.39:0/900086242 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fbd7c0a9f00 con 0x7fbd8410f7a0 2024-07-04T08:14:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:12 smithi039 ceph-mon[32062]: pgmap v450: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:13.119 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:12 smithi133 ceph-mon[37050]: pgmap v450: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:13.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.131+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fbd841a0f70 con 0x7fbd8410f7a0 2024-07-04T08:14:13.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.134+0000 7fbd80c00640 1 -- 172.21.15.39:0/900086242 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5993 (secure 0 0 0) 0x7fbd7c0a9800 con 0x7fbd8410f7a0 2024-07-04T08:14:13.137 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 92 2024-07-04T08:14:13.137 INFO:teuthology.orchestra.run.smithi039.stdout:fsid 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:14:13.137 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-07-04T07:59:21.010226+0000 2024-07-04T08:14:13.137 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-07-04T08:11:53.672125+0000 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:crush_version 27 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:full_ratio 0.95 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:backfillfull_ratio 0.9 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:nearfull_ratio 0.85 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:require_min_compat_client luminous 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:min_compat_client luminous 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:require_osd_release squid 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:stretch_mode_enabled false 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-07-04T08:14:13.138 INFO:teuthology.orchestra.run.smithi039.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:max_osd 12 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 up in weight 1 up_from 9 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6802/1924112753,v1:172.21.15.39:6803/1924112753] [v2:172.21.15.39:6804/1924112753,v1:172.21.15.39:6805/1924112753] exists,up cfc7b735-e66c-4ace-958a-f47b6df89f37 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 up in weight 1 up_from 14 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6800/735117817,v1:172.21.15.50:6801/735117817] [v2:172.21.15.50:6802/735117817,v1:172.21.15.50:6803/735117817] exists,up de735138-774d-4816-b14f-4ad189b8b79d 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 up in weight 1 up_from 19 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6800/3737341609,v1:172.21.15.133:6801/3737341609] [v2:172.21.15.133:6802/3737341609,v1:172.21.15.133:6803/3737341609] exists,up ad4f9bb6-fc12-476f-a359-b137919ed63d 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:osd.3 up in weight 1 up_from 27 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6810/4257278793,v1:172.21.15.39:6811/4257278793] [v2:172.21.15.39:6812/4257278793,v1:172.21.15.39:6813/4257278793] exists,up 7accf662-69bc-4e7d-a7bf-c174c683367a 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6808/1784161771,v1:172.21.15.50:6809/1784161771] [v2:172.21.15.50:6810/1784161771,v1:172.21.15.50:6811/1784161771] exists,up 9db0ba72-e313-4a94-a5d0-ff30e4e8028f 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:osd.5 up in weight 1 up_from 39 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6808/787209680,v1:172.21.15.133:6809/787209680] [v2:172.21.15.133:6810/787209680,v1:172.21.15.133:6811/787209680] exists,up 5a0b6a8f-16d6-4018-a67a-c654f6deb5dc 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:osd.6 up in weight 1 up_from 45 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6818/636897432,v1:172.21.15.39:6819/636897432] [v2:172.21.15.39:6820/636897432,v1:172.21.15.39:6821/636897432] exists,up fae5d481-b487-4e26-ae87-395fa655837c 2024-07-04T08:14:13.139 INFO:teuthology.orchestra.run.smithi039.stdout:osd.7 up in weight 1 up_from 50 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6816/1720421182,v1:172.21.15.50:6817/1720421182] [v2:172.21.15.50:6818/1720421182,v1:172.21.15.50:6819/1720421182] exists,up 8ff9a0e2-4610-450d-be51-0726c8bc14ac 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:osd.8 up in weight 1 up_from 56 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6816/341977020,v1:172.21.15.133:6817/341977020] [v2:172.21.15.133:6818/341977020,v1:172.21.15.133:6819/341977020] exists,up a1bdae27-2643-4838-ba2b-080777c325e9 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:osd.9 up in weight 1 up_from 62 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.39:6826/2701785798,v1:172.21.15.39:6827/2701785798] [v2:172.21.15.39:6828/2701785798,v1:172.21.15.39:6829/2701785798] exists,up 005c7632-7a25-4d09-9d4b-59675ba2cf70 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:osd.10 up in weight 1 up_from 67 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6824/2876845435,v1:172.21.15.50:6825/2876845435] [v2:172.21.15.50:6826/2876845435,v1:172.21.15.50:6827/2876845435] exists,up 191d4999-048f-4bd9-b484-e20564492846 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:osd.11 up in weight 1 up_from 73 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6824/3104157254,v1:172.21.15.133:6825/3104157254] [v2:172.21.15.133:6826/3104157254,v1:172.21.15.133:6827/3104157254] exists,up a3dddd70-6e8a-424f-a69d-7c2b41575aae 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.6 [8,11] 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.b [8,11] 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.c [2,1] 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 2.1d [2,4] 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.3 [4,9] 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.11 [5,9] 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 3.24 [4,9] 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 4.6 [9,11] 2024-07-04T08:14:13.140 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 4.f [9,8] 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 4.14 [9,8] 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:pg_upmap_items 4.15 [5,10] 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/2939871551 expires 2024-07-05T07:59:57.104896+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/3207006577 expires 2024-07-05T08:00:28.376822+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/2939871551 expires 2024-07-05T07:59:57.104896+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1920202184 expires 2024-07-05T07:59:57.104896+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/764812681 expires 2024-07-05T07:59:57.104896+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/3823748890 expires 2024-07-05T08:01:51.343953+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1949300646 expires 2024-07-05T08:00:28.376822+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/2171986895 expires 2024-07-05T07:59:57.104896+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1596793936 expires 2024-07-05T08:00:28.376822+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/3207006577 expires 2024-07-05T08:00:28.376822+0000 2024-07-04T08:14:13.141 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6801/1922588202 expires 2024-07-05T08:01:51.343953+0000 2024-07-04T08:14:13.142 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1763496185 expires 2024-07-05T08:01:51.343953+0000 2024-07-04T08:14:13.142 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/917994444 expires 2024-07-05T08:00:28.376822+0000 2024-07-04T08:14:13.142 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:0/1237664844 expires 2024-07-05T08:01:51.343953+0000 2024-07-04T08:14:13.142 INFO:teuthology.orchestra.run.smithi039.stdout:blocklist 172.21.15.39:6800/1922588202 expires 2024-07-05T08:01:51.343953+0000 2024-07-04T08:14:13.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.137+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd680b2840 msgr2=0x7fbd680b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.137+0000 7fbd89400640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd680b2840 0x7fbd680b4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009b50 tx=0x7fbd74005f90 comp rx=0 tx=0).stop 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.137+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd8410f7a0 msgr2=0x7fbd841a5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.137+0000 7fbd89400640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd8410f7a0 0x7fbd841a5680 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00efc0 tx=0x7fbd7c00c800 comp rx=0 tx=0).stop 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.138+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 shutdown_connections 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.138+0000 7fbd89400640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd680b2840 0x7fbd680b4d50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.138+0000 7fbd89400640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd8410f7a0 0x7fbd841a5680 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.138+0000 7fbd89400640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd8410ee50 0x7fbd841a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.138+0000 7fbd89400640 1 --2- 172.21.15.39:0/900086242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd84109510 0x7fbd8419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.138+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 >> 172.21.15.39:0/900086242 conn(0x7fbd84078b10 msgr2=0x7fbd8410d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.138+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 shutdown_connections 2024-07-04T08:14:13.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:13.138+0000 7fbd89400640 1 -- 172.21.15.39:0/900086242 wait complete. 2024-07-04T08:14:13.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:12 smithi050 ceph-mon[36577]: pgmap v450: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:14.052 INFO:teuthology.run_tasks:Running task fs.ready... 2024-07-04T08:14:14.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:14:14.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:13 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/900086242' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-04T08:14:14.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:13 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/900086242' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-04T08:14:14.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:13 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/900086242' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-04T08:14:14.605 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:15.010 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:14 smithi039 ceph-mon[32062]: pgmap v451: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:15.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:14 smithi050 ceph-mon[36577]: pgmap v451: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:15.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:14 smithi133 ceph-mon[37050]: pgmap v451: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.898+0000 7fe0a1c00640 1 -- 172.21.15.39:0/4256695297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe09c1089f0 msgr2=0x7fe09c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:16.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.898+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/4256695297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe09c1089f0 0x7fe09c108e00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe084009860 tx=0x7fe08402a340 comp rx=0 tx=0).stop 2024-07-04T08:14:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.898+0000 7fe0a1c00640 1 -- 172.21.15.39:0/4256695297 shutdown_connections 2024-07-04T08:14:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.898+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/4256695297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe09c109340 0x7fe09c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.898+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/4256695297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe09c1089f0 0x7fe09c108e00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.898+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/4256695297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe09c1030b0 0x7fe09c1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.898+0000 7fe0a1c00640 1 -- 172.21.15.39:0/4256695297 >> 172.21.15.39:0/4256695297 conn(0x7fe09c07ada0 msgr2=0x7fe09c07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.898+0000 7fe0a1c00640 1 -- 172.21.15.39:0/4256695297 shutdown_connections 2024-07-04T08:14:16.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.898+0000 7fe0a1c00640 1 -- 172.21.15.39:0/4256695297 wait complete. 2024-07-04T08:14:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.899+0000 7fe0a1c00640 1 Processor -- start 2024-07-04T08:14:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.900+0000 7fe0a1c00640 1 -- start start 2024-07-04T08:14:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.900+0000 7fe0a1c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe09c1030b0 0x7fe09c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.900+0000 7fe0a1c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe09c1089f0 0x7fe09c0751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:16.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.900+0000 7fe0a1c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe09c109340 0x7fe09c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.900+0000 7fe0a1c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c112fb0 con 0x7fe09c1030b0 2024-07-04T08:14:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.900+0000 7fe0a1c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c112e30 con 0x7fe09c1089f0 2024-07-04T08:14:16.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.900+0000 7fe0a1c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe09c113130 con 0x7fe09c109340 2024-07-04T08:14:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe09c1089f0 0x7fe09c0751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe09c1089f0 0x7fe09c0751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37286/0 (socket says 172.21.15.39:37286) 2024-07-04T08:14:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09a000640 1 -- 172.21.15.39:0/662506851 learned_addr learned my addr 172.21.15.39:0/662506851 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09b400640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe09c109340 0x7fe09c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09aa00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe09c1030b0 0x7fe09c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09a000640 1 -- 172.21.15.39:0/662506851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe09c109340 msgr2=0x7fe09c0756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09a000640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe09c109340 0x7fe09c0756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09a000640 1 -- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe09c1030b0 msgr2=0x7fe09c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:16.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09a000640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe09c1030b0 0x7fe09c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09a000640 1 -- 172.21.15.39:0/662506851 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09c1a7ce0 con 0x7fe09c1089f0 2024-07-04T08:14:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.901+0000 7fe09aa00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe09c1030b0 0x7fe09c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.902+0000 7fe09a000640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe09c1089f0 0x7fe09c0751a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe084035040 tx=0x7fe084032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.902+0000 7fe093400640 1 -- 172.21.15.39:0/662506851 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084037040 con 0x7fe09c1089f0 2024-07-04T08:14:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.902+0000 7fe093400640 1 -- 172.21.15.39:0/662506851 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe084030c10 con 0x7fe09c1089f0 2024-07-04T08:14:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.902+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09c1a7f70 con 0x7fe09c1089f0 2024-07-04T08:14:16.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.902+0000 7fe093400640 1 -- 172.21.15.39:0/662506851 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840035f0 con 0x7fe09c1089f0 2024-07-04T08:14:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.902+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09c1a8360 con 0x7fe09c1089f0 2024-07-04T08:14:16.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.903+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe09c1034c0 con 0x7fe09c1089f0 2024-07-04T08:14:16.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.906+0000 7fe093400640 1 -- 172.21.15.39:0/662506851 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fe084043020 con 0x7fe09c1089f0 2024-07-04T08:14:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.907+0000 7fe093400640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0800b28c0 0x7fe0800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:16.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.907+0000 7fe093400640 1 -- 172.21.15.39:0/662506851 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fe084101990 con 0x7fe09c1089f0 2024-07-04T08:14:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.908+0000 7fe09aa00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0800b28c0 0x7fe0800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:16.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.908+0000 7fe09aa00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0800b28c0 0x7fe0800b4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0766b0 tx=0x7fe08c0095c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:16.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:16.911+0000 7fe093400640 1 -- 172.21.15.39:0/662506851 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe0840c9200 con 0x7fe09c1089f0 2024-07-04T08:14:17.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:16 smithi039 ceph-mon[32062]: pgmap v452: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:17.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:16 smithi050 ceph-mon[36577]: pgmap v452: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:17.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.165+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe09c02de10 con 0x7fe09c1089f0 2024-07-04T08:14:17.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.166+0000 7fe093400640 1 -- 172.21.15.39:0/662506851 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11975 (secure 0 0 0) 0x7fe084030dc0 con 0x7fe09c1089f0 2024-07-04T08:14:17.168 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:14:17.168 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-07-04T08:11:09:862205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:11:09.668485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24673,24674,24679,24701,24707]},"id":1}]} 2024-07-04T08:14:17.168 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-07-04T08:14:17.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0800b28c0 msgr2=0x7fe0800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:17.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0800b28c0 0x7fe0800b4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0766b0 tx=0x7fe08c0095c0 comp rx=0 tx=0).stop 2024-07-04T08:14:17.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe09c1089f0 msgr2=0x7fe09c0751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:17.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe09c1089f0 0x7fe09c0751a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe084035040 tx=0x7fe084032d50 comp rx=0 tx=0).stop 2024-07-04T08:14:17.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 shutdown_connections 2024-07-04T08:14:17.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe0800b28c0 0x7fe0800b4dd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:17.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe09c109340 0x7fe09c0756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:17.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe09c1089f0 0x7fe09c0751a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 --2- 172.21.15.39:0/662506851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe09c1030b0 0x7fe09c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.169+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 >> 172.21.15.39:0/662506851 conn(0x7fe09c07ada0 msgr2=0x7fe09c106af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.170+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 shutdown_connections 2024-07-04T08:14:17.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:17.170+0000 7fe0a1c00640 1 -- 172.21.15.39:0/662506851 wait complete. 2024-07-04T08:14:17.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:16 smithi133 ceph-mon[37050]: pgmap v452: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:18.115 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:14:18.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:17 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/662506851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:14:18.197 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:17 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/662506851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:14:18.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:17 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/662506851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:14:18.618 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:19.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:18 smithi039 ceph-mon[32062]: pgmap v453: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:19.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:18 smithi050 ceph-mon[36577]: pgmap v453: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:19.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:18 smithi133 ceph-mon[37050]: pgmap v453: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:21.015 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:20 smithi039 ceph-mon[32062]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.122+0000 7f4e35800640 1 -- 172.21.15.39:0/344105527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30113aa0 msgr2=0x7f4e30115ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.122+0000 7f4e35800640 1 --2- 172.21.15.39:0/344105527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30113aa0 0x7f4e30115ec0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009ca0 tx=0x7f4e2802a8b0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.122+0000 7f4e35800640 1 -- 172.21.15.39:0/344105527 shutdown_connections 2024-07-04T08:14:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.122+0000 7f4e35800640 1 --2- 172.21.15.39:0/344105527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e300777e0 0x7f4e30113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.122+0000 7f4e35800640 1 --2- 172.21.15.39:0/344105527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30113aa0 0x7f4e30115ec0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.122+0000 7f4e35800640 1 --2- 172.21.15.39:0/344105527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e30076e90 0x7f4e300772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.122+0000 7f4e35800640 1 -- 172.21.15.39:0/344105527 >> 172.21.15.39:0/344105527 conn(0x7f4e30100b30 msgr2=0x7f4e30102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.123+0000 7f4e35800640 1 -- 172.21.15.39:0/344105527 shutdown_connections 2024-07-04T08:14:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.123+0000 7f4e35800640 1 -- 172.21.15.39:0/344105527 wait complete. 2024-07-04T08:14:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.124+0000 7f4e35800640 1 Processor -- start 2024-07-04T08:14:21.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.124+0000 7f4e35800640 1 -- start start 2024-07-04T08:14:21.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.124+0000 7f4e35800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30076e90 0x7f4e3019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:21.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.124+0000 7f4e35800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e300777e0 0x7f4e301a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:21.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e35800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e30113aa0 0x7f4e301a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:21.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e35800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30119460 con 0x7f4e30113aa0 2024-07-04T08:14:21.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e35800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301192e0 con 0x7f4e300777e0 2024-07-04T08:14:21.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e35800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e301195e0 con 0x7f4e30076e90 2024-07-04T08:14:21.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e2fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30076e90 0x7f4e3019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e2fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30076e90 0x7f4e3019fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56834/0 (socket says 172.21.15.39:56834) 2024-07-04T08:14:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e2f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e300777e0 0x7f4e301a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e2fe00640 1 -- 172.21.15.39:0/89922634 learned_addr learned my addr 172.21.15.39:0/89922634 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e34e00640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e30113aa0 0x7f4e301a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e2fe00640 1 -- 172.21.15.39:0/89922634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e300777e0 msgr2=0x7f4e301a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e2fe00640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e300777e0 0x7f4e301a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e2fe00640 1 -- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e30113aa0 msgr2=0x7f4e301a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e2fe00640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e30113aa0 0x7f4e301a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.125+0000 7f4e2fe00640 1 -- 172.21.15.39:0/89922634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e301a5d80 con 0x7f4e30076e90 2024-07-04T08:14:21.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.126+0000 7f4e2fe00640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30076e90 0x7f4e3019fe10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000e9f0 tx=0x7f4e2000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:21.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.127+0000 7f4e2cc00640 1 -- 172.21.15.39:0/89922634 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000cd80 con 0x7f4e30076e90 2024-07-04T08:14:21.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.127+0000 7f4e2cc00640 1 -- 172.21.15.39:0/89922634 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e200108e0 con 0x7f4e30076e90 2024-07-04T08:14:21.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.127+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e301a6070 con 0x7f4e30076e90 2024-07-04T08:14:21.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.127+0000 7f4e2cc00640 1 -- 172.21.15.39:0/89922634 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20018980 con 0x7f4e30076e90 2024-07-04T08:14:21.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.127+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e301ac7d0 con 0x7f4e30076e90 2024-07-04T08:14:21.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.128+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e300772a0 con 0x7f4e30076e90 2024-07-04T08:14:21.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.131+0000 7f4e2cc00640 1 -- 172.21.15.39:0/89922634 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f4e20010430 con 0x7f4e30076e90 2024-07-04T08:14:21.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.132+0000 7f4e2cc00640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e140b28c0 0x7f4e140b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:21.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.133+0000 7f4e2cc00640 1 -- 172.21.15.39:0/89922634 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4e200e29e0 con 0x7f4e30076e90 2024-07-04T08:14:21.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.133+0000 7f4e2f400640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e140b28c0 0x7f4e140b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:21.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.133+0000 7f4e2f400640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e140b28c0 0x7f4e140b4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4e30114180 tx=0x7f4e1c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:21.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.136+0000 7f4e2cc00640 1 -- 172.21.15.39:0/89922634 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f4e200a9c00 con 0x7f4e30076e90 2024-07-04T08:14:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:20 smithi133 ceph-mon[37050]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:21.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:20 smithi050 ceph-mon[36577]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:21.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.395+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e301a0f40 con 0x7f4e30076e90 2024-07-04T08:14:21.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.396+0000 7f4e2cc00640 1 -- 172.21.15.39:0/89922634 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11975 (secure 0 0 0) 0x7f4e200adf00 con 0x7f4e30076e90 2024-07-04T08:14:21.398 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:14:21.398 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":29,"btime":"2024-07-04T08:11:09:862205+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:11:09.668485+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24673,24674,24679,24701,24707]},"id":1}]} 2024-07-04T08:14:21.399 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 29 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e140b28c0 msgr2=0x7f4e140b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e140b28c0 0x7f4e140b4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4e30114180 tx=0x7f4e1c005f90 comp rx=0 tx=0).stop 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30076e90 msgr2=0x7f4e3019fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30076e90 0x7f4e3019fe10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000e9f0 tx=0x7f4e2000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 shutdown_connections 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4e140b28c0 0x7f4e140b4dd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e30113aa0 0x7f4e301a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4e300777e0 0x7f4e301a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 --2- 172.21.15.39:0/89922634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30076e90 0x7f4e3019fe10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.399+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 >> 172.21.15.39:0/89922634 conn(0x7f4e30100b30 msgr2=0x7f4e30115b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.400+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 shutdown_connections 2024-07-04T08:14:21.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:21.400+0000 7f4e35800640 1 -- 172.21.15.39:0/89922634 wait complete. 2024-07-04T08:14:22.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:21 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/89922634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:14:22.211 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:14:22.296 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:21 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/89922634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:14:22.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:21 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/89922634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:14:22.695 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:23.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:22 smithi039 ceph-mon[32062]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:23.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:22 smithi133 ceph-mon[37050]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:23.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:22 smithi050 ceph-mon[36577]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:25.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:24 smithi039 ceph-mon[32062]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.109+0000 7feaa5a00640 1 -- 172.21.15.39:0/175916405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa01019e0 msgr2=0x7feaa0101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.109+0000 7feaa5a00640 1 --2- 172.21.15.39:0/175916405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa01019e0 0x7feaa0101df0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea90009900 tx=0x7fea9002a480 comp rx=0 tx=0).stop 2024-07-04T08:14:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.109+0000 7feaa5a00640 1 -- 172.21.15.39:0/175916405 shutdown_connections 2024-07-04T08:14:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.109+0000 7feaa5a00640 1 --2- 172.21.15.39:0/175916405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaa010f590 0x7feaa01119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.109+0000 7feaa5a00640 1 --2- 172.21.15.39:0/175916405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa0102330 0x7feaa010f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.109+0000 7feaa5a00640 1 --2- 172.21.15.39:0/175916405 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa01019e0 0x7feaa0101df0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.109+0000 7feaa5a00640 1 -- 172.21.15.39:0/175916405 >> 172.21.15.39:0/175916405 conn(0x7feaa00fd8a0 msgr2=0x7feaa00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.109+0000 7feaa5a00640 1 -- 172.21.15.39:0/175916405 shutdown_connections 2024-07-04T08:14:25.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.110+0000 7feaa5a00640 1 -- 172.21.15.39:0/175916405 wait complete. 2024-07-04T08:14:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.111+0000 7feaa5a00640 1 Processor -- start 2024-07-04T08:14:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.111+0000 7feaa5a00640 1 -- start start 2024-07-04T08:14:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.111+0000 7feaa5a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa01019e0 0x7feaa019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.111+0000 7feaa5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaa0102330 0x7feaa019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.111+0000 7feaa5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa010f590 0x7feaa01a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7feaa5a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0115000 con 0x7feaa010f590 2024-07-04T08:14:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7feaa5a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0114e80 con 0x7feaa0102330 2024-07-04T08:14:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7feaa5a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feaa0115180 con 0x7feaa01019e0 2024-07-04T08:14:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7fea97e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaa0102330 0x7feaa019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7fea9fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa01019e0 0x7feaa019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7fea97e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaa0102330 0x7feaa019c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38040/0 (socket says 172.21.15.39:38040) 2024-07-04T08:14:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7fea97e00640 1 -- 172.21.15.39:0/2890051335 learned_addr learned my addr 172.21.15.39:0/2890051335 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7feaa5000640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa010f590 0x7feaa01a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:25.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7fea97e00640 1 -- 172.21.15.39:0/2890051335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa01019e0 msgr2=0x7feaa019bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7fea97e00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa01019e0 0x7feaa019bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7fea97e00640 1 -- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa010f590 msgr2=0x7feaa01a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7fea97e00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa010f590 0x7feaa01a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7fea97e00640 1 -- 172.21.15.39:0/2890051335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa01a1a20 con 0x7feaa0102330 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.112+0000 7feaa5000640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa010f590 0x7feaa01a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.113+0000 7fea97e00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaa0102330 0x7feaa019c050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea8800e9f0 tx=0x7fea8800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.113+0000 7fea9d600640 1 -- 172.21.15.39:0/2890051335 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8800cd80 con 0x7feaa0102330 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.113+0000 7fea9d600640 1 -- 172.21.15.39:0/2890051335 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea880054d0 con 0x7feaa0102330 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.113+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa01a1d10 con 0x7feaa0102330 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.113+0000 7fea9d600640 1 -- 172.21.15.39:0/2890051335 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88010430 con 0x7feaa0102330 2024-07-04T08:14:25.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.114+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa01a8470 con 0x7feaa0102330 2024-07-04T08:14:25.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.115+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa0101df0 con 0x7feaa0102330 2024-07-04T08:14:25.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.118+0000 7fea9d600640 1 -- 172.21.15.39:0/2890051335 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fea88005000 con 0x7feaa0102330 2024-07-04T08:14:25.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.120+0000 7fea9d600640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea840b2840 0x7fea840b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:25.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.120+0000 7fea9d600640 1 -- 172.21.15.39:0/2890051335 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fea880e2df0 con 0x7feaa0102330 2024-07-04T08:14:25.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.120+0000 7fea9fe00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea840b2840 0x7fea840b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.121+0000 7fea9d600640 1 -- 172.21.15.39:0/2890051335 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fea880052f0 con 0x7feaa0102330 2024-07-04T08:14:25.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.121+0000 7fea9fe00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea840b2840 0x7fea840b4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea90031440 tx=0x7fea90009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:24 smithi133 ceph-mon[37050]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:25.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.332+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaa019cbe0 con 0x7feaa0102330 2024-07-04T08:14:25.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.334+0000 7fea9d600640 1 -- 172.21.15.39:0/2890051335 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24957 (secure 0 0 0) 0x7feaa019cbe0 con 0x7feaa0102330 2024-07-04T08:14:25.337 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:14:25.337 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:11:53.672125+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:14:25.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea840b2840 msgr2=0x7fea840b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:25.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea840b2840 0x7fea840b4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fea90031440 tx=0x7fea90009560 comp rx=0 tx=0).stop 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaa0102330 msgr2=0x7feaa019c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaa0102330 0x7feaa019c050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea8800e9f0 tx=0x7fea8800edf0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 shutdown_connections 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fea840b2840 0x7fea840b4d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feaa010f590 0x7feaa01a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feaa0102330 0x7feaa019c050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 --2- 172.21.15.39:0/2890051335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa01019e0 0x7feaa019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 >> 172.21.15.39:0/2890051335 conn(0x7feaa00fd8a0 msgr2=0x7feaa0111730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 shutdown_connections 2024-07-04T08:14:25.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:25.337+0000 7feaa5a00640 1 -- 172.21.15.39:0/2890051335 wait complete. 2024-07-04T08:14:25.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:24 smithi050 ceph-mon[36577]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:26.310 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:25 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2890051335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:14:26.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:25 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2890051335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:14:26.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:25 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2890051335' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:14:26.403 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-04T08:09:41.252924+0000', 'modified': '2024-07-04T08:11:09.668485+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24707, 'mds_1': 24701, 'mds_2': 24679, 'mds_3': 24674, 'mds_4': 24673}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24619': {'gid': 24619, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6833/2411107381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6832', 'nonce': 2411107381}, {'type': 'v1', 'addr': '172.21.15.133:6833', 'nonce': 2411107381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24626': {'gid': 24626, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6833/264080488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6832', 'nonce': 264080488}, {'type': 'v1', 'addr': '172.21.15.50:6833', 'nonce': 264080488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6837/1240380045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 1240380045}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 1240380045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24656': {'gid': 24656, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6835/2710764279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 2710764279}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 2710764279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24661': {'gid': 24661, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/2187182943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2187182943}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 2187182943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24673': {'gid': 24673, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.133:6837/431595690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 431595690}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 431595690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.50:6837/4268659351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6836', 'nonce': 4268659351}, {'type': 'v1', 'addr': '172.21.15.50:6837', 'nonce': 4268659351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24679': {'gid': 24679, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.133:6839/1264801316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 1264801316}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 1264801316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24701': {'gid': 24701, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.39:6841/2753029904', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 2753029904}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 2753029904}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24707': {'gid': 24707, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.50:6839/2852157181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 2852157181}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 2852157181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24707, 'qdb_cluster': [24673, 24674, 24679, 24701, 24707]} 2024-07-04T08:14:26.403 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-04T08:14:26.403 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.c status 2024-07-04T08:14:26.893 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:27.208 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:26 smithi039 ceph-mon[32062]: pgmap v457: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:26 smithi133 ceph-mon[37050]: pgmap v457: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:27.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:26 smithi050 ceph-mon[36577]: pgmap v457: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:29.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.244+0000 7f50ebe00640 1 -- 172.21.15.39:0/662606942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 msgr2=0x7f50ec105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:29.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.244+0000 7f50ebe00640 1 --2- 172.21.15.39:0/662606942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 0x7f50ec105520 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50dc009860 tx=0x7f50dc02a340 comp rx=0 tx=0).stop 2024-07-04T08:14:29.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.244+0000 7f50ebe00640 1 -- 172.21.15.39:0/662606942 shutdown_connections 2024-07-04T08:14:29.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.244+0000 7f50ebe00640 1 --2- 172.21.15.39:0/662606942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ec10b3a0 0x7f50ec111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.244+0000 7f50ebe00640 1 --2- 172.21.15.39:0/662606942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec10aa50 0x7f50ec10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.244+0000 7f50ebe00640 1 --2- 172.21.15.39:0/662606942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 0x7f50ec105520 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.244+0000 7f50ebe00640 1 -- 172.21.15.39:0/662606942 >> 172.21.15.39:0/662606942 conn(0x7f50ec100b80 msgr2=0x7f50ec102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:29.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.244+0000 7f50ebe00640 1 -- 172.21.15.39:0/662606942 shutdown_connections 2024-07-04T08:14:29.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.245+0000 7f50ebe00640 1 -- 172.21.15.39:0/662606942 wait complete. 2024-07-04T08:14:29.248 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:28 smithi039 ceph-mon[32062]: pgmap v458: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:29.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.246+0000 7f50ebe00640 1 Processor -- start 2024-07-04T08:14:29.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.246+0000 7f50ebe00640 1 -- start start 2024-07-04T08:14:29.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50ebe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 0x7f50ec19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50ebe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec10aa50 0x7f50ec19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50ebe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ec10b3a0 0x7f50ec1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50ebe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec115050 con 0x7f50ec10b3a0 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50ebe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec114ed0 con 0x7f50ec10aa50 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50ebe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1151d0 con 0x7f50ec105110 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50eaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 0x7f50ec19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50eaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 0x7f50ec19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56866/0 (socket says 172.21.15.39:56866) 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50eaa00640 1 -- 172.21.15.39:0/307026545 learned_addr learned my addr 172.21.15.39:0/307026545 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50eb400640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ec10b3a0 0x7f50ec1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.247+0000 7f50ea000640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec10aa50 0x7f50ec19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50eaa00640 1 -- 172.21.15.39:0/307026545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec10aa50 msgr2=0x7f50ec19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:29.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50eaa00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec10aa50 0x7f50ec19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50eaa00640 1 -- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ec10b3a0 msgr2=0x7f50ec1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:29.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50eaa00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ec10b3a0 0x7f50ec1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50eaa00640 1 -- 172.21.15.39:0/307026545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec1a19d0 con 0x7f50ec105110 2024-07-04T08:14:29.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50eb400640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ec10b3a0 0x7f50ec1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:29.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50eaa00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 0x7f50ec19ba60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f50dc035040 tx=0x7f50dc032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:29.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50f0a00640 1 -- 172.21.15.39:0/307026545 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc037040 con 0x7f50ec105110 2024-07-04T08:14:29.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50f0a00640 1 -- 172.21.15.39:0/307026545 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50dc041070 con 0x7f50ec105110 2024-07-04T08:14:29.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.248+0000 7f50f0a00640 1 -- 172.21.15.39:0/307026545 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50dc003850 con 0x7f50ec105110 2024-07-04T08:14:29.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.249+0000 7f50ebe00640 1 -- 172.21.15.39:0/307026545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec1a1c60 con 0x7f50ec105110 2024-07-04T08:14:29.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.249+0000 7f50ebe00640 1 -- 172.21.15.39:0/307026545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec1a8420 con 0x7f50ec105110 2024-07-04T08:14:29.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.252+0000 7f50f84cc740 1 Processor -- start 2024-07-04T08:14:29.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.252+0000 7f50f84cc740 1 -- start start 2024-07-04T08:14:29.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50f0a00640 1 -- 172.21.15.39:0/307026545 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f50dc0039f0 con 0x7f50ec105110 2024-07-04T08:14:29.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9109df60 0x555a91442830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a91442d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 0x555a914432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50f84cc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555a913b8c80 con 0x555a910750a0 2024-07-04T08:14:29.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50f84cc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555a913a8220 con 0x555a9109df60 2024-07-04T08:14:29.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50f84cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555a913ac260 con 0x555a9107d280 2024-07-04T08:14:29.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50eaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a91442d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9109df60 0x555a91442830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50ea000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 0x555a914432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9109df60 0x555a91442830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38070/0 (socket says 172.21.15.39:38070) 2024-07-04T08:14:29.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.253+0000 7f50eb400640 1 -- 172.21.15.39:0/3141584250 learned_addr learned my addr 172.21.15.39:0/3141584250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:29.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.254+0000 7f50eb400640 1 -- 172.21.15.39:0/3141584250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 msgr2=0x555a914432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:29.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.254+0000 7f50eb400640 1 --2- 172.21.15.39:0/3141584250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 0x555a914432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.254+0000 7f50eb400640 1 -- 172.21.15.39:0/3141584250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 msgr2=0x555a91442d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:29.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.254+0000 7f50eb400640 1 --2- 172.21.15.39:0/3141584250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a91442d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.254+0000 7f50eb400640 1 -- 172.21.15.39:0/3141584250 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a91340530 con 0x555a9109df60 2024-07-04T08:14:29.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.254+0000 7f50eb400640 1 --2- 172.21.15.39:0/3141584250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9109df60 0x555a91442830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50e400f480 tx=0x7f50e400f940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48c81dd5711e3178 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:29.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.254+0000 7f50e0c00640 1 -- 172.21.15.39:0/3141584250 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e401d070 con 0x555a9109df60 2024-07-04T08:14:29.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.254+0000 7f50e0c00640 1 -- 172.21.15.39:0/3141584250 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e400cc30 con 0x555a9109df60 2024-07-04T08:14:29.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.254+0000 7f50e0c00640 1 -- 172.21.15.39:0/3141584250 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4018c50 con 0x555a9109df60 2024-07-04T08:14:29.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.255+0000 7f50f0a00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f50d00b28c0 0x7f50d00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.255+0000 7f50f0a00640 1 -- 172.21.15.39:0/307026545 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f50dc102220 con 0x7f50ec105110 2024-07-04T08:14:29.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.255+0000 7f50eaa00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f50d00b28c0 0x7f50d00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.255+0000 7f50f84cc740 1 -- 172.21.15.39:0/3141584250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9109df60 msgr2=0x555a91442830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.255+0000 7f50f84cc740 1 --2- 172.21.15.39:0/3141584250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9109df60 0x555a91442830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50e400f480 tx=0x7f50e400f940 comp rx=0 tx=0).stop 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.256+0000 7f50f84cc740 1 -- 172.21.15.39:0/3141584250 shutdown_connections 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.256+0000 7f50f84cc740 1 --2- 172.21.15.39:0/3141584250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a91442d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.256+0000 7f50f84cc740 1 --2- 172.21.15.39:0/3141584250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 0x555a914432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.256+0000 7f50f84cc740 1 --2- 172.21.15.39:0/3141584250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9109df60 0x555a91442830 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.256+0000 7f50f84cc740 1 -- 172.21.15.39:0/3141584250 >> 172.21.15.39:0/3141584250 conn(0x555a91092350 msgr2=0x555a91440720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.256+0000 7f50f84cc740 1 -- 172.21.15.39:0/3141584250 shutdown_connections 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.256+0000 7f50f84cc740 1 -- 172.21.15.39:0/3141584250 wait complete. 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.256+0000 7f50eaa00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f50d00b28c0 0x7f50d00b4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0c5410 tx=0x7f50dc105c60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:29.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.256+0000 7f50f84cc740 1 -- start start 2024-07-04T08:14:29.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.257+0000 7f50f84cc740 1 Processor -- start 2024-07-04T08:14:29.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.257+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a9144e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.257+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a9144ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.257+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9109df60 0x555a9145a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.257+0000 7f50f84cc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555a9120c5f0 con 0x555a910750a0 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.257+0000 7f50f84cc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555a9134f950 con 0x555a9107d280 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.257+0000 7f50f84cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555a91451010 con 0x555a9109df60 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.257+0000 7f50f84cc740 10 client.0 _ll_register_callbacks cb 0x555a913a1480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50ea000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a9144ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50eb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a9144e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50ea000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a9144ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38086/0 (socket says 172.21.15.39:38086) 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50ea000640 1 -- 172.21.15.39:0/2388932264 learned_addr learned my addr 172.21.15.39:0/2388932264 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50eb400640 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a9144e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47074/0 (socket says 172.21.15.39:47074) 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50ea000640 1 -- 172.21.15.39:0/2388932264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9109df60 msgr2=0x555a9145a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50ea000640 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9109df60 0x555a9145a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50ea000640 1 -- 172.21.15.39:0/2388932264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 msgr2=0x555a9144e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:29.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50ea000640 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a9144e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50ea000640 1 -- 172.21.15.39:0/2388932264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a91109540 con 0x555a9107d280 2024-07-04T08:14:29.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50ea000640 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a9144ea20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f50d8012700 tx=0x7f50d8012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b011c85ffeafcd8b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:29.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.258+0000 7f50d5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:14:29.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.259+0000 7f50d5600640 1 -- 172.21.15.39:0/2388932264 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d800f920 con 0x555a9107d280 2024-07-04T08:14:29.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.259+0000 7f50f84cc740 1 -- 172.21.15.39:0/2388932264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e400cc30 con 0x555a9107d280 2024-07-04T08:14:29.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.259+0000 7f50d5600640 1 -- 172.21.15.39:0/2388932264 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f50d800fac0 con 0x555a9107d280 2024-07-04T08:14:29.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.259+0000 7f50d5600640 1 -- 172.21.15.39:0/2388932264 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f50d800fcc0 con 0x555a9107d280 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.259+0000 7f50f84cc740 10 client.25298 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.259+0000 7f50f84cc740 1 -- 172.21.15.39:0/2388932264 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a9145d150 con 0x555a9107d280 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.260+0000 7f50d5600640 1 -- 172.21.15.39:0/2388932264 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e400cc30 con 0x555a9107d280 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.260+0000 7f50d5600640 1 -- 172.21.15.39:0/2388932264 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f50d801c420 con 0x555a9107d280 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.260+0000 7f50f84cc740 10 client.25298 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.260+0000 7f50f84cc740 10 client.25298 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.260+0000 7f50f84cc740 20 client.25298 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.260+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x555a9105c8e0 0x555a91460cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.260+0000 7f50f84cc740 4 client.25298 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.260+0000 7f50f84cc740 1 -- 172.21.15.39:0/2388932264 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a913babd0 con 0x555a9105c8e0 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.261+0000 7f50eb400640 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x555a9105c8e0 0x555a91460cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.261+0000 7f50eb400640 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x555a9105c8e0 0x555a91460cb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9abd7fa6331b83e6 server_cookie=c531140a9d74854b in_seq=0 out_seq=0 2024-07-04T08:14:29.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.261+0000 7f50d5600640 10 client.25298 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:14:29.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.262+0000 7f50d5600640 1 -- 172.21.15.39:0/2388932264 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a913babd0 con 0x555a9105c8e0 2024-07-04T08:14:29.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.262+0000 7f50d5600640 10 client.25298 handle_command_reply: tid=0 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.262+0000 7f50f84cc740 1 client.25298 shutdown 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.262+0000 7f50f84cc740 1 -- 172.21.15.39:0/2388932264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 msgr2=0x555a9144ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.262+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a9144ea20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f50d8012700 tx=0x7f50d8012bc0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50f84cc740 1 -- 172.21.15.39:0/2388932264 shutdown_connections 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x555a9105c8e0 0x555a91460cb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9109df60 0x555a9145a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a9144ea20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2388932264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a9144e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50d5600640 0 client.25298 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50f84cc740 1 -- 172.21.15.39:0/2388932264 >> 172.21.15.39:0/2388932264 conn(0x555a91092350 msgr2=0x555a914459c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:29.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50f84cc740 1 -- 172.21.15.39:0/2388932264 shutdown_connections 2024-07-04T08:14:29.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50f84cc740 1 -- 172.21.15.39:0/2388932264 wait complete. 2024-07-04T08:14:29.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.263+0000 7f50f84cc740 20 client.25298 trim_cache size 0 max 16384 2024-07-04T08:14:29.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.284+0000 7f50f84cc740 1 Processor -- start 2024-07-04T08:14:29.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.284+0000 7f50f84cc740 1 -- start start 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.284+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a910750a0 0x555a9145a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.284+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 0x555a9145e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.284+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a9109df60 0x555a91447940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.284+0000 7f50f84cc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555a9145eff0 con 0x555a9109df60 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.284+0000 7f50f84cc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555a9145f6c0 con 0x555a910750a0 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.284+0000 7f50f84cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555a9145fd90 con 0x555a9107d280 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a910750a0 0x555a9145a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50ea000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 0x555a9145e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50eb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a910750a0 0x555a9145a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38092/0 (socket says 172.21.15.39:38092) 2024-07-04T08:14:29.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50eb400640 1 -- 172.21.15.39:0/2561886011 learned_addr learned my addr 172.21.15.39:0/2561886011 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:29.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50eb400640 1 --2- 172.21.15.39:0/2561886011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a9109df60 0x555a91447940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50ea000640 1 -- 172.21.15.39:0/2561886011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a910750a0 msgr2=0x555a9145a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50ea000640 1 --2- 172.21.15.39:0/2561886011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a910750a0 0x555a9145a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50ea000640 1 -- 172.21.15.39:0/2561886011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a9109df60 msgr2=0x555a91447940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50ea000640 1 --2- 172.21.15.39:0/2561886011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a9109df60 0x555a91447940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50ea000640 1 -- 172.21.15.39:0/2561886011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a914450c0 con 0x555a9107d280 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.285+0000 7f50ea000640 1 --2- 172.21.15.39:0/2561886011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 0x555a9145e610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50d801ca00 tx=0x7f50d800fee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb6f07b57f0def02 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.286+0000 7f50d7e00640 1 -- 172.21.15.39:0/2561886011 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d800f920 con 0x555a9107d280 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.286+0000 7f50d7e00640 1 -- 172.21.15.39:0/2561886011 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d800fac0 con 0x555a9107d280 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.286+0000 7f50d7e00640 1 -- 172.21.15.39:0/2561886011 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d8005500 con 0x555a9107d280 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.286+0000 7f50f84cc740 1 -- 172.21.15.39:0/2561886011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 msgr2=0x555a9145e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.286+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2561886011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 0x555a9145e610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f50d801ca00 tx=0x7f50d800fee0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.287+0000 7f50f84cc740 1 -- 172.21.15.39:0/2561886011 shutdown_connections 2024-07-04T08:14:29.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.287+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2561886011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a9109df60 0x555a91447940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.287+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2561886011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9107d280 0x555a9145e610 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.287+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2561886011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a910750a0 0x555a9145a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.287+0000 7f50f84cc740 1 -- 172.21.15.39:0/2561886011 >> 172.21.15.39:0/2561886011 conn(0x555a91092350 msgr2=0x555a9144cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:29.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.287+0000 7f50f84cc740 1 -- 172.21.15.39:0/2561886011 shutdown_connections 2024-07-04T08:14:29.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.287+0000 7f50f84cc740 1 -- 172.21.15.39:0/2561886011 wait complete. 2024-07-04T08:14:29.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.287+0000 7f50f84cc740 1 -- start start 2024-07-04T08:14:29.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.287+0000 7f50f84cc740 1 Processor -- start 2024-07-04T08:14:29.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a91469310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a914627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50f84cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9109df60 0x555a91462d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50f84cc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555a91460b30 con 0x555a910750a0 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50f84cc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555a91460460 con 0x555a9107d280 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50f84cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555a91461200 con 0x555a9109df60 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50f84cc740 10 client.0 _ll_register_callbacks cb 0x7f50d8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50eb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a91469310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50eb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a91469310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47100/0 (socket says 172.21.15.39:47100) 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50eb400640 1 -- 172.21.15.39:0/2248523746 learned_addr learned my addr 172.21.15.39:0/2248523746 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.288+0000 7f50ea000640 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a914627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.289+0000 7f50eb400640 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9109df60 0x555a91462d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.289+0000 7f50ea000640 1 -- 172.21.15.39:0/2248523746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9109df60 msgr2=0x555a91462d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.289+0000 7f50ea000640 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9109df60 0x555a91462d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.289+0000 7f50ea000640 1 -- 172.21.15.39:0/2248523746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 msgr2=0x555a91469310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:29.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.289+0000 7f50ea000640 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a91469310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.289+0000 7f50ea000640 1 -- 172.21.15.39:0/2248523746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a91463240 con 0x555a9107d280 2024-07-04T08:14:29.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.289+0000 7f50ea000640 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a914627c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f50d801c810 tx=0x7f50d801ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4666bffeedf44b86 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:29.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.289+0000 7f50d5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.290+0000 7f50d5600640 1 -- 172.21.15.39:0/2248523746 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d800f920 con 0x555a9107d280 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.290+0000 7f50d5600640 1 -- 172.21.15.39:0/2248523746 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f50d8005470 con 0x555a9107d280 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.290+0000 7f50f84cc740 1 -- 172.21.15.39:0/2248523746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d800fac0 con 0x555a9107d280 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.290+0000 7f50d5600640 1 -- 172.21.15.39:0/2248523746 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f50d803b050 con 0x555a9107d280 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.290+0000 7f50f84cc740 10 client.25304 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.290+0000 7f50f84cc740 1 -- 172.21.15.39:0/2248523746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a91468b70 con 0x555a9107d280 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.290+0000 7f50d5600640 1 -- 172.21.15.39:0/2248523746 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d800fac0 con 0x555a9107d280 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.291+0000 7f50d5600640 1 -- 172.21.15.39:0/2248523746 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f50d802a480 con 0x555a9107d280 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.291+0000 7f50f84cc740 10 client.25304 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.291+0000 7f50f84cc740 10 client.25304 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.291+0000 7f50f84cc740 20 client.25304 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:29.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.291+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x555a91031c90 0x555a914ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.291+0000 7f50f84cc740 4 client.25304 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.291+0000 7f50f84cc740 1 -- 172.21.15.39:0/2248523746 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "status"}) -- 0x555a913b6670 con 0x555a91031c90 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.291+0000 7f50eb400640 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x555a91031c90 0x555a914ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50eb400640 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x555a91031c90 0x555a914ccfa0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77b4ba61959f87b7 server_cookie=5c28b0a076269506 in_seq=0 out_seq=0 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50d5600640 10 client.25304 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50d5600640 1 -- 172.21.15.39:0/2248523746 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555a913b6670 con 0x555a91031c90 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50d5600640 10 client.25304 handle_command_reply: tid=0 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50f84cc740 1 client.25304 shutdown 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50f84cc740 1 -- 172.21.15.39:0/2248523746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 msgr2=0x555a914627c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a914627c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f50d801c810 tx=0x7f50d801ccb0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50f84cc740 1 -- 172.21.15.39:0/2248523746 shutdown_connections 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x555a91031c90 0x555a914ccfa0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a9109df60 0x555a91462d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a9107d280 0x555a914627c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.292+0000 7f50f84cc740 1 --2- 172.21.15.39:0/2248523746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a910750a0 0x555a91469310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.293+0000 7f50f84cc740 1 -- 172.21.15.39:0/2248523746 >> 172.21.15.39:0/2248523746 conn(0x555a91092350 msgr2=0x555a9144e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:29.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.293+0000 7f50d5600640 0 client.25304 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:14:29.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.293+0000 7f50f84cc740 1 -- 172.21.15.39:0/2248523746 shutdown_connections 2024-07-04T08:14:29.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.293+0000 7f50f84cc740 1 -- 172.21.15.39:0/2248523746 wait complete. 2024-07-04T08:14:29.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.293+0000 7f50f84cc740 20 client.25304 trim_cache size 0 max 16384 2024-07-04T08:14:29.298 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:14:29.298 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24673, 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 204.446252719, 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 324.525237398 2024-07-04T08:14:29.299 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:14:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 -- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f50d00b28c0 msgr2=0x7f50d00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f50d00b28c0 0x7f50d00b4dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0c5410 tx=0x7f50dc105c60 comp rx=0 tx=0).stop 2024-07-04T08:14:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 -- 172.21.15.39:0/307026545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 msgr2=0x7f50ec19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 0x7f50ec19ba60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f50dc035040 tx=0x7f50dc032d50 comp rx=0 tx=0).stop 2024-07-04T08:14:29.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 -- 172.21.15.39:0/307026545 shutdown_connections 2024-07-04T08:14:29.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f50d00b28c0 0x7f50d00b4dd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50ec10b3a0 0x7f50ec1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f50ec10aa50 0x7f50ec19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 --2- 172.21.15.39:0/307026545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50ec105110 0x7f50ec19ba60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:29.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 -- 172.21.15.39:0/307026545 >> 172.21.15.39:0/307026545 conn(0x7f50ec100b80 msgr2=0x7f50ec102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:29.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 -- 172.21.15.39:0/307026545 shutdown_connections 2024-07-04T08:14:29.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:29.294+0000 7f50ebe00640 1 -- 172.21.15.39:0/307026545 wait complete. 2024-07-04T08:14:29.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:28 smithi133 ceph-mon[37050]: pgmap v458: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:29.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:28 smithi050 ceph-mon[36577]: pgmap v458: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:30.178 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.h status 2024-07-04T08:14:30.661 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:31.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:30 smithi039 ceph-mon[32062]: pgmap v459: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:31.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:30 smithi133 ceph-mon[37050]: pgmap v459: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:31.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:30 smithi050 ceph-mon[36577]: pgmap v459: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:33.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.029+0000 7f1a6f200640 1 -- 172.21.15.39:0/2819796745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 msgr2=0x7f1a70105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:33.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.029+0000 7f1a6f200640 1 --2- 172.21.15.39:0/2819796745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 0x7f1a70105520 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009900 tx=0x7f1a6002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.029+0000 7f1a6f200640 1 -- 172.21.15.39:0/2819796745 shutdown_connections 2024-07-04T08:14:33.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.029+0000 7f1a6f200640 1 --2- 172.21.15.39:0/2819796745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a7010aa50 0x7f1a7010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.029+0000 7f1a6f200640 1 --2- 172.21.15.39:0/2819796745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a7010b3a0 0x7f1a70111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.029+0000 7f1a6f200640 1 --2- 172.21.15.39:0/2819796745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 0x7f1a70105520 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.029+0000 7f1a6f200640 1 -- 172.21.15.39:0/2819796745 >> 172.21.15.39:0/2819796745 conn(0x7f1a70100b80 msgr2=0x7f1a70102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:33.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.029+0000 7f1a6f200640 1 -- 172.21.15.39:0/2819796745 shutdown_connections 2024-07-04T08:14:33.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.029+0000 7f1a6f200640 1 -- 172.21.15.39:0/2819796745 wait complete. 2024-07-04T08:14:33.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.031+0000 7f1a6f200640 1 Processor -- start 2024-07-04T08:14:33.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.031+0000 7f1a6f200640 1 -- start start 2024-07-04T08:14:33.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.031+0000 7f1a6f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 0x7f1a7019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.031+0000 7f1a6f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a7010aa50 0x7f1a7019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.031+0000 7f1a6f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a7010b3a0 0x7f1a701a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.031+0000 7f1a6f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a701150a0 con 0x7f1a7010aa50 2024-07-04T08:14:33.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.031+0000 7f1a6f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70114f20 con 0x7f1a7010b3a0 2024-07-04T08:14:33.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.031+0000 7f1a6f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70115220 con 0x7f1a70105110 2024-07-04T08:14:33.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a7010aa50 0x7f1a7019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 0x7f1a7019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 0x7f1a7019ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49706/0 (socket says 172.21.15.39:49706) 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6de00640 1 -- 172.21.15.39:0/970139577 learned_addr learned my addr 172.21.15.39:0/970139577 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a7010b3a0 0x7f1a701a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6e800640 1 -- 172.21.15.39:0/970139577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 msgr2=0x7f1a7019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6e800640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 0x7f1a7019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6e800640 1 -- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a7010aa50 msgr2=0x7f1a7019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6e800640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a7010aa50 0x7f1a7019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6e800640 1 -- 172.21.15.39:0/970139577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a701a19a0 con 0x7f1a7010b3a0 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.032+0000 7f1a6de00640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 0x7f1a7019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.033+0000 7f1a6d400640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a7010aa50 0x7f1a7019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:33.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.033+0000 7f1a6e800640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a7010b3a0 0x7f1a701a1270 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800e970 tx=0x7f1a6800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:33.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.033+0000 7f1a66a00640 1 -- 172.21.15.39:0/970139577 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6800ccc0 con 0x7f1a7010b3a0 2024-07-04T08:14:33.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.033+0000 7f1a66a00640 1 -- 172.21.15.39:0/970139577 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a68005020 con 0x7f1a7010b3a0 2024-07-04T08:14:33.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.033+0000 7f1a6f200640 1 -- 172.21.15.39:0/970139577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a701a1c90 con 0x7f1a7010b3a0 2024-07-04T08:14:33.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.033+0000 7f1a66a00640 1 -- 172.21.15.39:0/970139577 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68004a00 con 0x7f1a7010b3a0 2024-07-04T08:14:33.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.033+0000 7f1a6f200640 1 -- 172.21.15.39:0/970139577 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a701a83f0 con 0x7f1a7010b3a0 2024-07-04T08:14:33.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.037+0000 7f1a7b4ff740 1 Processor -- start 2024-07-04T08:14:33.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.037+0000 7f1a66a00640 1 -- 172.21.15.39:0/970139577 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f1a68004ba0 con 0x7f1a7010b3a0 2024-07-04T08:14:33.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.037+0000 7f1a7b4ff740 1 -- start start 2024-07-04T08:14:33.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 0x55bea63be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff9280 0x55bea63bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bea6334c80 con 0x55bea5ff10a0 2024-07-04T08:14:33.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bea6324220 con 0x55bea5ff9280 2024-07-04T08:14:33.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bea6328260 con 0x55bea6019f60 2024-07-04T08:14:33.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63bed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35086/0 (socket says 172.21.15.39:35086) 2024-07-04T08:14:33.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a6d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 0x55bea63be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a6de00640 1 -- 172.21.15.39:0/485556372 learned_addr learned my addr 172.21.15.39:0/485556372 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:33.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.038+0000 7f1a6d400640 1 --2- 172.21.15.39:0/485556372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff9280 0x55bea63bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.039+0000 7f1a6de00640 1 -- 172.21.15.39:0/485556372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 msgr2=0x55bea63be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:33.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.039+0000 7f1a6de00640 1 --2- 172.21.15.39:0/485556372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 0x55bea63be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.039+0000 7f1a6de00640 1 -- 172.21.15.39:0/485556372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff9280 msgr2=0x55bea63bf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:33.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.039+0000 7f1a6de00640 1 --2- 172.21.15.39:0/485556372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff9280 0x55bea63bf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.039+0000 7f1a6de00640 1 -- 172.21.15.39:0/485556372 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea62bc530 con 0x55bea5ff10a0 2024-07-04T08:14:33.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.039+0000 7f1a6d400640 1 --2- 172.21.15.39:0/485556372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff9280 0x55bea63bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:33.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.039+0000 7f1a6de00640 1 --2- 172.21.15.39:0/485556372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63bed70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1a60006010 tx=0x7f1a60004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28626c4789dc8fe4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:33.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.039+0000 7f1a66a00640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a500b28c0 0x7f1a500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.040+0000 7f1a5f400640 1 -- 172.21.15.39:0/485556372 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60040070 con 0x55bea5ff10a0 2024-07-04T08:14:33.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.040+0000 7f1a5f400640 1 -- 172.21.15.39:0/485556372 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a600047a0 con 0x55bea5ff10a0 2024-07-04T08:14:33.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.040+0000 7f1a5f400640 1 -- 172.21.15.39:0/485556372 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60035040 con 0x55bea5ff10a0 2024-07-04T08:14:33.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.040+0000 7f1a66a00640 1 -- 172.21.15.39:0/970139577 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1a680e2630 con 0x7f1a7010b3a0 2024-07-04T08:14:33.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.040+0000 7f1a6d400640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a500b28c0 0x7f1a500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/485556372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 msgr2=0x55bea63bed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/485556372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63bed70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1a60006010 tx=0x7f1a60004560 comp rx=0 tx=0).stop 2024-07-04T08:14:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a6d400640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a500b28c0 0x7f1a500b4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009fd0 tx=0x7f1a58004ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/485556372 shutdown_connections 2024-07-04T08:14:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/485556372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63bed70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/485556372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff9280 0x55bea63bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/485556372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 0x55bea63be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/485556372 >> 172.21.15.39:0/485556372 conn(0x55bea600e350 msgr2=0x55bea63bc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:33.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/485556372 shutdown_connections 2024-07-04T08:14:33.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.041+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/485556372 wait complete. 2024-07-04T08:14:33.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.042+0000 7f1a7b4ff740 1 -- start start 2024-07-04T08:14:33.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.042+0000 7f1a7b4ff740 1 Processor -- start 2024-07-04T08:14:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 0x55bea63ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea6019f60 0x55bea63d6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bea61885f0 con 0x55bea6019f60 2024-07-04T08:14:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bea62cb950 con 0x55bea5ff10a0 2024-07-04T08:14:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bea63ccfb0 con 0x55bea5ff9280 2024-07-04T08:14:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a7b4ff740 10 client.0 _ll_register_callbacks cb 0x55bea631d480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:33.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45630/0 (socket says 172.21.15.39:45630) 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a6de00640 1 -- 172.21.15.39:0/2905240848 learned_addr learned my addr 172.21.15.39:0/2905240848 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a6d400640 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 0x55bea63ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a6e800640 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea6019f60 0x55bea63d6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.043+0000 7f1a6de00640 1 -- 172.21.15.39:0/2905240848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 msgr2=0x55bea63ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.044+0000 7f1a6de00640 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 0x55bea63ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.044+0000 7f1a6de00640 1 -- 172.21.15.39:0/2905240848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea6019f60 msgr2=0x55bea63d6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.044+0000 7f1a6de00640 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea6019f60 0x55bea63d6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.044+0000 7f1a6de00640 1 -- 172.21.15.39:0/2905240848 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea6085540 con 0x55bea5ff10a0 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.044+0000 7f1a6de00640 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63ca3b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1a6002a9e0 tx=0x7f1a60004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dda506f10444c881 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:33.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.044+0000 7f1a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.044+0000 7f1a57e00640 1 -- 172.21.15.39:0/2905240848 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60040030 con 0x55bea5ff10a0 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.044+0000 7f1a57e00640 1 -- 172.21.15.39:0/2905240848 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1a60003d80 con 0x55bea5ff10a0 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.044+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/2905240848 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a600047a0 con 0x55bea5ff10a0 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.045+0000 7f1a57e00640 1 -- 172.21.15.39:0/2905240848 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a60056050 con 0x55bea5ff10a0 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.045+0000 7f1a7b4ff740 10 client.25316 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.045+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/2905240848 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bea63d8fd0 con 0x55bea5ff10a0 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.045+0000 7f1a57e00640 1 -- 172.21.15.39:0/2905240848 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a600047a0 con 0x55bea5ff10a0 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.045+0000 7f1a57e00640 1 -- 172.21.15.39:0/2905240848 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1a60031810 con 0x55bea5ff10a0 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.046+0000 7f1a7b4ff740 10 client.25316 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:33.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.046+0000 7f1a7b4ff740 10 client.25316 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-04T08:14:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.046+0000 7f1a7b4ff740 20 client.25316 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.046+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bea5fd88e0 0x55bea63dc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.046+0000 7f1a7b4ff740 4 client.25316 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:14:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.046+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/2905240848 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bea6336bd0 con 0x55bea5fd88e0 2024-07-04T08:14:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.046+0000 7f1a6d400640 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bea5fd88e0 0x55bea63dc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.047+0000 7f1a6d400640 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bea5fd88e0 0x55bea63dc600 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=36a5016f3b6f3415 server_cookie=e009b35039937d11 in_seq=0 out_seq=0 2024-07-04T08:14:33.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.047+0000 7f1a57e00640 10 client.25316 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:14:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a57e00640 1 -- 172.21.15.39:0/2905240848 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bea6336bd0 con 0x55bea5fd88e0 2024-07-04T08:14:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a57e00640 10 client.25316 handle_command_reply: tid=0 2024-07-04T08:14:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 client.25316 shutdown 2024-07-04T08:14:33.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/2905240848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 msgr2=0x55bea63ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63ca3b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1a6002a9e0 tx=0x7f1a60004c00 comp rx=0 tx=0).stop 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/2905240848 shutdown_connections 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bea5fd88e0 0x55bea63dc600 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea6019f60 0x55bea63d6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 0x55bea63ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a57e00640 0 client.25316 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/2905240848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63ca3b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/2905240848 >> 172.21.15.39:0/2905240848 conn(0x55bea600e350 msgr2=0x55bea63c19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.048+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/2905240848 shutdown_connections 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.049+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/2905240848 wait complete. 2024-07-04T08:14:33.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.049+0000 7f1a7b4ff740 20 client.25316 trim_cache size 0 max 16384 2024-07-04T08:14:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.070+0000 7f1a7b4ff740 1 Processor -- start 2024-07-04T08:14:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.070+0000 7f1a7b4ff740 1 -- start start 2024-07-04T08:14:33.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.070+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.070+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff9280 0x55bea63bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.070+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 0x55bea63d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.070+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bea63da940 con 0x55bea5ff9280 2024-07-04T08:14:33.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.070+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bea63db010 con 0x55bea5ff10a0 2024-07-04T08:14:33.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.070+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bea63db6e0 con 0x55bea6019f60 2024-07-04T08:14:33.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.071+0000 7f1a6d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff9280 0x55bea63bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.071+0000 7f1a6d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff9280 0x55bea63bff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35110/0 (socket says 172.21.15.39:35110) 2024-07-04T08:14:33.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.071+0000 7f1a6d400640 1 -- 172.21.15.39:0/4166005952 learned_addr learned my addr 172.21.15.39:0/4166005952 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:33.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.071+0000 7f1a6de00640 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.071+0000 7f1a6e800640 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 0x55bea63d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.071+0000 7f1a6de00640 1 -- 172.21.15.39:0/4166005952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 msgr2=0x55bea63d6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.072+0000 7f1a6de00640 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 0x55bea63d6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.072+0000 7f1a6de00640 1 -- 172.21.15.39:0/4166005952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff9280 msgr2=0x55bea63bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.072+0000 7f1a6de00640 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff9280 0x55bea63bff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.072+0000 7f1a6de00640 1 -- 172.21.15.39:0/4166005952 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea63c1690 con 0x55bea5ff10a0 2024-07-04T08:14:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.072+0000 7f1a6d400640 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff9280 0x55bea63bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.072+0000 7f1a6de00640 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63c5360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1a60031bb0 tx=0x7f1a60005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f326e442b00fce45 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.072+0000 7f1a5ea00640 1 -- 172.21.15.39:0/4166005952 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6005f070 con 0x55bea5ff10a0 2024-07-04T08:14:33.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.072+0000 7f1a5ea00640 1 -- 172.21.15.39:0/4166005952 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a6004d070 con 0x55bea5ff10a0 2024-07-04T08:14:33.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.072+0000 7f1a5ea00640 1 -- 172.21.15.39:0/4166005952 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60040070 con 0x55bea5ff10a0 2024-07-04T08:14:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.073+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/4166005952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 msgr2=0x55bea63c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.073+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63c5360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1a60031bb0 tx=0x7f1a60005a20 comp rx=0 tx=0).stop 2024-07-04T08:14:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.073+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/4166005952 shutdown_connections 2024-07-04T08:14:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.073+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff9280 0x55bea63bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.073+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea6019f60 0x55bea63d6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.073+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/4166005952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea5ff10a0 0x55bea63c5360 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.073+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/4166005952 >> 172.21.15.39:0/4166005952 conn(0x55bea600e350 msgr2=0x55bea63bf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.074+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/4166005952 shutdown_connections 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.074+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/4166005952 wait complete. 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.074+0000 7f1a7b4ff740 1 -- start start 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.074+0000 7f1a7b4ff740 1 Processor -- start 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 0x55bea63de520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a7b4ff740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea6019f60 0x55bea63deb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bea63dc480 con 0x55bea5ff10a0 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bea63dbdb0 con 0x55bea6019f60 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a7b4ff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bea63dcb50 con 0x55bea5ff9280 2024-07-04T08:14:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a7b4ff740 10 client.0 _ll_register_callbacks cb 0x7f1a58012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a6de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63e5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35118/0 (socket says 172.21.15.39:35118) 2024-07-04T08:14:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a6de00640 1 -- 172.21.15.39:0/3302187704 learned_addr learned my addr 172.21.15.39:0/3302187704 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a6d400640 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 0x55bea63de520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.075+0000 7f1a6e800640 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea6019f60 0x55bea63deb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a6de00640 1 -- 172.21.15.39:0/3302187704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 msgr2=0x55bea63de520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a6de00640 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 0x55bea63de520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a6de00640 1 -- 172.21.15.39:0/3302187704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea6019f60 msgr2=0x55bea63deb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a6de00640 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea6019f60 0x55bea63deb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a6de00640 1 -- 172.21.15.39:0/3302187704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea63df2e0 con 0x55bea5ff10a0 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a6e800640 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea6019f60 0x55bea63deb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a6de00640 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63e5070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1a60031df0 tx=0x7f1a60005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b52f98c9bd1e39 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a57e00640 1 -- 172.21.15.39:0/3302187704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6005f040 con 0x55bea5ff10a0 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a57e00640 1 -- 172.21.15.39:0/3302187704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1a600316a0 con 0x55bea5ff10a0 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.076+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/3302187704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a6004d070 con 0x55bea5ff10a0 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.077+0000 7f1a57e00640 1 -- 172.21.15.39:0/3302187704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a6006e050 con 0x55bea5ff10a0 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.077+0000 7f1a7b4ff740 10 client.15198 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.077+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/3302187704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bea6445900 con 0x55bea5ff10a0 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.077+0000 7f1a57e00640 1 -- 172.21.15.39:0/3302187704 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a6004d070 con 0x55bea5ff10a0 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.077+0000 7f1a57e00640 1 -- 172.21.15.39:0/3302187704 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1a60062070 con 0x55bea5ff10a0 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.077+0000 7f1a7b4ff740 10 client.15198 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.078+0000 7f1a7b4ff740 10 client.15198 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.078+0000 7f1a7b4ff740 20 client.15198 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.078+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bea5fadc90 0x55bea6448dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.078+0000 7f1a7b4ff740 4 client.15198 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:14:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.078+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/3302187704 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "status"}) -- 0x55bea6332670 con 0x55bea5fadc90 2024-07-04T08:14:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.078+0000 7f1a6d400640 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bea5fadc90 0x55bea6448dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a6d400640 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bea5fadc90 0x55bea6448dc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e7f376d32232120d server_cookie=34d8dc17bf50a072 in_seq=0 out_seq=0 2024-07-04T08:14:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a57e00640 10 client.15198 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:14:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a57e00640 1 -- 172.21.15.39:0/3302187704 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bea6332670 con 0x55bea5fadc90 2024-07-04T08:14:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a57e00640 10 client.15198 handle_command_reply: tid=0 2024-07-04T08:14:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a7b4ff740 1 client.15198 shutdown 2024-07-04T08:14:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/3302187704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 msgr2=0x55bea63e5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63e5070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1a60031df0 tx=0x7f1a60005fd0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/3302187704 shutdown_connections 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bea5fadc90 0x55bea6448dc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bea6019f60 0x55bea63deb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bea5ff9280 0x55bea63de520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a7b4ff740 1 --2- 172.21.15.39:0/3302187704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bea5ff10a0 0x55bea63e5070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a57e00640 0 client.15198 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.079+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/3302187704 >> 172.21.15.39:0/3302187704 conn(0x55bea600e350 msgr2=0x55bea63c8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.080+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/3302187704 shutdown_connections 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.080+0000 7f1a7b4ff740 1 -- 172.21.15.39:0/3302187704 wait complete. 2024-07-04T08:14:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.080+0000 7f1a7b4ff740 20 client.15198 trim_cache size 0 max 16384 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24674, 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 209.271165384, 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:14:33.086 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:14:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:14:33.087 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 325.56804758800001 2024-07-04T08:14:33.087 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:14:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.080+0000 7f1a6f200640 1 -- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a500b28c0 msgr2=0x7f1a500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.080+0000 7f1a6f200640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a500b28c0 0x7f1a500b4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009fd0 tx=0x7f1a58004ab0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.080+0000 7f1a6f200640 1 -- 172.21.15.39:0/970139577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a7010b3a0 msgr2=0x7f1a701a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.080+0000 7f1a6f200640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a7010b3a0 0x7f1a701a1270 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800e970 tx=0x7f1a6800ee30 comp rx=0 tx=0).stop 2024-07-04T08:14:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.081+0000 7f1a6f200640 1 -- 172.21.15.39:0/970139577 shutdown_connections 2024-07-04T08:14:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.081+0000 7f1a6f200640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a500b28c0 0x7f1a500b4dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.081+0000 7f1a6f200640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a7010aa50 0x7f1a7019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.081+0000 7f1a6f200640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a7010b3a0 0x7f1a701a1270 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.081+0000 7f1a6f200640 1 --2- 172.21.15.39:0/970139577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a70105110 0x7f1a7019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.081+0000 7f1a6f200640 1 -- 172.21.15.39:0/970139577 >> 172.21.15.39:0/970139577 conn(0x7f1a70100b80 msgr2=0x7f1a70102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.081+0000 7f1a6f200640 1 -- 172.21.15.39:0/970139577 shutdown_connections 2024-07-04T08:14:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:33.081+0000 7f1a6f200640 1 -- 172.21.15.39:0/970139577 wait complete. 2024-07-04T08:14:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:32 smithi133 ceph-mon[37050]: pgmap v460: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:33.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:32 smithi039 ceph-mon[32062]: pgmap v460: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:32 smithi050 ceph-mon[36577]: pgmap v460: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:33.976 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.i status 2024-07-04T08:14:34.488 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:35.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:34 smithi133 ceph-mon[37050]: pgmap v461: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:35.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:34 smithi039 ceph-mon[32062]: pgmap v461: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:35.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:34 smithi050 ceph-mon[36577]: pgmap v461: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.911+0000 7fadcaa00640 1 -- 172.21.15.39:0/1369723333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410ee50 msgr2=0x7fadc410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.911+0000 7fadcaa00640 1 --2- 172.21.15.39:0/1369723333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410ee50 0x7fadc410f260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009900 tx=0x7fadb002a400 comp rx=0 tx=0).stop 2024-07-04T08:14:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.911+0000 7fadcaa00640 1 -- 172.21.15.39:0/1369723333 shutdown_connections 2024-07-04T08:14:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.911+0000 7fadcaa00640 1 --2- 172.21.15.39:0/1369723333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410ee50 0x7fadc410f260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.911+0000 7fadcaa00640 1 --2- 172.21.15.39:0/1369723333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadc410f7a0 0x7fadc4115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.911+0000 7fadcaa00640 1 --2- 172.21.15.39:0/1369723333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4109510 0x7fadc4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.911+0000 7fadcaa00640 1 -- 172.21.15.39:0/1369723333 >> 172.21.15.39:0/1369723333 conn(0x7fadc4078b10 msgr2=0x7fadc4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.911+0000 7fadcaa00640 1 -- 172.21.15.39:0/1369723333 shutdown_connections 2024-07-04T08:14:36.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.911+0000 7fadcaa00640 1 -- 172.21.15.39:0/1369723333 wait complete. 2024-07-04T08:14:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.912+0000 7fadcaa00640 1 Processor -- start 2024-07-04T08:14:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.913+0000 7fadcaa00640 1 -- start start 2024-07-04T08:14:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.913+0000 7fadcaa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4109510 0x7fadc419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.913+0000 7fadcaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadc410ee50 0x7fadc41a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.913+0000 7fadcaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410f7a0 0x7fadc41a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.913+0000 7fadcaa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4119430 con 0x7fadc410f7a0 2024-07-04T08:14:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.913+0000 7fadcaa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41192b0 con 0x7fadc4109510 2024-07-04T08:14:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.913+0000 7fadcaa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41195b0 con 0x7fadc410ee50 2024-07-04T08:14:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadca000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410f7a0 0x7fadc41a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadc9600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4109510 0x7fadc419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadca000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410f7a0 0x7fadc41a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35142/0 (socket says 172.21.15.39:35142) 2024-07-04T08:14:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadc9600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4109510 0x7fadc419fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45662/0 (socket says 172.21.15.39:45662) 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadca000640 1 -- 172.21.15.39:0/3803290778 learned_addr learned my addr 172.21.15.39:0/3803290778 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadc8c00640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadc410ee50 0x7fadc41a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadc9600640 1 -- 172.21.15.39:0/3803290778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadc410ee50 msgr2=0x7fadc41a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadc9600640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadc410ee50 0x7fadc41a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadc9600640 1 -- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410f7a0 msgr2=0x7fadc41a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadc9600640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410f7a0 0x7fadc41a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadc9600640 1 -- 172.21.15.39:0/3803290778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc41a5db0 con 0x7fadc4109510 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.914+0000 7fadca000640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410f7a0 0x7fadc41a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.915+0000 7fadc9600640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4109510 0x7fadc419fe40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fadb8005fd0 tx=0x7fadb800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.915+0000 7fadc2000640 1 -- 172.21.15.39:0/3803290778 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8009c30 con 0x7fadc4109510 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.915+0000 7fadcaa00640 1 -- 172.21.15.39:0/3803290778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc41a60a0 con 0x7fadc4109510 2024-07-04T08:14:36.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.915+0000 7fadc2000640 1 -- 172.21.15.39:0/3803290778 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb8009dd0 con 0x7fadc4109510 2024-07-04T08:14:36.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.915+0000 7fadc2000640 1 -- 172.21.15.39:0/3803290778 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8003e20 con 0x7fadc4109510 2024-07-04T08:14:36.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.915+0000 7fadcaa00640 1 -- 172.21.15.39:0/3803290778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41ac800 con 0x7fadc4109510 2024-07-04T08:14:36.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.919+0000 7fadd259a740 1 Processor -- start 2024-07-04T08:14:36.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.919+0000 7fadd259a740 1 -- start start 2024-07-04T08:14:36.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.919+0000 7fadd259a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x5567375d3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.919+0000 7fadc2000640 1 -- 172.21.15.39:0/3803290778 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7fadb8004000 con 0x7fadc4109510 2024-07-04T08:14:36.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.919+0000 7fadd259a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.919+0000 7fadd259a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.919+0000 7fadd259a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556737549ca0 con 0x5567372060a0 2024-07-04T08:14:36.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.919+0000 7fadd259a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556737539240 con 0x55673722ef60 2024-07-04T08:14:36.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.919+0000 7fadd259a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55673753d280 con 0x55673720e280 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.920+0000 7fadca000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x5567375d3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.920+0000 7fadca000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x5567375d3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45678/0 (socket says 172.21.15.39:45678) 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.920+0000 7fadca000640 1 -- 172.21.15.39:0/3665832057 learned_addr learned my addr 172.21.15.39:0/3665832057 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.920+0000 7fadc8c00640 1 --2- 172.21.15.39:0/3665832057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.920+0000 7fadc9600640 1 --2- 172.21.15.39:0/3665832057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.920+0000 7fadc8c00640 1 -- 172.21.15.39:0/3665832057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 msgr2=0x5567375d3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.920+0000 7fadc8c00640 1 --2- 172.21.15.39:0/3665832057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x5567375d3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.920+0000 7fadc8c00640 1 -- 172.21.15.39:0/3665832057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 msgr2=0x5567375d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.920+0000 7fadc8c00640 1 --2- 172.21.15.39:0/3665832057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375d3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.921+0000 7fadc8c00640 1 -- 172.21.15.39:0/3665832057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567374d1550 con 0x55673720e280 2024-07-04T08:14:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.921+0000 7fadc8c00640 1 --2- 172.21.15.39:0/3665832057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375d42d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fadb002a920 tx=0x7fadb0032ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e450ab0cc6dc9371 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.921+0000 7fadb6a00640 1 -- 172.21.15.39:0/3665832057 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb003f070 con 0x55673720e280 2024-07-04T08:14:36.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.922+0000 7fadb6a00640 1 -- 172.21.15.39:0/3665832057 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb0004030 con 0x55673720e280 2024-07-04T08:14:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.922+0000 7fadb6a00640 1 -- 172.21.15.39:0/3665832057 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0035070 con 0x55673720e280 2024-07-04T08:14:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.922+0000 7fadc2000640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fadac0b2840 0x7fadac0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.922+0000 7fadd259a740 1 -- 172.21.15.39:0/3665832057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 msgr2=0x5567375d42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:36.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.922+0000 7fadd259a740 1 --2- 172.21.15.39:0/3665832057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375d42d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fadb002a920 tx=0x7fadb0032ba0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadc2000640 1 -- 172.21.15.39:0/3803290778 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fadb80e2320 con 0x7fadc4109510 2024-07-04T08:14:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadd259a740 1 -- 172.21.15.39:0/3665832057 shutdown_connections 2024-07-04T08:14:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadd259a740 1 --2- 172.21.15.39:0/3665832057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375d3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadd259a740 1 --2- 172.21.15.39:0/3665832057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375d42d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadd259a740 1 --2- 172.21.15.39:0/3665832057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x5567375d3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadd259a740 1 -- 172.21.15.39:0/3665832057 >> 172.21.15.39:0/3665832057 conn(0x556737223350 msgr2=0x5567375d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadca000640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fadac0b2840 0x7fadac0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadd259a740 1 -- 172.21.15.39:0/3665832057 shutdown_connections 2024-07-04T08:14:36.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadd259a740 1 -- 172.21.15.39:0/3665832057 wait complete. 2024-07-04T08:14:36.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.923+0000 7fadd259a740 1 -- start start 2024-07-04T08:14:36.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.924+0000 7fadd259a740 1 Processor -- start 2024-07-04T08:14:36.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.924+0000 7fadca000640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fadac0b2840 0x7fadac0b4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0049d0 tx=0x7fadbc009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:36.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.924+0000 7fadd259a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567372060a0 0x5567375df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.924+0000 7fadd259a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375df950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.924+0000 7fadd259a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673722ef60 0x5567375eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.924+0000 7fadd259a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55673739d5f0 con 0x55673722ef60 2024-07-04T08:14:36.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.924+0000 7fadd259a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5567374e0970 con 0x5567372060a0 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.924+0000 7fadd259a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5567375e1fd0 con 0x55673720e280 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.924+0000 7fadd259a740 10 client.0 _ll_register_callbacks cb 0x5567375324a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadc8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375df950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadc8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375df950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49784/0 (socket says 172.21.15.39:49784) 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadc8c00640 1 -- 172.21.15.39:0/2642925027 learned_addr learned my addr 172.21.15.39:0/2642925027 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadc9600640 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567372060a0 0x5567375df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadca000640 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673722ef60 0x5567375eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadc8c00640 1 -- 172.21.15.39:0/2642925027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567372060a0 msgr2=0x5567375df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadc8c00640 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567372060a0 0x5567375df410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadc8c00640 1 -- 172.21.15.39:0/2642925027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673722ef60 msgr2=0x5567375eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadc8c00640 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673722ef60 0x5567375eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadc8c00640 1 -- 172.21.15.39:0/2642925027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673729a540 con 0x55673720e280 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.925+0000 7fadca000640 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673722ef60 0x5567375eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.926+0000 7fadc8c00640 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375df950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fadb002ab50 tx=0x7fadb0005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc261aeb82cdac78 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.926+0000 7fadab400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.926+0000 7fadab400640 1 -- 172.21.15.39:0/2642925027 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0035070 con 0x55673720e280 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.926+0000 7fadab400640 1 -- 172.21.15.39:0/2642925027 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fadb0005140 con 0x55673720e280 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.926+0000 7fadd259a740 1 -- 172.21.15.39:0/2642925027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb0004030 con 0x55673720e280 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.926+0000 7fadab400640 1 -- 172.21.15.39:0/2642925027 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadb0055050 con 0x55673720e280 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.926+0000 7fadd259a740 10 client.25315 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.926+0000 7fadd259a740 1 -- 172.21.15.39:0/2642925027 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567375edf90 con 0x55673720e280 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.927+0000 7fadab400640 1 -- 172.21.15.39:0/2642925027 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb0004030 con 0x55673720e280 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.927+0000 7fadab400640 1 -- 172.21.15.39:0/2642925027 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fadb00035f0 con 0x55673720e280 2024-07-04T08:14:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.927+0000 7fadd259a740 10 client.25315 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.927+0000 7fadd259a740 10 client.25315 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-04T08:14:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.927+0000 7fadd259a740 20 client.25315 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.927+0000 7fadd259a740 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5567371ed8e0 0x5567375f1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.927+0000 7fadd259a740 4 client.25315 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:14:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.927+0000 7fadd259a740 1 -- 172.21.15.39:0/2642925027 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55673754bbf0 con 0x5567371ed8e0 2024-07-04T08:14:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.927+0000 7fadc9600640 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5567371ed8e0 0x5567375f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.928+0000 7fadc9600640 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5567371ed8e0 0x5567375f1570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fade0f18260f93b server_cookie=de70cb2b849571bb in_seq=0 out_seq=0 2024-07-04T08:14:36.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.928+0000 7fadab400640 10 client.25315 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:14:36.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadab400640 1 -- 172.21.15.39:0/2642925027 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55673754bbf0 con 0x5567371ed8e0 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadab400640 10 client.25315 handle_command_reply: tid=0 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 client.25315 shutdown 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 -- 172.21.15.39:0/2642925027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 msgr2=0x5567375df950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375df950 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fadb002ab50 tx=0x7fadb0005dd0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 -- 172.21.15.39:0/2642925027 shutdown_connections 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5567371ed8e0 0x5567375f1570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673722ef60 0x5567375eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375df950 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 --2- 172.21.15.39:0/2642925027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567372060a0 0x5567375df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 -- 172.21.15.39:0/2642925027 >> 172.21.15.39:0/2642925027 conn(0x556737223350 msgr2=0x5567375d6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadab400640 0 client.25315 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 -- 172.21.15.39:0/2642925027 shutdown_connections 2024-07-04T08:14:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.929+0000 7fadd259a740 1 -- 172.21.15.39:0/2642925027 wait complete. 2024-07-04T08:14:36.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.930+0000 7fadd259a740 20 client.25315 trim_cache size 0 max 16384 2024-07-04T08:14:36.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.951+0000 7fadd259a740 1 Processor -- start 2024-07-04T08:14:36.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.951+0000 7fadd259a740 1 -- start start 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.951+0000 7fadd259a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375f0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.951+0000 7fadd259a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375de720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.951+0000 7fadd259a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x5567375d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.951+0000 7fadd259a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5567375ef8b0 con 0x5567372060a0 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.951+0000 7fadd259a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5567375eff80 con 0x55673722ef60 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.951+0000 7fadd259a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5567375f0650 con 0x55673720e280 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375de720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375f0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35166/0 (socket says 172.21.15.39:35166) 2024-07-04T08:14:36.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc9600640 1 -- 172.21.15.39:0/16318024 learned_addr learned my addr 172.21.15.39:0/16318024 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:36.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadca000640 1 --2- 172.21.15.39:0/16318024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x5567375d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc8c00640 1 -- 172.21.15.39:0/16318024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 msgr2=0x5567375d9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:36.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc8c00640 1 --2- 172.21.15.39:0/16318024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x5567375d9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc8c00640 1 -- 172.21.15.39:0/16318024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 msgr2=0x5567375f0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:36.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc8c00640 1 --2- 172.21.15.39:0/16318024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375f0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc8c00640 1 -- 172.21.15.39:0/16318024 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567375daaa0 con 0x55673720e280 2024-07-04T08:14:36.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.952+0000 7fadc9600640 1 --2- 172.21.15.39:0/16318024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375f0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:36.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.953+0000 7fadc8c00640 1 --2- 172.21.15.39:0/16318024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375de720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fadb002a920 tx=0x7fadb002a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=537a9552dbd9ff38 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:36.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.953+0000 7fadb6000640 1 -- 172.21.15.39:0/16318024 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0035020 con 0x55673720e280 2024-07-04T08:14:36.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.953+0000 7fadb6000640 1 -- 172.21.15.39:0/16318024 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb004c070 con 0x55673720e280 2024-07-04T08:14:36.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.953+0000 7fadb6000640 1 -- 172.21.15.39:0/16318024 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb003f040 con 0x55673720e280 2024-07-04T08:14:36.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.954+0000 7fadd259a740 1 -- 172.21.15.39:0/16318024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 msgr2=0x5567375de720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:36.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.954+0000 7fadd259a740 1 --2- 172.21.15.39:0/16318024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375de720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fadb002a920 tx=0x7fadb002a400 comp rx=0 tx=0).stop 2024-07-04T08:14:36.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.954+0000 7fadd259a740 1 -- 172.21.15.39:0/16318024 shutdown_connections 2024-07-04T08:14:36.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.954+0000 7fadd259a740 1 --2- 172.21.15.39:0/16318024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x5567375d9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.954+0000 7fadd259a740 1 --2- 172.21.15.39:0/16318024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673720e280 0x5567375de720 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.954+0000 7fadd259a740 1 --2- 172.21.15.39:0/16318024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567372060a0 0x5567375f0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.954+0000 7fadd259a740 1 -- 172.21.15.39:0/16318024 >> 172.21.15.39:0/16318024 conn(0x556737223350 msgr2=0x5567375d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:36.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.954+0000 7fadd259a740 1 -- 172.21.15.39:0/16318024 shutdown_connections 2024-07-04T08:14:36.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.954+0000 7fadd259a740 1 -- 172.21.15.39:0/16318024 wait complete. 2024-07-04T08:14:36.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.955+0000 7fadd259a740 1 -- start start 2024-07-04T08:14:36.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.955+0000 7fadd259a740 1 Processor -- start 2024-07-04T08:14:36.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.955+0000 7fadd259a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567372060a0 0x5567375d8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.955+0000 7fadd259a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673720e280 0x5567375d8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadd259a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x556737658800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadd259a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5567375f13f0 con 0x55673720e280 2024-07-04T08:14:36.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadd259a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5567375f0d20 con 0x55673722ef60 2024-07-04T08:14:36.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadd259a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5567375f1ac0 con 0x5567372060a0 2024-07-04T08:14:36.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadd259a740 10 client.0 _ll_register_callbacks cb 0x7fadb80e5c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:36.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673720e280 0x5567375d8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673720e280 0x5567375d8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35172/0 (socket says 172.21.15.39:35172) 2024-07-04T08:14:36.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadca000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x556737658800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc8c00640 1 -- 172.21.15.39:0/533594613 learned_addr learned my addr 172.21.15.39:0/533594613 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:36.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc9600640 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567372060a0 0x5567375d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc9600640 1 -- 172.21.15.39:0/533594613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 msgr2=0x556737658800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:36.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc9600640 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x556737658800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc9600640 1 -- 172.21.15.39:0/533594613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673720e280 msgr2=0x5567375d8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:36.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc9600640 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673720e280 0x5567375d8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc9600640 1 -- 172.21.15.39:0/533594613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556737658d40 con 0x5567372060a0 2024-07-04T08:14:36.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.956+0000 7fadc8c00640 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673720e280 0x5567375d8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:36.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.957+0000 7fadc9600640 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567372060a0 0x5567375d8540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fadb80a5510 tx=0x7fadb80a5470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef7a766ee41e341 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:36.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.957+0000 7fadab400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:14:36.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.957+0000 7fadab400640 1 -- 172.21.15.39:0/533594613 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8020040 con 0x5567372060a0 2024-07-04T08:14:36.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.957+0000 7fadab400640 1 -- 172.21.15.39:0/533594613 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fadb80035f0 con 0x5567372060a0 2024-07-04T08:14:36.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.957+0000 7fadd259a740 1 -- 172.21.15.39:0/533594613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb004c070 con 0x5567372060a0 2024-07-04T08:14:36.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadab400640 1 -- 172.21.15.39:0/533594613 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fadb80a8fa0 con 0x5567372060a0 2024-07-04T08:14:36.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadd259a740 10 client.25321 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:36.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadd259a740 1 -- 172.21.15.39:0/533594613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556737659f50 con 0x5567372060a0 2024-07-04T08:14:36.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadab400640 1 -- 172.21.15.39:0/533594613 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb004c070 con 0x5567372060a0 2024-07-04T08:14:36.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadab400640 1 -- 172.21.15.39:0/533594613 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fadb80a5720 con 0x5567372060a0 2024-07-04T08:14:36.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadd259a740 10 client.25321 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:36.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadd259a740 10 client.25321 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-04T08:14:36.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadd259a740 20 client.25321 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadd259a740 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5567371c2c90 0x55673765d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadd259a740 4 client.25321 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.958+0000 7fadd259a740 1 -- 172.21.15.39:0/533594613 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "status"}) -- 0x556737547690 con 0x5567371c2c90 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.959+0000 7fadc8c00640 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5567371c2c90 0x55673765d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.959+0000 7fadab400640 10 client.25321 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.959+0000 7fadc8c00640 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5567371c2c90 0x55673765d890 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b934557b50e1b35d server_cookie=5351ac25e0b4910c in_seq=0 out_seq=0 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadab400640 1 -- 172.21.15.39:0/533594613 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556737547690 con 0x5567371c2c90 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadab400640 10 client.25321 handle_command_reply: tid=0 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 client.25321 shutdown 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 -- 172.21.15.39:0/533594613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567372060a0 msgr2=0x5567375d8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567372060a0 0x5567375d8540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fadb80a5510 tx=0x7fadb80a5470 comp rx=0 tx=0).stop 2024-07-04T08:14:36.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 -- 172.21.15.39:0/533594613 shutdown_connections 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5567371c2c90 0x55673765d890 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55673720e280 0x5567375d8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55673722ef60 0x556737658800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 --2- 172.21.15.39:0/533594613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567372060a0 0x5567375d8540 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 -- 172.21.15.39:0/533594613 >> 172.21.15.39:0/533594613 conn(0x556737223350 msgr2=0x5567375cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadab400640 0 client.25321 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.960+0000 7fadd259a740 1 -- 172.21.15.39:0/533594613 shutdown_connections 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.961+0000 7fadd259a740 1 -- 172.21.15.39:0/533594613 wait complete. 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.961+0000 7fadd259a740 20 client.25321 trim_cache size 0 max 16384 2024-07-04T08:14:36.964 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24679, 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 214.18154540800001, 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 326.77545432599999 2024-07-04T08:14:36.965 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:14:36.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 -- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fadac0b2840 msgr2=0x7fadac0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:36.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fadac0b2840 0x7fadac0b4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0049d0 tx=0x7fadbc009500 comp rx=0 tx=0).stop 2024-07-04T08:14:36.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 -- 172.21.15.39:0/3803290778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4109510 msgr2=0x7fadc419fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:36.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4109510 0x7fadc419fe40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fadb8005fd0 tx=0x7fadb800dde0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 -- 172.21.15.39:0/3803290778 shutdown_connections 2024-07-04T08:14:36.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fadac0b2840 0x7fadac0b4d50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fadc410f7a0 0x7fadc41a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadc410ee50 0x7fadc41a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 --2- 172.21.15.39:0/3803290778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fadc4109510 0x7fadc419fe40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:36.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 -- 172.21.15.39:0/3803290778 >> 172.21.15.39:0/3803290778 conn(0x7fadc4078b10 msgr2=0x7fadc4079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:36.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 -- 172.21.15.39:0/3803290778 shutdown_connections 2024-07-04T08:14:36.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:36.962+0000 7fadcaa00640 1 -- 172.21.15.39:0/3803290778 wait complete. 2024-07-04T08:14:37.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:36 smithi133 ceph-mon[37050]: pgmap v462: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:37.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:36 smithi039 ceph-mon[32062]: pgmap v462: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:37.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:36 smithi050 ceph-mon[36577]: pgmap v462: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:37.831 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.a status 2024-07-04T08:14:38.315 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:39.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:39 smithi039 ceph-mon[32062]: pgmap v463: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:39.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:39 smithi133 ceph-mon[37050]: pgmap v463: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:39 smithi050 ceph-mon[36577]: pgmap v463: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.628+0000 7f51b1000640 1 -- 172.21.15.39:0/4259196548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0777e0 msgr2=0x7f51ac113750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.628+0000 7f51b1000640 1 --2- 172.21.15.39:0/4259196548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0777e0 0x7f51ac113750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5194009900 tx=0x7f519402a410 comp rx=0 tx=0).stop 2024-07-04T08:14:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.628+0000 7f51b1000640 1 -- 172.21.15.39:0/4259196548 shutdown_connections 2024-07-04T08:14:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.628+0000 7f51b1000640 1 --2- 172.21.15.39:0/4259196548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0777e0 0x7f51ac113750 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.628+0000 7f51b1000640 1 --2- 172.21.15.39:0/4259196548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac113c90 0x7f51ac1160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.628+0000 7f51b1000640 1 --2- 172.21.15.39:0/4259196548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51ac076e90 0x7f51ac0772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.628+0000 7f51b1000640 1 -- 172.21.15.39:0/4259196548 >> 172.21.15.39:0/4259196548 conn(0x7f51ac100d60 msgr2=0x7f51ac1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.628+0000 7f51b1000640 1 -- 172.21.15.39:0/4259196548 shutdown_connections 2024-07-04T08:14:40.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.628+0000 7f51b1000640 1 -- 172.21.15.39:0/4259196548 wait complete. 2024-07-04T08:14:40.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.629+0000 7f51b1000640 1 Processor -- start 2024-07-04T08:14:40.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.630+0000 7f51b1000640 1 -- start start 2024-07-04T08:14:40.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.630+0000 7f51b1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51ac076e90 0x7f51ac1a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.630+0000 7f51b1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0777e0 0x7f51ac1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.630+0000 7f51b1000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac113c90 0x7f51ac1a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.630+0000 7f51b1000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac119630 con 0x7f51ac0777e0 2024-07-04T08:14:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.630+0000 7f51b1000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac1194b0 con 0x7f51ac076e90 2024-07-04T08:14:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.630+0000 7f51b1000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac1197b0 con 0x7f51ac113c90 2024-07-04T08:14:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51a9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51ac076e90 0x7f51ac1a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51a1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0777e0 0x7f51ac1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51a9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51ac076e90 0x7f51ac1a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45716/0 (socket says 172.21.15.39:45716) 2024-07-04T08:14:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51a9e00640 1 -- 172.21.15.39:0/4029214991 learned_addr learned my addr 172.21.15.39:0/4029214991 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:40.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51aa800640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac113c90 0x7f51ac1a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51a1400640 1 -- 172.21.15.39:0/4029214991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac113c90 msgr2=0x7f51ac1a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51a1400640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac113c90 0x7f51ac1a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51a1400640 1 -- 172.21.15.39:0/4029214991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51ac076e90 msgr2=0x7f51ac1a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:40.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51a1400640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51ac076e90 0x7f51ac1a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.631+0000 7f51a1400640 1 -- 172.21.15.39:0/4029214991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac1a5f90 con 0x7f51ac0777e0 2024-07-04T08:14:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.632+0000 7f51a1400640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0777e0 0x7f51ac1a05c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5194009230 tx=0x7f5194003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.632+0000 7f51a3400640 1 -- 172.21.15.39:0/4029214991 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519403f070 con 0x7f51ac0777e0 2024-07-04T08:14:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.632+0000 7f51b1000640 1 -- 172.21.15.39:0/4029214991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ac1a6220 con 0x7f51ac0777e0 2024-07-04T08:14:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.632+0000 7f51a3400640 1 -- 172.21.15.39:0/4029214991 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5194031b50 con 0x7f51ac0777e0 2024-07-04T08:14:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.632+0000 7f51a3400640 1 -- 172.21.15.39:0/4029214991 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194035030 con 0x7f51ac0777e0 2024-07-04T08:14:40.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.632+0000 7f51b1000640 1 -- 172.21.15.39:0/4029214991 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ac1ac9e0 con 0x7f51ac0777e0 2024-07-04T08:14:40.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.636+0000 7f51b76e5740 1 Processor -- start 2024-07-04T08:14:40.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.636+0000 7f51b76e5740 1 -- start start 2024-07-04T08:14:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.636+0000 7f51a3400640 1 -- 172.21.15.39:0/4029214991 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f5194042070 con 0x7f51ac0777e0 2024-07-04T08:14:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.636+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 0x5638424b7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51b76e5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56384242dca0 con 0x5638420ea0a0 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51b76e5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56384241d240 con 0x5638420f2280 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51b76e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563842421280 con 0x563842112f60 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51a1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51aa800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 0x5638424b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51a9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51a1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424b80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45718/0 (socket says 172.21.15.39:45718) 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.637+0000 7f51a1400640 1 -- 172.21.15.39:0/3456238332 learned_addr learned my addr 172.21.15.39:0/3456238332 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f51a1400640 1 -- 172.21.15.39:0/3456238332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 msgr2=0x5638424b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f51a1400640 1 --2- 172.21.15.39:0/3456238332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 0x5638424b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f51a1400640 1 -- 172.21.15.39:0/3456238332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 msgr2=0x5638424b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:40.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f51a1400640 1 --2- 172.21.15.39:0/3456238332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f51a1400640 1 -- 172.21.15.39:0/3456238332 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638423b5550 con 0x5638420f2280 2024-07-04T08:14:40.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f51a9e00640 1 --2- 172.21.15.39:0/3456238332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:40.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f51a1400640 1 --2- 172.21.15.39:0/3456238332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424b80f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5194108c20 tx=0x7f5194108d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=942a475f76c0e3a1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f519b400640 1 -- 172.21.15.39:0/3456238332 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194111070 con 0x5638420f2280 2024-07-04T08:14:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f51a3400640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f518c0b2840 0x7f518c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f519b400640 1 -- 172.21.15.39:0/3456238332 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f519410ad60 con 0x5638420f2280 2024-07-04T08:14:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.638+0000 7f519b400640 1 -- 172.21.15.39:0/3456238332 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51940c5910 con 0x5638420f2280 2024-07-04T08:14:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.639+0000 7f51a3400640 1 -- 172.21.15.39:0/4029214991 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f5194102830 con 0x7f51ac0777e0 2024-07-04T08:14:40.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.639+0000 7f51aa800640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f518c0b2840 0x7f518c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51b76e5740 1 -- 172.21.15.39:0/3456238332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 msgr2=0x5638424b80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51b76e5740 1 --2- 172.21.15.39:0/3456238332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424b80f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5194108c20 tx=0x7f5194108d10 comp rx=0 tx=0).stop 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51aa800640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f518c0b2840 0x7f518c0b4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005fd0 tx=0x7f51a4009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51b76e5740 1 -- 172.21.15.39:0/3456238332 shutdown_connections 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51b76e5740 1 --2- 172.21.15.39:0/3456238332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51b76e5740 1 --2- 172.21.15.39:0/3456238332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424b80f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51b76e5740 1 --2- 172.21.15.39:0/3456238332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 0x5638424b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51b76e5740 1 -- 172.21.15.39:0/3456238332 >> 172.21.15.39:0/3456238332 conn(0x563842107350 msgr2=0x5638424b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51b76e5740 1 -- 172.21.15.39:0/3456238332 shutdown_connections 2024-07-04T08:14:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.640+0000 7f51b76e5740 1 -- 172.21.15.39:0/3456238332 wait complete. 2024-07-04T08:14:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.641+0000 7f51b76e5740 1 -- start start 2024-07-04T08:14:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.641+0000 7f51b76e5740 1 Processor -- start 2024-07-04T08:14:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424c3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563842112f60 0x5638424ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51b76e5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5638422815f0 con 0x563842112f60 2024-07-04T08:14:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51b76e5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5638423c4970 con 0x5638420f2280 2024-07-04T08:14:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51b76e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5638424c5c10 con 0x5638420ea0a0 2024-07-04T08:14:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51b76e5740 10 client.0 _ll_register_callbacks cb 0x5638424164a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51a9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51a1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51a9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424c32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49846/0 (socket says 172.21.15.39:49846) 2024-07-04T08:14:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51a9e00640 1 -- 172.21.15.39:0/1932847581 learned_addr learned my addr 172.21.15.39:0/1932847581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.642+0000 7f51aa800640 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563842112f60 0x5638424ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.643+0000 7f51a9e00640 1 -- 172.21.15.39:0/1932847581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 msgr2=0x5638424c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.643+0000 7f51a9e00640 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.643+0000 7f51a9e00640 1 -- 172.21.15.39:0/1932847581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563842112f60 msgr2=0x5638424ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.643+0000 7f51a9e00640 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563842112f60 0x5638424ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.643+0000 7f51a9e00640 1 -- 172.21.15.39:0/1932847581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56384217e540 con 0x5638420ea0a0 2024-07-04T08:14:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.643+0000 7f51a9e00640 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424c32c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f519c0126d0 tx=0x7f519c012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b6d92caffce02f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.643+0000 7f5193e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:14:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.644+0000 7f5193e00640 1 -- 172.21.15.39:0/1932847581 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c004050 con 0x5638420ea0a0 2024-07-04T08:14:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.644+0000 7f51b76e5740 1 -- 172.21.15.39:0/1932847581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f519410ad60 con 0x5638420ea0a0 2024-07-04T08:14:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.644+0000 7f5193e00640 1 -- 172.21.15.39:0/1932847581 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f519c0041f0 con 0x5638420ea0a0 2024-07-04T08:14:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.644+0000 7f5193e00640 1 -- 172.21.15.39:0/1932847581 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f519c025050 con 0x5638420ea0a0 2024-07-04T08:14:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.644+0000 7f51b76e5740 10 client.25333 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.644+0000 7f51b76e5740 1 -- 172.21.15.39:0/1932847581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638424d1ce0 con 0x5638420ea0a0 2024-07-04T08:14:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.645+0000 7f5193e00640 1 -- 172.21.15.39:0/1932847581 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f519410ad60 con 0x5638420ea0a0 2024-07-04T08:14:40.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.645+0000 7f5193e00640 1 -- 172.21.15.39:0/1932847581 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f519c0039e0 con 0x5638420ea0a0 2024-07-04T08:14:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.645+0000 7f51b76e5740 10 client.25333 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.645+0000 7f51b76e5740 10 client.25333 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T08:14:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.645+0000 7f51b76e5740 20 client.25333 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.645+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5638420d18e0 0x5638424d5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.645+0000 7f51b76e5740 4 client.25333 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:14:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.645+0000 7f51b76e5740 1 -- 172.21.15.39:0/1932847581 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56384242fbf0 con 0x5638420d18e0 2024-07-04T08:14:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.645+0000 7f51a1400640 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5638420d18e0 0x5638424d5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.646+0000 7f51a1400640 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5638420d18e0 0x5638424d5780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81ae8375c9b97107 server_cookie=640f3708b593816 in_seq=0 out_seq=0 2024-07-04T08:14:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.646+0000 7f5193e00640 10 client.25333 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:14:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.647+0000 7f5193e00640 1 -- 172.21.15.39:0/1932847581 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56384242fbf0 con 0x5638420d18e0 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.647+0000 7f5193e00640 10 client.25333 handle_command_reply: tid=0 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.647+0000 7f51b76e5740 1 client.25333 shutdown 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.647+0000 7f51b76e5740 1 -- 172.21.15.39:0/1932847581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 msgr2=0x5638424c32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.647+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424c32c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f519c0126d0 tx=0x7f519c012b90 comp rx=0 tx=0).stop 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.647+0000 7f51b76e5740 1 -- 172.21.15.39:0/1932847581 shutdown_connections 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.647+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5638420d18e0 0x5638424d5780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.648+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563842112f60 0x5638424ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.648+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.648+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1932847581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424c32c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.648+0000 7f51b76e5740 1 -- 172.21.15.39:0/1932847581 >> 172.21.15.39:0/1932847581 conn(0x563842107350 msgr2=0x5638424ba800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.648+0000 7f5193e00640 0 client.25333 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:14:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.648+0000 7f51b76e5740 1 -- 172.21.15.39:0/1932847581 shutdown_connections 2024-07-04T08:14:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.648+0000 7f51b76e5740 1 -- 172.21.15.39:0/1932847581 wait complete. 2024-07-04T08:14:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.648+0000 7f51b76e5740 20 client.25333 trim_cache size 0 max 16384 2024-07-04T08:14:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.675+0000 7f51b76e5740 1 Processor -- start 2024-07-04T08:14:40.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.675+0000 7f51b76e5740 1 -- start start 2024-07-04T08:14:40.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 0x5638424c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51b76e5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5638424d3ac0 con 0x5638420ea0a0 2024-07-04T08:14:40.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51b76e5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5638424d4190 con 0x5638420f2280 2024-07-04T08:14:40.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51b76e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5638424d4860 con 0x563842112f60 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51a9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51a1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51a1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45728/0 (socket says 172.21.15.39:45728) 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51a1400640 1 -- 172.21.15.39:0/3472311566 learned_addr learned my addr 172.21.15.39:0/3472311566 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51a9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35226/0 (socket says 172.21.15.39:35226) 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.676+0000 7f51aa800640 1 --2- 172.21.15.39:0/3472311566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 0x5638424c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.677+0000 7f51a1400640 1 -- 172.21.15.39:0/3472311566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 msgr2=0x5638424c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.677+0000 7f51a1400640 1 --2- 172.21.15.39:0/3472311566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 0x5638424c1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.677+0000 7f51a1400640 1 -- 172.21.15.39:0/3472311566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 msgr2=0x5638424c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.677+0000 7f51a1400640 1 --2- 172.21.15.39:0/3472311566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.677+0000 7f51a1400640 1 -- 172.21.15.39:0/3472311566 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638424b8480 con 0x5638420f2280 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.677+0000 7f51a1400640 1 --2- 172.21.15.39:0/3472311566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424cee70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f51940c5ad0 tx=0x7f5194005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d986df19e52a3b22 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.677+0000 7f519aa00640 1 -- 172.21.15.39:0/3472311566 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194049070 con 0x5638420f2280 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.677+0000 7f519aa00640 1 -- 172.21.15.39:0/3472311566 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5194108300 con 0x5638420f2280 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.678+0000 7f519aa00640 1 -- 172.21.15.39:0/3472311566 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194111070 con 0x5638420f2280 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.678+0000 7f51b76e5740 1 -- 172.21.15.39:0/3472311566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 msgr2=0x5638424cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.678+0000 7f51b76e5740 1 --2- 172.21.15.39:0/3472311566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424cee70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f51940c5ad0 tx=0x7f5194005fd0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.679+0000 7f51b76e5740 1 -- 172.21.15.39:0/3472311566 shutdown_connections 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.679+0000 7f51b76e5740 1 --2- 172.21.15.39:0/3472311566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563842112f60 0x5638424c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.679+0000 7f51b76e5740 1 --2- 172.21.15.39:0/3472311566 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638420f2280 0x5638424cee70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.679+0000 7f51b76e5740 1 --2- 172.21.15.39:0/3472311566 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420ea0a0 0x5638424c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.679+0000 7f51b76e5740 1 -- 172.21.15.39:0/3472311566 >> 172.21.15.39:0/3472311566 conn(0x563842107350 msgr2=0x5638424be270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:40.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.679+0000 7f51b76e5740 1 -- 172.21.15.39:0/3472311566 shutdown_connections 2024-07-04T08:14:40.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.679+0000 7f51b76e5740 1 -- 172.21.15.39:0/3472311566 wait complete. 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.679+0000 7f51b76e5740 1 -- start start 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.680+0000 7f51b76e5740 1 Processor -- start 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.680+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.680+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420f2280 0x5638424de230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.680+0000 7f51b76e5740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563842112f60 0x5638424d7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.680+0000 7f51b76e5740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5638424d5600 con 0x5638420f2280 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.680+0000 7f51b76e5740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5638424d4f30 con 0x563842112f60 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.680+0000 7f51b76e5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5638424d5cd0 con 0x5638420ea0a0 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.680+0000 7f51b76e5740 10 client.0 _ll_register_callbacks cb 0x7f519c009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420f2280 0x5638424de230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424ddc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49870/0 (socket says 172.21.15.39:49870) 2024-07-04T08:14:40.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a9e00640 1 -- 172.21.15.39:0/1216654925 learned_addr learned my addr 172.21.15.39:0/1216654925 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:40.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51aa800640 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563842112f60 0x5638424d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a9e00640 1 -- 172.21.15.39:0/1216654925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563842112f60 msgr2=0x5638424d7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a9e00640 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563842112f60 0x5638424d7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a9e00640 1 -- 172.21.15.39:0/1216654925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420f2280 msgr2=0x5638424de230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a9e00640 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420f2280 0x5638424de230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a9e00640 1 -- 172.21.15.39:0/1216654925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638424d7970 con 0x5638420ea0a0 2024-07-04T08:14:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.681+0000 7f51a1400640 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420f2280 0x5638424de230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.682+0000 7f51a9e00640 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424ddc50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f519c0098d0 tx=0x7f519c009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232cca6596851938 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.682+0000 7f5193e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.682+0000 7f5193e00640 1 -- 172.21.15.39:0/1216654925 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c00f9a0 con 0x5638420ea0a0 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.682+0000 7f51b76e5740 1 -- 172.21.15.39:0/1216654925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5194108300 con 0x5638420ea0a0 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.682+0000 7f5193e00640 1 -- 172.21.15.39:0/1216654925 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f519c018040 con 0x5638420ea0a0 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f5193e00640 1 -- 172.21.15.39:0/1216654925 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f519c03a050 con 0x5638420ea0a0 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f51b76e5740 10 client.25339 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f51b76e5740 1 -- 172.21.15.39:0/1216654925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638424dd320 con 0x5638420ea0a0 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f5193e00640 1 -- 172.21.15.39:0/1216654925 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5194108300 con 0x5638420ea0a0 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f5193e00640 1 -- 172.21.15.39:0/1216654925 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f519c005140 con 0x5638420ea0a0 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f51b76e5740 10 client.25339 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f51b76e5740 10 client.25339 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T08:14:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f51b76e5740 20 client.25339 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:40.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5638420a6c90 0x563842541cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:40.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f51b76e5740 4 client.25339 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:14:40.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.683+0000 7f51b76e5740 1 -- 172.21.15.39:0/1216654925 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "status"}) -- 0x56384242b690 con 0x5638420a6c90 2024-07-04T08:14:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.684+0000 7f51a1400640 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5638420a6c90 0x563842541cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f5193e00640 10 client.25339 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:14:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51a1400640 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5638420a6c90 0x563842541cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19b9dd268be4fcab server_cookie=db1ef6a4af2d2bf3 in_seq=0 out_seq=0 2024-07-04T08:14:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f5193e00640 1 -- 172.21.15.39:0/1216654925 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56384242b690 con 0x5638420a6c90 2024-07-04T08:14:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f5193e00640 10 client.25339 handle_command_reply: tid=0 2024-07-04T08:14:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51b76e5740 1 client.25339 shutdown 2024-07-04T08:14:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51b76e5740 1 -- 172.21.15.39:0/1216654925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 msgr2=0x5638424ddc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424ddc50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f519c0098d0 tx=0x7f519c009350 comp rx=0 tx=0).stop 2024-07-04T08:14:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51b76e5740 1 -- 172.21.15.39:0/1216654925 shutdown_connections 2024-07-04T08:14:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5638420a6c90 0x563842541cf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5638420f2280 0x5638424de230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563842112f60 0x5638424d7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51b76e5740 1 --2- 172.21.15.39:0/1216654925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638420ea0a0 0x5638424ddc50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f5193e00640 0 client.25339 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:14:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.685+0000 7f51b76e5740 1 -- 172.21.15.39:0/1216654925 >> 172.21.15.39:0/1216654925 conn(0x563842107350 msgr2=0x5638424b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.686+0000 7f51b76e5740 1 -- 172.21.15.39:0/1216654925 shutdown_connections 2024-07-04T08:14:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.686+0000 7f51b76e5740 1 -- 172.21.15.39:0/1216654925 wait complete. 2024-07-04T08:14:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.686+0000 7f51b76e5740 20 client.25339 trim_cache size 0 max 16384 2024-07-04T08:14:40.689 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:14:40.690 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:14:40.690 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-07-04T08:14:40.690 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24701, 2024-07-04T08:14:40.690 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:14:40.690 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:14:40.690 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:14:40.690 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 218.916147446, 2024-07-04T08:14:40.691 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:14:40.691 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:14:40.691 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:14:40.691 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 320.97840456400002 2024-07-04T08:14:40.691 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:14:40.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.686+0000 7f51b1000640 1 -- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f518c0b2840 msgr2=0x7f518c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:40.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.686+0000 7f51b1000640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f518c0b2840 0x7f518c0b4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005fd0 tx=0x7f51a4009500 comp rx=0 tx=0).stop 2024-07-04T08:14:40.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.686+0000 7f51b1000640 1 -- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0777e0 msgr2=0x7f51ac1a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:40.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.687+0000 7f51b1000640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0777e0 0x7f51ac1a05c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5194009230 tx=0x7f5194003b90 comp rx=0 tx=0).stop 2024-07-04T08:14:40.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.687+0000 7f51b1000640 1 -- 172.21.15.39:0/4029214991 shutdown_connections 2024-07-04T08:14:40.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.687+0000 7f51b1000640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f518c0b2840 0x7f518c0b4d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.687+0000 7f51b1000640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51ac0777e0 0x7f51ac1a05c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.687+0000 7f51b1000640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac113c90 0x7f51ac1a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.687+0000 7f51b1000640 1 --2- 172.21.15.39:0/4029214991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51ac076e90 0x7f51ac1a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:40.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.687+0000 7f51b1000640 1 -- 172.21.15.39:0/4029214991 >> 172.21.15.39:0/4029214991 conn(0x7f51ac100d60 msgr2=0x7f51ac115d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:40.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.687+0000 7f51b1000640 1 -- 172.21.15.39:0/4029214991 shutdown_connections 2024-07-04T08:14:40.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:40.687+0000 7f51b1000640 1 -- 172.21.15.39:0/4029214991 wait complete. 2024-07-04T08:14:41.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:41 smithi133 ceph-mon[37050]: pgmap v464: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:41.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:41 smithi039 ceph-mon[32062]: pgmap v464: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:41.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:41 smithi050 ceph-mon[36577]: pgmap v464: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:41.648 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.b status 2024-07-04T08:14:42.085 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:43.291 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:43 smithi039 ceph-mon[32062]: pgmap v465: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:43.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:43 smithi133 ceph-mon[37050]: pgmap v465: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:43.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:43 smithi050 ceph-mon[36577]: pgmap v465: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:44.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.363+0000 7f29db600640 1 -- 172.21.15.39:0/3253958998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4109360 msgr2=0x7f29d410fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.363+0000 7f29db600640 1 --2- 172.21.15.39:0/3253958998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4109360 0x7f29d410fb90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009ca0 tx=0x7f29cc02a8b0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.363+0000 7f29db600640 1 -- 172.21.15.39:0/3253958998 shutdown_connections 2024-07-04T08:14:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.363+0000 7f29db600640 1 --2- 172.21.15.39:0/3253958998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4109360 0x7f29d410fb90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.363+0000 7f29db600640 1 --2- 172.21.15.39:0/3253958998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4108a10 0x7f29d4108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.363+0000 7f29db600640 1 --2- 172.21.15.39:0/3253958998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d41030d0 0x7f29d41034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.363+0000 7f29db600640 1 -- 172.21.15.39:0/3253958998 >> 172.21.15.39:0/3253958998 conn(0x7f29d40feb20 msgr2=0x7f29d4100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.363+0000 7f29db600640 1 -- 172.21.15.39:0/3253958998 shutdown_connections 2024-07-04T08:14:44.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.363+0000 7f29db600640 1 -- 172.21.15.39:0/3253958998 wait complete. 2024-07-04T08:14:44.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.364+0000 7f29db600640 1 Processor -- start 2024-07-04T08:14:44.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.364+0000 7f29db600640 1 -- start start 2024-07-04T08:14:44.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29db600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d41030d0 0x7f29d4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29db600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4108a10 0x7f29d40751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29db600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4109360 0x7f29d40756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29db600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4113080 con 0x7f29d4109360 2024-07-04T08:14:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29db600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4112f00 con 0x7f29d41030d0 2024-07-04T08:14:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29db600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4113200 con 0x7f29d4108a10 2024-07-04T08:14:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29d8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d41030d0 0x7f29d4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29d8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d41030d0 0x7f29d4074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48460/0 (socket says 172.21.15.39:48460) 2024-07-04T08:14:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29d8c00640 1 -- 172.21.15.39:0/2251367975 learned_addr learned my addr 172.21.15.39:0/2251367975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29d9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4109360 0x7f29d40756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.365+0000 7f29d3e00640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4108a10 0x7f29d40751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d3e00640 1 -- 172.21.15.39:0/2251367975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d41030d0 msgr2=0x7f29d4074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:44.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d3e00640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d41030d0 0x7f29d4074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d3e00640 1 -- 172.21.15.39:0/2251367975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4109360 msgr2=0x7f29d40756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:44.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d3e00640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4109360 0x7f29d40756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d3e00640 1 -- 172.21.15.39:0/2251367975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d41a7cb0 con 0x7f29d4108a10 2024-07-04T08:14:44.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d8c00640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d41030d0 0x7f29d4074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:44.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d3e00640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4108a10 0x7f29d40751a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f29d406bf60 tx=0x7f29c000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:44.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d1600640 1 -- 172.21.15.39:0/2251367975 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c000cc40 con 0x7f29d4108a10 2024-07-04T08:14:44.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d1600640 1 -- 172.21.15.39:0/2251367975 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c000cde0 con 0x7f29d4108a10 2024-07-04T08:14:44.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29db600640 1 -- 172.21.15.39:0/2251367975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d41a7fa0 con 0x7f29d4108a10 2024-07-04T08:14:44.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.366+0000 7f29d1600640 1 -- 172.21.15.39:0/2251367975 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0004a00 con 0x7f29d4108a10 2024-07-04T08:14:44.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.367+0000 7f29db600640 1 -- 172.21.15.39:0/2251367975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d41a83f0 con 0x7f29d4108a10 2024-07-04T08:14:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.370+0000 7f29e1bc8740 1 Processor -- start 2024-07-04T08:14:44.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.370+0000 7f29e1bc8740 1 -- start start 2024-07-04T08:14:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.370+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x56012248e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.370+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012248eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.370+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x56012248f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.370+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560122404c80 con 0x5601220e9f60 2024-07-04T08:14:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5601223f4220 con 0x5601220c10a0 2024-07-04T08:14:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5601223f8260 con 0x5601220c9280 2024-07-04T08:14:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x56012248e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012248eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x56012248e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52936/0 (socket says 172.21.15.39:52936) 2024-07-04T08:14:44.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d9600640 1 -- 172.21.15.39:0/3210601860 learned_addr learned my addr 172.21.15.39:0/3210601860 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:44.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d1600640 1 -- 172.21.15.39:0/2251367975 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f29c0004ba0 con 0x7f29d4108a10 2024-07-04T08:14:44.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d8c00640 1 -- 172.21.15.39:0/3210601860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 msgr2=0x56012248f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:44.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d8c00640 1 --2- 172.21.15.39:0/3210601860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x56012248f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d8c00640 1 -- 172.21.15.39:0/3210601860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 msgr2=0x56012248e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:44.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d8c00640 1 --2- 172.21.15.39:0/3210601860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x56012248e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.371+0000 7f29d8c00640 1 -- 172.21.15.39:0/3210601860 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012238c530 con 0x5601220c10a0 2024-07-04T08:14:44.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.372+0000 7f29d8c00640 1 --2- 172.21.15.39:0/3210601860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012248eb90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005fd0 tx=0x7f29c80109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a90b559f2a19d78c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:44.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.372+0000 7f29c6000640 1 -- 172.21.15.39:0/3210601860 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8004230 con 0x5601220c10a0 2024-07-04T08:14:44.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.372+0000 7f29c6000640 1 -- 172.21.15.39:0/3210601860 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c80043d0 con 0x5601220c10a0 2024-07-04T08:14:44.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.372+0000 7f29c6000640 1 -- 172.21.15.39:0/3210601860 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c80035f0 con 0x5601220c10a0 2024-07-04T08:14:44.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3210601860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 msgr2=0x56012248eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:44.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3210601860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012248eb90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f29c8005fd0 tx=0x7f29c80109e0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29d1600640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29bc0b2840 0x7f29bc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29d1600640 1 -- 172.21.15.39:0/2251367975 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f29c00e25e0 con 0x7f29d4108a10 2024-07-04T08:14:44.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3210601860 shutdown_connections 2024-07-04T08:14:44.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3210601860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x56012248f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3210601860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012248eb90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3210601860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x56012248e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3210601860 >> 172.21.15.39:0/3210601860 conn(0x5601220de350 msgr2=0x56012248c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:44.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.373+0000 7f29d9600640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29bc0b2840 0x7f29bc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.374+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3210601860 shutdown_connections 2024-07-04T08:14:44.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.374+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3210601860 wait complete. 2024-07-04T08:14:44.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.374+0000 7f29d9600640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29bc0b2840 0x7f29bc0b4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f29cc031040 tx=0x7f29cc005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:44.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.374+0000 7f29e1bc8740 1 -- start start 2024-07-04T08:14:44.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.375+0000 7f29e1bc8740 1 Processor -- start 2024-07-04T08:14:44.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.375+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012249a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.375+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x56012249a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.375+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x5601224a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.375+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5601222585f0 con 0x5601220e9f60 2024-07-04T08:14:44.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.375+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56012239b950 con 0x5601220c10a0 2024-07-04T08:14:44.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.375+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56012249ce30 con 0x5601220c9280 2024-07-04T08:14:44.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.375+0000 7f29e1bc8740 10 client.0 _ll_register_callbacks cb 0x5601223ed480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:44.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012249a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x56012249a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012249a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48474/0 (socket says 172.21.15.39:48474) 2024-07-04T08:14:44.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x56012249a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51098/0 (socket says 172.21.15.39:51098) 2024-07-04T08:14:44.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d8c00640 1 -- 172.21.15.39:0/3288212216 learned_addr learned my addr 172.21.15.39:0/3288212216 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:44.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d9600640 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x5601224a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d8c00640 1 -- 172.21.15.39:0/3288212216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 msgr2=0x56012249a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d8c00640 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x56012249a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d8c00640 1 -- 172.21.15.39:0/3288212216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 msgr2=0x5601224a6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d8c00640 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x5601224a6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.376+0000 7f29d8c00640 1 -- 172.21.15.39:0/3288212216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560122155540 con 0x5601220c10a0 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.377+0000 7f29d8c00640 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012249a320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f29c8004a40 tx=0x7f29c80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed2cbb667310d5d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.377+0000 7f29baa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.377+0000 7f29baa00640 1 -- 172.21.15.39:0/3288212216 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8029060 con 0x5601220c10a0 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.377+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3288212216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29c80043d0 con 0x5601220c10a0 2024-07-04T08:14:44.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.377+0000 7f29baa00640 1 -- 172.21.15.39:0/3288212216 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f29c800e980 con 0x5601220c10a0 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29baa00640 1 -- 172.21.15.39:0/3288212216 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29c8030050 con 0x5601220c10a0 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29e1bc8740 10 client.25370 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3288212216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5601224a8f40 con 0x5601220c10a0 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29baa00640 1 -- 172.21.15.39:0/3288212216 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c80043d0 con 0x5601220c10a0 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29baa00640 1 -- 172.21.15.39:0/3288212216 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f29c80053a0 con 0x5601220c10a0 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29e1bc8740 10 client.25370 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29e1bc8740 10 client.25370 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29e1bc8740 20 client.25370 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5601220a88e0 0x5601224ac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29e1bc8740 4 client.25370 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.378+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3288212216 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560122406bd0 con 0x5601220a88e0 2024-07-04T08:14:44.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.379+0000 7f29d3e00640 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5601220a88e0 0x5601224ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.379+0000 7f29d3e00640 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5601220a88e0 0x5601224ac710 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2589f13bf0521926 server_cookie=f1ae3d2385aaa0cd in_seq=0 out_seq=0 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.379+0000 7f29baa00640 10 client.25370 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.380+0000 7f29baa00640 1 -- 172.21.15.39:0/3288212216 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560122406bd0 con 0x5601220a88e0 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.380+0000 7f29baa00640 10 client.25370 handle_command_reply: tid=0 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 client.25370 shutdown 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3288212216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 msgr2=0x56012249a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012249a320 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f29c8004a40 tx=0x7f29c80046c0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3288212216 shutdown_connections 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5601220a88e0 0x5601224ac710 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x5601224a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x56012249a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/3288212216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x56012249a320 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29baa00640 0 client.25370 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3288212216 >> 172.21.15.39:0/3288212216 conn(0x5601220de350 msgr2=0x560122491830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:44.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3288212216 shutdown_connections 2024-07-04T08:14:44.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 1 -- 172.21.15.39:0/3288212216 wait complete. 2024-07-04T08:14:44.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.381+0000 7f29e1bc8740 20 client.25370 trim_cache size 0 max 16384 2024-07-04T08:14:44.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.402+0000 7f29e1bc8740 1 Processor -- start 2024-07-04T08:14:44.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.403+0000 7f29e1bc8740 1 -- start start 2024-07-04T08:14:44.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.403+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224ac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.403+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.403+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x560122493760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.403+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5601224aaa50 con 0x5601220e9f60 2024-07-04T08:14:44.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.403+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5601224ab120 con 0x5601220c10a0 2024-07-04T08:14:44.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.403+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5601224ab7f0 con 0x5601220c9280 2024-07-04T08:14:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x560122493760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51100/0 (socket says 172.21.15.39:51100) 2024-07-04T08:14:44.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d3e00640 1 -- 172.21.15.39:0/1298509605 learned_addr learned my addr 172.21.15.39:0/1298509605 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d8c00640 1 -- 172.21.15.39:0/1298509605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 msgr2=0x5601224a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d8c00640 1 --2- 172.21.15.39:0/1298509605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d8c00640 1 -- 172.21.15.39:0/1298509605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 msgr2=0x560122493760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d8c00640 1 --2- 172.21.15.39:0/1298509605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x560122493760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d8c00640 1 -- 172.21.15.39:0/1298509605 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012249b1c0 con 0x5601220c10a0 2024-07-04T08:14:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.404+0000 7f29d8c00640 1 --2- 172.21.15.39:0/1298509605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224ac710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29c800b4e0 tx=0x7f29c80109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0e02982360c37c9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.405+0000 7f29c5600640 1 -- 172.21.15.39:0/1298509605 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8029070 con 0x5601220c10a0 2024-07-04T08:14:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.405+0000 7f29c5600640 1 -- 172.21.15.39:0/1298509605 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c8004730 con 0x5601220c10a0 2024-07-04T08:14:44.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.405+0000 7f29c5600640 1 -- 172.21.15.39:0/1298509605 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c800ee60 con 0x5601220c10a0 2024-07-04T08:14:44.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.406+0000 7f29e1bc8740 1 -- 172.21.15.39:0/1298509605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 msgr2=0x5601224ac710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:44.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.406+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/1298509605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224ac710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29c800b4e0 tx=0x7f29c80109e0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.406+0000 7f29e1bc8740 1 -- 172.21.15.39:0/1298509605 shutdown_connections 2024-07-04T08:14:44.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.406+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/1298509605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x560122493760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.406+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/1298509605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.406+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/1298509605 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224ac710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.406+0000 7f29e1bc8740 1 -- 172.21.15.39:0/1298509605 >> 172.21.15.39:0/1298509605 conn(0x5601220de350 msgr2=0x5601224989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:44.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.406+0000 7f29e1bc8740 1 -- 172.21.15.39:0/1298509605 shutdown_connections 2024-07-04T08:14:44.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.406+0000 7f29e1bc8740 1 -- 172.21.15.39:0/1298509605 wait complete. 2024-07-04T08:14:44.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.407+0000 7f29e1bc8740 1 -- start start 2024-07-04T08:14:44.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.407+0000 7f29e1bc8740 1 Processor -- start 2024-07-04T08:14:44.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.407+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.407+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.407+0000 7f29e1bc8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x5601224ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.407+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5601224ac590 con 0x5601220e9f60 2024-07-04T08:14:44.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.407+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5601224abec0 con 0x5601220c10a0 2024-07-04T08:14:44.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.407+0000 7f29e1bc8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5601224acc60 con 0x5601220c9280 2024-07-04T08:14:44.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.407+0000 7f29e1bc8740 10 client.0 _ll_register_callbacks cb 0x7f29cc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:14:44.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51110/0 (socket says 172.21.15.39:51110) 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d3e00640 1 -- 172.21.15.39:0/4039193022 learned_addr learned my addr 172.21.15.39:0/4039193022 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d9600640 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x5601224ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d3e00640 1 -- 172.21.15.39:0/4039193022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 msgr2=0x5601224b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d3e00640 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d3e00640 1 -- 172.21.15.39:0/4039193022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 msgr2=0x5601224ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d3e00640 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x5601224ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d3e00640 1 -- 172.21.15.39:0/4039193022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601224aecc0 con 0x5601220c9280 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.408+0000 7f29d8c00640 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.409+0000 7f29d3e00640 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224ae240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f29c00a69c0 tx=0x7f29c0091040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec0bec4c559cb65 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.409+0000 7f29baa00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.409+0000 7f29baa00640 1 -- 172.21.15.39:0/4039193022 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c00c2070 con 0x5601220c9280 2024-07-04T08:14:44.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.409+0000 7f29e1bc8740 1 -- 172.21.15.39:0/4039193022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29c8004730 con 0x5601220c9280 2024-07-04T08:14:44.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.409+0000 7f29baa00640 1 -- 172.21.15.39:0/4039193022 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f29c00a59e0 con 0x5601220c9280 2024-07-04T08:14:44.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.409+0000 7f29baa00640 1 -- 172.21.15.39:0/4039193022 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29c001c040 con 0x5601220c9280 2024-07-04T08:14:44.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.409+0000 7f29e1bc8740 10 client.25354 fetch_fsmap learned FSMap version 29 2024-07-04T08:14:44.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.409+0000 7f29e1bc8740 1 -- 172.21.15.39:0/4039193022 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5601224b4670 con 0x5601220c9280 2024-07-04T08:14:44.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.410+0000 7f29baa00640 1 -- 172.21.15.39:0/4039193022 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c8004730 con 0x5601220c9280 2024-07-04T08:14:44.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.410+0000 7f29baa00640 1 -- 172.21.15.39:0/4039193022 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f29c00e21d0 con 0x5601220c9280 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.410+0000 7f29e1bc8740 10 client.25354 fetch_fsmap finished waiting for FSMap version 29 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.410+0000 7f29e1bc8740 10 client.25354 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.410+0000 7f29e1bc8740 20 client.25354 populate_metadata read hostname 'smithi039' 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.410+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56012207dc90 0x560122521710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.410+0000 7f29e1bc8740 4 client.25354 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.410+0000 7f29e1bc8740 1 -- 172.21.15.39:0/4039193022 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "status"}) -- 0x560122402670 con 0x56012207dc90 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.411+0000 7f29d8c00640 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56012207dc90 0x560122521710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.411+0000 7f29d8c00640 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56012207dc90 0x560122521710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56da333dc34ce99c server_cookie=7cb5df149dbc033d in_seq=0 out_seq=0 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.411+0000 7f29baa00640 10 client.25354 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29baa00640 1 -- 172.21.15.39:0/4039193022 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560122402670 con 0x56012207dc90 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29baa00640 10 client.25354 handle_command_reply: tid=0 2024-07-04T08:14:44.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 client.25354 shutdown 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 -- 172.21.15.39:0/4039193022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 msgr2=0x5601224ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224ae240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f29c00a69c0 tx=0x7f29c0091040 comp rx=0 tx=0).stop 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 -- 172.21.15.39:0/4039193022 shutdown_connections 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56012207dc90 0x560122521710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5601220e9f60 0x5601224ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5601220c9280 0x5601224ae240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 --2- 172.21.15.39:0/4039193022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5601220c10a0 0x5601224b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29baa00640 0 client.25354 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 -- 172.21.15.39:0/4039193022 >> 172.21.15.39:0/4039193022 conn(0x5601220de350 msgr2=0x56012248f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 -- 172.21.15.39:0/4039193022 shutdown_connections 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.412+0000 7f29e1bc8740 1 -- 172.21.15.39:0/4039193022 wait complete. 2024-07-04T08:14:44.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.413+0000 7f29e1bc8740 20 client.25354 trim_cache size 0 max 16384 2024-07-04T08:14:44.416 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:14:44.416 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:14:44.416 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-07-04T08:14:44.416 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24707, 2024-07-04T08:14:44.416 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:14:44.416 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:14:44.416 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:14:44.416 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 303.010760312, 2024-07-04T08:14:44.417 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 29, 2024-07-04T08:14:44.417 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:14:44.417 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 90, 2024-07-04T08:14:44.417 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 322.11051091100001 2024-07-04T08:14:44.417 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:14:44.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.413+0000 7f29db600640 1 -- 172.21.15.39:0/2251367975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29bc0b2840 msgr2=0x7f29bc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:44.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.413+0000 7f29db600640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29bc0b2840 0x7f29bc0b4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f29cc031040 tx=0x7f29cc005d80 comp rx=0 tx=0).stop 2024-07-04T08:14:44.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.413+0000 7f29db600640 1 -- 172.21.15.39:0/2251367975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4108a10 msgr2=0x7f29d40751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:44.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.413+0000 7f29db600640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4108a10 0x7f29d40751a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f29d406bf60 tx=0x7f29c000ee50 comp rx=0 tx=0).stop 2024-07-04T08:14:44.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.414+0000 7f29db600640 1 -- 172.21.15.39:0/2251367975 shutdown_connections 2024-07-04T08:14:44.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.414+0000 7f29db600640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f29bc0b2840 0x7f29bc0b4d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.414+0000 7f29db600640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29d4109360 0x7f29d40756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.414+0000 7f29db600640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4108a10 0x7f29d40751a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.414+0000 7f29db600640 1 --2- 172.21.15.39:0/2251367975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d41030d0 0x7f29d4074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:44.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.414+0000 7f29db600640 1 -- 172.21.15.39:0/2251367975 >> 172.21.15.39:0/2251367975 conn(0x7f29d40feb20 msgr2=0x7f29d4100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:44.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.414+0000 7f29db600640 1 -- 172.21.15.39:0/2251367975 shutdown_connections 2024-07-04T08:14:44.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:44.414+0000 7f29db600640 1 -- 172.21.15.39:0/2251367975 wait complete. 2024-07-04T08:14:45.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:45 smithi039 ceph-mon[32062]: pgmap v466: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:45.224 INFO:teuthology.run_tasks:Running task sequential... 2024-07-04T08:14:45.236 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-07-04T08:14:45.236 INFO:teuthology.task.sequential:In sequential, running task print... 2024-07-04T08:14:45.236 INFO:teuthology.task.print:Enabling mgr modules 2024-07-04T08:14:45.236 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-07-04T08:14:45.237 INFO:teuthology.task.exec:Executing custom commands... 2024-07-04T08:14:45.237 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T08:14:45.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-07-04T08:14:45.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:45 smithi133 ceph-mon[37050]: pgmap v466: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:45.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:45 smithi050 ceph-mon[36577]: pgmap v466: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:45.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.525+0000 7f60f0200640 1 -- 172.21.15.39:0/25548727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e810b4c0 msgr2=0x7f60e8111cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:45.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.525+0000 7f60f0200640 1 --2- 172.21.15.39:0/25548727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e810b4c0 0x7f60e8111cf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009be0 tx=0x7f60e002a6c0 comp rx=0 tx=0).stop 2024-07-04T08:14:45.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.525+0000 7f60f0200640 1 -- 172.21.15.39:0/25548727 shutdown_connections 2024-07-04T08:14:45.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.525+0000 7f60f0200640 1 --2- 172.21.15.39:0/25548727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e810b4c0 0x7f60e8111cf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:45.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.525+0000 7f60f0200640 1 --2- 172.21.15.39:0/25548727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60e810ab70 0x7f60e810af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:45.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.525+0000 7f60f0200640 1 --2- 172.21.15.39:0/25548727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60e8105240 0x7f60e8105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:45.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.525+0000 7f60f0200640 1 -- 172.21.15.39:0/25548727 >> 172.21.15.39:0/25548727 conn(0x7f60e8100c20 msgr2=0x7f60e8103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:45.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.525+0000 7f60f0200640 1 -- 172.21.15.39:0/25548727 shutdown_connections 2024-07-04T08:14:45.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.525+0000 7f60f0200640 1 -- 172.21.15.39:0/25548727 wait complete. 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.526+0000 7f60f0200640 1 Processor -- start 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.526+0000 7f60f0200640 1 -- start start 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60f0200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e8105240 0x7f60e819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60f0200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60e810ab70 0x7f60e819c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60f0200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60e810b4c0 0x7f60e81a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60f0200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8115170 con 0x7f60e810b4c0 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60f0200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60e8114ff0 con 0x7f60e8105240 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60f0200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f60e81152f0 con 0x7f60e810ab70 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60ee200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60e810b4c0 0x7f60e81a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:45.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60ee200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60e810b4c0 0x7f60e81a1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52988/0 (socket says 172.21.15.39:52988) 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60ee200640 1 -- 172.21.15.39:0/3356250693 learned_addr learned my addr 172.21.15.39:0/3356250693 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60ed800640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e8105240 0x7f60e819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60ece00640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60e810ab70 0x7f60e819c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60ed800640 1 -- 172.21.15.39:0/3356250693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60e810ab70 msgr2=0x7f60e819c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.527+0000 7f60ed800640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60e810ab70 0x7f60e819c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60ed800640 1 -- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60e810b4c0 msgr2=0x7f60e81a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60ed800640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60e810b4c0 0x7f60e81a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60ed800640 1 -- 172.21.15.39:0/3356250693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e81a1a60 con 0x7f60e8105240 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60ee200640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60e810b4c0 0x7f60e81a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60ece00640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60e810ab70 0x7f60e819c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60ed800640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e8105240 0x7f60e819bb30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e790 tx=0x7f60dc00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60e6000640 1 -- 172.21.15.39:0/3356250693 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00eea0 con 0x7f60e8105240 2024-07-04T08:14:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60e6000640 1 -- 172.21.15.39:0/3356250693 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60dc00ccc0 con 0x7f60e8105240 2024-07-04T08:14:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60e6000640 1 -- 172.21.15.39:0/3356250693 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc018680 con 0x7f60e8105240 2024-07-04T08:14:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e81a1d50 con 0x7f60e8105240 2024-07-04T08:14:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.528+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e81a84e0 con 0x7f60e8105240 2024-07-04T08:14:45.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.529+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e8105650 con 0x7f60e8105240 2024-07-04T08:14:45.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.532+0000 7f60e6000640 1 -- 172.21.15.39:0/3356250693 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 22) ==== 149669+0+0 (secure 0 0 0) 0x7f60dc010430 con 0x7f60e8105240 2024-07-04T08:14:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.534+0000 7f60e6000640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60d00b28c0 0x7f60d00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.534+0000 7f60e6000640 1 -- 172.21.15.39:0/3356250693 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f60dc0e2d10 con 0x7f60e8105240 2024-07-04T08:14:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.534+0000 7f60ece00640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60d00b28c0 0x7f60d00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.535+0000 7f60ece00640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60d00b28c0 0x7f60d00b4dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f60d40095f0 tx=0x7f60d4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:45.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.535+0000 7f60e6000640 1 -- 172.21.15.39:0/3356250693 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f60dc0a9f30 con 0x7f60e8105240 2024-07-04T08:14:45.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:45.802+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f60e802de10 con 0x7f60e8105240 2024-07-04T08:14:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.053+0000 7f60e6000640 1 -- 172.21.15.39:0/3356250693 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7f60dc0a61c0 con 0x7f60e8105240 2024-07-04T08:14:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.055+0000 7f60e6000640 1 -- 172.21.15.39:0/3356250693 <== mon.2 v2:172.21.15.50:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 module 'snap_schedule' is already enabled v23) ==== 133+0+0 (secure 0 0 0) 0x7f60dc0e4170 con 0x7f60e8105240 2024-07-04T08:14:46.056 INFO:teuthology.orchestra.run.smithi039.stderr:module 'snap_schedule' is already enabled 2024-07-04T08:14:46.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60d00b28c0 msgr2=0x7f60d00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:46.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60d00b28c0 0x7f60d00b4dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f60d40095f0 tx=0x7f60d4005f90 comp rx=0 tx=0).stop 2024-07-04T08:14:46.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e8105240 msgr2=0x7f60e819bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:46.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e8105240 0x7f60e819bb30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00e790 tx=0x7f60dc00ec50 comp rx=0 tx=0).stop 2024-07-04T08:14:46.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 shutdown_connections 2024-07-04T08:14:46.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60d00b28c0 0x7f60d00b4dd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:46.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60e810b4c0 0x7f60e81a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:46.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60e810ab70 0x7f60e819c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:46.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 --2- 172.21.15.39:0/3356250693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60e8105240 0x7f60e819bb30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:46.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.062+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 >> 172.21.15.39:0/3356250693 conn(0x7f60e8100c20 msgr2=0x7f60e810a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:46.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.063+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 shutdown_connections 2024-07-04T08:14:46.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:46.063+0000 7f60f0200640 1 -- 172.21.15.39:0/3356250693 wait complete. 2024-07-04T08:14:46.094 INFO:teuthology.run_tasks:Running task kclient... 2024-07-04T08:14:46.108 INFO:tasks.kclient:Mounting kernel clients... 2024-07-04T08:14:46.108 INFO:tasks.kclient:config is {'client.0': {}} 2024-07-04T08:14:46.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:46.109 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-07-04T08:14:46.161 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-07-04T08:14:46.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:46.163 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-04T08:14:46.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:46.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-07-04T08:14:46.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:46 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3356250693' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-04T08:14:46.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:46 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-04T08:14:46.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:46 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3356250693' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-04T08:14:46.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:46 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-04T08:14:46.366 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-07-04T08:14:46.366 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-04T08:14:46.366 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T08:14:46.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs ls 2024-07-04T08:14:46.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:46 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3356250693' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-04T08:14:46.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:46 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-04T08:14:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:47.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:47 smithi039 ceph-mon[32062]: pgmap v467: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:47.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:47 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-04T08:14:47.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:47 smithi039 ceph-mon[32062]: mgrmap e23: x(active, since 12m), standbys: z, y 2024-07-04T08:14:47.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:47 smithi050 ceph-mon[36577]: pgmap v467: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:47.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:47 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-04T08:14:47.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:47 smithi050 ceph-mon[36577]: mgrmap e23: x(active, since 12m), standbys: z, y 2024-07-04T08:14:47.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:47 smithi133 ceph-mon[37050]: pgmap v467: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:47.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:47 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-04T08:14:47.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:47 smithi133 ceph-mon[37050]: mgrmap e23: x(active, since 12m), standbys: z, y 2024-07-04T08:14:48.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:48 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:14:48.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:48 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:14:48.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:48 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:14:49.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.136+0000 7f0faf000640 1 -- 172.21.15.39:0/2086042039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa8101fb0 msgr2=0x7f0fa81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:49.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.136+0000 7f0faf000640 1 --2- 172.21.15.39:0/2086042039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa8101fb0 0x7f0fa81023c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009900 tx=0x7f0f9802a410 comp rx=0 tx=0).stop 2024-07-04T08:14:49.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.136+0000 7f0faf000640 1 -- 172.21.15.39:0/2086042039 shutdown_connections 2024-07-04T08:14:49.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.136+0000 7f0faf000640 1 --2- 172.21.15.39:0/2086042039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fa8108240 0x7f0fa810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:49.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.136+0000 7f0faf000640 1 --2- 172.21.15.39:0/2086042039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa81078f0 0x7f0fa8107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:49.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.136+0000 7f0faf000640 1 --2- 172.21.15.39:0/2086042039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa8101fb0 0x7f0fa81023c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:49.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.136+0000 7f0faf000640 1 -- 172.21.15.39:0/2086042039 >> 172.21.15.39:0/2086042039 conn(0x7f0fa8078c80 msgr2=0x7f0fa8079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:49.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.136+0000 7f0faf000640 1 -- 172.21.15.39:0/2086042039 shutdown_connections 2024-07-04T08:14:49.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.136+0000 7f0faf000640 1 -- 172.21.15.39:0/2086042039 wait complete. 2024-07-04T08:14:49.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.138+0000 7f0faf000640 1 Processor -- start 2024-07-04T08:14:49.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.138+0000 7f0faf000640 1 -- start start 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.138+0000 7f0faf000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa8101fb0 0x7f0fa81037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.138+0000 7f0faf000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa81078f0 0x7f0fa8103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.138+0000 7f0faf000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fa8108240 0x7f0fa8104220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.138+0000 7f0faf000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8111f00 con 0x7f0fa8101fb0 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.138+0000 7f0faf000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8111d80 con 0x7f0fa81078f0 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.138+0000 7f0faf000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8112080 con 0x7f0fa8108240 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0fa7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa8101fb0 0x7f0fa81037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0f9fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa81078f0 0x7f0fa8103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0fa7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa8101fb0 0x7f0fa81037a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52996/0 (socket says 172.21.15.39:52996) 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0fa7e00640 1 -- 172.21.15.39:0/933996147 learned_addr learned my addr 172.21.15.39:0/933996147 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:49.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0fad000640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fa8108240 0x7f0fa8104220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:49.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0f9fe00640 1 -- 172.21.15.39:0/933996147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fa8108240 msgr2=0x7f0fa8104220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:49.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0f9fe00640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fa8108240 0x7f0fa8104220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:49.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0f9fe00640 1 -- 172.21.15.39:0/933996147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa8101fb0 msgr2=0x7f0fa81037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:49.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0f9fe00640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa8101fb0 0x7f0fa81037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:49.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0f9fe00640 1 -- 172.21.15.39:0/933996147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa81a7c20 con 0x7f0fa81078f0 2024-07-04T08:14:49.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.139+0000 7f0fad000640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fa8108240 0x7f0fa8104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:49.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.140+0000 7f0f9fe00640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa81078f0 0x7f0fa8103ce0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000e9c0 tx=0x7f0f9000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:49.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.140+0000 7f0fa5600640 1 -- 172.21.15.39:0/933996147 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000ccc0 con 0x7f0fa81078f0 2024-07-04T08:14:49.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.140+0000 7f0fa5600640 1 -- 172.21.15.39:0/933996147 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f900108e0 con 0x7f0fa81078f0 2024-07-04T08:14:49.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.140+0000 7f0fa5600640 1 -- 172.21.15.39:0/933996147 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f900187c0 con 0x7f0fa81078f0 2024-07-04T08:14:49.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.140+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa81a7f10 con 0x7f0fa81078f0 2024-07-04T08:14:49.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.140+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81a8370 con 0x7f0fa81078f0 2024-07-04T08:14:49.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.141+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa81023c0 con 0x7f0fa81078f0 2024-07-04T08:14:49.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.144+0000 7f0fa5600640 1 -- 172.21.15.39:0/933996147 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7f0f90010430 con 0x7f0fa81078f0 2024-07-04T08:14:49.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.145+0000 7f0fa5600640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f8c0b2840 0x7f0f8c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:49.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.146+0000 7f0fa5600640 1 -- 172.21.15.39:0/933996147 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0f900e2aa0 con 0x7f0fa81078f0 2024-07-04T08:14:49.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.146+0000 7f0fa7e00640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f8c0b2840 0x7f0f8c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:49.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.146+0000 7f0fa7e00640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f8c0b2840 0x7f0f8c0b4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005ec0 tx=0x7f0f98005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:49.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.149+0000 7f0fa5600640 1 -- 172.21.15.39:0/933996147 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f0f900aa370 con 0x7f0fa81078f0 2024-07-04T08:14:49.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:49 smithi039 ceph-mon[32062]: pgmap v468: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:49.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:49 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:14:49.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:49 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:14:49.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:49 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:14:49.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:49 smithi050 ceph-mon[36577]: pgmap v468: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:49 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:14:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:49 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:14:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:49 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:14:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.409+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0fa81a8790 con 0x7f0fa81078f0 2024-07-04T08:14:49.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.410+0000 7f0fa5600640 1 -- 172.21.15.39:0/933996147 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f0f900a9c70 con 0x7f0fa81078f0 2024-07-04T08:14:49.411 INFO:teuthology.orchestra.run.smithi039.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-04T08:14:49.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f8c0b2840 msgr2=0x7f0f8c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f8c0b2840 0x7f0f8c0b4d50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005ec0 tx=0x7f0f98005a20 comp rx=0 tx=0).stop 2024-07-04T08:14:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa81078f0 msgr2=0x7f0fa8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa81078f0 0x7f0fa8103ce0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000e9c0 tx=0x7f0f9000ee80 comp rx=0 tx=0).stop 2024-07-04T08:14:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 shutdown_connections 2024-07-04T08:14:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0f8c0b2840 0x7f0f8c0b4d50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fa8108240 0x7f0fa8104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa81078f0 0x7f0fa8103ce0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 --2- 172.21.15.39:0/933996147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0fa8101fb0 0x7f0fa81037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 >> 172.21.15.39:0/933996147 conn(0x7f0fa8078c80 msgr2=0x7f0fa8105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 shutdown_connections 2024-07-04T08:14:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:49.413+0000 7f0faf000640 1 -- 172.21.15.39:0/933996147 wait complete. 2024-07-04T08:14:49.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:49 smithi133 ceph-mon[37050]: pgmap v468: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:14:49.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:49 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:14:49.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:49 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:14:49.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:49 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:14:50.186 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-04T08:14:50.186 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-04T08:14:50.186 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2024-07-04T08:14:50.186 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-04T08:14:50.186 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-04T08:14:50.186 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-04T08:14:50.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-04T08:14:50.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-04T08:14:50.187 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-04T08:14:50.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:50.187 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2024-07-04T08:14:50.245 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-04T08:14:50.245 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-04T08:14:50.245 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2024-07-04T08:14:50.245 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-07-04T08:14:50.245 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2024-07-04T08:14:50.245 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-07-04T08:14:50.245 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-07-04T08:14:50.246 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 41687sec preferred_lft 41687sec 2024-07-04T08:14:50.247 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link noprefixroute 2024-07-04T08:14:50.247 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2024-07-04T08:14:50.247 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-04T08:14:50.247 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2024-07-04T08:14:50.247 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2024-07-04T08:14:50.248 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-04T08:14:50.248 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:14:50.248 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:14:50.248 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2024-07-04T08:14:50.248 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2024-07-04T08:14:50.248 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2024-07-04T08:14:50.248 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-04T08:14:50.248 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:14:50.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:50 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/933996147' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-04T08:14:50.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:14:50.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:50 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/933996147' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-04T08:14:50.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:14:50.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:50.505 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-04T08:14:50.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:50 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/933996147' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-04T08:14:50.568 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-07-04T08:14:50.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:50.572 DEBUG:teuthology.orchestra.run.smithi039:> route 2024-07-04T08:14:50.619 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2024-07-04T08:14:50.619 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-04T08:14:50.619 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-07-04T08:14:50.619 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-07-04T08:14:50.620 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-04T08:14:50.620 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:14:50.620 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:14:50.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-07-04T08:14:50.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-07-04T08:14:50.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-07-04T08:14:50.620 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:14:50.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:14:50.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:14:50.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:50.842 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-07-04T08:14:50.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:50.898 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2024-07-04T08:14:50.954 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:14:50.954 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:14:50.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T08:14:50.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-04T08:14:50.954 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:14:51.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:14:51.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:14:51.151 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-04T08:14:51.151 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:14:51.151 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:14:51.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-04T08:14:51.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-04T08:14:51.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-04T08:14:51.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-04T08:14:51.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-04T08:14:51.152 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:14:51.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:14:51.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:51 smithi039 ceph-mon[32062]: pgmap v469: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:51.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:51 smithi050 ceph-mon[36577]: pgmap v469: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:51.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:14:51.432 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-04T08:14:51.432 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2024-07-04T08:14:51.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2024-07-04T08:14:51.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2024-07-04T08:14:51.433 DEBUG:teuthology.orchestra.run.smithi039:> ') 2024-07-04T08:14:51.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:14:51.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:51 smithi133 ceph-mon[37050]: pgmap v469: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:51.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:14:51.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T08:14:51.653 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-04T08:14:51.698 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-04T08:14:51.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-04T08:14:51.699 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-04T08:14:51.797 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-07-04T08:14:51.797 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-07-04T08:14:51.797 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/ea788566-52d9-4ba0-9ed3-9e28950c3dc4 2024-07-04T08:14:51.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:51.797 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/ea788566-52d9-4ba0-9ed3-9e28950c3dc4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-07-04T08:14:52.014 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-07-04T08:14:52.014 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-07-04T08:14:52.014 INFO:teuthology.orchestra.run.smithi039.stdout:invalid new device string format 2024-07-04T08:14:52.743 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-07-04T08:14:52.744 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-07-04T08:14:52.744 INFO:teuthology.orchestra.run.smithi039.stdout:invalid new device string format 2024-07-04T08:14:52.744 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.39:6789,172.21.15.50:6789,172.21.15.133:6789" 2024-07-04T08:14:52.744 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.39:6789,172.21.15.50:6789,172.21.15.133:6789:/volumes/qa/sv_1/ea788566-52d9-4ba0-9ed3-9e28950c3dc4 2024-07-04T08:14:52.744 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=2ee561ca-39db-11ef-bcab-c7b262605968" will pass to kernel 2024-07-04T08:14:52.747 INFO:teuthology.orchestra.run.smithi039.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-04T08:14:52.747 INFO:teuthology.orchestra.run.smithi039.stdout: You just mounted a file system that supports labels which does not 2024-07-04T08:14:52.747 INFO:teuthology.orchestra.run.smithi039.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-04T08:14:52.747 INFO:teuthology.orchestra.run.smithi039.stdout: applications will generate AVC messages and not be allowed access to 2024-07-04T08:14:52.748 INFO:teuthology.orchestra.run.smithi039.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-04T08:14:52.752 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-04T08:14:52.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:14:52.752 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-04T08:14:52.844 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-04T08:14:52.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-04T08:14:52.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:14:52.975 INFO:teuthology.orchestra.run.smithi039.stdout:client15267 2024-07-04T08:14:52.976 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:14:52.976 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T08:14:53.031 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-04T08:14:53.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-04T08:14:53.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:14:53.155 INFO:teuthology.orchestra.run.smithi039.stdout:client15267 2024-07-04T08:14:53.155 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-04T08:14:53.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-04T08:14:53.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:14:53.278 INFO:teuthology.orchestra.run.smithi039.stdout:client15267 2024-07-04T08:14:53.278 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2ee561ca-39db-11ef-bcab-c7b262605968.client15267/status') 2024-07-04T08:14:53.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-04T08:14:53.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:53 smithi050 ceph-mon[36577]: pgmap v470: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:53.414 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:53 smithi039 ceph-mon[32062]: pgmap v470: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:53.422 INFO:teuthology.orchestra.run.smithi039.stdout:instance: client.15267 (3)192.168.0.1:0/3995931503 2024-07-04T08:14:53.422 INFO:teuthology.orchestra.run.smithi039.stdout:blocklisted: false 2024-07-04T08:14:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records in 2024-07-04T08:14:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records out 2024-07-04T08:14:53.422 INFO:teuthology.orchestra.run.smithi039.stderr:70 bytes copied, 8.2418e-05 s, 849 kB/s 2024-07-04T08:14:53.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-04T08:14:53.436 INFO:teuthology.run_tasks:Running task exec... 2024-07-04T08:14:53.444 INFO:teuthology.task.exec:Executing custom commands... 2024-07-04T08:14:53.445 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T08:14:53.445 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-07-04T08:14:53.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:53 smithi133 ceph-mon[37050]: pgmap v470: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:14:53.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.733+0000 7f5d05000640 1 -- 172.21.15.39:0/1233842773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 msgr2=0x7f5d001040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:53.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.733+0000 7f5d05000640 1 --2- 172.21.15.39:0/1233842773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 0x7f5d001040a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009860 tx=0x7f5cf002a340 comp rx=0 tx=0).stop 2024-07-04T08:14:53.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.733+0000 7f5d05000640 1 -- 172.21.15.39:0/1233842773 shutdown_connections 2024-07-04T08:14:53.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.733+0000 7f5d05000640 1 --2- 172.21.15.39:0/1233842773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d0010d0f0 0x7f5d0010f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:53.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.733+0000 7f5d05000640 1 --2- 172.21.15.39:0/1233842773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d001045e0 0x7f5d0010cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:53.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.733+0000 7f5d05000640 1 --2- 172.21.15.39:0/1233842773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 0x7f5d001040a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:53.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.733+0000 7f5d05000640 1 -- 172.21.15.39:0/1233842773 >> 172.21.15.39:0/1233842773 conn(0x7f5d000fd780 msgr2=0x7f5d000ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:53.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.733+0000 7f5d05000640 1 -- 172.21.15.39:0/1233842773 shutdown_connections 2024-07-04T08:14:53.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.733+0000 7f5d05000640 1 -- 172.21.15.39:0/1233842773 wait complete. 2024-07-04T08:14:53.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.734+0000 7f5d05000640 1 Processor -- start 2024-07-04T08:14:53.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.735+0000 7f5d05000640 1 -- start start 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.735+0000 7f5d05000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 0x7f5d0019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.735+0000 7f5d05000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d001045e0 0x7f5d0019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.735+0000 7f5d05000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d0010d0f0 0x7f5d001a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.735+0000 7f5d05000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00112d70 con 0x7f5d001045e0 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5d05000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00112bf0 con 0x7f5d00103c90 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5d05000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00112ef0 con 0x7f5d0010d0f0 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cff400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 0x7f5d0019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cfea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d001045e0 0x7f5d0019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cff400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 0x7f5d0019b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48990/0 (socket says 172.21.15.39:48990) 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cff400640 1 -- 172.21.15.39:0/1592232403 learned_addr learned my addr 172.21.15.39:0/1592232403 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:53.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cffe00640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d0010d0f0 0x7f5d001a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:53.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cff400640 1 -- 172.21.15.39:0/1592232403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d0010d0f0 msgr2=0x7f5d001a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:53.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cff400640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d0010d0f0 0x7f5d001a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cff400640 1 -- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d001045e0 msgr2=0x7f5d0019bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cff400640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d001045e0 0x7f5d0019bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cff400640 1 -- 172.21.15.39:0/1592232403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d001a1900 con 0x7f5d00103c90 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.736+0000 7f5cfea00640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d001045e0 0x7f5d0019bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.737+0000 7f5cff400640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 0x7f5d0019b9d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00303f0 tx=0x7f5cf0032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.737+0000 7f5cf7e00640 1 -- 172.21.15.39:0/1592232403 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0040020 con 0x7f5d00103c90 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.737+0000 7f5cf7e00640 1 -- 172.21.15.39:0/1592232403 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf0030dd0 con 0x7f5d00103c90 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.737+0000 7f5cf7e00640 1 -- 172.21.15.39:0/1592232403 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0037040 con 0x7f5d00103c90 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.737+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d001a1b90 con 0x7f5d00103c90 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.737+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d001a2030 con 0x7f5d00103c90 2024-07-04T08:14:53.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.738+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d001040a0 con 0x7f5d00103c90 2024-07-04T08:14:53.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.741+0000 7f5cf7e00640 1 -- 172.21.15.39:0/1592232403 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7f5cf0043020 con 0x7f5d00103c90 2024-07-04T08:14:53.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.742+0000 7f5cf7e00640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5ce40b2840 0x7f5ce40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:53.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.743+0000 7f5cf7e00640 1 -- 172.21.15.39:0/1592232403 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f5cf0102750 con 0x7f5d00103c90 2024-07-04T08:14:53.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.743+0000 7f5cfea00640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5ce40b2840 0x7f5ce40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:53.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.744+0000 7f5cfea00640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5ce40b2840 0x7f5ce40b4d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5d00100f60 tx=0x7f5cec005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:53.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:53.744+0000 7f5cf7e00640 1 -- 172.21.15.39:0/1592232403 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5d001a2030 con 0x7f5d00103c90 2024-07-04T08:14:54.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.006+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f5d001a85b0 con 0x7f5d00103c90 2024-07-04T08:14:54.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.125+0000 7f5cf7e00640 1 -- 172.21.15.39:0/1592232403 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7f5cf004a020 con 0x7f5d00103c90 2024-07-04T08:14:54.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5ce40b2840 msgr2=0x7f5ce40b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:54.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5ce40b2840 0x7f5ce40b4d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5d00100f60 tx=0x7f5cec005f90 comp rx=0 tx=0).stop 2024-07-04T08:14:54.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 msgr2=0x7f5d0019b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:54.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 0x7f5d0019b9d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00303f0 tx=0x7f5cf0032da0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 shutdown_connections 2024-07-04T08:14:54.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5ce40b2840 0x7f5ce40b4d50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d001045e0 0x7f5d0019bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d0010d0f0 0x7f5d001a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 --2- 172.21.15.39:0/1592232403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5d00103c90 0x7f5d0019b9d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.128+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 >> 172.21.15.39:0/1592232403 conn(0x7f5d000fd780 msgr2=0x7f5d001018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.129+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 shutdown_connections 2024-07-04T08:14:54.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.129+0000 7f5d05000640 1 -- 172.21.15.39:0/1592232403 wait complete. 2024-07-04T08:14:54.161 INFO:teuthology.run_tasks:Running task check-counter... 2024-07-04T08:14:54.173 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-07-04T08:14:54.184 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T08:14:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:54 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1592232403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-04T08:14:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:54 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-04T08:14:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.474+0000 7fc4a5400640 1 -- 172.21.15.39:0/174856904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4a0109210 msgr2=0x7fc4a0109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.474+0000 7fc4a5400640 1 --2- 172.21.15.39:0/174856904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4a0109210 0x7fc4a0109620 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009900 tx=0x7fc48c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.474+0000 7fc4a5400640 1 -- 172.21.15.39:0/174856904 shutdown_connections 2024-07-04T08:14:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.474+0000 7fc4a5400640 1 --2- 172.21.15.39:0/174856904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a0109b60 0x7fc4a0115e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.474+0000 7fc4a5400640 1 --2- 172.21.15.39:0/174856904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4a0109210 0x7fc4a0109620 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.474+0000 7fc4a5400640 1 --2- 172.21.15.39:0/174856904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a010dfc0 0x7fc4a010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.474+0000 7fc4a5400640 1 -- 172.21.15.39:0/174856904 >> 172.21.15.39:0/174856904 conn(0x7fc4a0078bd0 msgr2=0x7fc4a0107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.474+0000 7fc4a5400640 1 -- 172.21.15.39:0/174856904 shutdown_connections 2024-07-04T08:14:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.474+0000 7fc4a5400640 1 -- 172.21.15.39:0/174856904 wait complete. 2024-07-04T08:14:54.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.475+0000 7fc4a5400640 1 Processor -- start 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.475+0000 7fc4a5400640 1 -- start start 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc4a5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a0109210 0x7fc4a019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc4a5400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4a0109b60 0x7fc4a01a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc4a5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a010dfc0 0x7fc4a01a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc4a5400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0119380 con 0x7fc4a0109210 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc4a5400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0119200 con 0x7fc4a010dfc0 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc4a5400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0119500 con 0x7fc4a0109b60 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc4a4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a010dfc0 0x7fc4a01a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc49fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a0109210 0x7fc4a019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc4a4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a010dfc0 0x7fc4a01a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:49008/0 (socket says 172.21.15.39:49008) 2024-07-04T08:14:54.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc4a4a00640 1 -- 172.21.15.39:0/1565285319 learned_addr learned my addr 172.21.15.39:0/1565285319 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.476+0000 7fc49f400640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4a0109b60 0x7fc4a01a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc4a4a00640 1 -- 172.21.15.39:0/1565285319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4a0109b60 msgr2=0x7fc4a01a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc4a4a00640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4a0109b60 0x7fc4a01a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc4a4a00640 1 -- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a0109210 msgr2=0x7fc4a019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc4a4a00640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a0109210 0x7fc4a019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc4a4a00640 1 -- 172.21.15.39:0/1565285319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a01a5d10 con 0x7fc4a010dfc0 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc49f400640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4a0109b60 0x7fc4a01a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc49fe00640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a0109210 0x7fc4a019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc4a4a00640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a010dfc0 0x7fc4a01a5610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc49800efd0 tx=0x7fc49800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc49cc00640 1 -- 172.21.15.39:0/1565285319 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498003b70 con 0x7fc4a010dfc0 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc49cc00640 1 -- 172.21.15.39:0/1565285319 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc498003d10 con 0x7fc4a010dfc0 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a01a6000 con 0x7fc4a010dfc0 2024-07-04T08:14:54.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.477+0000 7fc49cc00640 1 -- 172.21.15.39:0/1565285319 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980108a0 con 0x7fc4a010dfc0 2024-07-04T08:14:54.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.478+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a01ac790 con 0x7fc4a010dfc0 2024-07-04T08:14:54.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.479+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a0109620 con 0x7fc4a010dfc0 2024-07-04T08:14:54.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.481+0000 7fc49cc00640 1 -- 172.21.15.39:0/1565285319 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7fc498004b40 con 0x7fc4a010dfc0 2024-07-04T08:14:54.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.483+0000 7fc49cc00640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc4840b2840 0x7fc4840b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:54.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.483+0000 7fc49cc00640 1 -- 172.21.15.39:0/1565285319 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc4980e2fe0 con 0x7fc4a010dfc0 2024-07-04T08:14:54.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.483+0000 7fc49fe00640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc4840b2840 0x7fc4840b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:54.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.484+0000 7fc49fe00640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc4840b2840 0x7fc4840b4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4a010fda0 tx=0x7fc490005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:54.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.484+0000 7fc49cc00640 1 -- 172.21.15.39:0/1565285319 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc4980aa870 con 0x7fc4a010dfc0 2024-07-04T08:14:54.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:54 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1592232403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-04T08:14:54.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:54 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-04T08:14:54.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:54 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1592232403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-04T08:14:54.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:54 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-04T08:14:54.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.701+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4a0110f50 con 0x7fc4a010dfc0 2024-07-04T08:14:54.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.703+0000 7fc49cc00640 1 -- 172.21.15.39:0/1565285319 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24957 (secure 0 0 0) 0x7fc4980aa170 con 0x7fc4a010dfc0 2024-07-04T08:14:54.705 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:14:54.705 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:11:53.672125+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:14:54.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc4840b2840 msgr2=0x7fc4840b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:54.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc4840b2840 0x7fc4840b4d50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc4a010fda0 tx=0x7fc490005f90 comp rx=0 tx=0).stop 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a010dfc0 msgr2=0x7fc4a01a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a010dfc0 0x7fc4a01a5610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc49800efd0 tx=0x7fc49800c6a0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 shutdown_connections 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc4840b2840 0x7fc4840b4d50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4a010dfc0 0x7fc4a01a5610 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4a0109b60 0x7fc4a01a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 --2- 172.21.15.39:0/1565285319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a0109210 0x7fc4a019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 >> 172.21.15.39:0/1565285319 conn(0x7fc4a0078bd0 msgr2=0x7fc4a010c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.707+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 shutdown_connections 2024-07-04T08:14:54.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:54.708+0000 7fc4a5400640 1 -- 172.21.15.39:0/1565285319 wait complete. 2024-07-04T08:14:54.737 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-04T08:04:17.509668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-04T08:09:32.643834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-04T08:09:37.246918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-07-04T08:09:49.616671+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-07-04T08:14:54.737 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-04T08:14:55.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.010+0000 7f81a3200640 1 -- 172.21.15.39:0/66185416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81a410a980 msgr2=0x7f81a410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.010+0000 7f81a3200640 1 --2- 172.21.15.39:0/66185416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81a410a980 0x7f81a410ad90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8190009900 tx=0x7f819002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.010+0000 7f81a3200640 1 -- 172.21.15.39:0/66185416 shutdown_connections 2024-07-04T08:14:55.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.010+0000 7f81a3200640 1 --2- 172.21.15.39:0/66185416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a410b2d0 0x7f81a4111b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.010+0000 7f81a3200640 1 --2- 172.21.15.39:0/66185416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81a410a980 0x7f81a410ad90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.010+0000 7f81a3200640 1 --2- 172.21.15.39:0/66185416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81a4105050 0x7f81a4105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.010+0000 7f81a3200640 1 -- 172.21.15.39:0/66185416 >> 172.21.15.39:0/66185416 conn(0x7f81a4100ab0 msgr2=0x7f81a4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:55.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.010+0000 7f81a3200640 1 -- 172.21.15.39:0/66185416 shutdown_connections 2024-07-04T08:14:55.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.010+0000 7f81a3200640 1 -- 172.21.15.39:0/66185416 wait complete. 2024-07-04T08:14:55.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.011+0000 7f81a3200640 1 Processor -- start 2024-07-04T08:14:55.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.011+0000 7f81a3200640 1 -- start start 2024-07-04T08:14:55.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a3200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81a4105050 0x7f81a419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a410a980 0x7f81a419bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81a410b2d0 0x7f81a41a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4114f30 con 0x7f81a410b2d0 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a3200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4114db0 con 0x7f81a4105050 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41150b0 con 0x7f81a410a980 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81a410b2d0 0x7f81a41a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81a410b2d0 0x7f81a41a11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51568/0 (socket says 172.21.15.39:51568) 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a2800640 1 -- 172.21.15.39:0/3920632953 learned_addr learned my addr 172.21.15.39:0/3920632953 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a1400640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a410a980 0x7f81a419bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.012+0000 7f81a1e00640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81a4105050 0x7f81a419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f81a2800640 1 -- 172.21.15.39:0/3920632953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a410a980 msgr2=0x7f81a419bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f81a2800640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a410a980 0x7f81a419bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f81a2800640 1 -- 172.21.15.39:0/3920632953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81a4105050 msgr2=0x7f81a419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f81a2800640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81a4105050 0x7f81a419b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f81a2800640 1 -- 172.21.15.39:0/3920632953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a41a18e0 con 0x7f81a410b2d0 2024-07-04T08:14:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f81a1400640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a410a980 0x7f81a419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:55.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f81a1e00640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81a4105050 0x7f81a419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f81a2800640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81a410b2d0 0x7f81a41a11e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f819c00efd0 tx=0x7f819c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f819aa00640 1 -- 172.21.15.39:0/3920632953 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c003b70 con 0x7f81a410b2d0 2024-07-04T08:14:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a41a1bd0 con 0x7f81a410b2d0 2024-07-04T08:14:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f819aa00640 1 -- 172.21.15.39:0/3920632953 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819c003d10 con 0x7f81a410b2d0 2024-07-04T08:14:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.013+0000 7f819aa00640 1 -- 172.21.15.39:0/3920632953 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c0108a0 con 0x7f81a410b2d0 2024-07-04T08:14:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.014+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a41a8360 con 0x7f81a410b2d0 2024-07-04T08:14:55.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.015+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a4105460 con 0x7f81a410b2d0 2024-07-04T08:14:55.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.017+0000 7f819aa00640 1 -- 172.21.15.39:0/3920632953 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7f819c004b40 con 0x7f81a410b2d0 2024-07-04T08:14:55.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.019+0000 7f819aa00640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81880b2840 0x7f81880b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:55.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.019+0000 7f819aa00640 1 -- 172.21.15.39:0/3920632953 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f819c0e30d0 con 0x7f81a410b2d0 2024-07-04T08:14:55.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.019+0000 7f81a1e00640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81880b2840 0x7f81880b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:55.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.020+0000 7f81a1e00640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81880b2840 0x7f81880b4d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8194009c30 tx=0x7f8194005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:55.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.020+0000 7f819aa00640 1 -- 172.21.15.39:0/3920632953 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f819c0aa940 con 0x7f81a410b2d0 2024-07-04T08:14:55.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.222+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81a402de10 con 0x7f81a410b2d0 2024-07-04T08:14:55.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.223+0000 7f819aa00640 1 -- 172.21.15.39:0/3920632953 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f819c0aa240 con 0x7f81a410b2d0 2024-07-04T08:14:55.224 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-07-04T08:14:55.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81880b2840 msgr2=0x7f81880b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81880b2840 0x7f81880b4d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8194009c30 tx=0x7f8194005f90 comp rx=0 tx=0).stop 2024-07-04T08:14:55.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81a410b2d0 msgr2=0x7f81a41a11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81a410b2d0 0x7f81a41a11e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f819c00efd0 tx=0x7f819c00c6a0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 shutdown_connections 2024-07-04T08:14:55.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81880b2840 0x7f81880b4d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81a410b2d0 0x7f81a41a11e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a410a980 0x7f81a419bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 --2- 172.21.15.39:0/3920632953 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81a4105050 0x7f81a419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 >> 172.21.15.39:0/3920632953 conn(0x7f81a4100ab0 msgr2=0x7f81a4102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:55.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 shutdown_connections 2024-07-04T08:14:55.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.226+0000 7f81a3200640 1 -- 172.21.15.39:0/3920632953 wait complete. 2024-07-04T08:14:55.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-04T08:14:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:55 smithi050 ceph-mon[36577]: pgmap v471: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:55 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-04T08:14:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:55 smithi050 ceph-mon[36577]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-04T08:14:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:55 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1565285319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:14:55.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.531+0000 7fa11ca00640 1 -- 172.21.15.39:0/3194187851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 msgr2=0x7fa118109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.531+0000 7fa11ca00640 1 --2- 172.21.15.39:0/3194187851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 0x7fa118109620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa100009900 tx=0x7fa10002a410 comp rx=0 tx=0).stop 2024-07-04T08:14:55.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.531+0000 7fa11ca00640 1 -- 172.21.15.39:0/3194187851 shutdown_connections 2024-07-04T08:14:55.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.532+0000 7fa11ca00640 1 --2- 172.21.15.39:0/3194187851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 0x7fa118109620 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.532+0000 7fa11ca00640 1 --2- 172.21.15.39:0/3194187851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa118109b60 0x7fa118115e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.532+0000 7fa11ca00640 1 --2- 172.21.15.39:0/3194187851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11810dfc0 0x7fa11810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.532+0000 7fa11ca00640 1 -- 172.21.15.39:0/3194187851 >> 172.21.15.39:0/3194187851 conn(0x7fa118078bd0 msgr2=0x7fa118107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:55.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.532+0000 7fa11ca00640 1 -- 172.21.15.39:0/3194187851 shutdown_connections 2024-07-04T08:14:55.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.532+0000 7fa11ca00640 1 -- 172.21.15.39:0/3194187851 wait complete. 2024-07-04T08:14:55.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.533+0000 7fa11ca00640 1 Processor -- start 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.533+0000 7fa11ca00640 1 -- start start 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.533+0000 7fa11ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 0x7fa11819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.533+0000 7fa11ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa118109b60 0x7fa1181a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.533+0000 7fa11ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11810dfc0 0x7fa1181a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.533+0000 7fa11ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa118119380 con 0x7fa118109210 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.533+0000 7fa11ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa118119200 con 0x7fa118109b60 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.533+0000 7fa11ca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa118119500 con 0x7fa11810dfc0 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.533+0000 7fa116800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 0x7fa11819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa116800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 0x7fa11819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51578/0 (socket says 172.21.15.39:51578) 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa116800640 1 -- 172.21.15.39:0/465210745 learned_addr learned my addr 172.21.15.39:0/465210745 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa10fe00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa118109b60 0x7fa1181a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa117200640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11810dfc0 0x7fa1181a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:55.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa116800640 1 -- 172.21.15.39:0/465210745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11810dfc0 msgr2=0x7fa1181a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa116800640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11810dfc0 0x7fa1181a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa116800640 1 -- 172.21.15.39:0/465210745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa118109b60 msgr2=0x7fa1181a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa116800640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa118109b60 0x7fa1181a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa116800640 1 -- 172.21.15.39:0/465210745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1181a5cb0 con 0x7fa118109210 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa117200640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11810dfc0 0x7fa1181a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa10fe00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa118109b60 0x7fa1181a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.534+0000 7fa116800640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 0x7fa11819fd80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa10800d950 tx=0x7fa10800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.535+0000 7fa10f400640 1 -- 172.21.15.39:0/465210745 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108017020 con 0x7fa118109210 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.535+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1181a5fa0 con 0x7fa118109210 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.535+0000 7fa10f400640 1 -- 172.21.15.39:0/465210745 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa108009c30 con 0x7fa118109210 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.535+0000 7fa10f400640 1 -- 172.21.15.39:0/465210745 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10800f690 con 0x7fa118109210 2024-07-04T08:14:55.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.535+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1181ac730 con 0x7fa118109210 2024-07-04T08:14:55.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.536+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa118109620 con 0x7fa118109210 2024-07-04T08:14:55.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.538+0000 7fa10f400640 1 -- 172.21.15.39:0/465210745 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7fa10801a020 con 0x7fa118109210 2024-07-04T08:14:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.540+0000 7fa10f400640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2840 0x7fa0fc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.540+0000 7fa10f400640 1 -- 172.21.15.39:0/465210745 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa1080e2690 con 0x7fa118109210 2024-07-04T08:14:55.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.540+0000 7fa10fe00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2840 0x7fa0fc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:55.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.541+0000 7fa10fe00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2840 0x7fa0fc0b4d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa10002aa00 tx=0x7fa100031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:55.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.544+0000 7fa10f400640 1 -- 172.21.15.39:0/465210745 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fa108021020 con 0x7fa118109210 2024-07-04T08:14:55.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:55 smithi133 ceph-mon[37050]: pgmap v471: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:55.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:55 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-04T08:14:55.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:55 smithi133 ceph-mon[37050]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-04T08:14:55.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:55 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1565285319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:14:55.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:55 smithi039 ceph-mon[32062]: pgmap v471: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:14:55.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:55 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-04T08:14:55.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:55 smithi039 ceph-mon[32062]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-04T08:14:55.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:55 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1565285319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:14:55.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.762+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa11802de10 con 0x7fa118109210 2024-07-04T08:14:55.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.763+0000 7fa10f400640 1 -- 172.21.15.39:0/465210745 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fa108009de0 con 0x7fa118109210 2024-07-04T08:14:55.765 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.766+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2840 msgr2=0x7fa0fc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2840 0x7fa0fc0b4d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa10002aa00 tx=0x7fa100031400 comp rx=0 tx=0).stop 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 msgr2=0x7fa11819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 0x7fa11819fd80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa10800d950 tx=0x7fa10800de10 comp rx=0 tx=0).stop 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 shutdown_connections 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2840 0x7fa0fc0b4d50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11810dfc0 0x7fa1181a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa118109b60 0x7fa1181a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 --2- 172.21.15.39:0/465210745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa118109210 0x7fa11819fd80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:55.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 >> 172.21.15.39:0/465210745 conn(0x7fa118078bd0 msgr2=0x7fa11810c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:55.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 shutdown_connections 2024-07-04T08:14:55.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:55.767+0000 7fa11ca00640 1 -- 172.21.15.39:0/465210745 wait complete. 2024-07-04T08:14:55.797 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-04T08:14:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.082+0000 7f78baa00640 1 -- 172.21.15.39:0/3948616164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78b41090c0 msgr2=0x7f78b410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.082+0000 7f78baa00640 1 --2- 172.21.15.39:0/3948616164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78b41090c0 0x7f78b410f8f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009ca0 tx=0x7f78ac02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.082+0000 7f78baa00640 1 -- 172.21.15.39:0/3948616164 shutdown_connections 2024-07-04T08:14:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.082+0000 7f78baa00640 1 --2- 172.21.15.39:0/3948616164 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78b4108770 0x7f78b4108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.082+0000 7f78baa00640 1 --2- 172.21.15.39:0/3948616164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78b41090c0 0x7f78b410f8f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.082+0000 7f78baa00640 1 --2- 172.21.15.39:0/3948616164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b4102e40 0x7f78b4103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.082+0000 7f78baa00640 1 -- 172.21.15.39:0/3948616164 >> 172.21.15.39:0/3948616164 conn(0x7f78b40fe880 msgr2=0x7f78b4100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.082+0000 7f78baa00640 1 -- 172.21.15.39:0/3948616164 shutdown_connections 2024-07-04T08:14:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.083+0000 7f78baa00640 1 -- 172.21.15.39:0/3948616164 wait complete. 2024-07-04T08:14:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.084+0000 7f78baa00640 1 Processor -- start 2024-07-04T08:14:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.084+0000 7f78baa00640 1 -- start start 2024-07-04T08:14:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.084+0000 7f78baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78b4102e40 0x7f78b419b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.084+0000 7f78baa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b4108770 0x7f78b419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.084+0000 7f78baa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78b41090c0 0x7f78b41a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.084+0000 7f78baa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4112d80 con 0x7f78b41090c0 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.084+0000 7f78baa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4112c00 con 0x7f78b4102e40 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.084+0000 7f78baa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78b4112f00 con 0x7f78b4108770 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78ba000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78b41090c0 0x7f78b41a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78ba000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78b41090c0 0x7f78b41a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51600/0 (socket says 172.21.15.39:51600) 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78ba000640 1 -- 172.21.15.39:0/3868294244 learned_addr learned my addr 172.21.15.39:0/3868294244 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78b8c00640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b4108770 0x7f78b419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78b9600640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78b4102e40 0x7f78b419b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78ba000640 1 -- 172.21.15.39:0/3868294244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b4108770 msgr2=0x7f78b419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78ba000640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b4108770 0x7f78b419bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78ba000640 1 -- 172.21.15.39:0/3868294244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78b4102e40 msgr2=0x7f78b419b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78ba000640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78b4102e40 0x7f78b419b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.085+0000 7f78ba000640 1 -- 172.21.15.39:0/3868294244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b41a1870 con 0x7f78b41090c0 2024-07-04T08:14:56.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.086+0000 7f78ba000640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78b41090c0 0x7f78b41a1170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0098d0 tx=0x7f78ac0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:56.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.086+0000 7f78b2000640 1 -- 172.21.15.39:0/3868294244 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac003b70 con 0x7f78b41090c0 2024-07-04T08:14:56.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.086+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b41a1b00 con 0x7f78b41090c0 2024-07-04T08:14:56.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.086+0000 7f78b2000640 1 -- 172.21.15.39:0/3868294244 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78ac003d10 con 0x7f78b41090c0 2024-07-04T08:14:56.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.086+0000 7f78b2000640 1 -- 172.21.15.39:0/3868294244 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac02fa20 con 0x7f78b41090c0 2024-07-04T08:14:56.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.086+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b41a82f0 con 0x7f78b41090c0 2024-07-04T08:14:56.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.087+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b41040b0 con 0x7f78b41090c0 2024-07-04T08:14:56.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.090+0000 7f78b2000640 1 -- 172.21.15.39:0/3868294244 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7f78ac02fbe0 con 0x7f78b41090c0 2024-07-04T08:14:56.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.091+0000 7f78b2000640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f789c0b2840 0x7f789c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:56.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.092+0000 7f78b2000640 1 -- 172.21.15.39:0/3868294244 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f78ac103d10 con 0x7f78b41090c0 2024-07-04T08:14:56.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.092+0000 7f78b9600640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f789c0b2840 0x7f789c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:56.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.092+0000 7f78b9600640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f789c0b2840 0x7f789c0b4d50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f78a8005d00 tx=0x7f78a800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:56.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.094+0000 7f78b2000640 1 -- 172.21.15.39:0/3868294244 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f78ac0cb580 con 0x7f78b41090c0 2024-07-04T08:14:56.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.308+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f78b419ca60 con 0x7f78b41090c0 2024-07-04T08:14:56.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.309+0000 7f78b2000640 1 -- 172.21.15.39:0/3868294244 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f78ac0cae80 con 0x7f78b41090c0 2024-07-04T08:14:56.310 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-07-04T08:14:56.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f789c0b2840 msgr2=0x7f789c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f789c0b2840 0x7f789c0b4d50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f78a8005d00 tx=0x7f78a800b410 comp rx=0 tx=0).stop 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78b41090c0 msgr2=0x7f78b41a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78b41090c0 0x7f78b41a1170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0098d0 tx=0x7f78ac0317b0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 shutdown_connections 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f789c0b2840 0x7f789c0b4d50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78b41090c0 0x7f78b41a1170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b4108770 0x7f78b419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 --2- 172.21.15.39:0/3868294244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78b4102e40 0x7f78b419b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 >> 172.21.15.39:0/3868294244 conn(0x7f78b40fe880 msgr2=0x7f78b4107da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.312+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 shutdown_connections 2024-07-04T08:14:56.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.313+0000 7f78baa00640 1 -- 172.21.15.39:0/3868294244 wait complete. 2024-07-04T08:14:56.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-07-04T08:14:56.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:56 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3920632953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-04T08:14:56.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:56 smithi050 ceph-mon[36577]: pgmap v472: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:14:56.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:56 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/465210745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-04T08:14:56.413 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:56 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3920632953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-04T08:14:56.413 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:56 smithi039 ceph-mon[32062]: pgmap v472: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:14:56.414 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:56 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/465210745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-04T08:14:56.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:56 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3920632953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-04T08:14:56.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:56 smithi133 ceph-mon[37050]: pgmap v472: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:14:56.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:56 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/465210745' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-04T08:14:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.630+0000 7fc0c7e00640 1 -- 172.21.15.39:0/696761640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 msgr2=0x7fc0c8115e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.631+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/696761640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 0x7fc0c8115e20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009ca0 tx=0x7fc0c002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.631+0000 7fc0c7e00640 1 -- 172.21.15.39:0/696761640 shutdown_connections 2024-07-04T08:14:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.631+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/696761640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 0x7fc0c8115e20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.631+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/696761640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0c8109210 0x7fc0c8109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.631+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/696761640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0c810dfc0 0x7fc0c810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.631+0000 7fc0c7e00640 1 -- 172.21.15.39:0/696761640 >> 172.21.15.39:0/696761640 conn(0x7fc0c8078bd0 msgr2=0x7fc0c8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:56.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.631+0000 7fc0c7e00640 1 -- 172.21.15.39:0/696761640 shutdown_connections 2024-07-04T08:14:56.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.631+0000 7fc0c7e00640 1 -- 172.21.15.39:0/696761640 wait complete. 2024-07-04T08:14:56.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.632+0000 7fc0c7e00640 1 Processor -- start 2024-07-04T08:14:56.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.632+0000 7fc0c7e00640 1 -- start start 2024-07-04T08:14:56.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.632+0000 7fc0c7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0c8109210 0x7fc0c819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:56.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.632+0000 7fc0c7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 0x7fc0c81a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:56.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0c810dfc0 0x7fc0c81a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:56.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c8119370 con 0x7fc0c8109b60 2024-07-04T08:14:56.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c7e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c81191f0 con 0x7fc0c8109210 2024-07-04T08:14:56.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c81194f0 con 0x7fc0c810dfc0 2024-07-04T08:14:56.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 0x7fc0c81a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:56.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 0x7fc0c81a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51624/0 (socket says 172.21.15.39:51624) 2024-07-04T08:14:56.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6000640 1 -- 172.21.15.39:0/2091926338 learned_addr learned my addr 172.21.15.39:0/2091926338 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:56.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6a00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0c8109210 0x7fc0c819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6000640 1 -- 172.21.15.39:0/2091926338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0c810dfc0 msgr2=0x7fc0c81a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6000640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0c810dfc0 0x7fc0c81a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6000640 1 -- 172.21.15.39:0/2091926338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0c8109210 msgr2=0x7fc0c819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6000640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0c8109210 0x7fc0c819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6000640 1 -- 172.21.15.39:0/2091926338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c81a5c60 con 0x7fc0c8109b60 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.633+0000 7fc0c6a00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0c8109210 0x7fc0c819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.634+0000 7fc0c6000640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 0x7fc0c81a02d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e9f0 tx=0x7fc0b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.634+0000 7fc0bf400640 1 -- 172.21.15.39:0/2091926338 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b400ccc0 con 0x7fc0c8109b60 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.634+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c81a5f50 con 0x7fc0c8109b60 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.634+0000 7fc0bf400640 1 -- 172.21.15.39:0/2091926338 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0b40108e0 con 0x7fc0c8109b60 2024-07-04T08:14:56.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.634+0000 7fc0bf400640 1 -- 172.21.15.39:0/2091926338 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4018860 con 0x7fc0c8109b60 2024-07-04T08:14:56.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.634+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c81ac6e0 con 0x7fc0c8109b60 2024-07-04T08:14:56.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.635+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c8109620 con 0x7fc0c8109b60 2024-07-04T08:14:56.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.639+0000 7fc0bf400640 1 -- 172.21.15.39:0/2091926338 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7fc0b4010430 con 0x7fc0c8109b60 2024-07-04T08:14:56.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.640+0000 7fc0bf400640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0ac0b2840 0x7fc0ac0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:56.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.640+0000 7fc0bf400640 1 -- 172.21.15.39:0/2091926338 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc0b40e2a70 con 0x7fc0c8109b60 2024-07-04T08:14:56.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.641+0000 7fc0c6a00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0ac0b2840 0x7fc0ac0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:56.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.641+0000 7fc0c6a00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0ac0b2840 0x7fc0ac0b4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc0c810fda0 tx=0x7fc0b8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:56.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.644+0000 7fc0bf400640 1 -- 172.21.15.39:0/2091926338 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc0b40aa2e0 con 0x7fc0c8109b60 2024-07-04T08:14:56.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.863+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc0c802de10 con 0x7fc0c8109b60 2024-07-04T08:14:56.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.864+0000 7fc0bf400640 1 -- 172.21.15.39:0/2091926338 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fc0b40a9be0 con 0x7fc0c8109b60 2024-07-04T08:14:56.865 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-07-04T08:14:56.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0ac0b2840 msgr2=0x7fc0ac0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0ac0b2840 0x7fc0ac0b4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc0c810fda0 tx=0x7fc0b8005f90 comp rx=0 tx=0).stop 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 msgr2=0x7fc0c81a02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 0x7fc0c81a02d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e9f0 tx=0x7fc0b400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 shutdown_connections 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0ac0b2840 0x7fc0ac0b4d50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0c8109b60 0x7fc0c81a02d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0c810dfc0 0x7fc0c81a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 --2- 172.21.15.39:0/2091926338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc0c8109210 0x7fc0c819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 >> 172.21.15.39:0/2091926338 conn(0x7fc0c8078bd0 msgr2=0x7fc0c810c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 shutdown_connections 2024-07-04T08:14:56.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:56.867+0000 7fc0c7e00640 1 -- 172.21.15.39:0/2091926338 wait complete. 2024-07-04T08:14:56.897 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-07-04T08:14:56.898 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:14:57.394 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:14:57.550 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:57 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3868294244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-04T08:14:57.551 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:57 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2091926338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-04T08:14:57.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:57 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3868294244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-04T08:14:57.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:57 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2091926338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-04T08:14:57.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:57 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3868294244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-04T08:14:57.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:57 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2091926338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-04T08:14:58.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:14:58 smithi133 ceph-mon[37050]: pgmap v473: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-07-04T08:14:58.602 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:14:58 smithi039 ceph-mon[32062]: pgmap v473: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-07-04T08:14:58.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:14:58 smithi050 ceph-mon[36577]: pgmap v473: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-07-04T08:14:59.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.676+0000 7fad1d800640 1 -- 172.21.15.39:0/2547738573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad180777e0 msgr2=0x7fad18113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:59.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.676+0000 7fad1d800640 1 --2- 172.21.15.39:0/2547738573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad180777e0 0x7fad18113780 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fad00009860 tx=0x7fad0002a340 comp rx=0 tx=0).stop 2024-07-04T08:14:59.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.676+0000 7fad1d800640 1 -- 172.21.15.39:0/2547738573 shutdown_connections 2024-07-04T08:14:59.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.676+0000 7fad1d800640 1 --2- 172.21.15.39:0/2547738573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad18113cc0 0x7fad181160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.676+0000 7fad1d800640 1 --2- 172.21.15.39:0/2547738573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad180777e0 0x7fad18113780 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.676+0000 7fad1d800640 1 --2- 172.21.15.39:0/2547738573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad18076e90 0x7fad180772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.676+0000 7fad1d800640 1 -- 172.21.15.39:0/2547738573 >> 172.21.15.39:0/2547738573 conn(0x7fad18100d10 msgr2=0x7fad18103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:59.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.676+0000 7fad1d800640 1 -- 172.21.15.39:0/2547738573 shutdown_connections 2024-07-04T08:14:59.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.676+0000 7fad1d800640 1 -- 172.21.15.39:0/2547738573 wait complete. 2024-07-04T08:14:59.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.678+0000 7fad1d800640 1 Processor -- start 2024-07-04T08:14:59.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.678+0000 7fad1d800640 1 -- start start 2024-07-04T08:14:59.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad1d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad18076e90 0x7fad181a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:59.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad1d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad180777e0 0x7fad181a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:59.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad1d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad18113cc0 0x7fad181a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:59.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad1d800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fad181195f0 con 0x7fad180777e0 2024-07-04T08:14:59.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad1d800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fad18119470 con 0x7fad18113cc0 2024-07-04T08:14:59.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad1d800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad18119770 con 0x7fad18076e90 2024-07-04T08:14:59.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad16a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad18076e90 0x7fad181a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:59.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad16a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad18076e90 0x7fad181a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50436/0 (socket says 172.21.15.39:50436) 2024-07-04T08:14:59.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad16a00640 1 -- 172.21.15.39:0/2118061734 learned_addr learned my addr 172.21.15.39:0/2118061734 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:14:59.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad0fe00640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad180777e0 0x7fad181a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:59.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.679+0000 7fad17400640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad18113cc0 0x7fad181a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad16a00640 1 -- 172.21.15.39:0/2118061734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad18113cc0 msgr2=0x7fad181a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad16a00640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad18113cc0 0x7fad181a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad16a00640 1 -- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad180777e0 msgr2=0x7fad181a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad16a00640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad180777e0 0x7fad181a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad16a00640 1 -- 172.21.15.39:0/2118061734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad181a5f70 con 0x7fad18076e90 2024-07-04T08:14:59.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad16a00640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad18076e90 0x7fad181a0000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e9c0 tx=0x7fad0800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:59.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad0f400640 1 -- 172.21.15.39:0/2118061734 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0800cde0 con 0x7fad18076e90 2024-07-04T08:14:59.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad181a6260 con 0x7fad18076e90 2024-07-04T08:14:59.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad0f400640 1 -- 172.21.15.39:0/2118061734 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad080035f0 con 0x7fad18076e90 2024-07-04T08:14:59.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad0f400640 1 -- 172.21.15.39:0/2118061734 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08010690 con 0x7fad18076e90 2024-07-04T08:14:59.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.680+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad181ac9c0 con 0x7fad18076e90 2024-07-04T08:14:59.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.682+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad180772a0 con 0x7fad18076e90 2024-07-04T08:14:59.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.685+0000 7fad0f400640 1 -- 172.21.15.39:0/2118061734 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7fad08004000 con 0x7fad18076e90 2024-07-04T08:14:59.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.686+0000 7fad0f400640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7facfc0b2840 0x7facfc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:14:59.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.686+0000 7fad0f400640 1 -- 172.21.15.39:0/2118061734 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fad080e2a10 con 0x7fad18076e90 2024-07-04T08:14:59.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.686+0000 7fad0fe00640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7facfc0b2840 0x7facfc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:14:59.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.687+0000 7fad0fe00640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7facfc0b2840 0x7facfc0b4d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fad00005940 tx=0x7fad000058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:14:59.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.690+0000 7fad0f400640 1 -- 172.21.15.39:0/2118061734 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fad08009510 con 0x7fad18076e90 2024-07-04T08:14:59.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.949+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad181a1130 con 0x7fad18076e90 2024-07-04T08:14:59.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.950+0000 7fad0f400640 1 -- 172.21.15.39:0/2118061734 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fad080aa280 con 0x7fad18076e90 2024-07-04T08:14:59.952 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:14:59.952 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:14:59.952 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:14:59.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7facfc0b2840 msgr2=0x7facfc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:59.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7facfc0b2840 0x7facfc0b4d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fad00005940 tx=0x7fad000058d0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad18076e90 msgr2=0x7fad181a0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:14:59.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad18076e90 0x7fad181a0000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e9c0 tx=0x7fad0800ee80 comp rx=0 tx=0).stop 2024-07-04T08:14:59.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 shutdown_connections 2024-07-04T08:14:59.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7facfc0b2840 0x7facfc0b4d50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad180777e0 0x7fad181a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad18113cc0 0x7fad181a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 --2- 172.21.15.39:0/2118061734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad18076e90 0x7fad181a0000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:14:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.953+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 >> 172.21.15.39:0/2118061734 conn(0x7fad18100d10 msgr2=0x7fad18103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:14:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.954+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 shutdown_connections 2024-07-04T08:14:59.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:14:59.954+0000 7fad1d800640 1 -- 172.21.15.39:0/2118061734 wait complete. 2024-07-04T08:15:00.857 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-07-04T08:15:00.857 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-07-04T08:15:00.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T08:15:01.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:00 smithi133 ceph-mon[37050]: pgmap v474: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-04T08:15:01.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:00 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2118061734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:00 smithi039 ceph-mon[32062]: pgmap v474: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-04T08:15:01.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:00 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2118061734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:00 smithi050 ceph-mon[36577]: pgmap v474: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-04T08:15:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:00 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2118061734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:01.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.154+0000 7f83eea00640 1 -- 172.21.15.39:0/1548678752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83e8108870 msgr2=0x7f83e8108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:01.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.154+0000 7f83eea00640 1 --2- 172.21.15.39:0/1548678752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83e8108870 0x7f83e8108c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009900 tx=0x7f83d402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.154+0000 7f83eea00640 1 -- 172.21.15.39:0/1548678752 shutdown_connections 2024-07-04T08:15:01.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.154+0000 7f83eea00640 1 --2- 172.21.15.39:0/1548678752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e81091c0 0x7f83e810f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.154+0000 7f83eea00640 1 --2- 172.21.15.39:0/1548678752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83e8108870 0x7f83e8108c80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.154+0000 7f83eea00640 1 --2- 172.21.15.39:0/1548678752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e8102e40 0x7f83e8103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.154+0000 7f83eea00640 1 -- 172.21.15.39:0/1548678752 >> 172.21.15.39:0/1548678752 conn(0x7f83e80fe880 msgr2=0x7f83e8100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:01.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.154+0000 7f83eea00640 1 -- 172.21.15.39:0/1548678752 shutdown_connections 2024-07-04T08:15:01.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.154+0000 7f83eea00640 1 -- 172.21.15.39:0/1548678752 wait complete. 2024-07-04T08:15:01.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.155+0000 7f83eea00640 1 Processor -- start 2024-07-04T08:15:01.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83eea00640 1 -- start start 2024-07-04T08:15:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83eea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8102e40 0x7f83e8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83eea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83e8108870 0x7f83e8199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83eea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e81091c0 0x7f83e819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83eea00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8112ed0 con 0x7f83e8102e40 2024-07-04T08:15:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83eea00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8112d50 con 0x7f83e8108870 2024-07-04T08:15:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83eea00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8113050 con 0x7f83e81091c0 2024-07-04T08:15:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83ed600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8102e40 0x7f83e8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83ed600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8102e40 0x7f83e8199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59898/0 (socket says 172.21.15.39:59898) 2024-07-04T08:15:01.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83ed600640 1 -- 172.21.15.39:0/198265805 learned_addr learned my addr 172.21.15.39:0/198265805 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.156+0000 7f83ecc00640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83e8108870 0x7f83e8199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.157+0000 7f83ee000640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e81091c0 0x7f83e819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.157+0000 7f83ed600640 1 -- 172.21.15.39:0/198265805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e81091c0 msgr2=0x7f83e819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.157+0000 7f83ed600640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e81091c0 0x7f83e819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.157+0000 7f83ed600640 1 -- 172.21.15.39:0/198265805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83e8108870 msgr2=0x7f83e8199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.157+0000 7f83ed600640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83e8108870 0x7f83e8199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.157+0000 7f83ed600640 1 -- 172.21.15.39:0/198265805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e819f840 con 0x7f83e8102e40 2024-07-04T08:15:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.157+0000 7f83ed600640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8102e40 0x7f83e8199910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00ea60 tx=0x7f83dc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.158+0000 7f83e6000640 1 -- 172.21.15.39:0/198265805 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc00cd20 con 0x7f83e8102e40 2024-07-04T08:15:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.158+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e819fb30 con 0x7f83e8102e40 2024-07-04T08:15:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.158+0000 7f83e6000640 1 -- 172.21.15.39:0/198265805 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83dc0035f0 con 0x7f83e8102e40 2024-07-04T08:15:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.158+0000 7f83e6000640 1 -- 172.21.15.39:0/198265805 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc010690 con 0x7f83e8102e40 2024-07-04T08:15:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.158+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e81a62c0 con 0x7f83e8102e40 2024-07-04T08:15:01.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.159+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e8103250 con 0x7f83e8102e40 2024-07-04T08:15:01.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.163+0000 7f83e6000640 1 -- 172.21.15.39:0/198265805 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7f83dc0037a0 con 0x7f83e8102e40 2024-07-04T08:15:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.165+0000 7f83e6000640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83d00b2840 0x7f83d00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.165+0000 7f83e6000640 1 -- 172.21.15.39:0/198265805 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f83dc0e2b80 con 0x7f83e8102e40 2024-07-04T08:15:01.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.165+0000 7f83ecc00640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83d00b2840 0x7f83d00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:01.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.166+0000 7f83ecc00640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83d00b2840 0x7f83d00b4d50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83d402a9e0 tx=0x7f83d4005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:01.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.167+0000 7f83e6000640 1 -- 172.21.15.39:0/198265805 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f83dc055110 con 0x7f83e8102e40 2024-07-04T08:15:01.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.376+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83e8062d50 con 0x7f83d00b2840 2024-07-04T08:15:01.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.393+0000 7f83e6000640 1 -- 172.21.15.39:0/198265805 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340831 (secure 0 0 0) 0x7f83e8062d50 con 0x7f83d00b2840 2024-07-04T08:15:01.400 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:01.402 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:15:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83d00b2840 msgr2=0x7f83d00b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83d00b2840 0x7f83d00b4d50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83d402a9e0 tx=0x7f83d4005a20 comp rx=0 tx=0).stop 2024-07-04T08:15:01.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8102e40 msgr2=0x7f83e8199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8102e40 0x7f83e8199910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00ea60 tx=0x7f83dc00ee60 comp rx=0 tx=0).stop 2024-07-04T08:15:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 shutdown_connections 2024-07-04T08:15:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83d00b2840 0x7f83d00b4d50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83e81091c0 0x7f83e819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83e8108870 0x7f83e8199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 --2- 172.21.15.39:0/198265805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8102e40 0x7f83e8199910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 >> 172.21.15.39:0/198265805 conn(0x7f83e80fe880 msgr2=0x7f83e8100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:01.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 shutdown_connections 2024-07-04T08:15:01.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.402+0000 7f83eea00640 1 -- 172.21.15.39:0/198265805 wait complete. 2024-07-04T08:15:01.433 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-07-04T08:14:59.683575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662779,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2433,"num_read_kb":1326,"num_write":387,"num_write_kb":2305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":819,"ondisk_log_size":819,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374184,"kb_used_data":12552,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567640,"statfs":{"total":1151940427776,"available":1151557263360,"internally_reserved":0,"allocated":12853248,"data_stored":8485088,"data_compressed":498873,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":306,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":56,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006306"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":290,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:53.414925+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:14:53.414925+0000","last_peered":"2024-07-04T08:14:53.414925+0000","last_clean":"2024-07-04T08:14:53.414925+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:14:53.414925+0000","last_undegraded":"2024-07-04T08:14:53.414925+0000","last_fullsized":"2024-07-04T08:14:53.414925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553793+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:12:51.553793+0000","last_peered":"2024-07-04T08:12:51.553793+0000","last_clean":"2024-07-04T08:12:51.553793+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:12:51.553793+0000","last_undegraded":"2024-07-04T08:12:51.553793+0000","last_fullsized":"2024-07-04T08:12:51.553793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:16.118017+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:12:16.118017+0000","last_peered":"2024-07-04T08:12:16.118017+0000","last_clean":"2024-07-04T08:12:16.118017+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:12:16.118017+0000","last_undegraded":"2024-07-04T08:12:16.118017+0000","last_fullsized":"2024-07-04T08:12:16.118017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.698593+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:53.698593+0000","last_peered":"2024-07-04T08:11:53.698593+0000","last_clean":"2024-07-04T08:11:53.698593+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:53.698593+0000","last_undegraded":"2024-07-04T08:11:53.698593+0000","last_fullsized":"2024-07-04T08:11:53.698593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569975+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:12:51.569975+0000","last_peered":"2024-07-04T08:12:51.569975+0000","last_clean":"2024-07-04T08:12:51.569975+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:12:51.569975+0000","last_undegraded":"2024-07-04T08:12:51.569975+0000","last_fullsized":"2024-07-04T08:12:51.569975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:15.756154+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:12:15.756154+0000","last_peered":"2024-07-04T08:12:15.756154+0000","last_clean":"2024-07-04T08:12:15.756154+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:12:15.756154+0000","last_undegraded":"2024-07-04T08:12:15.756154+0000","last_fullsized":"2024-07-04T08:12:15.756154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247127+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:12:05.247127+0000","last_peered":"2024-07-04T08:12:05.247127+0000","last_clean":"2024-07-04T08:12:05.247127+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:12:05.247127+0000","last_undegraded":"2024-07-04T08:12:05.247127+0000","last_fullsized":"2024-07-04T08:12:05.247127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.549821+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:12:51.549821+0000","last_peered":"2024-07-04T08:12:51.549821+0000","last_clean":"2024-07-04T08:12:51.549821+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:12:51.549821+0000","last_undegraded":"2024-07-04T08:12:51.549821+0000","last_fullsized":"2024-07-04T08:12:51.549821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756728+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:12:20.756728+0000","last_peered":"2024-07-04T08:12:20.756728+0000","last_clean":"2024-07-04T08:12:20.756728+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:12:20.756728+0000","last_undegraded":"2024-07-04T08:12:20.756728+0000","last_fullsized":"2024-07-04T08:12:20.756728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247079+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:12:05.247079+0000","last_peered":"2024-07-04T08:12:05.247079+0000","last_clean":"2024-07-04T08:12:05.247079+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:12:05.247079+0000","last_undegraded":"2024-07-04T08:12:05.247079+0000","last_fullsized":"2024-07-04T08:12:05.247079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.535944+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:12:51.535944+0000","last_peered":"2024-07-04T08:12:51.535944+0000","last_clean":"2024-07-04T08:12:51.535944+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:12:51.535944+0000","last_undegraded":"2024-07-04T08:12:51.535944+0000","last_fullsized":"2024-07-04T08:12:51.535944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756706+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:12:20.756706+0000","last_peered":"2024-07-04T08:12:20.756706+0000","last_clean":"2024-07-04T08:12:20.756706+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:12:20.756706+0000","last_undegraded":"2024-07-04T08:12:20.756706+0000","last_fullsized":"2024-07-04T08:12:20.756706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562540+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:12:51.562540+0000","last_peered":"2024-07-04T08:12:51.562540+0000","last_clean":"2024-07-04T08:12:51.562540+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:12:51.562540+0000","last_undegraded":"2024-07-04T08:12:51.562540+0000","last_fullsized":"2024-07-04T08:12:51.562540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844569+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:53.844569+0000","last_peered":"2024-07-04T08:11:53.844569+0000","last_clean":"2024-07-04T08:11:53.844569+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:53.844569+0000","last_undegraded":"2024-07-04T08:11:53.844569+0000","last_fullsized":"2024-07-04T08:11:53.844569+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754878+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:12:05.754878+0000","last_peered":"2024-07-04T08:12:05.754878+0000","last_clean":"2024-07-04T08:12:05.754878+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:12:05.754878+0000","last_undegraded":"2024-07-04T08:12:05.754878+0000","last_fullsized":"2024-07-04T08:12:05.754878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570735+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:12:51.570735+0000","last_peered":"2024-07-04T08:12:51.570735+0000","last_clean":"2024-07-04T08:12:51.570735+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:12:51.570735+0000","last_undegraded":"2024-07-04T08:12:51.570735+0000","last_fullsized":"2024-07-04T08:12:51.570735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691548+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:53.691548+0000","last_peered":"2024-07-04T08:11:53.691548+0000","last_clean":"2024-07-04T08:11:53.691548+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:53.691548+0000","last_undegraded":"2024-07-04T08:11:53.691548+0000","last_fullsized":"2024-07-04T08:11:53.691548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754825+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:12:05.754825+0000","last_peered":"2024-07-04T08:12:05.754825+0000","last_clean":"2024-07-04T08:12:05.754825+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:12:05.754825+0000","last_undegraded":"2024-07-04T08:12:05.754825+0000","last_fullsized":"2024-07-04T08:12:05.754825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566667+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:12:51.566667+0000","last_peered":"2024-07-04T08:12:51.566667+0000","last_clean":"2024-07-04T08:12:51.566667+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:12:51.566667+0000","last_undegraded":"2024-07-04T08:12:51.566667+0000","last_fullsized":"2024-07-04T08:12:51.566667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.697029+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:53.697029+0000","last_peered":"2024-07-04T08:11:53.697029+0000","last_clean":"2024-07-04T08:11:53.697029+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:53.697029+0000","last_undegraded":"2024-07-04T08:11:53.697029+0000","last_fullsized":"2024-07-04T08:11:53.697029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771248+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:11:53.771248+0000","last_peered":"2024-07-04T08:11:53.771248+0000","last_clean":"2024-07-04T08:11:53.771248+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:11:53.771248+0000","last_undegraded":"2024-07-04T08:11:53.771248+0000","last_fullsized":"2024-07-04T08:11:53.771248+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.526597+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:12:51.526597+0000","last_peered":"2024-07-04T08:12:51.526597+0000","last_clean":"2024-07-04T08:12:51.526597+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:12:51.526597+0000","last_undegraded":"2024-07-04T08:12:51.526597+0000","last_fullsized":"2024-07-04T08:12:51.526597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936427+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:53.936427+0000","last_peered":"2024-07-04T08:11:53.936427+0000","last_clean":"2024-07-04T08:11:53.936427+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:53.936427+0000","last_undegraded":"2024-07-04T08:11:53.936427+0000","last_fullsized":"2024-07-04T08:11:53.936427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754887+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:12:05.754887+0000","last_peered":"2024-07-04T08:12:05.754887+0000","last_clean":"2024-07-04T08:12:05.754887+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:12:05.754887+0000","last_undegraded":"2024-07-04T08:12:05.754887+0000","last_fullsized":"2024-07-04T08:12:05.754887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568004+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:12:51.568004+0000","last_peered":"2024-07-04T08:12:51.568004+0000","last_clean":"2024-07-04T08:12:51.568004+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:12:51.568004+0000","last_undegraded":"2024-07-04T08:12:51.568004+0000","last_fullsized":"2024-07-04T08:12:51.568004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692160+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:53.692160+0000","last_peered":"2024-07-04T08:11:53.692160+0000","last_clean":"2024-07-04T08:11:53.692160+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:53.692160+0000","last_undegraded":"2024-07-04T08:11:53.692160+0000","last_fullsized":"2024-07-04T08:11:53.692160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115754+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:11:56.115754+0000","last_peered":"2024-07-04T08:11:56.115754+0000","last_clean":"2024-07-04T08:11:56.115754+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:11:56.115754+0000","last_undegraded":"2024-07-04T08:11:56.115754+0000","last_fullsized":"2024-07-04T08:11:56.115754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.556576+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:12:51.556576+0000","last_peered":"2024-07-04T08:12:51.556576+0000","last_clean":"2024-07-04T08:12:51.556576+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:12:51.556576+0000","last_undegraded":"2024-07-04T08:12:51.556576+0000","last_fullsized":"2024-07-04T08:12:51.556576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'17","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:54.818022+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:13:54.818022+0000","last_peered":"2024-07-04T08:13:54.818022+0000","last_clean":"2024-07-04T08:13:54.818022+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:13:54.818022+0000","last_undegraded":"2024-07-04T08:13:54.818022+0000","last_fullsized":"2024-07-04T08:13:54.818022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.322378+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:11:54.322378+0000","last_peered":"2024-07-04T08:11:54.322378+0000","last_clean":"2024-07-04T08:11:54.322378+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:11:54.322378+0000","last_undegraded":"2024-07-04T08:11:54.322378+0000","last_fullsized":"2024-07-04T08:11:54.322378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540856+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:12:51.540856+0000","last_peered":"2024-07-04T08:12:51.540856+0000","last_clean":"2024-07-04T08:12:51.540856+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:12:51.540856+0000","last_undegraded":"2024-07-04T08:12:51.540856+0000","last_fullsized":"2024-07-04T08:12:51.540856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771431+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:53.771431+0000","last_peered":"2024-07-04T08:11:53.771431+0000","last_clean":"2024-07-04T08:11:53.771431+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:53.771431+0000","last_undegraded":"2024-07-04T08:11:53.771431+0000","last_fullsized":"2024-07-04T08:11:53.771431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.238979+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:11:56.238979+0000","last_peered":"2024-07-04T08:11:56.238979+0000","last_clean":"2024-07-04T08:11:56.238979+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:11:56.238979+0000","last_undegraded":"2024-07-04T08:11:56.238979+0000","last_fullsized":"2024-07-04T08:11:56.238979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542562+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:12:51.542562+0000","last_peered":"2024-07-04T08:12:51.542562+0000","last_clean":"2024-07-04T08:12:51.542562+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:12:51.542562+0000","last_undegraded":"2024-07-04T08:12:51.542562+0000","last_fullsized":"2024-07-04T08:12:51.542562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993415+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:11:53.993415+0000","last_peered":"2024-07-04T08:11:53.993415+0000","last_clean":"2024-07-04T08:11:53.993415+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:11:53.993415+0000","last_undegraded":"2024-07-04T08:11:53.993415+0000","last_fullsized":"2024-07-04T08:11:53.993415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:00.246372+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:12:00.246372+0000","last_peered":"2024-07-04T08:12:00.246372+0000","last_clean":"2024-07-04T08:12:00.246372+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:12:00.246372+0000","last_undegraded":"2024-07-04T08:12:00.246372+0000","last_fullsized":"2024-07-04T08:12:00.246372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.543737+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:12:51.543737+0000","last_peered":"2024-07-04T08:12:51.543737+0000","last_clean":"2024-07-04T08:12:51.543737+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:12:51.543737+0000","last_undegraded":"2024-07-04T08:12:51.543737+0000","last_fullsized":"2024-07-04T08:12:51.543737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":286,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:54.418664+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:14:54.418664+0000","last_peered":"2024-07-04T08:14:54.418664+0000","last_clean":"2024-07-04T08:14:54.418664+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:14:54.418664+0000","last_undegraded":"2024-07-04T08:14:54.418664+0000","last_fullsized":"2024-07-04T08:14:54.418664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":233,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:58.137211+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:11:58.137211+0000","last_peered":"2024-07-04T08:11:58.137211+0000","last_clean":"2024-07-04T08:11:58.137211+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:11:58.137211+0000","last_undegraded":"2024-07-04T08:11:58.137211+0000","last_fullsized":"2024-07-04T08:11:58.137211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.516953+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:12:51.516953+0000","last_peered":"2024-07-04T08:12:51.516953+0000","last_clean":"2024-07-04T08:12:51.516953+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:12:51.516953+0000","last_undegraded":"2024-07-04T08:12:51.516953+0000","last_fullsized":"2024-07-04T08:12:51.516953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":285,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:56.479235+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:14:56.479235+0000","last_peered":"2024-07-04T08:14:56.479235+0000","last_clean":"2024-07-04T08:14:56.479235+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:14:56.479235+0000","last_undegraded":"2024-07-04T08:14:56.479235+0000","last_fullsized":"2024-07-04T08:14:56.479235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":233,"num_read_kb":233,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138305+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:12:03.138305+0000","last_peered":"2024-07-04T08:12:03.138305+0000","last_clean":"2024-07-04T08:12:03.138305+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:12:03.138305+0000","last_undegraded":"2024-07-04T08:12:03.138305+0000","last_fullsized":"2024-07-04T08:12:03.138305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534873+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:12:51.534873+0000","last_peered":"2024-07-04T08:12:51.534873+0000","last_clean":"2024-07-04T08:12:51.534873+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:12:51.534873+0000","last_undegraded":"2024-07-04T08:12:51.534873+0000","last_fullsized":"2024-07-04T08:12:51.534873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851437+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:53.851437+0000","last_peered":"2024-07-04T08:11:53.851437+0000","last_clean":"2024-07-04T08:11:53.851437+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:53.851437+0000","last_undegraded":"2024-07-04T08:11:53.851437+0000","last_fullsized":"2024-07-04T08:11:53.851437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975520+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:12:04.975520+0000","last_peered":"2024-07-04T08:12:04.975520+0000","last_clean":"2024-07-04T08:12:04.975520+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:12:04.975520+0000","last_undegraded":"2024-07-04T08:12:04.975520+0000","last_fullsized":"2024-07-04T08:12:04.975520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.558356+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:12:51.558356+0000","last_peered":"2024-07-04T08:12:51.558356+0000","last_clean":"2024-07-04T08:12:51.558356+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:12:51.558356+0000","last_undegraded":"2024-07-04T08:12:51.558356+0000","last_fullsized":"2024-07-04T08:12:51.558356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691883+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:53.691883+0000","last_peered":"2024-07-04T08:11:53.691883+0000","last_clean":"2024-07-04T08:11:53.691883+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:53.691883+0000","last_undegraded":"2024-07-04T08:11:53.691883+0000","last_fullsized":"2024-07-04T08:11:53.691883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":120,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:46.091830+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:12:46.091830+0000","last_peered":"2024-07-04T08:12:46.091830+0000","last_clean":"2024-07-04T08:12:46.091830+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:12:46.091830+0000","last_undegraded":"2024-07-04T08:12:46.091830+0000","last_fullsized":"2024-07-04T08:12:46.091830+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.085858+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:11:56.085858+0000","last_peered":"2024-07-04T08:11:56.085858+0000","last_clean":"2024-07-04T08:11:56.085858+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:11:56.085858+0000","last_undegraded":"2024-07-04T08:11:56.085858+0000","last_fullsized":"2024-07-04T08:11:56.085858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553863+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:12:51.553863+0000","last_peered":"2024-07-04T08:12:51.553863+0000","last_clean":"2024-07-04T08:12:51.553863+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:12:51.553863+0000","last_undegraded":"2024-07-04T08:12:51.553863+0000","last_fullsized":"2024-07-04T08:12:51.553863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":256,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:57.190728+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:14:57.190728+0000","last_peered":"2024-07-04T08:14:57.190728+0000","last_clean":"2024-07-04T08:14:57.190728+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:14:57.190728+0000","last_undegraded":"2024-07-04T08:14:57.190728+0000","last_fullsized":"2024-07-04T08:14:57.190728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.151807+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:12:04.151807+0000","last_peered":"2024-07-04T08:12:04.151807+0000","last_clean":"2024-07-04T08:12:04.151807+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:12:04.151807+0000","last_undegraded":"2024-07-04T08:12:04.151807+0000","last_fullsized":"2024-07-04T08:12:04.151807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.561500+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:12:51.561500+0000","last_peered":"2024-07-04T08:12:51.561500+0000","last_clean":"2024-07-04T08:12:51.561500+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:12:51.561500+0000","last_undegraded":"2024-07-04T08:12:51.561500+0000","last_fullsized":"2024-07-04T08:12:51.561500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939931+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:53.939931+0000","last_peered":"2024-07-04T08:11:53.939931+0000","last_clean":"2024-07-04T08:11:53.939931+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:11:53.939931+0000","last_undegraded":"2024-07-04T08:11:53.939931+0000","last_fullsized":"2024-07-04T08:11:53.939931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:09.641951+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:12:09.641951+0000","last_peered":"2024-07-04T08:12:09.641951+0000","last_clean":"2024-07-04T08:12:09.641951+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:12:09.641951+0000","last_undegraded":"2024-07-04T08:12:09.641951+0000","last_fullsized":"2024-07-04T08:12:09.641951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552528+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:12:51.552528+0000","last_peered":"2024-07-04T08:12:51.552528+0000","last_clean":"2024-07-04T08:12:51.552528+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:12:51.552528+0000","last_undegraded":"2024-07-04T08:12:51.552528+0000","last_fullsized":"2024-07-04T08:12:51.552528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995043+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:53.995043+0000","last_peered":"2024-07-04T08:11:53.995043+0000","last_clean":"2024-07-04T08:11:53.995043+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:53.995043+0000","last_undegraded":"2024-07-04T08:11:53.995043+0000","last_fullsized":"2024-07-04T08:11:53.995043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975432+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:12:04.975432+0000","last_peered":"2024-07-04T08:12:04.975432+0000","last_clean":"2024-07-04T08:12:04.975432+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:12:04.975432+0000","last_undegraded":"2024-07-04T08:12:04.975432+0000","last_fullsized":"2024-07-04T08:12:04.975432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531965+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:12:51.531965+0000","last_peered":"2024-07-04T08:12:51.531965+0000","last_clean":"2024-07-04T08:12:51.531965+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:12:51.531965+0000","last_undegraded":"2024-07-04T08:12:51.531965+0000","last_fullsized":"2024-07-04T08:12:51.531965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771354+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:53.771354+0000","last_peered":"2024-07-04T08:11:53.771354+0000","last_clean":"2024-07-04T08:11:53.771354+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:53.771354+0000","last_undegraded":"2024-07-04T08:11:53.771354+0000","last_fullsized":"2024-07-04T08:11:53.771354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975473+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:12:04.975473+0000","last_peered":"2024-07-04T08:12:04.975473+0000","last_clean":"2024-07-04T08:12:04.975473+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:12:04.975473+0000","last_undegraded":"2024-07-04T08:12:04.975473+0000","last_fullsized":"2024-07-04T08:12:04.975473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542769+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:12:51.542769+0000","last_peered":"2024-07-04T08:12:51.542769+0000","last_clean":"2024-07-04T08:12:51.542769+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:12:51.542769+0000","last_undegraded":"2024-07-04T08:12:51.542769+0000","last_fullsized":"2024-07-04T08:12:51.542769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:55.245889+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:55.245889+0000","last_peered":"2024-07-04T08:11:55.245889+0000","last_clean":"2024-07-04T08:11:55.245889+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:55.245889+0000","last_undegraded":"2024-07-04T08:11:55.245889+0000","last_fullsized":"2024-07-04T08:11:55.245889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.141070+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:12:18.141070+0000","last_peered":"2024-07-04T08:12:18.141070+0000","last_clean":"2024-07-04T08:12:18.141070+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:12:18.141070+0000","last_undegraded":"2024-07-04T08:12:18.141070+0000","last_fullsized":"2024-07-04T08:12:18.141070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552028+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:12:51.552028+0000","last_peered":"2024-07-04T08:12:51.552028+0000","last_clean":"2024-07-04T08:12:51.552028+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:12:51.552028+0000","last_undegraded":"2024-07-04T08:12:51.552028+0000","last_fullsized":"2024-07-04T08:12:51.552028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:54.475121+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:14:54.475121+0000","last_peered":"2024-07-04T08:14:54.475121+0000","last_clean":"2024-07-04T08:14:54.475121+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:14:54.475121+0000","last_undegraded":"2024-07-04T08:14:54.475121+0000","last_fullsized":"2024-07-04T08:14:54.475121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.666330+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:12:18.666330+0000","last_peered":"2024-07-04T08:12:18.666330+0000","last_clean":"2024-07-04T08:12:18.666330+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:12:18.666330+0000","last_undegraded":"2024-07-04T08:12:18.666330+0000","last_fullsized":"2024-07-04T08:12:18.666330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564982+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:12:51.564982+0000","last_peered":"2024-07-04T08:12:51.564982+0000","last_clean":"2024-07-04T08:12:51.564982+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:12:51.564982+0000","last_undegraded":"2024-07-04T08:12:51.564982+0000","last_fullsized":"2024-07-04T08:12:51.564982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.974209+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:54.974209+0000","last_peered":"2024-07-04T08:11:54.974209+0000","last_clean":"2024-07-04T08:11:54.974209+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:54.974209+0000","last_undegraded":"2024-07-04T08:11:54.974209+0000","last_fullsized":"2024-07-04T08:11:54.974209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:11.240822+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:12:11.240822+0000","last_peered":"2024-07-04T08:12:11.240822+0000","last_clean":"2024-07-04T08:12:11.240822+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:12:11.240822+0000","last_undegraded":"2024-07-04T08:12:11.240822+0000","last_fullsized":"2024-07-04T08:12:11.240822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562193+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:12:51.562193+0000","last_peered":"2024-07-04T08:12:51.562193+0000","last_clean":"2024-07-04T08:12:51.562193+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:12:51.562193+0000","last_undegraded":"2024-07-04T08:12:51.562193+0000","last_fullsized":"2024-07-04T08:12:51.562193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115841+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:56.115841+0000","last_peered":"2024-07-04T08:11:56.115841+0000","last_clean":"2024-07-04T08:11:56.115841+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:56.115841+0000","last_undegraded":"2024-07-04T08:11:56.115841+0000","last_fullsized":"2024-07-04T08:11:56.115841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.904331+0000","last_change":"2024-07-04T08:11:53.997034+0000","last_active":"2024-07-04T08:11:56.904331+0000","last_peered":"2024-07-04T08:11:56.904331+0000","last_clean":"2024-07-04T08:11:56.904331+0000","last_became_active":"2024-07-04T08:11:53.996456+0000","last_became_peered":"2024-07-04T08:11:53.996456+0000","last_unstale":"2024-07-04T08:11:56.904331+0000","last_undegraded":"2024-07-04T08:11:56.904331+0000","last_fullsized":"2024-07-04T08:11:56.904331+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.140549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.521874+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:12:51.521874+0000","last_peered":"2024-07-04T08:12:51.521874+0000","last_clean":"2024-07-04T08:12:51.521874+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:12:51.521874+0000","last_undegraded":"2024-07-04T08:12:51.521874+0000","last_fullsized":"2024-07-04T08:12:51.521874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":559,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:56.261606+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:14:56.261606+0000","last_peered":"2024-07-04T08:14:56.261606+0000","last_clean":"2024-07-04T08:14:56.261606+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:14:56.261606+0000","last_undegraded":"2024-07-04T08:14:56.261606+0000","last_fullsized":"2024-07-04T08:14:56.261606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64691,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":473,"num_read_kb":300,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800726+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:12:14.800726+0000","last_peered":"2024-07-04T08:12:14.800726+0000","last_clean":"2024-07-04T08:12:14.800726+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:12:14.800726+0000","last_undegraded":"2024-07-04T08:12:14.800726+0000","last_fullsized":"2024-07-04T08:12:14.800726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541913+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:12:51.541913+0000","last_peered":"2024-07-04T08:12:51.541913+0000","last_clean":"2024-07-04T08:12:51.541913+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:12:51.541913+0000","last_undegraded":"2024-07-04T08:12:51.541913+0000","last_fullsized":"2024-07-04T08:12:51.541913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974814+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:59.974814+0000","last_peered":"2024-07-04T08:11:59.974814+0000","last_clean":"2024-07-04T08:11:59.974814+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:59.974814+0000","last_undegraded":"2024-07-04T08:11:59.974814+0000","last_fullsized":"2024-07-04T08:11:59.974814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:10.755408+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:12:10.755408+0000","last_peered":"2024-07-04T08:12:10.755408+0000","last_clean":"2024-07-04T08:12:10.755408+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:12:10.755408+0000","last_undegraded":"2024-07-04T08:12:10.755408+0000","last_fullsized":"2024-07-04T08:12:10.755408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569907+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:12:51.569907+0000","last_peered":"2024-07-04T08:12:51.569907+0000","last_clean":"2024-07-04T08:12:51.569907+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:12:51.569907+0000","last_undegraded":"2024-07-04T08:12:51.569907+0000","last_fullsized":"2024-07-04T08:12:51.569907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:01.086478+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:12:01.086478+0000","last_peered":"2024-07-04T08:12:01.086478+0000","last_clean":"2024-07-04T08:12:01.086478+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:12:01.086478+0000","last_undegraded":"2024-07-04T08:12:01.086478+0000","last_fullsized":"2024-07-04T08:12:01.086478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800668+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:12:14.800668+0000","last_peered":"2024-07-04T08:12:14.800668+0000","last_clean":"2024-07-04T08:12:14.800668+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:12:14.800668+0000","last_undegraded":"2024-07-04T08:12:14.800668+0000","last_fullsized":"2024-07-04T08:12:14.800668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.547189+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:12:51.547189+0000","last_peered":"2024-07-04T08:12:51.547189+0000","last_clean":"2024-07-04T08:12:51.547189+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:12:51.547189+0000","last_undegraded":"2024-07-04T08:12:51.547189+0000","last_fullsized":"2024-07-04T08:12:51.547189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":512,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:56.188725+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:14:56.188725+0000","last_peered":"2024-07-04T08:14:56.188725+0000","last_clean":"2024-07-04T08:14:56.188725+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:14:56.188725+0000","last_undegraded":"2024-07-04T08:14:56.188725+0000","last_fullsized":"2024-07-04T08:14:56.188725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":462,"num_read_kb":462,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.326445+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:12:14.326445+0000","last_peered":"2024-07-04T08:12:14.326445+0000","last_clean":"2024-07-04T08:12:14.326445+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:12:14.326445+0000","last_undegraded":"2024-07-04T08:12:14.326445+0000","last_fullsized":"2024-07-04T08:12:14.326445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.639390+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:11:54.639390+0000","last_peered":"2024-07-04T08:11:54.639390+0000","last_clean":"2024-07-04T08:11:54.639390+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:11:54.639390+0000","last_undegraded":"2024-07-04T08:11:54.639390+0000","last_fullsized":"2024-07-04T08:11:54.639390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564562+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:12:51.564562+0000","last_peered":"2024-07-04T08:12:51.564562+0000","last_clean":"2024-07-04T08:12:51.564562+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:12:51.564562+0000","last_undegraded":"2024-07-04T08:12:51.564562+0000","last_fullsized":"2024-07-04T08:12:51.564562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974768+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:59.974768+0000","last_peered":"2024-07-04T08:11:59.974768+0000","last_clean":"2024-07-04T08:11:59.974768+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:59.974768+0000","last_undegraded":"2024-07-04T08:11:59.974768+0000","last_fullsized":"2024-07-04T08:11:59.974768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.239030+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:11:56.239030+0000","last_peered":"2024-07-04T08:11:56.239030+0000","last_clean":"2024-07-04T08:11:56.239030+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:11:56.239030+0000","last_undegraded":"2024-07-04T08:11:56.239030+0000","last_fullsized":"2024-07-04T08:11:56.239030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565645+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:12:51.565645+0000","last_peered":"2024-07-04T08:12:51.565645+0000","last_clean":"2024-07-04T08:12:51.565645+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:12:51.565645+0000","last_undegraded":"2024-07-04T08:12:51.565645+0000","last_fullsized":"2024-07-04T08:12:51.565645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138191+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:12:03.138191+0000","last_peered":"2024-07-04T08:12:03.138191+0000","last_clean":"2024-07-04T08:12:03.138191+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:12:03.138191+0000","last_undegraded":"2024-07-04T08:12:03.138191+0000","last_fullsized":"2024-07-04T08:12:03.138191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:06.905597+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:12:06.905597+0000","last_peered":"2024-07-04T08:12:06.905597+0000","last_clean":"2024-07-04T08:12:06.905597+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:12:06.905597+0000","last_undegraded":"2024-07-04T08:12:06.905597+0000","last_fullsized":"2024-07-04T08:12:06.905597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.554669+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:12:51.554669+0000","last_peered":"2024-07-04T08:12:51.554669+0000","last_clean":"2024-07-04T08:12:51.554669+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:12:51.554669+0000","last_undegraded":"2024-07-04T08:12:51.554669+0000","last_fullsized":"2024-07-04T08:12:51.554669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118570+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:12:21.118570+0000","last_peered":"2024-07-04T08:12:21.118570+0000","last_clean":"2024-07-04T08:12:21.118570+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:12:21.118570+0000","last_undegraded":"2024-07-04T08:12:21.118570+0000","last_fullsized":"2024-07-04T08:12:21.118570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":282,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:53.226636+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:14:53.226636+0000","last_peered":"2024-07-04T08:14:53.226636+0000","last_clean":"2024-07-04T08:14:53.226636+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:14:53.226636+0000","last_undegraded":"2024-07-04T08:14:53.226636+0000","last_fullsized":"2024-07-04T08:14:53.226636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":231,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.519199+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:12:51.519199+0000","last_peered":"2024-07-04T08:12:51.519199+0000","last_clean":"2024-07-04T08:12:51.519199+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:12:51.519199+0000","last_undegraded":"2024-07-04T08:12:51.519199+0000","last_fullsized":"2024-07-04T08:12:51.519199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118593+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:12:21.118593+0000","last_peered":"2024-07-04T08:12:21.118593+0000","last_clean":"2024-07-04T08:12:21.118593+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:12:21.118593+0000","last_undegraded":"2024-07-04T08:12:21.118593+0000","last_fullsized":"2024-07-04T08:12:21.118593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534235+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:12:51.534235+0000","last_peered":"2024-07-04T08:12:51.534235+0000","last_clean":"2024-07-04T08:12:51.534235+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:12:51.534235+0000","last_undegraded":"2024-07-04T08:12:51.534235+0000","last_fullsized":"2024-07-04T08:12:51.534235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.539681+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:12:51.539681+0000","last_peered":"2024-07-04T08:12:51.539681+0000","last_clean":"2024-07-04T08:12:51.539681+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:12:51.539681+0000","last_undegraded":"2024-07-04T08:12:51.539681+0000","last_fullsized":"2024-07-04T08:12:51.539681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566399+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:12:51.566399+0000","last_peered":"2024-07-04T08:12:51.566399+0000","last_clean":"2024-07-04T08:12:51.566399+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:12:51.566399+0000","last_undegraded":"2024-07-04T08:12:51.566399+0000","last_fullsized":"2024-07-04T08:12:51.566399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567851+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:12:51.567851+0000","last_peered":"2024-07-04T08:12:51.567851+0000","last_clean":"2024-07-04T08:12:51.567851+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:12:51.567851+0000","last_undegraded":"2024-07-04T08:12:51.567851+0000","last_fullsized":"2024-07-04T08:12:51.567851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569222+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:12:51.569222+0000","last_peered":"2024-07-04T08:12:51.569222+0000","last_clean":"2024-07-04T08:12:51.569222+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:12:51.569222+0000","last_undegraded":"2024-07-04T08:12:51.569222+0000","last_fullsized":"2024-07-04T08:12:51.569222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532408+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:12:51.532408+0000","last_peered":"2024-07-04T08:12:51.532408+0000","last_clean":"2024-07-04T08:12:51.532408+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:12:51.532408+0000","last_undegraded":"2024-07-04T08:12:51.532408+0000","last_fullsized":"2024-07-04T08:12:51.532408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540538+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:12:51.540538+0000","last_peered":"2024-07-04T08:12:51.540538+0000","last_clean":"2024-07-04T08:12:51.540538+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:12:51.540538+0000","last_undegraded":"2024-07-04T08:12:51.540538+0000","last_fullsized":"2024-07-04T08:12:51.540538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552348+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:12:51.552348+0000","last_peered":"2024-07-04T08:12:51.552348+0000","last_clean":"2024-07-04T08:12:51.552348+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:12:51.552348+0000","last_undegraded":"2024-07-04T08:12:51.552348+0000","last_fullsized":"2024-07-04T08:12:51.552348+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531503+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:12:51.531503+0000","last_peered":"2024-07-04T08:12:51.531503+0000","last_clean":"2024-07-04T08:12:51.531503+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:12:51.531503+0000","last_undegraded":"2024-07-04T08:12:51.531503+0000","last_fullsized":"2024-07-04T08:12:51.531503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.555574+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:12:51.555574+0000","last_peered":"2024-07-04T08:12:51.555574+0000","last_clean":"2024-07-04T08:12:51.555574+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:12:51.555574+0000","last_undegraded":"2024-07-04T08:12:51.555574+0000","last_fullsized":"2024-07-04T08:12:51.555574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560802+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:12:51.560802+0000","last_peered":"2024-07-04T08:12:51.560802+0000","last_clean":"2024-07-04T08:12:51.560802+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:12:51.560802+0000","last_undegraded":"2024-07-04T08:12:51.560802+0000","last_fullsized":"2024-07-04T08:12:51.560802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570948+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:12:51.570948+0000","last_peered":"2024-07-04T08:12:51.570948+0000","last_clean":"2024-07-04T08:12:51.570948+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:12:51.570948+0000","last_undegraded":"2024-07-04T08:12:51.570948+0000","last_fullsized":"2024-07-04T08:12:51.570948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532752+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:12:51.532752+0000","last_peered":"2024-07-04T08:12:51.532752+0000","last_clean":"2024-07-04T08:12:51.532752+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:12:51.532752+0000","last_undegraded":"2024-07-04T08:12:51.532752+0000","last_fullsized":"2024-07-04T08:12:51.532752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552705+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:12:51.552705+0000","last_peered":"2024-07-04T08:12:51.552705+0000","last_clean":"2024-07-04T08:12:51.552705+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:12:51.552705+0000","last_undegraded":"2024-07-04T08:12:51.552705+0000","last_fullsized":"2024-07-04T08:12:51.552705+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541640+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:12:51.541640+0000","last_peered":"2024-07-04T08:12:51.541640+0000","last_clean":"2024-07-04T08:12:51.541640+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:12:51.541640+0000","last_undegraded":"2024-07-04T08:12:51.541640+0000","last_fullsized":"2024-07-04T08:12:51.541640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569708+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:12:51.569708+0000","last_peered":"2024-07-04T08:12:51.569708+0000","last_clean":"2024-07-04T08:12:51.569708+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:12:51.569708+0000","last_undegraded":"2024-07-04T08:12:51.569708+0000","last_fullsized":"2024-07-04T08:12:51.569708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.571140+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:12:51.571140+0000","last_peered":"2024-07-04T08:12:51.571140+0000","last_clean":"2024-07-04T08:12:51.571140+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:12:51.571140+0000","last_undegraded":"2024-07-04T08:12:51.571140+0000","last_fullsized":"2024-07-04T08:12:51.571140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552227+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:12:51.552227+0000","last_peered":"2024-07-04T08:12:51.552227+0000","last_clean":"2024-07-04T08:12:51.552227+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:12:51.552227+0000","last_undegraded":"2024-07-04T08:12:51.552227+0000","last_fullsized":"2024-07-04T08:12:51.552227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567284+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:12:51.567284+0000","last_peered":"2024-07-04T08:12:51.567284+0000","last_clean":"2024-07-04T08:12:51.567284+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:12:51.567284+0000","last_undegraded":"2024-07-04T08:12:51.567284+0000","last_fullsized":"2024-07-04T08:12:51.567284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551582+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:12:51.551582+0000","last_peered":"2024-07-04T08:12:51.551582+0000","last_clean":"2024-07-04T08:12:51.551582+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:12:51.551582+0000","last_undegraded":"2024-07-04T08:12:51.551582+0000","last_fullsized":"2024-07-04T08:12:51.551582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534085+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:12:51.534085+0000","last_peered":"2024-07-04T08:12:51.534085+0000","last_clean":"2024-07-04T08:12:51.534085+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:12:51.534085+0000","last_undegraded":"2024-07-04T08:12:51.534085+0000","last_fullsized":"2024-07-04T08:12:51.534085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553666+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:12:51.553666+0000","last_peered":"2024-07-04T08:12:51.553666+0000","last_clean":"2024-07-04T08:12:51.553666+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:12:51.553666+0000","last_undegraded":"2024-07-04T08:12:51.553666+0000","last_fullsized":"2024-07-04T08:12:51.553666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540373+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:12:51.540373+0000","last_peered":"2024-07-04T08:12:51.540373+0000","last_clean":"2024-07-04T08:12:51.540373+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:12:51.540373+0000","last_undegraded":"2024-07-04T08:12:51.540373+0000","last_fullsized":"2024-07-04T08:12:51.540373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541173+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:12:51.541173+0000","last_peered":"2024-07-04T08:12:51.541173+0000","last_clean":"2024-07-04T08:12:51.541173+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:12:51.541173+0000","last_undegraded":"2024-07-04T08:12:51.541173+0000","last_fullsized":"2024-07-04T08:12:51.541173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560317+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:12:51.560317+0000","last_peered":"2024-07-04T08:12:51.560317+0000","last_clean":"2024-07-04T08:12:51.560317+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:12:51.560317+0000","last_undegraded":"2024-07-04T08:12:51.560317+0000","last_fullsized":"2024-07-04T08:12:51.560317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551363+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:12:51.551363+0000","last_peered":"2024-07-04T08:12:51.551363+0000","last_clean":"2024-07-04T08:12:51.551363+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:12:51.551363+0000","last_undegraded":"2024-07-04T08:12:51.551363+0000","last_fullsized":"2024-07-04T08:12:51.551363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541562+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:12:51.541562+0000","last_peered":"2024-07-04T08:12:51.541562+0000","last_clean":"2024-07-04T08:12:51.541562+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:12:51.541562+0000","last_undegraded":"2024-07-04T08:12:51.541562+0000","last_fullsized":"2024-07-04T08:12:51.541562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.536909+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:12:51.536909+0000","last_peered":"2024-07-04T08:12:51.536909+0000","last_clean":"2024-07-04T08:12:51.536909+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:12:51.536909+0000","last_undegraded":"2024-07-04T08:12:51.536909+0000","last_fullsized":"2024-07-04T08:12:51.536909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537941+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:12:51.537941+0000","last_peered":"2024-07-04T08:12:51.537941+0000","last_clean":"2024-07-04T08:12:51.537941+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:12:51.537941+0000","last_undegraded":"2024-07-04T08:12:51.537941+0000","last_fullsized":"2024-07-04T08:12:51.537941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565574+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:12:51.565574+0000","last_peered":"2024-07-04T08:12:51.565574+0000","last_clean":"2024-07-04T08:12:51.565574+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:12:51.565574+0000","last_undegraded":"2024-07-04T08:12:51.565574+0000","last_fullsized":"2024-07-04T08:12:51.565574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568095+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:12:51.568095+0000","last_peered":"2024-07-04T08:12:51.568095+0000","last_clean":"2024-07-04T08:12:51.568095+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:12:51.568095+0000","last_undegraded":"2024-07-04T08:12:51.568095+0000","last_fullsized":"2024-07-04T08:12:51.568095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569523+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:12:51.569523+0000","last_peered":"2024-07-04T08:12:51.569523+0000","last_clean":"2024-07-04T08:12:51.569523+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:12:51.569523+0000","last_undegraded":"2024-07-04T08:12:51.569523+0000","last_fullsized":"2024-07-04T08:12:51.569523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72177,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2352,"num_read_kb":1257,"num_write":262,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":221904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612689,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":543564,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1195603,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":547590,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168673,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":543872,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364903,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1141042,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528428,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":539530,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724657,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1135560,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920886,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":543604,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117116,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":538706,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378755,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":603769,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542280,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":611794,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705805,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":540454,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:15:01.440 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T08:15:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.766+0000 7fde23400640 1 -- 172.21.15.39:0/1982500499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c10f5f0 msgr2=0x7fde1c111a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.766+0000 7fde23400640 1 --2- 172.21.15.39:0/1982500499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c10f5f0 0x7fde1c111a10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fde14009ca0 tx=0x7fde1402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.767+0000 7fde23400640 1 -- 172.21.15.39:0/1982500499 shutdown_connections 2024-07-04T08:15:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.767+0000 7fde23400640 1 --2- 172.21.15.39:0/1982500499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde1c0755d0 0x7fde1c10f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.767+0000 7fde23400640 1 --2- 172.21.15.39:0/1982500499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c10f5f0 0x7fde1c111a10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.767+0000 7fde23400640 1 --2- 172.21.15.39:0/1982500499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde1c074c80 0x7fde1c075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.767+0000 7fde23400640 1 -- 172.21.15.39:0/1982500499 >> 172.21.15.39:0/1982500499 conn(0x7fde1c0fea60 msgr2=0x7fde1c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.767+0000 7fde23400640 1 -- 172.21.15.39:0/1982500499 shutdown_connections 2024-07-04T08:15:01.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.767+0000 7fde23400640 1 -- 172.21.15.39:0/1982500499 wait complete. 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.768+0000 7fde23400640 1 Processor -- start 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.768+0000 7fde23400640 1 -- start start 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.768+0000 7fde23400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde1c074c80 0x7fde1c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.768+0000 7fde23400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0755d0 0x7fde1c19c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.768+0000 7fde23400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde1c10f5f0 0x7fde1c1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.768+0000 7fde23400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c115140 con 0x7fde1c074c80 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.768+0000 7fde23400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c114fc0 con 0x7fde1c10f5f0 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde23400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1152c0 con 0x7fde1c0755d0 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde1be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0755d0 0x7fde1c19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde20a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde1c074c80 0x7fde1c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde1be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0755d0 0x7fde1c19c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33226/0 (socket says 172.21.15.39:33226) 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde1be00640 1 -- 172.21.15.39:0/2167824384 learned_addr learned my addr 172.21.15.39:0/2167824384 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde21400640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde1c10f5f0 0x7fde1c1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde1be00640 1 -- 172.21.15.39:0/2167824384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde1c10f5f0 msgr2=0x7fde1c1a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde1be00640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde1c10f5f0 0x7fde1c1a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde1be00640 1 -- 172.21.15.39:0/2167824384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde1c074c80 msgr2=0x7fde1c19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde1be00640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde1c074c80 0x7fde1c19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.769+0000 7fde1be00640 1 -- 172.21.15.39:0/2167824384 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c1a1af0 con 0x7fde1c0755d0 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.770+0000 7fde1be00640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0755d0 0x7fde1c19c160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e9c0 tx=0x7fde0400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.770+0000 7fde19600640 1 -- 172.21.15.39:0/2167824384 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0400cd80 con 0x7fde1c0755d0 2024-07-04T08:15:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.770+0000 7fde19600640 1 -- 172.21.15.39:0/2167824384 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde040054d0 con 0x7fde1c0755d0 2024-07-04T08:15:01.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.770+0000 7fde19600640 1 -- 172.21.15.39:0/2167824384 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04010430 con 0x7fde1c0755d0 2024-07-04T08:15:01.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.770+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c1a1de0 con 0x7fde1c0755d0 2024-07-04T08:15:01.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.770+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c1a8570 con 0x7fde1c0755d0 2024-07-04T08:15:01.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.771+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1c076190 con 0x7fde1c0755d0 2024-07-04T08:15:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.775+0000 7fde19600640 1 -- 172.21.15.39:0/2167824384 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7fde04004fd0 con 0x7fde1c0755d0 2024-07-04T08:15:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.776+0000 7fde19600640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fde000b2840 0x7fde000b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.777+0000 7fde19600640 1 -- 172.21.15.39:0/2167824384 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fde040e2a80 con 0x7fde1c0755d0 2024-07-04T08:15:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.777+0000 7fde20a00640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fde000b2840 0x7fde000b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.777+0000 7fde19600640 1 -- 172.21.15.39:0/2167824384 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fde040aa2f0 con 0x7fde1c0755d0 2024-07-04T08:15:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.778+0000 7fde20a00640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fde000b2840 0x7fde000b4d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009bd0 tx=0x7fde0c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:01.986+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fde1c109830 con 0x7fde000b2840 2024-07-04T08:15:02.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.004+0000 7fde19600640 1 -- 172.21.15.39:0/2167824384 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340808 (secure 0 0 0) 0x7fde1c109830 con 0x7fde000b2840 2024-07-04T08:15:02.006 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:02.013 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:15:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fde000b2840 msgr2=0x7fde000b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fde000b2840 0x7fde000b4d50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009bd0 tx=0x7fde0c005f90 comp rx=0 tx=0).stop 2024-07-04T08:15:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0755d0 msgr2=0x7fde1c19c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0755d0 0x7fde1c19c160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fde0400e9c0 tx=0x7fde0400ee80 comp rx=0 tx=0).stop 2024-07-04T08:15:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 shutdown_connections 2024-07-04T08:15:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fde000b2840 0x7fde000b4d50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fde1c10f5f0 0x7fde1c1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde1c0755d0 0x7fde1c19c160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 --2- 172.21.15.39:0/2167824384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde1c074c80 0x7fde1c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 >> 172.21.15.39:0/2167824384 conn(0x7fde1c0fea60 msgr2=0x7fde1c109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.012+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 shutdown_connections 2024-07-04T08:15:02.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.013+0000 7fde23400640 1 -- 172.21.15.39:0/2167824384 wait complete. 2024-07-04T08:15:02.043 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-07-04T08:15:01.684265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662779,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2453,"num_read_kb":1331,"num_write":387,"num_write_kb":2305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":819,"ondisk_log_size":819,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374188,"kb_used_data":12556,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567636,"statfs":{"total":1151940427776,"available":1151557259264,"internally_reserved":0,"allocated":12857344,"data_stored":8490461,"data_compressed":498873,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":306,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":56,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006143"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":295,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:58.426303+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:14:58.426303+0000","last_peered":"2024-07-04T08:14:58.426303+0000","last_clean":"2024-07-04T08:14:58.426303+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:14:58.426303+0000","last_undegraded":"2024-07-04T08:14:58.426303+0000","last_fullsized":"2024-07-04T08:14:58.426303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553793+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:12:51.553793+0000","last_peered":"2024-07-04T08:12:51.553793+0000","last_clean":"2024-07-04T08:12:51.553793+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:12:51.553793+0000","last_undegraded":"2024-07-04T08:12:51.553793+0000","last_fullsized":"2024-07-04T08:12:51.553793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:16.118017+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:12:16.118017+0000","last_peered":"2024-07-04T08:12:16.118017+0000","last_clean":"2024-07-04T08:12:16.118017+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:12:16.118017+0000","last_undegraded":"2024-07-04T08:12:16.118017+0000","last_fullsized":"2024-07-04T08:12:16.118017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.698593+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:53.698593+0000","last_peered":"2024-07-04T08:11:53.698593+0000","last_clean":"2024-07-04T08:11:53.698593+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:53.698593+0000","last_undegraded":"2024-07-04T08:11:53.698593+0000","last_fullsized":"2024-07-04T08:11:53.698593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569975+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:12:51.569975+0000","last_peered":"2024-07-04T08:12:51.569975+0000","last_clean":"2024-07-04T08:12:51.569975+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:12:51.569975+0000","last_undegraded":"2024-07-04T08:12:51.569975+0000","last_fullsized":"2024-07-04T08:12:51.569975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:15.756154+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:12:15.756154+0000","last_peered":"2024-07-04T08:12:15.756154+0000","last_clean":"2024-07-04T08:12:15.756154+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:12:15.756154+0000","last_undegraded":"2024-07-04T08:12:15.756154+0000","last_fullsized":"2024-07-04T08:12:15.756154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247127+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:12:05.247127+0000","last_peered":"2024-07-04T08:12:05.247127+0000","last_clean":"2024-07-04T08:12:05.247127+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:12:05.247127+0000","last_undegraded":"2024-07-04T08:12:05.247127+0000","last_fullsized":"2024-07-04T08:12:05.247127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.549821+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:12:51.549821+0000","last_peered":"2024-07-04T08:12:51.549821+0000","last_clean":"2024-07-04T08:12:51.549821+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:12:51.549821+0000","last_undegraded":"2024-07-04T08:12:51.549821+0000","last_fullsized":"2024-07-04T08:12:51.549821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756728+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:12:20.756728+0000","last_peered":"2024-07-04T08:12:20.756728+0000","last_clean":"2024-07-04T08:12:20.756728+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:12:20.756728+0000","last_undegraded":"2024-07-04T08:12:20.756728+0000","last_fullsized":"2024-07-04T08:12:20.756728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247079+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:12:05.247079+0000","last_peered":"2024-07-04T08:12:05.247079+0000","last_clean":"2024-07-04T08:12:05.247079+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:12:05.247079+0000","last_undegraded":"2024-07-04T08:12:05.247079+0000","last_fullsized":"2024-07-04T08:12:05.247079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.535944+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:12:51.535944+0000","last_peered":"2024-07-04T08:12:51.535944+0000","last_clean":"2024-07-04T08:12:51.535944+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:12:51.535944+0000","last_undegraded":"2024-07-04T08:12:51.535944+0000","last_fullsized":"2024-07-04T08:12:51.535944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756706+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:12:20.756706+0000","last_peered":"2024-07-04T08:12:20.756706+0000","last_clean":"2024-07-04T08:12:20.756706+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:12:20.756706+0000","last_undegraded":"2024-07-04T08:12:20.756706+0000","last_fullsized":"2024-07-04T08:12:20.756706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562540+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:12:51.562540+0000","last_peered":"2024-07-04T08:12:51.562540+0000","last_clean":"2024-07-04T08:12:51.562540+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:12:51.562540+0000","last_undegraded":"2024-07-04T08:12:51.562540+0000","last_fullsized":"2024-07-04T08:12:51.562540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844569+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:53.844569+0000","last_peered":"2024-07-04T08:11:53.844569+0000","last_clean":"2024-07-04T08:11:53.844569+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:53.844569+0000","last_undegraded":"2024-07-04T08:11:53.844569+0000","last_fullsized":"2024-07-04T08:11:53.844569+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754878+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:12:05.754878+0000","last_peered":"2024-07-04T08:12:05.754878+0000","last_clean":"2024-07-04T08:12:05.754878+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:12:05.754878+0000","last_undegraded":"2024-07-04T08:12:05.754878+0000","last_fullsized":"2024-07-04T08:12:05.754878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570735+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:12:51.570735+0000","last_peered":"2024-07-04T08:12:51.570735+0000","last_clean":"2024-07-04T08:12:51.570735+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:12:51.570735+0000","last_undegraded":"2024-07-04T08:12:51.570735+0000","last_fullsized":"2024-07-04T08:12:51.570735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691548+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:53.691548+0000","last_peered":"2024-07-04T08:11:53.691548+0000","last_clean":"2024-07-04T08:11:53.691548+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:53.691548+0000","last_undegraded":"2024-07-04T08:11:53.691548+0000","last_fullsized":"2024-07-04T08:11:53.691548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754825+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:12:05.754825+0000","last_peered":"2024-07-04T08:12:05.754825+0000","last_clean":"2024-07-04T08:12:05.754825+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:12:05.754825+0000","last_undegraded":"2024-07-04T08:12:05.754825+0000","last_fullsized":"2024-07-04T08:12:05.754825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566667+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:12:51.566667+0000","last_peered":"2024-07-04T08:12:51.566667+0000","last_clean":"2024-07-04T08:12:51.566667+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:12:51.566667+0000","last_undegraded":"2024-07-04T08:12:51.566667+0000","last_fullsized":"2024-07-04T08:12:51.566667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.697029+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:53.697029+0000","last_peered":"2024-07-04T08:11:53.697029+0000","last_clean":"2024-07-04T08:11:53.697029+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:53.697029+0000","last_undegraded":"2024-07-04T08:11:53.697029+0000","last_fullsized":"2024-07-04T08:11:53.697029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771248+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:11:53.771248+0000","last_peered":"2024-07-04T08:11:53.771248+0000","last_clean":"2024-07-04T08:11:53.771248+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:11:53.771248+0000","last_undegraded":"2024-07-04T08:11:53.771248+0000","last_fullsized":"2024-07-04T08:11:53.771248+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.526597+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:12:51.526597+0000","last_peered":"2024-07-04T08:12:51.526597+0000","last_clean":"2024-07-04T08:12:51.526597+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:12:51.526597+0000","last_undegraded":"2024-07-04T08:12:51.526597+0000","last_fullsized":"2024-07-04T08:12:51.526597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936427+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:53.936427+0000","last_peered":"2024-07-04T08:11:53.936427+0000","last_clean":"2024-07-04T08:11:53.936427+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:53.936427+0000","last_undegraded":"2024-07-04T08:11:53.936427+0000","last_fullsized":"2024-07-04T08:11:53.936427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754887+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:12:05.754887+0000","last_peered":"2024-07-04T08:12:05.754887+0000","last_clean":"2024-07-04T08:12:05.754887+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:12:05.754887+0000","last_undegraded":"2024-07-04T08:12:05.754887+0000","last_fullsized":"2024-07-04T08:12:05.754887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568004+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:12:51.568004+0000","last_peered":"2024-07-04T08:12:51.568004+0000","last_clean":"2024-07-04T08:12:51.568004+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:12:51.568004+0000","last_undegraded":"2024-07-04T08:12:51.568004+0000","last_fullsized":"2024-07-04T08:12:51.568004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692160+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:53.692160+0000","last_peered":"2024-07-04T08:11:53.692160+0000","last_clean":"2024-07-04T08:11:53.692160+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:53.692160+0000","last_undegraded":"2024-07-04T08:11:53.692160+0000","last_fullsized":"2024-07-04T08:11:53.692160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115754+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:11:56.115754+0000","last_peered":"2024-07-04T08:11:56.115754+0000","last_clean":"2024-07-04T08:11:56.115754+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:11:56.115754+0000","last_undegraded":"2024-07-04T08:11:56.115754+0000","last_fullsized":"2024-07-04T08:11:56.115754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.556576+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:12:51.556576+0000","last_peered":"2024-07-04T08:12:51.556576+0000","last_clean":"2024-07-04T08:12:51.556576+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:12:51.556576+0000","last_undegraded":"2024-07-04T08:12:51.556576+0000","last_fullsized":"2024-07-04T08:12:51.556576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'17","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:54.818022+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:13:54.818022+0000","last_peered":"2024-07-04T08:13:54.818022+0000","last_clean":"2024-07-04T08:13:54.818022+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:13:54.818022+0000","last_undegraded":"2024-07-04T08:13:54.818022+0000","last_fullsized":"2024-07-04T08:13:54.818022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.322378+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:11:54.322378+0000","last_peered":"2024-07-04T08:11:54.322378+0000","last_clean":"2024-07-04T08:11:54.322378+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:11:54.322378+0000","last_undegraded":"2024-07-04T08:11:54.322378+0000","last_fullsized":"2024-07-04T08:11:54.322378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540856+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:12:51.540856+0000","last_peered":"2024-07-04T08:12:51.540856+0000","last_clean":"2024-07-04T08:12:51.540856+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:12:51.540856+0000","last_undegraded":"2024-07-04T08:12:51.540856+0000","last_fullsized":"2024-07-04T08:12:51.540856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771431+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:53.771431+0000","last_peered":"2024-07-04T08:11:53.771431+0000","last_clean":"2024-07-04T08:11:53.771431+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:53.771431+0000","last_undegraded":"2024-07-04T08:11:53.771431+0000","last_fullsized":"2024-07-04T08:11:53.771431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.238979+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:11:56.238979+0000","last_peered":"2024-07-04T08:11:56.238979+0000","last_clean":"2024-07-04T08:11:56.238979+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:11:56.238979+0000","last_undegraded":"2024-07-04T08:11:56.238979+0000","last_fullsized":"2024-07-04T08:11:56.238979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542562+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:12:51.542562+0000","last_peered":"2024-07-04T08:12:51.542562+0000","last_clean":"2024-07-04T08:12:51.542562+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:12:51.542562+0000","last_undegraded":"2024-07-04T08:12:51.542562+0000","last_fullsized":"2024-07-04T08:12:51.542562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993415+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:11:53.993415+0000","last_peered":"2024-07-04T08:11:53.993415+0000","last_clean":"2024-07-04T08:11:53.993415+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:11:53.993415+0000","last_undegraded":"2024-07-04T08:11:53.993415+0000","last_fullsized":"2024-07-04T08:11:53.993415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:00.246372+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:12:00.246372+0000","last_peered":"2024-07-04T08:12:00.246372+0000","last_clean":"2024-07-04T08:12:00.246372+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:12:00.246372+0000","last_undegraded":"2024-07-04T08:12:00.246372+0000","last_fullsized":"2024-07-04T08:12:00.246372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.543737+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:12:51.543737+0000","last_peered":"2024-07-04T08:12:51.543737+0000","last_clean":"2024-07-04T08:12:51.543737+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:12:51.543737+0000","last_undegraded":"2024-07-04T08:12:51.543737+0000","last_fullsized":"2024-07-04T08:12:51.543737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":291,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:59.429621+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:14:59.429621+0000","last_peered":"2024-07-04T08:14:59.429621+0000","last_clean":"2024-07-04T08:14:59.429621+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:14:59.429621+0000","last_undegraded":"2024-07-04T08:14:59.429621+0000","last_fullsized":"2024-07-04T08:14:59.429621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":238,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:58.137211+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:11:58.137211+0000","last_peered":"2024-07-04T08:11:58.137211+0000","last_clean":"2024-07-04T08:11:58.137211+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:11:58.137211+0000","last_undegraded":"2024-07-04T08:11:58.137211+0000","last_fullsized":"2024-07-04T08:11:58.137211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.516953+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:12:51.516953+0000","last_peered":"2024-07-04T08:12:51.516953+0000","last_clean":"2024-07-04T08:12:51.516953+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:12:51.516953+0000","last_undegraded":"2024-07-04T08:12:51.516953+0000","last_fullsized":"2024-07-04T08:12:51.516953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":285,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:56.479235+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:14:56.479235+0000","last_peered":"2024-07-04T08:14:56.479235+0000","last_clean":"2024-07-04T08:14:56.479235+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:14:56.479235+0000","last_undegraded":"2024-07-04T08:14:56.479235+0000","last_fullsized":"2024-07-04T08:14:56.479235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":233,"num_read_kb":233,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138305+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:12:03.138305+0000","last_peered":"2024-07-04T08:12:03.138305+0000","last_clean":"2024-07-04T08:12:03.138305+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:12:03.138305+0000","last_undegraded":"2024-07-04T08:12:03.138305+0000","last_fullsized":"2024-07-04T08:12:03.138305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534873+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:12:51.534873+0000","last_peered":"2024-07-04T08:12:51.534873+0000","last_clean":"2024-07-04T08:12:51.534873+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:12:51.534873+0000","last_undegraded":"2024-07-04T08:12:51.534873+0000","last_fullsized":"2024-07-04T08:12:51.534873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851437+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:53.851437+0000","last_peered":"2024-07-04T08:11:53.851437+0000","last_clean":"2024-07-04T08:11:53.851437+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:53.851437+0000","last_undegraded":"2024-07-04T08:11:53.851437+0000","last_fullsized":"2024-07-04T08:11:53.851437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975520+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:12:04.975520+0000","last_peered":"2024-07-04T08:12:04.975520+0000","last_clean":"2024-07-04T08:12:04.975520+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:12:04.975520+0000","last_undegraded":"2024-07-04T08:12:04.975520+0000","last_fullsized":"2024-07-04T08:12:04.975520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.558356+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:12:51.558356+0000","last_peered":"2024-07-04T08:12:51.558356+0000","last_clean":"2024-07-04T08:12:51.558356+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:12:51.558356+0000","last_undegraded":"2024-07-04T08:12:51.558356+0000","last_fullsized":"2024-07-04T08:12:51.558356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691883+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:53.691883+0000","last_peered":"2024-07-04T08:11:53.691883+0000","last_clean":"2024-07-04T08:11:53.691883+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:53.691883+0000","last_undegraded":"2024-07-04T08:11:53.691883+0000","last_fullsized":"2024-07-04T08:11:53.691883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":120,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:46.091830+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:12:46.091830+0000","last_peered":"2024-07-04T08:12:46.091830+0000","last_clean":"2024-07-04T08:12:46.091830+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:12:46.091830+0000","last_undegraded":"2024-07-04T08:12:46.091830+0000","last_fullsized":"2024-07-04T08:12:46.091830+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.085858+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:11:56.085858+0000","last_peered":"2024-07-04T08:11:56.085858+0000","last_clean":"2024-07-04T08:11:56.085858+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:11:56.085858+0000","last_undegraded":"2024-07-04T08:11:56.085858+0000","last_fullsized":"2024-07-04T08:11:56.085858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553863+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:12:51.553863+0000","last_peered":"2024-07-04T08:12:51.553863+0000","last_clean":"2024-07-04T08:12:51.553863+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:12:51.553863+0000","last_undegraded":"2024-07-04T08:12:51.553863+0000","last_fullsized":"2024-07-04T08:12:51.553863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":256,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:57.190728+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:14:57.190728+0000","last_peered":"2024-07-04T08:14:57.190728+0000","last_clean":"2024-07-04T08:14:57.190728+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:14:57.190728+0000","last_undegraded":"2024-07-04T08:14:57.190728+0000","last_fullsized":"2024-07-04T08:14:57.190728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.151807+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:12:04.151807+0000","last_peered":"2024-07-04T08:12:04.151807+0000","last_clean":"2024-07-04T08:12:04.151807+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:12:04.151807+0000","last_undegraded":"2024-07-04T08:12:04.151807+0000","last_fullsized":"2024-07-04T08:12:04.151807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.561500+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:12:51.561500+0000","last_peered":"2024-07-04T08:12:51.561500+0000","last_clean":"2024-07-04T08:12:51.561500+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:12:51.561500+0000","last_undegraded":"2024-07-04T08:12:51.561500+0000","last_fullsized":"2024-07-04T08:12:51.561500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939931+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:53.939931+0000","last_peered":"2024-07-04T08:11:53.939931+0000","last_clean":"2024-07-04T08:11:53.939931+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:11:53.939931+0000","last_undegraded":"2024-07-04T08:11:53.939931+0000","last_fullsized":"2024-07-04T08:11:53.939931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:09.641951+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:12:09.641951+0000","last_peered":"2024-07-04T08:12:09.641951+0000","last_clean":"2024-07-04T08:12:09.641951+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:12:09.641951+0000","last_undegraded":"2024-07-04T08:12:09.641951+0000","last_fullsized":"2024-07-04T08:12:09.641951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552528+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:12:51.552528+0000","last_peered":"2024-07-04T08:12:51.552528+0000","last_clean":"2024-07-04T08:12:51.552528+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:12:51.552528+0000","last_undegraded":"2024-07-04T08:12:51.552528+0000","last_fullsized":"2024-07-04T08:12:51.552528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995043+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:53.995043+0000","last_peered":"2024-07-04T08:11:53.995043+0000","last_clean":"2024-07-04T08:11:53.995043+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:53.995043+0000","last_undegraded":"2024-07-04T08:11:53.995043+0000","last_fullsized":"2024-07-04T08:11:53.995043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975432+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:12:04.975432+0000","last_peered":"2024-07-04T08:12:04.975432+0000","last_clean":"2024-07-04T08:12:04.975432+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:12:04.975432+0000","last_undegraded":"2024-07-04T08:12:04.975432+0000","last_fullsized":"2024-07-04T08:12:04.975432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531965+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:12:51.531965+0000","last_peered":"2024-07-04T08:12:51.531965+0000","last_clean":"2024-07-04T08:12:51.531965+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:12:51.531965+0000","last_undegraded":"2024-07-04T08:12:51.531965+0000","last_fullsized":"2024-07-04T08:12:51.531965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771354+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:53.771354+0000","last_peered":"2024-07-04T08:11:53.771354+0000","last_clean":"2024-07-04T08:11:53.771354+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:53.771354+0000","last_undegraded":"2024-07-04T08:11:53.771354+0000","last_fullsized":"2024-07-04T08:11:53.771354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975473+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:12:04.975473+0000","last_peered":"2024-07-04T08:12:04.975473+0000","last_clean":"2024-07-04T08:12:04.975473+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:12:04.975473+0000","last_undegraded":"2024-07-04T08:12:04.975473+0000","last_fullsized":"2024-07-04T08:12:04.975473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542769+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:12:51.542769+0000","last_peered":"2024-07-04T08:12:51.542769+0000","last_clean":"2024-07-04T08:12:51.542769+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:12:51.542769+0000","last_undegraded":"2024-07-04T08:12:51.542769+0000","last_fullsized":"2024-07-04T08:12:51.542769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:55.245889+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:55.245889+0000","last_peered":"2024-07-04T08:11:55.245889+0000","last_clean":"2024-07-04T08:11:55.245889+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:55.245889+0000","last_undegraded":"2024-07-04T08:11:55.245889+0000","last_fullsized":"2024-07-04T08:11:55.245889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.141070+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:12:18.141070+0000","last_peered":"2024-07-04T08:12:18.141070+0000","last_clean":"2024-07-04T08:12:18.141070+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:12:18.141070+0000","last_undegraded":"2024-07-04T08:12:18.141070+0000","last_fullsized":"2024-07-04T08:12:18.141070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552028+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:12:51.552028+0000","last_peered":"2024-07-04T08:12:51.552028+0000","last_clean":"2024-07-04T08:12:51.552028+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:12:51.552028+0000","last_undegraded":"2024-07-04T08:12:51.552028+0000","last_fullsized":"2024-07-04T08:12:51.552028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":292,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:59.487825+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:14:59.487825+0000","last_peered":"2024-07-04T08:14:59.487825+0000","last_clean":"2024-07-04T08:14:59.487825+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:14:59.487825+0000","last_undegraded":"2024-07-04T08:14:59.487825+0000","last_fullsized":"2024-07-04T08:14:59.487825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.666330+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:12:18.666330+0000","last_peered":"2024-07-04T08:12:18.666330+0000","last_clean":"2024-07-04T08:12:18.666330+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:12:18.666330+0000","last_undegraded":"2024-07-04T08:12:18.666330+0000","last_fullsized":"2024-07-04T08:12:18.666330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564982+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:12:51.564982+0000","last_peered":"2024-07-04T08:12:51.564982+0000","last_clean":"2024-07-04T08:12:51.564982+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:12:51.564982+0000","last_undegraded":"2024-07-04T08:12:51.564982+0000","last_fullsized":"2024-07-04T08:12:51.564982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.974209+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:54.974209+0000","last_peered":"2024-07-04T08:11:54.974209+0000","last_clean":"2024-07-04T08:11:54.974209+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:54.974209+0000","last_undegraded":"2024-07-04T08:11:54.974209+0000","last_fullsized":"2024-07-04T08:11:54.974209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:11.240822+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:12:11.240822+0000","last_peered":"2024-07-04T08:12:11.240822+0000","last_clean":"2024-07-04T08:12:11.240822+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:12:11.240822+0000","last_undegraded":"2024-07-04T08:12:11.240822+0000","last_fullsized":"2024-07-04T08:12:11.240822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562193+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:12:51.562193+0000","last_peered":"2024-07-04T08:12:51.562193+0000","last_clean":"2024-07-04T08:12:51.562193+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:12:51.562193+0000","last_undegraded":"2024-07-04T08:12:51.562193+0000","last_fullsized":"2024-07-04T08:12:51.562193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115841+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:56.115841+0000","last_peered":"2024-07-04T08:11:56.115841+0000","last_clean":"2024-07-04T08:11:56.115841+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:56.115841+0000","last_undegraded":"2024-07-04T08:11:56.115841+0000","last_fullsized":"2024-07-04T08:11:56.115841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.904331+0000","last_change":"2024-07-04T08:11:53.997034+0000","last_active":"2024-07-04T08:11:56.904331+0000","last_peered":"2024-07-04T08:11:56.904331+0000","last_clean":"2024-07-04T08:11:56.904331+0000","last_became_active":"2024-07-04T08:11:53.996456+0000","last_became_peered":"2024-07-04T08:11:53.996456+0000","last_unstale":"2024-07-04T08:11:56.904331+0000","last_undegraded":"2024-07-04T08:11:56.904331+0000","last_fullsized":"2024-07-04T08:11:56.904331+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.140549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.521874+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:12:51.521874+0000","last_peered":"2024-07-04T08:12:51.521874+0000","last_clean":"2024-07-04T08:12:51.521874+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:12:51.521874+0000","last_undegraded":"2024-07-04T08:12:51.521874+0000","last_fullsized":"2024-07-04T08:12:51.521874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":559,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:56.261606+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:14:56.261606+0000","last_peered":"2024-07-04T08:14:56.261606+0000","last_clean":"2024-07-04T08:14:56.261606+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:14:56.261606+0000","last_undegraded":"2024-07-04T08:14:56.261606+0000","last_fullsized":"2024-07-04T08:14:56.261606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64691,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":473,"num_read_kb":300,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800726+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:12:14.800726+0000","last_peered":"2024-07-04T08:12:14.800726+0000","last_clean":"2024-07-04T08:12:14.800726+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:12:14.800726+0000","last_undegraded":"2024-07-04T08:12:14.800726+0000","last_fullsized":"2024-07-04T08:12:14.800726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541913+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:12:51.541913+0000","last_peered":"2024-07-04T08:12:51.541913+0000","last_clean":"2024-07-04T08:12:51.541913+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:12:51.541913+0000","last_undegraded":"2024-07-04T08:12:51.541913+0000","last_fullsized":"2024-07-04T08:12:51.541913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974814+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:59.974814+0000","last_peered":"2024-07-04T08:11:59.974814+0000","last_clean":"2024-07-04T08:11:59.974814+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:59.974814+0000","last_undegraded":"2024-07-04T08:11:59.974814+0000","last_fullsized":"2024-07-04T08:11:59.974814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:10.755408+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:12:10.755408+0000","last_peered":"2024-07-04T08:12:10.755408+0000","last_clean":"2024-07-04T08:12:10.755408+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:12:10.755408+0000","last_undegraded":"2024-07-04T08:12:10.755408+0000","last_fullsized":"2024-07-04T08:12:10.755408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569907+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:12:51.569907+0000","last_peered":"2024-07-04T08:12:51.569907+0000","last_clean":"2024-07-04T08:12:51.569907+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:12:51.569907+0000","last_undegraded":"2024-07-04T08:12:51.569907+0000","last_fullsized":"2024-07-04T08:12:51.569907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:01.086478+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:12:01.086478+0000","last_peered":"2024-07-04T08:12:01.086478+0000","last_clean":"2024-07-04T08:12:01.086478+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:12:01.086478+0000","last_undegraded":"2024-07-04T08:12:01.086478+0000","last_fullsized":"2024-07-04T08:12:01.086478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800668+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:12:14.800668+0000","last_peered":"2024-07-04T08:12:14.800668+0000","last_clean":"2024-07-04T08:12:14.800668+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:12:14.800668+0000","last_undegraded":"2024-07-04T08:12:14.800668+0000","last_fullsized":"2024-07-04T08:12:14.800668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.547189+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:12:51.547189+0000","last_peered":"2024-07-04T08:12:51.547189+0000","last_clean":"2024-07-04T08:12:51.547189+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:12:51.547189+0000","last_undegraded":"2024-07-04T08:12:51.547189+0000","last_fullsized":"2024-07-04T08:12:51.547189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":512,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:56.188725+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:14:56.188725+0000","last_peered":"2024-07-04T08:14:56.188725+0000","last_clean":"2024-07-04T08:14:56.188725+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:14:56.188725+0000","last_undegraded":"2024-07-04T08:14:56.188725+0000","last_fullsized":"2024-07-04T08:14:56.188725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":462,"num_read_kb":462,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.326445+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:12:14.326445+0000","last_peered":"2024-07-04T08:12:14.326445+0000","last_clean":"2024-07-04T08:12:14.326445+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:12:14.326445+0000","last_undegraded":"2024-07-04T08:12:14.326445+0000","last_fullsized":"2024-07-04T08:12:14.326445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.639390+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:11:54.639390+0000","last_peered":"2024-07-04T08:11:54.639390+0000","last_clean":"2024-07-04T08:11:54.639390+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:11:54.639390+0000","last_undegraded":"2024-07-04T08:11:54.639390+0000","last_fullsized":"2024-07-04T08:11:54.639390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564562+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:12:51.564562+0000","last_peered":"2024-07-04T08:12:51.564562+0000","last_clean":"2024-07-04T08:12:51.564562+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:12:51.564562+0000","last_undegraded":"2024-07-04T08:12:51.564562+0000","last_fullsized":"2024-07-04T08:12:51.564562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974768+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:59.974768+0000","last_peered":"2024-07-04T08:11:59.974768+0000","last_clean":"2024-07-04T08:11:59.974768+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:59.974768+0000","last_undegraded":"2024-07-04T08:11:59.974768+0000","last_fullsized":"2024-07-04T08:11:59.974768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.239030+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:11:56.239030+0000","last_peered":"2024-07-04T08:11:56.239030+0000","last_clean":"2024-07-04T08:11:56.239030+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:11:56.239030+0000","last_undegraded":"2024-07-04T08:11:56.239030+0000","last_fullsized":"2024-07-04T08:11:56.239030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565645+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:12:51.565645+0000","last_peered":"2024-07-04T08:12:51.565645+0000","last_clean":"2024-07-04T08:12:51.565645+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:12:51.565645+0000","last_undegraded":"2024-07-04T08:12:51.565645+0000","last_fullsized":"2024-07-04T08:12:51.565645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138191+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:12:03.138191+0000","last_peered":"2024-07-04T08:12:03.138191+0000","last_clean":"2024-07-04T08:12:03.138191+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:12:03.138191+0000","last_undegraded":"2024-07-04T08:12:03.138191+0000","last_fullsized":"2024-07-04T08:12:03.138191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:06.905597+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:12:06.905597+0000","last_peered":"2024-07-04T08:12:06.905597+0000","last_clean":"2024-07-04T08:12:06.905597+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:12:06.905597+0000","last_undegraded":"2024-07-04T08:12:06.905597+0000","last_fullsized":"2024-07-04T08:12:06.905597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.554669+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:12:51.554669+0000","last_peered":"2024-07-04T08:12:51.554669+0000","last_clean":"2024-07-04T08:12:51.554669+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:12:51.554669+0000","last_undegraded":"2024-07-04T08:12:51.554669+0000","last_fullsized":"2024-07-04T08:12:51.554669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118570+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:12:21.118570+0000","last_peered":"2024-07-04T08:12:21.118570+0000","last_clean":"2024-07-04T08:12:21.118570+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:12:21.118570+0000","last_undegraded":"2024-07-04T08:12:21.118570+0000","last_fullsized":"2024-07-04T08:12:21.118570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:58.234257+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:14:58.234257+0000","last_peered":"2024-07-04T08:14:58.234257+0000","last_clean":"2024-07-04T08:14:58.234257+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:14:58.234257+0000","last_undegraded":"2024-07-04T08:14:58.234257+0000","last_fullsized":"2024-07-04T08:14:58.234257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.519199+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:12:51.519199+0000","last_peered":"2024-07-04T08:12:51.519199+0000","last_clean":"2024-07-04T08:12:51.519199+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:12:51.519199+0000","last_undegraded":"2024-07-04T08:12:51.519199+0000","last_fullsized":"2024-07-04T08:12:51.519199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118593+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:12:21.118593+0000","last_peered":"2024-07-04T08:12:21.118593+0000","last_clean":"2024-07-04T08:12:21.118593+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:12:21.118593+0000","last_undegraded":"2024-07-04T08:12:21.118593+0000","last_fullsized":"2024-07-04T08:12:21.118593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534235+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:12:51.534235+0000","last_peered":"2024-07-04T08:12:51.534235+0000","last_clean":"2024-07-04T08:12:51.534235+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:12:51.534235+0000","last_undegraded":"2024-07-04T08:12:51.534235+0000","last_fullsized":"2024-07-04T08:12:51.534235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.539681+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:12:51.539681+0000","last_peered":"2024-07-04T08:12:51.539681+0000","last_clean":"2024-07-04T08:12:51.539681+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:12:51.539681+0000","last_undegraded":"2024-07-04T08:12:51.539681+0000","last_fullsized":"2024-07-04T08:12:51.539681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566399+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:12:51.566399+0000","last_peered":"2024-07-04T08:12:51.566399+0000","last_clean":"2024-07-04T08:12:51.566399+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:12:51.566399+0000","last_undegraded":"2024-07-04T08:12:51.566399+0000","last_fullsized":"2024-07-04T08:12:51.566399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567851+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:12:51.567851+0000","last_peered":"2024-07-04T08:12:51.567851+0000","last_clean":"2024-07-04T08:12:51.567851+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:12:51.567851+0000","last_undegraded":"2024-07-04T08:12:51.567851+0000","last_fullsized":"2024-07-04T08:12:51.567851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569222+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:12:51.569222+0000","last_peered":"2024-07-04T08:12:51.569222+0000","last_clean":"2024-07-04T08:12:51.569222+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:12:51.569222+0000","last_undegraded":"2024-07-04T08:12:51.569222+0000","last_fullsized":"2024-07-04T08:12:51.569222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532408+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:12:51.532408+0000","last_peered":"2024-07-04T08:12:51.532408+0000","last_clean":"2024-07-04T08:12:51.532408+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:12:51.532408+0000","last_undegraded":"2024-07-04T08:12:51.532408+0000","last_fullsized":"2024-07-04T08:12:51.532408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540538+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:12:51.540538+0000","last_peered":"2024-07-04T08:12:51.540538+0000","last_clean":"2024-07-04T08:12:51.540538+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:12:51.540538+0000","last_undegraded":"2024-07-04T08:12:51.540538+0000","last_fullsized":"2024-07-04T08:12:51.540538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552348+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:12:51.552348+0000","last_peered":"2024-07-04T08:12:51.552348+0000","last_clean":"2024-07-04T08:12:51.552348+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:12:51.552348+0000","last_undegraded":"2024-07-04T08:12:51.552348+0000","last_fullsized":"2024-07-04T08:12:51.552348+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531503+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:12:51.531503+0000","last_peered":"2024-07-04T08:12:51.531503+0000","last_clean":"2024-07-04T08:12:51.531503+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:12:51.531503+0000","last_undegraded":"2024-07-04T08:12:51.531503+0000","last_fullsized":"2024-07-04T08:12:51.531503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.555574+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:12:51.555574+0000","last_peered":"2024-07-04T08:12:51.555574+0000","last_clean":"2024-07-04T08:12:51.555574+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:12:51.555574+0000","last_undegraded":"2024-07-04T08:12:51.555574+0000","last_fullsized":"2024-07-04T08:12:51.555574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560802+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:12:51.560802+0000","last_peered":"2024-07-04T08:12:51.560802+0000","last_clean":"2024-07-04T08:12:51.560802+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:12:51.560802+0000","last_undegraded":"2024-07-04T08:12:51.560802+0000","last_fullsized":"2024-07-04T08:12:51.560802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570948+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:12:51.570948+0000","last_peered":"2024-07-04T08:12:51.570948+0000","last_clean":"2024-07-04T08:12:51.570948+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:12:51.570948+0000","last_undegraded":"2024-07-04T08:12:51.570948+0000","last_fullsized":"2024-07-04T08:12:51.570948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532752+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:12:51.532752+0000","last_peered":"2024-07-04T08:12:51.532752+0000","last_clean":"2024-07-04T08:12:51.532752+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:12:51.532752+0000","last_undegraded":"2024-07-04T08:12:51.532752+0000","last_fullsized":"2024-07-04T08:12:51.532752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552705+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:12:51.552705+0000","last_peered":"2024-07-04T08:12:51.552705+0000","last_clean":"2024-07-04T08:12:51.552705+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:12:51.552705+0000","last_undegraded":"2024-07-04T08:12:51.552705+0000","last_fullsized":"2024-07-04T08:12:51.552705+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541640+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:12:51.541640+0000","last_peered":"2024-07-04T08:12:51.541640+0000","last_clean":"2024-07-04T08:12:51.541640+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:12:51.541640+0000","last_undegraded":"2024-07-04T08:12:51.541640+0000","last_fullsized":"2024-07-04T08:12:51.541640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569708+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:12:51.569708+0000","last_peered":"2024-07-04T08:12:51.569708+0000","last_clean":"2024-07-04T08:12:51.569708+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:12:51.569708+0000","last_undegraded":"2024-07-04T08:12:51.569708+0000","last_fullsized":"2024-07-04T08:12:51.569708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.571140+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:12:51.571140+0000","last_peered":"2024-07-04T08:12:51.571140+0000","last_clean":"2024-07-04T08:12:51.571140+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:12:51.571140+0000","last_undegraded":"2024-07-04T08:12:51.571140+0000","last_fullsized":"2024-07-04T08:12:51.571140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552227+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:12:51.552227+0000","last_peered":"2024-07-04T08:12:51.552227+0000","last_clean":"2024-07-04T08:12:51.552227+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:12:51.552227+0000","last_undegraded":"2024-07-04T08:12:51.552227+0000","last_fullsized":"2024-07-04T08:12:51.552227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567284+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:12:51.567284+0000","last_peered":"2024-07-04T08:12:51.567284+0000","last_clean":"2024-07-04T08:12:51.567284+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:12:51.567284+0000","last_undegraded":"2024-07-04T08:12:51.567284+0000","last_fullsized":"2024-07-04T08:12:51.567284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551582+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:12:51.551582+0000","last_peered":"2024-07-04T08:12:51.551582+0000","last_clean":"2024-07-04T08:12:51.551582+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:12:51.551582+0000","last_undegraded":"2024-07-04T08:12:51.551582+0000","last_fullsized":"2024-07-04T08:12:51.551582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534085+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:12:51.534085+0000","last_peered":"2024-07-04T08:12:51.534085+0000","last_clean":"2024-07-04T08:12:51.534085+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:12:51.534085+0000","last_undegraded":"2024-07-04T08:12:51.534085+0000","last_fullsized":"2024-07-04T08:12:51.534085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553666+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:12:51.553666+0000","last_peered":"2024-07-04T08:12:51.553666+0000","last_clean":"2024-07-04T08:12:51.553666+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:12:51.553666+0000","last_undegraded":"2024-07-04T08:12:51.553666+0000","last_fullsized":"2024-07-04T08:12:51.553666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540373+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:12:51.540373+0000","last_peered":"2024-07-04T08:12:51.540373+0000","last_clean":"2024-07-04T08:12:51.540373+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:12:51.540373+0000","last_undegraded":"2024-07-04T08:12:51.540373+0000","last_fullsized":"2024-07-04T08:12:51.540373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541173+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:12:51.541173+0000","last_peered":"2024-07-04T08:12:51.541173+0000","last_clean":"2024-07-04T08:12:51.541173+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:12:51.541173+0000","last_undegraded":"2024-07-04T08:12:51.541173+0000","last_fullsized":"2024-07-04T08:12:51.541173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560317+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:12:51.560317+0000","last_peered":"2024-07-04T08:12:51.560317+0000","last_clean":"2024-07-04T08:12:51.560317+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:12:51.560317+0000","last_undegraded":"2024-07-04T08:12:51.560317+0000","last_fullsized":"2024-07-04T08:12:51.560317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551363+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:12:51.551363+0000","last_peered":"2024-07-04T08:12:51.551363+0000","last_clean":"2024-07-04T08:12:51.551363+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:12:51.551363+0000","last_undegraded":"2024-07-04T08:12:51.551363+0000","last_fullsized":"2024-07-04T08:12:51.551363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541562+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:12:51.541562+0000","last_peered":"2024-07-04T08:12:51.541562+0000","last_clean":"2024-07-04T08:12:51.541562+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:12:51.541562+0000","last_undegraded":"2024-07-04T08:12:51.541562+0000","last_fullsized":"2024-07-04T08:12:51.541562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.536909+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:12:51.536909+0000","last_peered":"2024-07-04T08:12:51.536909+0000","last_clean":"2024-07-04T08:12:51.536909+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:12:51.536909+0000","last_undegraded":"2024-07-04T08:12:51.536909+0000","last_fullsized":"2024-07-04T08:12:51.536909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537941+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:12:51.537941+0000","last_peered":"2024-07-04T08:12:51.537941+0000","last_clean":"2024-07-04T08:12:51.537941+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:12:51.537941+0000","last_undegraded":"2024-07-04T08:12:51.537941+0000","last_fullsized":"2024-07-04T08:12:51.537941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565574+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:12:51.565574+0000","last_peered":"2024-07-04T08:12:51.565574+0000","last_clean":"2024-07-04T08:12:51.565574+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:12:51.565574+0000","last_undegraded":"2024-07-04T08:12:51.565574+0000","last_fullsized":"2024-07-04T08:12:51.565574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568095+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:12:51.568095+0000","last_peered":"2024-07-04T08:12:51.568095+0000","last_clean":"2024-07-04T08:12:51.568095+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:12:51.568095+0000","last_undegraded":"2024-07-04T08:12:51.568095+0000","last_fullsized":"2024-07-04T08:12:51.568095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569523+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:12:51.569523+0000","last_peered":"2024-07-04T08:12:51.569523+0000","last_clean":"2024-07-04T08:12:51.569523+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:12:51.569523+0000","last_undegraded":"2024-07-04T08:12:51.569523+0000","last_fullsized":"2024-07-04T08:12:51.569523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72177,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2372,"num_read_kb":1262,"num_write":262,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":227277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612690,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":543564,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1195603,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":547590,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168674,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":543872,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364904,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1141042,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528428,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":539530,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724658,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1135560,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920887,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":543604,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117116,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":538706,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378756,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":609142,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542280,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":611794,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705805,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":540454,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:15:02.048 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-07-04T08:15:02.048 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-04T08:15:02.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.377+0000 7f8ef8000640 1 -- 172.21.15.39:0/84036922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef010b4a0 msgr2=0x7f8ef0111cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:02.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.377+0000 7f8ef8000640 1 --2- 172.21.15.39:0/84036922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef010b4a0 0x7f8ef0111cd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009ca0 tx=0x7f8ee802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.377+0000 7f8ef8000640 1 -- 172.21.15.39:0/84036922 shutdown_connections 2024-07-04T08:15:02.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.377+0000 7f8ef8000640 1 --2- 172.21.15.39:0/84036922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef010b4a0 0x7f8ef0111cd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.377+0000 7f8ef8000640 1 --2- 172.21.15.39:0/84036922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef010ab50 0x7f8ef010af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.377+0000 7f8ef8000640 1 --2- 172.21.15.39:0/84036922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef0105220 0x7f8ef0105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.377+0000 7f8ef8000640 1 -- 172.21.15.39:0/84036922 >> 172.21.15.39:0/84036922 conn(0x7f8ef0100c60 msgr2=0x7f8ef01030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:02.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.377+0000 7f8ef8000640 1 -- 172.21.15.39:0/84036922 shutdown_connections 2024-07-04T08:15:02.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.377+0000 7f8ef8000640 1 -- 172.21.15.39:0/84036922 wait complete. 2024-07-04T08:15:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.378+0000 7f8ef8000640 1 Processor -- start 2024-07-04T08:15:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.378+0000 7f8ef8000640 1 -- start start 2024-07-04T08:15:02.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef8000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef0105220 0x7f8ef019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef8000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef010ab50 0x7f8ef019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef8000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef010b4a0 0x7f8ef01a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef8000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0115170 con 0x7f8ef010b4a0 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef8000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0114ff0 con 0x7f8ef0105220 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef8000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01152f0 con 0x7f8ef010ab50 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef0105220 0x7f8ef019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef0105220 0x7f8ef019bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51038/0 (socket says 172.21.15.39:51038) 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef5600640 1 -- 172.21.15.39:0/2175139450 learned_addr learned my addr 172.21.15.39:0/2175139450 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef6000640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef010b4a0 0x7f8ef01a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef4c00640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef010ab50 0x7f8ef019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:02.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef5600640 1 -- 172.21.15.39:0/2175139450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef010ab50 msgr2=0x7f8ef019c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef5600640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef010ab50 0x7f8ef019c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef5600640 1 -- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef010b4a0 msgr2=0x7f8ef01a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.379+0000 7f8ef5600640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef010b4a0 0x7f8ef01a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.380+0000 7f8ef5600640 1 -- 172.21.15.39:0/2175139450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef01a1ac0 con 0x7f8ef0105220 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.380+0000 7f8ef5600640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef0105220 0x7f8ef019bb90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400e9f0 tx=0x7f8ee400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.380+0000 7f8eee000640 1 -- 172.21.15.39:0/2175139450 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400cd80 con 0x7f8ef0105220 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.380+0000 7f8eee000640 1 -- 172.21.15.39:0/2175139450 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee40054d0 con 0x7f8ef0105220 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.380+0000 7f8eee000640 1 -- 172.21.15.39:0/2175139450 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4010690 con 0x7f8ef0105220 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.380+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef01a1db0 con 0x7f8ef0105220 2024-07-04T08:15:02.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.380+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef01a8540 con 0x7f8ef0105220 2024-07-04T08:15:02.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.381+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef0105630 con 0x7f8ef0105220 2024-07-04T08:15:02.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.384+0000 7f8eee000640 1 -- 172.21.15.39:0/2175139450 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7f8ee4004e30 con 0x7f8ef0105220 2024-07-04T08:15:02.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.386+0000 7f8eee000640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ed80b2840 0x7f8ed80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:02.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.386+0000 7f8eee000640 1 -- 172.21.15.39:0/2175139450 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8ee40e2d30 con 0x7f8ef0105220 2024-07-04T08:15:02.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.386+0000 7f8ef4c00640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ed80b2840 0x7f8ed80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:02.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.387+0000 7f8ef4c00640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ed80b2840 0x7f8ed80b4d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0095f0 tx=0x7f8edc005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:02.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.387+0000 7f8eee000640 1 -- 172.21.15.39:0/2175139450 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f8ee4055110 con 0x7f8ef0105220 2024-07-04T08:15:02.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.600+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ef0062d50 con 0x7f8ed80b2840 2024-07-04T08:15:02.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.610+0000 7f8eee000640 1 -- 172.21.15.39:0/2175139450 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340808 (secure 0 0 0) 0x7f8ef0062d50 con 0x7f8ed80b2840 2024-07-04T08:15:02.616 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:02.618 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:15:02.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ed80b2840 msgr2=0x7f8ed80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:02.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ed80b2840 0x7f8ed80b4d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0095f0 tx=0x7f8edc005f90 comp rx=0 tx=0).stop 2024-07-04T08:15:02.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef0105220 msgr2=0x7f8ef019bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:02.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef0105220 0x7f8ef019bb90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400e9f0 tx=0x7f8ee400edf0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 shutdown_connections 2024-07-04T08:15:02.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ed80b2840 0x7f8ed80b4d50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef010b4a0 0x7f8ef01a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef010ab50 0x7f8ef019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 --2- 172.21.15.39:0/2175139450 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef0105220 0x7f8ef019bb90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:02.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 >> 172.21.15.39:0/2175139450 conn(0x7f8ef0100c60 msgr2=0x7f8ef010a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:02.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 shutdown_connections 2024-07-04T08:15:02.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:02.616+0000 7f8ef8000640 1 -- 172.21.15.39:0/2175139450 wait complete. 2024-07-04T08:15:02.649 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-07-04T08:15:01.684265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662779,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2453,"num_read_kb":1331,"num_write":387,"num_write_kb":2305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":819,"ondisk_log_size":819,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374188,"kb_used_data":12556,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567636,"statfs":{"total":1151940427776,"available":1151557259264,"internally_reserved":0,"allocated":12857344,"data_stored":8490461,"data_compressed":498873,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":306,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":56,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006143"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":295,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:58.426303+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:14:58.426303+0000","last_peered":"2024-07-04T08:14:58.426303+0000","last_clean":"2024-07-04T08:14:58.426303+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:14:58.426303+0000","last_undegraded":"2024-07-04T08:14:58.426303+0000","last_fullsized":"2024-07-04T08:14:58.426303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":235,"num_read_kb":235,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553793+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:12:51.553793+0000","last_peered":"2024-07-04T08:12:51.553793+0000","last_clean":"2024-07-04T08:12:51.553793+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:12:51.553793+0000","last_undegraded":"2024-07-04T08:12:51.553793+0000","last_fullsized":"2024-07-04T08:12:51.553793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:16.118017+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:12:16.118017+0000","last_peered":"2024-07-04T08:12:16.118017+0000","last_clean":"2024-07-04T08:12:16.118017+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:12:16.118017+0000","last_undegraded":"2024-07-04T08:12:16.118017+0000","last_fullsized":"2024-07-04T08:12:16.118017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.698593+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:53.698593+0000","last_peered":"2024-07-04T08:11:53.698593+0000","last_clean":"2024-07-04T08:11:53.698593+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:53.698593+0000","last_undegraded":"2024-07-04T08:11:53.698593+0000","last_fullsized":"2024-07-04T08:11:53.698593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569975+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:12:51.569975+0000","last_peered":"2024-07-04T08:12:51.569975+0000","last_clean":"2024-07-04T08:12:51.569975+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:12:51.569975+0000","last_undegraded":"2024-07-04T08:12:51.569975+0000","last_fullsized":"2024-07-04T08:12:51.569975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:15.756154+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:12:15.756154+0000","last_peered":"2024-07-04T08:12:15.756154+0000","last_clean":"2024-07-04T08:12:15.756154+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:12:15.756154+0000","last_undegraded":"2024-07-04T08:12:15.756154+0000","last_fullsized":"2024-07-04T08:12:15.756154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247127+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:12:05.247127+0000","last_peered":"2024-07-04T08:12:05.247127+0000","last_clean":"2024-07-04T08:12:05.247127+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:12:05.247127+0000","last_undegraded":"2024-07-04T08:12:05.247127+0000","last_fullsized":"2024-07-04T08:12:05.247127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.549821+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:12:51.549821+0000","last_peered":"2024-07-04T08:12:51.549821+0000","last_clean":"2024-07-04T08:12:51.549821+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:12:51.549821+0000","last_undegraded":"2024-07-04T08:12:51.549821+0000","last_fullsized":"2024-07-04T08:12:51.549821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756728+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:12:20.756728+0000","last_peered":"2024-07-04T08:12:20.756728+0000","last_clean":"2024-07-04T08:12:20.756728+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:12:20.756728+0000","last_undegraded":"2024-07-04T08:12:20.756728+0000","last_fullsized":"2024-07-04T08:12:20.756728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247079+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:12:05.247079+0000","last_peered":"2024-07-04T08:12:05.247079+0000","last_clean":"2024-07-04T08:12:05.247079+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:12:05.247079+0000","last_undegraded":"2024-07-04T08:12:05.247079+0000","last_fullsized":"2024-07-04T08:12:05.247079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.535944+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:12:51.535944+0000","last_peered":"2024-07-04T08:12:51.535944+0000","last_clean":"2024-07-04T08:12:51.535944+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:12:51.535944+0000","last_undegraded":"2024-07-04T08:12:51.535944+0000","last_fullsized":"2024-07-04T08:12:51.535944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756706+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:12:20.756706+0000","last_peered":"2024-07-04T08:12:20.756706+0000","last_clean":"2024-07-04T08:12:20.756706+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:12:20.756706+0000","last_undegraded":"2024-07-04T08:12:20.756706+0000","last_fullsized":"2024-07-04T08:12:20.756706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562540+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:12:51.562540+0000","last_peered":"2024-07-04T08:12:51.562540+0000","last_clean":"2024-07-04T08:12:51.562540+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:12:51.562540+0000","last_undegraded":"2024-07-04T08:12:51.562540+0000","last_fullsized":"2024-07-04T08:12:51.562540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844569+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:53.844569+0000","last_peered":"2024-07-04T08:11:53.844569+0000","last_clean":"2024-07-04T08:11:53.844569+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:53.844569+0000","last_undegraded":"2024-07-04T08:11:53.844569+0000","last_fullsized":"2024-07-04T08:11:53.844569+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754878+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:12:05.754878+0000","last_peered":"2024-07-04T08:12:05.754878+0000","last_clean":"2024-07-04T08:12:05.754878+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:12:05.754878+0000","last_undegraded":"2024-07-04T08:12:05.754878+0000","last_fullsized":"2024-07-04T08:12:05.754878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570735+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:12:51.570735+0000","last_peered":"2024-07-04T08:12:51.570735+0000","last_clean":"2024-07-04T08:12:51.570735+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:12:51.570735+0000","last_undegraded":"2024-07-04T08:12:51.570735+0000","last_fullsized":"2024-07-04T08:12:51.570735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691548+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:53.691548+0000","last_peered":"2024-07-04T08:11:53.691548+0000","last_clean":"2024-07-04T08:11:53.691548+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:53.691548+0000","last_undegraded":"2024-07-04T08:11:53.691548+0000","last_fullsized":"2024-07-04T08:11:53.691548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754825+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:12:05.754825+0000","last_peered":"2024-07-04T08:12:05.754825+0000","last_clean":"2024-07-04T08:12:05.754825+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:12:05.754825+0000","last_undegraded":"2024-07-04T08:12:05.754825+0000","last_fullsized":"2024-07-04T08:12:05.754825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566667+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:12:51.566667+0000","last_peered":"2024-07-04T08:12:51.566667+0000","last_clean":"2024-07-04T08:12:51.566667+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:12:51.566667+0000","last_undegraded":"2024-07-04T08:12:51.566667+0000","last_fullsized":"2024-07-04T08:12:51.566667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.697029+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:53.697029+0000","last_peered":"2024-07-04T08:11:53.697029+0000","last_clean":"2024-07-04T08:11:53.697029+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:53.697029+0000","last_undegraded":"2024-07-04T08:11:53.697029+0000","last_fullsized":"2024-07-04T08:11:53.697029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771248+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:11:53.771248+0000","last_peered":"2024-07-04T08:11:53.771248+0000","last_clean":"2024-07-04T08:11:53.771248+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:11:53.771248+0000","last_undegraded":"2024-07-04T08:11:53.771248+0000","last_fullsized":"2024-07-04T08:11:53.771248+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.526597+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:12:51.526597+0000","last_peered":"2024-07-04T08:12:51.526597+0000","last_clean":"2024-07-04T08:12:51.526597+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:12:51.526597+0000","last_undegraded":"2024-07-04T08:12:51.526597+0000","last_fullsized":"2024-07-04T08:12:51.526597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936427+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:53.936427+0000","last_peered":"2024-07-04T08:11:53.936427+0000","last_clean":"2024-07-04T08:11:53.936427+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:53.936427+0000","last_undegraded":"2024-07-04T08:11:53.936427+0000","last_fullsized":"2024-07-04T08:11:53.936427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754887+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:12:05.754887+0000","last_peered":"2024-07-04T08:12:05.754887+0000","last_clean":"2024-07-04T08:12:05.754887+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:12:05.754887+0000","last_undegraded":"2024-07-04T08:12:05.754887+0000","last_fullsized":"2024-07-04T08:12:05.754887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568004+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:12:51.568004+0000","last_peered":"2024-07-04T08:12:51.568004+0000","last_clean":"2024-07-04T08:12:51.568004+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:12:51.568004+0000","last_undegraded":"2024-07-04T08:12:51.568004+0000","last_fullsized":"2024-07-04T08:12:51.568004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692160+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:53.692160+0000","last_peered":"2024-07-04T08:11:53.692160+0000","last_clean":"2024-07-04T08:11:53.692160+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:53.692160+0000","last_undegraded":"2024-07-04T08:11:53.692160+0000","last_fullsized":"2024-07-04T08:11:53.692160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115754+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:11:56.115754+0000","last_peered":"2024-07-04T08:11:56.115754+0000","last_clean":"2024-07-04T08:11:56.115754+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:11:56.115754+0000","last_undegraded":"2024-07-04T08:11:56.115754+0000","last_fullsized":"2024-07-04T08:11:56.115754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.556576+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:12:51.556576+0000","last_peered":"2024-07-04T08:12:51.556576+0000","last_clean":"2024-07-04T08:12:51.556576+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:12:51.556576+0000","last_undegraded":"2024-07-04T08:12:51.556576+0000","last_fullsized":"2024-07-04T08:12:51.556576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'17","reported_seq":73,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:54.818022+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:13:54.818022+0000","last_peered":"2024-07-04T08:13:54.818022+0000","last_clean":"2024-07-04T08:13:54.818022+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:13:54.818022+0000","last_undegraded":"2024-07-04T08:13:54.818022+0000","last_fullsized":"2024-07-04T08:13:54.818022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.322378+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:11:54.322378+0000","last_peered":"2024-07-04T08:11:54.322378+0000","last_clean":"2024-07-04T08:11:54.322378+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:11:54.322378+0000","last_undegraded":"2024-07-04T08:11:54.322378+0000","last_fullsized":"2024-07-04T08:11:54.322378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540856+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:12:51.540856+0000","last_peered":"2024-07-04T08:12:51.540856+0000","last_clean":"2024-07-04T08:12:51.540856+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:12:51.540856+0000","last_undegraded":"2024-07-04T08:12:51.540856+0000","last_fullsized":"2024-07-04T08:12:51.540856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771431+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:53.771431+0000","last_peered":"2024-07-04T08:11:53.771431+0000","last_clean":"2024-07-04T08:11:53.771431+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:53.771431+0000","last_undegraded":"2024-07-04T08:11:53.771431+0000","last_fullsized":"2024-07-04T08:11:53.771431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.238979+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:11:56.238979+0000","last_peered":"2024-07-04T08:11:56.238979+0000","last_clean":"2024-07-04T08:11:56.238979+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:11:56.238979+0000","last_undegraded":"2024-07-04T08:11:56.238979+0000","last_fullsized":"2024-07-04T08:11:56.238979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542562+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:12:51.542562+0000","last_peered":"2024-07-04T08:12:51.542562+0000","last_clean":"2024-07-04T08:12:51.542562+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:12:51.542562+0000","last_undegraded":"2024-07-04T08:12:51.542562+0000","last_fullsized":"2024-07-04T08:12:51.542562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993415+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:11:53.993415+0000","last_peered":"2024-07-04T08:11:53.993415+0000","last_clean":"2024-07-04T08:11:53.993415+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:11:53.993415+0000","last_undegraded":"2024-07-04T08:11:53.993415+0000","last_fullsized":"2024-07-04T08:11:53.993415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:00.246372+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:12:00.246372+0000","last_peered":"2024-07-04T08:12:00.246372+0000","last_clean":"2024-07-04T08:12:00.246372+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:12:00.246372+0000","last_undegraded":"2024-07-04T08:12:00.246372+0000","last_fullsized":"2024-07-04T08:12:00.246372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.543737+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:12:51.543737+0000","last_peered":"2024-07-04T08:12:51.543737+0000","last_clean":"2024-07-04T08:12:51.543737+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:12:51.543737+0000","last_undegraded":"2024-07-04T08:12:51.543737+0000","last_fullsized":"2024-07-04T08:12:51.543737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":291,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:59.429621+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:14:59.429621+0000","last_peered":"2024-07-04T08:14:59.429621+0000","last_clean":"2024-07-04T08:14:59.429621+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:14:59.429621+0000","last_undegraded":"2024-07-04T08:14:59.429621+0000","last_fullsized":"2024-07-04T08:14:59.429621+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":238,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:58.137211+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:11:58.137211+0000","last_peered":"2024-07-04T08:11:58.137211+0000","last_clean":"2024-07-04T08:11:58.137211+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:11:58.137211+0000","last_undegraded":"2024-07-04T08:11:58.137211+0000","last_fullsized":"2024-07-04T08:11:58.137211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.516953+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:12:51.516953+0000","last_peered":"2024-07-04T08:12:51.516953+0000","last_clean":"2024-07-04T08:12:51.516953+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:12:51.516953+0000","last_undegraded":"2024-07-04T08:12:51.516953+0000","last_fullsized":"2024-07-04T08:12:51.516953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":285,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:56.479235+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:14:56.479235+0000","last_peered":"2024-07-04T08:14:56.479235+0000","last_clean":"2024-07-04T08:14:56.479235+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:14:56.479235+0000","last_undegraded":"2024-07-04T08:14:56.479235+0000","last_fullsized":"2024-07-04T08:14:56.479235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":233,"num_read_kb":233,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138305+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:12:03.138305+0000","last_peered":"2024-07-04T08:12:03.138305+0000","last_clean":"2024-07-04T08:12:03.138305+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:12:03.138305+0000","last_undegraded":"2024-07-04T08:12:03.138305+0000","last_fullsized":"2024-07-04T08:12:03.138305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534873+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:12:51.534873+0000","last_peered":"2024-07-04T08:12:51.534873+0000","last_clean":"2024-07-04T08:12:51.534873+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:12:51.534873+0000","last_undegraded":"2024-07-04T08:12:51.534873+0000","last_fullsized":"2024-07-04T08:12:51.534873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851437+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:53.851437+0000","last_peered":"2024-07-04T08:11:53.851437+0000","last_clean":"2024-07-04T08:11:53.851437+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:53.851437+0000","last_undegraded":"2024-07-04T08:11:53.851437+0000","last_fullsized":"2024-07-04T08:11:53.851437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975520+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:12:04.975520+0000","last_peered":"2024-07-04T08:12:04.975520+0000","last_clean":"2024-07-04T08:12:04.975520+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:12:04.975520+0000","last_undegraded":"2024-07-04T08:12:04.975520+0000","last_fullsized":"2024-07-04T08:12:04.975520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.558356+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:12:51.558356+0000","last_peered":"2024-07-04T08:12:51.558356+0000","last_clean":"2024-07-04T08:12:51.558356+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:12:51.558356+0000","last_undegraded":"2024-07-04T08:12:51.558356+0000","last_fullsized":"2024-07-04T08:12:51.558356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691883+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:53.691883+0000","last_peered":"2024-07-04T08:11:53.691883+0000","last_clean":"2024-07-04T08:11:53.691883+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:53.691883+0000","last_undegraded":"2024-07-04T08:11:53.691883+0000","last_fullsized":"2024-07-04T08:11:53.691883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":120,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:46.091830+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:12:46.091830+0000","last_peered":"2024-07-04T08:12:46.091830+0000","last_clean":"2024-07-04T08:12:46.091830+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:12:46.091830+0000","last_undegraded":"2024-07-04T08:12:46.091830+0000","last_fullsized":"2024-07-04T08:12:46.091830+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.085858+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:11:56.085858+0000","last_peered":"2024-07-04T08:11:56.085858+0000","last_clean":"2024-07-04T08:11:56.085858+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:11:56.085858+0000","last_undegraded":"2024-07-04T08:11:56.085858+0000","last_fullsized":"2024-07-04T08:11:56.085858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553863+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:12:51.553863+0000","last_peered":"2024-07-04T08:12:51.553863+0000","last_clean":"2024-07-04T08:12:51.553863+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:12:51.553863+0000","last_undegraded":"2024-07-04T08:12:51.553863+0000","last_fullsized":"2024-07-04T08:12:51.553863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":256,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:57.190728+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:14:57.190728+0000","last_peered":"2024-07-04T08:14:57.190728+0000","last_clean":"2024-07-04T08:14:57.190728+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:14:57.190728+0000","last_undegraded":"2024-07-04T08:14:57.190728+0000","last_fullsized":"2024-07-04T08:14:57.190728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":235,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.151807+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:12:04.151807+0000","last_peered":"2024-07-04T08:12:04.151807+0000","last_clean":"2024-07-04T08:12:04.151807+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:12:04.151807+0000","last_undegraded":"2024-07-04T08:12:04.151807+0000","last_fullsized":"2024-07-04T08:12:04.151807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.561500+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:12:51.561500+0000","last_peered":"2024-07-04T08:12:51.561500+0000","last_clean":"2024-07-04T08:12:51.561500+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:12:51.561500+0000","last_undegraded":"2024-07-04T08:12:51.561500+0000","last_fullsized":"2024-07-04T08:12:51.561500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.939931+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:53.939931+0000","last_peered":"2024-07-04T08:11:53.939931+0000","last_clean":"2024-07-04T08:11:53.939931+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:11:53.939931+0000","last_undegraded":"2024-07-04T08:11:53.939931+0000","last_fullsized":"2024-07-04T08:11:53.939931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:09.641951+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:12:09.641951+0000","last_peered":"2024-07-04T08:12:09.641951+0000","last_clean":"2024-07-04T08:12:09.641951+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:12:09.641951+0000","last_undegraded":"2024-07-04T08:12:09.641951+0000","last_fullsized":"2024-07-04T08:12:09.641951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552528+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:12:51.552528+0000","last_peered":"2024-07-04T08:12:51.552528+0000","last_clean":"2024-07-04T08:12:51.552528+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:12:51.552528+0000","last_undegraded":"2024-07-04T08:12:51.552528+0000","last_fullsized":"2024-07-04T08:12:51.552528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995043+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:53.995043+0000","last_peered":"2024-07-04T08:11:53.995043+0000","last_clean":"2024-07-04T08:11:53.995043+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:53.995043+0000","last_undegraded":"2024-07-04T08:11:53.995043+0000","last_fullsized":"2024-07-04T08:11:53.995043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975432+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:12:04.975432+0000","last_peered":"2024-07-04T08:12:04.975432+0000","last_clean":"2024-07-04T08:12:04.975432+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:12:04.975432+0000","last_undegraded":"2024-07-04T08:12:04.975432+0000","last_fullsized":"2024-07-04T08:12:04.975432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531965+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:12:51.531965+0000","last_peered":"2024-07-04T08:12:51.531965+0000","last_clean":"2024-07-04T08:12:51.531965+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:12:51.531965+0000","last_undegraded":"2024-07-04T08:12:51.531965+0000","last_fullsized":"2024-07-04T08:12:51.531965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771354+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:53.771354+0000","last_peered":"2024-07-04T08:11:53.771354+0000","last_clean":"2024-07-04T08:11:53.771354+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:53.771354+0000","last_undegraded":"2024-07-04T08:11:53.771354+0000","last_fullsized":"2024-07-04T08:11:53.771354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975473+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:12:04.975473+0000","last_peered":"2024-07-04T08:12:04.975473+0000","last_clean":"2024-07-04T08:12:04.975473+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:12:04.975473+0000","last_undegraded":"2024-07-04T08:12:04.975473+0000","last_fullsized":"2024-07-04T08:12:04.975473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542769+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:12:51.542769+0000","last_peered":"2024-07-04T08:12:51.542769+0000","last_clean":"2024-07-04T08:12:51.542769+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:12:51.542769+0000","last_undegraded":"2024-07-04T08:12:51.542769+0000","last_fullsized":"2024-07-04T08:12:51.542769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:55.245889+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:55.245889+0000","last_peered":"2024-07-04T08:11:55.245889+0000","last_clean":"2024-07-04T08:11:55.245889+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:55.245889+0000","last_undegraded":"2024-07-04T08:11:55.245889+0000","last_fullsized":"2024-07-04T08:11:55.245889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.141070+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:12:18.141070+0000","last_peered":"2024-07-04T08:12:18.141070+0000","last_clean":"2024-07-04T08:12:18.141070+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:12:18.141070+0000","last_undegraded":"2024-07-04T08:12:18.141070+0000","last_fullsized":"2024-07-04T08:12:18.141070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552028+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:12:51.552028+0000","last_peered":"2024-07-04T08:12:51.552028+0000","last_clean":"2024-07-04T08:12:51.552028+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:12:51.552028+0000","last_undegraded":"2024-07-04T08:12:51.552028+0000","last_fullsized":"2024-07-04T08:12:51.552028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":292,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:59.487825+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:14:59.487825+0000","last_peered":"2024-07-04T08:14:59.487825+0000","last_clean":"2024-07-04T08:14:59.487825+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:14:59.487825+0000","last_undegraded":"2024-07-04T08:14:59.487825+0000","last_fullsized":"2024-07-04T08:14:59.487825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.666330+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:12:18.666330+0000","last_peered":"2024-07-04T08:12:18.666330+0000","last_clean":"2024-07-04T08:12:18.666330+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:12:18.666330+0000","last_undegraded":"2024-07-04T08:12:18.666330+0000","last_fullsized":"2024-07-04T08:12:18.666330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564982+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:12:51.564982+0000","last_peered":"2024-07-04T08:12:51.564982+0000","last_clean":"2024-07-04T08:12:51.564982+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:12:51.564982+0000","last_undegraded":"2024-07-04T08:12:51.564982+0000","last_fullsized":"2024-07-04T08:12:51.564982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.974209+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:54.974209+0000","last_peered":"2024-07-04T08:11:54.974209+0000","last_clean":"2024-07-04T08:11:54.974209+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:54.974209+0000","last_undegraded":"2024-07-04T08:11:54.974209+0000","last_fullsized":"2024-07-04T08:11:54.974209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:11.240822+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:12:11.240822+0000","last_peered":"2024-07-04T08:12:11.240822+0000","last_clean":"2024-07-04T08:12:11.240822+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:12:11.240822+0000","last_undegraded":"2024-07-04T08:12:11.240822+0000","last_fullsized":"2024-07-04T08:12:11.240822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562193+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:12:51.562193+0000","last_peered":"2024-07-04T08:12:51.562193+0000","last_clean":"2024-07-04T08:12:51.562193+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:12:51.562193+0000","last_undegraded":"2024-07-04T08:12:51.562193+0000","last_fullsized":"2024-07-04T08:12:51.562193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115841+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:56.115841+0000","last_peered":"2024-07-04T08:11:56.115841+0000","last_clean":"2024-07-04T08:11:56.115841+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:56.115841+0000","last_undegraded":"2024-07-04T08:11:56.115841+0000","last_fullsized":"2024-07-04T08:11:56.115841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.904331+0000","last_change":"2024-07-04T08:11:53.997034+0000","last_active":"2024-07-04T08:11:56.904331+0000","last_peered":"2024-07-04T08:11:56.904331+0000","last_clean":"2024-07-04T08:11:56.904331+0000","last_became_active":"2024-07-04T08:11:53.996456+0000","last_became_peered":"2024-07-04T08:11:53.996456+0000","last_unstale":"2024-07-04T08:11:56.904331+0000","last_undegraded":"2024-07-04T08:11:56.904331+0000","last_fullsized":"2024-07-04T08:11:56.904331+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.140549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.521874+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:12:51.521874+0000","last_peered":"2024-07-04T08:12:51.521874+0000","last_clean":"2024-07-04T08:12:51.521874+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:12:51.521874+0000","last_undegraded":"2024-07-04T08:12:51.521874+0000","last_fullsized":"2024-07-04T08:12:51.521874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":559,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:56.261606+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:14:56.261606+0000","last_peered":"2024-07-04T08:14:56.261606+0000","last_clean":"2024-07-04T08:14:56.261606+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:14:56.261606+0000","last_undegraded":"2024-07-04T08:14:56.261606+0000","last_fullsized":"2024-07-04T08:14:56.261606+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64691,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":473,"num_read_kb":300,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800726+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:12:14.800726+0000","last_peered":"2024-07-04T08:12:14.800726+0000","last_clean":"2024-07-04T08:12:14.800726+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:12:14.800726+0000","last_undegraded":"2024-07-04T08:12:14.800726+0000","last_fullsized":"2024-07-04T08:12:14.800726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541913+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:12:51.541913+0000","last_peered":"2024-07-04T08:12:51.541913+0000","last_clean":"2024-07-04T08:12:51.541913+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:12:51.541913+0000","last_undegraded":"2024-07-04T08:12:51.541913+0000","last_fullsized":"2024-07-04T08:12:51.541913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974814+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:59.974814+0000","last_peered":"2024-07-04T08:11:59.974814+0000","last_clean":"2024-07-04T08:11:59.974814+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:59.974814+0000","last_undegraded":"2024-07-04T08:11:59.974814+0000","last_fullsized":"2024-07-04T08:11:59.974814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:10.755408+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:12:10.755408+0000","last_peered":"2024-07-04T08:12:10.755408+0000","last_clean":"2024-07-04T08:12:10.755408+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:12:10.755408+0000","last_undegraded":"2024-07-04T08:12:10.755408+0000","last_fullsized":"2024-07-04T08:12:10.755408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569907+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:12:51.569907+0000","last_peered":"2024-07-04T08:12:51.569907+0000","last_clean":"2024-07-04T08:12:51.569907+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:12:51.569907+0000","last_undegraded":"2024-07-04T08:12:51.569907+0000","last_fullsized":"2024-07-04T08:12:51.569907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:01.086478+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:12:01.086478+0000","last_peered":"2024-07-04T08:12:01.086478+0000","last_clean":"2024-07-04T08:12:01.086478+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:12:01.086478+0000","last_undegraded":"2024-07-04T08:12:01.086478+0000","last_fullsized":"2024-07-04T08:12:01.086478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800668+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:12:14.800668+0000","last_peered":"2024-07-04T08:12:14.800668+0000","last_clean":"2024-07-04T08:12:14.800668+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:12:14.800668+0000","last_undegraded":"2024-07-04T08:12:14.800668+0000","last_fullsized":"2024-07-04T08:12:14.800668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.547189+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:12:51.547189+0000","last_peered":"2024-07-04T08:12:51.547189+0000","last_clean":"2024-07-04T08:12:51.547189+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:12:51.547189+0000","last_undegraded":"2024-07-04T08:12:51.547189+0000","last_fullsized":"2024-07-04T08:12:51.547189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":512,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:56.188725+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:14:56.188725+0000","last_peered":"2024-07-04T08:14:56.188725+0000","last_clean":"2024-07-04T08:14:56.188725+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:14:56.188725+0000","last_undegraded":"2024-07-04T08:14:56.188725+0000","last_fullsized":"2024-07-04T08:14:56.188725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":462,"num_read_kb":462,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.326445+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:12:14.326445+0000","last_peered":"2024-07-04T08:12:14.326445+0000","last_clean":"2024-07-04T08:12:14.326445+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:12:14.326445+0000","last_undegraded":"2024-07-04T08:12:14.326445+0000","last_fullsized":"2024-07-04T08:12:14.326445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.639390+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:11:54.639390+0000","last_peered":"2024-07-04T08:11:54.639390+0000","last_clean":"2024-07-04T08:11:54.639390+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:11:54.639390+0000","last_undegraded":"2024-07-04T08:11:54.639390+0000","last_fullsized":"2024-07-04T08:11:54.639390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564562+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:12:51.564562+0000","last_peered":"2024-07-04T08:12:51.564562+0000","last_clean":"2024-07-04T08:12:51.564562+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:12:51.564562+0000","last_undegraded":"2024-07-04T08:12:51.564562+0000","last_fullsized":"2024-07-04T08:12:51.564562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974768+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:59.974768+0000","last_peered":"2024-07-04T08:11:59.974768+0000","last_clean":"2024-07-04T08:11:59.974768+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:59.974768+0000","last_undegraded":"2024-07-04T08:11:59.974768+0000","last_fullsized":"2024-07-04T08:11:59.974768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.239030+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:11:56.239030+0000","last_peered":"2024-07-04T08:11:56.239030+0000","last_clean":"2024-07-04T08:11:56.239030+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:11:56.239030+0000","last_undegraded":"2024-07-04T08:11:56.239030+0000","last_fullsized":"2024-07-04T08:11:56.239030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565645+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:12:51.565645+0000","last_peered":"2024-07-04T08:12:51.565645+0000","last_clean":"2024-07-04T08:12:51.565645+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:12:51.565645+0000","last_undegraded":"2024-07-04T08:12:51.565645+0000","last_fullsized":"2024-07-04T08:12:51.565645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138191+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:12:03.138191+0000","last_peered":"2024-07-04T08:12:03.138191+0000","last_clean":"2024-07-04T08:12:03.138191+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:12:03.138191+0000","last_undegraded":"2024-07-04T08:12:03.138191+0000","last_fullsized":"2024-07-04T08:12:03.138191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:06.905597+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:12:06.905597+0000","last_peered":"2024-07-04T08:12:06.905597+0000","last_clean":"2024-07-04T08:12:06.905597+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:12:06.905597+0000","last_undegraded":"2024-07-04T08:12:06.905597+0000","last_fullsized":"2024-07-04T08:12:06.905597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.554669+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:12:51.554669+0000","last_peered":"2024-07-04T08:12:51.554669+0000","last_clean":"2024-07-04T08:12:51.554669+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:12:51.554669+0000","last_undegraded":"2024-07-04T08:12:51.554669+0000","last_fullsized":"2024-07-04T08:12:51.554669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118570+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:12:21.118570+0000","last_peered":"2024-07-04T08:12:21.118570+0000","last_clean":"2024-07-04T08:12:21.118570+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:12:21.118570+0000","last_undegraded":"2024-07-04T08:12:21.118570+0000","last_fullsized":"2024-07-04T08:12:21.118570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:14:58.234257+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:14:58.234257+0000","last_peered":"2024-07-04T08:14:58.234257+0000","last_clean":"2024-07-04T08:14:58.234257+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:14:58.234257+0000","last_undegraded":"2024-07-04T08:14:58.234257+0000","last_fullsized":"2024-07-04T08:14:58.234257+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":236,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.519199+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:12:51.519199+0000","last_peered":"2024-07-04T08:12:51.519199+0000","last_clean":"2024-07-04T08:12:51.519199+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:12:51.519199+0000","last_undegraded":"2024-07-04T08:12:51.519199+0000","last_fullsized":"2024-07-04T08:12:51.519199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118593+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:12:21.118593+0000","last_peered":"2024-07-04T08:12:21.118593+0000","last_clean":"2024-07-04T08:12:21.118593+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:12:21.118593+0000","last_undegraded":"2024-07-04T08:12:21.118593+0000","last_fullsized":"2024-07-04T08:12:21.118593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534235+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:12:51.534235+0000","last_peered":"2024-07-04T08:12:51.534235+0000","last_clean":"2024-07-04T08:12:51.534235+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:12:51.534235+0000","last_undegraded":"2024-07-04T08:12:51.534235+0000","last_fullsized":"2024-07-04T08:12:51.534235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.539681+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:12:51.539681+0000","last_peered":"2024-07-04T08:12:51.539681+0000","last_clean":"2024-07-04T08:12:51.539681+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:12:51.539681+0000","last_undegraded":"2024-07-04T08:12:51.539681+0000","last_fullsized":"2024-07-04T08:12:51.539681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566399+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:12:51.566399+0000","last_peered":"2024-07-04T08:12:51.566399+0000","last_clean":"2024-07-04T08:12:51.566399+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:12:51.566399+0000","last_undegraded":"2024-07-04T08:12:51.566399+0000","last_fullsized":"2024-07-04T08:12:51.566399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567851+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:12:51.567851+0000","last_peered":"2024-07-04T08:12:51.567851+0000","last_clean":"2024-07-04T08:12:51.567851+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:12:51.567851+0000","last_undegraded":"2024-07-04T08:12:51.567851+0000","last_fullsized":"2024-07-04T08:12:51.567851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569222+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:12:51.569222+0000","last_peered":"2024-07-04T08:12:51.569222+0000","last_clean":"2024-07-04T08:12:51.569222+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:12:51.569222+0000","last_undegraded":"2024-07-04T08:12:51.569222+0000","last_fullsized":"2024-07-04T08:12:51.569222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532408+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:12:51.532408+0000","last_peered":"2024-07-04T08:12:51.532408+0000","last_clean":"2024-07-04T08:12:51.532408+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:12:51.532408+0000","last_undegraded":"2024-07-04T08:12:51.532408+0000","last_fullsized":"2024-07-04T08:12:51.532408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540538+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:12:51.540538+0000","last_peered":"2024-07-04T08:12:51.540538+0000","last_clean":"2024-07-04T08:12:51.540538+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:12:51.540538+0000","last_undegraded":"2024-07-04T08:12:51.540538+0000","last_fullsized":"2024-07-04T08:12:51.540538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552348+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:12:51.552348+0000","last_peered":"2024-07-04T08:12:51.552348+0000","last_clean":"2024-07-04T08:12:51.552348+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:12:51.552348+0000","last_undegraded":"2024-07-04T08:12:51.552348+0000","last_fullsized":"2024-07-04T08:12:51.552348+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531503+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:12:51.531503+0000","last_peered":"2024-07-04T08:12:51.531503+0000","last_clean":"2024-07-04T08:12:51.531503+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:12:51.531503+0000","last_undegraded":"2024-07-04T08:12:51.531503+0000","last_fullsized":"2024-07-04T08:12:51.531503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.555574+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:12:51.555574+0000","last_peered":"2024-07-04T08:12:51.555574+0000","last_clean":"2024-07-04T08:12:51.555574+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:12:51.555574+0000","last_undegraded":"2024-07-04T08:12:51.555574+0000","last_fullsized":"2024-07-04T08:12:51.555574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560802+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:12:51.560802+0000","last_peered":"2024-07-04T08:12:51.560802+0000","last_clean":"2024-07-04T08:12:51.560802+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:12:51.560802+0000","last_undegraded":"2024-07-04T08:12:51.560802+0000","last_fullsized":"2024-07-04T08:12:51.560802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570948+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:12:51.570948+0000","last_peered":"2024-07-04T08:12:51.570948+0000","last_clean":"2024-07-04T08:12:51.570948+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:12:51.570948+0000","last_undegraded":"2024-07-04T08:12:51.570948+0000","last_fullsized":"2024-07-04T08:12:51.570948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532752+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:12:51.532752+0000","last_peered":"2024-07-04T08:12:51.532752+0000","last_clean":"2024-07-04T08:12:51.532752+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:12:51.532752+0000","last_undegraded":"2024-07-04T08:12:51.532752+0000","last_fullsized":"2024-07-04T08:12:51.532752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552705+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:12:51.552705+0000","last_peered":"2024-07-04T08:12:51.552705+0000","last_clean":"2024-07-04T08:12:51.552705+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:12:51.552705+0000","last_undegraded":"2024-07-04T08:12:51.552705+0000","last_fullsized":"2024-07-04T08:12:51.552705+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541640+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:12:51.541640+0000","last_peered":"2024-07-04T08:12:51.541640+0000","last_clean":"2024-07-04T08:12:51.541640+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:12:51.541640+0000","last_undegraded":"2024-07-04T08:12:51.541640+0000","last_fullsized":"2024-07-04T08:12:51.541640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569708+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:12:51.569708+0000","last_peered":"2024-07-04T08:12:51.569708+0000","last_clean":"2024-07-04T08:12:51.569708+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:12:51.569708+0000","last_undegraded":"2024-07-04T08:12:51.569708+0000","last_fullsized":"2024-07-04T08:12:51.569708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.571140+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:12:51.571140+0000","last_peered":"2024-07-04T08:12:51.571140+0000","last_clean":"2024-07-04T08:12:51.571140+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:12:51.571140+0000","last_undegraded":"2024-07-04T08:12:51.571140+0000","last_fullsized":"2024-07-04T08:12:51.571140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552227+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:12:51.552227+0000","last_peered":"2024-07-04T08:12:51.552227+0000","last_clean":"2024-07-04T08:12:51.552227+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:12:51.552227+0000","last_undegraded":"2024-07-04T08:12:51.552227+0000","last_fullsized":"2024-07-04T08:12:51.552227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567284+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:12:51.567284+0000","last_peered":"2024-07-04T08:12:51.567284+0000","last_clean":"2024-07-04T08:12:51.567284+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:12:51.567284+0000","last_undegraded":"2024-07-04T08:12:51.567284+0000","last_fullsized":"2024-07-04T08:12:51.567284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551582+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:12:51.551582+0000","last_peered":"2024-07-04T08:12:51.551582+0000","last_clean":"2024-07-04T08:12:51.551582+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:12:51.551582+0000","last_undegraded":"2024-07-04T08:12:51.551582+0000","last_fullsized":"2024-07-04T08:12:51.551582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534085+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:12:51.534085+0000","last_peered":"2024-07-04T08:12:51.534085+0000","last_clean":"2024-07-04T08:12:51.534085+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:12:51.534085+0000","last_undegraded":"2024-07-04T08:12:51.534085+0000","last_fullsized":"2024-07-04T08:12:51.534085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553666+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:12:51.553666+0000","last_peered":"2024-07-04T08:12:51.553666+0000","last_clean":"2024-07-04T08:12:51.553666+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:12:51.553666+0000","last_undegraded":"2024-07-04T08:12:51.553666+0000","last_fullsized":"2024-07-04T08:12:51.553666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540373+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:12:51.540373+0000","last_peered":"2024-07-04T08:12:51.540373+0000","last_clean":"2024-07-04T08:12:51.540373+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:12:51.540373+0000","last_undegraded":"2024-07-04T08:12:51.540373+0000","last_fullsized":"2024-07-04T08:12:51.540373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541173+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:12:51.541173+0000","last_peered":"2024-07-04T08:12:51.541173+0000","last_clean":"2024-07-04T08:12:51.541173+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:12:51.541173+0000","last_undegraded":"2024-07-04T08:12:51.541173+0000","last_fullsized":"2024-07-04T08:12:51.541173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560317+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:12:51.560317+0000","last_peered":"2024-07-04T08:12:51.560317+0000","last_clean":"2024-07-04T08:12:51.560317+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:12:51.560317+0000","last_undegraded":"2024-07-04T08:12:51.560317+0000","last_fullsized":"2024-07-04T08:12:51.560317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551363+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:12:51.551363+0000","last_peered":"2024-07-04T08:12:51.551363+0000","last_clean":"2024-07-04T08:12:51.551363+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:12:51.551363+0000","last_undegraded":"2024-07-04T08:12:51.551363+0000","last_fullsized":"2024-07-04T08:12:51.551363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541562+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:12:51.541562+0000","last_peered":"2024-07-04T08:12:51.541562+0000","last_clean":"2024-07-04T08:12:51.541562+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:12:51.541562+0000","last_undegraded":"2024-07-04T08:12:51.541562+0000","last_fullsized":"2024-07-04T08:12:51.541562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.536909+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:12:51.536909+0000","last_peered":"2024-07-04T08:12:51.536909+0000","last_clean":"2024-07-04T08:12:51.536909+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:12:51.536909+0000","last_undegraded":"2024-07-04T08:12:51.536909+0000","last_fullsized":"2024-07-04T08:12:51.536909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537941+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:12:51.537941+0000","last_peered":"2024-07-04T08:12:51.537941+0000","last_clean":"2024-07-04T08:12:51.537941+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:12:51.537941+0000","last_undegraded":"2024-07-04T08:12:51.537941+0000","last_fullsized":"2024-07-04T08:12:51.537941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565574+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:12:51.565574+0000","last_peered":"2024-07-04T08:12:51.565574+0000","last_clean":"2024-07-04T08:12:51.565574+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:12:51.565574+0000","last_undegraded":"2024-07-04T08:12:51.565574+0000","last_fullsized":"2024-07-04T08:12:51.565574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568095+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:12:51.568095+0000","last_peered":"2024-07-04T08:12:51.568095+0000","last_clean":"2024-07-04T08:12:51.568095+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:12:51.568095+0000","last_undegraded":"2024-07-04T08:12:51.568095+0000","last_fullsized":"2024-07-04T08:12:51.568095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569523+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:12:51.569523+0000","last_peered":"2024-07-04T08:12:51.569523+0000","last_clean":"2024-07-04T08:12:51.569523+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:12:51.569523+0000","last_undegraded":"2024-07-04T08:12:51.569523+0000","last_fullsized":"2024-07-04T08:12:51.569523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72177,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2372,"num_read_kb":1262,"num_write":262,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":227277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612690,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":543564,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28164,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716988,"statfs":{"total":95995035648,"available":95966195712,"internally_reserved":0,"allocated":1175552,"data_stored":1195603,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":547590,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168674,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":543872,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364904,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1141042,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528428,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":539530,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724658,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1135560,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920887,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":543604,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117116,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":538706,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378756,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":609142,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542280,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":611794,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705805,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":540454,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:15:02.654 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:15:03.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:02 smithi133 ceph-mon[37050]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:03.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:02 smithi133 ceph-mon[37050]: pgmap v475: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-04T08:15:03.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:02 smithi133 ceph-mon[37050]: from='client.25417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:02 smithi039 ceph-mon[32062]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:02 smithi039 ceph-mon[32062]: pgmap v475: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-04T08:15:03.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:02 smithi039 ceph-mon[32062]: from='client.25417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:02 smithi050 ceph-mon[36577]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:02 smithi050 ceph-mon[36577]: pgmap v475: 129 pgs: 129 active+clean; 647 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-07-04T08:15:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:02 smithi050 ceph-mon[36577]: from='client.25417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:03.179 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:04.067 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:03 smithi133 ceph-mon[37050]: from='client.25448 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:04.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:03 smithi039 ceph-mon[32062]: from='client.25448 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:04.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:03 smithi050 ceph-mon[36577]: from='client.25448 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:05.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:04 smithi133 ceph-mon[37050]: pgmap v476: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-04T08:15:05.067 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:04 smithi039 ceph-mon[32062]: pgmap v476: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-04T08:15:05.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:04 smithi050 ceph-mon[36577]: pgmap v476: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-04T08:15:05.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.443+0000 7fd5d0400640 1 -- 172.21.15.39:0/2742029394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c81089f0 msgr2=0x7fd5c8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:05.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.443+0000 7fd5d0400640 1 --2- 172.21.15.39:0/2742029394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c81089f0 0x7fd5c8108e00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009ca0 tx=0x7fd5b402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.443+0000 7fd5d0400640 1 -- 172.21.15.39:0/2742029394 shutdown_connections 2024-07-04T08:15:05.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.443+0000 7fd5d0400640 1 --2- 172.21.15.39:0/2742029394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c8109340 0x7fd5c810fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.443+0000 7fd5d0400640 1 --2- 172.21.15.39:0/2742029394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c81089f0 0x7fd5c8108e00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.443+0000 7fd5d0400640 1 --2- 172.21.15.39:0/2742029394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c81030b0 0x7fd5c81034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.443+0000 7fd5d0400640 1 -- 172.21.15.39:0/2742029394 >> 172.21.15.39:0/2742029394 conn(0x7fd5c807ada0 msgr2=0x7fd5c807b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:05.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.443+0000 7fd5d0400640 1 -- 172.21.15.39:0/2742029394 shutdown_connections 2024-07-04T08:15:05.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.443+0000 7fd5d0400640 1 -- 172.21.15.39:0/2742029394 wait complete. 2024-07-04T08:15:05.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.445+0000 7fd5d0400640 1 Processor -- start 2024-07-04T08:15:05.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.445+0000 7fd5d0400640 1 -- start start 2024-07-04T08:15:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5d0400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81030b0 0x7fd5c8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5d0400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c81089f0 0x7fd5c80751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5d0400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c8109340 0x7fd5c80756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5d0400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8113000 con 0x7fd5c81030b0 2024-07-04T08:15:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5d0400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8112e80 con 0x7fd5c81089f0 2024-07-04T08:15:05.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5d0400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8113180 con 0x7fd5c8109340 2024-07-04T08:15:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5cda00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81030b0 0x7fd5c8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c81089f0 0x7fd5c80751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5cd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c81089f0 0x7fd5c80751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51070/0 (socket says 172.21.15.39:51070) 2024-07-04T08:15:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5cd000640 1 -- 172.21.15.39:0/1405222781 learned_addr learned my addr 172.21.15.39:0/1405222781 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5ce400640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c8109340 0x7fd5c80756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5cd000640 1 -- 172.21.15.39:0/1405222781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c8109340 msgr2=0x7fd5c80756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:05.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.446+0000 7fd5cd000640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c8109340 0x7fd5c80756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.447+0000 7fd5cd000640 1 -- 172.21.15.39:0/1405222781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81030b0 msgr2=0x7fd5c8074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:05.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.447+0000 7fd5cd000640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81030b0 0x7fd5c8074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.447+0000 7fd5cd000640 1 -- 172.21.15.39:0/1405222781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c81a7cd0 con 0x7fd5c81089f0 2024-07-04T08:15:05.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.447+0000 7fd5cd000640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c81089f0 0x7fd5c80751a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4000f80 tx=0x7fd5b4031750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:05.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.447+0000 7fd5c6000640 1 -- 172.21.15.39:0/1405222781 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4003b70 con 0x7fd5c81089f0 2024-07-04T08:15:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.447+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c81a7f60 con 0x7fd5c81089f0 2024-07-04T08:15:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.448+0000 7fd5c6000640 1 -- 172.21.15.39:0/1405222781 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5b4003d10 con 0x7fd5c81089f0 2024-07-04T08:15:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.448+0000 7fd5c6000640 1 -- 172.21.15.39:0/1405222781 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b402fa00 con 0x7fd5c81089f0 2024-07-04T08:15:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.448+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c81a8410 con 0x7fd5c81089f0 2024-07-04T08:15:05.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.449+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c81034c0 con 0x7fd5c81089f0 2024-07-04T08:15:05.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.452+0000 7fd5c6000640 1 -- 172.21.15.39:0/1405222781 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7fd5b4004580 con 0x7fd5c81089f0 2024-07-04T08:15:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.453+0000 7fd5c6000640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5ac0b2840 0x7fd5ac0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.453+0000 7fd5c6000640 1 -- 172.21.15.39:0/1405222781 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fd5b4103c00 con 0x7fd5c81089f0 2024-07-04T08:15:05.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.453+0000 7fd5cda00640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5ac0b2840 0x7fd5ac0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:05.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.454+0000 7fd5cda00640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5ac0b2840 0x7fd5ac0b4d50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80766b0 tx=0x7fd5b8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:05.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.455+0000 7fd5c6000640 1 -- 172.21.15.39:0/1405222781 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fd5b40cb4d0 con 0x7fd5c81089f0 2024-07-04T08:15:05.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.713+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5c8075f60 con 0x7fd5c81089f0 2024-07-04T08:15:05.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.715+0000 7fd5c6000640 1 -- 172.21.15.39:0/1405222781 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fd5c8075f60 con 0x7fd5c81089f0 2024-07-04T08:15:05.716 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:05.716 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:15:05.717 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:15:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5ac0b2840 msgr2=0x7fd5ac0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:05.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5ac0b2840 0x7fd5ac0b4d50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80766b0 tx=0x7fd5b8005f90 comp rx=0 tx=0).stop 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c81089f0 msgr2=0x7fd5c80751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c81089f0 0x7fd5c80751a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4000f80 tx=0x7fd5b4031750 comp rx=0 tx=0).stop 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 shutdown_connections 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5ac0b2840 0x7fd5ac0b4d50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c8109340 0x7fd5c80756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5c81089f0 0x7fd5c80751a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 --2- 172.21.15.39:0/1405222781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81030b0 0x7fd5c8074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.718+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 >> 172.21.15.39:0/1405222781 conn(0x7fd5c807ada0 msgr2=0x7fd5c8106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.719+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 shutdown_connections 2024-07-04T08:15:05.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:05.719+0000 7fd5d0400640 1 -- 172.21.15.39:0/1405222781 wait complete. 2024-07-04T08:15:06.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:05 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1405222781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:06.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:05 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1405222781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:06.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:05 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1405222781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:06.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:15:07.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:06 smithi133 ceph-mon[37050]: pgmap v477: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-04T08:15:07.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:06 smithi039 ceph-mon[32062]: pgmap v477: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-04T08:15:07.102 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:07.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:06 smithi050 ceph-mon[36577]: pgmap v477: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-04T08:15:09.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:08 smithi133 ceph-mon[37050]: pgmap v478: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-04T08:15:09.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:08 smithi039 ceph-mon[32062]: pgmap v478: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-04T08:15:09.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:08 smithi050 ceph-mon[36577]: pgmap v478: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-04T08:15:09.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.448+0000 7fb5f3200640 1 -- 172.21.15.39:0/701374789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f410ee50 msgr2=0x7fb5f410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:09.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.448+0000 7fb5f3200640 1 --2- 172.21.15.39:0/701374789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f410ee50 0x7fb5f410f260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009900 tx=0x7fb5dc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.448+0000 7fb5f3200640 1 -- 172.21.15.39:0/701374789 shutdown_connections 2024-07-04T08:15:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.448+0000 7fb5f3200640 1 --2- 172.21.15.39:0/701374789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f410ee50 0x7fb5f410f260 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.448+0000 7fb5f3200640 1 --2- 172.21.15.39:0/701374789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f410f7a0 0x7fb5f4115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.448+0000 7fb5f3200640 1 --2- 172.21.15.39:0/701374789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5f4109510 0x7fb5f4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.448+0000 7fb5f3200640 1 -- 172.21.15.39:0/701374789 >> 172.21.15.39:0/701374789 conn(0x7fb5f4078b10 msgr2=0x7fb5f4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.449+0000 7fb5f3200640 1 -- 172.21.15.39:0/701374789 shutdown_connections 2024-07-04T08:15:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.449+0000 7fb5f3200640 1 -- 172.21.15.39:0/701374789 wait complete. 2024-07-04T08:15:09.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.450+0000 7fb5f3200640 1 Processor -- start 2024-07-04T08:15:09.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.450+0000 7fb5f3200640 1 -- start start 2024-07-04T08:15:09.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f4109510 0x7fb5f419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:09.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f3200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f410ee50 0x7fb5f41a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5f410f7a0 0x7fb5f41a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4119460 con 0x7fb5f4109510 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f3200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41192e0 con 0x7fb5f410ee50 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41195e0 con 0x7fb5f410f7a0 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f4109510 0x7fb5f419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f4109510 0x7fb5f419fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59972/0 (socket says 172.21.15.39:59972) 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f1e00640 1 -- 172.21.15.39:0/3958550974 learned_addr learned my addr 172.21.15.39:0/3958550974 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5ebe00640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f410ee50 0x7fb5f41a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.451+0000 7fb5f2800640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5f410f7a0 0x7fb5f41a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5f1e00640 1 -- 172.21.15.39:0/3958550974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5f410f7a0 msgr2=0x7fb5f41a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5f1e00640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5f410f7a0 0x7fb5f41a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5f1e00640 1 -- 172.21.15.39:0/3958550974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f410ee50 msgr2=0x7fb5f41a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5f1e00640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f410ee50 0x7fb5f41a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5f1e00640 1 -- 172.21.15.39:0/3958550974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f41a5de0 con 0x7fb5f4109510 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5f1e00640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f4109510 0x7fb5f419fe70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400d950 tx=0x7fb5e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5eaa00640 1 -- 172.21.15.39:0/3958550974 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4017070 con 0x7fb5f4109510 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5eaa00640 1 -- 172.21.15.39:0/3958550974 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e4009c30 con 0x7fb5f4109510 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f41a60d0 con 0x7fb5f4109510 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.452+0000 7fb5eaa00640 1 -- 172.21.15.39:0/3958550974 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400f620 con 0x7fb5f4109510 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.453+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f41ac830 con 0x7fb5f4109510 2024-07-04T08:15:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.454+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f4109920 con 0x7fb5f4109510 2024-07-04T08:15:09.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.457+0000 7fb5eaa00640 1 -- 172.21.15.39:0/3958550974 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7fb5e4005030 con 0x7fb5f4109510 2024-07-04T08:15:09.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.458+0000 7fb5eaa00640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5d80b2840 0x7fb5d80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:09.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.458+0000 7fb5eaa00640 1 -- 172.21.15.39:0/3958550974 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb5e40e22a0 con 0x7fb5f4109510 2024-07-04T08:15:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.459+0000 7fb5ebe00640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5d80b2840 0x7fb5d80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.459+0000 7fb5ebe00640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5d80b2840 0x7fb5d80b4d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc02a9e0 tx=0x7fb5dc005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:09.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.460+0000 7fb5eaa00640 1 -- 172.21.15.39:0/3958550974 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb5e4009de0 con 0x7fb5f4109510 2024-07-04T08:15:09.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.678+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5f4110fc0 con 0x7fb5f4109510 2024-07-04T08:15:09.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.680+0000 7fb5eaa00640 1 -- 172.21.15.39:0/3958550974 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24957 (secure 0 0 0) 0x7fb5e40a94c0 con 0x7fb5f4109510 2024-07-04T08:15:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:09.682 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:11:53.672125+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:15:09.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5d80b2840 msgr2=0x7fb5d80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:09.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5d80b2840 0x7fb5d80b4d50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc02a9e0 tx=0x7fb5dc005a20 comp rx=0 tx=0).stop 2024-07-04T08:15:09.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f4109510 msgr2=0x7fb5f419fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:09.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f4109510 0x7fb5f419fe70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400d950 tx=0x7fb5e400de10 comp rx=0 tx=0).stop 2024-07-04T08:15:09.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 shutdown_connections 2024-07-04T08:15:09.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5d80b2840 0x7fb5d80b4d50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5f410f7a0 0x7fb5f41a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5f410ee50 0x7fb5f41a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 --2- 172.21.15.39:0/3958550974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5f4109510 0x7fb5f419fe70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:09.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 >> 172.21.15.39:0/3958550974 conn(0x7fb5f4078b10 msgr2=0x7fb5f40792e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:09.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 shutdown_connections 2024-07-04T08:15:09.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:09.683+0000 7fb5f3200640 1 -- 172.21.15.39:0/3958550974 wait complete. 2024-07-04T08:15:10.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:09 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3958550974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:15:10.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:09 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3958550974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:15:10.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:09 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3958550974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:15:10.653 DEBUG:tasks.fwd_scrub:Yielding 2024-07-04T08:15:10.654 INFO:teuthology.run_tasks:Running task exec... 2024-07-04T08:15:10.664 INFO:teuthology.task.exec:Executing custom commands... 2024-07-04T08:15:10.664 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T08:15:10.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-07-04T08:15:10.665 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-07-04T08:15:10.665 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:15:10.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:15:10.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.951+0000 7fc36b200640 1 -- 172.21.15.39:0/1469816668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364103c90 msgr2=0x7fc3641040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:10.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.951+0000 7fc36b200640 1 --2- 172.21.15.39:0/1469816668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364103c90 0x7fc3641040a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc358009900 tx=0x7fc35802a400 comp rx=0 tx=0).stop 2024-07-04T08:15:10.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.951+0000 7fc36b200640 1 -- 172.21.15.39:0/1469816668 shutdown_connections 2024-07-04T08:15:10.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.951+0000 7fc36b200640 1 --2- 172.21.15.39:0/1469816668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc36410d0f0 0x7fc36410f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:10.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.951+0000 7fc36b200640 1 --2- 172.21.15.39:0/1469816668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3641045e0 0x7fc36410cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:10.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.951+0000 7fc36b200640 1 --2- 172.21.15.39:0/1469816668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc364103c90 0x7fc3641040a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:10.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.951+0000 7fc36b200640 1 -- 172.21.15.39:0/1469816668 >> 172.21.15.39:0/1469816668 conn(0x7fc3640fd780 msgr2=0x7fc3640ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:10.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.951+0000 7fc36b200640 1 -- 172.21.15.39:0/1469816668 shutdown_connections 2024-07-04T08:15:10.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.951+0000 7fc36b200640 1 -- 172.21.15.39:0/1469816668 wait complete. 2024-07-04T08:15:10.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.952+0000 7fc36b200640 1 Processor -- start 2024-07-04T08:15:10.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc36b200640 1 -- start start 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc36b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc364103c90 0x7fc3641019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc36b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3641045e0 0x7fc364101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc36b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc36410d0f0 0x7fc364102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc36b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc364112d70 con 0x7fc36410d0f0 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc36b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc364112bf0 con 0x7fc364103c90 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc36b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc364112ef0 con 0x7fc3641045e0 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc369400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3641045e0 0x7fc364101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc36a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc36410d0f0 0x7fc364102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.953+0000 7fc369e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc364103c90 0x7fc3641019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.954+0000 7fc369400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3641045e0 0x7fc364101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33294/0 (socket says 172.21.15.39:33294) 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.954+0000 7fc369400640 1 -- 172.21.15.39:0/386444561 learned_addr learned my addr 172.21.15.39:0/386444561 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:10.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.954+0000 7fc369e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc364103c90 0x7fc3641019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51100/0 (socket says 172.21.15.39:51100) 2024-07-04T08:15:10.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.954+0000 7fc369400640 1 -- 172.21.15.39:0/386444561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc364103c90 msgr2=0x7fc3641019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.954+0000 7fc369400640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc364103c90 0x7fc3641019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.954+0000 7fc369400640 1 -- 172.21.15.39:0/386444561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc36410d0f0 msgr2=0x7fc364102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.954+0000 7fc369400640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc36410d0f0 0x7fc364102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.954+0000 7fc369400640 1 -- 172.21.15.39:0/386444561 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3641a7a40 con 0x7fc3641045e0 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.954+0000 7fc369400640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3641045e0 0x7fc364101f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc35400e9f0 tx=0x7fc35400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.955+0000 7fc35ea00640 1 -- 172.21.15.39:0/386444561 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35400ccc0 con 0x7fc3641045e0 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.955+0000 7fc35ea00640 1 -- 172.21.15.39:0/386444561 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3540108e0 con 0x7fc3641045e0 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.955+0000 7fc35ea00640 1 -- 172.21.15.39:0/386444561 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354018860 con 0x7fc3641045e0 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.955+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3641a7d30 con 0x7fc3641045e0 2024-07-04T08:15:10.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.955+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3641a81d0 con 0x7fc3641045e0 2024-07-04T08:15:10.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.956+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3641040a0 con 0x7fc3641045e0 2024-07-04T08:15:10.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.960+0000 7fc35ea00640 1 -- 172.21.15.39:0/386444561 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7fc354010430 con 0x7fc3641045e0 2024-07-04T08:15:10.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.962+0000 7fc35ea00640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc34c0b2870 0x7fc34c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:10.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.962+0000 7fc35ea00640 1 -- 172.21.15.39:0/386444561 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc3540e2e80 con 0x7fc3641045e0 2024-07-04T08:15:10.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.962+0000 7fc369e00640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc34c0b2870 0x7fc34c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:10.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.963+0000 7fc35ea00640 1 -- 172.21.15.39:0/386444561 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc3540e3300 con 0x7fc3641045e0 2024-07-04T08:15:10.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:10.963+0000 7fc369e00640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc34c0b2870 0x7fc34c0b4d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc358031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:11.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:10 smithi039 ceph-mon[32062]: pgmap v479: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-07-04T08:15:11.143 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:11.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:10 smithi050 ceph-mon[36577]: pgmap v479: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-07-04T08:15:11.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.207+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc364100a60 con 0x7fc34c0b2870 2024-07-04T08:15:11.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:10 smithi133 ceph-mon[37050]: pgmap v479: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-07-04T08:15:11.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.501+0000 7fc35ea00640 1 -- 172.21.15.39:0/386444561 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc364100a60 con 0x7fc34c0b2870 2024-07-04T08:15:11.503 INFO:teuthology.orchestra.run.smithi039.stdout:Schedule set for path / 2024-07-04T08:15:11.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc34c0b2870 msgr2=0x7fc34c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:11.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc34c0b2870 0x7fc34c0b4d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc358031400 comp rx=0 tx=0).stop 2024-07-04T08:15:11.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3641045e0 msgr2=0x7fc364101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:11.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3641045e0 0x7fc364101f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc35400e9f0 tx=0x7fc35400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 shutdown_connections 2024-07-04T08:15:11.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc34c0b2870 0x7fc34c0b4d80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc36410d0f0 0x7fc364102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3641045e0 0x7fc364101f00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 --2- 172.21.15.39:0/386444561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc364103c90 0x7fc3641019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.505+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 >> 172.21.15.39:0/386444561 conn(0x7fc3640fd780 msgr2=0x7fc364100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:11.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.506+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 shutdown_connections 2024-07-04T08:15:11.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.506+0000 7fc36b200640 1 -- 172.21.15.39:0/386444561 wait complete. 2024-07-04T08:15:11.538 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-07-04T08:15:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.819+0000 7f61a1400640 1 -- 172.21.15.39:0/3705118688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c1023b0 msgr2=0x7f619c10f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.819+0000 7f61a1400640 1 --2- 172.21.15.39:0/3705118688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c1023b0 0x7f619c10f0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6184009900 tx=0x7f618402a480 comp rx=0 tx=0).stop 2024-07-04T08:15:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.819+0000 7f61a1400640 1 -- 172.21.15.39:0/3705118688 shutdown_connections 2024-07-04T08:15:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.819+0000 7f61a1400640 1 --2- 172.21.15.39:0/3705118688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f619c10f610 0x7f619c111a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.819+0000 7f61a1400640 1 --2- 172.21.15.39:0/3705118688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c1023b0 0x7f619c10f0d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.819+0000 7f61a1400640 1 --2- 172.21.15.39:0/3705118688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f619c101a60 0x7f619c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.819+0000 7f61a1400640 1 -- 172.21.15.39:0/3705118688 >> 172.21.15.39:0/3705118688 conn(0x7f619c079c90 msgr2=0x7f619c07a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.819+0000 7f61a1400640 1 -- 172.21.15.39:0/3705118688 shutdown_connections 2024-07-04T08:15:11.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.819+0000 7f61a1400640 1 -- 172.21.15.39:0/3705118688 wait complete. 2024-07-04T08:15:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.820+0000 7f61a1400640 1 Processor -- start 2024-07-04T08:15:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.820+0000 7f61a1400640 1 -- start start 2024-07-04T08:15:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f61a1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c101a60 0x7f619c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f61a1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f619c1023b0 0x7f619c19c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:11.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f61a1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f619c10f610 0x7f619c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f61a1400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f619c115130 con 0x7f619c1023b0 2024-07-04T08:15:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f61a1400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f619c114fb0 con 0x7f619c10f610 2024-07-04T08:15:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f61a1400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1152b0 con 0x7f619c101a60 2024-07-04T08:15:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f6193e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f619c1023b0 0x7f619c19c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f619a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c101a60 0x7f619c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f619a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c101a60 0x7f619c19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34438/0 (socket says 172.21.15.39:34438) 2024-07-04T08:15:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f6193e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f619c1023b0 0x7f619c19c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34626/0 (socket says 172.21.15.39:34626) 2024-07-04T08:15:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f619a800640 1 -- 172.21.15.39:0/3710511452 learned_addr learned my addr 172.21.15.39:0/3710511452 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.821+0000 7f619b200640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f619c10f610 0x7f619c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.822+0000 7f619a800640 1 -- 172.21.15.39:0/3710511452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f619c10f610 msgr2=0x7f619c1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.822+0000 7f619a800640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f619c10f610 0x7f619c1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.822+0000 7f619a800640 1 -- 172.21.15.39:0/3710511452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f619c1023b0 msgr2=0x7f619c19c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.822+0000 7f619a800640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f619c1023b0 0x7f619c19c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.822+0000 7f619a800640 1 -- 172.21.15.39:0/3710511452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c1a1ae0 con 0x7f619c101a60 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.822+0000 7f619a800640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c101a60 0x7f619c19bbb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9f0 tx=0x7f618c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.823+0000 7f6193400640 1 -- 172.21.15.39:0/3710511452 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c00cd80 con 0x7f619c101a60 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.823+0000 7f6193400640 1 -- 172.21.15.39:0/3710511452 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c0054d0 con 0x7f619c101a60 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.823+0000 7f6193400640 1 -- 172.21.15.39:0/3710511452 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c010690 con 0x7f619c101a60 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.823+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c1a1dd0 con 0x7f619c101a60 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.823+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c1a8560 con 0x7f619c101a60 2024-07-04T08:15:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.825+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619c101e70 con 0x7f619c101a60 2024-07-04T08:15:11.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.826+0000 7f6193400640 1 -- 172.21.15.39:0/3710511452 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7f618c004e90 con 0x7f619c101a60 2024-07-04T08:15:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.828+0000 7f6193400640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61800b28c0 0x7f61800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.828+0000 7f6193400640 1 -- 172.21.15.39:0/3710511452 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f618c0e2a60 con 0x7f619c101a60 2024-07-04T08:15:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.828+0000 7f6193e00640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61800b28c0 0x7f61800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:11.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.829+0000 7f6193e00640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61800b28c0 0x7f61800b4dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6184005ab0 tx=0x7f6184005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:11.831+0000 7f6193400640 1 -- 172.21.15.39:0/3710511452 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f618c0aa330 con 0x7f619c101a60 2024-07-04T08:15:12.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.109+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f619c1098c0 con 0x7f61800b28c0 2024-07-04T08:15:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.225+0000 7f6193400640 1 -- 172.21.15.39:0/3710511452 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f619c1098c0 con 0x7f61800b28c0 2024-07-04T08:15:12.226 INFO:teuthology.orchestra.run.smithi039.stdout:Retention added to path / 2024-07-04T08:15:12.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61800b28c0 msgr2=0x7f61800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:12.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61800b28c0 0x7f61800b4dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6184005ab0 tx=0x7f6184005a20 comp rx=0 tx=0).stop 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c101a60 msgr2=0x7f619c19bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c101a60 0x7f619c19bbb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f618c00e9f0 tx=0x7f618c00edf0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 shutdown_connections 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61800b28c0 0x7f61800b4dd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f619c1023b0 0x7f619c19c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f619c10f610 0x7f619c1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 --2- 172.21.15.39:0/3710511452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f619c101a60 0x7f619c19bbb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.227+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 >> 172.21.15.39:0/3710511452 conn(0x7f619c079c90 msgr2=0x7f619c1091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.228+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 shutdown_connections 2024-07-04T08:15:12.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.228+0000 7f61a1400640 1 -- 172.21.15.39:0/3710511452 wait complete. 2024-07-04T08:15:12.253 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-04T08:15:12.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.516+0000 7f91e9800640 1 -- 172.21.15.39:0/2909475882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e4109100 msgr2=0x7f91e410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:12.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.516+0000 7f91e9800640 1 --2- 172.21.15.39:0/2909475882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e4109100 0x7f91e410f930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009be0 tx=0x7f91dc02a6c0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.516+0000 7f91e9800640 1 -- 172.21.15.39:0/2909475882 shutdown_connections 2024-07-04T08:15:12.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.516+0000 7f91e9800640 1 --2- 172.21.15.39:0/2909475882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e41087b0 0x7f91e4108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.516+0000 7f91e9800640 1 --2- 172.21.15.39:0/2909475882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e4109100 0x7f91e410f930 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.516+0000 7f91e9800640 1 --2- 172.21.15.39:0/2909475882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e4102e80 0x7f91e4103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.516+0000 7f91e9800640 1 -- 172.21.15.39:0/2909475882 >> 172.21.15.39:0/2909475882 conn(0x7f91e40fe8e0 msgr2=0x7f91e4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:12.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.516+0000 7f91e9800640 1 -- 172.21.15.39:0/2909475882 shutdown_connections 2024-07-04T08:15:12.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.516+0000 7f91e9800640 1 -- 172.21.15.39:0/2909475882 wait complete. 2024-07-04T08:15:12.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.518+0000 7f91e9800640 1 Processor -- start 2024-07-04T08:15:12.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.518+0000 7f91e9800640 1 -- start start 2024-07-04T08:15:12.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e4102e80 0x7f91e4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:12.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e41087b0 0x7f91e40751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:12.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e9800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e4109100 0x7f91e40756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:12.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e9800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91e4112dc0 con 0x7f91e4102e80 2024-07-04T08:15:12.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e9800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91e4112c40 con 0x7f91e4109100 2024-07-04T08:15:12.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e9800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91e4112f40 con 0x7f91e41087b0 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e4102e80 0x7f91e4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e4102e80 0x7f91e4074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34640/0 (socket says 172.21.15.39:34640) 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e41087b0 0x7f91e40751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e3e00640 1 -- 172.21.15.39:0/2084073852 learned_addr learned my addr 172.21.15.39:0/2084073852 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.519+0000 7f91e8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e4109100 0x7f91e40756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e3400640 1 -- 172.21.15.39:0/2084073852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e4109100 msgr2=0x7f91e40756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e3400640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e4109100 0x7f91e40756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e3400640 1 -- 172.21.15.39:0/2084073852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e4102e80 msgr2=0x7f91e4074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e3400640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e4102e80 0x7f91e4074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e3400640 1 -- 172.21.15.39:0/2084073852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e41a7a30 con 0x7f91e41087b0 2024-07-04T08:15:12.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e8e00640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e4109100 0x7f91e40756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:12.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e3400640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e41087b0 0x7f91e40751a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f91e406c2d0 tx=0x7f91d000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:12.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e0c00640 1 -- 172.21.15.39:0/2084073852 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d000ccd0 con 0x7f91e41087b0 2024-07-04T08:15:12.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e0c00640 1 -- 172.21.15.39:0/2084073852 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d00035f0 con 0x7f91e41087b0 2024-07-04T08:15:12.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e41a7d20 con 0x7f91e41087b0 2024-07-04T08:15:12.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.520+0000 7f91e0c00640 1 -- 172.21.15.39:0/2084073852 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0010690 con 0x7f91e41087b0 2024-07-04T08:15:12.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.521+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e41a8200 con 0x7f91e41087b0 2024-07-04T08:15:12.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.523+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e41040f0 con 0x7f91e41087b0 2024-07-04T08:15:12.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.526+0000 7f91e0c00640 1 -- 172.21.15.39:0/2084073852 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 149669+0+0 (secure 0 0 0) 0x7f91d0003a80 con 0x7f91e41087b0 2024-07-04T08:15:12.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.528+0000 7f91e0c00640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91c80b28f0 0x7f91c80b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:12.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.529+0000 7f91e0c00640 1 -- 172.21.15.39:0/2084073852 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f91d00e27b0 con 0x7f91e41087b0 2024-07-04T08:15:12.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.529+0000 7f91e3e00640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91c80b28f0 0x7f91c80b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:12.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.530+0000 7f91e3e00640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91c80b28f0 0x7f91c80b4e00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f91e40766b0 tx=0x7f91d4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:12.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.532+0000 7f91e0c00640 1 -- 172.21.15.39:0/2084073852 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f91d0009510 con 0x7f91e41087b0 2024-07-04T08:15:12.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.786+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f91e4062d50 con 0x7f91c80b28f0 2024-07-04T08:15:12.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.798+0000 7f91e0c00640 1 -- 172.21.15.39:0/2084073852 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f91e4062d50 con 0x7f91c80b28f0 2024-07-04T08:15:12.799 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-04T00:00:00", "created": "2024-07-04T08:15:11", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.800+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91c80b28f0 msgr2=0x7f91c80b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.800+0000 7f91e9800640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91c80b28f0 0x7f91c80b4e00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f91e40766b0 tx=0x7f91d4005f90 comp rx=0 tx=0).stop 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.800+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e41087b0 msgr2=0x7f91e40751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.800+0000 7f91e9800640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e41087b0 0x7f91e40751a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f91e406c2d0 tx=0x7f91d000ee50 comp rx=0 tx=0).stop 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.801+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 shutdown_connections 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.801+0000 7f91e9800640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91c80b28f0 0x7f91c80b4e00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.801+0000 7f91e9800640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91e4109100 0x7f91e40756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.801+0000 7f91e9800640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91e41087b0 0x7f91e40751a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.801+0000 7f91e9800640 1 --2- 172.21.15.39:0/2084073852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91e4102e80 0x7f91e4074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:12.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.801+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 >> 172.21.15.39:0/2084073852 conn(0x7f91e40fe8e0 msgr2=0x7f91e4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:12.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.801+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 shutdown_connections 2024-07-04T08:15:12.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:12.801+0000 7f91e9800640 1 -- 172.21.15.39:0/2084073852 wait complete. 2024-07-04T08:15:12.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-04T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:12 smithi039 ceph-mon[32062]: from='client.25441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:12 smithi039 ceph-mon[32062]: pgmap v480: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-07-04T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:12 smithi039 ceph-mon[32062]: from='client.25453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:13.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.103+0000 7f608ee00640 1 -- 172.21.15.39:0/2900323538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 msgr2=0x7f6088108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.103+0000 7f608ee00640 1 --2- 172.21.15.39:0/2900323538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 0x7f6088108bc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6074009900 tx=0x7f607402a400 comp rx=0 tx=0).stop 2024-07-04T08:15:13.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.103+0000 7f608ee00640 1 -- 172.21.15.39:0/2900323538 shutdown_connections 2024-07-04T08:15:13.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.103+0000 7f608ee00640 1 --2- 172.21.15.39:0/2900323538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 0x7f6088108bc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.103+0000 7f608ee00640 1 --2- 172.21.15.39:0/2900323538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6088109100 0x7f608810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.103+0000 7f608ee00640 1 --2- 172.21.15.39:0/2900323538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6088102e80 0x7f6088103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.103+0000 7f608ee00640 1 -- 172.21.15.39:0/2900323538 >> 172.21.15.39:0/2900323538 conn(0x7f60880fe8e0 msgr2=0x7f6088100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:13.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.103+0000 7f608ee00640 1 -- 172.21.15.39:0/2900323538 shutdown_connections 2024-07-04T08:15:13.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.103+0000 7f608ee00640 1 -- 172.21.15.39:0/2900323538 wait complete. 2024-07-04T08:15:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.104+0000 7f608ee00640 1 Processor -- start 2024-07-04T08:15:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.104+0000 7f608ee00640 1 -- start start 2024-07-04T08:15:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608ee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6088102e80 0x7f608819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608ee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 0x7f608819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608ee00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6088109100 0x7f60881a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608ee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6088112dc0 con 0x7f60881087b0 2024-07-04T08:15:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608ee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6088112c40 con 0x7f6088102e80 2024-07-04T08:15:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608ee00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6088112f40 con 0x7f6088109100 2024-07-04T08:15:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 0x7f608819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 0x7f608819bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34662/0 (socket says 172.21.15.39:34662) 2024-07-04T08:15:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608d000640 1 -- 172.21.15.39:0/3035979786 learned_addr learned my addr 172.21.15.39:0/3035979786 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608e400640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6088109100 0x7f60881a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.105+0000 7f608da00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6088102e80 0x7f608819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f608d000640 1 -- 172.21.15.39:0/3035979786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6088109100 msgr2=0x7f60881a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f608d000640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6088109100 0x7f60881a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f608d000640 1 -- 172.21.15.39:0/3035979786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6088102e80 msgr2=0x7f608819ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f608d000640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6088102e80 0x7f608819ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f608d000640 1 -- 172.21.15.39:0/3035979786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60881a1950 con 0x7f60881087b0 2024-07-04T08:15:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f608da00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6088102e80 0x7f608819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:13.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f608d000640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 0x7f608819bfc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f607402a920 tx=0x7f6074031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f6086000640 1 -- 172.21.15.39:0/3035979786 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607403f030 con 0x7f60881087b0 2024-07-04T08:15:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f6086000640 1 -- 172.21.15.39:0/3035979786 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6074004490 con 0x7f60881087b0 2024-07-04T08:15:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60881a1be0 con 0x7f60881087b0 2024-07-04T08:15:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.106+0000 7f6086000640 1 -- 172.21.15.39:0/3035979786 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074035070 con 0x7f60881087b0 2024-07-04T08:15:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.107+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60881a83d0 con 0x7f60881087b0 2024-07-04T08:15:13.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.108+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6088103290 con 0x7f60881087b0 2024-07-04T08:15:13.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.110+0000 7f6086000640 1 -- 172.21.15.39:0/3035979786 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7f6074042030 con 0x7f60881087b0 2024-07-04T08:15:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.111+0000 7f6086000640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60700b28d0 0x7f60700b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.112+0000 7f6086000640 1 -- 172.21.15.39:0/3035979786 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f6074102b50 con 0x7f60881087b0 2024-07-04T08:15:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.112+0000 7f608da00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60700b28d0 0x7f60700b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.113+0000 7f608da00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60700b28d0 0x7f60700b4de0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f607c009fc0 tx=0x7f607c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:13.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.116+0000 7f6086000640 1 -- 172.21.15.39:0/3035979786 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f60740c9c10 con 0x7f60881087b0 2024-07-04T08:15:13.118 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:12 smithi133 ceph-mon[37050]: from='client.25441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:13.118 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:12 smithi133 ceph-mon[37050]: pgmap v480: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-07-04T08:15:13.118 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:12 smithi133 ceph-mon[37050]: from='client.25453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:13.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:12 smithi050 ceph-mon[36577]: from='client.25441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:13.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:12 smithi050 ceph-mon[36577]: pgmap v480: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 10 op/s 2024-07-04T08:15:13.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:12 smithi050 ceph-mon[36577]: from='client.25453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.367+0000 7fbf3fe00640 1 -- 172.21.15.39:0/2231394860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40109190 msgr2=0x7fbf4010f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.367+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/2231394860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40109190 0x7fbf4010f9c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009ca0 tx=0x7fbf3802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.368+0000 7fbf3fe00640 1 -- 172.21.15.39:0/2231394860 shutdown_connections 2024-07-04T08:15:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.368+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/2231394860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf40108840 0x7fbf40108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.368+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/2231394860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40109190 0x7fbf4010f9c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.368+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/2231394860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf40102f00 0x7fbf40103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.368+0000 7fbf3fe00640 1 -- 172.21.15.39:0/2231394860 >> 172.21.15.39:0/2231394860 conn(0x7fbf400fe990 msgr2=0x7fbf40100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.368+0000 7fbf3fe00640 1 -- 172.21.15.39:0/2231394860 shutdown_connections 2024-07-04T08:15:13.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.368+0000 7fbf3fe00640 1 -- 172.21.15.39:0/2231394860 wait complete. 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.369+0000 7fbf3fe00640 1 Processor -- start 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.369+0000 7fbf3fe00640 1 -- start start 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.369+0000 7fbf3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf40102f00 0x7fbf4019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.369+0000 7fbf3fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf40108840 0x7fbf4019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.369+0000 7fbf3fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40109190 0x7fbf401a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.369+0000 7fbf3fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40112ea0 con 0x7fbf40102f00 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.369+0000 7fbf3fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40112d20 con 0x7fbf40108840 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.369+0000 7fbf3fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40113020 con 0x7fbf40109190 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf40102f00 0x7fbf4019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf40102f00 0x7fbf4019ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34678/0 (socket says 172.21.15.39:34678) 2024-07-04T08:15:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3ea00640 1 -- 172.21.15.39:0/3962974505 learned_addr learned my addr 172.21.15.39:0/3962974505 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3e000640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf40108840 0x7fbf4019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3f400640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40109190 0x7fbf401a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3ea00640 1 -- 172.21.15.39:0/3962974505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40109190 msgr2=0x7fbf401a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3ea00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40109190 0x7fbf401a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3ea00640 1 -- 172.21.15.39:0/3962974505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf40108840 msgr2=0x7fbf4019bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3ea00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf40108840 0x7fbf4019bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3ea00640 1 -- 172.21.15.39:0/3962974505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf401a1980 con 0x7fbf40102f00 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.370+0000 7fbf3e000640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf40108840 0x7fbf4019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.371+0000 7fbf3ea00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf40102f00 0x7fbf4019ba10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000e9f0 tx=0x7fbf3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.371+0000 7fbf37400640 1 -- 172.21.15.39:0/3962974505 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000cd80 con 0x7fbf40102f00 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.371+0000 7fbf37400640 1 -- 172.21.15.39:0/3962974505 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf300108e0 con 0x7fbf40102f00 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.371+0000 7fbf3fe00640 1 -- 172.21.15.39:0/3962974505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf401a1c70 con 0x7fbf40102f00 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.371+0000 7fbf37400640 1 -- 172.21.15.39:0/3962974505 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30018980 con 0x7fbf40102f00 2024-07-04T08:15:13.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.371+0000 7fbf3fe00640 1 -- 172.21.15.39:0/3962974505 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf401a83d0 con 0x7fbf40102f00 2024-07-04T08:15:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.375+0000 7fbf4bb84740 1 Processor -- start 2024-07-04T08:15:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.375+0000 7fbf4bb84740 1 -- start start 2024-07-04T08:15:13.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.375+0000 7fbf37400640 1 -- 172.21.15.39:0/3962974505 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7fbf30010430 con 0x7fbf40102f00 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.375+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5cb01f350 0x55a5cb3c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.375+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 0x55a5cb3c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.375+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 0x55a5cb3c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.375+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb2ed8e0 con 0x55a5cb01f350 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.375+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb339ec0 con 0x55a5caffe670 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.375+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb329460 con 0x55a5caff6490 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5cb01f350 0x55a5cb3c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5cb01f350 0x55a5cb3c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34682/0 (socket says 172.21.15.39:34682) 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3f400640 1 -- 172.21.15.39:0/2578611285 learned_addr learned my addr 172.21.15.39:0/2578611285 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:13.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3ea00640 1 --2- 172.21.15.39:0/2578611285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 0x55a5cb3c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3e000640 1 --2- 172.21.15.39:0/2578611285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 0x55a5cb3c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3f400640 1 -- 172.21.15.39:0/2578611285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 msgr2=0x55a5cb3c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:13.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3f400640 1 --2- 172.21.15.39:0/2578611285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 0x55a5cb3c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3f400640 1 -- 172.21.15.39:0/2578611285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 msgr2=0x55a5cb3c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3f400640 1 --2- 172.21.15.39:0/2578611285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 0x55a5cb3c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.376+0000 7fbf3f400640 1 -- 172.21.15.39:0/2578611285 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5cb2c1280 con 0x55a5cb01f350 2024-07-04T08:15:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.377+0000 7fbf3f400640 1 --2- 172.21.15.39:0/2578611285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5cb01f350 0x55a5cb3c37f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009ca0 tx=0x7fbf38009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3305b8ac2d281d5a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.377+0000 7fbf2be00640 1 -- 172.21.15.39:0/2578611285 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38031cf0 con 0x55a5cb01f350 2024-07-04T08:15:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.377+0000 7fbf2be00640 1 -- 172.21.15.39:0/2578611285 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf38036030 con 0x55a5cb01f350 2024-07-04T08:15:13.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.377+0000 7fbf2be00640 1 -- 172.21.15.39:0/2578611285 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38004260 con 0x55a5cb01f350 2024-07-04T08:15:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.378+0000 7fbf37400640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbf240b28d0 0x7fbf240b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.378+0000 7fbf3ea00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbf240b28d0 0x7fbf240b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.378+0000 7fbf37400640 1 -- 172.21.15.39:0/3962974505 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fbf300e2ec0 con 0x7fbf40102f00 2024-07-04T08:15:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.378+0000 7fbf4bb84740 1 -- 172.21.15.39:0/2578611285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5cb01f350 msgr2=0x55a5cb3c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.378+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/2578611285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5cb01f350 0x55a5cb3c37f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009ca0 tx=0x7fbf38009430 comp rx=0 tx=0).stop 2024-07-04T08:15:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.379+0000 7fbf4bb84740 1 -- 172.21.15.39:0/2578611285 shutdown_connections 2024-07-04T08:15:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.379+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/2578611285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 0x55a5cb3c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.379+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/2578611285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 0x55a5cb3c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.379+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/2578611285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5cb01f350 0x55a5cb3c37f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.379+0000 7fbf4bb84740 1 -- 172.21.15.39:0/2578611285 >> 172.21.15.39:0/2578611285 conn(0x55a5cb013740 msgr2=0x55a5cb3c16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:13.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.379+0000 7fbf4bb84740 1 -- 172.21.15.39:0/2578611285 shutdown_connections 2024-07-04T08:15:13.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.379+0000 7fbf4bb84740 1 -- 172.21.15.39:0/2578611285 wait complete. 2024-07-04T08:15:13.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.379+0000 7fbf3ea00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbf240b28d0 0x7fbf240b4de0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbf300e3160 tx=0x7fbf300e6970 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.380+0000 7fbf4bb84740 1 -- start start 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.380+0000 7fbf4bb84740 1 Processor -- start 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.380+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caff6490 0x55a5cb3cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.380+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 0x55a5cb3cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.380+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 0x55a5cb3daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.380+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb2d0760 con 0x55a5caff6490 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.380+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb32d4a0 con 0x55a5caffe670 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.380+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a5cb18d890 con 0x55a5cb01f350 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.380+0000 7fbf4bb84740 10 client.0 _ll_register_callbacks cb 0x55a5cb2d12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:13.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caff6490 0x55a5cb3cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 0x55a5cb3cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caff6490 0x55a5cb3cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34690/0 (socket says 172.21.15.39:34690) 2024-07-04T08:15:13.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3f400640 1 -- 172.21.15.39:0/3235216558 learned_addr learned my addr 172.21.15.39:0/3235216558 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:13.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3f400640 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 0x55a5cb3daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3f400640 1 -- 172.21.15.39:0/3235216558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 msgr2=0x55a5cb3daf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3f400640 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 0x55a5cb3daf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3f400640 1 -- 172.21.15.39:0/3235216558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 msgr2=0x55a5cb3cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3f400640 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 0x55a5cb3cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3f400640 1 -- 172.21.15.39:0/3235216558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5cb08a930 con 0x55a5caff6490 2024-07-04T08:15:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.381+0000 7fbf3e000640 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 0x55a5cb3cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.382+0000 7fbf3f400640 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caff6490 0x55a5cb3cf080 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbf38003e80 tx=0x7fbf38003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f59501ae403381 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:13.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.382+0000 7fbf28c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.382+0000 7fbf28c00640 1 -- 172.21.15.39:0/3235216558 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38058060 con 0x55a5caff6490 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.382+0000 7fbf28c00640 1 -- 172.21.15.39:0/3235216558 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fbf38031860 con 0x55a5caff6490 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.382+0000 7fbf4bb84740 1 -- 172.21.15.39:0/3235216558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf38036030 con 0x55a5caff6490 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.382+0000 7fbf28c00640 1 -- 172.21.15.39:0/3235216558 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf3805f050 con 0x55a5caff6490 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.382+0000 7fbf4bb84740 10 client.15381 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.382+0000 7fbf4bb84740 1 -- 172.21.15.39:0/3235216558 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a5cb3ddb30 con 0x55a5caff6490 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.383+0000 7fbf28c00640 1 -- 172.21.15.39:0/3235216558 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf38036030 con 0x55a5caff6490 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.383+0000 7fbf28c00640 1 -- 172.21.15.39:0/3235216558 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbf38043070 con 0x55a5caff6490 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.383+0000 7fbf4bb84740 10 client.15381 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.383+0000 7fbf4bb84740 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.383+0000 7fbf4bb84740 20 client.15381 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:13.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.383+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a5cafddcd0 0x55a5cb3e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.383+0000 7fbf4bb84740 4 client.15381 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:15:13.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.383+0000 7fbf4bb84740 1 -- 172.21.15.39:0/3235216558 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5cb33be10 con 0x55a5cafddcd0 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.384+0000 7fbf3e000640 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a5cafddcd0 0x55a5cb3e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.384+0000 7fbf3e000640 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a5cafddcd0 0x55a5cb3e0e90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c33d6cb0e207c1 server_cookie=9015c0a5c7e10153 in_seq=0 out_seq=0 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.384+0000 7fbf28c00640 10 client.15381 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf28c00640 1 -- 172.21.15.39:0/3235216558 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a5cb33be10 con 0x55a5cafddcd0 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf28c00640 10 client.15381 handle_command_reply: tid=0 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf4bb84740 1 client.15381 shutdown 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf4bb84740 1 -- 172.21.15.39:0/3235216558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caff6490 msgr2=0x55a5cb3cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caff6490 0x55a5cb3cf080 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbf38003e80 tx=0x7fbf38003d80 comp rx=0 tx=0).stop 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf4bb84740 1 -- 172.21.15.39:0/3235216558 shutdown_connections 2024-07-04T08:15:13.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a5cafddcd0 0x55a5cb3e0e90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 0x55a5cb3daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caffe670 0x55a5cb3cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/3235216558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caff6490 0x55a5cb3cf080 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf4bb84740 1 -- 172.21.15.39:0/3235216558 >> 172.21.15.39:0/3235216558 conn(0x55a5cb013740 msgr2=0x55a5cb3c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.385+0000 7fbf28c00640 0 client.15381 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.386+0000 7fbf4bb84740 1 -- 172.21.15.39:0/3235216558 shutdown_connections 2024-07-04T08:15:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.386+0000 7fbf4bb84740 1 -- 172.21.15.39:0/3235216558 wait complete. 2024-07-04T08:15:13.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.386+0000 7fbf4bb84740 20 client.15381 trim_cache size 0 max 16384 2024-07-04T08:15:13.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.393+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f608810ebe0 con 0x7f60700b28d0 2024-07-04T08:15:13.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.401+0000 7f6086000640 1 -- 172.21.15.39:0/3035979786 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f608810ebe0 con 0x7f60700b28d0 2024-07-04T08:15:13.403 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1m 6m3h 2024-07-04T08:15:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60700b28d0 msgr2=0x7f60700b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60700b28d0 0x7f60700b4de0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f607c009fc0 tx=0x7f607c005f90 comp rx=0 tx=0).stop 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 msgr2=0x7f608819bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 0x7f608819bfc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f607402a920 tx=0x7f6074031a20 comp rx=0 tx=0).stop 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 shutdown_connections 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60700b28d0 0x7f60700b4de0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60881087b0 0x7f608819bfc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6088109100 0x7f60881a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 --2- 172.21.15.39:0/3035979786 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6088102e80 0x7f608819ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 >> 172.21.15.39:0/3035979786 conn(0x7f60880fe8e0 msgr2=0x7f6088105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.404+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 shutdown_connections 2024-07-04T08:15:13.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.405+0000 7f608ee00640 1 -- 172.21.15.39:0/3035979786 wait complete. 2024-07-04T08:15:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.408+0000 7fbf4bb84740 1 Processor -- start 2024-07-04T08:15:13.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.408+0000 7fbf4bb84740 1 -- start start 2024-07-04T08:15:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.408+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caff6490 0x55a5cb3c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.408+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.408+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 0x55a5cb3daf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b2230 con 0x55a5caffe670 2024-07-04T08:15:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b3d70 con 0x55a5caff6490 2024-07-04T08:15:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b2900 con 0x55a5cb01f350 2024-07-04T08:15:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caff6490 0x55a5cb3c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caff6490 0x55a5cb3c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45912/0 (socket says 172.21.15.39:45912) 2024-07-04T08:15:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3f400640 1 -- 172.21.15.39:0/939449418 learned_addr learned my addr 172.21.15.39:0/939449418 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3e000640 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3f400640 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 0x55a5cb3daf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3f400640 1 -- 172.21.15.39:0/939449418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 msgr2=0x55a5cb3daf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3f400640 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 0x55a5cb3daf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3f400640 1 -- 172.21.15.39:0/939449418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 msgr2=0x55a5cb3d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3f400640 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3f400640 1 -- 172.21.15.39:0/939449418 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5cafddf00 con 0x55a5caff6490 2024-07-04T08:15:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.409+0000 7fbf3e000640 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:15:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.410+0000 7fbf3f400640 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caff6490 0x55a5cb3c6f60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf38002230 tx=0x7fbf38005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307b5640a0caa52c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.410+0000 7fbf2b400640 1 -- 172.21.15.39:0/939449418 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38058070 con 0x55a5caff6490 2024-07-04T08:15:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.410+0000 7fbf2b400640 1 -- 172.21.15.39:0/939449418 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf38036070 con 0x55a5caff6490 2024-07-04T08:15:13.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.410+0000 7fbf2b400640 1 -- 172.21.15.39:0/939449418 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf380318c0 con 0x55a5caff6490 2024-07-04T08:15:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.411+0000 7fbf4bb84740 1 -- 172.21.15.39:0/939449418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caff6490 msgr2=0x55a5cb3c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.411+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caff6490 0x55a5cb3c6f60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf38002230 tx=0x7fbf38005fd0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.411+0000 7fbf4bb84740 1 -- 172.21.15.39:0/939449418 shutdown_connections 2024-07-04T08:15:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.411+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.411+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5cb01f350 0x55a5cb3daf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.411+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/939449418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5caff6490 0x55a5cb3c6f60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.411+0000 7fbf4bb84740 1 -- 172.21.15.39:0/939449418 >> 172.21.15.39:0/939449418 conn(0x55a5cb013740 msgr2=0x55a5cb3d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.411+0000 7fbf4bb84740 1 -- 172.21.15.39:0/939449418 shutdown_connections 2024-07-04T08:15:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.411+0000 7fbf4bb84740 1 -- 172.21.15.39:0/939449418 wait complete. 2024-07-04T08:15:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.412+0000 7fbf4bb84740 1 -- start start 2024-07-04T08:15:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.412+0000 7fbf4bb84740 1 Processor -- start 2024-07-04T08:15:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 0x55a5cb3e1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf4bb84740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5cb01f350 0x55a5cb441550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b36a0 con 0x55a5caffe670 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b2fd0 con 0x55a5cb01f350 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf4bb84740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b0210 con 0x55a5caff6490 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf4bb84740 10 client.0 _ll_register_callbacks cb 0x7fbf38009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf3e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 0x55a5cb3e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf3e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3e1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34714/0 (socket says 172.21.15.39:34714) 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf3f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 0x55a5cb3e1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34532/0 (socket says 172.21.15.39:34532) 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf3e000640 1 -- 172.21.15.39:0/1326657914 learned_addr learned my addr 172.21.15.39:0/1326657914 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:13.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.413+0000 7fbf3f400640 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5cb01f350 0x55a5cb441550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.414+0000 7fbf3e000640 1 -- 172.21.15.39:0/1326657914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 msgr2=0x55a5cb3e1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.414+0000 7fbf3e000640 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 0x55a5cb3e1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.414+0000 7fbf3e000640 1 -- 172.21.15.39:0/1326657914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5cb01f350 msgr2=0x55a5cb441550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.414+0000 7fbf3e000640 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5cb01f350 0x55a5cb441550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.414+0000 7fbf3e000640 1 -- 172.21.15.39:0/1326657914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5cb441bc0 con 0x55a5caffe670 2024-07-04T08:15:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.414+0000 7fbf3e000640 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3e1ee0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbf40104f80 tx=0x7fbf2c01f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdda5d7c1f23a06 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.414+0000 7fbf28c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:15:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.415+0000 7fbf28c00640 1 -- 172.21.15.39:0/1326657914 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c01fd10 con 0x55a5caffe670 2024-07-04T08:15:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.415+0000 7fbf28c00640 1 -- 172.21.15.39:0/1326657914 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fbf2c004030 con 0x55a5caffe670 2024-07-04T08:15:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.415+0000 7fbf4bb84740 1 -- 172.21.15.39:0/1326657914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf38036070 con 0x55a5caffe670 2024-07-04T08:15:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.415+0000 7fbf28c00640 1 -- 172.21.15.39:0/1326657914 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf2c004230 con 0x55a5caffe670 2024-07-04T08:15:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.415+0000 7fbf4bb84740 10 client.15387 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.415+0000 7fbf4bb84740 1 -- 172.21.15.39:0/1326657914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a5cb449020 con 0x55a5caffe670 2024-07-04T08:15:13.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.415+0000 7fbf28c00640 1 -- 172.21.15.39:0/1326657914 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf38036070 con 0x55a5caffe670 2024-07-04T08:15:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.416+0000 7fbf28c00640 1 -- 172.21.15.39:0/1326657914 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbf2c004930 con 0x55a5caffe670 2024-07-04T08:15:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.416+0000 7fbf4bb84740 10 client.15387 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.416+0000 7fbf4bb84740 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:15:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.416+0000 7fbf4bb84740 20 client.15387 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.416+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a5cafb3080 0x55a5cb44b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.416+0000 7fbf4bb84740 4 client.15387 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:15:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.416+0000 7fbf4bb84740 1 -- 172.21.15.39:0/1326657914 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a5cb3378b0 con 0x55a5cafb3080 2024-07-04T08:15:13.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.416+0000 7fbf3f400640 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a5cafb3080 0x55a5cb44b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:13.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.417+0000 7fbf28c00640 10 client.15387 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:13.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.417+0000 7fbf3f400640 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a5cafb3080 0x55a5cb44b7c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a2ae0e3617bc588 server_cookie=35ebc3d6bb5e5bc2 in_seq=0 out_seq=0 2024-07-04T08:15:13.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.418+0000 7fbf28c00640 1 -- 172.21.15.39:0/1326657914 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a5cb3378b0 con 0x55a5cafb3080 2024-07-04T08:15:13.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.418+0000 7fbf28c00640 10 client.15387 handle_command_reply: tid=0 2024-07-04T08:15:13.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.418+0000 7fbf4bb84740 1 client.15387 shutdown 2024-07-04T08:15:13.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 -- 172.21.15.39:0/1326657914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 msgr2=0x55a5cb3e1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3e1ee0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbf40104f80 tx=0x7fbf2c01f390 comp rx=0 tx=0).stop 2024-07-04T08:15:13.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 -- 172.21.15.39:0/1326657914 shutdown_connections 2024-07-04T08:15:13.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a5cafb3080 0x55a5cb44b7c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a5caffe670 0x55a5cb3e1ee0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a5cb01f350 0x55a5cb441550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 --2- 172.21.15.39:0/1326657914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a5caff6490 0x55a5cb3e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 -- 172.21.15.39:0/1326657914 >> 172.21.15.39:0/1326657914 conn(0x55a5cb013740 msgr2=0x55a5cb3c44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:13.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf28c00640 0 client.15387 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:13.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 -- 172.21.15.39:0/1326657914 shutdown_connections 2024-07-04T08:15:13.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.419+0000 7fbf4bb84740 1 -- 172.21.15.39:0/1326657914 wait complete. 2024-07-04T08:15:13.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.420+0000 7fbf4bb84740 20 client.15387 trim_cache size 0 max 16384 2024-07-04T08:15:13.423 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:15:13.423 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:15:13.423 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "fab3c176-38e1-47a7-9e54-878dbfb4800a", 2024-07-04T08:15:13.423 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:15:13.423 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:15:13.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.420+0000 7fbf3fe00640 1 -- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbf240b28d0 msgr2=0x7fbf240b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.420+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbf240b28d0 0x7fbf240b4de0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbf300e3160 tx=0x7fbf300e6970 comp rx=0 tx=0).stop 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.420+0000 7fbf3fe00640 1 -- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf40102f00 msgr2=0x7fbf4019ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.420+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf40102f00 0x7fbf4019ba10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000e9f0 tx=0x7fbf3000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.421+0000 7fbf3fe00640 1 -- 172.21.15.39:0/3962974505 shutdown_connections 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.421+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbf240b28d0 0x7fbf240b4de0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.421+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40109190 0x7fbf401a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.421+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbf40108840 0x7fbf4019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.421+0000 7fbf3fe00640 1 --2- 172.21.15.39:0/3962974505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf40102f00 0x7fbf4019ba10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.421+0000 7fbf3fe00640 1 -- 172.21.15.39:0/3962974505 >> 172.21.15.39:0/3962974505 conn(0x7fbf400fe990 msgr2=0x7fbf40107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.422+0000 7fbf3fe00640 1 -- 172.21.15.39:0/3962974505 shutdown_connections 2024-07-04T08:15:13.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:13.422+0000 7fbf3fe00640 1 -- 172.21.15.39:0/3962974505 wait complete. 2024-07-04T08:15:13.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-07-04T08:15:13.505 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-07-04T08:15:13.516 INFO:teuthology.run_tasks:Running task quiescer... 2024-07-04T08:15:13.529 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-04T08:15:13.529 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph pg dump --format=json 2024-07-04T08:15:14.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:13 smithi039 ceph-mon[32062]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:14.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:13 smithi039 ceph-mon[32062]: mgrmap e24: x(active, since 13m), standbys: z, y 2024-07-04T08:15:14.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:13 smithi050 ceph-mon[36577]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:14.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:13 smithi050 ceph-mon[36577]: mgrmap e24: x(active, since 13m), standbys: z, y 2024-07-04T08:15:14.239 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:15:14.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:13 smithi133 ceph-mon[37050]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:14.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:13 smithi133 ceph-mon[37050]: mgrmap e24: x(active, since 13m), standbys: z, y 2024-07-04T08:15:14.363 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:15.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:14 smithi039 ceph-mon[32062]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:15.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:14 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:15:15.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:14 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:15:15.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:14 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:15:15.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:14 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:15:15.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:14 smithi039 ceph-mon[32062]: pgmap v481: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-07-04T08:15:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:14 smithi050 ceph-mon[36577]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:14 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:15:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:14 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:15:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:14 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:15:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:14 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:15:15.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:14 smithi050 ceph-mon[36577]: pgmap v481: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-07-04T08:15:15.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:14 smithi133 ceph-mon[37050]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:15:15.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:14 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:15:15.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:14 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:15:15.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:14 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:15:15.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:14 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:15:15.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:14 smithi133 ceph-mon[37050]: pgmap v481: 129 pgs: 129 active+clean; 652 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 10 op/s 2024-07-04T08:15:15.783 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:16.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.828+0000 7f6054e00640 1 -- 172.21.15.39:0/1982314708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6050075650 msgr2=0x7f605010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:16.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.828+0000 7f6054e00640 1 --2- 172.21.15.39:0/1982314708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6050075650 0x7f605010ef20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6038009900 tx=0x7f603802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:15:16.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.828+0000 7f6054e00640 1 -- 172.21.15.39:0/1982314708 shutdown_connections 2024-07-04T08:15:16.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.828+0000 7f6054e00640 1 --2- 172.21.15.39:0/1982314708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f605010f460 0x7f60501118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:16.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.828+0000 7f6054e00640 1 --2- 172.21.15.39:0/1982314708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6050075650 0x7f605010ef20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:16.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.828+0000 7f6054e00640 1 --2- 172.21.15.39:0/1982314708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6050074d00 0x7f6050075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:16.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.828+0000 7f6054e00640 1 -- 172.21.15.39:0/1982314708 >> 172.21.15.39:0/1982314708 conn(0x7f60500fe990 msgr2=0x7f6050100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:16.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.828+0000 7f6054e00640 1 -- 172.21.15.39:0/1982314708 shutdown_connections 2024-07-04T08:15:16.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.828+0000 7f6054e00640 1 -- 172.21.15.39:0/1982314708 wait complete. 2024-07-04T08:15:16.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.830+0000 7f6054e00640 1 Processor -- start 2024-07-04T08:15:16.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.830+0000 7f6054e00640 1 -- start start 2024-07-04T08:15:16.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.830+0000 7f6054e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6050074d00 0x7f605019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:16.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f6054e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6050075650 0x7f605019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:16.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f6054e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605010f460 0x7f60501a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:16.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f6054e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6050115050 con 0x7f6050074d00 2024-07-04T08:15:16.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f6054e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6050114ed0 con 0x7f605010f460 2024-07-04T08:15:16.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f6054e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f60501151d0 con 0x7f6050075650 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f604f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6050074d00 0x7f605019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f604f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6050074d00 0x7f605019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34730/0 (socket says 172.21.15.39:34730) 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f604f400640 1 -- 172.21.15.39:0/207578527 learned_addr learned my addr 172.21.15.39:0/207578527 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f604ea00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6050075650 0x7f605019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.831+0000 7f604fe00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605010f460 0x7f60501a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.832+0000 7f604f400640 1 -- 172.21.15.39:0/207578527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6050075650 msgr2=0x7f605019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.832+0000 7f604f400640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6050075650 0x7f605019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.832+0000 7f604f400640 1 -- 172.21.15.39:0/207578527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605010f460 msgr2=0x7f60501a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.832+0000 7f604f400640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605010f460 0x7f60501a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:16.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.832+0000 7f604f400640 1 -- 172.21.15.39:0/207578527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60501a1990 con 0x7f6050074d00 2024-07-04T08:15:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.832+0000 7f604f400640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6050074d00 0x7f605019ba20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f604000e9f0 tx=0x7f604000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.832+0000 7f6047e00640 1 -- 172.21.15.39:0/207578527 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604000cd80 con 0x7f6050074d00 2024-07-04T08:15:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.832+0000 7f6047e00640 1 -- 172.21.15.39:0/207578527 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60400108e0 con 0x7f6050074d00 2024-07-04T08:15:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.832+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60501a1c80 con 0x7f6050074d00 2024-07-04T08:15:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.833+0000 7f6047e00640 1 -- 172.21.15.39:0/207578527 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040018980 con 0x7f6050074d00 2024-07-04T08:15:16.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.833+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60501a83e0 con 0x7f6050074d00 2024-07-04T08:15:16.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.834+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6050075110 con 0x7f6050074d00 2024-07-04T08:15:16.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.836+0000 7f6047e00640 1 -- 172.21.15.39:0/207578527 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7f6040010430 con 0x7f6050074d00 2024-07-04T08:15:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.838+0000 7f6047e00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60340b28d0 0x7f60340b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.838+0000 7f6047e00640 1 -- 172.21.15.39:0/207578527 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f60400e2ec0 con 0x7f6050074d00 2024-07-04T08:15:16.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.838+0000 7f604ea00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60340b28d0 0x7f60340b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:16.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.839+0000 7f604ea00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60340b28d0 0x7f60340b4de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6038031440 tx=0x7f6038005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:16.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:16.843+0000 7f6047e00640 1 -- 172.21.15.39:0/207578527 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f60400aa5d0 con 0x7f6050074d00 2024-07-04T08:15:17.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.061+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6050109710 con 0x7f60340b28d0 2024-07-04T08:15:17.063 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:16 smithi039 ceph-mon[32062]: pgmap v482: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-04T08:15:17.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.072+0000 7f6047e00640 1 -- 172.21.15.39:0/207578527 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340952 (secure 0 0 0) 0x7f6050109710 con 0x7f60340b28d0 2024-07-04T08:15:17.078 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:17.080 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:15:17.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60340b28d0 msgr2=0x7f60340b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60340b28d0 0x7f60340b4de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6038031440 tx=0x7f6038005a20 comp rx=0 tx=0).stop 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6050074d00 msgr2=0x7f605019ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6050074d00 0x7f605019ba20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f604000e9f0 tx=0x7f604000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 shutdown_connections 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60340b28d0 0x7f60340b4de0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f605010f460 0x7f60501a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6050075650 0x7f605019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 --2- 172.21.15.39:0/207578527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6050074d00 0x7f605019ba20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 >> 172.21.15.39:0/207578527 conn(0x7f60500fe990 msgr2=0x7f6050109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.079+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 shutdown_connections 2024-07-04T08:15:17.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:17.080+0000 7f6054e00640 1 -- 172.21.15.39:0/207578527 wait complete. 2024-07-04T08:15:17.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:16 smithi050 ceph-mon[36577]: pgmap v482: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-04T08:15:17.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:16 smithi133 ceph-mon[37050]: pgmap v482: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-04T08:15:18.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:17 smithi050 ceph-mon[36577]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:18.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:17 smithi133 ceph-mon[37050]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:18.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:17 smithi039 ceph-mon[32062]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:18.512 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-07-04T08:15:15.691293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1455144,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2929,"num_read_kb":1712,"num_write":612,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":963,"ondisk_log_size":963,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378512,"kb_used_data":12712,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563312,"statfs":{"total":1151940427776,"available":1151552831488,"internally_reserved":0,"allocated":13017088,"data_stored":10069818,"data_compressed":573633,"data_compressed_allocated":774144,"data_compressed_original":4374528,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":450,"num_read_kb":355,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005665"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":311,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:13.460034+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:15:13.460034+0000","last_peered":"2024-07-04T08:15:13.460034+0000","last_clean":"2024-07-04T08:15:13.460034+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:15:13.460034+0000","last_undegraded":"2024-07-04T08:15:13.460034+0000","last_fullsized":"2024-07-04T08:15:13.460034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":251,"num_read_kb":251,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553793+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:12:51.553793+0000","last_peered":"2024-07-04T08:12:51.553793+0000","last_clean":"2024-07-04T08:12:51.553793+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:12:51.553793+0000","last_undegraded":"2024-07-04T08:12:51.553793+0000","last_fullsized":"2024-07-04T08:12:51.553793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:16.118017+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:12:16.118017+0000","last_peered":"2024-07-04T08:12:16.118017+0000","last_clean":"2024-07-04T08:12:16.118017+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:12:16.118017+0000","last_undegraded":"2024-07-04T08:12:16.118017+0000","last_fullsized":"2024-07-04T08:12:16.118017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.698593+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:53.698593+0000","last_peered":"2024-07-04T08:11:53.698593+0000","last_clean":"2024-07-04T08:11:53.698593+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:53.698593+0000","last_undegraded":"2024-07-04T08:11:53.698593+0000","last_fullsized":"2024-07-04T08:11:53.698593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569975+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:12:51.569975+0000","last_peered":"2024-07-04T08:12:51.569975+0000","last_clean":"2024-07-04T08:12:51.569975+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:12:51.569975+0000","last_undegraded":"2024-07-04T08:12:51.569975+0000","last_fullsized":"2024-07-04T08:12:51.569975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:15.756154+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:12:15.756154+0000","last_peered":"2024-07-04T08:12:15.756154+0000","last_clean":"2024-07-04T08:12:15.756154+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:12:15.756154+0000","last_undegraded":"2024-07-04T08:12:15.756154+0000","last_fullsized":"2024-07-04T08:12:15.756154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247127+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:12:05.247127+0000","last_peered":"2024-07-04T08:12:05.247127+0000","last_clean":"2024-07-04T08:12:05.247127+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:12:05.247127+0000","last_undegraded":"2024-07-04T08:12:05.247127+0000","last_fullsized":"2024-07-04T08:12:05.247127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.549821+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:12:51.549821+0000","last_peered":"2024-07-04T08:12:51.549821+0000","last_clean":"2024-07-04T08:12:51.549821+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:12:51.549821+0000","last_undegraded":"2024-07-04T08:12:51.549821+0000","last_fullsized":"2024-07-04T08:12:51.549821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756728+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:12:20.756728+0000","last_peered":"2024-07-04T08:12:20.756728+0000","last_clean":"2024-07-04T08:12:20.756728+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:12:20.756728+0000","last_undegraded":"2024-07-04T08:12:20.756728+0000","last_fullsized":"2024-07-04T08:12:20.756728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247079+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:12:05.247079+0000","last_peered":"2024-07-04T08:12:05.247079+0000","last_clean":"2024-07-04T08:12:05.247079+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:12:05.247079+0000","last_undegraded":"2024-07-04T08:12:05.247079+0000","last_fullsized":"2024-07-04T08:12:05.247079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.535944+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:12:51.535944+0000","last_peered":"2024-07-04T08:12:51.535944+0000","last_clean":"2024-07-04T08:12:51.535944+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:12:51.535944+0000","last_undegraded":"2024-07-04T08:12:51.535944+0000","last_fullsized":"2024-07-04T08:12:51.535944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756706+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:12:20.756706+0000","last_peered":"2024-07-04T08:12:20.756706+0000","last_clean":"2024-07-04T08:12:20.756706+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:12:20.756706+0000","last_undegraded":"2024-07-04T08:12:20.756706+0000","last_fullsized":"2024-07-04T08:12:20.756706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562540+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:12:51.562540+0000","last_peered":"2024-07-04T08:12:51.562540+0000","last_clean":"2024-07-04T08:12:51.562540+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:12:51.562540+0000","last_undegraded":"2024-07-04T08:12:51.562540+0000","last_fullsized":"2024-07-04T08:12:51.562540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844569+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:53.844569+0000","last_peered":"2024-07-04T08:11:53.844569+0000","last_clean":"2024-07-04T08:11:53.844569+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:53.844569+0000","last_undegraded":"2024-07-04T08:11:53.844569+0000","last_fullsized":"2024-07-04T08:11:53.844569+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754878+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:12:05.754878+0000","last_peered":"2024-07-04T08:12:05.754878+0000","last_clean":"2024-07-04T08:12:05.754878+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:12:05.754878+0000","last_undegraded":"2024-07-04T08:12:05.754878+0000","last_fullsized":"2024-07-04T08:12:05.754878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570735+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:12:51.570735+0000","last_peered":"2024-07-04T08:12:51.570735+0000","last_clean":"2024-07-04T08:12:51.570735+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:12:51.570735+0000","last_undegraded":"2024-07-04T08:12:51.570735+0000","last_fullsized":"2024-07-04T08:12:51.570735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691548+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:53.691548+0000","last_peered":"2024-07-04T08:11:53.691548+0000","last_clean":"2024-07-04T08:11:53.691548+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:53.691548+0000","last_undegraded":"2024-07-04T08:11:53.691548+0000","last_fullsized":"2024-07-04T08:11:53.691548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754825+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:12:05.754825+0000","last_peered":"2024-07-04T08:12:05.754825+0000","last_clean":"2024-07-04T08:12:05.754825+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:12:05.754825+0000","last_undegraded":"2024-07-04T08:12:05.754825+0000","last_fullsized":"2024-07-04T08:12:05.754825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566667+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:12:51.566667+0000","last_peered":"2024-07-04T08:12:51.566667+0000","last_clean":"2024-07-04T08:12:51.566667+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:12:51.566667+0000","last_undegraded":"2024-07-04T08:12:51.566667+0000","last_fullsized":"2024-07-04T08:12:51.566667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.697029+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:53.697029+0000","last_peered":"2024-07-04T08:11:53.697029+0000","last_clean":"2024-07-04T08:11:53.697029+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:53.697029+0000","last_undegraded":"2024-07-04T08:11:53.697029+0000","last_fullsized":"2024-07-04T08:11:53.697029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771248+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:11:53.771248+0000","last_peered":"2024-07-04T08:11:53.771248+0000","last_clean":"2024-07-04T08:11:53.771248+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:11:53.771248+0000","last_undegraded":"2024-07-04T08:11:53.771248+0000","last_fullsized":"2024-07-04T08:11:53.771248+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.526597+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:12:51.526597+0000","last_peered":"2024-07-04T08:12:51.526597+0000","last_clean":"2024-07-04T08:12:51.526597+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:12:51.526597+0000","last_undegraded":"2024-07-04T08:12:51.526597+0000","last_fullsized":"2024-07-04T08:12:51.526597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936427+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:53.936427+0000","last_peered":"2024-07-04T08:11:53.936427+0000","last_clean":"2024-07-04T08:11:53.936427+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:53.936427+0000","last_undegraded":"2024-07-04T08:11:53.936427+0000","last_fullsized":"2024-07-04T08:11:53.936427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754887+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:12:05.754887+0000","last_peered":"2024-07-04T08:12:05.754887+0000","last_clean":"2024-07-04T08:12:05.754887+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:12:05.754887+0000","last_undegraded":"2024-07-04T08:12:05.754887+0000","last_fullsized":"2024-07-04T08:12:05.754887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568004+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:12:51.568004+0000","last_peered":"2024-07-04T08:12:51.568004+0000","last_clean":"2024-07-04T08:12:51.568004+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:12:51.568004+0000","last_undegraded":"2024-07-04T08:12:51.568004+0000","last_fullsized":"2024-07-04T08:12:51.568004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692160+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:53.692160+0000","last_peered":"2024-07-04T08:11:53.692160+0000","last_clean":"2024-07-04T08:11:53.692160+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:53.692160+0000","last_undegraded":"2024-07-04T08:11:53.692160+0000","last_fullsized":"2024-07-04T08:11:53.692160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115754+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:11:56.115754+0000","last_peered":"2024-07-04T08:11:56.115754+0000","last_clean":"2024-07-04T08:11:56.115754+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:11:56.115754+0000","last_undegraded":"2024-07-04T08:11:56.115754+0000","last_fullsized":"2024-07-04T08:11:56.115754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.556576+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:12:51.556576+0000","last_peered":"2024-07-04T08:12:51.556576+0000","last_clean":"2024-07-04T08:12:51.556576+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:12:51.556576+0000","last_undegraded":"2024-07-04T08:12:51.556576+0000","last_fullsized":"2024-07-04T08:12:51.556576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'101","reported_seq":246,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:13.400609+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:15:13.400609+0000","last_peered":"2024-07-04T08:15:13.400609+0000","last_clean":"2024-07-04T08:15:13.400609+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:15:13.400609+0000","last_undegraded":"2024-07-04T08:15:13.400609+0000","last_fullsized":"2024-07-04T08:15:13.400609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":185,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.322378+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:11:54.322378+0000","last_peered":"2024-07-04T08:11:54.322378+0000","last_clean":"2024-07-04T08:11:54.322378+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:11:54.322378+0000","last_undegraded":"2024-07-04T08:11:54.322378+0000","last_fullsized":"2024-07-04T08:11:54.322378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540856+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:12:51.540856+0000","last_peered":"2024-07-04T08:12:51.540856+0000","last_clean":"2024-07-04T08:12:51.540856+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:12:51.540856+0000","last_undegraded":"2024-07-04T08:12:51.540856+0000","last_fullsized":"2024-07-04T08:12:51.540856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771431+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:53.771431+0000","last_peered":"2024-07-04T08:11:53.771431+0000","last_clean":"2024-07-04T08:11:53.771431+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:53.771431+0000","last_undegraded":"2024-07-04T08:11:53.771431+0000","last_fullsized":"2024-07-04T08:11:53.771431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.238979+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:11:56.238979+0000","last_peered":"2024-07-04T08:11:56.238979+0000","last_clean":"2024-07-04T08:11:56.238979+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:11:56.238979+0000","last_undegraded":"2024-07-04T08:11:56.238979+0000","last_fullsized":"2024-07-04T08:11:56.238979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542562+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:12:51.542562+0000","last_peered":"2024-07-04T08:12:51.542562+0000","last_clean":"2024-07-04T08:12:51.542562+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:12:51.542562+0000","last_undegraded":"2024-07-04T08:12:51.542562+0000","last_fullsized":"2024-07-04T08:12:51.542562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993415+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:11:53.993415+0000","last_peered":"2024-07-04T08:11:53.993415+0000","last_clean":"2024-07-04T08:11:53.993415+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:11:53.993415+0000","last_undegraded":"2024-07-04T08:11:53.993415+0000","last_fullsized":"2024-07-04T08:11:53.993415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:00.246372+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:12:00.246372+0000","last_peered":"2024-07-04T08:12:00.246372+0000","last_clean":"2024-07-04T08:12:00.246372+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:12:00.246372+0000","last_undegraded":"2024-07-04T08:12:00.246372+0000","last_fullsized":"2024-07-04T08:12:00.246372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.543737+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:12:51.543737+0000","last_peered":"2024-07-04T08:12:51.543737+0000","last_clean":"2024-07-04T08:12:51.543737+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:12:51.543737+0000","last_undegraded":"2024-07-04T08:12:51.543737+0000","last_fullsized":"2024-07-04T08:12:51.543737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:09.452758+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:15:09.452758+0000","last_peered":"2024-07-04T08:15:09.452758+0000","last_clean":"2024-07-04T08:15:09.452758+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:15:09.452758+0000","last_undegraded":"2024-07-04T08:15:09.452758+0000","last_fullsized":"2024-07-04T08:15:09.452758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":248,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:58.137211+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:11:58.137211+0000","last_peered":"2024-07-04T08:11:58.137211+0000","last_clean":"2024-07-04T08:11:58.137211+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:11:58.137211+0000","last_undegraded":"2024-07-04T08:11:58.137211+0000","last_fullsized":"2024-07-04T08:11:58.137211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.516953+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:12:51.516953+0000","last_peered":"2024-07-04T08:12:51.516953+0000","last_clean":"2024-07-04T08:12:51.516953+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:12:51.516953+0000","last_undegraded":"2024-07-04T08:12:51.516953+0000","last_fullsized":"2024-07-04T08:12:51.516953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":300,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:11.520009+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:15:11.520009+0000","last_peered":"2024-07-04T08:15:11.520009+0000","last_clean":"2024-07-04T08:15:11.520009+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:15:11.520009+0000","last_undegraded":"2024-07-04T08:15:11.520009+0000","last_fullsized":"2024-07-04T08:15:11.520009+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":248,"num_read_kb":248,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138305+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:12:03.138305+0000","last_peered":"2024-07-04T08:12:03.138305+0000","last_clean":"2024-07-04T08:12:03.138305+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:12:03.138305+0000","last_undegraded":"2024-07-04T08:12:03.138305+0000","last_fullsized":"2024-07-04T08:12:03.138305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534873+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:12:51.534873+0000","last_peered":"2024-07-04T08:12:51.534873+0000","last_clean":"2024-07-04T08:12:51.534873+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:12:51.534873+0000","last_undegraded":"2024-07-04T08:12:51.534873+0000","last_fullsized":"2024-07-04T08:12:51.534873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851437+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:53.851437+0000","last_peered":"2024-07-04T08:11:53.851437+0000","last_clean":"2024-07-04T08:11:53.851437+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:53.851437+0000","last_undegraded":"2024-07-04T08:11:53.851437+0000","last_fullsized":"2024-07-04T08:11:53.851437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975520+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:12:04.975520+0000","last_peered":"2024-07-04T08:12:04.975520+0000","last_clean":"2024-07-04T08:12:04.975520+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:12:04.975520+0000","last_undegraded":"2024-07-04T08:12:04.975520+0000","last_fullsized":"2024-07-04T08:12:04.975520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.558356+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:12:51.558356+0000","last_peered":"2024-07-04T08:12:51.558356+0000","last_clean":"2024-07-04T08:12:51.558356+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:12:51.558356+0000","last_undegraded":"2024-07-04T08:12:51.558356+0000","last_fullsized":"2024-07-04T08:12:51.558356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691883+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:53.691883+0000","last_peered":"2024-07-04T08:11:53.691883+0000","last_clean":"2024-07-04T08:11:53.691883+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:53.691883+0000","last_undegraded":"2024-07-04T08:11:53.691883+0000","last_fullsized":"2024-07-04T08:11:53.691883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":120,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:46.091830+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:12:46.091830+0000","last_peered":"2024-07-04T08:12:46.091830+0000","last_clean":"2024-07-04T08:12:46.091830+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:12:46.091830+0000","last_undegraded":"2024-07-04T08:12:46.091830+0000","last_fullsized":"2024-07-04T08:12:46.091830+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.085858+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:11:56.085858+0000","last_peered":"2024-07-04T08:11:56.085858+0000","last_clean":"2024-07-04T08:11:56.085858+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:11:56.085858+0000","last_undegraded":"2024-07-04T08:11:56.085858+0000","last_fullsized":"2024-07-04T08:11:56.085858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553863+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:12:51.553863+0000","last_peered":"2024-07-04T08:12:51.553863+0000","last_clean":"2024-07-04T08:12:51.553863+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:12:51.553863+0000","last_undegraded":"2024-07-04T08:12:51.553863+0000","last_fullsized":"2024-07-04T08:12:51.553863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":271,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:12.211979+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:15:12.211979+0000","last_peered":"2024-07-04T08:15:12.211979+0000","last_clean":"2024-07-04T08:15:12.211979+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:15:12.211979+0000","last_undegraded":"2024-07-04T08:15:12.211979+0000","last_fullsized":"2024-07-04T08:15:12.211979+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":250,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.151807+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:12:04.151807+0000","last_peered":"2024-07-04T08:12:04.151807+0000","last_clean":"2024-07-04T08:12:04.151807+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:12:04.151807+0000","last_undegraded":"2024-07-04T08:12:04.151807+0000","last_fullsized":"2024-07-04T08:12:04.151807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.561500+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:12:51.561500+0000","last_peered":"2024-07-04T08:12:51.561500+0000","last_clean":"2024-07-04T08:12:51.561500+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:12:51.561500+0000","last_undegraded":"2024-07-04T08:12:51.561500+0000","last_fullsized":"2024-07-04T08:12:51.561500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'62","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:12.794616+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:15:12.794616+0000","last_peered":"2024-07-04T08:15:12.794616+0000","last_clean":"2024-07-04T08:15:12.794616+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:15:12.794616+0000","last_undegraded":"2024-07-04T08:15:12.794616+0000","last_fullsized":"2024-07-04T08:15:12.794616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:09.641951+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:12:09.641951+0000","last_peered":"2024-07-04T08:12:09.641951+0000","last_clean":"2024-07-04T08:12:09.641951+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:12:09.641951+0000","last_undegraded":"2024-07-04T08:12:09.641951+0000","last_fullsized":"2024-07-04T08:12:09.641951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552528+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:12:51.552528+0000","last_peered":"2024-07-04T08:12:51.552528+0000","last_clean":"2024-07-04T08:12:51.552528+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:12:51.552528+0000","last_undegraded":"2024-07-04T08:12:51.552528+0000","last_fullsized":"2024-07-04T08:12:51.552528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995043+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:53.995043+0000","last_peered":"2024-07-04T08:11:53.995043+0000","last_clean":"2024-07-04T08:11:53.995043+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:53.995043+0000","last_undegraded":"2024-07-04T08:11:53.995043+0000","last_fullsized":"2024-07-04T08:11:53.995043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975432+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:12:04.975432+0000","last_peered":"2024-07-04T08:12:04.975432+0000","last_clean":"2024-07-04T08:12:04.975432+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:12:04.975432+0000","last_undegraded":"2024-07-04T08:12:04.975432+0000","last_fullsized":"2024-07-04T08:12:04.975432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531965+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:12:51.531965+0000","last_peered":"2024-07-04T08:12:51.531965+0000","last_clean":"2024-07-04T08:12:51.531965+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:12:51.531965+0000","last_undegraded":"2024-07-04T08:12:51.531965+0000","last_fullsized":"2024-07-04T08:12:51.531965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771354+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:53.771354+0000","last_peered":"2024-07-04T08:11:53.771354+0000","last_clean":"2024-07-04T08:11:53.771354+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:53.771354+0000","last_undegraded":"2024-07-04T08:11:53.771354+0000","last_fullsized":"2024-07-04T08:11:53.771354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975473+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:12:04.975473+0000","last_peered":"2024-07-04T08:12:04.975473+0000","last_clean":"2024-07-04T08:12:04.975473+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:12:04.975473+0000","last_undegraded":"2024-07-04T08:12:04.975473+0000","last_fullsized":"2024-07-04T08:12:04.975473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542769+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:12:51.542769+0000","last_peered":"2024-07-04T08:12:51.542769+0000","last_clean":"2024-07-04T08:12:51.542769+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:12:51.542769+0000","last_undegraded":"2024-07-04T08:12:51.542769+0000","last_fullsized":"2024-07-04T08:12:51.542769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:55.245889+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:55.245889+0000","last_peered":"2024-07-04T08:11:55.245889+0000","last_clean":"2024-07-04T08:11:55.245889+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:55.245889+0000","last_undegraded":"2024-07-04T08:11:55.245889+0000","last_fullsized":"2024-07-04T08:11:55.245889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.141070+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:12:18.141070+0000","last_peered":"2024-07-04T08:12:18.141070+0000","last_clean":"2024-07-04T08:12:18.141070+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:12:18.141070+0000","last_undegraded":"2024-07-04T08:12:18.141070+0000","last_fullsized":"2024-07-04T08:12:18.141070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552028+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:12:51.552028+0000","last_peered":"2024-07-04T08:12:51.552028+0000","last_clean":"2024-07-04T08:12:51.552028+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:12:51.552028+0000","last_undegraded":"2024-07-04T08:12:51.552028+0000","last_fullsized":"2024-07-04T08:12:51.552028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":302,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:09.515802+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:15:09.515802+0000","last_peered":"2024-07-04T08:15:09.515802+0000","last_clean":"2024-07-04T08:15:09.515802+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:15:09.515802+0000","last_undegraded":"2024-07-04T08:15:09.515802+0000","last_fullsized":"2024-07-04T08:15:09.515802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":250,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.666330+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:12:18.666330+0000","last_peered":"2024-07-04T08:12:18.666330+0000","last_clean":"2024-07-04T08:12:18.666330+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:12:18.666330+0000","last_undegraded":"2024-07-04T08:12:18.666330+0000","last_fullsized":"2024-07-04T08:12:18.666330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564982+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:12:51.564982+0000","last_peered":"2024-07-04T08:12:51.564982+0000","last_clean":"2024-07-04T08:12:51.564982+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:12:51.564982+0000","last_undegraded":"2024-07-04T08:12:51.564982+0000","last_fullsized":"2024-07-04T08:12:51.564982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.974209+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:54.974209+0000","last_peered":"2024-07-04T08:11:54.974209+0000","last_clean":"2024-07-04T08:11:54.974209+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:54.974209+0000","last_undegraded":"2024-07-04T08:11:54.974209+0000","last_fullsized":"2024-07-04T08:11:54.974209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:11.240822+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:12:11.240822+0000","last_peered":"2024-07-04T08:12:11.240822+0000","last_clean":"2024-07-04T08:12:11.240822+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:12:11.240822+0000","last_undegraded":"2024-07-04T08:12:11.240822+0000","last_fullsized":"2024-07-04T08:12:11.240822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562193+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:12:51.562193+0000","last_peered":"2024-07-04T08:12:51.562193+0000","last_clean":"2024-07-04T08:12:51.562193+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:12:51.562193+0000","last_undegraded":"2024-07-04T08:12:51.562193+0000","last_fullsized":"2024-07-04T08:12:51.562193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115841+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:56.115841+0000","last_peered":"2024-07-04T08:11:56.115841+0000","last_clean":"2024-07-04T08:11:56.115841+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:56.115841+0000","last_undegraded":"2024-07-04T08:11:56.115841+0000","last_fullsized":"2024-07-04T08:11:56.115841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.904331+0000","last_change":"2024-07-04T08:11:53.997034+0000","last_active":"2024-07-04T08:11:56.904331+0000","last_peered":"2024-07-04T08:11:56.904331+0000","last_clean":"2024-07-04T08:11:56.904331+0000","last_became_active":"2024-07-04T08:11:53.996456+0000","last_became_peered":"2024-07-04T08:11:53.996456+0000","last_unstale":"2024-07-04T08:11:56.904331+0000","last_undegraded":"2024-07-04T08:11:56.904331+0000","last_fullsized":"2024-07-04T08:11:56.904331+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.140549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.521874+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:12:51.521874+0000","last_peered":"2024-07-04T08:12:51.521874+0000","last_clean":"2024-07-04T08:12:51.521874+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:12:51.521874+0000","last_undegraded":"2024-07-04T08:12:51.521874+0000","last_fullsized":"2024-07-04T08:12:51.521874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":591,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:11.287710+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:15:11.287710+0000","last_peered":"2024-07-04T08:15:11.287710+0000","last_clean":"2024-07-04T08:15:11.287710+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:15:11.287710+0000","last_undegraded":"2024-07-04T08:15:11.287710+0000","last_fullsized":"2024-07-04T08:15:11.287710+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70064,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":504,"num_read_kb":321,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800726+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:12:14.800726+0000","last_peered":"2024-07-04T08:12:14.800726+0000","last_clean":"2024-07-04T08:12:14.800726+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:12:14.800726+0000","last_undegraded":"2024-07-04T08:12:14.800726+0000","last_fullsized":"2024-07-04T08:12:14.800726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541913+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:12:51.541913+0000","last_peered":"2024-07-04T08:12:51.541913+0000","last_clean":"2024-07-04T08:12:51.541913+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:12:51.541913+0000","last_undegraded":"2024-07-04T08:12:51.541913+0000","last_fullsized":"2024-07-04T08:12:51.541913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974814+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:59.974814+0000","last_peered":"2024-07-04T08:11:59.974814+0000","last_clean":"2024-07-04T08:11:59.974814+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:59.974814+0000","last_undegraded":"2024-07-04T08:11:59.974814+0000","last_fullsized":"2024-07-04T08:11:59.974814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:10.755408+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:12:10.755408+0000","last_peered":"2024-07-04T08:12:10.755408+0000","last_clean":"2024-07-04T08:12:10.755408+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:12:10.755408+0000","last_undegraded":"2024-07-04T08:12:10.755408+0000","last_fullsized":"2024-07-04T08:12:10.755408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569907+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:12:51.569907+0000","last_peered":"2024-07-04T08:12:51.569907+0000","last_clean":"2024-07-04T08:12:51.569907+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:12:51.569907+0000","last_undegraded":"2024-07-04T08:12:51.569907+0000","last_fullsized":"2024-07-04T08:12:51.569907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:01.086478+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:12:01.086478+0000","last_peered":"2024-07-04T08:12:01.086478+0000","last_clean":"2024-07-04T08:12:01.086478+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:12:01.086478+0000","last_undegraded":"2024-07-04T08:12:01.086478+0000","last_fullsized":"2024-07-04T08:12:01.086478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800668+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:12:14.800668+0000","last_peered":"2024-07-04T08:12:14.800668+0000","last_clean":"2024-07-04T08:12:14.800668+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:12:14.800668+0000","last_undegraded":"2024-07-04T08:12:14.800668+0000","last_fullsized":"2024-07-04T08:12:14.800668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.547189+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:12:51.547189+0000","last_peered":"2024-07-04T08:12:51.547189+0000","last_clean":"2024-07-04T08:12:51.547189+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:12:51.547189+0000","last_undegraded":"2024-07-04T08:12:51.547189+0000","last_fullsized":"2024-07-04T08:12:51.547189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":542,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:11.210328+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:15:11.210328+0000","last_peered":"2024-07-04T08:15:11.210328+0000","last_clean":"2024-07-04T08:15:11.210328+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:15:11.210328+0000","last_undegraded":"2024-07-04T08:15:11.210328+0000","last_fullsized":"2024-07-04T08:15:11.210328+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":492,"num_read_kb":492,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.326445+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:12:14.326445+0000","last_peered":"2024-07-04T08:12:14.326445+0000","last_clean":"2024-07-04T08:12:14.326445+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:12:14.326445+0000","last_undegraded":"2024-07-04T08:12:14.326445+0000","last_fullsized":"2024-07-04T08:12:14.326445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.639390+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:11:54.639390+0000","last_peered":"2024-07-04T08:11:54.639390+0000","last_clean":"2024-07-04T08:11:54.639390+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:11:54.639390+0000","last_undegraded":"2024-07-04T08:11:54.639390+0000","last_fullsized":"2024-07-04T08:11:54.639390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564562+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:12:51.564562+0000","last_peered":"2024-07-04T08:12:51.564562+0000","last_clean":"2024-07-04T08:12:51.564562+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:12:51.564562+0000","last_undegraded":"2024-07-04T08:12:51.564562+0000","last_fullsized":"2024-07-04T08:12:51.564562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974768+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:59.974768+0000","last_peered":"2024-07-04T08:11:59.974768+0000","last_clean":"2024-07-04T08:11:59.974768+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:59.974768+0000","last_undegraded":"2024-07-04T08:11:59.974768+0000","last_fullsized":"2024-07-04T08:11:59.974768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.239030+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:11:56.239030+0000","last_peered":"2024-07-04T08:11:56.239030+0000","last_clean":"2024-07-04T08:11:56.239030+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:11:56.239030+0000","last_undegraded":"2024-07-04T08:11:56.239030+0000","last_fullsized":"2024-07-04T08:11:56.239030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565645+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:12:51.565645+0000","last_peered":"2024-07-04T08:12:51.565645+0000","last_clean":"2024-07-04T08:12:51.565645+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:12:51.565645+0000","last_undegraded":"2024-07-04T08:12:51.565645+0000","last_fullsized":"2024-07-04T08:12:51.565645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138191+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:12:03.138191+0000","last_peered":"2024-07-04T08:12:03.138191+0000","last_clean":"2024-07-04T08:12:03.138191+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:12:03.138191+0000","last_undegraded":"2024-07-04T08:12:03.138191+0000","last_fullsized":"2024-07-04T08:12:03.138191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:06.905597+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:12:06.905597+0000","last_peered":"2024-07-04T08:12:06.905597+0000","last_clean":"2024-07-04T08:12:06.905597+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:12:06.905597+0000","last_undegraded":"2024-07-04T08:12:06.905597+0000","last_fullsized":"2024-07-04T08:12:06.905597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.554669+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:12:51.554669+0000","last_peered":"2024-07-04T08:12:51.554669+0000","last_clean":"2024-07-04T08:12:51.554669+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:12:51.554669+0000","last_undegraded":"2024-07-04T08:12:51.554669+0000","last_fullsized":"2024-07-04T08:12:51.554669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118570+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:12:21.118570+0000","last_peered":"2024-07-04T08:12:21.118570+0000","last_clean":"2024-07-04T08:12:21.118570+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:12:21.118570+0000","last_undegraded":"2024-07-04T08:12:21.118570+0000","last_fullsized":"2024-07-04T08:12:21.118570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":302,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:13.257558+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:15:13.257558+0000","last_peered":"2024-07-04T08:15:13.257558+0000","last_clean":"2024-07-04T08:15:13.257558+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:15:13.257558+0000","last_undegraded":"2024-07-04T08:15:13.257558+0000","last_fullsized":"2024-07-04T08:15:13.257558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":251,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.519199+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:12:51.519199+0000","last_peered":"2024-07-04T08:12:51.519199+0000","last_clean":"2024-07-04T08:12:51.519199+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:12:51.519199+0000","last_undegraded":"2024-07-04T08:12:51.519199+0000","last_fullsized":"2024-07-04T08:12:51.519199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118593+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:12:21.118593+0000","last_peered":"2024-07-04T08:12:21.118593+0000","last_clean":"2024-07-04T08:12:21.118593+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:12:21.118593+0000","last_undegraded":"2024-07-04T08:12:21.118593+0000","last_fullsized":"2024-07-04T08:12:21.118593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534235+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:12:51.534235+0000","last_peered":"2024-07-04T08:12:51.534235+0000","last_clean":"2024-07-04T08:12:51.534235+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:12:51.534235+0000","last_undegraded":"2024-07-04T08:12:51.534235+0000","last_fullsized":"2024-07-04T08:12:51.534235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.539681+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:12:51.539681+0000","last_peered":"2024-07-04T08:12:51.539681+0000","last_clean":"2024-07-04T08:12:51.539681+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:12:51.539681+0000","last_undegraded":"2024-07-04T08:12:51.539681+0000","last_fullsized":"2024-07-04T08:12:51.539681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566399+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:12:51.566399+0000","last_peered":"2024-07-04T08:12:51.566399+0000","last_clean":"2024-07-04T08:12:51.566399+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:12:51.566399+0000","last_undegraded":"2024-07-04T08:12:51.566399+0000","last_fullsized":"2024-07-04T08:12:51.566399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567851+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:12:51.567851+0000","last_peered":"2024-07-04T08:12:51.567851+0000","last_clean":"2024-07-04T08:12:51.567851+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:12:51.567851+0000","last_undegraded":"2024-07-04T08:12:51.567851+0000","last_fullsized":"2024-07-04T08:12:51.567851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569222+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:12:51.569222+0000","last_peered":"2024-07-04T08:12:51.569222+0000","last_clean":"2024-07-04T08:12:51.569222+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:12:51.569222+0000","last_undegraded":"2024-07-04T08:12:51.569222+0000","last_fullsized":"2024-07-04T08:12:51.569222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532408+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:12:51.532408+0000","last_peered":"2024-07-04T08:12:51.532408+0000","last_clean":"2024-07-04T08:12:51.532408+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:12:51.532408+0000","last_undegraded":"2024-07-04T08:12:51.532408+0000","last_fullsized":"2024-07-04T08:12:51.532408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540538+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:12:51.540538+0000","last_peered":"2024-07-04T08:12:51.540538+0000","last_clean":"2024-07-04T08:12:51.540538+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:12:51.540538+0000","last_undegraded":"2024-07-04T08:12:51.540538+0000","last_fullsized":"2024-07-04T08:12:51.540538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552348+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:12:51.552348+0000","last_peered":"2024-07-04T08:12:51.552348+0000","last_clean":"2024-07-04T08:12:51.552348+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:12:51.552348+0000","last_undegraded":"2024-07-04T08:12:51.552348+0000","last_fullsized":"2024-07-04T08:12:51.552348+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531503+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:12:51.531503+0000","last_peered":"2024-07-04T08:12:51.531503+0000","last_clean":"2024-07-04T08:12:51.531503+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:12:51.531503+0000","last_undegraded":"2024-07-04T08:12:51.531503+0000","last_fullsized":"2024-07-04T08:12:51.531503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.555574+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:12:51.555574+0000","last_peered":"2024-07-04T08:12:51.555574+0000","last_clean":"2024-07-04T08:12:51.555574+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:12:51.555574+0000","last_undegraded":"2024-07-04T08:12:51.555574+0000","last_fullsized":"2024-07-04T08:12:51.555574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560802+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:12:51.560802+0000","last_peered":"2024-07-04T08:12:51.560802+0000","last_clean":"2024-07-04T08:12:51.560802+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:12:51.560802+0000","last_undegraded":"2024-07-04T08:12:51.560802+0000","last_fullsized":"2024-07-04T08:12:51.560802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570948+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:12:51.570948+0000","last_peered":"2024-07-04T08:12:51.570948+0000","last_clean":"2024-07-04T08:12:51.570948+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:12:51.570948+0000","last_undegraded":"2024-07-04T08:12:51.570948+0000","last_fullsized":"2024-07-04T08:12:51.570948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532752+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:12:51.532752+0000","last_peered":"2024-07-04T08:12:51.532752+0000","last_clean":"2024-07-04T08:12:51.532752+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:12:51.532752+0000","last_undegraded":"2024-07-04T08:12:51.532752+0000","last_fullsized":"2024-07-04T08:12:51.532752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552705+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:12:51.552705+0000","last_peered":"2024-07-04T08:12:51.552705+0000","last_clean":"2024-07-04T08:12:51.552705+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:12:51.552705+0000","last_undegraded":"2024-07-04T08:12:51.552705+0000","last_fullsized":"2024-07-04T08:12:51.552705+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541640+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:12:51.541640+0000","last_peered":"2024-07-04T08:12:51.541640+0000","last_clean":"2024-07-04T08:12:51.541640+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:12:51.541640+0000","last_undegraded":"2024-07-04T08:12:51.541640+0000","last_fullsized":"2024-07-04T08:12:51.541640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569708+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:12:51.569708+0000","last_peered":"2024-07-04T08:12:51.569708+0000","last_clean":"2024-07-04T08:12:51.569708+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:12:51.569708+0000","last_undegraded":"2024-07-04T08:12:51.569708+0000","last_fullsized":"2024-07-04T08:12:51.569708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.571140+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:12:51.571140+0000","last_peered":"2024-07-04T08:12:51.571140+0000","last_clean":"2024-07-04T08:12:51.571140+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:12:51.571140+0000","last_undegraded":"2024-07-04T08:12:51.571140+0000","last_fullsized":"2024-07-04T08:12:51.571140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552227+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:12:51.552227+0000","last_peered":"2024-07-04T08:12:51.552227+0000","last_clean":"2024-07-04T08:12:51.552227+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:12:51.552227+0000","last_undegraded":"2024-07-04T08:12:51.552227+0000","last_fullsized":"2024-07-04T08:12:51.552227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567284+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:12:51.567284+0000","last_peered":"2024-07-04T08:12:51.567284+0000","last_clean":"2024-07-04T08:12:51.567284+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:12:51.567284+0000","last_undegraded":"2024-07-04T08:12:51.567284+0000","last_fullsized":"2024-07-04T08:12:51.567284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551582+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:12:51.551582+0000","last_peered":"2024-07-04T08:12:51.551582+0000","last_clean":"2024-07-04T08:12:51.551582+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:12:51.551582+0000","last_undegraded":"2024-07-04T08:12:51.551582+0000","last_fullsized":"2024-07-04T08:12:51.551582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534085+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:12:51.534085+0000","last_peered":"2024-07-04T08:12:51.534085+0000","last_clean":"2024-07-04T08:12:51.534085+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:12:51.534085+0000","last_undegraded":"2024-07-04T08:12:51.534085+0000","last_fullsized":"2024-07-04T08:12:51.534085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553666+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:12:51.553666+0000","last_peered":"2024-07-04T08:12:51.553666+0000","last_clean":"2024-07-04T08:12:51.553666+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:12:51.553666+0000","last_undegraded":"2024-07-04T08:12:51.553666+0000","last_fullsized":"2024-07-04T08:12:51.553666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540373+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:12:51.540373+0000","last_peered":"2024-07-04T08:12:51.540373+0000","last_clean":"2024-07-04T08:12:51.540373+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:12:51.540373+0000","last_undegraded":"2024-07-04T08:12:51.540373+0000","last_fullsized":"2024-07-04T08:12:51.540373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541173+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:12:51.541173+0000","last_peered":"2024-07-04T08:12:51.541173+0000","last_clean":"2024-07-04T08:12:51.541173+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:12:51.541173+0000","last_undegraded":"2024-07-04T08:12:51.541173+0000","last_fullsized":"2024-07-04T08:12:51.541173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560317+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:12:51.560317+0000","last_peered":"2024-07-04T08:12:51.560317+0000","last_clean":"2024-07-04T08:12:51.560317+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:12:51.560317+0000","last_undegraded":"2024-07-04T08:12:51.560317+0000","last_fullsized":"2024-07-04T08:12:51.560317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551363+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:12:51.551363+0000","last_peered":"2024-07-04T08:12:51.551363+0000","last_clean":"2024-07-04T08:12:51.551363+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:12:51.551363+0000","last_undegraded":"2024-07-04T08:12:51.551363+0000","last_fullsized":"2024-07-04T08:12:51.551363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541562+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:12:51.541562+0000","last_peered":"2024-07-04T08:12:51.541562+0000","last_clean":"2024-07-04T08:12:51.541562+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:12:51.541562+0000","last_undegraded":"2024-07-04T08:12:51.541562+0000","last_fullsized":"2024-07-04T08:12:51.541562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.536909+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:12:51.536909+0000","last_peered":"2024-07-04T08:12:51.536909+0000","last_clean":"2024-07-04T08:12:51.536909+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:12:51.536909+0000","last_undegraded":"2024-07-04T08:12:51.536909+0000","last_fullsized":"2024-07-04T08:12:51.536909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537941+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:12:51.537941+0000","last_peered":"2024-07-04T08:12:51.537941+0000","last_clean":"2024-07-04T08:12:51.537941+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:12:51.537941+0000","last_undegraded":"2024-07-04T08:12:51.537941+0000","last_fullsized":"2024-07-04T08:12:51.537941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565574+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:12:51.565574+0000","last_peered":"2024-07-04T08:12:51.565574+0000","last_clean":"2024-07-04T08:12:51.565574+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:12:51.565574+0000","last_undegraded":"2024-07-04T08:12:51.565574+0000","last_fullsized":"2024-07-04T08:12:51.565574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568095+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:12:51.568095+0000","last_peered":"2024-07-04T08:12:51.568095+0000","last_clean":"2024-07-04T08:12:51.568095+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:12:51.568095+0000","last_undegraded":"2024-07-04T08:12:51.568095+0000","last_fullsized":"2024-07-04T08:12:51.568095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569523+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:12:51.569523+0000","last_peered":"2024-07-04T08:12:51.569523+0000","last_clean":"2024-07-04T08:12:51.569523+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:12:51.569523+0000","last_undegraded":"2024-07-04T08:12:51.569523+0000","last_fullsized":"2024-07-04T08:12:51.569523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":864542,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2848,"num_read_kb":1643,"num_write":487,"num_write_kb":2247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2397018,"data_compressed":102009,"data_compressed_allocated":135168,"data_compressed_original":2088960,"omap_allocated":0,"internal_metadata":0},"log_size":369,"ondisk_log_size":369,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612692,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":543564,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808919,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1179648,"data_stored":1200976,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":940806,"data_compressed":51617,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168677,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":543872,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364906,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1141042,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528431,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":539530,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724661,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36472,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708680,"statfs":{"total":95995035648,"available":95957688320,"internally_reserved":0,"allocated":1159168,"data_stored":1529336,"data_compressed":87398,"data_compressed_allocated":114688,"data_compressed_original":1048576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920889,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":543604,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117119,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1114112,"data_stored":1325698,"data_compressed":69708,"data_compressed_allocated":94208,"data_compressed_original":851968,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378758,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":609142,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542283,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":611794,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705808,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":540454,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":787422,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":396596,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":466108,"data_compressed":17729,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":197800,"data_compressed":9520,"data_compressed_allocated":12288,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:15:18.518 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph pg dump --format=json 2024-07-04T08:15:19.025 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:18 smithi039 ceph-mon[32062]: pgmap v483: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-07-04T08:15:19.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.136+0000 7f7798600640 1 -- 172.21.15.39:0/2722788120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109340 msgr2=0x7f779010fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:19.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.136+0000 7f7798600640 1 --2- 172.21.15.39:0/2722788120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109340 0x7f779010fb70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7788009900 tx=0x7f778802a400 comp rx=0 tx=0).stop 2024-07-04T08:15:19.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.136+0000 7f7798600640 1 -- 172.21.15.39:0/2722788120 shutdown_connections 2024-07-04T08:15:19.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.136+0000 7f7798600640 1 --2- 172.21.15.39:0/2722788120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109340 0x7f779010fb70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.136+0000 7f7798600640 1 --2- 172.21.15.39:0/2722788120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77901089f0 0x7f7790108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.136+0000 7f7798600640 1 --2- 172.21.15.39:0/2722788120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77901030b0 0x7f77901034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.136+0000 7f7798600640 1 -- 172.21.15.39:0/2722788120 >> 172.21.15.39:0/2722788120 conn(0x7f779007ada0 msgr2=0x7f779007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:19.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.136+0000 7f7798600640 1 -- 172.21.15.39:0/2722788120 shutdown_connections 2024-07-04T08:15:19.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.136+0000 7f7798600640 1 -- 172.21.15.39:0/2722788120 wait complete. 2024-07-04T08:15:19.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.138+0000 7f7798600640 1 Processor -- start 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.138+0000 7f7798600640 1 -- start start 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.138+0000 7f7798600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77901030b0 0x7f7790074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.138+0000 7f7798600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77901089f0 0x7f77900751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.138+0000 7f7798600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109340 0x7f77900756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.138+0000 7f7798600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7790113040 con 0x7f7790109340 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.138+0000 7f7798600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7790112ec0 con 0x7f77901089f0 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.138+0000 7f7798600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77901131c0 con 0x7f77901030b0 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77901089f0 0x7f77900751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7796600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109340 0x7f77900756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77901089f0 0x7f77900751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45972/0 (socket says 172.21.15.39:45972) 2024-07-04T08:15:19.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795200640 1 -- 172.21.15.39:0/2464832051 learned_addr learned my addr 172.21.15.39:0/2464832051 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:19.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77901030b0 0x7f7790074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795200640 1 -- 172.21.15.39:0/2464832051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77901030b0 msgr2=0x7f7790074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:19.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795200640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77901030b0 0x7f7790074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795200640 1 -- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109340 msgr2=0x7f77900756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:19.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795200640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109340 0x7f77900756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795200640 1 -- 172.21.15.39:0/2464832051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77901a7c30 con 0x7f77901089f0 2024-07-04T08:15:19.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7796600640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109340 0x7f77900756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:19.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.139+0000 7f7795c00640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77901030b0 0x7f7790074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:19.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.140+0000 7f7795200640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77901089f0 0x7f77900751a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f777c00e9f0 tx=0x7f777c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:19.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.140+0000 7f778e000640 1 -- 172.21.15.39:0/2464832051 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c00ccc0 con 0x7f77901089f0 2024-07-04T08:15:19.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.140+0000 7f7798600640 1 -- 172.21.15.39:0/2464832051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77901a7f20 con 0x7f77901089f0 2024-07-04T08:15:19.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.140+0000 7f778e000640 1 -- 172.21.15.39:0/2464832051 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f777c0108e0 con 0x7f77901089f0 2024-07-04T08:15:19.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.140+0000 7f778e000640 1 -- 172.21.15.39:0/2464832051 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c018860 con 0x7f77901089f0 2024-07-04T08:15:19.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.140+0000 7f7798600640 1 -- 172.21.15.39:0/2464832051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77901a83d0 con 0x7f77901089f0 2024-07-04T08:15:19.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:18 smithi050 ceph-mon[36577]: pgmap v483: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-07-04T08:15:19.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.144+0000 7f779ecaa740 1 Processor -- start 2024-07-04T08:15:19.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.144+0000 7f779ecaa740 1 -- start start 2024-07-04T08:15:19.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.144+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ae8070 0x55f166e8c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.144+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e8ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ac7390 0x55f166e8d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f779ecaa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f166e02ea0 con 0x55f166abf1b0 2024-07-04T08:15:19.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f779ecaa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f166df2440 con 0x55f166ac7390 2024-07-04T08:15:19.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f779ecaa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f166df6480 con 0x55f166ae8070 2024-07-04T08:15:19.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f7795c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e8ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f7796600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ae8070 0x55f166e8c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f7795c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e8ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34752/0 (socket says 172.21.15.39:34752) 2024-07-04T08:15:19.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f7795c00640 1 -- 172.21.15.39:0/1923231381 learned_addr learned my addr 172.21.15.39:0/1923231381 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:19.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f7796600640 1 -- 172.21.15.39:0/1923231381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ac7390 msgr2=0x55f166e8d1f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:15:19.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f7796600640 1 --2- 172.21.15.39:0/1923231381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ac7390 0x55f166e8d1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f7796600640 1 -- 172.21.15.39:0/1923231381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 msgr2=0x55f166e8ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:19.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f7796600640 1 --2- 172.21.15.39:0/1923231381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e8ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.145+0000 7f7796600640 1 -- 172.21.15.39:0/1923231381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f166d8a610 con 0x55f166ae8070 2024-07-04T08:15:19.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.146+0000 7f7796600640 1 --2- 172.21.15.39:0/1923231381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ae8070 0x55f166e8c770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f778802a920 tx=0x7f7788004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13459c993141e4a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:19.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.146+0000 7f7782a00640 1 -- 172.21.15.39:0/1923231381 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788040070 con 0x55f166ae8070 2024-07-04T08:15:19.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.146+0000 7f778e000640 1 -- 172.21.15.39:0/2464832051 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7f777c010430 con 0x7f77901089f0 2024-07-04T08:15:19.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.146+0000 7f7782a00640 1 -- 172.21.15.39:0/1923231381 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77880047a0 con 0x55f166ae8070 2024-07-04T08:15:19.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.146+0000 7f7782a00640 1 -- 172.21.15.39:0/1923231381 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788035040 con 0x55f166ae8070 2024-07-04T08:15:19.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.147+0000 7f779ecaa740 1 -- 172.21.15.39:0/1923231381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ae8070 msgr2=0x55f166e8c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:19.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.147+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1923231381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ae8070 0x55f166e8c770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f778802a920 tx=0x7f7788004430 comp rx=0 tx=0).stop 2024-07-04T08:15:19.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.148+0000 7f779ecaa740 1 -- 172.21.15.39:0/1923231381 shutdown_connections 2024-07-04T08:15:19.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.148+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1923231381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e8ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.148+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1923231381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ac7390 0x55f166e8d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.148+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1923231381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ae8070 0x55f166e8c770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.148+0000 7f779ecaa740 1 -- 172.21.15.39:0/1923231381 >> 172.21.15.39:0/1923231381 conn(0x55f166adc460 msgr2=0x55f166e8a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:19.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.148+0000 7f779ecaa740 1 -- 172.21.15.39:0/1923231381 shutdown_connections 2024-07-04T08:15:19.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.148+0000 7f779ecaa740 1 -- 172.21.15.39:0/1923231381 wait complete. 2024-07-04T08:15:19.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.148+0000 7f778e000640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2860 0x7f77780b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.149+0000 7f778e000640 1 -- 172.21.15.39:0/2464832051 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f777c0e2ab0 con 0x7f77901089f0 2024-07-04T08:15:19.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.149+0000 7f779ecaa740 1 -- start start 2024-07-04T08:15:19.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.149+0000 7f7795c00640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2860 0x7f77780b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.149+0000 7f779ecaa740 1 Processor -- start 2024-07-04T08:15:19.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f7795c00640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2860 0x7f77780b4d70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55f166e8b3f0 tx=0x7f7784009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:19.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166abf1b0 0x55f166e98400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166ae8070 0x55f166ea4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f779ecaa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f166c56700 con 0x55f166ae8070 2024-07-04T08:15:19.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f779ecaa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f166d99b70 con 0x55f166abf1b0 2024-07-04T08:15:19.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f779ecaa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f166e9aef0 con 0x55f166ac7390 2024-07-04T08:15:19.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f779ecaa740 10 client.0 _ll_register_callbacks cb 0x55f166deb530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:19.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f7795c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166abf1b0 0x55f166e98400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f7795200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f7795200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34594/0 (socket says 172.21.15.39:34594) 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f7795200640 1 -- 172.21.15.39:0/3688580622 learned_addr learned my addr 172.21.15.39:0/3688580622 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f7795c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166abf1b0 0x55f166e98400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45986/0 (socket says 172.21.15.39:45986) 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.150+0000 7f7796600640 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166ae8070 0x55f166ea4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.151+0000 7f7795c00640 1 -- 172.21.15.39:0/3688580622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 msgr2=0x55f166e989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.151+0000 7f7795c00640 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.151+0000 7f7795c00640 1 -- 172.21.15.39:0/3688580622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166ae8070 msgr2=0x55f166ea4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.151+0000 7f7795c00640 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166ae8070 0x55f166ea4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.151+0000 7f7795c00640 1 -- 172.21.15.39:0/3688580622 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f166b53650 con 0x55f166abf1b0 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.151+0000 7f7795c00640 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166abf1b0 0x55f166e98400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7784003f50 tx=0x7f7784004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e94b08c85532a33f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.151+0000 7f7777400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:15:19.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.152+0000 7f7777400640 1 -- 172.21.15.39:0/3688580622 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784004550 con 0x55f166abf1b0 2024-07-04T08:15:19.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.152+0000 7f7777400640 1 -- 172.21.15.39:0/3688580622 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f77840046f0 con 0x55f166abf1b0 2024-07-04T08:15:19.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.152+0000 7f779ecaa740 1 -- 172.21.15.39:0/3688580622 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77880047a0 con 0x55f166abf1b0 2024-07-04T08:15:19.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.152+0000 7f7777400640 1 -- 172.21.15.39:0/3688580622 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7784004b20 con 0x55f166abf1b0 2024-07-04T08:15:19.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.152+0000 7f779ecaa740 10 client.25523 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:19.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.152+0000 7f779ecaa740 1 -- 172.21.15.39:0/3688580622 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f166ea7010 con 0x55f166abf1b0 2024-07-04T08:15:19.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.153+0000 7f7777400640 1 -- 172.21.15.39:0/3688580622 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77880047a0 con 0x55f166abf1b0 2024-07-04T08:15:19.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.153+0000 7f7777400640 1 -- 172.21.15.39:0/3688580622 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7784027020 con 0x55f166abf1b0 2024-07-04T08:15:19.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.153+0000 7f779ecaa740 10 client.25523 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:19.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.153+0000 7f779ecaa740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:15:19.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.153+0000 7f779ecaa740 20 client.25523 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:19.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.153+0000 7f779ecaa740 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f166aa69f0 0x55f166eaa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.153+0000 7f779ecaa740 4 client.25523 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.153+0000 7f779ecaa740 1 -- 172.21.15.39:0/3688580622 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f166e04df0 con 0x55f166aa69f0 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.154+0000 7f7795200640 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f166aa69f0 0x55f166eaa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.154+0000 7f7795200640 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f166aa69f0 0x55f166eaa530 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d72c7824f45916e6 server_cookie=c49454d63a4d61fd in_seq=0 out_seq=0 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.154+0000 7f7777400640 10 client.25523 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.155+0000 7f7777400640 1 -- 172.21.15.39:0/3688580622 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f166e04df0 con 0x55f166aa69f0 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.155+0000 7f7777400640 10 client.25523 handle_command_reply: tid=0 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.155+0000 7f779ecaa740 1 client.25523 shutdown 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.155+0000 7f779ecaa740 1 -- 172.21.15.39:0/3688580622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166abf1b0 msgr2=0x55f166e98400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.155+0000 7f779ecaa740 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166abf1b0 0x55f166e98400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7784003f50 tx=0x7f7784004ab0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.155+0000 7f779ecaa740 1 -- 172.21.15.39:0/3688580622 shutdown_connections 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.155+0000 7f779ecaa740 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f166aa69f0 0x55f166eaa530 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.155+0000 7f779ecaa740 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166ae8070 0x55f166ea4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.155+0000 7f779ecaa740 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.156+0000 7f779ecaa740 1 --2- 172.21.15.39:0/3688580622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166abf1b0 0x55f166e98400 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.156+0000 7f779ecaa740 1 -- 172.21.15.39:0/3688580622 >> 172.21.15.39:0/3688580622 conn(0x55f166adc460 msgr2=0x55f166e8f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:19.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.156+0000 7f7777400640 0 client.25523 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:19.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.156+0000 7f779ecaa740 1 -- 172.21.15.39:0/3688580622 shutdown_connections 2024-07-04T08:15:19.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.156+0000 7f779ecaa740 1 -- 172.21.15.39:0/3688580622 wait complete. 2024-07-04T08:15:19.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.156+0000 7f779ecaa740 20 client.25523 trim_cache size 0 max 16384 2024-07-04T08:15:19.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.177+0000 7f779ecaa740 1 Processor -- start 2024-07-04T08:15:19.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.177+0000 7f779ecaa740 1 -- start start 2024-07-04T08:15:19.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.178+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e99e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.178+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166eafd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.178+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166eb0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.178+0000 7f779ecaa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b2480 con 0x55f166abf1b0 2024-07-04T08:15:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.178+0000 7f779ecaa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b3fc0 con 0x55f166ae8070 2024-07-04T08:15:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.178+0000 7f779ecaa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b2b50 con 0x55f166ac7390 2024-07-04T08:15:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7795c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e99e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7796600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166eb0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7796600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166eb0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:46000/0 (socket says 172.21.15.39:46000) 2024-07-04T08:15:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7795c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e99e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:34778/0 (socket says 172.21.15.39:34778) 2024-07-04T08:15:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7796600640 1 -- 172.21.15.39:0/2660691254 learned_addr learned my addr 172.21.15.39:0/2660691254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7795200640 1 --2- 172.21.15.39:0/2660691254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166eafd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7796600640 1 -- 172.21.15.39:0/2660691254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 msgr2=0x55f166eafd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7796600640 1 --2- 172.21.15.39:0/2660691254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166eafd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7796600640 1 -- 172.21.15.39:0/2660691254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 msgr2=0x55f166e99e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7796600640 1 --2- 172.21.15.39:0/2660691254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e99e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.179+0000 7f7796600640 1 -- 172.21.15.39:0/2660691254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f166e93ab0 con 0x55f166ae8070 2024-07-04T08:15:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.180+0000 7f7796600640 1 --2- 172.21.15.39:0/2660691254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166eb0270 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7788009900 tx=0x7f7788031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16c40d27e7679e91 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.180+0000 7f7782000640 1 -- 172.21.15.39:0/2660691254 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788040070 con 0x55f166ae8070 2024-07-04T08:15:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.180+0000 7f7782000640 1 -- 172.21.15.39:0/2660691254 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77880054f0 con 0x55f166ae8070 2024-07-04T08:15:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.180+0000 7f7782000640 1 -- 172.21.15.39:0/2660691254 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788035040 con 0x55f166ae8070 2024-07-04T08:15:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.181+0000 7f779ecaa740 1 -- 172.21.15.39:0/2660691254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 msgr2=0x55f166eb0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.181+0000 7f779ecaa740 1 --2- 172.21.15.39:0/2660691254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166eb0270 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7788009900 tx=0x7f7788031400 comp rx=0 tx=0).stop 2024-07-04T08:15:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.181+0000 7f779ecaa740 1 -- 172.21.15.39:0/2660691254 shutdown_connections 2024-07-04T08:15:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.181+0000 7f779ecaa740 1 --2- 172.21.15.39:0/2660691254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166eb0270 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.181+0000 7f779ecaa740 1 --2- 172.21.15.39:0/2660691254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166eafd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.181+0000 7f779ecaa740 1 --2- 172.21.15.39:0/2660691254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166e99e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.181+0000 7f779ecaa740 1 -- 172.21.15.39:0/2660691254 >> 172.21.15.39:0/2660691254 conn(0x55f166adc460 msgr2=0x55f166e99720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.181+0000 7f779ecaa740 1 -- 172.21.15.39:0/2660691254 shutdown_connections 2024-07-04T08:15:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.181+0000 7f779ecaa740 1 -- 172.21.15.39:0/2660691254 wait complete. 2024-07-04T08:15:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.182+0000 7f779ecaa740 1 -- start start 2024-07-04T08:15:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.182+0000 7f779ecaa740 1 Processor -- start 2024-07-04T08:15:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166ea0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e9bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f779ecaa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166e9c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f779ecaa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b38f0 con 0x55f166abf1b0 2024-07-04T08:15:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f779ecaa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b3220 con 0x55f166ae8070 2024-07-04T08:15:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f779ecaa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b0460 con 0x55f166ac7390 2024-07-04T08:15:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f779ecaa740 10 client.0 _ll_register_callbacks cb 0x7f7784012f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:19.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f7795200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e9bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f7795c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166ea0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f7796600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166e9c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f7795200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e9bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34610/0 (socket says 172.21.15.39:34610) 2024-07-04T08:15:19.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f7796600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166e9c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:46006/0 (socket says 172.21.15.39:46006) 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.183+0000 7f7795200640 1 -- 172.21.15.39:0/1585191563 learned_addr learned my addr 172.21.15.39:0/1585191563 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.184+0000 7f7795c00640 1 -- 172.21.15.39:0/1585191563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 msgr2=0x55f166e9bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.184+0000 7f7795c00640 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e9bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.184+0000 7f7795c00640 1 -- 172.21.15.39:0/1585191563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 msgr2=0x55f166e9c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.184+0000 7f7795c00640 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166e9c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.184+0000 7f7795c00640 1 -- 172.21.15.39:0/1585191563 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f166e9cba0 con 0x55f166abf1b0 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.184+0000 7f7796600640 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166e9c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.184+0000 7f7795200640 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e9bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.184+0000 7f7795c00640 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166ea0930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7784005b40 tx=0x7f7784004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cb2cc558a668d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.184+0000 7f7777400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.185+0000 7f7777400640 1 -- 172.21.15.39:0/1585191563 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778403a070 con 0x55f166abf1b0 2024-07-04T08:15:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.185+0000 7f7777400640 1 -- 172.21.15.39:0/1585191563 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7784029a90 con 0x55f166abf1b0 2024-07-04T08:15:19.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.185+0000 7f779ecaa740 1 -- 172.21.15.39:0/1585191563 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77880054f0 con 0x55f166abf1b0 2024-07-04T08:15:19.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.185+0000 7f7777400640 1 -- 172.21.15.39:0/1585191563 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7784046050 con 0x55f166abf1b0 2024-07-04T08:15:19.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.185+0000 7f779ecaa740 10 client.15411 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:19.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.185+0000 7f779ecaa740 1 -- 172.21.15.39:0/1585191563 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f166f12d50 con 0x55f166abf1b0 2024-07-04T08:15:19.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.186+0000 7f7777400640 1 -- 172.21.15.39:0/1585191563 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77880054f0 con 0x55f166abf1b0 2024-07-04T08:15:19.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.187+0000 7f7777400640 1 -- 172.21.15.39:0/1585191563 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7784027070 con 0x55f166abf1b0 2024-07-04T08:15:19.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.187+0000 7f779ecaa740 10 client.15411 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:19.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.187+0000 7f779ecaa740 10 client.15411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:15:19.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.187+0000 7f779ecaa740 20 client.15411 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:19.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.187+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f166a7bda0 0x55f166f15510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:19.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.187+0000 7f779ecaa740 4 client.15411 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:15:19.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.187+0000 7f779ecaa740 1 -- 172.21.15.39:0/1585191563 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f166e00890 con 0x55f166a7bda0 2024-07-04T08:15:19.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.187+0000 7f7795200640 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f166a7bda0 0x55f166f15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:19.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.188+0000 7f7777400640 10 client.15411 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:19.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.188+0000 7f7795200640 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f166a7bda0 0x55f166f15510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e0a31dcc67795e2 server_cookie=a7d566c9374b4a72 in_seq=0 out_seq=0 2024-07-04T08:15:19.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.188+0000 7f7777400640 1 -- 172.21.15.39:0/1585191563 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f166e00890 con 0x55f166a7bda0 2024-07-04T08:15:19.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.188+0000 7f7777400640 10 client.15411 handle_command_reply: tid=0 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.188+0000 7f779ecaa740 1 client.15411 shutdown 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.188+0000 7f779ecaa740 1 -- 172.21.15.39:0/1585191563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 msgr2=0x55f166ea0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.188+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166ea0930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7784005b40 tx=0x7f7784004650 comp rx=0 tx=0).stop 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.188+0000 7f779ecaa740 1 -- 172.21.15.39:0/1585191563 shutdown_connections 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.189+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f166a7bda0 0x55f166f15510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.189+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f166ae8070 0x55f166e9c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.189+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f166ac7390 0x55f166e9bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.189+0000 7f779ecaa740 1 --2- 172.21.15.39:0/1585191563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f166abf1b0 0x55f166ea0930 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7784005b40 tx=0x7f7784004650 comp rx=0 tx=0).stop 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.189+0000 7f779ecaa740 1 -- 172.21.15.39:0/1585191563 >> 172.21.15.39:0/1585191563 conn(0x55f166adc460 msgr2=0x55f166e996b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.189+0000 7f7777400640 0 client.15411 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.189+0000 7f779ecaa740 1 -- 172.21.15.39:0/1585191563 shutdown_connections 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.189+0000 7f779ecaa740 1 -- 172.21.15.39:0/1585191563 wait complete. 2024-07-04T08:15:19.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.189+0000 7f779ecaa740 20 client.15411 trim_cache size 0 max 16384 2024-07-04T08:15:19.193 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:15:19.193 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-04T08:15:19.193 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-07-04T08:15:19.193 INFO:teuthology.orchestra.run.smithi039.stdout: "fab3c176-38e1-47a7-9e54-878dbfb4800a": { 2024-07-04T08:15:19.193 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-07-04T08:15:19.193 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "fab3c176-38e1-47a7-9e54-878dbfb4800a", 2024-07-04T08:15:19.193 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-07-04T08:15:19.193 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:15:19.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:15:19.194 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:15:19.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.190+0000 7f7798600640 1 -- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2860 msgr2=0x7f77780b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:19.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.190+0000 7f7798600640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2860 0x7f77780b4d70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55f166e8b3f0 tx=0x7f7784009500 comp rx=0 tx=0).stop 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.190+0000 7f7798600640 1 -- 172.21.15.39:0/2464832051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77901089f0 msgr2=0x7f77900751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.190+0000 7f7798600640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77901089f0 0x7f77900751a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f777c00e9f0 tx=0x7f777c00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.191+0000 7f7798600640 1 -- 172.21.15.39:0/2464832051 shutdown_connections 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.191+0000 7f7798600640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2860 0x7f77780b4d70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.191+0000 7f7798600640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109340 0x7f77900756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.191+0000 7f7798600640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f77901089f0 0x7f77900751a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.191+0000 7f7798600640 1 --2- 172.21.15.39:0/2464832051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77901030b0 0x7f7790074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.191+0000 7f7798600640 1 -- 172.21.15.39:0/2464832051 >> 172.21.15.39:0/2464832051 conn(0x7f779007ada0 msgr2=0x7f77901069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.191+0000 7f7798600640 1 -- 172.21.15.39:0/2464832051 shutdown_connections 2024-07-04T08:15:19.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:19.191+0000 7f7798600640 1 -- 172.21.15.39:0/2464832051 wait complete. 2024-07-04T08:15:19.205 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:19.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:18 smithi133 ceph-mon[37050]: pgmap v483: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-07-04T08:15:20.343 INFO:tasks.cephfs.filesystem:scrub status for tag:fab3c176-38e1-47a7-9e54-878dbfb4800a - {'path': '/', 'tag': 'fab3c176-38e1-47a7-9e54-878dbfb4800a', 'options': 'recursive,force'} 2024-07-04T08:15:21.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:20 smithi039 ceph-mon[32062]: pgmap v484: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 55 op/s 2024-07-04T08:15:21.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:20 smithi050 ceph-mon[36577]: pgmap v484: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 55 op/s 2024-07-04T08:15:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:20 smithi133 ceph-mon[37050]: pgmap v484: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 55 op/s 2024-07-04T08:15:22.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.138+0000 7f859d200640 1 -- 172.21.15.39:0/879157099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598111200 msgr2=0x7f8598111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:22.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.138+0000 7f859d200640 1 --2- 172.21.15.39:0/879157099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598111200 0x7f8598111610 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8580009be0 tx=0x7f858002a6c0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.138+0000 7f859d200640 1 -- 172.21.15.39:0/879157099 shutdown_connections 2024-07-04T08:15:22.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.138+0000 7f859d200640 1 --2- 172.21.15.39:0/879157099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8598111b50 0x7f8598118380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.138+0000 7f859d200640 1 --2- 172.21.15.39:0/879157099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598111200 0x7f8598111610 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.138+0000 7f859d200640 1 --2- 172.21.15.39:0/879157099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f859810b8c0 0x7f859810bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.138+0000 7f859d200640 1 -- 172.21.15.39:0/879157099 >> 172.21.15.39:0/879157099 conn(0x7f8598070fd0 msgr2=0x7f8598109910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:22.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.138+0000 7f859d200640 1 -- 172.21.15.39:0/879157099 shutdown_connections 2024-07-04T08:15:22.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.138+0000 7f859d200640 1 -- 172.21.15.39:0/879157099 wait complete. 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.139+0000 7f859d200640 1 Processor -- start 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.139+0000 7f859d200640 1 -- start start 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f859d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f859810b8c0 0x7f859806de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f859d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598111200 0x7f859806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f859d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8598111b50 0x7f859806c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f859d200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f859811b7b0 con 0x7f8598111b50 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f859d200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f859811b630 con 0x7f859810b8c0 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f859d200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f859811b930 con 0x7f8598111200 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f8596800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8598111b50 0x7f859806c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:22.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f858d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598111200 0x7f859806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f8596800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8598111b50 0x7f859806c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50940/0 (socket says 172.21.15.39:50940) 2024-07-04T08:15:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f8596800640 1 -- 172.21.15.39:0/1108775428 learned_addr learned my addr 172.21.15.39:0/1108775428 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.140+0000 7f8595e00640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f859810b8c0 0x7f859806de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.141+0000 7f8596800640 1 -- 172.21.15.39:0/1108775428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598111200 msgr2=0x7f859806c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.141+0000 7f8596800640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598111200 0x7f859806c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.141+0000 7f8596800640 1 -- 172.21.15.39:0/1108775428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f859810b8c0 msgr2=0x7f859806de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.141+0000 7f8596800640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f859810b8c0 0x7f859806de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.141+0000 7f8596800640 1 -- 172.21.15.39:0/1108775428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859806d150 con 0x7f8598111b50 2024-07-04T08:15:22.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.141+0000 7f8595e00640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f859810b8c0 0x7f859806de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.141+0000 7f8596800640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8598111b50 0x7f859806c930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f859000e970 tx=0x7f859000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.141+0000 7f858f400640 1 -- 172.21.15.39:0/1108775428 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859000ccc0 con 0x7f8598111b50 2024-07-04T08:15:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.142+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85981b4be0 con 0x7f8598111b50 2024-07-04T08:15:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.142+0000 7f858f400640 1 -- 172.21.15.39:0/1108775428 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f859000ce60 con 0x7f8598111b50 2024-07-04T08:15:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.142+0000 7f858f400640 1 -- 172.21.15.39:0/1108775428 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590010430 con 0x7f8598111b50 2024-07-04T08:15:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.142+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85981b50f0 con 0x7f8598111b50 2024-07-04T08:15:22.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.143+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859810bcd0 con 0x7f8598111b50 2024-07-04T08:15:22.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.146+0000 7f858f400640 1 -- 172.21.15.39:0/1108775428 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7f8590004fb0 con 0x7f8598111b50 2024-07-04T08:15:22.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.148+0000 7f858f400640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f857c0b28d0 0x7f857c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.149+0000 7f858f400640 1 -- 172.21.15.39:0/1108775428 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f85900e2e40 con 0x7f8598111b50 2024-07-04T08:15:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.149+0000 7f8595e00640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f857c0b28d0 0x7f857c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:22.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.149+0000 7f8595e00640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f857c0b28d0 0x7f857c0b4de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85880096c0 tx=0x7f8588009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:22.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.150+0000 7f858f400640 1 -- 172.21.15.39:0/1108775428 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f8590055110 con 0x7f8598111b50 2024-07-04T08:15:22.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.363+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8598062d50 con 0x7f857c0b28d0 2024-07-04T08:15:22.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.375+0000 7f858f400640 1 -- 172.21.15.39:0/1108775428 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340956 (secure 0 0 0) 0x7f8598062d50 con 0x7f857c0b28d0 2024-07-04T08:15:22.378 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:22.387 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:15:22.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f857c0b28d0 msgr2=0x7f857c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:22.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f857c0b28d0 0x7f857c0b4de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f85880096c0 tx=0x7f8588009260 comp rx=0 tx=0).stop 2024-07-04T08:15:22.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8598111b50 msgr2=0x7f859806c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:22.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8598111b50 0x7f859806c930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f859000e970 tx=0x7f859000ed70 comp rx=0 tx=0).stop 2024-07-04T08:15:22.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 shutdown_connections 2024-07-04T08:15:22.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f857c0b28d0 0x7f857c0b4de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8598111b50 0x7f859806c930 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8598111200 0x7f859806c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 --2- 172.21.15.39:0/1108775428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f859810b8c0 0x7f859806de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:22.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.386+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 >> 172.21.15.39:0/1108775428 conn(0x7f8598070fd0 msgr2=0x7f8598071880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:22.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.387+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 shutdown_connections 2024-07-04T08:15:22.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:22.387+0000 7f859d200640 1 -- 172.21.15.39:0/1108775428 wait complete. 2024-07-04T08:15:23.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:22 smithi050 ceph-mon[36577]: pgmap v485: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-04T08:15:23.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:22 smithi039 ceph-mon[32062]: pgmap v485: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-04T08:15:23.267 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-07-04T08:15:21.695044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1455144,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2990,"num_read_kb":1738,"num_write":612,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":963,"ondisk_log_size":963,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378592,"kb_used_data":12788,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563232,"statfs":{"total":1151940427776,"available":1151552749568,"internally_reserved":0,"allocated":13094912,"data_stored":10856810,"data_compressed":611013,"data_compressed_allocated":823296,"data_compressed_original":5136384,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":456,"num_read_kb":356,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006061"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":316,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:18.470810+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:15:18.470810+0000","last_peered":"2024-07-04T08:15:18.470810+0000","last_clean":"2024-07-04T08:15:18.470810+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:15:18.470810+0000","last_undegraded":"2024-07-04T08:15:18.470810+0000","last_fullsized":"2024-07-04T08:15:18.470810+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":256,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553793+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:12:51.553793+0000","last_peered":"2024-07-04T08:12:51.553793+0000","last_clean":"2024-07-04T08:12:51.553793+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:12:51.553793+0000","last_undegraded":"2024-07-04T08:12:51.553793+0000","last_fullsized":"2024-07-04T08:12:51.553793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:16.118017+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:12:16.118017+0000","last_peered":"2024-07-04T08:12:16.118017+0000","last_clean":"2024-07-04T08:12:16.118017+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:12:16.118017+0000","last_undegraded":"2024-07-04T08:12:16.118017+0000","last_fullsized":"2024-07-04T08:12:16.118017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.698593+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:53.698593+0000","last_peered":"2024-07-04T08:11:53.698593+0000","last_clean":"2024-07-04T08:11:53.698593+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:53.698593+0000","last_undegraded":"2024-07-04T08:11:53.698593+0000","last_fullsized":"2024-07-04T08:11:53.698593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569975+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:12:51.569975+0000","last_peered":"2024-07-04T08:12:51.569975+0000","last_clean":"2024-07-04T08:12:51.569975+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:12:51.569975+0000","last_undegraded":"2024-07-04T08:12:51.569975+0000","last_fullsized":"2024-07-04T08:12:51.569975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:15.756154+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:12:15.756154+0000","last_peered":"2024-07-04T08:12:15.756154+0000","last_clean":"2024-07-04T08:12:15.756154+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:12:15.756154+0000","last_undegraded":"2024-07-04T08:12:15.756154+0000","last_fullsized":"2024-07-04T08:12:15.756154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247127+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:12:05.247127+0000","last_peered":"2024-07-04T08:12:05.247127+0000","last_clean":"2024-07-04T08:12:05.247127+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:12:05.247127+0000","last_undegraded":"2024-07-04T08:12:05.247127+0000","last_fullsized":"2024-07-04T08:12:05.247127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.549821+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:12:51.549821+0000","last_peered":"2024-07-04T08:12:51.549821+0000","last_clean":"2024-07-04T08:12:51.549821+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:12:51.549821+0000","last_undegraded":"2024-07-04T08:12:51.549821+0000","last_fullsized":"2024-07-04T08:12:51.549821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756728+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:12:20.756728+0000","last_peered":"2024-07-04T08:12:20.756728+0000","last_clean":"2024-07-04T08:12:20.756728+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:12:20.756728+0000","last_undegraded":"2024-07-04T08:12:20.756728+0000","last_fullsized":"2024-07-04T08:12:20.756728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247079+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:12:05.247079+0000","last_peered":"2024-07-04T08:12:05.247079+0000","last_clean":"2024-07-04T08:12:05.247079+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:12:05.247079+0000","last_undegraded":"2024-07-04T08:12:05.247079+0000","last_fullsized":"2024-07-04T08:12:05.247079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.535944+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:12:51.535944+0000","last_peered":"2024-07-04T08:12:51.535944+0000","last_clean":"2024-07-04T08:12:51.535944+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:12:51.535944+0000","last_undegraded":"2024-07-04T08:12:51.535944+0000","last_fullsized":"2024-07-04T08:12:51.535944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756706+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:12:20.756706+0000","last_peered":"2024-07-04T08:12:20.756706+0000","last_clean":"2024-07-04T08:12:20.756706+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:12:20.756706+0000","last_undegraded":"2024-07-04T08:12:20.756706+0000","last_fullsized":"2024-07-04T08:12:20.756706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562540+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:12:51.562540+0000","last_peered":"2024-07-04T08:12:51.562540+0000","last_clean":"2024-07-04T08:12:51.562540+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:12:51.562540+0000","last_undegraded":"2024-07-04T08:12:51.562540+0000","last_fullsized":"2024-07-04T08:12:51.562540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844569+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:53.844569+0000","last_peered":"2024-07-04T08:11:53.844569+0000","last_clean":"2024-07-04T08:11:53.844569+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:53.844569+0000","last_undegraded":"2024-07-04T08:11:53.844569+0000","last_fullsized":"2024-07-04T08:11:53.844569+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754878+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:12:05.754878+0000","last_peered":"2024-07-04T08:12:05.754878+0000","last_clean":"2024-07-04T08:12:05.754878+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:12:05.754878+0000","last_undegraded":"2024-07-04T08:12:05.754878+0000","last_fullsized":"2024-07-04T08:12:05.754878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570735+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:12:51.570735+0000","last_peered":"2024-07-04T08:12:51.570735+0000","last_clean":"2024-07-04T08:12:51.570735+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:12:51.570735+0000","last_undegraded":"2024-07-04T08:12:51.570735+0000","last_fullsized":"2024-07-04T08:12:51.570735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691548+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:53.691548+0000","last_peered":"2024-07-04T08:11:53.691548+0000","last_clean":"2024-07-04T08:11:53.691548+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:53.691548+0000","last_undegraded":"2024-07-04T08:11:53.691548+0000","last_fullsized":"2024-07-04T08:11:53.691548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754825+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:12:05.754825+0000","last_peered":"2024-07-04T08:12:05.754825+0000","last_clean":"2024-07-04T08:12:05.754825+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:12:05.754825+0000","last_undegraded":"2024-07-04T08:12:05.754825+0000","last_fullsized":"2024-07-04T08:12:05.754825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566667+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:12:51.566667+0000","last_peered":"2024-07-04T08:12:51.566667+0000","last_clean":"2024-07-04T08:12:51.566667+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:12:51.566667+0000","last_undegraded":"2024-07-04T08:12:51.566667+0000","last_fullsized":"2024-07-04T08:12:51.566667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.697029+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:53.697029+0000","last_peered":"2024-07-04T08:11:53.697029+0000","last_clean":"2024-07-04T08:11:53.697029+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:53.697029+0000","last_undegraded":"2024-07-04T08:11:53.697029+0000","last_fullsized":"2024-07-04T08:11:53.697029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771248+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:11:53.771248+0000","last_peered":"2024-07-04T08:11:53.771248+0000","last_clean":"2024-07-04T08:11:53.771248+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:11:53.771248+0000","last_undegraded":"2024-07-04T08:11:53.771248+0000","last_fullsized":"2024-07-04T08:11:53.771248+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.526597+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:12:51.526597+0000","last_peered":"2024-07-04T08:12:51.526597+0000","last_clean":"2024-07-04T08:12:51.526597+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:12:51.526597+0000","last_undegraded":"2024-07-04T08:12:51.526597+0000","last_fullsized":"2024-07-04T08:12:51.526597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936427+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:53.936427+0000","last_peered":"2024-07-04T08:11:53.936427+0000","last_clean":"2024-07-04T08:11:53.936427+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:53.936427+0000","last_undegraded":"2024-07-04T08:11:53.936427+0000","last_fullsized":"2024-07-04T08:11:53.936427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754887+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:12:05.754887+0000","last_peered":"2024-07-04T08:12:05.754887+0000","last_clean":"2024-07-04T08:12:05.754887+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:12:05.754887+0000","last_undegraded":"2024-07-04T08:12:05.754887+0000","last_fullsized":"2024-07-04T08:12:05.754887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568004+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:12:51.568004+0000","last_peered":"2024-07-04T08:12:51.568004+0000","last_clean":"2024-07-04T08:12:51.568004+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:12:51.568004+0000","last_undegraded":"2024-07-04T08:12:51.568004+0000","last_fullsized":"2024-07-04T08:12:51.568004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692160+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:53.692160+0000","last_peered":"2024-07-04T08:11:53.692160+0000","last_clean":"2024-07-04T08:11:53.692160+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:53.692160+0000","last_undegraded":"2024-07-04T08:11:53.692160+0000","last_fullsized":"2024-07-04T08:11:53.692160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115754+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:11:56.115754+0000","last_peered":"2024-07-04T08:11:56.115754+0000","last_clean":"2024-07-04T08:11:56.115754+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:11:56.115754+0000","last_undegraded":"2024-07-04T08:11:56.115754+0000","last_fullsized":"2024-07-04T08:11:56.115754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.556576+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:12:51.556576+0000","last_peered":"2024-07-04T08:12:51.556576+0000","last_clean":"2024-07-04T08:12:51.556576+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:12:51.556576+0000","last_undegraded":"2024-07-04T08:12:51.556576+0000","last_fullsized":"2024-07-04T08:12:51.556576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'101","reported_seq":246,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:13.400609+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:15:13.400609+0000","last_peered":"2024-07-04T08:15:13.400609+0000","last_clean":"2024-07-04T08:15:13.400609+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:15:13.400609+0000","last_undegraded":"2024-07-04T08:15:13.400609+0000","last_fullsized":"2024-07-04T08:15:13.400609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":185,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.322378+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:11:54.322378+0000","last_peered":"2024-07-04T08:11:54.322378+0000","last_clean":"2024-07-04T08:11:54.322378+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:11:54.322378+0000","last_undegraded":"2024-07-04T08:11:54.322378+0000","last_fullsized":"2024-07-04T08:11:54.322378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540856+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:12:51.540856+0000","last_peered":"2024-07-04T08:12:51.540856+0000","last_clean":"2024-07-04T08:12:51.540856+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:12:51.540856+0000","last_undegraded":"2024-07-04T08:12:51.540856+0000","last_fullsized":"2024-07-04T08:12:51.540856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771431+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:53.771431+0000","last_peered":"2024-07-04T08:11:53.771431+0000","last_clean":"2024-07-04T08:11:53.771431+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:53.771431+0000","last_undegraded":"2024-07-04T08:11:53.771431+0000","last_fullsized":"2024-07-04T08:11:53.771431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.238979+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:11:56.238979+0000","last_peered":"2024-07-04T08:11:56.238979+0000","last_clean":"2024-07-04T08:11:56.238979+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:11:56.238979+0000","last_undegraded":"2024-07-04T08:11:56.238979+0000","last_fullsized":"2024-07-04T08:11:56.238979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542562+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:12:51.542562+0000","last_peered":"2024-07-04T08:12:51.542562+0000","last_clean":"2024-07-04T08:12:51.542562+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:12:51.542562+0000","last_undegraded":"2024-07-04T08:12:51.542562+0000","last_fullsized":"2024-07-04T08:12:51.542562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993415+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:11:53.993415+0000","last_peered":"2024-07-04T08:11:53.993415+0000","last_clean":"2024-07-04T08:11:53.993415+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:11:53.993415+0000","last_undegraded":"2024-07-04T08:11:53.993415+0000","last_fullsized":"2024-07-04T08:11:53.993415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:00.246372+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:12:00.246372+0000","last_peered":"2024-07-04T08:12:00.246372+0000","last_clean":"2024-07-04T08:12:00.246372+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:12:00.246372+0000","last_undegraded":"2024-07-04T08:12:00.246372+0000","last_fullsized":"2024-07-04T08:12:00.246372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.543737+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:12:51.543737+0000","last_peered":"2024-07-04T08:12:51.543737+0000","last_clean":"2024-07-04T08:12:51.543737+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:12:51.543737+0000","last_undegraded":"2024-07-04T08:12:51.543737+0000","last_fullsized":"2024-07-04T08:12:51.543737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":311,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:19.474591+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:15:19.474591+0000","last_peered":"2024-07-04T08:15:19.474591+0000","last_clean":"2024-07-04T08:15:19.474591+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:15:19.474591+0000","last_undegraded":"2024-07-04T08:15:19.474591+0000","last_fullsized":"2024-07-04T08:15:19.474591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":258,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:58.137211+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:11:58.137211+0000","last_peered":"2024-07-04T08:11:58.137211+0000","last_clean":"2024-07-04T08:11:58.137211+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:11:58.137211+0000","last_undegraded":"2024-07-04T08:11:58.137211+0000","last_fullsized":"2024-07-04T08:11:58.137211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.516953+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:12:51.516953+0000","last_peered":"2024-07-04T08:12:51.516953+0000","last_clean":"2024-07-04T08:12:51.516953+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:12:51.516953+0000","last_undegraded":"2024-07-04T08:12:51.516953+0000","last_fullsized":"2024-07-04T08:12:51.516953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":305,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:16.531291+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:15:16.531291+0000","last_peered":"2024-07-04T08:15:16.531291+0000","last_clean":"2024-07-04T08:15:16.531291+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:15:16.531291+0000","last_undegraded":"2024-07-04T08:15:16.531291+0000","last_fullsized":"2024-07-04T08:15:16.531291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":253,"num_read_kb":253,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138305+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:12:03.138305+0000","last_peered":"2024-07-04T08:12:03.138305+0000","last_clean":"2024-07-04T08:12:03.138305+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:12:03.138305+0000","last_undegraded":"2024-07-04T08:12:03.138305+0000","last_fullsized":"2024-07-04T08:12:03.138305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534873+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:12:51.534873+0000","last_peered":"2024-07-04T08:12:51.534873+0000","last_clean":"2024-07-04T08:12:51.534873+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:12:51.534873+0000","last_undegraded":"2024-07-04T08:12:51.534873+0000","last_fullsized":"2024-07-04T08:12:51.534873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851437+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:53.851437+0000","last_peered":"2024-07-04T08:11:53.851437+0000","last_clean":"2024-07-04T08:11:53.851437+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:53.851437+0000","last_undegraded":"2024-07-04T08:11:53.851437+0000","last_fullsized":"2024-07-04T08:11:53.851437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975520+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:12:04.975520+0000","last_peered":"2024-07-04T08:12:04.975520+0000","last_clean":"2024-07-04T08:12:04.975520+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:12:04.975520+0000","last_undegraded":"2024-07-04T08:12:04.975520+0000","last_fullsized":"2024-07-04T08:12:04.975520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.558356+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:12:51.558356+0000","last_peered":"2024-07-04T08:12:51.558356+0000","last_clean":"2024-07-04T08:12:51.558356+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:12:51.558356+0000","last_undegraded":"2024-07-04T08:12:51.558356+0000","last_fullsized":"2024-07-04T08:12:51.558356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691883+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:53.691883+0000","last_peered":"2024-07-04T08:11:53.691883+0000","last_clean":"2024-07-04T08:11:53.691883+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:53.691883+0000","last_undegraded":"2024-07-04T08:11:53.691883+0000","last_fullsized":"2024-07-04T08:11:53.691883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":120,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:46.091830+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:12:46.091830+0000","last_peered":"2024-07-04T08:12:46.091830+0000","last_clean":"2024-07-04T08:12:46.091830+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:12:46.091830+0000","last_undegraded":"2024-07-04T08:12:46.091830+0000","last_fullsized":"2024-07-04T08:12:46.091830+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.085858+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:11:56.085858+0000","last_peered":"2024-07-04T08:11:56.085858+0000","last_clean":"2024-07-04T08:11:56.085858+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:11:56.085858+0000","last_undegraded":"2024-07-04T08:11:56.085858+0000","last_fullsized":"2024-07-04T08:11:56.085858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553863+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:12:51.553863+0000","last_peered":"2024-07-04T08:12:51.553863+0000","last_clean":"2024-07-04T08:12:51.553863+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:12:51.553863+0000","last_undegraded":"2024-07-04T08:12:51.553863+0000","last_fullsized":"2024-07-04T08:12:51.553863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":276,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:17.219053+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:15:17.219053+0000","last_peered":"2024-07-04T08:15:17.219053+0000","last_clean":"2024-07-04T08:15:17.219053+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:15:17.219053+0000","last_undegraded":"2024-07-04T08:15:17.219053+0000","last_fullsized":"2024-07-04T08:15:17.219053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":255,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.151807+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:12:04.151807+0000","last_peered":"2024-07-04T08:12:04.151807+0000","last_clean":"2024-07-04T08:12:04.151807+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:12:04.151807+0000","last_undegraded":"2024-07-04T08:12:04.151807+0000","last_fullsized":"2024-07-04T08:12:04.151807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.561500+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:12:51.561500+0000","last_peered":"2024-07-04T08:12:51.561500+0000","last_clean":"2024-07-04T08:12:51.561500+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:12:51.561500+0000","last_undegraded":"2024-07-04T08:12:51.561500+0000","last_fullsized":"2024-07-04T08:12:51.561500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'62","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:12.794616+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:15:12.794616+0000","last_peered":"2024-07-04T08:15:12.794616+0000","last_clean":"2024-07-04T08:15:12.794616+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:15:12.794616+0000","last_undegraded":"2024-07-04T08:15:12.794616+0000","last_fullsized":"2024-07-04T08:15:12.794616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:09.641951+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:12:09.641951+0000","last_peered":"2024-07-04T08:12:09.641951+0000","last_clean":"2024-07-04T08:12:09.641951+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:12:09.641951+0000","last_undegraded":"2024-07-04T08:12:09.641951+0000","last_fullsized":"2024-07-04T08:12:09.641951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552528+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:12:51.552528+0000","last_peered":"2024-07-04T08:12:51.552528+0000","last_clean":"2024-07-04T08:12:51.552528+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:12:51.552528+0000","last_undegraded":"2024-07-04T08:12:51.552528+0000","last_fullsized":"2024-07-04T08:12:51.552528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995043+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:53.995043+0000","last_peered":"2024-07-04T08:11:53.995043+0000","last_clean":"2024-07-04T08:11:53.995043+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:53.995043+0000","last_undegraded":"2024-07-04T08:11:53.995043+0000","last_fullsized":"2024-07-04T08:11:53.995043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975432+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:12:04.975432+0000","last_peered":"2024-07-04T08:12:04.975432+0000","last_clean":"2024-07-04T08:12:04.975432+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:12:04.975432+0000","last_undegraded":"2024-07-04T08:12:04.975432+0000","last_fullsized":"2024-07-04T08:12:04.975432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531965+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:12:51.531965+0000","last_peered":"2024-07-04T08:12:51.531965+0000","last_clean":"2024-07-04T08:12:51.531965+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:12:51.531965+0000","last_undegraded":"2024-07-04T08:12:51.531965+0000","last_fullsized":"2024-07-04T08:12:51.531965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771354+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:53.771354+0000","last_peered":"2024-07-04T08:11:53.771354+0000","last_clean":"2024-07-04T08:11:53.771354+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:53.771354+0000","last_undegraded":"2024-07-04T08:11:53.771354+0000","last_fullsized":"2024-07-04T08:11:53.771354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975473+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:12:04.975473+0000","last_peered":"2024-07-04T08:12:04.975473+0000","last_clean":"2024-07-04T08:12:04.975473+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:12:04.975473+0000","last_undegraded":"2024-07-04T08:12:04.975473+0000","last_fullsized":"2024-07-04T08:12:04.975473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542769+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:12:51.542769+0000","last_peered":"2024-07-04T08:12:51.542769+0000","last_clean":"2024-07-04T08:12:51.542769+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:12:51.542769+0000","last_undegraded":"2024-07-04T08:12:51.542769+0000","last_fullsized":"2024-07-04T08:12:51.542769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:55.245889+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:55.245889+0000","last_peered":"2024-07-04T08:11:55.245889+0000","last_clean":"2024-07-04T08:11:55.245889+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:55.245889+0000","last_undegraded":"2024-07-04T08:11:55.245889+0000","last_fullsized":"2024-07-04T08:11:55.245889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.141070+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:12:18.141070+0000","last_peered":"2024-07-04T08:12:18.141070+0000","last_clean":"2024-07-04T08:12:18.141070+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:12:18.141070+0000","last_undegraded":"2024-07-04T08:12:18.141070+0000","last_fullsized":"2024-07-04T08:12:18.141070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552028+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:12:51.552028+0000","last_peered":"2024-07-04T08:12:51.552028+0000","last_clean":"2024-07-04T08:12:51.552028+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:12:51.552028+0000","last_undegraded":"2024-07-04T08:12:51.552028+0000","last_fullsized":"2024-07-04T08:12:51.552028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":312,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:19.541751+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:15:19.541751+0000","last_peered":"2024-07-04T08:15:19.541751+0000","last_clean":"2024-07-04T08:15:19.541751+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:15:19.541751+0000","last_undegraded":"2024-07-04T08:15:19.541751+0000","last_fullsized":"2024-07-04T08:15:19.541751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":260,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.666330+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:12:18.666330+0000","last_peered":"2024-07-04T08:12:18.666330+0000","last_clean":"2024-07-04T08:12:18.666330+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:12:18.666330+0000","last_undegraded":"2024-07-04T08:12:18.666330+0000","last_fullsized":"2024-07-04T08:12:18.666330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564982+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:12:51.564982+0000","last_peered":"2024-07-04T08:12:51.564982+0000","last_clean":"2024-07-04T08:12:51.564982+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:12:51.564982+0000","last_undegraded":"2024-07-04T08:12:51.564982+0000","last_fullsized":"2024-07-04T08:12:51.564982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.974209+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:54.974209+0000","last_peered":"2024-07-04T08:11:54.974209+0000","last_clean":"2024-07-04T08:11:54.974209+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:54.974209+0000","last_undegraded":"2024-07-04T08:11:54.974209+0000","last_fullsized":"2024-07-04T08:11:54.974209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:11.240822+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:12:11.240822+0000","last_peered":"2024-07-04T08:12:11.240822+0000","last_clean":"2024-07-04T08:12:11.240822+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:12:11.240822+0000","last_undegraded":"2024-07-04T08:12:11.240822+0000","last_fullsized":"2024-07-04T08:12:11.240822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562193+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:12:51.562193+0000","last_peered":"2024-07-04T08:12:51.562193+0000","last_clean":"2024-07-04T08:12:51.562193+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:12:51.562193+0000","last_undegraded":"2024-07-04T08:12:51.562193+0000","last_fullsized":"2024-07-04T08:12:51.562193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115841+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:56.115841+0000","last_peered":"2024-07-04T08:11:56.115841+0000","last_clean":"2024-07-04T08:11:56.115841+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:56.115841+0000","last_undegraded":"2024-07-04T08:11:56.115841+0000","last_fullsized":"2024-07-04T08:11:56.115841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.904331+0000","last_change":"2024-07-04T08:11:53.997034+0000","last_active":"2024-07-04T08:11:56.904331+0000","last_peered":"2024-07-04T08:11:56.904331+0000","last_clean":"2024-07-04T08:11:56.904331+0000","last_became_active":"2024-07-04T08:11:53.996456+0000","last_became_peered":"2024-07-04T08:11:53.996456+0000","last_unstale":"2024-07-04T08:11:56.904331+0000","last_undegraded":"2024-07-04T08:11:56.904331+0000","last_fullsized":"2024-07-04T08:11:56.904331+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.140549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.521874+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:12:51.521874+0000","last_peered":"2024-07-04T08:12:51.521874+0000","last_clean":"2024-07-04T08:12:51.521874+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:12:51.521874+0000","last_undegraded":"2024-07-04T08:12:51.521874+0000","last_fullsized":"2024-07-04T08:12:51.521874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":602,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:16.296128+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:15:16.296128+0000","last_peered":"2024-07-04T08:15:16.296128+0000","last_clean":"2024-07-04T08:15:16.296128+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:15:16.296128+0000","last_undegraded":"2024-07-04T08:15:16.296128+0000","last_fullsized":"2024-07-04T08:15:16.296128+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70064,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":515,"num_read_kb":327,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800726+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:12:14.800726+0000","last_peered":"2024-07-04T08:12:14.800726+0000","last_clean":"2024-07-04T08:12:14.800726+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:12:14.800726+0000","last_undegraded":"2024-07-04T08:12:14.800726+0000","last_fullsized":"2024-07-04T08:12:14.800726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541913+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:12:51.541913+0000","last_peered":"2024-07-04T08:12:51.541913+0000","last_clean":"2024-07-04T08:12:51.541913+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:12:51.541913+0000","last_undegraded":"2024-07-04T08:12:51.541913+0000","last_fullsized":"2024-07-04T08:12:51.541913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974814+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:59.974814+0000","last_peered":"2024-07-04T08:11:59.974814+0000","last_clean":"2024-07-04T08:11:59.974814+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:59.974814+0000","last_undegraded":"2024-07-04T08:11:59.974814+0000","last_fullsized":"2024-07-04T08:11:59.974814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:10.755408+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:12:10.755408+0000","last_peered":"2024-07-04T08:12:10.755408+0000","last_clean":"2024-07-04T08:12:10.755408+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:12:10.755408+0000","last_undegraded":"2024-07-04T08:12:10.755408+0000","last_fullsized":"2024-07-04T08:12:10.755408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569907+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:12:51.569907+0000","last_peered":"2024-07-04T08:12:51.569907+0000","last_clean":"2024-07-04T08:12:51.569907+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:12:51.569907+0000","last_undegraded":"2024-07-04T08:12:51.569907+0000","last_fullsized":"2024-07-04T08:12:51.569907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:01.086478+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:12:01.086478+0000","last_peered":"2024-07-04T08:12:01.086478+0000","last_clean":"2024-07-04T08:12:01.086478+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:12:01.086478+0000","last_undegraded":"2024-07-04T08:12:01.086478+0000","last_fullsized":"2024-07-04T08:12:01.086478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800668+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:12:14.800668+0000","last_peered":"2024-07-04T08:12:14.800668+0000","last_clean":"2024-07-04T08:12:14.800668+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:12:14.800668+0000","last_undegraded":"2024-07-04T08:12:14.800668+0000","last_fullsized":"2024-07-04T08:12:14.800668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.547189+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:12:51.547189+0000","last_peered":"2024-07-04T08:12:51.547189+0000","last_clean":"2024-07-04T08:12:51.547189+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:12:51.547189+0000","last_undegraded":"2024-07-04T08:12:51.547189+0000","last_fullsized":"2024-07-04T08:12:51.547189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":552,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:16.217198+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:15:16.217198+0000","last_peered":"2024-07-04T08:15:16.217198+0000","last_clean":"2024-07-04T08:15:16.217198+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:15:16.217198+0000","last_undegraded":"2024-07-04T08:15:16.217198+0000","last_fullsized":"2024-07-04T08:15:16.217198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":502,"num_read_kb":502,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.326445+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:12:14.326445+0000","last_peered":"2024-07-04T08:12:14.326445+0000","last_clean":"2024-07-04T08:12:14.326445+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:12:14.326445+0000","last_undegraded":"2024-07-04T08:12:14.326445+0000","last_fullsized":"2024-07-04T08:12:14.326445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.639390+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:11:54.639390+0000","last_peered":"2024-07-04T08:11:54.639390+0000","last_clean":"2024-07-04T08:11:54.639390+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:11:54.639390+0000","last_undegraded":"2024-07-04T08:11:54.639390+0000","last_fullsized":"2024-07-04T08:11:54.639390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564562+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:12:51.564562+0000","last_peered":"2024-07-04T08:12:51.564562+0000","last_clean":"2024-07-04T08:12:51.564562+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:12:51.564562+0000","last_undegraded":"2024-07-04T08:12:51.564562+0000","last_fullsized":"2024-07-04T08:12:51.564562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974768+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:59.974768+0000","last_peered":"2024-07-04T08:11:59.974768+0000","last_clean":"2024-07-04T08:11:59.974768+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:59.974768+0000","last_undegraded":"2024-07-04T08:11:59.974768+0000","last_fullsized":"2024-07-04T08:11:59.974768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.239030+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:11:56.239030+0000","last_peered":"2024-07-04T08:11:56.239030+0000","last_clean":"2024-07-04T08:11:56.239030+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:11:56.239030+0000","last_undegraded":"2024-07-04T08:11:56.239030+0000","last_fullsized":"2024-07-04T08:11:56.239030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565645+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:12:51.565645+0000","last_peered":"2024-07-04T08:12:51.565645+0000","last_clean":"2024-07-04T08:12:51.565645+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:12:51.565645+0000","last_undegraded":"2024-07-04T08:12:51.565645+0000","last_fullsized":"2024-07-04T08:12:51.565645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138191+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:12:03.138191+0000","last_peered":"2024-07-04T08:12:03.138191+0000","last_clean":"2024-07-04T08:12:03.138191+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:12:03.138191+0000","last_undegraded":"2024-07-04T08:12:03.138191+0000","last_fullsized":"2024-07-04T08:12:03.138191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:06.905597+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:12:06.905597+0000","last_peered":"2024-07-04T08:12:06.905597+0000","last_clean":"2024-07-04T08:12:06.905597+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:12:06.905597+0000","last_undegraded":"2024-07-04T08:12:06.905597+0000","last_fullsized":"2024-07-04T08:12:06.905597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.554669+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:12:51.554669+0000","last_peered":"2024-07-04T08:12:51.554669+0000","last_clean":"2024-07-04T08:12:51.554669+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:12:51.554669+0000","last_undegraded":"2024-07-04T08:12:51.554669+0000","last_fullsized":"2024-07-04T08:12:51.554669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118570+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:12:21.118570+0000","last_peered":"2024-07-04T08:12:21.118570+0000","last_clean":"2024-07-04T08:12:21.118570+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:12:21.118570+0000","last_undegraded":"2024-07-04T08:12:21.118570+0000","last_fullsized":"2024-07-04T08:12:21.118570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":307,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:18.265549+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:15:18.265549+0000","last_peered":"2024-07-04T08:15:18.265549+0000","last_clean":"2024-07-04T08:15:18.265549+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:15:18.265549+0000","last_undegraded":"2024-07-04T08:15:18.265549+0000","last_fullsized":"2024-07-04T08:15:18.265549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":256,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.519199+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:12:51.519199+0000","last_peered":"2024-07-04T08:12:51.519199+0000","last_clean":"2024-07-04T08:12:51.519199+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:12:51.519199+0000","last_undegraded":"2024-07-04T08:12:51.519199+0000","last_fullsized":"2024-07-04T08:12:51.519199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118593+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:12:21.118593+0000","last_peered":"2024-07-04T08:12:21.118593+0000","last_clean":"2024-07-04T08:12:21.118593+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:12:21.118593+0000","last_undegraded":"2024-07-04T08:12:21.118593+0000","last_fullsized":"2024-07-04T08:12:21.118593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534235+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:12:51.534235+0000","last_peered":"2024-07-04T08:12:51.534235+0000","last_clean":"2024-07-04T08:12:51.534235+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:12:51.534235+0000","last_undegraded":"2024-07-04T08:12:51.534235+0000","last_fullsized":"2024-07-04T08:12:51.534235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.539681+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:12:51.539681+0000","last_peered":"2024-07-04T08:12:51.539681+0000","last_clean":"2024-07-04T08:12:51.539681+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:12:51.539681+0000","last_undegraded":"2024-07-04T08:12:51.539681+0000","last_fullsized":"2024-07-04T08:12:51.539681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566399+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:12:51.566399+0000","last_peered":"2024-07-04T08:12:51.566399+0000","last_clean":"2024-07-04T08:12:51.566399+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:12:51.566399+0000","last_undegraded":"2024-07-04T08:12:51.566399+0000","last_fullsized":"2024-07-04T08:12:51.566399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567851+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:12:51.567851+0000","last_peered":"2024-07-04T08:12:51.567851+0000","last_clean":"2024-07-04T08:12:51.567851+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:12:51.567851+0000","last_undegraded":"2024-07-04T08:12:51.567851+0000","last_fullsized":"2024-07-04T08:12:51.567851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569222+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:12:51.569222+0000","last_peered":"2024-07-04T08:12:51.569222+0000","last_clean":"2024-07-04T08:12:51.569222+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:12:51.569222+0000","last_undegraded":"2024-07-04T08:12:51.569222+0000","last_fullsized":"2024-07-04T08:12:51.569222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532408+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:12:51.532408+0000","last_peered":"2024-07-04T08:12:51.532408+0000","last_clean":"2024-07-04T08:12:51.532408+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:12:51.532408+0000","last_undegraded":"2024-07-04T08:12:51.532408+0000","last_fullsized":"2024-07-04T08:12:51.532408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540538+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:12:51.540538+0000","last_peered":"2024-07-04T08:12:51.540538+0000","last_clean":"2024-07-04T08:12:51.540538+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:12:51.540538+0000","last_undegraded":"2024-07-04T08:12:51.540538+0000","last_fullsized":"2024-07-04T08:12:51.540538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552348+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:12:51.552348+0000","last_peered":"2024-07-04T08:12:51.552348+0000","last_clean":"2024-07-04T08:12:51.552348+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:12:51.552348+0000","last_undegraded":"2024-07-04T08:12:51.552348+0000","last_fullsized":"2024-07-04T08:12:51.552348+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531503+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:12:51.531503+0000","last_peered":"2024-07-04T08:12:51.531503+0000","last_clean":"2024-07-04T08:12:51.531503+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:12:51.531503+0000","last_undegraded":"2024-07-04T08:12:51.531503+0000","last_fullsized":"2024-07-04T08:12:51.531503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.555574+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:12:51.555574+0000","last_peered":"2024-07-04T08:12:51.555574+0000","last_clean":"2024-07-04T08:12:51.555574+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:12:51.555574+0000","last_undegraded":"2024-07-04T08:12:51.555574+0000","last_fullsized":"2024-07-04T08:12:51.555574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560802+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:12:51.560802+0000","last_peered":"2024-07-04T08:12:51.560802+0000","last_clean":"2024-07-04T08:12:51.560802+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:12:51.560802+0000","last_undegraded":"2024-07-04T08:12:51.560802+0000","last_fullsized":"2024-07-04T08:12:51.560802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570948+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:12:51.570948+0000","last_peered":"2024-07-04T08:12:51.570948+0000","last_clean":"2024-07-04T08:12:51.570948+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:12:51.570948+0000","last_undegraded":"2024-07-04T08:12:51.570948+0000","last_fullsized":"2024-07-04T08:12:51.570948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532752+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:12:51.532752+0000","last_peered":"2024-07-04T08:12:51.532752+0000","last_clean":"2024-07-04T08:12:51.532752+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:12:51.532752+0000","last_undegraded":"2024-07-04T08:12:51.532752+0000","last_fullsized":"2024-07-04T08:12:51.532752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552705+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:12:51.552705+0000","last_peered":"2024-07-04T08:12:51.552705+0000","last_clean":"2024-07-04T08:12:51.552705+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:12:51.552705+0000","last_undegraded":"2024-07-04T08:12:51.552705+0000","last_fullsized":"2024-07-04T08:12:51.552705+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541640+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:12:51.541640+0000","last_peered":"2024-07-04T08:12:51.541640+0000","last_clean":"2024-07-04T08:12:51.541640+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:12:51.541640+0000","last_undegraded":"2024-07-04T08:12:51.541640+0000","last_fullsized":"2024-07-04T08:12:51.541640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569708+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:12:51.569708+0000","last_peered":"2024-07-04T08:12:51.569708+0000","last_clean":"2024-07-04T08:12:51.569708+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:12:51.569708+0000","last_undegraded":"2024-07-04T08:12:51.569708+0000","last_fullsized":"2024-07-04T08:12:51.569708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.571140+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:12:51.571140+0000","last_peered":"2024-07-04T08:12:51.571140+0000","last_clean":"2024-07-04T08:12:51.571140+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:12:51.571140+0000","last_undegraded":"2024-07-04T08:12:51.571140+0000","last_fullsized":"2024-07-04T08:12:51.571140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552227+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:12:51.552227+0000","last_peered":"2024-07-04T08:12:51.552227+0000","last_clean":"2024-07-04T08:12:51.552227+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:12:51.552227+0000","last_undegraded":"2024-07-04T08:12:51.552227+0000","last_fullsized":"2024-07-04T08:12:51.552227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567284+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:12:51.567284+0000","last_peered":"2024-07-04T08:12:51.567284+0000","last_clean":"2024-07-04T08:12:51.567284+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:12:51.567284+0000","last_undegraded":"2024-07-04T08:12:51.567284+0000","last_fullsized":"2024-07-04T08:12:51.567284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551582+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:12:51.551582+0000","last_peered":"2024-07-04T08:12:51.551582+0000","last_clean":"2024-07-04T08:12:51.551582+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:12:51.551582+0000","last_undegraded":"2024-07-04T08:12:51.551582+0000","last_fullsized":"2024-07-04T08:12:51.551582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534085+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:12:51.534085+0000","last_peered":"2024-07-04T08:12:51.534085+0000","last_clean":"2024-07-04T08:12:51.534085+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:12:51.534085+0000","last_undegraded":"2024-07-04T08:12:51.534085+0000","last_fullsized":"2024-07-04T08:12:51.534085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553666+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:12:51.553666+0000","last_peered":"2024-07-04T08:12:51.553666+0000","last_clean":"2024-07-04T08:12:51.553666+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:12:51.553666+0000","last_undegraded":"2024-07-04T08:12:51.553666+0000","last_fullsized":"2024-07-04T08:12:51.553666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540373+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:12:51.540373+0000","last_peered":"2024-07-04T08:12:51.540373+0000","last_clean":"2024-07-04T08:12:51.540373+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:12:51.540373+0000","last_undegraded":"2024-07-04T08:12:51.540373+0000","last_fullsized":"2024-07-04T08:12:51.540373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541173+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:12:51.541173+0000","last_peered":"2024-07-04T08:12:51.541173+0000","last_clean":"2024-07-04T08:12:51.541173+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:12:51.541173+0000","last_undegraded":"2024-07-04T08:12:51.541173+0000","last_fullsized":"2024-07-04T08:12:51.541173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560317+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:12:51.560317+0000","last_peered":"2024-07-04T08:12:51.560317+0000","last_clean":"2024-07-04T08:12:51.560317+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:12:51.560317+0000","last_undegraded":"2024-07-04T08:12:51.560317+0000","last_fullsized":"2024-07-04T08:12:51.560317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551363+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:12:51.551363+0000","last_peered":"2024-07-04T08:12:51.551363+0000","last_clean":"2024-07-04T08:12:51.551363+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:12:51.551363+0000","last_undegraded":"2024-07-04T08:12:51.551363+0000","last_fullsized":"2024-07-04T08:12:51.551363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541562+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:12:51.541562+0000","last_peered":"2024-07-04T08:12:51.541562+0000","last_clean":"2024-07-04T08:12:51.541562+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:12:51.541562+0000","last_undegraded":"2024-07-04T08:12:51.541562+0000","last_fullsized":"2024-07-04T08:12:51.541562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.536909+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:12:51.536909+0000","last_peered":"2024-07-04T08:12:51.536909+0000","last_clean":"2024-07-04T08:12:51.536909+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:12:51.536909+0000","last_undegraded":"2024-07-04T08:12:51.536909+0000","last_fullsized":"2024-07-04T08:12:51.536909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537941+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:12:51.537941+0000","last_peered":"2024-07-04T08:12:51.537941+0000","last_clean":"2024-07-04T08:12:51.537941+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:12:51.537941+0000","last_undegraded":"2024-07-04T08:12:51.537941+0000","last_fullsized":"2024-07-04T08:12:51.537941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565574+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:12:51.565574+0000","last_peered":"2024-07-04T08:12:51.565574+0000","last_clean":"2024-07-04T08:12:51.565574+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:12:51.565574+0000","last_undegraded":"2024-07-04T08:12:51.565574+0000","last_fullsized":"2024-07-04T08:12:51.565574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568095+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:12:51.568095+0000","last_peered":"2024-07-04T08:12:51.568095+0000","last_clean":"2024-07-04T08:12:51.568095+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:12:51.568095+0000","last_undegraded":"2024-07-04T08:12:51.568095+0000","last_fullsized":"2024-07-04T08:12:51.568095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569523+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:12:51.569523+0000","last_peered":"2024-07-04T08:12:51.569523+0000","last_clean":"2024-07-04T08:12:51.569523+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:12:51.569523+0000","last_undegraded":"2024-07-04T08:12:51.569523+0000","last_fullsized":"2024-07-04T08:12:51.569523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":864542,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2909,"num_read_kb":1669,"num_write":487,"num_write_kb":2247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823296,"data_stored":2593626,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":369,"ondisk_log_size":369,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612694,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":936780,"data_compressed":51617,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808920,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1232896,"data_stored":1594752,"data_compressed":87398,"data_compressed_allocated":114688,"data_compressed_original":1048576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":940806,"data_compressed":51617,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168678,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":543872,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364908,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1141042,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528432,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":539530,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724662,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36472,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708680,"statfs":{"total":95995035648,"available":95957688320,"internally_reserved":0,"allocated":1159168,"data_stored":1529336,"data_compressed":87398,"data_compressed_allocated":114688,"data_compressed_original":1048576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920891,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":543604,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117120,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1114112,"data_stored":1325698,"data_compressed":69708,"data_compressed_allocated":94208,"data_compressed_original":851968,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378760,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":609142,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542284,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":611794,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705809,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":540454,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":787422,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":396596,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":466108,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":394408,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:15:23.273 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-04T08:15:23.273 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph pg dump --format=json 2024-07-04T08:15:23.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:22 smithi133 ceph-mon[37050]: pgmap v485: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-04T08:15:23.793 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:24.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:23 smithi050 ceph-mon[36577]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:24.149 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:23 smithi039 ceph-mon[32062]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:24.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:23 smithi133 ceph-mon[37050]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:25.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:24 smithi039 ceph-mon[32062]: pgmap v486: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-07-04T08:15:25.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:24 smithi050 ceph-mon[36577]: pgmap v486: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-07-04T08:15:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:24 smithi133 ceph-mon[37050]: pgmap v486: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-07-04T08:15:26.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.088+0000 7fc91e000640 1 -- 172.21.15.39:0/3114939653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91810aa30 msgr2=0x7fc91810ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:26.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.088+0000 7fc91e000640 1 --2- 172.21.15.39:0/3114939653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91810aa30 0x7fc91810ae40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc90002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.088+0000 7fc91e000640 1 -- 172.21.15.39:0/3114939653 shutdown_connections 2024-07-04T08:15:26.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.088+0000 7fc91e000640 1 --2- 172.21.15.39:0/3114939653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91810aa30 0x7fc91810ae40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.088+0000 7fc91e000640 1 --2- 172.21.15.39:0/3114939653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc91810b380 0x7fc918111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.088+0000 7fc91e000640 1 --2- 172.21.15.39:0/3114939653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9181050f0 0x7fc918105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.088+0000 7fc91e000640 1 -- 172.21.15.39:0/3114939653 >> 172.21.15.39:0/3114939653 conn(0x7fc918100b60 msgr2=0x7fc918102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:26.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.088+0000 7fc91e000640 1 -- 172.21.15.39:0/3114939653 shutdown_connections 2024-07-04T08:15:26.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.088+0000 7fc91e000640 1 -- 172.21.15.39:0/3114939653 wait complete. 2024-07-04T08:15:26.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.089+0000 7fc91e000640 1 Processor -- start 2024-07-04T08:15:26.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc91e000640 1 -- start start 2024-07-04T08:15:26.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc91e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9181050f0 0x7fc91819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:26.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc91e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91810aa30 0x7fc91819bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:26.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc91e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc91810b380 0x7fc9181a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:26.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc91e000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc918114ff0 con 0x7fc91810aa30 2024-07-04T08:15:26.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc91e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc918114e70 con 0x7fc9181050f0 2024-07-04T08:15:26.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc91e000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc918115170 con 0x7fc91810b380 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc917e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91810aa30 0x7fc91819bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc917e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91810aa30 0x7fc91819bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50956/0 (socket says 172.21.15.39:50956) 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc917e00640 1 -- 172.21.15.39:0/236556525 learned_addr learned my addr 172.21.15.39:0/236556525 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.090+0000 7fc91cc00640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9181050f0 0x7fc91819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.091+0000 7fc91d600640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc91810b380 0x7fc9181a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.091+0000 7fc91cc00640 1 -- 172.21.15.39:0/236556525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc91810b380 msgr2=0x7fc9181a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.091+0000 7fc91cc00640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc91810b380 0x7fc9181a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.091+0000 7fc91cc00640 1 -- 172.21.15.39:0/236556525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91810aa30 msgr2=0x7fc91819bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.091+0000 7fc91cc00640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91810aa30 0x7fc91819bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.091+0000 7fc91cc00640 1 -- 172.21.15.39:0/236556525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9181a19b0 con 0x7fc9181050f0 2024-07-04T08:15:26.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.091+0000 7fc91cc00640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9181050f0 0x7fc91819ba40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc90800e9f0 tx=0x7fc90800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:26.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.092+0000 7fc915600640 1 -- 172.21.15.39:0/236556525 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90800ccc0 con 0x7fc9181050f0 2024-07-04T08:15:26.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.092+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9181a1ca0 con 0x7fc9181050f0 2024-07-04T08:15:26.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.092+0000 7fc915600640 1 -- 172.21.15.39:0/236556525 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9080108e0 con 0x7fc9181050f0 2024-07-04T08:15:26.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.092+0000 7fc915600640 1 -- 172.21.15.39:0/236556525 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908018880 con 0x7fc9181050f0 2024-07-04T08:15:26.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.092+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9181a8400 con 0x7fc9181050f0 2024-07-04T08:15:26.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.093+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc918105500 con 0x7fc9181050f0 2024-07-04T08:15:26.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.095+0000 7fc915600640 1 -- 172.21.15.39:0/236556525 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7fc908010430 con 0x7fc9181050f0 2024-07-04T08:15:26.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.097+0000 7fc915600640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc8fc0b28d0 0x7fc8fc0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:26.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.097+0000 7fc915600640 1 -- 172.21.15.39:0/236556525 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc9080e3100 con 0x7fc9181050f0 2024-07-04T08:15:26.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.097+0000 7fc917e00640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc8fc0b28d0 0x7fc8fc0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:26.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.098+0000 7fc917e00640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc8fc0b28d0 0x7fc8fc0b4de0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc91810a510 tx=0x7fc900009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:26.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.098+0000 7fc915600640 1 -- 172.21.15.39:0/236556525 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc9080aa810 con 0x7fc9181050f0 2024-07-04T08:15:26.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.310+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc918062d50 con 0x7fc8fc0b28d0 2024-07-04T08:15:26.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.321+0000 7fc915600640 1 -- 172.21.15.39:0/236556525 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340954 (secure 0 0 0) 0x7fc918062d50 con 0x7fc8fc0b28d0 2024-07-04T08:15:26.329 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:26.330 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-07-04T08:15:26.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.328+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc8fc0b28d0 msgr2=0x7fc8fc0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:26.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.328+0000 7fc91e000640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc8fc0b28d0 0x7fc8fc0b4de0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc91810a510 tx=0x7fc900009560 comp rx=0 tx=0).stop 2024-07-04T08:15:26.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.328+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9181050f0 msgr2=0x7fc91819ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:26.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.328+0000 7fc91e000640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9181050f0 0x7fc91819ba40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc90800e9f0 tx=0x7fc90800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.329+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 shutdown_connections 2024-07-04T08:15:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.329+0000 7fc91e000640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc8fc0b28d0 0x7fc8fc0b4de0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.329+0000 7fc91e000640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91810aa30 0x7fc91819bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.329+0000 7fc91e000640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc91810b380 0x7fc9181a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.329+0000 7fc91e000640 1 --2- 172.21.15.39:0/236556525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9181050f0 0x7fc91819ba40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.329+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 >> 172.21.15.39:0/236556525 conn(0x7fc918100b60 msgr2=0x7fc918102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.329+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 shutdown_connections 2024-07-04T08:15:26.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:26.329+0000 7fc91e000640 1 -- 172.21.15.39:0/236556525 wait complete. 2024-07-04T08:15:27.169 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:26 smithi039 ceph-mon[32062]: pgmap v487: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-04T08:15:27.222 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-07-04T08:15:25.697000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1455144,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":3030,"num_read_kb":1763,"num_write":612,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":963,"ondisk_log_size":963,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378592,"kb_used_data":12788,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563232,"statfs":{"total":1151940427776,"available":1151552749568,"internally_reserved":0,"allocated":13094912,"data_stored":10856810,"data_compressed":611013,"data_compressed_allocated":823296,"data_compressed_original":5136384,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":451,"num_read_kb":356,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006670"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":321,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:23.483400+0000","last_change":"2024-07-04T08:09:34.294576+0000","last_active":"2024-07-04T08:15:23.483400+0000","last_peered":"2024-07-04T08:15:23.483400+0000","last_clean":"2024-07-04T08:15:23.483400+0000","last_became_active":"2024-07-04T08:09:34.292183+0000","last_became_peered":"2024-07-04T08:09:34.292183+0000","last_unstale":"2024-07-04T08:15:23.483400+0000","last_undegraded":"2024-07-04T08:15:23.483400+0000","last_fullsized":"2024-07-04T08:15:23.483400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:06:50.767573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":261,"num_read_kb":261,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553793+0000","last_change":"2024-07-04T08:09:38.375692+0000","last_active":"2024-07-04T08:12:51.553793+0000","last_peered":"2024-07-04T08:12:51.553793+0000","last_clean":"2024-07-04T08:12:51.553793+0000","last_became_active":"2024-07-04T08:09:38.374692+0000","last_became_peered":"2024-07-04T08:09:38.374692+0000","last_unstale":"2024-07-04T08:12:51.553793+0000","last_undegraded":"2024-07-04T08:12:51.553793+0000","last_fullsized":"2024-07-04T08:12:51.553793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:12:09.785516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:16.118017+0000","last_change":"2024-07-04T08:09:51.631088+0000","last_active":"2024-07-04T08:12:16.118017+0000","last_peered":"2024-07-04T08:12:16.118017+0000","last_clean":"2024-07-04T08:12:16.118017+0000","last_became_active":"2024-07-04T08:09:51.630730+0000","last_became_peered":"2024-07-04T08:09:51.630730+0000","last_unstale":"2024-07-04T08:12:16.118017+0000","last_undegraded":"2024-07-04T08:12:16.118017+0000","last_fullsized":"2024-07-04T08:12:16.118017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:26:01.108312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.698593+0000","last_change":"2024-07-04T08:09:34.528322+0000","last_active":"2024-07-04T08:11:53.698593+0000","last_peered":"2024-07-04T08:11:53.698593+0000","last_clean":"2024-07-04T08:11:53.698593+0000","last_became_active":"2024-07-04T08:09:34.526996+0000","last_became_peered":"2024-07-04T08:09:34.526996+0000","last_unstale":"2024-07-04T08:11:53.698593+0000","last_undegraded":"2024-07-04T08:11:53.698593+0000","last_fullsized":"2024-07-04T08:11:53.698593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:27:06.382042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569975+0000","last_change":"2024-07-04T08:09:38.372547+0000","last_active":"2024-07-04T08:12:51.569975+0000","last_peered":"2024-07-04T08:12:51.569975+0000","last_clean":"2024-07-04T08:12:51.569975+0000","last_became_active":"2024-07-04T08:09:38.371163+0000","last_became_peered":"2024-07-04T08:09:38.371163+0000","last_unstale":"2024-07-04T08:12:51.569975+0000","last_undegraded":"2024-07-04T08:12:51.569975+0000","last_fullsized":"2024-07-04T08:12:51.569975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:41:59.058145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:15.756154+0000","last_change":"2024-07-04T08:09:51.621076+0000","last_active":"2024-07-04T08:12:15.756154+0000","last_peered":"2024-07-04T08:12:15.756154+0000","last_clean":"2024-07-04T08:12:15.756154+0000","last_became_active":"2024-07-04T08:09:51.620298+0000","last_became_peered":"2024-07-04T08:09:51.620298+0000","last_unstale":"2024-07-04T08:12:15.756154+0000","last_undegraded":"2024-07-04T08:12:15.756154+0000","last_fullsized":"2024-07-04T08:12:15.756154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:45.315667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":69,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247127+0000","last_change":"2024-07-04T08:10:53.130984+0000","last_active":"2024-07-04T08:12:05.247127+0000","last_peered":"2024-07-04T08:12:05.247127+0000","last_clean":"2024-07-04T08:12:05.247127+0000","last_became_active":"2024-07-04T08:10:53.127948+0000","last_became_peered":"2024-07-04T08:10:53.127948+0000","last_unstale":"2024-07-04T08:12:05.247127+0000","last_undegraded":"2024-07-04T08:12:05.247127+0000","last_fullsized":"2024-07-04T08:12:05.247127+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:04:55.934500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.549821+0000","last_change":"2024-07-04T08:09:38.376256+0000","last_active":"2024-07-04T08:12:51.549821+0000","last_peered":"2024-07-04T08:12:51.549821+0000","last_clean":"2024-07-04T08:12:51.549821+0000","last_became_active":"2024-07-04T08:09:38.375810+0000","last_became_peered":"2024-07-04T08:09:38.375810+0000","last_unstale":"2024-07-04T08:12:51.549821+0000","last_undegraded":"2024-07-04T08:12:51.549821+0000","last_fullsized":"2024-07-04T08:12:51.549821+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:16:56.399533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756728+0000","last_change":"2024-07-04T08:09:51.629356+0000","last_active":"2024-07-04T08:12:20.756728+0000","last_peered":"2024-07-04T08:12:20.756728+0000","last_clean":"2024-07-04T08:12:20.756728+0000","last_became_active":"2024-07-04T08:09:51.628827+0000","last_became_peered":"2024-07-04T08:09:51.628827+0000","last_unstale":"2024-07-04T08:12:20.756728+0000","last_undegraded":"2024-07-04T08:12:20.756728+0000","last_fullsized":"2024-07-04T08:12:20.756728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:10:22.374365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.247079+0000","last_change":"2024-07-04T08:09:34.286561+0000","last_active":"2024-07-04T08:12:05.247079+0000","last_peered":"2024-07-04T08:12:05.247079+0000","last_clean":"2024-07-04T08:12:05.247079+0000","last_became_active":"2024-07-04T08:09:34.286148+0000","last_became_peered":"2024-07-04T08:09:34.286148+0000","last_unstale":"2024-07-04T08:12:05.247079+0000","last_undegraded":"2024-07-04T08:12:05.247079+0000","last_fullsized":"2024-07-04T08:12:05.247079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:00.315145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.535944+0000","last_change":"2024-07-04T08:09:38.354071+0000","last_active":"2024-07-04T08:12:51.535944+0000","last_peered":"2024-07-04T08:12:51.535944+0000","last_clean":"2024-07-04T08:12:51.535944+0000","last_became_active":"2024-07-04T08:09:38.353655+0000","last_became_peered":"2024-07-04T08:09:38.353655+0000","last_unstale":"2024-07-04T08:12:51.535944+0000","last_undegraded":"2024-07-04T08:12:51.535944+0000","last_fullsized":"2024-07-04T08:12:51.535944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:33:58.126524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:20.756706+0000","last_change":"2024-07-04T08:09:51.623322+0000","last_active":"2024-07-04T08:12:20.756706+0000","last_peered":"2024-07-04T08:12:20.756706+0000","last_clean":"2024-07-04T08:12:20.756706+0000","last_became_active":"2024-07-04T08:09:51.622828+0000","last_became_peered":"2024-07-04T08:09:51.622828+0000","last_unstale":"2024-07-04T08:12:20.756706+0000","last_undegraded":"2024-07-04T08:12:20.756706+0000","last_fullsized":"2024-07-04T08:12:20.756706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:20:26.155525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562540+0000","last_change":"2024-07-04T08:09:38.381162+0000","last_active":"2024-07-04T08:12:51.562540+0000","last_peered":"2024-07-04T08:12:51.562540+0000","last_clean":"2024-07-04T08:12:51.562540+0000","last_became_active":"2024-07-04T08:09:38.380770+0000","last_became_peered":"2024-07-04T08:09:38.380770+0000","last_unstale":"2024-07-04T08:12:51.562540+0000","last_undegraded":"2024-07-04T08:12:51.562540+0000","last_fullsized":"2024-07-04T08:12:51.562540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:19.847676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.844569+0000","last_change":"2024-07-04T08:10:52.664251+0000","last_active":"2024-07-04T08:11:53.844569+0000","last_peered":"2024-07-04T08:11:53.844569+0000","last_clean":"2024-07-04T08:11:53.844569+0000","last_became_active":"2024-07-04T08:10:52.660647+0000","last_became_peered":"2024-07-04T08:10:52.660647+0000","last_unstale":"2024-07-04T08:11:53.844569+0000","last_undegraded":"2024-07-04T08:11:53.844569+0000","last_fullsized":"2024-07-04T08:11:53.844569+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:54:35.099102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754878+0000","last_change":"2024-07-04T08:09:51.616261+0000","last_active":"2024-07-04T08:12:05.754878+0000","last_peered":"2024-07-04T08:12:05.754878+0000","last_clean":"2024-07-04T08:12:05.754878+0000","last_became_active":"2024-07-04T08:09:51.615869+0000","last_became_peered":"2024-07-04T08:09:51.615869+0000","last_unstale":"2024-07-04T08:12:05.754878+0000","last_undegraded":"2024-07-04T08:12:05.754878+0000","last_fullsized":"2024-07-04T08:12:05.754878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:38:01.601450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570735+0000","last_change":"2024-07-04T08:09:38.383523+0000","last_active":"2024-07-04T08:12:51.570735+0000","last_peered":"2024-07-04T08:12:51.570735+0000","last_clean":"2024-07-04T08:12:51.570735+0000","last_became_active":"2024-07-04T08:09:38.381636+0000","last_became_peered":"2024-07-04T08:09:38.381636+0000","last_unstale":"2024-07-04T08:12:51.570735+0000","last_undegraded":"2024-07-04T08:12:51.570735+0000","last_fullsized":"2024-07-04T08:12:51.570735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:30:00.771288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691548+0000","last_change":"2024-07-04T08:09:34.300281+0000","last_active":"2024-07-04T08:11:53.691548+0000","last_peered":"2024-07-04T08:11:53.691548+0000","last_clean":"2024-07-04T08:11:53.691548+0000","last_became_active":"2024-07-04T08:09:34.299864+0000","last_became_peered":"2024-07-04T08:09:34.299864+0000","last_unstale":"2024-07-04T08:11:53.691548+0000","last_undegraded":"2024-07-04T08:11:53.691548+0000","last_fullsized":"2024-07-04T08:11:53.691548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:28:10.267153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754825+0000","last_change":"2024-07-04T08:09:51.627929+0000","last_active":"2024-07-04T08:12:05.754825+0000","last_peered":"2024-07-04T08:12:05.754825+0000","last_clean":"2024-07-04T08:12:05.754825+0000","last_became_active":"2024-07-04T08:09:51.627231+0000","last_became_peered":"2024-07-04T08:09:51.627231+0000","last_unstale":"2024-07-04T08:12:05.754825+0000","last_undegraded":"2024-07-04T08:12:05.754825+0000","last_fullsized":"2024-07-04T08:12:05.754825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:57:00.540819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566667+0000","last_change":"2024-07-04T08:09:38.361605+0000","last_active":"2024-07-04T08:12:51.566667+0000","last_peered":"2024-07-04T08:12:51.566667+0000","last_clean":"2024-07-04T08:12:51.566667+0000","last_became_active":"2024-07-04T08:09:38.361204+0000","last_became_peered":"2024-07-04T08:09:38.361204+0000","last_unstale":"2024-07-04T08:12:51.566667+0000","last_undegraded":"2024-07-04T08:12:51.566667+0000","last_fullsized":"2024-07-04T08:12:51.566667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:07:10.152558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.697029+0000","last_change":"2024-07-04T08:09:34.294439+0000","last_active":"2024-07-04T08:11:53.697029+0000","last_peered":"2024-07-04T08:11:53.697029+0000","last_clean":"2024-07-04T08:11:53.697029+0000","last_became_active":"2024-07-04T08:09:34.293044+0000","last_became_peered":"2024-07-04T08:09:34.293044+0000","last_unstale":"2024-07-04T08:11:53.697029+0000","last_undegraded":"2024-07-04T08:11:53.697029+0000","last_fullsized":"2024-07-04T08:11:53.697029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:08:17.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:13:10.254268+0000","last_change":"2024-07-04T08:10:52.682137+0000","last_active":"2024-07-04T08:13:10.254268+0000","last_peered":"2024-07-04T08:13:10.254268+0000","last_clean":"2024-07-04T08:13:10.254268+0000","last_became_active":"2024-07-04T08:10:52.681676+0000","last_became_peered":"2024-07-04T08:10:52.681676+0000","last_unstale":"2024-07-04T08:13:10.254268+0000","last_undegraded":"2024-07-04T08:13:10.254268+0000","last_fullsized":"2024-07-04T08:13:10.254268+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:31:50.243382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.526597+0000","last_change":"2024-07-04T08:09:38.356481+0000","last_active":"2024-07-04T08:12:51.526597+0000","last_peered":"2024-07-04T08:12:51.526597+0000","last_clean":"2024-07-04T08:12:51.526597+0000","last_became_active":"2024-07-04T08:09:38.356106+0000","last_became_peered":"2024-07-04T08:09:38.356106+0000","last_unstale":"2024-07-04T08:12:51.526597+0000","last_undegraded":"2024-07-04T08:12:51.526597+0000","last_fullsized":"2024-07-04T08:12:51.526597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:50:06.009332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.936427+0000","last_change":"2024-07-04T08:09:34.526586+0000","last_active":"2024-07-04T08:11:53.936427+0000","last_peered":"2024-07-04T08:11:53.936427+0000","last_clean":"2024-07-04T08:11:53.936427+0000","last_became_active":"2024-07-04T08:09:34.526253+0000","last_became_peered":"2024-07-04T08:09:34.526253+0000","last_unstale":"2024-07-04T08:11:53.936427+0000","last_undegraded":"2024-07-04T08:11:53.936427+0000","last_fullsized":"2024-07-04T08:11:53.936427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:17:11.836361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:05.754887+0000","last_change":"2024-07-04T08:09:51.621134+0000","last_active":"2024-07-04T08:12:05.754887+0000","last_peered":"2024-07-04T08:12:05.754887+0000","last_clean":"2024-07-04T08:12:05.754887+0000","last_became_active":"2024-07-04T08:09:51.620425+0000","last_became_peered":"2024-07-04T08:09:51.620425+0000","last_unstale":"2024-07-04T08:12:05.754887+0000","last_undegraded":"2024-07-04T08:12:05.754887+0000","last_fullsized":"2024-07-04T08:12:05.754887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:46:51.742964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568004+0000","last_change":"2024-07-04T08:09:38.383023+0000","last_active":"2024-07-04T08:12:51.568004+0000","last_peered":"2024-07-04T08:12:51.568004+0000","last_clean":"2024-07-04T08:12:51.568004+0000","last_became_active":"2024-07-04T08:09:38.382624+0000","last_became_peered":"2024-07-04T08:09:38.382624+0000","last_unstale":"2024-07-04T08:12:51.568004+0000","last_undegraded":"2024-07-04T08:12:51.568004+0000","last_fullsized":"2024-07-04T08:12:51.568004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:37:54.305662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.692160+0000","last_change":"2024-07-04T08:09:34.291327+0000","last_active":"2024-07-04T08:11:53.692160+0000","last_peered":"2024-07-04T08:11:53.692160+0000","last_clean":"2024-07-04T08:11:53.692160+0000","last_became_active":"2024-07-04T08:09:34.290832+0000","last_became_peered":"2024-07-04T08:09:34.290832+0000","last_unstale":"2024-07-04T08:11:53.692160+0000","last_undegraded":"2024-07-04T08:11:53.692160+0000","last_fullsized":"2024-07-04T08:11:53.692160+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:22.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115754+0000","last_change":"2024-07-04T08:09:51.613607+0000","last_active":"2024-07-04T08:11:56.115754+0000","last_peered":"2024-07-04T08:11:56.115754+0000","last_clean":"2024-07-04T08:11:56.115754+0000","last_became_active":"2024-07-04T08:09:51.613076+0000","last_became_peered":"2024-07-04T08:09:51.613076+0000","last_unstale":"2024-07-04T08:11:56.115754+0000","last_undegraded":"2024-07-04T08:11:56.115754+0000","last_fullsized":"2024-07-04T08:11:56.115754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:14:15.108439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.556576+0000","last_change":"2024-07-04T08:09:38.367811+0000","last_active":"2024-07-04T08:12:51.556576+0000","last_peered":"2024-07-04T08:12:51.556576+0000","last_clean":"2024-07-04T08:12:51.556576+0000","last_became_active":"2024-07-04T08:09:38.367400+0000","last_became_peered":"2024-07-04T08:09:38.367400+0000","last_unstale":"2024-07-04T08:12:51.556576+0000","last_undegraded":"2024-07-04T08:12:51.556576+0000","last_fullsized":"2024-07-04T08:12:51.556576+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:35:43.272386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'101","reported_seq":246,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:13.400609+0000","last_change":"2024-07-04T08:10:52.688000+0000","last_active":"2024-07-04T08:15:13.400609+0000","last_peered":"2024-07-04T08:15:13.400609+0000","last_clean":"2024-07-04T08:15:13.400609+0000","last_became_active":"2024-07-04T08:10:52.680131+0000","last_became_peered":"2024-07-04T08:10:52.680131+0000","last_unstale":"2024-07-04T08:15:13.400609+0000","last_undegraded":"2024-07-04T08:15:13.400609+0000","last_fullsized":"2024-07-04T08:15:13.400609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:19.626441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":185,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.322378+0000","last_change":"2024-07-04T08:09:51.634340+0000","last_active":"2024-07-04T08:11:54.322378+0000","last_peered":"2024-07-04T08:11:54.322378+0000","last_clean":"2024-07-04T08:11:54.322378+0000","last_became_active":"2024-07-04T08:09:51.633216+0000","last_became_peered":"2024-07-04T08:09:51.633216+0000","last_unstale":"2024-07-04T08:11:54.322378+0000","last_undegraded":"2024-07-04T08:11:54.322378+0000","last_fullsized":"2024-07-04T08:11:54.322378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:28:15.055696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540856+0000","last_change":"2024-07-04T08:09:38.380685+0000","last_active":"2024-07-04T08:12:51.540856+0000","last_peered":"2024-07-04T08:12:51.540856+0000","last_clean":"2024-07-04T08:12:51.540856+0000","last_became_active":"2024-07-04T08:09:38.380223+0000","last_became_peered":"2024-07-04T08:09:38.380223+0000","last_unstale":"2024-07-04T08:12:51.540856+0000","last_undegraded":"2024-07-04T08:12:51.540856+0000","last_fullsized":"2024-07-04T08:12:51.540856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:15:21.702875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771431+0000","last_change":"2024-07-04T08:09:34.527824+0000","last_active":"2024-07-04T08:11:53.771431+0000","last_peered":"2024-07-04T08:11:53.771431+0000","last_clean":"2024-07-04T08:11:53.771431+0000","last_became_active":"2024-07-04T08:09:34.527238+0000","last_became_peered":"2024-07-04T08:09:34.527238+0000","last_unstale":"2024-07-04T08:11:53.771431+0000","last_undegraded":"2024-07-04T08:11:53.771431+0000","last_fullsized":"2024-07-04T08:11:53.771431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:15:03.781882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.238979+0000","last_change":"2024-07-04T08:09:51.642244+0000","last_active":"2024-07-04T08:11:56.238979+0000","last_peered":"2024-07-04T08:11:56.238979+0000","last_clean":"2024-07-04T08:11:56.238979+0000","last_became_active":"2024-07-04T08:09:51.641690+0000","last_became_peered":"2024-07-04T08:09:51.641690+0000","last_unstale":"2024-07-04T08:11:56.238979+0000","last_undegraded":"2024-07-04T08:11:56.238979+0000","last_fullsized":"2024-07-04T08:11:56.238979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:08:45.980854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542562+0000","last_change":"2024-07-04T08:09:38.384772+0000","last_active":"2024-07-04T08:12:51.542562+0000","last_peered":"2024-07-04T08:12:51.542562+0000","last_clean":"2024-07-04T08:12:51.542562+0000","last_became_active":"2024-07-04T08:09:38.384302+0000","last_became_peered":"2024-07-04T08:09:38.384302+0000","last_unstale":"2024-07-04T08:12:51.542562+0000","last_undegraded":"2024-07-04T08:12:51.542562+0000","last_fullsized":"2024-07-04T08:12:51.542562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:16:38.976770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.993415+0000","last_change":"2024-07-04T08:09:34.530804+0000","last_active":"2024-07-04T08:11:53.993415+0000","last_peered":"2024-07-04T08:11:53.993415+0000","last_clean":"2024-07-04T08:11:53.993415+0000","last_became_active":"2024-07-04T08:09:34.530325+0000","last_became_peered":"2024-07-04T08:09:34.530325+0000","last_unstale":"2024-07-04T08:11:53.993415+0000","last_undegraded":"2024-07-04T08:11:53.993415+0000","last_fullsized":"2024-07-04T08:11:53.993415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:56:37.158559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:00.246372+0000","last_change":"2024-07-04T08:09:51.625286+0000","last_active":"2024-07-04T08:12:00.246372+0000","last_peered":"2024-07-04T08:12:00.246372+0000","last_clean":"2024-07-04T08:12:00.246372+0000","last_became_active":"2024-07-04T08:09:51.624708+0000","last_became_peered":"2024-07-04T08:09:51.624708+0000","last_unstale":"2024-07-04T08:12:00.246372+0000","last_undegraded":"2024-07-04T08:12:00.246372+0000","last_fullsized":"2024-07-04T08:12:00.246372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:42:17.403603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.543737+0000","last_change":"2024-07-04T08:10:52.676059+0000","last_active":"2024-07-04T08:12:51.543737+0000","last_peered":"2024-07-04T08:12:51.543737+0000","last_clean":"2024-07-04T08:12:51.543737+0000","last_became_active":"2024-07-04T08:10:52.675698+0000","last_became_peered":"2024-07-04T08:10:52.675698+0000","last_unstale":"2024-07-04T08:12:51.543737+0000","last_undegraded":"2024-07-04T08:12:51.543737+0000","last_fullsized":"2024-07-04T08:12:51.543737+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:40:12.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":311,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:19.474591+0000","last_change":"2024-07-04T08:09:34.291861+0000","last_active":"2024-07-04T08:15:19.474591+0000","last_peered":"2024-07-04T08:15:19.474591+0000","last_clean":"2024-07-04T08:15:19.474591+0000","last_became_active":"2024-07-04T08:09:34.291351+0000","last_became_peered":"2024-07-04T08:09:34.291351+0000","last_unstale":"2024-07-04T08:15:19.474591+0000","last_undegraded":"2024-07-04T08:15:19.474591+0000","last_fullsized":"2024-07-04T08:15:19.474591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:13:23.173089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":258,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:58.137211+0000","last_change":"2024-07-04T08:09:51.642150+0000","last_active":"2024-07-04T08:11:58.137211+0000","last_peered":"2024-07-04T08:11:58.137211+0000","last_clean":"2024-07-04T08:11:58.137211+0000","last_became_active":"2024-07-04T08:09:51.640652+0000","last_became_peered":"2024-07-04T08:09:51.640652+0000","last_unstale":"2024-07-04T08:11:58.137211+0000","last_undegraded":"2024-07-04T08:11:58.137211+0000","last_fullsized":"2024-07-04T08:11:58.137211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:46:08.543887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.516953+0000","last_change":"2024-07-04T08:09:38.356557+0000","last_active":"2024-07-04T08:12:51.516953+0000","last_peered":"2024-07-04T08:12:51.516953+0000","last_clean":"2024-07-04T08:12:51.516953+0000","last_became_active":"2024-07-04T08:09:38.356207+0000","last_became_peered":"2024-07-04T08:09:38.356207+0000","last_unstale":"2024-07-04T08:12:51.516953+0000","last_undegraded":"2024-07-04T08:12:51.516953+0000","last_fullsized":"2024-07-04T08:12:51.516953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:39:16.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":310,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:21.546283+0000","last_change":"2024-07-04T08:09:34.529848+0000","last_active":"2024-07-04T08:15:21.546283+0000","last_peered":"2024-07-04T08:15:21.546283+0000","last_clean":"2024-07-04T08:15:21.546283+0000","last_became_active":"2024-07-04T08:09:34.529074+0000","last_became_peered":"2024-07-04T08:09:34.529074+0000","last_unstale":"2024-07-04T08:15:21.546283+0000","last_undegraded":"2024-07-04T08:15:21.546283+0000","last_fullsized":"2024-07-04T08:15:21.546283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:15:44.051941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":258,"num_read_kb":258,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138305+0000","last_change":"2024-07-04T08:09:51.643216+0000","last_active":"2024-07-04T08:12:03.138305+0000","last_peered":"2024-07-04T08:12:03.138305+0000","last_clean":"2024-07-04T08:12:03.138305+0000","last_became_active":"2024-07-04T08:09:51.640010+0000","last_became_peered":"2024-07-04T08:09:51.640010+0000","last_unstale":"2024-07-04T08:12:03.138305+0000","last_undegraded":"2024-07-04T08:12:03.138305+0000","last_fullsized":"2024-07-04T08:12:03.138305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:04:30.115049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534873+0000","last_change":"2024-07-04T08:09:38.379597+0000","last_active":"2024-07-04T08:12:51.534873+0000","last_peered":"2024-07-04T08:12:51.534873+0000","last_clean":"2024-07-04T08:12:51.534873+0000","last_became_active":"2024-07-04T08:09:38.379143+0000","last_became_peered":"2024-07-04T08:09:38.379143+0000","last_unstale":"2024-07-04T08:12:51.534873+0000","last_undegraded":"2024-07-04T08:12:51.534873+0000","last_fullsized":"2024-07-04T08:12:51.534873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:57:02.376954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.851437+0000","last_change":"2024-07-04T08:09:34.287964+0000","last_active":"2024-07-04T08:11:53.851437+0000","last_peered":"2024-07-04T08:11:53.851437+0000","last_clean":"2024-07-04T08:11:53.851437+0000","last_became_active":"2024-07-04T08:09:34.286542+0000","last_became_peered":"2024-07-04T08:09:34.286542+0000","last_unstale":"2024-07-04T08:11:53.851437+0000","last_undegraded":"2024-07-04T08:11:53.851437+0000","last_fullsized":"2024-07-04T08:11:53.851437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:38.290885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975520+0000","last_change":"2024-07-04T08:10:52.684905+0000","last_active":"2024-07-04T08:12:04.975520+0000","last_peered":"2024-07-04T08:12:04.975520+0000","last_clean":"2024-07-04T08:12:04.975520+0000","last_became_active":"2024-07-04T08:10:52.684276+0000","last_became_peered":"2024-07-04T08:10:52.684276+0000","last_unstale":"2024-07-04T08:12:04.975520+0000","last_undegraded":"2024-07-04T08:12:04.975520+0000","last_fullsized":"2024-07-04T08:12:04.975520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:05:18.413497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.558356+0000","last_change":"2024-07-04T08:09:38.370879+0000","last_active":"2024-07-04T08:12:51.558356+0000","last_peered":"2024-07-04T08:12:51.558356+0000","last_clean":"2024-07-04T08:12:51.558356+0000","last_became_active":"2024-07-04T08:09:38.370426+0000","last_became_peered":"2024-07-04T08:09:38.370426+0000","last_unstale":"2024-07-04T08:12:51.558356+0000","last_undegraded":"2024-07-04T08:12:51.558356+0000","last_fullsized":"2024-07-04T08:12:51.558356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:32:56.782352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.691883+0000","last_change":"2024-07-04T08:09:34.296329+0000","last_active":"2024-07-04T08:11:53.691883+0000","last_peered":"2024-07-04T08:11:53.691883+0000","last_clean":"2024-07-04T08:11:53.691883+0000","last_became_active":"2024-07-04T08:09:34.295820+0000","last_became_peered":"2024-07-04T08:09:34.295820+0000","last_unstale":"2024-07-04T08:11:53.691883+0000","last_undegraded":"2024-07-04T08:11:53.691883+0000","last_fullsized":"2024-07-04T08:11:53.691883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:26.715370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":120,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:46.091830+0000","last_change":"2024-07-04T08:08:08.233769+0000","last_active":"2024-07-04T08:12:46.091830+0000","last_peered":"2024-07-04T08:12:46.091830+0000","last_clean":"2024-07-04T08:12:46.091830+0000","last_became_active":"2024-07-04T08:08:08.227173+0000","last_became_peered":"2024-07-04T08:08:08.227173+0000","last_unstale":"2024-07-04T08:12:46.091830+0000","last_undegraded":"2024-07-04T08:12:46.091830+0000","last_fullsized":"2024-07-04T08:12:46.091830+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:04:17.542334+0000","last_clean_scrub_stamp":"2024-07-04T08:04:17.542334+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:35:38.441697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.085858+0000","last_change":"2024-07-04T08:09:51.625132+0000","last_active":"2024-07-04T08:11:56.085858+0000","last_peered":"2024-07-04T08:11:56.085858+0000","last_clean":"2024-07-04T08:11:56.085858+0000","last_became_active":"2024-07-04T08:09:51.624658+0000","last_became_peered":"2024-07-04T08:09:51.624658+0000","last_unstale":"2024-07-04T08:11:56.085858+0000","last_undegraded":"2024-07-04T08:11:56.085858+0000","last_fullsized":"2024-07-04T08:11:56.085858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:01:50.431044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553863+0000","last_change":"2024-07-04T08:09:38.358450+0000","last_active":"2024-07-04T08:12:51.553863+0000","last_peered":"2024-07-04T08:12:51.553863+0000","last_clean":"2024-07-04T08:12:51.553863+0000","last_became_active":"2024-07-04T08:09:38.358086+0000","last_became_peered":"2024-07-04T08:09:38.358086+0000","last_unstale":"2024-07-04T08:12:51.553863+0000","last_undegraded":"2024-07-04T08:12:51.553863+0000","last_fullsized":"2024-07-04T08:12:51.553863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:35:14.559474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":282,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:23.227312+0000","last_change":"2024-07-04T08:10:52.670382+0000","last_active":"2024-07-04T08:15:23.227312+0000","last_peered":"2024-07-04T08:15:23.227312+0000","last_clean":"2024-07-04T08:15:23.227312+0000","last_became_active":"2024-07-04T08:10:52.670013+0000","last_became_peered":"2024-07-04T08:10:52.670013+0000","last_unstale":"2024-07-04T08:15:23.227312+0000","last_undegraded":"2024-07-04T08:15:23.227312+0000","last_fullsized":"2024-07-04T08:15:23.227312+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:11:49.654026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":260,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.151807+0000","last_change":"2024-07-04T08:09:51.622079+0000","last_active":"2024-07-04T08:12:04.151807+0000","last_peered":"2024-07-04T08:12:04.151807+0000","last_clean":"2024-07-04T08:12:04.151807+0000","last_became_active":"2024-07-04T08:09:51.621400+0000","last_became_peered":"2024-07-04T08:09:51.621400+0000","last_unstale":"2024-07-04T08:12:04.151807+0000","last_undegraded":"2024-07-04T08:12:04.151807+0000","last_fullsized":"2024-07-04T08:12:04.151807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:54:10.822810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.561500+0000","last_change":"2024-07-04T08:09:38.379208+0000","last_active":"2024-07-04T08:12:51.561500+0000","last_peered":"2024-07-04T08:12:51.561500+0000","last_clean":"2024-07-04T08:12:51.561500+0000","last_became_active":"2024-07-04T08:09:38.378760+0000","last_became_peered":"2024-07-04T08:09:38.378760+0000","last_unstale":"2024-07-04T08:12:51.561500+0000","last_undegraded":"2024-07-04T08:12:51.561500+0000","last_fullsized":"2024-07-04T08:12:51.561500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:25:56.223072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'62","reported_seq":138,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:12.794616+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:15:12.794616+0000","last_peered":"2024-07-04T08:15:12.794616+0000","last_clean":"2024-07-04T08:15:12.794616+0000","last_became_active":"2024-07-04T08:09:34.295759+0000","last_became_peered":"2024-07-04T08:09:34.295759+0000","last_unstale":"2024-07-04T08:15:12.794616+0000","last_undegraded":"2024-07-04T08:15:12.794616+0000","last_fullsized":"2024-07-04T08:15:12.794616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:05:49.173454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:09.641951+0000","last_change":"2024-07-04T08:09:51.632077+0000","last_active":"2024-07-04T08:12:09.641951+0000","last_peered":"2024-07-04T08:12:09.641951+0000","last_clean":"2024-07-04T08:12:09.641951+0000","last_became_active":"2024-07-04T08:09:51.629663+0000","last_became_peered":"2024-07-04T08:09:51.629663+0000","last_unstale":"2024-07-04T08:12:09.641951+0000","last_undegraded":"2024-07-04T08:12:09.641951+0000","last_fullsized":"2024-07-04T08:12:09.641951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:41.360549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552528+0000","last_change":"2024-07-04T08:09:38.360636+0000","last_active":"2024-07-04T08:12:51.552528+0000","last_peered":"2024-07-04T08:12:51.552528+0000","last_clean":"2024-07-04T08:12:51.552528+0000","last_became_active":"2024-07-04T08:09:38.360213+0000","last_became_peered":"2024-07-04T08:09:38.360213+0000","last_unstale":"2024-07-04T08:12:51.552528+0000","last_undegraded":"2024-07-04T08:12:51.552528+0000","last_fullsized":"2024-07-04T08:12:51.552528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:47:16.182393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.995043+0000","last_change":"2024-07-04T08:09:34.297622+0000","last_active":"2024-07-04T08:11:53.995043+0000","last_peered":"2024-07-04T08:11:53.995043+0000","last_clean":"2024-07-04T08:11:53.995043+0000","last_became_active":"2024-07-04T08:09:34.296179+0000","last_became_peered":"2024-07-04T08:09:34.296179+0000","last_unstale":"2024-07-04T08:11:53.995043+0000","last_undegraded":"2024-07-04T08:11:53.995043+0000","last_fullsized":"2024-07-04T08:11:53.995043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:46:03.430565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975432+0000","last_change":"2024-07-04T08:09:51.611989+0000","last_active":"2024-07-04T08:12:04.975432+0000","last_peered":"2024-07-04T08:12:04.975432+0000","last_clean":"2024-07-04T08:12:04.975432+0000","last_became_active":"2024-07-04T08:09:51.611520+0000","last_became_peered":"2024-07-04T08:09:51.611520+0000","last_unstale":"2024-07-04T08:12:04.975432+0000","last_undegraded":"2024-07-04T08:12:04.975432+0000","last_fullsized":"2024-07-04T08:12:04.975432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:52:38.517496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531965+0000","last_change":"2024-07-04T08:09:38.359652+0000","last_active":"2024-07-04T08:12:51.531965+0000","last_peered":"2024-07-04T08:12:51.531965+0000","last_clean":"2024-07-04T08:12:51.531965+0000","last_became_active":"2024-07-04T08:09:38.359123+0000","last_became_peered":"2024-07-04T08:09:38.359123+0000","last_unstale":"2024-07-04T08:12:51.531965+0000","last_undegraded":"2024-07-04T08:12:51.531965+0000","last_fullsized":"2024-07-04T08:12:51.531965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:08:18.101895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:53.771354+0000","last_change":"2024-07-04T08:09:34.287922+0000","last_active":"2024-07-04T08:11:53.771354+0000","last_peered":"2024-07-04T08:11:53.771354+0000","last_clean":"2024-07-04T08:11:53.771354+0000","last_became_active":"2024-07-04T08:09:34.287527+0000","last_became_peered":"2024-07-04T08:09:34.287527+0000","last_unstale":"2024-07-04T08:11:53.771354+0000","last_undegraded":"2024-07-04T08:11:53.771354+0000","last_fullsized":"2024-07-04T08:11:53.771354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:44:23.268385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:04.975473+0000","last_change":"2024-07-04T08:09:51.625263+0000","last_active":"2024-07-04T08:12:04.975473+0000","last_peered":"2024-07-04T08:12:04.975473+0000","last_clean":"2024-07-04T08:12:04.975473+0000","last_became_active":"2024-07-04T08:09:51.624773+0000","last_became_peered":"2024-07-04T08:09:51.624773+0000","last_unstale":"2024-07-04T08:12:04.975473+0000","last_undegraded":"2024-07-04T08:12:04.975473+0000","last_fullsized":"2024-07-04T08:12:04.975473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:51.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.542769+0000","last_change":"2024-07-04T08:10:53.127531+0000","last_active":"2024-07-04T08:12:51.542769+0000","last_peered":"2024-07-04T08:12:51.542769+0000","last_clean":"2024-07-04T08:12:51.542769+0000","last_became_active":"2024-07-04T08:10:53.126553+0000","last_became_peered":"2024-07-04T08:10:53.126553+0000","last_unstale":"2024-07-04T08:12:51.542769+0000","last_undegraded":"2024-07-04T08:12:51.542769+0000","last_fullsized":"2024-07-04T08:12:51.542769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:00:49.116400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:55.245889+0000","last_change":"2024-07-04T08:09:34.527687+0000","last_active":"2024-07-04T08:11:55.245889+0000","last_peered":"2024-07-04T08:11:55.245889+0000","last_clean":"2024-07-04T08:11:55.245889+0000","last_became_active":"2024-07-04T08:09:34.527296+0000","last_became_peered":"2024-07-04T08:09:34.527296+0000","last_unstale":"2024-07-04T08:11:55.245889+0000","last_undegraded":"2024-07-04T08:11:55.245889+0000","last_fullsized":"2024-07-04T08:11:55.245889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:03:04.869985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.141070+0000","last_change":"2024-07-04T08:09:51.642937+0000","last_active":"2024-07-04T08:12:18.141070+0000","last_peered":"2024-07-04T08:12:18.141070+0000","last_clean":"2024-07-04T08:12:18.141070+0000","last_became_active":"2024-07-04T08:09:51.642120+0000","last_became_peered":"2024-07-04T08:09:51.642120+0000","last_unstale":"2024-07-04T08:12:18.141070+0000","last_undegraded":"2024-07-04T08:12:18.141070+0000","last_fullsized":"2024-07-04T08:12:18.141070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:09:07.964204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552028+0000","last_change":"2024-07-04T08:09:38.356539+0000","last_active":"2024-07-04T08:12:51.552028+0000","last_peered":"2024-07-04T08:12:51.552028+0000","last_clean":"2024-07-04T08:12:51.552028+0000","last_became_active":"2024-07-04T08:09:38.356136+0000","last_became_peered":"2024-07-04T08:09:38.356136+0000","last_unstale":"2024-07-04T08:12:51.552028+0000","last_undegraded":"2024-07-04T08:12:51.552028+0000","last_fullsized":"2024-07-04T08:12:51.552028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:02:34.067846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'7","reported_seq":312,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:19.541751+0000","last_change":"2024-07-04T08:09:34.287990+0000","last_active":"2024-07-04T08:15:19.541751+0000","last_peered":"2024-07-04T08:15:19.541751+0000","last_clean":"2024-07-04T08:15:19.541751+0000","last_became_active":"2024-07-04T08:09:34.287462+0000","last_became_peered":"2024-07-04T08:09:34.287462+0000","last_unstale":"2024-07-04T08:15:19.541751+0000","last_undegraded":"2024-07-04T08:15:19.541751+0000","last_fullsized":"2024-07-04T08:15:19.541751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:06:10.294738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":260,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:18.666330+0000","last_change":"2024-07-04T08:09:51.631588+0000","last_active":"2024-07-04T08:12:18.666330+0000","last_peered":"2024-07-04T08:12:18.666330+0000","last_clean":"2024-07-04T08:12:18.666330+0000","last_became_active":"2024-07-04T08:09:51.631087+0000","last_became_peered":"2024-07-04T08:09:51.631087+0000","last_unstale":"2024-07-04T08:12:18.666330+0000","last_undegraded":"2024-07-04T08:12:18.666330+0000","last_fullsized":"2024-07-04T08:12:18.666330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:42:39.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564982+0000","last_change":"2024-07-04T08:09:38.390850+0000","last_active":"2024-07-04T08:12:51.564982+0000","last_peered":"2024-07-04T08:12:51.564982+0000","last_clean":"2024-07-04T08:12:51.564982+0000","last_became_active":"2024-07-04T08:09:38.390432+0000","last_became_peered":"2024-07-04T08:09:38.390432+0000","last_unstale":"2024-07-04T08:12:51.564982+0000","last_undegraded":"2024-07-04T08:12:51.564982+0000","last_fullsized":"2024-07-04T08:12:51.564982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:33:09.782967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.974209+0000","last_change":"2024-07-04T08:09:34.296238+0000","last_active":"2024-07-04T08:11:54.974209+0000","last_peered":"2024-07-04T08:11:54.974209+0000","last_clean":"2024-07-04T08:11:54.974209+0000","last_became_active":"2024-07-04T08:09:34.295792+0000","last_became_peered":"2024-07-04T08:09:34.295792+0000","last_unstale":"2024-07-04T08:11:54.974209+0000","last_undegraded":"2024-07-04T08:11:54.974209+0000","last_fullsized":"2024-07-04T08:11:54.974209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:39:37.190286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:11.240822+0000","last_change":"2024-07-04T08:10:52.683951+0000","last_active":"2024-07-04T08:12:11.240822+0000","last_peered":"2024-07-04T08:12:11.240822+0000","last_clean":"2024-07-04T08:12:11.240822+0000","last_became_active":"2024-07-04T08:10:52.683493+0000","last_became_peered":"2024-07-04T08:10:52.683493+0000","last_unstale":"2024-07-04T08:12:11.240822+0000","last_undegraded":"2024-07-04T08:12:11.240822+0000","last_fullsized":"2024-07-04T08:12:11.240822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:01:11.647758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.562193+0000","last_change":"2024-07-04T08:09:38.385889+0000","last_active":"2024-07-04T08:12:51.562193+0000","last_peered":"2024-07-04T08:12:51.562193+0000","last_clean":"2024-07-04T08:12:51.562193+0000","last_became_active":"2024-07-04T08:09:38.385400+0000","last_became_peered":"2024-07-04T08:09:38.385400+0000","last_unstale":"2024-07-04T08:12:51.562193+0000","last_undegraded":"2024-07-04T08:12:51.562193+0000","last_fullsized":"2024-07-04T08:12:51.562193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:51:31.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.115841+0000","last_change":"2024-07-04T08:09:34.293048+0000","last_active":"2024-07-04T08:11:56.115841+0000","last_peered":"2024-07-04T08:11:56.115841+0000","last_clean":"2024-07-04T08:11:56.115841+0000","last_became_active":"2024-07-04T08:09:34.292682+0000","last_became_peered":"2024-07-04T08:09:34.292682+0000","last_unstale":"2024-07-04T08:11:56.115841+0000","last_undegraded":"2024-07-04T08:11:56.115841+0000","last_fullsized":"2024-07-04T08:11:56.115841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:45:12.335486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.904331+0000","last_change":"2024-07-04T08:11:53.997034+0000","last_active":"2024-07-04T08:11:56.904331+0000","last_peered":"2024-07-04T08:11:56.904331+0000","last_clean":"2024-07-04T08:11:56.904331+0000","last_became_active":"2024-07-04T08:11:53.996456+0000","last_became_peered":"2024-07-04T08:11:53.996456+0000","last_unstale":"2024-07-04T08:11:56.904331+0000","last_undegraded":"2024-07-04T08:11:56.904331+0000","last_fullsized":"2024-07-04T08:11:56.904331+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:44:33.140549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,3,6],"acting":[10,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.521874+0000","last_change":"2024-07-04T08:09:38.356807+0000","last_active":"2024-07-04T08:12:51.521874+0000","last_peered":"2024-07-04T08:12:51.521874+0000","last_clean":"2024-07-04T08:12:51.521874+0000","last_became_active":"2024-07-04T08:09:38.356451+0000","last_became_peered":"2024-07-04T08:09:38.356451+0000","last_unstale":"2024-07-04T08:12:51.521874+0000","last_undegraded":"2024-07-04T08:12:51.521874+0000","last_fullsized":"2024-07-04T08:12:51.521874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:27:25.453288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":612,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:21.304504+0000","last_change":"2024-07-04T08:09:34.285204+0000","last_active":"2024-07-04T08:15:21.304504+0000","last_peered":"2024-07-04T08:15:21.304504+0000","last_clean":"2024-07-04T08:15:21.304504+0000","last_became_active":"2024-07-04T08:09:34.284801+0000","last_became_peered":"2024-07-04T08:09:34.284801+0000","last_unstale":"2024-07-04T08:15:21.304504+0000","last_undegraded":"2024-07-04T08:15:21.304504+0000","last_fullsized":"2024-07-04T08:15:21.304504+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:21:18.959996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70064,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":525,"num_read_kb":332,"num_write":43,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800726+0000","last_change":"2024-07-04T08:09:51.623449+0000","last_active":"2024-07-04T08:12:14.800726+0000","last_peered":"2024-07-04T08:12:14.800726+0000","last_clean":"2024-07-04T08:12:14.800726+0000","last_became_active":"2024-07-04T08:09:51.620077+0000","last_became_peered":"2024-07-04T08:09:51.620077+0000","last_unstale":"2024-07-04T08:12:14.800726+0000","last_undegraded":"2024-07-04T08:12:14.800726+0000","last_fullsized":"2024-07-04T08:12:14.800726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:27:03.730464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541913+0000","last_change":"2024-07-04T08:09:38.356728+0000","last_active":"2024-07-04T08:12:51.541913+0000","last_peered":"2024-07-04T08:12:51.541913+0000","last_clean":"2024-07-04T08:12:51.541913+0000","last_became_active":"2024-07-04T08:09:38.356293+0000","last_became_peered":"2024-07-04T08:09:38.356293+0000","last_unstale":"2024-07-04T08:12:51.541913+0000","last_undegraded":"2024-07-04T08:12:51.541913+0000","last_fullsized":"2024-07-04T08:12:51.541913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:45:11.437198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974814+0000","last_change":"2024-07-04T08:09:34.528519+0000","last_active":"2024-07-04T08:11:59.974814+0000","last_peered":"2024-07-04T08:11:59.974814+0000","last_clean":"2024-07-04T08:11:59.974814+0000","last_became_active":"2024-07-04T08:09:34.527975+0000","last_became_peered":"2024-07-04T08:09:34.527975+0000","last_unstale":"2024-07-04T08:11:59.974814+0000","last_undegraded":"2024-07-04T08:11:59.974814+0000","last_fullsized":"2024-07-04T08:11:59.974814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:19:21.441244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:10.755408+0000","last_change":"2024-07-04T08:09:51.629288+0000","last_active":"2024-07-04T08:12:10.755408+0000","last_peered":"2024-07-04T08:12:10.755408+0000","last_clean":"2024-07-04T08:12:10.755408+0000","last_became_active":"2024-07-04T08:09:51.628801+0000","last_became_peered":"2024-07-04T08:09:51.628801+0000","last_unstale":"2024-07-04T08:12:10.755408+0000","last_undegraded":"2024-07-04T08:12:10.755408+0000","last_fullsized":"2024-07-04T08:12:10.755408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:07:52.029606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569907+0000","last_change":"2024-07-04T08:09:38.367424+0000","last_active":"2024-07-04T08:12:51.569907+0000","last_peered":"2024-07-04T08:12:51.569907+0000","last_clean":"2024-07-04T08:12:51.569907+0000","last_became_active":"2024-07-04T08:09:38.365985+0000","last_became_peered":"2024-07-04T08:09:38.365985+0000","last_unstale":"2024-07-04T08:12:51.569907+0000","last_undegraded":"2024-07-04T08:12:51.569907+0000","last_fullsized":"2024-07-04T08:12:51.569907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:28:19.797007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:01.086478+0000","last_change":"2024-07-04T08:09:34.289776+0000","last_active":"2024-07-04T08:12:01.086478+0000","last_peered":"2024-07-04T08:12:01.086478+0000","last_clean":"2024-07-04T08:12:01.086478+0000","last_became_active":"2024-07-04T08:09:34.289176+0000","last_became_peered":"2024-07-04T08:09:34.289176+0000","last_unstale":"2024-07-04T08:12:01.086478+0000","last_undegraded":"2024-07-04T08:12:01.086478+0000","last_fullsized":"2024-07-04T08:12:01.086478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:17:53.292622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.800668+0000","last_change":"2024-07-04T08:09:51.637195+0000","last_active":"2024-07-04T08:12:14.800668+0000","last_peered":"2024-07-04T08:12:14.800668+0000","last_clean":"2024-07-04T08:12:14.800668+0000","last_became_active":"2024-07-04T08:09:51.636590+0000","last_became_peered":"2024-07-04T08:09:51.636590+0000","last_unstale":"2024-07-04T08:12:14.800668+0000","last_undegraded":"2024-07-04T08:12:14.800668+0000","last_fullsized":"2024-07-04T08:12:14.800668+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:33:32.474597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.547189+0000","last_change":"2024-07-04T08:09:38.362520+0000","last_active":"2024-07-04T08:12:51.547189+0000","last_peered":"2024-07-04T08:12:51.547189+0000","last_clean":"2024-07-04T08:12:51.547189+0000","last_became_active":"2024-07-04T08:09:38.362090+0000","last_became_peered":"2024-07-04T08:09:38.362090+0000","last_unstale":"2024-07-04T08:12:51.547189+0000","last_undegraded":"2024-07-04T08:12:51.547189+0000","last_fullsized":"2024-07-04T08:12:51.547189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:15:10.467666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":562,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:21.224105+0000","last_change":"2024-07-04T08:09:34.301985+0000","last_active":"2024-07-04T08:15:21.224105+0000","last_peered":"2024-07-04T08:15:21.224105+0000","last_clean":"2024-07-04T08:15:21.224105+0000","last_became_active":"2024-07-04T08:09:34.301585+0000","last_became_peered":"2024-07-04T08:09:34.301585+0000","last_unstale":"2024-07-04T08:15:21.224105+0000","last_undegraded":"2024-07-04T08:15:21.224105+0000","last_fullsized":"2024-07-04T08:15:21.224105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:29:36.116115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":512,"num_read_kb":512,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:14.326445+0000","last_change":"2024-07-04T08:09:51.616505+0000","last_active":"2024-07-04T08:12:14.326445+0000","last_peered":"2024-07-04T08:12:14.326445+0000","last_clean":"2024-07-04T08:12:14.326445+0000","last_became_active":"2024-07-04T08:09:51.615890+0000","last_became_peered":"2024-07-04T08:09:51.615890+0000","last_unstale":"2024-07-04T08:12:14.326445+0000","last_undegraded":"2024-07-04T08:12:14.326445+0000","last_fullsized":"2024-07-04T08:12:14.326445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:58:25.861899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:54.639390+0000","last_change":"2024-07-04T08:09:51.643233+0000","last_active":"2024-07-04T08:11:54.639390+0000","last_peered":"2024-07-04T08:11:54.639390+0000","last_clean":"2024-07-04T08:11:54.639390+0000","last_became_active":"2024-07-04T08:09:51.642735+0000","last_became_peered":"2024-07-04T08:09:51.642735+0000","last_unstale":"2024-07-04T08:11:54.639390+0000","last_undegraded":"2024-07-04T08:11:54.639390+0000","last_fullsized":"2024-07-04T08:11:54.639390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:21:47.502966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.564562+0000","last_change":"2024-07-04T08:09:38.385010+0000","last_active":"2024-07-04T08:12:51.564562+0000","last_peered":"2024-07-04T08:12:51.564562+0000","last_clean":"2024-07-04T08:12:51.564562+0000","last_became_active":"2024-07-04T08:09:38.384629+0000","last_became_peered":"2024-07-04T08:09:38.384629+0000","last_unstale":"2024-07-04T08:12:51.564562+0000","last_undegraded":"2024-07-04T08:12:51.564562+0000","last_fullsized":"2024-07-04T08:12:51.564562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T20:03:20.005869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:59.974768+0000","last_change":"2024-07-04T08:09:34.284742+0000","last_active":"2024-07-04T08:11:59.974768+0000","last_peered":"2024-07-04T08:11:59.974768+0000","last_clean":"2024-07-04T08:11:59.974768+0000","last_became_active":"2024-07-04T08:09:34.284365+0000","last_became_peered":"2024-07-04T08:09:34.284365+0000","last_unstale":"2024-07-04T08:11:59.974768+0000","last_undegraded":"2024-07-04T08:11:59.974768+0000","last_fullsized":"2024-07-04T08:11:59.974768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:23:02.751480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:11:56.239030+0000","last_change":"2024-07-04T08:09:51.622175+0000","last_active":"2024-07-04T08:11:56.239030+0000","last_peered":"2024-07-04T08:11:56.239030+0000","last_clean":"2024-07-04T08:11:56.239030+0000","last_became_active":"2024-07-04T08:09:51.621642+0000","last_became_peered":"2024-07-04T08:09:51.621642+0000","last_unstale":"2024-07-04T08:11:56.239030+0000","last_undegraded":"2024-07-04T08:11:56.239030+0000","last_fullsized":"2024-07-04T08:11:56.239030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:56:08.626362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565645+0000","last_change":"2024-07-04T08:09:38.390604+0000","last_active":"2024-07-04T08:12:51.565645+0000","last_peered":"2024-07-04T08:12:51.565645+0000","last_clean":"2024-07-04T08:12:51.565645+0000","last_became_active":"2024-07-04T08:09:38.389594+0000","last_became_peered":"2024-07-04T08:09:38.389594+0000","last_unstale":"2024-07-04T08:12:51.565645+0000","last_undegraded":"2024-07-04T08:12:51.565645+0000","last_fullsized":"2024-07-04T08:12:51.565645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:14:18.436818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:03.138191+0000","last_change":"2024-07-04T08:09:34.294615+0000","last_active":"2024-07-04T08:12:03.138191+0000","last_peered":"2024-07-04T08:12:03.138191+0000","last_clean":"2024-07-04T08:12:03.138191+0000","last_became_active":"2024-07-04T08:09:34.294019+0000","last_became_peered":"2024-07-04T08:09:34.294019+0000","last_unstale":"2024-07-04T08:12:03.138191+0000","last_undegraded":"2024-07-04T08:12:03.138191+0000","last_fullsized":"2024-07-04T08:12:03.138191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:17:12.375622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:06.905597+0000","last_change":"2024-07-04T08:09:34.291157+0000","last_active":"2024-07-04T08:12:06.905597+0000","last_peered":"2024-07-04T08:12:06.905597+0000","last_clean":"2024-07-04T08:12:06.905597+0000","last_became_active":"2024-07-04T08:09:34.290653+0000","last_became_peered":"2024-07-04T08:09:34.290653+0000","last_unstale":"2024-07-04T08:12:06.905597+0000","last_undegraded":"2024-07-04T08:12:06.905597+0000","last_fullsized":"2024-07-04T08:12:06.905597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:52:02.020242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.554669+0000","last_change":"2024-07-04T08:09:38.382532+0000","last_active":"2024-07-04T08:12:51.554669+0000","last_peered":"2024-07-04T08:12:51.554669+0000","last_clean":"2024-07-04T08:12:51.554669+0000","last_became_active":"2024-07-04T08:09:38.382176+0000","last_became_peered":"2024-07-04T08:09:38.382176+0000","last_unstale":"2024-07-04T08:12:51.554669+0000","last_undegraded":"2024-07-04T08:12:51.554669+0000","last_fullsized":"2024-07-04T08:12:51.554669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:34:53.413870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118570+0000","last_change":"2024-07-04T08:09:51.642772+0000","last_active":"2024-07-04T08:12:21.118570+0000","last_peered":"2024-07-04T08:12:21.118570+0000","last_clean":"2024-07-04T08:12:21.118570+0000","last_became_active":"2024-07-04T08:09:51.642054+0000","last_became_peered":"2024-07-04T08:09:51.642054+0000","last_unstale":"2024-07-04T08:12:21.118570+0000","last_undegraded":"2024-07-04T08:12:21.118570+0000","last_fullsized":"2024-07-04T08:12:21.118570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:00:56.484453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":312,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:15:23.273320+0000","last_change":"2024-07-04T08:09:34.288190+0000","last_active":"2024-07-04T08:15:23.273320+0000","last_peered":"2024-07-04T08:15:23.273320+0000","last_clean":"2024-07-04T08:15:23.273320+0000","last_became_active":"2024-07-04T08:09:34.287773+0000","last_became_peered":"2024-07-04T08:09:34.287773+0000","last_unstale":"2024-07-04T08:15:23.273320+0000","last_undegraded":"2024-07-04T08:15:23.273320+0000","last_fullsized":"2024-07-04T08:15:23.273320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:33.236020+0000","last_clean_scrub_stamp":"2024-07-04T08:09:33.236020+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:46:52.985552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":261,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.519199+0000","last_change":"2024-07-04T08:09:38.365253+0000","last_active":"2024-07-04T08:12:51.519199+0000","last_peered":"2024-07-04T08:12:51.519199+0000","last_clean":"2024-07-04T08:12:51.519199+0000","last_became_active":"2024-07-04T08:09:38.364708+0000","last_became_peered":"2024-07-04T08:09:38.364708+0000","last_unstale":"2024-07-04T08:12:51.519199+0000","last_undegraded":"2024-07-04T08:12:51.519199+0000","last_fullsized":"2024-07-04T08:12:51.519199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:19:28.351604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:21.118593+0000","last_change":"2024-07-04T08:09:51.612161+0000","last_active":"2024-07-04T08:12:21.118593+0000","last_peered":"2024-07-04T08:12:21.118593+0000","last_clean":"2024-07-04T08:12:21.118593+0000","last_became_active":"2024-07-04T08:09:51.611649+0000","last_became_peered":"2024-07-04T08:09:51.611649+0000","last_unstale":"2024-07-04T08:12:21.118593+0000","last_undegraded":"2024-07-04T08:12:21.118593+0000","last_fullsized":"2024-07-04T08:12:21.118593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:50.585122+0000","last_clean_scrub_stamp":"2024-07-04T08:09:50.585122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:45.614244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534235+0000","last_change":"2024-07-04T08:09:38.357276+0000","last_active":"2024-07-04T08:12:51.534235+0000","last_peered":"2024-07-04T08:12:51.534235+0000","last_clean":"2024-07-04T08:12:51.534235+0000","last_became_active":"2024-07-04T08:09:38.356632+0000","last_became_peered":"2024-07-04T08:09:38.356632+0000","last_unstale":"2024-07-04T08:12:51.534235+0000","last_undegraded":"2024-07-04T08:12:51.534235+0000","last_fullsized":"2024-07-04T08:12:51.534235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:10:14.160606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.539681+0000","last_change":"2024-07-04T08:09:38.358086+0000","last_active":"2024-07-04T08:12:51.539681+0000","last_peered":"2024-07-04T08:12:51.539681+0000","last_clean":"2024-07-04T08:12:51.539681+0000","last_became_active":"2024-07-04T08:09:38.357675+0000","last_became_peered":"2024-07-04T08:09:38.357675+0000","last_unstale":"2024-07-04T08:12:51.539681+0000","last_undegraded":"2024-07-04T08:12:51.539681+0000","last_fullsized":"2024-07-04T08:12:51.539681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:55:35.962016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.566399+0000","last_change":"2024-07-04T08:09:38.381031+0000","last_active":"2024-07-04T08:12:51.566399+0000","last_peered":"2024-07-04T08:12:51.566399+0000","last_clean":"2024-07-04T08:12:51.566399+0000","last_became_active":"2024-07-04T08:09:38.380662+0000","last_became_peered":"2024-07-04T08:09:38.380662+0000","last_unstale":"2024-07-04T08:12:51.566399+0000","last_undegraded":"2024-07-04T08:12:51.566399+0000","last_fullsized":"2024-07-04T08:12:51.566399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:48:42.432669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567851+0000","last_change":"2024-07-04T08:09:38.387576+0000","last_active":"2024-07-04T08:12:51.567851+0000","last_peered":"2024-07-04T08:12:51.567851+0000","last_clean":"2024-07-04T08:12:51.567851+0000","last_became_active":"2024-07-04T08:09:38.386700+0000","last_became_peered":"2024-07-04T08:09:38.386700+0000","last_unstale":"2024-07-04T08:12:51.567851+0000","last_undegraded":"2024-07-04T08:12:51.567851+0000","last_fullsized":"2024-07-04T08:12:51.567851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T08:52:42.879057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569222+0000","last_change":"2024-07-04T08:10:52.685603+0000","last_active":"2024-07-04T08:12:51.569222+0000","last_peered":"2024-07-04T08:12:51.569222+0000","last_clean":"2024-07-04T08:12:51.569222+0000","last_became_active":"2024-07-04T08:10:52.685057+0000","last_became_peered":"2024-07-04T08:10:52.685057+0000","last_unstale":"2024-07-04T08:12:51.569222+0000","last_undegraded":"2024-07-04T08:12:51.569222+0000","last_fullsized":"2024-07-04T08:12:51.569222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:02:19.636234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532408+0000","last_change":"2024-07-04T08:09:38.359821+0000","last_active":"2024-07-04T08:12:51.532408+0000","last_peered":"2024-07-04T08:12:51.532408+0000","last_clean":"2024-07-04T08:12:51.532408+0000","last_became_active":"2024-07-04T08:09:38.359221+0000","last_became_peered":"2024-07-04T08:09:38.359221+0000","last_unstale":"2024-07-04T08:12:51.532408+0000","last_undegraded":"2024-07-04T08:12:51.532408+0000","last_fullsized":"2024-07-04T08:12:51.532408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:46:34.452059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540538+0000","last_change":"2024-07-04T08:09:38.360511+0000","last_active":"2024-07-04T08:12:51.540538+0000","last_peered":"2024-07-04T08:12:51.540538+0000","last_clean":"2024-07-04T08:12:51.540538+0000","last_became_active":"2024-07-04T08:09:38.360089+0000","last_became_peered":"2024-07-04T08:09:38.360089+0000","last_unstale":"2024-07-04T08:12:51.540538+0000","last_undegraded":"2024-07-04T08:12:51.540538+0000","last_fullsized":"2024-07-04T08:12:51.540538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:00:01.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552348+0000","last_change":"2024-07-04T08:09:38.378363+0000","last_active":"2024-07-04T08:12:51.552348+0000","last_peered":"2024-07-04T08:12:51.552348+0000","last_clean":"2024-07-04T08:12:51.552348+0000","last_became_active":"2024-07-04T08:09:38.377656+0000","last_became_peered":"2024-07-04T08:09:38.377656+0000","last_unstale":"2024-07-04T08:12:51.552348+0000","last_undegraded":"2024-07-04T08:12:51.552348+0000","last_fullsized":"2024-07-04T08:12:51.552348+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T16:41:51.977062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.531503+0000","last_change":"2024-07-04T08:09:38.382490+0000","last_active":"2024-07-04T08:12:51.531503+0000","last_peered":"2024-07-04T08:12:51.531503+0000","last_clean":"2024-07-04T08:12:51.531503+0000","last_became_active":"2024-07-04T08:09:38.382121+0000","last_became_peered":"2024-07-04T08:09:38.382121+0000","last_unstale":"2024-07-04T08:12:51.531503+0000","last_undegraded":"2024-07-04T08:12:51.531503+0000","last_fullsized":"2024-07-04T08:12:51.531503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:19:43.229658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.555574+0000","last_change":"2024-07-04T08:09:38.372915+0000","last_active":"2024-07-04T08:12:51.555574+0000","last_peered":"2024-07-04T08:12:51.555574+0000","last_clean":"2024-07-04T08:12:51.555574+0000","last_became_active":"2024-07-04T08:09:38.372340+0000","last_became_peered":"2024-07-04T08:09:38.372340+0000","last_unstale":"2024-07-04T08:12:51.555574+0000","last_undegraded":"2024-07-04T08:12:51.555574+0000","last_fullsized":"2024-07-04T08:12:51.555574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:13:26.775626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560802+0000","last_change":"2024-07-04T08:09:38.378389+0000","last_active":"2024-07-04T08:12:51.560802+0000","last_peered":"2024-07-04T08:12:51.560802+0000","last_clean":"2024-07-04T08:12:51.560802+0000","last_became_active":"2024-07-04T08:09:38.377964+0000","last_became_peered":"2024-07-04T08:09:38.377964+0000","last_unstale":"2024-07-04T08:12:51.560802+0000","last_undegraded":"2024-07-04T08:12:51.560802+0000","last_fullsized":"2024-07-04T08:12:51.560802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:58:11.312614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.570948+0000","last_change":"2024-07-04T08:09:38.382402+0000","last_active":"2024-07-04T08:12:51.570948+0000","last_peered":"2024-07-04T08:12:51.570948+0000","last_clean":"2024-07-04T08:12:51.570948+0000","last_became_active":"2024-07-04T08:09:38.382015+0000","last_became_peered":"2024-07-04T08:09:38.382015+0000","last_unstale":"2024-07-04T08:12:51.570948+0000","last_undegraded":"2024-07-04T08:12:51.570948+0000","last_fullsized":"2024-07-04T08:12:51.570948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T12:45:57.014102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.532752+0000","last_change":"2024-07-04T08:09:38.360128+0000","last_active":"2024-07-04T08:12:51.532752+0000","last_peered":"2024-07-04T08:12:51.532752+0000","last_clean":"2024-07-04T08:12:51.532752+0000","last_became_active":"2024-07-04T08:09:38.359595+0000","last_became_peered":"2024-07-04T08:09:38.359595+0000","last_unstale":"2024-07-04T08:12:51.532752+0000","last_undegraded":"2024-07-04T08:12:51.532752+0000","last_fullsized":"2024-07-04T08:12:51.532752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:29:27.671548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552705+0000","last_change":"2024-07-04T08:09:38.384444+0000","last_active":"2024-07-04T08:12:51.552705+0000","last_peered":"2024-07-04T08:12:51.552705+0000","last_clean":"2024-07-04T08:12:51.552705+0000","last_became_active":"2024-07-04T08:09:38.384184+0000","last_became_peered":"2024-07-04T08:09:38.384184+0000","last_unstale":"2024-07-04T08:12:51.552705+0000","last_undegraded":"2024-07-04T08:12:51.552705+0000","last_fullsized":"2024-07-04T08:12:51.552705+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T19:01:25.205215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541640+0000","last_change":"2024-07-04T08:09:38.363572+0000","last_active":"2024-07-04T08:12:51.541640+0000","last_peered":"2024-07-04T08:12:51.541640+0000","last_clean":"2024-07-04T08:12:51.541640+0000","last_became_active":"2024-07-04T08:09:38.363220+0000","last_became_peered":"2024-07-04T08:09:38.363220+0000","last_unstale":"2024-07-04T08:12:51.541640+0000","last_undegraded":"2024-07-04T08:12:51.541640+0000","last_fullsized":"2024-07-04T08:12:51.541640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:23:55.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569708+0000","last_change":"2024-07-04T08:09:38.374451+0000","last_active":"2024-07-04T08:12:51.569708+0000","last_peered":"2024-07-04T08:12:51.569708+0000","last_clean":"2024-07-04T08:12:51.569708+0000","last_became_active":"2024-07-04T08:09:38.374087+0000","last_became_peered":"2024-07-04T08:09:38.374087+0000","last_unstale":"2024-07-04T08:12:51.569708+0000","last_undegraded":"2024-07-04T08:12:51.569708+0000","last_fullsized":"2024-07-04T08:12:51.569708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:30:15.916623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.571140+0000","last_change":"2024-07-04T08:09:38.357698+0000","last_active":"2024-07-04T08:12:51.571140+0000","last_peered":"2024-07-04T08:12:51.571140+0000","last_clean":"2024-07-04T08:12:51.571140+0000","last_became_active":"2024-07-04T08:09:38.357287+0000","last_became_peered":"2024-07-04T08:09:38.357287+0000","last_unstale":"2024-07-04T08:12:51.571140+0000","last_undegraded":"2024-07-04T08:12:51.571140+0000","last_fullsized":"2024-07-04T08:12:51.571140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:16:04.700295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.552227+0000","last_change":"2024-07-04T08:09:38.366175+0000","last_active":"2024-07-04T08:12:51.552227+0000","last_peered":"2024-07-04T08:12:51.552227+0000","last_clean":"2024-07-04T08:12:51.552227+0000","last_became_active":"2024-07-04T08:09:38.365710+0000","last_became_peered":"2024-07-04T08:09:38.365710+0000","last_unstale":"2024-07-04T08:12:51.552227+0000","last_undegraded":"2024-07-04T08:12:51.552227+0000","last_fullsized":"2024-07-04T08:12:51.552227+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T17:49:26.526787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.567284+0000","last_change":"2024-07-04T08:09:38.366459+0000","last_active":"2024-07-04T08:12:51.567284+0000","last_peered":"2024-07-04T08:12:51.567284+0000","last_clean":"2024-07-04T08:12:51.567284+0000","last_became_active":"2024-07-04T08:09:38.361692+0000","last_became_peered":"2024-07-04T08:09:38.361692+0000","last_unstale":"2024-07-04T08:12:51.567284+0000","last_undegraded":"2024-07-04T08:12:51.567284+0000","last_fullsized":"2024-07-04T08:12:51.567284+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:12:46.930940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551582+0000","last_change":"2024-07-04T08:09:38.378802+0000","last_active":"2024-07-04T08:12:51.551582+0000","last_peered":"2024-07-04T08:12:51.551582+0000","last_clean":"2024-07-04T08:12:51.551582+0000","last_became_active":"2024-07-04T08:09:38.378378+0000","last_became_peered":"2024-07-04T08:09:38.378378+0000","last_unstale":"2024-07-04T08:12:51.551582+0000","last_undegraded":"2024-07-04T08:12:51.551582+0000","last_fullsized":"2024-07-04T08:12:51.551582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:10:16.782529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.534085+0000","last_change":"2024-07-04T08:09:38.358372+0000","last_active":"2024-07-04T08:12:51.534085+0000","last_peered":"2024-07-04T08:12:51.534085+0000","last_clean":"2024-07-04T08:12:51.534085+0000","last_became_active":"2024-07-04T08:09:38.357990+0000","last_became_peered":"2024-07-04T08:09:38.357990+0000","last_unstale":"2024-07-04T08:12:51.534085+0000","last_undegraded":"2024-07-04T08:12:51.534085+0000","last_fullsized":"2024-07-04T08:12:51.534085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:50:38.371092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.553666+0000","last_change":"2024-07-04T08:09:38.361761+0000","last_active":"2024-07-04T08:12:51.553666+0000","last_peered":"2024-07-04T08:12:51.553666+0000","last_clean":"2024-07-04T08:12:51.553666+0000","last_became_active":"2024-07-04T08:09:38.361367+0000","last_became_peered":"2024-07-04T08:09:38.361367+0000","last_unstale":"2024-07-04T08:12:51.553666+0000","last_undegraded":"2024-07-04T08:12:51.553666+0000","last_fullsized":"2024-07-04T08:12:51.553666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:44:33.960174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.540373+0000","last_change":"2024-07-04T08:09:38.360295+0000","last_active":"2024-07-04T08:12:51.540373+0000","last_peered":"2024-07-04T08:12:51.540373+0000","last_clean":"2024-07-04T08:12:51.540373+0000","last_became_active":"2024-07-04T08:09:38.359880+0000","last_became_peered":"2024-07-04T08:09:38.359880+0000","last_unstale":"2024-07-04T08:12:51.540373+0000","last_undegraded":"2024-07-04T08:12:51.540373+0000","last_fullsized":"2024-07-04T08:12:51.540373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T13:17:50.599760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541173+0000","last_change":"2024-07-04T08:09:38.350129+0000","last_active":"2024-07-04T08:12:51.541173+0000","last_peered":"2024-07-04T08:12:51.541173+0000","last_clean":"2024-07-04T08:12:51.541173+0000","last_became_active":"2024-07-04T08:09:38.349715+0000","last_became_peered":"2024-07-04T08:09:38.349715+0000","last_unstale":"2024-07-04T08:12:51.541173+0000","last_undegraded":"2024-07-04T08:12:51.541173+0000","last_fullsized":"2024-07-04T08:12:51.541173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:32:34.970531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.560317+0000","last_change":"2024-07-04T08:09:38.383685+0000","last_active":"2024-07-04T08:12:51.560317+0000","last_peered":"2024-07-04T08:12:51.560317+0000","last_clean":"2024-07-04T08:12:51.560317+0000","last_became_active":"2024-07-04T08:09:38.383173+0000","last_became_peered":"2024-07-04T08:09:38.383173+0000","last_unstale":"2024-07-04T08:12:51.560317+0000","last_undegraded":"2024-07-04T08:12:51.560317+0000","last_fullsized":"2024-07-04T08:12:51.560317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:45:22.372652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.551363+0000","last_change":"2024-07-04T08:09:38.363906+0000","last_active":"2024-07-04T08:12:51.551363+0000","last_peered":"2024-07-04T08:12:51.551363+0000","last_clean":"2024-07-04T08:12:51.551363+0000","last_became_active":"2024-07-04T08:09:38.363550+0000","last_became_peered":"2024-07-04T08:09:38.363550+0000","last_unstale":"2024-07-04T08:12:51.551363+0000","last_undegraded":"2024-07-04T08:12:51.551363+0000","last_fullsized":"2024-07-04T08:12:51.551363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T18:55:04.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.541562+0000","last_change":"2024-07-04T08:09:38.369996+0000","last_active":"2024-07-04T08:12:51.541562+0000","last_peered":"2024-07-04T08:12:51.541562+0000","last_clean":"2024-07-04T08:12:51.541562+0000","last_became_active":"2024-07-04T08:09:38.369480+0000","last_became_peered":"2024-07-04T08:09:38.369480+0000","last_unstale":"2024-07-04T08:12:51.541562+0000","last_undegraded":"2024-07-04T08:12:51.541562+0000","last_fullsized":"2024-07-04T08:12:51.541562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T11:03:47.225521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.536909+0000","last_change":"2024-07-04T08:09:38.389401+0000","last_active":"2024-07-04T08:12:51.536909+0000","last_peered":"2024-07-04T08:12:51.536909+0000","last_clean":"2024-07-04T08:12:51.536909+0000","last_became_active":"2024-07-04T08:09:38.389101+0000","last_became_peered":"2024-07-04T08:09:38.389101+0000","last_unstale":"2024-07-04T08:12:51.536909+0000","last_undegraded":"2024-07-04T08:12:51.536909+0000","last_fullsized":"2024-07-04T08:12:51.536909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T15:07:11.888637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.537941+0000","last_change":"2024-07-04T08:09:38.363703+0000","last_active":"2024-07-04T08:12:51.537941+0000","last_peered":"2024-07-04T08:12:51.537941+0000","last_clean":"2024-07-04T08:12:51.537941+0000","last_became_active":"2024-07-04T08:09:38.363269+0000","last_became_peered":"2024-07-04T08:09:38.363269+0000","last_unstale":"2024-07-04T08:12:51.537941+0000","last_undegraded":"2024-07-04T08:12:51.537941+0000","last_fullsized":"2024-07-04T08:12:51.537941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T14:43:11.723022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.565574+0000","last_change":"2024-07-04T08:09:38.361229+0000","last_active":"2024-07-04T08:12:51.565574+0000","last_peered":"2024-07-04T08:12:51.565574+0000","last_clean":"2024-07-04T08:12:51.565574+0000","last_became_active":"2024-07-04T08:09:38.360823+0000","last_became_peered":"2024-07-04T08:09:38.360823+0000","last_unstale":"2024-07-04T08:12:51.565574+0000","last_undegraded":"2024-07-04T08:12:51.565574+0000","last_fullsized":"2024-07-04T08:12:51.565574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:15:15.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.568095+0000","last_change":"2024-07-04T08:09:38.364104+0000","last_active":"2024-07-04T08:12:51.568095+0000","last_peered":"2024-07-04T08:12:51.568095+0000","last_clean":"2024-07-04T08:12:51.568095+0000","last_became_active":"2024-07-04T08:09:38.361350+0000","last_became_peered":"2024-07-04T08:09:38.361350+0000","last_unstale":"2024-07-04T08:12:51.568095+0000","last_undegraded":"2024-07-04T08:12:51.568095+0000","last_fullsized":"2024-07-04T08:12:51.568095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T09:00:58.320767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"92'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-07-04T08:12:51.569523+0000","last_change":"2024-07-04T08:09:38.375291+0000","last_active":"2024-07-04T08:12:51.569523+0000","last_peered":"2024-07-04T08:12:51.569523+0000","last_clean":"2024-07-04T08:12:51.569523+0000","last_became_active":"2024-07-04T08:09:38.374971+0000","last_became_peered":"2024-07-04T08:09:38.374971+0000","last_unstale":"2024-07-04T08:12:51.569523+0000","last_undegraded":"2024-07-04T08:12:51.569523+0000","last_fullsized":"2024-07-04T08:12:51.569523+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-04T08:09:37.310461+0000","last_clean_scrub_stamp":"2024-07-04T08:09:37.310461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-05T10:54:24.434789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":864542,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2949,"num_read_kb":1694,"num_write":487,"num_write_kb":2247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823296,"data_stored":2593626,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":369,"ondisk_log_size":369,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":290978,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612694,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":936780,"data_compressed":51617,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808921,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1232896,"data_stored":1594752,"data_compressed":87398,"data_compressed_allocated":114688,"data_compressed_original":1048576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":940806,"data_compressed":51617,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168679,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":543872,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364908,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1141042,"data_compressed":69307,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528433,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":539530,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724663,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36472,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708680,"statfs":{"total":95995035648,"available":95957688320,"internally_reserved":0,"allocated":1159168,"data_stored":1529336,"data_compressed":87398,"data_compressed_allocated":114688,"data_compressed_original":1048576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920891,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":543604,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117121,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1114112,"data_stored":1325698,"data_compressed":69708,"data_compressed_allocated":94208,"data_compressed_original":851968,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378760,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":609142,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542285,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":611794,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705810,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":540454,"data_compressed":32328,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":35361,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36979,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":73518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":787422,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":396596,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":466108,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":394408,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-04T08:15:27.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:15:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:26 smithi133 ceph-mon[37050]: pgmap v487: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-04T08:15:27.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:26 smithi050 ceph-mon[36577]: pgmap v487: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-07-04T08:15:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:28.207 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:27 smithi039 ceph-mon[32062]: from='client.25541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:28.208 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:27 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:15:28.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:27 smithi133 ceph-mon[37050]: from='client.25541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:28.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:27 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:15:28.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:27 smithi050 ceph-mon[36577]: from='client.25541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-04T08:15:28.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:27 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:15:29.084 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:28 smithi039 ceph-mon[32062]: pgmap v488: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-07-04T08:15:29.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:28 smithi133 ceph-mon[37050]: pgmap v488: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-07-04T08:15:29.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:28 smithi050 ceph-mon[36577]: pgmap v488: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-07-04T08:15:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.969+0000 7fb1c4c00640 1 -- 172.21.15.39:0/3854957684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010b3a0 msgr2=0x7fb1c0111bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.969+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/3854957684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010b3a0 0x7fb1c0111bd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009900 tx=0x7fb1b802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:15:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.970+0000 7fb1c4c00640 1 -- 172.21.15.39:0/3854957684 shutdown_connections 2024-07-04T08:15:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.970+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/3854957684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010b3a0 0x7fb1c0111bd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.970+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/3854957684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1c010aa50 0x7fb1c010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.970+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/3854957684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1c0105110 0x7fb1c0105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.970+0000 7fb1c4c00640 1 -- 172.21.15.39:0/3854957684 >> 172.21.15.39:0/3854957684 conn(0x7fb1c0100b60 msgr2=0x7fb1c0102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.970+0000 7fb1c4c00640 1 -- 172.21.15.39:0/3854957684 shutdown_connections 2024-07-04T08:15:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.970+0000 7fb1c4c00640 1 -- 172.21.15.39:0/3854957684 wait complete. 2024-07-04T08:15:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.971+0000 7fb1c4c00640 1 Processor -- start 2024-07-04T08:15:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.971+0000 7fb1c4c00640 1 -- start start 2024-07-04T08:15:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1c4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1c0105110 0x7fb1c019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1c4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010aa50 0x7fb1c019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1c4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1c010b3a0 0x7fb1c01a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1c4c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0115050 con 0x7fb1c010aa50 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1c4c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0114ed0 con 0x7fb1c010b3a0 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1c4c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01151d0 con 0x7fb1c0105110 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1b6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010aa50 0x7fb1c019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1bf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1c0105110 0x7fb1c019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1b6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010aa50 0x7fb1c019bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50964/0 (socket says 172.21.15.39:50964) 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1b6a00640 1 -- 172.21.15.39:0/2815320706 learned_addr learned my addr 172.21.15.39:0/2815320706 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.972+0000 7fb1bfe00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1c010b3a0 0x7fb1c01a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.973+0000 7fb1b6a00640 1 -- 172.21.15.39:0/2815320706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1c0105110 msgr2=0x7fb1c019b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.973+0000 7fb1b6a00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1c0105110 0x7fb1c019b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.973+0000 7fb1b6a00640 1 -- 172.21.15.39:0/2815320706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1c010b3a0 msgr2=0x7fb1c01a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.973+0000 7fb1b6a00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1c010b3a0 0x7fb1c01a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.973+0000 7fb1b6a00640 1 -- 172.21.15.39:0/2815320706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c01a1930 con 0x7fb1c010aa50 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.973+0000 7fb1b6a00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010aa50 0x7fb1c019bf60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800ebc0 tx=0x7fb1a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.973+0000 7fb1bcc00640 1 -- 172.21.15.39:0/2815320706 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8010630 con 0x7fb1c010aa50 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.973+0000 7fb1bcc00640 1 -- 172.21.15.39:0/2815320706 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1a8010c80 con 0x7fb1c010aa50 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.974+0000 7fb1bcc00640 1 -- 172.21.15.39:0/2815320706 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8017ba0 con 0x7fb1c010aa50 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.974+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c01a1c20 con 0x7fb1c010aa50 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.974+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c01a8380 con 0x7fb1c010aa50 2024-07-04T08:15:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.975+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c0105520 con 0x7fb1c010aa50 2024-07-04T08:15:29.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.978+0000 7fb1bcc00640 1 -- 172.21.15.39:0/2815320706 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7fb1a80107d0 con 0x7fb1c010aa50 2024-07-04T08:15:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.979+0000 7fb1bcc00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1a40b28d0 0x7fb1a40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:29.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.979+0000 7fb1bcc00640 1 -- 172.21.15.39:0/2815320706 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb1a80e3310 con 0x7fb1c010aa50 2024-07-04T08:15:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.979+0000 7fb1bf400640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1a40b28d0 0x7fb1a40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:29.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.980+0000 7fb1bf400640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1a40b28d0 0x7fb1a40b4de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009bd0 tx=0x7fb1b0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:29.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:29.983+0000 7fb1bcc00640 1 -- 172.21.15.39:0/2815320706 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb1a8055110 con 0x7fb1c010aa50 2024-07-04T08:15:30.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.241+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1c019caf0 con 0x7fb1c010aa50 2024-07-04T08:15:30.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.242+0000 7fb1bcc00640 1 -- 172.21.15.39:0/2815320706 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fb1c019caf0 con 0x7fb1c010aa50 2024-07-04T08:15:30.244 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:30.244 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:15:30.245 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:15:30.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.245+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1a40b28d0 msgr2=0x7fb1a40b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.245+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1a40b28d0 0x7fb1a40b4de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009bd0 tx=0x7fb1b0005f90 comp rx=0 tx=0).stop 2024-07-04T08:15:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.245+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010aa50 msgr2=0x7fb1c019bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.245+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010aa50 0x7fb1c019bf60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800ebc0 tx=0x7fb1a800c6a0 comp rx=0 tx=0).stop 2024-07-04T08:15:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.246+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 shutdown_connections 2024-07-04T08:15:30.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.246+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1a40b28d0 0x7fb1a40b4de0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.246+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c010aa50 0x7fb1c019bf60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.246+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1c010b3a0 0x7fb1c01a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.246+0000 7fb1c4c00640 1 --2- 172.21.15.39:0/2815320706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1c0105110 0x7fb1c019b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.246+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 >> 172.21.15.39:0/2815320706 conn(0x7fb1c0100b60 msgr2=0x7fb1c0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.246+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 shutdown_connections 2024-07-04T08:15:30.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:30.246+0000 7fb1c4c00640 1 -- 172.21.15.39:0/2815320706 wait complete. 2024-07-04T08:15:31.012 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:15:31.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:30 smithi133 ceph-mon[37050]: pgmap v489: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:31.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:30 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2815320706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:31.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:30 smithi039 ceph-mon[32062]: pgmap v489: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:31.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:30 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2815320706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:31.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:30 smithi050 ceph-mon[36577]: pgmap v489: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:31.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:30 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2815320706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:31.488 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:33.295 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:32 smithi039 ceph-mon[32062]: pgmap v490: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:32 smithi133 ceph-mon[37050]: pgmap v490: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:33.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:32 smithi050 ceph-mon[36577]: pgmap v490: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:33.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.894+0000 7ff5c7600640 1 -- 172.21.15.39:0/1158250379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5c010b590 msgr2=0x7ff5c0111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:33.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.894+0000 7ff5c7600640 1 --2- 172.21.15.39:0/1158250379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5c010b590 0x7ff5c0111dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009be0 tx=0x7ff5b802a6d0 comp rx=0 tx=0).stop 2024-07-04T08:15:33.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.894+0000 7ff5c7600640 1 -- 172.21.15.39:0/1158250379 shutdown_connections 2024-07-04T08:15:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.894+0000 7ff5c7600640 1 --2- 172.21.15.39:0/1158250379 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5c010b590 0x7ff5c0111dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.894+0000 7ff5c7600640 1 --2- 172.21.15.39:0/1158250379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c010ac40 0x7ff5c010b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.894+0000 7ff5c7600640 1 --2- 172.21.15.39:0/1158250379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c0105300 0x7ff5c0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.894+0000 7ff5c7600640 1 -- 172.21.15.39:0/1158250379 >> 172.21.15.39:0/1158250379 conn(0x7ff5c0100d10 msgr2=0x7ff5c0103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.894+0000 7ff5c7600640 1 -- 172.21.15.39:0/1158250379 shutdown_connections 2024-07-04T08:15:33.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.894+0000 7ff5c7600640 1 -- 172.21.15.39:0/1158250379 wait complete. 2024-07-04T08:15:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.895+0000 7ff5c7600640 1 Processor -- start 2024-07-04T08:15:33.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.895+0000 7ff5c7600640 1 -- start start 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5c7600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c0105300 0x7ff5c019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5c7600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c010ac40 0x7ff5c019c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5c7600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5c010b590 0x7ff5c01a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5c7600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0115240 con 0x7ff5c010ac40 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5c7600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01150c0 con 0x7ff5c010b590 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5c7600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01153c0 con 0x7ff5c0105300 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c010ac40 0x7ff5c019c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5c4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c0105300 0x7ff5c019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5c4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c0105300 0x7ff5c019bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:45618/0 (socket says 172.21.15.39:45618) 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c010ac40 0x7ff5c019c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45124/0 (socket says 172.21.15.39:45124) 2024-07-04T08:15:33.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5bfe00640 1 -- 172.21.15.39:0/13671855 learned_addr learned my addr 172.21.15.39:0/13671855 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:33.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.896+0000 7ff5c5600640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5c010b590 0x7ff5c01a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5c4c00640 1 -- 172.21.15.39:0/13671855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5c010b590 msgr2=0x7ff5c01a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5c4c00640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5c010b590 0x7ff5c01a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5c4c00640 1 -- 172.21.15.39:0/13671855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c010ac40 msgr2=0x7ff5c019c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5c4c00640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c010ac40 0x7ff5c019c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5c4c00640 1 -- 172.21.15.39:0/13671855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c01a1b40 con 0x7ff5c0105300 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5c4c00640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c0105300 0x7ff5c019bbd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005fd0 tx=0x7ff5b400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5bd600640 1 -- 172.21.15.39:0/13671855 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400cd00 con 0x7ff5c0105300 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5bd600640 1 -- 172.21.15.39:0/13671855 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5b40035f0 con 0x7ff5c0105300 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5bd600640 1 -- 172.21.15.39:0/13671855 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4010690 con 0x7ff5c0105300 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.897+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c01a1e30 con 0x7ff5c0105300 2024-07-04T08:15:33.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.898+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c01a8590 con 0x7ff5c0105300 2024-07-04T08:15:33.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.899+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c0105710 con 0x7ff5c0105300 2024-07-04T08:15:33.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.903+0000 7ff5bd600640 1 -- 172.21.15.39:0/13671855 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7ff5b4003aa0 con 0x7ff5c0105300 2024-07-04T08:15:33.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.904+0000 7ff5bd600640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5a80b28d0 0x7ff5a80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:33.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.904+0000 7ff5bd600640 1 -- 172.21.15.39:0/13671855 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff5b40e2bd0 con 0x7ff5c0105300 2024-07-04T08:15:33.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.904+0000 7ff5bfe00640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5a80b28d0 0x7ff5a80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:33.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.905+0000 7ff5bd600640 1 -- 172.21.15.39:0/13671855 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7ff5b4009510 con 0x7ff5c0105300 2024-07-04T08:15:33.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:33.905+0000 7ff5bfe00640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5a80b28d0 0x7ff5a80b4de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0103620 tx=0x7ff5ac005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:34.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.162+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5c019cd00 con 0x7ff5c0105300 2024-07-04T08:15:34.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.164+0000 7ff5bd600640 1 -- 172.21.15.39:0/13671855 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7ff5b40b0020 con 0x7ff5c0105300 2024-07-04T08:15:34.165 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:34.166 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:15:34.166 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:15:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.166+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5a80b28d0 msgr2=0x7ff5a80b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.166+0000 7ff5c7600640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5a80b28d0 0x7ff5a80b4de0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0103620 tx=0x7ff5ac005f90 comp rx=0 tx=0).stop 2024-07-04T08:15:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.166+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c0105300 msgr2=0x7ff5c019bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.166+0000 7ff5c7600640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c0105300 0x7ff5c019bbd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4005fd0 tx=0x7ff5b400ee50 comp rx=0 tx=0).stop 2024-07-04T08:15:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.167+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 shutdown_connections 2024-07-04T08:15:34.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.167+0000 7ff5c7600640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5a80b28d0 0x7ff5a80b4de0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:34.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.167+0000 7ff5c7600640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff5c010ac40 0x7ff5c019c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:34.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.167+0000 7ff5c7600640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff5c010b590 0x7ff5c01a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:34.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.167+0000 7ff5c7600640 1 --2- 172.21.15.39:0/13671855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c0105300 0x7ff5c019bbd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:34.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.167+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 >> 172.21.15.39:0/13671855 conn(0x7ff5c0100d10 msgr2=0x7ff5c01091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:34.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.167+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 shutdown_connections 2024-07-04T08:15:34.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:34.167+0000 7ff5c7600640 1 -- 172.21.15.39:0/13671855 wait complete. 2024-07-04T08:15:35.105 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:15:35.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:34 smithi039 ceph-mon[32062]: pgmap v491: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:15:35.187 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:34 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/13671855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:35.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:34 smithi133 ceph-mon[37050]: pgmap v491: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:15:35.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:34 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/13671855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:35.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:34 smithi050 ceph-mon[36577]: pgmap v491: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:15:35.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:34 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/13671855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:35.596 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:37.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:36 smithi133 ceph-mon[37050]: pgmap v492: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:37.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:36 smithi039 ceph-mon[32062]: pgmap v492: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:37.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:36 smithi050 ceph-mon[36577]: pgmap v492: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:37.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.902+0000 7fe4fd400640 1 -- 172.21.15.39:0/2537783920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4f8101fb0 msgr2=0x7fe4f81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:37.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.902+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2537783920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4f8101fb0 0x7fe4f81023c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009900 tx=0x7fe4e802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:15:37.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.902+0000 7fe4fd400640 1 -- 172.21.15.39:0/2537783920 shutdown_connections 2024-07-04T08:15:37.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.902+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2537783920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f8108240 0x7fe4f810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:37.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.902+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2537783920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f81078f0 0x7fe4f8107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:37.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.902+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2537783920 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4f8101fb0 0x7fe4f81023c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:37.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.902+0000 7fe4fd400640 1 -- 172.21.15.39:0/2537783920 >> 172.21.15.39:0/2537783920 conn(0x7fe4f8078c80 msgr2=0x7fe4f8079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:37.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.903+0000 7fe4fd400640 1 -- 172.21.15.39:0/2537783920 shutdown_connections 2024-07-04T08:15:37.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.903+0000 7fe4fd400640 1 -- 172.21.15.39:0/2537783920 wait complete. 2024-07-04T08:15:37.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.904+0000 7fe4fd400640 1 Processor -- start 2024-07-04T08:15:37.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.904+0000 7fe4fd400640 1 -- start start 2024-07-04T08:15:37.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4fd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4f8101fb0 0x7fe4f8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:37.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4fd400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f81078f0 0x7fe4f8199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:37.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4fd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f8108240 0x7fe4f819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:37.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4fd400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8111eb0 con 0x7fe4f8101fb0 2024-07-04T08:15:37.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4fd400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8111d30 con 0x7fe4f8108240 2024-07-04T08:15:37.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4fd400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f8112030 con 0x7fe4f81078f0 2024-07-04T08:15:37.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4f6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4f8101fb0 0x7fe4f8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:37.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4f6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4f8101fb0 0x7fe4f8199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45134/0 (socket says 172.21.15.39:45134) 2024-07-04T08:15:37.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4f6800640 1 -- 172.21.15.39:0/2220934859 learned_addr learned my addr 172.21.15.39:0/2220934859 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:37.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4f5e00640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f81078f0 0x7fe4f8199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:37.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.905+0000 7fe4f7200640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f8108240 0x7fe4f819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:37.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.906+0000 7fe4f5e00640 1 -- 172.21.15.39:0/2220934859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f8108240 msgr2=0x7fe4f819f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:37.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.906+0000 7fe4f5e00640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f8108240 0x7fe4f819f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:37.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.906+0000 7fe4f5e00640 1 -- 172.21.15.39:0/2220934859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4f8101fb0 msgr2=0x7fe4f8199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:37.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.906+0000 7fe4f5e00640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4f8101fb0 0x7fe4f8199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:37.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.906+0000 7fe4f5e00640 1 -- 172.21.15.39:0/2220934859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f819f900 con 0x7fe4f81078f0 2024-07-04T08:15:37.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.906+0000 7fe4f5e00640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f81078f0 0x7fe4f8199f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400e9c0 tx=0x7fe4e400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:37.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.906+0000 7fe4ef400640 1 -- 172.21.15.39:0/2220934859 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400ccc0 con 0x7fe4f81078f0 2024-07-04T08:15:37.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.906+0000 7fe4ef400640 1 -- 172.21.15.39:0/2220934859 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4e4005020 con 0x7fe4f81078f0 2024-07-04T08:15:37.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.907+0000 7fe4ef400640 1 -- 172.21.15.39:0/2220934859 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4004a00 con 0x7fe4f81078f0 2024-07-04T08:15:37.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.907+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f819fbf0 con 0x7fe4f81078f0 2024-07-04T08:15:37.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.907+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f8103680 con 0x7fe4f81078f0 2024-07-04T08:15:37.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.908+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f81023c0 con 0x7fe4f81078f0 2024-07-04T08:15:37.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.911+0000 7fe4ef400640 1 -- 172.21.15.39:0/2220934859 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7fe4e40051d0 con 0x7fe4f81078f0 2024-07-04T08:15:37.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.913+0000 7fe4ef400640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4dc0b2950 0x7fe4dc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:37.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.913+0000 7fe4ef400640 1 -- 172.21.15.39:0/2220934859 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fe4e40e2880 con 0x7fe4f81078f0 2024-07-04T08:15:37.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.913+0000 7fe4f6800640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4dc0b2950 0x7fe4dc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.914+0000 7fe4ef400640 1 -- 172.21.15.39:0/2220934859 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe4e40aa020 con 0x7fe4f81078f0 2024-07-04T08:15:37.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:37.914+0000 7fe4f6800640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4dc0b2950 0x7fe4dc0b4e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8031440 tx=0x7fe4e80058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.130+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4f8103940 con 0x7fe4f81078f0 2024-07-04T08:15:38.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.132+0000 7fe4ef400640 1 -- 172.21.15.39:0/2220934859 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24957 (secure 0 0 0) 0x7fe4e40a9920 con 0x7fe4f81078f0 2024-07-04T08:15:38.134 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:38.135 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":92,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:11:53.672125+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:15:38.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4dc0b2950 msgr2=0x7fe4dc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:38.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4dc0b2950 0x7fe4dc0b4e60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8031440 tx=0x7fe4e80058d0 comp rx=0 tx=0).stop 2024-07-04T08:15:38.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f81078f0 msgr2=0x7fe4f8199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f81078f0 0x7fe4f8199f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400e9c0 tx=0x7fe4e400ee80 comp rx=0 tx=0).stop 2024-07-04T08:15:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 shutdown_connections 2024-07-04T08:15:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe4dc0b2950 0x7fe4dc0b4e60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4f8108240 0x7fe4f819f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4f81078f0 0x7fe4f8199f30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 --2- 172.21.15.39:0/2220934859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4f8101fb0 0x7fe4f8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 >> 172.21.15.39:0/2220934859 conn(0x7fe4f8078c80 msgr2=0x7fe4f8105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 shutdown_connections 2024-07-04T08:15:38.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:38.136+0000 7fe4fd400640 1 -- 172.21.15.39:0/2220934859 wait complete. 2024-07-04T08:15:39.004 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-07-04T08:15:39.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:38 smithi133 ceph-mon[37050]: pgmap v493: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:15:39.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:38 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2220934859' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:15:39.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:38 smithi039 ceph-mon[32062]: pgmap v493: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:15:39.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:38 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2220934859' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:15:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:38 smithi050 ceph-mon[36577]: pgmap v493: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:15:39.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:38 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2220934859' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:15:39.505 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:41.301 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:40 smithi039 ceph-mon[32062]: pgmap v494: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:41.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:40 smithi133 ceph-mon[37050]: pgmap v494: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:41.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:40 smithi050 ceph-mon[36577]: pgmap v494: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:42.735 INFO:teuthology.orchestra.run.smithi039.stdout:{"run_dir":"/var/run/ceph"} 2024-07-04T08:15:42.735 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-07-04T08:15:42.735 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3860 2024-07-04T08:15:42.736 DEBUG:tasks.quiescer:Yielding 2024-07-04T08:15:42.736 INFO:teuthology.run_tasks:Running task workunit... 2024-07-04T08:15:42.749 INFO:tasks.workunit:Pulling workunits from ref 2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T08:15:42.749 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-07-04T08:15:42.750 INFO:tasks.workunit:timeout=3h 2024-07-04T08:15:42.750 INFO:tasks.workunit:cleanup=True 2024-07-04T08:15:42.750 DEBUG:teuthology.orchestra.run.smithi039:> stat -- /home/ubuntu/cephtest/mnt.0 2024-07-04T08:15:42.750 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:15:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-07-04T08:15:42.804 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-07-04T08:15:42.805 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-07-04T08:15:42.805 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-04T08:15:42.805 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-07-04T08:15:42.805 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-07-04 08:11:25.448265893 +0000 2024-07-04T08:15:42.805 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-07-04 08:11:25.448265893 +0000 2024-07-04T08:15:42.805 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-07-04 08:14:52.837401464 +0000 2024-07-04T08:15:42.805 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-07-04 08:11:25.448265893 +0000 2024-07-04T08:15:42.805 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-07-04T08:15:42.805 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-07-04T08:15:42.939 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 2e9f68b5506c607b2f7d69d380cb7887cf00dc84 2024-07-04T08:15:42.993 INFO:tasks.workunit.client.0.smithi039.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-07-04T08:15:43.228 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:43.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:42 smithi133 ceph-mon[37050]: pgmap v495: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:43.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:42 smithi039 ceph-mon[32062]: pgmap v495: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:43.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:42 smithi050 ceph-mon[36577]: pgmap v495: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:15:45.194 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:44 smithi039 ceph-mon[32062]: pgmap v496: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:15:45.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:44 smithi133 ceph-mon[37050]: pgmap v496: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:15:45.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:44 smithi050 ceph-mon[36577]: pgmap v496: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-04T08:15:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.528+0000 7fb777200640 1 -- 172.21.15.39:0/673597448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb770103f20 msgr2=0x7fb770104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:45.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.528+0000 7fb777200640 1 --2- 172.21.15.39:0/673597448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb770103f20 0x7fb770104330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb760009900 tx=0x7fb76002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.528+0000 7fb777200640 1 -- 172.21.15.39:0/673597448 shutdown_connections 2024-07-04T08:15:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.528+0000 7fb777200640 1 --2- 172.21.15.39:0/673597448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77010d380 0x7fb77010f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.528+0000 7fb777200640 1 --2- 172.21.15.39:0/673597448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb770104870 0x7fb77010cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.528+0000 7fb777200640 1 --2- 172.21.15.39:0/673597448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb770103f20 0x7fb770104330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.528+0000 7fb777200640 1 -- 172.21.15.39:0/673597448 >> 172.21.15.39:0/673597448 conn(0x7fb770078c90 msgr2=0x7fb7700790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.528+0000 7fb777200640 1 -- 172.21.15.39:0/673597448 shutdown_connections 2024-07-04T08:15:45.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.528+0000 7fb777200640 1 -- 172.21.15.39:0/673597448 wait complete. 2024-07-04T08:15:45.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.530+0000 7fb777200640 1 Processor -- start 2024-07-04T08:15:45.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.531+0000 7fb777200640 1 -- start start 2024-07-04T08:15:45.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb777200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb770103f20 0x7fb77019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:45.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb777200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb770104870 0x7fb77019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:45.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb777200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77010d380 0x7fb7701a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:45.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb777200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb770112ff0 con 0x7fb77010d380 2024-07-04T08:15:45.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb777200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb770112e70 con 0x7fb770103f20 2024-07-04T08:15:45.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb777200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb770113170 con 0x7fb770104870 2024-07-04T08:15:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb770103f20 0x7fb77019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb770103f20 0x7fb77019bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38266/0 (socket says 172.21.15.39:38266) 2024-07-04T08:15:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb76fe00640 1 -- 172.21.15.39:0/4182794629 learned_addr learned my addr 172.21.15.39:0/4182794629 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb767e00640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb770104870 0x7fb77019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:45.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.532+0000 7fb775200640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77010d380 0x7fb7701a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.533+0000 7fb76fe00640 1 -- 172.21.15.39:0/4182794629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb770104870 msgr2=0x7fb77019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.533+0000 7fb76fe00640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb770104870 0x7fb77019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.533+0000 7fb76fe00640 1 -- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77010d380 msgr2=0x7fb7701a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.533+0000 7fb76fe00640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77010d380 0x7fb7701a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.533+0000 7fb76fe00640 1 -- 172.21.15.39:0/4182794629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7701a1c90 con 0x7fb770103f20 2024-07-04T08:15:45.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.533+0000 7fb76fe00640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb770103f20 0x7fb77019bd20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb760032fd0 tx=0x7fb760031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:45.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.534+0000 7fb76d600640 1 -- 172.21.15.39:0/4182794629 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76003f030 con 0x7fb770103f20 2024-07-04T08:15:45.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.534+0000 7fb76d600640 1 -- 172.21.15.39:0/4182794629 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7600043d0 con 0x7fb770103f20 2024-07-04T08:15:45.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.534+0000 7fb777200640 1 -- 172.21.15.39:0/4182794629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7701a1f20 con 0x7fb770103f20 2024-07-04T08:15:45.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.534+0000 7fb76d600640 1 -- 172.21.15.39:0/4182794629 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760035070 con 0x7fb770103f20 2024-07-04T08:15:45.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.534+0000 7fb777200640 1 -- 172.21.15.39:0/4182794629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7701a86e0 con 0x7fb770103f20 2024-07-04T08:15:45.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.535+0000 7fb766000640 1 -- 172.21.15.39:0/4182794629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77006b790 con 0x7fb770103f20 2024-07-04T08:15:45.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.538+0000 7fb76d600640 1 -- 172.21.15.39:0/4182794629 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7fb760042030 con 0x7fb770103f20 2024-07-04T08:15:45.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.540+0000 7fb76d600640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7540b28d0 0x7fb7540b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:45.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.540+0000 7fb76d600640 1 -- 172.21.15.39:0/4182794629 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb760102ea0 con 0x7fb770103f20 2024-07-04T08:15:45.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.540+0000 7fb767e00640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7540b28d0 0x7fb7540b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:45.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.541+0000 7fb767e00640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7540b28d0 0x7fb7540b4de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb77010f010 tx=0x7fb758005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:45.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.541+0000 7fb76d600640 1 -- 172.21.15.39:0/4182794629 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb7600ca5b0 con 0x7fb770103f20 2024-07-04T08:15:45.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.816+0000 7fb766000640 1 -- 172.21.15.39:0/4182794629 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb77019ce50 con 0x7fb770103f20 2024-07-04T08:15:45.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.817+0000 7fb76d600640 1 -- 172.21.15.39:0/4182794629 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fb7600c9eb0 con 0x7fb770103f20 2024-07-04T08:15:45.819 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:15:45.819 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:15:45.819 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:15:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 -- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7540b28d0 msgr2=0x7fb7540b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7540b28d0 0x7fb7540b4de0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb77010f010 tx=0x7fb758005f90 comp rx=0 tx=0).stop 2024-07-04T08:15:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 -- 172.21.15.39:0/4182794629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb770103f20 msgr2=0x7fb77019bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:45.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb770103f20 0x7fb77019bd20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb760032fd0 tx=0x7fb760031a20 comp rx=0 tx=0).stop 2024-07-04T08:15:45.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 -- 172.21.15.39:0/4182794629 shutdown_connections 2024-07-04T08:15:45.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7540b28d0 0x7fb7540b4de0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb77010d380 0x7fb7701a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb770104870 0x7fb77019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 --2- 172.21.15.39:0/4182794629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb770103f20 0x7fb77019bd20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:45.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 -- 172.21.15.39:0/4182794629 >> 172.21.15.39:0/4182794629 conn(0x7fb770078c90 msgr2=0x7fb7700ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:45.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 -- 172.21.15.39:0/4182794629 shutdown_connections 2024-07-04T08:15:45.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:45.820+0000 7fb766000640 1 -- 172.21.15.39:0/4182794629 wait complete. 2024-07-04T08:15:46.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:45 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/4182794629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:46.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:45 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/4182794629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:46.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:45 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/4182794629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:15:46.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-04T08:09:41.252924+0000', 'modified': '2024-07-04T08:14:54.008520+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24707, 'mds_1': 24701, 'mds_2': 24679, 'mds_3': 24674, 'mds_4': 24673}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24619': {'gid': 24619, 'name': 'l', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6833/2411107381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6832', 'nonce': 2411107381}, {'type': 'v1', 'addr': '172.21.15.133:6833', 'nonce': 2411107381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24626': {'gid': 24626, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6833/264080488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6832', 'nonce': 264080488}, {'type': 'v1', 'addr': '172.21.15.50:6833', 'nonce': 264080488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24644': {'gid': 24644, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.39:6837/1240380045', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6836', 'nonce': 1240380045}, {'type': 'v1', 'addr': '172.21.15.39:6837', 'nonce': 1240380045}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24656': {'gid': 24656, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6835/2710764279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 2710764279}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 2710764279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24661': {'gid': 24661, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/2187182943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2187182943}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 2187182943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24673': {'gid': 24673, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.133:6837/431595690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 431595690}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 431595690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24674': {'gid': 24674, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 31, 'addr': '172.21.15.50:6837/4268659351', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6836', 'nonce': 4268659351}, {'type': 'v1', 'addr': '172.21.15.50:6837', 'nonce': 4268659351}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24679': {'gid': 24679, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.133:6839/1264801316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 1264801316}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 1264801316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24701': {'gid': 24701, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.39:6841/2753029904', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6840', 'nonce': 2753029904}, {'type': 'v1', 'addr': '172.21.15.39:6841', 'nonce': 2753029904}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24707': {'gid': 24707, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.50:6839/2852157181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6838', 'nonce': 2852157181}, {'type': 'v1', 'addr': '172.21.15.50:6839', 'nonce': 2852157181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24707, 'qdb_cluster': [24707, 24701, 24679, 24674, 24673]} 2024-07-04T08:15:46.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-04T08:15:46.713 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.c status 2024-07-04T08:15:47.217 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:47.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:46 smithi133 ceph-mon[37050]: pgmap v497: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-04T08:15:47.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:46 smithi039 ceph-mon[32062]: pgmap v497: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-04T08:15:47.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:46 smithi050 ceph-mon[36577]: pgmap v497: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-04T08:15:49.251 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:48 smithi039 ceph-mon[32062]: pgmap v498: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-07-04T08:15:49.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:48 smithi133 ceph-mon[37050]: pgmap v498: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-07-04T08:15:49.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:48 smithi050 ceph-mon[36577]: pgmap v498: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-07-04T08:15:49.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.456+0000 7fc631e00640 1 -- 172.21.15.39:0/1083653923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc62c101b20 msgr2=0x7fc62c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:49.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.456+0000 7fc631e00640 1 --2- 172.21.15.39:0/1083653923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc62c101b20 0x7fc62c101f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009860 tx=0x7fc61c02a3d0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.456+0000 7fc631e00640 1 -- 172.21.15.39:0/1083653923 shutdown_connections 2024-07-04T08:15:49.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.456+0000 7fc631e00640 1 --2- 172.21.15.39:0/1083653923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc62c102470 0x7fc62c10f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.456+0000 7fc631e00640 1 --2- 172.21.15.39:0/1083653923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c10f6d0 0x7fc62c111af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.456+0000 7fc631e00640 1 --2- 172.21.15.39:0/1083653923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc62c101b20 0x7fc62c101f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.456+0000 7fc631e00640 1 -- 172.21.15.39:0/1083653923 >> 172.21.15.39:0/1083653923 conn(0x7fc62c078c80 msgr2=0x7fc62c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:49.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.456+0000 7fc631e00640 1 -- 172.21.15.39:0/1083653923 shutdown_connections 2024-07-04T08:15:49.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.456+0000 7fc631e00640 1 -- 172.21.15.39:0/1083653923 wait complete. 2024-07-04T08:15:49.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.458+0000 7fc631e00640 1 Processor -- start 2024-07-04T08:15:49.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.458+0000 7fc631e00640 1 -- start start 2024-07-04T08:15:49.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.458+0000 7fc631e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c101b20 0x7fc62c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.458+0000 7fc631e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc62c102470 0x7fc62c19c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.458+0000 7fc631e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc62c10f6d0 0x7fc62c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.458+0000 7fc631e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c1151b0 con 0x7fc62c10f6d0 2024-07-04T08:15:49.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.458+0000 7fc631e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c115030 con 0x7fc62c102470 2024-07-04T08:15:49.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.458+0000 7fc631e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c115330 con 0x7fc62c101b20 2024-07-04T08:15:49.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.459+0000 7fc62aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c101b20 0x7fc62c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.459+0000 7fc62aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c101b20 0x7fc62c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:47948/0 (socket says 172.21.15.39:47948) 2024-07-04T08:15:49.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.459+0000 7fc62aa00640 1 -- 172.21.15.39:0/172386051 learned_addr learned my addr 172.21.15.39:0/172386051 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:49.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.459+0000 7fc62a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc62c102470 0x7fc62c19c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.459+0000 7fc62b400640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc62c10f6d0 0x7fc62c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.459+0000 7fc62a000640 1 -- 172.21.15.39:0/172386051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c101b20 msgr2=0x7fc62c19bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:49.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.459+0000 7fc62a000640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c101b20 0x7fc62c19bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc62a000640 1 -- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc62c10f6d0 msgr2=0x7fc62c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:49.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc62a000640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc62c10f6d0 0x7fc62c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc62a000640 1 -- 172.21.15.39:0/172386051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c1a1af0 con 0x7fc62c102470 2024-07-04T08:15:49.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc62aa00640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c101b20 0x7fc62c19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:49.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc62a000640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc62c102470 0x7fc62c19c120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc61800e790 tx=0x7fc61800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:49.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc630a00640 1 -- 172.21.15.39:0/172386051 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61800eea0 con 0x7fc62c102470 2024-07-04T08:15:49.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc630a00640 1 -- 172.21.15.39:0/172386051 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61800ccc0 con 0x7fc62c102470 2024-07-04T08:15:49.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc631e00640 1 -- 172.21.15.39:0/172386051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c1a1de0 con 0x7fc62c102470 2024-07-04T08:15:49.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc630a00640 1 -- 172.21.15.39:0/172386051 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618018680 con 0x7fc62c102470 2024-07-04T08:15:49.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.460+0000 7fc631e00640 1 -- 172.21.15.39:0/172386051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c1a8540 con 0x7fc62c102470 2024-07-04T08:15:49.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.465+0000 7fc63844e740 1 Processor -- start 2024-07-04T08:15:49.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.466+0000 7fc63844e740 1 -- start start 2024-07-04T08:15:49.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.466+0000 7fc63844e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592869b1f60 0x559286d56670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.466+0000 7fc63844e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869890a0 0x559286d56bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.466+0000 7fc63844e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559286991280 0x559286d570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.466+0000 7fc63844e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559286cccca0 con 0x5592869b1f60 2024-07-04T08:15:49.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.466+0000 7fc63844e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559286cbc240 con 0x559286991280 2024-07-04T08:15:49.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.466+0000 7fc63844e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559286cc0280 con 0x5592869890a0 2024-07-04T08:15:49.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.466+0000 7fc630a00640 1 -- 172.21.15.39:0/172386051 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7fc618010430 con 0x7fc62c102470 2024-07-04T08:15:49.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592869b1f60 0x559286d56670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559286991280 0x559286d570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592869b1f60 0x559286d56670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58322/0 (socket says 172.21.15.39:58322) 2024-07-04T08:15:49.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62b400640 1 -- 172.21.15.39:0/346133838 learned_addr learned my addr 172.21.15.39:0/346133838 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:49.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62aa00640 1 --2- 172.21.15.39:0/346133838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869890a0 0x559286d56bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62a000640 1 -- 172.21.15.39:0/346133838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869890a0 msgr2=0x559286d56bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:49.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62a000640 1 --2- 172.21.15.39:0/346133838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869890a0 0x559286d56bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62a000640 1 -- 172.21.15.39:0/346133838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592869b1f60 msgr2=0x559286d56670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:49.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62a000640 1 --2- 172.21.15.39:0/346133838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592869b1f60 0x559286d56670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.467+0000 7fc62a000640 1 -- 172.21.15.39:0/346133838 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559286c54550 con 0x559286991280 2024-07-04T08:15:49.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.468+0000 7fc62a000640 1 --2- 172.21.15.39:0/346133838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559286991280 0x559286d570f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc6180dfe90 tx=0x7fc6180ea610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9726abd27b49eb1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:49.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.468+0000 7fc620c00640 1 -- 172.21.15.39:0/346133838 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180f2020 con 0x559286991280 2024-07-04T08:15:49.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.468+0000 7fc620c00640 1 -- 172.21.15.39:0/346133838 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6180eae10 con 0x559286991280 2024-07-04T08:15:49.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.468+0000 7fc620c00640 1 -- 172.21.15.39:0/346133838 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6180ed820 con 0x559286991280 2024-07-04T08:15:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.469+0000 7fc63844e740 1 -- 172.21.15.39:0/346133838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559286991280 msgr2=0x559286d570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.469+0000 7fc63844e740 1 --2- 172.21.15.39:0/346133838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559286991280 0x559286d570f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc6180dfe90 tx=0x7fc6180ea610 comp rx=0 tx=0).stop 2024-07-04T08:15:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.469+0000 7fc630a00640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6100b2950 0x7fc6100b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.470+0000 7fc630a00640 1 -- 172.21.15.39:0/172386051 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc6180e29c0 con 0x7fc62c102470 2024-07-04T08:15:49.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.470+0000 7fc62b400640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6100b2950 0x7fc6100b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.470+0000 7fc63844e740 1 -- 172.21.15.39:0/346133838 shutdown_connections 2024-07-04T08:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.470+0000 7fc63844e740 1 --2- 172.21.15.39:0/346133838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559286991280 0x559286d570f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.470+0000 7fc63844e740 1 --2- 172.21.15.39:0/346133838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869890a0 0x559286d56bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.470+0000 7fc63844e740 1 --2- 172.21.15.39:0/346133838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592869b1f60 0x559286d56670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.470+0000 7fc63844e740 1 -- 172.21.15.39:0/346133838 >> 172.21.15.39:0/346133838 conn(0x5592869a6350 msgr2=0x559286d54560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.471+0000 7fc63844e740 1 -- 172.21.15.39:0/346133838 shutdown_connections 2024-07-04T08:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.471+0000 7fc63844e740 1 -- 172.21.15.39:0/346133838 wait complete. 2024-07-04T08:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.471+0000 7fc62b400640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6100b2950 0x7fc6100b4e60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc624011670 tx=0x7fc624009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.472+0000 7fc63844e740 1 -- start start 2024-07-04T08:15:49.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.472+0000 7fc63844e740 1 Processor -- start 2024-07-04T08:15:49.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc63844e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d62900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc63844e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc63844e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869b1f60 0x559286d67f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc63844e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559286b205f0 con 0x559286991280 2024-07-04T08:15:49.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc63844e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559286c63970 con 0x5592869890a0 2024-07-04T08:15:49.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc63844e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559286d5cd20 con 0x5592869b1f60 2024-07-04T08:15:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc63844e740 10 client.0 _ll_register_callbacks cb 0x559286cb54a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc62b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869b1f60 0x559286d67f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc62a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc62b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869b1f60 0x559286d67f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:47966/0 (socket says 172.21.15.39:47966) 2024-07-04T08:15:49.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc62b400640 1 -- 172.21.15.39:0/873095620 learned_addr learned my addr 172.21.15.39:0/873095620 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:49.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.473+0000 7fc62aa00640 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d62900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.474+0000 7fc62b400640 1 -- 172.21.15.39:0/873095620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 msgr2=0x559286d62900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.474+0000 7fc62b400640 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d62900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.474+0000 7fc62b400640 1 -- 172.21.15.39:0/873095620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 msgr2=0x559286d62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.474+0000 7fc62b400640 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d62ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.474+0000 7fc62b400640 1 -- 172.21.15.39:0/873095620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559286a1d540 con 0x5592869b1f60 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.474+0000 7fc62aa00640 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d62900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.474+0000 7fc62b400640 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869b1f60 0x559286d67f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc624003f60 tx=0x7fc624004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5451d52dbfbf8663 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.474+0000 7fc615600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.475+0000 7fc615600640 1 -- 172.21.15.39:0/873095620 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62400f9d0 con 0x5592869b1f60 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.475+0000 7fc615600640 1 -- 172.21.15.39:0/873095620 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc62400fb70 con 0x5592869b1f60 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.475+0000 7fc63844e740 1 -- 172.21.15.39:0/873095620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6180eae10 con 0x5592869b1f60 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.475+0000 7fc615600640 1 -- 172.21.15.39:0/873095620 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc62400fd70 con 0x5592869b1f60 2024-07-04T08:15:49.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.475+0000 7fc63844e740 10 client.25555 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:49.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.475+0000 7fc63844e740 1 -- 172.21.15.39:0/873095620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559286d81ec0 con 0x5592869b1f60 2024-07-04T08:15:49.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.476+0000 7fc615600640 1 -- 172.21.15.39:0/873095620 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6180eae10 con 0x5592869b1f60 2024-07-04T08:15:49.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.476+0000 7fc615600640 1 -- 172.21.15.39:0/873095620 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc624026070 con 0x5592869b1f60 2024-07-04T08:15:49.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.476+0000 7fc63844e740 10 client.25555 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:49.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.476+0000 7fc63844e740 10 client.25555 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T08:15:49.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.476+0000 7fc63844e740 20 client.25555 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:49.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.476+0000 7fc63844e740 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592869708e0 0x559286d85a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.477+0000 7fc63844e740 4 client.25555 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:15:49.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.477+0000 7fc63844e740 1 -- 172.21.15.39:0/873095620 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559286ccebf0 con 0x5592869708e0 2024-07-04T08:15:49.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.478+0000 7fc62aa00640 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592869708e0 0x559286d85a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.482+0000 7fc62aa00640 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592869708e0 0x559286d85a20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98938b4f9af25b87 server_cookie=eca8c896c772c3e4 in_seq=0 out_seq=0 2024-07-04T08:15:49.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.482+0000 7fc615600640 10 client.25555 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:15:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.483+0000 7fc615600640 1 -- 172.21.15.39:0/873095620 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559286ccebf0 con 0x5592869708e0 2024-07-04T08:15:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.483+0000 7fc615600640 10 client.25555 handle_command_reply: tid=0 2024-07-04T08:15:49.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.483+0000 7fc63844e740 1 client.25555 shutdown 2024-07-04T08:15:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 -- 172.21.15.39:0/873095620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869b1f60 msgr2=0x559286d67f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869b1f60 0x559286d67f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc624003f60 tx=0x7fc624004ab0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 -- 172.21.15.39:0/873095620 shutdown_connections 2024-07-04T08:15:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592869708e0 0x559286d85a20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869b1f60 0x559286d67f00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 --2- 172.21.15.39:0/873095620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d62900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc615600640 0 client.25555 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:15:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 -- 172.21.15.39:0/873095620 >> 172.21.15.39:0/873095620 conn(0x5592869a6350 msgr2=0x559286d52ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 -- 172.21.15.39:0/873095620 shutdown_connections 2024-07-04T08:15:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 1 -- 172.21.15.39:0/873095620 wait complete. 2024-07-04T08:15:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.484+0000 7fc63844e740 20 client.25555 trim_cache size 0 max 16384 2024-07-04T08:15:49.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.508+0000 7fc63844e740 1 Processor -- start 2024-07-04T08:15:49.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.508+0000 7fc63844e740 1 -- start start 2024-07-04T08:15:49.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.508+0000 7fc63844e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc63844e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc63844e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d5f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc63844e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559286d83d60 con 0x559286991280 2024-07-04T08:15:49.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc63844e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559286d84430 con 0x5592869890a0 2024-07-04T08:15:49.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc63844e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559286d84b00 con 0x5592869a6350 2024-07-04T08:15:49.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc62aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc62aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:47972/0 (socket says 172.21.15.39:47972) 2024-07-04T08:15:49.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc62aa00640 1 -- 172.21.15.39:0/932658824 learned_addr learned my addr 172.21.15.39:0/932658824 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:49.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc62b400640 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d5f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc62a000640 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.509+0000 7fc62aa00640 1 -- 172.21.15.39:0/932658824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 msgr2=0x559286d628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:49.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.510+0000 7fc62aa00640 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d628e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.510+0000 7fc62aa00640 1 -- 172.21.15.39:0/932658824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 msgr2=0x559286d5f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:49.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.510+0000 7fc62aa00640 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d5f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.510+0000 7fc62aa00640 1 -- 172.21.15.39:0/932658824 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559286d54ee0 con 0x5592869a6350 2024-07-04T08:15:49.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.510+0000 7fc62b400640 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d5f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:15:49.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.510+0000 7fc62aa00640 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d532b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc61c032ff0 tx=0x7fc61c0058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1d4cd528ca3834 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:49.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.511+0000 7fc617e00640 1 -- 172.21.15.39:0/932658824 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c004250 con 0x5592869a6350 2024-07-04T08:15:49.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.511+0000 7fc617e00640 1 -- 172.21.15.39:0/932658824 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61c030e80 con 0x5592869a6350 2024-07-04T08:15:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.511+0000 7fc617e00640 1 -- 172.21.15.39:0/932658824 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c004b80 con 0x5592869a6350 2024-07-04T08:15:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.512+0000 7fc63844e740 1 -- 172.21.15.39:0/932658824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 msgr2=0x559286d532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.512+0000 7fc63844e740 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d532b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc61c032ff0 tx=0x7fc61c0058d0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.512+0000 7fc63844e740 1 -- 172.21.15.39:0/932658824 shutdown_connections 2024-07-04T08:15:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.512+0000 7fc63844e740 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d5f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.513+0000 7fc63844e740 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.513+0000 7fc63844e740 1 --2- 172.21.15.39:0/932658824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d532b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.513+0000 7fc63844e740 1 -- 172.21.15.39:0/932658824 >> 172.21.15.39:0/932658824 conn(0x559286d68640 msgr2=0x559286d52ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.513+0000 7fc63844e740 1 -- 172.21.15.39:0/932658824 shutdown_connections 2024-07-04T08:15:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.513+0000 7fc63844e740 1 -- 172.21.15.39:0/932658824 wait complete. 2024-07-04T08:15:49.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.514+0000 7fc63844e740 1 -- start start 2024-07-04T08:15:49.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.514+0000 7fc63844e740 1 Processor -- start 2024-07-04T08:15:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc63844e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d8e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc63844e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc63844e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d89c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc63844e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559286d858a0 con 0x559286991280 2024-07-04T08:15:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc63844e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559286d851d0 con 0x5592869890a0 2024-07-04T08:15:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc63844e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559286d85f70 con 0x5592869a6350 2024-07-04T08:15:49.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc63844e740 10 client.0 _ll_register_callbacks cb 0x7fc624009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:49.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d89c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d89c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:47978/0 (socket says 172.21.15.39:47978) 2024-07-04T08:15:49.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62b400640 1 -- 172.21.15.39:0/3640736721 learned_addr learned my addr 172.21.15.39:0/3640736721 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:49.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62aa00640 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d8e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62b400640 1 -- 172.21.15.39:0/3640736721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 msgr2=0x559286d8e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62b400640 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d8e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62b400640 1 -- 172.21.15.39:0/3640736721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 msgr2=0x559286d896e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62b400640 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d896e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.515+0000 7fc62b400640 1 -- 172.21.15.39:0/3640736721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559286d8a260 con 0x5592869a6350 2024-07-04T08:15:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.516+0000 7fc62b400640 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d89c20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc624000f80 tx=0x7fc624009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=210fd816aa289f37 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.516+0000 7fc615600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:15:49.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.516+0000 7fc615600640 1 -- 172.21.15.39:0/3640736721 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62403b060 con 0x5592869a6350 2024-07-04T08:15:49.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.516+0000 7fc615600640 1 -- 172.21.15.39:0/3640736721 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fc624036640 con 0x5592869a6350 2024-07-04T08:15:49.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.516+0000 7fc63844e740 1 -- 172.21.15.39:0/3640736721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61c030e80 con 0x5592869a6350 2024-07-04T08:15:49.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.517+0000 7fc615600640 1 -- 172.21.15.39:0/3640736721 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc624003d50 con 0x5592869a6350 2024-07-04T08:15:49.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.517+0000 7fc63844e740 10 client.25561 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:49.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.517+0000 7fc63844e740 1 -- 172.21.15.39:0/3640736721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559286deea70 con 0x5592869a6350 2024-07-04T08:15:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.517+0000 7fc615600640 1 -- 172.21.15.39:0/3640736721 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61c030e80 con 0x5592869a6350 2024-07-04T08:15:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.518+0000 7fc615600640 1 -- 172.21.15.39:0/3640736721 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc624026040 con 0x5592869a6350 2024-07-04T08:15:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.518+0000 7fc63844e740 10 client.25561 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.518+0000 7fc63844e740 10 client.25561 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-04T08:15:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.518+0000 7fc63844e740 20 client.25561 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.518+0000 7fc63844e740 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592869b1f60 0x559286df2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.518+0000 7fc63844e740 4 client.25561 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:15:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.518+0000 7fc63844e740 1 -- 172.21.15.39:0/3640736721 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "status"}) -- 0x559286cca690 con 0x5592869b1f60 2024-07-04T08:15:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.518+0000 7fc62aa00640 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592869b1f60 0x559286df2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.519+0000 7fc62aa00640 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592869b1f60 0x559286df2390 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=48f96bdc3a4c130b server_cookie=9c310b903f1ab507 in_seq=0 out_seq=0 2024-07-04T08:15:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.519+0000 7fc615600640 10 client.25561 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:15:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.519+0000 7fc615600640 1 -- 172.21.15.39:0/3640736721 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559286cca690 con 0x5592869b1f60 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.519+0000 7fc615600640 10 client.25561 handle_command_reply: tid=0 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.519+0000 7fc63844e740 1 client.25561 shutdown 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 -- 172.21.15.39:0/3640736721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 msgr2=0x559286d89c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d89c20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc624000f80 tx=0x7fc624009570 comp rx=0 tx=0).stop 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 -- 172.21.15.39:0/3640736721 shutdown_connections 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592869b1f60 0x559286df2390 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559286991280 0x559286d896e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592869a6350 0x559286d89c20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 --2- 172.21.15.39:0/3640736721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592869890a0 0x559286d8e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc615600640 0 client.25561 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 -- 172.21.15.39:0/3640736721 >> 172.21.15.39:0/3640736721 conn(0x559286d68640 msgr2=0x559286d576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 -- 172.21.15.39:0/3640736721 shutdown_connections 2024-07-04T08:15:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 1 -- 172.21.15.39:0/3640736721 wait complete. 2024-07-04T08:15:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.520+0000 7fc63844e740 20 client.25561 trim_cache size 0 max 16384 2024-07-04T08:15:49.523 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:15:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:15:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 4, 2024-07-04T08:15:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24673, 2024-07-04T08:15:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:15:49.523 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:15:49.524 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:15:49.524 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 284.67232837300003, 2024-07-04T08:15:49.524 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 30, 2024-07-04T08:15:49.524 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:15:49.524 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 92, 2024-07-04T08:15:49.524 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 404.751311853 2024-07-04T08:15:49.524 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:15:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.521+0000 7fc631e00640 1 -- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6100b2950 msgr2=0x7fc6100b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.521+0000 7fc631e00640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6100b2950 0x7fc6100b4e60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc624011670 tx=0x7fc624009500 comp rx=0 tx=0).stop 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.521+0000 7fc631e00640 1 -- 172.21.15.39:0/172386051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc62c102470 msgr2=0x7fc62c19c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.521+0000 7fc631e00640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc62c102470 0x7fc62c19c120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc61800e790 tx=0x7fc61800ec50 comp rx=0 tx=0).stop 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.522+0000 7fc631e00640 1 -- 172.21.15.39:0/172386051 shutdown_connections 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.522+0000 7fc631e00640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc6100b2950 0x7fc6100b4e60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.522+0000 7fc631e00640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc62c10f6d0 0x7fc62c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.522+0000 7fc631e00640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc62c102470 0x7fc62c19c120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.522+0000 7fc631e00640 1 --2- 172.21.15.39:0/172386051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc62c101b20 0x7fc62c19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.522+0000 7fc631e00640 1 -- 172.21.15.39:0/172386051 >> 172.21.15.39:0/172386051 conn(0x7fc62c078c80 msgr2=0x7fc62c108140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.522+0000 7fc631e00640 1 -- 172.21.15.39:0/172386051 shutdown_connections 2024-07-04T08:15:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:49.522+0000 7fc631e00640 1 -- 172.21.15.39:0/172386051 wait complete. 2024-07-04T08:15:50.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:50 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:15:50.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:50 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:15:50.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:50 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:15:50.279 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:50 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:15:50.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:50 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:15:50.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:50 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:15:50.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:50 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:15:50.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:50 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:15:50.344 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:15:50.351 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.h status 2024-07-04T08:15:50.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:50 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:15:50.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:50 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:15:50.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:50 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:15:50.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:50 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:15:50.835 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:50.854 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:51.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:51 smithi039 ceph-mon[32062]: pgmap v499: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:15:51.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:51 smithi133 ceph-mon[37050]: pgmap v499: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:15:51.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:51 smithi050 ceph-mon[36577]: pgmap v499: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:15:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:53 smithi039 ceph-mon[32062]: pgmap v500: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:15:53.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:53 smithi133 ceph-mon[37050]: pgmap v500: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:15:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:53 smithi050 ceph-mon[36577]: pgmap v500: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:15:53.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.991+0000 7fa2a6800640 1 -- 172.21.15.39:0/469909093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 msgr2=0x7fa2a0103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:53.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.991+0000 7fa2a6800640 1 --2- 172.21.15.39:0/469909093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 0x7fa2a0103310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa294009900 tx=0x7fa29402a400 comp rx=0 tx=0).stop 2024-07-04T08:15:53.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.991+0000 7fa2a6800640 1 -- 172.21.15.39:0/469909093 shutdown_connections 2024-07-04T08:15:53.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.991+0000 7fa2a6800640 1 --2- 172.21.15.39:0/469909093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0109190 0x7fa2a010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:53.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.991+0000 7fa2a6800640 1 --2- 172.21.15.39:0/469909093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a0108840 0x7fa2a0108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:53.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.991+0000 7fa2a6800640 1 --2- 172.21.15.39:0/469909093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 0x7fa2a0103310 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:53.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.991+0000 7fa2a6800640 1 -- 172.21.15.39:0/469909093 >> 172.21.15.39:0/469909093 conn(0x7fa2a00fe990 msgr2=0x7fa2a0100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:53.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.991+0000 7fa2a6800640 1 -- 172.21.15.39:0/469909093 shutdown_connections 2024-07-04T08:15:53.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.991+0000 7fa2a6800640 1 -- 172.21.15.39:0/469909093 wait complete. 2024-07-04T08:15:53.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.993+0000 7fa2a6800640 1 Processor -- start 2024-07-04T08:15:53.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.993+0000 7fa2a6800640 1 -- start start 2024-07-04T08:15:53.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.993+0000 7fa2a6800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 0x7fa2a0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:53.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.993+0000 7fa2a6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0108840 0x7fa2a0199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:53.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.993+0000 7fa2a6800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a0109190 0x7fa2a019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.993+0000 7fa2a6800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0112ea0 con 0x7fa2a0108840 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.993+0000 7fa2a6800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0112d20 con 0x7fa2a0102f00 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.993+0000 7fa2a6800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0113020 con 0x7fa2a0109190 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 0x7fa2a0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a4a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0108840 0x7fa2a0199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 0x7fa2a0199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58902/0 (socket says 172.21.15.39:58902) 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a5400640 1 -- 172.21.15.39:0/1252192240 learned_addr learned my addr 172.21.15.39:0/1252192240 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a0109190 0x7fa2a019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a5400640 1 -- 172.21.15.39:0/1252192240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a0109190 msgr2=0x7fa2a019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a5400640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a0109190 0x7fa2a019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a5400640 1 -- 172.21.15.39:0/1252192240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0108840 msgr2=0x7fa2a0199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:53.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a5400640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0108840 0x7fa2a0199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.994+0000 7fa2a5400640 1 -- 172.21.15.39:0/1252192240 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a019f7a0 con 0x7fa2a0102f00 2024-07-04T08:15:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.995+0000 7fa2a5400640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 0x7fa2a0199830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa294005ab0 tx=0x7fa294003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.995+0000 7fa29e000640 1 -- 172.21.15.39:0/1252192240 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29403f070 con 0x7fa2a0102f00 2024-07-04T08:15:53.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.995+0000 7fa2a6800640 1 -- 172.21.15.39:0/1252192240 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a019fa30 con 0x7fa2a0102f00 2024-07-04T08:15:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.995+0000 7fa29e000640 1 -- 172.21.15.39:0/1252192240 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa294031b60 con 0x7fa2a0102f00 2024-07-04T08:15:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.995+0000 7fa29e000640 1 -- 172.21.15.39:0/1252192240 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294035030 con 0x7fa2a0102f00 2024-07-04T08:15:53.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.995+0000 7fa2a6800640 1 -- 172.21.15.39:0/1252192240 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a01a61f0 con 0x7fa2a0102f00 2024-07-04T08:15:54.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.999+0000 7fa2ae361740 1 Processor -- start 2024-07-04T08:15:54.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.999+0000 7fa2ae361740 1 -- start start 2024-07-04T08:15:54.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.999+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f66940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.999+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 0x5563d8f66e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.999+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f673c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.999+0000 7fa29e000640 1 -- 172.21.15.39:0/1252192240 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7fa294042070 con 0x7fa2a0102f00 2024-07-04T08:15:54.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.999+0000 7fa2ae361740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5563d8edced0 con 0x5563d8b991b0 2024-07-04T08:15:54.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.999+0000 7fa2ae361740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5563d8ecc470 con 0x5563d8bc2070 2024-07-04T08:15:54.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:53.999+0000 7fa2ae361740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5563d8ed04b0 con 0x5563d8ba1390 2024-07-04T08:15:54.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f66940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 0x5563d8f66e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f66940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58914/0 (socket says 172.21.15.39:58914) 2024-07-04T08:15:54.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a5e00640 1 -- 172.21.15.39:0/452987384 learned_addr learned my addr 172.21.15.39:0/452987384 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:54.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/452987384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a5e00640 1 -- 172.21.15.39:0/452987384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 msgr2=0x5563d8f673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/452987384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a5e00640 1 -- 172.21.15.39:0/452987384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 msgr2=0x5563d8f66e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/452987384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 0x5563d8f66e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.000+0000 7fa2a5e00640 1 -- 172.21.15.39:0/452987384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d8e64640 con 0x5563d8bc2070 2024-07-04T08:15:54.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.001+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/452987384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f66940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa298009fd0 tx=0x7fa2980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d774dedb15d5e9cc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.001+0000 7fa292a00640 1 -- 172.21.15.39:0/452987384 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298004d20 con 0x5563d8bc2070 2024-07-04T08:15:54.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.002+0000 7fa292a00640 1 -- 172.21.15.39:0/452987384 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa298004050 con 0x5563d8bc2070 2024-07-04T08:15:54.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.002+0000 7fa292a00640 1 -- 172.21.15.39:0/452987384 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298005280 con 0x5563d8bc2070 2024-07-04T08:15:54.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.002+0000 7fa29e000640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2880b28d0 0x7fa2880b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.003+0000 7fa29e000640 1 -- 172.21.15.39:0/1252192240 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa294102fd0 con 0x7fa2a0102f00 2024-07-04T08:15:54.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.003+0000 7fa2ae361740 1 -- 172.21.15.39:0/452987384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 msgr2=0x5563d8f66940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.003+0000 7fa2ae361740 1 --2- 172.21.15.39:0/452987384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f66940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa298009fd0 tx=0x7fa2980118d0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.003+0000 7fa2a5400640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2880b28d0 0x7fa2880b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.004+0000 7fa2ae361740 1 -- 172.21.15.39:0/452987384 shutdown_connections 2024-07-04T08:15:54.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.004+0000 7fa2ae361740 1 --2- 172.21.15.39:0/452987384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 0x5563d8f66e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.004+0000 7fa2ae361740 1 --2- 172.21.15.39:0/452987384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.004+0000 7fa2ae361740 1 --2- 172.21.15.39:0/452987384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f66940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.004+0000 7fa2ae361740 1 -- 172.21.15.39:0/452987384 >> 172.21.15.39:0/452987384 conn(0x5563d8bb6460 msgr2=0x5563d8f64830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.004+0000 7fa2ae361740 1 -- 172.21.15.39:0/452987384 shutdown_connections 2024-07-04T08:15:54.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.004+0000 7fa2ae361740 1 -- 172.21.15.39:0/452987384 wait complete. 2024-07-04T08:15:54.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.004+0000 7fa2a5400640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2880b28d0 0x7fa2880b4de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa2940c9d90 tx=0x7fa294106a80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.005+0000 7fa2ae361740 1 -- start start 2024-07-04T08:15:54.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.005+0000 7fa2ae361740 1 Processor -- start 2024-07-04T08:15:54.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.005+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 0x5563d8f72500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.005+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f72a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.005+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f7e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.005+0000 7fa2ae361740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5563d8d30700 con 0x5563d8b991b0 2024-07-04T08:15:54.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.005+0000 7fa2ae361740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5563d8e73ba0 con 0x5563d8bc2070 2024-07-04T08:15:54.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.005+0000 7fa2ae361740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5563d8f75120 con 0x5563d8ba1390 2024-07-04T08:15:54.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.005+0000 7fa2ae361740 10 client.0 _ll_register_callbacks cb 0x5563d8ec5560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:54.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a4a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f72a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 0x5563d8f72500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a4a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f72a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60798/0 (socket says 172.21.15.39:60798) 2024-07-04T08:15:54.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a4a00640 1 -- 172.21.15.39:0/494504030 learned_addr learned my addr 172.21.15.39:0/494504030 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:54.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f7e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a4a00640 1 -- 172.21.15.39:0/494504030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 msgr2=0x5563d8f7e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f7e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a4a00640 1 -- 172.21.15.39:0/494504030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 msgr2=0x5563d8f72500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 0x5563d8f72500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.006+0000 7fa2a4a00640 1 -- 172.21.15.39:0/494504030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d8c2d650 con 0x5563d8ba1390 2024-07-04T08:15:54.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.007+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f72a40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa28c012700 tx=0x7fa28c012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3063354fc0f60e0f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.007+0000 7fa287400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:15:54.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.007+0000 7fa287400640 1 -- 172.21.15.39:0/494504030 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c00f920 con 0x5563d8ba1390 2024-07-04T08:15:54.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.008+0000 7fa2ae361740 1 -- 172.21.15.39:0/494504030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa298004050 con 0x5563d8ba1390 2024-07-04T08:15:54.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.008+0000 7fa287400640 1 -- 172.21.15.39:0/494504030 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa28c00fac0 con 0x5563d8ba1390 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.008+0000 7fa287400640 1 -- 172.21.15.39:0/494504030 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa28c00fcc0 con 0x5563d8ba1390 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.008+0000 7fa2ae361740 10 client.25573 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.008+0000 7fa2ae361740 1 -- 172.21.15.39:0/494504030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5563d8f81130 con 0x5563d8ba1390 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.009+0000 7fa287400640 1 -- 172.21.15.39:0/494504030 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa298004050 con 0x5563d8ba1390 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.009+0000 7fa287400640 1 -- 172.21.15.39:0/494504030 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa28c01c420 con 0x5563d8ba1390 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.009+0000 7fa2ae361740 10 client.25573 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.009+0000 7fa2ae361740 10 client.25573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.009+0000 7fa2ae361740 20 client.25573 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.009+0000 7fa2ae361740 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563d8b809f0 0x5563d8f84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.009+0000 7fa2ae361740 4 client.25573 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:15:54.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.009+0000 7fa2ae361740 1 -- 172.21.15.39:0/494504030 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563d8edee20 con 0x5563d8b809f0 2024-07-04T08:15:54.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.010+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563d8b809f0 0x5563d8f84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.010+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563d8b809f0 0x5563d8f84690 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126f0fe7a93e63fa server_cookie=c051c295ee381579 in_seq=0 out_seq=0 2024-07-04T08:15:54.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.010+0000 7fa287400640 10 client.25573 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:54.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.011+0000 7fa287400640 1 -- 172.21.15.39:0/494504030 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5563d8edee20 con 0x5563d8b809f0 2024-07-04T08:15:54.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.011+0000 7fa287400640 10 client.25573 handle_command_reply: tid=0 2024-07-04T08:15:54.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.011+0000 7fa2ae361740 1 client.25573 shutdown 2024-07-04T08:15:54.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.011+0000 7fa2ae361740 1 -- 172.21.15.39:0/494504030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 msgr2=0x5563d8f72a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.011+0000 7fa2ae361740 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f72a40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa28c012700 tx=0x7fa28c012bc0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa2ae361740 1 -- 172.21.15.39:0/494504030 shutdown_connections 2024-07-04T08:15:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa2ae361740 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563d8b809f0 0x5563d8f84690 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa2ae361740 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8bc2070 0x5563d8f7e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa2ae361740 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f72a40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa2ae361740 1 --2- 172.21.15.39:0/494504030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8b991b0 0x5563d8f72500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa2ae361740 1 -- 172.21.15.39:0/494504030 >> 172.21.15.39:0/494504030 conn(0x5563d8bb6460 msgr2=0x5563d8f69b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa287400640 0 client.25573 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa2ae361740 1 -- 172.21.15.39:0/494504030 shutdown_connections 2024-07-04T08:15:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa2ae361740 1 -- 172.21.15.39:0/494504030 wait complete. 2024-07-04T08:15:54.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.012+0000 7fa2ae361740 20 client.25573 trim_cache size 0 max 16384 2024-07-04T08:15:54.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.033+0000 7fa2ae361740 1 Processor -- start 2024-07-04T08:15:54.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.033+0000 7fa2ae361740 1 -- start start 2024-07-04T08:15:54.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8b991b0 0x5563d8f68080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f7e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 0x5563d8f7e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2ae361740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01ade40 con 0x5563d8bc2070 2024-07-04T08:15:54.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2ae361740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01af980 con 0x5563d8b991b0 2024-07-04T08:15:54.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2ae361740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01ae510 con 0x5563d8ba1390 2024-07-04T08:15:54.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8b991b0 0x5563d8f68080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2a4a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f7e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8b991b0 0x5563d8f68080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58934/0 (socket says 172.21.15.39:58934) 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2a5e00640 1 -- 172.21.15.39:0/2918084874 learned_addr learned my addr 172.21.15.39:0/2918084874 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.034+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/2918084874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 0x5563d8f7e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.035+0000 7fa2a4a00640 1 -- 172.21.15.39:0/2918084874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8b991b0 msgr2=0x5563d8f68080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.035+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/2918084874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8b991b0 0x5563d8f68080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.035+0000 7fa2a4a00640 1 -- 172.21.15.39:0/2918084874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 msgr2=0x5563d8f7e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.035+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/2918084874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 0x5563d8f7e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.035+0000 7fa2a4a00640 1 -- 172.21.15.39:0/2918084874 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d8f6db90 con 0x5563d8ba1390 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.035+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/2918084874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f7e200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa28c01c960 tx=0x7fa28c005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f66ec495eaba443 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.035+0000 7fa292000640 1 -- 172.21.15.39:0/2918084874 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c02f070 con 0x5563d8ba1390 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.036+0000 7fa292000640 1 -- 172.21.15.39:0/2918084874 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c005590 con 0x5563d8ba1390 2024-07-04T08:15:54.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.036+0000 7fa292000640 1 -- 172.21.15.39:0/2918084874 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c01c680 con 0x5563d8ba1390 2024-07-04T08:15:54.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.036+0000 7fa2ae361740 1 -- 172.21.15.39:0/2918084874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 msgr2=0x5563d8f7e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.036+0000 7fa2ae361740 1 --2- 172.21.15.39:0/2918084874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f7e200 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa28c01c960 tx=0x7fa28c005db0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.037+0000 7fa2ae361740 1 -- 172.21.15.39:0/2918084874 shutdown_connections 2024-07-04T08:15:54.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.037+0000 7fa2ae361740 1 --2- 172.21.15.39:0/2918084874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 0x5563d8f7e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.037+0000 7fa2ae361740 1 --2- 172.21.15.39:0/2918084874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8ba1390 0x5563d8f7e200 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.037+0000 7fa2ae361740 1 --2- 172.21.15.39:0/2918084874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8b991b0 0x5563d8f68080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.037+0000 7fa2ae361740 1 -- 172.21.15.39:0/2918084874 >> 172.21.15.39:0/2918084874 conn(0x5563d8bb6460 msgr2=0x5563d8f67970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.037+0000 7fa2ae361740 1 -- 172.21.15.39:0/2918084874 shutdown_connections 2024-07-04T08:15:54.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.037+0000 7fa2ae361740 1 -- 172.21.15.39:0/2918084874 wait complete. 2024-07-04T08:15:54.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.037+0000 7fa2ae361740 1 -- start start 2024-07-04T08:15:54.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.038+0000 7fa2ae361740 1 Processor -- start 2024-07-04T08:15:54.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.038+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8b991b0 0x5563d8f7a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.038+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8ba1390 0x5563d8f75c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.038+0000 7fa2ae361740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 0x5563d8f76160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.038+0000 7fa2ae361740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01af2b0 con 0x5563d8bc2070 2024-07-04T08:15:54.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.038+0000 7fa2ae361740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01aebe0 con 0x5563d8ba1390 2024-07-04T08:15:54.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.038+0000 7fa2ae361740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01abe20 con 0x5563d8b991b0 2024-07-04T08:15:54.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.038+0000 7fa2ae361740 10 client.0 _ll_register_callbacks cb 0x7fa28c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8ba1390 0x5563d8f75c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 0x5563d8f76160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8ba1390 0x5563d8f75c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58940/0 (socket says 172.21.15.39:58940) 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a4a00640 1 -- 172.21.15.39:0/3983946639 learned_addr learned my addr 172.21.15.39:0/3983946639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8b991b0 0x5563d8f7a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a4a00640 1 -- 172.21.15.39:0/3983946639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8b991b0 msgr2=0x5563d8f7a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8b991b0 0x5563d8f7a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a4a00640 1 -- 172.21.15.39:0/3983946639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 msgr2=0x5563d8f76160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 0x5563d8f76160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a4a00640 1 -- 172.21.15.39:0/3983946639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d8f766a0 con 0x5563d8ba1390 2024-07-04T08:15:54.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8b991b0 0x5563d8f7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:15:54.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa2a4a00640 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8ba1390 0x5563d8f75c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00f890 tx=0x7fa28c01ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fcdce4c020f29e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.039+0000 7fa287400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.040+0000 7fa287400640 1 -- 172.21.15.39:0/3983946639 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c02f030 con 0x5563d8ba1390 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.040+0000 7fa287400640 1 -- 172.21.15.39:0/3983946639 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa28c0050c0 con 0x5563d8ba1390 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.040+0000 7fa2ae361740 1 -- 172.21.15.39:0/3983946639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28c005590 con 0x5563d8ba1390 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.040+0000 7fa287400640 1 -- 172.21.15.39:0/3983946639 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa28c0052c0 con 0x5563d8ba1390 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.040+0000 7fa2ae361740 10 client.25598 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.040+0000 7fa2ae361740 1 -- 172.21.15.39:0/3983946639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5563d8ff52a0 con 0x5563d8ba1390 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.040+0000 7fa287400640 1 -- 172.21.15.39:0/3983946639 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c005590 con 0x5563d8ba1390 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.041+0000 7fa287400640 1 -- 172.21.15.39:0/3983946639 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa28c02a640 con 0x5563d8ba1390 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.041+0000 7fa2ae361740 10 client.25598 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.041+0000 7fa2ae361740 10 client.25598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.041+0000 7fa2ae361740 20 client.25598 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:54.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.041+0000 7fa2ae361740 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563d8b55da0 0x5563d8ff7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.041+0000 7fa2ae361740 4 client.25598 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.041+0000 7fa2ae361740 1 -- 172.21.15.39:0/3983946639 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563d8eda8c0 con 0x5563d8b55da0 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.041+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563d8b55da0 0x5563d8ff7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.041+0000 7fa2a5e00640 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563d8b55da0 0x5563d8ff7830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2aa8ae56a66f90 server_cookie=11d08d82deb344cb in_seq=0 out_seq=0 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa287400640 10 client.25598 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa287400640 1 -- 172.21.15.39:0/3983946639 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5563d8eda8c0 con 0x5563d8b55da0 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa287400640 10 client.25598 handle_command_reply: tid=0 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 client.25598 shutdown 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 -- 172.21.15.39:0/3983946639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8ba1390 msgr2=0x5563d8f75c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8ba1390 0x5563d8f75c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00f890 tx=0x7fa28c01ce60 comp rx=0 tx=0).stop 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 -- 172.21.15.39:0/3983946639 shutdown_connections 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563d8b55da0 0x5563d8ff7830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5563d8bc2070 0x5563d8f76160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa287400640 0 client.25598 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:15:54.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5563d8ba1390 0x5563d8f75c20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 --2- 172.21.15.39:0/3983946639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563d8b991b0 0x5563d8f7a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 -- 172.21.15.39:0/3983946639 >> 172.21.15.39:0/3983946639 conn(0x5563d8bb6460 msgr2=0x5563d8f69a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 -- 172.21.15.39:0/3983946639 shutdown_connections 2024-07-04T08:15:54.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.042+0000 7fa2ae361740 1 -- 172.21.15.39:0/3983946639 wait complete. 2024-07-04T08:15:54.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.043+0000 7fa2ae361740 20 client.25598 trim_cache size 0 max 16384 2024-07-04T08:15:54.047 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:15:54.047 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:15:54.047 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:15:54.047 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:15:54.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.043+0000 7fa2a6800640 1 -- 172.21.15.39:0/1252192240 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2880b28d0 msgr2=0x7fa2880b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.043+0000 7fa2a6800640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2880b28d0 0x7fa2880b4de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa2940c9d90 tx=0x7fa294106a80 comp rx=0 tx=0).stop 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.043+0000 7fa2a6800640 1 -- 172.21.15.39:0/1252192240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 msgr2=0x7fa2a0199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.043+0000 7fa2a6800640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 0x7fa2a0199830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa294005ab0 tx=0x7fa294003c10 comp rx=0 tx=0).stop 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.044+0000 7fa2a6800640 1 -- 172.21.15.39:0/1252192240 shutdown_connections 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.044+0000 7fa2a6800640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2880b28d0 0x7fa2880b4de0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.044+0000 7fa2a6800640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2a0108840 0x7fa2a0199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.044+0000 7fa2a6800640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2a0109190 0x7fa2a019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.044+0000 7fa2a6800640 1 --2- 172.21.15.39:0/1252192240 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2a0102f00 0x7fa2a0199830 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.044+0000 7fa2a6800640 1 -- 172.21.15.39:0/1252192240 >> 172.21.15.39:0/1252192240 conn(0x7fa2a00fe990 msgr2=0x7fa2a0100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.044+0000 7fa2a6800640 1 -- 172.21.15.39:0/1252192240 shutdown_connections 2024-07-04T08:15:54.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.044+0000 7fa2a6800640 1 -- 172.21.15.39:0/1252192240 wait complete. 2024-07-04T08:15:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.060+0000 7f2ff6000640 1 -- 172.21.15.39:0/2598163548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff010f6d0 msgr2=0x7f2ff0111af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.060+0000 7f2ff6000640 1 --2- 172.21.15.39:0/2598163548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff010f6d0 0x7f2ff0111af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009ca0 tx=0x7f2fe802a950 comp rx=0 tx=0).stop 2024-07-04T08:15:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.061+0000 7f2ff6000640 1 -- 172.21.15.39:0/2598163548 shutdown_connections 2024-07-04T08:15:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.061+0000 7f2ff6000640 1 --2- 172.21.15.39:0/2598163548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff0102470 0x7f2ff010f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.061+0000 7f2ff6000640 1 --2- 172.21.15.39:0/2598163548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff010f6d0 0x7f2ff0111af0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.061+0000 7f2ff6000640 1 --2- 172.21.15.39:0/2598163548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff0101b20 0x7f2ff0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.061+0000 7f2ff6000640 1 -- 172.21.15.39:0/2598163548 >> 172.21.15.39:0/2598163548 conn(0x7f2ff0078c80 msgr2=0x7f2ff0079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.061+0000 7f2ff6000640 1 -- 172.21.15.39:0/2598163548 shutdown_connections 2024-07-04T08:15:54.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.061+0000 7f2ff6000640 1 -- 172.21.15.39:0/2598163548 wait complete. 2024-07-04T08:15:54.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.062+0000 7f2ff6000640 1 Processor -- start 2024-07-04T08:15:54.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.062+0000 7f2ff6000640 1 -- start start 2024-07-04T08:15:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2ff6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff0101b20 0x7f2ff019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2ff6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff0102470 0x7f2ff019c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2ff6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff010f6d0 0x7f2ff01a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2ff6000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff01151c0 con 0x7f2ff010f6d0 2024-07-04T08:15:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2ff6000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0115040 con 0x7f2ff0101b20 2024-07-04T08:15:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2ff6000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0115340 con 0x7f2ff0102470 2024-07-04T08:15:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2fef400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff0101b20 0x7f2ff019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2fef400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff0101b20 0x7f2ff019bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58970/0 (socket says 172.21.15.39:58970) 2024-07-04T08:15:54.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2fef400640 1 -- 172.21.15.39:0/1101000755 learned_addr learned my addr 172.21.15.39:0/1101000755 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:54.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2feea00640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff0102470 0x7f2ff019c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2fefe00640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff010f6d0 0x7f2ff01a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2fef400640 1 -- 172.21.15.39:0/1101000755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff0102470 msgr2=0x7f2ff019c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2fef400640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff0102470 0x7f2ff019c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2fef400640 1 -- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff010f6d0 msgr2=0x7f2ff01a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2fef400640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff010f6d0 0x7f2ff01a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.063+0000 7f2fef400640 1 -- 172.21.15.39:0/1101000755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff01a1a40 con 0x7f2ff0101b20 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.064+0000 7f2fef400640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff0101b20 0x7f2ff019bad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000e9f0 tx=0x7f2fe000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.064+0000 7f2fe7e00640 1 -- 172.21.15.39:0/1101000755 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000cd80 con 0x7f2ff0101b20 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.064+0000 7f2fe7e00640 1 -- 172.21.15.39:0/1101000755 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe00108e0 con 0x7f2ff0101b20 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.064+0000 7f2ff6000640 1 -- 172.21.15.39:0/1101000755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff01a1d30 con 0x7f2ff0101b20 2024-07-04T08:15:54.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.064+0000 7f2ff6000640 1 -- 172.21.15.39:0/1101000755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff01a8490 con 0x7f2ff0101b20 2024-07-04T08:15:54.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.064+0000 7f2fe7e00640 1 -- 172.21.15.39:0/1101000755 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0018980 con 0x7f2ff0101b20 2024-07-04T08:15:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.068+0000 7f2ffc682740 1 Processor -- start 2024-07-04T08:15:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.068+0000 7f2ffc682740 1 -- start start 2024-07-04T08:15:54.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.068+0000 7f2fe7e00640 1 -- 172.21.15.39:0/1101000755 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7f2fe0010430 con 0x7f2ff0101b20 2024-07-04T08:15:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.068+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99dbf60 0x55beb9d80650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d80b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99bb280 0x55beb9d810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2ffc682740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55beb9cf6c80 con 0x55beb99bb280 2024-07-04T08:15:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2ffc682740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55beb9ce6220 con 0x55beb99b30a0 2024-07-04T08:15:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2ffc682740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55beb9cea260 con 0x55beb99dbf60 2024-07-04T08:15:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2fefe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99dbf60 0x55beb9d80650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2fefe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99dbf60 0x55beb9d80650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60828/0 (socket says 172.21.15.39:60828) 2024-07-04T08:15:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2fefe00640 1 -- 172.21.15.39:0/1359267899 learned_addr learned my addr 172.21.15.39:0/1359267899 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2fef400640 1 --2- 172.21.15.39:0/1359267899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d80b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.069+0000 7f2feea00640 1 --2- 172.21.15.39:0/1359267899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99bb280 0x55beb9d810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.070+0000 7f2fefe00640 1 -- 172.21.15.39:0/1359267899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 msgr2=0x55beb9d80b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.070+0000 7f2fefe00640 1 --2- 172.21.15.39:0/1359267899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d80b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.070+0000 7f2fefe00640 1 -- 172.21.15.39:0/1359267899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99bb280 msgr2=0x55beb9d810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.070+0000 7f2fefe00640 1 --2- 172.21.15.39:0/1359267899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99bb280 0x55beb9d810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.070+0000 7f2fefe00640 1 -- 172.21.15.39:0/1359267899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb9c7e530 con 0x55beb99dbf60 2024-07-04T08:15:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.070+0000 7f2fefe00640 1 --2- 172.21.15.39:0/1359267899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99dbf60 0x55beb9d80650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8035000 tx=0x7f2fe8009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d017e5566150717 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.071+0000 7f2fe7e00640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2fd40b2950 0x7f2fd40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.071+0000 7f2fe7e00640 1 -- 172.21.15.39:0/1101000755 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2fe00e2c00 con 0x7f2ff0101b20 2024-07-04T08:15:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.071+0000 7f2fe4c00640 1 -- 172.21.15.39:0/1359267899 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8034040 con 0x55beb99dbf60 2024-07-04T08:15:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.071+0000 7f2fe4c00640 1 -- 172.21.15.39:0/1359267899 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe8003e20 con 0x55beb99dbf60 2024-07-04T08:15:54.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.071+0000 7f2fe4c00640 1 -- 172.21.15.39:0/1359267899 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80042a0 con 0x55beb99dbf60 2024-07-04T08:15:54.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.072+0000 7f2fef400640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2fd40b2950 0x7f2fd40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.072+0000 7f2ffc682740 1 -- 172.21.15.39:0/1359267899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99dbf60 msgr2=0x55beb9d80650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.072+0000 7f2ffc682740 1 --2- 172.21.15.39:0/1359267899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99dbf60 0x55beb9d80650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8035000 tx=0x7f2fe8009a20 comp rx=0 tx=0).stop 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.072+0000 7f2ffc682740 1 -- 172.21.15.39:0/1359267899 shutdown_connections 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.072+0000 7f2ffc682740 1 --2- 172.21.15.39:0/1359267899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99bb280 0x55beb9d810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.072+0000 7f2ffc682740 1 --2- 172.21.15.39:0/1359267899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d80b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.072+0000 7f2ffc682740 1 --2- 172.21.15.39:0/1359267899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99dbf60 0x55beb9d80650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.072+0000 7f2ffc682740 1 -- 172.21.15.39:0/1359267899 >> 172.21.15.39:0/1359267899 conn(0x55beb99d0350 msgr2=0x55beb9d7e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.072+0000 7f2ffc682740 1 -- 172.21.15.39:0/1359267899 shutdown_connections 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.073+0000 7f2fef400640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2fd40b2950 0x7f2fd40b4e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00a9ba0 tx=0x7f2fe00e66b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.073+0000 7f2ffc682740 1 -- 172.21.15.39:0/1359267899 wait complete. 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.073+0000 7f2ffc682740 1 -- start start 2024-07-04T08:15:54.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.073+0000 7f2ffc682740 1 Processor -- start 2024-07-04T08:15:54.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.074+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d8c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.074+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d8c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.074+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 0x55beb9d98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.074+0000 7f2ffc682740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55beb9b4a5f0 con 0x55beb99dbf60 2024-07-04T08:15:54.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.074+0000 7f2ffc682740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55beb9c8d950 con 0x55beb99b30a0 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.074+0000 7f2ffc682740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55beb9d8edd0 con 0x55beb99bb280 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.074+0000 7f2ffc682740 10 client.0 _ll_register_callbacks cb 0x55beb9cdf480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2feea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d8c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2feea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d8c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60830/0 (socket says 172.21.15.39:60830) 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2feea00640 1 -- 172.21.15.39:0/4236583632 learned_addr learned my addr 172.21.15.39:0/4236583632 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2fefe00640 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d8c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2fefe00640 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 0x55beb9d98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2feea00640 1 -- 172.21.15.39:0/4236583632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 msgr2=0x55beb9d8c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2feea00640 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d8c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2feea00640 1 -- 172.21.15.39:0/4236583632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 msgr2=0x55beb9d98120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2feea00640 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 0x55beb9d98120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.075+0000 7f2feea00640 1 -- 172.21.15.39:0/4236583632 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb9a47540 con 0x55beb99bb280 2024-07-04T08:15:54.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.076+0000 7f2feea00640 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d8c6f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009fd0 tx=0x7f2fdc012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=110f78ea59594670 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.076+0000 7f2fd9600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:15:54.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.076+0000 7f2fd9600640 1 -- 172.21.15.39:0/4236583632 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00f920 con 0x55beb99bb280 2024-07-04T08:15:54.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.076+0000 7f2fd9600640 1 -- 172.21.15.39:0/4236583632 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2fdc00fac0 con 0x55beb99bb280 2024-07-04T08:15:54.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.076+0000 7f2ffc682740 1 -- 172.21.15.39:0/4236583632 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fe8003e20 con 0x55beb99bb280 2024-07-04T08:15:54.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.077+0000 7f2fd9600640 1 -- 172.21.15.39:0/4236583632 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2fdc00fcc0 con 0x55beb99bb280 2024-07-04T08:15:54.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.077+0000 7f2ffc682740 10 client.25591 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:54.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.077+0000 7f2ffc682740 1 -- 172.21.15.39:0/4236583632 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beb9d9ae40 con 0x55beb99bb280 2024-07-04T08:15:54.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.077+0000 7f2fd9600640 1 -- 172.21.15.39:0/4236583632 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe8003e20 con 0x55beb99bb280 2024-07-04T08:15:54.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.078+0000 7f2fd9600640 1 -- 172.21.15.39:0/4236583632 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2fdc01c420 con 0x55beb99bb280 2024-07-04T08:15:54.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.078+0000 7f2ffc682740 10 client.25591 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:54.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.078+0000 7f2ffc682740 10 client.25591 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-04T08:15:54.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.078+0000 7f2ffc682740 20 client.25591 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:54.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.078+0000 7f2ffc682740 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55beb999a8e0 0x55beb9d9e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.078+0000 7f2ffc682740 4 client.25591 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:15:54.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.078+0000 7f2ffc682740 1 -- 172.21.15.39:0/4236583632 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beb9cf8bd0 con 0x55beb999a8e0 2024-07-04T08:15:54.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.078+0000 7f2fefe00640 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55beb999a8e0 0x55beb9d9e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.079+0000 7f2fefe00640 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55beb999a8e0 0x55beb9d9e920 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62225a5f722e695d server_cookie=2271dea639d9f49d in_seq=0 out_seq=0 2024-07-04T08:15:54.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.079+0000 7f2fd9600640 10 client.25591 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:15:54.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2fd9600640 1 -- 172.21.15.39:0/4236583632 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55beb9cf8bd0 con 0x55beb999a8e0 2024-07-04T08:15:54.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2fd9600640 10 client.25591 handle_command_reply: tid=0 2024-07-04T08:15:54.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2ffc682740 1 client.25591 shutdown 2024-07-04T08:15:54.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2ffc682740 1 -- 172.21.15.39:0/4236583632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 msgr2=0x55beb9d8c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2ffc682740 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d8c6f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009fd0 tx=0x7f2fdc012b60 comp rx=0 tx=0).stop 2024-07-04T08:15:54.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2ffc682740 1 -- 172.21.15.39:0/4236583632 shutdown_connections 2024-07-04T08:15:54.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2ffc682740 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55beb999a8e0 0x55beb9d9e920 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2ffc682740 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 0x55beb9d98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2ffc682740 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d8c6f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2ffc682740 1 --2- 172.21.15.39:0/4236583632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d8c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2ffc682740 1 -- 172.21.15.39:0/4236583632 >> 172.21.15.39:0/4236583632 conn(0x55beb99d0350 msgr2=0x55beb9d837e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.080+0000 7f2fd9600640 0 client.25591 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:15:54.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.081+0000 7f2ffc682740 1 -- 172.21.15.39:0/4236583632 shutdown_connections 2024-07-04T08:15:54.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.081+0000 7f2ffc682740 1 -- 172.21.15.39:0/4236583632 wait complete. 2024-07-04T08:15:54.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.081+0000 7f2ffc682740 20 client.25591 trim_cache size 0 max 16384 2024-07-04T08:15:54.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.103+0000 7f2ffc682740 1 Processor -- start 2024-07-04T08:15:54.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.103+0000 7f2ffc682740 1 -- start start 2024-07-04T08:15:54.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.103+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99b30a0 0x55beb9d9d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.103+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d86770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.103+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99dbf60 0x55beb9d98090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.103+0000 7f2ffc682740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55beb9d9cc60 con 0x55beb99b30a0 2024-07-04T08:15:54.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.103+0000 7f2ffc682740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55beb9d9d330 con 0x55beb99dbf60 2024-07-04T08:15:54.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.103+0000 7f2ffc682740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55beb9d9da00 con 0x55beb99bb280 2024-07-04T08:15:54.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2fefe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99b30a0 0x55beb9d9d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2fefe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99b30a0 0x55beb9d9d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45654/0 (socket says 172.21.15.39:45654) 2024-07-04T08:15:54.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2fefe00640 1 -- 172.21.15.39:0/421551505 learned_addr learned my addr 172.21.15.39:0/421551505 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:54.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2feea00640 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d86770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2fefe00640 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99dbf60 0x55beb9d98090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2feea00640 1 -- 172.21.15.39:0/421551505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99dbf60 msgr2=0x55beb9d98090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2feea00640 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99dbf60 0x55beb9d98090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2feea00640 1 -- 172.21.15.39:0/421551505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99b30a0 msgr2=0x55beb9d9d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2feea00640 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99b30a0 0x55beb9d9d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2feea00640 1 -- 172.21.15.39:0/421551505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb9d834b0 con 0x55beb99bb280 2024-07-04T08:15:54.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.104+0000 7f2fefe00640 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99b30a0 0x55beb9d9d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:54.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.105+0000 7f2feea00640 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d86770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc01c960 tx=0x7f2fdc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e4ce1eb79d6277d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.105+0000 7f2fdbe00640 1 -- 172.21.15.39:0/421551505 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00f920 con 0x55beb99bb280 2024-07-04T08:15:54.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.105+0000 7f2fdbe00640 1 -- 172.21.15.39:0/421551505 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fdc00fac0 con 0x55beb99bb280 2024-07-04T08:15:54.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.105+0000 7f2fdbe00640 1 -- 172.21.15.39:0/421551505 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc01c680 con 0x55beb99bb280 2024-07-04T08:15:54.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.106+0000 7f2ffc682740 1 -- 172.21.15.39:0/421551505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 msgr2=0x55beb9d86770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.106+0000 7f2ffc682740 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d86770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc01c960 tx=0x7f2fdc00f790 comp rx=0 tx=0).stop 2024-07-04T08:15:54.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.106+0000 7f2ffc682740 1 -- 172.21.15.39:0/421551505 shutdown_connections 2024-07-04T08:15:54.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.106+0000 7f2ffc682740 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99dbf60 0x55beb9d98090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.106+0000 7f2ffc682740 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9d86770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.106+0000 7f2ffc682740 1 --2- 172.21.15.39:0/421551505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99b30a0 0x55beb9d9d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.106+0000 7f2ffc682740 1 -- 172.21.15.39:0/421551505 >> 172.21.15.39:0/421551505 conn(0x55beb99d0350 msgr2=0x55beb9d870f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.107+0000 7f2ffc682740 1 -- 172.21.15.39:0/421551505 shutdown_connections 2024-07-04T08:15:54.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.107+0000 7f2ffc682740 1 -- 172.21.15.39:0/421551505 wait complete. 2024-07-04T08:15:54.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.107+0000 7f2ffc682740 1 -- start start 2024-07-04T08:15:54.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.107+0000 7f2ffc682740 1 Processor -- start 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.108+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d9fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.108+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9da0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.108+0000 7f2ffc682740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 0x55beb9e05bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.108+0000 7f2ffc682740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55beb9d9e7a0 con 0x55beb99dbf60 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.108+0000 7f2ffc682740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55beb9d9e0d0 con 0x55beb99b30a0 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.108+0000 7f2ffc682740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55beb9d9ee70 con 0x55beb99bb280 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.108+0000 7f2ffc682740 10 client.0 _ll_register_callbacks cb 0x7f2fdc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2fefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d9fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2fefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d9fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59016/0 (socket says 172.21.15.39:59016) 2024-07-04T08:15:54.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2fefe00640 1 -- 172.21.15.39:0/3608326085 learned_addr learned my addr 172.21.15.39:0/3608326085 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:54.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2fefe00640 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 0x55beb9e05bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2feea00640 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9da0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2fefe00640 1 -- 172.21.15.39:0/3608326085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 msgr2=0x55beb9da0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2fefe00640 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9da0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2fefe00640 1 -- 172.21.15.39:0/3608326085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 msgr2=0x55beb9e05bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:54.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2fefe00640 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 0x55beb9e05bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.109+0000 7f2fefe00640 1 -- 172.21.15.39:0/3608326085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb9da0d00 con 0x55beb99b30a0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.110+0000 7f2fefe00640 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d9fed0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8004af0 tx=0x7f2fe8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0eeae4a683195c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.110+0000 7f2fd9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.110+0000 7f2fd9600640 1 -- 172.21.15.39:0/3608326085 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8063060 con 0x55beb99b30a0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.110+0000 7f2fd9600640 1 -- 172.21.15.39:0/3608326085 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2fe802f870 con 0x55beb99b30a0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.110+0000 7f2ffc682740 1 -- 172.21.15.39:0/3608326085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fdc00fac0 con 0x55beb99b30a0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.110+0000 7f2fd9600640 1 -- 172.21.15.39:0/3608326085 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2fe8003e40 con 0x55beb99b30a0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.110+0000 7f2ffc682740 10 client.25616 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.110+0000 7f2ffc682740 1 -- 172.21.15.39:0/3608326085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beb9e0f9b0 con 0x55beb99b30a0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.111+0000 7f2fd9600640 1 -- 172.21.15.39:0/3608326085 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fdc00fac0 con 0x55beb99b30a0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.111+0000 7f2fd9600640 1 -- 172.21.15.39:0/3608326085 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2fe804f040 con 0x55beb99b30a0 2024-07-04T08:15:54.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.111+0000 7f2ffc682740 10 client.25616 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:54.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.111+0000 7f2ffc682740 10 client.25616 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-04T08:15:54.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.111+0000 7f2ffc682740 20 client.25616 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:54.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.111+0000 7f2ffc682740 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55beb996fc90 0x55beb9e12d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:54.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.111+0000 7f2ffc682740 4 client.25616 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:15:54.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.111+0000 7f2ffc682740 1 -- 172.21.15.39:0/3608326085 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "status"}) -- 0x55beb9cf4670 con 0x55beb996fc90 2024-07-04T08:15:54.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.112+0000 7f2feea00640 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55beb996fc90 0x55beb9e12d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:54.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2feea00640 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55beb996fc90 0x55beb9e12d90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c7d799a9180edb0 server_cookie=5785c3d93fe449bf in_seq=0 out_seq=0 2024-07-04T08:15:54.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2fd9600640 10 client.25616 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2fd9600640 1 -- 172.21.15.39:0/3608326085 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55beb9cf4670 con 0x55beb996fc90 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2fd9600640 10 client.25616 handle_command_reply: tid=0 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2ffc682740 1 client.25616 shutdown 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2ffc682740 1 -- 172.21.15.39:0/3608326085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 msgr2=0x55beb9d9fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2ffc682740 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d9fed0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8004af0 tx=0x7f2fe8005fd0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2ffc682740 1 -- 172.21.15.39:0/3608326085 shutdown_connections 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2ffc682740 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55beb996fc90 0x55beb9e12d90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2ffc682740 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55beb99dbf60 0x55beb9e05bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2ffc682740 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55beb99bb280 0x55beb9da0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2ffc682740 1 --2- 172.21.15.39:0/3608326085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beb99b30a0 0x55beb9d9fed0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2ffc682740 1 -- 172.21.15.39:0/3608326085 >> 172.21.15.39:0/3608326085 conn(0x55beb99d0350 msgr2=0x55beb9d7de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.113+0000 7f2fd9600640 0 client.25616 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.114+0000 7f2ffc682740 1 -- 172.21.15.39:0/3608326085 shutdown_connections 2024-07-04T08:15:54.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.114+0000 7f2ffc682740 1 -- 172.21.15.39:0/3608326085 wait complete. 2024-07-04T08:15:54.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.114+0000 7f2ffc682740 20 client.25616 trim_cache size 0 max 16384 2024-07-04T08:15:54.117 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:15:54.117 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:15:54.117 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 3, 2024-07-04T08:15:54.117 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24674, 2024-07-04T08:15:54.117 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:15:54.117 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:15:54.117 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:15:54.117 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 290.30422794700002, 2024-07-04T08:15:54.118 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 30, 2024-07-04T08:15:54.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:15:54.118 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 92, 2024-07-04T08:15:54.118 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 406.60111080500002 2024-07-04T08:15:54.118 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:15:54.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 -- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2fd40b2950 msgr2=0x7f2fd40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2fd40b2950 0x7f2fd40b4e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00a9ba0 tx=0x7f2fe00e66b0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 -- 172.21.15.39:0/1101000755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff0101b20 msgr2=0x7f2ff019bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff0101b20 0x7f2ff019bad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000e9f0 tx=0x7f2fe000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 -- 172.21.15.39:0/1101000755 shutdown_connections 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2fd40b2950 0x7f2fd40b4e60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff010f6d0 0x7f2ff01a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ff0102470 0x7f2ff019c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 --2- 172.21.15.39:0/1101000755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2ff0101b20 0x7f2ff019bad0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 -- 172.21.15.39:0/1101000755 >> 172.21.15.39:0/1101000755 conn(0x7f2ff0078c80 msgr2=0x7f2ff01091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 -- 172.21.15.39:0/1101000755 shutdown_connections 2024-07-04T08:15:54.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:54.115+0000 7f2ff6000640 1 -- 172.21.15.39:0/1101000755 wait complete. 2024-07-04T08:15:55.297 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:55 smithi039 ceph-mon[32062]: pgmap v501: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-07-04T08:15:55.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:55 smithi133 ceph-mon[37050]: pgmap v501: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-07-04T08:15:55.381 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.i status 2024-07-04T08:15:55.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:55 smithi050 ceph-mon[36577]: pgmap v501: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-07-04T08:15:55.676 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:15:55.677 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:15:55.874 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:57.302 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:57 smithi039 ceph-mon[32062]: pgmap v502: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:15:57.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:57 smithi133 ceph-mon[37050]: pgmap v502: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:15:57.323 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:15:57.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:57 smithi050 ceph-mon[36577]: pgmap v502: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:15:58.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.634+0000 7fa34a600640 1 -- 172.21.15.39:0/2170986716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 msgr2=0x7fa344108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:58.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.634+0000 7fa34a600640 1 --2- 172.21.15.39:0/2170986716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 0x7fa344108c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa330009900 tx=0x7fa33002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.634+0000 7fa34a600640 1 -- 172.21.15.39:0/2170986716 shutdown_connections 2024-07-04T08:15:58.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.634+0000 7fa34a600640 1 --2- 172.21.15.39:0/2170986716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 0x7fa344108c30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.634+0000 7fa34a600640 1 --2- 172.21.15.39:0/2170986716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344109170 0x7fa34410f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.634+0000 7fa34a600640 1 --2- 172.21.15.39:0/2170986716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa344102ee0 0x7fa3441032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.634+0000 7fa34a600640 1 -- 172.21.15.39:0/2170986716 >> 172.21.15.39:0/2170986716 conn(0x7fa3440fe930 msgr2=0x7fa344100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:58.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.634+0000 7fa34a600640 1 -- 172.21.15.39:0/2170986716 shutdown_connections 2024-07-04T08:15:58.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.634+0000 7fa34a600640 1 -- 172.21.15.39:0/2170986716 wait complete. 2024-07-04T08:15:58.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.636+0000 7fa34a600640 1 Processor -- start 2024-07-04T08:15:58.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.636+0000 7fa34a600640 1 -- start start 2024-07-04T08:15:58.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa34a600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102ee0 0x7fa344199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa34a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 0x7fa344199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa34a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa344109170 0x7fa34419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa34a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa344112e30 con 0x7fa344108820 2024-07-04T08:15:58.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa34a600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa344112cb0 con 0x7fa344102ee0 2024-07-04T08:15:58.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa34a600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa344112fb0 con 0x7fa344109170 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa343e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 0x7fa344199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa343e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 0x7fa344199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45674/0 (socket says 172.21.15.39:45674) 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa343e00640 1 -- 172.21.15.39:0/1578571951 learned_addr learned my addr 172.21.15.39:0/1578571951 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa349c00640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa344109170 0x7fa34419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.637+0000 7fa349200640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102ee0 0x7fa344199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.638+0000 7fa343e00640 1 -- 172.21.15.39:0/1578571951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa344109170 msgr2=0x7fa34419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.638+0000 7fa343e00640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa344109170 0x7fa34419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.638+0000 7fa343e00640 1 -- 172.21.15.39:0/1578571951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102ee0 msgr2=0x7fa344199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.638+0000 7fa343e00640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102ee0 0x7fa344199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.638+0000 7fa343e00640 1 -- 172.21.15.39:0/1578571951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34419f790 con 0x7fa344108820 2024-07-04T08:15:58.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.638+0000 7fa349c00640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa344109170 0x7fa34419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:58.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.638+0000 7fa343e00640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 0x7fa344199dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa33002a910 tx=0x7fa330031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:58.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.639+0000 7fa341600640 1 -- 172.21.15.39:0/1578571951 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33003f030 con 0x7fa344108820 2024-07-04T08:15:58.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.639+0000 7fa34a600640 1 -- 172.21.15.39:0/1578571951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34419fa20 con 0x7fa344108820 2024-07-04T08:15:58.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.639+0000 7fa341600640 1 -- 172.21.15.39:0/1578571951 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3300043d0 con 0x7fa344108820 2024-07-04T08:15:58.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.639+0000 7fa341600640 1 -- 172.21.15.39:0/1578571951 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330035070 con 0x7fa344108820 2024-07-04T08:15:58.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.639+0000 7fa34a600640 1 -- 172.21.15.39:0/1578571951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3441a61e0 con 0x7fa344108820 2024-07-04T08:15:58.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.642+0000 7fa352222740 1 Processor -- start 2024-07-04T08:15:58.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.642+0000 7fa352222740 1 -- start start 2024-07-04T08:15:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa341600640 1 -- 172.21.15.39:0/1578571951 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7fa330042030 con 0x7fa344108820 2024-07-04T08:15:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa352222740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08764850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa352222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08764d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa352222740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac087652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa352222740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ac086daca0 con 0x55ac083bff60 2024-07-04T08:15:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa352222740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac086ca240 con 0x55ac0839f280 2024-07-04T08:15:58.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa352222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac086ce280 con 0x55ac083970a0 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa349c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08764850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa349c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08764850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45680/0 (socket says 172.21.15.39:45680) 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa349c00640 1 -- 172.21.15.39:0/3711992362 learned_addr learned my addr 172.21.15.39:0/3711992362 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.643+0000 7fa349200640 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08764d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.644+0000 7fa343e00640 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac087652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.644+0000 7fa349c00640 1 -- 172.21.15.39:0/3711992362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 msgr2=0x55ac08764d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.644+0000 7fa349c00640 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08764d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.644+0000 7fa349c00640 1 -- 172.21.15.39:0/3711992362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 msgr2=0x55ac087652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.644+0000 7fa349c00640 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac087652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.644+0000 7fa349c00640 1 -- 172.21.15.39:0/3711992362 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac08662550 con 0x55ac083bff60 2024-07-04T08:15:58.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.644+0000 7fa349c00640 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08764850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa33c011810 tx=0x7fa33c011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b9758202bf40a1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.644+0000 7fa343e00640 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac087652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:15:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.644+0000 7fa349200640 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08764d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.645+0000 7fa336000640 1 -- 172.21.15.39:0/3711992362 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c011310 con 0x55ac083bff60 2024-07-04T08:15:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.645+0000 7fa336000640 1 -- 172.21.15.39:0/3711992362 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c00f9a0 con 0x55ac083bff60 2024-07-04T08:15:58.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.645+0000 7fa336000640 1 -- 172.21.15.39:0/3711992362 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c005650 con 0x55ac083bff60 2024-07-04T08:15:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.645+0000 7fa341600640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa32c0b2950 0x7fa32c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa341600640 1 -- 172.21.15.39:0/1578571951 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa330102b20 con 0x7fa344108820 2024-07-04T08:15:58.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa349200640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa32c0b2950 0x7fa32c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa352222740 1 -- 172.21.15.39:0/3711992362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 msgr2=0x55ac08764850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa352222740 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08764850 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa33c011810 tx=0x7fa33c011cd0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa352222740 1 -- 172.21.15.39:0/3711992362 shutdown_connections 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa352222740 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac087652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa352222740 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08764d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa352222740 1 --2- 172.21.15.39:0/3711992362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08764850 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa352222740 1 -- 172.21.15.39:0/3711992362 >> 172.21.15.39:0/3711992362 conn(0x55ac083b4350 msgr2=0x55ac08762740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa352222740 1 -- 172.21.15.39:0/3711992362 shutdown_connections 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.646+0000 7fa352222740 1 -- 172.21.15.39:0/3711992362 wait complete. 2024-07-04T08:15:58.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.647+0000 7fa349200640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa32c0b2950 0x7fa32c0b4e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55ac087634d0 tx=0x7fa338005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:58.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.647+0000 7fa352222740 1 -- start start 2024-07-04T08:15:58.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.647+0000 7fa352222740 1 Processor -- start 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.648+0000 7fa352222740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 0x55ac08770420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.648+0000 7fa352222740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08770960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.648+0000 7fa352222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083bff60 0x55ac0877c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.648+0000 7fa352222740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ac0852e5f0 con 0x55ac083970a0 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.648+0000 7fa352222740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac08671970 con 0x55ac0839f280 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.648+0000 7fa352222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac08773030 con 0x55ac083bff60 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.648+0000 7fa352222740 10 client.0 _ll_register_callbacks cb 0x55ac086c34a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa349200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 0x55ac08770420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa343e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08770960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa343e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08770960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59056/0 (socket says 172.21.15.39:59056) 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa343e00640 1 -- 172.21.15.39:0/1400355042 learned_addr learned my addr 172.21.15.39:0/1400355042 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:58.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa349200640 1 -- 172.21.15.39:0/1400355042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083bff60 msgr2=0x55ac0877c350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:15:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa349200640 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083bff60 0x55ac0877c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa349200640 1 -- 172.21.15.39:0/1400355042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 msgr2=0x55ac08770960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa349200640 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08770960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa349200640 1 -- 172.21.15.39:0/1400355042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0842b540 con 0x55ac083970a0 2024-07-04T08:15:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.649+0000 7fa343e00640 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08770960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:15:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.650+0000 7fa349200640 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 0x55ac08770420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa3380045f0 tx=0x7fa338004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5130081b8c09c453 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:58.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.650+0000 7fa32aa00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:15:58.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.650+0000 7fa32aa00640 1 -- 172.21.15.39:0/1400355042 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33801f640 con 0x55ac083970a0 2024-07-04T08:15:58.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.650+0000 7fa32aa00640 1 -- 172.21.15.39:0/1400355042 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa33801f7e0 con 0x55ac083970a0 2024-07-04T08:15:58.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.650+0000 7fa352222740 1 -- 172.21.15.39:0/1400355042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa33c00f9a0 con 0x55ac083970a0 2024-07-04T08:15:58.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.650+0000 7fa32aa00640 1 -- 172.21.15.39:0/1400355042 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa33801f9e0 con 0x55ac083970a0 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.650+0000 7fa352222740 10 client.15513 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.650+0000 7fa352222740 1 -- 172.21.15.39:0/1400355042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac0877f080 con 0x55ac083970a0 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.651+0000 7fa32aa00640 1 -- 172.21.15.39:0/1400355042 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c00f9a0 con 0x55ac083970a0 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.652+0000 7fa32aa00640 1 -- 172.21.15.39:0/1400355042 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa338027070 con 0x55ac083970a0 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.652+0000 7fa352222740 10 client.15513 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.652+0000 7fa352222740 10 client.15513 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.652+0000 7fa352222740 20 client.15513 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.652+0000 7fa352222740 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ac0837e8e0 0x55ac08782ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.652+0000 7fa352222740 4 client.15513 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:15:58.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.652+0000 7fa352222740 1 -- 172.21.15.39:0/1400355042 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac086dcbf0 con 0x55ac0837e8e0 2024-07-04T08:15:58.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.653+0000 7fa343e00640 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ac0837e8e0 0x55ac08782ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.654+0000 7fa343e00640 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ac0837e8e0 0x55ac08782ba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b3257d99f825091 server_cookie=df96e31f57da3b71 in_seq=0 out_seq=0 2024-07-04T08:15:58.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.654+0000 7fa32aa00640 10 client.15513 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:15:58.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.654+0000 7fa32aa00640 1 -- 172.21.15.39:0/1400355042 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ac086dcbf0 con 0x55ac0837e8e0 2024-07-04T08:15:58.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.654+0000 7fa32aa00640 10 client.15513 handle_command_reply: tid=0 2024-07-04T08:15:58.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.654+0000 7fa352222740 1 client.15513 shutdown 2024-07-04T08:15:58.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 -- 172.21.15.39:0/1400355042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 msgr2=0x55ac08770420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:58.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 0x55ac08770420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa3380045f0 tx=0x7fa338004730 comp rx=0 tx=0).stop 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 -- 172.21.15.39:0/1400355042 shutdown_connections 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ac0837e8e0 0x55ac08782ba0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083bff60 0x55ac0877c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08770960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 --2- 172.21.15.39:0/1400355042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 0x55ac08770420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa32aa00640 0 client.15513 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 -- 172.21.15.39:0/1400355042 >> 172.21.15.39:0/1400355042 conn(0x55ac083b4350 msgr2=0x55ac08767a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 -- 172.21.15.39:0/1400355042 shutdown_connections 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.655+0000 7fa352222740 1 -- 172.21.15.39:0/1400355042 wait complete. 2024-07-04T08:15:58.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.656+0000 7fa352222740 20 client.15513 trim_cache size 0 max 16384 2024-07-04T08:15:58.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.686+0000 7fa352222740 1 Processor -- start 2024-07-04T08:15:58.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.686+0000 7fa352222740 1 -- start start 2024-07-04T08:15:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.686+0000 7fa352222740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 0x55ac0877c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.686+0000 7fa352222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac0839f280 0x55ac0878b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.686+0000 7fa352222740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac083bff60 0x55ac08761d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.686+0000 7fa352222740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ac08780ee0 con 0x55ac083970a0 2024-07-04T08:15:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.686+0000 7fa352222740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac087815b0 con 0x55ac083bff60 2024-07-04T08:15:58.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.686+0000 7fa352222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac08781c80 con 0x55ac0839f280 2024-07-04T08:15:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa349200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 0x55ac0877c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa343e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac0839f280 0x55ac0878b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa343e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac0839f280 0x55ac0878b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60884/0 (socket says 172.21.15.39:60884) 2024-07-04T08:15:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa343e00640 1 -- 172.21.15.39:0/2928672616 learned_addr learned my addr 172.21.15.39:0/2928672616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa349c00640 1 --2- 172.21.15.39:0/2928672616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac083bff60 0x55ac08761d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa343e00640 1 -- 172.21.15.39:0/2928672616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac083bff60 msgr2=0x55ac08761d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa343e00640 1 --2- 172.21.15.39:0/2928672616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac083bff60 0x55ac08761d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa343e00640 1 -- 172.21.15.39:0/2928672616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 msgr2=0x55ac0877c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa343e00640 1 --2- 172.21.15.39:0/2928672616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 0x55ac0877c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.687+0000 7fa343e00640 1 -- 172.21.15.39:0/2928672616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac08766d70 con 0x55ac0839f280 2024-07-04T08:15:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.688+0000 7fa343e00640 1 --2- 172.21.15.39:0/2928672616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac0839f280 0x55ac0878b270 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa3300ec010 tx=0x7fa330104f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eb82e5180f375ff server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.688+0000 7fa335600640 1 -- 172.21.15.39:0/2928672616 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3300cae60 con 0x55ac0839f280 2024-07-04T08:15:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.688+0000 7fa335600640 1 -- 172.21.15.39:0/2928672616 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3300c9640 con 0x55ac0839f280 2024-07-04T08:15:58.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.688+0000 7fa335600640 1 -- 172.21.15.39:0/2928672616 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330108070 con 0x55ac0839f280 2024-07-04T08:15:58.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.689+0000 7fa352222740 1 -- 172.21.15.39:0/2928672616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac0839f280 msgr2=0x55ac0878b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.689+0000 7fa352222740 1 --2- 172.21.15.39:0/2928672616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac0839f280 0x55ac0878b270 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa3300ec010 tx=0x7fa330104f70 comp rx=0 tx=0).stop 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.689+0000 7fa352222740 1 -- 172.21.15.39:0/2928672616 shutdown_connections 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.689+0000 7fa352222740 1 --2- 172.21.15.39:0/2928672616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac083bff60 0x55ac08761d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.689+0000 7fa352222740 1 --2- 172.21.15.39:0/2928672616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac0839f280 0x55ac0878b270 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.689+0000 7fa352222740 1 --2- 172.21.15.39:0/2928672616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083970a0 0x55ac0877c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.689+0000 7fa352222740 1 -- 172.21.15.39:0/2928672616 >> 172.21.15.39:0/2928672616 conn(0x55ac083b4350 msgr2=0x55ac0876eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.689+0000 7fa352222740 1 -- 172.21.15.39:0/2928672616 shutdown_connections 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.689+0000 7fa352222740 1 -- 172.21.15.39:0/2928672616 wait complete. 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.690+0000 7fa352222740 1 -- start start 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.690+0000 7fa352222740 1 Processor -- start 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.690+0000 7fa352222740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08785610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa352222740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08761990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa352222740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08761ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa352222740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ac08782a20 con 0x55ac083bff60 2024-07-04T08:15:58.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa352222740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac08782350 con 0x55ac0839f280 2024-07-04T08:15:58.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa352222740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac087830f0 con 0x55ac083970a0 2024-07-04T08:15:58.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa352222740 10 client.0 _ll_register_callbacks cb 0x7fa338009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:15:58.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa349200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08785610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa349200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08785610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60890/0 (socket says 172.21.15.39:60890) 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa349200640 1 -- 172.21.15.39:0/2753870742 learned_addr learned my addr 172.21.15.39:0/2753870742 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa343e00640 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08761990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa349c00640 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08761ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa349200640 1 -- 172.21.15.39:0/2753870742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 msgr2=0x55ac08761990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa349200640 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08761990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa349200640 1 -- 172.21.15.39:0/2753870742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 msgr2=0x55ac08761ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa349200640 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08761ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.691+0000 7fa349200640 1 -- 172.21.15.39:0/2753870742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac087625c0 con 0x55ac083970a0 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.692+0000 7fa349200640 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08785610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa33800a010 tx=0x7fa33801d720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0820da5129622f0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.692+0000 7fa32aa00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:15:58.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.692+0000 7fa32aa00640 1 -- 172.21.15.39:0/2753870742 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33803b070 con 0x55ac083970a0 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.692+0000 7fa352222740 1 -- 172.21.15.39:0/2753870742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3300c9640 con 0x55ac083970a0 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.692+0000 7fa32aa00640 1 -- 172.21.15.39:0/2753870742 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa33801d8c0 con 0x55ac083970a0 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.692+0000 7fa32aa00640 1 -- 172.21.15.39:0/2753870742 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa33801dac0 con 0x55ac083970a0 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.692+0000 7fa352222740 10 client.25612 fetch_fsmap learned FSMap version 30 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.693+0000 7fa352222740 1 -- 172.21.15.39:0/2753870742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac087efd40 con 0x55ac083970a0 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.693+0000 7fa32aa00640 1 -- 172.21.15.39:0/2753870742 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3300c9640 con 0x55ac083970a0 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.693+0000 7fa32aa00640 1 -- 172.21.15.39:0/2753870742 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa338027040 con 0x55ac083970a0 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.693+0000 7fa352222740 10 client.25612 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.693+0000 7fa352222740 10 client.25612 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.693+0000 7fa352222740 20 client.25612 populate_metadata read hostname 'smithi039' 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.693+0000 7fa352222740 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ac08353c90 0x55ac087f3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:15:58.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.693+0000 7fa352222740 4 client.25612 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:15:58.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.693+0000 7fa352222740 1 -- 172.21.15.39:0/2753870742 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "status"}) -- 0x55ac086d8690 con 0x55ac08353c90 2024-07-04T08:15:58.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.694+0000 7fa343e00640 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ac08353c90 0x55ac087f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.694+0000 7fa343e00640 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ac08353c90 0x55ac087f3680 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=354b151d5382ecfe server_cookie=df8c91ec8d51eb4e in_seq=0 out_seq=0 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.694+0000 7fa32aa00640 10 client.25612 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.694+0000 7fa32aa00640 1 -- 172.21.15.39:0/2753870742 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ac086d8690 con 0x55ac08353c90 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.694+0000 7fa32aa00640 10 client.25612 handle_command_reply: tid=0 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.694+0000 7fa352222740 1 client.25612 shutdown 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 -- 172.21.15.39:0/2753870742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 msgr2=0x55ac08785610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08785610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa33800a010 tx=0x7fa33801d720 comp rx=0 tx=0).stop 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 -- 172.21.15.39:0/2753870742 shutdown_connections 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ac08353c90 0x55ac087f3680 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac083bff60 0x55ac08761ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac0839f280 0x55ac08761990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 --2- 172.21.15.39:0/2753870742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac083970a0 0x55ac08785610 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 -- 172.21.15.39:0/2753870742 >> 172.21.15.39:0/2753870742 conn(0x55ac083b4350 msgr2=0x55ac08770400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:58.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa32aa00640 0 client.25612 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:15:58.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 -- 172.21.15.39:0/2753870742 shutdown_connections 2024-07-04T08:15:58.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 1 -- 172.21.15.39:0/2753870742 wait complete. 2024-07-04T08:15:58.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.695+0000 7fa352222740 20 client.25612 trim_cache size 0 max 16384 2024-07-04T08:15:58.699 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:15:58.699 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 2, 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24679, 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 295.91629335099998, 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 30, 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 92, 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 408.51020269200001 2024-07-04T08:15:58.700 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 -- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa32c0b2950 msgr2=0x7fa32c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa32c0b2950 0x7fa32c0b4e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55ac087634d0 tx=0x7fa338005f90 comp rx=0 tx=0).stop 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 -- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 msgr2=0x7fa344199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 0x7fa344199dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa33002a910 tx=0x7fa330031a20 comp rx=0 tx=0).stop 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 -- 172.21.15.39:0/1578571951 shutdown_connections 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa32c0b2950 0x7fa32c0b4e60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa344108820 0x7fa344199dc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa344109170 0x7fa34419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 --2- 172.21.15.39:0/1578571951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa344102ee0 0x7fa344199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 -- 172.21.15.39:0/1578571951 >> 172.21.15.39:0/1578571951 conn(0x7fa3440fe930 msgr2=0x7fa344100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 -- 172.21.15.39:0/1578571951 shutdown_connections 2024-07-04T08:15:58.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:15:58.696+0000 7fa34a600640 1 -- 172.21.15.39:0/1578571951 wait complete. 2024-07-04T08:15:59.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:15:59 smithi133 ceph-mon[37050]: pgmap v503: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-04T08:15:59.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:15:59 smithi039 ceph-mon[32062]: pgmap v503: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-04T08:15:59.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:15:59 smithi050 ceph-mon[36577]: pgmap v503: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-04T08:16:00.018 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.a status 2024-07-04T08:16:00.508 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:16:00 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:16:00.242+0000 7fdf8be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:16:00.508 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:16:00 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:16:00.242+0000 7fdf8be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:16:00.508 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:16:00 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:16:00.242+0000 7fdf8be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:16:00.508 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:16:00 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:16:00.242+0000 7fdf8be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:16:00.508 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:16:00 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:16:00.242+0000 7fdf8be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-04T08:16:00.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.605+0000 7f24f3e00640 1 -- 172.21.15.39:0/697531415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f410f4c0 msgr2=0x7f24f41118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:00.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.605+0000 7f24f3e00640 1 --2- 172.21.15.39:0/697531415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f410f4c0 0x7f24f41118e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009be0 tx=0x7f24ec02a6c0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.605+0000 7f24f3e00640 1 -- 172.21.15.39:0/697531415 shutdown_connections 2024-07-04T08:16:00.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.605+0000 7f24f3e00640 1 --2- 172.21.15.39:0/697531415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f410f4c0 0x7f24f41118e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.605+0000 7f24f3e00640 1 --2- 172.21.15.39:0/697531415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f40755f0 0x7f24f410ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.605+0000 7f24f3e00640 1 --2- 172.21.15.39:0/697531415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f4074ca0 0x7f24f40750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.605+0000 7f24f3e00640 1 -- 172.21.15.39:0/697531415 >> 172.21.15.39:0/697531415 conn(0x7f24f40fe950 msgr2=0x7f24f4100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:00.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.605+0000 7f24f3e00640 1 -- 172.21.15.39:0/697531415 shutdown_connections 2024-07-04T08:16:00.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.605+0000 7f24f3e00640 1 -- 172.21.15.39:0/697531415 wait complete. 2024-07-04T08:16:00.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.607+0000 7f24f3e00640 1 Processor -- start 2024-07-04T08:16:00.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.607+0000 7f24f3e00640 1 -- start start 2024-07-04T08:16:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.607+0000 7f24f3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f4074ca0 0x7f24f419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.607+0000 7f24f3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f40755f0 0x7f24f419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.607+0000 7f24f3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f410f4c0 0x7f24f41a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.608+0000 7f24f3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4115020 con 0x7f24f410f4c0 2024-07-04T08:16:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.608+0000 7f24f3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4114ea0 con 0x7f24f40755f0 2024-07-04T08:16:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.608+0000 7f24f3e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24f41151a0 con 0x7f24f4074ca0 2024-07-04T08:16:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.608+0000 7f24f2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f4074ca0 0x7f24f419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.608+0000 7f24f2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f4074ca0 0x7f24f419b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60926/0 (socket says 172.21.15.39:60926) 2024-07-04T08:16:00.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.608+0000 7f24f2a00640 1 -- 172.21.15.39:0/2755211174 learned_addr learned my addr 172.21.15.39:0/2755211174 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:00.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.608+0000 7f24f2000640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f40755f0 0x7f24f419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:00.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.608+0000 7f24f3400640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f410f4c0 0x7f24f41a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:00.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.609+0000 7f24f2a00640 1 -- 172.21.15.39:0/2755211174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f40755f0 msgr2=0x7f24f419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:00.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.609+0000 7f24f2a00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f40755f0 0x7f24f419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.609+0000 7f24f2a00640 1 -- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f410f4c0 msgr2=0x7f24f41a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:00.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.609+0000 7f24f2a00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f410f4c0 0x7f24f41a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.609+0000 7f24f2a00640 1 -- 172.21.15.39:0/2755211174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f41a18e0 con 0x7f24f4074ca0 2024-07-04T08:16:00.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.609+0000 7f24f2a00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f4074ca0 0x7f24f419b970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e9c0 tx=0x7f24e400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:00.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.610+0000 7f24f8a00640 1 -- 172.21.15.39:0/2755211174 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e400ccc0 con 0x7f24f4074ca0 2024-07-04T08:16:00.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.610+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f41a1bd0 con 0x7f24f4074ca0 2024-07-04T08:16:00.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.610+0000 7f24f8a00640 1 -- 172.21.15.39:0/2755211174 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e40054d0 con 0x7f24f4074ca0 2024-07-04T08:16:00.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.610+0000 7f24f8a00640 1 -- 172.21.15.39:0/2755211174 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4010430 con 0x7f24f4074ca0 2024-07-04T08:16:00.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.610+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f41a8330 con 0x7f24f4074ca0 2024-07-04T08:16:00.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.612+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f40750b0 con 0x7f24f4074ca0 2024-07-04T08:16:00.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.614+0000 7f24f8a00640 1 -- 172.21.15.39:0/2755211174 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 24) ==== 149726+0+0 (secure 0 0 0) 0x7f24e4004f80 con 0x7f24f4074ca0 2024-07-04T08:16:00.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.616+0000 7f24f8a00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24d80b2950 0x7f24d80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.617+0000 7f24f8a00640 1 -- 172.21.15.39:0/2755211174 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f24e40e2be0 con 0x7f24f4074ca0 2024-07-04T08:16:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.617+0000 7f24f2000640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24d80b2950 0x7f24d80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:00.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.618+0000 7f24f2000640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24d80b2950 0x7f24d80b4e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24f4111920 tx=0x7f24e0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:00.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.620+0000 7f24f8a00640 1 -- 172.21.15.39:0/2755211174 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f24e40aa2f0 con 0x7f24f4074ca0 2024-07-04T08:16:00.638 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:00.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.884+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24f419caa0 con 0x7f24f4074ca0 2024-07-04T08:16:00.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.885+0000 7f24f8a00640 1 -- 172.21.15.39:0/2755211174 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f24e40a9bf0 con 0x7f24f4074ca0 2024-07-04T08:16:00.887 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:16:00.887 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:16:00.888 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:16:00.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.888+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24d80b2950 msgr2=0x7f24d80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:00.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.888+0000 7f24f3e00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24d80b2950 0x7f24d80b4e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24f4111920 tx=0x7f24e0005f90 comp rx=0 tx=0).stop 2024-07-04T08:16:00.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.888+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f4074ca0 msgr2=0x7f24f419b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:00.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.888+0000 7f24f3e00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f4074ca0 0x7f24f419b970 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24e400e9c0 tx=0x7f24e400ee80 comp rx=0 tx=0).stop 2024-07-04T08:16:00.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.888+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 shutdown_connections 2024-07-04T08:16:00.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.889+0000 7f24f3e00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24d80b2950 0x7f24d80b4e60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.889+0000 7f24f3e00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f410f4c0 0x7f24f41a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.889+0000 7f24f3e00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f40755f0 0x7f24f419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.889+0000 7f24f3e00640 1 --2- 172.21.15.39:0/2755211174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f4074ca0 0x7f24f419b970 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:00.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.889+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 >> 172.21.15.39:0/2755211174 conn(0x7f24f40fe950 msgr2=0x7f24f41004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:00.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.889+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 shutdown_connections 2024-07-04T08:16:00.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:00.889+0000 7f24f3e00640 1 -- 172.21.15.39:0/2755211174 wait complete. 2024-07-04T08:16:01.307 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:01 smithi039 ceph-mon[32062]: pgmap v504: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:16:01.307 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:01 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2755211174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:01.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:01 smithi133 ceph-mon[37050]: pgmap v504: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:16:01.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:01 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2755211174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:01.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:01 smithi050 ceph-mon[36577]: pgmap v504: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:16:01.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:01 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2755211174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:02.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:16:02.807 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:03.237 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:03 smithi039 ceph-mon[32062]: pgmap v505: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:03.237 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:03 smithi039 ceph-mon[32062]: mgrmap e25: x(active, since 14m), standbys: z, y 2024-07-04T08:16:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:03 smithi133 ceph-mon[37050]: pgmap v505: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:03 smithi133 ceph-mon[37050]: mgrmap e25: x(active, since 14m), standbys: z, y 2024-07-04T08:16:03.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:03 smithi050 ceph-mon[36577]: pgmap v505: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:03.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:03 smithi050 ceph-mon[36577]: mgrmap e25: x(active, since 14m), standbys: z, y 2024-07-04T08:16:04.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.335+0000 7f7467200640 1 -- 172.21.15.39:0/191175413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f746810aa50 msgr2=0x7f746810ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:04.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.335+0000 7f7467200640 1 --2- 172.21.15.39:0/191175413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f746810aa50 0x7f746810ae60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7454009900 tx=0x7f745402a400 comp rx=0 tx=0).stop 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.335+0000 7f7467200640 1 -- 172.21.15.39:0/191175413 shutdown_connections 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.335+0000 7f7467200640 1 --2- 172.21.15.39:0/191175413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f746810b3a0 0x7f7468111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.335+0000 7f7467200640 1 --2- 172.21.15.39:0/191175413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f746810aa50 0x7f746810ae60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.335+0000 7f7467200640 1 --2- 172.21.15.39:0/191175413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7468105110 0x7f7468105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.335+0000 7f7467200640 1 -- 172.21.15.39:0/191175413 >> 172.21.15.39:0/191175413 conn(0x7f7468100b80 msgr2=0x7f7468102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.335+0000 7f7467200640 1 -- 172.21.15.39:0/191175413 shutdown_connections 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.335+0000 7f7467200640 1 -- 172.21.15.39:0/191175413 wait complete. 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.337+0000 7f7467200640 1 Processor -- start 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.337+0000 7f7467200640 1 -- start start 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7467200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7468105110 0x7f746819baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7467200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746810aa50 0x7f746819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7467200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f746810b3a0 0x7f74681a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7467200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7468115010 con 0x7f746810aa50 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7467200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7468114e90 con 0x7f7468105110 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7467200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7468115190 con 0x7f746810b3a0 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7465400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746810aa50 0x7f746819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7465400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746810aa50 0x7f746819c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58022/0 (socket says 172.21.15.39:58022) 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7465400640 1 -- 172.21.15.39:0/1626948514 learned_addr learned my addr 172.21.15.39:0/1626948514 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7466800640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f746810b3a0 0x7f74681a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.338+0000 7f7465e00640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7468105110 0x7f746819baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.339+0000 7f7465400640 1 -- 172.21.15.39:0/1626948514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f746810b3a0 msgr2=0x7f74681a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.339+0000 7f7465400640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f746810b3a0 0x7f74681a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.339+0000 7f7465400640 1 -- 172.21.15.39:0/1626948514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7468105110 msgr2=0x7f746819baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:04.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.339+0000 7f7465400640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7468105110 0x7f746819baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.339+0000 7f7465400640 1 -- 172.21.15.39:0/1626948514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74681a1950 con 0x7f746810aa50 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.339+0000 7f7465400640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746810aa50 0x7f746819c040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7454006010 tx=0x7f7454004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.340+0000 7f745ea00640 1 -- 172.21.15.39:0/1626948514 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745403f030 con 0x7f746810aa50 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.340+0000 7f7467200640 1 -- 172.21.15.39:0/1626948514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74681a1be0 con 0x7f746810aa50 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.340+0000 7f745ea00640 1 -- 172.21.15.39:0/1626948514 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7454003aa0 con 0x7f746810aa50 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.340+0000 7f745ea00640 1 -- 172.21.15.39:0/1626948514 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454035070 con 0x7f746810aa50 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.340+0000 7f7467200640 1 -- 172.21.15.39:0/1626948514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681a2110 con 0x7f746810aa50 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.344+0000 7f7473362740 1 Processor -- start 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.344+0000 7f7473362740 1 -- start start 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.344+0000 7f7473362740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3e80060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.344+0000 7f7473362740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.344+0000 7f7473362740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 0x5574c3e80ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.344+0000 7f7473362740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5574c3dedca0 con 0x5574c3ad2f60 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.344+0000 7f7473362740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5574c3ddd240 con 0x5574c3ab2280 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.344+0000 7f7473362740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5574c3de1280 con 0x5574c3aaa0a0 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f745ea00640 1 -- 172.21.15.39:0/1626948514 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7454042030 con 0x7f746810aa50 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7465e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7466800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3e80060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7465e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e805a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43882/0 (socket says 172.21.15.39:43882) 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7465e00640 1 -- 172.21.15.39:0/1524813288 learned_addr learned my addr 172.21.15.39:0/1524813288 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7465e00640 1 -- 172.21.15.39:0/1524813288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 msgr2=0x5574c3e80ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7465e00640 1 --2- 172.21.15.39:0/1524813288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 0x5574c3e80ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7465e00640 1 -- 172.21.15.39:0/1524813288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 msgr2=0x5574c3e80060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7465e00640 1 --2- 172.21.15.39:0/1524813288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3e80060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7465e00640 1 -- 172.21.15.39:0/1524813288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c3d75550 con 0x5574c3aaa0a0 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.345+0000 7f7465e00640 1 --2- 172.21.15.39:0/1524813288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e805a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7458009cf0 tx=0x7f7458011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97a6705001cc603 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.346+0000 7f7453400640 1 -- 172.21.15.39:0/1524813288 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458004230 con 0x5574c3aaa0a0 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.346+0000 7f7453400640 1 -- 172.21.15.39:0/1524813288 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74580043d0 con 0x5574c3aaa0a0 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.346+0000 7f7453400640 1 -- 172.21.15.39:0/1524813288 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580197e0 con 0x5574c3aaa0a0 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f745ea00640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f744c0b2a10 0x7f744c0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f745ea00640 1 -- 172.21.15.39:0/1626948514 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7454102a00 con 0x7f746810aa50 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f7466800640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f744c0b2a10 0x7f744c0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f7473362740 1 -- 172.21.15.39:0/1524813288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 msgr2=0x5574c3e805a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f7473362740 1 --2- 172.21.15.39:0/1524813288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e805a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7458009cf0 tx=0x7f7458011810 comp rx=0 tx=0).stop 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f7473362740 1 -- 172.21.15.39:0/1524813288 shutdown_connections 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f7473362740 1 --2- 172.21.15.39:0/1524813288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 0x5574c3e80ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f7473362740 1 --2- 172.21.15.39:0/1524813288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e805a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f7473362740 1 --2- 172.21.15.39:0/1524813288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3e80060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f7473362740 1 -- 172.21.15.39:0/1524813288 >> 172.21.15.39:0/1524813288 conn(0x5574c3ac7350 msgr2=0x5574c3e7df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:04.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.347+0000 7f7473362740 1 -- 172.21.15.39:0/1524813288 shutdown_connections 2024-07-04T08:16:04.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.348+0000 7f7473362740 1 -- 172.21.15.39:0/1524813288 wait complete. 2024-07-04T08:16:04.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.348+0000 7f7466800640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f744c0b2a10 0x7f744c0b4f20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7460007f00 tx=0x7f7460007520 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:04.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.348+0000 7f7473362740 1 -- start start 2024-07-04T08:16:04.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.348+0000 7f7473362740 1 Processor -- start 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7473362740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e8bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7473362740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ab2280 0x5574c3e8c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7473362740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ad2f60 0x5574c3e97b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7473362740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5574c3c415f0 con 0x5574c3ab2280 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7473362740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5574c3d84970 con 0x5574c3aaa0a0 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7473362740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5574c3e8e840 con 0x5574c3ad2f60 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7473362740 10 client.0 _ll_register_callbacks cb 0x5574c3dd64a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7465e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e8bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7465400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ab2280 0x5574c3e8c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7465e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e8bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60444/0 (socket says 172.21.15.39:60444) 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7465e00640 1 -- 172.21.15.39:0/2695730537 learned_addr learned my addr 172.21.15.39:0/2695730537 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.349+0000 7f7465400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ab2280 0x5574c3e8c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58046/0 (socket says 172.21.15.39:58046) 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7466800640 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ad2f60 0x5574c3e97b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7465e00640 1 -- 172.21.15.39:0/2695730537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ad2f60 msgr2=0x5574c3e97b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7465e00640 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ad2f60 0x5574c3e97b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7465e00640 1 -- 172.21.15.39:0/2695730537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ab2280 msgr2=0x5574c3e8c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7465e00640 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ab2280 0x5574c3e8c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7465e00640 1 -- 172.21.15.39:0/2695730537 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c3b3e540 con 0x5574c3aaa0a0 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7466800640 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ad2f60 0x5574c3e97b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7465400640 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ab2280 0x5574c3e8c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:04.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7465e00640 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e8bc30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74580038b0 tx=0x7f74580038e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15396fcf06d19642 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.350+0000 7f7447e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.351+0000 7f7447e00640 1 -- 172.21.15.39:0/2695730537 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458027070 con 0x5574c3aaa0a0 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.351+0000 7f7473362740 1 -- 172.21.15.39:0/2695730537 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74580043d0 con 0x5574c3aaa0a0 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.351+0000 7f7447e00640 1 -- 172.21.15.39:0/2695730537 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7458005240 con 0x5574c3aaa0a0 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.351+0000 7f7447e00640 1 -- 172.21.15.39:0/2695730537 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7458033050 con 0x5574c3aaa0a0 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.351+0000 7f7473362740 10 client.25655 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.351+0000 7f7473362740 1 -- 172.21.15.39:0/2695730537 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574c3e9a860 con 0x5574c3aaa0a0 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.351+0000 7f7447e00640 1 -- 172.21.15.39:0/2695730537 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74580043d0 con 0x5574c3aaa0a0 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.352+0000 7f7447e00640 1 -- 172.21.15.39:0/2695730537 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7458004c00 con 0x5574c3aaa0a0 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.352+0000 7f7473362740 10 client.25655 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.352+0000 7f7473362740 10 client.25655 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.352+0000 7f7473362740 20 client.25655 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:04.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.352+0000 7f7473362740 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5574c3a918e0 0x5574c3e9e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.352+0000 7f7473362740 4 client.25655 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.352+0000 7f7473362740 1 -- 172.21.15.39:0/2695730537 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c3defbf0 con 0x5574c3a918e0 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.352+0000 7f7465400640 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5574c3a918e0 0x5574c3e9e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.353+0000 7f7465400640 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5574c3a918e0 0x5574c3e9e0e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5bbdc65ea195336 server_cookie=ec014422de3f5f56 in_seq=0 out_seq=0 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.353+0000 7f7447e00640 10 client.25655 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7447e00640 1 -- 172.21.15.39:0/2695730537 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5574c3defbf0 con 0x5574c3a918e0 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7447e00640 10 client.25655 handle_command_reply: tid=0 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 client.25655 shutdown 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 -- 172.21.15.39:0/2695730537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 msgr2=0x5574c3e8bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e8bc30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74580038b0 tx=0x7f74580038e0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 -- 172.21.15.39:0/2695730537 shutdown_connections 2024-07-04T08:16:04.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5574c3a918e0 0x5574c3e9e0e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ab2280 0x5574c3e8c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ad2f60 0x5574c3e97b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 --2- 172.21.15.39:0/2695730537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e8bc30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7447e00640 0 client.25655 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 -- 172.21.15.39:0/2695730537 >> 172.21.15.39:0/2695730537 conn(0x5574c3ac7350 msgr2=0x5574c3e83240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 -- 172.21.15.39:0/2695730537 shutdown_connections 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 1 -- 172.21.15.39:0/2695730537 wait complete. 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.354+0000 7f7473362740 20 client.25655 trim_cache size 0 max 16384 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.375+0000 7f7473362740 1 Processor -- start 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.375+0000 7f7473362740 1 -- start start 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.375+0000 7f7473362740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7473362740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ab2280 0x5574c3e97b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7473362740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3e8c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7473362740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5574c3e9c420 con 0x5574c3ad2f60 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7473362740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5574c3e9caf0 con 0x5574c3aaa0a0 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7473362740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5574c3e9d1c0 con 0x5574c3ab2280 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7465e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7465400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ab2280 0x5574c3e97b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7465e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e871e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60452/0 (socket says 172.21.15.39:60452) 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7465e00640 1 -- 172.21.15.39:0/3602277237 learned_addr learned my addr 172.21.15.39:0/3602277237 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7466800640 1 --2- 172.21.15.39:0/3602277237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3e8c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7465e00640 1 -- 172.21.15.39:0/3602277237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ab2280 msgr2=0x5574c3e97b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7465e00640 1 --2- 172.21.15.39:0/3602277237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ab2280 0x5574c3e97b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7465e00640 1 -- 172.21.15.39:0/3602277237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 msgr2=0x5574c3e8c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:04.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7465e00640 1 --2- 172.21.15.39:0/3602277237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3e8c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.376+0000 7f7465e00640 1 -- 172.21.15.39:0/3602277237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c3e82ec0 con 0x5574c3aaa0a0 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.377+0000 7f7466800640 1 --2- 172.21.15.39:0/3602277237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3e8c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.377+0000 7f7465e00640 1 --2- 172.21.15.39:0/3602277237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e871e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f74580058c0 tx=0x7f7458009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9398c084d4ae1863 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.377+0000 7f7452a00640 1 -- 172.21.15.39:0/3602277237 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458027070 con 0x5574c3aaa0a0 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.377+0000 7f7452a00640 1 -- 172.21.15.39:0/3602277237 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f745803c070 con 0x5574c3aaa0a0 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.377+0000 7f7452a00640 1 -- 172.21.15.39:0/3602277237 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458005000 con 0x5574c3aaa0a0 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.378+0000 7f7473362740 1 -- 172.21.15.39:0/3602277237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 msgr2=0x5574c3e871e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.378+0000 7f7473362740 1 --2- 172.21.15.39:0/3602277237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e871e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f74580058c0 tx=0x7f7458009bf0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.378+0000 7f7473362740 1 -- 172.21.15.39:0/3602277237 shutdown_connections 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.378+0000 7f7473362740 1 --2- 172.21.15.39:0/3602277237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3e8c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.378+0000 7f7473362740 1 --2- 172.21.15.39:0/3602277237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3ab2280 0x5574c3e97b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.378+0000 7f7473362740 1 --2- 172.21.15.39:0/3602277237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3aaa0a0 0x5574c3e871e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.378+0000 7f7473362740 1 -- 172.21.15.39:0/3602277237 >> 172.21.15.39:0/3602277237 conn(0x5574c3ac7350 msgr2=0x5574c3e8bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.378+0000 7f7473362740 1 -- 172.21.15.39:0/3602277237 shutdown_connections 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.378+0000 7f7473362740 1 -- 172.21.15.39:0/3602277237 wait complete. 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.379+0000 7f7473362740 1 -- start start 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.379+0000 7f7473362740 1 Processor -- start 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7473362740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e9f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7473362740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 0x5574c3e9fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7473362740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3ea4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7473362740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5574c3e9df60 con 0x5574c3ad2f60 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7473362740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5574c3e9d890 con 0x5574c3ab2280 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7473362740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5574c3e9e630 con 0x5574c3aaa0a0 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7473362740 10 client.0 _ll_register_callbacks cb 0x7f7460007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:04.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7465400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 0x5574c3e9fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7465e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e9f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7465400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 0x5574c3e9fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60458/0 (socket says 172.21.15.39:60458) 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7465400640 1 -- 172.21.15.39:0/2072620732 learned_addr learned my addr 172.21.15.39:0/2072620732 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.380+0000 7f7466800640 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3ea4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7465400640 1 -- 172.21.15.39:0/2072620732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 msgr2=0x5574c3e9f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7465400640 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e9f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7465400640 1 -- 172.21.15.39:0/2072620732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 msgr2=0x5574c3ea4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7465400640 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3ea4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7465400640 1 -- 172.21.15.39:0/2072620732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c3ea5380 con 0x5574c3ab2280 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7465e00640 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e9f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7465400640 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 0x5574c3e9fe40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f74540c98c0 tx=0x7f7454003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa639ea43afc835f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:04.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7447e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 1, 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24701, 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 302.61574851400002, 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 30, 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:16:04.862 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 92, 2024-07-04T08:16:04.863 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 404.67800440500002 2024-07-04T08:16:04.863 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:16:04.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7447e00640 1 -- 172.21.15.39:0/2072620732 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454049040 con 0x5574c3ab2280 2024-07-04T08:16:04.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7447e00640 1 -- 172.21.15.39:0/2072620732 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f74540ec0b0 con 0x5574c3ab2280 2024-07-04T08:16:04.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7473362740 1 -- 172.21.15.39:0/2072620732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f745803c070 con 0x5574c3ab2280 2024-07-04T08:16:04.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7447e00640 1 -- 172.21.15.39:0/2072620732 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74540ec2b0 con 0x5574c3ab2280 2024-07-04T08:16:04.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7473362740 10 client.25661 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:04.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.381+0000 7f7473362740 1 -- 172.21.15.39:0/2072620732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574c3ea66b0 con 0x5574c3ab2280 2024-07-04T08:16:04.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.382+0000 7f7447e00640 1 -- 172.21.15.39:0/2072620732 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f745803c070 con 0x5574c3ab2280 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.382+0000 7f7447e00640 1 -- 172.21.15.39:0/2072620732 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f745404c060 con 0x5574c3ab2280 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.382+0000 7f7473362740 10 client.25661 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.382+0000 7f7473362740 10 client.25661 resolve_mds: resolved name 'a' to daemon mds.a 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.382+0000 7f7473362740 20 client.25661 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.382+0000 7f7473362740 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5574c3a66c90 0x5574c3f0a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.382+0000 7f7473362740 4 client.25661 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.382+0000 7f7473362740 1 -- 172.21.15.39:0/2072620732 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "status"}) -- 0x5574c3e9c270 con 0x5574c3a66c90 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.383+0000 7f7465e00640 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5574c3a66c90 0x5574c3f0a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.383+0000 7f7465e00640 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5574c3a66c90 0x5574c3f0a0e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74cef92e2d589301 server_cookie=422517f319fb8fb0 in_seq=0 out_seq=0 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.383+0000 7f7447e00640 10 client.25661 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.383+0000 7f7447e00640 1 -- 172.21.15.39:0/2072620732 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7f7458030160 con 0x5574c3a66c90 2024-07-04T08:16:04.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.383+0000 7f7447e00640 10 client.25661 handle_command_reply: tid=0 2024-07-04T08:16:04.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 client.25661 shutdown 2024-07-04T08:16:04.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 -- 172.21.15.39:0/2072620732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 msgr2=0x5574c3e9fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:04.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 0x5574c3e9fe40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f74540c98c0 tx=0x7f7454003f50 comp rx=0 tx=0).stop 2024-07-04T08:16:04.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 -- 172.21.15.39:0/2072620732 shutdown_connections 2024-07-04T08:16:04.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5574c3a66c90 0x5574c3f0a0e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5574c3ad2f60 0x5574c3ea4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5574c3ab2280 0x5574c3e9fe40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7447e00640 0 client.25661 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 --2- 172.21.15.39:0/2072620732 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5574c3aaa0a0 0x5574c3e9f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 -- 172.21.15.39:0/2072620732 >> 172.21.15.39:0/2072620732 conn(0x5574c3ac7350 msgr2=0x5574c3e7d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 -- 172.21.15.39:0/2072620732 shutdown_connections 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 1 -- 172.21.15.39:0/2072620732 wait complete. 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.384+0000 7f7473362740 20 client.25661 trim_cache size 0 max 16384 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 -- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f744c0b2a10 msgr2=0x7f744c0b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f744c0b2a10 0x7f744c0b4f20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7460007f00 tx=0x7f7460007520 comp rx=0 tx=0).stop 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 -- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746810aa50 msgr2=0x7f746819c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746810aa50 0x7f746819c040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7454006010 tx=0x7f7454004060 comp rx=0 tx=0).stop 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 -- 172.21.15.39:0/1626948514 shutdown_connections 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f744c0b2a10 0x7f744c0b4f20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746810aa50 0x7f746819c040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f746810b3a0 0x7f74681a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 --2- 172.21.15.39:0/1626948514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7468105110 0x7f746819baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:04.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.385+0000 7f7467200640 1 -- 172.21.15.39:0/1626948514 >> 172.21.15.39:0/1626948514 conn(0x7f7468100b80 msgr2=0x7f7468102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:04.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.386+0000 7f7467200640 1 -- 172.21.15.39:0/1626948514 shutdown_connections 2024-07-04T08:16:04.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:04.386+0000 7f7467200640 1 -- 172.21.15.39:0/1626948514 wait complete. 2024-07-04T08:16:05.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:05 smithi039 ceph-mon[32062]: pgmap v506: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-04T08:16:05.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:05 smithi050 ceph-mon[36577]: pgmap v506: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-04T08:16:05.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:05 smithi133 ceph-mon[37050]: pgmap v506: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-04T08:16:05.660 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.b status 2024-07-04T08:16:06.305 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:06.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.314+0000 7f0981600640 1 -- 172.21.15.39:0/2049396265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c105110 msgr2=0x7f097c105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:06.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.314+0000 7f0981600640 1 --2- 172.21.15.39:0/2049396265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c105110 0x7f097c105520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f096c009900 tx=0x7f096c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.314+0000 7f0981600640 1 -- 172.21.15.39:0/2049396265 shutdown_connections 2024-07-04T08:16:06.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.314+0000 7f0981600640 1 --2- 172.21.15.39:0/2049396265 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f097c10b3a0 0x7f097c111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.314+0000 7f0981600640 1 --2- 172.21.15.39:0/2049396265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097c10aa50 0x7f097c10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.314+0000 7f0981600640 1 --2- 172.21.15.39:0/2049396265 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c105110 0x7f097c105520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.314+0000 7f0981600640 1 -- 172.21.15.39:0/2049396265 >> 172.21.15.39:0/2049396265 conn(0x7f097c100b80 msgr2=0x7f097c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:06.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.314+0000 7f0981600640 1 -- 172.21.15.39:0/2049396265 shutdown_connections 2024-07-04T08:16:06.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.314+0000 7f0981600640 1 -- 172.21.15.39:0/2049396265 wait complete. 2024-07-04T08:16:06.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.315+0000 7f0981600640 1 Processor -- start 2024-07-04T08:16:06.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.315+0000 7f0981600640 1 -- start start 2024-07-04T08:16:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.316+0000 7f0981600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097c105110 0x7f097c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.316+0000 7f0981600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f097c10aa50 0x7f097c19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.316+0000 7f0981600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c10b3a0 0x7f097c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.316+0000 7f0981600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f097c115070 con 0x7f097c10aa50 2024-07-04T08:16:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.316+0000 7f0981600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f097c114ef0 con 0x7f097c10b3a0 2024-07-04T08:16:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.316+0000 7f0981600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1151f0 con 0x7f097c105110 2024-07-04T08:16:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.316+0000 7f0980c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c10b3a0 0x7f097c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f097be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097c105110 0x7f097c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f0980c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c10b3a0 0x7f097c1a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60464/0 (socket says 172.21.15.39:60464) 2024-07-04T08:16:06.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f0980c00640 1 -- 172.21.15.39:0/1212403730 learned_addr learned my addr 172.21.15.39:0/1212403730 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:06.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f0973e00640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f097c10aa50 0x7f097c19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f0980c00640 1 -- 172.21.15.39:0/1212403730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097c105110 msgr2=0x7f097c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f0980c00640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097c105110 0x7f097c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f0980c00640 1 -- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f097c10aa50 msgr2=0x7f097c19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f0980c00640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f097c10aa50 0x7f097c19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f0980c00640 1 -- 172.21.15.39:0/1212403730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c1a1910 con 0x7f097c10b3a0 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.317+0000 7f0980c00640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c10b3a0 0x7f097c1a12a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f097400cce0 tx=0x7f097400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.318+0000 7f0979600640 1 -- 172.21.15.39:0/1212403730 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097400f450 con 0x7f097c10b3a0 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.318+0000 7f0981600640 1 -- 172.21.15.39:0/1212403730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c1a1c00 con 0x7f097c10b3a0 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.318+0000 7f0981600640 1 -- 172.21.15.39:0/1212403730 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c1a8420 con 0x7f097c10b3a0 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.318+0000 7f0979600640 1 -- 172.21.15.39:0/1212403730 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f097400faa0 con 0x7f097c10b3a0 2024-07-04T08:16:06.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.318+0000 7f0979600640 1 -- 172.21.15.39:0/1212403730 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974018a90 con 0x7f097c10b3a0 2024-07-04T08:16:06.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.322+0000 7f098927e740 1 Processor -- start 2024-07-04T08:16:06.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.324+0000 7f0979600640 1 -- 172.21.15.39:0/1212403730 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f097400fc50 con 0x7f097c10b3a0 2024-07-04T08:16:06.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.324+0000 7f098927e740 1 -- start start 2024-07-04T08:16:06.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.324+0000 7f098927e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb804950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.324+0000 7f098927e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb804e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.324+0000 7f098927e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb8053d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.324+0000 7f098927e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bacb77aee0 con 0x55bacb4371b0 2024-07-04T08:16:06.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.324+0000 7f098927e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bacb76a480 con 0x55bacb460070 2024-07-04T08:16:06.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.324+0000 7f098927e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bacb76e4c0 con 0x55bacb43f390 2024-07-04T08:16:06.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.325+0000 7f0973e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb804950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.325+0000 7f0973e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb804950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60472/0 (socket says 172.21.15.39:60472) 2024-07-04T08:16:06.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.325+0000 7f0973e00640 1 -- 172.21.15.39:0/968888548 learned_addr learned my addr 172.21.15.39:0/968888548 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:06.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.325+0000 7f0973e00640 1 --2- 172.21.15.39:0/968888548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb8053d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.325+0000 7f097be00640 1 --2- 172.21.15.39:0/968888548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb804e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.326+0000 7f0973e00640 1 -- 172.21.15.39:0/968888548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 msgr2=0x55bacb8053d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:06.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.326+0000 7f0973e00640 1 --2- 172.21.15.39:0/968888548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb8053d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.326+0000 7f0973e00640 1 -- 172.21.15.39:0/968888548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 msgr2=0x55bacb804e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:06.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.326+0000 7f0973e00640 1 --2- 172.21.15.39:0/968888548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb804e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.326+0000 7f0973e00640 1 -- 172.21.15.39:0/968888548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacb702650 con 0x55bacb460070 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.327+0000 7f0979600640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09600b2990 0x7f09600b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.327+0000 7f0973e00640 1 --2- 172.21.15.39:0/968888548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb804950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f09640058c0 tx=0x7f0964005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d29ef7423b5280 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.327+0000 7f0979600640 1 -- 172.21.15.39:0/1212403730 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f09740e2c10 con 0x7f097c10b3a0 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.327+0000 7f0971600640 1 -- 172.21.15.39:0/968888548 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964005020 con 0x55bacb460070 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.327+0000 7f097be00640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09600b2990 0x7f09600b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.327+0000 7f0971600640 1 -- 172.21.15.39:0/968888548 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09640047a0 con 0x55bacb460070 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.327+0000 7f0971600640 1 -- 172.21.15.39:0/968888548 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096400fa20 con 0x55bacb460070 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.328+0000 7f098927e740 1 -- 172.21.15.39:0/968888548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 msgr2=0x55bacb804950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.328+0000 7f098927e740 1 --2- 172.21.15.39:0/968888548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb804950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f09640058c0 tx=0x7f0964005850 comp rx=0 tx=0).stop 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.328+0000 7f098927e740 1 -- 172.21.15.39:0/968888548 shutdown_connections 2024-07-04T08:16:06.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.328+0000 7f098927e740 1 --2- 172.21.15.39:0/968888548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb804e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.328+0000 7f098927e740 1 --2- 172.21.15.39:0/968888548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb8053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.328+0000 7f098927e740 1 --2- 172.21.15.39:0/968888548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb804950 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.328+0000 7f098927e740 1 -- 172.21.15.39:0/968888548 >> 172.21.15.39:0/968888548 conn(0x55bacb454460 msgr2=0x55bacb802840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:06.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.328+0000 7f097be00640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09600b2990 0x7f09600b4ea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f096c031440 tx=0x7f096c009640 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:06.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.329+0000 7f098927e740 1 -- 172.21.15.39:0/968888548 shutdown_connections 2024-07-04T08:16:06.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.329+0000 7f098927e740 1 -- 172.21.15.39:0/968888548 wait complete. 2024-07-04T08:16:06.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.330+0000 7f098927e740 1 -- start start 2024-07-04T08:16:06.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.330+0000 7f098927e740 1 Processor -- start 2024-07-04T08:16:06.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.331+0000 7f098927e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb810530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.331+0000 7f098927e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb810a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.331+0000 7f098927e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb82d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.331+0000 7f098927e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bacb5ce700 con 0x55bacb4371b0 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.331+0000 7f098927e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bacb711bb0 con 0x55bacb460070 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.331+0000 7f098927e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bacb824140 con 0x55bacb43f390 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f097be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb810530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.331+0000 7f098927e740 10 client.0 _ll_register_callbacks cb 0x55bacb763570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f097be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb810530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58092/0 (socket says 172.21.15.39:58092) 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f097be00640 1 -- 172.21.15.39:0/1989782539 learned_addr learned my addr 172.21.15.39:0/1989782539 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f0973e00640 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb810a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f0980c00640 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb82d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f0973e00640 1 -- 172.21.15.39:0/1989782539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 msgr2=0x55bacb82d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f0973e00640 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb82d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f0973e00640 1 -- 172.21.15.39:0/1989782539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 msgr2=0x55bacb810530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f0973e00640 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb810530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f0973e00640 1 -- 172.21.15.39:0/1989782539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacb4cb650 con 0x55bacb43f390 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f0980c00640 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb82d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.332+0000 7f097be00640 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb810530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.333+0000 7f0973e00640 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb810a70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f09640058c0 tx=0x7f0964009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be79ecec737e526f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.333+0000 7f096a000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.333+0000 7f096a000640 1 -- 172.21.15.39:0/1989782539 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964004a90 con 0x55bacb43f390 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.333+0000 7f098927e740 1 -- 172.21.15.39:0/1989782539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09640047a0 con 0x55bacb43f390 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.333+0000 7f096a000640 1 -- 172.21.15.39:0/1989782539 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0964019040 con 0x55bacb43f390 2024-07-04T08:16:06.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.334+0000 7f096a000640 1 -- 172.21.15.39:0/1989782539 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0964035050 con 0x55bacb43f390 2024-07-04T08:16:06.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.334+0000 7f098927e740 10 client.25651 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:06.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.334+0000 7f098927e740 1 -- 172.21.15.39:0/1989782539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bacb830180 con 0x55bacb43f390 2024-07-04T08:16:06.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.334+0000 7f096a000640 1 -- 172.21.15.39:0/1989782539 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09640047a0 con 0x55bacb43f390 2024-07-04T08:16:06.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.334+0000 7f096a000640 1 -- 172.21.15.39:0/1989782539 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0964030030 con 0x55bacb43f390 2024-07-04T08:16:06.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.334+0000 7f098927e740 10 client.25651 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:06.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.335+0000 7f098927e740 10 client.25651 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:16:06.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.335+0000 7f098927e740 20 client.25651 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.335+0000 7f098927e740 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55bacb41e9f0 0x55bacb8334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.335+0000 7f098927e740 4 client.25651 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.335+0000 7f098927e740 1 -- 172.21.15.39:0/1989782539 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bacb77ce30 con 0x55bacb41e9f0 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.335+0000 7f097be00640 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55bacb41e9f0 0x55bacb8334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.336+0000 7f096a000640 10 client.25651 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.336+0000 7f097be00640 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55bacb41e9f0 0x55bacb8334e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f2d9b84d1c42787f server_cookie=d62b2a267f0f4ecd in_seq=0 out_seq=0 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.336+0000 7f096a000640 1 -- 172.21.15.39:0/1989782539 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bacb77ce30 con 0x55bacb41e9f0 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.336+0000 7f096a000640 10 client.25651 handle_command_reply: tid=0 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 client.25651 shutdown 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 -- 172.21.15.39:0/1989782539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 msgr2=0x55bacb810a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:06.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb810a70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f09640058c0 tx=0x7f0964009620 comp rx=0 tx=0).stop 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 -- 172.21.15.39:0/1989782539 shutdown_connections 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55bacb41e9f0 0x55bacb8334e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb82d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb810a70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 --2- 172.21.15.39:0/1989782539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb810530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 -- 172.21.15.39:0/1989782539 >> 172.21.15.39:0/1989782539 conn(0x55bacb454460 msgr2=0x55bacb807ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f096a000640 0 client.25651 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.337+0000 7f098927e740 1 -- 172.21.15.39:0/1989782539 shutdown_connections 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.338+0000 7f098927e740 1 -- 172.21.15.39:0/1989782539 wait complete. 2024-07-04T08:16:06.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.338+0000 7f098927e740 20 client.25651 trim_cache size 0 max 16384 2024-07-04T08:16:06.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.355+0000 7f098927e740 1 Processor -- start 2024-07-04T08:16:06.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.355+0000 7f098927e740 1 -- start start 2024-07-04T08:16:06.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.355+0000 7f098927e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb808410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.355+0000 7f098927e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb811f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.355+0000 7f098927e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb838c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.355+0000 7f098927e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f095800fdc0 con 0x55bacb4371b0 2024-07-04T08:16:06.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.355+0000 7f098927e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0958011900 con 0x55bacb460070 2024-07-04T08:16:06.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.355+0000 7f098927e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0958010490 con 0x55bacb43f390 2024-07-04T08:16:06.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f097be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb808410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f097be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb808410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58104/0 (socket says 172.21.15.39:58104) 2024-07-04T08:16:06.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f097be00640 1 -- 172.21.15.39:0/4129810880 learned_addr learned my addr 172.21.15.39:0/4129810880 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:06.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f0980c00640 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb838c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f0973e00640 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb811f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f097be00640 1 -- 172.21.15.39:0/4129810880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 msgr2=0x55bacb811f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f097be00640 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb811f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f097be00640 1 -- 172.21.15.39:0/4129810880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 msgr2=0x55bacb838c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f097be00640 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb838c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.356+0000 7f097be00640 1 -- 172.21.15.39:0/4129810880 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacb806c40 con 0x55bacb4371b0 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.357+0000 7f0973e00640 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb811f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.357+0000 7f097be00640 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb808410 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f096c000c00 tx=0x7f096c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee29e30a474d8552 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.357+0000 7f0970c00640 1 -- 172.21.15.39:0/4129810880 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c056070 con 0x55bacb4371b0 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.357+0000 7f0970c00640 1 -- 172.21.15.39:0/4129810880 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f096c004c00 con 0x55bacb4371b0 2024-07-04T08:16:06.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.357+0000 7f0970c00640 1 -- 172.21.15.39:0/4129810880 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c035040 con 0x55bacb4371b0 2024-07-04T08:16:06.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.358+0000 7f098927e740 1 -- 172.21.15.39:0/4129810880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 msgr2=0x55bacb808410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:06.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.358+0000 7f098927e740 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb808410 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f096c000c00 tx=0x7f096c03e000 comp rx=0 tx=0).stop 2024-07-04T08:16:06.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.358+0000 7f098927e740 1 -- 172.21.15.39:0/4129810880 shutdown_connections 2024-07-04T08:16:06.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.358+0000 7f098927e740 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb460070 0x55bacb838c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.358+0000 7f098927e740 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb43f390 0x55bacb811f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.358+0000 7f098927e740 1 --2- 172.21.15.39:0/4129810880 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb4371b0 0x55bacb808410 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.358+0000 7f098927e740 1 -- 172.21.15.39:0/4129810880 >> 172.21.15.39:0/4129810880 conn(0x55bacb454460 msgr2=0x55bacb811850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:06.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.359+0000 7f098927e740 1 -- 172.21.15.39:0/4129810880 shutdown_connections 2024-07-04T08:16:06.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.359+0000 7f098927e740 1 -- 172.21.15.39:0/4129810880 wait complete. 2024-07-04T08:16:06.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.359+0000 7f098927e740 1 -- start start 2024-07-04T08:16:06.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.360+0000 7f098927e740 1 Processor -- start 2024-07-04T08:16:06.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f098927e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb4371b0 0x55bacb829ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f098927e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb43f390 0x55bacb824fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f098927e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb460070 0x55bacb825510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f098927e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0958011230 con 0x55bacb43f390 2024-07-04T08:16:06.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f098927e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0958010b60 con 0x55bacb4371b0 2024-07-04T08:16:06.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f098927e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f095800dda0 con 0x55bacb460070 2024-07-04T08:16:06.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f098927e740 10 client.0 _ll_register_callbacks cb 0x7f0964009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:06.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f097be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb4371b0 0x55bacb829ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f0980c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb460070 0x55bacb825510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f097be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb4371b0 0x55bacb829ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60506/0 (socket says 172.21.15.39:60506) 2024-07-04T08:16:06.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f097be00640 1 -- 172.21.15.39:0/1116025441 learned_addr learned my addr 172.21.15.39:0/1116025441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:06.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.361+0000 7f0973e00640 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb43f390 0x55bacb824fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f097be00640 1 -- 172.21.15.39:0/1116025441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb460070 msgr2=0x55bacb825510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:06.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f097be00640 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb460070 0x55bacb825510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f097be00640 1 -- 172.21.15.39:0/1116025441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb43f390 msgr2=0x55bacb824fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f097be00640 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb43f390 0x55bacb824fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f097be00640 1 -- 172.21.15.39:0/1116025441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacb825ca0 con 0x55bacb4371b0 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f0973e00640 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb43f390 0x55bacb824fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f097be00640 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb4371b0 0x55bacb829ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f096c0057c0 tx=0x7f096c031740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a16a2f6dff03f48 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f096a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f096a000640 1 -- 172.21.15.39:0/1116025441 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c056040 con 0x55bacb4371b0 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f098927e740 1 -- 172.21.15.39:0/1116025441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096c004c00 con 0x55bacb4371b0 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.362+0000 7f096a000640 1 -- 172.21.15.39:0/1116025441 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f096c0502f0 con 0x55bacb4371b0 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.363+0000 7f096a000640 1 -- 172.21.15.39:0/1116025441 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f096c065050 con 0x55bacb4371b0 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.363+0000 7f098927e740 10 client.25676 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.363+0000 7f098927e740 1 -- 172.21.15.39:0/1116025441 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bacb89bae0 con 0x55bacb4371b0 2024-07-04T08:16:06.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.363+0000 7f096a000640 1 -- 172.21.15.39:0/1116025441 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f096c004c00 con 0x55bacb4371b0 2024-07-04T08:16:06.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.363+0000 7f096a000640 1 -- 172.21.15.39:0/1116025441 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f096c03c440 con 0x55bacb4371b0 2024-07-04T08:16:06.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.363+0000 7f098927e740 10 client.25676 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:06.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.363+0000 7f098927e740 10 client.25676 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:16:06.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.364+0000 7f098927e740 20 client.25676 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:06.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.364+0000 7f098927e740 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55bacb3f3da0 0x55bacb89e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:06.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.364+0000 7f098927e740 4 client.25676 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:16:06.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.364+0000 7f098927e740 1 -- 172.21.15.39:0/1116025441 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bacb7788d0 con 0x55bacb3f3da0 2024-07-04T08:16:06.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.364+0000 7f0973e00640 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55bacb3f3da0 0x55bacb89e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:06.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.364+0000 7f0973e00640 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55bacb3f3da0 0x55bacb89e2d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8063b3152177d24f server_cookie=7bc885eb6481067 in_seq=0 out_seq=0 2024-07-04T08:16:06.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f096a000640 10 client.25676 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:16:06.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f096a000640 1 -- 172.21.15.39:0/1116025441 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bacb7788d0 con 0x55bacb3f3da0 2024-07-04T08:16:06.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f096a000640 10 client.25676 handle_command_reply: tid=0 2024-07-04T08:16:06.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 client.25676 shutdown 2024-07-04T08:16:06.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 -- 172.21.15.39:0/1116025441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb4371b0 msgr2=0x55bacb829ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:06.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb4371b0 0x55bacb829ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f096c0057c0 tx=0x7f096c031740 comp rx=0 tx=0).stop 2024-07-04T08:16:06.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 -- 172.21.15.39:0/1116025441 shutdown_connections 2024-07-04T08:16:06.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55bacb3f3da0 0x55bacb89e2d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bacb43f390 0x55bacb824fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bacb460070 0x55bacb825510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 --2- 172.21.15.39:0/1116025441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bacb4371b0 0x55bacb829ad0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 -- 172.21.15.39:0/1116025441 >> 172.21.15.39:0/1116025441 conn(0x55bacb454460 msgr2=0x55bacb8062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:06.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f096a000640 0 client.25676 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:16:06.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.365+0000 7f098927e740 1 -- 172.21.15.39:0/1116025441 shutdown_connections 2024-07-04T08:16:06.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.366+0000 7f098927e740 1 -- 172.21.15.39:0/1116025441 wait complete. 2024-07-04T08:16:06.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.366+0000 7f098927e740 20 client.25676 trim_cache size 0 max 16384 2024-07-04T08:16:06.370 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:16:06.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.366+0000 7f0981600640 1 -- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09600b2990 msgr2=0x7f09600b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:06.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.366+0000 7f0981600640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09600b2990 0x7f09600b4ea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f096c031440 tx=0x7f096c009640 comp rx=0 tx=0).stop 2024-07-04T08:16:06.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.366+0000 7f0981600640 1 -- 172.21.15.39:0/1212403730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c10b3a0 msgr2=0x7f097c1a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:06.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.366+0000 7f0981600640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c10b3a0 0x7f097c1a12a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f097400cce0 tx=0x7f097400a620 comp rx=0 tx=0).stop 2024-07-04T08:16:06.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.367+0000 7f0981600640 1 -- 172.21.15.39:0/1212403730 shutdown_connections 2024-07-04T08:16:06.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.367+0000 7f0981600640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09600b2990 0x7f09600b4ea0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.367+0000 7f0981600640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f097c10aa50 0x7f097c19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.367+0000 7f0981600640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f097c10b3a0 0x7f097c1a12a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.367+0000 7f0981600640 1 --2- 172.21.15.39:0/1212403730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f097c105110 0x7f097c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:06.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.367+0000 7f0981600640 1 -- 172.21.15.39:0/1212403730 >> 172.21.15.39:0/1212403730 conn(0x7f097c100b80 msgr2=0x7f097c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:06.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.367+0000 7f0981600640 1 -- 172.21.15.39:0/1212403730 shutdown_connections 2024-07-04T08:16:06.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:06.367+0000 7f0981600640 1 -- 172.21.15.39:0/1212403730 wait complete. 2024-07-04T08:16:07.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:07 smithi039 ceph-mon[32062]: pgmap v507: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:16:07.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:07 smithi050 ceph-mon[36577]: pgmap v507: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:16:07.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:07 smithi133 ceph-mon[37050]: pgmap v507: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:16:07.572 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:16:08.545 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:09.184 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:09 smithi039 ceph-mon[32062]: pgmap v508: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-07-04T08:16:09.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:09 smithi050 ceph-mon[36577]: pgmap v508: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-07-04T08:16:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.407+0000 7f56d1e00640 1 -- 172.21.15.39:0/2168743078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 msgr2=0x7f56cc10f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.407+0000 7f56d1e00640 1 --2- 172.21.15.39:0/2168743078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 0x7f56cc10f1f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009900 tx=0x7f56b402a480 comp rx=0 tx=0).stop 2024-07-04T08:16:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.407+0000 7f56d1e00640 1 -- 172.21.15.39:0/2168743078 shutdown_connections 2024-07-04T08:16:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.407+0000 7f56d1e00640 1 --2- 172.21.15.39:0/2168743078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 0x7f56cc10f1f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.407+0000 7f56d1e00640 1 --2- 172.21.15.39:0/2168743078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56cc10f730 0x7f56cc111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.407+0000 7f56d1e00640 1 --2- 172.21.15.39:0/2168743078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56cc101b80 0x7f56cc101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.407+0000 7f56d1e00640 1 -- 172.21.15.39:0/2168743078 >> 172.21.15.39:0/2168743078 conn(0x7f56cc078c80 msgr2=0x7f56cc079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.408+0000 7f56d1e00640 1 -- 172.21.15.39:0/2168743078 shutdown_connections 2024-07-04T08:16:09.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.408+0000 7f56d1e00640 1 -- 172.21.15.39:0/2168743078 wait complete. 2024-07-04T08:16:09.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.409+0000 7f56d1e00640 1 Processor -- start 2024-07-04T08:16:09.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.409+0000 7f56d1e00640 1 -- start start 2024-07-04T08:16:09.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56d1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56cc101b80 0x7f56cc19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56d1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 0x7f56cc19c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56d1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56cc10f730 0x7f56cc1a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56d1e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc115190 con 0x7f56cc1024d0 2024-07-04T08:16:09.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56d1e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc115010 con 0x7f56cc10f730 2024-07-04T08:16:09.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56d1e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc115310 con 0x7f56cc101b80 2024-07-04T08:16:09.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56c2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 0x7f56cc19c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56c2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 0x7f56cc19c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58126/0 (socket says 172.21.15.39:58126) 2024-07-04T08:16:09.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56c2000640 1 -- 172.21.15.39:0/1501579492 learned_addr learned my addr 172.21.15.39:0/1501579492 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:09.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.410+0000 7f56caa00640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56cc101b80 0x7f56cc19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.411+0000 7f56cb400640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56cc10f730 0x7f56cc1a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.411+0000 7f56c2000640 1 -- 172.21.15.39:0/1501579492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56cc101b80 msgr2=0x7f56cc19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:09.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.411+0000 7f56c2000640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56cc101b80 0x7f56cc19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.411+0000 7f56c2000640 1 -- 172.21.15.39:0/1501579492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56cc10f730 msgr2=0x7f56cc1a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.411+0000 7f56c2000640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56cc10f730 0x7f56cc1a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.411+0000 7f56c2000640 1 -- 172.21.15.39:0/1501579492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc1a1ad0 con 0x7f56cc1024d0 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.411+0000 7f56cb400640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56cc10f730 0x7f56cc1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.412+0000 7f56c2000640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 0x7f56cc19c100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005940 tx=0x7f56b40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.412+0000 7f56d0a00640 1 -- 172.21.15.39:0/1501579492 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b403f030 con 0x7f56cc1024d0 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.412+0000 7f56d0a00640 1 -- 172.21.15.39:0/1501579492 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b40044c0 con 0x7f56cc1024d0 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.412+0000 7f56d1e00640 1 -- 172.21.15.39:0/1501579492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc1a1d60 con 0x7f56cc1024d0 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.412+0000 7f56d1e00640 1 -- 172.21.15.39:0/1501579492 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc1a8520 con 0x7f56cc1024d0 2024-07-04T08:16:09.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.413+0000 7f56d0a00640 1 -- 172.21.15.39:0/1501579492 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4035070 con 0x7f56cc1024d0 2024-07-04T08:16:09.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.416+0000 7f56d833e740 1 Processor -- start 2024-07-04T08:16:09.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.416+0000 7f56d833e740 1 -- start start 2024-07-04T08:16:09.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.416+0000 7f56d833e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c56ef60 0x55583c913670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.416+0000 7f56d833e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c913bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.416+0000 7f56d833e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c9140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.416+0000 7f56d833e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55583c889ca0 con 0x55583c56ef60 2024-07-04T08:16:09.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.416+0000 7f56d833e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55583c879240 con 0x55583c54e280 2024-07-04T08:16:09.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.416+0000 7f56d833e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55583c87d280 con 0x55583c5460a0 2024-07-04T08:16:09.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56d0a00640 1 -- 172.21.15.39:0/1501579492 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f56b4042030 con 0x7f56cc1024d0 2024-07-04T08:16:09.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56caa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c913bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56caa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c913bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43994/0 (socket says 172.21.15.39:43994) 2024-07-04T08:16:09.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56caa00640 1 -- 172.21.15.39:0/4048475175 learned_addr learned my addr 172.21.15.39:0/4048475175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:09.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56c2000640 1 --2- 172.21.15.39:0/4048475175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c9140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56caa00640 1 -- 172.21.15.39:0/4048475175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 msgr2=0x55583c9140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:09.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56caa00640 1 --2- 172.21.15.39:0/4048475175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c9140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56caa00640 1 -- 172.21.15.39:0/4048475175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c56ef60 msgr2=0x55583c913670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:09.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56caa00640 1 --2- 172.21.15.39:0/4048475175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c56ef60 0x55583c913670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56caa00640 1 -- 172.21.15.39:0/4048475175 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55583c811550 con 0x55583c5460a0 2024-07-04T08:16:09.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.417+0000 7f56c2000640 1 --2- 172.21.15.39:0/4048475175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c9140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:09.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.418+0000 7f56caa00640 1 --2- 172.21.15.39:0/4048475175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c913bb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0114f0 tx=0x7f56bc0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5e77ef06ae380fb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:09.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.418+0000 7f56c0c00640 1 -- 172.21.15.39:0/4048475175 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc01e070 con 0x55583c5460a0 2024-07-04T08:16:09.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.418+0000 7f56c0c00640 1 -- 172.21.15.39:0/4048475175 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56bc003c30 con 0x55583c5460a0 2024-07-04T08:16:09.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.418+0000 7f56c0c00640 1 -- 172.21.15.39:0/4048475175 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00fa00 con 0x55583c5460a0 2024-07-04T08:16:09.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.419+0000 7f56d0a00640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56b00b2960 0x7f56b00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.419+0000 7f56d0a00640 1 -- 172.21.15.39:0/1501579492 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f56b4102cf0 con 0x7f56cc1024d0 2024-07-04T08:16:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.419+0000 7f56cb400640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56b00b2960 0x7f56b00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.419+0000 7f56d833e740 1 -- 172.21.15.39:0/4048475175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 msgr2=0x55583c913bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.419+0000 7f56d833e740 1 --2- 172.21.15.39:0/4048475175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c913bb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0114f0 tx=0x7f56bc0119b0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.420+0000 7f56d833e740 1 -- 172.21.15.39:0/4048475175 shutdown_connections 2024-07-04T08:16:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.420+0000 7f56d833e740 1 --2- 172.21.15.39:0/4048475175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c9140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.420+0000 7f56d833e740 1 --2- 172.21.15.39:0/4048475175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c913bb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.420+0000 7f56d833e740 1 --2- 172.21.15.39:0/4048475175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c56ef60 0x55583c913670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.420+0000 7f56d833e740 1 -- 172.21.15.39:0/4048475175 >> 172.21.15.39:0/4048475175 conn(0x55583c563350 msgr2=0x55583c911560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.420+0000 7f56d833e740 1 -- 172.21.15.39:0/4048475175 shutdown_connections 2024-07-04T08:16:09.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.420+0000 7f56d833e740 1 -- 172.21.15.39:0/4048475175 wait complete. 2024-07-04T08:16:09.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.420+0000 7f56cb400640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56b00b2960 0x7f56b00b4e70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56c4011550 tx=0x7f56c4009c30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:09.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.421+0000 7f56d833e740 1 -- start start 2024-07-04T08:16:09.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.421+0000 7f56d833e740 1 Processor -- start 2024-07-04T08:16:09.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56d833e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c91f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56d833e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c54e280 0x55583c91f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56d833e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c56ef60 0x55583c9449f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56d833e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55583c6dd5f0 con 0x55583c5460a0 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56d833e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55583c820970 con 0x55583c56ef60 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56d833e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55583c932e60 con 0x55583c54e280 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56d833e740 10 client.0 _ll_register_callbacks cb 0x55583c8724a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56caa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c91f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56caa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c91f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58142/0 (socket says 172.21.15.39:58142) 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56caa00640 1 -- 172.21.15.39:0/4267324185 learned_addr learned my addr 172.21.15.39:0/4267324185 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56cb400640 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c56ef60 0x55583c9449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56c2000640 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c54e280 0x55583c91f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56cb400640 1 -- 172.21.15.39:0/4267324185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c54e280 msgr2=0x55583c91f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.422+0000 7f56cb400640 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c54e280 0x55583c91f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.423+0000 7f56cb400640 1 -- 172.21.15.39:0/4267324185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 msgr2=0x55583c91f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.423+0000 7f56cb400640 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c91f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.423+0000 7f56cb400640 1 -- 172.21.15.39:0/4267324185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55583c5da540 con 0x55583c56ef60 2024-07-04T08:16:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.423+0000 7f56cb400640 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c56ef60 0x55583c9449f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56c4011de0 tx=0x7f56c4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac173e6aa3ab47ff server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.423+0000 7f56b9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.423+0000 7f56b9600640 1 -- 172.21.15.39:0/4267324185 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c402b060 con 0x55583c56ef60 2024-07-04T08:16:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.423+0000 7f56d833e740 1 -- 172.21.15.39:0/4267324185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56bc003c30 con 0x55583c56ef60 2024-07-04T08:16:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.423+0000 7f56b9600640 1 -- 172.21.15.39:0/4267324185 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f56c401ab10 con 0x55583c56ef60 2024-07-04T08:16:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.424+0000 7f56b9600640 1 -- 172.21.15.39:0/4267324185 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56c4032050 con 0x55583c56ef60 2024-07-04T08:16:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.424+0000 7f56d833e740 10 client.25688 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.424+0000 7f56d833e740 1 -- 172.21.15.39:0/4267324185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55583c9477a0 con 0x55583c56ef60 2024-07-04T08:16:09.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.424+0000 7f56b9600640 1 -- 172.21.15.39:0/4267324185 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56bc003c30 con 0x55583c56ef60 2024-07-04T08:16:09.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.424+0000 7f56b9600640 1 -- 172.21.15.39:0/4267324185 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f56c400f3c0 con 0x55583c56ef60 2024-07-04T08:16:09.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.425+0000 7f56d833e740 10 client.25688 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:09.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.425+0000 7f56d833e740 10 client.25688 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T08:16:09.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.425+0000 7f56d833e740 20 client.25688 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:09.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.425+0000 7f56d833e740 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55583c52d8e0 0x55583c94b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.425+0000 7f56d833e740 4 client.25688 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:09.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.425+0000 7f56d833e740 1 -- 172.21.15.39:0/4267324185 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55583c88bbf0 con 0x55583c52d8e0 2024-07-04T08:16:09.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.425+0000 7f56caa00640 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55583c52d8e0 0x55583c94b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.426+0000 7f56caa00640 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55583c52d8e0 0x55583c94b2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a196d2b8e372d7f server_cookie=fbf737d2d70af095 in_seq=0 out_seq=0 2024-07-04T08:16:09.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.426+0000 7f56b9600640 10 client.25688 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:09.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56b9600640 1 -- 172.21.15.39:0/4267324185 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55583c88bbf0 con 0x55583c52d8e0 2024-07-04T08:16:09.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56b9600640 10 client.25688 handle_command_reply: tid=0 2024-07-04T08:16:09.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56d833e740 1 client.25688 shutdown 2024-07-04T08:16:09.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56d833e740 1 -- 172.21.15.39:0/4267324185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c56ef60 msgr2=0x55583c9449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:09.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56d833e740 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c56ef60 0x55583c9449f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f56c4011de0 tx=0x7f56c4005f90 comp rx=0 tx=0).stop 2024-07-04T08:16:09.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56d833e740 1 -- 172.21.15.39:0/4267324185 shutdown_connections 2024-07-04T08:16:09.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56d833e740 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55583c52d8e0 0x55583c94b2c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56d833e740 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c56ef60 0x55583c9449f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56d833e740 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c54e280 0x55583c91f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56d833e740 1 --2- 172.21.15.39:0/4267324185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c91f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.427+0000 7f56d833e740 1 -- 172.21.15.39:0/4267324185 >> 172.21.15.39:0/4267324185 conn(0x55583c563350 msgr2=0x55583c916800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.428+0000 7f56b9600640 0 client.25688 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.428+0000 7f56d833e740 1 -- 172.21.15.39:0/4267324185 shutdown_connections 2024-07-04T08:16:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.428+0000 7f56d833e740 1 -- 172.21.15.39:0/4267324185 wait complete. 2024-07-04T08:16:09.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.428+0000 7f56d833e740 20 client.25688 trim_cache size 0 max 16384 2024-07-04T08:16:09.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.448+0000 7f56d833e740 1 Processor -- start 2024-07-04T08:16:09.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.449+0000 7f56d833e740 1 -- start start 2024-07-04T08:16:09.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.449+0000 7f56d833e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c9449e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.449+0000 7f56d833e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c944f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.449+0000 7f56d833e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c56ef60 0x55583c91a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.449+0000 7f56d833e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55583c949600 con 0x55583c56ef60 2024-07-04T08:16:09.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.449+0000 7f56d833e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55583c949cd0 con 0x55583c54e280 2024-07-04T08:16:09.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.449+0000 7f56d833e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55583c94a3a0 con 0x55583c5460a0 2024-07-04T08:16:09.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.449+0000 7f56c2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c944f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.450+0000 7f56c2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c944f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60570/0 (socket says 172.21.15.39:60570) 2024-07-04T08:16:09.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.450+0000 7f56c2000640 1 -- 172.21.15.39:0/4128426997 learned_addr learned my addr 172.21.15.39:0/4128426997 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:09.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.450+0000 7f56cb400640 1 --2- 172.21.15.39:0/4128426997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c56ef60 0x55583c91a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.450+0000 7f56caa00640 1 --2- 172.21.15.39:0/4128426997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c9449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.450+0000 7f56c2000640 1 -- 172.21.15.39:0/4128426997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 msgr2=0x55583c9449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:09.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.450+0000 7f56c2000640 1 --2- 172.21.15.39:0/4128426997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c9449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.450+0000 7f56c2000640 1 -- 172.21.15.39:0/4128426997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c56ef60 msgr2=0x55583c91a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:09.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.450+0000 7f56c2000640 1 --2- 172.21.15.39:0/4128426997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c56ef60 0x55583c91a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.450+0000 7f56c2000640 1 -- 172.21.15.39:0/4128426997 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55583c9475d0 con 0x55583c54e280 2024-07-04T08:16:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.451+0000 7f56c2000640 1 --2- 172.21.15.39:0/4128426997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c944f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f56b4032af0 tx=0x7f56b403b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b841c530eaf06de server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.451+0000 7f56bbe00640 1 -- 172.21.15.39:0/4128426997 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4049040 con 0x55583c54e280 2024-07-04T08:16:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.451+0000 7f56bbe00640 1 -- 172.21.15.39:0/4128426997 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b40c9800 con 0x55583c54e280 2024-07-04T08:16:09.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.451+0000 7f56bbe00640 1 -- 172.21.15.39:0/4128426997 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b40c60d0 con 0x55583c54e280 2024-07-04T08:16:09.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.452+0000 7f56d833e740 1 -- 172.21.15.39:0/4128426997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 msgr2=0x55583c944f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:09.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.452+0000 7f56d833e740 1 --2- 172.21.15.39:0/4128426997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c944f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f56b4032af0 tx=0x7f56b403b4f0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.452+0000 7f56d833e740 1 -- 172.21.15.39:0/4128426997 shutdown_connections 2024-07-04T08:16:09.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.452+0000 7f56d833e740 1 --2- 172.21.15.39:0/4128426997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c56ef60 0x55583c91a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.452+0000 7f56d833e740 1 --2- 172.21.15.39:0/4128426997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c944f20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.452+0000 7f56d833e740 1 --2- 172.21.15.39:0/4128426997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c5460a0 0x55583c9449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.452+0000 7f56d833e740 1 -- 172.21.15.39:0/4128426997 >> 172.21.15.39:0/4128426997 conn(0x55583c563350 msgr2=0x55583c948c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:09.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.452+0000 7f56d833e740 1 -- 172.21.15.39:0/4128426997 shutdown_connections 2024-07-04T08:16:09.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.452+0000 7f56d833e740 1 -- 172.21.15.39:0/4128426997 wait complete. 2024-07-04T08:16:09.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.453+0000 7f56d833e740 1 -- start start 2024-07-04T08:16:09.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.453+0000 7f56d833e740 1 Processor -- start 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56d833e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c953c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56d833e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c94f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56d833e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c56ef60 0x55583c94fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56d833e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55583c94b140 con 0x55583c5460a0 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56d833e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55583c94aa70 con 0x55583c54e280 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56d833e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55583c94b810 con 0x55583c56ef60 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56d833e740 10 client.0 _ll_register_callbacks cb 0x7f56c4009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56c2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c94f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56caa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c953c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56c2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c94f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60584/0 (socket says 172.21.15.39:60584) 2024-07-04T08:16:09.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56c2000640 1 -- 172.21.15.39:0/439025522 learned_addr learned my addr 172.21.15.39:0/439025522 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.454+0000 7f56cb400640 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c56ef60 0x55583c94fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56c2000640 1 -- 172.21.15.39:0/439025522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c56ef60 msgr2=0x55583c94fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56c2000640 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c56ef60 0x55583c94fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56c2000640 1 -- 172.21.15.39:0/439025522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 msgr2=0x55583c953c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56c2000640 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c953c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56c2000640 1 -- 172.21.15.39:0/439025522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55583c950380 con 0x55583c54e280 2024-07-04T08:16:09.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56caa00640 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c953c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:09.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56c2000640 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c94f6d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f56b40c9c80 tx=0x7f56b4100540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1109f7d586b43758 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:09.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56b9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:09.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56b9600640 1 -- 172.21.15.39:0/439025522 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4049070 con 0x55583c54e280 2024-07-04T08:16:09.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.455+0000 7f56b9600640 1 -- 172.21.15.39:0/439025522 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f56b40d2820 con 0x55583c54e280 2024-07-04T08:16:09.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56d833e740 1 -- 172.21.15.39:0/439025522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b40c9800 con 0x55583c54e280 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56b9600640 1 -- 172.21.15.39:0/439025522 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56b40ab050 con 0x55583c54e280 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56d833e740 10 client.25694 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56d833e740 1 -- 172.21.15.39:0/439025522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55583c9b3470 con 0x55583c54e280 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56b9600640 1 -- 172.21.15.39:0/439025522 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b40c9800 con 0x55583c54e280 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56b9600640 1 -- 172.21.15.39:0/439025522 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f56b40c5660 con 0x55583c54e280 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56d833e740 10 client.25694 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56d833e740 10 client.25694 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56d833e740 20 client.25694 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56d833e740 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55583c502c90 0x55583c9b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:09.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56d833e740 4 client.25694 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-04T08:16:09.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.456+0000 7f56d833e740 1 -- 172.21.15.39:0/439025522 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "status"}) -- 0x55583c887690 con 0x55583c502c90 2024-07-04T08:16:09.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.457+0000 7f56caa00640 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55583c502c90 0x55583c9b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56b9600640 10 client.25694 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56caa00640 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55583c502c90 0x55583c9b6bf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3151c48f19ef4a server_cookie=483a461732b75855 in_seq=0 out_seq=0 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56b9600640 1 -- 172.21.15.39:0/439025522 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55583c887690 con 0x55583c502c90 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56b9600640 10 client.25694 handle_command_reply: tid=0 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56d833e740 1 client.25694 shutdown 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56d833e740 1 -- 172.21.15.39:0/439025522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 msgr2=0x55583c94f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56d833e740 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c94f6d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f56b40c9c80 tx=0x7f56b4100540 comp rx=0 tx=0).stop 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56d833e740 1 -- 172.21.15.39:0/439025522 shutdown_connections 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56d833e740 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55583c502c90 0x55583c9b6bf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56d833e740 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55583c56ef60 0x55583c94fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56d833e740 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55583c54e280 0x55583c94f6d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56d833e740 1 --2- 172.21.15.39:0/439025522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55583c5460a0 0x55583c953c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56b9600640 0 client.25694 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:09.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.458+0000 7f56d833e740 1 -- 172.21.15.39:0/439025522 >> 172.21.15.39:0/439025522 conn(0x55583c563350 msgr2=0x55583c91a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:09.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.459+0000 7f56d833e740 1 -- 172.21.15.39:0/439025522 shutdown_connections 2024-07-04T08:16:09.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.459+0000 7f56d833e740 1 -- 172.21.15.39:0/439025522 wait complete. 2024-07-04T08:16:09.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.459+0000 7f56d833e740 20 client.25694 trim_cache size 0 max 16384 2024-07-04T08:16:09.463 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:16:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "cluster_fsid": "2ee561ca-39db-11ef-bcab-c7b262605968", 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "whoami": 0, 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "id": 24707, 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "want_state": "up:active", 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "up:active", 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "fs_name": "cephfs", 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "rank_uptime": 388.05805662199998, 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "mdsmap_epoch": 30, 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch": 92, 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_epoch_barrier": 92, 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "uptime": 407.157800689 2024-07-04T08:16:09.464 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:16:09.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 -- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56b00b2960 msgr2=0x7f56b00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:09.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56b00b2960 0x7f56b00b4e70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f56c4011550 tx=0x7f56c4009c30 comp rx=0 tx=0).stop 2024-07-04T08:16:09.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 -- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 msgr2=0x7f56cc19c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:09.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 0x7f56cc19c100 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005940 tx=0x7f56b40040c0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 -- 172.21.15.39:0/1501579492 shutdown_connections 2024-07-04T08:16:09.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56b00b2960 0x7f56b00b4e70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56cc1024d0 0x7f56cc19c100 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56cc10f730 0x7f56cc1a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 --2- 172.21.15.39:0/1501579492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56cc101b80 0x7f56cc19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:09.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 -- 172.21.15.39:0/1501579492 >> 172.21.15.39:0/1501579492 conn(0x7f56cc078c80 msgr2=0x7f56cc1081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:09.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 -- 172.21.15.39:0/1501579492 shutdown_connections 2024-07-04T08:16:09.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:09.460+0000 7f56d1e00640 1 -- 172.21.15.39:0/1501579492 wait complete. 2024-07-04T08:16:09.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:09 smithi133 ceph-mon[37050]: pgmap v508: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-07-04T08:16:10.733 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-07-04T08:16:11.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:11 smithi039 ceph-mon[32062]: pgmap v509: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:16:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:11 smithi050 ceph-mon[36577]: pgmap v509: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:16:11.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:11 smithi133 ceph-mon[37050]: pgmap v509: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:16:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.200+0000 7f0d67e00640 1 -- 172.21.15.39:0/2644848006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d68074c80 msgr2=0x7f0d68075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.200+0000 7f0d67e00640 1 --2- 172.21.15.39:0/2644848006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d68074c80 0x7f0d68075090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009900 tx=0x7f0d5802a410 comp rx=0 tx=0).stop 2024-07-04T08:16:12.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.201+0000 7f0d67e00640 1 -- 172.21.15.39:0/2644848006 shutdown_connections 2024-07-04T08:16:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.201+0000 7f0d67e00640 1 --2- 172.21.15.39:0/2644848006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d6810f4e0 0x7f0d68111900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.201+0000 7f0d67e00640 1 --2- 172.21.15.39:0/2644848006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d680755d0 0x7f0d6810efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.201+0000 7f0d67e00640 1 --2- 172.21.15.39:0/2644848006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d68074c80 0x7f0d68075090 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.201+0000 7f0d67e00640 1 -- 172.21.15.39:0/2644848006 >> 172.21.15.39:0/2644848006 conn(0x7f0d680fe950 msgr2=0x7f0d68100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.201+0000 7f0d67e00640 1 -- 172.21.15.39:0/2644848006 shutdown_connections 2024-07-04T08:16:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.201+0000 7f0d67e00640 1 -- 172.21.15.39:0/2644848006 wait complete. 2024-07-04T08:16:12.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.203+0000 7f0d67e00640 1 Processor -- start 2024-07-04T08:16:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.203+0000 7f0d67e00640 1 -- start start 2024-07-04T08:16:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.203+0000 7f0d67e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d68074c80 0x7f0d6819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d67e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d680755d0 0x7f0d6819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d67e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d6810f4e0 0x7f0d681a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d67e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68115080 con 0x7f0d680755d0 2024-07-04T08:16:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d67e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68114f00 con 0x7f0d6810f4e0 2024-07-04T08:16:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d67e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68115200 con 0x7f0d68074c80 2024-07-04T08:16:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d66a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d68074c80 0x7f0d6819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d66a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d68074c80 0x7f0d6819ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41374/0 (socket says 172.21.15.39:41374) 2024-07-04T08:16:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d66a00640 1 -- 172.21.15.39:0/3095133650 learned_addr learned my addr 172.21.15.39:0/3095133650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d66000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d680755d0 0x7f0d6819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.204+0000 7f0d67400640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d6810f4e0 0x7f0d681a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.205+0000 7f0d67400640 1 -- 172.21.15.39:0/3095133650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d68074c80 msgr2=0x7f0d6819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.205+0000 7f0d67400640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d68074c80 0x7f0d6819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.205+0000 7f0d67400640 1 -- 172.21.15.39:0/3095133650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d680755d0 msgr2=0x7f0d6819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.205+0000 7f0d67400640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d680755d0 0x7f0d6819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.205+0000 7f0d67400640 1 -- 172.21.15.39:0/3095133650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d681a19c0 con 0x7f0d6810f4e0 2024-07-04T08:16:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.205+0000 7f0d67400640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d6810f4e0 0x7f0d681a1290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009fd0 tx=0x7f0d6000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.205+0000 7f0d5f400640 1 -- 172.21.15.39:0/3095133650 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000ce20 con 0x7f0d6810f4e0 2024-07-04T08:16:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.205+0000 7f0d5f400640 1 -- 172.21.15.39:0/3095133650 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d60005020 con 0x7f0d6810f4e0 2024-07-04T08:16:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.205+0000 7f0d67e00640 1 -- 172.21.15.39:0/3095133650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d681a1cb0 con 0x7f0d6810f4e0 2024-07-04T08:16:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.206+0000 7f0d5f400640 1 -- 172.21.15.39:0/3095133650 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60010690 con 0x7f0d6810f4e0 2024-07-04T08:16:12.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.206+0000 7f0d67e00640 1 -- 172.21.15.39:0/3095133650 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d681a8410 con 0x7f0d6810f4e0 2024-07-04T08:16:12.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.210+0000 7f0d74295740 1 Processor -- start 2024-07-04T08:16:12.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.210+0000 7f0d5f400640 1 -- 172.21.15.39:0/3095133650 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0d600054d0 con 0x7f0d6810f4e0 2024-07-04T08:16:12.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.211+0000 7f0d74295740 1 -- start start 2024-07-04T08:16:12.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.211+0000 7f0d74295740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 0x56365944b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.211+0000 7f0d74295740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56365907e1b0 0x56365944beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.211+0000 7f0d74295740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563659086390 0x56365944c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.211+0000 7f0d74295740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5636593c1f00 con 0x5636590a7070 2024-07-04T08:16:12.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.211+0000 7f0d74295740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5636593b14a0 con 0x563659086390 2024-07-04T08:16:12.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.211+0000 7f0d74295740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5636593b54e0 con 0x56365907e1b0 2024-07-04T08:16:12.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56365907e1b0 0x56365944beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 0x56365944b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56365907e1b0 0x56365944beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41378/0 (socket says 172.21.15.39:41378) 2024-07-04T08:16:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66a00640 1 -- 172.21.15.39:0/3302979142 learned_addr learned my addr 172.21.15.39:0/3302979142 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66000640 1 --2- 172.21.15.39:0/3302979142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563659086390 0x56365944c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66000640 1 -- 172.21.15.39:0/3302979142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56365907e1b0 msgr2=0x56365944beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66000640 1 --2- 172.21.15.39:0/3302979142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56365907e1b0 0x56365944beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d5f400640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d4c0b29e0 0x7f0d4c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66000640 1 -- 172.21.15.39:0/3302979142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 msgr2=0x56365944b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66000640 1 --2- 172.21.15.39:0/3302979142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 0x56365944b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.212+0000 7f0d66000640 1 -- 172.21.15.39:0/3302979142 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563659349670 con 0x563659086390 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.213+0000 7f0d5f400640 1 -- 172.21.15.39:0/3095133650 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0d600e2f60 con 0x7f0d6810f4e0 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.213+0000 7f0d66000640 1 --2- 172.21.15.39:0/3302979142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563659086390 0x56365944c3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0d68111940 tx=0x7f0d54004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec6cb44bf7b8294 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.213+0000 7f0d53e00640 1 -- 172.21.15.39:0/3302979142 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54017020 con 0x563659086390 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.213+0000 7f0d53e00640 1 -- 172.21.15.39:0/3302979142 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d540130d0 con 0x563659086390 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.213+0000 7f0d53e00640 1 -- 172.21.15.39:0/3302979142 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d540035f0 con 0x563659086390 2024-07-04T08:16:12.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.214+0000 7f0d67400640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d4c0b29e0 0x7f0d4c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.214+0000 7f0d74295740 1 -- 172.21.15.39:0/3302979142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563659086390 msgr2=0x56365944c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.214+0000 7f0d67400640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d4c0b29e0 0x7f0d4c0b4ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0d600a9ea0 tx=0x7f0d60005b70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.214+0000 7f0d74295740 1 --2- 172.21.15.39:0/3302979142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563659086390 0x56365944c3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0d68111940 tx=0x7f0d54004730 comp rx=0 tx=0).stop 2024-07-04T08:16:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.215+0000 7f0d74295740 1 -- 172.21.15.39:0/3302979142 shutdown_connections 2024-07-04T08:16:12.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.215+0000 7f0d74295740 1 --2- 172.21.15.39:0/3302979142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563659086390 0x56365944c3f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.215+0000 7f0d74295740 1 --2- 172.21.15.39:0/3302979142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56365907e1b0 0x56365944beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.215+0000 7f0d74295740 1 --2- 172.21.15.39:0/3302979142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 0x56365944b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.215+0000 7f0d74295740 1 -- 172.21.15.39:0/3302979142 >> 172.21.15.39:0/3302979142 conn(0x56365909b460 msgr2=0x563659449860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:12.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.215+0000 7f0d74295740 1 -- 172.21.15.39:0/3302979142 shutdown_connections 2024-07-04T08:16:12.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.215+0000 7f0d74295740 1 -- 172.21.15.39:0/3302979142 wait complete. 2024-07-04T08:16:12.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.216+0000 7f0d74295740 1 -- start start 2024-07-04T08:16:12.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.216+0000 7f0d74295740 1 Processor -- start 2024-07-04T08:16:12.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d74295740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56365907e1b0 0x563659457610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d74295740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 0x563659457bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d74295740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636590659f0 0x5636594634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d74295740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563659215700 con 0x56365907e1b0 2024-07-04T08:16:12.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d74295740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563659358bd0 con 0x5636590659f0 2024-07-04T08:16:12.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d74295740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56365945a150 con 0x5636590a7070 2024-07-04T08:16:12.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d74295740 10 client.0 _ll_register_callbacks cb 0x5636593aa590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:12.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d66000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56365907e1b0 0x563659457610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d66000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56365907e1b0 0x563659457610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54832/0 (socket says 172.21.15.39:54832) 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d66a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 0x563659457bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.217+0000 7f0d66000640 1 -- 172.21.15.39:0/1084795259 learned_addr learned my addr 172.21.15.39:0/1084795259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.218+0000 7f0d66a00640 1 -- 172.21.15.39:0/1084795259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636590659f0 msgr2=0x5636594634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.218+0000 7f0d66a00640 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636590659f0 0x5636594634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.218+0000 7f0d66a00640 1 -- 172.21.15.39:0/1084795259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56365907e1b0 msgr2=0x563659457610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.218+0000 7f0d66a00640 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56365907e1b0 0x563659457610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.218+0000 7f0d66a00640 1 -- 172.21.15.39:0/1084795259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563659112650 con 0x5636590a7070 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.218+0000 7f0d66a00640 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 0x563659457bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x56365944a5f0 tx=0x7f0d580043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b3eec279323c4a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:12.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.218+0000 7f0d50c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:16:12.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.218+0000 7f0d50c00640 1 -- 172.21.15.39:0/1084795259 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58035040 con 0x5636590a7070 2024-07-04T08:16:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.219+0000 7f0d50c00640 1 -- 172.21.15.39:0/1084795259 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0d58003ac0 con 0x5636590a7070 2024-07-04T08:16:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.219+0000 7f0d74295740 1 -- 172.21.15.39:0/1084795259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d540130d0 con 0x5636590a7070 2024-07-04T08:16:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.219+0000 7f0d50c00640 1 -- 172.21.15.39:0/1084795259 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d58003cc0 con 0x5636590a7070 2024-07-04T08:16:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.219+0000 7f0d74295740 10 client.25684 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.219+0000 7f0d74295740 1 -- 172.21.15.39:0/1084795259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563659466250 con 0x5636590a7070 2024-07-04T08:16:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.219+0000 7f0d50c00640 1 -- 172.21.15.39:0/1084795259 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d540130d0 con 0x5636590a7070 2024-07-04T08:16:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.220+0000 7f0d50c00640 1 -- 172.21.15.39:0/1084795259 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0d58004b80 con 0x5636590a7070 2024-07-04T08:16:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.220+0000 7f0d74295740 10 client.25684 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:12.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.220+0000 7f0d74295740 10 client.25684 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:16:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.220+0000 7f0d74295740 20 client.25684 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.220+0000 7f0d74295740 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56365903ada0 0x563659469870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.220+0000 7f0d74295740 4 client.25684 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.220+0000 7f0d74295740 1 -- 172.21.15.39:0/1084795259 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636593c3e50 con 0x56365903ada0 2024-07-04T08:16:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.220+0000 7f0d66000640 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56365903ada0 0x563659469870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.221+0000 7f0d66000640 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56365903ada0 0x563659469870 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b91bb7b93109d330 server_cookie=a61b12255e1ee544 in_seq=0 out_seq=0 2024-07-04T08:16:12.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.221+0000 7f0d50c00640 10 client.25684 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:16:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d50c00640 1 -- 172.21.15.39:0/1084795259 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5636593c3e50 con 0x56365903ada0 2024-07-04T08:16:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d50c00640 10 client.25684 handle_command_reply: tid=0 2024-07-04T08:16:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d74295740 1 client.25684 shutdown 2024-07-04T08:16:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d74295740 1 -- 172.21.15.39:0/1084795259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 msgr2=0x563659457bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d74295740 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 0x563659457bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x56365944a5f0 tx=0x7f0d580043f0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d74295740 1 -- 172.21.15.39:0/1084795259 shutdown_connections 2024-07-04T08:16:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d74295740 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56365903ada0 0x563659469870 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d74295740 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5636590659f0 0x5636594634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d74295740 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 0x563659457bd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d74295740 1 --2- 172.21.15.39:0/1084795259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56365907e1b0 0x563659457610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d50c00640 0 client.25684 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:16:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.222+0000 7f0d74295740 1 -- 172.21.15.39:0/1084795259 >> 172.21.15.39:0/1084795259 conn(0x56365909b460 msgr2=0x56365944eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.223+0000 7f0d74295740 1 -- 172.21.15.39:0/1084795259 shutdown_connections 2024-07-04T08:16:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.223+0000 7f0d74295740 1 -- 172.21.15.39:0/1084795259 wait complete. 2024-07-04T08:16:12.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.223+0000 7f0d74295740 20 client.25684 trim_cache size 0 max 16384 2024-07-04T08:16:12.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.238+0000 7f0d74295740 1 Processor -- start 2024-07-04T08:16:12.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.239+0000 7f0d74295740 1 -- start start 2024-07-04T08:16:12.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.239+0000 7f0d74295740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590659f0 0x5636594487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.239+0000 7f0d74295740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 0x563659463290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.239+0000 7f0d74295740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 0x563659459040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.239+0000 7f0d74295740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d681b22a0 con 0x5636590659f0 2024-07-04T08:16:12.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.239+0000 7f0d74295740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d681b3de0 con 0x56365907e1b0 2024-07-04T08:16:12.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.239+0000 7f0d74295740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d681b2970 con 0x5636590a7070 2024-07-04T08:16:12.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.240+0000 7f0d66000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590659f0 0x5636594487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.240+0000 7f0d66000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590659f0 0x5636594487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54840/0 (socket says 172.21.15.39:54840) 2024-07-04T08:16:12.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.240+0000 7f0d66000640 1 -- 172.21.15.39:0/652276771 learned_addr learned my addr 172.21.15.39:0/652276771 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:12.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.240+0000 7f0d66a00640 1 --2- 172.21.15.39:0/652276771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 0x563659463290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.240+0000 7f0d66000640 1 -- 172.21.15.39:0/652276771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 msgr2=0x563659459040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:12.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.240+0000 7f0d66000640 1 --2- 172.21.15.39:0/652276771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 0x563659459040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.240+0000 7f0d66000640 1 -- 172.21.15.39:0/652276771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 msgr2=0x563659463290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:12.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.240+0000 7f0d66000640 1 --2- 172.21.15.39:0/652276771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 0x563659463290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.240+0000 7f0d66000640 1 -- 172.21.15.39:0/652276771 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365944a150 con 0x5636590659f0 2024-07-04T08:16:12.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.241+0000 7f0d66a00640 1 --2- 172.21.15.39:0/652276771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 0x563659463290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:12.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.241+0000 7f0d66000640 1 --2- 172.21.15.39:0/652276771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590659f0 0x5636594487f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400a010 tx=0x7f0d5401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62350fe18aa1b07 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:12.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.241+0000 7f0d53400640 1 -- 172.21.15.39:0/652276771 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54017040 con 0x5636590659f0 2024-07-04T08:16:12.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.241+0000 7f0d53400640 1 -- 172.21.15.39:0/652276771 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d5400f790 con 0x5636590659f0 2024-07-04T08:16:12.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.241+0000 7f0d53400640 1 -- 172.21.15.39:0/652276771 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54005020 con 0x5636590659f0 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.242+0000 7f0d74295740 1 -- 172.21.15.39:0/652276771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590659f0 msgr2=0x5636594487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.242+0000 7f0d74295740 1 --2- 172.21.15.39:0/652276771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590659f0 0x5636594487f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400a010 tx=0x7f0d5401c040 comp rx=0 tx=0).stop 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.243+0000 7f0d74295740 1 -- 172.21.15.39:0/652276771 shutdown_connections 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.243+0000 7f0d74295740 1 --2- 172.21.15.39:0/652276771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590a7070 0x563659459040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.243+0000 7f0d74295740 1 --2- 172.21.15.39:0/652276771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 0x563659463290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.243+0000 7f0d74295740 1 --2- 172.21.15.39:0/652276771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590659f0 0x5636594487f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.243+0000 7f0d74295740 1 -- 172.21.15.39:0/652276771 >> 172.21.15.39:0/652276771 conn(0x56365909b460 msgr2=0x563659458930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.243+0000 7f0d74295740 1 -- 172.21.15.39:0/652276771 shutdown_connections 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.243+0000 7f0d74295740 1 -- 172.21.15.39:0/652276771 wait complete. 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.243+0000 7f0d74295740 1 -- start start 2024-07-04T08:16:12.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.244+0000 7f0d74295740 1 Processor -- start 2024-07-04T08:16:12.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.244+0000 7f0d74295740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590659f0 0x56365946edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.244+0000 7f0d74295740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 0x56365946f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.244+0000 7f0d74295740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 0x56365946f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.244+0000 7f0d74295740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d681b3710 con 0x5636590a7070 2024-07-04T08:16:12.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.244+0000 7f0d74295740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d681b3040 con 0x56365907e1b0 2024-07-04T08:16:12.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.244+0000 7f0d74295740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d681b0280 con 0x5636590659f0 2024-07-04T08:16:12.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.244+0000 7f0d74295740 10 client.0 _ll_register_callbacks cb 0x7f0d58009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:12.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.245+0000 7f0d66000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590659f0 0x56365946edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.245+0000 7f0d66000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590659f0 0x56365946edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41398/0 (socket says 172.21.15.39:41398) 2024-07-04T08:16:12.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.245+0000 7f0d66000640 1 -- 172.21.15.39:0/382593486 learned_addr learned my addr 172.21.15.39:0/382593486 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:12.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.245+0000 7f0d66000640 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 0x56365946f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.245+0000 7f0d66a00640 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 0x56365946f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d66000640 1 -- 172.21.15.39:0/382593486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590659f0 msgr2=0x56365946edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:12.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d66000640 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590659f0 0x56365946edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d66000640 1 -- 172.21.15.39:0/382593486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 msgr2=0x56365946f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:12.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d66000640 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 0x56365946f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d66000640 1 -- 172.21.15.39:0/382593486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365945b970 con 0x5636590a7070 2024-07-04T08:16:12.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d66000640 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 0x56365946f870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x56365945fbc0 tx=0x7f0d54013440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ca7de92cf74756 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:12.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d50c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:16:12.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d50c00640 1 -- 172.21.15.39:0/382593486 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54039060 con 0x5636590a7070 2024-07-04T08:16:12.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d50c00640 1 -- 172.21.15.39:0/382593486 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0d54005370 con 0x5636590a7070 2024-07-04T08:16:12.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.246+0000 7f0d74295740 1 -- 172.21.15.39:0/382593486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d5400f790 con 0x5636590a7070 2024-07-04T08:16:12.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.247+0000 7f0d50c00640 1 -- 172.21.15.39:0/382593486 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d54040050 con 0x5636590a7070 2024-07-04T08:16:12.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.247+0000 7f0d74295740 10 client.15591 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:12.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.247+0000 7f0d74295740 1 -- 172.21.15.39:0/382593486 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636594dab00 con 0x5636590a7070 2024-07-04T08:16:12.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.247+0000 7f0d50c00640 1 -- 172.21.15.39:0/382593486 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d5400f790 con 0x5636590a7070 2024-07-04T08:16:12.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.247+0000 7f0d50c00640 1 -- 172.21.15.39:0/382593486 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0d54003bf0 con 0x5636590a7070 2024-07-04T08:16:12.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.248+0000 7f0d74295740 10 client.15591 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:12.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.248+0000 7f0d74295740 10 client.15591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:16:12.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.248+0000 7f0d74295740 20 client.15591 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:12.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.248+0000 7f0d74295740 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5636594db000 0x5636594dd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:12.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.248+0000 7f0d74295740 4 client.15591 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:16:12.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.248+0000 7f0d74295740 1 -- 172.21.15.39:0/382593486 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636593bf8f0 con 0x5636594db000 2024-07-04T08:16:12.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.249+0000 7f0d66a00640 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5636594db000 0x5636594dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:12.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.249+0000 7f0d66a00640 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5636594db000 0x5636594dd420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c52fb4b66946b95c server_cookie=55f799600d0fd487 in_seq=0 out_seq=0 2024-07-04T08:16:12.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.249+0000 7f0d50c00640 10 client.15591 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:16:12.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.249+0000 7f0d50c00640 1 -- 172.21.15.39:0/382593486 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636593bf8f0 con 0x5636594db000 2024-07-04T08:16:12.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d50c00640 10 client.15591 handle_command_reply: tid=0 2024-07-04T08:16:12.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d74295740 1 client.15591 shutdown 2024-07-04T08:16:12.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d74295740 1 -- 172.21.15.39:0/382593486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 msgr2=0x56365946f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:12.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d74295740 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 0x56365946f870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x56365945fbc0 tx=0x7f0d54013440 comp rx=0 tx=0).stop 2024-07-04T08:16:12.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d74295740 1 -- 172.21.15.39:0/382593486 shutdown_connections 2024-07-04T08:16:12.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d74295740 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5636594db000 0x5636594dd420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d74295740 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5636590a7070 0x56365946f870 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d74295740 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56365907e1b0 0x56365946f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d50c00640 0 client.15591 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:16:12.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d74295740 1 --2- 172.21.15.39:0/382593486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5636590659f0 0x56365946edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.250+0000 7f0d74295740 1 -- 172.21.15.39:0/382593486 >> 172.21.15.39:0/382593486 conn(0x56365909b460 msgr2=0x5636594485c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:12.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.251+0000 7f0d74295740 1 -- 172.21.15.39:0/382593486 shutdown_connections 2024-07-04T08:16:12.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.251+0000 7f0d74295740 1 -- 172.21.15.39:0/382593486 wait complete. 2024-07-04T08:16:12.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.251+0000 7f0d74295740 20 client.15591 trim_cache size 0 max 16384 2024-07-04T08:16:12.254 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:16:12.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 -- 172.21.15.39:0/3095133650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d4c0b29e0 msgr2=0x7f0d4c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:12.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d4c0b29e0 0x7f0d4c0b4ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0d600a9ea0 tx=0x7f0d60005b70 comp rx=0 tx=0).stop 2024-07-04T08:16:12.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 -- 172.21.15.39:0/3095133650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d6810f4e0 msgr2=0x7f0d681a1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:12.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d6810f4e0 0x7f0d681a1290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009fd0 tx=0x7f0d6000ef10 comp rx=0 tx=0).stop 2024-07-04T08:16:12.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 -- 172.21.15.39:0/3095133650 shutdown_connections 2024-07-04T08:16:12.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d4c0b29e0 0x7f0d4c0b4ef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d680755d0 0x7f0d6819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d6810f4e0 0x7f0d681a1290 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 --2- 172.21.15.39:0/3095133650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d68074c80 0x7f0d6819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:12.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.252+0000 7f0d67e00640 1 -- 172.21.15.39:0/3095133650 >> 172.21.15.39:0/3095133650 conn(0x7f0d680fe950 msgr2=0x7f0d68109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:12.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.253+0000 7f0d67e00640 1 -- 172.21.15.39:0/3095133650 shutdown_connections 2024-07-04T08:16:12.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:12.253+0000 7f0d67e00640 1 -- 172.21.15.39:0/3095133650 wait complete. 2024-07-04T08:16:13.057 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:16:13.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:13 smithi039 ceph-mon[32062]: pgmap v510: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:16:13.369 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:13 smithi133 ceph-mon[37050]: pgmap v510: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:16:13.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:13 smithi050 ceph-mon[36577]: pgmap v510: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:16:13.596 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:15.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:15 smithi039 ceph-mon[32062]: pgmap v511: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-07-04T08:16:15.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:15 smithi050 ceph-mon[36577]: pgmap v511: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-07-04T08:16:15.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:15 smithi133 ceph-mon[37050]: pgmap v511: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-07-04T08:16:15.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.851+0000 7f4567e00640 1 -- 172.21.15.39:0/1722449369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456810f460 msgr2=0x7f45681118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:15.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.851+0000 7f4567e00640 1 --2- 172.21.15.39:0/1722449369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456810f460 0x7f45681118a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4560009be0 tx=0x7f456002a6c0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.852+0000 7f4567e00640 1 -- 172.21.15.39:0/1722449369 shutdown_connections 2024-07-04T08:16:15.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.852+0000 7f4567e00640 1 --2- 172.21.15.39:0/1722449369 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4568075650 0x7f456810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.852+0000 7f4567e00640 1 --2- 172.21.15.39:0/1722449369 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456810f460 0x7f45681118a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.852+0000 7f4567e00640 1 --2- 172.21.15.39:0/1722449369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4568074d00 0x7f4568075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.852+0000 7f4567e00640 1 -- 172.21.15.39:0/1722449369 >> 172.21.15.39:0/1722449369 conn(0x7f45680fe990 msgr2=0x7f4568100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:15.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.852+0000 7f4567e00640 1 -- 172.21.15.39:0/1722449369 shutdown_connections 2024-07-04T08:16:15.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.853+0000 7f4567e00640 1 -- 172.21.15.39:0/1722449369 wait complete. 2024-07-04T08:16:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.854+0000 7f4567e00640 1 Processor -- start 2024-07-04T08:16:15.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.854+0000 7f4567e00640 1 -- start start 2024-07-04T08:16:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4567e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4568074d00 0x7f456819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4567e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4568075650 0x7f456819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4567e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456810f460 0x7f45681a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4567e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4568114ff0 con 0x7f4568075650 2024-07-04T08:16:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4567e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4568114e70 con 0x7f456810f460 2024-07-04T08:16:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4567e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4568115170 con 0x7f4568074d00 2024-07-04T08:16:15.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4566a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4568074d00 0x7f456819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4566000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4568075650 0x7f456819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4566a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4568074d00 0x7f456819b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41422/0 (socket says 172.21.15.39:41422) 2024-07-04T08:16:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.855+0000 7f4566a00640 1 -- 172.21.15.39:0/2397060198 learned_addr learned my addr 172.21.15.39:0/2397060198 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:15.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.856+0000 7f4567400640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456810f460 0x7f45681a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.856+0000 7f4566000640 1 -- 172.21.15.39:0/2397060198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4568074d00 msgr2=0x7f456819b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.856+0000 7f4566000640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4568074d00 0x7f456819b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.856+0000 7f4566000640 1 -- 172.21.15.39:0/2397060198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456810f460 msgr2=0x7f45681a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.856+0000 7f4566000640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456810f460 0x7f45681a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.856+0000 7f4566000640 1 -- 172.21.15.39:0/2397060198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45681a1930 con 0x7f4568075650 2024-07-04T08:16:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.856+0000 7f4566000640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4568075650 0x7f456819bf60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f456806ac50 tx=0x7f455400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.857+0000 7f455f400640 1 -- 172.21.15.39:0/2397060198 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554004440 con 0x7f4568075650 2024-07-04T08:16:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.857+0000 7f455f400640 1 -- 172.21.15.39:0/2397060198 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f455400cd70 con 0x7f4568075650 2024-07-04T08:16:15.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.857+0000 7f455f400640 1 -- 172.21.15.39:0/2397060198 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554019480 con 0x7f4568075650 2024-07-04T08:16:15.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.857+0000 7f4567e00640 1 -- 172.21.15.39:0/2397060198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45681a1c20 con 0x7f4568075650 2024-07-04T08:16:15.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.857+0000 7f4567e00640 1 -- 172.21.15.39:0/2397060198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45681a8380 con 0x7f4568075650 2024-07-04T08:16:15.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.862+0000 7f4573ca9740 1 Processor -- start 2024-07-04T08:16:15.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.862+0000 7f4573ca9740 1 -- start start 2024-07-04T08:16:15.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.863+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 0x55668ad77970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.863+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9aa1b0 0x55668ad77eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.863+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9b2390 0x55668ad783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.863+0000 7f4573ca9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55668acedf00 con 0x55668a9b2390 2024-07-04T08:16:15.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.863+0000 7f4573ca9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55668acdd4a0 con 0x55668a9d3070 2024-07-04T08:16:15.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.863+0000 7f4573ca9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55668ace14e0 con 0x55668a9aa1b0 2024-07-04T08:16:15.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.863+0000 7f4566a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9aa1b0 0x55668ad77eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.864+0000 7f4566a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9aa1b0 0x55668ad77eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41432/0 (socket says 172.21.15.39:41432) 2024-07-04T08:16:15.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.864+0000 7f4566a00640 1 -- 172.21.15.39:0/1199113650 learned_addr learned my addr 172.21.15.39:0/1199113650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:15.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.864+0000 7f4567400640 1 --2- 172.21.15.39:0/1199113650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 0x55668ad77970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.864+0000 7f455f400640 1 -- 172.21.15.39:0/2397060198 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4554019620 con 0x7f4568075650 2024-07-04T08:16:15.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.864+0000 7f4566a00640 1 -- 172.21.15.39:0/1199113650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 msgr2=0x55668ad77970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:15.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.864+0000 7f4566a00640 1 --2- 172.21.15.39:0/1199113650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 0x55668ad77970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.864+0000 7f4566a00640 1 -- 172.21.15.39:0/1199113650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9b2390 msgr2=0x55668ad783f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:15.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.864+0000 7f4566a00640 1 --2- 172.21.15.39:0/1199113650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9b2390 0x55668ad783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.864+0000 7f4566a00640 1 -- 172.21.15.39:0/1199113650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668ac75670 con 0x55668a9aa1b0 2024-07-04T08:16:15.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.865+0000 7f4566a00640 1 --2- 172.21.15.39:0/1199113650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9aa1b0 0x55668ad77eb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4558009dc0 tx=0x7f4558011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be9012fd4f600670 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:15.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.865+0000 7f4553e00640 1 -- 172.21.15.39:0/1199113650 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558005220 con 0x55668a9aa1b0 2024-07-04T08:16:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.867+0000 7f455f400640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f454c0b29e0 0x7f454c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.867+0000 7f4553e00640 1 -- 172.21.15.39:0/1199113650 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4558004c50 con 0x55668a9aa1b0 2024-07-04T08:16:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.867+0000 7f4553e00640 1 -- 172.21.15.39:0/1199113650 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45580197f0 con 0x55668a9aa1b0 2024-07-04T08:16:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.867+0000 7f455f400640 1 -- 172.21.15.39:0/2397060198 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f45540e2d70 con 0x7f4568075650 2024-07-04T08:16:15.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.867+0000 7f4567400640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f454c0b29e0 0x7f454c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.868+0000 7f4573ca9740 1 -- 172.21.15.39:0/1199113650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9aa1b0 msgr2=0x55668ad77eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:15.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.868+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1199113650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9aa1b0 0x55668ad77eb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4558009dc0 tx=0x7f4558011810 comp rx=0 tx=0).stop 2024-07-04T08:16:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.869+0000 7f4567400640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f454c0b29e0 0x7f454c0b4ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4560009370 tx=0x7f4560004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.869+0000 7f4573ca9740 1 -- 172.21.15.39:0/1199113650 shutdown_connections 2024-07-04T08:16:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.869+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1199113650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9b2390 0x55668ad783f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.869+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1199113650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9aa1b0 0x55668ad77eb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.869+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1199113650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 0x55668ad77970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.869+0000 7f4573ca9740 1 -- 172.21.15.39:0/1199113650 >> 172.21.15.39:0/1199113650 conn(0x55668a9c7460 msgr2=0x55668ad75860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:15.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.870+0000 7f4573ca9740 1 -- 172.21.15.39:0/1199113650 shutdown_connections 2024-07-04T08:16:15.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.870+0000 7f4573ca9740 1 -- 172.21.15.39:0/1199113650 wait complete. 2024-07-04T08:16:15.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.871+0000 7f4573ca9740 1 -- start start 2024-07-04T08:16:15.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.871+0000 7f4573ca9740 1 Processor -- start 2024-07-04T08:16:15.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 0x55668ad835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad83b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 0x55668ad8f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4573ca9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55668ab41700 con 0x55668a9aa1b0 2024-07-04T08:16:15.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4573ca9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55668ac84bd0 con 0x55668a9d3070 2024-07-04T08:16:15.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4573ca9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55668ad86150 con 0x55668a9b2390 2024-07-04T08:16:15.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4573ca9740 10 client.0 _ll_register_callbacks cb 0x55668acd6590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:15.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4566000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad83b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4566000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad83b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41448/0 (socket says 172.21.15.39:41448) 2024-07-04T08:16:15.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.872+0000 7f4566000640 1 -- 172.21.15.39:0/1724809510 learned_addr learned my addr 172.21.15.39:0/1724809510 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:15.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.873+0000 7f4566a00640 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 0x55668ad835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.873+0000 7f4567400640 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 0x55668ad8f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.873+0000 7f4566000640 1 -- 172.21.15.39:0/1724809510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 msgr2=0x55668ad8f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:15.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.873+0000 7f4566000640 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 0x55668ad8f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.873+0000 7f4566000640 1 -- 172.21.15.39:0/1724809510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 msgr2=0x55668ad835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:15.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.873+0000 7f4566000640 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 0x55668ad835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.873+0000 7f4566000640 1 -- 172.21.15.39:0/1724809510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668aa3e650 con 0x55668a9b2390 2024-07-04T08:16:15.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.874+0000 7f4567400640 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 0x55668ad8f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:15.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.874+0000 7f4566a00640 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 0x55668ad835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:15.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.874+0000 7f4566000640 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad83b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f45540a99c0 tx=0x7f45540a94f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca6c1a37d4af264 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:15.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.874+0000 7f4550c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:16:15.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.875+0000 7f4550c00640 1 -- 172.21.15.39:0/1724809510 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540c1070 con 0x55668a9b2390 2024-07-04T08:16:15.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.875+0000 7f4550c00640 1 -- 172.21.15.39:0/1724809510 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f45540e03b0 con 0x55668a9b2390 2024-07-04T08:16:15.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.875+0000 7f4573ca9740 1 -- 172.21.15.39:0/1724809510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4558004c50 con 0x55668a9b2390 2024-07-04T08:16:15.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.875+0000 7f4550c00640 1 -- 172.21.15.39:0/1724809510 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4554009510 con 0x55668a9b2390 2024-07-04T08:16:15.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.875+0000 7f4573ca9740 10 client.25699 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:15.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.876+0000 7f4573ca9740 1 -- 172.21.15.39:0/1724809510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55668ad922a0 con 0x55668a9b2390 2024-07-04T08:16:15.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.876+0000 7f4550c00640 1 -- 172.21.15.39:0/1724809510 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4558004c50 con 0x55668a9b2390 2024-07-04T08:16:15.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.876+0000 7f4550c00640 1 -- 172.21.15.39:0/1724809510 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f45540c4020 con 0x55668a9b2390 2024-07-04T08:16:15.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.877+0000 7f4573ca9740 10 client.25699 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:15.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.877+0000 7f4573ca9740 10 client.25699 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:16:15.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.877+0000 7f4573ca9740 20 client.25699 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:15.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.877+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55668a9919f0 0x55668ad956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.877+0000 7f4573ca9740 4 client.25699 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:15.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.877+0000 7f4573ca9740 1 -- 172.21.15.39:0/1724809510 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55668acefe50 con 0x55668a9919f0 2024-07-04T08:16:15.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.877+0000 7f4566a00640 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55668a9919f0 0x55668ad956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.878+0000 7f4566a00640 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55668a9919f0 0x55668ad956c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d680d8c5f02f67c0 server_cookie=2cb86dcc4d8013c9 in_seq=0 out_seq=0 2024-07-04T08:16:15.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.878+0000 7f4550c00640 10 client.25699 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:16:15.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.879+0000 7f4550c00640 1 -- 172.21.15.39:0/1724809510 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55668acefe50 con 0x55668a9919f0 2024-07-04T08:16:15.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.879+0000 7f4550c00640 10 client.25699 handle_command_reply: tid=0 2024-07-04T08:16:15.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.879+0000 7f4573ca9740 1 client.25699 shutdown 2024-07-04T08:16:15.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.880+0000 7f4573ca9740 1 -- 172.21.15.39:0/1724809510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 msgr2=0x55668ad83b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.880+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad83b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f45540a99c0 tx=0x7f45540a94f0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.880+0000 7f4573ca9740 1 -- 172.21.15.39:0/1724809510 shutdown_connections 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.880+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55668a9919f0 0x55668ad956c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.880+0000 7f4550c00640 0 client.25699 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.880+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9d3070 0x55668ad8f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.880+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad83b50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.880+0000 7f4573ca9740 1 --2- 172.21.15.39:0/1724809510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 0x55668ad835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.880+0000 7f4573ca9740 1 -- 172.21.15.39:0/1724809510 >> 172.21.15.39:0/1724809510 conn(0x55668a9c7460 msgr2=0x55668ad7ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.881+0000 7f4573ca9740 1 -- 172.21.15.39:0/1724809510 shutdown_connections 2024-07-04T08:16:15.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.881+0000 7f4573ca9740 1 -- 172.21.15.39:0/1724809510 wait complete. 2024-07-04T08:16:15.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.881+0000 7f4573ca9740 20 client.25699 trim_cache size 0 max 16384 2024-07-04T08:16:15.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.910+0000 7f4573ca9740 1 Processor -- start 2024-07-04T08:16:15.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.910+0000 7f4573ca9740 1 -- start start 2024-07-04T08:16:15.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9919f0 0x55668ad853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9aa1b0 0x55668ad8f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9b2390 0x55668ad7cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4573ca9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b2270 con 0x55668a9b2390 2024-07-04T08:16:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4573ca9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b3db0 con 0x55668a9aa1b0 2024-07-04T08:16:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4573ca9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b2940 con 0x55668a9919f0 2024-07-04T08:16:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4566000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9aa1b0 0x55668ad8f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4566000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9aa1b0 0x55668ad8f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33860/0 (socket says 172.21.15.39:33860) 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4566000640 1 -- 172.21.15.39:0/4284177175 learned_addr learned my addr 172.21.15.39:0/4284177175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4566a00640 1 --2- 172.21.15.39:0/4284177175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9919f0 0x55668ad853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4566000640 1 -- 172.21.15.39:0/4284177175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9919f0 msgr2=0x55668ad853c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4566000640 1 --2- 172.21.15.39:0/4284177175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9919f0 0x55668ad853c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4566000640 1 -- 172.21.15.39:0/4284177175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9b2390 msgr2=0x55668ad7cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.911+0000 7f4566000640 1 --2- 172.21.15.39:0/4284177175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9b2390 0x55668ad7cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.912+0000 7f4566000640 1 -- 172.21.15.39:0/4284177175 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668ad79c60 con 0x55668a9aa1b0 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.912+0000 7f4566000640 1 --2- 172.21.15.39:0/4284177175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9aa1b0 0x55668ad8f2c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45540c6040 tx=0x7f45540a9560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fca17eb9aa9107b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.912+0000 7f4553400640 1 -- 172.21.15.39:0/4284177175 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455408a070 con 0x55668a9aa1b0 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.912+0000 7f4553400640 1 -- 172.21.15.39:0/4284177175 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45540b28d0 con 0x55668a9aa1b0 2024-07-04T08:16:15.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.912+0000 7f4553400640 1 -- 172.21.15.39:0/4284177175 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540c1070 con 0x55668a9aa1b0 2024-07-04T08:16:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.913+0000 7f4573ca9740 1 -- 172.21.15.39:0/4284177175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9aa1b0 msgr2=0x55668ad8f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.913+0000 7f4573ca9740 1 --2- 172.21.15.39:0/4284177175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9aa1b0 0x55668ad8f2c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f45540c6040 tx=0x7f45540a9560 comp rx=0 tx=0).stop 2024-07-04T08:16:15.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.913+0000 7f4573ca9740 1 -- 172.21.15.39:0/4284177175 shutdown_connections 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.913+0000 7f4573ca9740 1 --2- 172.21.15.39:0/4284177175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9b2390 0x55668ad7cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.913+0000 7f4573ca9740 1 --2- 172.21.15.39:0/4284177175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9aa1b0 0x55668ad8f2c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.913+0000 7f4573ca9740 1 --2- 172.21.15.39:0/4284177175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9919f0 0x55668ad853c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.913+0000 7f4573ca9740 1 -- 172.21.15.39:0/4284177175 >> 172.21.15.39:0/4284177175 conn(0x55668a9c7460 msgr2=0x55668ad79360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.914+0000 7f4573ca9740 1 -- 172.21.15.39:0/4284177175 shutdown_connections 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.914+0000 7f4573ca9740 1 -- 172.21.15.39:0/4284177175 wait complete. 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.914+0000 7f4573ca9740 1 -- start start 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.915+0000 7f4573ca9740 1 Processor -- start 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.915+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9919f0 0x55668ad74610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.915+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 0x55668ad74bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.915+0000 7f4573ca9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad8a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.915+0000 7f4573ca9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b36e0 con 0x55668a9aa1b0 2024-07-04T08:16:15.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.916+0000 7f4573ca9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b3010 con 0x55668a9919f0 2024-07-04T08:16:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.916+0000 7f4573ca9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45681b0250 con 0x55668a9b2390 2024-07-04T08:16:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.916+0000 7f4573ca9740 10 client.0 _ll_register_callbacks cb 0x7f45540a9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.916+0000 7f4566000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 0x55668ad74bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.916+0000 7f4566a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9919f0 0x55668ad74610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.916+0000 7f4566a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9919f0 0x55668ad74610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33876/0 (socket says 172.21.15.39:33876) 2024-07-04T08:16:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.916+0000 7f4566a00640 1 -- 172.21.15.39:0/3364671151 learned_addr learned my addr 172.21.15.39:0/3364671151 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:15.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.916+0000 7f4567400640 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad8a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.917+0000 7f4566a00640 1 -- 172.21.15.39:0/3364671151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 msgr2=0x55668ad8a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.917+0000 7f4566a00640 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad8a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.917+0000 7f4566a00640 1 -- 172.21.15.39:0/3364671151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 msgr2=0x55668ad74bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.917+0000 7f4566a00640 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 0x55668ad74bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.917+0000 7f4566a00640 1 -- 172.21.15.39:0/3364671151 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668ad8aaa0 con 0x55668a9919f0 2024-07-04T08:16:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.917+0000 7f4566a00640 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9919f0 0x55668ad74610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4558011d10 tx=0x7f45580056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84ffe8ad660c1cf9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.917+0000 7f4550c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:15.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.918+0000 7f4550c00640 1 -- 172.21.15.39:0/3364671151 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455802f070 con 0x55668a9919f0 2024-07-04T08:16:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.918+0000 7f4550c00640 1 -- 172.21.15.39:0/3364671151 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4558004450 con 0x55668a9919f0 2024-07-04T08:16:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.918+0000 7f4573ca9740 1 -- 172.21.15.39:0/3364671151 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45540b28d0 con 0x55668a9919f0 2024-07-04T08:16:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.918+0000 7f4550c00640 1 -- 172.21.15.39:0/3364671151 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4558004fc0 con 0x55668a9919f0 2024-07-04T08:16:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.918+0000 7f4573ca9740 10 client.25724 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:15.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.918+0000 7f4573ca9740 1 -- 172.21.15.39:0/3364671151 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55668ad8bd20 con 0x55668a9919f0 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.919+0000 7f4550c00640 1 -- 172.21.15.39:0/3364671151 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45540b28d0 con 0x55668a9919f0 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.919+0000 7f4550c00640 1 -- 172.21.15.39:0/3364671151 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4558003a00 con 0x55668a9919f0 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.919+0000 7f4573ca9740 10 client.25724 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.919+0000 7f4573ca9740 10 client.25724 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.919+0000 7f4573ca9740 20 client.25724 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.919+0000 7f4573ca9740 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55668a9d3070 0x55668adffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.920+0000 7f4573ca9740 4 client.25724 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.920+0000 7f4573ca9740 1 -- 172.21.15.39:0/3364671151 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55668aceb8f0 con 0x55668a9d3070 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.920+0000 7f4566000640 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55668a9d3070 0x55668adffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.920+0000 7f4550c00640 10 client.25724 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.920+0000 7f4566000640 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55668a9d3070 0x55668adffe60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62c2fecbd4c176df server_cookie=1cc239bfb3b6297 in_seq=0 out_seq=0 2024-07-04T08:16:15.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4550c00640 1 -- 172.21.15.39:0/3364671151 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55668aceb8f0 con 0x55668a9d3070 2024-07-04T08:16:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4550c00640 10 client.25724 handle_command_reply: tid=0 2024-07-04T08:16:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4573ca9740 1 client.25724 shutdown 2024-07-04T08:16:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4573ca9740 1 -- 172.21.15.39:0/3364671151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9919f0 msgr2=0x55668ad74610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4573ca9740 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9919f0 0x55668ad74610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4558011d10 tx=0x7f45580056b0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4573ca9740 1 -- 172.21.15.39:0/3364671151 shutdown_connections 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4573ca9740 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55668a9d3070 0x55668adffe60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4573ca9740 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55668a9aa1b0 0x55668ad74bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4573ca9740 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55668a9b2390 0x55668ad8a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4573ca9740 1 --2- 172.21.15.39:0/3364671151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55668a9919f0 0x55668ad74610 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4573ca9740 1 -- 172.21.15.39:0/3364671151 >> 172.21.15.39:0/3364671151 conn(0x55668a9c7460 msgr2=0x55668ad79360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.921+0000 7f4550c00640 0 client.25724 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.922+0000 7f4573ca9740 1 -- 172.21.15.39:0/3364671151 shutdown_connections 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.922+0000 7f4573ca9740 1 -- 172.21.15.39:0/3364671151 wait complete. 2024-07-04T08:16:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.922+0000 7f4573ca9740 20 client.25724 trim_cache size 0 max 16384 2024-07-04T08:16:15.926 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:16:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 -- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f454c0b29e0 msgr2=0x7f454c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f454c0b29e0 0x7f454c0b4ef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4560009370 tx=0x7f4560004a40 comp rx=0 tx=0).stop 2024-07-04T08:16:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 -- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4568075650 msgr2=0x7f456819bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4568075650 0x7f456819bf60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f456806ac50 tx=0x7f455400ee50 comp rx=0 tx=0).stop 2024-07-04T08:16:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 -- 172.21.15.39:0/2397060198 shutdown_connections 2024-07-04T08:16:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f454c0b29e0 0x7f454c0b4ef0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4568075650 0x7f456819bf60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456810f460 0x7f45681a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 --2- 172.21.15.39:0/2397060198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4568074d00 0x7f456819b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 -- 172.21.15.39:0/2397060198 >> 172.21.15.39:0/2397060198 conn(0x7f45680fe990 msgr2=0x7f4568109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 -- 172.21.15.39:0/2397060198 shutdown_connections 2024-07-04T08:16:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:15.923+0000 7f4567e00640 1 -- 172.21.15.39:0/2397060198 wait complete. 2024-07-04T08:16:16.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:16:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:17.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:17 smithi050 ceph-mon[36577]: pgmap v512: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-04T08:16:17.491 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:17 smithi039 ceph-mon[32062]: pgmap v512: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-04T08:16:17.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:17 smithi133 ceph-mon[37050]: pgmap v512: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-07-04T08:16:19.361 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:19 smithi039 ceph-mon[32062]: pgmap v513: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-07-04T08:16:19.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:19 smithi050 ceph-mon[36577]: pgmap v513: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-07-04T08:16:19.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:19 smithi133 ceph-mon[37050]: pgmap v513: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-07-04T08:16:19.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.694+0000 7f34d2800640 1 -- 172.21.15.39:0/1548303958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc104660 msgr2=0x7f34cc10cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.694+0000 7f34d2800640 1 --2- 172.21.15.39:0/1548303958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc104660 0x7f34cc10cbb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009900 tx=0x7f34b802a410 comp rx=0 tx=0).stop 2024-07-04T08:16:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.695+0000 7f34d2800640 1 -- 172.21.15.39:0/1548303958 shutdown_connections 2024-07-04T08:16:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.695+0000 7f34d2800640 1 --2- 172.21.15.39:0/1548303958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc10d170 0x7f34cc10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.695+0000 7f34d2800640 1 --2- 172.21.15.39:0/1548303958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc104660 0x7f34cc10cbb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.695+0000 7f34d2800640 1 --2- 172.21.15.39:0/1548303958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34cc103d10 0x7f34cc104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.695+0000 7f34d2800640 1 -- 172.21.15.39:0/1548303958 >> 172.21.15.39:0/1548303958 conn(0x7f34cc0fd810 msgr2=0x7f34cc0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.695+0000 7f34d2800640 1 -- 172.21.15.39:0/1548303958 shutdown_connections 2024-07-04T08:16:19.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.695+0000 7f34d2800640 1 -- 172.21.15.39:0/1548303958 wait complete. 2024-07-04T08:16:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.696+0000 7f34d2800640 1 Processor -- start 2024-07-04T08:16:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.696+0000 7f34d2800640 1 -- start start 2024-07-04T08:16:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc103d10 0x7f34cc19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc104660 0x7f34cc19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34cc10d170 0x7f34cc1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d2800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc112dd0 con 0x7f34cc104660 2024-07-04T08:16:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d2800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc112c50 con 0x7f34cc103d10 2024-07-04T08:16:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d2800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc112f50 con 0x7f34cc10d170 2024-07-04T08:16:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc103d10 0x7f34cc19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc103d10 0x7f34cc19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33900/0 (socket says 172.21.15.39:33900) 2024-07-04T08:16:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d1400640 1 -- 172.21.15.39:0/3540377247 learned_addr learned my addr 172.21.15.39:0/3540377247 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:19.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.697+0000 7f34d1e00640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34cc10d170 0x7f34cc1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.698+0000 7f34d0a00640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc104660 0x7f34cc19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.698+0000 7f34d1e00640 1 -- 172.21.15.39:0/3540377247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc103d10 msgr2=0x7f34cc19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.698+0000 7f34d1e00640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc103d10 0x7f34cc19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.698+0000 7f34d1e00640 1 -- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc104660 msgr2=0x7f34cc19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.698+0000 7f34d1e00640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc104660 0x7f34cc19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.698+0000 7f34d1e00640 1 -- 172.21.15.39:0/3540377247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc1a1970 con 0x7f34cc10d170 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.698+0000 7f34d1e00640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34cc10d170 0x7f34cc1a1240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009fd0 tx=0x7f34c400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.699+0000 7f34d1400640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc103d10 0x7f34cc19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.699+0000 7f34ca000640 1 -- 172.21.15.39:0/3540377247 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c400ce20 con 0x7f34cc10d170 2024-07-04T08:16:19.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.699+0000 7f34ca000640 1 -- 172.21.15.39:0/3540377247 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c4005020 con 0x7f34cc10d170 2024-07-04T08:16:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.699+0000 7f34ca000640 1 -- 172.21.15.39:0/3540377247 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4010670 con 0x7f34cc10d170 2024-07-04T08:16:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.700+0000 7f34d2800640 1 -- 172.21.15.39:0/3540377247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc1a1c60 con 0x7f34cc10d170 2024-07-04T08:16:19.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.700+0000 7f34d2800640 1 -- 172.21.15.39:0/3540377247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc1a83c0 con 0x7f34cc10d170 2024-07-04T08:16:19.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.704+0000 7f34da45d740 1 Processor -- start 2024-07-04T08:16:19.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.705+0000 7f34ca000640 1 -- 172.21.15.39:0/3540377247 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f34c40054b0 con 0x7f34cc10d170 2024-07-04T08:16:19.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.706+0000 7f34da45d740 1 -- start start 2024-07-04T08:16:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.706+0000 7f34da45d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e5468070 0x5614e580c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.706+0000 7f34da45d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e543f1b0 0x5614e580ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.706+0000 7f34da45d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e580d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.706+0000 7f34da45d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5614e5782ee0 con 0x5614e5468070 2024-07-04T08:16:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.706+0000 7f34da45d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5614e5772480 con 0x5614e5447390 2024-07-04T08:16:19.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.706+0000 7f34da45d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5614e57764c0 con 0x5614e543f1b0 2024-07-04T08:16:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.707+0000 7f34d1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e543f1b0 0x5614e580ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.707+0000 7f34d1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e543f1b0 0x5614e580ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41478/0 (socket says 172.21.15.39:41478) 2024-07-04T08:16:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.707+0000 7f34d1400640 1 -- 172.21.15.39:0/2447927269 learned_addr learned my addr 172.21.15.39:0/2447927269 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.707+0000 7f34d0a00640 1 --2- 172.21.15.39:0/2447927269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e580d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.707+0000 7f34d0a00640 1 --2- 172.21.15.39:0/2447927269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e5468070 0x5614e580c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.708+0000 7f34d0a00640 1 -- 172.21.15.39:0/2447927269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e543f1b0 msgr2=0x5614e580ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.708+0000 7f34d0a00640 1 --2- 172.21.15.39:0/2447927269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e543f1b0 0x5614e580ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.708+0000 7f34d0a00640 1 -- 172.21.15.39:0/2447927269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 msgr2=0x5614e580d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.708+0000 7f34d0a00640 1 --2- 172.21.15.39:0/2447927269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e580d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.708+0000 7f34d0a00640 1 -- 172.21.15.39:0/2447927269 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e570a650 con 0x5614e5468070 2024-07-04T08:16:19.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.708+0000 7f34ca000640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34b40b29e0 0x7f34b40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.708+0000 7f34ca000640 1 -- 172.21.15.39:0/3540377247 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f34c40e2eb0 con 0x7f34cc10d170 2024-07-04T08:16:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.708+0000 7f34d0a00640 1 --2- 172.21.15.39:0/2447927269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e5468070 0x5614e580c950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f34b802a930 tx=0x7f34b8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf74cd02bccae33 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.709+0000 7f34d1e00640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34b40b29e0 0x7f34b40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.709+0000 7f34bea00640 1 -- 172.21.15.39:0/2447927269 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8041070 con 0x5614e5468070 2024-07-04T08:16:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.709+0000 7f34bea00640 1 -- 172.21.15.39:0/2447927269 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b80044d0 con 0x5614e5468070 2024-07-04T08:16:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.709+0000 7f34bea00640 1 -- 172.21.15.39:0/2447927269 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8035070 con 0x5614e5468070 2024-07-04T08:16:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34d1e00640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34b40b29e0 0x7f34b40b4ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009230 tx=0x7f34c4005f70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34da45d740 1 -- 172.21.15.39:0/2447927269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e5468070 msgr2=0x5614e580c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:19.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34da45d740 1 --2- 172.21.15.39:0/2447927269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e5468070 0x5614e580c950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f34b802a930 tx=0x7f34b8031400 comp rx=0 tx=0).stop 2024-07-04T08:16:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34da45d740 1 -- 172.21.15.39:0/2447927269 shutdown_connections 2024-07-04T08:16:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34da45d740 1 --2- 172.21.15.39:0/2447927269 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e580d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34da45d740 1 --2- 172.21.15.39:0/2447927269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e543f1b0 0x5614e580ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34da45d740 1 --2- 172.21.15.39:0/2447927269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e5468070 0x5614e580c950 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34da45d740 1 -- 172.21.15.39:0/2447927269 >> 172.21.15.39:0/2447927269 conn(0x5614e545c460 msgr2=0x5614e580a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34da45d740 1 -- 172.21.15.39:0/2447927269 shutdown_connections 2024-07-04T08:16:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.711+0000 7f34da45d740 1 -- 172.21.15.39:0/2447927269 wait complete. 2024-07-04T08:16:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.712+0000 7f34da45d740 1 -- start start 2024-07-04T08:16:19.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.712+0000 7f34da45d740 1 Processor -- start 2024-07-04T08:16:19.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.713+0000 7f34da45d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5820d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.713+0000 7f34da45d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 0x5614e58212f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.713+0000 7f34da45d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 0x5614e582cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.713+0000 7f34da45d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5614e55d6700 con 0x5614e543f1b0 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.713+0000 7f34da45d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5614e5719bb0 con 0x5614e5468070 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.713+0000 7f34da45d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5614e5823800 con 0x5614e5447390 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.713+0000 7f34da45d740 10 client.0 _ll_register_callbacks cb 0x5614e576b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.714+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5820d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.714+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 0x5614e582cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.714+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 0x5614e582cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33912/0 (socket says 172.21.15.39:33912) 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.714+0000 7f34d0a00640 1 -- 172.21.15.39:0/1054616837 learned_addr learned my addr 172.21.15.39:0/1054616837 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:19.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.714+0000 7f34d1400640 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 0x5614e58212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.715+0000 7f34d1400640 1 -- 172.21.15.39:0/1054616837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 msgr2=0x5614e582cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.715+0000 7f34d1400640 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 0x5614e582cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.715+0000 7f34d1400640 1 -- 172.21.15.39:0/1054616837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 msgr2=0x5614e5820d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.715+0000 7f34d1400640 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5820d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.715+0000 7f34d1400640 1 -- 172.21.15.39:0/1054616837 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e54d3650 con 0x5614e5447390 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.715+0000 7f34d0a00640 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5820d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.716+0000 7f34d1400640 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 0x5614e58212f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f34c0012cf0 tx=0x7f34c000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d40bbce74d24b08 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.716+0000 7f34b3400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.716+0000 7f34b3400640 1 -- 172.21.15.39:0/1054616837 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0005020 con 0x5614e5447390 2024-07-04T08:16:19.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.716+0000 7f34da45d740 1 -- 172.21.15.39:0/1054616837 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b80044d0 con 0x5614e5447390 2024-07-04T08:16:19.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.717+0000 7f34b3400640 1 -- 172.21.15.39:0/1054616837 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f34c00051c0 con 0x5614e5447390 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.717+0000 7f34b3400640 1 -- 172.21.15.39:0/1054616837 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34c00053c0 con 0x5614e5447390 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.717+0000 7f34da45d740 10 client.25717 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.717+0000 7f34da45d740 1 -- 172.21.15.39:0/1054616837 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5614e582f950 con 0x5614e5447390 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.717+0000 7f34b3400640 1 -- 172.21.15.39:0/1054616837 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b80044d0 con 0x5614e5447390 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.718+0000 7f34b3400640 1 -- 172.21.15.39:0/1054616837 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f34c0010430 con 0x5614e5447390 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.718+0000 7f34da45d740 10 client.25717 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.718+0000 7f34da45d740 10 client.25717 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.718+0000 7f34da45d740 20 client.25717 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.718+0000 7f34da45d740 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5614e54269f0 0x5614e5832eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.718+0000 7f34da45d740 4 client.25717 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:19.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.718+0000 7f34da45d740 1 -- 172.21.15.39:0/1054616837 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614e5784e30 con 0x5614e54269f0 2024-07-04T08:16:19.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.719+0000 7f34d0a00640 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5614e54269f0 0x5614e5832eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.720+0000 7f34d0a00640 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5614e54269f0 0x5614e5832eb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf006b71ed1d3610 server_cookie=2fa7a6fe10eaf39a in_seq=0 out_seq=0 2024-07-04T08:16:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.720+0000 7f34b3400640 10 client.25717 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:16:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34b3400640 1 -- 172.21.15.39:0/1054616837 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5614e5784e30 con 0x5614e54269f0 2024-07-04T08:16:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34b3400640 10 client.25717 handle_command_reply: tid=0 2024-07-04T08:16:19.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34da45d740 1 client.25717 shutdown 2024-07-04T08:16:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34da45d740 1 -- 172.21.15.39:0/1054616837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 msgr2=0x5614e58212f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34da45d740 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 0x5614e58212f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f34c0012cf0 tx=0x7f34c000f790 comp rx=0 tx=0).stop 2024-07-04T08:16:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34da45d740 1 -- 172.21.15.39:0/1054616837 shutdown_connections 2024-07-04T08:16:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34da45d740 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5614e54269f0 0x5614e5832eb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34da45d740 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 0x5614e582cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34da45d740 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 0x5614e58212f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34da45d740 1 --2- 172.21.15.39:0/1054616837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5820d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34da45d740 1 -- 172.21.15.39:0/1054616837 >> 172.21.15.39:0/1054616837 conn(0x5614e545c460 msgr2=0x5614e580fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:19.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.721+0000 7f34b3400640 0 client.25717 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:16:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.722+0000 7f34da45d740 1 -- 172.21.15.39:0/1054616837 shutdown_connections 2024-07-04T08:16:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.722+0000 7f34da45d740 1 -- 172.21.15.39:0/1054616837 wait complete. 2024-07-04T08:16:19.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.722+0000 7f34da45d740 20 client.25717 trim_cache size 0 max 16384 2024-07-04T08:16:19.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.743+0000 7f34da45d740 1 Processor -- start 2024-07-04T08:16:19.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.743+0000 7f34da45d740 1 -- start start 2024-07-04T08:16:19.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.744+0000 7f34da45d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e582c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.744+0000 7f34da45d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 0x5614e580e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.744+0000 7f34da45d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 0x5614e58386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.744+0000 7f34da45d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1ade20 con 0x5614e543f1b0 2024-07-04T08:16:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.744+0000 7f34da45d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1af960 con 0x5614e5468070 2024-07-04T08:16:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.744+0000 7f34da45d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1ae4f0 con 0x5614e5447390 2024-07-04T08:16:19.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.744+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e582c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.745+0000 7f34d0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e582c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54944/0 (socket says 172.21.15.39:54944) 2024-07-04T08:16:19.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.745+0000 7f34d0a00640 1 -- 172.21.15.39:0/3440340314 learned_addr learned my addr 172.21.15.39:0/3440340314 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:19.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.745+0000 7f34d0a00640 1 --2- 172.21.15.39:0/3440340314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 0x5614e58386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.745+0000 7f34d0a00640 1 -- 172.21.15.39:0/3440340314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 msgr2=0x5614e580e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:19.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.745+0000 7f34d0a00640 1 --2- 172.21.15.39:0/3440340314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 0x5614e580e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.745+0000 7f34d0a00640 1 -- 172.21.15.39:0/3440340314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 msgr2=0x5614e58386b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:19.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.745+0000 7f34d0a00640 1 --2- 172.21.15.39:0/3440340314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 0x5614e58386b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.745+0000 7f34d0a00640 1 -- 172.21.15.39:0/3440340314 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e580b1c0 con 0x5614e543f1b0 2024-07-04T08:16:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.746+0000 7f34d0a00640 1 --2- 172.21.15.39:0/3440340314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e582c920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009800 tx=0x7f34b8005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4371bef7339ae42d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.746+0000 7f34be000640 1 -- 172.21.15.39:0/3440340314 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8035040 con 0x5614e543f1b0 2024-07-04T08:16:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.746+0000 7f34be000640 1 -- 172.21.15.39:0/3440340314 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b8058070 con 0x5614e543f1b0 2024-07-04T08:16:19.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.746+0000 7f34be000640 1 -- 172.21.15.39:0/3440340314 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8041040 con 0x5614e543f1b0 2024-07-04T08:16:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.748+0000 7f34da45d740 1 -- 172.21.15.39:0/3440340314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 msgr2=0x5614e582c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.748+0000 7f34da45d740 1 --2- 172.21.15.39:0/3440340314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e582c920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009800 tx=0x7f34b8005e10 comp rx=0 tx=0).stop 2024-07-04T08:16:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.748+0000 7f34da45d740 1 -- 172.21.15.39:0/3440340314 shutdown_connections 2024-07-04T08:16:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.748+0000 7f34da45d740 1 --2- 172.21.15.39:0/3440340314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5468070 0x5614e58386b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.748+0000 7f34da45d740 1 --2- 172.21.15.39:0/3440340314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5447390 0x5614e580e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.748+0000 7f34da45d740 1 --2- 172.21.15.39:0/3440340314 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e582c920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.748+0000 7f34da45d740 1 -- 172.21.15.39:0/3440340314 >> 172.21.15.39:0/3440340314 conn(0x5614e545c460 msgr2=0x5614e580d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:19.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.748+0000 7f34da45d740 1 -- 172.21.15.39:0/3440340314 shutdown_connections 2024-07-04T08:16:19.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.748+0000 7f34da45d740 1 -- 172.21.15.39:0/3440340314 wait complete. 2024-07-04T08:16:19.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.749+0000 7f34da45d740 1 -- start start 2024-07-04T08:16:19.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.749+0000 7f34da45d740 1 Processor -- start 2024-07-04T08:16:19.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.750+0000 7f34da45d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5829210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.750+0000 7f34da45d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e58246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.750+0000 7f34da45d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5468070 0x5614e5824ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.750+0000 7f34da45d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1af290 con 0x5614e543f1b0 2024-07-04T08:16:19.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.750+0000 7f34da45d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1aebc0 con 0x5614e5447390 2024-07-04T08:16:19.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.750+0000 7f34da45d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1abdc0 con 0x5614e5468070 2024-07-04T08:16:19.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.750+0000 7f34da45d740 10 client.0 _ll_register_callbacks cb 0x7f34c0009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:19.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.750+0000 7f34d1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e58246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.751+0000 7f34d1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e58246c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33930/0 (socket says 172.21.15.39:33930) 2024-07-04T08:16:19.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.751+0000 7f34d1400640 1 -- 172.21.15.39:0/233803499 learned_addr learned my addr 172.21.15.39:0/233803499 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:19.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.751+0000 7f34d0a00640 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5829210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.751+0000 7f34d1400640 1 -- 172.21.15.39:0/233803499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5468070 msgr2=0x5614e5824ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:16:19.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.751+0000 7f34d1400640 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5468070 0x5614e5824ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.751+0000 7f34d1400640 1 -- 172.21.15.39:0/233803499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 msgr2=0x5614e5829210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.752+0000 7f34d1400640 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5829210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.752+0000 7f34d1400640 1 -- 172.21.15.39:0/233803499 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e5825430 con 0x5614e5447390 2024-07-04T08:16:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.752+0000 7f34d0a00640 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5829210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.752+0000 7f34d1400640 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e58246c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005e00 tx=0x7f34c00122e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc8e7600a7cc7cb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.752+0000 7f34b3400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.752+0000 7f34b3400640 1 -- 172.21.15.39:0/233803499 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00053c0 con 0x5614e5447390 2024-07-04T08:16:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.752+0000 7f34b3400640 1 -- 172.21.15.39:0/233803499 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f34c00051c0 con 0x5614e5447390 2024-07-04T08:16:19.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.753+0000 7f34da45d740 1 -- 172.21.15.39:0/233803499 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b8058070 con 0x5614e5447390 2024-07-04T08:16:19.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.753+0000 7f34b3400640 1 -- 172.21.15.39:0/233803499 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34c0034050 con 0x5614e5447390 2024-07-04T08:16:19.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34da45d740 10 client.25742 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:19.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34da45d740 1 -- 172.21.15.39:0/233803499 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5614e589b6f0 con 0x5614e5447390 2024-07-04T08:16:19.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34b3400640 1 -- 172.21.15.39:0/233803499 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b8058070 con 0x5614e5447390 2024-07-04T08:16:19.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34b3400640 1 -- 172.21.15.39:0/233803499 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f34c0037020 con 0x5614e5447390 2024-07-04T08:16:19.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34da45d740 10 client.25742 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:19.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34da45d740 10 client.25742 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:16:19.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34da45d740 20 client.25742 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:19.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34da45d740 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5614e53fbda0 0x5614e589df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:19.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34da45d740 4 client.25742 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:16:19.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.754+0000 7f34da45d740 1 -- 172.21.15.39:0/233803499 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614e57808d0 con 0x5614e53fbda0 2024-07-04T08:16:19.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.755+0000 7f34d0a00640 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5614e53fbda0 0x5614e589df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:19.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.755+0000 7f34d0a00640 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5614e53fbda0 0x5614e589df50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2eaf43fd2e74d151 server_cookie=ed9b016a008b6c80 in_seq=0 out_seq=0 2024-07-04T08:16:19.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.755+0000 7f34b3400640 10 client.25742 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:16:19.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.756+0000 7f34b3400640 1 -- 172.21.15.39:0/233803499 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614e57808d0 con 0x5614e53fbda0 2024-07-04T08:16:19.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.756+0000 7f34b3400640 10 client.25742 handle_command_reply: tid=0 2024-07-04T08:16:19.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.756+0000 7f34da45d740 1 client.25742 shutdown 2024-07-04T08:16:19.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34da45d740 1 -- 172.21.15.39:0/233803499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 msgr2=0x5614e58246c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:19.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34da45d740 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e58246c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005e00 tx=0x7f34c00122e0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34da45d740 1 -- 172.21.15.39:0/233803499 shutdown_connections 2024-07-04T08:16:19.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34da45d740 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5614e53fbda0 0x5614e589df50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34da45d740 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614e5468070 0x5614e5824ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34da45d740 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5614e5447390 0x5614e58246c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34da45d740 1 --2- 172.21.15.39:0/233803499 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e543f1b0 0x5614e5829210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34da45d740 1 -- 172.21.15.39:0/233803499 >> 172.21.15.39:0/233803499 conn(0x5614e545c460 msgr2=0x5614e580f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:19.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34b3400640 0 client.25742 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:16:19.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.757+0000 7f34da45d740 1 -- 172.21.15.39:0/233803499 shutdown_connections 2024-07-04T08:16:19.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.758+0000 7f34da45d740 1 -- 172.21.15.39:0/233803499 wait complete. 2024-07-04T08:16:19.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.758+0000 7f34da45d740 20 client.25742 trim_cache size 0 max 16384 2024-07-04T08:16:19.760 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.758+0000 7f34d2800640 1 -- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34b40b29e0 msgr2=0x7f34b40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.758+0000 7f34d2800640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34b40b29e0 0x7f34b40b4ef0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009230 tx=0x7f34c4005f70 comp rx=0 tx=0).stop 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.758+0000 7f34d2800640 1 -- 172.21.15.39:0/3540377247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34cc10d170 msgr2=0x7f34cc1a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.758+0000 7f34d2800640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34cc10d170 0x7f34cc1a1240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009fd0 tx=0x7f34c400ef10 comp rx=0 tx=0).stop 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.759+0000 7f34d2800640 1 -- 172.21.15.39:0/3540377247 shutdown_connections 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.759+0000 7f34d2800640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34b40b29e0 0x7f34b40b4ef0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.759+0000 7f34d2800640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34cc104660 0x7f34cc19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.759+0000 7f34d2800640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34cc10d170 0x7f34cc1a1240 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.759+0000 7f34d2800640 1 --2- 172.21.15.39:0/3540377247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34cc103d10 0x7f34cc19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.759+0000 7f34d2800640 1 -- 172.21.15.39:0/3540377247 >> 172.21.15.39:0/3540377247 conn(0x7f34cc0fd810 msgr2=0x7f34cc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:19.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.759+0000 7f34d2800640 1 -- 172.21.15.39:0/3540377247 shutdown_connections 2024-07-04T08:16:19.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:19.759+0000 7f34d2800640 1 -- 172.21.15.39:0/3540377247 wait complete. 2024-07-04T08:16:20.632 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:16:21.142 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:21.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:21 smithi050 ceph-mon[36577]: pgmap v514: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 9 op/s 2024-07-04T08:16:21.460 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:21 smithi039 ceph-mon[32062]: pgmap v514: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 9 op/s 2024-07-04T08:16:21.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:21 smithi133 ceph-mon[37050]: pgmap v514: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 9 op/s 2024-07-04T08:16:23.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:23 smithi050 ceph-mon[36577]: pgmap v515: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:16:23.523 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:23 smithi039 ceph-mon[32062]: pgmap v515: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:16:23.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.522+0000 7f4669a00640 1 -- 172.21.15.39:0/2630621721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664074c80 msgr2=0x7f4664075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:23.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.522+0000 7f4669a00640 1 --2- 172.21.15.39:0/2630621721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664074c80 0x7f4664075090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4654009900 tx=0x7f465402a420 comp rx=0 tx=0).stop 2024-07-04T08:16:23.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.522+0000 7f4669a00640 1 -- 172.21.15.39:0/2630621721 shutdown_connections 2024-07-04T08:16:23.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.522+0000 7f4669a00640 1 --2- 172.21.15.39:0/2630621721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46640755d0 0x7f466410efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.522+0000 7f4669a00640 1 --2- 172.21.15.39:0/2630621721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f466410f4e0 0x7f4664111900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.522+0000 7f4669a00640 1 --2- 172.21.15.39:0/2630621721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4664074c80 0x7f4664075090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.522+0000 7f4669a00640 1 -- 172.21.15.39:0/2630621721 >> 172.21.15.39:0/2630621721 conn(0x7f46640fe950 msgr2=0x7f4664100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.522+0000 7f4669a00640 1 -- 172.21.15.39:0/2630621721 shutdown_connections 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.522+0000 7f4669a00640 1 -- 172.21.15.39:0/2630621721 wait complete. 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.524+0000 7f4669a00640 1 Processor -- start 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.524+0000 7f4669a00640 1 -- start start 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.525+0000 7f4669a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4664074c80 0x7f466419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.525+0000 7f4669a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46640755d0 0x7f466419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.525+0000 7f4669a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466410f4e0 0x7f46641a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.525+0000 7f4669a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4664115030 con 0x7f466410f4e0 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.525+0000 7f4669a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4664114eb0 con 0x7f46640755d0 2024-07-04T08:16:23.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.525+0000 7f4669a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46641151b0 con 0x7f4664074c80 2024-07-04T08:16:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.525+0000 7f4669000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466410f4e0 0x7f46641a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.525+0000 7f4669000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466410f4e0 0x7f46641a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46838/0 (socket says 172.21.15.39:46838) 2024-07-04T08:16:23.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.525+0000 7f4669000640 1 -- 172.21.15.39:0/3201862246 learned_addr learned my addr 172.21.15.39:0/3201862246 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:23.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.527+0000 7f4669000640 1 -- 172.21.15.39:0/3201862246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4664074c80 msgr2=0x7f466419ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-04T08:16:23.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.527+0000 7f4669000640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4664074c80 0x7f466419ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.527+0000 7f4669000640 1 -- 172.21.15.39:0/3201862246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46640755d0 msgr2=0x7f466419bfc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-04T08:16:23.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.527+0000 7f4669000640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46640755d0 0x7f466419bfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.527+0000 7f4669000640 1 -- 172.21.15.39:0/3201862246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46641a1990 con 0x7f466410f4e0 2024-07-04T08:16:23.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.528+0000 7f4669000640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466410f4e0 0x7f46641a1260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f465c00eb10 tx=0x7f465c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:23.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.529+0000 7f4660c00640 1 -- 172.21.15.39:0/3201862246 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c004b20 con 0x7f466410f4e0 2024-07-04T08:16:23.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.529+0000 7f4660c00640 1 -- 172.21.15.39:0/3201862246 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c004cc0 con 0x7f466410f4e0 2024-07-04T08:16:23.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.529+0000 7f4669a00640 1 -- 172.21.15.39:0/3201862246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46641a1c80 con 0x7f466410f4e0 2024-07-04T08:16:23.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.529+0000 7f4669a00640 1 -- 172.21.15.39:0/3201862246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641a83e0 con 0x7f466410f4e0 2024-07-04T08:16:23.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.529+0000 7f4660c00640 1 -- 172.21.15.39:0/3201862246 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c010720 con 0x7f466410f4e0 2024-07-04T08:16:23.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.534+0000 7f467160f740 1 Processor -- start 2024-07-04T08:16:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.537+0000 7f4660c00640 1 -- 172.21.15.39:0/3201862246 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f465c005020 con 0x7f466410f4e0 2024-07-04T08:16:23.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.537+0000 7f467160f740 1 -- start start 2024-07-04T08:16:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f467160f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f467160f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f467160f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484fe390 0x55b6488c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f467160f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b648839f00 con 0x55b64851f070 2024-07-04T08:16:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f467160f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b6488294a0 con 0x55b6484f61b0 2024-07-04T08:16:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f467160f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b64882d4e0 con 0x55b6484fe390 2024-07-04T08:16:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f4663e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f4663e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47714/0 (socket says 172.21.15.39:47714) 2024-07-04T08:16:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f4663e00640 1 -- 172.21.15.39:0/550840191 learned_addr learned my addr 172.21.15.39:0/550840191 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:23.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.538+0000 7f4663400640 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.539+0000 7f4663400640 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484fe390 0x55b6488c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.539+0000 7f4663e00640 1 -- 172.21.15.39:0/550840191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484fe390 msgr2=0x55b6488c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.539+0000 7f4663e00640 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484fe390 0x55b6488c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.539+0000 7f4663e00640 1 -- 172.21.15.39:0/550840191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 msgr2=0x55b6488c3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.539+0000 7f4663e00640 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488c3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.539+0000 7f4663e00640 1 -- 172.21.15.39:0/550840191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6487c1670 con 0x55b6484f61b0 2024-07-04T08:16:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.539+0000 7f4663400640 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484fe390 0x55b6488c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.540+0000 7f4663e00640 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488c3eb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4654006010 tx=0x7f4654009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840bda2d3c146aff server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:23.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.540+0000 7f4660c00640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f46480b29e0 0x7f46480b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.540+0000 7f4659600640 1 -- 172.21.15.39:0/550840191 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654040070 con 0x55b6484f61b0 2024-07-04T08:16:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.540+0000 7f4659600640 1 -- 172.21.15.39:0/550840191 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4654032e50 con 0x55b6484f61b0 2024-07-04T08:16:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.540+0000 7f4659600640 1 -- 172.21.15.39:0/550840191 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654035040 con 0x55b6484f61b0 2024-07-04T08:16:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.540+0000 7f4660c00640 1 -- 172.21.15.39:0/3201862246 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f465c0e2e70 con 0x7f466410f4e0 2024-07-04T08:16:23.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.541+0000 7f4663400640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f46480b29e0 0x7f46480b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.542+0000 7f467160f740 1 -- 172.21.15.39:0/550840191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 msgr2=0x55b6488c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.542+0000 7f467160f740 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488c3eb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4654006010 tx=0x7f4654009680 comp rx=0 tx=0).stop 2024-07-04T08:16:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.542+0000 7f467160f740 1 -- 172.21.15.39:0/550840191 shutdown_connections 2024-07-04T08:16:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.542+0000 7f467160f740 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484fe390 0x55b6488c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.542+0000 7f467160f740 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488c3eb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.542+0000 7f467160f740 1 --2- 172.21.15.39:0/550840191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488c3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.542+0000 7f467160f740 1 -- 172.21.15.39:0/550840191 >> 172.21.15.39:0/550840191 conn(0x55b648513460 msgr2=0x55b6488c1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.543+0000 7f4663400640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f46480b29e0 0x7f46480b4ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4664111950 tx=0x7f4650009630 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.543+0000 7f467160f740 1 -- 172.21.15.39:0/550840191 shutdown_connections 2024-07-04T08:16:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.543+0000 7f467160f740 1 -- 172.21.15.39:0/550840191 wait complete. 2024-07-04T08:16:23.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.544+0000 7f467160f740 1 -- start start 2024-07-04T08:16:23.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.544+0000 7f467160f740 1 Processor -- start 2024-07-04T08:16:23.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f467160f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6484f61b0 0x55b6488d8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f467160f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484fe390 0x55b6488d8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f467160f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b64851f070 0x55b6488dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f4663400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484fe390 0x55b6488d8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f467160f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b64868d700 con 0x55b6484f61b0 2024-07-04T08:16:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f467160f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b6487d0bd0 con 0x55b6484fe390 2024-07-04T08:16:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f467160f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b6488d20f0 con 0x55b64851f070 2024-07-04T08:16:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f4663e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6484f61b0 0x55b6488d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f4669000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b64851f070 0x55b6488dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f4669000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b64851f070 0x55b6488dda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58734/0 (socket says 172.21.15.39:58734) 2024-07-04T08:16:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f4663400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484fe390 0x55b6488d8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47726/0 (socket says 172.21.15.39:47726) 2024-07-04T08:16:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.546+0000 7f4669000640 1 -- 172.21.15.39:0/950208731 learned_addr learned my addr 172.21.15.39:0/950208731 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.547+0000 7f467160f740 10 client.0 _ll_register_callbacks cb 0x55b648822590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.548+0000 7f4663400640 1 -- 172.21.15.39:0/950208731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b64851f070 msgr2=0x55b6488dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:23.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.548+0000 7f4663400640 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b64851f070 0x55b6488dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.548+0000 7f4663400640 1 -- 172.21.15.39:0/950208731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6484f61b0 msgr2=0x55b6488d8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.548+0000 7f4663400640 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6484f61b0 0x55b6488d8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.548+0000 7f4663400640 1 -- 172.21.15.39:0/950208731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b64858a650 con 0x55b6484fe390 2024-07-04T08:16:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.548+0000 7f4663e00640 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6484f61b0 0x55b6488d8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.548+0000 7f4669000640 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b64851f070 0x55b6488dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.548+0000 7f4663400640 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484fe390 0x55b6488d8a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f466406ae80 tx=0x7f46500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9886c1f50c6dfa4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:23.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.548+0000 7f464e000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.549+0000 7f464e000640 1 -- 172.21.15.39:0/950208731 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650018070 con 0x55b6484fe390 2024-07-04T08:16:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.549+0000 7f464e000640 1 -- 172.21.15.39:0/950208731 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4650012860 con 0x55b6484fe390 2024-07-04T08:16:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.549+0000 7f467160f740 1 -- 172.21.15.39:0/950208731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4654032e50 con 0x55b6484fe390 2024-07-04T08:16:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.549+0000 7f464e000640 1 -- 172.21.15.39:0/950208731 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4650009a10 con 0x55b6484fe390 2024-07-04T08:16:23.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.549+0000 7f467160f740 10 client.25751 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:23.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.549+0000 7f467160f740 1 -- 172.21.15.39:0/950208731 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b6488e69c0 con 0x55b6484fe390 2024-07-04T08:16:23.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.550+0000 7f464e000640 1 -- 172.21.15.39:0/950208731 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4654032e50 con 0x55b6484fe390 2024-07-04T08:16:23.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.551+0000 7f464e000640 1 -- 172.21.15.39:0/950208731 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f465000f040 con 0x55b6484fe390 2024-07-04T08:16:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.552+0000 7f467160f740 10 client.25751 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.552+0000 7f467160f740 10 client.25751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:16:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.552+0000 7f467160f740 20 client.25751 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.552+0000 7f467160f740 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b6484dd9f0 0x55b6488e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.552+0000 7f467160f740 4 client.25751 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.552+0000 7f467160f740 1 -- 172.21.15.39:0/950208731 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b64883be50 con 0x55b6484dd9f0 2024-07-04T08:16:23.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.552+0000 7f4663e00640 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b6484dd9f0 0x55b6488e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.553+0000 7f4663e00640 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b6484dd9f0 0x55b6488e9ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d427164dd3b9c6a7 server_cookie=92f09098938a37d9 in_seq=0 out_seq=0 2024-07-04T08:16:23.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.553+0000 7f464e000640 10 client.25751 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:23.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.554+0000 7f464e000640 1 -- 172.21.15.39:0/950208731 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b64883be50 con 0x55b6484dd9f0 2024-07-04T08:16:23.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.554+0000 7f464e000640 10 client.25751 handle_command_reply: tid=0 2024-07-04T08:16:23.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.554+0000 7f467160f740 1 client.25751 shutdown 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.555+0000 7f467160f740 1 -- 172.21.15.39:0/950208731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484fe390 msgr2=0x55b6488d8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.555+0000 7f467160f740 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484fe390 0x55b6488d8a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f466406ae80 tx=0x7f46500046c0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.555+0000 7f467160f740 1 -- 172.21.15.39:0/950208731 shutdown_connections 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.555+0000 7f467160f740 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b6484dd9f0 0x55b6488e9ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.555+0000 7f467160f740 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b64851f070 0x55b6488dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.555+0000 7f467160f740 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484fe390 0x55b6488d8a50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.555+0000 7f467160f740 1 --2- 172.21.15.39:0/950208731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b6484f61b0 0x55b6488d8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.555+0000 7f467160f740 1 -- 172.21.15.39:0/950208731 >> 172.21.15.39:0/950208731 conn(0x55b648513460 msgr2=0x55b6488c6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.555+0000 7f464e000640 0 client.25751 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.556+0000 7f467160f740 1 -- 172.21.15.39:0/950208731 shutdown_connections 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.556+0000 7f467160f740 1 -- 172.21.15.39:0/950208731 wait complete. 2024-07-04T08:16:23.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.556+0000 7f467160f740 20 client.25751 trim_cache size 0 max 16384 2024-07-04T08:16:23.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:23 smithi133 ceph-mon[37050]: pgmap v515: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:16:23.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.585+0000 7f467160f740 1 Processor -- start 2024-07-04T08:16:23.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.586+0000 7f467160f740 1 -- start start 2024-07-04T08:16:23.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.586+0000 7f467160f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484f61b0 0x55b6488d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.586+0000 7f467160f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.586+0000 7f467160f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484b2da0 0x55b6488d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.586+0000 7f467160f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f464400fdb0 con 0x55b64851f070 2024-07-04T08:16:23.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.586+0000 7f467160f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f46440118f0 con 0x55b6484b2da0 2024-07-04T08:16:23.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.586+0000 7f467160f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4644010480 con 0x55b6484f61b0 2024-07-04T08:16:23.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.586+0000 7f4663400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.586+0000 7f4663e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484f61b0 0x55b6488d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4663e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484f61b0 0x55b6488d2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58742/0 (socket says 172.21.15.39:58742) 2024-07-04T08:16:23.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4663e00640 1 -- 172.21.15.39:0/3251845511 learned_addr learned my addr 172.21.15.39:0/3251845511 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:23.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4669000640 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484b2da0 0x55b6488d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4663e00640 1 -- 172.21.15.39:0/3251845511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484b2da0 msgr2=0x55b6488d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:23.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4663e00640 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484b2da0 0x55b6488d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4663e00640 1 -- 172.21.15.39:0/3251845511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 msgr2=0x55b6488e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:23.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4663e00640 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4663e00640 1 -- 172.21.15.39:0/3251845511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6488c5c60 con 0x55b6484f61b0 2024-07-04T08:16:23.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4669000640 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484b2da0 0x55b6488d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:23.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.587+0000 7f4663400640 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:23.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.588+0000 7f4663e00640 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484f61b0 0x55b6488d2aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55b6488c25f0 tx=0x7f4654005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cff20826f625b610 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:23.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.588+0000 7f4658c00640 1 -- 172.21.15.39:0/3251845511 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654035040 con 0x55b6484f61b0 2024-07-04T08:16:23.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.588+0000 7f4658c00640 1 -- 172.21.15.39:0/3251845511 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4654053070 con 0x55b6484f61b0 2024-07-04T08:16:23.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.588+0000 7f4658c00640 1 -- 172.21.15.39:0/3251845511 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654040040 con 0x55b6484f61b0 2024-07-04T08:16:23.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.590+0000 7f467160f740 1 -- 172.21.15.39:0/3251845511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484f61b0 msgr2=0x55b6488d2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:23.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.590+0000 7f467160f740 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484f61b0 0x55b6488d2aa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55b6488c25f0 tx=0x7f4654005a20 comp rx=0 tx=0).stop 2024-07-04T08:16:23.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.590+0000 7f467160f740 1 -- 172.21.15.39:0/3251845511 shutdown_connections 2024-07-04T08:16:23.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.590+0000 7f467160f740 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.590+0000 7f467160f740 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484b2da0 0x55b6488d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.590+0000 7f467160f740 1 --2- 172.21.15.39:0/3251845511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484f61b0 0x55b6488d2aa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.590+0000 7f467160f740 1 -- 172.21.15.39:0/3251845511 >> 172.21.15.39:0/3251845511 conn(0x55b648513460 msgr2=0x55b6488d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:23.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.590+0000 7f467160f740 1 -- 172.21.15.39:0/3251845511 shutdown_connections 2024-07-04T08:16:23.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.590+0000 7f467160f740 1 -- 172.21.15.39:0/3251845511 wait complete. 2024-07-04T08:16:23.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.591+0000 7f467160f740 1 -- start start 2024-07-04T08:16:23.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.591+0000 7f467160f740 1 Processor -- start 2024-07-04T08:16:23.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f467160f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484b2da0 0x55b6488de750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f467160f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488dec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f467160f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f467160f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4644011220 con 0x55b64851f070 2024-07-04T08:16:23.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f467160f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4644010b50 con 0x55b6484f61b0 2024-07-04T08:16:23.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f467160f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f464400dd90 con 0x55b6484b2da0 2024-07-04T08:16:23.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f467160f740 10 client.0 _ll_register_callbacks cb 0x7f4650011f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:23.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f4663400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f4663400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488dec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47734/0 (socket says 172.21.15.39:47734) 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f4663400640 1 -- 172.21.15.39:0/1411461711 learned_addr learned my addr 172.21.15.39:0/1411461711 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.592+0000 7f4663e00640 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484b2da0 0x55b6488de750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.593+0000 7f4669000640 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.593+0000 7f4663400640 1 -- 172.21.15.39:0/1411461711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484b2da0 msgr2=0x55b6488de750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.593+0000 7f4663400640 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484b2da0 0x55b6488de750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.593+0000 7f4663400640 1 -- 172.21.15.39:0/1411461711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 msgr2=0x55b6488e37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.593+0000 7f4663400640 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.593+0000 7f4663400640 1 -- 172.21.15.39:0/1411461711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6488df510 con 0x55b6484f61b0 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.593+0000 7f4669000640 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:23.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.593+0000 7f4663400640 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488dec90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4650005780 tx=0x7f46500044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3686fd7956bb2cda server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:23.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.594+0000 7f464e000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.594+0000 7f464e000640 1 -- 172.21.15.39:0/1411461711 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465003a070 con 0x55b6484f61b0 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.594+0000 7f464e000640 1 -- 172.21.15.39:0/1411461711 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f465001f610 con 0x55b6484f61b0 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.594+0000 7f467160f740 1 -- 172.21.15.39:0/1411461711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4654053070 con 0x55b6484f61b0 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.594+0000 7f464e000640 1 -- 172.21.15.39:0/1411461711 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4650046050 con 0x55b6484f61b0 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.594+0000 7f467160f740 10 client.25754 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.594+0000 7f467160f740 1 -- 172.21.15.39:0/1411461711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b6488e4c90 con 0x55b6484f61b0 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.595+0000 7f464e000640 1 -- 172.21.15.39:0/1411461711 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4654053070 con 0x55b6484f61b0 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.595+0000 7f464e000640 1 -- 172.21.15.39:0/1411461711 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4650036410 con 0x55b6484f61b0 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.595+0000 7f467160f740 10 client.25754 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.595+0000 7f467160f740 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:16:23.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.595+0000 7f467160f740 20 client.25754 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:23.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.595+0000 7f467160f740 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b648952c30 0x55b648955050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:23.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.596+0000 7f467160f740 4 client.25754 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:16:23.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.596+0000 7f467160f740 1 -- 172.21.15.39:0/1411461711 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6488378f0 con 0x55b648952c30 2024-07-04T08:16:23.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.596+0000 7f4663e00640 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b648952c30 0x55b648955050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:23.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.597+0000 7f4663e00640 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b648952c30 0x55b648955050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e9beeede4579e51 server_cookie=3d9eab0b81651654 in_seq=0 out_seq=0 2024-07-04T08:16:23.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.597+0000 7f464e000640 10 client.25754 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:23.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.597+0000 7f464e000640 1 -- 172.21.15.39:0/1411461711 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6488378f0 con 0x55b648952c30 2024-07-04T08:16:23.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.597+0000 7f464e000640 10 client.25754 handle_command_reply: tid=0 2024-07-04T08:16:23.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.597+0000 7f467160f740 1 client.25754 shutdown 2024-07-04T08:16:23.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 -- 172.21.15.39:0/1411461711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 msgr2=0x55b6488dec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488dec90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4650005780 tx=0x7f46500044c0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 -- 172.21.15.39:0/1411461711 shutdown_connections 2024-07-04T08:16:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b648952c30 0x55b648955050 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b64851f070 0x55b6488e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b6484f61b0 0x55b6488dec90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 --2- 172.21.15.39:0/1411461711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b6484b2da0 0x55b6488de750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 -- 172.21.15.39:0/1411461711 >> 172.21.15.39:0/1411461711 conn(0x55b648513460 msgr2=0x55b6488c6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f464e000640 0 client.25754 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 -- 172.21.15.39:0/1411461711 shutdown_connections 2024-07-04T08:16:23.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.598+0000 7f467160f740 1 -- 172.21.15.39:0/1411461711 wait complete. 2024-07-04T08:16:23.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.599+0000 7f467160f740 20 client.25754 trim_cache size 0 max 16384 2024-07-04T08:16:23.601 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 -- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f46480b29e0 msgr2=0x7f46480b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f46480b29e0 0x7f46480b4ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4664111950 tx=0x7f4650009630 comp rx=0 tx=0).stop 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 -- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466410f4e0 msgr2=0x7f46641a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466410f4e0 0x7f46641a1260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f465c00eb10 tx=0x7f465c00c2d0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 -- 172.21.15.39:0/3201862246 shutdown_connections 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f46480b29e0 0x7f46480b4ef0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466410f4e0 0x7f46641a1260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f46640755d0 0x7f466419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 --2- 172.21.15.39:0/3201862246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4664074c80 0x7f466419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.600+0000 7f4669a00640 1 -- 172.21.15.39:0/3201862246 >> 172.21.15.39:0/3201862246 conn(0x7f46640fe950 msgr2=0x7f4664109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.601+0000 7f4669a00640 1 -- 172.21.15.39:0/3201862246 shutdown_connections 2024-07-04T08:16:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:23.601+0000 7f4669a00640 1 -- 172.21.15.39:0/3201862246 wait complete. 2024-07-04T08:16:25.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:25 smithi050 ceph-mon[36577]: pgmap v516: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:16:25.481 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:16:25.482 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:16:25.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:25 smithi133 ceph-mon[37050]: pgmap v516: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:16:25.567 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:25 smithi039 ceph-mon[32062]: pgmap v516: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-04T08:16:25.734 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-04T08:16:25.734 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-07-04T08:16:25.734 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:16:26.131 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:27.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:27 smithi050 ceph-mon[36577]: pgmap v517: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-07-04T08:16:27.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:27 smithi133 ceph-mon[37050]: pgmap v517: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-07-04T08:16:27.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:27 smithi039 ceph-mon[32062]: pgmap v517: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 9 op/s 2024-07-04T08:16:27.708 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:28.958 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:16:28.959 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:16:28.959 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "2d0b4930-c3d6-4f1b-b2ae-270e9be607d1", 2024-07-04T08:16:28.959 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:16:28.959 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:16:28.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.814+0000 7efc9a800640 1 -- 172.21.15.39:0/888800973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94104660 msgr2=0x7efc9410cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.814+0000 7efc9a800640 1 --2- 172.21.15.39:0/888800973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94104660 0x7efc9410cbb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc80009860 tx=0x7efc8002a340 comp rx=0 tx=0).stop 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.814+0000 7efc9a800640 1 -- 172.21.15.39:0/888800973 shutdown_connections 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.814+0000 7efc9a800640 1 --2- 172.21.15.39:0/888800973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94104660 0x7efc9410cbb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.814+0000 7efc9a800640 1 --2- 172.21.15.39:0/888800973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc9410d170 0x7efc9410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.814+0000 7efc9a800640 1 --2- 172.21.15.39:0/888800973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc94103d10 0x7efc94104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.814+0000 7efc9a800640 1 -- 172.21.15.39:0/888800973 >> 172.21.15.39:0/888800973 conn(0x7efc940fd810 msgr2=0x7efc940ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.816+0000 7efc9a800640 1 -- 172.21.15.39:0/888800973 shutdown_connections 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.816+0000 7efc9a800640 1 -- 172.21.15.39:0/888800973 wait complete. 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.817+0000 7efc9a800640 1 Processor -- start 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.818+0000 7efc9a800640 1 -- start start 2024-07-04T08:16:28.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.818+0000 7efc9a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc94103d10 0x7efc94197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.818+0000 7efc9a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94104660 0x7efc94197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.818+0000 7efc9a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc9410d170 0x7efc9419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.818+0000 7efc9a800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc94110c10 con 0x7efc94104660 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.818+0000 7efc9a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc94110a90 con 0x7efc9410d170 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc9a800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efc94110d90 con 0x7efc94103d10 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc99400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc94103d10 0x7efc94197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc99400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc94103d10 0x7efc94197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58774/0 (socket says 172.21.15.39:58774) 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc99400640 1 -- 172.21.15.39:0/2179418466 learned_addr learned my addr 172.21.15.39:0/2179418466 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc98a00640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94104660 0x7efc94197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc99400640 1 -- 172.21.15.39:0/2179418466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc9410d170 msgr2=0x7efc9419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:28.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc99400640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc9410d170 0x7efc9419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc99400640 1 -- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94104660 msgr2=0x7efc94197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc99400640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94104660 0x7efc94197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.819+0000 7efc99400640 1 -- 172.21.15.39:0/2179418466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9419d5a0 con 0x7efc94103d10 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.820+0000 7efc99400640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc94103d10 0x7efc94197660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efc8800e9f0 tx=0x7efc8800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.820+0000 7efc92000640 1 -- 172.21.15.39:0/2179418466 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8800cde0 con 0x7efc94103d10 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.820+0000 7efc9a800640 1 -- 172.21.15.39:0/2179418466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9419d890 con 0x7efc94103d10 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.820+0000 7efc9a800640 1 -- 172.21.15.39:0/2179418466 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9406c330 con 0x7efc94103d10 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.823+0000 7efc92000640 1 -- 172.21.15.39:0/2179418466 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc880035f0 con 0x7efc94103d10 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.823+0000 7efc92000640 1 -- 172.21.15.39:0/2179418466 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88010690 con 0x7efc94103d10 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.825+0000 7efca2401740 1 Processor -- start 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.826+0000 7efca2401740 1 -- start start 2024-07-04T08:16:28.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.826+0000 7efca2401740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db3b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.826+0000 7efca2401740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d76e490 0x55bb9db3bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.826+0000 7efca2401740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d776670 0x55bb9db3c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.826+0000 7efca2401740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bb9da658e0 con 0x55bb9d797350 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.826+0000 7efca2401740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bb9dab1ec0 con 0x55bb9d76e490 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.826+0000 7efca2401740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bb9daa1460 con 0x55bb9d776670 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.827+0000 7efc99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db3b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.827+0000 7efc99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db3b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46906/0 (socket says 172.21.15.39:46906) 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.827+0000 7efc99e00640 1 -- 172.21.15.39:0/1363805335 learned_addr learned my addr 172.21.15.39:0/1363805335 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.827+0000 7efc98a00640 1 --2- 172.21.15.39:0/1363805335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d776670 0x55bb9db3c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.828+0000 7efc99e00640 1 -- 172.21.15.39:0/1363805335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d776670 msgr2=0x55bb9db3c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:28.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.828+0000 7efc99e00640 1 --2- 172.21.15.39:0/1363805335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d776670 0x55bb9db3c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.828+0000 7efc99e00640 1 -- 172.21.15.39:0/1363805335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d76e490 msgr2=0x55bb9db3bd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.828+0000 7efc99e00640 1 --2- 172.21.15.39:0/1363805335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d76e490 0x55bb9db3bd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.828+0000 7efc99e00640 1 -- 172.21.15.39:0/1363805335 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9da39280 con 0x55bb9d797350 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.829+0000 7efc92000640 1 -- 172.21.15.39:0/2179418466 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7efc88004000 con 0x7efc94103d10 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.829+0000 7efc99e00640 1 --2- 172.21.15.39:0/1363805335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db3b7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005c40 tx=0x7efc8c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e80d0bdbc4ab89 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.829+0000 7efc86a00640 1 -- 172.21.15.39:0/1363805335 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c003b70 con 0x55bb9d797350 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.830+0000 7efc86a00640 1 -- 172.21.15.39:0/1363805335 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc8c00ee60 con 0x55bb9d797350 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.830+0000 7efc86a00640 1 -- 172.21.15.39:0/1363805335 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c010850 con 0x55bb9d797350 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.831+0000 7efca2401740 1 -- 172.21.15.39:0/1363805335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 msgr2=0x55bb9db3b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.831+0000 7efca2401740 1 --2- 172.21.15.39:0/1363805335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db3b7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005c40 tx=0x7efc8c009f90 comp rx=0 tx=0).stop 2024-07-04T08:16:28.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.831+0000 7efc92000640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efc7c0b2ab0 0x7efc7c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.831+0000 7efc92000640 1 -- 172.21.15.39:0/2179418466 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7efc880e2b80 con 0x7efc94103d10 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.831+0000 7efca2401740 1 -- 172.21.15.39:0/1363805335 shutdown_connections 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.831+0000 7efca2401740 1 --2- 172.21.15.39:0/1363805335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d776670 0x55bb9db3c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.831+0000 7efca2401740 1 --2- 172.21.15.39:0/1363805335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d76e490 0x55bb9db3bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.831+0000 7efca2401740 1 --2- 172.21.15.39:0/1363805335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db3b7f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.831+0000 7efca2401740 1 -- 172.21.15.39:0/1363805335 >> 172.21.15.39:0/1363805335 conn(0x55bb9d78b740 msgr2=0x55bb9db396e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.832+0000 7efc99e00640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efc7c0b2ab0 0x7efc7c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.832+0000 7efca2401740 1 -- 172.21.15.39:0/1363805335 shutdown_connections 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.832+0000 7efca2401740 1 -- 172.21.15.39:0/1363805335 wait complete. 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.833+0000 7efca2401740 1 -- start start 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.833+0000 7efc99e00640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efc7c0b2ab0 0x7efc7c0b4fc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006010 tx=0x7efc8c004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:28.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.833+0000 7efca2401740 1 Processor -- start 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.834+0000 7efca2401740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d76e490 0x55bb9db47860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.834+0000 7efca2401740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db47e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.834+0000 7efca2401740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d797350 0x55bb9db4ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.834+0000 7efca2401740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bb9da48760 con 0x55bb9d76e490 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.834+0000 7efca2401740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bb9daa54a0 con 0x55bb9d776670 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.834+0000 7efca2401740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bb9d905890 con 0x55bb9d797350 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.834+0000 7efca2401740 10 client.0 _ll_register_callbacks cb 0x55bb9da492b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.834+0000 7efc98a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db47e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc98a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db47e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47760/0 (socket says 172.21.15.39:47760) 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc98a00640 1 -- 172.21.15.39:0/2641190991 learned_addr learned my addr 172.21.15.39:0/2641190991 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:28.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc99400640 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d76e490 0x55bb9db47860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc99e00640 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d797350 0x55bb9db4ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc98a00640 1 -- 172.21.15.39:0/2641190991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d797350 msgr2=0x55bb9db4ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc98a00640 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d797350 0x55bb9db4ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc98a00640 1 -- 172.21.15.39:0/2641190991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d76e490 msgr2=0x55bb9db47860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc98a00640 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d76e490 0x55bb9db47860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc98a00640 1 -- 172.21.15.39:0/2641190991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9d802930 con 0x55bb9d776670 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc99400640 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d76e490 0x55bb9db47860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.835+0000 7efc98a00640 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db47e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efc8002a930 tx=0x7efc800097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b7fc0dcb2a0efdf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.836+0000 7efc7b400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.836+0000 7efc7b400640 1 -- 172.21.15.39:0/2641190991 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80041070 con 0x55bb9d776670 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.836+0000 7efca2401740 1 -- 172.21.15.39:0/2641190991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc8c00ee60 con 0x55bb9d776670 2024-07-04T08:16:28.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.836+0000 7efc7b400640 1 -- 172.21.15.39:0/2641190991 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7efc80030450 con 0x55bb9d776670 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.836+0000 7efc7b400640 1 -- 172.21.15.39:0/2641190991 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc8004d050 con 0x55bb9d776670 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.836+0000 7efca2401740 10 client.25760 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.836+0000 7efca2401740 1 -- 172.21.15.39:0/2641190991 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb9db66e00 con 0x55bb9d776670 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.837+0000 7efc7b400640 1 -- 172.21.15.39:0/2641190991 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc8c00ee60 con 0x55bb9d776670 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.837+0000 7efc7b400640 1 -- 172.21.15.39:0/2641190991 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc800046c0 con 0x55bb9d776670 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.837+0000 7efca2401740 10 client.25760 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.837+0000 7efca2401740 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.837+0000 7efca2401740 20 client.25760 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.837+0000 7efca2401740 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bb9d755cd0 0x55bb9db6a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.837+0000 7efca2401740 4 client.25760 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.837+0000 7efca2401740 1 -- 172.21.15.39:0/2641190991 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb9dab3e10 con 0x55bb9d755cd0 2024-07-04T08:16:28.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.837+0000 7efc99400640 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bb9d755cd0 0x55bb9db6a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.838+0000 7efc99400640 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bb9d755cd0 0x55bb9db6a2f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c4adb59bbfd5f5 server_cookie=3aea4feeeea327d1 in_seq=0 out_seq=0 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.838+0000 7efc7b400640 10 client.25760 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efc7b400640 1 -- 172.21.15.39:0/2641190991 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bb9dab3e10 con 0x55bb9d755cd0 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efc7b400640 10 client.25760 handle_command_reply: tid=0 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efca2401740 1 client.25760 shutdown 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efca2401740 1 -- 172.21.15.39:0/2641190991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 msgr2=0x55bb9db47e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efca2401740 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db47e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efc8002a930 tx=0x7efc800097b0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efca2401740 1 -- 172.21.15.39:0/2641190991 shutdown_connections 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efca2401740 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bb9d755cd0 0x55bb9db6a2f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efca2401740 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d797350 0x55bb9db4ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efca2401740 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db47e20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efca2401740 1 --2- 172.21.15.39:0/2641190991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d76e490 0x55bb9db47860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efca2401740 1 -- 172.21.15.39:0/2641190991 >> 172.21.15.39:0/2641190991 conn(0x55bb9d78b740 msgr2=0x55bb9db37c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.839+0000 7efc7b400640 0 client.25760 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.840+0000 7efca2401740 1 -- 172.21.15.39:0/2641190991 shutdown_connections 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.840+0000 7efca2401740 1 -- 172.21.15.39:0/2641190991 wait complete. 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.840+0000 7efca2401740 20 client.25760 trim_cache size 0 max 16384 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.863+0000 7efca2401740 1 Processor -- start 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.863+0000 7efca2401740 1 -- start start 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.863+0000 7efca2401740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 0x55bb9db38320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.863+0000 7efca2401740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d776670 0x55bb9db48b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.863+0000 7efca2401740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d797350 0x55bb9db490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.863+0000 7efca2401740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc7400fd70 con 0x55bb9d776670 2024-07-04T08:16:28.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.864+0000 7efca2401740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc740118b0 con 0x55bb9d797350 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.864+0000 7efca2401740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efc74010440 con 0x55bb9d76e490 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.864+0000 7efc99e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d797350 0x55bb9db490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.864+0000 7efc99e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d797350 0x55bb9db490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47772/0 (socket says 172.21.15.39:47772) 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.864+0000 7efc99e00640 1 -- 172.21.15.39:0/2527847860 learned_addr learned my addr 172.21.15.39:0/2527847860 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.864+0000 7efc98a00640 1 --2- 172.21.15.39:0/2527847860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d776670 0x55bb9db48b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.864+0000 7efc99400640 1 --2- 172.21.15.39:0/2527847860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 0x55bb9db38320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.865+0000 7efc98a00640 1 -- 172.21.15.39:0/2527847860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 msgr2=0x55bb9db38320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.865+0000 7efc98a00640 1 --2- 172.21.15.39:0/2527847860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 0x55bb9db38320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.865+0000 7efc98a00640 1 -- 172.21.15.39:0/2527847860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d797350 msgr2=0x55bb9db490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.865+0000 7efc98a00640 1 --2- 172.21.15.39:0/2527847860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d797350 0x55bb9db490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.865+0000 7efc98a00640 1 -- 172.21.15.39:0/2527847860 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9db41d80 con 0x55bb9d776670 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.866+0000 7efc98a00640 1 --2- 172.21.15.39:0/2527847860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d776670 0x55bb9db48b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efc80004a60 tx=0x7efc8002aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d8d998900aaafe server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.866+0000 7efc86000640 1 -- 172.21.15.39:0/2527847860 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8004a030 con 0x55bb9d776670 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.866+0000 7efc86000640 1 -- 172.21.15.39:0/2527847860 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc80030900 con 0x55bb9d776670 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.866+0000 7efc86000640 1 -- 172.21.15.39:0/2527847860 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80041070 con 0x55bb9d776670 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.867+0000 7efca2401740 1 -- 172.21.15.39:0/2527847860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d776670 msgr2=0x55bb9db48b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.867+0000 7efca2401740 1 --2- 172.21.15.39:0/2527847860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d776670 0x55bb9db48b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efc80004a60 tx=0x7efc8002aa40 comp rx=0 tx=0).stop 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.867+0000 7efca2401740 1 -- 172.21.15.39:0/2527847860 shutdown_connections 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.867+0000 7efca2401740 1 --2- 172.21.15.39:0/2527847860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d776670 0x55bb9db48b80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.867+0000 7efca2401740 1 --2- 172.21.15.39:0/2527847860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d797350 0x55bb9db490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.867+0000 7efca2401740 1 --2- 172.21.15.39:0/2527847860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 0x55bb9db38320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.867+0000 7efca2401740 1 -- 172.21.15.39:0/2527847860 >> 172.21.15.39:0/2527847860 conn(0x55bb9d78b740 msgr2=0x55bb9db37c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:28.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.868+0000 7efca2401740 1 -- 172.21.15.39:0/2527847860 shutdown_connections 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.868+0000 7efca2401740 1 -- 172.21.15.39:0/2527847860 wait complete. 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.868+0000 7efca2401740 1 -- start start 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.869+0000 7efca2401740 1 Processor -- start 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.869+0000 7efca2401740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 0x55bb9db7b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.869+0000 7efca2401740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db7bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.869+0000 7efca2401740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.869+0000 7efca2401740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efc740111e0 con 0x55bb9d797350 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.869+0000 7efca2401740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efc74010b10 con 0x55bb9d776670 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.869+0000 7efca2401740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efc7400dd50 con 0x55bb9d76e490 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.869+0000 7efca2401740 10 client.0 _ll_register_callbacks cb 0x7efc800096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc99e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46942/0 (socket says 172.21.15.39:46942) 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc99e00640 1 -- 172.21.15.39:0/2441368374 learned_addr learned my addr 172.21.15.39:0/2441368374 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc98a00640 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db7bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc99400640 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 0x55bb9db7b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc99e00640 1 -- 172.21.15.39:0/2441368374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 msgr2=0x55bb9db7b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc99e00640 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 0x55bb9db7b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc99e00640 1 -- 172.21.15.39:0/2441368374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 msgr2=0x55bb9db7bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc99e00640 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db7bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.870+0000 7efc99e00640 1 -- 172.21.15.39:0/2441368374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9db7c360 con 0x55bb9d797350 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.871+0000 7efc99e00640 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db767e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005c10 tx=0x7efc8c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a26fbc48259430 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.871+0000 7efc7b400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.871+0000 7efc7b400640 1 -- 172.21.15.39:0/2441368374 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c01d040 con 0x55bb9d797350 2024-07-04T08:16:28.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.871+0000 7efc7b400640 1 -- 172.21.15.39:0/2441368374 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7efc8c02d040 con 0x55bb9d797350 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.872+0000 7efca2401740 1 -- 172.21.15.39:0/2441368374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc80030900 con 0x55bb9d797350 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.872+0000 7efc7b400640 1 -- 172.21.15.39:0/2441368374 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc8c004b20 con 0x55bb9d797350 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.872+0000 7efca2401740 10 client.15654 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.872+0000 7efca2401740 1 -- 172.21.15.39:0/2441368374 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb9dbd2de0 con 0x55bb9d797350 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.872+0000 7efc7b400640 1 -- 172.21.15.39:0/2441368374 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc80030900 con 0x55bb9d797350 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.873+0000 7efc7b400640 1 -- 172.21.15.39:0/2441368374 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efc8c023aa0 con 0x55bb9d797350 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.873+0000 7efca2401740 10 client.15654 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.873+0000 7efca2401740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.873+0000 7efca2401740 20 client.15654 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.873+0000 7efca2401740 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bb9d72b080 0x55bb9dbd5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.873+0000 7efca2401740 4 client.15654 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:16:28.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.873+0000 7efca2401740 1 -- 172.21.15.39:0/2441368374 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bb9daaf8b0 con 0x55bb9d72b080 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.873+0000 7efc99400640 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bb9d72b080 0x55bb9dbd5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.874+0000 7efc99400640 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bb9d72b080 0x55bb9dbd5480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f20302b982b41 server_cookie=ef6748ea0e2e9958 in_seq=0 out_seq=0 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.874+0000 7efc7b400640 10 client.15654 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.875+0000 7efc7b400640 1 -- 172.21.15.39:0/2441368374 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bb9daaf8b0 con 0x55bb9d72b080 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.875+0000 7efc7b400640 10 client.15654 handle_command_reply: tid=0 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.875+0000 7efca2401740 1 client.15654 shutdown 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.876+0000 7efca2401740 1 -- 172.21.15.39:0/2441368374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 msgr2=0x55bb9db767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.876+0000 7efca2401740 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db767e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005c10 tx=0x7efc8c010430 comp rx=0 tx=0).stop 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.876+0000 7efca2401740 1 -- 172.21.15.39:0/2441368374 shutdown_connections 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.876+0000 7efca2401740 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bb9d72b080 0x55bb9dbd5480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.876+0000 7efca2401740 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bb9d797350 0x55bb9db767e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.876+0000 7efca2401740 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bb9d776670 0x55bb9db7bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.876+0000 7efc7b400640 0 client.15654 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.876+0000 7efca2401740 1 --2- 172.21.15.39:0/2441368374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bb9d76e490 0x55bb9db7b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.876+0000 7efca2401740 1 -- 172.21.15.39:0/2441368374 >> 172.21.15.39:0/2441368374 conn(0x55bb9d78b740 msgr2=0x55bb9db49c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.877+0000 7efca2401740 1 -- 172.21.15.39:0/2441368374 shutdown_connections 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.877+0000 7efca2401740 1 -- 172.21.15.39:0/2441368374 wait complete. 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.877+0000 7efca2401740 20 client.15654 trim_cache size 0 max 16384 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.878+0000 7efc9a800640 1 -- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efc7c0b2ab0 msgr2=0x7efc7c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.878+0000 7efc9a800640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efc7c0b2ab0 0x7efc7c0b4fc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006010 tx=0x7efc8c004a40 comp rx=0 tx=0).stop 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.878+0000 7efc9a800640 1 -- 172.21.15.39:0/2179418466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc94103d10 msgr2=0x7efc94197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.878+0000 7efc9a800640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc94103d10 0x7efc94197660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efc8800e9f0 tx=0x7efc8800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.879+0000 7efc9a800640 1 -- 172.21.15.39:0/2179418466 shutdown_connections 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.879+0000 7efc9a800640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efc7c0b2ab0 0x7efc7c0b4fc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.879+0000 7efc9a800640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efc94104660 0x7efc94197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.879+0000 7efc9a800640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efc9410d170 0x7efc9419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.879+0000 7efc9a800640 1 --2- 172.21.15.39:0/2179418466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc94103d10 0x7efc94197660 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:28.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.879+0000 7efc9a800640 1 -- 172.21.15.39:0/2179418466 >> 172.21.15.39:0/2179418466 conn(0x7efc940fd810 msgr2=0x7efc94106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:28.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.880+0000 7efc9a800640 1 -- 172.21.15.39:0/2179418466 shutdown_connections 2024-07-04T08:16:28.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:28.880+0000 7efc9a800640 1 -- 172.21.15.39:0/2179418466 wait complete. 2024-07-04T08:16:29.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:29 smithi050 ceph-mon[36577]: pgmap v518: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-07-04T08:16:29.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:29 smithi133 ceph-mon[37050]: pgmap v518: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-07-04T08:16:29.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:29 smithi039 ceph-mon[32062]: pgmap v518: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-07-04T08:16:30.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:30 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:16:30.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:30 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:16:30.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:30 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:16:30.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:30 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:16:30.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:30 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:16:30.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:30 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:16:30.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:30 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:16:30.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:30 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:16:30.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:30 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:16:30.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:30 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:16:30.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:30 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:16:30.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:30 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:16:31.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:31 smithi133 ceph-mon[37050]: pgmap v519: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:16:31.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:31 smithi039 ceph-mon[32062]: pgmap v519: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:16:31.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:31 smithi050 ceph-mon[36577]: pgmap v519: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:16:32.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:32 smithi133 ceph-mon[37050]: pgmap v520: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:32.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:32 smithi039 ceph-mon[32062]: pgmap v520: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:32.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:32 smithi050 ceph-mon[36577]: pgmap v520: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:33.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:16:33.494 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:33 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:16:33.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:33 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:16:33.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:33 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:16:33.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.852+0000 7fb471c00640 1 -- 172.21.15.39:0/318582651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c0768d0 msgr2=0x7fb46c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.852+0000 7fb471c00640 1 --2- 172.21.15.39:0/318582651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c0768d0 0x7fb46c074d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009900 tx=0x7fb45c02a410 comp rx=0 tx=0).stop 2024-07-04T08:16:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.853+0000 7fb471c00640 1 -- 172.21.15.39:0/318582651 shutdown_connections 2024-07-04T08:16:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.853+0000 7fb471c00640 1 --2- 172.21.15.39:0/318582651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46c1137d0 0x7fb46c115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.853+0000 7fb471c00640 1 --2- 172.21.15.39:0/318582651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c075320 0x7fb46c075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.853+0000 7fb471c00640 1 --2- 172.21.15.39:0/318582651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c0768d0 0x7fb46c074d00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.853+0000 7fb471c00640 1 -- 172.21.15.39:0/318582651 >> 172.21.15.39:0/318582651 conn(0x7fb46c0fe990 msgr2=0x7fb46c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.853+0000 7fb471c00640 1 -- 172.21.15.39:0/318582651 shutdown_connections 2024-07-04T08:16:33.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.853+0000 7fb471c00640 1 -- 172.21.15.39:0/318582651 wait complete. 2024-07-04T08:16:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.855+0000 7fb471c00640 1 Processor -- start 2024-07-04T08:16:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.855+0000 7fb471c00640 1 -- start start 2024-07-04T08:16:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.856+0000 7fb471c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c075320 0x7fb46c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.856+0000 7fb471c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46c0768d0 0x7fb46c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.856+0000 7fb471c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c1137d0 0x7fb46c1a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.856+0000 7fb471c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c119470 con 0x7fb46c075320 2024-07-04T08:16:33.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.856+0000 7fb471c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1192f0 con 0x7fb46c0768d0 2024-07-04T08:16:33.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.856+0000 7fb471c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1195f0 con 0x7fb46c1137d0 2024-07-04T08:16:33.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.856+0000 7fb46b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46c0768d0 0x7fb46c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:33.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.856+0000 7fb46be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c075320 0x7fb46c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:33.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.857+0000 7fb46be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c075320 0x7fb46c19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40368/0 (socket says 172.21.15.39:40368) 2024-07-04T08:16:33.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.857+0000 7fb46be00640 1 -- 172.21.15.39:0/2570278226 learned_addr learned my addr 172.21.15.39:0/2570278226 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:33.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.857+0000 7fb471200640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c1137d0 0x7fb46c1a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:33.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.858+0000 7fb46be00640 1 -- 172.21.15.39:0/2570278226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c1137d0 msgr2=0x7fb46c1a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:33.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.858+0000 7fb46be00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c1137d0 0x7fb46c1a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.858+0000 7fb46be00640 1 -- 172.21.15.39:0/2570278226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46c0768d0 msgr2=0x7fb46c1a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.858+0000 7fb46be00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46c0768d0 0x7fb46c1a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.858+0000 7fb46be00640 1 -- 172.21.15.39:0/2570278226 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c1a5d70 con 0x7fb46c075320 2024-07-04T08:16:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.859+0000 7fb46be00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c075320 0x7fb46c19fe00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb45c031440 tx=0x7fb45c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.859+0000 7fb468c00640 1 -- 172.21.15.39:0/2570278226 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c03f070 con 0x7fb46c075320 2024-07-04T08:16:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.859+0000 7fb468c00640 1 -- 172.21.15.39:0/2570278226 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c031b50 con 0x7fb46c075320 2024-07-04T08:16:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.859+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c1a6000 con 0x7fb46c075320 2024-07-04T08:16:33.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.859+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c1ac7c0 con 0x7fb46c075320 2024-07-04T08:16:33.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.860+0000 7fb468c00640 1 -- 172.21.15.39:0/2570278226 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c035030 con 0x7fb46c075320 2024-07-04T08:16:33.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.862+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44c0052c0 con 0x7fb46c075320 2024-07-04T08:16:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.867+0000 7fb468c00640 1 -- 172.21.15.39:0/2570278226 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb45c042070 con 0x7fb46c075320 2024-07-04T08:16:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.869+0000 7fb468c00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4500b2a90 0x7fb4500b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:33.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.870+0000 7fb468c00640 1 -- 172.21.15.39:0/2570278226 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb45c103210 con 0x7fb46c075320 2024-07-04T08:16:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.872+0000 7fb46b400640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4500b2a90 0x7fb4500b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:33.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.873+0000 7fb46b400640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4500b2a90 0x7fb4500b4fa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb46c115730 tx=0x7fb458005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:33.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:33.877+0000 7fb468c00640 1 -- 172.21.15.39:0/2570278226 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb45c049030 con 0x7fb46c075320 2024-07-04T08:16:33.907 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:34.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.285+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb44c0055b0 con 0x7fb46c075320 2024-07-04T08:16:34.287 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:16:34.288 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:16:34.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.286+0000 7fb468c00640 1 -- 172.21.15.39:0/2570278226 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fb45c0ca860 con 0x7fb46c075320 2024-07-04T08:16:34.288 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:16:34.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4500b2a90 msgr2=0x7fb4500b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:34.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4500b2a90 0x7fb4500b4fa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb46c115730 tx=0x7fb458005f90 comp rx=0 tx=0).stop 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c075320 msgr2=0x7fb46c19fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c075320 0x7fb46c19fe00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb45c031440 tx=0x7fb45c003b90 comp rx=0 tx=0).stop 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 shutdown_connections 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4500b2a90 0x7fb4500b4fa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c1137d0 0x7fb46c1a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46c0768d0 0x7fb46c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 --2- 172.21.15.39:0/2570278226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46c075320 0x7fb46c19fe00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 >> 172.21.15.39:0/2570278226 conn(0x7fb46c0fe990 msgr2=0x7fb46c10f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.289+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 shutdown_connections 2024-07-04T08:16:34.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:34.290+0000 7fb471c00640 1 -- 172.21.15.39:0/2570278226 wait complete. 2024-07-04T08:16:34.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:34 smithi133 ceph-mon[37050]: pgmap v521: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-07-04T08:16:34.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:34 smithi039 ceph-mon[32062]: pgmap v521: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-07-04T08:16:34.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:34 smithi050 ceph-mon[36577]: pgmap v521: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-07-04T08:16:35.373 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:35 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2570278226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:35.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:35 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2570278226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:35.617 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-07-04T08:16:35.617 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-07-04T08:16:35.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:35 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2570278226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:36.395 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:36.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:36 smithi039 ceph-mon[32062]: pgmap v522: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-04T08:16:36.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:36 smithi133 ceph-mon[37050]: pgmap v522: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-04T08:16:36.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:36 smithi050 ceph-mon[36577]: pgmap v522: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-04T08:16:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.246+0000 7f1e34c00640 1 -- 172.21.15.39:0/607322899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e30074d00 msgr2=0x7f1e30075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.246+0000 7f1e34c00640 1 --2- 172.21.15.39:0/607322899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e30074d00 0x7f1e30075110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009900 tx=0x7f1e2002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.246+0000 7f1e34c00640 1 -- 172.21.15.39:0/607322899 shutdown_connections 2024-07-04T08:16:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.246+0000 7f1e34c00640 1 --2- 172.21.15.39:0/607322899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e30075650 0x7f1e3010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.246+0000 7f1e34c00640 1 --2- 172.21.15.39:0/607322899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e3010f610 0x7f1e30111a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.246+0000 7f1e34c00640 1 --2- 172.21.15.39:0/607322899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e30074d00 0x7f1e30075110 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.246+0000 7f1e34c00640 1 -- 172.21.15.39:0/607322899 >> 172.21.15.39:0/607322899 conn(0x7f1e3007ada0 msgr2=0x7f1e3007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:37.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.246+0000 7f1e34c00640 1 -- 172.21.15.39:0/607322899 shutdown_connections 2024-07-04T08:16:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.247+0000 7f1e34c00640 1 -- 172.21.15.39:0/607322899 wait complete. 2024-07-04T08:16:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.248+0000 7f1e34c00640 1 Processor -- start 2024-07-04T08:16:37.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.248+0000 7f1e34c00640 1 -- start start 2024-07-04T08:16:37.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e34c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30074d00 0x7f1e3019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e34c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e30075650 0x7f1e3019c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e34c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3010f610 0x7f1e301a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e34c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301151f0 con 0x7f1e3010f610 2024-07-04T08:16:37.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e34c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30115070 con 0x7f1e30075650 2024-07-04T08:16:37.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e34c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1e30115370 con 0x7f1e30074d00 2024-07-04T08:16:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e2de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30074d00 0x7f1e3019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e2de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30074d00 0x7f1e3019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34980/0 (socket says 172.21.15.39:34980) 2024-07-04T08:16:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e2de00640 1 -- 172.21.15.39:0/4289147650 learned_addr learned my addr 172.21.15.39:0/4289147650 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.249+0000 7f1e2e800640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3010f610 0x7f1e301a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.250+0000 7f1e2d400640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e30075650 0x7f1e3019c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.250+0000 7f1e2de00640 1 -- 172.21.15.39:0/4289147650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e30075650 msgr2=0x7f1e3019c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.250+0000 7f1e2de00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e30075650 0x7f1e3019c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.250+0000 7f1e2de00640 1 -- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3010f610 msgr2=0x7f1e301a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.250+0000 7f1e2de00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3010f610 0x7f1e301a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.250+0000 7f1e2de00640 1 -- 172.21.15.39:0/4289147650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e301a1b50 con 0x7f1e30074d00 2024-07-04T08:16:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.251+0000 7f1e2de00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30074d00 0x7f1e3019bbe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005ab0 tx=0x7f1e20003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.251+0000 7f1e26a00640 1 -- 172.21.15.39:0/4289147650 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2003f070 con 0x7f1e30074d00 2024-07-04T08:16:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.252+0000 7f1e26a00640 1 -- 172.21.15.39:0/4289147650 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e20031b60 con 0x7f1e30074d00 2024-07-04T08:16:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.252+0000 7f1e26a00640 1 -- 172.21.15.39:0/4289147650 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e20035030 con 0x7f1e30074d00 2024-07-04T08:16:37.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.253+0000 7f1e34c00640 1 -- 172.21.15.39:0/4289147650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e301a1de0 con 0x7f1e30074d00 2024-07-04T08:16:37.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.253+0000 7f1e34c00640 1 -- 172.21.15.39:0/4289147650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e301a85a0 con 0x7f1e30074d00 2024-07-04T08:16:37.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.258+0000 7f1e3b2a8740 1 Processor -- start 2024-07-04T08:16:37.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.258+0000 7f1e3b2a8740 1 -- start start 2024-07-04T08:16:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.258+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f93319a070 0x55f93353e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.258+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93353ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.258+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93353f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.258+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f9334b4ed0 con 0x55f93319a070 2024-07-04T08:16:37.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.258+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f9334a4470 con 0x55f933179390 2024-07-04T08:16:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.258+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f9334a84b0 con 0x55f9331711b0 2024-07-04T08:16:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.259+0000 7f1e2d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93353f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.259+0000 7f1e2e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f93319a070 0x55f93353e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.259+0000 7f1e2e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f93319a070 0x55f93353e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40404/0 (socket says 172.21.15.39:40404) 2024-07-04T08:16:37.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.259+0000 7f1e2e800640 1 -- 172.21.15.39:0/1403034825 learned_addr learned my addr 172.21.15.39:0/1403034825 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.260+0000 7f1e2e800640 1 -- 172.21.15.39:0/1403034825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 msgr2=0x55f93353ece0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:16:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.260+0000 7f1e2e800640 1 --2- 172.21.15.39:0/1403034825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93353ece0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.260+0000 7f1e2e800640 1 -- 172.21.15.39:0/1403034825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 msgr2=0x55f93353f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.260+0000 7f1e2e800640 1 --2- 172.21.15.39:0/1403034825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93353f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.260+0000 7f1e2e800640 1 -- 172.21.15.39:0/1403034825 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93343c640 con 0x55f93319a070 2024-07-04T08:16:37.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.261+0000 7f1e26a00640 1 -- 172.21.15.39:0/4289147650 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1e20042070 con 0x7f1e30074d00 2024-07-04T08:16:37.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.261+0000 7f1e2e800640 1 --2- 172.21.15.39:0/1403034825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f93319a070 0x55f93353e7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009fd0 tx=0x7f1e280118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75a8dc9d6be23a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:37.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.261+0000 7f1e1b400640 1 -- 172.21.15.39:0/1403034825 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28004d20 con 0x55f93319a070 2024-07-04T08:16:37.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.261+0000 7f1e1b400640 1 -- 172.21.15.39:0/1403034825 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e28004050 con 0x55f93319a070 2024-07-04T08:16:37.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.261+0000 7f1e1b400640 1 -- 172.21.15.39:0/1403034825 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28005280 con 0x55f93319a070 2024-07-04T08:16:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.263+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1403034825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f93319a070 msgr2=0x55f93353e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.263+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/1403034825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f93319a070 0x55f93353e7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009fd0 tx=0x7f1e280118d0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.263+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1403034825 shutdown_connections 2024-07-04T08:16:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.263+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/1403034825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93353f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.263+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/1403034825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93353ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.263+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/1403034825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f93319a070 0x55f93353e7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.263+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1403034825 >> 172.21.15.39:0/1403034825 conn(0x55f93318e460 msgr2=0x55f93353c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.264+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1403034825 shutdown_connections 2024-07-04T08:16:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.264+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1403034825 wait complete. 2024-07-04T08:16:37.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.264+0000 7f1e26a00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1e140b2970 0x7f1e140b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.264+0000 7f1e26a00640 1 -- 172.21.15.39:0/4289147650 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1e201033f0 con 0x7f1e30074d00 2024-07-04T08:16:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.264+0000 7f1e3b2a8740 1 -- start start 2024-07-04T08:16:37.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.265+0000 7f1e3b2a8740 1 Processor -- start 2024-07-04T08:16:37.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.265+0000 7f1e2d400640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1e140b2970 0x7f1e140b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.266+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93354a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.266+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 0x55f93354ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.266+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f93319a070 0x55f93354feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.266+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f933308700 con 0x55f933179390 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.266+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f93344bba0 con 0x55f93319a070 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.266+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f933546d30 con 0x55f9331711b0 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.266+0000 7f1e3b2a8740 10 client.0 _ll_register_callbacks cb 0x55f93349d560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2d400640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1e140b2970 0x7f1e140b4e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1e301112e0 tx=0x7f1e1c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f93319a070 0x55f93354feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f93319a070 0x55f93354feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48766/0 (socket says 172.21.15.39:48766) 2024-07-04T08:16:37.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2e800640 1 -- 172.21.15.39:0/3090065414 learned_addr learned my addr 172.21.15.39:0/3090065414 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:37.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2de00640 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93354a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2e800640 1 -- 172.21.15.39:0/3090065414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 msgr2=0x55f93354a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2e800640 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93354a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2e800640 1 -- 172.21.15.39:0/3090065414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 msgr2=0x55f93354ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2e800640 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 0x55f93354ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.267+0000 7f1e2e800640 1 -- 172.21.15.39:0/3090065414 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f933205650 con 0x55f93319a070 2024-07-04T08:16:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.268+0000 7f1e2de00640 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93354a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.268+0000 7f1e2e800640 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f93319a070 0x55f93354feb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1e2801b040 tx=0x7f1e28004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e79d595a784e6de server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:37.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.268+0000 7f1e0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.269+0000 7f1e0fe00640 1 -- 172.21.15.39:0/3090065414 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28028020 con 0x55f93319a070 2024-07-04T08:16:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.269+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/3090065414 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e28004050 con 0x55f93319a070 2024-07-04T08:16:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.269+0000 7f1e0fe00640 1 -- 172.21.15.39:0/3090065414 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1e28005040 con 0x55f93319a070 2024-07-04T08:16:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.269+0000 7f1e0fe00640 1 -- 172.21.15.39:0/3090065414 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e28033050 con 0x55f93319a070 2024-07-04T08:16:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.269+0000 7f1e3b2a8740 10 client.25784 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.269+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/3090065414 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f933569fa0 con 0x55f93319a070 2024-07-04T08:16:37.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.270+0000 7f1e0fe00640 1 -- 172.21.15.39:0/3090065414 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e28004050 con 0x55f93319a070 2024-07-04T08:16:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.271+0000 7f1e0fe00640 1 -- 172.21.15.39:0/3090065414 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1e2800f570 con 0x55f93319a070 2024-07-04T08:16:37.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.271+0000 7f1e3b2a8740 10 client.25784 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.271+0000 7f1e3b2a8740 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:16:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.271+0000 7f1e3b2a8740 20 client.25784 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.271+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f9331589f0 0x55f93356d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.271+0000 7f1e3b2a8740 4 client.25784 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.271+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/3090065414 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9334b6e20 con 0x55f9331589f0 2024-07-04T08:16:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.272+0000 7f1e2de00640 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f9331589f0 0x55f93356d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.272+0000 7f1e2de00640 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f9331589f0 0x55f93356d360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d6330d6c0c4ac server_cookie=7ba73ca7d26347a5 in_seq=0 out_seq=0 2024-07-04T08:16:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.272+0000 7f1e0fe00640 10 client.25784 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.273+0000 7f1e0fe00640 1 -- 172.21.15.39:0/3090065414 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f9334b6e20 con 0x55f9331589f0 2024-07-04T08:16:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.273+0000 7f1e0fe00640 10 client.25784 handle_command_reply: tid=0 2024-07-04T08:16:37.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.273+0000 7f1e3b2a8740 1 client.25784 shutdown 2024-07-04T08:16:37.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.274+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/3090065414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f93319a070 msgr2=0x55f93354feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:37.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.274+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f93319a070 0x55f93354feb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1e2801b040 tx=0x7f1e28004430 comp rx=0 tx=0).stop 2024-07-04T08:16:37.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.274+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/3090065414 shutdown_connections 2024-07-04T08:16:37.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.274+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f9331589f0 0x55f93356d360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.274+0000 7f1e0fe00640 0 client.25784 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.274+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 0x55f93354ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.274+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f93319a070 0x55f93354feb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.274+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/3090065414 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93354a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.274+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/3090065414 >> 172.21.15.39:0/3090065414 conn(0x55f93318e460 msgr2=0x55f933541980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.275+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/3090065414 shutdown_connections 2024-07-04T08:16:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.275+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/3090065414 wait complete. 2024-07-04T08:16:37.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.275+0000 7f1e3b2a8740 20 client.25784 trim_cache size 0 max 16384 2024-07-04T08:16:37.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.309+0000 7f1e3b2a8740 1 Processor -- start 2024-07-04T08:16:37.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.309+0000 7f1e3b2a8740 1 -- start start 2024-07-04T08:16:37.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.310+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f9331589f0 0x55f93353b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.310+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93353c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.310+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 0x55f933547f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.310+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301ae120 con 0x55f933179390 2024-07-04T08:16:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.310+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301afc60 con 0x55f9331589f0 2024-07-04T08:16:37.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.310+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301ae7f0 con 0x55f9331711b0 2024-07-04T08:16:37.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.310+0000 7f1e2de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f9331589f0 0x55f93353b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f9331589f0 0x55f93353b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48768/0 (socket says 172.21.15.39:48768) 2024-07-04T08:16:37.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2de00640 1 -- 172.21.15.39:0/1035909115 learned_addr learned my addr 172.21.15.39:0/1035909115 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:37.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2e800640 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 0x55f933547f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2d400640 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93353c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2de00640 1 -- 172.21.15.39:0/1035909115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 msgr2=0x55f93353c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:37.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2de00640 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93353c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2de00640 1 -- 172.21.15.39:0/1035909115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 msgr2=0x55f933547f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:37.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2de00640 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 0x55f933547f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2de00640 1 -- 172.21.15.39:0/1035909115 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f93354b970 con 0x55f9331589f0 2024-07-04T08:16:37.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.311+0000 7f1e2e800640 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 0x55f933547f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:37.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.312+0000 7f1e2d400640 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93353c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:37.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.312+0000 7f1e2de00640 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f9331589f0 0x55f93353b930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e20000c00 tx=0x7f1e2002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9e7bbb1d23da82 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:37.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.312+0000 7f1e1aa00640 1 -- 172.21.15.39:0/1035909115 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e200e4070 con 0x55f9331589f0 2024-07-04T08:16:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.312+0000 7f1e1aa00640 1 -- 172.21.15.39:0/1035909115 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e200ce6f0 con 0x55f9331589f0 2024-07-04T08:16:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.312+0000 7f1e1aa00640 1 -- 172.21.15.39:0/1035909115 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e200c6740 con 0x55f9331589f0 2024-07-04T08:16:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.313+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1035909115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f9331589f0 msgr2=0x55f93353b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.313+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f9331589f0 0x55f93353b930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1e20000c00 tx=0x7f1e2002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.314+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1035909115 shutdown_connections 2024-07-04T08:16:37.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.314+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f933179390 0x55f933547f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.314+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f93353c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.314+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/1035909115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f9331589f0 0x55f93353b930 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.314+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1035909115 >> 172.21.15.39:0/1035909115 conn(0x55f93318e460 msgr2=0x55f93353bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.314+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1035909115 shutdown_connections 2024-07-04T08:16:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.314+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/1035909115 wait complete. 2024-07-04T08:16:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.315+0000 7f1e3b2a8740 1 -- start start 2024-07-04T08:16:37.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.315+0000 7f1e3b2a8740 1 Processor -- start 2024-07-04T08:16:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.316+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f9331589f0 0x55f9335729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.316+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f933572f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.316+0000 7f1e3b2a8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93357e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.316+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301af590 con 0x55f9331589f0 2024-07-04T08:16:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.316+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301aeec0 con 0x55f933179390 2024-07-04T08:16:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.316+0000 7f1e3b2a8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1e301ac140 con 0x55f9331711b0 2024-07-04T08:16:37.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.316+0000 7f1e3b2a8740 10 client.0 _ll_register_callbacks cb 0x7f1e28009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.317+0000 7f1e2e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93357e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.317+0000 7f1e2de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f9331589f0 0x55f9335729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.317+0000 7f1e2d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f933572f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.317+0000 7f1e2e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93357e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48774/0 (socket says 172.21.15.39:48774) 2024-07-04T08:16:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.317+0000 7f1e2e800640 1 -- 172.21.15.39:0/200008218 learned_addr learned my addr 172.21.15.39:0/200008218 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:37.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.317+0000 7f1e2d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f933572f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:35014/0 (socket says 172.21.15.39:35014) 2024-07-04T08:16:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.318+0000 7f1e2e800640 1 -- 172.21.15.39:0/200008218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 msgr2=0x55f933572f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.318+0000 7f1e2e800640 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f933572f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.318+0000 7f1e2e800640 1 -- 172.21.15.39:0/200008218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f9331589f0 msgr2=0x55f9335729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:37.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.318+0000 7f1e2e800640 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f9331589f0 0x55f9335729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.318+0000 7f1e2e800640 1 -- 172.21.15.39:0/200008218 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9335735a0 con 0x55f933179390 2024-07-04T08:16:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.318+0000 7f1e2d400640 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f933572f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.318+0000 7f1e2e800640 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93357e6f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e28002230 tx=0x7f1e28022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55204aa02189e40 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.318+0000 7f1e0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:37.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.319+0000 7f1e0fe00640 1 -- 172.21.15.39:0/200008218 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28028070 con 0x55f933179390 2024-07-04T08:16:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.319+0000 7f1e0fe00640 1 -- 172.21.15.39:0/200008218 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1e2800f6e0 con 0x55f933179390 2024-07-04T08:16:37.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.319+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/200008218 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e200ce6f0 con 0x55f933179390 2024-07-04T08:16:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.322+0000 7f1e0fe00640 1 -- 172.21.15.39:0/200008218 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1e28046050 con 0x55f933179390 2024-07-04T08:16:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.322+0000 7f1e3b2a8740 10 client.25790 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.322+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/200008218 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f933542ed0 con 0x55f933179390 2024-07-04T08:16:37.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.322+0000 7f1e0fe00640 1 -- 172.21.15.39:0/200008218 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e200ce6f0 con 0x55f933179390 2024-07-04T08:16:37.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.323+0000 7f1e0fe00640 1 -- 172.21.15.39:0/200008218 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1e28005380 con 0x55f933179390 2024-07-04T08:16:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.323+0000 7f1e3b2a8740 10 client.25790 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.323+0000 7f1e3b2a8740 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:16:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.323+0000 7f1e3b2a8740 20 client.25790 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.323+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f93319a070 0x55f9335e8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.324+0000 7f1e3b2a8740 4 client.25790 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:16:37.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.324+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/200008218 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f9334b28c0 con 0x55f93319a070 2024-07-04T08:16:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.324+0000 7f1e2de00640 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f93319a070 0x55f9335e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.325+0000 7f1e0fe00640 10 client.25790 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.325+0000 7f1e2de00640 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f93319a070 0x55f9335e8830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d27f75aa122d9ee server_cookie=bfce0f96638d56f0 in_seq=0 out_seq=0 2024-07-04T08:16:37.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.325+0000 7f1e0fe00640 1 -- 172.21.15.39:0/200008218 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f9334b28c0 con 0x55f93319a070 2024-07-04T08:16:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.325+0000 7f1e0fe00640 10 client.25790 handle_command_reply: tid=0 2024-07-04T08:16:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.325+0000 7f1e3b2a8740 1 client.25790 shutdown 2024-07-04T08:16:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/200008218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 msgr2=0x55f93357e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93357e6f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e28002230 tx=0x7f1e28022000 comp rx=0 tx=0).stop 2024-07-04T08:16:37.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/200008218 shutdown_connections 2024-07-04T08:16:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f93319a070 0x55f9335e8830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f933179390 0x55f93357e6f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f9331711b0 0x55f933572f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 --2- 172.21.15.39:0/200008218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f9331589f0 0x55f9335729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/200008218 >> 172.21.15.39:0/200008218 conn(0x55f93318e460 msgr2=0x55f93354bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e0fe00640 0 client.25790 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/200008218 shutdown_connections 2024-07-04T08:16:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.326+0000 7f1e3b2a8740 1 -- 172.21.15.39:0/200008218 wait complete. 2024-07-04T08:16:37.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.327+0000 7f1e3b2a8740 20 client.25790 trim_cache size 0 max 16384 2024-07-04T08:16:37.329 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:16:37.329 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:16:37.329 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:16:37.329 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:16:37.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 -- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1e140b2970 msgr2=0x7f1e140b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:37.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1e140b2970 0x7f1e140b4e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1e301112e0 tx=0x7f1e1c005f90 comp rx=0 tx=0).stop 2024-07-04T08:16:37.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 -- 172.21.15.39:0/4289147650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30074d00 msgr2=0x7f1e3019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:37.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30074d00 0x7f1e3019bbe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005ab0 tx=0x7f1e20003c10 comp rx=0 tx=0).stop 2024-07-04T08:16:37.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 -- 172.21.15.39:0/4289147650 shutdown_connections 2024-07-04T08:16:37.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1e140b2970 0x7f1e140b4e80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e3010f610 0x7f1e301a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e30075650 0x7f1e3019c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 --2- 172.21.15.39:0/4289147650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e30074d00 0x7f1e3019bbe0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:37.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.328+0000 7f1e34c00640 1 -- 172.21.15.39:0/4289147650 >> 172.21.15.39:0/4289147650 conn(0x7f1e3007ada0 msgr2=0x7f1e300ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:37.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.330+0000 7f1e34c00640 1 -- 172.21.15.39:0/4289147650 shutdown_connections 2024-07-04T08:16:37.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:37.330+0000 7f1e34c00640 1 -- 172.21.15.39:0/4289147650 wait complete. 2024-07-04T08:16:39.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:38 smithi133 ceph-mon[37050]: pgmap v523: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-04T08:16:39.074 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:38 smithi039 ceph-mon[32062]: pgmap v523: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-04T08:16:39.077 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:16:39.077 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:16:39.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:38 smithi050 ceph-mon[36577]: pgmap v523: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-07-04T08:16:39.805 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:39.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.890+0000 7fa81cc00640 1 -- 172.21.15.39:0/1618720300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818108850 msgr2=0x7fa818108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:39.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.890+0000 7fa81cc00640 1 --2- 172.21.15.39:0/1618720300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818108850 0x7fa818108c60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa804009900 tx=0x7fa80402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.891+0000 7fa81cc00640 1 -- 172.21.15.39:0/1618720300 shutdown_connections 2024-07-04T08:16:39.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.891+0000 7fa81cc00640 1 --2- 172.21.15.39:0/1618720300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8181091a0 0x7fa81810f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.891+0000 7fa81cc00640 1 --2- 172.21.15.39:0/1618720300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818108850 0x7fa818108c60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.891+0000 7fa81cc00640 1 --2- 172.21.15.39:0/1618720300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa818102f10 0x7fa818103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.891+0000 7fa81cc00640 1 -- 172.21.15.39:0/1618720300 >> 172.21.15.39:0/1618720300 conn(0x7fa8180fe980 msgr2=0x7fa818100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:39.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.891+0000 7fa81cc00640 1 -- 172.21.15.39:0/1618720300 shutdown_connections 2024-07-04T08:16:39.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.891+0000 7fa81cc00640 1 -- 172.21.15.39:0/1618720300 wait complete. 2024-07-04T08:16:39.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.893+0000 7fa81cc00640 1 Processor -- start 2024-07-04T08:16:39.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.893+0000 7fa81cc00640 1 -- start start 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.893+0000 7fa81cc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818102f10 0x7fa818076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.893+0000 7fa81cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa818108850 0x7fa818074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.893+0000 7fa81cc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8181091a0 0x7fa8180751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa81cc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa818112eb0 con 0x7fa818108850 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa81cc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa818112d30 con 0x7fa8181091a0 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa81cc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa818113030 con 0x7fa818102f10 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa817400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818102f10 0x7fa818076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa816a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa818108850 0x7fa818074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa817400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818102f10 0x7fa818076670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:35030/0 (socket says 172.21.15.39:35030) 2024-07-04T08:16:39.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa817400640 1 -- 172.21.15.39:0/529711926 learned_addr learned my addr 172.21.15.39:0/529711926 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:39.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa817400640 1 -- 172.21.15.39:0/529711926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8181091a0 msgr2=0x7fa8180751a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-07-04T08:16:39.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa817400640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8181091a0 0x7fa8180751a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa817400640 1 -- 172.21.15.39:0/529711926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa818108850 msgr2=0x7fa818074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:39.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa817400640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa818108850 0x7fa818074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.894+0000 7fa817400640 1 -- 172.21.15.39:0/529711926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8180759c0 con 0x7fa818102f10 2024-07-04T08:16:39.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.895+0000 7fa817400640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818102f10 0x7fa818076670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa80800d950 tx=0x7fa80800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:39.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.895+0000 7fa80fe00640 1 -- 172.21.15.39:0/529711926 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808017030 con 0x7fa818102f10 2024-07-04T08:16:39.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.896+0000 7fa80fe00640 1 -- 172.21.15.39:0/529711926 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa808004020 con 0x7fa818102f10 2024-07-04T08:16:39.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.896+0000 7fa81cc00640 1 -- 172.21.15.39:0/529711926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81819d9a0 con 0x7fa818102f10 2024-07-04T08:16:39.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.896+0000 7fa81cc00640 1 -- 172.21.15.39:0/529711926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81819df00 con 0x7fa818102f10 2024-07-04T08:16:39.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.896+0000 7fa80fe00640 1 -- 172.21.15.39:0/529711926 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080047a0 con 0x7fa818102f10 2024-07-04T08:16:39.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.903+0000 7fa82481d740 1 Processor -- start 2024-07-04T08:16:39.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.904+0000 7fa82481d740 1 -- start start 2024-07-04T08:16:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.904+0000 7fa80fe00640 1 -- 172.21.15.39:0/529711926 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa808009a30 con 0x7fa818102f10 2024-07-04T08:16:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.907+0000 7fa80fe00640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa7fc0b2a50 0x7fa7fc0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.907+0000 7fa80fe00640 1 -- 172.21.15.39:0/529711926 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa8080e2230 con 0x7fa818102f10 2024-07-04T08:16:39.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.908+0000 7fa82481d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed884b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.908+0000 7fa82481d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 0x556ed884be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.908+0000 7fa82481d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 0x556ed884c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.908+0000 7fa82481d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556ed8811140 con 0x556ed84a6850 2024-07-04T08:16:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.908+0000 7fa82481d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556ed8732380 con 0x556ed847d990 2024-07-04T08:16:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.908+0000 7fa82481d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ed87c0c30 con 0x556ed8485b70 2024-07-04T08:16:39.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.910+0000 7fa817400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed884b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.910+0000 7fa817400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed884b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40452/0 (socket says 172.21.15.39:40452) 2024-07-04T08:16:39.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.910+0000 7fa817400640 1 -- 172.21.15.39:0/2336418152 learned_addr learned my addr 172.21.15.39:0/2336418152 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:39.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.911+0000 7fa817e00640 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 0x556ed884c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.911+0000 7fa817e00640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa7fc0b2a50 0x7fa7fc0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.911+0000 7fa816a00640 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 0x556ed884be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.912+0000 7fa817400640 1 -- 172.21.15.39:0/2336418152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 msgr2=0x556ed884c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:39.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.912+0000 7fa817400640 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 0x556ed884c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.912+0000 7fa817400640 1 -- 172.21.15.39:0/2336418152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 msgr2=0x556ed884be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:39.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.912+0000 7fa817400640 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 0x556ed884be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.912+0000 7fa817400640 1 -- 172.21.15.39:0/2336418152 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed8520ba0 con 0x556ed84a6850 2024-07-04T08:16:39.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.912+0000 7fa816a00640 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 0x556ed884be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:39.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.912+0000 7fa817400640 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed884b950 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa8080a9110 tx=0x7fa8080a8a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28a4c2660f5818d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:39.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.913+0000 7fa817e00640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa7fc0b2a50 0x7fa7fc0b4f60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa810009fd0 tx=0x7fa8100097a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:39.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.913+0000 7fa80cc00640 1 -- 172.21.15.39:0/2336418152 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080ad530 con 0x556ed84a6850 2024-07-04T08:16:39.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.914+0000 7fa80cc00640 1 -- 172.21.15.39:0/2336418152 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8080b1fe0 con 0x556ed84a6850 2024-07-04T08:16:39.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.914+0000 7fa80cc00640 1 -- 172.21.15.39:0/2336418152 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080ad120 con 0x556ed84a6850 2024-07-04T08:16:39.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.915+0000 7fa82481d740 1 -- 172.21.15.39:0/2336418152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 msgr2=0x556ed884b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:39.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.915+0000 7fa82481d740 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed884b950 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa8080a9110 tx=0x7fa8080a8a20 comp rx=0 tx=0).stop 2024-07-04T08:16:39.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.915+0000 7fa82481d740 1 -- 172.21.15.39:0/2336418152 shutdown_connections 2024-07-04T08:16:39.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.915+0000 7fa82481d740 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 0x556ed884c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.915+0000 7fa82481d740 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 0x556ed884be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.915+0000 7fa82481d740 1 --2- 172.21.15.39:0/2336418152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed884b950 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.915+0000 7fa82481d740 1 -- 172.21.15.39:0/2336418152 >> 172.21.15.39:0/2336418152 conn(0x556ed849ac40 msgr2=0x556ed8849840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:39.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.916+0000 7fa82481d740 1 -- 172.21.15.39:0/2336418152 shutdown_connections 2024-07-04T08:16:39.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.916+0000 7fa82481d740 1 -- 172.21.15.39:0/2336418152 wait complete. 2024-07-04T08:16:39.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.917+0000 7fa82481d740 1 -- start start 2024-07-04T08:16:39.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.917+0000 7fa82481d740 1 Processor -- start 2024-07-04T08:16:39.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.918+0000 7fa82481d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed847d990 0x556ed8857520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa82481d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 0x556ed8857a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa82481d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed84a6850 0x556ed88742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa82481d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556ed87b4210 con 0x556ed847d990 2024-07-04T08:16:39.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa82481d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556ed87b01d0 con 0x556ed84a6850 2024-07-04T08:16:39.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa82481d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ed87575d0 con 0x556ed8485b70 2024-07-04T08:16:39.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa82481d740 10 client.0 _ll_register_callbacks cb 0x556ed84be0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:39.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa817400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed847d990 0x556ed8857520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa817400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed847d990 0x556ed8857520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40466/0 (socket says 172.21.15.39:40466) 2024-07-04T08:16:39.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa817400640 1 -- 172.21.15.39:0/1183939985 learned_addr learned my addr 172.21.15.39:0/1183939985 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:39.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.919+0000 7fa817e00640 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed84a6850 0x556ed88742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.920+0000 7fa816a00640 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 0x556ed8857a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.920+0000 7fa817400640 1 -- 172.21.15.39:0/1183939985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 msgr2=0x556ed8857a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:39.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.920+0000 7fa817400640 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 0x556ed8857a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.920+0000 7fa817400640 1 -- 172.21.15.39:0/1183939985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed84a6850 msgr2=0x556ed88742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.920+0000 7fa817400640 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed84a6850 0x556ed88742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.920+0000 7fa817400640 1 -- 172.21.15.39:0/1183939985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed8747960 con 0x556ed847d990 2024-07-04T08:16:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.921+0000 7fa817400640 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed847d990 0x556ed8857520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa8080a9340 tx=0x7fa8080df640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e8092b69c750e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.921+0000 7fa801600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:16:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.921+0000 7fa801600640 1 -- 172.21.15.39:0/1183939985 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080c2020 con 0x556ed847d990 2024-07-04T08:16:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.921+0000 7fa82481d740 1 -- 172.21.15.39:0/1183939985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8080b1fe0 con 0x556ed847d990 2024-07-04T08:16:39.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.922+0000 7fa801600640 1 -- 172.21.15.39:0/1183939985 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa8080a5770 con 0x556ed847d990 2024-07-04T08:16:39.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.922+0000 7fa801600640 1 -- 172.21.15.39:0/1183939985 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8080c2020 con 0x556ed847d990 2024-07-04T08:16:39.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.922+0000 7fa82481d740 10 client.15687 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:39.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.922+0000 7fa82481d740 1 -- 172.21.15.39:0/1183939985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ed88773a0 con 0x556ed847d990 2024-07-04T08:16:39.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.923+0000 7fa801600640 1 -- 172.21.15.39:0/1183939985 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8080b1fe0 con 0x556ed847d990 2024-07-04T08:16:39.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.923+0000 7fa801600640 1 -- 172.21.15.39:0/1183939985 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa8080ad120 con 0x556ed847d990 2024-07-04T08:16:39.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.923+0000 7fa82481d740 10 client.15687 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:39.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.923+0000 7fa82481d740 10 client.15687 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:16:39.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.923+0000 7fa82481d740 20 client.15687 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:39.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.923+0000 7fa82481d740 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ed84651d0 0x556ed887a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.923+0000 7fa82481d740 4 client.15687 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:39.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.923+0000 7fa82481d740 1 -- 172.21.15.39:0/1183939985 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ed87c2b80 con 0x556ed84651d0 2024-07-04T08:16:39.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.924+0000 7fa816a00640 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ed84651d0 0x556ed887a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.924+0000 7fa816a00640 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ed84651d0 0x556ed887a700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96fb3abfb3b1a1ea server_cookie=70864f444164b2f8 in_seq=0 out_seq=0 2024-07-04T08:16:39.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.924+0000 7fa801600640 10 client.15687 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:39.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.925+0000 7fa801600640 1 -- 172.21.15.39:0/1183939985 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556ed87c2b80 con 0x556ed84651d0 2024-07-04T08:16:39.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.925+0000 7fa801600640 10 client.15687 handle_command_reply: tid=0 2024-07-04T08:16:39.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.925+0000 7fa82481d740 1 client.15687 shutdown 2024-07-04T08:16:39.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 -- 172.21.15.39:0/1183939985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed847d990 msgr2=0x556ed8857520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed847d990 0x556ed8857520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa8080a9340 tx=0x7fa8080df640 comp rx=0 tx=0).stop 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 -- 172.21.15.39:0/1183939985 shutdown_connections 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ed84651d0 0x556ed887a700 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed84a6850 0x556ed88742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed8485b70 0x556ed8857a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 --2- 172.21.15.39:0/1183939985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed847d990 0x556ed8857520 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 -- 172.21.15.39:0/1183939985 >> 172.21.15.39:0/1183939985 conn(0x556ed849ac40 msgr2=0x556ed8848000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa801600640 0 client.15687 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 -- 172.21.15.39:0/1183939985 shutdown_connections 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.928+0000 7fa82481d740 1 -- 172.21.15.39:0/1183939985 wait complete. 2024-07-04T08:16:39.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.929+0000 7fa82481d740 20 client.15687 trim_cache size 0 max 16384 2024-07-04T08:16:39.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.963+0000 7fa82481d740 1 Processor -- start 2024-07-04T08:16:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.963+0000 7fa82481d740 1 -- start start 2024-07-04T08:16:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.964+0000 7fa82481d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 0x556ed8851170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.964+0000 7fa82481d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed8485b70 0x556ed8874250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.964+0000 7fa82481d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed84a6850 0x556ed8849160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.964+0000 7fa82481d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8011960 con 0x556ed8485b70 2024-07-04T08:16:39.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.964+0000 7fa82481d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f80104f0 con 0x556ed847d990 2024-07-04T08:16:39.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.964+0000 7fa82481d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8010bc0 con 0x556ed84a6850 2024-07-04T08:16:39.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.965+0000 7fa817e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed84a6850 0x556ed8849160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.965+0000 7fa817e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed84a6850 0x556ed8849160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:35070/0 (socket says 172.21.15.39:35070) 2024-07-04T08:16:39.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.965+0000 7fa817e00640 1 -- 172.21.15.39:0/1055333550 learned_addr learned my addr 172.21.15.39:0/1055333550 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:39.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.965+0000 7fa817e00640 1 -- 172.21.15.39:0/1055333550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 msgr2=0x556ed8851170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:16:39.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.965+0000 7fa817e00640 1 --2- 172.21.15.39:0/1055333550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 0x556ed8851170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.966+0000 7fa817e00640 1 -- 172.21.15.39:0/1055333550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed8485b70 msgr2=0x556ed8874250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:16:39.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.966+0000 7fa817e00640 1 --2- 172.21.15.39:0/1055333550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed8485b70 0x556ed8874250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.966+0000 7fa817e00640 1 -- 172.21.15.39:0/1055333550 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed8857be0 con 0x556ed84a6850 2024-07-04T08:16:39.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.966+0000 7fa817e00640 1 --2- 172.21.15.39:0/1055333550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed84a6850 0x556ed8849160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa810003d10 tx=0x7fa810004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435213da75e3d850 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.966+0000 7fa803e00640 1 -- 172.21.15.39:0/1055333550 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810004b20 con 0x556ed84a6850 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.966+0000 7fa803e00640 1 -- 172.21.15.39:0/1055333550 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa810004cc0 con 0x556ed84a6850 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.966+0000 7fa803e00640 1 -- 172.21.15.39:0/1055333550 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810005240 con 0x556ed84a6850 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.968+0000 7fa82481d740 1 -- 172.21.15.39:0/1055333550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed84a6850 msgr2=0x556ed8849160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.968+0000 7fa82481d740 1 --2- 172.21.15.39:0/1055333550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed84a6850 0x556ed8849160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa810003d10 tx=0x7fa810004a40 comp rx=0 tx=0).stop 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.968+0000 7fa82481d740 1 -- 172.21.15.39:0/1055333550 shutdown_connections 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.968+0000 7fa82481d740 1 --2- 172.21.15.39:0/1055333550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed8485b70 0x556ed8874250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.968+0000 7fa82481d740 1 --2- 172.21.15.39:0/1055333550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed84a6850 0x556ed8849160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.968+0000 7fa82481d740 1 --2- 172.21.15.39:0/1055333550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed847d990 0x556ed8851170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.968+0000 7fa82481d740 1 -- 172.21.15.39:0/1055333550 >> 172.21.15.39:0/1055333550 conn(0x556ed849ac40 msgr2=0x556ed8850a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:39.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.969+0000 7fa82481d740 1 -- 172.21.15.39:0/1055333550 shutdown_connections 2024-07-04T08:16:39.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.969+0000 7fa82481d740 1 -- 172.21.15.39:0/1055333550 wait complete. 2024-07-04T08:16:39.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.969+0000 7fa82481d740 1 -- start start 2024-07-04T08:16:39.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.970+0000 7fa82481d740 1 Processor -- start 2024-07-04T08:16:39.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.970+0000 7fa82481d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed8485b70 0x556ed887a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.970+0000 7fa82481d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed887ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.970+0000 7fa82481d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed843a580 0x556ed88e0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.970+0000 7fa82481d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f800de00 con 0x556ed84a6850 2024-07-04T08:16:39.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.970+0000 7fa82481d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8011290 con 0x556ed8485b70 2024-07-04T08:16:39.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.970+0000 7fa82481d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f800e470 con 0x556ed843a580 2024-07-04T08:16:39.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.971+0000 7fa82481d740 10 client.0 _ll_register_callbacks cb 0x7fa8080a88f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:39.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.971+0000 7fa817400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed8485b70 0x556ed887a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.972+0000 7fa817400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed8485b70 0x556ed887a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48814/0 (socket says 172.21.15.39:48814) 2024-07-04T08:16:39.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.972+0000 7fa817400640 1 -- 172.21.15.39:0/3205127286 learned_addr learned my addr 172.21.15.39:0/3205127286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:39.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.972+0000 7fa817e00640 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed843a580 0x556ed88e0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.972+0000 7fa816a00640 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed887ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.973+0000 7fa816a00640 1 -- 172.21.15.39:0/3205127286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed843a580 msgr2=0x556ed88e0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.973+0000 7fa816a00640 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed843a580 0x556ed88e0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.973+0000 7fa816a00640 1 -- 172.21.15.39:0/3205127286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed8485b70 msgr2=0x556ed887a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.973+0000 7fa816a00640 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed8485b70 0x556ed887a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.973+0000 7fa816a00640 1 -- 172.21.15.39:0/3205127286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed887b750 con 0x556ed84a6850 2024-07-04T08:16:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.973+0000 7fa817400640 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed8485b70 0x556ed887a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.974+0000 7fa816a00640 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed887ae80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa804009230 tx=0x7fa804004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5a23562824ada server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:39.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.974+0000 7fa801600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:16:39.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.975+0000 7fa801600640 1 -- 172.21.15.39:0/3205127286 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804035040 con 0x556ed84a6850 2024-07-04T08:16:39.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.975+0000 7fa801600640 1 -- 172.21.15.39:0/3205127286 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa804005510 con 0x556ed84a6850 2024-07-04T08:16:39.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.975+0000 7fa82481d740 1 -- 172.21.15.39:0/3205127286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa810004cc0 con 0x556ed84a6850 2024-07-04T08:16:39.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.975+0000 7fa801600640 1 -- 172.21.15.39:0/3205127286 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa804053050 con 0x556ed84a6850 2024-07-04T08:16:39.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.975+0000 7fa82481d740 10 client.15690 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:39.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.975+0000 7fa82481d740 1 -- 172.21.15.39:0/3205127286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ed88e21e0 con 0x556ed84a6850 2024-07-04T08:16:39.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.976+0000 7fa801600640 1 -- 172.21.15.39:0/3205127286 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa810004cc0 con 0x556ed84a6850 2024-07-04T08:16:39.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.976+0000 7fa801600640 1 -- 172.21.15.39:0/3205127286 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa804045650 con 0x556ed84a6850 2024-07-04T08:16:39.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.976+0000 7fa82481d740 10 client.15690 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:39.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.976+0000 7fa82481d740 10 client.15690 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:16:39.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.976+0000 7fa82481d740 20 client.15690 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:39.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.976+0000 7fa82481d740 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ed88e29f0 0x556ed88e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:39.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.977+0000 7fa82481d740 4 client.15690 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-07-04T08:16:39.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.977+0000 7fa82481d740 1 -- 172.21.15.39:0/3205127286 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x556ed87be620 con 0x556ed88e29f0 2024-07-04T08:16:39.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.977+0000 7fa817400640 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ed88e29f0 0x556ed88e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:39.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.978+0000 7fa817400640 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ed88e29f0 0x556ed88e4e10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6dd2483e5db993 server_cookie=bfe41a97b18e49f6 in_seq=0 out_seq=0 2024-07-04T08:16:39.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.978+0000 7fa801600640 10 client.15690 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:39.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.986+0000 7fa801600640 1 -- 172.21.15.39:0/3205127286 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556ed87be620 con 0x556ed88e29f0 2024-07-04T08:16:39.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.986+0000 7fa801600640 10 client.15690 handle_command_reply: tid=0 2024-07-04T08:16:39.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.986+0000 7fa82481d740 1 client.15690 shutdown 2024-07-04T08:16:39.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.987+0000 7fa82481d740 1 -- 172.21.15.39:0/3205127286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 msgr2=0x556ed887ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:39.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.987+0000 7fa82481d740 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed887ae80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa804009230 tx=0x7fa804004560 comp rx=0 tx=0).stop 2024-07-04T08:16:39.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.987+0000 7fa82481d740 1 -- 172.21.15.39:0/3205127286 shutdown_connections 2024-07-04T08:16:39.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.987+0000 7fa82481d740 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ed88e29f0 0x556ed88e4e10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.988+0000 7fa82481d740 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ed84a6850 0x556ed887ae80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.988+0000 7fa82481d740 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ed843a580 0x556ed88e0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.988+0000 7fa82481d740 1 --2- 172.21.15.39:0/3205127286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ed8485b70 0x556ed887a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.988+0000 7fa82481d740 1 -- 172.21.15.39:0/3205127286 >> 172.21.15.39:0/3205127286 conn(0x556ed849ac40 msgr2=0x556ed8849120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:39.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.988+0000 7fa801600640 0 client.15690 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:39.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.989+0000 7fa82481d740 1 -- 172.21.15.39:0/3205127286 shutdown_connections 2024-07-04T08:16:39.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.989+0000 7fa82481d740 1 -- 172.21.15.39:0/3205127286 wait complete. 2024-07-04T08:16:39.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.989+0000 7fa82481d740 20 client.15690 trim_cache size 0 max 16384 2024-07-04T08:16:39.991 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:16:39.991 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:16:39.991 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:16:39.991 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 2, 2024-07-04T08:16:39.991 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:16:39.991 INFO:teuthology.orchestra.run.smithi039.stdout: "1ec7ed84": { 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 2, 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 142.8, 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:16:39.992 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:16:39.993 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:39.993 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:39.993 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:39.993 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:39.993 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:39.993 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:16:39.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.990+0000 7fa81cc00640 1 -- 172.21.15.39:0/529711926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa7fc0b2a50 msgr2=0x7fa7fc0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:39.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.990+0000 7fa81cc00640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa7fc0b2a50 0x7fa7fc0b4f60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa810009fd0 tx=0x7fa8100097a0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.990+0000 7fa81cc00640 1 -- 172.21.15.39:0/529711926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818102f10 msgr2=0x7fa818076670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.990+0000 7fa81cc00640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818102f10 0x7fa818076670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa80800d950 tx=0x7fa80800de10 comp rx=0 tx=0).stop 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.991+0000 7fa81cc00640 1 -- 172.21.15.39:0/529711926 shutdown_connections 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.991+0000 7fa81cc00640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa7fc0b2a50 0x7fa7fc0b4f60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.991+0000 7fa81cc00640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa818108850 0x7fa818074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.991+0000 7fa81cc00640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8181091a0 0x7fa8180751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.991+0000 7fa81cc00640 1 --2- 172.21.15.39:0/529711926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa818102f10 0x7fa818076670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.991+0000 7fa81cc00640 1 -- 172.21.15.39:0/529711926 >> 172.21.15.39:0/529711926 conn(0x7fa8180fe980 msgr2=0x7fa818100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.993+0000 7fa81cc00640 1 -- 172.21.15.39:0/529711926 shutdown_connections 2024-07-04T08:16:39.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:39.993+0000 7fa81cc00640 1 -- 172.21.15.39:0/529711926 wait complete. 2024-07-04T08:16:41.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:40 smithi133 ceph-mon[37050]: pgmap v524: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-04T08:16:41.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:40 smithi039 ceph-mon[32062]: pgmap v524: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-04T08:16:41.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:40 smithi050 ceph-mon[36577]: pgmap v524: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-04T08:16:41.475 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1ec7ed84', quiesce took 15.7 seconds. Will release after: 7.050000000000001 2024-07-04T08:16:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.855+0000 7fefb6c00640 1 -- 172.21.15.39:0/4107685820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb0103f00 msgr2=0x7fefb0104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.855+0000 7fefb6c00640 1 --2- 172.21.15.39:0/4107685820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb0103f00 0x7fefb0104310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009860 tx=0x7fefa002a340 comp rx=0 tx=0).stop 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.855+0000 7fefb6c00640 1 -- 172.21.15.39:0/4107685820 shutdown_connections 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.855+0000 7fefb6c00640 1 --2- 172.21.15.39:0/4107685820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb0104850 0x7fefb010cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.855+0000 7fefb6c00640 1 --2- 172.21.15.39:0/4107685820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb010d360 0x7fefb010f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.855+0000 7fefb6c00640 1 --2- 172.21.15.39:0/4107685820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb0103f00 0x7fefb0104310 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.855+0000 7fefb6c00640 1 -- 172.21.15.39:0/4107685820 >> 172.21.15.39:0/4107685820 conn(0x7fefb0078c80 msgr2=0x7fefb0079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.855+0000 7fefb6c00640 1 -- 172.21.15.39:0/4107685820 shutdown_connections 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.855+0000 7fefb6c00640 1 -- 172.21.15.39:0/4107685820 wait complete. 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.857+0000 7fefb6c00640 1 Processor -- start 2024-07-04T08:16:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.857+0000 7fefb6c00640 1 -- start start 2024-07-04T08:16:42.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefb6c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb0103f00 0x7fefb0103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefb6c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb0104850 0x7fefb0101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefb6c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb010d360 0x7fefb0102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefb6c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0112fd0 con 0x7fefb0103f00 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefb6c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0112e50 con 0x7fefb010d360 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefb6c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fefb0113150 con 0x7fefb0104850 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefaf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb0104850 0x7fefb0101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefb4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb010d360 0x7fefb0102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefaf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb0104850 0x7fefb0101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50680/0 (socket says 172.21.15.39:50680) 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.858+0000 7fefaf400640 1 -- 172.21.15.39:0/2593799485 learned_addr learned my addr 172.21.15.39:0/2593799485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.859+0000 7fefb4c00640 1 -- 172.21.15.39:0/2593799485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb0104850 msgr2=0x7fefb0101c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.859+0000 7fefb4c00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb0104850 0x7fefb0101c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.859+0000 7fefb4c00640 1 -- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb0103f00 msgr2=0x7fefb0103650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.859+0000 7fefafe00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb0103f00 0x7fefb0103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.859+0000 7fefb4c00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb0103f00 0x7fefb0103650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.859+0000 7fefb4c00640 1 -- 172.21.15.39:0/2593799485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb01029a0 con 0x7fefb010d360 2024-07-04T08:16:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.859+0000 7fefafe00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb0103f00 0x7fefb0103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:16:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.860+0000 7fefb4c00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb010d360 0x7fefb0102180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009fd0 tx=0x7fefa800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.860+0000 7fefacc00640 1 -- 172.21.15.39:0/2593799485 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa800ce20 con 0x7fefb010d360 2024-07-04T08:16:42.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.860+0000 7fefacc00640 1 -- 172.21.15.39:0/2593799485 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa8005020 con 0x7fefb010d360 2024-07-04T08:16:42.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.860+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb01ac400 con 0x7fefb010d360 2024-07-04T08:16:42.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.860+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb01ac940 con 0x7fefb010d360 2024-07-04T08:16:42.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.860+0000 7fefacc00640 1 -- 172.21.15.39:0/2593799485 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa8010690 con 0x7fefb010d360 2024-07-04T08:16:42.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.862+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef900052c0 con 0x7fefb010d360 2024-07-04T08:16:42.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.864+0000 7fefacc00640 1 -- 172.21.15.39:0/2593799485 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fefa8004b20 con 0x7fefb010d360 2024-07-04T08:16:42.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.867+0000 7fefacc00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef940b29e0 0x7fef940b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:42.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.867+0000 7fefafe00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef940b29e0 0x7fef940b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:42.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.867+0000 7fefacc00640 1 -- 172.21.15.39:0/2593799485 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fefa80e2f70 con 0x7fefb010d360 2024-07-04T08:16:42.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.868+0000 7fefafe00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef940b29e0 0x7fef940b4ef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fefa002a860 tx=0x7fefa0030340 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:42.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:42.870+0000 7fefacc00640 1 -- 172.21.15.39:0/2593799485 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fefa80aa5c0 con 0x7fefb010d360 2024-07-04T08:16:43.066 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:42 smithi133 ceph-mon[37050]: pgmap v525: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-07-04T08:16:43.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:42 smithi039 ceph-mon[32062]: pgmap v525: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-07-04T08:16:43.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.117+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef90005880 con 0x7fefb010d360 2024-07-04T08:16:43.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.118+0000 7fefacc00640 1 -- 172.21.15.39:0/2593799485 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fefa80a9ec0 con 0x7fefb010d360 2024-07-04T08:16:43.120 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:16:43.120 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:16:43.121 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:16:43.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.121+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef940b29e0 msgr2=0x7fef940b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:43.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.121+0000 7fefb6c00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef940b29e0 0x7fef940b4ef0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fefa002a860 tx=0x7fefa0030340 comp rx=0 tx=0).stop 2024-07-04T08:16:43.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.121+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb010d360 msgr2=0x7fefb0102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:43.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.121+0000 7fefb6c00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb010d360 0x7fefb0102180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009fd0 tx=0x7fefa800ef10 comp rx=0 tx=0).stop 2024-07-04T08:16:43.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.122+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 shutdown_connections 2024-07-04T08:16:43.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.122+0000 7fefb6c00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef940b29e0 0x7fef940b4ef0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:43.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.122+0000 7fefb6c00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb010d360 0x7fefb0102180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:43.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.122+0000 7fefb6c00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb0104850 0x7fefb0101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:43.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.122+0000 7fefb6c00640 1 --2- 172.21.15.39:0/2593799485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefb0103f00 0x7fefb0103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:43.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.122+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 >> 172.21.15.39:0/2593799485 conn(0x7fefb0078c80 msgr2=0x7fefb00ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:43.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.122+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 shutdown_connections 2024-07-04T08:16:43.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:43.122+0000 7fefb6c00640 1 -- 172.21.15.39:0/2593799485 wait complete. 2024-07-04T08:16:43.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:42 smithi050 ceph-mon[36577]: pgmap v525: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-07-04T08:16:44.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:16:44.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:43 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2593799485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:44.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:43 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2593799485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:44.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:43 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2593799485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:44.496 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:44.941 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:44 smithi039 ceph-mon[32062]: pgmap v526: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-07-04T08:16:45.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:44 smithi050 ceph-mon[36577]: pgmap v526: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-07-04T08:16:45.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:44 smithi133 ceph-mon[37050]: pgmap v526: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-07-04T08:16:46.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.794+0000 7fa627e00640 1 -- 172.21.15.39:0/4174222699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa628103d10 msgr2=0x7fa628104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:46.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.794+0000 7fa627e00640 1 --2- 172.21.15.39:0/4174222699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa628103d10 0x7fa628104120 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa618009900 tx=0x7fa61802a410 comp rx=0 tx=0).stop 2024-07-04T08:16:46.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.794+0000 7fa627e00640 1 -- 172.21.15.39:0/4174222699 shutdown_connections 2024-07-04T08:16:46.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.794+0000 7fa627e00640 1 --2- 172.21.15.39:0/4174222699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa62810d170 0x7fa62810f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.794+0000 7fa627e00640 1 --2- 172.21.15.39:0/4174222699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa628104660 0x7fa62810cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.794+0000 7fa627e00640 1 --2- 172.21.15.39:0/4174222699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa628103d10 0x7fa628104120 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.794+0000 7fa627e00640 1 -- 172.21.15.39:0/4174222699 >> 172.21.15.39:0/4174222699 conn(0x7fa6280fd810 msgr2=0x7fa6280ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:46.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.795+0000 7fa627e00640 1 -- 172.21.15.39:0/4174222699 shutdown_connections 2024-07-04T08:16:46.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.796+0000 7fa627e00640 1 -- 172.21.15.39:0/4174222699 wait complete. 2024-07-04T08:16:46.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.797+0000 7fa627e00640 1 Processor -- start 2024-07-04T08:16:46.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.797+0000 7fa627e00640 1 -- start start 2024-07-04T08:16:46.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.798+0000 7fa627e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa628103d10 0x7fa6281997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.798+0000 7fa627e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa628104660 0x7fa628199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.798+0000 7fa627e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa62810d170 0x7fa62819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.798+0000 7fa627e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa628112e20 con 0x7fa62810d170 2024-07-04T08:16:46.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.798+0000 7fa627e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa628112ca0 con 0x7fa628103d10 2024-07-04T08:16:46.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.798+0000 7fa627e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa628112fa0 con 0x7fa628104660 2024-07-04T08:16:46.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.798+0000 7fa626a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa628103d10 0x7fa6281997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.798+0000 7fa626a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa628103d10 0x7fa6281997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57438/0 (socket says 172.21.15.39:57438) 2024-07-04T08:16:46.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.798+0000 7fa626a00640 1 -- 172.21.15.39:0/2415704874 learned_addr learned my addr 172.21.15.39:0/2415704874 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:46.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.799+0000 7fa626000640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa628104660 0x7fa628199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.799+0000 7fa627400640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa62810d170 0x7fa62819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.799+0000 7fa626a00640 1 -- 172.21.15.39:0/2415704874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa628104660 msgr2=0x7fa628199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:46.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.799+0000 7fa626a00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa628104660 0x7fa628199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.799+0000 7fa626a00640 1 -- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa62810d170 msgr2=0x7fa62819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:46.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.799+0000 7fa626a00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa62810d170 0x7fa62819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.799+0000 7fa626a00640 1 -- 172.21.15.39:0/2415704874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62819f760 con 0x7fa628103d10 2024-07-04T08:16:46.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.800+0000 7fa626a00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa628103d10 0x7fa6281997f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa618005ab0 tx=0x7fa618003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:46.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.800+0000 7fa61f400640 1 -- 172.21.15.39:0/2415704874 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61803f070 con 0x7fa628103d10 2024-07-04T08:16:46.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.801+0000 7fa61f400640 1 -- 172.21.15.39:0/2415704874 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa618031d60 con 0x7fa628103d10 2024-07-04T08:16:46.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.801+0000 7fa61f400640 1 -- 172.21.15.39:0/2415704874 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618035030 con 0x7fa628103d10 2024-07-04T08:16:46.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.801+0000 7fa627e00640 1 -- 172.21.15.39:0/2415704874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62819f9f0 con 0x7fa628103d10 2024-07-04T08:16:46.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.802+0000 7fa627e00640 1 -- 172.21.15.39:0/2415704874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62819fe50 con 0x7fa628103d10 2024-07-04T08:16:46.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.805+0000 7fa633bdb740 1 Processor -- start 2024-07-04T08:16:46.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.806+0000 7fa633bdb740 1 -- start start 2024-07-04T08:16:46.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.806+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d9c070 0x5615b9140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.806+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d731b0 0x5615b9140e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.807+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d7b390 0x5615b91413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.807+0000 7fa633bdb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5615b90b6ee0 con 0x5615b8d9c070 2024-07-04T08:16:46.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.807+0000 7fa633bdb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5615b90a6480 con 0x5615b8d731b0 2024-07-04T08:16:46.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.807+0000 7fa633bdb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5615b90aa4c0 con 0x5615b8d7b390 2024-07-04T08:16:46.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.807+0000 7fa627400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d9c070 0x5615b9140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.807+0000 7fa627400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d9c070 0x5615b9140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39564/0 (socket says 172.21.15.39:39564) 2024-07-04T08:16:46.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.807+0000 7fa627400640 1 -- 172.21.15.39:0/2880658389 learned_addr learned my addr 172.21.15.39:0/2880658389 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:46.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.808+0000 7fa627400640 1 -- 172.21.15.39:0/2880658389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d7b390 msgr2=0x5615b91413d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:16:46.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.808+0000 7fa627400640 1 --2- 172.21.15.39:0/2880658389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d7b390 0x5615b91413d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.808+0000 7fa627400640 1 -- 172.21.15.39:0/2880658389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d731b0 msgr2=0x5615b9140e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:16:46.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.808+0000 7fa627400640 1 --2- 172.21.15.39:0/2880658389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d731b0 0x5615b9140e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.808+0000 7fa627400640 1 -- 172.21.15.39:0/2880658389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615b903e650 con 0x5615b8d9c070 2024-07-04T08:16:46.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.808+0000 7fa627400640 1 --2- 172.21.15.39:0/2880658389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d9c070 0x5615b9140950 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa620009fd0 tx=0x7fa620015ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436311b52cb4090a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:46.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.809+0000 7fa613e00640 1 -- 172.21.15.39:0/2880658389 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620004d20 con 0x5615b8d9c070 2024-07-04T08:16:46.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.809+0000 7fa613e00640 1 -- 172.21.15.39:0/2880658389 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62000f9a0 con 0x5615b8d9c070 2024-07-04T08:16:46.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.809+0000 7fa613e00640 1 -- 172.21.15.39:0/2880658389 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620004470 con 0x5615b8d9c070 2024-07-04T08:16:46.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.809+0000 7fa61f400640 1 -- 172.21.15.39:0/2415704874 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa618042070 con 0x7fa628103d10 2024-07-04T08:16:46.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.810+0000 7fa633bdb740 1 -- 172.21.15.39:0/2880658389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d9c070 msgr2=0x5615b9140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:46.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.810+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2880658389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d9c070 0x5615b9140950 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa620009fd0 tx=0x7fa620015ce0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.810+0000 7fa633bdb740 1 -- 172.21.15.39:0/2880658389 shutdown_connections 2024-07-04T08:16:46.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.810+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2880658389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d7b390 0x5615b91413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.810+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2880658389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d731b0 0x5615b9140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.810+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2880658389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d9c070 0x5615b9140950 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.810+0000 7fa633bdb740 1 -- 172.21.15.39:0/2880658389 >> 172.21.15.39:0/2880658389 conn(0x5615b8d90460 msgr2=0x5615b913e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:46.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.810+0000 7fa633bdb740 1 -- 172.21.15.39:0/2880658389 shutdown_connections 2024-07-04T08:16:46.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.810+0000 7fa633bdb740 1 -- 172.21.15.39:0/2880658389 wait complete. 2024-07-04T08:16:46.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.811+0000 7fa633bdb740 1 -- start start 2024-07-04T08:16:46.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.811+0000 7fa633bdb740 1 Processor -- start 2024-07-04T08:16:46.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.812+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.812+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d7b390 0x5615b9145580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.812+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 0x5615b914a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.812+0000 7fa633bdb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5615b8f0a700 con 0x5615b8d731b0 2024-07-04T08:16:46.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.812+0000 7fa633bdb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5615b904dbb0 con 0x5615b8d7b390 2024-07-04T08:16:46.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.812+0000 7fa633bdb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5615b914f130 con 0x5615b8d9c070 2024-07-04T08:16:46.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.813+0000 7fa627400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.813+0000 7fa627400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9144fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39580/0 (socket says 172.21.15.39:39580) 2024-07-04T08:16:46.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.813+0000 7fa627400640 1 -- 172.21.15.39:0/4053062707 learned_addr learned my addr 172.21.15.39:0/4053062707 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:46.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.813+0000 7fa627400640 1 -- 172.21.15.39:0/4053062707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 msgr2=0x5615b914a5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:16:46.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.813+0000 7fa627400640 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 0x5615b914a5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.813+0000 7fa627400640 1 -- 172.21.15.39:0/4053062707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d7b390 msgr2=0x5615b9145580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:16:46.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.813+0000 7fa627400640 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d7b390 0x5615b9145580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.813+0000 7fa627400640 1 -- 172.21.15.39:0/4053062707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615b8e07650 con 0x5615b8d731b0 2024-07-04T08:16:46.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.814+0000 7fa627400640 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9144fc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa620006010 tx=0x7fa620009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff86f4a36c363a01 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:46.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.814+0000 7fa61f400640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa60c0b2970 0x7fa60c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.814+0000 7fa610c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:16:46.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.814+0000 7fa61f400640 1 -- 172.21.15.39:0/2415704874 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa618103130 con 0x7fa628103d10 2024-07-04T08:16:46.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.814+0000 7fa610c00640 1 -- 172.21.15.39:0/4053062707 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62002b060 con 0x5615b8d731b0 2024-07-04T08:16:46.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.814+0000 7fa610c00640 1 -- 172.21.15.39:0/4053062707 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa620003c40 con 0x5615b8d731b0 2024-07-04T08:16:46.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.815+0000 7fa626a00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa60c0b2970 0x7fa60c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.816+0000 7fa626a00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa60c0b2970 0x7fa60c0b4e80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5615b913f5d0 tx=0x7fa618000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:46.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.816+0000 7fa633bdb740 10 client.0 _ll_register_callbacks cb 0x5615b909f570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:46.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.816+0000 7fa633bdb740 1 -- 172.21.15.39:0/4053062707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa62000f9a0 con 0x5615b8d731b0 2024-07-04T08:16:46.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.817+0000 7fa610c00640 1 -- 172.21.15.39:0/4053062707 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa620003e40 con 0x5615b8d731b0 2024-07-04T08:16:46.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.817+0000 7fa633bdb740 10 client.15705 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:46.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.817+0000 7fa633bdb740 1 -- 172.21.15.39:0/4053062707 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5615b916c090 con 0x5615b8d731b0 2024-07-04T08:16:46.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.817+0000 7fa610c00640 1 -- 172.21.15.39:0/4053062707 <== mon.0 v2:172.21.15.39:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa62001c070 con 0x5615b8d731b0 2024-07-04T08:16:46.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.817+0000 7fa633bdb740 10 client.15705 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:46.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.817+0000 7fa633bdb740 10 client.15705 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:16:46.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.817+0000 7fa633bdb740 20 client.15705 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:46.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.817+0000 7fa633bdb740 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5615b8d5a9f0 0x5615b916f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.818+0000 7fa633bdb740 4 client.15705 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:46.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.818+0000 7fa633bdb740 1 -- 172.21.15.39:0/4053062707 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa62000f9a0 con 0x5615b8d5a9f0 2024-07-04T08:16:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.818+0000 7fa626000640 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5615b8d5a9f0 0x5615b916f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.819+0000 7fa610c00640 10 client.15705 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:16:46.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.819+0000 7fa626000640 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5615b8d5a9f0 0x5615b916f3f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=593ea99cdce9c086 server_cookie=dc30ecc4af50b24c in_seq=0 out_seq=0 2024-07-04T08:16:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.820+0000 7fa610c00640 1 -- 172.21.15.39:0/4053062707 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fa62000f9a0 con 0x5615b8d5a9f0 2024-07-04T08:16:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.820+0000 7fa610c00640 10 client.15705 handle_command_reply: tid=0 2024-07-04T08:16:46.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.820+0000 7fa633bdb740 1 client.15705 shutdown 2024-07-04T08:16:46.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.820+0000 7fa633bdb740 1 -- 172.21.15.39:0/4053062707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 msgr2=0x5615b9144fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:46.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.820+0000 7fa633bdb740 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9144fc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa620006010 tx=0x7fa620009be0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa633bdb740 1 -- 172.21.15.39:0/4053062707 shutdown_connections 2024-07-04T08:16:46.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa633bdb740 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5615b8d5a9f0 0x5615b916f3f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa633bdb740 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 0x5615b914a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa633bdb740 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d7b390 0x5615b9145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa633bdb740 1 --2- 172.21.15.39:0/4053062707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9144fc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa633bdb740 1 -- 172.21.15.39:0/4053062707 >> 172.21.15.39:0/4053062707 conn(0x5615b8d90460 msgr2=0x5615b9143b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa610c00640 0 client.15705 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:16:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa633bdb740 1 -- 172.21.15.39:0/4053062707 shutdown_connections 2024-07-04T08:16:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa633bdb740 1 -- 172.21.15.39:0/4053062707 wait complete. 2024-07-04T08:16:46.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.821+0000 7fa633bdb740 20 client.15705 trim_cache size 0 max 16384 2024-07-04T08:16:46.851 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:46 smithi039 ceph-mon[32062]: pgmap v527: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-07-04T08:16:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.851+0000 7fa633bdb740 1 Processor -- start 2024-07-04T08:16:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.851+0000 7fa633bdb740 1 -- start start 2024-07-04T08:16:46.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.851+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d731b0 0x5615b914e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.851+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d7b390 0x5615b9142340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.851+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d9c070 0x5615b914a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.851+0000 7fa633bdb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281ade40 con 0x5615b8d7b390 2024-07-04T08:16:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.851+0000 7fa633bdb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281af980 con 0x5615b8d9c070 2024-07-04T08:16:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.851+0000 7fa633bdb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281ae510 con 0x5615b8d731b0 2024-07-04T08:16:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.852+0000 7fa627400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d9c070 0x5615b914a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.852+0000 7fa627400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d9c070 0x5615b914a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57446/0 (socket says 172.21.15.39:57446) 2024-07-04T08:16:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.852+0000 7fa627400640 1 -- 172.21.15.39:0/2765422089 learned_addr learned my addr 172.21.15.39:0/2765422089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.852+0000 7fa627400640 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d731b0 0x5615b914e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.852+0000 7fa626000640 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d7b390 0x5615b9142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.853+0000 7fa627400640 1 -- 172.21.15.39:0/2765422089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d731b0 msgr2=0x5615b914e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.853+0000 7fa627400640 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d731b0 0x5615b914e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.853+0000 7fa627400640 1 -- 172.21.15.39:0/2765422089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d7b390 msgr2=0x5615b9142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.853+0000 7fa627400640 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d7b390 0x5615b9142340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.853+0000 7fa627400640 1 -- 172.21.15.39:0/2765422089 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615b9146080 con 0x5615b8d9c070 2024-07-04T08:16:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.853+0000 7fa626000640 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d7b390 0x5615b9142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.853+0000 7fa627400640 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d9c070 0x5615b914a3a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa620009de0 tx=0x7fa620023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6abc3e1aae3338e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.853+0000 7fa613400640 1 -- 172.21.15.39:0/2765422089 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62002b040 con 0x5615b8d9c070 2024-07-04T08:16:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.854+0000 7fa613400640 1 -- 172.21.15.39:0/2765422089 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6200155c0 con 0x5615b8d9c070 2024-07-04T08:16:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.854+0000 7fa613400640 1 -- 172.21.15.39:0/2765422089 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620035060 con 0x5615b8d9c070 2024-07-04T08:16:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.855+0000 7fa633bdb740 1 -- 172.21.15.39:0/2765422089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d9c070 msgr2=0x5615b914a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.855+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d9c070 0x5615b914a3a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa620009de0 tx=0x7fa620023000 comp rx=0 tx=0).stop 2024-07-04T08:16:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.855+0000 7fa633bdb740 1 -- 172.21.15.39:0/2765422089 shutdown_connections 2024-07-04T08:16:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.855+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d7b390 0x5615b9142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.855+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d9c070 0x5615b914a3a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.855+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2765422089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d731b0 0x5615b914e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.855+0000 7fa633bdb740 1 -- 172.21.15.39:0/2765422089 >> 172.21.15.39:0/2765422089 conn(0x5615b8d90460 msgr2=0x5615b9143a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.855+0000 7fa633bdb740 1 -- 172.21.15.39:0/2765422089 shutdown_connections 2024-07-04T08:16:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.855+0000 7fa633bdb740 1 -- 172.21.15.39:0/2765422089 wait complete. 2024-07-04T08:16:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.856+0000 7fa633bdb740 1 -- start start 2024-07-04T08:16:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.856+0000 7fa633bdb740 1 Processor -- start 2024-07-04T08:16:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.857+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9166650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.857+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d7b390 0x5615b9166c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.857+0000 7fa633bdb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 0x5615b91d5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.857+0000 7fa633bdb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281af2b0 con 0x5615b8d731b0 2024-07-04T08:16:46.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.857+0000 7fa633bdb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281aebe0 con 0x5615b8d7b390 2024-07-04T08:16:46.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.857+0000 7fa633bdb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281abe20 con 0x5615b8d9c070 2024-07-04T08:16:46.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.857+0000 7fa633bdb740 10 client.0 _ll_register_callbacks cb 0x7fa620009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:46.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.858+0000 7fa627400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9166650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.858+0000 7fa627400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 0x5615b91d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.858+0000 7fa627400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 0x5615b91d5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50734/0 (socket says 172.21.15.39:50734) 2024-07-04T08:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.858+0000 7fa627400640 1 -- 172.21.15.39:0/2261077392 learned_addr learned my addr 172.21.15.39:0/2261077392 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:46.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.858+0000 7fa626000640 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d7b390 0x5615b9166c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.859+0000 7fa627400640 1 -- 172.21.15.39:0/2261077392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 msgr2=0x5615b91d5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:46.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.859+0000 7fa627400640 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 0x5615b91d5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.859+0000 7fa627400640 1 -- 172.21.15.39:0/2261077392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d7b390 msgr2=0x5615b9166c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:46.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.859+0000 7fa627400640 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d7b390 0x5615b9166c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.860+0000 7fa627400640 1 -- 172.21.15.39:0/2261077392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615b91d5f80 con 0x5615b8d731b0 2024-07-04T08:16:46.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.860+0000 7fa627400640 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9166650 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa620009690 tx=0x7fa620015050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab3d7147f0e7620 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.860+0000 7fa610c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:16:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.861+0000 7fa610c00640 1 -- 172.21.15.39:0/2261077392 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62002b040 con 0x5615b8d731b0 2024-07-04T08:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.861+0000 7fa610c00640 1 -- 172.21.15.39:0/2261077392 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa620004820 con 0x5615b8d731b0 2024-07-04T08:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.862+0000 7fa633bdb740 1 -- 172.21.15.39:0/2261077392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6200155c0 con 0x5615b8d731b0 2024-07-04T08:16:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.862+0000 7fa610c00640 1 -- 172.21.15.39:0/2261077392 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa6200040c0 con 0x5615b8d731b0 2024-07-04T08:16:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.862+0000 7fa633bdb740 10 client.15711 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.862+0000 7fa633bdb740 1 -- 172.21.15.39:0/2261077392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615b91d7110 con 0x5615b8d731b0 2024-07-04T08:16:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.863+0000 7fa610c00640 1 -- 172.21.15.39:0/2261077392 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6200155c0 con 0x5615b8d731b0 2024-07-04T08:16:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.863+0000 7fa610c00640 1 -- 172.21.15.39:0/2261077392 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa62001c070 con 0x5615b8d731b0 2024-07-04T08:16:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.864+0000 7fa633bdb740 10 client.15711 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.864+0000 7fa633bdb740 10 client.15711 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:16:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.864+0000 7fa633bdb740 20 client.15711 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.864+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5615b8d2fda0 0x5615b91d9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:46.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.864+0000 7fa633bdb740 4 client.15711 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:16:46.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.864+0000 7fa633bdb740 1 -- 172.21.15.39:0/2261077392 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615b90b8e30 con 0x5615b8d2fda0 2024-07-04T08:16:46.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.865+0000 7fa626000640 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5615b8d2fda0 0x5615b91d9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.868+0000 7fa626000640 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5615b8d2fda0 0x5615b91d9730 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f51db9da90b65639 server_cookie=6f57181940837cad in_seq=0 out_seq=0 2024-07-04T08:16:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.868+0000 7fa610c00640 10 client.15711 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:16:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.869+0000 7fa610c00640 1 -- 172.21.15.39:0/2261077392 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615b90b8e30 con 0x5615b8d2fda0 2024-07-04T08:16:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.869+0000 7fa610c00640 10 client.15711 handle_command_reply: tid=0 2024-07-04T08:16:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.869+0000 7fa633bdb740 1 client.15711 shutdown 2024-07-04T08:16:46.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.869+0000 7fa633bdb740 1 -- 172.21.15.39:0/2261077392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 msgr2=0x5615b9166650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:46.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.869+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9166650 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa620009690 tx=0x7fa620015050 comp rx=0 tx=0).stop 2024-07-04T08:16:46.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa633bdb740 1 -- 172.21.15.39:0/2261077392 shutdown_connections 2024-07-04T08:16:46.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5615b8d2fda0 0x5615b91d9730 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5615b8d9c070 0x5615b91d5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5615b8d7b390 0x5615b9166c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa633bdb740 1 --2- 172.21.15.39:0/2261077392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5615b8d731b0 0x5615b9166650 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa633bdb740 1 -- 172.21.15.39:0/2261077392 >> 172.21.15.39:0/2261077392 conn(0x5615b8d90460 msgr2=0x5615b9142110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa610c00640 0 client.15711 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:16:46.872 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa633bdb740 1 -- 172.21.15.39:0/2261077392 shutdown_connections 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa633bdb740 1 -- 172.21.15.39:0/2261077392 wait complete. 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.870+0000 7fa633bdb740 20 client.15711 trim_cache size 0 max 16384 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.871+0000 7fa627e00640 1 -- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa60c0b2970 msgr2=0x7fa60c0b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.871+0000 7fa627e00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa60c0b2970 0x7fa60c0b4e80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5615b913f5d0 tx=0x7fa618000bc0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.871+0000 7fa627e00640 1 -- 172.21.15.39:0/2415704874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa628103d10 msgr2=0x7fa6281997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.871+0000 7fa627e00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa628103d10 0x7fa6281997f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa618005ab0 tx=0x7fa618003c10 comp rx=0 tx=0).stop 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.872+0000 7fa627e00640 1 -- 172.21.15.39:0/2415704874 shutdown_connections 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.872+0000 7fa627e00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa60c0b2970 0x7fa60c0b4e80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.872+0000 7fa627e00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa62810d170 0x7fa62819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.872+0000 7fa627e00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa628104660 0x7fa628199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.872+0000 7fa627e00640 1 --2- 172.21.15.39:0/2415704874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa628103d10 0x7fa6281997f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.872+0000 7fa627e00640 1 -- 172.21.15.39:0/2415704874 >> 172.21.15.39:0/2415704874 conn(0x7fa6280fd810 msgr2=0x7fa628106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.872+0000 7fa627e00640 1 -- 172.21.15.39:0/2415704874 shutdown_connections 2024-07-04T08:16:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:46.872+0000 7fa627e00640 1 -- 172.21.15.39:0/2415704874 wait complete. 2024-07-04T08:16:47.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:46 smithi050 ceph-mon[36577]: pgmap v527: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-07-04T08:16:47.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:46 smithi133 ceph-mon[37050]: pgmap v527: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-07-04T08:16:47.732 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:16:48.186 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:48.526 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:16:49.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:48 smithi039 ceph-mon[32062]: pgmap v528: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-04T08:16:49.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:48 smithi050 ceph-mon[36577]: pgmap v528: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-04T08:16:49.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:48 smithi133 ceph-mon[37050]: pgmap v528: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 10 op/s 2024-07-04T08:16:49.656 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:51.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:50 smithi039 ceph-mon[32062]: pgmap v529: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-07-04T08:16:51.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:50 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:16:51.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:50 smithi050 ceph-mon[36577]: pgmap v529: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-07-04T08:16:51.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:50 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:16:51.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:50 smithi133 ceph-mon[37050]: pgmap v529: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-07-04T08:16:51.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:50 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:16:52.024 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:16:52.024 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:16:52.024 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:51 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:16:52.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.095+0000 7fa25de00640 1 -- 172.21.15.39:0/3978920972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa258103d10 msgr2=0x7fa258104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.095+0000 7fa25de00640 1 --2- 172.21.15.39:0/3978920972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa258103d10 0x7fa258104120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009860 tx=0x7fa24c02a340 comp rx=0 tx=0).stop 2024-07-04T08:16:52.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.095+0000 7fa25de00640 1 -- 172.21.15.39:0/3978920972 shutdown_connections 2024-07-04T08:16:52.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.095+0000 7fa25de00640 1 --2- 172.21.15.39:0/3978920972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa25810d170 0x7fa25810f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.095+0000 7fa25de00640 1 --2- 172.21.15.39:0/3978920972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa258104660 0x7fa25810cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.095+0000 7fa25de00640 1 --2- 172.21.15.39:0/3978920972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa258103d10 0x7fa258104120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.095+0000 7fa25de00640 1 -- 172.21.15.39:0/3978920972 >> 172.21.15.39:0/3978920972 conn(0x7fa2580fd810 msgr2=0x7fa2580ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.095+0000 7fa25de00640 1 -- 172.21.15.39:0/3978920972 shutdown_connections 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.095+0000 7fa25de00640 1 -- 172.21.15.39:0/3978920972 wait complete. 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.096+0000 7fa25de00640 1 Processor -- start 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.097+0000 7fa25de00640 1 -- start start 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.097+0000 7fa25de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa258103d10 0x7fa25819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.097+0000 7fa25de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa258104660 0x7fa25819bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.097+0000 7fa25de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa25810d170 0x7fa2581a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.097+0000 7fa25de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa258112df0 con 0x7fa258103d10 2024-07-04T08:16:52.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.097+0000 7fa25de00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa258112c70 con 0x7fa258104660 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.097+0000 7fa25de00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa258112f70 con 0x7fa25810d170 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.097+0000 7fa25ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa258103d10 0x7fa25819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa25ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa258103d10 0x7fa25819ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56042/0 (socket says 172.21.15.39:56042) 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa25ca00640 1 -- 172.21.15.39:0/3737859081 learned_addr learned my addr 172.21.15.39:0/3737859081 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa25d400640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa25810d170 0x7fa2581a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa257e00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa258104660 0x7fa25819bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa25ca00640 1 -- 172.21.15.39:0/3737859081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa25810d170 msgr2=0x7fa2581a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa25ca00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa25810d170 0x7fa2581a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa25ca00640 1 -- 172.21.15.39:0/3737859081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa258104660 msgr2=0x7fa25819bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa25ca00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa258104660 0x7fa25819bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa25ca00640 1 -- 172.21.15.39:0/3737859081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2581a19b0 con 0x7fa258103d10 2024-07-04T08:16:52.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa257e00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa258104660 0x7fa25819bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:52.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.098+0000 7fa25ca00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa258103d10 0x7fa25819ba40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0303f0 tx=0x7fa24c032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:52.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.099+0000 7fa255600640 1 -- 172.21.15.39:0/3737859081 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c040020 con 0x7fa258103d10 2024-07-04T08:16:52.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.099+0000 7fa25de00640 1 -- 172.21.15.39:0/3737859081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2581a1c40 con 0x7fa258103d10 2024-07-04T08:16:52.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.099+0000 7fa255600640 1 -- 172.21.15.39:0/3737859081 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa24c030dd0 con 0x7fa258103d10 2024-07-04T08:16:52.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.099+0000 7fa255600640 1 -- 172.21.15.39:0/3737859081 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c037040 con 0x7fa258103d10 2024-07-04T08:16:52.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.099+0000 7fa25de00640 1 -- 172.21.15.39:0/3737859081 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581a8400 con 0x7fa258103d10 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.102+0000 7fa265a67740 1 Processor -- start 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.102+0000 7fa265a67740 1 -- start start 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa265a67740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 0x5602af4d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa265a67740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 0x5602af4d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa265a67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4d43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa265a67740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5602af449f00 con 0x5602af12f070 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa265a67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5602af4394a0 con 0x5602af10e390 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa265a67740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5602af43d4e0 con 0x5602af1061b0 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa255600640 1 -- 172.21.15.39:0/3737859081 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa24c043020 con 0x7fa258103d10 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa257e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4d43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa257e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4d43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47950/0 (socket says 172.21.15.39:47950) 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.103+0000 7fa257e00640 1 -- 172.21.15.39:0/3988789013 learned_addr learned my addr 172.21.15.39:0/3988789013 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.104+0000 7fa25ca00640 1 --2- 172.21.15.39:0/3988789013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 0x5602af4d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.104+0000 7fa25d400640 1 --2- 172.21.15.39:0/3988789013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 0x5602af4d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.104+0000 7fa257e00640 1 -- 172.21.15.39:0/3988789013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 msgr2=0x5602af4d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.104+0000 7fa257e00640 1 --2- 172.21.15.39:0/3988789013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 0x5602af4d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.104+0000 7fa257e00640 1 -- 172.21.15.39:0/3988789013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 msgr2=0x5602af4d3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.104+0000 7fa257e00640 1 --2- 172.21.15.39:0/3988789013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 0x5602af4d3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.104+0000 7fa257e00640 1 -- 172.21.15.39:0/3988789013 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602af3d1670 con 0x5602af10e390 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.104+0000 7fa257e00640 1 --2- 172.21.15.39:0/3988789013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4d43f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa244011470 tx=0x7fa244011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1349e9e2660cd869 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.105+0000 7fa24a000640 1 -- 172.21.15.39:0/3988789013 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24401e070 con 0x5602af10e390 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.105+0000 7fa24a000640 1 -- 172.21.15.39:0/3988789013 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa244004310 con 0x5602af10e390 2024-07-04T08:16:52.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.105+0000 7fa24a000640 1 -- 172.21.15.39:0/3988789013 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24400f900 con 0x5602af10e390 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.105+0000 7fa255600640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2400b2960 0x7fa2400b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa265a67740 1 -- 172.21.15.39:0/3988789013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 msgr2=0x5602af4d43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa265a67740 1 --2- 172.21.15.39:0/3988789013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4d43f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa244011470 tx=0x7fa244011930 comp rx=0 tx=0).stop 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa255600640 1 -- 172.21.15.39:0/3737859081 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa24c102580 con 0x7fa258103d10 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa25d400640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2400b2960 0x7fa2400b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa265a67740 1 -- 172.21.15.39:0/3988789013 shutdown_connections 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa265a67740 1 --2- 172.21.15.39:0/3988789013 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4d43f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa265a67740 1 --2- 172.21.15.39:0/3988789013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 0x5602af4d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa265a67740 1 --2- 172.21.15.39:0/3988789013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 0x5602af4d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa265a67740 1 -- 172.21.15.39:0/3988789013 >> 172.21.15.39:0/3988789013 conn(0x5602af123460 msgr2=0x5602af4d1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa265a67740 1 -- 172.21.15.39:0/3988789013 shutdown_connections 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.106+0000 7fa265a67740 1 -- 172.21.15.39:0/3988789013 wait complete. 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.107+0000 7fa25d400640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2400b2960 0x7fa2400b4e70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa250005fd0 tx=0x7fa250010040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.107+0000 7fa265a67740 1 -- start start 2024-07-04T08:16:52.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.107+0000 7fa265a67740 1 Processor -- start 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa265a67740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af1061b0 0x5602af4df520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa265a67740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af10e390 0x5602af4dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa265a67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af12f070 0x5602af4eb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa265a67740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5602af29d700 con 0x5602af1061b0 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa265a67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5602af3e0bd0 con 0x5602af12f070 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa265a67740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5602af4e2150 con 0x5602af10e390 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa265a67740 10 client.0 _ll_register_callbacks cb 0x5602af432590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af1061b0 0x5602af4df520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af1061b0 0x5602af4df520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56070/0 (socket says 172.21.15.39:56070) 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25ca00640 1 -- 172.21.15.39:0/2247634398 learned_addr learned my addr 172.21.15.39:0/2247634398 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:52.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa257e00640 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af10e390 0x5602af4dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25d400640 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af12f070 0x5602af4eb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25ca00640 1 -- 172.21.15.39:0/2247634398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af10e390 msgr2=0x5602af4dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25ca00640 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af10e390 0x5602af4dfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25ca00640 1 -- 172.21.15.39:0/2247634398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af12f070 msgr2=0x5602af4eb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25ca00640 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af12f070 0x5602af4eb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25ca00640 1 -- 172.21.15.39:0/2247634398 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602af19a650 con 0x5602af1061b0 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa257e00640 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af10e390 0x5602af4dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.108+0000 7fa25d400640 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af12f070 0x5602af4eb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.109+0000 7fa25ca00640 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af1061b0 0x5602af4df520 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0ffab0 tx=0x7fa24c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522b0baf476d8a1d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.109+0000 7fa23ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.109+0000 7fa23ea00640 1 -- 172.21.15.39:0/2247634398 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0c8f30 con 0x5602af1061b0 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.109+0000 7fa23ea00640 1 -- 172.21.15.39:0/2247634398 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa24c102190 con 0x5602af1061b0 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.109+0000 7fa265a67740 1 -- 172.21.15.39:0/2247634398 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa244004310 con 0x5602af1061b0 2024-07-04T08:16:52.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.109+0000 7fa23ea00640 1 -- 172.21.15.39:0/2247634398 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa24c0c8d50 con 0x5602af1061b0 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.109+0000 7fa265a67740 10 client.15723 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.109+0000 7fa265a67740 1 -- 172.21.15.39:0/2247634398 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5602af4ee140 con 0x5602af1061b0 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.110+0000 7fa23ea00640 1 -- 172.21.15.39:0/2247634398 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa244004310 con 0x5602af1061b0 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.110+0000 7fa23ea00640 1 -- 172.21.15.39:0/2247634398 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa24c04d060 con 0x5602af1061b0 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.110+0000 7fa265a67740 10 client.15723 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.110+0000 7fa265a67740 10 client.15723 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.110+0000 7fa265a67740 20 client.15723 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.110+0000 7fa265a67740 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5602af0ed9f0 0x5602af4f1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.110+0000 7fa265a67740 4 client.15723 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.110+0000 7fa265a67740 1 -- 172.21.15.39:0/2247634398 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602af44be50 con 0x5602af0ed9f0 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.110+0000 7fa257e00640 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5602af0ed9f0 0x5602af4f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.111+0000 7fa257e00640 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5602af0ed9f0 0x5602af4f1740 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a2a3e288e04e08f9 server_cookie=dff1eb2887476b0e in_seq=0 out_seq=0 2024-07-04T08:16:52.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.111+0000 7fa23ea00640 10 client.15723 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa23ea00640 1 -- 172.21.15.39:0/2247634398 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5602af44be50 con 0x5602af0ed9f0 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa23ea00640 10 client.15723 handle_command_reply: tid=0 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 client.15723 shutdown 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 -- 172.21.15.39:0/2247634398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af1061b0 msgr2=0x5602af4df520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af1061b0 0x5602af4df520 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0ffab0 tx=0x7fa24c004610 comp rx=0 tx=0).stop 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 -- 172.21.15.39:0/2247634398 shutdown_connections 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5602af0ed9f0 0x5602af4f1740 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af12f070 0x5602af4eb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af10e390 0x5602af4dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 --2- 172.21.15.39:0/2247634398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af1061b0 0x5602af4df520 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 -- 172.21.15.39:0/2247634398 >> 172.21.15.39:0/2247634398 conn(0x5602af123460 msgr2=0x5602af4d6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa23ea00640 0 client.15723 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.112+0000 7fa265a67740 1 -- 172.21.15.39:0/2247634398 shutdown_connections 2024-07-04T08:16:52.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.113+0000 7fa265a67740 1 -- 172.21.15.39:0/2247634398 wait complete. 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.113+0000 7fa265a67740 20 client.15723 trim_cache size 0 max 16384 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.127+0000 7fa265a67740 1 Processor -- start 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.127+0000 7fa265a67740 1 -- start start 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa265a67740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 0x5602af4df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa265a67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4eb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa265a67740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 0x5602af4e0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa265a67740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b2260 con 0x5602af12f070 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa265a67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b3da0 con 0x5602af10e390 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa265a67740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b2930 con 0x5602af1061b0 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa25d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 0x5602af4e0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa257e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa257e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4eb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47968/0 (socket says 172.21.15.39:47968) 2024-07-04T08:16:52.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa25d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 0x5602af4e0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56078/0 (socket says 172.21.15.39:56078) 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa257e00640 1 -- 172.21.15.39:0/2551623521 learned_addr learned my addr 172.21.15.39:0/2551623521 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.128+0000 7fa25ca00640 1 --2- 172.21.15.39:0/2551623521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 0x5602af4df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.129+0000 7fa257e00640 1 -- 172.21.15.39:0/2551623521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 msgr2=0x5602af4df960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.129+0000 7fa257e00640 1 --2- 172.21.15.39:0/2551623521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 0x5602af4df960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.129+0000 7fa257e00640 1 -- 172.21.15.39:0/2551623521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 msgr2=0x5602af4e0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.129+0000 7fa257e00640 1 --2- 172.21.15.39:0/2551623521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 0x5602af4e0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.129+0000 7fa257e00640 1 -- 172.21.15.39:0/2551623521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602af4de280 con 0x5602af10e390 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.129+0000 7fa257e00640 1 --2- 172.21.15.39:0/2551623521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4eb260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa244009710 tx=0x7fa244005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa959fe7e03fb50 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.129+0000 7fa249600640 1 -- 172.21.15.39:0/2551623521 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24401e040 con 0x5602af10e390 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.129+0000 7fa249600640 1 -- 172.21.15.39:0/2551623521 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa244005480 con 0x5602af10e390 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.129+0000 7fa249600640 1 -- 172.21.15.39:0/2551623521 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24400f960 con 0x5602af10e390 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.130+0000 7fa265a67740 1 -- 172.21.15.39:0/2551623521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 msgr2=0x5602af4eb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.130+0000 7fa265a67740 1 --2- 172.21.15.39:0/2551623521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4eb260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa244009710 tx=0x7fa244005850 comp rx=0 tx=0).stop 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.130+0000 7fa265a67740 1 -- 172.21.15.39:0/2551623521 shutdown_connections 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.130+0000 7fa265a67740 1 --2- 172.21.15.39:0/2551623521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af12f070 0x5602af4e0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.130+0000 7fa265a67740 1 --2- 172.21.15.39:0/2551623521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af10e390 0x5602af4eb260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.130+0000 7fa265a67740 1 --2- 172.21.15.39:0/2551623521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af1061b0 0x5602af4df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.130+0000 7fa265a67740 1 -- 172.21.15.39:0/2551623521 >> 172.21.15.39:0/2551623521 conn(0x5602af123460 msgr2=0x5602af4e0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.130+0000 7fa265a67740 1 -- 172.21.15.39:0/2551623521 shutdown_connections 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.130+0000 7fa265a67740 1 -- 172.21.15.39:0/2551623521 wait complete. 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.131+0000 7fa265a67740 1 -- start start 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.131+0000 7fa265a67740 1 Processor -- start 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.132+0000 7fa265a67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af1061b0 0x5602af4f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.132+0000 7fa265a67740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af10e390 0x5602af4f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.132+0000 7fa265a67740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af12f070 0x5602af4e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.132+0000 7fa265a67740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b36d0 con 0x5602af10e390 2024-07-04T08:16:52.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.132+0000 7fa265a67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b3000 con 0x5602af1061b0 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.132+0000 7fa265a67740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b0240 con 0x5602af12f070 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.132+0000 7fa265a67740 10 client.0 _ll_register_callbacks cb 0x7fa24c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.132+0000 7fa257e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af10e390 0x5602af4f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa257e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af10e390 0x5602af4f6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56082/0 (socket says 172.21.15.39:56082) 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa257e00640 1 -- 172.21.15.39:0/3156181400 learned_addr learned my addr 172.21.15.39:0/3156181400 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa25ca00640 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af1061b0 0x5602af4f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa25d400640 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af12f070 0x5602af4e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa25ca00640 1 -- 172.21.15.39:0/3156181400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af12f070 msgr2=0x5602af4e63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa25ca00640 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af12f070 0x5602af4e63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa25ca00640 1 -- 172.21.15.39:0/3156181400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af10e390 msgr2=0x5602af4f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa25ca00640 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af10e390 0x5602af4f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa25ca00640 1 -- 172.21.15.39:0/3156181400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602af4e6910 con 0x5602af1061b0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa25ca00640 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af1061b0 0x5602af4f6500 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0c5660 tx=0x7fa24c0ffea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab72ddc756c7ae3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.133+0000 7fa23ea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.134+0000 7fa23ea00640 1 -- 172.21.15.39:0/3156181400 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c0cd9e0 con 0x5602af1061b0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.134+0000 7fa265a67740 1 -- 172.21.15.39:0/3156181400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa244005480 con 0x5602af1061b0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.134+0000 7fa23ea00640 1 -- 172.21.15.39:0/3156181400 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa24c0d21c0 con 0x5602af1061b0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.134+0000 7fa23ea00640 1 -- 172.21.15.39:0/3156181400 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa24c0c90f0 con 0x5602af1061b0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.134+0000 7fa265a67740 10 client.25838 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.134+0000 7fa265a67740 1 -- 172.21.15.39:0/3156181400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5602af4e6910 con 0x5602af1061b0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.134+0000 7fa23ea00640 1 -- 172.21.15.39:0/3156181400 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa244005480 con 0x5602af1061b0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.135+0000 7fa23ea00640 1 -- 172.21.15.39:0/3156181400 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa24c0c4f50 con 0x5602af1061b0 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.135+0000 7fa265a67740 10 client.25838 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:52.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.135+0000 7fa265a67740 10 client.25838 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.135+0000 7fa265a67740 20 client.25838 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.135+0000 7fa265a67740 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5602af0c2da0 0x5602af55bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.135+0000 7fa265a67740 4 client.25838 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.135+0000 7fa265a67740 1 -- 172.21.15.39:0/3156181400 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602af4478f0 con 0x5602af0c2da0 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.136+0000 7fa257e00640 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5602af0c2da0 0x5602af55bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.136+0000 7fa257e00640 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5602af0c2da0 0x5602af55bee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17490dc29d28db42 server_cookie=db14cff1fa70632a in_seq=0 out_seq=0 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.136+0000 7fa23ea00640 10 client.25838 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa23ea00640 1 -- 172.21.15.39:0/3156181400 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602af4478f0 con 0x5602af0c2da0 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa23ea00640 10 client.25838 handle_command_reply: tid=0 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 client.25838 shutdown 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 -- 172.21.15.39:0/3156181400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af1061b0 msgr2=0x5602af4f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af1061b0 0x5602af4f6500 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0c5660 tx=0x7fa24c0ffea0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 -- 172.21.15.39:0/3156181400 shutdown_connections 2024-07-04T08:16:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5602af0c2da0 0x5602af55bee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602af10e390 0x5602af4f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602af12f070 0x5602af4e63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 --2- 172.21.15.39:0/3156181400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602af1061b0 0x5602af4f6500 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 -- 172.21.15.39:0/3156181400 >> 172.21.15.39:0/3156181400 conn(0x5602af123460 msgr2=0x5602af4d6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa23ea00640 0 client.25838 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:16:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 -- 172.21.15.39:0/3156181400 shutdown_connections 2024-07-04T08:16:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 1 -- 172.21.15.39:0/3156181400 wait complete. 2024-07-04T08:16:52.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.137+0000 7fa265a67740 20 client.25838 trim_cache size 0 max 16384 2024-07-04T08:16:52.153 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:16:52.153 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:16:52.153 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:51 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:16:52.153 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 -- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2400b2960 msgr2=0x7fa2400b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2400b2960 0x7fa2400b4e70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa250005fd0 tx=0x7fa250010040 comp rx=0 tx=0).stop 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 -- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa258103d10 msgr2=0x7fa25819ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa258103d10 0x7fa25819ba40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0303f0 tx=0x7fa24c032da0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 -- 172.21.15.39:0/3737859081 shutdown_connections 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa2400b2960 0x7fa2400b4e70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa25810d170 0x7fa2581a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa258104660 0x7fa25819bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 --2- 172.21.15.39:0/3737859081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa258103d10 0x7fa25819ba40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.138+0000 7fa25de00640 1 -- 172.21.15.39:0/3737859081 >> 172.21.15.39:0/3737859081 conn(0x7fa2580fd810 msgr2=0x7fa2581019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:52.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.139+0000 7fa25de00640 1 -- 172.21.15.39:0/3737859081 shutdown_connections 2024-07-04T08:16:52.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.139+0000 7fa25de00640 1 -- 172.21.15.39:0/3737859081 wait complete. 2024-07-04T08:16:52.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:16:52.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:16:52.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:51 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:16:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.453+0000 7fe2e1200640 1 -- 172.21.15.39:0/2797753644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2dc074ca0 msgr2=0x7fe2dc0750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.453+0000 7fe2e1200640 1 --2- 172.21.15.39:0/2797753644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2dc074ca0 0x7fe2dc0750b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009900 tx=0x7fe2cc02a410 comp rx=0 tx=0).stop 2024-07-04T08:16:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.453+0000 7fe2e1200640 1 -- 172.21.15.39:0/2797753644 shutdown_connections 2024-07-04T08:16:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.453+0000 7fe2e1200640 1 --2- 172.21.15.39:0/2797753644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2dc10f670 0x7fe2dc111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.455 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.453+0000 7fe2e1200640 1 --2- 172.21.15.39:0/2797753644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2dc0755f0 0x7fe2dc10f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.453+0000 7fe2e1200640 1 --2- 172.21.15.39:0/2797753644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2dc074ca0 0x7fe2dc0750b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.453+0000 7fe2e1200640 1 -- 172.21.15.39:0/2797753644 >> 172.21.15.39:0/2797753644 conn(0x7fe2dc07ada0 msgr2=0x7fe2dc07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.454+0000 7fe2e1200640 1 -- 172.21.15.39:0/2797753644 shutdown_connections 2024-07-04T08:16:52.456 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.454+0000 7fe2e1200640 1 -- 172.21.15.39:0/2797753644 wait complete. 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.455+0000 7fe2e1200640 1 Processor -- start 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.455+0000 7fe2e1200640 1 -- start start 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.455+0000 7fe2e1200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2dc074ca0 0x7fe2dc19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.455+0000 7fe2e1200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2dc0755f0 0x7fe2dc19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.455+0000 7fe2e1200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2dc10f670 0x7fe2dc1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.455+0000 7fe2e1200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1151d0 con 0x7fe2dc10f670 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.455+0000 7fe2e1200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc115050 con 0x7fe2dc0755f0 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.455+0000 7fe2e1200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc115350 con 0x7fe2dc074ca0 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2d9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2dc074ca0 0x7fe2dc19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.457 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2d9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2dc074ca0 0x7fe2dc19bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43160/0 (socket says 172.21.15.39:43160) 2024-07-04T08:16:52.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2d9e00640 1 -- 172.21.15.39:0/612655074 learned_addr learned my addr 172.21.15.39:0/612655074 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:52.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2da800640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2dc10f670 0x7fe2dc1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2d9400640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2dc0755f0 0x7fe2dc19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2d9e00640 1 -- 172.21.15.39:0/612655074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2dc0755f0 msgr2=0x7fe2dc19c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2d9e00640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2dc0755f0 0x7fe2dc19c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2d9e00640 1 -- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2dc10f670 msgr2=0x7fe2dc1a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2d9e00640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2dc10f670 0x7fe2dc1a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.456+0000 7fe2d9e00640 1 -- 172.21.15.39:0/612655074 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc1a1b90 con 0x7fe2dc074ca0 2024-07-04T08:16:52.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.457+0000 7fe2d9e00640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2dc074ca0 0x7fe2dc19bc20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc005940 tx=0x7fe2cc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:52.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.457+0000 7fe2d2a00640 1 -- 172.21.15.39:0/612655074 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc03f030 con 0x7fe2dc074ca0 2024-07-04T08:16:52.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.457+0000 7fe2d2a00640 1 -- 172.21.15.39:0/612655074 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2cc003aa0 con 0x7fe2dc074ca0 2024-07-04T08:16:52.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.457+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc1a1e20 con 0x7fe2dc074ca0 2024-07-04T08:16:52.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.457+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc1a85e0 con 0x7fe2dc074ca0 2024-07-04T08:16:52.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.457+0000 7fe2d2a00640 1 -- 172.21.15.39:0/612655074 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc035070 con 0x7fe2dc074ca0 2024-07-04T08:16:52.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.459+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2dc0750b0 con 0x7fe2dc074ca0 2024-07-04T08:16:52.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.461+0000 7fe2d2a00640 1 -- 172.21.15.39:0/612655074 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe2cc042030 con 0x7fe2dc074ca0 2024-07-04T08:16:52.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.463+0000 7fe2d2a00640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe2c00b29e0 0x7fe2c00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:52.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.463+0000 7fe2d2a00640 1 -- 172.21.15.39:0/612655074 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fe2cc103270 con 0x7fe2dc074ca0 2024-07-04T08:16:52.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.463+0000 7fe2d9400640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe2c00b29e0 0x7fe2c00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:52.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.464+0000 7fe2d9400640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe2c00b29e0 0x7fe2c00b4ef0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc1113b0 tx=0x7fe2c800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:52.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.465+0000 7fe2d2a00640 1 -- 172.21.15.39:0/612655074 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe2cc0ca270 con 0x7fe2dc074ca0 2024-07-04T08:16:52.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.741+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2dc1a8810 con 0x7fe2dc074ca0 2024-07-04T08:16:52.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.742+0000 7fe2d2a00640 1 -- 172.21.15.39:0/612655074 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fe2cc0ce570 con 0x7fe2dc074ca0 2024-07-04T08:16:52.744 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:16:52.744 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:16:52.745 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:16:52.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe2c00b29e0 msgr2=0x7fe2c00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe2c00b29e0 0x7fe2c00b4ef0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc1113b0 tx=0x7fe2c800b410 comp rx=0 tx=0).stop 2024-07-04T08:16:52.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2dc074ca0 msgr2=0x7fe2dc19bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:52.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2dc074ca0 0x7fe2dc19bc20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc005940 tx=0x7fe2cc004270 comp rx=0 tx=0).stop 2024-07-04T08:16:52.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 shutdown_connections 2024-07-04T08:16:52.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe2c00b29e0 0x7fe2c00b4ef0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2dc10f670 0x7fe2dc1a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe2dc0755f0 0x7fe2dc19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 --2- 172.21.15.39:0/612655074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2dc074ca0 0x7fe2dc19bc20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:52.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.745+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 >> 172.21.15.39:0/612655074 conn(0x7fe2dc07ada0 msgr2=0x7fe2dc109210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:52.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.746+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 shutdown_connections 2024-07-04T08:16:52.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:52.746+0000 7fe2e1200640 1 -- 172.21.15.39:0/612655074 wait complete. 2024-07-04T08:16:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:52 smithi039 ceph-mon[32062]: pgmap v530: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:53.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:52 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/612655074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:53.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:52 smithi050 ceph-mon[36577]: pgmap v530: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:53.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:52 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/612655074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:53.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:52 smithi133 ceph-mon[37050]: pgmap v530: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:53.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:52 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/612655074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:16:53.743 INFO:tasks.workunit.client.0.smithi039.stderr:Updating files: 52% (6503/12271) Updating files: 53% (6504/12271) Updating files: 54% (6627/12271) Updating files: 55% (6750/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 87% (10787/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-07-04T08:16:53.781 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id 1ec7ed84 --release --await' 2024-07-04T08:16:53.781 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id 1ec7ed84 --release --await 2024-07-04T08:16:54.061 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:16:54.267 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:54.987 INFO:tasks.workunit.client.0.smithi039.stderr:Note: switching to '2e9f68b5506c607b2f7d69d380cb7887cf00dc84'. 2024-07-04T08:16:54.987 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-07-04T08:16:54.987 INFO:tasks.workunit.client.0.smithi039.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-04T08:16:54.987 INFO:tasks.workunit.client.0.smithi039.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-04T08:16:54.987 INFO:tasks.workunit.client.0.smithi039.stderr:state without impacting any branches by switching back to a branch. 2024-07-04T08:16:54.987 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-07-04T08:16:54.987 INFO:tasks.workunit.client.0.smithi039.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-04T08:16:54.987 INFO:tasks.workunit.client.0.smithi039.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-04T08:16:54.987 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-07-04T08:16:54.988 INFO:tasks.workunit.client.0.smithi039.stderr: git switch -c 2024-07-04T08:16:54.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-07-04T08:16:54.988 INFO:tasks.workunit.client.0.smithi039.stderr:Or undo this operation with: 2024-07-04T08:16:54.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-07-04T08:16:54.988 INFO:tasks.workunit.client.0.smithi039.stderr: git switch - 2024-07-04T08:16:54.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-07-04T08:16:54.988 INFO:tasks.workunit.client.0.smithi039.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-04T08:16:54.988 INFO:tasks.workunit.client.0.smithi039.stderr: 2024-07-04T08:16:54.988 INFO:tasks.workunit.client.0.smithi039.stderr:HEAD is now at 2e9f68b5506 Merge PR #52939 into wip-pdonnell-testing-20240703.143006-debug 2024-07-04T08:16:55.003 DEBUG:teuthology.orchestra.run.smithi039:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-07-04T08:16:55.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:54 smithi039 ceph-mon[32062]: pgmap v531: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:55.154 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:54 smithi050 ceph-mon[36577]: pgmap v531: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:55.155 INFO:tasks.workunit.client.0.smithi039.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-04T08:16:55.155 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-04T08:16:55.155 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-04T08:16:55.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:54 smithi133 ceph-mon[37050]: pgmap v531: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:56.010 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:16:57.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:56 smithi050 ceph-mon[36577]: pgmap v532: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:16:57.250 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-04T08:16:57.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:56 smithi133 ceph-mon[37050]: pgmap v532: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:16:57.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:56 smithi039 ceph-mon[32062]: pgmap v532: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:16:57.375 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-04T08:16:57.463 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-04T08:16:57.468 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-04T08:16:57.469 INFO:tasks.workunit.client.0.smithi039.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-04T08:16:57.548 INFO:tasks.workunit.client.0.smithi039.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-04T08:16:57.557 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:16:57.557 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-07-04T08:16:57.602 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-07-04T08:16:57.603 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-07-04T08:16:57.603 DEBUG:teuthology.orchestra.run.smithi039:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2e9f68b5506c607b2f7d69d380cb7887cf00dc84 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-07-04T08:16:57.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.623+0000 7fabc0e00640 1 -- 172.21.15.39:0/2157067567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabbc108820 msgr2=0x7fabbc108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:57.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.623+0000 7fabc0e00640 1 --2- 172.21.15.39:0/2157067567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabbc108820 0x7fabbc108c30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faba8009900 tx=0x7faba802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.623+0000 7fabc0e00640 1 -- 172.21.15.39:0/2157067567 shutdown_connections 2024-07-04T08:16:57.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.623+0000 7fabc0e00640 1 --2- 172.21.15.39:0/2157067567 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc109170 0x7fabbc10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.623+0000 7fabc0e00640 1 --2- 172.21.15.39:0/2157067567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabbc108820 0x7fabbc108c30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.623+0000 7fabc0e00640 1 --2- 172.21.15.39:0/2157067567 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabbc102ee0 0x7fabbc1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.623+0000 7fabc0e00640 1 -- 172.21.15.39:0/2157067567 >> 172.21.15.39:0/2157067567 conn(0x7fabbc0fe930 msgr2=0x7fabbc100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:57.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.623+0000 7fabc0e00640 1 -- 172.21.15.39:0/2157067567 shutdown_connections 2024-07-04T08:16:57.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.623+0000 7fabc0e00640 1 -- 172.21.15.39:0/2157067567 wait complete. 2024-07-04T08:16:57.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.625+0000 7fabc0e00640 1 Processor -- start 2024-07-04T08:16:57.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.625+0000 7fabc0e00640 1 -- start start 2024-07-04T08:16:57.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.625+0000 7fabc0e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc102ee0 0x7fabbc074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.625+0000 7fabc0e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabbc108820 0x7fabbc0751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.625+0000 7fabc0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabbc109170 0x7fabbc0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.625+0000 7fabc0e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc112e80 con 0x7fabbc102ee0 2024-07-04T08:16:57.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.625+0000 7fabc0e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc112d00 con 0x7fabbc109170 2024-07-04T08:16:57.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.625+0000 7fabc0e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc113000 con 0x7fabbc108820 2024-07-04T08:16:57.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc102ee0 0x7fabbc074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabbc108820 0x7fabbc0751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc102ee0 0x7fabbc074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56106/0 (socket says 172.21.15.39:56106) 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbb400640 1 -- 172.21.15.39:0/546119842 learned_addr learned my addr 172.21.15.39:0/546119842 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbbe00640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabbc109170 0x7fabbc0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbb400640 1 -- 172.21.15.39:0/546119842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabbc108820 msgr2=0x7fabbc0751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbb400640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabbc108820 0x7fabbc0751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbb400640 1 -- 172.21.15.39:0/546119842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabbc109170 msgr2=0x7fabbc0756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbb400640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabbc109170 0x7fabbc0756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.626+0000 7fabbb400640 1 -- 172.21.15.39:0/546119842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc1a7b10 con 0x7fabbc102ee0 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.627+0000 7fabbb400640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc102ee0 0x7fabbc074c60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fabac00e9f0 tx=0x7fabac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.627+0000 7fabb3e00640 1 -- 172.21.15.39:0/546119842 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac00cd80 con 0x7fabbc102ee0 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.627+0000 7fabb3e00640 1 -- 172.21.15.39:0/546119842 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabac0108e0 con 0x7fabbc102ee0 2024-07-04T08:16:57.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.627+0000 7fabb3e00640 1 -- 172.21.15.39:0/546119842 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac018980 con 0x7fabbc102ee0 2024-07-04T08:16:57.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.627+0000 7fabc0e00640 1 -- 172.21.15.39:0/546119842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc1a7e00 con 0x7fabbc102ee0 2024-07-04T08:16:57.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.627+0000 7fabc0e00640 1 -- 172.21.15.39:0/546119842 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc1a8260 con 0x7fabbc102ee0 2024-07-04T08:16:57.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.631+0000 7fabc8982740 1 Processor -- start 2024-07-04T08:16:57.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.631+0000 7fabb3e00640 1 -- 172.21.15.39:0/546119842 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fabac010430 con 0x7fabbc102ee0 2024-07-04T08:16:57.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.631+0000 7fabc8982740 1 -- start start 2024-07-04T08:16:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabc8982740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b3f4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabc8982740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabc8982740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b3f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabc8982740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b81b30eb20 con 0x55b81b04f520 2024-07-04T08:16:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabc8982740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b81b36a2b0 con 0x55b81b026660 2024-07-04T08:16:57.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabc8982740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b81b359850 con 0x55b81b02e840 2024-07-04T08:16:57.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b3f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b3f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56108/0 (socket says 172.21.15.39:56108) 2024-07-04T08:16:57.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabbbe00640 1 -- 172.21.15.39:0/3051166230 learned_addr learned my addr 172.21.15.39:0/3051166230 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:57.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.632+0000 7fabbb400640 1 --2- 172.21.15.39:0/3051166230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.633+0000 7fabbaa00640 1 --2- 172.21.15.39:0/3051166230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b3f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.633+0000 7fabbbe00640 1 -- 172.21.15.39:0/3051166230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 msgr2=0x55b81b3f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:57.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.633+0000 7fabbbe00640 1 --2- 172.21.15.39:0/3051166230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b3f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.633+0000 7fabbbe00640 1 -- 172.21.15.39:0/3051166230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 msgr2=0x55b81b3f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:57.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.633+0000 7fabbbe00640 1 --2- 172.21.15.39:0/3051166230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.633+0000 7fabbbe00640 1 -- 172.21.15.39:0/3051166230 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81b313780 con 0x55b81b04f520 2024-07-04T08:16:57.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.633+0000 7fabbbe00640 1 --2- 172.21.15.39:0/3051166230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b3f4490 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009fd0 tx=0x7fabb40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc450eaf02b087c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:57.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.634+0000 7fabb3e00640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faba00b29e0 0x7faba00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.634+0000 7fabb0c00640 1 -- 172.21.15.39:0/3051166230 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4004d20 con 0x55b81b04f520 2024-07-04T08:16:57.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.634+0000 7fabb3e00640 1 -- 172.21.15.39:0/546119842 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fabac0e2c70 con 0x7fabbc102ee0 2024-07-04T08:16:57.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.634+0000 7fabb0c00640 1 -- 172.21.15.39:0/3051166230 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb4004050 con 0x55b81b04f520 2024-07-04T08:16:57.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.634+0000 7fabb0c00640 1 -- 172.21.15.39:0/3051166230 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4005260 con 0x55b81b04f520 2024-07-04T08:16:57.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.634+0000 7fabbb400640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faba00b29e0 0x7faba00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.635+0000 7fabc8982740 1 -- 172.21.15.39:0/3051166230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 msgr2=0x55b81b3f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.635+0000 7fabc8982740 1 --2- 172.21.15.39:0/3051166230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b3f4490 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009fd0 tx=0x7fabb40118d0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.635+0000 7fabbb400640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faba00b29e0 0x7faba00b4ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55b81b3f3110 tx=0x7fabac0e6720 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.635+0000 7fabc8982740 1 -- 172.21.15.39:0/3051166230 shutdown_connections 2024-07-04T08:16:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.635+0000 7fabc8982740 1 --2- 172.21.15.39:0/3051166230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b3f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.635+0000 7fabc8982740 1 --2- 172.21.15.39:0/3051166230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.635+0000 7fabc8982740 1 --2- 172.21.15.39:0/3051166230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b3f4490 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.635+0000 7fabc8982740 1 -- 172.21.15.39:0/3051166230 >> 172.21.15.39:0/3051166230 conn(0x55b81b043910 msgr2=0x55b81b3f2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.635+0000 7fabc8982740 1 -- 172.21.15.39:0/3051166230 shutdown_connections 2024-07-04T08:16:57.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.636+0000 7fabc8982740 1 -- 172.21.15.39:0/3051166230 wait complete. 2024-07-04T08:16:57.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.636+0000 7fabc8982740 1 -- start start 2024-07-04T08:16:57.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.637+0000 7fabc8982740 1 Processor -- start 2024-07-04T08:16:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.637+0000 7fabc8982740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.637+0000 7fabc8982740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b4000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.637+0000 7fabc8982740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b40be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.637+0000 7fabc8982740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b81b300d20 con 0x55b81b04f520 2024-07-04T08:16:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.637+0000 7fabc8982740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b81b35d890 con 0x55b81b026660 2024-07-04T08:16:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.637+0000 7fabc8982740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b81b1bdbb0 con 0x55b81b02e840 2024-07-04T08:16:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.637+0000 7fabc8982740 10 client.0 _ll_register_callbacks cb 0x55b81b3019e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:57.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3ffb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48048/0 (socket says 172.21.15.39:48048) 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbbe00640 1 -- 172.21.15.39:0/2148595194 learned_addr learned my addr 172.21.15.39:0/2148595194 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbbe00640 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b40be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbaa00640 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b4000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbbe00640 1 -- 172.21.15.39:0/2148595194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 msgr2=0x55b81b4000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbbe00640 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b4000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbbe00640 1 -- 172.21.15.39:0/2148595194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 msgr2=0x55b81b3ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbbe00640 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.638+0000 7fabbbe00640 1 -- 172.21.15.39:0/2148595194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81b2f0660 con 0x55b81b04f520 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.639+0000 7fabbbe00640 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b40be10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fabb4011dd0 tx=0x7fabb4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b028088267efff1b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:57.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.639+0000 7faba5600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.639+0000 7faba5600640 1 -- 172.21.15.39:0/2148595194 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4004a40 con 0x55b81b04f520 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.639+0000 7faba5600640 1 -- 172.21.15.39:0/2148595194 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fabb4004be0 con 0x55b81b04f520 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.639+0000 7fabc8982740 1 -- 172.21.15.39:0/2148595194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb4004050 con 0x55b81b04f520 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.640+0000 7faba5600640 1 -- 172.21.15.39:0/2148595194 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fabb4033050 con 0x55b81b04f520 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.640+0000 7fabc8982740 10 client.15747 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.640+0000 7fabc8982740 1 -- 172.21.15.39:0/2148595194 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b81b0bab00 con 0x55b81b04f520 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.640+0000 7faba5600640 1 -- 172.21.15.39:0/2148595194 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb4004050 con 0x55b81b04f520 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.640+0000 7faba5600640 1 -- 172.21.15.39:0/2148595194 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fabb4028450 con 0x55b81b04f520 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.640+0000 7fabc8982740 10 client.15747 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.640+0000 7fabc8982740 10 client.15747 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:16:57.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.640+0000 7fabc8982740 20 client.15747 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:57.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.641+0000 7fabc8982740 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b81b00dea0 0x55b81b411a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.641+0000 7fabc8982740 4 client.15747 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:57.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.641+0000 7fabc8982740 1 -- 172.21.15.39:0/2148595194 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b81b36c200 con 0x55b81b00dea0 2024-07-04T08:16:57.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.641+0000 7fabbaa00640 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b81b00dea0 0x55b81b411a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.642+0000 7fabbaa00640 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b81b00dea0 0x55b81b411a50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9174423809802cee server_cookie=c0bdcd69d6fdb330 in_seq=0 out_seq=0 2024-07-04T08:16:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.642+0000 7faba5600640 10 client.15747 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.643+0000 7faba5600640 1 -- 172.21.15.39:0/2148595194 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b81b36c200 con 0x55b81b00dea0 2024-07-04T08:16:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.643+0000 7faba5600640 10 client.15747 handle_command_reply: tid=0 2024-07-04T08:16:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.643+0000 7fabc8982740 1 client.15747 shutdown 2024-07-04T08:16:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.644+0000 7fabc8982740 1 -- 172.21.15.39:0/2148595194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 msgr2=0x55b81b40be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:57.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.644+0000 7fabc8982740 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b40be10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fabb4011dd0 tx=0x7fabb4005fd0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7fabc8982740 1 -- 172.21.15.39:0/2148595194 shutdown_connections 2024-07-04T08:16:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7fabc8982740 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b81b00dea0 0x55b81b411a50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7fabc8982740 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b40be10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7fabc8982740 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b4000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7fabc8982740 1 --2- 172.21.15.39:0/2148595194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7faba5600640 0 client.15747 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7fabc8982740 1 -- 172.21.15.39:0/2148595194 >> 172.21.15.39:0/2148595194 conn(0x55b81b043910 msgr2=0x55b81b3f7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7fabc8982740 1 -- 172.21.15.39:0/2148595194 shutdown_connections 2024-07-04T08:16:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7fabc8982740 1 -- 172.21.15.39:0/2148595194 wait complete. 2024-07-04T08:16:57.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.646+0000 7fabc8982740 20 client.15747 trim_cache size 0 max 16384 2024-07-04T08:16:57.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.667+0000 7fabc8982740 1 Processor -- start 2024-07-04T08:16:57.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.667+0000 7fabc8982740 1 -- start start 2024-07-04T08:16:57.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.667+0000 7fabc8982740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3fb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.667+0000 7fabc8982740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b401a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.667+0000 7fabc8982740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b4170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.667+0000 7fabc8982740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1b22e0 con 0x55b81b04f520 2024-07-04T08:16:57.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.667+0000 7fabc8982740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1b3e20 con 0x55b81b026660 2024-07-04T08:16:57.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.667+0000 7fabc8982740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1b29b0 con 0x55b81b02e840 2024-07-04T08:16:57.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.667+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3fb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b4170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b401a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b4170b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56128/0 (socket says 172.21.15.39:56128) 2024-07-04T08:16:57.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbbe00640 1 -- 172.21.15.39:0/1159281831 learned_addr learned my addr 172.21.15.39:0/1159281831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:57.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbaa00640 1 -- 172.21.15.39:0/1159281831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 msgr2=0x55b81b3fb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:57.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbaa00640 1 --2- 172.21.15.39:0/1159281831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3fb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbaa00640 1 -- 172.21.15.39:0/1159281831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 msgr2=0x55b81b4170b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbaa00640 1 --2- 172.21.15.39:0/1159281831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b4170b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbaa00640 1 -- 172.21.15.39:0/1159281831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81b131960 con 0x55b81b02e840 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbbe00640 1 --2- 172.21.15.39:0/1159281831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b4170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabbaa00640 1 --2- 172.21.15.39:0/1159281831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b401a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faba8009230 tx=0x7faba8005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1278a20911c9c633 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7faba7e00640 1 -- 172.21.15.39:0/1159281831 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8035040 con 0x55b81b02e840 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7faba7e00640 1 -- 172.21.15.39:0/1159281831 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba804b070 con 0x55b81b02e840 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7faba7e00640 1 -- 172.21.15.39:0/1159281831 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80035f0 con 0x55b81b02e840 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabc8982740 1 -- 172.21.15.39:0/1159281831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 msgr2=0x55b81b401a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabc8982740 1 --2- 172.21.15.39:0/1159281831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b401a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faba8009230 tx=0x7faba8005a20 comp rx=0 tx=0).stop 2024-07-04T08:16:57.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabc8982740 1 -- 172.21.15.39:0/1159281831 shutdown_connections 2024-07-04T08:16:57.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabc8982740 1 --2- 172.21.15.39:0/1159281831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b04f520 0x55b81b4170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabc8982740 1 --2- 172.21.15.39:0/1159281831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b02e840 0x55b81b401a70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabc8982740 1 --2- 172.21.15.39:0/1159281831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b026660 0x55b81b3fb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabc8982740 1 -- 172.21.15.39:0/1159281831 >> 172.21.15.39:0/1159281831 conn(0x55b81b043910 msgr2=0x55b81b3f5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:57.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabc8982740 1 -- 172.21.15.39:0/1159281831 shutdown_connections 2024-07-04T08:16:57.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.672+0000 7fabc8982740 1 -- 172.21.15.39:0/1159281831 wait complete. 2024-07-04T08:16:57.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 -- start start 2024-07-04T08:16:57.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 Processor -- start 2024-07-04T08:16:57.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b026660 0x55b81b403d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b02e840 0x55b81b4042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b04f520 0x55b81b4047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1b3750 con 0x55b81b02e840 2024-07-04T08:16:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1b3080 con 0x55b81b04f520 2024-07-04T08:16:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1b02c0 con 0x55b81b026660 2024-07-04T08:16:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 10 client.0 _ll_register_callbacks cb 0x7fabb4003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b026660 0x55b81b403d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b026660 0x55b81b403d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43214/0 (socket says 172.21.15.39:43214) 2024-07-04T08:16:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b02e840 0x55b81b4042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 -- 172.21.15.39:0/4067272028 learned_addr learned my addr 172.21.15.39:0/4067272028 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b04f520 0x55b81b4047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 -- 172.21.15.39:0/4067272028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b04f520 msgr2=0x55b81b4047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b04f520 0x55b81b4047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 -- 172.21.15.39:0/4067272028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b02e840 msgr2=0x55b81b4042a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b02e840 0x55b81b4042a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 -- 172.21.15.39:0/4067272028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81b3268a0 con 0x55b81b026660 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbbe00640 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b026660 0x55b81b403d60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fabb4004260 tx=0x7fabb400fac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d973236e466d38fb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7faba5600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7faba5600640 1 -- 172.21.15.39:0/4067272028 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4028450 con 0x55b81b026660 2024-07-04T08:16:57.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 -- 172.21.15.39:0/4067272028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba804b070 con 0x55b81b026660 2024-07-04T08:16:57.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7faba5600640 1 -- 172.21.15.39:0/4067272028 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fabb4009a50 con 0x55b81b026660 2024-07-04T08:16:57.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7faba5600640 1 -- 172.21.15.39:0/4067272028 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fabb4009c50 con 0x55b81b026660 2024-07-04T08:16:57.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 10 client.25837 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:57.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 -- 172.21.15.39:0/4067272028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b81b322450 con 0x55b81b026660 2024-07-04T08:16:57.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7faba5600640 1 -- 172.21.15.39:0/4067272028 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba804b070 con 0x55b81b026660 2024-07-04T08:16:57.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7faba5600640 1 -- 172.21.15.39:0/4067272028 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fabb4028a40 con 0x55b81b026660 2024-07-04T08:16:57.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 10 client.25837 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:57.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 10 client.25837 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:16:57.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 20 client.25837 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:57.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b81afe3250 0x55b81b47c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:57.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 4 client.25837 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1ec7ed84", "release": true, "await": true}] 2024-07-04T08:16:57.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabc8982740 1 -- 172.21.15.39:0/4067272028 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1ec7ed84", "release": true, "await": true}) -- 0x55b81b3ff3f0 con 0x55b81afe3250 2024-07-04T08:16:57.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.675+0000 7fabbaa00640 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b81afe3250 0x55b81b47c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:57.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.680+0000 7fabbaa00640 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b81afe3250 0x55b81b47c7b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27f0f11bd73a78d server_cookie=c289ce59c070ca68 in_seq=0 out_seq=0 2024-07-04T08:16:57.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.680+0000 7faba5600640 10 client.25837 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:57.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7faba5600640 1 -- 172.21.15.39:0/4067272028 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b81b3ff3f0 con 0x55b81afe3250 2024-07-04T08:16:57.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7faba5600640 10 client.25837 handle_command_reply: tid=0 2024-07-04T08:16:57.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 client.25837 shutdown 2024-07-04T08:16:57.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 -- 172.21.15.39:0/4067272028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b026660 msgr2=0x55b81b403d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:57.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b026660 0x55b81b403d60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fabb4004260 tx=0x7fabb400fac0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 -- 172.21.15.39:0/4067272028 shutdown_connections 2024-07-04T08:16:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b81afe3250 0x55b81b47c7b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b81b02e840 0x55b81b4042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b81b04f520 0x55b81b4047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 --2- 172.21.15.39:0/4067272028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b81b026660 0x55b81b403d60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7faba5600640 0 client.25837 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:16:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 -- 172.21.15.39:0/4067272028 >> 172.21.15.39:0/4067272028 conn(0x55b81b043910 msgr2=0x55b81b3fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 -- 172.21.15.39:0/4067272028 shutdown_connections 2024-07-04T08:16:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.683+0000 7fabc8982740 1 -- 172.21.15.39:0/4067272028 wait complete. 2024-07-04T08:16:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.684+0000 7fabc8982740 20 client.25837 trim_cache size 0 max 16384 2024-07-04T08:16:57.688 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:16:57.688 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:16:57.688 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:16:57.688 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 4, 2024-07-04T08:16:57.688 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:16:57.688 INFO:teuthology.orchestra.run.smithi039.stdout: "1ec7ed84": { 2024-07-04T08:16:57.688 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 4, 2024-07-04T08:16:57.688 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 142.8, 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:16:57.689 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:16:57.690 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:16:57.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:57.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:57.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:57.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:57.690 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:16:57.690 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:16:57.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 -- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faba00b29e0 msgr2=0x7faba00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:57.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faba00b29e0 0x7faba00b4ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55b81b3f3110 tx=0x7fabac0e6720 comp rx=0 tx=0).stop 2024-07-04T08:16:57.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 -- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc102ee0 msgr2=0x7fabbc074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:57.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc102ee0 0x7fabbc074c60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fabac00e9f0 tx=0x7fabac00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 -- 172.21.15.39:0/546119842 shutdown_connections 2024-07-04T08:16:57.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faba00b29e0 0x7faba00b4ef0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fabbc109170 0x7fabbc0756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabbc108820 0x7fabbc0751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 --2- 172.21.15.39:0/546119842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc102ee0 0x7fabbc074c60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:57.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 -- 172.21.15.39:0/546119842 >> 172.21.15.39:0/546119842 conn(0x7fabbc0fe930 msgr2=0x7fabbc100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:57.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 -- 172.21.15.39:0/546119842 shutdown_connections 2024-07-04T08:16:57.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:57.685+0000 7fabc0e00640 1 -- 172.21.15.39:0/546119842 wait complete. 2024-07-04T08:16:57.700 INFO:tasks.workunit.client.0.smithi039.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-07-04T08:16:57.701 INFO:tasks.workunit.client.0.smithi039.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-07-04T08:16:57.701 INFO:tasks.workunit.client.0.smithi039.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-07-04T08:16:59.025 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1ec7ed84', total seconds elapsed: 33.3 2024-07-04T08:16:59.025 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-07-04T08:16:59.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:16:59.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:16:58 smithi050 ceph-mon[36577]: pgmap v533: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:59.150 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:16:58 smithi039 ceph-mon[32062]: pgmap v533: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:59.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:16:58 smithi133 ceph-mon[37050]: pgmap v533: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:16:59.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.498+0000 7f2d9be00640 1 -- 172.21.15.39:0/2545953406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c108820 msgr2=0x7f2d9c108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:59.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.498+0000 7f2d9be00640 1 --2- 172.21.15.39:0/2545953406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c108820 0x7f2d9c108c30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009900 tx=0x7f2d8402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.498+0000 7f2d9be00640 1 -- 172.21.15.39:0/2545953406 shutdown_connections 2024-07-04T08:16:59.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.498+0000 7f2d9be00640 1 --2- 172.21.15.39:0/2545953406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c108820 0x7f2d9c108c30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.498+0000 7f2d9be00640 1 --2- 172.21.15.39:0/2545953406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d9c109170 0x7f2d9c10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.498+0000 7f2d9be00640 1 --2- 172.21.15.39:0/2545953406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c102ee0 0x7f2d9c1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.498+0000 7f2d9be00640 1 -- 172.21.15.39:0/2545953406 >> 172.21.15.39:0/2545953406 conn(0x7f2d9c0fe930 msgr2=0x7f2d9c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.498+0000 7f2d9be00640 1 -- 172.21.15.39:0/2545953406 shutdown_connections 2024-07-04T08:16:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.498+0000 7f2d9be00640 1 -- 172.21.15.39:0/2545953406 wait complete. 2024-07-04T08:16:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.500+0000 7f2d9be00640 1 Processor -- start 2024-07-04T08:16:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.500+0000 7f2d9be00640 1 -- start start 2024-07-04T08:16:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.500+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d9c102ee0 0x7f2d9c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.500+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c108820 0x7f2d9c19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c109170 0x7f2d9c1a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d9be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c112e80 con 0x7f2d9c109170 2024-07-04T08:16:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d9be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c112d00 con 0x7f2d9c102ee0 2024-07-04T08:16:59.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d9be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c113000 con 0x7f2d9c108820 2024-07-04T08:16:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d9c102ee0 0x7f2d9c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d93e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c108820 0x7f2d9c19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d9c102ee0 0x7f2d9c19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48088/0 (socket says 172.21.15.39:48088) 2024-07-04T08:16:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d93e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c108820 0x7f2d9c19c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43232/0 (socket says 172.21.15.39:43232) 2024-07-04T08:16:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d9aa00640 1 -- 172.21.15.39:0/207062832 learned_addr learned my addr 172.21.15.39:0/207062832 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:59.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.501+0000 7f2d9b400640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c109170 0x7f2d9c1a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.502+0000 7f2d93e00640 1 -- 172.21.15.39:0/207062832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d9c102ee0 msgr2=0x7f2d9c19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.502+0000 7f2d93e00640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d9c102ee0 0x7f2d9c19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.502+0000 7f2d93e00640 1 -- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c109170 msgr2=0x7f2d9c1a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.502+0000 7f2d93e00640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c109170 0x7f2d9c1a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.502+0000 7f2d93e00640 1 -- 172.21.15.39:0/207062832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c1a19e0 con 0x7f2d9c108820 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.502+0000 7f2d93e00640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c108820 0x7f2d9c19c010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009230 tx=0x7f2d84003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.503+0000 7f2d93400640 1 -- 172.21.15.39:0/207062832 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8403f070 con 0x7f2d9c108820 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.503+0000 7f2d9be00640 1 -- 172.21.15.39:0/207062832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9c1a1c70 con 0x7f2d9c108820 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.503+0000 7f2d93400640 1 -- 172.21.15.39:0/207062832 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d84031b50 con 0x7f2d9c108820 2024-07-04T08:16:59.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.503+0000 7f2d93400640 1 -- 172.21.15.39:0/207062832 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84035030 con 0x7f2d9c108820 2024-07-04T08:16:59.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.503+0000 7f2d9be00640 1 -- 172.21.15.39:0/207062832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9c1a8430 con 0x7f2d9c108820 2024-07-04T08:16:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.506+0000 7f2da7f12740 1 Processor -- start 2024-07-04T08:16:59.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.507+0000 7f2da7f12740 1 -- start start 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.507+0000 7f2d93400640 1 -- 172.21.15.39:0/207062832 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2d84042070 con 0x7f2d9c108820 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.507+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 0x563a65297950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.507+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a65297e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.507+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.507+0000 7f2da7f12740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563a6520dee0 con 0x563a64ed2390 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.507+0000 7f2da7f12740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563a651fd480 con 0x563a64ef3070 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.507+0000 7f2da7f12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563a652014c0 con 0x563a64eca1b0 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 0x563a65297950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a65297e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a65297e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43242/0 (socket says 172.21.15.39:43242) 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 0x563a65297950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48102/0 (socket says 172.21.15.39:48102) 2024-07-04T08:16:59.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9aa00640 1 -- 172.21.15.39:0/1261293109 learned_addr learned my addr 172.21.15.39:0/1261293109 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d93e00640 1 --2- 172.21.15.39:0/1261293109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9aa00640 1 -- 172.21.15.39:0/1261293109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 msgr2=0x563a65297950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/1261293109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 0x563a65297950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9aa00640 1 -- 172.21.15.39:0/1261293109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 msgr2=0x563a652983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/1261293109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.508+0000 7f2d9aa00640 1 -- 172.21.15.39:0/1261293109 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a65195650 con 0x563a64eca1b0 2024-07-04T08:16:59.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.509+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/1261293109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a65297e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009dc0 tx=0x7f2d8c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=684eda79809906dc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:59.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.509+0000 7f2d8be00640 1 -- 172.21.15.39:0/1261293109 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c005220 con 0x563a64eca1b0 2024-07-04T08:16:59.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.509+0000 7f2d8be00640 1 -- 172.21.15.39:0/1261293109 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c004c50 con 0x563a64eca1b0 2024-07-04T08:16:59.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.509+0000 7f2d8be00640 1 -- 172.21.15.39:0/1261293109 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c019810 con 0x563a64eca1b0 2024-07-04T08:16:59.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.509+0000 7f2d93400640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2d800b29e0 0x7f2d800b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.510+0000 7f2d93400640 1 -- 172.21.15.39:0/207062832 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2d84103220 con 0x7f2d9c108820 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.510+0000 7f2d9b400640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2d800b29e0 0x7f2d800b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.510+0000 7f2da7f12740 1 -- 172.21.15.39:0/1261293109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 msgr2=0x563a65297e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.510+0000 7f2da7f12740 1 --2- 172.21.15.39:0/1261293109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a65297e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009dc0 tx=0x7f2d8c011810 comp rx=0 tx=0).stop 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.511+0000 7f2da7f12740 1 -- 172.21.15.39:0/1261293109 shutdown_connections 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.511+0000 7f2da7f12740 1 --2- 172.21.15.39:0/1261293109 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.511+0000 7f2da7f12740 1 --2- 172.21.15.39:0/1261293109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a65297e90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.511+0000 7f2da7f12740 1 --2- 172.21.15.39:0/1261293109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 0x563a65297950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.511+0000 7f2da7f12740 1 -- 172.21.15.39:0/1261293109 >> 172.21.15.39:0/1261293109 conn(0x563a64ee7460 msgr2=0x563a65295840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.511+0000 7f2d9b400640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2d800b29e0 0x7f2d800b4ef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005fd0 tx=0x7f2d94009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.511+0000 7f2da7f12740 1 -- 172.21.15.39:0/1261293109 shutdown_connections 2024-07-04T08:16:59.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.511+0000 7f2da7f12740 1 -- 172.21.15.39:0/1261293109 wait complete. 2024-07-04T08:16:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.512+0000 7f2da7f12740 1 -- start start 2024-07-04T08:16:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.512+0000 7f2da7f12740 1 Processor -- start 2024-07-04T08:16:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.512+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a652ac400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.512+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.512+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 0x563a652b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.512+0000 7f2da7f12740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563a65061700 con 0x563a64ed2390 2024-07-04T08:16:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.512+0000 7f2da7f12740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563a651a4bb0 con 0x563a64ef3070 2024-07-04T08:16:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.512+0000 7f2da7f12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563a652a6130 con 0x563a64eca1b0 2024-07-04T08:16:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2da7f12740 10 client.0 _ll_register_callbacks cb 0x563a651f6570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2d93e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2d93e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652ac9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56156/0 (socket says 172.21.15.39:56156) 2024-07-04T08:16:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2d93e00640 1 -- 172.21.15.39:0/971235871 learned_addr learned my addr 172.21.15.39:0/971235871 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:59.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2d9b400640 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 0x563a652b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2d93e00640 1 -- 172.21.15.39:0/971235871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 msgr2=0x563a652ac400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2d93e00640 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a652ac400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2d93e00640 1 -- 172.21.15.39:0/971235871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 msgr2=0x563a652b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2d93e00640 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 0x563a652b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.513+0000 7f2d93e00640 1 -- 172.21.15.39:0/971235871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a64f5e650 con 0x563a64ed2390 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.514+0000 7f2d93e00640 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652ac9e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2d840c9e70 tx=0x7f2d84003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bc5cdd1ffb4c40 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.514+0000 7f2d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.514+0000 7f2d88c00640 1 -- 172.21.15.39:0/971235871 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84049070 con 0x563a64ed2390 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.514+0000 7f2d88c00640 1 -- 172.21.15.39:0/971235871 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2d840ca3d0 con 0x563a64ed2390 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.514+0000 7f2da7f12740 1 -- 172.21.15.39:0/971235871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d8c004c50 con 0x563a64ed2390 2024-07-04T08:16:59.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.514+0000 7f2d88c00640 1 -- 172.21.15.39:0/971235871 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d840c98e0 con 0x563a64ed2390 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.514+0000 7f2da7f12740 10 client.15765 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.514+0000 7f2da7f12740 1 -- 172.21.15.39:0/971235871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a652ba950 con 0x563a64ed2390 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.515+0000 7f2d88c00640 1 -- 172.21.15.39:0/971235871 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c004c50 con 0x563a64ed2390 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.515+0000 7f2d88c00640 1 -- 172.21.15.39:0/971235871 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2d840ce170 con 0x563a64ed2390 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.515+0000 7f2da7f12740 10 client.15765 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.515+0000 7f2da7f12740 10 client.15765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.515+0000 7f2da7f12740 20 client.15765 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.515+0000 7f2da7f12740 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563a64eb19f0 0x563a652bde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.515+0000 7f2da7f12740 4 client.15765 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.515+0000 7f2da7f12740 1 -- 172.21.15.39:0/971235871 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a6520fe30 con 0x563a64eb19f0 2024-07-04T08:16:59.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.516+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563a64eb19f0 0x563a652bde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.517+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563a64eb19f0 0x563a652bde00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2317548f95048746 server_cookie=3b089a9bf79b44f2 in_seq=0 out_seq=0 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.517+0000 7f2d88c00640 10 client.15765 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.517+0000 7f2d88c00640 1 -- 172.21.15.39:0/971235871 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563a6520fe30 con 0x563a64eb19f0 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.517+0000 7f2d88c00640 10 client.15765 handle_command_reply: tid=0 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 client.15765 shutdown 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 -- 172.21.15.39:0/971235871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 msgr2=0x563a652ac9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652ac9e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2d840c9e70 tx=0x7f2d84003f90 comp rx=0 tx=0).stop 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 -- 172.21.15.39:0/971235871 shutdown_connections 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563a64eb19f0 0x563a652bde00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ed2390 0x563a652ac9e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ef3070 0x563a652b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 --2- 172.21.15.39:0/971235871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a652ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2d88c00640 0 client.15765 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:16:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 -- 172.21.15.39:0/971235871 >> 172.21.15.39:0/971235871 conn(0x563a64ee7460 msgr2=0x563a6529ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 -- 172.21.15.39:0/971235871 shutdown_connections 2024-07-04T08:16:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.518+0000 7f2da7f12740 1 -- 172.21.15.39:0/971235871 wait complete. 2024-07-04T08:16:59.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.519+0000 7f2da7f12740 20 client.15765 trim_cache size 0 max 16384 2024-07-04T08:16:59.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.534+0000 7f2da7f12740 1 Processor -- start 2024-07-04T08:16:59.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.534+0000 7f2da7f12740 1 -- start start 2024-07-04T08:16:59.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.535+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a652a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.535+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ed2390 0x563a652b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.535+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 0x563a6529aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.535+0000 7f2da7f12740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1ade10 con 0x563a64ef3070 2024-07-04T08:16:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.535+0000 7f2da7f12740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1af950 con 0x563a64ed2390 2024-07-04T08:16:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.535+0000 7f2da7f12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1ae4e0 con 0x563a64eca1b0 2024-07-04T08:16:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.535+0000 7f2d9b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 0x563a6529aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.535+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a652a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.535+0000 7f2d9b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 0x563a6529aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56162/0 (socket says 172.21.15.39:56162) 2024-07-04T08:16:59.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.536+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a652a6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43256/0 (socket says 172.21.15.39:43256) 2024-07-04T08:16:59.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.536+0000 7f2d9b400640 1 -- 172.21.15.39:0/3604069220 learned_addr learned my addr 172.21.15.39:0/3604069220 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:59.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.536+0000 7f2d9aa00640 1 -- 172.21.15.39:0/3604069220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ed2390 msgr2=0x563a652b8800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:16:59.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.536+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/3604069220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ed2390 0x563a652b8800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.536+0000 7f2d9aa00640 1 -- 172.21.15.39:0/3604069220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 msgr2=0x563a6529aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:59.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.536+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/3604069220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 0x563a6529aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.536+0000 7f2d9aa00640 1 -- 172.21.15.39:0/3604069220 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a652a6010 con 0x563a64eca1b0 2024-07-04T08:16:59.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.536+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/3604069220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a652a6a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0097b0 tx=0x7f2d8c0041f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=994f19714ae4d81c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:59.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.537+0000 7f2d8b400640 1 -- 172.21.15.39:0/3604069220 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0035f0 con 0x563a64eca1b0 2024-07-04T08:16:59.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.537+0000 7f2d8b400640 1 -- 172.21.15.39:0/3604069220 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c003c40 con 0x563a64eca1b0 2024-07-04T08:16:59.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.537+0000 7f2d8b400640 1 -- 172.21.15.39:0/3604069220 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0196e0 con 0x563a64eca1b0 2024-07-04T08:16:59.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.538+0000 7f2da7f12740 1 -- 172.21.15.39:0/3604069220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 msgr2=0x563a652a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:59.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.538+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3604069220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a652a6a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0097b0 tx=0x7f2d8c0041f0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.538+0000 7f2da7f12740 1 -- 172.21.15.39:0/3604069220 shutdown_connections 2024-07-04T08:16:59.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.538+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3604069220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 0x563a6529aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.538+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3604069220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64ed2390 0x563a652b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.538+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3604069220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64eca1b0 0x563a652a6a50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.538+0000 7f2da7f12740 1 -- 172.21.15.39:0/3604069220 >> 172.21.15.39:0/3604069220 conn(0x563a64ee7460 msgr2=0x563a652a6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:59.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.538+0000 7f2da7f12740 1 -- 172.21.15.39:0/3604069220 shutdown_connections 2024-07-04T08:16:59.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.538+0000 7f2da7f12740 1 -- 172.21.15.39:0/3604069220 wait complete. 2024-07-04T08:16:59.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.539+0000 7f2da7f12740 1 -- start start 2024-07-04T08:16:59.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.539+0000 7f2da7f12740 1 Processor -- start 2024-07-04T08:16:59.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64eca1b0 0x563a652b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64ed2390 0x563a652b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2da7f12740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 0x563a6531d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2da7f12740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1af280 con 0x563a64ef3070 2024-07-04T08:16:59.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2da7f12740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1aebb0 con 0x563a64eca1b0 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2da7f12740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c1abe50 con 0x563a64ed2390 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2da7f12740 10 client.0 _ll_register_callbacks cb 0x7f2d84031ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64eca1b0 0x563a652b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64eca1b0 0x563a652b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48114/0 (socket says 172.21.15.39:48114) 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2d9aa00640 1 -- 172.21.15.39:0/3988354432 learned_addr learned my addr 172.21.15.39:0/3988354432 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2d9b400640 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 0x563a6531d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.540+0000 7f2d93e00640 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64ed2390 0x563a652b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.541+0000 7f2d93e00640 1 -- 172.21.15.39:0/3988354432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64eca1b0 msgr2=0x563a652b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.541+0000 7f2d93e00640 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64eca1b0 0x563a652b59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.541+0000 7f2d93e00640 1 -- 172.21.15.39:0/3988354432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 msgr2=0x563a6531d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.541+0000 7f2d93e00640 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 0x563a6531d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.541+0000 7f2d93e00640 1 -- 172.21.15.39:0/3988354432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a6531dc30 con 0x563a64ed2390 2024-07-04T08:16:59.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.541+0000 7f2d93e00640 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64ed2390 0x563a652b5f90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2d8402a9b0 tx=0x7f2d840ca600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4348eb18c362ca32 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:16:59.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.541+0000 7f2d88c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.541+0000 7f2d88c00640 1 -- 172.21.15.39:0/3988354432 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84049070 con 0x563a64ed2390 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2da7f12740 1 -- 172.21.15.39:0/3988354432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d8c003c40 con 0x563a64ed2390 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2d88c00640 1 -- 172.21.15.39:0/3988354432 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2d840d2d50 con 0x563a64ed2390 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2d88c00640 1 -- 172.21.15.39:0/3988354432 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d84100a70 con 0x563a64ed2390 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2da7f12740 10 client.25852 fetch_fsmap learned FSMap version 30 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2da7f12740 1 -- 172.21.15.39:0/3988354432 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563a65326130 con 0x563a64ed2390 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2d88c00640 1 -- 172.21.15.39:0/3988354432 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c003c40 con 0x563a64ed2390 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2d88c00640 1 -- 172.21.15.39:0/3988354432 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2d840c9460 con 0x563a64ed2390 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2da7f12740 10 client.25852 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2da7f12740 10 client.25852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2da7f12740 20 client.25852 populate_metadata read hostname 'smithi039' 2024-07-04T08:16:59.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563a64e86da0 0x563a65328700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:16:59.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2da7f12740 4 client.25852 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:16:59.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.542+0000 7f2da7f12740 1 -- 172.21.15.39:0/3988354432 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a6520b8d0 con 0x563a64e86da0 2024-07-04T08:16:59.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.543+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563a64e86da0 0x563a65328700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:16:59.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.543+0000 7f2d9aa00640 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563a64e86da0 0x563a65328700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7c908b25f935957 server_cookie=a502b50a956f6afc in_seq=0 out_seq=0 2024-07-04T08:16:59.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.543+0000 7f2d88c00640 10 client.25852 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:16:59.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2d88c00640 1 -- 172.21.15.39:0/3988354432 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a6520b8d0 con 0x563a64e86da0 2024-07-04T08:16:59.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2d88c00640 10 client.25852 handle_command_reply: tid=0 2024-07-04T08:16:59.550 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 client.25852 shutdown 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 -- 172.21.15.39:0/3988354432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64ed2390 msgr2=0x563a652b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64ed2390 0x563a652b5f90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2d8402a9b0 tx=0x7f2d840ca600 comp rx=0 tx=0).stop 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 -- 172.21.15.39:0/3988354432 shutdown_connections 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563a64e86da0 0x563a65328700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563a64ef3070 0x563a6531d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563a64ed2390 0x563a652b5f90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 --2- 172.21.15.39:0/3988354432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563a64eca1b0 0x563a652b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2d88c00640 0 client.25852 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 -- 172.21.15.39:0/3988354432 >> 172.21.15.39:0/3988354432 conn(0x563a64ee7460 msgr2=0x563a65298980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 -- 172.21.15.39:0/3988354432 shutdown_connections 2024-07-04T08:16:59.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 1 -- 172.21.15.39:0/3988354432 wait complete. 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.544+0000 7f2da7f12740 20 client.25852 trim_cache size 0 max 16384 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 -- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2d800b29e0 msgr2=0x7f2d800b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2d800b29e0 0x7f2d800b4ef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2d94005fd0 tx=0x7f2d94009500 comp rx=0 tx=0).stop 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 -- 172.21.15.39:0/207062832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c108820 msgr2=0x7f2d9c19c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c108820 0x7f2d9c19c010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009230 tx=0x7f2d84003b90 comp rx=0 tx=0).stop 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 -- 172.21.15.39:0/207062832 shutdown_connections 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2d800b29e0 0x7f2d800b4ef0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c109170 0x7f2d9c1a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d9c108820 0x7f2d9c19c010 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 --2- 172.21.15.39:0/207062832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d9c102ee0 0x7f2d9c19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.545+0000 7f2d9be00640 1 -- 172.21.15.39:0/207062832 >> 172.21.15.39:0/207062832 conn(0x7f2d9c0fe930 msgr2=0x7f2d9c107f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.546+0000 7f2d9be00640 1 -- 172.21.15.39:0/207062832 shutdown_connections 2024-07-04T08:16:59.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:16:59.546+0000 7f2d9be00640 1 -- 172.21.15.39:0/207062832 wait complete. 2024-07-04T08:16:59.580 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:00.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:17:01.017 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:00 smithi039 ceph-mon[32062]: pgmap v534: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:17:01.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:00 smithi050 ceph-mon[36577]: pgmap v534: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:17:01.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:00 smithi133 ceph-mon[37050]: pgmap v534: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:17:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.775+0000 7f3ca3200640 1 -- 172.21.15.39:0/3302254935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca41050f0 msgr2=0x7f3ca4105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.775+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3302254935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca41050f0 0x7f3ca4105500 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009900 tx=0x7f3c9402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:17:02.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.776+0000 7f3ca3200640 1 -- 172.21.15.39:0/3302254935 shutdown_connections 2024-07-04T08:17:02.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.776+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3302254935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410b380 0x7f3ca4111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:02.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.776+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3302254935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca410aa30 0x7f3ca410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:02.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.776+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3302254935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca41050f0 0x7f3ca4105500 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:02.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.776+0000 7f3ca3200640 1 -- 172.21.15.39:0/3302254935 >> 172.21.15.39:0/3302254935 conn(0x7f3ca4100b60 msgr2=0x7f3ca4102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:02.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.776+0000 7f3ca3200640 1 -- 172.21.15.39:0/3302254935 shutdown_connections 2024-07-04T08:17:02.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.776+0000 7f3ca3200640 1 -- 172.21.15.39:0/3302254935 wait complete. 2024-07-04T08:17:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.777+0000 7f3ca3200640 1 Processor -- start 2024-07-04T08:17:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.777+0000 7f3ca3200640 1 -- start start 2024-07-04T08:17:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.777+0000 7f3ca3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca41050f0 0x7f3ca419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.777+0000 7f3ca3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410aa30 0x7f3ca419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca3200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b380 0x7f3ca41a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4115030 con 0x7f3ca410aa30 2024-07-04T08:17:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca3200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4114eb0 con 0x7f3ca410b380 2024-07-04T08:17:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41151b0 con 0x7f3ca41050f0 2024-07-04T08:17:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca41050f0 0x7f3ca419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca41050f0 0x7f3ca419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33340/0 (socket says 172.21.15.39:33340) 2024-07-04T08:17:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca1e00640 1 -- 172.21.15.39:0/3766791576 learned_addr learned my addr 172.21.15.39:0/3766791576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca1400640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410aa30 0x7f3ca419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca2800640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b380 0x7f3ca41a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:02.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca1e00640 1 -- 172.21.15.39:0/3766791576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b380 msgr2=0x7f3ca41a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:02.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca1e00640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b380 0x7f3ca41a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:02.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca1e00640 1 -- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410aa30 msgr2=0x7f3ca419bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:02.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca1e00640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410aa30 0x7f3ca419bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:02.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.778+0000 7f3ca1e00640 1 -- 172.21.15.39:0/3766791576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca41a19b0 con 0x7f3ca41050f0 2024-07-04T08:17:02.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.779+0000 7f3ca1e00640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca41050f0 0x7f3ca419ba40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c94005940 tx=0x7f3c940040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.779+0000 7f3c9aa00640 1 -- 172.21.15.39:0/3766791576 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9403f030 con 0x7f3ca41050f0 2024-07-04T08:17:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.779+0000 7f3c9aa00640 1 -- 172.21.15.39:0/3766791576 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c940044c0 con 0x7f3ca41050f0 2024-07-04T08:17:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.779+0000 7f3c9aa00640 1 -- 172.21.15.39:0/3766791576 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94035070 con 0x7f3ca41050f0 2024-07-04T08:17:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.779+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca41a1c40 con 0x7f3ca41050f0 2024-07-04T08:17:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.779+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca41a8400 con 0x7f3ca41050f0 2024-07-04T08:17:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.781+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca4105500 con 0x7f3ca41050f0 2024-07-04T08:17:02.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.784+0000 7f3c9aa00640 1 -- 172.21.15.39:0/3766791576 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3c94042030 con 0x7f3ca41050f0 2024-07-04T08:17:02.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.786+0000 7f3c9aa00640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b29e0 0x7f3c880b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:02.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.786+0000 7f3c9aa00640 1 -- 172.21.15.39:0/3766791576 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f3c94102e90 con 0x7f3ca41050f0 2024-07-04T08:17:02.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.786+0000 7f3ca1400640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b29e0 0x7f3c880b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:02.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.787+0000 7f3c9aa00640 1 -- 172.21.15.39:0/3766791576 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f3c940c9e90 con 0x7f3ca41050f0 2024-07-04T08:17:02.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:02.787+0000 7f3ca1400640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b29e0 0x7f3c880b4ef0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3ca410a5c0 tx=0x7f3c90005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:03.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.046+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ca419cb70 con 0x7f3ca41050f0 2024-07-04T08:17:03.049 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:02 smithi039 ceph-mon[32062]: pgmap v535: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-04T08:17:03.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.048+0000 7f3c9aa00640 1 -- 172.21.15.39:0/3766791576 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f3c940ce190 con 0x7f3ca41050f0 2024-07-04T08:17:03.050 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:17:03.051 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:17:03.051 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:17:03.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b29e0 msgr2=0x7f3c880b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:03.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b29e0 0x7f3c880b4ef0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3ca410a5c0 tx=0x7f3c90005f90 comp rx=0 tx=0).stop 2024-07-04T08:17:03.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca41050f0 msgr2=0x7f3ca419ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:03.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca41050f0 0x7f3ca419ba40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c94005940 tx=0x7f3c940040c0 comp rx=0 tx=0).stop 2024-07-04T08:17:03.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 shutdown_connections 2024-07-04T08:17:03.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3c880b29e0 0x7f3c880b4ef0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:03.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ca410aa30 0x7f3ca419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:03.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ca410b380 0x7f3ca41a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:03.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 --2- 172.21.15.39:0/3766791576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca41050f0 0x7f3ca419ba40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:03.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 >> 172.21.15.39:0/3766791576 conn(0x7f3ca4100b60 msgr2=0x7f3ca4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:03.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 shutdown_connections 2024-07-04T08:17:03.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:03.051+0000 7f3ca3200640 1 -- 172.21.15.39:0/3766791576 wait complete. 2024-07-04T08:17:03.145 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:02 smithi050 ceph-mon[36577]: pgmap v535: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-04T08:17:03.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:02 smithi133 ceph-mon[37050]: pgmap v535: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-04T08:17:04.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:03 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3766791576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:17:04.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:03 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3766791576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:17:04.393 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-07-04T08:17:04.393 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-07-04T08:17:04.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:03 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3766791576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:17:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.005+0000 7fed6ac00640 1 -- 172.21.15.39:0/4055600767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 msgr2=0x7fed641056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.005+0000 7fed6ac00640 1 --2- 172.21.15.39:0/4055600767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 0x7fed641056e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fed54009900 tx=0x7fed5402a410 comp rx=0 tx=0).stop 2024-07-04T08:17:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.005+0000 7fed6ac00640 1 -- 172.21.15.39:0/4055600767 shutdown_connections 2024-07-04T08:17:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.005+0000 7fed6ac00640 1 --2- 172.21.15.39:0/4055600767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed6410b560 0x7fed64111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.005+0000 7fed6ac00640 1 --2- 172.21.15.39:0/4055600767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed6410ac10 0x7fed6410b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.005+0000 7fed6ac00640 1 --2- 172.21.15.39:0/4055600767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 0x7fed641056e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.005+0000 7fed6ac00640 1 -- 172.21.15.39:0/4055600767 >> 172.21.15.39:0/4055600767 conn(0x7fed64100d60 msgr2=0x7fed641031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.005+0000 7fed6ac00640 1 -- 172.21.15.39:0/4055600767 shutdown_connections 2024-07-04T08:17:05.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.005+0000 7fed6ac00640 1 -- 172.21.15.39:0/4055600767 wait complete. 2024-07-04T08:17:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.006+0000 7fed6ac00640 1 Processor -- start 2024-07-04T08:17:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.006+0000 7fed6ac00640 1 -- start start 2024-07-04T08:17:05.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed6ac00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 0x7fed6419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed6ac00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed6410ac10 0x7fed6419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed6ac00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed6410b560 0x7fed641a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed6ac00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed64115260 con 0x7fed641052d0 2024-07-04T08:17:05.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed6ac00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed641150e0 con 0x7fed6410ac10 2024-07-04T08:17:05.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed6ac00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed641153e0 con 0x7fed6410b560 2024-07-04T08:17:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed63e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 0x7fed6419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed68c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed6410b560 0x7fed641a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed63e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 0x7fed6419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42888/0 (socket says 172.21.15.39:42888) 2024-07-04T08:17:05.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed5be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed6410ac10 0x7fed6419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.008+0000 7fed63e00640 1 -- 172.21.15.39:0/337130861 learned_addr learned my addr 172.21.15.39:0/337130861 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.007+0000 7fed68c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed6410b560 0x7fed641a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33362/0 (socket says 172.21.15.39:33362) 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.008+0000 7fed63e00640 1 -- 172.21.15.39:0/337130861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed6410b560 msgr2=0x7fed641a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.008+0000 7fed63e00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed6410b560 0x7fed641a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.008+0000 7fed63e00640 1 -- 172.21.15.39:0/337130861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed6410ac10 msgr2=0x7fed6419c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.008+0000 7fed63e00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed6410ac10 0x7fed6419c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.008+0000 7fed63e00640 1 -- 172.21.15.39:0/337130861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed641a1be0 con 0x7fed641052d0 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.009+0000 7fed63e00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 0x7fed6419bc70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fed54005ab0 tx=0x7fed54003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.009+0000 7fed61600640 1 -- 172.21.15.39:0/337130861 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5403f070 con 0x7fed641052d0 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.009+0000 7fed61600640 1 -- 172.21.15.39:0/337130861 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed54031b60 con 0x7fed641052d0 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.009+0000 7fed6ac00640 1 -- 172.21.15.39:0/337130861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed641a1e70 con 0x7fed641052d0 2024-07-04T08:17:05.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.009+0000 7fed61600640 1 -- 172.21.15.39:0/337130861 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54035030 con 0x7fed641052d0 2024-07-04T08:17:05.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.009+0000 7fed6ac00640 1 -- 172.21.15.39:0/337130861 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed641a8630 con 0x7fed641052d0 2024-07-04T08:17:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.015+0000 7fed61600640 1 -- 172.21.15.39:0/337130861 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fed54042070 con 0x7fed641052d0 2024-07-04T08:17:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.015+0000 7fed7132a740 1 Processor -- start 2024-07-04T08:17:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.015+0000 7fed7132a740 1 -- start start 2024-07-04T08:17:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.015+0000 7fed7132a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8791d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.015+0000 7fed7132a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c875501b0 0x559c8791dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.015+0000 7fed7132a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.015+0000 7fed7132a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559c87893f00 con 0x559c87558390 2024-07-04T08:17:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.015+0000 7fed7132a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559c878834a0 con 0x559c87579070 2024-07-04T08:17:05.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.015+0000 7fed7132a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559c878874e0 con 0x559c875501b0 2024-07-04T08:17:05.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.016+0000 7fed63e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c875501b0 0x559c8791dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.016+0000 7fed63e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c875501b0 0x559c8791dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33374/0 (socket says 172.21.15.39:33374) 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.016+0000 7fed63e00640 1 -- 172.21.15.39:0/13624254 learned_addr learned my addr 172.21.15.39:0/13624254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.016+0000 7fed5be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.016+0000 7fed68c00640 1 --2- 172.21.15.39:0/13624254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8791d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed68c00640 1 -- 172.21.15.39:0/13624254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c875501b0 msgr2=0x559c8791dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed68c00640 1 --2- 172.21.15.39:0/13624254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c875501b0 0x559c8791dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed68c00640 1 -- 172.21.15.39:0/13624254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 msgr2=0x559c8791e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed68c00640 1 --2- 172.21.15.39:0/13624254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed68c00640 1 -- 172.21.15.39:0/13624254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c8781b670 con 0x559c87579070 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed61600640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed480b29e0 0x7fed480b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed5be00640 1 --2- 172.21.15.39:0/13624254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:05.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed61600640 1 -- 172.21.15.39:0/337130861 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fed54102f30 con 0x7fed641052d0 2024-07-04T08:17:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed68c00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed480b29e0 0x7fed480b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.017+0000 7fed68c00640 1 --2- 172.21.15.39:0/13624254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8791d7d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009f00 tx=0x7fed5c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1acf4f4bd8ff62 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.018+0000 7fed59600640 1 -- 172.21.15.39:0/13624254 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c003cb0 con 0x559c87579070 2024-07-04T08:17:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.018+0000 7fed59600640 1 -- 172.21.15.39:0/13624254 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed5c01bae0 con 0x559c87579070 2024-07-04T08:17:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.018+0000 7fed59600640 1 -- 172.21.15.39:0/13624254 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c0199d0 con 0x559c87579070 2024-07-04T08:17:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed7132a740 1 -- 172.21.15.39:0/13624254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 msgr2=0x559c8791d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed7132a740 1 --2- 172.21.15.39:0/13624254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8791d7d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009f00 tx=0x7fed5c011810 comp rx=0 tx=0).stop 2024-07-04T08:17:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed68c00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed480b29e0 0x7fed480b4ef0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fed5c020000 tx=0x7fed5c021000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed7132a740 1 -- 172.21.15.39:0/13624254 shutdown_connections 2024-07-04T08:17:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed7132a740 1 --2- 172.21.15.39:0/13624254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed7132a740 1 --2- 172.21.15.39:0/13624254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c875501b0 0x559c8791dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed7132a740 1 --2- 172.21.15.39:0/13624254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8791d7d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed7132a740 1 -- 172.21.15.39:0/13624254 >> 172.21.15.39:0/13624254 conn(0x559c8756d460 msgr2=0x559c8791b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:05.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed7132a740 1 -- 172.21.15.39:0/13624254 shutdown_connections 2024-07-04T08:17:05.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.019+0000 7fed7132a740 1 -- 172.21.15.39:0/13624254 wait complete. 2024-07-04T08:17:05.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.020+0000 7fed7132a740 1 -- start start 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.020+0000 7fed7132a740 1 Processor -- start 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed7132a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87929360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed7132a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87558390 0x559c879298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed7132a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c87935300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed7132a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559c876e7700 con 0x559c875501b0 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed7132a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559c8782abd0 con 0x559c87558390 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed7132a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559c8792bfb0 con 0x559c87579070 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed7132a740 10 client.0 _ll_register_callbacks cb 0x559c8787c590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed63e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87929360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed63e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87929360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42912/0 (socket says 172.21.15.39:42912) 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed63e00640 1 -- 172.21.15.39:0/2791095316 learned_addr learned my addr 172.21.15.39:0/2791095316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed68c00640 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c87935300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.021+0000 7fed5be00640 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87558390 0x559c879298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.022+0000 7fed63e00640 1 -- 172.21.15.39:0/2791095316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 msgr2=0x559c87935300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.022+0000 7fed63e00640 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c87935300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.022+0000 7fed63e00640 1 -- 172.21.15.39:0/2791095316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87558390 msgr2=0x559c879298a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.022+0000 7fed63e00640 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87558390 0x559c879298a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.022+0000 7fed63e00640 1 -- 172.21.15.39:0/2791095316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c875e4650 con 0x559c875501b0 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.022+0000 7fed63e00640 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87929360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fed54100400 tx=0x7fed54009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4702880ed50f2a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.022+0000 7fed52000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:17:05.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.023+0000 7fed52000640 1 -- 172.21.15.39:0/2791095316 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54049050 con 0x559c875501b0 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.023+0000 7fed52000640 1 -- 172.21.15.39:0/2791095316 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fed540c6220 con 0x559c875501b0 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.023+0000 7fed7132a740 1 -- 172.21.15.39:0/2791095316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed5c01bae0 con 0x559c875501b0 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.023+0000 7fed52000640 1 -- 172.21.15.39:0/2791095316 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed540c98e0 con 0x559c875501b0 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.023+0000 7fed7132a740 10 client.15789 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.023+0000 7fed7132a740 1 -- 172.21.15.39:0/2791095316 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559c87937ff0 con 0x559c875501b0 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.023+0000 7fed52000640 1 -- 172.21.15.39:0/2791095316 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed5c01bae0 con 0x559c875501b0 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.024+0000 7fed52000640 1 -- 172.21.15.39:0/2791095316 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fed5403b460 con 0x559c875501b0 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.024+0000 7fed7132a740 10 client.15789 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.024+0000 7fed7132a740 10 client.15789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.024+0000 7fed7132a740 20 client.15789 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.024+0000 7fed7132a740 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559c875379f0 0x559c8793b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.024+0000 7fed7132a740 4 client.15789 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:05.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.024+0000 7fed7132a740 1 -- 172.21.15.39:0/2791095316 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c87895e50 con 0x559c875379f0 2024-07-04T08:17:05.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.024+0000 7fed5be00640 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559c875379f0 0x559c8793b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.025+0000 7fed52000640 10 client.15789 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:17:05.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.025+0000 7fed5be00640 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559c875379f0 0x559c8793b4f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7122d3a886622480 server_cookie=5798d70d083b3135 in_seq=0 out_seq=0 2024-07-04T08:17:05.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed52000640 1 -- 172.21.15.39:0/2791095316 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c87895e50 con 0x559c875379f0 2024-07-04T08:17:05.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed52000640 10 client.15789 handle_command_reply: tid=0 2024-07-04T08:17:05.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed7132a740 1 client.15789 shutdown 2024-07-04T08:17:05.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed7132a740 1 -- 172.21.15.39:0/2791095316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 msgr2=0x559c87929360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:05.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed7132a740 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87929360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fed54100400 tx=0x7fed54009620 comp rx=0 tx=0).stop 2024-07-04T08:17:05.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed7132a740 1 -- 172.21.15.39:0/2791095316 shutdown_connections 2024-07-04T08:17:05.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed7132a740 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559c875379f0 0x559c8793b4f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed7132a740 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c87935300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed7132a740 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87558390 0x559c879298a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed7132a740 1 --2- 172.21.15.39:0/2791095316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87929360 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed52000640 0 client.15789 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:17:05.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.026+0000 7fed7132a740 1 -- 172.21.15.39:0/2791095316 >> 172.21.15.39:0/2791095316 conn(0x559c8756d460 msgr2=0x559c87920960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:05.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.027+0000 7fed7132a740 1 -- 172.21.15.39:0/2791095316 shutdown_connections 2024-07-04T08:17:05.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.027+0000 7fed7132a740 1 -- 172.21.15.39:0/2791095316 wait complete. 2024-07-04T08:17:05.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.027+0000 7fed7132a740 20 client.15789 trim_cache size 0 max 16384 2024-07-04T08:17:05.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.043+0000 7fed7132a740 1 Processor -- start 2024-07-04T08:17:05.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.043+0000 7fed7132a740 1 -- start start 2024-07-04T08:17:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.043+0000 7fed7132a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c875501b0 0x559c8791a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.043+0000 7fed7132a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.043+0000 7fed7132a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c879350f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.043+0000 7fed7132a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed641b24f0 con 0x559c87558390 2024-07-04T08:17:05.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.043+0000 7fed7132a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed641b4030 con 0x559c875501b0 2024-07-04T08:17:05.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.043+0000 7fed7132a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed641b2bc0 con 0x559c87579070 2024-07-04T08:17:05.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed63e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c875501b0 0x559c8791a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed68c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c879350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed68c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c879350f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33392/0 (socket says 172.21.15.39:33392) 2024-07-04T08:17:05.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed68c00640 1 -- 172.21.15.39:0/550640819 learned_addr learned my addr 172.21.15.39:0/550640819 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:05.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed5be00640 1 --2- 172.21.15.39:0/550640819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed68c00640 1 -- 172.21.15.39:0/550640819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c875501b0 msgr2=0x559c8791a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:05.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed68c00640 1 --2- 172.21.15.39:0/550640819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c875501b0 0x559c8791a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed68c00640 1 -- 172.21.15.39:0/550640819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 msgr2=0x559c8791b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:05.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed68c00640 1 --2- 172.21.15.39:0/550640819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed68c00640 1 -- 172.21.15.39:0/550640819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c8791fcf0 con 0x559c87579070 2024-07-04T08:17:05.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.044+0000 7fed5be00640 1 --2- 172.21.15.39:0/550640819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:05.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.045+0000 7fed68c00640 1 --2- 172.21.15.39:0/550640819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c879350f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009730 tx=0x7fed5c027000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594bdce7a1f9f542 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:05.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.045+0000 7fed58c00640 1 -- 172.21.15.39:0/550640819 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c035070 con 0x559c87579070 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.045+0000 7fed58c00640 1 -- 172.21.15.39:0/550640819 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed5c019420 con 0x559c87579070 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.045+0000 7fed58c00640 1 -- 172.21.15.39:0/550640819 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c029040 con 0x559c87579070 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.046+0000 7fed7132a740 1 -- 172.21.15.39:0/550640819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 msgr2=0x559c879350f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.046+0000 7fed7132a740 1 --2- 172.21.15.39:0/550640819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c879350f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009730 tx=0x7fed5c027000 comp rx=0 tx=0).stop 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.046+0000 7fed7132a740 1 -- 172.21.15.39:0/550640819 shutdown_connections 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.046+0000 7fed7132a740 1 --2- 172.21.15.39:0/550640819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c87558390 0x559c8791b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.046+0000 7fed7132a740 1 --2- 172.21.15.39:0/550640819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87579070 0x559c879350f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.046+0000 7fed7132a740 1 --2- 172.21.15.39:0/550640819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c875501b0 0x559c8791a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.046+0000 7fed7132a740 1 -- 172.21.15.39:0/550640819 >> 172.21.15.39:0/550640819 conn(0x559c8756d460 msgr2=0x559c8791b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:05.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.047+0000 7fed7132a740 1 -- 172.21.15.39:0/550640819 shutdown_connections 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.047+0000 7fed7132a740 1 -- 172.21.15.39:0/550640819 wait complete. 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.047+0000 7fed7132a740 1 -- start start 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.047+0000 7fed7132a740 1 Processor -- start 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.048+0000 7fed7132a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87931270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.048+0000 7fed7132a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87558390 0x559c879317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.048+0000 7fed7132a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8792c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.048+0000 7fed7132a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed641b3960 con 0x559c875501b0 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.048+0000 7fed7132a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed641b3290 con 0x559c87579070 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.048+0000 7fed7132a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed641b04d0 con 0x559c87558390 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.048+0000 7fed7132a740 10 client.0 _ll_register_callbacks cb 0x7fed54005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:05.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed68c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8792c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed5be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87558390 0x559c879317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed5be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87558390 0x559c879317b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33402/0 (socket says 172.21.15.39:33402) 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed5be00640 1 -- 172.21.15.39:0/3647772973 learned_addr learned my addr 172.21.15.39:0/3647772973 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed68c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8792c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44906/0 (socket says 172.21.15.39:44906) 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed63e00640 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87931270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed5be00640 1 -- 172.21.15.39:0/3647772973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 msgr2=0x559c8792c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed5be00640 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8792c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed5be00640 1 -- 172.21.15.39:0/3647772973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 msgr2=0x559c87931270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed5be00640 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87931270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed5be00640 1 -- 172.21.15.39:0/3647772973 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c87931da0 con 0x559c87558390 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed68c00640 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8792c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:05.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.049+0000 7fed5be00640 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87558390 0x559c879317b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fed4c000c00 tx=0x7fed4c0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3ca8e727f75807 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.050+0000 7fed52000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.050+0000 7fed52000640 1 -- 172.21.15.39:0/3647772973 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c010840 con 0x559c87558390 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.050+0000 7fed52000640 1 -- 172.21.15.39:0/3647772973 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fed4c0109e0 con 0x559c87558390 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.050+0000 7fed7132a740 1 -- 172.21.15.39:0/3647772973 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed5c019420 con 0x559c87558390 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.050+0000 7fed52000640 1 -- 172.21.15.39:0/3647772973 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed4c030050 con 0x559c87558390 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.050+0000 7fed7132a740 10 client.25876 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.050+0000 7fed7132a740 1 -- 172.21.15.39:0/3647772973 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559c879a4160 con 0x559c87558390 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.051+0000 7fed52000640 1 -- 172.21.15.39:0/3647772973 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed5c019420 con 0x559c87558390 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.051+0000 7fed52000640 1 -- 172.21.15.39:0/3647772973 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fed4c01e3f0 con 0x559c87558390 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.051+0000 7fed7132a740 10 client.25876 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:05.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.051+0000 7fed7132a740 10 client.25876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:17:05.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.051+0000 7fed7132a740 20 client.25876 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:05.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.051+0000 7fed7132a740 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559c8750cda0 0x559c879a6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:05.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.051+0000 7fed7132a740 4 client.25876 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:17:05.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.051+0000 7fed7132a740 1 -- 172.21.15.39:0/3647772973 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c878918f0 con 0x559c8750cda0 2024-07-04T08:17:05.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.051+0000 7fed63e00640 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559c8750cda0 0x559c879a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:05.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.052+0000 7fed63e00640 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559c8750cda0 0x559c879a6900 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c901e14690d5908d server_cookie=8ef5b3cb8e097982 in_seq=0 out_seq=0 2024-07-04T08:17:05.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.052+0000 7fed52000640 10 client.25876 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:17:05.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.052+0000 7fed52000640 1 -- 172.21.15.39:0/3647772973 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c878918f0 con 0x559c8750cda0 2024-07-04T08:17:05.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.052+0000 7fed52000640 10 client.25876 handle_command_reply: tid=0 2024-07-04T08:17:05.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.052+0000 7fed7132a740 1 client.25876 shutdown 2024-07-04T08:17:05.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 -- 172.21.15.39:0/3647772973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87558390 msgr2=0x559c879317b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:05.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87558390 0x559c879317b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fed4c000c00 tx=0x7fed4c0039c0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 -- 172.21.15.39:0/3647772973 shutdown_connections 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559c8750cda0 0x559c879a6900 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c87579070 0x559c8792c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559c87558390 0x559c879317b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 --2- 172.21.15.39:0/3647772973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559c875501b0 0x559c87931270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed52000640 0 client.25876 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 -- 172.21.15.39:0/3647772973 >> 172.21.15.39:0/3647772973 conn(0x559c8756d460 msgr2=0x559c8791e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 -- 172.21.15.39:0/3647772973 shutdown_connections 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.053+0000 7fed7132a740 1 -- 172.21.15.39:0/3647772973 wait complete. 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed7132a740 20 client.25876 trim_cache size 0 max 16384 2024-07-04T08:17:05.060 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:17:05.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 -- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed480b29e0 msgr2=0x7fed480b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:05.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed480b29e0 0x7fed480b4ef0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fed5c020000 tx=0x7fed5c021000 comp rx=0 tx=0).stop 2024-07-04T08:17:05.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 -- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 msgr2=0x7fed6419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:05.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 0x7fed6419bc70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fed54005ab0 tx=0x7fed54003c10 comp rx=0 tx=0).stop 2024-07-04T08:17:05.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 -- 172.21.15.39:0/337130861 shutdown_connections 2024-07-04T08:17:05.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed480b29e0 0x7fed480b4ef0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed6410b560 0x7fed641a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed6410ac10 0x7fed6419c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 --2- 172.21.15.39:0/337130861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed641052d0 0x7fed6419bc70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:05.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.054+0000 7fed6ac00640 1 -- 172.21.15.39:0/337130861 >> 172.21.15.39:0/337130861 conn(0x7fed64100d60 msgr2=0x7fed64103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:05.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.055+0000 7fed6ac00640 1 -- 172.21.15.39:0/337130861 shutdown_connections 2024-07-04T08:17:05.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:05.055+0000 7fed6ac00640 1 -- 172.21.15.39:0/337130861 wait complete. 2024-07-04T08:17:05.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:04 smithi133 ceph-mon[37050]: pgmap v536: 129 pgs: 129 active+clean; 1.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-07-04T08:17:05.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:04 smithi039 ceph-mon[32062]: pgmap v536: 129 pgs: 129 active+clean; 1.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-07-04T08:17:05.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:04 smithi050 ceph-mon[36577]: pgmap v536: 129 pgs: 129 active+clean; 1.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-07-04T08:17:05.884 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:17:05.885 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:07.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:06 smithi133 ceph-mon[37050]: pgmap v537: 129 pgs: 129 active+clean; 1.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-07-04T08:17:07.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:06 smithi039 ceph-mon[32062]: pgmap v537: 129 pgs: 129 active+clean; 1.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-07-04T08:17:07.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:06 smithi050 ceph-mon[36577]: pgmap v537: 129 pgs: 129 active+clean; 1.5 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 30 KiB/s wr, 26 op/s 2024-07-04T08:17:07.404 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:09.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:08 smithi133 ceph-mon[37050]: pgmap v538: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 29 op/s 2024-07-04T08:17:09.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:08 smithi039 ceph-mon[32062]: pgmap v538: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 29 op/s 2024-07-04T08:17:09.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:08 smithi050 ceph-mon[36577]: pgmap v538: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 29 op/s 2024-07-04T08:17:09.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.395+0000 7efdf5600640 1 -- 172.21.15.39:0/3367924914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdf010aa60 msgr2=0x7efdf010ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:09.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.395+0000 7efdf5600640 1 --2- 172.21.15.39:0/3367924914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdf010aa60 0x7efdf010ae70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efddc009900 tx=0x7efddc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.395+0000 7efdf5600640 1 -- 172.21.15.39:0/3367924914 shutdown_connections 2024-07-04T08:17:09.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.395+0000 7efdf5600640 1 --2- 172.21.15.39:0/3367924914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdf010aa60 0x7efdf010ae70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.395+0000 7efdf5600640 1 --2- 172.21.15.39:0/3367924914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf010b3b0 0x7efdf0111be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.395+0000 7efdf5600640 1 --2- 172.21.15.39:0/3367924914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf0105120 0x7efdf0105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.395+0000 7efdf5600640 1 -- 172.21.15.39:0/3367924914 >> 172.21.15.39:0/3367924914 conn(0x7efdf0100b90 msgr2=0x7efdf0102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:09.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.395+0000 7efdf5600640 1 -- 172.21.15.39:0/3367924914 shutdown_connections 2024-07-04T08:17:09.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.395+0000 7efdf5600640 1 -- 172.21.15.39:0/3367924914 wait complete. 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.396+0000 7efdf5600640 1 Processor -- start 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.396+0000 7efdf5600640 1 -- start start 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdf5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf0105120 0x7efdf019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdf5600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf010aa60 0x7efdf019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdf5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdf010b3b0 0x7efdf01a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdf5600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0115070 con 0x7efdf010b3b0 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdf5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0114ef0 con 0x7efdf0105120 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdf5600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01151f0 con 0x7efdf010aa60 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf0105120 0x7efdf019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdf4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdf010b3b0 0x7efdf01a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf010aa60 0x7efdf019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf0105120 0x7efdf019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44916/0 (socket says 172.21.15.39:44916) 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdefe00640 1 -- 172.21.15.39:0/511138990 learned_addr learned my addr 172.21.15.39:0/511138990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.397+0000 7efdef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf010aa60 0x7efdf019c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33414/0 (socket says 172.21.15.39:33414) 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.398+0000 7efdef400640 1 -- 172.21.15.39:0/511138990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf0105120 msgr2=0x7efdf019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.398+0000 7efdef400640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf0105120 0x7efdf019ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.398+0000 7efdef400640 1 -- 172.21.15.39:0/511138990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdf010b3b0 msgr2=0x7efdf01a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.398+0000 7efdef400640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdf010b3b0 0x7efdf01a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.398+0000 7efdef400640 1 -- 172.21.15.39:0/511138990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf01a19d0 con 0x7efdf010aa60 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.398+0000 7efdef400640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf010aa60 0x7efdf019c000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efddc009230 tx=0x7efddc003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.399+0000 7efdecc00640 1 -- 172.21.15.39:0/511138990 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc03f070 con 0x7efdf010aa60 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.399+0000 7efdf5600640 1 -- 172.21.15.39:0/511138990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf01a1c60 con 0x7efdf010aa60 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.399+0000 7efdecc00640 1 -- 172.21.15.39:0/511138990 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efddc031b50 con 0x7efdf010aa60 2024-07-04T08:17:09.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.399+0000 7efdecc00640 1 -- 172.21.15.39:0/511138990 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc035030 con 0x7efdf010aa60 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.399+0000 7efdf5600640 1 -- 172.21.15.39:0/511138990 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf01a8420 con 0x7efdf010aa60 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.403+0000 7efdfd2ad740 1 Processor -- start 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.403+0000 7efdfd2ad740 1 -- start start 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.404+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.404+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570babe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.404+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.404+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560570b71140 con 0x5605707e5b70 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.404+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560570a92380 con 0x5605707dd990 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.404+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560570b20c30 con 0x560570806850 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.404+0000 7efdecc00640 1 -- 172.21.15.39:0/511138990 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7efddc042070 con 0x7efdf010aa60 2024-07-04T08:17:09.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.404+0000 7efdf4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570babe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdf4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33418/0 (socket says 172.21.15.39:33418) 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdf4c00640 1 -- 172.21.15.39:0/2852009574 learned_addr learned my addr 172.21.15.39:0/2852009574 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdef400640 1 --2- 172.21.15.39:0/2852009574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdf4c00640 1 -- 172.21.15.39:0/2852009574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 msgr2=0x560570babe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdf4c00640 1 --2- 172.21.15.39:0/2852009574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570babe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdf4c00640 1 -- 172.21.15.39:0/2852009574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 msgr2=0x560570bac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdf4c00640 1 --2- 172.21.15.39:0/2852009574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdf4c00640 1 -- 172.21.15.39:0/2852009574 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560570880ba0 con 0x560570806850 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdef400640 1 --2- 172.21.15.39:0/2852009574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.405+0000 7efdf4c00640 1 --2- 172.21.15.39:0/2852009574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bab950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efde8011440 tx=0x7efde8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8be018481b5f814 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.406+0000 7efde5600640 1 -- 172.21.15.39:0/2852009574 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8003c30 con 0x560570806850 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.406+0000 7efde5600640 1 -- 172.21.15.39:0/2852009574 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde8003dd0 con 0x560570806850 2024-07-04T08:17:09.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.406+0000 7efde5600640 1 -- 172.21.15.39:0/2852009574 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde800fae0 con 0x560570806850 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.406+0000 7efdecc00640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdd40b2950 0x7efdd40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.406+0000 7efdecc00640 1 -- 172.21.15.39:0/511138990 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7efddc102ed0 con 0x7efdf010aa60 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.406+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2852009574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 msgr2=0x560570bab950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.406+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2852009574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bab950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efde8011440 tx=0x7efde8011900 comp rx=0 tx=0).stop 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.407+0000 7efdefe00640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdd40b2950 0x7efdd40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.407+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2852009574 shutdown_connections 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.407+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2852009574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.407+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2852009574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570babe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.407+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2852009574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bab950 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.407+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2852009574 >> 172.21.15.39:0/2852009574 conn(0x5605707fac40 msgr2=0x560570ba9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.407+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2852009574 shutdown_connections 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.407+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2852009574 wait complete. 2024-07-04T08:17:09.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.407+0000 7efdefe00640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdd40b2950 0x7efdd40b4e60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x560570baa5d0 tx=0x7efde0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.408+0000 7efdfd2ad740 1 -- start start 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.408+0000 7efdfd2ad740 1 Processor -- start 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.408+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bb74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bb7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bc32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560570b14210 con 0x5605707e5b70 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560570b101d0 con 0x5605707dd990 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560570ab75d0 con 0x560570806850 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdfd2ad740 10 client.0 _ll_register_callbacks cb 0x56057081e0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bb74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bb7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bb74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44946/0 (socket says 172.21.15.39:44946) 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdefe00640 1 -- 172.21.15.39:0/3263225432 learned_addr learned my addr 172.21.15.39:0/3263225432 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdf4c00640 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bc32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdef400640 1 -- 172.21.15.39:0/3263225432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 msgr2=0x560570bc32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdef400640 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bc32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdef400640 1 -- 172.21.15.39:0/3263225432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 msgr2=0x560570bb74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdef400640 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bb74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.409+0000 7efdef400640 1 -- 172.21.15.39:0/3263225432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560570aa7960 con 0x5605707e5b70 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.410+0000 7efdef400640 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bb7a10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efddc100290 tx=0x7efddc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144c98ff106f0d2f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.410+0000 7efdda000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.410+0000 7efdda000640 1 -- 172.21.15.39:0/3263225432 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc049070 con 0x5605707e5b70 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.410+0000 7efdda000640 1 -- 172.21.15.39:0/3263225432 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7efddc0d27f0 con 0x5605707e5b70 2024-07-04T08:17:09.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.410+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3263225432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efde8003dd0 con 0x5605707e5b70 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.410+0000 7efdda000640 1 -- 172.21.15.39:0/3263225432 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efddc100720 con 0x5605707e5b70 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.410+0000 7efdfd2ad740 10 client.15804 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.410+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3263225432 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560570bc62c0 con 0x5605707e5b70 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.411+0000 7efdda000640 1 -- 172.21.15.39:0/3263225432 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde8003dd0 con 0x5605707e5b70 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.411+0000 7efdda000640 1 -- 172.21.15.39:0/3263225432 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efddc0c9080 con 0x5605707e5b70 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.411+0000 7efdfd2ad740 10 client.15804 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.411+0000 7efdfd2ad740 10 client.15804 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.411+0000 7efdfd2ad740 20 client.15804 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.411+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5605707c51d0 0x560570bc9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.411+0000 7efdfd2ad740 4 client.15804 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.411+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3263225432 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560570b22b80 con 0x5605707c51d0 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.412+0000 7efdefe00640 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5605707c51d0 0x560570bc9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.412+0000 7efdefe00640 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5605707c51d0 0x560570bc9740 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368d1a78f9564d3b server_cookie=86036e95fe970e5e in_seq=0 out_seq=0 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.412+0000 7efdda000640 10 client.15804 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.413+0000 7efdda000640 1 -- 172.21.15.39:0/3263225432 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560570b22b80 con 0x5605707c51d0 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.413+0000 7efdda000640 10 client.15804 handle_command_reply: tid=0 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.413+0000 7efdfd2ad740 1 client.15804 shutdown 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3263225432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 msgr2=0x560570bb7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bb7a10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efddc100290 tx=0x7efddc031400 comp rx=0 tx=0).stop 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3263225432 shutdown_connections 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5605707c51d0 0x560570bc9740 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bb7a10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bc32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdda000640 0 client.15804 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:09.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3263225432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bb74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3263225432 >> 172.21.15.39:0/3263225432 conn(0x5605707fac40 msgr2=0x560570ba8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3263225432 shutdown_connections 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3263225432 wait complete. 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.414+0000 7efdfd2ad740 20 client.15804 trim_cache size 0 max 16384 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.434+0000 7efdfd2ad740 1 Processor -- start 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.434+0000 7efdfd2ad740 1 -- start start 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bc30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01b3d30 con 0x5605707e5b70 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01b28c0 con 0x5605707dd990 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01b2f90 con 0x560570806850 2024-07-04T08:17:09.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bc30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bc30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44958/0 (socket says 172.21.15.39:44958) 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdefe00640 1 -- 172.21.15.39:0/3071279246 learned_addr learned my addr 172.21.15.39:0/3071279246 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdf4c00640 1 --2- 172.21.15.39:0/3071279246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdefe00640 1 -- 172.21.15.39:0/3071279246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 msgr2=0x560570bceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdefe00640 1 --2- 172.21.15.39:0/3071279246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdefe00640 1 -- 172.21.15.39:0/3071279246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 msgr2=0x560570bad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdefe00640 1 --2- 172.21.15.39:0/3071279246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdefe00640 1 -- 172.21.15.39:0/3071279246 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560570bb8590 con 0x5605707dd990 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.435+0000 7efdf4c00640 1 --2- 172.21.15.39:0/3071279246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.436+0000 7efdefe00640 1 --2- 172.21.15.39:0/3071279246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bc30c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x560570bc9e40 tx=0x7efde001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9965cef3708895a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:09.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.436+0000 7efde4c00640 1 -- 172.21.15.39:0/3071279246 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0004030 con 0x5605707dd990 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.436+0000 7efde4c00640 1 -- 172.21.15.39:0/3071279246 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde0005020 con 0x5605707dd990 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.436+0000 7efde4c00640 1 -- 172.21.15.39:0/3071279246 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0004c00 con 0x5605707dd990 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.437+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3071279246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 msgr2=0x560570bc30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.437+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3071279246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bc30c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x560570bc9e40 tx=0x7efde001b000 comp rx=0 tx=0).stop 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.437+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3071279246 shutdown_connections 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.437+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3071279246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5605707e5b70 0x560570bad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.437+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3071279246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560570806850 0x560570bceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.437+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/3071279246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bc30c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.437+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3071279246 >> 172.21.15.39:0/3071279246 conn(0x5605707fac40 msgr2=0x560570bac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.437+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3071279246 shutdown_connections 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.437+0000 7efdfd2ad740 1 -- 172.21.15.39:0/3071279246 wait complete. 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.438+0000 7efdfd2ad740 1 -- start start 2024-07-04T08:17:09.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.438+0000 7efdfd2ad740 1 Processor -- start 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.438+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bbba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.438+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605707e5b70 0x560570bbbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.438+0000 7efdfd2ad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560570806850 0x560570c33680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.438+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01b01d0 con 0x560570806850 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.438+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01b3660 con 0x5605707dd990 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.438+0000 7efdfd2ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01b0840 con 0x5605707e5b70 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdfd2ad740 10 client.0 _ll_register_callbacks cb 0x7efddc009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdf4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560570806850 0x560570c33680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdefe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bbba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605707e5b70 0x560570bbbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdf4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560570806850 0x560570c33680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42994/0 (socket says 172.21.15.39:42994) 2024-07-04T08:17:09.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdf4c00640 1 -- 172.21.15.39:0/2057144919 learned_addr learned my addr 172.21.15.39:0/2057144919 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdef400640 1 -- 172.21.15.39:0/2057144919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 msgr2=0x560570bbba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdef400640 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bbba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdef400640 1 -- 172.21.15.39:0/2057144919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560570806850 msgr2=0x560570c33680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdef400640 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560570806850 0x560570c33680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.439+0000 7efdef400640 1 -- 172.21.15.39:0/2057144919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560570c33cf0 con 0x5605707e5b70 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.440+0000 7efdef400640 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605707e5b70 0x560570bbbff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efddc005ab0 tx=0x7efddc0d2a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d737b0fdb5c8b8d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.440+0000 7efdda000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.440+0000 7efdda000640 1 -- 172.21.15.39:0/2057144919 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc049040 con 0x5605707e5b70 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.440+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2057144919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efde0005020 con 0x5605707e5b70 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.440+0000 7efdda000640 1 -- 172.21.15.39:0/2057144919 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7efddc0318b0 con 0x5605707e5b70 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.440+0000 7efdda000640 1 -- 172.21.15.39:0/2057144919 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efddc0ac050 con 0x5605707e5b70 2024-07-04T08:17:09.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.440+0000 7efdfd2ad740 10 client.25894 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.440+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2057144919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560570c3b010 con 0x5605707e5b70 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.441+0000 7efdda000640 1 -- 172.21.15.39:0/2057144919 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde0005020 con 0x5605707e5b70 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.441+0000 7efdda000640 1 -- 172.21.15.39:0/2057144919 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efddc03bad0 con 0x5605707e5b70 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.441+0000 7efdfd2ad740 10 client.25894 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.441+0000 7efdfd2ad740 10 client.25894 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.441+0000 7efdfd2ad740 20 client.25894 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.441+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56057079a580 0x560570c3d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.441+0000 7efdfd2ad740 4 client.25894 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.441+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2057144919 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x560570b1e620 con 0x56057079a580 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.441+0000 7efdefe00640 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56057079a580 0x560570c3d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.442+0000 7efdefe00640 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56057079a580 0x560570c3d6b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927c168c576acbea server_cookie=6c3542ee08a6da09 in_seq=0 out_seq=0 2024-07-04T08:17:09.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.442+0000 7efdda000640 10 client.25894 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdda000640 1 -- 172.21.15.39:0/2057144919 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x560570b1e620 con 0x56057079a580 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdda000640 10 client.25894 handle_command_reply: tid=0 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 client.25894 shutdown 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2057144919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605707e5b70 msgr2=0x560570bbbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605707e5b70 0x560570bbbff0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efddc005ab0 tx=0x7efddc0d2a20 comp rx=0 tx=0).stop 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2057144919 shutdown_connections 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56057079a580 0x560570c3d6b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560570806850 0x560570c33680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605707e5b70 0x560570bbbff0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 --2- 172.21.15.39:0/2057144919 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5605707dd990 0x560570bbba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2057144919 >> 172.21.15.39:0/2057144919 conn(0x5605707fac40 msgr2=0x560570ba9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdda000640 0 client.25894 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:09.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.453+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2057144919 shutdown_connections 2024-07-04T08:17:09.504 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:17:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:17:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:17:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 6, 2024-07-04T08:17:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:17:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "ad947b9a": { 2024-07-04T08:17:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 6, 2024-07-04T08:17:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 142.8, 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:17:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.454+0000 7efdfd2ad740 1 -- 172.21.15.39:0/2057144919 wait complete. 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.454+0000 7efdfd2ad740 20 client.25894 trim_cache size 0 max 16384 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.454+0000 7efdf5600640 1 -- 172.21.15.39:0/511138990 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdd40b2950 msgr2=0x7efdd40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.454+0000 7efdf5600640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdd40b2950 0x7efdd40b4e60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x560570baa5d0 tx=0x7efde0005f90 comp rx=0 tx=0).stop 2024-07-04T08:17:09.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 -- 172.21.15.39:0/511138990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf010aa60 msgr2=0x7efdf019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:09.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf010aa60 0x7efdf019c000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efddc009230 tx=0x7efddc003b90 comp rx=0 tx=0).stop 2024-07-04T08:17:09.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 -- 172.21.15.39:0/511138990 shutdown_connections 2024-07-04T08:17:09.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efdd40b2950 0x7efdd40b4e60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdf010b3b0 0x7efdf01a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf010aa60 0x7efdf019c000 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 --2- 172.21.15.39:0/511138990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efdf0105120 0x7efdf019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:09.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 -- 172.21.15.39:0/511138990 >> 172.21.15.39:0/511138990 conn(0x7efdf0100b90 msgr2=0x7efdf0102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:09.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 -- 172.21.15.39:0/511138990 shutdown_connections 2024-07-04T08:17:09.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:08.455+0000 7efdf5600640 1 -- 172.21.15.39:0/511138990 wait complete. 2024-07-04T08:17:10.459 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ad947b9a', quiesce took 11.4 seconds. Will release after: 11.35 2024-07-04T08:17:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.731+0000 7fcfb4000640 1 -- 172.21.15.39:0/2866609535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac075650 msgr2=0x7fcfac10f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.731+0000 7fcfb4000640 1 --2- 172.21.15.39:0/2866609535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac075650 0x7fcfac10f0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009900 tx=0x7fcf9802a410 comp rx=0 tx=0).stop 2024-07-04T08:17:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.731+0000 7fcfb4000640 1 -- 172.21.15.39:0/2866609535 shutdown_connections 2024-07-04T08:17:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.731+0000 7fcfb4000640 1 --2- 172.21.15.39:0/2866609535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfac10f620 0x7fcfac111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.731+0000 7fcfb4000640 1 --2- 172.21.15.39:0/2866609535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac075650 0x7fcfac10f0e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.731+0000 7fcfb4000640 1 --2- 172.21.15.39:0/2866609535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfac074d00 0x7fcfac075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.731+0000 7fcfb4000640 1 -- 172.21.15.39:0/2866609535 >> 172.21.15.39:0/2866609535 conn(0x7fcfac0feb30 msgr2=0x7fcfac100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.731+0000 7fcfb4000640 1 -- 172.21.15.39:0/2866609535 shutdown_connections 2024-07-04T08:17:10.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.732+0000 7fcfb4000640 1 -- 172.21.15.39:0/2866609535 wait complete. 2024-07-04T08:17:10.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.733+0000 7fcfb4000640 1 Processor -- start 2024-07-04T08:17:10.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.733+0000 7fcfb4000640 1 -- start start 2024-07-04T08:17:10.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.733+0000 7fcfb4000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac074d00 0x7fcfac19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.733+0000 7fcfb4000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfac075650 0x7fcfac19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.733+0000 7fcfb4000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfac10f620 0x7fcfac1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.733+0000 7fcfb4000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1151c0 con 0x7fcfac10f620 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.733+0000 7fcfb4000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac115040 con 0x7fcfac075650 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.733+0000 7fcfb4000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac115340 con 0x7fcfac074d00 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb1600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac074d00 0x7fcfac19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfac075650 0x7fcfac19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb1600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac074d00 0x7fcfac19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33472/0 (socket says 172.21.15.39:33472) 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb1600640 1 -- 172.21.15.39:0/4272141343 learned_addr learned my addr 172.21.15.39:0/4272141343 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb2000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfac10f620 0x7fcfac1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb1600640 1 -- 172.21.15.39:0/4272141343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfac075650 msgr2=0x7fcfac19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb1600640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfac075650 0x7fcfac19c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb1600640 1 -- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfac10f620 msgr2=0x7fcfac1a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:10.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb1600640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfac10f620 0x7fcfac1a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb1600640 1 -- 172.21.15.39:0/4272141343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac1a1c20 con 0x7fcfac074d00 2024-07-04T08:17:10.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.734+0000 7fcfb2000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfac10f620 0x7fcfac1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:10.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.735+0000 7fcfb1600640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac074d00 0x7fcfac19bcb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9f0 tx=0x7fcfa000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:10.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.735+0000 7fcfaa000640 1 -- 172.21.15.39:0/4272141343 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000cd80 con 0x7fcfac074d00 2024-07-04T08:17:10.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.735+0000 7fcfaa000640 1 -- 172.21.15.39:0/4272141343 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa00054d0 con 0x7fcfac074d00 2024-07-04T08:17:10.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.735+0000 7fcfaa000640 1 -- 172.21.15.39:0/4272141343 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0010690 con 0x7fcfac074d00 2024-07-04T08:17:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.735+0000 7fcfb4000640 1 -- 172.21.15.39:0/4272141343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfac1a1f10 con 0x7fcfac074d00 2024-07-04T08:17:10.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.735+0000 7fcfb4000640 1 -- 172.21.15.39:0/4272141343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfac1a8670 con 0x7fcfac074d00 2024-07-04T08:17:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.739+0000 7fcfba5ad740 1 Processor -- start 2024-07-04T08:17:10.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.739+0000 7fcfba5ad740 1 -- start start 2024-07-04T08:17:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.739+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba1417070 0x556ba17bb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.740+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba13ee1b0 0x556ba17bbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.740+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17bc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.740+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556ba1731f00 con 0x556ba1417070 2024-07-04T08:17:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.740+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556ba17214a0 con 0x556ba13f6390 2024-07-04T08:17:10.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.740+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ba17254e0 con 0x556ba13ee1b0 2024-07-04T08:17:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.740+0000 7fcfb0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17bc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.740+0000 7fcfb0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17bc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44984/0 (socket says 172.21.15.39:44984) 2024-07-04T08:17:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.740+0000 7fcfb0c00640 1 -- 172.21.15.39:0/3400747431 learned_addr learned my addr 172.21.15.39:0/3400747431 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.741+0000 7fcfb0c00640 1 -- 172.21.15.39:0/3400747431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba13ee1b0 msgr2=0x556ba17bbd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:17:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.741+0000 7fcfb0c00640 1 --2- 172.21.15.39:0/3400747431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba13ee1b0 0x556ba17bbd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.741+0000 7fcfb0c00640 1 -- 172.21.15.39:0/3400747431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba1417070 msgr2=0x556ba17bb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.741+0000 7fcfb0c00640 1 --2- 172.21.15.39:0/3400747431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba1417070 0x556ba17bb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.741+0000 7fcfb0c00640 1 -- 172.21.15.39:0/3400747431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba16b9670 con 0x556ba13f6390 2024-07-04T08:17:10.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.741+0000 7fcfb0c00640 1 --2- 172.21.15.39:0/3400747431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17bc250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcfac06b000 tx=0x7fcf98032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7166942c45b33e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:10.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.741+0000 7fcf9ea00640 1 -- 172.21.15.39:0/3400747431 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98040070 con 0x556ba13f6390 2024-07-04T08:17:10.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.742+0000 7fcf9ea00640 1 -- 172.21.15.39:0/3400747431 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf98004030 con 0x556ba13f6390 2024-07-04T08:17:10.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.742+0000 7fcf9ea00640 1 -- 172.21.15.39:0/3400747431 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98035040 con 0x556ba13f6390 2024-07-04T08:17:10.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.742+0000 7fcfaa000640 1 -- 172.21.15.39:0/4272141343 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcfa0004e90 con 0x7fcfac074d00 2024-07-04T08:17:10.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.743+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3400747431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 msgr2=0x556ba17bc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:10.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.743+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3400747431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17bc250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcfac06b000 tx=0x7fcf98032b20 comp rx=0 tx=0).stop 2024-07-04T08:17:10.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.743+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3400747431 shutdown_connections 2024-07-04T08:17:10.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.743+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3400747431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17bc250 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.743+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3400747431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba13ee1b0 0x556ba17bbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.743+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3400747431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba1417070 0x556ba17bb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.743+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3400747431 >> 172.21.15.39:0/3400747431 conn(0x556ba140b460 msgr2=0x556ba17b96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.743+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3400747431 shutdown_connections 2024-07-04T08:17:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.743+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3400747431 wait complete. 2024-07-04T08:17:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.744+0000 7fcfba5ad740 1 -- start start 2024-07-04T08:17:10.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.744+0000 7fcfba5ad740 1 Processor -- start 2024-07-04T08:17:10.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.745+0000 7fcfaa000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf940b28f0 0x7fcf940b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.745+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 0x556ba17d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.745+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.745+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.745+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556ba1585700 con 0x556ba13ee1b0 2024-07-04T08:17:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.745+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556ba16c8bd0 con 0x556ba13f6390 2024-07-04T08:17:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.745+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ba17c9f50 con 0x556ba1417070 2024-07-04T08:17:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.745+0000 7fcfba5ad740 10 client.0 _ll_register_callbacks cb 0x556ba171a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.745+0000 7fcfaa000640 1 -- 172.21.15.39:0/4272141343 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fcfa00e2dd0 con 0x7fcfac074d00 2024-07-04T08:17:10.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.746+0000 7fcfb2000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf940b28f0 0x7fcf940b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.746+0000 7fcfb2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.746+0000 7fcfb2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33482/0 (socket says 172.21.15.39:33482) 2024-07-04T08:17:10.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.746+0000 7fcfb2000640 1 -- 172.21.15.39:0/2609195582 learned_addr learned my addr 172.21.15.39:0/2609195582 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.746+0000 7fcfb1600640 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 0x556ba17d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.746+0000 7fcfb0c00640 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.747+0000 7fcfb2000640 1 -- 172.21.15.39:0/2609195582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 msgr2=0x556ba17d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.747+0000 7fcfb2000640 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.747+0000 7fcfb2000640 1 -- 172.21.15.39:0/2609195582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 msgr2=0x556ba17d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.747+0000 7fcfb2000640 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 0x556ba17d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.747+0000 7fcfb2000640 1 -- 172.21.15.39:0/2609195582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba1482650 con 0x556ba1417070 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.747+0000 7fcfb2000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf940b28f0 0x7fcf940b4e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009fd0 tx=0x7fcfa4009dc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.747+0000 7fcfb2000640 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d58c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcfa401f580 tx=0x7fcfa4009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28aa80f40146cf8b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:10.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.747+0000 7fcf93400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:17:10.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.748+0000 7fcf93400640 1 -- 172.21.15.39:0/2609195582 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4012240 con 0x556ba1417070 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.748+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2609195582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf98004030 con 0x556ba1417070 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.748+0000 7fcf93400640 1 -- 172.21.15.39:0/2609195582 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fcfa4011440 con 0x556ba1417070 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.748+0000 7fcf93400640 1 -- 172.21.15.39:0/2609195582 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfa402e050 con 0x556ba1417070 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.748+0000 7fcfba5ad740 10 client.25903 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.748+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2609195582 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ba17de860 con 0x556ba1417070 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.749+0000 7fcf93400640 1 -- 172.21.15.39:0/2609195582 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf98004030 con 0x556ba1417070 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.749+0000 7fcf93400640 1 -- 172.21.15.39:0/2609195582 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcfa401f050 con 0x556ba1417070 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.749+0000 7fcfba5ad740 10 client.25903 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.749+0000 7fcfba5ad740 10 client.25903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.749+0000 7fcfba5ad740 20 client.25903 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.749+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ba13d59f0 0x556ba17e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.749+0000 7fcfba5ad740 4 client.25903 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:10.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.749+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2609195582 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ba1733e50 con 0x556ba13d59f0 2024-07-04T08:17:10.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.750+0000 7fcfb1600640 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ba13d59f0 0x556ba17e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.750+0000 7fcfb1600640 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ba13d59f0 0x556ba17e1e40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4567c54ba9f5f81d server_cookie=3194bcb57190231f in_seq=0 out_seq=0 2024-07-04T08:17:10.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.750+0000 7fcf93400640 10 client.25903 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:10.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcf93400640 1 -- 172.21.15.39:0/2609195582 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556ba1733e50 con 0x556ba13d59f0 2024-07-04T08:17:10.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcf93400640 10 client.25903 handle_command_reply: tid=0 2024-07-04T08:17:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcfba5ad740 1 client.25903 shutdown 2024-07-04T08:17:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2609195582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 msgr2=0x556ba17d58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d58c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcfa401f580 tx=0x7fcfa4009ad0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2609195582 shutdown_connections 2024-07-04T08:17:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ba13d59f0 0x556ba17e1e40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d58c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2609195582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 0x556ba17d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.751+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2609195582 >> 172.21.15.39:0/2609195582 conn(0x556ba140b460 msgr2=0x556ba17bc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.752+0000 7fcf93400640 0 client.25903 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.752+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2609195582 shutdown_connections 2024-07-04T08:17:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.752+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2609195582 wait complete. 2024-07-04T08:17:10.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.752+0000 7fcfba5ad740 20 client.25903 trim_cache size 0 max 16384 2024-07-04T08:17:10.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.767+0000 7fcfba5ad740 1 Processor -- start 2024-07-04T08:17:10.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.768+0000 7fcfba5ad740 1 -- start start 2024-07-04T08:17:10.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.768+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13ee1b0 0x556ba17c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.768+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13f6390 0x556ba17cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.768+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.768+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1ae0f0 con 0x556ba13f6390 2024-07-04T08:17:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.768+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1afc30 con 0x556ba13ee1b0 2024-07-04T08:17:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.768+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1ae7c0 con 0x556ba1417070 2024-07-04T08:17:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.768+0000 7fcfb0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13f6390 0x556ba17cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13ee1b0 0x556ba17c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13ee1b0 0x556ba17c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44996/0 (socket says 172.21.15.39:44996) 2024-07-04T08:17:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb1600640 1 -- 172.21.15.39:0/2200403167 learned_addr learned my addr 172.21.15.39:0/2200403167 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:10.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb2000640 1 --2- 172.21.15.39:0/2200403167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb1600640 1 -- 172.21.15.39:0/2200403167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 msgr2=0x556ba17d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb1600640 1 --2- 172.21.15.39:0/2200403167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb1600640 1 -- 172.21.15.39:0/2200403167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13f6390 msgr2=0x556ba17cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb1600640 1 --2- 172.21.15.39:0/2200403167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13f6390 0x556ba17cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb1600640 1 -- 172.21.15.39:0/2200403167 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba17d1380 con 0x556ba13ee1b0 2024-07-04T08:17:10.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.769+0000 7fcfb1600640 1 --2- 172.21.15.39:0/2200403167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13ee1b0 0x556ba17c92d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00a9a20 tx=0x7fcfa00cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0198e2d1b27758b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:10.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.770+0000 7fcf9e000640 1 -- 172.21.15.39:0/2200403167 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00c9070 con 0x556ba13ee1b0 2024-07-04T08:17:10.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.770+0000 7fcf9e000640 1 -- 172.21.15.39:0/2200403167 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa00aa420 con 0x556ba13ee1b0 2024-07-04T08:17:10.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.770+0000 7fcf9e000640 1 -- 172.21.15.39:0/2200403167 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00c0070 con 0x556ba13ee1b0 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2200403167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13ee1b0 msgr2=0x556ba17c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2200403167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13ee1b0 0x556ba17c92d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00a9a20 tx=0x7fcfa00cc040 comp rx=0 tx=0).stop 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2200403167 shutdown_connections 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2200403167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13f6390 0x556ba17cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2200403167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/2200403167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13ee1b0 0x556ba17c92d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2200403167 >> 172.21.15.39:0/2200403167 conn(0x556ba140b460 msgr2=0x556ba17d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2200403167 shutdown_connections 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 -- 172.21.15.39:0/2200403167 wait complete. 2024-07-04T08:17:10.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.771+0000 7fcfba5ad740 1 -- start start 2024-07-04T08:17:10.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.772+0000 7fcfba5ad740 1 Processor -- start 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.772+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 0x556ba17d5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.772+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.772+0000 7fcfba5ad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.772+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1af560 con 0x556ba13ee1b0 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.772+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1aee90 con 0x556ba13f6390 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.772+0000 7fcfba5ad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac1ac130 con 0x556ba1417070 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.772+0000 7fcfba5ad740 10 client.0 _ll_register_callbacks cb 0x7fcfa4012770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 0x556ba17d5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17d5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45008/0 (socket says 172.21.15.39:45008) 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb0c00640 1 -- 172.21.15.39:0/3587949655 learned_addr learned my addr 172.21.15.39:0/3587949655 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb2000640 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb1600640 1 -- 172.21.15.39:0/3587949655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 msgr2=0x556ba17cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb1600640 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb1600640 1 -- 172.21.15.39:0/3587949655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 msgr2=0x556ba17d5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb1600640 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17d5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.773+0000 7fcfb1600640 1 -- 172.21.15.39:0/3587949655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba17cd060 con 0x556ba13ee1b0 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.774+0000 7fcfb1600640 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 0x556ba17d5930 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00a99d0 tx=0x7fcfa00a9960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcf8f36744a0223 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.774+0000 7fcf93400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.774+0000 7fcf93400640 1 -- 172.21.15.39:0/3587949655 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00b7060 con 0x556ba13ee1b0 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.774+0000 7fcf93400640 1 -- 172.21.15.39:0/3587949655 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fcfa00e0410 con 0x556ba13ee1b0 2024-07-04T08:17:10.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.774+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3587949655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa00aa420 con 0x556ba13ee1b0 2024-07-04T08:17:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.774+0000 7fcf93400640 1 -- 172.21.15.39:0/3587949655 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfa00c9070 con 0x556ba13ee1b0 2024-07-04T08:17:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.774+0000 7fcfba5ad740 10 client.15825 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:10.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.774+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3587949655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ba17dc290 con 0x556ba13ee1b0 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.775+0000 7fcf93400640 1 -- 172.21.15.39:0/3587949655 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa00aa420 con 0x556ba13ee1b0 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.775+0000 7fcf93400640 1 -- 172.21.15.39:0/3587949655 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcfa0091bb0 con 0x556ba13ee1b0 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.775+0000 7fcfba5ad740 10 client.15825 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.775+0000 7fcfba5ad740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.775+0000 7fcfba5ad740 20 client.15825 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.775+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ba13aada0 0x556ba184c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.775+0000 7fcfba5ad740 4 client.15825 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.775+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3587949655 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ba172f8f0 con 0x556ba13aada0 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.775+0000 7fcfb0c00640 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ba13aada0 0x556ba184c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.776+0000 7fcfb0c00640 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ba13aada0 0x556ba184c5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c205adcd7d97bd69 server_cookie=71cac7d228dff5ab in_seq=0 out_seq=0 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.776+0000 7fcf93400640 10 client.15825 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:10.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.776+0000 7fcf93400640 1 -- 172.21.15.39:0/3587949655 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ba172f8f0 con 0x556ba13aada0 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.776+0000 7fcf93400640 10 client.15825 handle_command_reply: tid=0 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.776+0000 7fcfba5ad740 1 client.15825 shutdown 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3587949655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 msgr2=0x556ba17d5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 0x556ba17d5930 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00a99d0 tx=0x7fcfa00a9960 comp rx=0 tx=0).stop 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3587949655 shutdown_connections 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ba13aada0 0x556ba184c5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ba1417070 0x556ba17cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ba13f6390 0x556ba17d5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 --2- 172.21.15.39:0/3587949655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ba13ee1b0 0x556ba17d5930 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3587949655 >> 172.21.15.39:0/3587949655 conn(0x556ba140b460 msgr2=0x556ba17b9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcf93400640 0 client.15825 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:10.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3587949655 shutdown_connections 2024-07-04T08:17:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 1 -- 172.21.15.39:0/3587949655 wait complete. 2024-07-04T08:17:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.777+0000 7fcfba5ad740 20 client.15825 trim_cache size 0 max 16384 2024-07-04T08:17:10.781 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:17:10.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 -- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf940b28f0 msgr2=0x7fcf940b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf940b28f0 0x7fcf940b4e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009fd0 tx=0x7fcfa4009dc0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 -- 172.21.15.39:0/4272141343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac074d00 msgr2=0x7fcfac19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac074d00 0x7fcfac19bcb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e9f0 tx=0x7fcfa000edf0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 -- 172.21.15.39:0/4272141343 shutdown_connections 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf940b28f0 0x7fcf940b4e00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfac10f620 0x7fcfac1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcfac075650 0x7fcfac19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 --2- 172.21.15.39:0/4272141343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfac074d00 0x7fcfac19bcb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 -- 172.21.15.39:0/4272141343 >> 172.21.15.39:0/4272141343 conn(0x7fcfac0feb30 msgr2=0x7fcfac1091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 -- 172.21.15.39:0/4272141343 shutdown_connections 2024-07-04T08:17:10.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:10.778+0000 7fcfb4000640 1 -- 172.21.15.39:0/4272141343 wait complete. 2024-07-04T08:17:11.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:10 smithi039 ceph-mon[32062]: pgmap v539: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-07-04T08:17:11.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:10 smithi133 ceph-mon[37050]: pgmap v539: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-07-04T08:17:11.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:10 smithi050 ceph-mon[36577]: pgmap v539: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 28 op/s 2024-07-04T08:17:12.819 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:17:12.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:17:13.301 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:13.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:12 smithi133 ceph-mon[37050]: pgmap v540: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 29 op/s 2024-07-04T08:17:13.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:12 smithi039 ceph-mon[32062]: pgmap v540: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 29 op/s 2024-07-04T08:17:13.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:12 smithi050 ceph-mon[36577]: pgmap v540: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 29 op/s 2024-07-04T08:17:15.207 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:14 smithi039 ceph-mon[32062]: pgmap v541: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 29 op/s 2024-07-04T08:17:15.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:14 smithi133 ceph-mon[37050]: pgmap v541: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 29 op/s 2024-07-04T08:17:15.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:14 smithi050 ceph-mon[36577]: pgmap v541: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 29 op/s 2024-07-04T08:17:15.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.578+0000 7f91abe00640 1 -- 172.21.15.39:0/1464591036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91ac108820 msgr2=0x7f91ac108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:15.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.578+0000 7f91abe00640 1 --2- 172.21.15.39:0/1464591036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91ac108820 0x7f91ac108c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9194009900 tx=0x7f919402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.578+0000 7f91abe00640 1 -- 172.21.15.39:0/1464591036 shutdown_connections 2024-07-04T08:17:15.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.578+0000 7f91abe00640 1 --2- 172.21.15.39:0/1464591036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91ac108820 0x7f91ac108c30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.578+0000 7f91abe00640 1 --2- 172.21.15.39:0/1464591036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91ac109170 0x7f91ac10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.578+0000 7f91abe00640 1 --2- 172.21.15.39:0/1464591036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91ac102ee0 0x7f91ac1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.578+0000 7f91abe00640 1 -- 172.21.15.39:0/1464591036 >> 172.21.15.39:0/1464591036 conn(0x7f91ac0fe930 msgr2=0x7f91ac100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:15.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.578+0000 7f91abe00640 1 -- 172.21.15.39:0/1464591036 shutdown_connections 2024-07-04T08:17:15.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.578+0000 7f91abe00640 1 -- 172.21.15.39:0/1464591036 wait complete. 2024-07-04T08:17:15.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.579+0000 7f91abe00640 1 Processor -- start 2024-07-04T08:17:15.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.580+0000 7f91abe00640 1 -- start start 2024-07-04T08:17:15.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.580+0000 7f91abe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91ac102ee0 0x7f91ac074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.580+0000 7f91abe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91ac108820 0x7f91ac0751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.580+0000 7f91abe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91ac109170 0x7f91ac0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.580+0000 7f91abe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac112e30 con 0x7f91ac109170 2024-07-04T08:17:15.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.580+0000 7f91abe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac112cb0 con 0x7f91ac108820 2024-07-04T08:17:15.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.580+0000 7f91abe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac112fb0 con 0x7f91ac102ee0 2024-07-04T08:17:15.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91aaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91ac102ee0 0x7f91ac074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91ab400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91ac109170 0x7f91ac0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91aaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91ac102ee0 0x7f91ac074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:45512/0 (socket says 172.21.15.39:45512) 2024-07-04T08:17:15.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91aaa00640 1 -- 172.21.15.39:0/2570190723 learned_addr learned my addr 172.21.15.39:0/2570190723 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:15.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91a2000640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91ac108820 0x7f91ac0751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91aaa00640 1 -- 172.21.15.39:0/2570190723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91ac108820 msgr2=0x7f91ac0751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91aaa00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91ac108820 0x7f91ac0751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91aaa00640 1 -- 172.21.15.39:0/2570190723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91ac109170 msgr2=0x7f91ac0756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91aaa00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91ac109170 0x7f91ac0756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91aaa00640 1 -- 172.21.15.39:0/2570190723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac1a7b40 con 0x7f91ac102ee0 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.581+0000 7f91aaa00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91ac102ee0 0x7f91ac074c60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f919c00ea60 tx=0x7f919c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.582+0000 7f91a3e00640 1 -- 172.21.15.39:0/2570190723 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c00cd20 con 0x7f91ac102ee0 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.582+0000 7f91abe00640 1 -- 172.21.15.39:0/2570190723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ac1a7e30 con 0x7f91ac102ee0 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.582+0000 7f91a3e00640 1 -- 172.21.15.39:0/2570190723 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f919c0035f0 con 0x7f91ac102ee0 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.582+0000 7f91a3e00640 1 -- 172.21.15.39:0/2570190723 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c010690 con 0x7f91ac102ee0 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.582+0000 7f91abe00640 1 -- 172.21.15.39:0/2570190723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ac1a8290 con 0x7f91ac102ee0 2024-07-04T08:17:15.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.585+0000 7f91b7c81740 1 Processor -- start 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.586+0000 7f91b7c81740 1 -- start start 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.586+0000 7f91a3e00640 1 -- 172.21.15.39:0/2570190723 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f919c0037a0 con 0x7f91ac102ee0 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.586+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d25096350 0x563d2543a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.586+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d2506d490 0x563d2543ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.586+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d2543b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.586+0000 7f91b7c81740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563d253648f0 con 0x563d25096350 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.586+0000 7f91b7c81740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563d253b0ed0 con 0x563d25075670 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.586+0000 7f91b7c81740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563d253a0470 con 0x563d2506d490 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91ab400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d25096350 0x563d2543a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91ab400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d25096350 0x563d2543a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53872/0 (socket says 172.21.15.39:53872) 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91ab400640 1 -- 172.21.15.39:0/683205273 learned_addr learned my addr 172.21.15.39:0/683205273 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:15.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91a2000640 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d2543b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91aaa00640 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d2506d490 0x563d2543ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91a2000640 1 -- 172.21.15.39:0/683205273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d2506d490 msgr2=0x563d2543ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91a2000640 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d2506d490 0x563d2543ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91a2000640 1 -- 172.21.15.39:0/683205273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d25096350 msgr2=0x563d2543a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91a2000640 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d25096350 0x563d2543a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91a2000640 1 -- 172.21.15.39:0/683205273 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d25338290 con 0x563d25075670 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91ab400640 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d25096350 0x563d2543a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.587+0000 7f91aaa00640 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d2506d490 0x563d2543ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.588+0000 7f91a2000640 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d2543b280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f919402a910 tx=0x7f9194000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58366a5cd829b6f7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.588+0000 7f919be00640 1 -- 172.21.15.39:0/683205273 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194040070 con 0x563d25075670 2024-07-04T08:17:15.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.588+0000 7f919be00640 1 -- 172.21.15.39:0/683205273 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9194004030 con 0x563d25075670 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.588+0000 7f919be00640 1 -- 172.21.15.39:0/683205273 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194035040 con 0x563d25075670 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.588+0000 7f91a3e00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91900b2960 0x7f91900b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.589+0000 7f91a3e00640 1 -- 172.21.15.39:0/2570190723 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f919c0e2bd0 con 0x7f91ac102ee0 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.589+0000 7f91ab400640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91900b2960 0x7f91900b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.589+0000 7f91b7c81740 1 -- 172.21.15.39:0/683205273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 msgr2=0x563d2543b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.589+0000 7f91b7c81740 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d2543b280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f919402a910 tx=0x7f9194000bc0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.589+0000 7f91b7c81740 1 -- 172.21.15.39:0/683205273 shutdown_connections 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.589+0000 7f91b7c81740 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d2543b280 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.589+0000 7f91b7c81740 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d2506d490 0x563d2543ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.589+0000 7f91b7c81740 1 --2- 172.21.15.39:0/683205273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d25096350 0x563d2543a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.589+0000 7f91b7c81740 1 -- 172.21.15.39:0/683205273 >> 172.21.15.39:0/683205273 conn(0x563d2508a740 msgr2=0x563d254386f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.590+0000 7f91b7c81740 1 -- 172.21.15.39:0/683205273 shutdown_connections 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.590+0000 7f91ab400640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91900b2960 0x7f91900b4e70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91a4012840 tx=0x7f91a4009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.590+0000 7f91b7c81740 1 -- 172.21.15.39:0/683205273 wait complete. 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.590+0000 7f91b7c81740 1 -- start start 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.590+0000 7f91b7c81740 1 Processor -- start 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.591+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d25446380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.591+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d254468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.591+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d254520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.591+0000 7f91b7c81740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563d25347770 con 0x563d2506d490 2024-07-04T08:17:15.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.591+0000 7f91b7c81740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563d253a44b0 con 0x563d25075670 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.591+0000 7f91b7c81740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563d25204890 con 0x563d25096350 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.591+0000 7f91b7c81740 10 client.0 _ll_register_callbacks cb 0x563d253482c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d25446380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91a2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d254468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d25446380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53884/0 (socket says 172.21.15.39:53884) 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91aaa00640 1 -- 172.21.15.39:0/1401198778 learned_addr learned my addr 172.21.15.39:0/1401198778 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91ab400640 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d254520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91a2000640 1 -- 172.21.15.39:0/1401198778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 msgr2=0x563d254520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91a2000640 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d254520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91a2000640 1 -- 172.21.15.39:0/1401198778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 msgr2=0x563d25446380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91a2000640 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d25446380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91a2000640 1 -- 172.21.15.39:0/1401198778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d25101930 con 0x563d25075670 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91ab400640 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d254520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91aaa00640 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d25446380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f91a2000640 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d254468c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f919402a910 tx=0x7f9194032f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc9848eff15bba9c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.592+0000 7f9198c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.593+0000 7f9198c00640 1 -- 172.21.15.39:0/1401198778 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194040070 con 0x563d25075670 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.593+0000 7f91b7c81740 1 -- 172.21.15.39:0/1401198778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9194004030 con 0x563d25075670 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.593+0000 7f9198c00640 1 -- 172.21.15.39:0/1401198778 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f91940316a0 con 0x563d25075670 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.593+0000 7f9198c00640 1 -- 172.21.15.39:0/1401198778 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9194056050 con 0x563d25075670 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.593+0000 7f91b7c81740 10 client.25949 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:15.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.593+0000 7f91b7c81740 1 -- 172.21.15.39:0/1401198778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d25454d50 con 0x563d25075670 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.594+0000 7f9198c00640 1 -- 172.21.15.39:0/1401198778 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9194004030 con 0x563d25075670 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.594+0000 7f9198c00640 1 -- 172.21.15.39:0/1401198778 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9194004cb0 con 0x563d25075670 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.594+0000 7f91b7c81740 10 client.25949 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.594+0000 7f91b7c81740 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.594+0000 7f91b7c81740 20 client.25949 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.594+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563d25054cd0 0x563d254580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.594+0000 7f91b7c81740 4 client.25949 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.594+0000 7f91b7c81740 1 -- 172.21.15.39:0/1401198778 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d253b2e20 con 0x563d25054cd0 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.595+0000 7f91aaa00640 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563d25054cd0 0x563d254580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.595+0000 7f91aaa00640 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563d25054cd0 0x563d254580b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7307faa55592e294 server_cookie=6c1ece53047dd424 in_seq=0 out_seq=0 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.595+0000 7f9198c00640 10 client.25949 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:15.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f9198c00640 1 -- 172.21.15.39:0/1401198778 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563d253b2e20 con 0x563d25054cd0 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f9198c00640 10 client.25949 handle_command_reply: tid=0 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f91b7c81740 1 client.25949 shutdown 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f91b7c81740 1 -- 172.21.15.39:0/1401198778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 msgr2=0x563d254468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d254468c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f919402a910 tx=0x7f9194032f50 comp rx=0 tx=0).stop 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f91b7c81740 1 -- 172.21.15.39:0/1401198778 shutdown_connections 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563d25054cd0 0x563d254580b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d254520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d254468c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1401198778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d25446380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f9198c00640 0 client.25949 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.596+0000 7f91b7c81740 1 -- 172.21.15.39:0/1401198778 >> 172.21.15.39:0/1401198778 conn(0x563d2508a740 msgr2=0x563d2543d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.597+0000 7f91b7c81740 1 -- 172.21.15.39:0/1401198778 shutdown_connections 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.597+0000 7f91b7c81740 1 -- 172.21.15.39:0/1401198778 wait complete. 2024-07-04T08:17:15.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.597+0000 7f91b7c81740 20 client.25949 trim_cache size 0 max 16384 2024-07-04T08:17:15.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.617+0000 7f91b7c81740 1 Processor -- start 2024-07-04T08:17:15.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91b7c81740 1 -- start start 2024-07-04T08:17:15.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2543c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25448060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25437330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91b7c81740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1b2280 con 0x563d2506d490 2024-07-04T08:17:15.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91b7c81740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1b3dc0 con 0x563d25075670 2024-07-04T08:17:15.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91b7c81740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1b2950 con 0x563d25096350 2024-07-04T08:17:15.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2543c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2543c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53888/0 (socket says 172.21.15.39:53888) 2024-07-04T08:17:15.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91aaa00640 1 -- 172.21.15.39:0/1237691267 learned_addr learned my addr 172.21.15.39:0/1237691267 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:15.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.618+0000 7f91a2000640 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25448060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.619+0000 7f91ab400640 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25437330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.619+0000 7f91aaa00640 1 -- 172.21.15.39:0/1237691267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 msgr2=0x563d25437330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.619+0000 7f91aaa00640 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25437330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.619+0000 7f91aaa00640 1 -- 172.21.15.39:0/1237691267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 msgr2=0x563d25448060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.619+0000 7f91aaa00640 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25448060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.619+0000 7f91aaa00640 1 -- 172.21.15.39:0/1237691267 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2543caf0 con 0x563d2506d490 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.619+0000 7f91ab400640 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25437330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.619+0000 7f91a2000640 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25448060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.619+0000 7f91aaa00640 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2543c300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f919c009920 tx=0x7f919c0e50b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6019cc57fb07829c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.620+0000 7f919b400640 1 -- 172.21.15.39:0/1237691267 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c0b2320 con 0x563d2506d490 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.620+0000 7f919b400640 1 -- 172.21.15.39:0/1237691267 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f919c0a9690 con 0x563d2506d490 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.620+0000 7f919b400640 1 -- 172.21.15.39:0/1237691267 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c0a5e10 con 0x563d2506d490 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.620+0000 7f91b7c81740 1 -- 172.21.15.39:0/1237691267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 msgr2=0x563d2543c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:15.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.620+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2543c300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f919c009920 tx=0x7f919c0e50b0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.621+0000 7f91b7c81740 1 -- 172.21.15.39:0/1237691267 shutdown_connections 2024-07-04T08:17:15.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.621+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25437330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.621+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25448060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.621+0000 7f91b7c81740 1 --2- 172.21.15.39:0/1237691267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2543c300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.621+0000 7f91b7c81740 1 -- 172.21.15.39:0/1237691267 >> 172.21.15.39:0/1237691267 conn(0x563d2508a740 msgr2=0x563d25436c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:15.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.621+0000 7f91b7c81740 1 -- 172.21.15.39:0/1237691267 shutdown_connections 2024-07-04T08:17:15.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.621+0000 7f91b7c81740 1 -- 172.21.15.39:0/1237691267 wait complete. 2024-07-04T08:17:15.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.621+0000 7f91b7c81740 1 -- start start 2024-07-04T08:17:15.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.622+0000 7f91b7c81740 1 Processor -- start 2024-07-04T08:17:15.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.622+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2545b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.622+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25458be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.622+0000 7f91b7c81740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25459120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.622+0000 7f91b7c81740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1b36f0 con 0x563d2506d490 2024-07-04T08:17:15.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.622+0000 7f91b7c81740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1b3020 con 0x563d25075670 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.622+0000 7f91b7c81740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1b0260 con 0x563d25096350 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.622+0000 7f91b7c81740 10 client.0 _ll_register_callbacks cb 0x7f919402aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91ab400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25459120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91aaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2545b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91ab400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25459120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:45550/0 (socket says 172.21.15.39:45550) 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91ab400640 1 -- 172.21.15.39:0/2232065 learned_addr learned my addr 172.21.15.39:0/2232065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91a2000640 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25458be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91aaa00640 1 -- 172.21.15.39:0/2232065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 msgr2=0x563d25459120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91aaa00640 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25459120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91aaa00640 1 -- 172.21.15.39:0/2232065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 msgr2=0x563d25458be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91aaa00640 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25458be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91aaa00640 1 -- 172.21.15.39:0/2232065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d25459810 con 0x563d2506d490 2024-07-04T08:17:15.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91a2000640 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25458be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f91aaa00640 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2545b710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x563d25439410 tx=0x7f919c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87364085d8fd9f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.623+0000 7f9198c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:17:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.624+0000 7f9198c00640 1 -- 172.21.15.39:0/2232065 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c0b2320 con 0x563d2506d490 2024-07-04T08:17:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.624+0000 7f91b7c81740 1 -- 172.21.15.39:0/2232065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f919c0a9690 con 0x563d2506d490 2024-07-04T08:17:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.624+0000 7f9198c00640 1 -- 172.21.15.39:0/2232065 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f919c0a5e10 con 0x563d2506d490 2024-07-04T08:17:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.624+0000 7f9198c00640 1 -- 172.21.15.39:0/2232065 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f919c0adf30 con 0x563d2506d490 2024-07-04T08:17:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.624+0000 7f91b7c81740 10 client.15843 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.624+0000 7f91b7c81740 1 -- 172.21.15.39:0/2232065 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d254c0450 con 0x563d2506d490 2024-07-04T08:17:15.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.624+0000 7f9198c00640 1 -- 172.21.15.39:0/2232065 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f919c0a9690 con 0x563d2506d490 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.625+0000 7f9198c00640 1 -- 172.21.15.39:0/2232065 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f919c004000 con 0x563d2506d490 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.625+0000 7f91b7c81740 10 client.15843 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.625+0000 7f91b7c81740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.625+0000 7f91b7c81740 20 client.15843 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.625+0000 7f91b7c81740 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563d2502a080 0x563d254c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.625+0000 7f91b7c81740 4 client.15843 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.625+0000 7f91b7c81740 1 -- 172.21.15.39:0/2232065 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563d253ae8c0 con 0x563d2502a080 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.625+0000 7f91a2000640 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563d2502a080 0x563d254c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.626+0000 7f91a2000640 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563d2502a080 0x563d254c2a30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6dee21fea268605 server_cookie=b98f19405d4c19ac in_seq=0 out_seq=0 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.626+0000 7f9198c00640 10 client.15843 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f9198c00640 1 -- 172.21.15.39:0/2232065 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563d253ae8c0 con 0x563d2502a080 2024-07-04T08:17:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f9198c00640 10 client.15843 handle_command_reply: tid=0 2024-07-04T08:17:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f91b7c81740 1 client.15843 shutdown 2024-07-04T08:17:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f91b7c81740 1 -- 172.21.15.39:0/2232065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 msgr2=0x563d2545b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f91b7c81740 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2545b710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x563d25439410 tx=0x7f919c009310 comp rx=0 tx=0).stop 2024-07-04T08:17:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f91b7c81740 1 -- 172.21.15.39:0/2232065 shutdown_connections 2024-07-04T08:17:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f91b7c81740 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563d2502a080 0x563d254c2a30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f91b7c81740 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563d25096350 0x563d25459120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f91b7c81740 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563d25075670 0x563d25458be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f91b7c81740 1 --2- 172.21.15.39:0/2232065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563d2506d490 0x563d2545b710 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f9198c00640 0 client.15843 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.627+0000 7f91b7c81740 1 -- 172.21.15.39:0/2232065 >> 172.21.15.39:0/2232065 conn(0x563d2508a740 msgr2=0x563d25447dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.628+0000 7f91b7c81740 1 -- 172.21.15.39:0/2232065 shutdown_connections 2024-07-04T08:17:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.628+0000 7f91b7c81740 1 -- 172.21.15.39:0/2232065 wait complete. 2024-07-04T08:17:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.628+0000 7f91b7c81740 20 client.15843 trim_cache size 0 max 16384 2024-07-04T08:17:15.633 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:17:15.633 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:17:15.634 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "cb5e59ae-389c-4013-9629-03c75a493e05", 2024-07-04T08:17:15.634 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:17:15.634 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:17:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.628+0000 7f91abe00640 1 -- 172.21.15.39:0/2570190723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91900b2960 msgr2=0x7f91900b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.628+0000 7f91abe00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91900b2960 0x7f91900b4e70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91a4012840 tx=0x7f91a4009500 comp rx=0 tx=0).stop 2024-07-04T08:17:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.628+0000 7f91abe00640 1 -- 172.21.15.39:0/2570190723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91ac102ee0 msgr2=0x7f91ac074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.628+0000 7f91abe00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91ac102ee0 0x7f91ac074c60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f919c00ea60 tx=0x7f919c00ee60 comp rx=0 tx=0).stop 2024-07-04T08:17:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.629+0000 7f91abe00640 1 -- 172.21.15.39:0/2570190723 shutdown_connections 2024-07-04T08:17:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.629+0000 7f91abe00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f91900b2960 0x7f91900b4e70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.629+0000 7f91abe00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91ac109170 0x7f91ac0756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.629+0000 7f91abe00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91ac108820 0x7f91ac0751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.629+0000 7f91abe00640 1 --2- 172.21.15.39:0/2570190723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91ac102ee0 0x7f91ac074c60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.629+0000 7f91abe00640 1 -- 172.21.15.39:0/2570190723 >> 172.21.15.39:0/2570190723 conn(0x7f91ac0fe930 msgr2=0x7f91ac100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.629+0000 7f91abe00640 1 -- 172.21.15.39:0/2570190723 shutdown_connections 2024-07-04T08:17:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:15.629+0000 7f91abe00640 1 -- 172.21.15.39:0/2570190723 wait complete. 2024-07-04T08:17:16.515 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:17:17.003 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:17.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:16 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:17:17.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:16 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:17.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:16 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:17:17.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:16 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:17.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:16 smithi133 ceph-mon[37050]: pgmap v542: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 KiB/s wr, 27 op/s 2024-07-04T08:17:17.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:16 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:17:17.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:16 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:17.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:16 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:17:17.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:16 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:17.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:16 smithi039 ceph-mon[32062]: pgmap v542: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 KiB/s wr, 27 op/s 2024-07-04T08:17:17.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:16 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:17:17.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:16 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:17.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:16 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:17:17.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:16 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:17.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:16 smithi050 ceph-mon[36577]: pgmap v542: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 KiB/s wr, 27 op/s 2024-07-04T08:17:18.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:17 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:17:18.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:17 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:17:18.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:17 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:17:19.251 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:18 smithi039 ceph-mon[32062]: pgmap v543: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-07-04T08:17:19.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.248+0000 7eff78600640 1 -- 172.21.15.39:0/4068592719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff70109340 msgr2=0x7eff7010fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.248+0000 7eff78600640 1 --2- 172.21.15.39:0/4068592719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff70109340 0x7eff7010fb70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7eff68009ca0 tx=0x7eff6802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.249+0000 7eff78600640 1 -- 172.21.15.39:0/4068592719 shutdown_connections 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.249+0000 7eff78600640 1 --2- 172.21.15.39:0/4068592719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff70109340 0x7eff7010fb70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.249+0000 7eff78600640 1 --2- 172.21.15.39:0/4068592719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff701089f0 0x7eff70108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.249+0000 7eff78600640 1 --2- 172.21.15.39:0/4068592719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff701030b0 0x7eff701034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.249+0000 7eff78600640 1 -- 172.21.15.39:0/4068592719 >> 172.21.15.39:0/4068592719 conn(0x7eff7007ada0 msgr2=0x7eff7007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.249+0000 7eff78600640 1 -- 172.21.15.39:0/4068592719 shutdown_connections 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.249+0000 7eff78600640 1 -- 172.21.15.39:0/4068592719 wait complete. 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.250+0000 7eff78600640 1 Processor -- start 2024-07-04T08:17:19.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.250+0000 7eff78600640 1 -- start start 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff78600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff701030b0 0x7eff70199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff78600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff701089f0 0x7eff70199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff78600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff70109340 0x7eff7019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff78600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff70113040 con 0x7eff701089f0 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff78600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff70112ec0 con 0x7eff701030b0 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff78600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff701131c0 con 0x7eff70109340 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff75200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff701089f0 0x7eff70199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff75c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff701030b0 0x7eff70199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff75200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff701089f0 0x7eff70199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53932/0 (socket says 172.21.15.39:53932) 2024-07-04T08:17:19.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff75200640 1 -- 172.21.15.39:0/2214321996 learned_addr learned my addr 172.21.15.39:0/2214321996 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.251+0000 7eff76600640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff70109340 0x7eff7019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.252+0000 7eff75c00640 1 -- 172.21.15.39:0/2214321996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff70109340 msgr2=0x7eff7019f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.252+0000 7eff75c00640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff70109340 0x7eff7019f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.252+0000 7eff75c00640 1 -- 172.21.15.39:0/2214321996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff701089f0 msgr2=0x7eff70199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.252+0000 7eff75c00640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff701089f0 0x7eff70199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.252+0000 7eff75c00640 1 -- 172.21.15.39:0/2214321996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7019f8e0 con 0x7eff701030b0 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.252+0000 7eff75c00640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff701030b0 0x7eff70199970 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9f0 tx=0x7eff6000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.252+0000 7eff6e000640 1 -- 172.21.15.39:0/2214321996 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6000ccc0 con 0x7eff701030b0 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.253+0000 7eff6e000640 1 -- 172.21.15.39:0/2214321996 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff600108e0 con 0x7eff701030b0 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.253+0000 7eff78600640 1 -- 172.21.15.39:0/2214321996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7019fbd0 con 0x7eff701030b0 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.253+0000 7eff6e000640 1 -- 172.21.15.39:0/2214321996 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60018880 con 0x7eff701030b0 2024-07-04T08:17:19.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.253+0000 7eff78600640 1 -- 172.21.15.39:0/2214321996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff701a6330 con 0x7eff701030b0 2024-07-04T08:17:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.256+0000 7eff7ecd6740 1 Processor -- start 2024-07-04T08:17:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.256+0000 7eff7ecd6740 1 -- start start 2024-07-04T08:17:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff6e000640 1 -- 172.21.15.39:0/2214321996 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7eff60010430 con 0x7eff701030b0 2024-07-04T08:17:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef034070 0x561cef3d8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef00b1b0 0x561cef3d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef013390 0x561cef3d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561cef34eea0 con 0x561cef00b1b0 2024-07-04T08:17:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561cef33e440 con 0x561cef013390 2024-07-04T08:17:19.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561cef342480 con 0x561cef034070 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff75c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef00b1b0 0x561cef3d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff75c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef00b1b0 0x561cef3d8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:53946/0 (socket says 172.21.15.39:53946) 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff75c00640 1 -- 172.21.15.39:0/296155084 learned_addr learned my addr 172.21.15.39:0/296155084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff76600640 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef034070 0x561cef3d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.257+0000 7eff75200640 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef013390 0x561cef3d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.258+0000 7eff76600640 1 -- 172.21.15.39:0/296155084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef013390 msgr2=0x561cef3d91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.258+0000 7eff76600640 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef013390 0x561cef3d91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.258+0000 7eff76600640 1 -- 172.21.15.39:0/296155084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef00b1b0 msgr2=0x561cef3d8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.258+0000 7eff76600640 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef00b1b0 0x561cef3d8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.258+0000 7eff76600640 1 -- 172.21.15.39:0/296155084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cef2d6610 con 0x561cef034070 2024-07-04T08:17:19.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.258+0000 7eff75200640 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef013390 0x561cef3d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.258+0000 7eff76600640 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef034070 0x561cef3d8770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff68005e30 tx=0x7eff68009430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90497eebd133d3c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.259+0000 7eff66a00640 1 -- 172.21.15.39:0/296155084 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68031cf0 con 0x561cef034070 2024-07-04T08:17:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.259+0000 7eff66a00640 1 -- 172.21.15.39:0/296155084 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff68036030 con 0x561cef034070 2024-07-04T08:17:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.259+0000 7eff66a00640 1 -- 172.21.15.39:0/296155084 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff680042a0 con 0x561cef034070 2024-07-04T08:17:19.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.259+0000 7eff6e000640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7eff540b2960 0x7eff540b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.259+0000 7eff6e000640 1 -- 172.21.15.39:0/2214321996 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7eff600e2cd0 con 0x7eff701030b0 2024-07-04T08:17:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.259+0000 7eff75c00640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7eff540b2960 0x7eff540b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff7ecd6740 1 -- 172.21.15.39:0/296155084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef034070 msgr2=0x561cef3d8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef034070 0x561cef3d8770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff68005e30 tx=0x7eff68009430 comp rx=0 tx=0).stop 2024-07-04T08:17:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff7ecd6740 1 -- 172.21.15.39:0/296155084 shutdown_connections 2024-07-04T08:17:19.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef00b1b0 0x561cef3d8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef013390 0x561cef3d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/296155084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef034070 0x561cef3d8770 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff7ecd6740 1 -- 172.21.15.39:0/296155084 >> 172.21.15.39:0/296155084 conn(0x561cef028460 msgr2=0x561cef3d6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:19.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff75c00640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7eff540b2960 0x7eff540b4e70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7eff600a99d0 tx=0x7eff600e6780 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:19.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff7ecd6740 1 -- 172.21.15.39:0/296155084 shutdown_connections 2024-07-04T08:17:19.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.260+0000 7eff7ecd6740 1 -- 172.21.15.39:0/296155084 wait complete. 2024-07-04T08:17:19.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.261+0000 7eff7ecd6740 1 -- start start 2024-07-04T08:17:19.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.261+0000 7eff7ecd6740 1 Processor -- start 2024-07-04T08:17:19.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 0x561cef3e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 0x561cef3f4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561cef1a2700 con 0x561cef034070 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561cef2e5b70 con 0x561cef00b1b0 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561cef3e6d70 con 0x561cef013390 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff7ecd6740 10 client.0 _ll_register_callbacks cb 0x561cef337530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff75200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff76600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 0x561cef3e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff75200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3e47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:45604/0 (socket says 172.21.15.39:45604) 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff75200640 1 -- 172.21.15.39:0/1857722040 learned_addr learned my addr 172.21.15.39:0/1857722040 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.262+0000 7eff76600640 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 0x561cef3f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff75200640 1 -- 172.21.15.39:0/1857722040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 msgr2=0x561cef3e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff75200640 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 0x561cef3e42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff75200640 1 -- 172.21.15.39:0/1857722040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 msgr2=0x561cef3f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff75200640 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 0x561cef3f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff75200640 1 -- 172.21.15.39:0/1857722040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cef09f650 con 0x561cef013390 2024-07-04T08:17:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff75200640 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3e47f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0127d0 tx=0x7eff5c012c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c9a5cf9e23485e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff5b400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:17:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff5b400640 1 -- 172.21.15.39:0/1857722040 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004030 con 0x561cef013390 2024-07-04T08:17:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff5b400640 1 -- 172.21.15.39:0/1857722040 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7eff5c0041d0 con 0x561cef013390 2024-07-04T08:17:19.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.263+0000 7eff7ecd6740 1 -- 172.21.15.39:0/1857722040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff68036030 con 0x561cef013390 2024-07-04T08:17:19.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.264+0000 7eff5b400640 1 -- 172.21.15.39:0/1857722040 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff5c025050 con 0x561cef013390 2024-07-04T08:17:19.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.264+0000 7eff7ecd6740 10 client.25933 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:19.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.264+0000 7eff7ecd6740 1 -- 172.21.15.39:0/1857722040 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561cef3f7170 con 0x561cef013390 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.264+0000 7eff5b400640 1 -- 172.21.15.39:0/1857722040 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff68036030 con 0x561cef013390 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.265+0000 7eff5b400640 1 -- 172.21.15.39:0/1857722040 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7eff5c0039b0 con 0x561cef013390 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.265+0000 7eff7ecd6740 10 client.25933 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.265+0000 7eff7ecd6740 10 client.25933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.265+0000 7eff7ecd6740 20 client.25933 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.265+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561ceeff29f0 0x561cef3fa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.265+0000 7eff7ecd6740 4 client.25933 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.265+0000 7eff7ecd6740 1 -- 172.21.15.39:0/1857722040 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cef350df0 con 0x561ceeff29f0 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.265+0000 7eff76600640 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561ceeff29f0 0x561cef3fa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.266+0000 7eff76600640 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561ceeff29f0 0x561cef3fa710 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d02e1d9dc92867 server_cookie=bb7389284e982cad in_seq=0 out_seq=0 2024-07-04T08:17:19.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.266+0000 7eff5b400640 10 client.25933 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:19.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff5b400640 1 -- 172.21.15.39:0/1857722040 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561cef350df0 con 0x561ceeff29f0 2024-07-04T08:17:19.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff5b400640 10 client.25933 handle_command_reply: tid=0 2024-07-04T08:17:19.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 client.25933 shutdown 2024-07-04T08:17:19.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 -- 172.21.15.39:0/1857722040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 msgr2=0x561cef3e47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:19.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3e47f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0127d0 tx=0x7eff5c012c90 comp rx=0 tx=0).stop 2024-07-04T08:17:19.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 -- 172.21.15.39:0/1857722040 shutdown_connections 2024-07-04T08:17:19.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561ceeff29f0 0x561cef3fa710 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 0x561cef3f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3e47f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/1857722040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 0x561cef3e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff5b400640 0 client.25933 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 -- 172.21.15.39:0/1857722040 >> 172.21.15.39:0/1857722040 conn(0x561cef028460 msgr2=0x561cef3db950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.267+0000 7eff7ecd6740 1 -- 172.21.15.39:0/1857722040 shutdown_connections 2024-07-04T08:17:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.268+0000 7eff7ecd6740 1 -- 172.21.15.39:0/1857722040 wait complete. 2024-07-04T08:17:19.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.268+0000 7eff7ecd6740 20 client.25933 trim_cache size 0 max 16384 2024-07-04T08:17:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.288+0000 7eff7ecd6740 1 Processor -- start 2024-07-04T08:17:19.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.288+0000 7eff7ecd6740 1 -- start start 2024-07-04T08:17:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef00b1b0 0x561cef3d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef013390 0x561cef3d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef034070 0x561cef3e4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff701adf90 con 0x561cef013390 2024-07-04T08:17:19.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff701afad0 con 0x561cef034070 2024-07-04T08:17:19.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff701ae660 con 0x561cef00b1b0 2024-07-04T08:17:19.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff76600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef00b1b0 0x561cef3d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff75200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef013390 0x561cef3d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff76600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef00b1b0 0x561cef3d5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:45618/0 (socket says 172.21.15.39:45618) 2024-07-04T08:17:19.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff76600640 1 -- 172.21.15.39:0/2682066576 learned_addr learned my addr 172.21.15.39:0/2682066576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:19.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.289+0000 7eff76600640 1 --2- 172.21.15.39:0/2682066576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef034070 0x561cef3e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.290+0000 7eff75200640 1 -- 172.21.15.39:0/2682066576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef00b1b0 msgr2=0x561cef3d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.290+0000 7eff75200640 1 --2- 172.21.15.39:0/2682066576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef00b1b0 0x561cef3d5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.290+0000 7eff75200640 1 -- 172.21.15.39:0/2682066576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef034070 msgr2=0x561cef3e4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.290+0000 7eff75200640 1 --2- 172.21.15.39:0/2682066576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef034070 0x561cef3e4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.290+0000 7eff75200640 1 -- 172.21.15.39:0/2682066576 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cef3e3010 con 0x561cef013390 2024-07-04T08:17:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.290+0000 7eff75200640 1 --2- 172.21.15.39:0/2682066576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef013390 0x561cef3d6690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0059f0 tx=0x7eff5c0124b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7a3f74c7d0f16a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.290+0000 7eff66000640 1 -- 172.21.15.39:0/2682066576 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c022040 con 0x561cef013390 2024-07-04T08:17:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.290+0000 7eff66000640 1 -- 172.21.15.39:0/2682066576 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5c0105f0 con 0x561cef013390 2024-07-04T08:17:19.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.290+0000 7eff66000640 1 -- 172.21.15.39:0/2682066576 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c003850 con 0x561cef013390 2024-07-04T08:17:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.291+0000 7eff7ecd6740 1 -- 172.21.15.39:0/2682066576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef013390 msgr2=0x561cef3d6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.291+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/2682066576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef013390 0x561cef3d6690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0059f0 tx=0x7eff5c0124b0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.291+0000 7eff7ecd6740 1 -- 172.21.15.39:0/2682066576 shutdown_connections 2024-07-04T08:17:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.291+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/2682066576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef013390 0x561cef3d6690 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.291+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/2682066576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef034070 0x561cef3e4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.291+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/2682066576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef00b1b0 0x561cef3d5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.291+0000 7eff7ecd6740 1 -- 172.21.15.39:0/2682066576 >> 172.21.15.39:0/2682066576 conn(0x561cef028460 msgr2=0x561cef3d5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:19.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.291+0000 7eff7ecd6740 1 -- 172.21.15.39:0/2682066576 shutdown_connections 2024-07-04T08:17:19.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.291+0000 7eff7ecd6740 1 -- 172.21.15.39:0/2682066576 wait complete. 2024-07-04T08:17:19.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.292+0000 7eff7ecd6740 1 -- start start 2024-07-04T08:17:19.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.292+0000 7eff7ecd6740 1 Processor -- start 2024-07-04T08:17:19.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 0x561cef3f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff7ecd6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 0x561cef3eb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff701af400 con 0x561cef034070 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff701aed30 con 0x561cef00b1b0 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff7ecd6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff701abf70 con 0x561cef013390 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff7ecd6740 10 client.0 _ll_register_callbacks cb 0x7eff5c0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff76600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 0x561cef3f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff76600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 0x561cef3f0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:40350/0 (socket says 172.21.15.39:40350) 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff75200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff76600640 1 -- 172.21.15.39:0/3267751017 learned_addr learned my addr 172.21.15.39:0/3267751017 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff76600640 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 0x561cef3eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff75200640 1 -- 172.21.15.39:0/3267751017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 msgr2=0x561cef3f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff75200640 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 0x561cef3f0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff75200640 1 -- 172.21.15.39:0/3267751017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 msgr2=0x561cef3eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:19.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff75200640 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 0x561cef3eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.293+0000 7eff75200640 1 -- 172.21.15.39:0/3267751017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cef400950 con 0x561cef013390 2024-07-04T08:17:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.294+0000 7eff75200640 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3f09a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x561cef07d520 tx=0x7eff5c010a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de1948182336d4e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.294+0000 7eff5b400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:17:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.294+0000 7eff5b400640 1 -- 172.21.15.39:0/3267751017 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c010e50 con 0x561cef013390 2024-07-04T08:17:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.294+0000 7eff5b400640 1 -- 172.21.15.39:0/3267751017 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7eff5c00fb30 con 0x561cef013390 2024-07-04T08:17:19.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.294+0000 7eff7ecd6740 1 -- 172.21.15.39:0/3267751017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5c0105f0 con 0x561cef013390 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff5b400640 1 -- 172.21.15.39:0/3267751017 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff5c037050 con 0x561cef013390 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff7ecd6740 10 client.25939 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff7ecd6740 1 -- 172.21.15.39:0/3267751017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561cef463250 con 0x561cef013390 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff5b400640 1 -- 172.21.15.39:0/3267751017 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5c0105f0 con 0x561cef013390 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff5b400640 1 -- 172.21.15.39:0/3267751017 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7eff5c005110 con 0x561cef013390 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff7ecd6740 10 client.25939 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff7ecd6740 10 client.25939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff7ecd6740 20 client.25939 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561ceefc7da0 0x561cef465890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff7ecd6740 4 client.25939 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.295+0000 7eff7ecd6740 1 -- 172.21.15.39:0/3267751017 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561cef34c890 con 0x561ceefc7da0 2024-07-04T08:17:19.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.296+0000 7eff76600640 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561ceefc7da0 0x561cef465890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.296+0000 7eff76600640 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561ceefc7da0 0x561cef465890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32fa977077ef0f9c server_cookie=d18c4fe35f73e9b2 in_seq=0 out_seq=0 2024-07-04T08:17:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.296+0000 7eff5b400640 10 client.25939 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.296+0000 7eff5b400640 1 -- 172.21.15.39:0/3267751017 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561cef34c890 con 0x561ceefc7da0 2024-07-04T08:17:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.296+0000 7eff5b400640 10 client.25939 handle_command_reply: tid=0 2024-07-04T08:17:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.296+0000 7eff7ecd6740 1 client.25939 shutdown 2024-07-04T08:17:19.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 -- 172.21.15.39:0/3267751017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 msgr2=0x561cef3f09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3f09a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x561cef07d520 tx=0x7eff5c010a90 comp rx=0 tx=0).stop 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 -- 172.21.15.39:0/3267751017 shutdown_connections 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561ceefc7da0 0x561cef465890 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561cef034070 0x561cef3eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561cef013390 0x561cef3f09a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 --2- 172.21.15.39:0/3267751017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561cef00b1b0 0x561cef3f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff5b400640 0 client.25939 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 -- 172.21.15.39:0/3267751017 >> 172.21.15.39:0/3267751017 conn(0x561cef028460 msgr2=0x561cef3d97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 -- 172.21.15.39:0/3267751017 shutdown_connections 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 1 -- 172.21.15.39:0/3267751017 wait complete. 2024-07-04T08:17:19.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.297+0000 7eff7ecd6740 20 client.25939 trim_cache size 0 max 16384 2024-07-04T08:17:19.301 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:17:19.301 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:17:19.301 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:17:19.301 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 -- 172.21.15.39:0/2214321996 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7eff540b2960 msgr2=0x7eff540b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7eff540b2960 0x7eff540b4e70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7eff600a99d0 tx=0x7eff600e6780 comp rx=0 tx=0).stop 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 -- 172.21.15.39:0/2214321996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff701030b0 msgr2=0x7eff70199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff701030b0 0x7eff70199970 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff6000e9f0 tx=0x7eff6000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 -- 172.21.15.39:0/2214321996 shutdown_connections 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7eff540b2960 0x7eff540b4e70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff701089f0 0x7eff70199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff70109340 0x7eff7019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 --2- 172.21.15.39:0/2214321996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff701030b0 0x7eff70199970 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 -- 172.21.15.39:0/2214321996 >> 172.21.15.39:0/2214321996 conn(0x7eff7007ada0 msgr2=0x7eff701069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:19.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 -- 172.21.15.39:0/2214321996 shutdown_connections 2024-07-04T08:17:19.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:19.298+0000 7eff78600640 1 -- 172.21.15.39:0/2214321996 wait complete. 2024-07-04T08:17:19.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:18 smithi133 ceph-mon[37050]: pgmap v543: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-07-04T08:17:19.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:18 smithi050 ceph-mon[36577]: pgmap v543: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-07-04T08:17:20.237 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:17:20.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:17:20.723 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:21.224 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:20 smithi039 ceph-mon[32062]: pgmap v544: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:17:21.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:20 smithi133 ceph-mon[37050]: pgmap v544: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:17:21.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:20 smithi050 ceph-mon[36577]: pgmap v544: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:17:21.810 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:17:22.288 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:23.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:22 smithi133 ceph-mon[37050]: pgmap v545: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:17:23.352 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:22 smithi039 ceph-mon[32062]: pgmap v545: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:17:23.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:22 smithi050 ceph-mon[36577]: pgmap v545: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:17:23.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.166+0000 7f5cf5400640 1 -- 172.21.15.39:0/1678300304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf010d170 msgr2=0x7f5cf010f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:23.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.166+0000 7f5cf5400640 1 --2- 172.21.15.39:0/1678300304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf010d170 0x7f5cf010f660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009900 tx=0x7f5ce802a410 comp rx=0 tx=0).stop 2024-07-04T08:17:23.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.166+0000 7f5cf5400640 1 -- 172.21.15.39:0/1678300304 shutdown_connections 2024-07-04T08:17:23.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.166+0000 7f5cf5400640 1 --2- 172.21.15.39:0/1678300304 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf010d170 0x7f5cf010f660 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.166+0000 7f5cf5400640 1 --2- 172.21.15.39:0/1678300304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf0104660 0x7f5cf010cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.166+0000 7f5cf5400640 1 --2- 172.21.15.39:0/1678300304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cf0103d10 0x7f5cf0104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.166+0000 7f5cf5400640 1 -- 172.21.15.39:0/1678300304 >> 172.21.15.39:0/1678300304 conn(0x7f5cf00fd810 msgr2=0x7f5cf00ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:23.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.166+0000 7f5cf5400640 1 -- 172.21.15.39:0/1678300304 shutdown_connections 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.166+0000 7f5cf5400640 1 -- 172.21.15.39:0/1678300304 wait complete. 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.167+0000 7f5cf5400640 1 Processor -- start 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.168+0000 7f5cf5400640 1 -- start start 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.168+0000 7f5cf5400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cf0103d10 0x7f5cf019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.168+0000 7f5cf5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf0104660 0x7f5cf019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.168+0000 7f5cf5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf010d170 0x7f5cf01a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.168+0000 7f5cf5400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0112df0 con 0x7f5cf0104660 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.168+0000 7f5cf5400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0112c70 con 0x7f5cf010d170 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.168+0000 7f5cf5400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0112f70 con 0x7f5cf0103d10 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf0104660 0x7f5cf019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf0104660 0x7f5cf019bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47340/0 (socket says 172.21.15.39:47340) 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cef400640 1 -- 172.21.15.39:0/881134616 learned_addr learned my addr 172.21.15.39:0/881134616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:23.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cefe00640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cf0103d10 0x7f5cf019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:23.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cf4a00640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf010d170 0x7f5cf01a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:23.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cef400640 1 -- 172.21.15.39:0/881134616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cf0103d10 msgr2=0x7f5cf019ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:23.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cef400640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cf0103d10 0x7f5cf019ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.473 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:17:23.473 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cef400640 1 -- 172.21.15.39:0/881134616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf010d170 msgr2=0x7f5cf01a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cef400640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf010d170 0x7f5cf01a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.169+0000 7f5cef400640 1 -- 172.21.15.39:0/881134616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf01a19b0 con 0x7f5cf0104660 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.170+0000 7f5cef400640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf0104660 0x7f5cf019bfe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9f0 tx=0x7f5cd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.170+0000 7f5cecc00640 1 -- 172.21.15.39:0/881134616 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd800ccc0 con 0x7f5cf0104660 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.170+0000 7f5cecc00640 1 -- 172.21.15.39:0/881134616 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cd80108e0 con 0x7f5cf0104660 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.170+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf01a1ca0 con 0x7f5cf0104660 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.170+0000 7f5cecc00640 1 -- 172.21.15.39:0/881134616 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8018860 con 0x7f5cf0104660 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.170+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf01a8400 con 0x7f5cf0104660 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.171+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf0104120 con 0x7f5cf0104660 2024-07-04T08:17:23.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.175+0000 7f5cecc00640 1 -- 172.21.15.39:0/881134616 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5cd8010430 con 0x7f5cf0104660 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.176+0000 7f5cecc00640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5cd40b2960 0x7f5cd40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.177+0000 7f5cecc00640 1 -- 172.21.15.39:0/881134616 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f5cd80e30c0 con 0x7f5cf0104660 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.177+0000 7f5cefe00640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5cd40b2960 0x7f5cd40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.177+0000 7f5cecc00640 1 -- 172.21.15.39:0/881134616 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5cd80aa0c0 con 0x7f5cf0104660 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.178+0000 7f5cefe00640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5cd40b2960 0x7f5cd40b4e70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005fd0 tx=0x7f5ce0009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.435+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cf01a86e0 con 0x7f5cf0104660 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.437+0000 7f5cecc00640 1 -- 172.21.15.39:0/881134616 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f5cd80ae3c0 con 0x7f5cf0104660 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5cd40b2960 msgr2=0x7f5cd40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5cd40b2960 0x7f5cd40b4e70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005fd0 tx=0x7f5ce0009680 comp rx=0 tx=0).stop 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf0104660 msgr2=0x7f5cf019bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:23.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf0104660 0x7f5cf019bfe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800e9f0 tx=0x7f5cd800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 shutdown_connections 2024-07-04T08:17:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5cd40b2960 0x7f5cd40b4e70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cf0104660 0x7f5cf019bfe0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5cf010d170 0x7f5cf01a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 --2- 172.21.15.39:0/881134616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cf0103d10 0x7f5cf019ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 >> 172.21.15.39:0/881134616 conn(0x7f5cf00fd810 msgr2=0x7f5cf010c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 shutdown_connections 2024-07-04T08:17:23.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:23.440+0000 7f5cf5400640 1 -- 172.21.15.39:0/881134616 wait complete. 2024-07-04T08:17:24.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:23 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/881134616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:17:24.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:23 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/881134616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:17:24.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:23 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/881134616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:17:25.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:25 smithi039 ceph-mon[32062]: pgmap v546: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-07-04T08:17:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:25 smithi133 ceph-mon[37050]: pgmap v546: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-07-04T08:17:25.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:25 smithi050 ceph-mon[36577]: pgmap v546: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 KiB/s wr, 11 op/s 2024-07-04T08:17:25.922 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:17:26.651 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:27.051 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:27 smithi039 ceph-mon[32062]: pgmap v547: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-04T08:17:27.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:27 smithi133 ceph-mon[37050]: pgmap v547: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-04T08:17:27.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.369+0000 7fee45800640 1 -- 172.21.15.39:0/4012296244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40107700 msgr2=0x7fee40107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:27.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.369+0000 7fee45800640 1 --2- 172.21.15.39:0/4012296244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40107700 0x7fee40107b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009900 tx=0x7fee2c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.370+0000 7fee45800640 1 -- 172.21.15.39:0/4012296244 shutdown_connections 2024-07-04T08:17:27.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.370+0000 7fee45800640 1 --2- 172.21.15.39:0/4012296244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40107700 0x7fee40107b10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.370+0000 7fee45800640 1 --2- 172.21.15.39:0/4012296244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40108050 0x7fee4010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.370+0000 7fee45800640 1 --2- 172.21.15.39:0/4012296244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee40101dc0 0x7fee401021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.370+0000 7fee45800640 1 -- 172.21.15.39:0/4012296244 >> 172.21.15.39:0/4012296244 conn(0x7fee400fd810 msgr2=0x7fee400ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:27.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.370+0000 7fee45800640 1 -- 172.21.15.39:0/4012296244 shutdown_connections 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.370+0000 7fee45800640 1 -- 172.21.15.39:0/4012296244 wait complete. 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.371+0000 7fee45800640 1 Processor -- start 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.372+0000 7fee45800640 1 -- start start 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.372+0000 7fee45800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40101dc0 0x7fee401997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.372+0000 7fee45800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee40107700 0x7fee40199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.372+0000 7fee45800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40108050 0x7fee4019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.372+0000 7fee45800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fee40111d10 con 0x7fee40101dc0 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.372+0000 7fee45800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fee40111b90 con 0x7fee40108050 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.372+0000 7fee45800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee40111e90 con 0x7fee40107700 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40101dc0 0x7fee401997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40101dc0 0x7fee401997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47352/0 (socket says 172.21.15.39:47352) 2024-07-04T08:17:27.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3fe00640 1 -- 172.21.15.39:0/1885239631 learned_addr learned my addr 172.21.15.39:0/1885239631 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee40107700 0x7fee40199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee44e00640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40108050 0x7fee4019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3fe00640 1 -- 172.21.15.39:0/1885239631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee40107700 msgr2=0x7fee40199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3fe00640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee40107700 0x7fee40199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3fe00640 1 -- 172.21.15.39:0/1885239631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40108050 msgr2=0x7fee4019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3fe00640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40108050 0x7fee4019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3fe00640 1 -- 172.21.15.39:0/1885239631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4019f740 con 0x7fee40101dc0 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.373+0000 7fee3fe00640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40101dc0 0x7fee401997d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fee3000e9f0 tx=0x7fee3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.374+0000 7fee3cc00640 1 -- 172.21.15.39:0/1885239631 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3000cd80 con 0x7fee40101dc0 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.374+0000 7fee3cc00640 1 -- 172.21.15.39:0/1885239631 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee300108e0 con 0x7fee40101dc0 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.374+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4019fa30 con 0x7fee40101dc0 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.374+0000 7fee3cc00640 1 -- 172.21.15.39:0/1885239631 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30018980 con 0x7fee40101dc0 2024-07-04T08:17:27.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.374+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee401a6190 con 0x7fee40101dc0 2024-07-04T08:17:27.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.375+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee401021d0 con 0x7fee40101dc0 2024-07-04T08:17:27.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.378+0000 7fee3cc00640 1 -- 172.21.15.39:0/1885239631 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fee30010430 con 0x7fee40101dc0 2024-07-04T08:17:27.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.379+0000 7fee3cc00640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee240b29e0 0x7fee240b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:27.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.380+0000 7fee3cc00640 1 -- 172.21.15.39:0/1885239631 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fee300e3130 con 0x7fee40101dc0 2024-07-04T08:17:27.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.380+0000 7fee3f400640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee240b29e0 0x7fee240b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:27.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.381+0000 7fee3f400640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee240b29e0 0x7fee240b4ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009230 tx=0x7fee2c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:27.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.382+0000 7fee3cc00640 1 -- 172.21.15.39:0/1885239631 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fee300aa130 con 0x7fee40101dc0 2024-07-04T08:17:27.398 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:27 smithi050 ceph-mon[36577]: pgmap v547: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-04T08:17:27.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.645+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee401a6480 con 0x7fee40101dc0 2024-07-04T08:17:27.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.647+0000 7fee3cc00640 1 -- 172.21.15.39:0/1885239631 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fee300ae430 con 0x7fee40101dc0 2024-07-04T08:17:27.649 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:17:27.649 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:17:27.649 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:17:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.650+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee240b29e0 msgr2=0x7fee240b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.650+0000 7fee45800640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee240b29e0 0x7fee240b4ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009230 tx=0x7fee2c031400 comp rx=0 tx=0).stop 2024-07-04T08:17:27.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.650+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40101dc0 msgr2=0x7fee401997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.650+0000 7fee45800640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40101dc0 0x7fee401997d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fee3000e9f0 tx=0x7fee3000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.651+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 shutdown_connections 2024-07-04T08:17:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.651+0000 7fee45800640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee240b29e0 0x7fee240b4ef0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.651+0000 7fee45800640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee40108050 0x7fee4019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.651+0000 7fee45800640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee40107700 0x7fee40199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.651+0000 7fee45800640 1 --2- 172.21.15.39:0/1885239631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee40101dc0 0x7fee401997d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.651+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 >> 172.21.15.39:0/1885239631 conn(0x7fee400fd810 msgr2=0x7fee40100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.651+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 shutdown_connections 2024-07-04T08:17:27.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:27.651+0000 7fee45800640 1 -- 172.21.15.39:0/1885239631 wait complete. 2024-07-04T08:17:28.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:28 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1885239631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:17:28.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:28 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1885239631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:17:28.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:28 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1885239631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:17:28.661 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id ad947b9a --release --await' 2024-07-04T08:17:28.661 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id ad947b9a --release --await 2024-07-04T08:17:29.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:29 smithi133 ceph-mon[37050]: pgmap v548: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-07-04T08:17:29.327 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:29.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:29 smithi039 ceph-mon[32062]: pgmap v548: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-07-04T08:17:29.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:29 smithi050 ceph-mon[36577]: pgmap v548: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-07-04T08:17:30.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.028+0000 7ff923000640 1 -- 172.21.15.39:0/2016235783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff91c10ee50 msgr2=0x7ff91c10f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.028+0000 7ff923000640 1 --2- 172.21.15.39:0/2016235783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff91c10ee50 0x7ff91c10f260 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff908009900 tx=0x7ff90802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.028+0000 7ff923000640 1 -- 172.21.15.39:0/2016235783 shutdown_connections 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.028+0000 7ff923000640 1 --2- 172.21.15.39:0/2016235783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff91c10f7a0 0x7ff91c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.028+0000 7ff923000640 1 --2- 172.21.15.39:0/2016235783 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff91c10ee50 0x7ff91c10f260 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.028+0000 7ff923000640 1 --2- 172.21.15.39:0/2016235783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff91c109510 0x7ff91c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.028+0000 7ff923000640 1 -- 172.21.15.39:0/2016235783 >> 172.21.15.39:0/2016235783 conn(0x7ff91c078b10 msgr2=0x7ff91c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.029+0000 7ff923000640 1 -- 172.21.15.39:0/2016235783 shutdown_connections 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.029+0000 7ff923000640 1 -- 172.21.15.39:0/2016235783 wait complete. 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.030+0000 7ff923000640 1 Processor -- start 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.030+0000 7ff923000640 1 -- start start 2024-07-04T08:17:30.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.030+0000 7ff923000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff91c109510 0x7ff91c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.030+0000 7ff923000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff91c10ee50 0x7ff91c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.030+0000 7ff923000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff91c10f7a0 0x7ff91c1a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.030+0000 7ff923000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c119430 con 0x7ff91c109510 2024-07-04T08:17:30.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.030+0000 7ff923000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1192b0 con 0x7ff91c10ee50 2024-07-04T08:17:30.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.030+0000 7ff923000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1195b0 con 0x7ff91c10f7a0 2024-07-04T08:17:30.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff921c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff91c109510 0x7ff91c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff921c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff91c109510 0x7ff91c19fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47368/0 (socket says 172.21.15.39:47368) 2024-07-04T08:17:30.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff921c00640 1 -- 172.21.15.39:0/4131027233 learned_addr learned my addr 172.21.15.39:0/4131027233 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff922600640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff91c10f7a0 0x7ff91c1a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff921200640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff91c10ee50 0x7ff91c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff921c00640 1 -- 172.21.15.39:0/4131027233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff91c10f7a0 msgr2=0x7ff91c1a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff921c00640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff91c10f7a0 0x7ff91c1a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff921c00640 1 -- 172.21.15.39:0/4131027233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff91c10ee50 msgr2=0x7ff91c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff921c00640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff91c10ee50 0x7ff91c1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff921c00640 1 -- 172.21.15.39:0/4131027233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c1a5db0 con 0x7ff91c109510 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.031+0000 7ff922600640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff91c10f7a0 0x7ff91c1a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.032+0000 7ff921c00640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff91c109510 0x7ff91c19fe40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff91000e9f0 tx=0x7ff91000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.032+0000 7ff91a000640 1 -- 172.21.15.39:0/4131027233 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91000cd80 con 0x7ff91c109510 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.032+0000 7ff91a000640 1 -- 172.21.15.39:0/4131027233 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9100108e0 con 0x7ff91c109510 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.032+0000 7ff923000640 1 -- 172.21.15.39:0/4131027233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c1a60a0 con 0x7ff91c109510 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.032+0000 7ff91a000640 1 -- 172.21.15.39:0/4131027233 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910018980 con 0x7ff91c109510 2024-07-04T08:17:30.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.032+0000 7ff923000640 1 -- 172.21.15.39:0/4131027233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c1ac800 con 0x7ff91c109510 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.036+0000 7ff92abca740 1 Processor -- start 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.036+0000 7ff92abca740 1 -- start start 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.036+0000 7ff91a000640 1 -- 172.21.15.39:0/4131027233 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff910010430 con 0x7ff91c109510 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.036+0000 7ff92abca740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f7e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.036+0000 7ff92abca740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f7ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.036+0000 7ff92abca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 0x563876f7f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.036+0000 7ff92abca740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563876ef4ee0 con 0x563876bb11b0 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.036+0000 7ff92abca740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563876ee4480 con 0x563876bda070 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.036+0000 7ff92abca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563876ee84c0 con 0x563876bb9390 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff921c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f7ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff921c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f7ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47370/0 (socket says 172.21.15.39:47370) 2024-07-04T08:17:30.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff922600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f7e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff921c00640 1 -- 172.21.15.39:0/616928439 learned_addr learned my addr 172.21.15.39:0/616928439 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:30.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff921200640 1 --2- 172.21.15.39:0/616928439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 0x563876f7f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff921c00640 1 -- 172.21.15.39:0/616928439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 msgr2=0x563876f7f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:30.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff921c00640 1 --2- 172.21.15.39:0/616928439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 0x563876f7f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff921c00640 1 -- 172.21.15.39:0/616928439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 msgr2=0x563876f7e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:30.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff921c00640 1 --2- 172.21.15.39:0/616928439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f7e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff921c00640 1 -- 172.21.15.39:0/616928439 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563876e7c650 con 0x563876bb11b0 2024-07-04T08:17:30.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.037+0000 7ff922600640 1 --2- 172.21.15.39:0/616928439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f7e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:30.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.038+0000 7ff921c00640 1 --2- 172.21.15.39:0/616928439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f7ee90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff9100dffa0 tx=0x7ff9100eb750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc44442c9acd436 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.038+0000 7ff91a000640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9040b29e0 0x7ff9040b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.038+0000 7ff90ea00640 1 -- 172.21.15.39:0/616928439 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100a62c0 con 0x563876bb11b0 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.038+0000 7ff91a000640 1 -- 172.21.15.39:0/4131027233 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff9100e2c10 con 0x7ff91c109510 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.038+0000 7ff90ea00640 1 -- 172.21.15.39:0/616928439 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9100f0070 con 0x563876bb11b0 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.038+0000 7ff90ea00640 1 -- 172.21.15.39:0/616928439 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100a93e0 con 0x563876bb11b0 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.038+0000 7ff922600640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9040b29e0 0x7ff9040b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.039+0000 7ff92abca740 1 -- 172.21.15.39:0/616928439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 msgr2=0x563876f7ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.039+0000 7ff922600640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9040b29e0 0x7ff9040b4ef0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff914009fd0 tx=0x7ff914009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.039+0000 7ff92abca740 1 --2- 172.21.15.39:0/616928439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f7ee90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff9100dffa0 tx=0x7ff9100eb750 comp rx=0 tx=0).stop 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.039+0000 7ff92abca740 1 -- 172.21.15.39:0/616928439 shutdown_connections 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.040+0000 7ff92abca740 1 --2- 172.21.15.39:0/616928439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f7ee90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.040+0000 7ff92abca740 1 --2- 172.21.15.39:0/616928439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 0x563876f7f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.040+0000 7ff92abca740 1 --2- 172.21.15.39:0/616928439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f7e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.040+0000 7ff92abca740 1 -- 172.21.15.39:0/616928439 >> 172.21.15.39:0/616928439 conn(0x563876bce460 msgr2=0x563876f7c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.040+0000 7ff92abca740 1 -- 172.21.15.39:0/616928439 shutdown_connections 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.040+0000 7ff92abca740 1 -- 172.21.15.39:0/616928439 wait complete. 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.040+0000 7ff92abca740 1 -- start start 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.040+0000 7ff92abca740 1 Processor -- start 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff92abca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb11b0 0x563876f8a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff92abca740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb9390 0x563876f8ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff92abca740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f96450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff92abca740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563876d48700 con 0x563876bb9390 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff92abca740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563876e8bbb0 con 0x563876bda070 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff92abca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563876f8d130 con 0x563876bb11b0 2024-07-04T08:17:30.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff92abca740 10 client.0 _ll_register_callbacks cb 0x563876edd570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff921200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb9390 0x563876f8ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff921200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb9390 0x563876f8ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47382/0 (socket says 172.21.15.39:47382) 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff921200640 1 -- 172.21.15.39:0/4145563708 learned_addr learned my addr 172.21.15.39:0/4145563708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.041+0000 7ff921c00640 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb11b0 0x563876f8a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.042+0000 7ff922600640 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f96450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.042+0000 7ff921200640 1 -- 172.21.15.39:0/4145563708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb11b0 msgr2=0x563876f8a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.042+0000 7ff921200640 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb11b0 0x563876f8a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.042+0000 7ff921200640 1 -- 172.21.15.39:0/4145563708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 msgr2=0x563876f96450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.042+0000 7ff921200640 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f96450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.042+0000 7ff921200640 1 -- 172.21.15.39:0/4145563708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563876c45650 con 0x563876bb9390 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.042+0000 7ff922600640 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f96450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:30.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.042+0000 7ff921200640 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb9390 0x563876f8ab20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff908009900 tx=0x7ff908032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3294ba46ba50e4b5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.042+0000 7ff903400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.043+0000 7ff903400640 1 -- 172.21.15.39:0/4145563708 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908035040 con 0x563876bb9390 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.043+0000 7ff92abca740 1 -- 172.21.15.39:0/4145563708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9100f0070 con 0x563876bb9390 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.043+0000 7ff903400640 1 -- 172.21.15.39:0/4145563708 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff908031760 con 0x563876bb9390 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.043+0000 7ff903400640 1 -- 172.21.15.39:0/4145563708 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff90804f050 con 0x563876bb9390 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.043+0000 7ff92abca740 10 client.15885 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.043+0000 7ff92abca740 1 -- 172.21.15.39:0/4145563708 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563876f991c0 con 0x563876bb9390 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.043+0000 7ff903400640 1 -- 172.21.15.39:0/4145563708 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9100f0070 con 0x563876bb9390 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.044+0000 7ff903400640 1 -- 172.21.15.39:0/4145563708 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff908004750 con 0x563876bb9390 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.044+0000 7ff92abca740 10 client.15885 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:30.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.044+0000 7ff92abca740 10 client.15885 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:17:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.044+0000 7ff92abca740 20 client.15885 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.044+0000 7ff92abca740 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x563876b989f0 0x563876f9c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.044+0000 7ff92abca740 4 client.15885 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.044+0000 7ff92abca740 1 -- 172.21.15.39:0/4145563708 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563876ef6e30 con 0x563876b989f0 2024-07-04T08:17:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.044+0000 7ff921c00640 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x563876b989f0 0x563876f9c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.045+0000 7ff921c00640 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x563876b989f0 0x563876f9c6e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d78ab444485ef2f server_cookie=730aea395704df5f in_seq=0 out_seq=0 2024-07-04T08:17:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.045+0000 7ff903400640 10 client.15885 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:17:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.045+0000 7ff903400640 1 -- 172.21.15.39:0/4145563708 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563876ef6e30 con 0x563876b989f0 2024-07-04T08:17:30.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.045+0000 7ff903400640 10 client.15885 handle_command_reply: tid=0 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 client.15885 shutdown 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 -- 172.21.15.39:0/4145563708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb9390 msgr2=0x563876f8ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb9390 0x563876f8ab20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff908009900 tx=0x7ff908032e90 comp rx=0 tx=0).stop 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 -- 172.21.15.39:0/4145563708 shutdown_connections 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x563876b989f0 0x563876f9c6e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb9390 0x563876f8ab20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bda070 0x563876f96450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 --2- 172.21.15.39:0/4145563708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb11b0 0x563876f8a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff903400640 0 client.15885 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 -- 172.21.15.39:0/4145563708 >> 172.21.15.39:0/4145563708 conn(0x563876bce460 msgr2=0x563876f81ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 -- 172.21.15.39:0/4145563708 shutdown_connections 2024-07-04T08:17:30.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 1 -- 172.21.15.39:0/4145563708 wait complete. 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.046+0000 7ff92abca740 20 client.15885 trim_cache size 0 max 16384 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff92abca740 1 Processor -- start 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff92abca740 1 -- start start 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff92abca740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f85b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff92abca740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb9390 0x563876f80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff92abca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bda070 0x563876f96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff92abca740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b6670 con 0x563876bb11b0 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff92abca740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b81b0 con 0x563876bb9390 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff92abca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b6d40 con 0x563876bda070 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff921c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f85b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff921c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f85b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47396/0 (socket says 172.21.15.39:47396) 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.062+0000 7ff921c00640 1 -- 172.21.15.39:0/4287581422 learned_addr learned my addr 172.21.15.39:0/4287581422 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff921200640 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb9390 0x563876f80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff922600640 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bda070 0x563876f96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff921c00640 1 -- 172.21.15.39:0/4287581422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bda070 msgr2=0x563876f96240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff921c00640 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bda070 0x563876f96240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff921c00640 1 -- 172.21.15.39:0/4287581422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb9390 msgr2=0x563876f80090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff921c00640 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb9390 0x563876f80090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff921c00640 1 -- 172.21.15.39:0/4287581422 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563876f80c40 con 0x563876bb11b0 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff922600640 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bda070 0x563876f96240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff921c00640 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f85b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9100a9c00 tx=0x7ff9100e6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e30896190574a23 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.063+0000 7ff90e000640 1 -- 172.21.15.39:0/4287581422 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100a95c0 con 0x563876bb11b0 2024-07-04T08:17:30.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.064+0000 7ff90e000640 1 -- 172.21.15.39:0/4287581422 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9100f0040 con 0x563876bb11b0 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.064+0000 7ff90e000640 1 -- 172.21.15.39:0/4287581422 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100a97b0 con 0x563876bb11b0 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.064+0000 7ff92abca740 1 -- 172.21.15.39:0/4287581422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 msgr2=0x563876f85b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.064+0000 7ff92abca740 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f85b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9100a9c00 tx=0x7ff9100e6760 comp rx=0 tx=0).stop 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.064+0000 7ff92abca740 1 -- 172.21.15.39:0/4287581422 shutdown_connections 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.064+0000 7ff92abca740 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bda070 0x563876f96240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.064+0000 7ff92abca740 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb9390 0x563876f80090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.064+0000 7ff92abca740 1 --2- 172.21.15.39:0/4287581422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bb11b0 0x563876f85b50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.064+0000 7ff92abca740 1 -- 172.21.15.39:0/4287581422 >> 172.21.15.39:0/4287581422 conn(0x563876bce460 msgr2=0x563876f7f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.065+0000 7ff92abca740 1 -- 172.21.15.39:0/4287581422 shutdown_connections 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.065+0000 7ff92abca740 1 -- 172.21.15.39:0/4287581422 wait complete. 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.065+0000 7ff92abca740 1 -- start start 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.065+0000 7ff92abca740 1 Processor -- start 2024-07-04T08:17:30.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff92abca740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb11b0 0x563876f8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff92abca740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 0x563876f8e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff92abca740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bda070 0x563876ffce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff92abca740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b7ae0 con 0x563876bda070 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff92abca740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b7410 con 0x563876bb11b0 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff92abca740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b4650 con 0x563876bb9390 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff92abca740 10 client.0 _ll_register_callbacks cb 0x7ff908009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff921c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb11b0 0x563876f8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff922600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bda070 0x563876ffce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff921c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb11b0 0x563876f8db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53608/0 (socket says 172.21.15.39:53608) 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff921c00640 1 -- 172.21.15.39:0/3689513426 learned_addr learned my addr 172.21.15.39:0/3689513426 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff921200640 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 0x563876f8e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff921c00640 1 -- 172.21.15.39:0/3689513426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 msgr2=0x563876f8e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.066+0000 7ff921c00640 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 0x563876f8e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff921c00640 1 -- 172.21.15.39:0/3689513426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bda070 msgr2=0x563876ffce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff921c00640 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bda070 0x563876ffce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff921c00640 1 -- 172.21.15.39:0/3689513426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563876ffd4a0 con 0x563876bb11b0 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff921c00640 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb11b0 0x563876f8db80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff9100e0630 tx=0x7ff910005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fefdcfdf96e4b05 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff903400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff903400640 1 -- 172.21.15.39:0/3689513426 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9100c9060 con 0x563876bb11b0 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff92abca740 1 -- 172.21.15.39:0/3689513426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9100f0040 con 0x563876bb11b0 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff903400640 1 -- 172.21.15.39:0/3689513426 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff9100e6530 con 0x563876bb11b0 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff903400640 1 -- 172.21.15.39:0/3689513426 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff9100a9a40 con 0x563876bb11b0 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff92abca740 10 client.26003 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:30.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.067+0000 7ff92abca740 1 -- 172.21.15.39:0/3689513426 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638770047e0 con 0x563876bb11b0 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.068+0000 7ff903400640 1 -- 172.21.15.39:0/3689513426 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9100f0040 con 0x563876bb11b0 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.068+0000 7ff903400640 1 -- 172.21.15.39:0/3689513426 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff9100b2040 con 0x563876bb11b0 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.068+0000 7ff92abca740 10 client.26003 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.068+0000 7ff92abca740 10 client.26003 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.068+0000 7ff92abca740 20 client.26003 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.068+0000 7ff92abca740 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x563876b6dda0 0x563877006870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.068+0000 7ff92abca740 4 client.26003 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.068+0000 7ff92abca740 1 -- 172.21.15.39:0/3689513426 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563876ef28d0 con 0x563876b6dda0 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.068+0000 7ff921200640 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x563876b6dda0 0x563877006870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff921200640 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x563876b6dda0 0x563877006870 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b396166736dc35d2 server_cookie=a172217912214ac5 in_seq=0 out_seq=0 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff903400640 10 client.26003 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:17:30.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff903400640 1 -- 172.21.15.39:0/3689513426 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563876ef28d0 con 0x563876b6dda0 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff903400640 10 client.26003 handle_command_reply: tid=0 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff92abca740 1 client.26003 shutdown 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff92abca740 1 -- 172.21.15.39:0/3689513426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb11b0 msgr2=0x563876f8db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff92abca740 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb11b0 0x563876f8db80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff9100e0630 tx=0x7ff910005ad0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff92abca740 1 -- 172.21.15.39:0/3689513426 shutdown_connections 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff92abca740 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x563876b6dda0 0x563877006870 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff92abca740 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563876bda070 0x563876ffce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff92abca740 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563876bb9390 0x563876f8e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff92abca740 1 --2- 172.21.15.39:0/3689513426 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563876bb11b0 0x563876f8db80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff903400640 0 client.26003 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.069+0000 7ff92abca740 1 -- 172.21.15.39:0/3689513426 >> 172.21.15.39:0/3689513426 conn(0x563876bce460 msgr2=0x563876f8b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:30.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.070+0000 7ff92abca740 1 -- 172.21.15.39:0/3689513426 shutdown_connections 2024-07-04T08:17:30.145 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:17:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.070+0000 7ff92abca740 1 -- 172.21.15.39:0/3689513426 wait complete. 2024-07-04T08:17:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.070+0000 7ff92abca740 20 client.26003 trim_cache size 0 max 16384 2024-07-04T08:17:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 -- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9040b29e0 msgr2=0x7ff9040b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9040b29e0 0x7ff9040b4ef0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff914009fd0 tx=0x7ff914009500 comp rx=0 tx=0).stop 2024-07-04T08:17:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 -- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff91c109510 msgr2=0x7ff91c19fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff91c109510 0x7ff91c19fe40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff91000e9f0 tx=0x7ff91000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 -- 172.21.15.39:0/4131027233 shutdown_connections 2024-07-04T08:17:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9040b29e0 0x7ff9040b4ef0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff91c10f7a0 0x7ff91c1a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff91c10ee50 0x7ff91c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 --2- 172.21.15.39:0/4131027233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff91c109510 0x7ff91c19fe40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:30.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 -- 172.21.15.39:0/4131027233 >> 172.21.15.39:0/4131027233 conn(0x7ff91c078b10 msgr2=0x7ff91c079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:30.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 -- 172.21.15.39:0/4131027233 shutdown_connections 2024-07-04T08:17:30.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:30.071+0000 7ff923000640 1 -- 172.21.15.39:0/4131027233 wait complete. 2024-07-04T08:17:31.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:31 smithi133 ceph-mon[37050]: pgmap v549: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-07-04T08:17:31.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:31 smithi039 ceph-mon[32062]: pgmap v549: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-07-04T08:17:31.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:31 smithi050 ceph-mon[36577]: pgmap v549: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-07-04T08:17:31.956 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:17:32.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.623+0000 7f8832000640 1 -- 172.21.15.39:0/1368737643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f882c109370 msgr2=0x7f882c10fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.623+0000 7f8832000640 1 --2- 172.21.15.39:0/1368737643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f882c109370 0x7f882c10fba0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8824009ca0 tx=0x7f882402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.624+0000 7f8832000640 1 -- 172.21.15.39:0/1368737643 shutdown_connections 2024-07-04T08:17:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.624+0000 7f8832000640 1 --2- 172.21.15.39:0/1368737643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f882c109370 0x7f882c10fba0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.624+0000 7f8832000640 1 --2- 172.21.15.39:0/1368737643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f882c108a20 0x7f882c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.624+0000 7f8832000640 1 --2- 172.21.15.39:0/1368737643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f882c1030e0 0x7f882c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.624+0000 7f8832000640 1 -- 172.21.15.39:0/1368737643 >> 172.21.15.39:0/1368737643 conn(0x7f882c0feb30 msgr2=0x7f882c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:32.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.624+0000 7f8832000640 1 -- 172.21.15.39:0/1368737643 shutdown_connections 2024-07-04T08:17:32.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.624+0000 7f8832000640 1 -- 172.21.15.39:0/1368737643 wait complete. 2024-07-04T08:17:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.625+0000 7f8832000640 1 Processor -- start 2024-07-04T08:17:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.625+0000 7f8832000640 1 -- start start 2024-07-04T08:17:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f8832000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f882c1030e0 0x7f882c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f8832000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f882c108a20 0x7f882c19c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f8832000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f882c109370 0x7f882c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f8832000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f882c113040 con 0x7f882c109370 2024-07-04T08:17:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f8832000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f882c112ec0 con 0x7f882c1030e0 2024-07-04T08:17:32.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f8832000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1131c0 con 0x7f882c108a20 2024-07-04T08:17:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f882b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f882c1030e0 0x7f882c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f882b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f882c1030e0 0x7f882c19bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59382/0 (socket says 172.21.15.39:59382) 2024-07-04T08:17:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f882b400640 1 -- 172.21.15.39:0/439916777 learned_addr learned my addr 172.21.15.39:0/439916777 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f882be00640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f882c109370 0x7f882c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.626+0000 7f882aa00640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f882c108a20 0x7f882c19c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.627+0000 7f882b400640 1 -- 172.21.15.39:0/439916777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f882c108a20 msgr2=0x7f882c19c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.627+0000 7f882b400640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f882c108a20 0x7f882c19c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.627+0000 7f882b400640 1 -- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f882c109370 msgr2=0x7f882c1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:32.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.627+0000 7f882b400640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f882c109370 0x7f882c1a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.627+0000 7f882b400640 1 -- 172.21.15.39:0/439916777 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c1a1b40 con 0x7f882c1030e0 2024-07-04T08:17:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.627+0000 7f882b400640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f882c1030e0 0x7f882c19bbd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f881c00e9f0 tx=0x7f881c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.628+0000 7f8823e00640 1 -- 172.21.15.39:0/439916777 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c00cd80 con 0x7f882c1030e0 2024-07-04T08:17:32.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.628+0000 7f8832000640 1 -- 172.21.15.39:0/439916777 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882c1a1e30 con 0x7f882c1030e0 2024-07-04T08:17:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.628+0000 7f8823e00640 1 -- 172.21.15.39:0/439916777 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f881c0054d0 con 0x7f882c1030e0 2024-07-04T08:17:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.628+0000 7f8823e00640 1 -- 172.21.15.39:0/439916777 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c010690 con 0x7f882c1030e0 2024-07-04T08:17:32.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.628+0000 7f8832000640 1 -- 172.21.15.39:0/439916777 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882c1a8590 con 0x7f882c1030e0 2024-07-04T08:17:32.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.631+0000 7f88386f3740 1 Processor -- start 2024-07-04T08:17:32.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.631+0000 7f88386f3740 1 -- start start 2024-07-04T08:17:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f88386f3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f8823e00640 1 -- 172.21.15.39:0/439916777 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f881c010830 con 0x7f882c1030e0 2024-07-04T08:17:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f88386f3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f88386f3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f88386f3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b0d4202b00 con 0x55b0d3f43520 2024-07-04T08:17:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f88386f3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b0d425e290 con 0x55b0d3f22840 2024-07-04T08:17:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f88386f3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b0d424d830 con 0x55b0d3f1a660 2024-07-04T08:17:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f882be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f882aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f882aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59396/0 (socket says 172.21.15.39:59396) 2024-07-04T08:17:32.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f882aa00640 1 -- 172.21.15.39:0/2446963385 learned_addr learned my addr 172.21.15.39:0/2446963385 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:32.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.632+0000 7f882b400640 1 --2- 172.21.15.39:0/2446963385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.633+0000 7f882aa00640 1 -- 172.21.15.39:0/2446963385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 msgr2=0x55b0d42e87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:32.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.633+0000 7f882aa00640 1 --2- 172.21.15.39:0/2446963385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42e87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.633+0000 7f882aa00640 1 -- 172.21.15.39:0/2446963385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 msgr2=0x55b0d42e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:32.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.633+0000 7f882aa00640 1 --2- 172.21.15.39:0/2446963385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.633+0000 7f882aa00640 1 -- 172.21.15.39:0/2446963385 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d4207760 con 0x55b0d3f22840 2024-07-04T08:17:32.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.633+0000 7f882be00640 1 --2- 172.21.15.39:0/2446963385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:32.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.634+0000 7f882aa00640 1 --2- 172.21.15.39:0/2446963385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e8d20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8818009c20 tx=0x7f8818011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a69c0adf82ae3b32 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:32.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.634+0000 7f8823e00640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88100b2960 0x7f88100b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.634+0000 7f8820c00640 1 -- 172.21.15.39:0/2446963385 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818004230 con 0x55b0d3f22840 2024-07-04T08:17:32.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.635+0000 7f8820c00640 1 -- 172.21.15.39:0/2446963385 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88180043d0 con 0x55b0d3f22840 2024-07-04T08:17:32.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.635+0000 7f8820c00640 1 -- 172.21.15.39:0/2446963385 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818004d50 con 0x55b0d3f22840 2024-07-04T08:17:32.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.635+0000 7f882be00640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88100b2960 0x7f88100b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.635+0000 7f8823e00640 1 -- 172.21.15.39:0/439916777 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f881c0e2de0 con 0x7f882c1030e0 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f88386f3740 1 -- 172.21.15.39:0/2446963385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 msgr2=0x55b0d42e8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f88386f3740 1 --2- 172.21.15.39:0/2446963385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e8d20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8818009c20 tx=0x7f8818011810 comp rx=0 tx=0).stop 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f882be00640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88100b2960 0x7f88100b4e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f882402f500 tx=0x7f882402f490 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f88386f3740 1 -- 172.21.15.39:0/2446963385 shutdown_connections 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f88386f3740 1 --2- 172.21.15.39:0/2446963385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e8d20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f88386f3740 1 --2- 172.21.15.39:0/2446963385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f88386f3740 1 --2- 172.21.15.39:0/2446963385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f88386f3740 1 -- 172.21.15.39:0/2446963385 >> 172.21.15.39:0/2446963385 conn(0x55b0d3f37910 msgr2=0x55b0d42e6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f88386f3740 1 -- 172.21.15.39:0/2446963385 shutdown_connections 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.636+0000 7f88386f3740 1 -- 172.21.15.39:0/2446963385 wait complete. 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.637+0000 7f88386f3740 1 -- start start 2024-07-04T08:17:32.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.637+0000 7f88386f3740 1 Processor -- start 2024-07-04T08:17:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.637+0000 7f88386f3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f88386f3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f88386f3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f88386f3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b0d41f4d00 con 0x55b0d3f43520 2024-07-04T08:17:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f88386f3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b0d4251870 con 0x55b0d3f22840 2024-07-04T08:17:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f88386f3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b0d40b1bb0 con 0x55b0d3f1a660 2024-07-04T08:17:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f88386f3740 10 client.0 _ll_register_callbacks cb 0x55b0d41f59c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59402/0 (socket says 172.21.15.39:59402) 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882aa00640 1 -- 172.21.15.39:0/834817530 learned_addr learned my addr 172.21.15.39:0/834817530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882b400640 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882be00640 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882aa00640 1 -- 172.21.15.39:0/834817530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 msgr2=0x55b0d42f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882aa00640 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882aa00640 1 -- 172.21.15.39:0/834817530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 msgr2=0x55b0d42f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882aa00640 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882aa00640 1 -- 172.21.15.39:0/834817530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d41e4640 con 0x55b0d3f22840 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.638+0000 7f882be00640 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.639+0000 7f882aa00640 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42f3e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8818019a50 tx=0x7f8818019a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12fe4b341dfdae2a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.639+0000 7f8815600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.639+0000 7f8815600640 1 -- 172.21.15.39:0/834817530 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818028020 con 0x55b0d3f22840 2024-07-04T08:17:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.639+0000 7f8815600640 1 -- 172.21.15.39:0/834817530 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8818024620 con 0x55b0d3f22840 2024-07-04T08:17:32.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.639+0000 7f88386f3740 1 -- 172.21.15.39:0/834817530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88180043d0 con 0x55b0d3f22840 2024-07-04T08:17:32.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.640+0000 7f8815600640 1 -- 172.21.15.39:0/834817530 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8818033050 con 0x55b0d3f22840 2024-07-04T08:17:32.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.640+0000 7f88386f3740 10 client.26015 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:32.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.640+0000 7f88386f3740 1 -- 172.21.15.39:0/834817530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0d3faeb00 con 0x55b0d3f22840 2024-07-04T08:17:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.640+0000 7f8815600640 1 -- 172.21.15.39:0/834817530 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88180043d0 con 0x55b0d3f22840 2024-07-04T08:17:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.640+0000 7f8815600640 1 -- 172.21.15.39:0/834817530 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8818018070 con 0x55b0d3f22840 2024-07-04T08:17:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.641+0000 7f88386f3740 10 client.26015 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.641+0000 7f88386f3740 10 client.26015 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:17:32.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.641+0000 7f88386f3740 20 client.26015 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.641+0000 7f88386f3740 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b0d3f01ea0 0x55b0d43057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.641+0000 7f88386f3740 4 client.26015 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.641+0000 7f88386f3740 1 -- 172.21.15.39:0/834817530 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0d42601e0 con 0x55b0d3f01ea0 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.641+0000 7f882b400640 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b0d3f01ea0 0x55b0d43057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.642+0000 7f882b400640 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b0d3f01ea0 0x55b0d43057e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0aa80d8d13c12ba server_cookie=338436260bedd120 in_seq=0 out_seq=0 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.642+0000 7f8815600640 10 client.26015 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.642+0000 7f8815600640 1 -- 172.21.15.39:0/834817530 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b0d42601e0 con 0x55b0d3f01ea0 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.642+0000 7f8815600640 10 client.26015 handle_command_reply: tid=0 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.642+0000 7f88386f3740 1 client.26015 shutdown 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 -- 172.21.15.39:0/834817530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 msgr2=0x55b0d42f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42f3e20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8818019a50 tx=0x7f8818019a80 comp rx=0 tx=0).stop 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 -- 172.21.15.39:0/834817530 shutdown_connections 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b0d3f01ea0 0x55b0d43057e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42f3e20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f8815600640 0 client.26015 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 --2- 172.21.15.39:0/834817530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 -- 172.21.15.39:0/834817530 >> 172.21.15.39:0/834817530 conn(0x55b0d3f37910 msgr2=0x55b0d42eb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 -- 172.21.15.39:0/834817530 shutdown_connections 2024-07-04T08:17:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 1 -- 172.21.15.39:0/834817530 wait complete. 2024-07-04T08:17:32.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.643+0000 7f88386f3740 20 client.26015 trim_cache size 0 max 16384 2024-07-04T08:17:32.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.663+0000 7f88386f3740 1 Processor -- start 2024-07-04T08:17:32.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.663+0000 7f88386f3740 1 -- start start 2024-07-04T08:17:32.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f88386f3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42ef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f88386f3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f88386f3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f88386f3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1b2380 con 0x55b0d3f43520 2024-07-04T08:17:32.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f88386f3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1b3ec0 con 0x55b0d3f22840 2024-07-04T08:17:32.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f88386f3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1b2a50 con 0x55b0d3f1a660 2024-07-04T08:17:32.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f882b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42ef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f882be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f882b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42ef0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56696/0 (socket says 172.21.15.39:56696) 2024-07-04T08:17:32.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.664+0000 7f882b400640 1 -- 172.21.15.39:0/3093395571 learned_addr learned my addr 172.21.15.39:0/3093395571 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:32.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.665+0000 7f882aa00640 1 --2- 172.21.15.39:0/3093395571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.665+0000 7f882b400640 1 -- 172.21.15.39:0/3093395571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 msgr2=0x55b0d42e54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:32.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.665+0000 7f882b400640 1 --2- 172.21.15.39:0/3093395571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.665+0000 7f882b400640 1 -- 172.21.15.39:0/3093395571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 msgr2=0x55b0d42e99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:32.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.665+0000 7f882b400640 1 --2- 172.21.15.39:0/3093395571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.665+0000 7f882b400640 1 -- 172.21.15.39:0/3093395571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d4025960 con 0x55b0d3f1a660 2024-07-04T08:17:32.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.665+0000 7f882be00640 1 --2- 172.21.15.39:0/3093395571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:32.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.665+0000 7f882b400640 1 --2- 172.21.15.39:0/3093395571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42ef0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f881c00e9c0 tx=0x7f881c0e5230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11d61a6bb7a3716 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:32.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.666+0000 7f8817e00640 1 -- 172.21.15.39:0/3093395571 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c0a9680 con 0x55b0d3f1a660 2024-07-04T08:17:32.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.666+0000 7f8817e00640 1 -- 172.21.15.39:0/3093395571 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f881c0c0070 con 0x55b0d3f1a660 2024-07-04T08:17:32.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.666+0000 7f8817e00640 1 -- 172.21.15.39:0/3093395571 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c0a94a0 con 0x55b0d3f1a660 2024-07-04T08:17:32.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.667+0000 7f88386f3740 1 -- 172.21.15.39:0/3093395571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 msgr2=0x55b0d42ef0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:32.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.667+0000 7f88386f3740 1 --2- 172.21.15.39:0/3093395571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42ef0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f881c00e9c0 tx=0x7f881c0e5230 comp rx=0 tx=0).stop 2024-07-04T08:17:32.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.667+0000 7f88386f3740 1 -- 172.21.15.39:0/3093395571 shutdown_connections 2024-07-04T08:17:32.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.667+0000 7f88386f3740 1 --2- 172.21.15.39:0/3093395571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42e99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.667+0000 7f88386f3740 1 --2- 172.21.15.39:0/3093395571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f22840 0x55b0d42e54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.667+0000 7f88386f3740 1 --2- 172.21.15.39:0/3093395571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f1a660 0x55b0d42ef0a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.667+0000 7f88386f3740 1 -- 172.21.15.39:0/3093395571 >> 172.21.15.39:0/3093395571 conn(0x55b0d3f37910 msgr2=0x55b0d42e92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:32.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.668+0000 7f88386f3740 1 -- 172.21.15.39:0/3093395571 shutdown_connections 2024-07-04T08:17:32.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.668+0000 7f88386f3740 1 -- 172.21.15.39:0/3093395571 wait complete. 2024-07-04T08:17:32.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.668+0000 7f88386f3740 1 -- start start 2024-07-04T08:17:32.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.669+0000 7f88386f3740 1 Processor -- start 2024-07-04T08:17:32.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.669+0000 7f88386f3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f1a660 0x55b0d42fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f88386f3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f22840 0x55b0d42ede30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f88386f3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f88386f3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1b37f0 con 0x55b0d3f43520 2024-07-04T08:17:32.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f88386f3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1b3120 con 0x55b0d3f1a660 2024-07-04T08:17:32.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f88386f3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1b0360 con 0x55b0d3f22840 2024-07-04T08:17:32.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f88386f3740 10 client.0 _ll_register_callbacks cb 0x7f8818009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:32.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f882be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f882aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f22840 0x55b0d42ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f882aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f22840 0x55b0d42ede30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56704/0 (socket says 172.21.15.39:56704) 2024-07-04T08:17:32.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f882aa00640 1 -- 172.21.15.39:0/3368353551 learned_addr learned my addr 172.21.15.39:0/3368353551 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:32.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f882b400640 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f1a660 0x55b0d42fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f882aa00640 1 -- 172.21.15.39:0/3368353551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f1a660 msgr2=0x55b0d42fbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:32.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.670+0000 7f882aa00640 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f1a660 0x55b0d42fbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.671+0000 7f882aa00640 1 -- 172.21.15.39:0/3368353551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 msgr2=0x55b0d42ee370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:32.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.671+0000 7f882aa00640 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42ee370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.671+0000 7f882aa00640 1 -- 172.21.15.39:0/3368353551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d421a880 con 0x55b0d3f22840 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.671+0000 7f882aa00640 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f22840 0x55b0d42ede30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8818018360 tx=0x7f8818019780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68191ad66fff3887 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.671+0000 7f8815600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.671+0000 7f8815600640 1 -- 172.21.15.39:0/3368353551 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818028040 con 0x55b0d3f22840 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.671+0000 7f88386f3740 1 -- 172.21.15.39:0/3368353551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f881c0c0070 con 0x55b0d3f22840 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.671+0000 7f8815600640 1 -- 172.21.15.39:0/3368353551 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8818018af0 con 0x55b0d3f22840 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.672+0000 7f8815600640 1 -- 172.21.15.39:0/3368353551 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8818044050 con 0x55b0d3f22840 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.672+0000 7f88386f3740 10 client.25987 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.672+0000 7f88386f3740 1 -- 172.21.15.39:0/3368353551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0d4216430 con 0x55b0d3f22840 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.672+0000 7f8815600640 1 -- 172.21.15.39:0/3368353551 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f881c0c0070 con 0x55b0d3f22840 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.672+0000 7f8815600640 1 -- 172.21.15.39:0/3368353551 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f881803f030 con 0x55b0d3f22840 2024-07-04T08:17:32.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.673+0000 7f88386f3740 10 client.25987 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:32.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.673+0000 7f88386f3740 10 client.25987 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:17:32.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.673+0000 7f88386f3740 20 client.25987 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:32.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.673+0000 7f88386f3740 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b0d3ed7250 0x55b0d436fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:32.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.673+0000 7f88386f3740 4 client.25987 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ad947b9a", "release": true, "await": true}] 2024-07-04T08:17:32.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.673+0000 7f88386f3740 1 -- 172.21.15.39:0/3368353551 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ad947b9a", "release": true, "await": true}) -- 0x55b0d425bc80 con 0x55b0d3ed7250 2024-07-04T08:17:32.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.673+0000 7f882b400640 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b0d3ed7250 0x55b0d436fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:32.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.674+0000 7f882b400640 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b0d3ed7250 0x55b0d436fe50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72f41d3a2586deec server_cookie=fc42303b49d43897 in_seq=0 out_seq=0 2024-07-04T08:17:32.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.674+0000 7f8815600640 10 client.25987 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:32.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.675+0000 7f8815600640 1 -- 172.21.15.39:0/3368353551 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b0d425bc80 con 0x55b0d3ed7250 2024-07-04T08:17:32.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.675+0000 7f8815600640 10 client.25987 handle_command_reply: tid=0 2024-07-04T08:17:32.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 client.25987 shutdown 2024-07-04T08:17:32.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 -- 172.21.15.39:0/3368353551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f22840 msgr2=0x55b0d42ede30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:32.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f22840 0x55b0d42ede30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8818018360 tx=0x7f8818019780 comp rx=0 tx=0).stop 2024-07-04T08:17:32.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 -- 172.21.15.39:0/3368353551 shutdown_connections 2024-07-04T08:17:32.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b0d3ed7250 0x55b0d436fe50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b0d3f43520 0x55b0d42ee370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0d3f22840 0x55b0d42ede30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 --2- 172.21.15.39:0/3368353551 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b0d3f1a660 0x55b0d42fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f8815600640 0 client.25987 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:32.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 -- 172.21.15.39:0/3368353551 >> 172.21.15.39:0/3368353551 conn(0x55b0d3f37910 msgr2=0x55b0d42f2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:32.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.676+0000 7f88386f3740 1 -- 172.21.15.39:0/3368353551 shutdown_connections 2024-07-04T08:17:32.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.677+0000 7f88386f3740 1 -- 172.21.15.39:0/3368353551 wait complete. 2024-07-04T08:17:32.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.677+0000 7f88386f3740 20 client.25987 trim_cache size 0 max 16384 2024-07-04T08:17:32.679 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:17:32.679 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:17:32.679 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:17:32.679 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 8, 2024-07-04T08:17:32.679 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "ad947b9a": { 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 8, 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 142.8, 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:17:32.680 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:17:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:17:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:17:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:17:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:32.681 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:17:32.681 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:17:32.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 -- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88100b2960 msgr2=0x7f88100b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:32.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88100b2960 0x7f88100b4e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f882402f500 tx=0x7f882402f490 comp rx=0 tx=0).stop 2024-07-04T08:17:32.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 -- 172.21.15.39:0/439916777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f882c1030e0 msgr2=0x7f882c19bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:32.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f882c1030e0 0x7f882c19bbd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f881c00e9f0 tx=0x7f881c00edf0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 -- 172.21.15.39:0/439916777 shutdown_connections 2024-07-04T08:17:32.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88100b2960 0x7f88100b4e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f882c109370 0x7f882c1a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f882c108a20 0x7f882c19c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 --2- 172.21.15.39:0/439916777 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f882c1030e0 0x7f882c19bbd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:32.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 -- 172.21.15.39:0/439916777 >> 172.21.15.39:0/439916777 conn(0x7f882c0feb30 msgr2=0x7f882c10de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:32.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 -- 172.21.15.39:0/439916777 shutdown_connections 2024-07-04T08:17:32.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:32.678+0000 7f8832000640 1 -- 172.21.15.39:0/439916777 wait complete. 2024-07-04T08:17:32.685 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:33.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:33 smithi133 ceph-mon[37050]: pgmap v550: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-04T08:17:33.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:33 smithi039 ceph-mon[32062]: pgmap v550: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-04T08:17:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:33 smithi050 ceph-mon[36577]: pgmap v550: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-04T08:17:34.026 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ad947b9a', total seconds elapsed: 35.0 2024-07-04T08:17:35.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:35 smithi039 ceph-mon[32062]: pgmap v551: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-07-04T08:17:35.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:35 smithi133 ceph-mon[37050]: pgmap v551: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-07-04T08:17:35.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:35 smithi050 ceph-mon[36577]: pgmap v551: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-07-04T08:17:35.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.626+0000 7f2500800640 1 -- 172.21.15.39:0/1177668345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f810b590 msgr2=0x7f24f8111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:35.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.626+0000 7f2500800640 1 --2- 172.21.15.39:0/1177668345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f810b590 0x7f24f8111dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f24f4009ca0 tx=0x7f24f402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.626+0000 7f2500800640 1 -- 172.21.15.39:0/1177668345 shutdown_connections 2024-07-04T08:17:35.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.626+0000 7f2500800640 1 --2- 172.21.15.39:0/1177668345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f810b590 0x7f24f8111dc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.626+0000 7f2500800640 1 --2- 172.21.15.39:0/1177668345 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f810ac40 0x7f24f810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.626+0000 7f2500800640 1 --2- 172.21.15.39:0/1177668345 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f8105300 0x7f24f8105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.626+0000 7f2500800640 1 -- 172.21.15.39:0/1177668345 >> 172.21.15.39:0/1177668345 conn(0x7f24f8100d10 msgr2=0x7f24f8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:35.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.626+0000 7f2500800640 1 -- 172.21.15.39:0/1177668345 shutdown_connections 2024-07-04T08:17:35.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.626+0000 7f2500800640 1 -- 172.21.15.39:0/1177668345 wait complete. 2024-07-04T08:17:35.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.628+0000 7f2500800640 1 Processor -- start 2024-07-04T08:17:35.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.628+0000 7f2500800640 1 -- start start 2024-07-04T08:17:35.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.628+0000 7f2500800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f8105300 0x7f24f819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.628+0000 7f2500800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f810ac40 0x7f24f819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.628+0000 7f2500800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f810b590 0x7f24f81a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.628+0000 7f2500800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24f8115260 con 0x7f24f810b590 2024-07-04T08:17:35.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.628+0000 7f2500800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81150e0 con 0x7f24f810ac40 2024-07-04T08:17:35.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.628+0000 7f2500800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81153e0 con 0x7f24f8105300 2024-07-04T08:17:35.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f810b590 0x7f24f81a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fde00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f8105300 0x7f24f819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f810ac40 0x7f24f819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fde00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f8105300 0x7f24f819bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56724/0 (socket says 172.21.15.39:56724) 2024-07-04T08:17:35.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fde00640 1 -- 172.21.15.39:0/2780282839 learned_addr learned my addr 172.21.15.39:0/2780282839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:35.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fde00640 1 -- 172.21.15.39:0/2780282839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f810ac40 msgr2=0x7f24f819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:35.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fde00640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f810ac40 0x7f24f819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fde00640 1 -- 172.21.15.39:0/2780282839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f810b590 msgr2=0x7f24f81a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:35.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fde00640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f810b590 0x7f24f81a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.629+0000 7f24fde00640 1 -- 172.21.15.39:0/2780282839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f81a1c20 con 0x7f24f8105300 2024-07-04T08:17:35.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.630+0000 7f24fde00640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f8105300 0x7f24f819bcb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00e9f0 tx=0x7f24ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:35.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.630+0000 7f24f2a00640 1 -- 172.21.15.39:0/2780282839 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00cd80 con 0x7f24f8105300 2024-07-04T08:17:35.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.630+0000 7f24f2a00640 1 -- 172.21.15.39:0/2780282839 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24ec0108e0 con 0x7f24f8105300 2024-07-04T08:17:35.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.630+0000 7f2500800640 1 -- 172.21.15.39:0/2780282839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f81a1f10 con 0x7f24f8105300 2024-07-04T08:17:35.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.630+0000 7f24f2a00640 1 -- 172.21.15.39:0/2780282839 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec018980 con 0x7f24f8105300 2024-07-04T08:17:35.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.630+0000 7f2500800640 1 -- 172.21.15.39:0/2780282839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f81a8670 con 0x7f24f8105300 2024-07-04T08:17:35.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.634+0000 7f2506db4740 1 Processor -- start 2024-07-04T08:17:35.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.634+0000 7f2506db4740 1 -- start start 2024-07-04T08:17:35.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.634+0000 7f24f2a00640 1 -- 172.21.15.39:0/2780282839 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f24ec010430 con 0x7f24f8105300 2024-07-04T08:17:35.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.634+0000 7f2506db4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be56027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.634+0000 7f2506db4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5602cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.634+0000 7f2506db4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 0x555be5603230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.634+0000 7f2506db4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555be5578ee0 con 0x555be525e070 2024-07-04T08:17:35.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.634+0000 7f2506db4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555be5568480 con 0x555be52351b0 2024-07-04T08:17:35.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.634+0000 7f2506db4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555be556c4c0 con 0x555be523d390 2024-07-04T08:17:35.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be56027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5602cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be56027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54186/0 (socket says 172.21.15.39:54186) 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fe800640 1 -- 172.21.15.39:0/523723332 learned_addr learned my addr 172.21.15.39:0/523723332 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fd400640 1 --2- 172.21.15.39:0/523723332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 0x555be5603230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fde00640 1 -- 172.21.15.39:0/523723332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 msgr2=0x555be5603230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fde00640 1 --2- 172.21.15.39:0/523723332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 0x555be5603230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fde00640 1 -- 172.21.15.39:0/523723332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 msgr2=0x555be56027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fde00640 1 --2- 172.21.15.39:0/523723332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be56027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fde00640 1 -- 172.21.15.39:0/523723332 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be5500650 con 0x555be52351b0 2024-07-04T08:17:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.635+0000 7f24fe800640 1 --2- 172.21.15.39:0/523723332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be56027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.636+0000 7f24fde00640 1 --2- 172.21.15.39:0/523723332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5602cf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0e0010 tx=0x7f24ec0eb730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25501ba2c74f1cf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.636+0000 7f24f2a00640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24e00b29e0 0x7f24e00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.636+0000 7f24e7400640 1 -- 172.21.15.39:0/523723332 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec0ed850 con 0x555be52351b0 2024-07-04T08:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.636+0000 7f24e7400640 1 -- 172.21.15.39:0/523723332 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24ec0a9760 con 0x555be52351b0 2024-07-04T08:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.636+0000 7f24e7400640 1 -- 172.21.15.39:0/523723332 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec0ebe10 con 0x555be52351b0 2024-07-04T08:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.637+0000 7f24fe800640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24e00b29e0 0x7f24e00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.637+0000 7f24f2a00640 1 -- 172.21.15.39:0/2780282839 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f24ec0e2bf0 con 0x7f24f8105300 2024-07-04T08:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.637+0000 7f2506db4740 1 -- 172.21.15.39:0/523723332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 msgr2=0x555be5602cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.637+0000 7f2506db4740 1 --2- 172.21.15.39:0/523723332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5602cf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0e0010 tx=0x7f24ec0eb730 comp rx=0 tx=0).stop 2024-07-04T08:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.637+0000 7f24fe800640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24e00b29e0 0x7f24e00b4ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f24f402f500 tx=0x7f24f402f490 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.638+0000 7f2506db4740 1 -- 172.21.15.39:0/523723332 shutdown_connections 2024-07-04T08:17:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.638+0000 7f2506db4740 1 --2- 172.21.15.39:0/523723332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 0x555be5603230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.638+0000 7f2506db4740 1 --2- 172.21.15.39:0/523723332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5602cf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.638+0000 7f2506db4740 1 --2- 172.21.15.39:0/523723332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be56027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.638+0000 7f2506db4740 1 -- 172.21.15.39:0/523723332 >> 172.21.15.39:0/523723332 conn(0x555be5252460 msgr2=0x555be56006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.638+0000 7f2506db4740 1 -- 172.21.15.39:0/523723332 shutdown_connections 2024-07-04T08:17:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.638+0000 7f2506db4740 1 -- 172.21.15.39:0/523723332 wait complete. 2024-07-04T08:17:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.638+0000 7f2506db4740 1 -- start start 2024-07-04T08:17:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.639+0000 7f2506db4740 1 Processor -- start 2024-07-04T08:17:35.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f2506db4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be560e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f2506db4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 0x555be560e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f2506db4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be5622a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f2506db4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555be53cc700 con 0x555be525e070 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f2506db4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555be550fbb0 con 0x555be52351b0 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f2506db4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555be5610f90 con 0x555be523d390 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f2506db4740 10 client.0 _ll_register_callbacks cb 0x555be5561570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f24fe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be5622a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f24fde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be560e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f24fe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be5622a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54190/0 (socket says 172.21.15.39:54190) 2024-07-04T08:17:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f24fe800640 1 -- 172.21.15.39:0/416713127 learned_addr learned my addr 172.21.15.39:0/416713127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.640+0000 7f24fd400640 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 0x555be560e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.641+0000 7f24fde00640 1 -- 172.21.15.39:0/416713127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 msgr2=0x555be560e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.641+0000 7f24fde00640 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 0x555be560e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.641+0000 7f24fde00640 1 -- 172.21.15.39:0/416713127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 msgr2=0x555be5622a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.641+0000 7f24fde00640 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be5622a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.641+0000 7f24fde00640 1 -- 172.21.15.39:0/416713127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be52c9650 con 0x555be52351b0 2024-07-04T08:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.641+0000 7f24fde00640 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be560e350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0a42a0 tx=0x7f24ec0a9a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66c3e309f1cf708c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.641+0000 7f24dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f24dbe00640 1 -- 172.21.15.39:0/416713127 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec091040 con 0x555be52351b0 2024-07-04T08:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f24dbe00640 1 -- 172.21.15.39:0/416713127 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f24ec0e6250 con 0x555be52351b0 2024-07-04T08:17:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f2506db4740 1 -- 172.21.15.39:0/416713127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24ec0a9760 con 0x555be52351b0 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f24dbe00640 1 -- 172.21.15.39:0/416713127 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24ec0a5b10 con 0x555be52351b0 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f2506db4740 10 client.26033 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f2506db4740 1 -- 172.21.15.39:0/416713127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555be56257b0 con 0x555be52351b0 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f24dbe00640 1 -- 172.21.15.39:0/416713127 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24ec0a9760 con 0x555be52351b0 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f24dbe00640 1 -- 172.21.15.39:0/416713127 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f24ec0e0290 con 0x555be52351b0 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f2506db4740 10 client.26033 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.642+0000 7f2506db4740 10 client.26033 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.643+0000 7f2506db4740 20 client.26033 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.643+0000 7f2506db4740 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x555be521c9f0 0x555be5628b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.643+0000 7f2506db4740 4 client.26033 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:35.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.643+0000 7f2506db4740 1 -- 172.21.15.39:0/416713127 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555be557ae30 con 0x555be521c9f0 2024-07-04T08:17:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.643+0000 7f24fd400640 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x555be521c9f0 0x555be5628b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.644+0000 7f24dbe00640 10 client.26033 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:17:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.644+0000 7f24fd400640 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x555be521c9f0 0x555be5628b10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=74541fd22c9b6845 server_cookie=9f7937ed20ab2247 in_seq=0 out_seq=0 2024-07-04T08:17:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f24dbe00640 1 -- 172.21.15.39:0/416713127 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555be557ae30 con 0x555be521c9f0 2024-07-04T08:17:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f24dbe00640 10 client.26033 handle_command_reply: tid=0 2024-07-04T08:17:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f2506db4740 1 client.26033 shutdown 2024-07-04T08:17:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f2506db4740 1 -- 172.21.15.39:0/416713127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 msgr2=0x555be560e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f2506db4740 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be560e350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0a42a0 tx=0x7f24ec0a9a20 comp rx=0 tx=0).stop 2024-07-04T08:17:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f2506db4740 1 -- 172.21.15.39:0/416713127 shutdown_connections 2024-07-04T08:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f2506db4740 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x555be521c9f0 0x555be5628b10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f2506db4740 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be5622a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f2506db4740 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be523d390 0x555be560e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f2506db4740 1 --2- 172.21.15.39:0/416713127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be560e350 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f24dbe00640 0 client.26033 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.645+0000 7f2506db4740 1 -- 172.21.15.39:0/416713127 >> 172.21.15.39:0/416713127 conn(0x555be5252460 msgr2=0x555be5605990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.646+0000 7f2506db4740 1 -- 172.21.15.39:0/416713127 shutdown_connections 2024-07-04T08:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.646+0000 7f2506db4740 1 -- 172.21.15.39:0/416713127 wait complete. 2024-07-04T08:17:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.646+0000 7f2506db4740 20 client.26033 trim_cache size 0 max 16384 2024-07-04T08:17:35.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.663+0000 7f2506db4740 1 Processor -- start 2024-07-04T08:17:35.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.663+0000 7f2506db4740 1 -- start start 2024-07-04T08:17:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.663+0000 7f2506db4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5622a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.663+0000 7f2506db4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be523d390 0x555be55ff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.663+0000 7f2506db4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be525e070 0x555be56006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.663+0000 7f2506db4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81ae0b0 con 0x555be523d390 2024-07-04T08:17:35.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.663+0000 7f2506db4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81afbf0 con 0x555be52351b0 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.663+0000 7f2506db4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81ae780 con 0x555be525e070 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be523d390 0x555be55ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5622a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be523d390 0x555be55ff940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54196/0 (socket says 172.21.15.39:54196) 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fd400640 1 -- 172.21.15.39:0/1520100323 learned_addr learned my addr 172.21.15.39:0/1520100323 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fe800640 1 --2- 172.21.15.39:0/1520100323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be525e070 0x555be56006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fde00640 1 -- 172.21.15.39:0/1520100323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be525e070 msgr2=0x555be56006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fde00640 1 --2- 172.21.15.39:0/1520100323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be525e070 0x555be56006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fde00640 1 -- 172.21.15.39:0/1520100323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be523d390 msgr2=0x555be55ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fde00640 1 --2- 172.21.15.39:0/1520100323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be523d390 0x555be55ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.664+0000 7f24fde00640 1 -- 172.21.15.39:0/1520100323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555be5608df0 con 0x555be52351b0 2024-07-04T08:17:35.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.665+0000 7f24fde00640 1 --2- 172.21.15.39:0/1520100323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5622a80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0a9950 tx=0x7f24ec018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac48b5e8821d4c9e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:35.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.665+0000 7f24e6a00640 1 -- 172.21.15.39:0/1520100323 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec091040 con 0x555be52351b0 2024-07-04T08:17:35.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.665+0000 7f24e6a00640 1 -- 172.21.15.39:0/1520100323 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24ec0cae50 con 0x555be52351b0 2024-07-04T08:17:35.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.665+0000 7f24e6a00640 1 -- 172.21.15.39:0/1520100323 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec0ebc30 con 0x555be52351b0 2024-07-04T08:17:35.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.666+0000 7f2506db4740 1 -- 172.21.15.39:0/1520100323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 msgr2=0x555be5622a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:35.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.666+0000 7f2506db4740 1 --2- 172.21.15.39:0/1520100323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5622a80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0a9950 tx=0x7f24ec018420 comp rx=0 tx=0).stop 2024-07-04T08:17:35.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.666+0000 7f2506db4740 1 -- 172.21.15.39:0/1520100323 shutdown_connections 2024-07-04T08:17:35.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.666+0000 7f2506db4740 1 --2- 172.21.15.39:0/1520100323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be523d390 0x555be55ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.666+0000 7f2506db4740 1 --2- 172.21.15.39:0/1520100323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be525e070 0x555be56006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.666+0000 7f2506db4740 1 --2- 172.21.15.39:0/1520100323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be52351b0 0x555be5622a80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.666+0000 7f2506db4740 1 -- 172.21.15.39:0/1520100323 >> 172.21.15.39:0/1520100323 conn(0x555be5252460 msgr2=0x555be55fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.667+0000 7f2506db4740 1 -- 172.21.15.39:0/1520100323 shutdown_connections 2024-07-04T08:17:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.667+0000 7f2506db4740 1 -- 172.21.15.39:0/1520100323 wait complete. 2024-07-04T08:17:35.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.667+0000 7f2506db4740 1 -- start start 2024-07-04T08:17:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.668+0000 7f2506db4740 1 Processor -- start 2024-07-04T08:17:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.668+0000 7f2506db4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be52351b0 0x555be562c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.668+0000 7f2506db4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be523d390 0x555be5627530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.668+0000 7f2506db4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be5627a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.668+0000 7f2506db4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81af520 con 0x555be525e070 2024-07-04T08:17:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.668+0000 7f2506db4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81aee50 con 0x555be523d390 2024-07-04T08:17:35.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.668+0000 7f2506db4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81ac090 con 0x555be52351b0 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.668+0000 7f2506db4740 10 client.0 _ll_register_callbacks cb 0x7f24e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be523d390 0x555be5627530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fe800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be5627a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be523d390 0x555be5627530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59478/0 (socket says 172.21.15.39:59478) 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fd400640 1 -- 172.21.15.39:0/85795899 learned_addr learned my addr 172.21.15.39:0/85795899 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fde00640 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be52351b0 0x555be562c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fd400640 1 -- 172.21.15.39:0/85795899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be52351b0 msgr2=0x555be562c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fd400640 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be52351b0 0x555be562c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fd400640 1 -- 172.21.15.39:0/85795899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 msgr2=0x555be5627a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:35.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fd400640 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be5627a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fd400640 1 -- 172.21.15.39:0/85795899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555be5628060 con 0x555be523d390 2024-07-04T08:17:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24fd400640 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be523d390 0x555be5627530 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f24f806bc30 tx=0x7f24e8010960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a38cf7a269a12a6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.669+0000 7f24dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.670+0000 7f24dbe00640 1 -- 172.21.15.39:0/85795899 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8005220 con 0x555be523d390 2024-07-04T08:17:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.670+0000 7f24dbe00640 1 -- 172.21.15.39:0/85795899 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f24e80053c0 con 0x555be523d390 2024-07-04T08:17:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.670+0000 7f2506db4740 1 -- 172.21.15.39:0/85795899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24ec0cae50 con 0x555be523d390 2024-07-04T08:17:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.670+0000 7f24dbe00640 1 -- 172.21.15.39:0/85795899 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24e80055c0 con 0x555be523d390 2024-07-04T08:17:35.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.670+0000 7f2506db4740 10 client.26039 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.670+0000 7f2506db4740 1 -- 172.21.15.39:0/85795899 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555be5690e50 con 0x555be523d390 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.671+0000 7f24dbe00640 1 -- 172.21.15.39:0/85795899 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24ec0cae50 con 0x555be523d390 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.671+0000 7f24dbe00640 1 -- 172.21.15.39:0/85795899 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f24e8025070 con 0x555be523d390 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.671+0000 7f2506db4740 10 client.26039 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.671+0000 7f2506db4740 10 client.26039 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.671+0000 7f2506db4740 20 client.26039 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.671+0000 7f2506db4740 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x555be51f1da0 0x555be56936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.671+0000 7f2506db4740 4 client.26039 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.671+0000 7f2506db4740 1 -- 172.21.15.39:0/85795899 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555be55768d0 con 0x555be51f1da0 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.671+0000 7f24fde00640 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x555be51f1da0 0x555be56936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.672+0000 7f24fde00640 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x555be51f1da0 0x555be56936c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=630d72e803b19ce9 server_cookie=351d17e43da38f55 in_seq=0 out_seq=0 2024-07-04T08:17:35.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.672+0000 7f24dbe00640 10 client.26039 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.672+0000 7f24dbe00640 1 -- 172.21.15.39:0/85795899 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555be55768d0 con 0x555be51f1da0 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.672+0000 7f24dbe00640 10 client.26039 handle_command_reply: tid=0 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.672+0000 7f2506db4740 1 client.26039 shutdown 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.672+0000 7f2506db4740 1 -- 172.21.15.39:0/85795899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be523d390 msgr2=0x555be5627530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.672+0000 7f2506db4740 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be523d390 0x555be5627530 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f24f806bc30 tx=0x7f24e8010960 comp rx=0 tx=0).stop 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f2506db4740 1 -- 172.21.15.39:0/85795899 shutdown_connections 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f2506db4740 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x555be51f1da0 0x555be56936c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f2506db4740 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555be525e070 0x555be5627a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f2506db4740 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555be523d390 0x555be5627530 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f2506db4740 1 --2- 172.21.15.39:0/85795899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555be52351b0 0x555be562c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f24dbe00640 0 client.26039 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:17:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f2506db4740 1 -- 172.21.15.39:0/85795899 >> 172.21.15.39:0/85795899 conn(0x555be5252460 msgr2=0x555be560fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f2506db4740 1 -- 172.21.15.39:0/85795899 shutdown_connections 2024-07-04T08:17:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f2506db4740 1 -- 172.21.15.39:0/85795899 wait complete. 2024-07-04T08:17:35.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.673+0000 7f2506db4740 20 client.26039 trim_cache size 0 max 16384 2024-07-04T08:17:35.677 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:17:35.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 -- 172.21.15.39:0/2780282839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24e00b29e0 msgr2=0x7f24e00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:35.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24e00b29e0 0x7f24e00b4ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f24f402f500 tx=0x7f24f402f490 comp rx=0 tx=0).stop 2024-07-04T08:17:35.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 -- 172.21.15.39:0/2780282839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f8105300 msgr2=0x7f24f819bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:35.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f8105300 0x7f24f819bcb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00e9f0 tx=0x7f24ec00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 -- 172.21.15.39:0/2780282839 shutdown_connections 2024-07-04T08:17:35.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24e00b29e0 0x7f24e00b4ef0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24f810b590 0x7f24f81a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24f810ac40 0x7f24f819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 --2- 172.21.15.39:0/2780282839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24f8105300 0x7f24f819bcb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 -- 172.21.15.39:0/2780282839 >> 172.21.15.39:0/2780282839 conn(0x7f24f8100d10 msgr2=0x7f24f810a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 -- 172.21.15.39:0/2780282839 shutdown_connections 2024-07-04T08:17:35.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:35.674+0000 7f2500800640 1 -- 172.21.15.39:0/2780282839 wait complete. 2024-07-04T08:17:36.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:17:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:37.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:37 smithi133 ceph-mon[37050]: pgmap v552: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-07-04T08:17:37.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:37 smithi039 ceph-mon[32062]: pgmap v552: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-07-04T08:17:37.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:37 smithi050 ceph-mon[36577]: pgmap v552: 129 pgs: 129 active+clean; 1.6 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 9 op/s 2024-07-04T08:17:39.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:39 smithi039 ceph-mon[32062]: pgmap v553: 129 pgs: 129 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 KiB/s wr, 13 op/s 2024-07-04T08:17:39.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:39 smithi133 ceph-mon[37050]: pgmap v553: 129 pgs: 129 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 KiB/s wr, 13 op/s 2024-07-04T08:17:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.390+0000 7f8af4000640 1 -- 172.21.15.39:0/645577383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec108a10 msgr2=0x7f8aec108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.390+0000 7f8af4000640 1 --2- 172.21.15.39:0/645577383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec108a10 0x7f8aec108e20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009860 tx=0x7f8ad802a340 comp rx=0 tx=0).stop 2024-07-04T08:17:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.390+0000 7f8af4000640 1 -- 172.21.15.39:0/645577383 shutdown_connections 2024-07-04T08:17:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.390+0000 7f8af4000640 1 --2- 172.21.15.39:0/645577383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aec109360 0x7f8aec10fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.390+0000 7f8af4000640 1 --2- 172.21.15.39:0/645577383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec108a10 0x7f8aec108e20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.390+0000 7f8af4000640 1 --2- 172.21.15.39:0/645577383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aec1030d0 0x7f8aec1034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.390+0000 7f8af4000640 1 -- 172.21.15.39:0/645577383 >> 172.21.15.39:0/645577383 conn(0x7f8aec0feb20 msgr2=0x7f8aec100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.390+0000 7f8af4000640 1 -- 172.21.15.39:0/645577383 shutdown_connections 2024-07-04T08:17:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.390+0000 7f8af4000640 1 -- 172.21.15.39:0/645577383 wait complete. 2024-07-04T08:17:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.392+0000 7f8af4000640 1 Processor -- start 2024-07-04T08:17:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.392+0000 7f8af4000640 1 -- start start 2024-07-04T08:17:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af4000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec1030d0 0x7f8aec19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af4000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aec108a10 0x7f8aec19c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af4000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aec109360 0x7f8aec1a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af4000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec112fd0 con 0x7f8aec108a10 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af4000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec112e50 con 0x7f8aec1030d0 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af4000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec113150 con 0x7f8aec109360 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aec108a10 0x7f8aec19c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec1030d0 0x7f8aec19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aec108a10 0x7f8aec19c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54222/0 (socket says 172.21.15.39:54222) 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec1030d0 0x7f8aec19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59488/0 (socket says 172.21.15.39:59488) 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af0c00640 1 -- 172.21.15.39:0/3326485509 learned_addr learned my addr 172.21.15.39:0/3326485509 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.393+0000 7f8af2000640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aec109360 0x7f8aec1a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.396 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:39 smithi050 ceph-mon[36577]: pgmap v553: 129 pgs: 129 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 KiB/s wr, 13 op/s 2024-07-04T08:17:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.394+0000 7f8af1600640 1 -- 172.21.15.39:0/3326485509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aec109360 msgr2=0x7f8aec1a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.394+0000 7f8af1600640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aec109360 0x7f8aec1a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.394+0000 7f8af1600640 1 -- 172.21.15.39:0/3326485509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aec108a10 msgr2=0x7f8aec19c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.394+0000 7f8af1600640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aec108a10 0x7f8aec19c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.394+0000 7f8af1600640 1 -- 172.21.15.39:0/3326485509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec1a1b50 con 0x7f8aec1030d0 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.394+0000 7f8af1600640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec1030d0 0x7f8aec19bbe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00e790 tx=0x7f8adc00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.394+0000 7f8aea000640 1 -- 172.21.15.39:0/3326485509 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00eea0 con 0x7f8aec1030d0 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.394+0000 7f8aea000640 1 -- 172.21.15.39:0/3326485509 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8adc00ccc0 con 0x7f8aec1030d0 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.394+0000 7f8af4000640 1 -- 172.21.15.39:0/3326485509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec1a1e40 con 0x7f8aec1030d0 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.395+0000 7f8aea000640 1 -- 172.21.15.39:0/3326485509 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc018680 con 0x7f8aec1030d0 2024-07-04T08:17:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.395+0000 7f8af4000640 1 -- 172.21.15.39:0/3326485509 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec1a85a0 con 0x7f8aec1030d0 2024-07-04T08:17:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.398+0000 7f8afa6e1740 1 Processor -- start 2024-07-04T08:17:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.398+0000 7f8afa6e1740 1 -- start start 2024-07-04T08:17:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.398+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c472f070 0x5579c4ad37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c47061b0 0x5579c4ad3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8aea000640 1 -- 172.21.15.39:0/3326485509 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8adc010430 con 0x7f8aec1030d0 2024-07-04T08:17:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 0x5579c4ad4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5579c4a49ee0 con 0x5579c472f070 2024-07-04T08:17:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5579c4a39480 con 0x5579c47061b0 2024-07-04T08:17:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5579c4a3d4c0 con 0x5579c470e390 2024-07-04T08:17:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8af2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c472f070 0x5579c4ad37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8af1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c47061b0 0x5579c4ad3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8af2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c472f070 0x5579c4ad37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54224/0 (socket says 172.21.15.39:54224) 2024-07-04T08:17:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8af2000640 1 -- 172.21.15.39:0/3313202726 learned_addr learned my addr 172.21.15.39:0/3313202726 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8af1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c47061b0 0x5579c4ad3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59500/0 (socket says 172.21.15.39:59500) 2024-07-04T08:17:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.399+0000 7f8af0c00640 1 --2- 172.21.15.39:0/3313202726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 0x5579c4ad4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.400+0000 7f8af2000640 1 -- 172.21.15.39:0/3313202726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 msgr2=0x5579c4ad4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.400+0000 7f8af2000640 1 --2- 172.21.15.39:0/3313202726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 0x5579c4ad4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.400+0000 7f8af2000640 1 -- 172.21.15.39:0/3313202726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c47061b0 msgr2=0x5579c4ad3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.400+0000 7f8af2000640 1 --2- 172.21.15.39:0/3313202726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c47061b0 0x5579c4ad3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.400+0000 7f8af2000640 1 -- 172.21.15.39:0/3313202726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579c49d1650 con 0x5579c472f070 2024-07-04T08:17:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.400+0000 7f8af1600640 1 --2- 172.21.15.39:0/3313202726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c47061b0 0x5579c4ad3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.400+0000 7f8af2000640 1 --2- 172.21.15.39:0/3313202726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c472f070 0x5579c4ad37b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009fd0 tx=0x7f8ae40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12daff3f03eb5a34 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.400+0000 7f8ae2a00640 1 -- 172.21.15.39:0/3313202726 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4004d20 con 0x5579c472f070 2024-07-04T08:17:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.401+0000 7f8ae2a00640 1 -- 172.21.15.39:0/3313202726 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae401e070 con 0x5579c472f070 2024-07-04T08:17:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.401+0000 7f8ae2a00640 1 -- 172.21.15.39:0/3313202726 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40042b0 con 0x5579c472f070 2024-07-04T08:17:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.401+0000 7f8aea000640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ad00b29e0 0x7f8ad00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.401+0000 7f8aea000640 1 -- 172.21.15.39:0/3326485509 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8adc0e2aa0 con 0x7f8aec1030d0 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.401+0000 7f8af1600640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ad00b29e0 0x7f8ad00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.401+0000 7f8afa6e1740 1 -- 172.21.15.39:0/3313202726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c472f070 msgr2=0x5579c4ad37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.402+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/3313202726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c472f070 0x5579c4ad37b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009fd0 tx=0x7f8ae40118d0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.402+0000 7f8afa6e1740 1 -- 172.21.15.39:0/3313202726 shutdown_connections 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.402+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/3313202726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 0x5579c4ad4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.402+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/3313202726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c47061b0 0x5579c4ad3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.402+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/3313202726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c472f070 0x5579c4ad37b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.402+0000 7f8afa6e1740 1 -- 172.21.15.39:0/3313202726 >> 172.21.15.39:0/3313202726 conn(0x5579c4723460 msgr2=0x5579c4ad16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.402+0000 7f8afa6e1740 1 -- 172.21.15.39:0/3313202726 shutdown_connections 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.402+0000 7f8af1600640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ad00b29e0 0x7f8ad00b4ef0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5579c4ad2430 tx=0x7f8adc0e64a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:39.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.402+0000 7f8afa6e1740 1 -- 172.21.15.39:0/3313202726 wait complete. 2024-07-04T08:17:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.403+0000 7f8afa6e1740 1 -- start start 2024-07-04T08:17:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.403+0000 7f8afa6e1740 1 Processor -- start 2024-07-04T08:17:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.403+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 0x5579c4adf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.403+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4adf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.403+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4aeb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.403+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5579c489d700 con 0x5579c470e390 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5579c49e0bb0 con 0x5579c472f070 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5579c4ae1fe0 con 0x5579c47061b0 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8afa6e1740 10 client.0 _ll_register_callbacks cb 0x5579c4a32570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 0x5579c4adf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4adf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 0x5579c4adf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56796/0 (socket says 172.21.15.39:56796) 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af2000640 1 -- 172.21.15.39:0/700036050 learned_addr learned my addr 172.21.15.39:0/700036050 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af2000640 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4aeb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af0c00640 1 -- 172.21.15.39:0/700036050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 msgr2=0x5579c4adf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af0c00640 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 0x5579c4adf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af0c00640 1 -- 172.21.15.39:0/700036050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 msgr2=0x5579c4aeb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:39.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af0c00640 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4aeb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.404+0000 7f8af0c00640 1 -- 172.21.15.39:0/700036050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579c479a650 con 0x5579c470e390 2024-07-04T08:17:39.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.405+0000 7f8af0c00640 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4adf880 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009830 tx=0x7f8ad8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a38034e1e1fb65 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:39.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.405+0000 7f8ad7400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:17:39.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.405+0000 7f8ad7400640 1 -- 172.21.15.39:0/700036050 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8037040 con 0x5579c470e390 2024-07-04T08:17:39.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.405+0000 7f8afa6e1740 1 -- 172.21.15.39:0/700036050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae401e070 con 0x5579c470e390 2024-07-04T08:17:39.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.405+0000 7f8ad7400640 1 -- 172.21.15.39:0/700036050 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8ad8030450 con 0x5579c470e390 2024-07-04T08:17:39.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8ad7400640 1 -- 172.21.15.39:0/700036050 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ad804d050 con 0x5579c470e390 2024-07-04T08:17:39.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8afa6e1740 10 client.15936 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:39.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8afa6e1740 1 -- 172.21.15.39:0/700036050 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579c4aedff0 con 0x5579c470e390 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8ad7400640 1 -- 172.21.15.39:0/700036050 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae401e070 con 0x5579c470e390 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8ad7400640 1 -- 172.21.15.39:0/700036050 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ad803d450 con 0x5579c470e390 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8afa6e1740 10 client.15936 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8afa6e1740 10 client.15936 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8afa6e1740 20 client.15936 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5579c46ed9f0 0x5579c4af14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8afa6e1740 4 client.15936 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.406+0000 7f8afa6e1740 1 -- 172.21.15.39:0/700036050 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579c4a4be30 con 0x5579c46ed9f0 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.407+0000 7f8af2000640 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5579c46ed9f0 0x5579c4af14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.407+0000 7f8af2000640 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5579c46ed9f0 0x5579c4af14f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a729eef2c7897a4 server_cookie=afd87c7e9c4e4bd in_seq=0 out_seq=0 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.407+0000 7f8ad7400640 10 client.15936 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:17:39.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.408+0000 7f8ad7400640 1 -- 172.21.15.39:0/700036050 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5579c4a4be30 con 0x5579c46ed9f0 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.408+0000 7f8ad7400640 10 client.15936 handle_command_reply: tid=0 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 client.15936 shutdown 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 -- 172.21.15.39:0/700036050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 msgr2=0x5579c4adf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4adf880 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009830 tx=0x7f8ad8005d80 comp rx=0 tx=0).stop 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 -- 172.21.15.39:0/700036050 shutdown_connections 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5579c46ed9f0 0x5579c4af14f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4adf880 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4aeb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8ad7400640 0 client.15936 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/700036050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 0x5579c4adf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 -- 172.21.15.39:0/700036050 >> 172.21.15.39:0/700036050 conn(0x5579c4723460 msgr2=0x5579c4ad47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 -- 172.21.15.39:0/700036050 shutdown_connections 2024-07-04T08:17:39.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 1 -- 172.21.15.39:0/700036050 wait complete. 2024-07-04T08:17:39.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.409+0000 7f8afa6e1740 20 client.15936 trim_cache size 0 max 16384 2024-07-04T08:17:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.425+0000 7f8afa6e1740 1 Processor -- start 2024-07-04T08:17:39.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.425+0000 7f8afa6e1740 1 -- start start 2024-07-04T08:17:39.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.425+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 0x5579c4ad0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.425+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4ad16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.425+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4aeb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.425+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1ae0f0 con 0x5579c470e390 2024-07-04T08:17:39.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.425+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1afc30 con 0x5579c472f070 2024-07-04T08:17:39.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.425+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1ae7c0 con 0x5579c47061b0 2024-07-04T08:17:39.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4ad16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af0c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4ad16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54228/0 (socket says 172.21.15.39:54228) 2024-07-04T08:17:39.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af0c00640 1 -- 172.21.15.39:0/1682748822 learned_addr learned my addr 172.21.15.39:0/1682748822 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:39.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af2000640 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 0x5579c4ad0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af2000640 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4aeb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af0c00640 1 -- 172.21.15.39:0/1682748822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 msgr2=0x5579c4ad0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af0c00640 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 0x5579c4ad0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af0c00640 1 -- 172.21.15.39:0/1682748822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 msgr2=0x5579c4aeb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af0c00640 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4aeb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af0c00640 1 -- 172.21.15.39:0/1682748822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579c4ad5aa0 con 0x5579c470e390 2024-07-04T08:17:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.426+0000 7f8af2000640 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4aeb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.427+0000 7f8af0c00640 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4ad16f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8000c00 tx=0x7f8ad8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b23624e33895627 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:39.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.427+0000 7f8ae2000640 1 -- 172.21.15.39:0/1682748822 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8037070 con 0x5579c470e390 2024-07-04T08:17:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.427+0000 7f8ae2000640 1 -- 172.21.15.39:0/1682748822 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ad8030900 con 0x5579c470e390 2024-07-04T08:17:39.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.427+0000 7f8ae2000640 1 -- 172.21.15.39:0/1682748822 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8005290 con 0x5579c470e390 2024-07-04T08:17:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.428+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1682748822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 msgr2=0x5579c4ad16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.428+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4ad16f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8000c00 tx=0x7f8ad8001e70 comp rx=0 tx=0).stop 2024-07-04T08:17:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.428+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1682748822 shutdown_connections 2024-07-04T08:17:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.428+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c470e390 0x5579c4ad16f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.428+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4aeb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.428+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1682748822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c47061b0 0x5579c4ad0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.428+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1682748822 >> 172.21.15.39:0/1682748822 conn(0x5579c4723460 msgr2=0x5579c4ad0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.428+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1682748822 shutdown_connections 2024-07-04T08:17:39.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.428+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1682748822 wait complete. 2024-07-04T08:17:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.429+0000 7f8afa6e1740 1 -- start start 2024-07-04T08:17:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.429+0000 7f8afa6e1740 1 Processor -- start 2024-07-04T08:17:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c47061b0 0x5579c4b51eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 0x5579c4b52450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8afa6e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4ae6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1af560 con 0x5579c47061b0 2024-07-04T08:17:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1aee90 con 0x5579c472f070 2024-07-04T08:17:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8afa6e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1ac130 con 0x5579c470e390 2024-07-04T08:17:39.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8afa6e1740 10 client.0 _ll_register_callbacks cb 0x7f8ae40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8af2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c47061b0 0x5579c4b51eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8af2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c47061b0 0x5579c4b51eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54244/0 (socket says 172.21.15.39:54244) 2024-07-04T08:17:39.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8af2000640 1 -- 172.21.15.39:0/1044842684 learned_addr learned my addr 172.21.15.39:0/1044842684 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8af0c00640 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 0x5579c4b52450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.430+0000 7f8af2000640 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4ae6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.431+0000 7f8af0c00640 1 -- 172.21.15.39:0/1044842684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 msgr2=0x5579c4ae6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.431+0000 7f8af0c00640 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4ae6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.431+0000 7f8af0c00640 1 -- 172.21.15.39:0/1044842684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c47061b0 msgr2=0x5579c4b51eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.431+0000 7f8af0c00640 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c47061b0 0x5579c4b51eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.431+0000 7f8af0c00640 1 -- 172.21.15.39:0/1044842684 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579c4ae75e0 con 0x5579c470e390 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.431+0000 7f8af2000640 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c47061b0 0x5579c4b51eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.431+0000 7f8af0c00640 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 0x5579c4b52450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009830 tx=0x7f8ad803d450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a22c69559abfe6c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.431+0000 7f8ad7400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.431+0000 7f8ad7400640 1 -- 172.21.15.39:0/1044842684 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad805f060 con 0x5579c470e390 2024-07-04T08:17:39.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.432+0000 7f8ad7400640 1 -- 172.21.15.39:0/1044842684 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8ad8030650 con 0x5579c470e390 2024-07-04T08:17:39.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.432+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1044842684 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ad8030900 con 0x5579c470e390 2024-07-04T08:17:39.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.432+0000 7f8ad7400640 1 -- 172.21.15.39:0/1044842684 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ad8066050 con 0x5579c470e390 2024-07-04T08:17:39.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.432+0000 7f8afa6e1740 10 client.26023 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:39.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.432+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1044842684 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579c4b596d0 con 0x5579c470e390 2024-07-04T08:17:39.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.433+0000 7f8ad7400640 1 -- 172.21.15.39:0/1044842684 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ad8030900 con 0x5579c470e390 2024-07-04T08:17:39.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.433+0000 7f8ad7400640 1 -- 172.21.15.39:0/1044842684 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ad8004750 con 0x5579c470e390 2024-07-04T08:17:39.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.433+0000 7f8afa6e1740 10 client.26023 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:39.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.433+0000 7f8afa6e1740 10 client.26023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.433+0000 7f8afa6e1740 20 client.26023 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.433+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5579c46c2da0 0x5579c4b5bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.433+0000 7f8afa6e1740 4 client.26023 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.433+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1044842684 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579c4a478d0 con 0x5579c46c2da0 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.434+0000 7f8af2000640 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5579c46c2da0 0x5579c4b5bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.434+0000 7f8af2000640 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5579c46c2da0 0x5579c4b5bd20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1faf16c4c143c619 server_cookie=70c43a1702744b90 in_seq=0 out_seq=0 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.434+0000 7f8ad7400640 10 client.26023 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.434+0000 7f8ad7400640 1 -- 172.21.15.39:0/1044842684 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8ae401e020 con 0x5579c46c2da0 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.434+0000 7f8ad7400640 10 client.26023 handle_command_reply: tid=0 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.434+0000 7f8afa6e1740 1 client.26023 shutdown 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1044842684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 msgr2=0x5579c4b52450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 0x5579c4b52450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009830 tx=0x7f8ad803d450 comp rx=0 tx=0).stop 2024-07-04T08:17:39.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1044842684 shutdown_connections 2024-07-04T08:17:39.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5579c46c2da0 0x5579c4b5bd20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5579c472f070 0x5579c4ae6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579c470e390 0x5579c4b52450 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 --2- 172.21.15.39:0/1044842684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5579c47061b0 0x5579c4b51eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1044842684 >> 172.21.15.39:0/1044842684 conn(0x5579c4723460 msgr2=0x5579c4ae05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:39.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8ad7400640 0 client.26023 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:17:39.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1044842684 shutdown_connections 2024-07-04T08:17:39.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 1 -- 172.21.15.39:0/1044842684 wait complete. 2024-07-04T08:17:39.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.435+0000 7f8afa6e1740 20 client.26023 trim_cache size 0 max 16384 2024-07-04T08:17:39.439 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:17:39.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 -- 172.21.15.39:0/3326485509 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ad00b29e0 msgr2=0x7f8ad00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:39.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ad00b29e0 0x7f8ad00b4ef0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5579c4ad2430 tx=0x7f8adc0e64a0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 -- 172.21.15.39:0/3326485509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec1030d0 msgr2=0x7f8aec19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:39.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec1030d0 0x7f8aec19bbe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00e790 tx=0x7f8adc00ec50 comp rx=0 tx=0).stop 2024-07-04T08:17:39.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 -- 172.21.15.39:0/3326485509 shutdown_connections 2024-07-04T08:17:39.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ad00b29e0 0x7f8ad00b4ef0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aec108a10 0x7f8aec19c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aec109360 0x7f8aec1a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 --2- 172.21.15.39:0/3326485509 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8aec1030d0 0x7f8aec19bbe0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:39.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 -- 172.21.15.39:0/3326485509 >> 172.21.15.39:0/3326485509 conn(0x7f8aec0feb20 msgr2=0x7f8aec105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:39.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.436+0000 7f8af4000640 1 -- 172.21.15.39:0/3326485509 shutdown_connections 2024-07-04T08:17:39.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:39.437+0000 7f8af4000640 1 -- 172.21.15.39:0/3326485509 wait complete. 2024-07-04T08:17:40.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:17:40.935 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:41.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:41 smithi133 ceph-mon[37050]: pgmap v554: 129 pgs: 129 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-07-04T08:17:41.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:41 smithi039 ceph-mon[32062]: pgmap v554: 129 pgs: 129 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-07-04T08:17:41.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:41 smithi050 ceph-mon[36577]: pgmap v554: 129 pgs: 129 active+clean; 1.7 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2024-07-04T08:17:43.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.245+0000 7f4028e00640 1 -- 172.21.15.39:0/3366314746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4024109510 msgr2=0x7f4024109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:43.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.245+0000 7f4028e00640 1 --2- 172.21.15.39:0/3366314746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4024109510 0x7f4024109920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4014009900 tx=0x7f401402a410 comp rx=0 tx=0).stop 2024-07-04T08:17:43.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.245+0000 7f4028e00640 1 -- 172.21.15.39:0/3366314746 shutdown_connections 2024-07-04T08:17:43.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.245+0000 7f4028e00640 1 --2- 172.21.15.39:0/3366314746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402410f7a0 0x7f4024115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.245+0000 7f4028e00640 1 --2- 172.21.15.39:0/3366314746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f402410ee50 0x7f402410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.245+0000 7f4028e00640 1 --2- 172.21.15.39:0/3366314746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4024109510 0x7f4024109920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.245+0000 7f4028e00640 1 -- 172.21.15.39:0/3366314746 >> 172.21.15.39:0/3366314746 conn(0x7f4024078b10 msgr2=0x7f4024107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:43.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.246+0000 7f4028e00640 1 -- 172.21.15.39:0/3366314746 shutdown_connections 2024-07-04T08:17:43.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.246+0000 7f4028e00640 1 -- 172.21.15.39:0/3366314746 wait complete. 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.247+0000 7f4028e00640 1 Processor -- start 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.247+0000 7f4028e00640 1 -- start start 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.247+0000 7f4028e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4024109510 0x7f402419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.247+0000 7f4028e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f402410ee50 0x7f40241a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.247+0000 7f4028e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402410f7a0 0x7f40241a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.247+0000 7f4028e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4024119470 con 0x7f402410ee50 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.247+0000 7f4028e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40241192f0 con 0x7f4024109510 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.247+0000 7f4028e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40241195f0 con 0x7f402410f7a0 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4022a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f402410ee50 0x7f40241a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4022a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f402410ee50 0x7f40241a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52768/0 (socket says 172.21.15.39:52768) 2024-07-04T08:17:43.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4022a00640 1 -- 172.21.15.39:0/762784430 learned_addr learned my addr 172.21.15.39:0/762784430 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:43.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4023400640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4024109510 0x7f402419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4023e00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402410f7a0 0x7f40241a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4022a00640 1 -- 172.21.15.39:0/762784430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402410f7a0 msgr2=0x7f40241a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4022a00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402410f7a0 0x7f40241a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4022a00640 1 -- 172.21.15.39:0/762784430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4024109510 msgr2=0x7f402419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4022a00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4024109510 0x7f402419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.248+0000 7f4022a00640 1 -- 172.21.15.39:0/762784430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40241a5db0 con 0x7f402410ee50 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.249+0000 7f4022a00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f402410ee50 0x7f40241a03e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f401000e9f0 tx=0x7f401000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.249+0000 7f401be00640 1 -- 172.21.15.39:0/762784430 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401000ccc0 con 0x7f402410ee50 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.249+0000 7f401be00640 1 -- 172.21.15.39:0/762784430 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40100108e0 con 0x7f402410ee50 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.249+0000 7f4028e00640 1 -- 172.21.15.39:0/762784430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40241a60a0 con 0x7f402410ee50 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.249+0000 7f401be00640 1 -- 172.21.15.39:0/762784430 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010018860 con 0x7f402410ee50 2024-07-04T08:17:43.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.249+0000 7f4028e00640 1 -- 172.21.15.39:0/762784430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40241ac800 con 0x7f402410ee50 2024-07-04T08:17:43.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.253+0000 7f40309a4740 1 Processor -- start 2024-07-04T08:17:43.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.253+0000 7f40309a4740 1 -- start start 2024-07-04T08:17:43.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.253+0000 7f40309a4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 0x5653057ff970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f40309a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653054321b0 0x5653057ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f40309a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 0x5653058003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f40309a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x565305775f00 con 0x56530543a390 2024-07-04T08:17:43.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f40309a4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5653057654a0 con 0x56530545b070 2024-07-04T08:17:43.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f40309a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5653057694e0 con 0x5653054321b0 2024-07-04T08:17:43.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f4023e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 0x5653057ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f4023400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653054321b0 0x5653057ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f4023e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 0x5653057ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45744/0 (socket says 172.21.15.39:45744) 2024-07-04T08:17:43.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f4023400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653054321b0 0x5653057ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39324/0 (socket says 172.21.15.39:39324) 2024-07-04T08:17:43.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f4023e00640 1 -- 172.21.15.39:0/921664322 learned_addr learned my addr 172.21.15.39:0/921664322 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:43.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f401be00640 1 -- 172.21.15.39:0/762784430 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4010010430 con 0x7f402410ee50 2024-07-04T08:17:43.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f4023400640 1 -- 172.21.15.39:0/921664322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 msgr2=0x5653057ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:43.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f4023400640 1 --2- 172.21.15.39:0/921664322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 0x5653057ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.254+0000 7f4023400640 1 -- 172.21.15.39:0/921664322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 msgr2=0x5653058003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:43.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.255+0000 7f4023400640 1 --2- 172.21.15.39:0/921664322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 0x5653058003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.255+0000 7f4023400640 1 -- 172.21.15.39:0/921664322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653056fd670 con 0x5653054321b0 2024-07-04T08:17:43.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.255+0000 7f4023400640 1 --2- 172.21.15.39:0/921664322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653054321b0 0x5653057ffeb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4014009230 tx=0x7f4014032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f550515503fad2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.256+0000 7f4018c00640 1 -- 172.21.15.39:0/921664322 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014040070 con 0x5653054321b0 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.256+0000 7f4018c00640 1 -- 172.21.15.39:0/921664322 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4014004030 con 0x5653054321b0 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.256+0000 7f4018c00640 1 -- 172.21.15.39:0/921664322 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014035040 con 0x5653054321b0 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.256+0000 7f401be00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40080b29e0 0x7f40080b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f40309a4740 1 -- 172.21.15.39:0/921664322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653054321b0 msgr2=0x5653057ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f40309a4740 1 --2- 172.21.15.39:0/921664322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653054321b0 0x5653057ffeb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4014009230 tx=0x7f4014032b90 comp rx=0 tx=0).stop 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f401be00640 1 -- 172.21.15.39:0/762784430 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f40100e2bd0 con 0x7f402410ee50 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f4023e00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40080b29e0 0x7f40080b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f40309a4740 1 -- 172.21.15.39:0/921664322 shutdown_connections 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f40309a4740 1 --2- 172.21.15.39:0/921664322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 0x5653058003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f40309a4740 1 --2- 172.21.15.39:0/921664322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5653054321b0 0x5653057ffeb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f40309a4740 1 --2- 172.21.15.39:0/921664322 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 0x5653057ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f40309a4740 1 -- 172.21.15.39:0/921664322 >> 172.21.15.39:0/921664322 conn(0x56530544f460 msgr2=0x5653057fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:43.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f40309a4740 1 -- 172.21.15.39:0/921664322 shutdown_connections 2024-07-04T08:17:43.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.257+0000 7f40309a4740 1 -- 172.21.15.39:0/921664322 wait complete. 2024-07-04T08:17:43.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.258+0000 7f4023e00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40080b29e0 0x7f40080b4ef0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f401c005fd0 tx=0x7f401c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.258+0000 7f40309a4740 1 -- start start 2024-07-04T08:17:43.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.258+0000 7f40309a4740 1 Processor -- start 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f40309a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x56530580b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f40309a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530543a390 0x56530580ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f40309a4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 0x565305817410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f40309a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5653055c9700 con 0x5653054321b0 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f40309a4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56530570cbd0 con 0x56530545b070 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f40309a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56530580e0f0 con 0x56530543a390 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f40309a4740 10 client.0 _ll_register_callbacks cb 0x56530575e590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f4023400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x56530580b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f4023400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x56530580b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52782/0 (socket says 172.21.15.39:52782) 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f4023400640 1 -- 172.21.15.39:0/2265591532 learned_addr learned my addr 172.21.15.39:0/2265591532 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f4023e00640 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 0x565305817410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.259+0000 7f4022a00640 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530543a390 0x56530580ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f4023400640 1 -- 172.21.15.39:0/2265591532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530543a390 msgr2=0x56530580ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:43.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f4023400640 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530543a390 0x56530580ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f4023400640 1 -- 172.21.15.39:0/2265591532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 msgr2=0x565305817410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:43.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f4023400640 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 0x565305817410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f4023400640 1 -- 172.21.15.39:0/2265591532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653054c6650 con 0x5653054321b0 2024-07-04T08:17:43.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f4023400640 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x56530580b530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4014032fb0 tx=0x7f4014009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725131ce97f6ceb2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:43.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f400d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:17:43.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f400d600640 1 -- 172.21.15.39:0/2265591532 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014040070 con 0x5653054321b0 2024-07-04T08:17:43.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f400d600640 1 -- 172.21.15.39:0/2265591532 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4014005460 con 0x5653054321b0 2024-07-04T08:17:43.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.260+0000 7f40309a4740 1 -- 172.21.15.39:0/2265591532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4014004030 con 0x5653054321b0 2024-07-04T08:17:43.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f400d600640 1 -- 172.21.15.39:0/2265591532 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4014056050 con 0x5653054321b0 2024-07-04T08:17:43.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f40309a4740 10 client.15951 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f40309a4740 1 -- 172.21.15.39:0/2265591532 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56530581a140 con 0x5653054321b0 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f400d600640 1 -- 172.21.15.39:0/2265591532 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4014004030 con 0x5653054321b0 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f400d600640 1 -- 172.21.15.39:0/2265591532 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4014004c40 con 0x5653054321b0 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f40309a4740 10 client.15951 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f40309a4740 10 client.15951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f40309a4740 20 client.15951 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f40309a4740 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5653054199f0 0x56530581d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f40309a4740 4 client.15951 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.261+0000 7f40309a4740 1 -- 172.21.15.39:0/2265591532 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565305777e50 con 0x5653054199f0 2024-07-04T08:17:43.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.262+0000 7f4022a00640 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5653054199f0 0x56530581d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.262+0000 7f4022a00640 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5653054199f0 0x56530581d4a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d96f135431034373 server_cookie=5812e9a3a307d659 in_seq=0 out_seq=0 2024-07-04T08:17:43.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.262+0000 7f400d600640 10 client.15951 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:17:43.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.263+0000 7f400d600640 1 -- 172.21.15.39:0/2265591532 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565305777e50 con 0x5653054199f0 2024-07-04T08:17:43.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.263+0000 7f400d600640 10 client.15951 handle_command_reply: tid=0 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.263+0000 7f40309a4740 1 client.15951 shutdown 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.263+0000 7f40309a4740 1 -- 172.21.15.39:0/2265591532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 msgr2=0x56530580b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.263+0000 7f40309a4740 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x56530580b530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4014032fb0 tx=0x7f4014009260 comp rx=0 tx=0).stop 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f40309a4740 1 -- 172.21.15.39:0/2265591532 shutdown_connections 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f40309a4740 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5653054199f0 0x56530581d4a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f40309a4740 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530545b070 0x565305817410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f40309a4740 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530543a390 0x56530580ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f40309a4740 1 --2- 172.21.15.39:0/2265591532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x56530580b530 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f40309a4740 1 -- 172.21.15.39:0/2265591532 >> 172.21.15.39:0/2265591532 conn(0x56530544f460 msgr2=0x565305802b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f400d600640 0 client.15951 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f40309a4740 1 -- 172.21.15.39:0/2265591532 shutdown_connections 2024-07-04T08:17:43.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f40309a4740 1 -- 172.21.15.39:0/2265591532 wait complete. 2024-07-04T08:17:43.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.264+0000 7f40309a4740 20 client.15951 trim_cache size 0 max 16384 2024-07-04T08:17:43.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.279+0000 7f40309a4740 1 Processor -- start 2024-07-04T08:17:43.281 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:43 smithi039 ceph-mon[32062]: pgmap v555: 129 pgs: 129 active+clean; 1.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 12 op/s 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f40309a4740 1 -- start start 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f40309a4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653054321b0 0x565305805190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f40309a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 0x565305817380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f40309a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x5653058178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f40309a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b66a0 con 0x56530543a390 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f40309a4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b81e0 con 0x5653054321b0 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f40309a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b6d70 con 0x56530545b070 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f4023e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x5653058178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f4022a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 0x565305817380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.280+0000 7f4023400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653054321b0 0x565305805190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.281+0000 7f4022a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 0x565305817380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52792/0 (socket says 172.21.15.39:52792) 2024-07-04T08:17:43.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.281+0000 7f4023e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x5653058178c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39344/0 (socket says 172.21.15.39:39344) 2024-07-04T08:17:43.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.281+0000 7f4022a00640 1 -- 172.21.15.39:0/3031045956 learned_addr learned my addr 172.21.15.39:0/3031045956 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:43.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.281+0000 7f4023400640 1 -- 172.21.15.39:0/3031045956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 msgr2=0x5653058178c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:43.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.281+0000 7f4023400640 1 --2- 172.21.15.39:0/3031045956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x5653058178c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.281+0000 7f4023400640 1 -- 172.21.15.39:0/3031045956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 msgr2=0x565305817380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.281+0000 7f4023400640 1 --2- 172.21.15.39:0/3031045956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 0x565305817380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.281+0000 7f4023400640 1 -- 172.21.15.39:0/3031045956 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565305817e00 con 0x5653054321b0 2024-07-04T08:17:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.281+0000 7f4023400640 1 --2- 172.21.15.39:0/3031045956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653054321b0 0x565305805190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4014009900 tx=0x7f4014031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93c8c75c7610595f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.282+0000 7f400fe00640 1 -- 172.21.15.39:0/3031045956 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401405f070 con 0x5653054321b0 2024-07-04T08:17:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.282+0000 7f400fe00640 1 -- 172.21.15.39:0/3031045956 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f401404d070 con 0x5653054321b0 2024-07-04T08:17:43.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.282+0000 7f400fe00640 1 -- 172.21.15.39:0/3031045956 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014040040 con 0x5653054321b0 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.282+0000 7f40309a4740 1 -- 172.21.15.39:0/3031045956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653054321b0 msgr2=0x565305805190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.282+0000 7f40309a4740 1 --2- 172.21.15.39:0/3031045956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653054321b0 0x565305805190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4014009900 tx=0x7f4014031400 comp rx=0 tx=0).stop 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.283+0000 7f40309a4740 1 -- 172.21.15.39:0/3031045956 shutdown_connections 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.283+0000 7f40309a4740 1 --2- 172.21.15.39:0/3031045956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56530543a390 0x565305817380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.283+0000 7f40309a4740 1 --2- 172.21.15.39:0/3031045956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x5653058178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.283+0000 7f40309a4740 1 --2- 172.21.15.39:0/3031045956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653054321b0 0x565305805190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.283+0000 7f40309a4740 1 -- 172.21.15.39:0/3031045956 >> 172.21.15.39:0/3031045956 conn(0x56530544f460 msgr2=0x565305804a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.283+0000 7f40309a4740 1 -- 172.21.15.39:0/3031045956 shutdown_connections 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.283+0000 7f40309a4740 1 -- 172.21.15.39:0/3031045956 wait complete. 2024-07-04T08:17:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.283+0000 7f40309a4740 1 -- start start 2024-07-04T08:17:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f40309a4740 1 Processor -- start 2024-07-04T08:17:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f40309a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x565305820cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f40309a4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530543a390 0x56530581e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f40309a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x56530581e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f40309a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b7b10 con 0x5653054321b0 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f40309a4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b7440 con 0x56530543a390 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f40309a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40241b4680 con 0x56530545b070 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f40309a4740 10 client.0 _ll_register_callbacks cb 0x7f401c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f4022a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530543a390 0x56530581e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f4022a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530543a390 0x56530581e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45780/0 (socket says 172.21.15.39:45780) 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f4022a00640 1 -- 172.21.15.39:0/3945470084 learned_addr learned my addr 172.21.15.39:0/3945470084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.284+0000 7f4023400640 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x565305820cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f4023e00640 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x56530581e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f4022a00640 1 -- 172.21.15.39:0/3945470084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 msgr2=0x56530581e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f4022a00640 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x56530581e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f4022a00640 1 -- 172.21.15.39:0/3945470084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 msgr2=0x565305820cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f4022a00640 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x565305820cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f4022a00640 1 -- 172.21.15.39:0/3945470084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56530581f000 con 0x56530543a390 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f4023e00640 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x56530581e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f4023400640 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x565305820cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f4022a00640 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530543a390 0x56530581e1f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40100a93f0 tx=0x7f4010009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=834c762a94943a02 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.285+0000 7f400d600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.286+0000 7f400d600640 1 -- 172.21.15.39:0/3945470084 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40100a96e0 con 0x56530543a390 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.286+0000 7f40309a4740 1 -- 172.21.15.39:0/3945470084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f401404d070 con 0x56530543a390 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.286+0000 7f400d600640 1 -- 172.21.15.39:0/3945470084 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f40100e0210 con 0x56530543a390 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.286+0000 7f400d600640 1 -- 172.21.15.39:0/3945470084 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40100e0410 con 0x56530543a390 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.286+0000 7f40309a4740 10 client.26075 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.286+0000 7f40309a4740 1 -- 172.21.15.39:0/3945470084 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565305885a50 con 0x56530543a390 2024-07-04T08:17:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.286+0000 7f400d600640 1 -- 172.21.15.39:0/3945470084 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f401404d070 con 0x56530543a390 2024-07-04T08:17:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.287+0000 7f400d600640 1 -- 172.21.15.39:0/3945470084 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f40100a5040 con 0x56530543a390 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.287+0000 7f40309a4740 10 client.26075 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.287+0000 7f40309a4740 10 client.26075 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.287+0000 7f40309a4740 20 client.26075 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.287+0000 7f40309a4740 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5653053eeda0 0x565305888140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.287+0000 7f40309a4740 4 client.26075 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.287+0000 7f40309a4740 1 -- 172.21.15.39:0/3945470084 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653057738f0 con 0x5653053eeda0 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.287+0000 7f4023400640 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5653053eeda0 0x565305888140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.288+0000 7f4023400640 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5653053eeda0 0x565305888140 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b3f589e0aa937be server_cookie=e53a26e484b9e8d1 in_seq=0 out_seq=0 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.288+0000 7f400d600640 10 client.26075 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:17:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.288+0000 7f400d600640 1 -- 172.21.15.39:0/3945470084 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653057738f0 con 0x5653053eeda0 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.288+0000 7f400d600640 10 client.26075 handle_command_reply: tid=0 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.288+0000 7f40309a4740 1 client.26075 shutdown 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.288+0000 7f40309a4740 1 -- 172.21.15.39:0/3945470084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530543a390 msgr2=0x56530581e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.288+0000 7f40309a4740 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530543a390 0x56530581e1f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40100a93f0 tx=0x7f4010009740 comp rx=0 tx=0).stop 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f40309a4740 1 -- 172.21.15.39:0/3945470084 shutdown_connections 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f40309a4740 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5653053eeda0 0x565305888140 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f40309a4740 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56530545b070 0x56530581e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f40309a4740 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56530543a390 0x56530581e1f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f40309a4740 1 --2- 172.21.15.39:0/3945470084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5653054321b0 0x565305820cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f400d600640 0 client.26075 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f40309a4740 1 -- 172.21.15.39:0/3945470084 >> 172.21.15.39:0/3945470084 conn(0x56530544f460 msgr2=0x56530580c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f40309a4740 1 -- 172.21.15.39:0/3945470084 shutdown_connections 2024-07-04T08:17:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f40309a4740 1 -- 172.21.15.39:0/3945470084 wait complete. 2024-07-04T08:17:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.289+0000 7f40309a4740 20 client.26075 trim_cache size 0 max 16384 2024-07-04T08:17:43.296 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:17:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 -- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40080b29e0 msgr2=0x7f40080b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40080b29e0 0x7f40080b4ef0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f401c005fd0 tx=0x7f401c009500 comp rx=0 tx=0).stop 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 -- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f402410ee50 msgr2=0x7f40241a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f402410ee50 0x7f40241a03e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f401000e9f0 tx=0x7f401000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 -- 172.21.15.39:0/762784430 shutdown_connections 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40080b29e0 0x7f40080b4ef0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f402410ee50 0x7f40241a03e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402410f7a0 0x7f40241a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 --2- 172.21.15.39:0/762784430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4024109510 0x7f402419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 -- 172.21.15.39:0/762784430 >> 172.21.15.39:0/762784430 conn(0x7f4024078b10 msgr2=0x7f4024107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 -- 172.21.15.39:0/762784430 shutdown_connections 2024-07-04T08:17:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:43.290+0000 7f4028e00640 1 -- 172.21.15.39:0/762784430 wait complete. 2024-07-04T08:17:43.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:43 smithi050 ceph-mon[36577]: pgmap v555: 129 pgs: 129 active+clean; 1.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 12 op/s 2024-07-04T08:17:43.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:43 smithi133 ceph-mon[37050]: pgmap v555: 129 pgs: 129 active+clean; 1.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 12 op/s 2024-07-04T08:17:44.175 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:17:44.675 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:45.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:45 smithi039 ceph-mon[32062]: pgmap v556: 129 pgs: 129 active+clean; 1.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-07-04T08:17:45.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:45 smithi050 ceph-mon[36577]: pgmap v556: 129 pgs: 129 active+clean; 1.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-07-04T08:17:45.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:45 smithi133 ceph-mon[37050]: pgmap v556: 129 pgs: 129 active+clean; 1.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-07-04T08:17:46.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.918+0000 7fef8bc00640 1 -- 172.21.15.39:0/2282900051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef84105300 msgr2=0x7fef84105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:46.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.918+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2282900051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef84105300 0x7fef84105710 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fef78009860 tx=0x7fef7802a340 comp rx=0 tx=0).stop 2024-07-04T08:17:46.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.918+0000 7fef8bc00640 1 -- 172.21.15.39:0/2282900051 shutdown_connections 2024-07-04T08:17:46.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.918+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2282900051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef8410ac40 0x7fef8410b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.918+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2282900051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef8410b590 0x7fef84111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.918+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2282900051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef84105300 0x7fef84105710 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.918+0000 7fef8bc00640 1 -- 172.21.15.39:0/2282900051 >> 172.21.15.39:0/2282900051 conn(0x7fef84100d10 msgr2=0x7fef84103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:46.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.918+0000 7fef8bc00640 1 -- 172.21.15.39:0/2282900051 shutdown_connections 2024-07-04T08:17:46.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.918+0000 7fef8bc00640 1 -- 172.21.15.39:0/2282900051 wait complete. 2024-07-04T08:17:46.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.919+0000 7fef8bc00640 1 Processor -- start 2024-07-04T08:17:46.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.920+0000 7fef8bc00640 1 -- start start 2024-07-04T08:17:46.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.920+0000 7fef8bc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef84105300 0x7fef8419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.920+0000 7fef8bc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef8410ac40 0x7fef8419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.920+0000 7fef8bc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef8410b590 0x7fef841a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.920+0000 7fef8bc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef841152b0 con 0x7fef84105300 2024-07-04T08:17:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.920+0000 7fef8bc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef84115130 con 0x7fef8410b590 2024-07-04T08:17:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.920+0000 7fef8bc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fef84115430 con 0x7fef8410ac40 2024-07-04T08:17:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.920+0000 7fef89200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef84105300 0x7fef8419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.921+0000 7fef89200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef84105300 0x7fef8419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52810/0 (socket says 172.21.15.39:52810) 2024-07-04T08:17:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.921+0000 7fef89200640 1 -- 172.21.15.39:0/2236133614 learned_addr learned my addr 172.21.15.39:0/2236133614 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.921+0000 7fef83e00640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef8410ac40 0x7fef8419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.921+0000 7fef89c00640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef8410b590 0x7fef841a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.921+0000 7fef89200640 1 -- 172.21.15.39:0/2236133614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef8410ac40 msgr2=0x7fef8419c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.921+0000 7fef89200640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef8410ac40 0x7fef8419c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.921+0000 7fef89200640 1 -- 172.21.15.39:0/2236133614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef8410b590 msgr2=0x7fef841a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.921+0000 7fef89200640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef8410b590 0x7fef841a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.921+0000 7fef89200640 1 -- 172.21.15.39:0/2236133614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef841a1c30 con 0x7fef84105300 2024-07-04T08:17:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.922+0000 7fef83e00640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef8410ac40 0x7fef8419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.922+0000 7fef89200640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef84105300 0x7fef8419bcc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fef780303f0 tx=0x7fef78032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.922+0000 7fef81600640 1 -- 172.21.15.39:0/2236133614 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78040020 con 0x7fef84105300 2024-07-04T08:17:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.922+0000 7fef81600640 1 -- 172.21.15.39:0/2236133614 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef78030450 con 0x7fef84105300 2024-07-04T08:17:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.923+0000 7fef8bc00640 1 -- 172.21.15.39:0/2236133614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef841a1ec0 con 0x7fef84105300 2024-07-04T08:17:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.923+0000 7fef81600640 1 -- 172.21.15.39:0/2236133614 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78037040 con 0x7fef84105300 2024-07-04T08:17:46.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.923+0000 7fef8bc00640 1 -- 172.21.15.39:0/2236133614 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef841a8680 con 0x7fef84105300 2024-07-04T08:17:46.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.927+0000 7fef92293740 1 Processor -- start 2024-07-04T08:17:46.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.928+0000 7fef92293740 1 -- start start 2024-07-04T08:17:46.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.928+0000 7fef92293740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.928+0000 7fef92293740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac21d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.928+0000 7fef92293740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac22250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.928+0000 7fef92293740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bc9ab97ee0 con 0x55bc9a85c390 2024-07-04T08:17:46.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.928+0000 7fef92293740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bc9ab87480 con 0x55bc9a87d070 2024-07-04T08:17:46.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.928+0000 7fef92293740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bc9ab8b4c0 con 0x55bc9a8541b0 2024-07-04T08:17:46.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef81600640 1 -- 172.21.15.39:0/2236133614 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fef78043020 con 0x7fef84105300 2024-07-04T08:17:46.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45802/0 (socket says 172.21.15.39:45802) 2024-07-04T08:17:46.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89c00640 1 -- 172.21.15.39:0/3851474002 learned_addr learned my addr 172.21.15.39:0/3851474002 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:46.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89200640 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac21d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef83e00640 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac22250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89c00640 1 -- 172.21.15.39:0/3851474002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 msgr2=0x55bc9ac21d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:46.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89c00640 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac21d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89c00640 1 -- 172.21.15.39:0/3851474002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 msgr2=0x55bc9ac22250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:46.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89c00640 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac22250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89c00640 1 -- 172.21.15.39:0/3851474002 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc9ab1f650 con 0x55bc9a87d070 2024-07-04T08:17:46.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.929+0000 7fef89200640 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac21d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:46.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.930+0000 7fef89c00640 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac217d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00a430 tx=0x7fef7c009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce8eebbcd830be7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:46.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.930+0000 7fef72000640 1 -- 172.21.15.39:0/3851474002 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c013790 con 0x55bc9a87d070 2024-07-04T08:17:46.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.930+0000 7fef72000640 1 -- 172.21.15.39:0/3851474002 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c019070 con 0x55bc9a87d070 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.930+0000 7fef72000640 1 -- 172.21.15.39:0/3851474002 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c014ce0 con 0x55bc9a87d070 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.930+0000 7fef81600640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef6c0b2960 0x7fef6c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.931+0000 7fef81600640 1 -- 172.21.15.39:0/2236133614 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fef78102950 con 0x7fef84105300 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.931+0000 7fef89200640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef6c0b2960 0x7fef6c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.931+0000 7fef92293740 1 -- 172.21.15.39:0/3851474002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 msgr2=0x55bc9ac217d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.931+0000 7fef92293740 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac217d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00a430 tx=0x7fef7c009650 comp rx=0 tx=0).stop 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.932+0000 7fef92293740 1 -- 172.21.15.39:0/3851474002 shutdown_connections 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.932+0000 7fef92293740 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac22250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.932+0000 7fef92293740 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac21d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.932+0000 7fef92293740 1 --2- 172.21.15.39:0/3851474002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac217d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.932+0000 7fef92293740 1 -- 172.21.15.39:0/3851474002 >> 172.21.15.39:0/3851474002 conn(0x55bc9a871460 msgr2=0x55bc9ac1f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.932+0000 7fef89200640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef6c0b2960 0x7fef6c0b4e70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fef780fff30 tx=0x7fef7802a340 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.932+0000 7fef92293740 1 -- 172.21.15.39:0/3851474002 shutdown_connections 2024-07-04T08:17:46.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.932+0000 7fef92293740 1 -- 172.21.15.39:0/3851474002 wait complete. 2024-07-04T08:17:46.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.932+0000 7fef92293740 1 -- start start 2024-07-04T08:17:46.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.933+0000 7fef92293740 1 Processor -- start 2024-07-04T08:17:46.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.933+0000 7fef92293740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a8541b0 0x55bc9ac2d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.933+0000 7fef92293740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac2d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.933+0000 7fef92293740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a87d070 0x55bc9ac392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.933+0000 7fef92293740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bc9a9eb700 con 0x55bc9a85c390 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.933+0000 7fef92293740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bc9ab2ebb0 con 0x55bc9a8541b0 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.933+0000 7fef92293740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bc9ac2ffb0 con 0x55bc9a87d070 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.933+0000 7fef92293740 10 client.0 _ll_register_callbacks cb 0x55bc9ab80570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef83e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac2d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef89c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a8541b0 0x55bc9ac2d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef83e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac2d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52836/0 (socket says 172.21.15.39:52836) 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef83e00640 1 -- 172.21.15.39:0/2469374053 learned_addr learned my addr 172.21.15.39:0/2469374053 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef89c00640 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a87d070 0x55bc9ac392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef83e00640 1 -- 172.21.15.39:0/2469374053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a87d070 msgr2=0x55bc9ac392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef83e00640 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a87d070 0x55bc9ac392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef83e00640 1 -- 172.21.15.39:0/2469374053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a8541b0 msgr2=0x55bc9ac2d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef83e00640 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a8541b0 0x55bc9ac2d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef83e00640 1 -- 172.21.15.39:0/2469374053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc9a8e8650 con 0x55bc9a85c390 2024-07-04T08:17:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef83e00640 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac2d910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fef74011960 tx=0x7fef74011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c362cc46092e37c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.934+0000 7fef66a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:17:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.935+0000 7fef66a00640 1 -- 172.21.15.39:0/2469374053 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7400fbf0 con 0x55bc9a85c390 2024-07-04T08:17:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.935+0000 7fef92293740 1 -- 172.21.15.39:0/2469374053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7c019070 con 0x55bc9a85c390 2024-07-04T08:17:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.935+0000 7fef66a00640 1 -- 172.21.15.39:0/2469374053 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fef7400fd90 con 0x55bc9a85c390 2024-07-04T08:17:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.935+0000 7fef66a00640 1 -- 172.21.15.39:0/2469374053 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fef74027050 con 0x55bc9a85c390 2024-07-04T08:17:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.935+0000 7fef92293740 10 client.15969 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:46.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.935+0000 7fef92293740 1 -- 172.21.15.39:0/2469374053 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc9ac3c0a0 con 0x55bc9a85c390 2024-07-04T08:17:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.936+0000 7fef66a00640 1 -- 172.21.15.39:0/2469374053 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c019070 con 0x55bc9a85c390 2024-07-04T08:17:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.936+0000 7fef66a00640 1 -- 172.21.15.39:0/2469374053 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fef74022030 con 0x55bc9a85c390 2024-07-04T08:17:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.936+0000 7fef92293740 10 client.15969 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.936+0000 7fef92293740 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.936+0000 7fef92293740 20 client.15969 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.936+0000 7fef92293740 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bc9a83b9f0 0x55bc9ac3f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.936+0000 7fef92293740 4 client.15969 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.936+0000 7fef92293740 1 -- 172.21.15.39:0/2469374053 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc9ab99e30 con 0x55bc9a83b9f0 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.936+0000 7fef89c00640 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bc9a83b9f0 0x55bc9ac3f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.937+0000 7fef89c00640 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bc9a83b9f0 0x55bc9ac3f4e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b34cd9682792737 server_cookie=2999328f6b28b9f3 in_seq=0 out_seq=0 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.937+0000 7fef66a00640 10 client.15969 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef66a00640 1 -- 172.21.15.39:0/2469374053 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bc9ab99e30 con 0x55bc9a83b9f0 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef66a00640 10 client.15969 handle_command_reply: tid=0 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef92293740 1 client.15969 shutdown 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef92293740 1 -- 172.21.15.39:0/2469374053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 msgr2=0x55bc9ac2d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef92293740 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac2d910 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fef74011960 tx=0x7fef74011e20 comp rx=0 tx=0).stop 2024-07-04T08:17:46.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef92293740 1 -- 172.21.15.39:0/2469374053 shutdown_connections 2024-07-04T08:17:46.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef92293740 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bc9a83b9f0 0x55bc9ac3f4e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef92293740 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac2d910 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef92293740 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a87d070 0x55bc9ac392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef92293740 1 --2- 172.21.15.39:0/2469374053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a8541b0 0x55bc9ac2d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef66a00640 0 client.15969 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:46.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.938+0000 7fef92293740 1 -- 172.21.15.39:0/2469374053 >> 172.21.15.39:0/2469374053 conn(0x55bc9a871460 msgr2=0x55bc9ac249b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:46.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.939+0000 7fef92293740 1 -- 172.21.15.39:0/2469374053 shutdown_connections 2024-07-04T08:17:46.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.939+0000 7fef92293740 1 -- 172.21.15.39:0/2469374053 wait complete. 2024-07-04T08:17:46.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.939+0000 7fef92293740 20 client.15969 trim_cache size 0 max 16384 2024-07-04T08:17:46.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.954+0000 7fef92293740 1 Processor -- start 2024-07-04T08:17:46.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.954+0000 7fef92293740 1 -- start start 2024-07-04T08:17:46.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef92293740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac1e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef92293740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac1f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef92293740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef92293740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b2530 con 0x55bc9a85c390 2024-07-04T08:17:46.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef92293740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b4070 con 0x55bc9a87d070 2024-07-04T08:17:46.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef92293740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b2c00 con 0x55bc9a8541b0 2024-07-04T08:17:46.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef83e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac1f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef83e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac1f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52840/0 (socket says 172.21.15.39:52840) 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef83e00640 1 -- 172.21.15.39:0/914633504 learned_addr learned my addr 172.21.15.39:0/914633504 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef89c00640 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac1e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.955+0000 7fef89c00640 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.956+0000 7fef89c00640 1 -- 172.21.15.39:0/914633504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 msgr2=0x55bc9ac268b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.956+0000 7fef89c00640 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac268b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.956+0000 7fef89c00640 1 -- 172.21.15.39:0/914633504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 msgr2=0x55bc9ac1f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.956+0000 7fef89c00640 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac1f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.956+0000 7fef89c00640 1 -- 172.21.15.39:0/914633504 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc9ac2e470 con 0x55bc9a8541b0 2024-07-04T08:17:46.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.956+0000 7fef83e00640 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac1f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:17:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.956+0000 7fef89c00640 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac1e9d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0149c0 tx=0x7fef7c022040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579f0dff38252ad5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.956+0000 7fef71600640 1 -- 172.21.15.39:0/914633504 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c013850 con 0x55bc9a8541b0 2024-07-04T08:17:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.956+0000 7fef71600640 1 -- 172.21.15.39:0/914633504 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c019070 con 0x55bc9a8541b0 2024-07-04T08:17:46.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.957+0000 7fef71600640 1 -- 172.21.15.39:0/914633504 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c02c710 con 0x55bc9a8541b0 2024-07-04T08:17:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.957+0000 7fef92293740 1 -- 172.21.15.39:0/914633504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 msgr2=0x55bc9ac1e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.957+0000 7fef92293740 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac1e9d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0149c0 tx=0x7fef7c022040 comp rx=0 tx=0).stop 2024-07-04T08:17:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.957+0000 7fef92293740 1 -- 172.21.15.39:0/914633504 shutdown_connections 2024-07-04T08:17:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.957+0000 7fef92293740 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a85c390 0x55bc9ac1f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.957+0000 7fef92293740 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a87d070 0x55bc9ac268b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.957+0000 7fef92293740 1 --2- 172.21.15.39:0/914633504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac1e9d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.957+0000 7fef92293740 1 -- 172.21.15.39:0/914633504 >> 172.21.15.39:0/914633504 conn(0x55bc9a871460 msgr2=0x55bc9ac1efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.958+0000 7fef92293740 1 -- 172.21.15.39:0/914633504 shutdown_connections 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.958+0000 7fef92293740 1 -- 172.21.15.39:0/914633504 wait complete. 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.958+0000 7fef92293740 1 -- start start 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.958+0000 7fef92293740 1 Processor -- start 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef92293740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef92293740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a85c390 0x55bc9ac38ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef92293740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a87d070 0x55bc9ac392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef92293740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b39a0 con 0x55bc9a87d070 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef92293740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b32d0 con 0x55bc9a85c390 2024-07-04T08:17:46.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef92293740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b0510 con 0x55bc9a8541b0 2024-07-04T08:17:46.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef92293740 10 client.0 _ll_register_callbacks cb 0x7fef74009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:46.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef89c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef83e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a85c390 0x55bc9ac38ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef89c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac359f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39410/0 (socket says 172.21.15.39:39410) 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.959+0000 7fef89c00640 1 -- 172.21.15.39:0/4180421118 learned_addr learned my addr 172.21.15.39:0/4180421118 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.960+0000 7fef89c00640 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a87d070 0x55bc9ac392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.960+0000 7fef83e00640 1 -- 172.21.15.39:0/4180421118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 msgr2=0x55bc9ac359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.960+0000 7fef83e00640 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.960+0000 7fef83e00640 1 -- 172.21.15.39:0/4180421118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a87d070 msgr2=0x55bc9ac392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.960+0000 7fef83e00640 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a87d070 0x55bc9ac392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.960+0000 7fef83e00640 1 -- 172.21.15.39:0/4180421118 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc9ac39a80 con 0x55bc9a85c390 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.960+0000 7fef83e00640 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a85c390 0x55bc9ac38ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef74019530 tx=0x7fef74005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be82db5608137b1d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.960+0000 7fef66a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:46.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.961+0000 7fef66a00640 1 -- 172.21.15.39:0/4180421118 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7402f070 con 0x55bc9a85c390 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.961+0000 7fef92293740 1 -- 172.21.15.39:0/4180421118 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef7c019070 con 0x55bc9a85c390 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.961+0000 7fef66a00640 1 -- 172.21.15.39:0/4180421118 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fef7400fbf0 con 0x55bc9a85c390 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.961+0000 7fef66a00640 1 -- 172.21.15.39:0/4180421118 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fef7403b050 con 0x55bc9a85c390 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.961+0000 7fef92293740 10 client.26093 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.961+0000 7fef92293740 1 -- 172.21.15.39:0/4180421118 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc9aca7d60 con 0x55bc9a85c390 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.962+0000 7fef66a00640 1 -- 172.21.15.39:0/4180421118 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c019070 con 0x55bc9a85c390 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.962+0000 7fef66a00640 1 -- 172.21.15.39:0/4180421118 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fef74022040 con 0x55bc9a85c390 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.962+0000 7fef92293740 10 client.26093 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.962+0000 7fef92293740 10 client.26093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.962+0000 7fef92293740 20 client.26093 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.962+0000 7fef92293740 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bc9a810da0 0x55bc9acaa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.962+0000 7fef92293740 4 client.26093 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:17:46.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.962+0000 7fef92293740 1 -- 172.21.15.39:0/4180421118 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc9ac28d60 con 0x55bc9a810da0 2024-07-04T08:17:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.962+0000 7fef89c00640 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bc9a810da0 0x55bc9acaa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.963+0000 7fef89c00640 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bc9a810da0 0x55bc9acaa520 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc1bc735371efd0 server_cookie=e58f09948d438008 in_seq=0 out_seq=0 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.963+0000 7fef66a00640 10 client.26093 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.963+0000 7fef66a00640 1 -- 172.21.15.39:0/4180421118 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc9ac28d60 con 0x55bc9a810da0 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.963+0000 7fef66a00640 10 client.26093 handle_command_reply: tid=0 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.963+0000 7fef92293740 1 client.26093 shutdown 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 -- 172.21.15.39:0/4180421118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a85c390 msgr2=0x55bc9ac38ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a85c390 0x55bc9ac38ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef74019530 tx=0x7fef74005700 comp rx=0 tx=0).stop 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 -- 172.21.15.39:0/4180421118 shutdown_connections 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bc9a810da0 0x55bc9acaa520 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bc9a87d070 0x55bc9ac392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bc9a85c390 0x55bc9ac38ca0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 --2- 172.21.15.39:0/4180421118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bc9a8541b0 0x55bc9ac359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef66a00640 0 client.26093 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 -- 172.21.15.39:0/4180421118 >> 172.21.15.39:0/4180421118 conn(0x55bc9a871460 msgr2=0x55bc9ac22800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 -- 172.21.15.39:0/4180421118 shutdown_connections 2024-07-04T08:17:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 1 -- 172.21.15.39:0/4180421118 wait complete. 2024-07-04T08:17:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.964+0000 7fef92293740 20 client.26093 trim_cache size 0 max 16384 2024-07-04T08:17:46.969 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 -- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef6c0b2960 msgr2=0x7fef6c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef6c0b2960 0x7fef6c0b4e70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fef780fff30 tx=0x7fef7802a340 comp rx=0 tx=0).stop 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 -- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef84105300 msgr2=0x7fef8419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef84105300 0x7fef8419bcc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fef780303f0 tx=0x7fef78032eb0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 -- 172.21.15.39:0/2236133614 shutdown_connections 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fef6c0b2960 0x7fef6c0b4e70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef8410b590 0x7fef841a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef8410ac40 0x7fef8419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 --2- 172.21.15.39:0/2236133614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fef84105300 0x7fef8419bcc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 -- 172.21.15.39:0/2236133614 >> 172.21.15.39:0/2236133614 conn(0x7fef84100d10 msgr2=0x7fef84109410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 -- 172.21.15.39:0/2236133614 shutdown_connections 2024-07-04T08:17:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:46.965+0000 7fef8bc00640 1 -- 172.21.15.39:0/2236133614 wait complete. 2024-07-04T08:17:47.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:47 smithi039 ceph-mon[32062]: pgmap v557: 129 pgs: 129 active+clean; 1.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-07-04T08:17:47.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:47 smithi050 ceph-mon[36577]: pgmap v557: 129 pgs: 129 active+clean; 1.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-07-04T08:17:47.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:47 smithi133 ceph-mon[37050]: pgmap v557: 129 pgs: 129 active+clean; 1.8 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-07-04T08:17:48.896 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:17:48.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:17:49.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:49 smithi039 ceph-mon[32062]: pgmap v558: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-07-04T08:17:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:49.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:49 smithi050 ceph-mon[36577]: pgmap v558: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-07-04T08:17:49.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:49 smithi133 ceph-mon[37050]: pgmap v558: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-07-04T08:17:51.365 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:51 smithi039 ceph-mon[32062]: pgmap v559: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-07-04T08:17:51.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:51 smithi050 ceph-mon[36577]: pgmap v559: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-07-04T08:17:51.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:51 smithi133 ceph-mon[37050]: pgmap v559: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-07-04T08:17:51.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.717+0000 7f61a5600640 1 -- 172.21.15.39:0/3775734178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a0074ca0 msgr2=0x7f61a00750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:51.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.717+0000 7f61a5600640 1 --2- 172.21.15.39:0/3775734178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a0074ca0 0x7f61a00750b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6190009900 tx=0x7f619002a410 comp rx=0 tx=0).stop 2024-07-04T08:17:51.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.717+0000 7f61a5600640 1 -- 172.21.15.39:0/3775734178 shutdown_connections 2024-07-04T08:17:51.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.717+0000 7f61a5600640 1 --2- 172.21.15.39:0/3775734178 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a00755f0 0x7f61a010f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.717+0000 7f61a5600640 1 --2- 172.21.15.39:0/3775734178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a010f670 0x7f61a0111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.717+0000 7f61a5600640 1 --2- 172.21.15.39:0/3775734178 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a0074ca0 0x7f61a00750b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.717+0000 7f61a5600640 1 -- 172.21.15.39:0/3775734178 >> 172.21.15.39:0/3775734178 conn(0x7f61a007ada0 msgr2=0x7f61a007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:51.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.717+0000 7f61a5600640 1 -- 172.21.15.39:0/3775734178 shutdown_connections 2024-07-04T08:17:51.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.717+0000 7f61a5600640 1 -- 172.21.15.39:0/3775734178 wait complete. 2024-07-04T08:17:51.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.718+0000 7f61a5600640 1 Processor -- start 2024-07-04T08:17:51.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.718+0000 7f61a5600640 1 -- start start 2024-07-04T08:17:51.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f61a5600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a0074ca0 0x7f61a019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f61a5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a00755f0 0x7f61a019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f61a5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a010f670 0x7f61a01a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f61a5600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61a0115240 con 0x7f61a010f670 2024-07-04T08:17:51.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f61a5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61a01150c0 con 0x7f61a00755f0 2024-07-04T08:17:51.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f61a5600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61a01153c0 con 0x7f61a0074ca0 2024-07-04T08:17:51.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f6197e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a00755f0 0x7f61a019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f6197e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a00755f0 0x7f61a019c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54302/0 (socket says 172.21.15.39:54302) 2024-07-04T08:17:51.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f6197e00640 1 -- 172.21.15.39:0/3473481545 learned_addr learned my addr 172.21.15.39:0/3473481545 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:51.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f619f400640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a010f670 0x7f61a01a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.719+0000 7f619ea00640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a0074ca0 0x7f61a019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.720+0000 7f6197e00640 1 -- 172.21.15.39:0/3473481545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a0074ca0 msgr2=0x7f61a019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:51.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.720+0000 7f6197e00640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a0074ca0 0x7f61a019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.720+0000 7f6197e00640 1 -- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a010f670 msgr2=0x7f61a01a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:51.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.720+0000 7f6197e00640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a010f670 0x7f61a01a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.720+0000 7f6197e00640 1 -- 172.21.15.39:0/3473481545 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a01a1c00 con 0x7f61a00755f0 2024-07-04T08:17:51.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.720+0000 7f6197e00640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a00755f0 0x7f61a019c230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f618800e9f0 tx=0x7f618800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:51.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.721+0000 7f6197400640 1 -- 172.21.15.39:0/3473481545 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618800ccc0 con 0x7f61a00755f0 2024-07-04T08:17:51.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.721+0000 7f61a5600640 1 -- 172.21.15.39:0/3473481545 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a01a1ef0 con 0x7f61a00755f0 2024-07-04T08:17:51.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.721+0000 7f6197400640 1 -- 172.21.15.39:0/3473481545 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880108e0 con 0x7f61a00755f0 2024-07-04T08:17:51.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.721+0000 7f6197400640 1 -- 172.21.15.39:0/3473481545 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188018860 con 0x7f61a00755f0 2024-07-04T08:17:51.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.721+0000 7f61a5600640 1 -- 172.21.15.39:0/3473481545 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a01a8650 con 0x7f61a00755f0 2024-07-04T08:17:51.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.724+0000 7f61abc14740 1 Processor -- start 2024-07-04T08:17:51.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.724+0000 7f61abc14740 1 -- start start 2024-07-04T08:17:51.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f6197400640 1 -- 172.21.15.39:0/3473481545 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6188010430 con 0x7f61a00755f0 2024-07-04T08:17:51.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f61abc14740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ec0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f61abc14740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ec0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f61abc14740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ec10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f61abc14740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5567f6dea8f0 con 0x5567f6af3490 2024-07-04T08:17:51.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f61abc14740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5567f6e36ed0 con 0x5567f6afb670 2024-07-04T08:17:51.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f61abc14740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5567f6e26470 con 0x5567f6b1c350 2024-07-04T08:17:51.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f619f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ec0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f619f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ec0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42756/0 (socket says 172.21.15.39:42756) 2024-07-04T08:17:51.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f619f400640 1 -- 172.21.15.39:0/3049555236 learned_addr learned my addr 172.21.15.39:0/3049555236 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:51.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f6197e00640 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ec10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.725+0000 7f619ea00640 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ec0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f619f400640 1 -- 172.21.15.39:0/3049555236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 msgr2=0x5567f6ec10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f619f400640 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ec10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f619f400640 1 -- 172.21.15.39:0/3049555236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 msgr2=0x5567f6ec0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f619f400640 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ec0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f619f400640 1 -- 172.21.15.39:0/3049555236 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f6dbe290 con 0x5567f6b1c350 2024-07-04T08:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f619ea00640 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ec0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:51.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f619f400640 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ec0650 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6198009f50 tx=0x7f61980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb92d2aa7a418617 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f618fe00640 1 -- 172.21.15.39:0/3049555236 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198004d20 con 0x5567f6b1c350 2024-07-04T08:17:51.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f618fe00640 1 -- 172.21.15.39:0/3049555236 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6198004050 con 0x5567f6b1c350 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.726+0000 7f618fe00640 1 -- 172.21.15.39:0/3049555236 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198005280 con 0x5567f6b1c350 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.727+0000 7f6197400640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61840b29e0 0x7f61840b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.727+0000 7f6197400640 1 -- 172.21.15.39:0/3473481545 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f61880e2bb0 con 0x7f61a00755f0 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.727+0000 7f619ea00640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61840b29e0 0x7f61840b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f61abc14740 1 -- 172.21.15.39:0/3049555236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 msgr2=0x5567f6ec0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f61abc14740 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ec0650 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6198009f50 tx=0x7f61980118d0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f61abc14740 1 -- 172.21.15.39:0/3049555236 shutdown_connections 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f61abc14740 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ec0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f61abc14740 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ec10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f61abc14740 1 --2- 172.21.15.39:0/3049555236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ec0650 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f61abc14740 1 -- 172.21.15.39:0/3049555236 >> 172.21.15.39:0/3049555236 conn(0x5567f6b10740 msgr2=0x5567f6ebe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f61abc14740 1 -- 172.21.15.39:0/3049555236 shutdown_connections 2024-07-04T08:17:51.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f61abc14740 1 -- 172.21.15.39:0/3049555236 wait complete. 2024-07-04T08:17:51.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.728+0000 7f619ea00640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61840b29e0 0x7f61840b4ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6190005ab0 tx=0x7f6190005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.729+0000 7f61abc14740 1 -- start start 2024-07-04T08:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.729+0000 7f61abc14740 1 Processor -- start 2024-07-04T08:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f61abc14740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ecbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f61abc14740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6afb670 0x5567f6ecc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f61abc14740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6b1c350 0x5567f6ed7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f61abc14740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5567f6dcd770 con 0x5567f6af3490 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f61abc14740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5567f6e2a4b0 con 0x5567f6b1c350 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f61abc14740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5567f6c8a890 con 0x5567f6afb670 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f61abc14740 10 client.0 _ll_register_callbacks cb 0x5567f6dce2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f6197e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6afb670 0x5567f6ecc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f6197e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6afb670 0x5567f6ecc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42764/0 (socket says 172.21.15.39:42764) 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f6197e00640 1 -- 172.21.15.39:0/2347795282 learned_addr learned my addr 172.21.15.39:0/2347795282 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f619f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6b1c350 0x5567f6ed7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.730+0000 7f619ea00640 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ecbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f6197e00640 1 -- 172.21.15.39:0/2347795282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6b1c350 msgr2=0x5567f6ed7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f6197e00640 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6b1c350 0x5567f6ed7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f6197e00640 1 -- 172.21.15.39:0/2347795282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 msgr2=0x5567f6ecbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f6197e00640 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ecbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f6197e00640 1 -- 172.21.15.39:0/2347795282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f6b87930 con 0x5567f6afb670 2024-07-04T08:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f619f400640 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6b1c350 0x5567f6ed7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f6197e00640 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6afb670 0x5567f6ecc3a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f61880a6e20 tx=0x7f6188010cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b9b9f467d7f78a0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f618cc00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f618cc00640 1 -- 172.21.15.39:0/2347795282 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880a9320 con 0x5567f6afb670 2024-07-04T08:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f618cc00640 1 -- 172.21.15.39:0/2347795282 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f61880a8e10 con 0x5567f6afb670 2024-07-04T08:17:51.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.731+0000 7f61abc14740 1 -- 172.21.15.39:0/2347795282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6198004050 con 0x5567f6afb670 2024-07-04T08:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f618cc00640 1 -- 172.21.15.39:0/2347795282 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61880e0400 con 0x5567f6afb670 2024-07-04T08:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f61abc14740 10 client.26068 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f61abc14740 1 -- 172.21.15.39:0/2347795282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567f6edaae0 con 0x5567f6afb670 2024-07-04T08:17:51.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f618cc00640 1 -- 172.21.15.39:0/2347795282 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6198004050 con 0x5567f6afb670 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f618cc00640 1 -- 172.21.15.39:0/2347795282 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f61880cc0b0 con 0x5567f6afb670 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f61abc14740 10 client.26068 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f61abc14740 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f61abc14740 20 client.26068 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f61abc14740 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5567f6adacd0 0x5567f6eddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f61abc14740 4 client.26068 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.732+0000 7f61abc14740 1 -- 172.21.15.39:0/2347795282 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567f6e38e20 con 0x5567f6adacd0 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.733+0000 7f619ea00640 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5567f6adacd0 0x5567f6eddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.733+0000 7f619ea00640 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5567f6adacd0 0x5567f6eddfa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc5d39b967800ea6 server_cookie=3b3d4b3a7d376e79 in_seq=0 out_seq=0 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.733+0000 7f618cc00640 10 client.26068 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.734+0000 7f618cc00640 1 -- 172.21.15.39:0/2347795282 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5567f6e38e20 con 0x5567f6adacd0 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.734+0000 7f618cc00640 10 client.26068 handle_command_reply: tid=0 2024-07-04T08:17:51.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.734+0000 7f61abc14740 1 client.26068 shutdown 2024-07-04T08:17:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 -- 172.21.15.39:0/2347795282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6afb670 msgr2=0x5567f6ecc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6afb670 0x5567f6ecc3a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f61880a6e20 tx=0x7f6188010cc0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 -- 172.21.15.39:0/2347795282 shutdown_connections 2024-07-04T08:17:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5567f6adacd0 0x5567f6eddfa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6b1c350 0x5567f6ed7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6afb670 0x5567f6ecc3a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f618cc00640 0 client.26068 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 --2- 172.21.15.39:0/2347795282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ecbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 -- 172.21.15.39:0/2347795282 >> 172.21.15.39:0/2347795282 conn(0x5567f6b10740 msgr2=0x5567f6ec37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 -- 172.21.15.39:0/2347795282 shutdown_connections 2024-07-04T08:17:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 1 -- 172.21.15.39:0/2347795282 wait complete. 2024-07-04T08:17:51.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.735+0000 7f61abc14740 20 client.26068 trim_cache size 0 max 16384 2024-07-04T08:17:51.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.772+0000 7f61abc14740 1 Processor -- start 2024-07-04T08:17:51.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.772+0000 7f61abc14740 1 -- start start 2024-07-04T08:17:51.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.772+0000 7f61abc14740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6af3490 0x5567f6ec2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.772+0000 7f61abc14740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.772+0000 7f61abc14740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6b1c350 0x5567f6ed7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.772+0000 7f61abc14740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61a01b25c0 con 0x5567f6b1c350 2024-07-04T08:17:51.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.772+0000 7f61abc14740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61a01b4100 con 0x5567f6afb670 2024-07-04T08:17:51.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.772+0000 7f61abc14740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61a01b2c90 con 0x5567f6af3490 2024-07-04T08:17:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f6197e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f6197e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54340/0 (socket says 172.21.15.39:54340) 2024-07-04T08:17:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f6197e00640 1 -- 172.21.15.39:0/2034817372 learned_addr learned my addr 172.21.15.39:0/2034817372 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f619ea00640 1 --2- 172.21.15.39:0/2034817372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6af3490 0x5567f6ec2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f619f400640 1 --2- 172.21.15.39:0/2034817372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6b1c350 0x5567f6ed7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f6197e00640 1 -- 172.21.15.39:0/2034817372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6af3490 msgr2=0x5567f6ec2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:51.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f6197e00640 1 --2- 172.21.15.39:0/2034817372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6af3490 0x5567f6ec2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f6197e00640 1 -- 172.21.15.39:0/2034817372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6b1c350 msgr2=0x5567f6ed7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f6197e00640 1 --2- 172.21.15.39:0/2034817372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6b1c350 0x5567f6ed7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.773+0000 7f6197e00640 1 -- 172.21.15.39:0/2034817372 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f6ece5a0 con 0x5567f6afb670 2024-07-04T08:17:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.774+0000 7f6197e00640 1 --2- 172.21.15.39:0/2034817372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecda90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f618800c6a0 tx=0x7f61880096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4986fa90d33db149 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.774+0000 7f618f400640 1 -- 172.21.15.39:0/2034817372 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880c9070 con 0x5567f6afb670 2024-07-04T08:17:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.774+0000 7f618f400640 1 -- 172.21.15.39:0/2034817372 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61880b22a0 con 0x5567f6afb670 2024-07-04T08:17:51.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.774+0000 7f618f400640 1 -- 172.21.15.39:0/2034817372 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880a9bb0 con 0x5567f6afb670 2024-07-04T08:17:51.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.775+0000 7f61abc14740 1 -- 172.21.15.39:0/2034817372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 msgr2=0x5567f6ecda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:51.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.775+0000 7f61abc14740 1 --2- 172.21.15.39:0/2034817372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecda90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f618800c6a0 tx=0x7f61880096a0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.775+0000 7f61abc14740 1 -- 172.21.15.39:0/2034817372 shutdown_connections 2024-07-04T08:17:51.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.775+0000 7f61abc14740 1 --2- 172.21.15.39:0/2034817372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6b1c350 0x5567f6ed7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.775+0000 7f61abc14740 1 --2- 172.21.15.39:0/2034817372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecda90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.775+0000 7f61abc14740 1 --2- 172.21.15.39:0/2034817372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6af3490 0x5567f6ec2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.775+0000 7f61abc14740 1 -- 172.21.15.39:0/2034817372 >> 172.21.15.39:0/2034817372 conn(0x5567f6b10740 msgr2=0x5567f6ecd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:51.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.776+0000 7f61abc14740 1 -- 172.21.15.39:0/2034817372 shutdown_connections 2024-07-04T08:17:51.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.776+0000 7f61abc14740 1 -- 172.21.15.39:0/2034817372 wait complete. 2024-07-04T08:17:51.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.776+0000 7f61abc14740 1 -- start start 2024-07-04T08:17:51.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.777+0000 7f61abc14740 1 Processor -- start 2024-07-04T08:17:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f61abc14740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ed4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f61abc14740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f61abc14740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ed0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f61abc14740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f61a01b3a30 con 0x5567f6af3490 2024-07-04T08:17:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f61abc14740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f61a01b3360 con 0x5567f6afb670 2024-07-04T08:17:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f61abc14740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61a01b05a0 con 0x5567f6b1c350 2024-07-04T08:17:51.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f61abc14740 10 client.0 _ll_register_callbacks cb 0x7f61880a5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:51.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f6197e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f6197e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54354/0 (socket says 172.21.15.39:54354) 2024-07-04T08:17:51.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f6197e00640 1 -- 172.21.15.39:0/906361711 learned_addr learned my addr 172.21.15.39:0/906361711 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:51.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f619ea00640 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ed4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f619f400640 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ed0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f6197e00640 1 -- 172.21.15.39:0/906361711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 msgr2=0x5567f6ed0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f6197e00640 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ed0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f6197e00640 1 -- 172.21.15.39:0/906361711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 msgr2=0x5567f6ed4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f6197e00640 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ed4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f6197e00640 1 -- 172.21.15.39:0/906361711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f6f45730 con 0x5567f6afb670 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.778+0000 7f619f400640 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ed0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.779+0000 7f6197e00640 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecfe30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f61880a6150 tx=0x7f61880107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2856f0de2a174 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.779+0000 7f618cc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.779+0000 7f618cc00640 1 -- 172.21.15.39:0/906361711 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880c9050 con 0x5567f6afb670 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.779+0000 7f618cc00640 1 -- 172.21.15.39:0/906361711 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f61880a5730 con 0x5567f6afb670 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.779+0000 7f61abc14740 1 -- 172.21.15.39:0/906361711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567f6e348c0 con 0x5567f6afb670 2024-07-04T08:17:51.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.779+0000 7f618cc00640 1 -- 172.21.15.39:0/906361711 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f618808f050 con 0x5567f6afb670 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.779+0000 7f61abc14740 10 client.26111 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.779+0000 7f61abc14740 1 -- 172.21.15.39:0/906361711 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567f6f45730 con 0x5567f6afb670 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.780+0000 7f618cc00640 1 -- 172.21.15.39:0/906361711 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5567f6e348c0 con 0x5567f6afb670 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.780+0000 7f618cc00640 1 -- 172.21.15.39:0/906361711 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f61880af040 con 0x5567f6afb670 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.780+0000 7f61abc14740 10 client.26111 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.780+0000 7f61abc14740 10 client.26111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.780+0000 7f61abc14740 20 client.26111 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.780+0000 7f61abc14740 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5567f6ab0080 0x5567f6f48bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.780+0000 7f61abc14740 4 client.26111 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.780+0000 7f61abc14740 1 -- 172.21.15.39:0/906361711 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5567f6e32000 con 0x5567f6ab0080 2024-07-04T08:17:51.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.780+0000 7f619ea00640 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5567f6ab0080 0x5567f6f48bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:51.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.781+0000 7f619ea00640 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5567f6ab0080 0x5567f6f48bf0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee84e37a72d537b5 server_cookie=29cc397b96737e5f in_seq=0 out_seq=0 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.781+0000 7f618cc00640 10 client.26111 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f618cc00640 1 -- 172.21.15.39:0/906361711 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5567f6e32000 con 0x5567f6ab0080 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f618cc00640 10 client.26111 handle_command_reply: tid=0 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f61abc14740 1 client.26111 shutdown 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f61abc14740 1 -- 172.21.15.39:0/906361711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 msgr2=0x5567f6ecfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f61abc14740 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecfe30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f61880a6150 tx=0x7f61880107e0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f61abc14740 1 -- 172.21.15.39:0/906361711 shutdown_connections 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f61abc14740 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5567f6ab0080 0x5567f6f48bf0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f61abc14740 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567f6b1c350 0x5567f6ed0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f61abc14740 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567f6afb670 0x5567f6ecfe30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f61abc14740 1 --2- 172.21.15.39:0/906361711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567f6af3490 0x5567f6ed4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f61abc14740 1 -- 172.21.15.39:0/906361711 >> 172.21.15.39:0/906361711 conn(0x5567f6b10740 msgr2=0x5567f6ec1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:51.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.782+0000 7f618cc00640 0 client.26111 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:51.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.783+0000 7f61abc14740 1 -- 172.21.15.39:0/906361711 shutdown_connections 2024-07-04T08:17:51.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.783+0000 7f61abc14740 1 -- 172.21.15.39:0/906361711 wait complete. 2024-07-04T08:17:51.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.783+0000 7f61abc14740 20 client.26111 trim_cache size 0 max 16384 2024-07-04T08:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "07fed365-313e-4c03-be80-97d774e4f0ea", 2024-07-04T08:17:51.787 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:17:51.788 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:17:51.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 -- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61840b29e0 msgr2=0x7f61840b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:51.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61840b29e0 0x7f61840b4ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6190005ab0 tx=0x7f6190005a20 comp rx=0 tx=0).stop 2024-07-04T08:17:51.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 -- 172.21.15.39:0/3473481545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a00755f0 msgr2=0x7f61a019c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a00755f0 0x7f61a019c230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f618800e9f0 tx=0x7f618800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 -- 172.21.15.39:0/3473481545 shutdown_connections 2024-07-04T08:17:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61840b29e0 0x7f61840b4ef0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61a010f670 0x7f61a01a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f61a00755f0 0x7f61a019c230 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 --2- 172.21.15.39:0/3473481545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61a0074ca0 0x7f61a019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 -- 172.21.15.39:0/3473481545 >> 172.21.15.39:0/3473481545 conn(0x7f61a007ada0 msgr2=0x7f61a01091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 -- 172.21.15.39:0/3473481545 shutdown_connections 2024-07-04T08:17:51.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:51.784+0000 7f61a5600640 1 -- 172.21.15.39:0/3473481545 wait complete. 2024-07-04T08:17:52.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:17:52.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:17:52.601 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:17:52.741 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:17:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:53.380 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: pgmap v560: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-07-04T08:17:53.380 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:17:53.381 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: pgmap v560: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:17:53.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:17:53.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: pgmap v560: 129 pgs: 129 active+clean; 1.9 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-07-04T08:17:53.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:17:53.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:53.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:17:53.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:17:53.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:17:53.567 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:17:53.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:17:53.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:17:53.568 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:17:54.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:54 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi039 to 176.4M 2024-07-04T08:17:54.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:54 smithi039 ceph-mon[32062]: Unable to set osd_memory_target on smithi039 to 185068441: error parsing value: Value '185068441' is below minimum 939524096 2024-07-04T08:17:54.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:54 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi039 to 176.4M 2024-07-04T08:17:54.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:54 smithi050 ceph-mon[36577]: Unable to set osd_memory_target on smithi039 to 185068441: error parsing value: Value '185068441' is below minimum 939524096 2024-07-04T08:17:54.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:54 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi039 to 176.4M 2024-07-04T08:17:54.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:54 smithi133 ceph-mon[37050]: Unable to set osd_memory_target on smithi039 to 185068441: error parsing value: Value '185068441' is below minimum 939524096 2024-07-04T08:17:55.351 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:55 smithi039 ceph-mon[32062]: pgmap v561: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-07-04T08:17:55.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:55 smithi050 ceph-mon[36577]: pgmap v561: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-07-04T08:17:55.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:55 smithi133 ceph-mon[37050]: pgmap v561: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 16 op/s 2024-07-04T08:17:55.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.678+0000 7f0b8d000640 1 -- 172.21.15.39:0/3930448592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b8810ee50 msgr2=0x7f0b8810f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:55.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.678+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3930448592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b8810ee50 0x7f0b8810f260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009900 tx=0x7f0b7002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.678+0000 7f0b8d000640 1 -- 172.21.15.39:0/3930448592 shutdown_connections 2024-07-04T08:17:55.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.678+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3930448592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8810f7a0 0x7f0b88115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.678+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3930448592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b8810ee50 0x7f0b8810f260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.678+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3930448592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b88109510 0x7f0b88109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.678+0000 7f0b8d000640 1 -- 172.21.15.39:0/3930448592 >> 172.21.15.39:0/3930448592 conn(0x7f0b88078b10 msgr2=0x7f0b88107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:55.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.678+0000 7f0b8d000640 1 -- 172.21.15.39:0/3930448592 shutdown_connections 2024-07-04T08:17:55.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.678+0000 7f0b8d000640 1 -- 172.21.15.39:0/3930448592 wait complete. 2024-07-04T08:17:55.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.680+0000 7f0b8d000640 1 Processor -- start 2024-07-04T08:17:55.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.681+0000 7f0b8d000640 1 -- start start 2024-07-04T08:17:55.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.681+0000 7f0b8d000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b88109510 0x7f0b8819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.681+0000 7f0b8d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b8810ee50 0x7f0b881a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.681+0000 7f0b8d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8810f7a0 0x7f0b881a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b8d000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0b88119430 con 0x7f0b8810ee50 2024-07-04T08:17:55.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b8d000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881192b0 con 0x7f0b8810f7a0 2024-07-04T08:17:55.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b8d000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881195b0 con 0x7f0b88109510 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b87400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b88109510 0x7f0b8819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b87400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b88109510 0x7f0b8819fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42800/0 (socket says 172.21.15.39:42800) 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b87400640 1 -- 172.21.15.39:0/3044092764 learned_addr learned my addr 172.21.15.39:0/3044092764 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b87e00640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8810f7a0 0x7f0b881a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b7fe00640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b8810ee50 0x7f0b881a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b87400640 1 -- 172.21.15.39:0/3044092764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8810f7a0 msgr2=0x7f0b881a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b87400640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8810f7a0 0x7f0b881a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b87400640 1 -- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b8810ee50 msgr2=0x7f0b881a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b87400640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b8810ee50 0x7f0b881a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b87400640 1 -- 172.21.15.39:0/3044092764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b881a5dd0 con 0x7f0b88109510 2024-07-04T08:17:55.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.682+0000 7f0b7fe00640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b8810ee50 0x7f0b881a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:55.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.683+0000 7f0b87400640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b88109510 0x7f0b8819fe60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800e9f0 tx=0x7f0b7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:55.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.683+0000 7f0b84c00640 1 -- 172.21.15.39:0/3044092764 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800cd80 con 0x7f0b88109510 2024-07-04T08:17:55.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.683+0000 7f0b84c00640 1 -- 172.21.15.39:0/3044092764 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b780108e0 con 0x7f0b88109510 2024-07-04T08:17:55.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.683+0000 7f0b8d000640 1 -- 172.21.15.39:0/3044092764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b881a60c0 con 0x7f0b88109510 2024-07-04T08:17:55.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.683+0000 7f0b84c00640 1 -- 172.21.15.39:0/3044092764 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78018980 con 0x7f0b88109510 2024-07-04T08:17:55.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.683+0000 7f0b8d000640 1 -- 172.21.15.39:0/3044092764 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b881ac820 con 0x7f0b88109510 2024-07-04T08:17:55.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.688+0000 7f0b94be2740 1 Processor -- start 2024-07-04T08:17:55.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.688+0000 7f0b94be2740 1 -- start start 2024-07-04T08:17:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaaaa070 0x563bdae4e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae4ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae4f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b94be2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563bdadc4ed0 con 0x563bdaa811b0 2024-07-04T08:17:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b94be2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563bdadb4470 con 0x563bdaaaa070 2024-07-04T08:17:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b94be2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563bdadb84b0 con 0x563bdaa89390 2024-07-04T08:17:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaaaa070 0x563bdae4e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaaaa070 0x563bdae4e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54378/0 (socket says 172.21.15.39:54378) 2024-07-04T08:17:55.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b87e00640 1 -- 172.21.15.39:0/781221254 learned_addr learned my addr 172.21.15.39:0/781221254 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:55.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b84c00640 1 -- 172.21.15.39:0/3044092764 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0b78010430 con 0x7f0b88109510 2024-07-04T08:17:55.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b7fe00640 1 --2- 172.21.15.39:0/781221254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae4f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.689+0000 7f0b87e00640 1 -- 172.21.15.39:0/781221254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 msgr2=0x563bdae4f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.690+0000 7f0b87e00640 1 --2- 172.21.15.39:0/781221254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae4f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.690+0000 7f0b87e00640 1 -- 172.21.15.39:0/781221254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 msgr2=0x563bdae4ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.690+0000 7f0b87e00640 1 --2- 172.21.15.39:0/781221254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae4ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.690+0000 7f0b87e00640 1 -- 172.21.15.39:0/781221254 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bdad4c640 con 0x563bdaaaa070 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.690+0000 7f0b7fe00640 1 --2- 172.21.15.39:0/781221254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae4f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.690+0000 7f0b87e00640 1 --2- 172.21.15.39:0/781221254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaaaa070 0x563bdae4e940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009fd0 tx=0x7f0b800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae5a789eb21cef9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.690+0000 7f0b7cc00640 1 -- 172.21.15.39:0/781221254 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80004d20 con 0x563bdaaaa070 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.691+0000 7f0b7cc00640 1 -- 172.21.15.39:0/781221254 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b80004050 con 0x563bdaaaa070 2024-07-04T08:17:55.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.691+0000 7f0b7cc00640 1 -- 172.21.15.39:0/781221254 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80005280 con 0x563bdaaaa070 2024-07-04T08:17:55.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.691+0000 7f0b94be2740 1 -- 172.21.15.39:0/781221254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaaaa070 msgr2=0x563bdae4e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.691+0000 7f0b94be2740 1 --2- 172.21.15.39:0/781221254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaaaa070 0x563bdae4e940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009fd0 tx=0x7f0b800118d0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.691+0000 7f0b84c00640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0b6c0b2960 0x7f0b6c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b94be2740 1 -- 172.21.15.39:0/781221254 shutdown_connections 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b94be2740 1 --2- 172.21.15.39:0/781221254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae4ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b94be2740 1 --2- 172.21.15.39:0/781221254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae4f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b94be2740 1 --2- 172.21.15.39:0/781221254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaaaa070 0x563bdae4e940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b84c00640 1 -- 172.21.15.39:0/3044092764 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0b780e2d80 con 0x7f0b88109510 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b94be2740 1 -- 172.21.15.39:0/781221254 >> 172.21.15.39:0/781221254 conn(0x563bdaa9e460 msgr2=0x563bdae4c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b87e00640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0b6c0b2960 0x7f0b6c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b94be2740 1 -- 172.21.15.39:0/781221254 shutdown_connections 2024-07-04T08:17:55.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b94be2740 1 -- 172.21.15.39:0/781221254 wait complete. 2024-07-04T08:17:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.692+0000 7f0b94be2740 1 -- start start 2024-07-04T08:17:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.693+0000 7f0b87e00640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0b6c0b2960 0x7f0b6c0b4e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b80005fd0 tx=0x7f0b800048d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:55.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.693+0000 7f0b94be2740 1 Processor -- start 2024-07-04T08:17:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.693+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa811b0 0x563bdae5a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.693+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.693+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaaaa070 0x563bdae66440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.693+0000 7f0b94be2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563bdac18700 con 0x563bdaaaa070 2024-07-04T08:17:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.693+0000 7f0b94be2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563bdad5bba0 con 0x563bdaa811b0 2024-07-04T08:17:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.693+0000 7f0b94be2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563bdae5d120 con 0x563bdaa89390 2024-07-04T08:17:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.693+0000 7f0b94be2740 10 client.0 _ll_register_callbacks cb 0x563bdadad560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaaaa070 0x563bdae66440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b87400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa811b0 0x563bdae5a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaaaa070 0x563bdae66440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47738/0 (socket says 172.21.15.39:47738) 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b87e00640 1 -- 172.21.15.39:0/3245725474 learned_addr learned my addr 172.21.15.39:0/3245725474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b7fe00640 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b87400640 1 -- 172.21.15.39:0/3245725474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 msgr2=0x563bdae5ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b87400640 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae5ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b87400640 1 -- 172.21.15.39:0/3245725474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaaaa070 msgr2=0x563bdae66440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b87400640 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaaaa070 0x563bdae66440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.694+0000 7f0b87400640 1 -- 172.21.15.39:0/3245725474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bdab15650 con 0x563bdaa811b0 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.695+0000 7f0b87400640 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa811b0 0x563bdae5a610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0b780e0140 tx=0x7f0b78010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0872ff854a31ba9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.695+0000 7f0b75600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:55.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.695+0000 7f0b75600640 1 -- 172.21.15.39:0/3245725474 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780e4070 con 0x563bdaa811b0 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.695+0000 7f0b75600640 1 -- 172.21.15.39:0/3245725474 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0b780a8f30 con 0x563bdaa811b0 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.695+0000 7f0b94be2740 1 -- 172.21.15.39:0/3245725474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b80004050 con 0x563bdaa811b0 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.695+0000 7f0b75600640 1 -- 172.21.15.39:0/3245725474 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b780e05d0 con 0x563bdaa811b0 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.695+0000 7f0b94be2740 10 client.26123 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.695+0000 7f0b94be2740 1 -- 172.21.15.39:0/3245725474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563bdae69120 con 0x563bdaa811b0 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.696+0000 7f0b75600640 1 -- 172.21.15.39:0/3245725474 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b80004050 con 0x563bdaa811b0 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.696+0000 7f0b75600640 1 -- 172.21.15.39:0/3245725474 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0b780c4030 con 0x563bdaa811b0 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.696+0000 7f0b94be2740 10 client.26123 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.696+0000 7f0b94be2740 10 client.26123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.696+0000 7f0b94be2740 20 client.26123 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:55.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.696+0000 7f0b94be2740 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563bdaa689f0 0x563bdae6c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.696+0000 7f0b94be2740 4 client.26123 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:17:55.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.696+0000 7f0b94be2740 1 -- 172.21.15.39:0/3245725474 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bdadc6e20 con 0x563bdaa689f0 2024-07-04T08:17:55.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.696+0000 7f0b7fe00640 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563bdaa689f0 0x563bdae6c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.697+0000 7f0b7fe00640 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563bdaa689f0 0x563bdae6c640 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdaf641cb4f0e935 server_cookie=9fe66b58a18c024a in_seq=0 out_seq=0 2024-07-04T08:17:55.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.697+0000 7f0b75600640 10 client.26123 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:55.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b75600640 1 -- 172.21.15.39:0/3245725474 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563bdadc6e20 con 0x563bdaa689f0 2024-07-04T08:17:55.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b75600640 10 client.26123 handle_command_reply: tid=0 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b94be2740 1 client.26123 shutdown 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b94be2740 1 -- 172.21.15.39:0/3245725474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa811b0 msgr2=0x563bdae5a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b94be2740 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa811b0 0x563bdae5a610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0b780e0140 tx=0x7f0b78010cc0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b94be2740 1 -- 172.21.15.39:0/3245725474 shutdown_connections 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b94be2740 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563bdaa689f0 0x563bdae6c640 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b75600640 0 client.26123 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b94be2740 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaaaa070 0x563bdae66440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b94be2740 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b94be2740 1 --2- 172.21.15.39:0/3245725474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa811b0 0x563bdae5a610 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.698+0000 7f0b94be2740 1 -- 172.21.15.39:0/3245725474 >> 172.21.15.39:0/3245725474 conn(0x563bdaa9e460 msgr2=0x563bdae51ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:55.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.699+0000 7f0b94be2740 1 -- 172.21.15.39:0/3245725474 shutdown_connections 2024-07-04T08:17:55.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.699+0000 7f0b94be2740 1 -- 172.21.15.39:0/3245725474 wait complete. 2024-07-04T08:17:55.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.699+0000 7f0b94be2740 20 client.26123 trim_cache size 0 max 16384 2024-07-04T08:17:55.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.719+0000 7f0b94be2740 1 Processor -- start 2024-07-04T08:17:55.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.719+0000 7f0b94be2740 1 -- start start 2024-07-04T08:17:55.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa689f0 0x563bdae5c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae59920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa89390 0x563bdae66230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b94be2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881b6720 con 0x563bdaa811b0 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b94be2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881b8260 con 0x563bdaa89390 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b94be2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881b6df0 con 0x563bdaa689f0 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b87400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa689f0 0x563bdae5c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b7fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae59920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b87400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa689f0 0x563bdae5c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42826/0 (socket says 172.21.15.39:42826) 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b87400640 1 -- 172.21.15.39:0/2730699358 learned_addr learned my addr 172.21.15.39:0/2730699358 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b87e00640 1 --2- 172.21.15.39:0/2730699358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa89390 0x563bdae66230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b87400640 1 -- 172.21.15.39:0/2730699358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa89390 msgr2=0x563bdae66230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b87400640 1 --2- 172.21.15.39:0/2730699358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa89390 0x563bdae66230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b87400640 1 -- 172.21.15.39:0/2730699358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 msgr2=0x563bdae59920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.720+0000 7f0b87400640 1 --2- 172.21.15.39:0/2730699358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae59920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.721+0000 7f0b87400640 1 -- 172.21.15.39:0/2730699358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bdae511d0 con 0x563bdaa689f0 2024-07-04T08:17:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.721+0000 7f0b87400640 1 --2- 172.21.15.39:0/2730699358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa689f0 0x563bdae5c2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b780a5d50 tx=0x7f0b780a5cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe90d1893119c581 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.721+0000 7f0b77e00640 1 -- 172.21.15.39:0/2730699358 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780c4390 con 0x563bdaa689f0 2024-07-04T08:17:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.721+0000 7f0b77e00640 1 -- 172.21.15.39:0/2730699358 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b780c1070 con 0x563bdaa689f0 2024-07-04T08:17:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.721+0000 7f0b77e00640 1 -- 172.21.15.39:0/2730699358 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780a9190 con 0x563bdaa689f0 2024-07-04T08:17:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.722+0000 7f0b94be2740 1 -- 172.21.15.39:0/2730699358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa689f0 msgr2=0x563bdae5c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.722+0000 7f0b94be2740 1 --2- 172.21.15.39:0/2730699358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa689f0 0x563bdae5c2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b780a5d50 tx=0x7f0b780a5cc0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.722+0000 7f0b94be2740 1 -- 172.21.15.39:0/2730699358 shutdown_connections 2024-07-04T08:17:55.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.722+0000 7f0b94be2740 1 --2- 172.21.15.39:0/2730699358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae59920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.722+0000 7f0b94be2740 1 --2- 172.21.15.39:0/2730699358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa89390 0x563bdae66230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.722+0000 7f0b94be2740 1 --2- 172.21.15.39:0/2730699358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa689f0 0x563bdae5c2f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.722+0000 7f0b94be2740 1 -- 172.21.15.39:0/2730699358 >> 172.21.15.39:0/2730699358 conn(0x563bdaa9e460 msgr2=0x563bdae4f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.722+0000 7f0b94be2740 1 -- 172.21.15.39:0/2730699358 shutdown_connections 2024-07-04T08:17:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.722+0000 7f0b94be2740 1 -- 172.21.15.39:0/2730699358 wait complete. 2024-07-04T08:17:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.723+0000 7f0b94be2740 1 -- start start 2024-07-04T08:17:55.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.723+0000 7f0b94be2740 1 Processor -- start 2024-07-04T08:17:55.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa689f0 0x563bdae629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae71720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b94be2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae71c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b94be2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881b7b90 con 0x563bdaa811b0 2024-07-04T08:17:55.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b94be2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881b74c0 con 0x563bdaa689f0 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b94be2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881b4700 con 0x563bdaa89390 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b94be2740 10 client.0 _ll_register_callbacks cb 0x7f0b78009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b87400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa689f0 0x563bdae629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b87400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa689f0 0x563bdae629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54412/0 (socket says 172.21.15.39:54412) 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b7fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae71720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b87400640 1 -- 172.21.15.39:0/374774151 learned_addr learned my addr 172.21.15.39:0/374774151 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b87e00640 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae71c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b87400640 1 -- 172.21.15.39:0/374774151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 msgr2=0x563bdae71c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b87400640 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae71c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b87400640 1 -- 172.21.15.39:0/374774151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 msgr2=0x563bdae71720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:17:55.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b87400640 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae71720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.724+0000 7f0b87400640 1 -- 172.21.15.39:0/374774151 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bdae72350 con 0x563bdaa689f0 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.725+0000 7f0b87e00640 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae71c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.725+0000 7f0b87400640 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa689f0 0x563bdae629c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b780a6270 tx=0x7f0b780e0220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96c9c2a0d4ed0bc1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.725+0000 7f0b75600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.725+0000 7f0b75600640 1 -- 172.21.15.39:0/374774151 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780b7060 con 0x563bdaa689f0 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.725+0000 7f0b75600640 1 -- 172.21.15.39:0/374774151 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0b78018d30 con 0x563bdaa689f0 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.725+0000 7f0b94be2740 1 -- 172.21.15.39:0/374774151 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b780c1070 con 0x563bdaa689f0 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.725+0000 7f0b75600640 1 -- 172.21.15.39:0/374774151 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b780be050 con 0x563bdaa689f0 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b94be2740 10 client.26129 fetch_fsmap learned FSMap version 30 2024-07-04T08:17:55.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b94be2740 1 -- 172.21.15.39:0/374774151 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563bdaed4d60 con 0x563bdaa689f0 2024-07-04T08:17:55.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b75600640 1 -- 172.21.15.39:0/374774151 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b780c1070 con 0x563bdaa689f0 2024-07-04T08:17:55.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b75600640 1 -- 172.21.15.39:0/374774151 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0b780c4070 con 0x563bdaa689f0 2024-07-04T08:17:55.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b94be2740 10 client.26129 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:17:55.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b94be2740 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:17:55.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b94be2740 20 client.26129 populate_metadata read hostname 'smithi039' 2024-07-04T08:17:55.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b94be2740 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563bdaaaa070 0x563bdaed7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:55.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b94be2740 4 client.26129 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:17:55.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.726+0000 7f0b94be2740 1 -- 172.21.15.39:0/374774151 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563bdadc28c0 con 0x563bdaaaa070 2024-07-04T08:17:55.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.727+0000 7f0b7fe00640 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563bdaaaa070 0x563bdaed7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:55.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.727+0000 7f0b7fe00640 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563bdaaaa070 0x563bdaed7010 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ce797123d69fcd server_cookie=4264eb46d5926094 in_seq=0 out_seq=0 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.727+0000 7f0b75600640 10 client.26129 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.727+0000 7f0b75600640 1 -- 172.21.15.39:0/374774151 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563bdadc28c0 con 0x563bdaaaa070 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.727+0000 7f0b75600640 10 client.26129 handle_command_reply: tid=0 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.727+0000 7f0b94be2740 1 client.26129 shutdown 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 -- 172.21.15.39:0/374774151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa689f0 msgr2=0x563bdae629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa689f0 0x563bdae629c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0b780a6270 tx=0x7f0b780e0220 comp rx=0 tx=0).stop 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 -- 172.21.15.39:0/374774151 shutdown_connections 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563bdaaaa070 0x563bdaed7010 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563bdaa811b0 0x563bdae71720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563bdaa89390 0x563bdae71c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 --2- 172.21.15.39:0/374774151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563bdaa689f0 0x563bdae629c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b75600640 0 client.26129 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:17:55.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 -- 172.21.15.39:0/374774151 >> 172.21.15.39:0/374774151 conn(0x563bdaa9e460 msgr2=0x563bdae5c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:55.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 -- 172.21.15.39:0/374774151 shutdown_connections 2024-07-04T08:17:55.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 1 -- 172.21.15.39:0/374774151 wait complete. 2024-07-04T08:17:55.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.728+0000 7f0b94be2740 20 client.26129 trim_cache size 0 max 16384 2024-07-04T08:17:55.734 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:17:55.734 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:17:55.735 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:17:55.735 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:17:55.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 -- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0b6c0b2960 msgr2=0x7f0b6c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:55.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0b6c0b2960 0x7f0b6c0b4e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b80005fd0 tx=0x7f0b800048d0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 -- 172.21.15.39:0/3044092764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b88109510 msgr2=0x7f0b8819fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:55.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b88109510 0x7f0b8819fe60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800e9f0 tx=0x7f0b7800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 -- 172.21.15.39:0/3044092764 shutdown_connections 2024-07-04T08:17:55.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0b6c0b2960 0x7f0b6c0b4e70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b8810ee50 0x7f0b881a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0b8810f7a0 0x7f0b881a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 --2- 172.21.15.39:0/3044092764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b88109510 0x7f0b8819fe60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:55.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.729+0000 7f0b8d000640 1 -- 172.21.15.39:0/3044092764 >> 172.21.15.39:0/3044092764 conn(0x7f0b88078b10 msgr2=0x7f0b88079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:55.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.730+0000 7f0b8d000640 1 -- 172.21.15.39:0/3044092764 shutdown_connections 2024-07-04T08:17:55.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:55.730+0000 7f0b8d000640 1 -- 172.21.15.39:0/3044092764 wait complete. 2024-07-04T08:17:56.606 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:17:56.606 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:17:57.103 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:17:57.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:57 smithi050 ceph-mon[36577]: pgmap v562: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-07-04T08:17:57.454 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:57 smithi039 ceph-mon[32062]: pgmap v562: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-07-04T08:17:57.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:57 smithi133 ceph-mon[37050]: pgmap v562: 129 pgs: 129 active+clean; 2.0 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-07-04T08:17:59.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:17:59 smithi050 ceph-mon[36577]: pgmap v563: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-07-04T08:17:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.463+0000 7feaeec00640 1 -- 172.21.15.39:0/171432434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8102f00 msgr2=0x7feae8103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.463+0000 7feaeec00640 1 --2- 172.21.15.39:0/171432434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8102f00 0x7feae8103310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feadc009900 tx=0x7feadc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.463+0000 7feaeec00640 1 -- 172.21.15.39:0/171432434 shutdown_connections 2024-07-04T08:17:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.463+0000 7feaeec00640 1 --2- 172.21.15.39:0/171432434 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feae8109190 0x7feae810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.463+0000 7feaeec00640 1 --2- 172.21.15.39:0/171432434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae8108840 0x7feae8108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.463+0000 7feaeec00640 1 --2- 172.21.15.39:0/171432434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8102f00 0x7feae8103310 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.463+0000 7feaeec00640 1 -- 172.21.15.39:0/171432434 >> 172.21.15.39:0/171432434 conn(0x7feae80fe990 msgr2=0x7feae8100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:59.465 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:17:59 smithi039 ceph-mon[32062]: pgmap v563: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-07-04T08:17:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.464+0000 7feaeec00640 1 -- 172.21.15.39:0/171432434 shutdown_connections 2024-07-04T08:17:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.464+0000 7feaeec00640 1 -- 172.21.15.39:0/171432434 wait complete. 2024-07-04T08:17:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.465+0000 7feaeec00640 1 Processor -- start 2024-07-04T08:17:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.465+0000 7feaeec00640 1 -- start start 2024-07-04T08:17:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaeec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feae8102f00 0x7feae8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaeec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae8108840 0x7feae8199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaeec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8109190 0x7feae819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaeec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7feae8112ea0 con 0x7feae8109190 2024-07-04T08:17:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaeec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feae8112d20 con 0x7feae8102f00 2024-07-04T08:17:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaeec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feae8113020 con 0x7feae8108840 2024-07-04T08:17:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaece00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae8108840 0x7feae8199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaed800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feae8102f00 0x7feae8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaece00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae8108840 0x7feae8199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42860/0 (socket says 172.21.15.39:42860) 2024-07-04T08:17:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaee200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8109190 0x7feae819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaece00640 1 -- 172.21.15.39:0/34441611 learned_addr learned my addr 172.21.15.39:0/34441611 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaece00640 1 -- 172.21.15.39:0/34441611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feae8102f00 msgr2=0x7feae8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaece00640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feae8102f00 0x7feae8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaece00640 1 -- 172.21.15.39:0/34441611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8109190 msgr2=0x7feae819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaece00640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8109190 0x7feae819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.466+0000 7feaece00640 1 -- 172.21.15.39:0/34441611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae819f7c0 con 0x7feae8108840 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.467+0000 7feaece00640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae8108840 0x7feae8199df0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fead400e9f0 tx=0x7fead400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.467+0000 7feae6000640 1 -- 172.21.15.39:0/34441611 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead400ccc0 con 0x7feae8108840 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.467+0000 7feae6000640 1 -- 172.21.15.39:0/34441611 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fead40108e0 con 0x7feae8108840 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.467+0000 7feae6000640 1 -- 172.21.15.39:0/34441611 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4018860 con 0x7feae8108840 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.467+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae819fab0 con 0x7feae8108840 2024-07-04T08:17:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.467+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae8074b40 con 0x7feae8108840 2024-07-04T08:17:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.469+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feae8103310 con 0x7feae8108840 2024-07-04T08:17:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.471+0000 7feae6000640 1 -- 172.21.15.39:0/34441611 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fead4010430 con 0x7feae8108840 2024-07-04T08:17:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.473+0000 7feae6000640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fead00b29e0 0x7fead00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:17:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.473+0000 7feae6000640 1 -- 172.21.15.39:0/34441611 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fead40e2be0 con 0x7feae8108840 2024-07-04T08:17:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.474+0000 7feaed800640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fead00b29e0 0x7fead00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:17:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.474+0000 7feaed800640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fead00b29e0 0x7fead00b4ef0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feadc005ab0 tx=0x7feadc005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:17:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.477+0000 7feae6000640 1 -- 172.21.15.39:0/34441611 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fead40a9be0 con 0x7feae8108840 2024-07-04T08:17:59.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:17:59 smithi133 ceph-mon[37050]: pgmap v563: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 KiB/s wr, 18 op/s 2024-07-04T08:17:59.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.741+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feae802de10 con 0x7feae8108840 2024-07-04T08:17:59.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.742+0000 7feae6000640 1 -- 172.21.15.39:0/34441611 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fead40adee0 con 0x7feae8108840 2024-07-04T08:17:59.744 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:17:59.744 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:17:59.744 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:17:59.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fead00b29e0 msgr2=0x7fead00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:59.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fead00b29e0 0x7fead00b4ef0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feadc005ab0 tx=0x7feadc005a20 comp rx=0 tx=0).stop 2024-07-04T08:17:59.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae8108840 msgr2=0x7feae8199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:17:59.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae8108840 0x7feae8199df0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fead400e9f0 tx=0x7fead400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 shutdown_connections 2024-07-04T08:17:59.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fead00b29e0 0x7fead00b4ef0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feae8109190 0x7feae819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feae8108840 0x7feae8199df0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 --2- 172.21.15.39:0/34441611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feae8102f00 0x7feae8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:17:59.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 >> 172.21.15.39:0/34441611 conn(0x7feae80fe990 msgr2=0x7feae8100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:17:59.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 shutdown_connections 2024-07-04T08:17:59.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:17:59.746+0000 7feaeec00640 1 -- 172.21.15.39:0/34441611 wait complete. 2024-07-04T08:18:00.560 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:00 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/34441611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:18:00.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:00 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/34441611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:18:00.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:00 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/34441611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:18:00.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:18:01.104 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:01.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:01 smithi039 ceph-mon[32062]: pgmap v564: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-07-04T08:18:01.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:01 smithi133 ceph-mon[37050]: pgmap v564: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-07-04T08:18:01.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:01 smithi050 ceph-mon[36577]: pgmap v564: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-07-04T08:18:02.542 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:02 smithi039 ceph-mon[32062]: pgmap v565: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-07-04T08:18:02.566 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:02 smithi133 ceph-mon[37050]: pgmap v565: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-07-04T08:18:02.644 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:02 smithi050 ceph-mon[36577]: pgmap v565: 129 pgs: 129 active+clean; 2.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-07-04T08:18:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.507+0000 7f64e3e00640 1 -- 172.21.15.39:0/286660717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e410ee50 msgr2=0x7f64e410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.507+0000 7f64e3e00640 1 --2- 172.21.15.39:0/286660717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e410ee50 0x7f64e410f260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009900 tx=0x7f64cc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.508+0000 7f64e3e00640 1 -- 172.21.15.39:0/286660717 shutdown_connections 2024-07-04T08:18:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.508+0000 7f64e3e00640 1 --2- 172.21.15.39:0/286660717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e410ee50 0x7f64e410f260 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.508+0000 7f64e3e00640 1 --2- 172.21.15.39:0/286660717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e410f7a0 0x7f64e4115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.508+0000 7f64e3e00640 1 --2- 172.21.15.39:0/286660717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4109510 0x7f64e4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.508+0000 7f64e3e00640 1 -- 172.21.15.39:0/286660717 >> 172.21.15.39:0/286660717 conn(0x7f64e4078b10 msgr2=0x7f64e4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:03.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.508+0000 7f64e3e00640 1 -- 172.21.15.39:0/286660717 shutdown_connections 2024-07-04T08:18:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.508+0000 7f64e3e00640 1 -- 172.21.15.39:0/286660717 wait complete. 2024-07-04T08:18:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.509+0000 7f64e3e00640 1 Processor -- start 2024-07-04T08:18:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.509+0000 7f64e3e00640 1 -- start start 2024-07-04T08:18:03.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4109510 0x7f64e419fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e410ee50 0x7f64e41a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e410f7a0 0x7f64e41a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64e4119410 con 0x7f64e410f7a0 2024-07-04T08:18:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f64e4119290 con 0x7f64e4109510 2024-07-04T08:18:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e3e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f64e4119590 con 0x7f64e410ee50 2024-07-04T08:18:03.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e410f7a0 0x7f64e41a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4109510 0x7f64e419fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4109510 0x7f64e419fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48102/0 (socket says 172.21.15.39:48102) 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e410f7a0 0x7f64e41a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47044/0 (socket says 172.21.15.39:47044) 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e2a00640 1 -- 172.21.15.39:0/3546392352 learned_addr learned my addr 172.21.15.39:0/3546392352 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e410ee50 0x7f64e41a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e2a00640 1 -- 172.21.15.39:0/3546392352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e410ee50 msgr2=0x7f64e41a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.510+0000 7f64e2a00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e410ee50 0x7f64e41a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.511+0000 7f64e2a00640 1 -- 172.21.15.39:0/3546392352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e410f7a0 msgr2=0x7f64e41a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.511+0000 7f64e2a00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e410f7a0 0x7f64e41a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.511+0000 7f64e2a00640 1 -- 172.21.15.39:0/3546392352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e41a5e30 con 0x7f64e4109510 2024-07-04T08:18:03.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.511+0000 7f64e2a00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4109510 0x7f64e419fec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f64d400e9f0 tx=0x7f64d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:03.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.511+0000 7f64e8a00640 1 -- 172.21.15.39:0/3546392352 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d400cd80 con 0x7f64e4109510 2024-07-04T08:18:03.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.511+0000 7f64e3e00640 1 -- 172.21.15.39:0/3546392352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e41a6120 con 0x7f64e4109510 2024-07-04T08:18:03.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.511+0000 7f64e3e00640 1 -- 172.21.15.39:0/3546392352 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e41ac880 con 0x7f64e4109510 2024-07-04T08:18:03.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.511+0000 7f64e8a00640 1 -- 172.21.15.39:0/3546392352 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d40108e0 con 0x7f64e4109510 2024-07-04T08:18:03.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.512+0000 7f64e8a00640 1 -- 172.21.15.39:0/3546392352 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d40188e0 con 0x7f64e4109510 2024-07-04T08:18:03.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.515+0000 7f64f051f740 1 Processor -- start 2024-07-04T08:18:03.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.515+0000 7f64f051f740 1 -- start start 2024-07-04T08:18:03.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.515+0000 7f64e8a00640 1 -- 172.21.15.39:0/3546392352 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f64d4010430 con 0x7f64e4109510 2024-07-04T08:18:03.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64f051f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b32950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64f051f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 0x55d0e6b32e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64f051f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e676d390 0x55d0e6b333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64f051f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d0e6aa8ee0 con 0x55d0e678e070 2024-07-04T08:18:03.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64f051f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d0e6a98480 con 0x55d0e67651b0 2024-07-04T08:18:03.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64f051f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d0e6a9c4c0 con 0x55d0e676d390 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64e3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b32950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64e3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b32950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47054/0 (socket says 172.21.15.39:47054) 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64e3400640 1 -- 172.21.15.39:0/1588643916 learned_addr learned my addr 172.21.15.39:0/1588643916 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.516+0000 7f64e2a00640 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 0x55d0e6b32e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.517+0000 7f64dbe00640 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e676d390 0x55d0e6b333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.517+0000 7f64e3400640 1 -- 172.21.15.39:0/1588643916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e676d390 msgr2=0x55d0e6b333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.517+0000 7f64e3400640 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e676d390 0x55d0e6b333d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.517+0000 7f64e3400640 1 -- 172.21.15.39:0/1588643916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 msgr2=0x55d0e6b32e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.517+0000 7f64e3400640 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 0x55d0e6b32e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.517+0000 7f64e3400640 1 -- 172.21.15.39:0/1588643916 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e6a30650 con 0x55d0e678e070 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.517+0000 7f64dbe00640 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e676d390 0x55d0e6b333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.517+0000 7f64e3400640 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b32950 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009f50 tx=0x7f64dc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d4c80005db9329 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:03.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.517+0000 7f64d8c00640 1 -- 172.21.15.39:0/1588643916 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc004b20 con 0x55d0e678e070 2024-07-04T08:18:03.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.518+0000 7f64d8c00640 1 -- 172.21.15.39:0/1588643916 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc004cc0 con 0x55d0e678e070 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.518+0000 7f64d8c00640 1 -- 172.21.15.39:0/1588643916 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc004850 con 0x55d0e678e070 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.518+0000 7f64e8a00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f64c80b2960 0x7f64c80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.518+0000 7f64e8a00640 1 -- 172.21.15.39:0/3546392352 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f64d40e2ca0 con 0x7f64e4109510 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.518+0000 7f64e2a00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f64c80b2960 0x7f64c80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64f051f740 1 -- 172.21.15.39:0/1588643916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 msgr2=0x55d0e6b32950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64f051f740 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b32950 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009f50 tx=0x7f64dc0118d0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64f051f740 1 -- 172.21.15.39:0/1588643916 shutdown_connections 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64f051f740 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e676d390 0x55d0e6b333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64f051f740 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 0x55d0e6b32e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64f051f740 1 --2- 172.21.15.39:0/1588643916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b32950 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64f051f740 1 -- 172.21.15.39:0/1588643916 >> 172.21.15.39:0/1588643916 conn(0x55d0e6782460 msgr2=0x55d0e6b30840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64f051f740 1 -- 172.21.15.39:0/1588643916 shutdown_connections 2024-07-04T08:18:03.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64f051f740 1 -- 172.21.15.39:0/1588643916 wait complete. 2024-07-04T08:18:03.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.519+0000 7f64e2a00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f64c80b2960 0x7f64c80b4e70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55d0e6b315d0 tx=0x7f64d40e6750 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.520+0000 7f64f051f740 1 -- start start 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.520+0000 7f64f051f740 1 Processor -- start 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.520+0000 7f64f051f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e67651b0 0x55d0e6b3e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64f051f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b3ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64f051f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b4a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64f051f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d0e68fc700 con 0x55d0e678e070 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64f051f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d0e6a3fbb0 con 0x55d0e676d390 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64f051f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d0e6b41130 con 0x55d0e67651b0 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64f051f740 10 client.0 _ll_register_callbacks cb 0x55d0e6a91570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64e3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e67651b0 0x55d0e6b3e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b3ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b3ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48110/0 (socket says 172.21.15.39:48110) 2024-07-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64dbe00640 1 -- 172.21.15.39:0/3678901587 learned_addr learned my addr 172.21.15.39:0/3678901587 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:03.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64e3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e67651b0 0x55d0e6b3e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:52216/0 (socket says 172.21.15.39:52216) 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64e3400640 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b4a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64dbe00640 1 -- 172.21.15.39:0/3678901587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e67651b0 msgr2=0x55d0e6b3e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e67651b0 0x55d0e6b3e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64dbe00640 1 -- 172.21.15.39:0/3678901587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 msgr2=0x55d0e6b4a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b4a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.521+0000 7f64dbe00640 1 -- 172.21.15.39:0/3678901587 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e67f9650 con 0x55d0e676d390 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.522+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b3ea10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009900 tx=0x7f64cc032d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636bb98cdd18b9b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.522+0000 7f64d1600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.522+0000 7f64d1600640 1 -- 172.21.15.39:0/3678901587 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc040070 con 0x55d0e676d390 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.522+0000 7f64d1600640 1 -- 172.21.15.39:0/3678901587 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f64cc0318b0 con 0x55d0e676d390 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.522+0000 7f64f051f740 1 -- 172.21.15.39:0/3678901587 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64dc004cc0 con 0x55d0e676d390 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.523+0000 7f64d1600640 1 -- 172.21.15.39:0/3678901587 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64cc035040 con 0x55d0e676d390 2024-07-04T08:18:03.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.523+0000 7f64f051f740 10 client.26147 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:03.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.523+0000 7f64f051f740 1 -- 172.21.15.39:0/3678901587 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0e6b4d070 con 0x55d0e676d390 2024-07-04T08:18:03.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.523+0000 7f64d1600640 1 -- 172.21.15.39:0/3678901587 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc004cc0 con 0x55d0e676d390 2024-07-04T08:18:03.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.523+0000 7f64d1600640 1 -- 172.21.15.39:0/3678901587 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f64cc004750 con 0x55d0e676d390 2024-07-04T08:18:03.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.523+0000 7f64f051f740 10 client.26147 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:03.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.523+0000 7f64f051f740 10 client.26147 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:18:03.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.524+0000 7f64f051f740 20 client.26147 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:03.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.524+0000 7f64f051f740 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55d0e674c9f0 0x55d0e6b50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.524+0000 7f64f051f740 4 client.26147 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:03.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.524+0000 7f64f051f740 1 -- 172.21.15.39:0/3678901587 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0e6aaae30 con 0x55d0e674c9f0 2024-07-04T08:18:03.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.524+0000 7f64e3400640 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55d0e674c9f0 0x55d0e6b50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.525+0000 7f64e3400640 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55d0e674c9f0 0x55d0e6b50560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f02eeeb4f1bb67d3 server_cookie=84d5b366c095a2ed in_seq=0 out_seq=0 2024-07-04T08:18:03.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.525+0000 7f64d1600640 10 client.26147 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:18:03.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64d1600640 1 -- 172.21.15.39:0/3678901587 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d0e6aaae30 con 0x55d0e674c9f0 2024-07-04T08:18:03.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64d1600640 10 client.26147 handle_command_reply: tid=0 2024-07-04T08:18:03.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 client.26147 shutdown 2024-07-04T08:18:03.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 -- 172.21.15.39:0/3678901587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 msgr2=0x55d0e6b3ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:03.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b3ea10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009900 tx=0x7f64cc032d30 comp rx=0 tx=0).stop 2024-07-04T08:18:03.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 -- 172.21.15.39:0/3678901587 shutdown_connections 2024-07-04T08:18:03.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55d0e674c9f0 0x55d0e6b50560 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e678e070 0x55d0e6b4a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b3ea10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64d1600640 0 client.26147 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:18:03.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 --2- 172.21.15.39:0/3678901587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e67651b0 0x55d0e6b3e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 -- 172.21.15.39:0/3678901587 >> 172.21.15.39:0/3678901587 conn(0x55d0e6782460 msgr2=0x55d0e6b35b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:03.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 -- 172.21.15.39:0/3678901587 shutdown_connections 2024-07-04T08:18:03.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 1 -- 172.21.15.39:0/3678901587 wait complete. 2024-07-04T08:18:03.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.526+0000 7f64f051f740 20 client.26147 trim_cache size 0 max 16384 2024-07-04T08:18:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.543+0000 7f64f051f740 1 Processor -- start 2024-07-04T08:18:03.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.543+0000 7f64f051f740 1 -- start start 2024-07-04T08:18:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.543+0000 7f64f051f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 0x55d0e6b2f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.543+0000 7f64f051f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e676d390 0x55d0e6b34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.543+0000 7f64f051f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 0x55d0e6b4a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.543+0000 7f64f051f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41b67a0 con 0x55d0e676d390 2024-07-04T08:18:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.543+0000 7f64f051f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41b82e0 con 0x55d0e67651b0 2024-07-04T08:18:03.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.543+0000 7f64f051f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41b6e70 con 0x55d0e678e070 2024-07-04T08:18:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.543+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e676d390 0x55d0e6b34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e676d390 0x55d0e6b34090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47070/0 (socket says 172.21.15.39:47070) 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64dbe00640 1 -- 172.21.15.39:0/3607164380 learned_addr learned my addr 172.21.15.39:0/3607164380 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64e3400640 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 0x55d0e6b2f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64e3400640 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 0x55d0e6b4a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64dbe00640 1 -- 172.21.15.39:0/3607164380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 msgr2=0x55d0e6b4a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 0x55d0e6b4a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64dbe00640 1 -- 172.21.15.39:0/3607164380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 msgr2=0x55d0e6b2f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 0x55d0e6b2f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64dbe00640 1 -- 172.21.15.39:0/3607164380 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e6b34c40 con 0x55d0e676d390 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64e3400640 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 0x55d0e6b2f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.544+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e676d390 0x55d0e6b34090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009800 tx=0x7f64cc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597527be4f355aed server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.545+0000 7f64d3e00640 1 -- 172.21.15.39:0/3607164380 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc040040 con 0x55d0e676d390 2024-07-04T08:18:03.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.545+0000 7f64d3e00640 1 -- 172.21.15.39:0/3607164380 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64cc004750 con 0x55d0e676d390 2024-07-04T08:18:03.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.545+0000 7f64d3e00640 1 -- 172.21.15.39:0/3607164380 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0035f0 con 0x55d0e676d390 2024-07-04T08:18:03.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 -- 172.21.15.39:0/3607164380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e676d390 msgr2=0x55d0e6b34090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:03.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e676d390 0x55d0e6b34090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009800 tx=0x7f64cc001e70 comp rx=0 tx=0).stop 2024-07-04T08:18:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 -- 172.21.15.39:0/3607164380 shutdown_connections 2024-07-04T08:18:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e676d390 0x55d0e6b34090 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 0x55d0e6b4a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 --2- 172.21.15.39:0/3607164380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e67651b0 0x55d0e6b2f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 -- 172.21.15.39:0/3607164380 >> 172.21.15.39:0/3607164380 conn(0x55d0e6782460 msgr2=0x55d0e6b33980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 -- 172.21.15.39:0/3607164380 shutdown_connections 2024-07-04T08:18:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 -- 172.21.15.39:0/3607164380 wait complete. 2024-07-04T08:18:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.546+0000 7f64f051f740 1 -- start start 2024-07-04T08:18:03.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.547+0000 7f64f051f740 1 Processor -- start 2024-07-04T08:18:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.547+0000 7f64f051f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e67651b0 0x55d0e6b46c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.547+0000 7f64f051f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.547+0000 7f64f051f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 0x55d0e6b42840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.547+0000 7f64f051f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41b7c10 con 0x55d0e67651b0 2024-07-04T08:18:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.547+0000 7f64f051f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41b7540 con 0x55d0e676d390 2024-07-04T08:18:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.547+0000 7f64f051f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41b4780 con 0x55d0e678e070 2024-07-04T08:18:03.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.547+0000 7f64f051f740 10 client.0 _ll_register_callbacks cb 0x7f64dc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64e3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e67651b0 0x55d0e6b46c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64e3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e67651b0 0x55d0e6b46c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47080/0 (socket says 172.21.15.39:47080) 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64e3400640 1 -- 172.21.15.39:0/3302670901 learned_addr learned my addr 172.21.15.39:0/3302670901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64e3400640 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 0x55d0e6b42840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64dbe00640 1 -- 172.21.15.39:0/3302670901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 msgr2=0x55d0e6b42840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 0x55d0e6b42840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64dbe00640 1 -- 172.21.15.39:0/3302670901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e67651b0 msgr2=0x55d0e6b46c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e67651b0 0x55d0e6b46c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.548+0000 7f64dbe00640 1 -- 172.21.15.39:0/3302670901 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e6bb7cf0 con 0x55d0e676d390 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.549+0000 7f64dbe00640 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b42130 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f64cc004cc0 tx=0x7f64cc032d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b77a6675abd2c981 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:03.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.549+0000 7f64d1600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.549+0000 7f64d1600640 1 -- 172.21.15.39:0/3302670901 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc05f060 con 0x55d0e676d390 2024-07-04T08:18:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.549+0000 7f64d1600640 1 -- 172.21.15.39:0/3302670901 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f64cc0035f0 con 0x55d0e676d390 2024-07-04T08:18:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.549+0000 7f64f051f740 1 -- 172.21.15.39:0/3302670901 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64cc004750 con 0x55d0e676d390 2024-07-04T08:18:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.549+0000 7f64d1600640 1 -- 172.21.15.39:0/3302670901 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64cc040070 con 0x55d0e676d390 2024-07-04T08:18:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.549+0000 7f64f051f740 10 client.26153 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:03.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.549+0000 7f64f051f740 1 -- 172.21.15.39:0/3302670901 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0e6bb8e10 con 0x55d0e676d390 2024-07-04T08:18:03.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.550+0000 7f64d1600640 1 -- 172.21.15.39:0/3302670901 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64cc004750 con 0x55d0e676d390 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.550+0000 7f64d1600640 1 -- 172.21.15.39:0/3302670901 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f64cc005090 con 0x55d0e676d390 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.550+0000 7f64f051f740 10 client.26153 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.550+0000 7f64f051f740 10 client.26153 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.550+0000 7f64f051f740 20 client.26153 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.550+0000 7f64f051f740 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55d0e6721da0 0x55d0e6bbb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.550+0000 7f64f051f740 4 client.26153 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.550+0000 7f64f051f740 1 -- 172.21.15.39:0/3302670901 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0e6aa68d0 con 0x55d0e6721da0 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.551+0000 7f64e3400640 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55d0e6721da0 0x55d0e6bbb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.551+0000 7f64e3400640 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55d0e6721da0 0x55d0e6bbb460 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f4c127d7973f288 server_cookie=92d5304c89597dc5 in_seq=0 out_seq=0 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.551+0000 7f64d1600640 10 client.26153 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.551+0000 7f64d1600640 1 -- 172.21.15.39:0/3302670901 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0e6aa68d0 con 0x55d0e6721da0 2024-07-04T08:18:03.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.551+0000 7f64d1600640 10 client.26153 handle_command_reply: tid=0 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.551+0000 7f64f051f740 1 client.26153 shutdown 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 -- 172.21.15.39:0/3302670901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 msgr2=0x55d0e6b42130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b42130 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f64cc004cc0 tx=0x7f64cc032d30 comp rx=0 tx=0).stop 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 -- 172.21.15.39:0/3302670901 shutdown_connections 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55d0e6721da0 0x55d0e6bbb460 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0e678e070 0x55d0e6b42840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0e676d390 0x55d0e6b42130 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64d1600640 0 client.26153 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 --2- 172.21.15.39:0/3302670901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0e67651b0 0x55d0e6b46c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 -- 172.21.15.39:0/3302670901 >> 172.21.15.39:0/3302670901 conn(0x55d0e6782460 msgr2=0x55d0e6b2f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 -- 172.21.15.39:0/3302670901 shutdown_connections 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 1 -- 172.21.15.39:0/3302670901 wait complete. 2024-07-04T08:18:03.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.552+0000 7f64f051f740 20 client.26153 trim_cache size 0 max 16384 2024-07-04T08:18:03.556 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:03.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 -- 172.21.15.39:0/3546392352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f64c80b2960 msgr2=0x7f64c80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f64c80b2960 0x7f64c80b4e70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55d0e6b315d0 tx=0x7f64d40e6750 comp rx=0 tx=0).stop 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 -- 172.21.15.39:0/3546392352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4109510 msgr2=0x7f64e419fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4109510 0x7f64e419fec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f64d400e9f0 tx=0x7f64d400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 -- 172.21.15.39:0/3546392352 shutdown_connections 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f64c80b2960 0x7f64c80b4e70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64e410f7a0 0x7f64e41a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64e410ee50 0x7f64e41a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 --2- 172.21.15.39:0/3546392352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f64e4109510 0x7f64e419fec0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 -- 172.21.15.39:0/3546392352 >> 172.21.15.39:0/3546392352 conn(0x7f64e4078b10 msgr2=0x7f64e4079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 -- 172.21.15.39:0/3546392352 shutdown_connections 2024-07-04T08:18:03.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:03.553+0000 7f64e3e00640 1 -- 172.21.15.39:0/3546392352 wait complete. 2024-07-04T08:18:04.517 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:18:05.004 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:05.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:04 smithi039 ceph-mon[32062]: pgmap v566: 129 pgs: 129 active+clean; 2.2 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-07-04T08:18:05.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:04 smithi050 ceph-mon[36577]: pgmap v566: 129 pgs: 129 active+clean; 2.2 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-07-04T08:18:05.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:04 smithi133 ceph-mon[37050]: pgmap v566: 129 pgs: 129 active+clean; 2.2 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-07-04T08:18:07.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:06 smithi039 ceph-mon[32062]: pgmap v567: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 29 op/s 2024-07-04T08:18:07.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:06 smithi050 ceph-mon[36577]: pgmap v567: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 29 op/s 2024-07-04T08:18:07.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:06 smithi133 ceph-mon[37050]: pgmap v567: 129 pgs: 129 active+clean; 2.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 29 op/s 2024-07-04T08:18:07.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.458+0000 7f3b26800640 1 -- 172.21.15.39:0/4022338611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b20075650 msgr2=0x7f3b2010f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:07.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.458+0000 7f3b26800640 1 --2- 172.21.15.39:0/4022338611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b20075650 0x7f3b2010f0d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009900 tx=0x7f3b0802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.458+0000 7f3b26800640 1 -- 172.21.15.39:0/4022338611 shutdown_connections 2024-07-04T08:18:07.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.458+0000 7f3b26800640 1 --2- 172.21.15.39:0/4022338611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b20075650 0x7f3b2010f0d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.459+0000 7f3b26800640 1 --2- 172.21.15.39:0/4022338611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b2010f610 0x7f3b20111a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.459+0000 7f3b26800640 1 --2- 172.21.15.39:0/4022338611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b20074d00 0x7f3b20075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.459+0000 7f3b26800640 1 -- 172.21.15.39:0/4022338611 >> 172.21.15.39:0/4022338611 conn(0x7f3b2007ada0 msgr2=0x7f3b2007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:07.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.459+0000 7f3b26800640 1 -- 172.21.15.39:0/4022338611 shutdown_connections 2024-07-04T08:18:07.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.459+0000 7f3b26800640 1 -- 172.21.15.39:0/4022338611 wait complete. 2024-07-04T08:18:07.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.460+0000 7f3b26800640 1 Processor -- start 2024-07-04T08:18:07.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.460+0000 7f3b26800640 1 -- start start 2024-07-04T08:18:07.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b26800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b20074d00 0x7f3b2019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b26800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b20075650 0x7f3b2019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b26800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b2010f610 0x7f3b201a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b26800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3b20115200 con 0x7f3b2010f610 2024-07-04T08:18:07.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b26800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3b20115080 con 0x7f3b20075650 2024-07-04T08:18:07.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b26800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3b20115380 con 0x7f3b20074d00 2024-07-04T08:18:07.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b16a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b20075650 0x7f3b2019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b16a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b20075650 0x7f3b2019c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48160/0 (socket says 172.21.15.39:48160) 2024-07-04T08:18:07.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b16a00640 1 -- 172.21.15.39:0/2260209939 learned_addr learned my addr 172.21.15.39:0/2260209939 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:07.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b1fe00640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b2010f610 0x7f3b201a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.461+0000 7f3b1f400640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b20074d00 0x7f3b2019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.462+0000 7f3b16a00640 1 -- 172.21.15.39:0/2260209939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b20074d00 msgr2=0x7f3b2019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:07.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.462+0000 7f3b16a00640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b20074d00 0x7f3b2019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.462+0000 7f3b16a00640 1 -- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b2010f610 msgr2=0x7f3b201a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:07.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.462+0000 7f3b16a00640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b2010f610 0x7f3b201a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.462+0000 7f3b16a00640 1 -- 172.21.15.39:0/2260209939 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b201a1c20 con 0x7f3b20075650 2024-07-04T08:18:07.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.462+0000 7f3b16a00640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b20075650 0x7f3b2019c250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b08005ab0 tx=0x7f3b08003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:07.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.462+0000 7f3b1cc00640 1 -- 172.21.15.39:0/2260209939 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0803f070 con 0x7f3b20075650 2024-07-04T08:18:07.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.463+0000 7f3b26800640 1 -- 172.21.15.39:0/2260209939 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b201a1eb0 con 0x7f3b20075650 2024-07-04T08:18:07.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.463+0000 7f3b26800640 1 -- 172.21.15.39:0/2260209939 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b201a8670 con 0x7f3b20075650 2024-07-04T08:18:07.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.463+0000 7f3b1cc00640 1 -- 172.21.15.39:0/2260209939 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b0803b910 con 0x7f3b20075650 2024-07-04T08:18:07.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.463+0000 7f3b1cc00640 1 -- 172.21.15.39:0/2260209939 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08049060 con 0x7f3b20075650 2024-07-04T08:18:07.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.466+0000 7f3b2cec8740 1 Processor -- start 2024-07-04T08:18:07.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.466+0000 7f3b2cec8740 1 -- start start 2024-07-04T08:18:07.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d99b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 0x55e37d99bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d99c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e37d911f00 con 0x55e37d5d6390 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e37d9014a0 con 0x55e37d5f7070 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e37d9054e0 con 0x55e37d5ce1b0 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b1cc00640 1 -- 172.21.15.39:0/2260209939 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3b08042070 con 0x7f3b20075650 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b1fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d99b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b1fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d99b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48168/0 (socket says 172.21.15.39:48168) 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b1fe00640 1 -- 172.21.15.39:0/2743584663 learned_addr learned my addr 172.21.15.39:0/2743584663 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b16a00640 1 --2- 172.21.15.39:0/2743584663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d99c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b1fe00640 1 -- 172.21.15.39:0/2743584663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 msgr2=0x55e37d99bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.467+0000 7f3b1fe00640 1 --2- 172.21.15.39:0/2743584663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 0x55e37d99bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.468+0000 7f3b1fe00640 1 -- 172.21.15.39:0/2743584663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 msgr2=0x55e37d99c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.468+0000 7f3b1fe00640 1 --2- 172.21.15.39:0/2743584663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d99c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.468+0000 7f3b1fe00640 1 -- 172.21.15.39:0/2743584663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37d899670 con 0x55e37d5f7070 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.468+0000 7f3b16a00640 1 --2- 172.21.15.39:0/2743584663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d99c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.468+0000 7f3b1fe00640 1 --2- 172.21.15.39:0/2743584663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d99b7d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009f90 tx=0x7f3b18010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c55aa914740927 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.468+0000 7f3b14c00640 1 -- 172.21.15.39:0/2743584663 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18011700 con 0x55e37d5f7070 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.468+0000 7f3b14c00640 1 -- 172.21.15.39:0/2743584663 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b18011d50 con 0x55e37d5f7070 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.468+0000 7f3b14c00640 1 -- 172.21.15.39:0/2743584663 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18019da0 con 0x55e37d5f7070 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.469+0000 7f3b1cc00640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3b040b2960 0x7f3b040b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.469+0000 7f3b1cc00640 1 -- 172.21.15.39:0/2260209939 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f3b08102ff0 con 0x7f3b20075650 2024-07-04T08:18:07.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.469+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2743584663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 msgr2=0x55e37d99b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.469+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2743584663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d99b7d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009f90 tx=0x7f3b18010860 comp rx=0 tx=0).stop 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.470+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2743584663 shutdown_connections 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.470+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2743584663 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d99c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.470+0000 7f3b1f400640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3b040b2960 0x7f3b040b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.470+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2743584663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 0x55e37d99bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.470+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2743584663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d99b7d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.470+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2743584663 >> 172.21.15.39:0/2743584663 conn(0x55e37d5eb460 msgr2=0x55e37d9996c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.470+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2743584663 shutdown_connections 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.470+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2743584663 wait complete. 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.471+0000 7f3b1f400640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3b040b2960 0x7f3b040b4e70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55e37d99a450 tx=0x7f3b100046c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.471+0000 7f3b2cec8740 1 -- start start 2024-07-04T08:18:07.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.471+0000 7f3b2cec8740 1 Processor -- start 2024-07-04T08:18:07.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 0x55e37d9a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5d6390 0x55e37d9a7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5f7070 0x55e37d9b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e37d765700 con 0x55e37d5f7070 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e37d8a8bd0 con 0x55e37d5d6390 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e37d9a9fb0 con 0x55e37d5ce1b0 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b2cec8740 10 client.0 _ll_register_callbacks cb 0x55e37d8fa590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b1fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5f7070 0x55e37d9b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b1fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5f7070 0x55e37d9b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47132/0 (socket says 172.21.15.39:47132) 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b1fe00640 1 -- 172.21.15.39:0/2026812502 learned_addr learned my addr 172.21.15.39:0/2026812502 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.472+0000 7f3b1f400640 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 0x55e37d9a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.473+0000 7f3b16a00640 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5d6390 0x55e37d9a7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.473+0000 7f3b1f400640 1 -- 172.21.15.39:0/2026812502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5d6390 msgr2=0x55e37d9a7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.473+0000 7f3b1f400640 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5d6390 0x55e37d9a7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.473+0000 7f3b1f400640 1 -- 172.21.15.39:0/2026812502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5f7070 msgr2=0x55e37d9b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.473+0000 7f3b1f400640 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5f7070 0x55e37d9b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.473+0000 7f3b1f400640 1 -- 172.21.15.39:0/2026812502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37d662650 con 0x55e37d5ce1b0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.473+0000 7f3b1f400640 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 0x55e37d9a7320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3b100126d0 tx=0x7f3b10005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a55c664e15361a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.473+0000 7f3b0d600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.474+0000 7f3b0d600640 1 -- 172.21.15.39:0/2026812502 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000fa60 con 0x55e37d5ce1b0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.474+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2026812502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b18011d50 con 0x55e37d5ce1b0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.474+0000 7f3b0d600640 1 -- 172.21.15.39:0/2026812502 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f3b1000fc00 con 0x55e37d5ce1b0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.474+0000 7f3b0d600640 1 -- 172.21.15.39:0/2026812502 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b10030050 con 0x55e37d5ce1b0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.474+0000 7f3b2cec8740 10 client.26119 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.474+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2026812502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e37d9b5fd0 con 0x55e37d5ce1b0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.474+0000 7f3b0d600640 1 -- 172.21.15.39:0/2026812502 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b18011d50 con 0x55e37d5ce1b0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.475+0000 7f3b0d600640 1 -- 172.21.15.39:0/2026812502 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3b10027400 con 0x55e37d5ce1b0 2024-07-04T08:18:07.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.475+0000 7f3b2cec8740 10 client.26119 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.475+0000 7f3b2cec8740 10 client.26119 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.475+0000 7f3b2cec8740 20 client.26119 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.475+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55e37d5b59f0 0x55e37d9b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.475+0000 7f3b2cec8740 4 client.26119 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.475+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2026812502 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e37d913e50 con 0x55e37d5b59f0 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.475+0000 7f3b16a00640 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55e37d5b59f0 0x55e37d9b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.476+0000 7f3b16a00640 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55e37d5b59f0 0x55e37d9b94c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d584313a5d90cb9 server_cookie=dee7fcc025690648 in_seq=0 out_seq=0 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.476+0000 7f3b0d600640 10 client.26119 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b0d600640 1 -- 172.21.15.39:0/2026812502 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e37d913e50 con 0x55e37d5b59f0 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b0d600640 10 client.26119 handle_command_reply: tid=0 2024-07-04T08:18:07.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 client.26119 shutdown 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2026812502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 msgr2=0x55e37d9a7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 0x55e37d9a7320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3b100126d0 tx=0x7f3b10005f90 comp rx=0 tx=0).stop 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2026812502 shutdown_connections 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55e37d5b59f0 0x55e37d9b94c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5f7070 0x55e37d9b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5d6390 0x55e37d9a7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b0d600640 0 client.26119 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/2026812502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5ce1b0 0x55e37d9a7320 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2026812502 >> 172.21.15.39:0/2026812502 conn(0x55e37d5eb460 msgr2=0x55e37d99e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2026812502 shutdown_connections 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 1 -- 172.21.15.39:0/2026812502 wait complete. 2024-07-04T08:18:07.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.477+0000 7f3b2cec8740 20 client.26119 trim_cache size 0 max 16384 2024-07-04T08:18:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.493+0000 7f3b2cec8740 1 Processor -- start 2024-07-04T08:18:07.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.493+0000 7f3b2cec8740 1 -- start start 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5ce1b0 0x55e37d998960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d999730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5f7070 0x55e37d9b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc00fdd0 con 0x55e37d5d6390 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc011910 con 0x55e37d5ce1b0 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc0104a0 con 0x55e37d5f7070 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b1f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5ce1b0 0x55e37d998960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5f7070 0x55e37d9b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b1f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5ce1b0 0x55e37d998960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48190/0 (socket says 172.21.15.39:48190) 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b1f400640 1 -- 172.21.15.39:0/1367276581 learned_addr learned my addr 172.21.15.39:0/1367276581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:07.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b16a00640 1 --2- 172.21.15.39:0/1367276581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d999730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b1f400640 1 -- 172.21.15.39:0/1367276581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5f7070 msgr2=0x55e37d9b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:07.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.494+0000 7f3b1f400640 1 --2- 172.21.15.39:0/1367276581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5f7070 0x55e37d9b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.495+0000 7f3b1f400640 1 -- 172.21.15.39:0/1367276581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 msgr2=0x55e37d999730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:07.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.495+0000 7f3b1f400640 1 --2- 172.21.15.39:0/1367276581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d999730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.495+0000 7f3b1f400640 1 -- 172.21.15.39:0/1367276581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e37d99e060 con 0x55e37d5ce1b0 2024-07-04T08:18:07.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.495+0000 7f3b16a00640 1 --2- 172.21.15.39:0/1367276581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d999730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:18:07.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.495+0000 7f3b1f400640 1 --2- 172.21.15.39:0/1367276581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5ce1b0 0x55e37d998960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000a010 tx=0x7f3b10009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b52a80091f2262 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:07.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.495+0000 7f3b0fe00640 1 -- 172.21.15.39:0/1367276581 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10027400 con 0x55e37d5ce1b0 2024-07-04T08:18:07.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.495+0000 7f3b0fe00640 1 -- 172.21.15.39:0/1367276581 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b10038070 con 0x55e37d5ce1b0 2024-07-04T08:18:07.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.495+0000 7f3b0fe00640 1 -- 172.21.15.39:0/1367276581 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000fac0 con 0x55e37d5ce1b0 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.496+0000 7f3b2cec8740 1 -- 172.21.15.39:0/1367276581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5ce1b0 msgr2=0x55e37d998960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.496+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/1367276581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5ce1b0 0x55e37d998960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000a010 tx=0x7f3b10009280 comp rx=0 tx=0).stop 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.496+0000 7f3b2cec8740 1 -- 172.21.15.39:0/1367276581 shutdown_connections 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.496+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/1367276581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5d6390 0x55e37d999730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.496+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/1367276581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5f7070 0x55e37d9b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.496+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/1367276581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5ce1b0 0x55e37d998960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.496+0000 7f3b2cec8740 1 -- 172.21.15.39:0/1367276581 >> 172.21.15.39:0/1367276581 conn(0x55e37d5eb460 msgr2=0x55e37d999020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.496+0000 7f3b2cec8740 1 -- 172.21.15.39:0/1367276581 shutdown_connections 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.496+0000 7f3b2cec8740 1 -- 172.21.15.39:0/1367276581 wait complete. 2024-07-04T08:18:07.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.497+0000 7f3b2cec8740 1 -- start start 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.497+0000 7f3b2cec8740 1 Processor -- start 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5ce1b0 0x55e37d9be980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5d6390 0x55e37d9beec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b2cec8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d9bf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc011240 con 0x55e37d5ce1b0 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc010b70 con 0x55e37d5f7070 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b2cec8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc00ddb0 con 0x55e37d5d6390 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b2cec8740 10 client.0 _ll_register_callbacks cb 0x7f3b100095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5ce1b0 0x55e37d9be980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5ce1b0 0x55e37d9be980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47144/0 (socket says 172.21.15.39:47144) 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b1f400640 1 -- 172.21.15.39:0/4161294892 learned_addr learned my addr 172.21.15.39:0/4161294892 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:07.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.498+0000 7f3b1fe00640 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d9bf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.499+0000 7f3b1f400640 1 -- 172.21.15.39:0/4161294892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5d6390 msgr2=0x55e37d9beec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:07.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.499+0000 7f3b1f400640 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5d6390 0x55e37d9beec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.499+0000 7f3b1f400640 1 -- 172.21.15.39:0/4161294892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 msgr2=0x55e37d9bf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:07.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.499+0000 7f3b1f400640 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d9bf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.499+0000 7f3b1f400640 1 -- 172.21.15.39:0/4161294892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37d9ae850 con 0x55e37d5ce1b0 2024-07-04T08:18:07.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.499+0000 7f3b1f400640 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5ce1b0 0x55e37d9be980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b10012fd0 tx=0x7f3b10027630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdfa96f8a5d67da server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:07.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.499+0000 7f3b0d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:07.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.499+0000 7f3b0d600640 1 -- 172.21.15.39:0/4161294892 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10041070 con 0x55e37d5ce1b0 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.500+0000 7f3b2cec8740 1 -- 172.21.15.39:0/4161294892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b10038070 con 0x55e37d5ce1b0 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.500+0000 7f3b0d600640 1 -- 172.21.15.39:0/4161294892 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f3b10005250 con 0x55e37d5ce1b0 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.500+0000 7f3b0d600640 1 -- 172.21.15.39:0/4161294892 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3b1004d050 con 0x55e37d5ce1b0 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.500+0000 7f3b2cec8740 10 client.16041 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.500+0000 7f3b2cec8740 1 -- 172.21.15.39:0/4161294892 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e37da21e10 con 0x55e37d5ce1b0 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.500+0000 7f3b0d600640 1 -- 172.21.15.39:0/4161294892 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b10038070 con 0x55e37d5ce1b0 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.500+0000 7f3b0d600640 1 -- 172.21.15.39:0/4161294892 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3b10004850 con 0x55e37d5ce1b0 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.500+0000 7f3b2cec8740 10 client.16041 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.500+0000 7f3b2cec8740 10 client.16041 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.501+0000 7f3b2cec8740 20 client.16041 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.501+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55e37d58ada0 0x55e37da24510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.501+0000 7f3b2cec8740 4 client.16041 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:07.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.501+0000 7f3b2cec8740 1 -- 172.21.15.39:0/4161294892 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e37d90f8f0 con 0x55e37d58ada0 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.501+0000 7f3b16a00640 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55e37d58ada0 0x55e37da24510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.501+0000 7f3b16a00640 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55e37d58ada0 0x55e37da24510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=299523e6567bc06f server_cookie=fa164fa4353805b7 in_seq=0 out_seq=0 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.501+0000 7f3b0d600640 10 client.16041 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b0d600640 1 -- 172.21.15.39:0/4161294892 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e37d90f8f0 con 0x55e37d58ada0 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b0d600640 10 client.16041 handle_command_reply: tid=0 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 client.16041 shutdown 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 -- 172.21.15.39:0/4161294892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5ce1b0 msgr2=0x55e37d9be980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5ce1b0 0x55e37d9be980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b10012fd0 tx=0x7f3b10027630 comp rx=0 tx=0).stop 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 -- 172.21.15.39:0/4161294892 shutdown_connections 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55e37d58ada0 0x55e37da24510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e37d5f7070 0x55e37d9bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e37d5d6390 0x55e37d9beec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b0d600640 0 client.16041 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:18:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 --2- 172.21.15.39:0/4161294892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e37d5ce1b0 0x55e37d9be980 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 -- 172.21.15.39:0/4161294892 >> 172.21.15.39:0/4161294892 conn(0x55e37d5eb460 msgr2=0x55e37d99c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 -- 172.21.15.39:0/4161294892 shutdown_connections 2024-07-04T08:18:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 1 -- 172.21.15.39:0/4161294892 wait complete. 2024-07-04T08:18:07.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.502+0000 7f3b2cec8740 20 client.16041 trim_cache size 0 max 16384 2024-07-04T08:18:07.506 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 -- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3b040b2960 msgr2=0x7f3b040b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3b040b2960 0x7f3b040b4e70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55e37d99a450 tx=0x7f3b100046c0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 -- 172.21.15.39:0/2260209939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b20075650 msgr2=0x7f3b2019c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b20075650 0x7f3b2019c250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3b08005ab0 tx=0x7f3b08003b90 comp rx=0 tx=0).stop 2024-07-04T08:18:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 -- 172.21.15.39:0/2260209939 shutdown_connections 2024-07-04T08:18:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3b040b2960 0x7f3b040b4e70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b2010f610 0x7f3b201a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3b20075650 0x7f3b2019c250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 --2- 172.21.15.39:0/2260209939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b20074d00 0x7f3b2019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:07.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 -- 172.21.15.39:0/2260209939 >> 172.21.15.39:0/2260209939 conn(0x7f3b2007ada0 msgr2=0x7f3b201081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:07.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.503+0000 7f3b26800640 1 -- 172.21.15.39:0/2260209939 shutdown_connections 2024-07-04T08:18:07.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:07.504+0000 7f3b26800640 1 -- 172.21.15.39:0/2260209939 wait complete. 2024-07-04T08:18:08.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:18:08.979 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:09.132 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:08 smithi039 ceph-mon[32062]: pgmap v568: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-07-04T08:18:09.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:08 smithi050 ceph-mon[36577]: pgmap v568: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-07-04T08:18:09.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:08 smithi133 ceph-mon[37050]: pgmap v568: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-07-04T08:18:11.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:10 smithi039 ceph-mon[32062]: pgmap v569: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-07-04T08:18:11.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:10 smithi050 ceph-mon[36577]: pgmap v569: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-07-04T08:18:11.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:10 smithi133 ceph-mon[37050]: pgmap v569: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 28 op/s 2024-07-04T08:18:11.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.521+0000 7f0238a00640 1 -- 172.21.15.39:0/1673527556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0234103d10 msgr2=0x7f0234104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.521+0000 7f0238a00640 1 --2- 172.21.15.39:0/1673527556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0234103d10 0x7f0234104120 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0224009900 tx=0x7f022402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.522+0000 7f0238a00640 1 -- 172.21.15.39:0/1673527556 shutdown_connections 2024-07-04T08:18:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.522+0000 7f0238a00640 1 --2- 172.21.15.39:0/1673527556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f023410d170 0x7f023410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.522+0000 7f0238a00640 1 --2- 172.21.15.39:0/1673527556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0234104660 0x7f023410cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.522+0000 7f0238a00640 1 --2- 172.21.15.39:0/1673527556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0234103d10 0x7f0234104120 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.522+0000 7f0238a00640 1 -- 172.21.15.39:0/1673527556 >> 172.21.15.39:0/1673527556 conn(0x7f02340fd810 msgr2=0x7f02340ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.522+0000 7f0238a00640 1 -- 172.21.15.39:0/1673527556 shutdown_connections 2024-07-04T08:18:11.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.522+0000 7f0238a00640 1 -- 172.21.15.39:0/1673527556 wait complete. 2024-07-04T08:18:11.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.523+0000 7f0238a00640 1 Processor -- start 2024-07-04T08:18:11.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.523+0000 7f0238a00640 1 -- start start 2024-07-04T08:18:11.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0238a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0234103d10 0x7f023419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0238a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0234104660 0x7f023419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0238a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023410d170 0x7f02341a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0238a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0234112dd0 con 0x7f0234103d10 2024-07-04T08:18:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0238a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0234112c50 con 0x7f0234104660 2024-07-04T08:18:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0238a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0234112f50 con 0x7f023410d170 2024-07-04T08:18:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0229e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0234104660 0x7f023419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0229e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0234104660 0x7f023419bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53578/0 (socket says 172.21.15.39:53578) 2024-07-04T08:18:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0229e00640 1 -- 172.21.15.39:0/3955639046 learned_addr learned my addr 172.21.15.39:0/3955639046 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0232800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0234103d10 0x7f023419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.524+0000 7f0233200640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023410d170 0x7f02341a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.525+0000 7f0229e00640 1 -- 172.21.15.39:0/3955639046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023410d170 msgr2=0x7f02341a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.525+0000 7f0229e00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023410d170 0x7f02341a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.525+0000 7f0229e00640 1 -- 172.21.15.39:0/3955639046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0234103d10 msgr2=0x7f023419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.525+0000 7f0229e00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0234103d10 0x7f023419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.525+0000 7f0229e00640 1 -- 172.21.15.39:0/3955639046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02341a1990 con 0x7f0234104660 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.525+0000 7f0229e00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0234104660 0x7f023419bfc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f021c00e970 tx=0x7f021c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.526+0000 7f022be00640 1 -- 172.21.15.39:0/3955639046 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c00cd80 con 0x7f0234104660 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.526+0000 7f0238a00640 1 -- 172.21.15.39:0/3955639046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02341a1c80 con 0x7f0234104660 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.526+0000 7f022be00640 1 -- 172.21.15.39:0/3955639046 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f021c0035f0 con 0x7f0234104660 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.526+0000 7f022be00640 1 -- 172.21.15.39:0/3955639046 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c010690 con 0x7f0234104660 2024-07-04T08:18:11.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.526+0000 7f0238a00640 1 -- 172.21.15.39:0/3955639046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02341a83e0 con 0x7f0234104660 2024-07-04T08:18:11.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.529+0000 7f023fac4740 1 Processor -- start 2024-07-04T08:18:11.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.530+0000 7f023fac4740 1 -- start start 2024-07-04T08:18:11.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.530+0000 7f022be00640 1 -- 172.21.15.39:0/3955639046 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f021c003aa0 con 0x7f0234104660 2024-07-04T08:18:11.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.530+0000 7f023fac4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c15950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.530+0000 7f023fac4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c15e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.530+0000 7f023fac4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.530+0000 7f023fac4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ad04b8bee0 con 0x55ad04850390 2024-07-04T08:18:11.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.530+0000 7f023fac4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad04b7b480 con 0x55ad04871070 2024-07-04T08:18:11.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.530+0000 7f023fac4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad04b7f4c0 con 0x55ad048481b0 2024-07-04T08:18:11.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.530+0000 7f0233200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c15950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0229e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0233200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c15950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53592/0 (socket says 172.21.15.39:53592) 2024-07-04T08:18:11.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0233200640 1 -- 172.21.15.39:0/1147400838 learned_addr learned my addr 172.21.15.39:0/1147400838 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:11.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0232800640 1 --2- 172.21.15.39:0/1147400838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c15e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0233200640 1 -- 172.21.15.39:0/1147400838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 msgr2=0x55ad04c15e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:11.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0233200640 1 --2- 172.21.15.39:0/1147400838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c15e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0233200640 1 -- 172.21.15.39:0/1147400838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 msgr2=0x55ad04c163d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:11.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0233200640 1 --2- 172.21.15.39:0/1147400838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c163d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0233200640 1 -- 172.21.15.39:0/1147400838 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad04b13650 con 0x55ad04871070 2024-07-04T08:18:11.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0232800640 1 --2- 172.21.15.39:0/1147400838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c15e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:11.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.531+0000 7f0233200640 1 --2- 172.21.15.39:0/1147400838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c15950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f022c009fd0 tx=0x7f022c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7109c0f0aa8e9336 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:11.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.532+0000 7f0223e00640 1 -- 172.21.15.39:0/1147400838 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c004d20 con 0x55ad04871070 2024-07-04T08:18:11.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.532+0000 7f0223e00640 1 -- 172.21.15.39:0/1147400838 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f022c004050 con 0x55ad04871070 2024-07-04T08:18:11.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.532+0000 7f0223e00640 1 -- 172.21.15.39:0/1147400838 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c005280 con 0x55ad04871070 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.532+0000 7f022be00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f02140b2960 0x7f02140b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f023fac4740 1 -- 172.21.15.39:0/1147400838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 msgr2=0x55ad04c15950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f023fac4740 1 --2- 172.21.15.39:0/1147400838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c15950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f022c009fd0 tx=0x7f022c0118d0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f022be00640 1 -- 172.21.15.39:0/3955639046 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f021c0e2c30 con 0x7f0234104660 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f0232800640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f02140b2960 0x7f02140b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f023fac4740 1 -- 172.21.15.39:0/1147400838 shutdown_connections 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f023fac4740 1 --2- 172.21.15.39:0/1147400838 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f023fac4740 1 --2- 172.21.15.39:0/1147400838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c15e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f023fac4740 1 --2- 172.21.15.39:0/1147400838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c15950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f023fac4740 1 -- 172.21.15.39:0/1147400838 >> 172.21.15.39:0/1147400838 conn(0x55ad04865460 msgr2=0x55ad04c13840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f023fac4740 1 -- 172.21.15.39:0/1147400838 shutdown_connections 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.533+0000 7f023fac4740 1 -- 172.21.15.39:0/1147400838 wait complete. 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.534+0000 7f0232800640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f02140b2960 0x7f02140b4e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0224031440 tx=0x7f0224005890 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.534+0000 7f023fac4740 1 -- start start 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.534+0000 7f023fac4740 1 Processor -- start 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f023fac4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad048481b0 0x55ad04c21550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f023fac4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c21ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f023fac4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad04871070 0x55ad04c2d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f023fac4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ad049df700 con 0x55ad04850390 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f023fac4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad04b22bb0 con 0x55ad048481b0 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f023fac4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad04c24130 con 0x55ad04871070 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f023fac4740 10 client.0 _ll_register_callbacks cb 0x55ad04b74570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f0229e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c21ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f0232800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad048481b0 0x55ad04c21550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f0229e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c21ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57232/0 (socket says 172.21.15.39:57232) 2024-07-04T08:18:11.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f0229e00640 1 -- 172.21.15.39:0/444371472 learned_addr learned my addr 172.21.15.39:0/444371472 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:11.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.535+0000 7f0233200640 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad04871070 0x55ad04c2d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.536+0000 7f0232800640 1 -- 172.21.15.39:0/444371472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad04871070 msgr2=0x55ad04c2d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:11.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.536+0000 7f0232800640 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad04871070 0x55ad04c2d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.536+0000 7f0232800640 1 -- 172.21.15.39:0/444371472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 msgr2=0x55ad04c21ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:11.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.536+0000 7f0232800640 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c21ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.536+0000 7f0232800640 1 -- 172.21.15.39:0/444371472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad048dc650 con 0x55ad048481b0 2024-07-04T08:18:11.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.536+0000 7f0233200640 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad04871070 0x55ad04c2d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:11.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.536+0000 7f0229e00640 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c21ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.536+0000 7f0232800640 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad048481b0 0x55ad04c21550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0224009230 tx=0x7f0224009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c12db0d038cc045 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.536+0000 7f0220c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.537+0000 7f0220c00640 1 -- 172.21.15.39:0/444371472 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022404c070 con 0x55ad048481b0 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.537+0000 7f0220c00640 1 -- 172.21.15.39:0/444371472 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0224005230 con 0x55ad048481b0 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.537+0000 7f023fac4740 1 -- 172.21.15.39:0/444371472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f022c004050 con 0x55ad048481b0 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.537+0000 7f0220c00640 1 -- 172.21.15.39:0/444371472 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0224058050 con 0x55ad048481b0 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.537+0000 7f023fac4740 10 client.26183 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.537+0000 7f023fac4740 1 -- 172.21.15.39:0/444371472 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad04c30190 con 0x55ad048481b0 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.538+0000 7f0220c00640 1 -- 172.21.15.39:0/444371472 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f022c004050 con 0x55ad048481b0 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.538+0000 7f0220c00640 1 -- 172.21.15.39:0/444371472 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f02240035f0 con 0x55ad048481b0 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.538+0000 7f023fac4740 10 client.26183 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.538+0000 7f023fac4740 10 client.26183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:18:11.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.538+0000 7f023fac4740 20 client.26183 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:11.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.538+0000 7f023fac4740 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ad0482f9f0 0x55ad04c336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.538+0000 7f023fac4740 4 client.26183 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:11.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.538+0000 7f023fac4740 1 -- 172.21.15.39:0/444371472 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad04b8de30 con 0x55ad0482f9f0 2024-07-04T08:18:11.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.539+0000 7f0229e00640 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ad0482f9f0 0x55ad04c336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.541+0000 7f0229e00640 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ad0482f9f0 0x55ad04c336d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0e9ca8ac63cf33 server_cookie=5acdde00181dd163 in_seq=0 out_seq=0 2024-07-04T08:18:11.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.541+0000 7f0220c00640 10 client.26183 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:18:11.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.542+0000 7f0220c00640 1 -- 172.21.15.39:0/444371472 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ad04b8de30 con 0x55ad0482f9f0 2024-07-04T08:18:11.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.542+0000 7f0220c00640 10 client.26183 handle_command_reply: tid=0 2024-07-04T08:18:11.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.542+0000 7f023fac4740 1 client.26183 shutdown 2024-07-04T08:18:11.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.542+0000 7f023fac4740 1 -- 172.21.15.39:0/444371472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad048481b0 msgr2=0x55ad04c21550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.542+0000 7f023fac4740 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad048481b0 0x55ad04c21550 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0224009230 tx=0x7f0224009260 comp rx=0 tx=0).stop 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f023fac4740 1 -- 172.21.15.39:0/444371472 shutdown_connections 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f023fac4740 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ad0482f9f0 0x55ad04c336d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f023fac4740 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c21ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f0220c00640 0 client.26183 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f023fac4740 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad04871070 0x55ad04c2d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f023fac4740 1 --2- 172.21.15.39:0/444371472 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad048481b0 0x55ad04c21550 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f023fac4740 1 -- 172.21.15.39:0/444371472 >> 172.21.15.39:0/444371472 conn(0x55ad04865460 msgr2=0x55ad04c18b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f023fac4740 1 -- 172.21.15.39:0/444371472 shutdown_connections 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f023fac4740 1 -- 172.21.15.39:0/444371472 wait complete. 2024-07-04T08:18:11.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.543+0000 7f023fac4740 20 client.26183 trim_cache size 0 max 16384 2024-07-04T08:18:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.559+0000 7f023fac4740 1 Processor -- start 2024-07-04T08:18:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.559+0000 7f023fac4740 1 -- start start 2024-07-04T08:18:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.559+0000 7f023fac4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f023fac4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c2d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f023fac4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c17090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f023fac4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b2320 con 0x55ad04850390 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f023fac4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b3e60 con 0x55ad04871070 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f023fac4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b29f0 con 0x55ad048481b0 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0229e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c2d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0229e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c2d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57238/0 (socket says 172.21.15.39:57238) 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0229e00640 1 -- 172.21.15.39:0/1115050537 learned_addr learned my addr 172.21.15.39:0/1115050537 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0232800640 1 --2- 172.21.15.39:0/1115050537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0233200640 1 --2- 172.21.15.39:0/1115050537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c17090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0229e00640 1 -- 172.21.15.39:0/1115050537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 msgr2=0x55ad04c19410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0229e00640 1 --2- 172.21.15.39:0/1115050537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c19410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0229e00640 1 -- 172.21.15.39:0/1115050537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 msgr2=0x55ad04c17090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0229e00640 1 --2- 172.21.15.39:0/1115050537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c17090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.560+0000 7f0229e00640 1 -- 172.21.15.39:0/1115050537 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad04c14520 con 0x55ad04850390 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.561+0000 7f0229e00640 1 --2- 172.21.15.39:0/1115050537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c2d210 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f021c0a6120 tx=0x7f021c0e5080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4371838f9523c1c2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.561+0000 7f0223400640 1 -- 172.21.15.39:0/1115050537 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c0a5310 con 0x55ad04850390 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.561+0000 7f0223400640 1 -- 172.21.15.39:0/1115050537 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f021c009460 con 0x55ad04850390 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.561+0000 7f0223400640 1 -- 172.21.15.39:0/1115050537 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c0b29e0 con 0x55ad04850390 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.562+0000 7f023fac4740 1 -- 172.21.15.39:0/1115050537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 msgr2=0x55ad04c2d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.562+0000 7f023fac4740 1 --2- 172.21.15.39:0/1115050537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c2d210 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f021c0a6120 tx=0x7f021c0e5080 comp rx=0 tx=0).stop 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.562+0000 7f023fac4740 1 -- 172.21.15.39:0/1115050537 shutdown_connections 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.562+0000 7f023fac4740 1 --2- 172.21.15.39:0/1115050537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c2d210 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.562+0000 7f023fac4740 1 --2- 172.21.15.39:0/1115050537 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c17090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.562+0000 7f023fac4740 1 --2- 172.21.15.39:0/1115050537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c19410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.562+0000 7f023fac4740 1 -- 172.21.15.39:0/1115050537 >> 172.21.15.39:0/1115050537 conn(0x55ad04865460 msgr2=0x55ad04c16980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.562+0000 7f023fac4740 1 -- 172.21.15.39:0/1115050537 shutdown_connections 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.562+0000 7f023fac4740 1 -- 172.21.15.39:0/1115050537 wait complete. 2024-07-04T08:18:11.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.563+0000 7f023fac4740 1 -- start start 2024-07-04T08:18:11.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.563+0000 7f023fac4740 1 Processor -- start 2024-07-04T08:18:11.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.563+0000 7f023fac4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c24b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.563+0000 7f023fac4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c25070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.563+0000 7f023fac4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c99e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.563+0000 7f023fac4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b3790 con 0x55ad04850390 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.563+0000 7f023fac4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b30c0 con 0x55ad04871070 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.563+0000 7f023fac4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f02341b0300 con 0x55ad048481b0 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f023fac4740 10 client.0 _ll_register_callbacks cb 0x7f021c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0229e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c25070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0233200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c99e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0229e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c25070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57242/0 (socket says 172.21.15.39:57242) 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0229e00640 1 -- 172.21.15.39:0/3961620724 learned_addr learned my addr 172.21.15.39:0/3961620724 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0232800640 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c24b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0233200640 1 -- 172.21.15.39:0/3961620724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 msgr2=0x55ad04c24b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0233200640 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c24b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0233200640 1 -- 172.21.15.39:0/3961620724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 msgr2=0x55ad04c25070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0233200640 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c25070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0233200640 1 -- 172.21.15.39:0/3961620724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad04c9a440 con 0x55ad04871070 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0232800640 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c24b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.564+0000 7f0229e00640 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c25070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.565+0000 7f0233200640 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c99e50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f022c009fa0 tx=0x7f022c003f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=356c6c7c6cdbcb96 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.565+0000 7f0220c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.565+0000 7f0220c00640 1 -- 172.21.15.39:0/3961620724 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c0054d0 con 0x55ad04871070 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.565+0000 7f0220c00640 1 -- 172.21.15.39:0/3961620724 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f022c005670 con 0x55ad04871070 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.565+0000 7f023fac4740 1 -- 172.21.15.39:0/3961620724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ad04b898d0 con 0x55ad04871070 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.566+0000 7f0220c00640 1 -- 172.21.15.39:0/3961620724 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f022c034050 con 0x55ad04871070 2024-07-04T08:18:11.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.566+0000 7f023fac4740 10 client.26189 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.566+0000 7f023fac4740 1 -- 172.21.15.39:0/3961620724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad04c9b6a0 con 0x55ad04871070 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.566+0000 7f0220c00640 1 -- 172.21.15.39:0/3961620724 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ad04b898d0 con 0x55ad04871070 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.567+0000 7f0220c00640 1 -- 172.21.15.39:0/3961620724 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f022c019430 con 0x55ad04871070 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.567+0000 7f023fac4740 10 client.26189 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.567+0000 7f023fac4740 10 client.26189 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.567+0000 7f023fac4740 20 client.26189 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.567+0000 7f023fac4740 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ad04804da0 0x55ad04c9db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.567+0000 7f023fac4740 4 client.26189 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.567+0000 7f023fac4740 1 -- 172.21.15.39:0/3961620724 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad04b87010 con 0x55ad04804da0 2024-07-04T08:18:11.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.567+0000 7f0232800640 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ad04804da0 0x55ad04c9db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:11.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f0232800640 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ad04804da0 0x55ad04c9db80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=428113794647d6ca server_cookie=f211118bb4ed3f42 in_seq=0 out_seq=0 2024-07-04T08:18:11.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f0220c00640 10 client.26189 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:18:11.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f0220c00640 1 -- 172.21.15.39:0/3961620724 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad04b87010 con 0x55ad04804da0 2024-07-04T08:18:11.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f0220c00640 10 client.26189 handle_command_reply: tid=0 2024-07-04T08:18:11.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f023fac4740 1 client.26189 shutdown 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f023fac4740 1 -- 172.21.15.39:0/3961620724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 msgr2=0x55ad04c99e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f023fac4740 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c99e50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f022c009fa0 tx=0x7f022c003f00 comp rx=0 tx=0).stop 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f023fac4740 1 -- 172.21.15.39:0/3961620724 shutdown_connections 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f023fac4740 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55ad04804da0 0x55ad04c9db80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f023fac4740 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad04850390 0x55ad04c25070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f023fac4740 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad04871070 0x55ad04c99e50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f023fac4740 1 --2- 172.21.15.39:0/3961620724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad048481b0 0x55ad04c24b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f0220c00640 0 client.26189 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.568+0000 7f023fac4740 1 -- 172.21.15.39:0/3961620724 >> 172.21.15.39:0/3961620724 conn(0x55ad04865460 msgr2=0x55ad04c191e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.569+0000 7f023fac4740 1 -- 172.21.15.39:0/3961620724 shutdown_connections 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.569+0000 7f023fac4740 1 -- 172.21.15.39:0/3961620724 wait complete. 2024-07-04T08:18:11.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.569+0000 7f023fac4740 20 client.26189 trim_cache size 0 max 16384 2024-07-04T08:18:11.573 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:11.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.569+0000 7f0238a00640 1 -- 172.21.15.39:0/3955639046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f02140b2960 msgr2=0x7f02140b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:11.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.569+0000 7f0238a00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f02140b2960 0x7f02140b4e70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0224031440 tx=0x7f0224005890 comp rx=0 tx=0).stop 2024-07-04T08:18:11.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 -- 172.21.15.39:0/3955639046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0234104660 msgr2=0x7f023419bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:11.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0234104660 0x7f023419bfc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f021c00e970 tx=0x7f021c00ee30 comp rx=0 tx=0).stop 2024-07-04T08:18:11.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 -- 172.21.15.39:0/3955639046 shutdown_connections 2024-07-04T08:18:11.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f02140b2960 0x7f02140b4e70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023410d170 0x7f02341a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0234104660 0x7f023419bfc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 --2- 172.21.15.39:0/3955639046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0234103d10 0x7f023419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:11.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 -- 172.21.15.39:0/3955639046 >> 172.21.15.39:0/3955639046 conn(0x7f02340fd810 msgr2=0x7f0234101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:11.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 -- 172.21.15.39:0/3955639046 shutdown_connections 2024-07-04T08:18:11.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:11.570+0000 7f0238a00640 1 -- 172.21.15.39:0/3955639046 wait complete. 2024-07-04T08:18:12.574 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:18:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:13.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:12 smithi050 ceph-mon[36577]: pgmap v570: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 29 op/s 2024-07-04T08:18:13.220 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:12 smithi039 ceph-mon[32062]: pgmap v570: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 29 op/s 2024-07-04T08:18:13.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:12 smithi133 ceph-mon[37050]: pgmap v570: 129 pgs: 129 active+clean; 2.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 42 KiB/s wr, 29 op/s 2024-07-04T08:18:15.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:14 smithi039 ceph-mon[32062]: pgmap v571: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-07-04T08:18:15.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:14 smithi133 ceph-mon[37050]: pgmap v571: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-07-04T08:18:15.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:14 smithi050 ceph-mon[36577]: pgmap v571: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-07-04T08:18:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.400+0000 7fdf2c000640 1 -- 172.21.15.39:0/1424419773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf241089f0 msgr2=0x7fdf24108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.400+0000 7fdf2c000640 1 --2- 172.21.15.39:0/1424419773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf241089f0 0x7fdf24108e00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009900 tx=0x7fdf1002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.401+0000 7fdf2c000640 1 -- 172.21.15.39:0/1424419773 shutdown_connections 2024-07-04T08:18:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.401+0000 7fdf2c000640 1 --2- 172.21.15.39:0/1424419773 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf241089f0 0x7fdf24108e00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.401+0000 7fdf2c000640 1 --2- 172.21.15.39:0/1424419773 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf24109340 0x7fdf2410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.401+0000 7fdf2c000640 1 --2- 172.21.15.39:0/1424419773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf241030b0 0x7fdf241034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.401+0000 7fdf2c000640 1 -- 172.21.15.39:0/1424419773 >> 172.21.15.39:0/1424419773 conn(0x7fdf2407ada0 msgr2=0x7fdf2407b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:15.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.401+0000 7fdf2c000640 1 -- 172.21.15.39:0/1424419773 shutdown_connections 2024-07-04T08:18:15.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.401+0000 7fdf2c000640 1 -- 172.21.15.39:0/1424419773 wait complete. 2024-07-04T08:18:15.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.402+0000 7fdf2c000640 1 Processor -- start 2024-07-04T08:18:15.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.402+0000 7fdf2c000640 1 -- start start 2024-07-04T08:18:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf2c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf241030b0 0x7fdf2419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf2c000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf241089f0 0x7fdf2419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf2c000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf24109340 0x7fdf241a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf2c000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24113020 con 0x7fdf241089f0 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf2c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24112ea0 con 0x7fdf241030b0 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf2c000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241131a0 con 0x7fdf24109340 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf28c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf241089f0 0x7fdf2419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf29600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf241030b0 0x7fdf2419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf29600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf241030b0 0x7fdf2419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53626/0 (socket says 172.21.15.39:53626) 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf28c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf241089f0 0x7fdf2419c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57250/0 (socket says 172.21.15.39:57250) 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf29600640 1 -- 172.21.15.39:0/444491636 learned_addr learned my addr 172.21.15.39:0/444491636 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf2a000640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf24109340 0x7fdf241a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf29600640 1 -- 172.21.15.39:0/444491636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf24109340 msgr2=0x7fdf241a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:15.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.403+0000 7fdf29600640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf24109340 0x7fdf241a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf29600640 1 -- 172.21.15.39:0/444491636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf241089f0 msgr2=0x7fdf2419c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:15.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf29600640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf241089f0 0x7fdf2419c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf29600640 1 -- 172.21.15.39:0/444491636 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf241a1bc0 con 0x7fdf241030b0 2024-07-04T08:18:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf2a000640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf24109340 0x7fdf241a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf29600640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf241030b0 0x7fdf2419bc50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400e9f0 tx=0x7fdf1400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf22000640 1 -- 172.21.15.39:0/444491636 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400cd80 con 0x7fdf241030b0 2024-07-04T08:18:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf2c000640 1 -- 172.21.15.39:0/444491636 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf241a1eb0 con 0x7fdf241030b0 2024-07-04T08:18:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf22000640 1 -- 172.21.15.39:0/444491636 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf140054d0 con 0x7fdf241030b0 2024-07-04T08:18:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf22000640 1 -- 172.21.15.39:0/444491636 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14010690 con 0x7fdf241030b0 2024-07-04T08:18:15.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.404+0000 7fdf2c000640 1 -- 172.21.15.39:0/444491636 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf241a8610 con 0x7fdf241030b0 2024-07-04T08:18:15.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.408+0000 7fdf32721740 1 Processor -- start 2024-07-04T08:18:15.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.408+0000 7fdf32721740 1 -- start start 2024-07-04T08:18:15.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.408+0000 7fdf32721740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ebc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.408+0000 7fdf32721740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470aef1b0 0x561470ebccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.408+0000 7fdf32721740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470af7390 0x561470ebd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.408+0000 7fdf32721740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561470e32ee0 con 0x561470b18070 2024-07-04T08:18:15.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.408+0000 7fdf32721740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561470e22480 con 0x561470af7390 2024-07-04T08:18:15.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.408+0000 7fdf32721740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561470e264c0 con 0x561470aef1b0 2024-07-04T08:18:15.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ebc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ebc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57258/0 (socket says 172.21.15.39:57258) 2024-07-04T08:18:15.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf2a000640 1 -- 172.21.15.39:0/4224663370 learned_addr learned my addr 172.21.15.39:0/4224663370 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:15.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf28c00640 1 --2- 172.21.15.39:0/4224663370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470af7390 0x561470ebd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf22000640 1 -- 172.21.15.39:0/444491636 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdf14004e90 con 0x7fdf241030b0 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf28c00640 1 -- 172.21.15.39:0/4224663370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470aef1b0 msgr2=0x561470ebccf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf28c00640 1 --2- 172.21.15.39:0/4224663370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470aef1b0 0x561470ebccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf28c00640 1 -- 172.21.15.39:0/4224663370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 msgr2=0x561470ebc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf28c00640 1 --2- 172.21.15.39:0/4224663370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ebc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf28c00640 1 -- 172.21.15.39:0/4224663370 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561470dba650 con 0x561470af7390 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.409+0000 7fdf2a000640 1 --2- 172.21.15.39:0/4224663370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ebc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.410+0000 7fdf28c00640 1 --2- 172.21.15.39:0/4224663370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470af7390 0x561470ebd230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009230 tx=0x7fdf10004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf61aa0b6c426f09 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.410+0000 7fdf1aa00640 1 -- 172.21.15.39:0/4224663370 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10035040 con 0x561470af7390 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.410+0000 7fdf1aa00640 1 -- 172.21.15.39:0/4224663370 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf100054f0 con 0x561470af7390 2024-07-04T08:18:15.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.410+0000 7fdf1aa00640 1 -- 172.21.15.39:0/4224663370 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf100035f0 con 0x561470af7390 2024-07-04T08:18:15.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.411+0000 7fdf32721740 1 -- 172.21.15.39:0/4224663370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470af7390 msgr2=0x561470ebd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.411+0000 7fdf32721740 1 --2- 172.21.15.39:0/4224663370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470af7390 0x561470ebd230 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009230 tx=0x7fdf10004040 comp rx=0 tx=0).stop 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf32721740 1 -- 172.21.15.39:0/4224663370 shutdown_connections 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf32721740 1 --2- 172.21.15.39:0/4224663370 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470af7390 0x561470ebd230 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf32721740 1 --2- 172.21.15.39:0/4224663370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470aef1b0 0x561470ebccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf32721740 1 --2- 172.21.15.39:0/4224663370 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ebc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf32721740 1 -- 172.21.15.39:0/4224663370 >> 172.21.15.39:0/4224663370 conn(0x561470b0c460 msgr2=0x561470eba6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf32721740 1 -- 172.21.15.39:0/4224663370 shutdown_connections 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf32721740 1 -- 172.21.15.39:0/4224663370 wait complete. 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf22000640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf080b28f0 0x7fdf080b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf22000640 1 -- 172.21.15.39:0/444491636 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdf140e2dd0 con 0x7fdf241030b0 2024-07-04T08:18:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.412+0000 7fdf28c00640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf080b28f0 0x7fdf080b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.413+0000 7fdf32721740 1 -- start start 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.413+0000 7fdf32721740 1 Processor -- start 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.413+0000 7fdf28c00640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf080b28f0 0x7fdf080b4e00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdf1002a9e0 tx=0x7fdf10000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf32721740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aef1b0 0x561470ec83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf32721740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 0x561470ec88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf32721740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ed4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf32721740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561470c86700 con 0x561470b18070 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf32721740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561470dc9bb0 con 0x561470aef1b0 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf32721740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561470ecaf30 con 0x561470af7390 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf32721740 10 client.0 _ll_register_callbacks cb 0x561470e1b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf29600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aef1b0 0x561470ec83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ed4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf29600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aef1b0 0x561470ec83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53630/0 (socket says 172.21.15.39:53630) 2024-07-04T08:18:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf29600640 1 -- 172.21.15.39:0/1642223094 learned_addr learned my addr 172.21.15.39:0/1642223094 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf2a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ed4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57264/0 (socket says 172.21.15.39:57264) 2024-07-04T08:18:15.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.414+0000 7fdf28c00640 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 0x561470ec88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf29600640 1 -- 172.21.15.39:0/1642223094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 msgr2=0x561470ec88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf29600640 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 0x561470ec88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf29600640 1 -- 172.21.15.39:0/1642223094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 msgr2=0x561470ed4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf29600640 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ed4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf29600640 1 -- 172.21.15.39:0/1642223094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561470b83650 con 0x561470aef1b0 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf28c00640 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 0x561470ec88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf29600640 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aef1b0 0x561470ec83a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400c6a0 tx=0x7fdf140048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fef8f9dc9bd8288f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf0f400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf0f400640 1 -- 172.21.15.39:0/1642223094 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf140c0030 con 0x561470aef1b0 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf0f400640 1 -- 172.21.15.39:0/1642223094 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdf140a6310 con 0x561470aef1b0 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.415+0000 7fdf32721740 1 -- 172.21.15.39:0/1642223094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf100054f0 con 0x561470aef1b0 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf0f400640 1 -- 172.21.15.39:0/1642223094 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf140e0620 con 0x561470aef1b0 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf32721740 10 client.26201 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf32721740 1 -- 172.21.15.39:0/1642223094 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561470ed6fa0 con 0x561470aef1b0 2024-07-04T08:18:15.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf0f400640 1 -- 172.21.15.39:0/1642223094 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf100054f0 con 0x561470aef1b0 2024-07-04T08:18:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf0f400640 1 -- 172.21.15.39:0/1642223094 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdf140b26f0 con 0x561470aef1b0 2024-07-04T08:18:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf32721740 10 client.26201 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf32721740 10 client.26201 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:18:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf32721740 20 client.26201 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf32721740 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x561470ad69f0 0x561470eda3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf32721740 4 client.26201 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.416+0000 7fdf32721740 1 -- 172.21.15.39:0/1642223094 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561470e34e30 con 0x561470ad69f0 2024-07-04T08:18:15.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.417+0000 7fdf28c00640 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x561470ad69f0 0x561470eda3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.418+0000 7fdf28c00640 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x561470ad69f0 0x561470eda3f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c68a932fcd63365f server_cookie=e42f66563717a1f in_seq=0 out_seq=0 2024-07-04T08:18:15.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.418+0000 7fdf0f400640 10 client.26201 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:18:15.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.418+0000 7fdf0f400640 1 -- 172.21.15.39:0/1642223094 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561470e34e30 con 0x561470ad69f0 2024-07-04T08:18:15.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.418+0000 7fdf0f400640 10 client.26201 handle_command_reply: tid=0 2024-07-04T08:18:15.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.418+0000 7fdf32721740 1 client.26201 shutdown 2024-07-04T08:18:15.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 -- 172.21.15.39:0/1642223094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aef1b0 msgr2=0x561470ec83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aef1b0 0x561470ec83a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400c6a0 tx=0x7fdf140048f0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 -- 172.21.15.39:0/1642223094 shutdown_connections 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x561470ad69f0 0x561470eda3f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470b18070 0x561470ed4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 0x561470ec88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 --2- 172.21.15.39:0/1642223094 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aef1b0 0x561470ec83a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf0f400640 0 client.26201 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 -- 172.21.15.39:0/1642223094 >> 172.21.15.39:0/1642223094 conn(0x561470b0c460 msgr2=0x561470ebf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 -- 172.21.15.39:0/1642223094 shutdown_connections 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 1 -- 172.21.15.39:0/1642223094 wait complete. 2024-07-04T08:18:15.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.419+0000 7fdf32721740 20 client.26201 trim_cache size 0 max 16384 2024-07-04T08:18:15.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf32721740 1 Processor -- start 2024-07-04T08:18:15.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf32721740 1 -- start start 2024-07-04T08:18:15.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf32721740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470aef1b0 0x561470eb9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf32721740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 0x561470eba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf32721740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470b18070 0x561470ec1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf32721740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241b2400 con 0x561470aef1b0 2024-07-04T08:18:15.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf32721740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241b3f40 con 0x561470b18070 2024-07-04T08:18:15.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf32721740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241b2ad0 con 0x561470af7390 2024-07-04T08:18:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf29600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470aef1b0 0x561470eb9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf29600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470aef1b0 0x561470eb9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57280/0 (socket says 172.21.15.39:57280) 2024-07-04T08:18:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.435+0000 7fdf29600640 1 -- 172.21.15.39:0/3225394000 learned_addr learned my addr 172.21.15.39:0/3225394000 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.436+0000 7fdf28c00640 1 --2- 172.21.15.39:0/3225394000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 0x561470eba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.436+0000 7fdf2a000640 1 --2- 172.21.15.39:0/3225394000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470b18070 0x561470ec1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.436+0000 7fdf29600640 1 -- 172.21.15.39:0/3225394000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 msgr2=0x561470eba6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.436+0000 7fdf29600640 1 --2- 172.21.15.39:0/3225394000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 0x561470eba6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.436+0000 7fdf29600640 1 -- 172.21.15.39:0/3225394000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470b18070 msgr2=0x561470ec1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:15.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.436+0000 7fdf29600640 1 --2- 172.21.15.39:0/3225394000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470b18070 0x561470ec1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.436+0000 7fdf29600640 1 -- 172.21.15.39:0/3225394000 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561470ebeaa0 con 0x561470aef1b0 2024-07-04T08:18:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.436+0000 7fdf29600640 1 --2- 172.21.15.39:0/3225394000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470aef1b0 0x561470eb9940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf14004a40 tx=0x7fdf140a9950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b6e2fd5508cdcb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.437+0000 7fdf1a000640 1 -- 172.21.15.39:0/3225394000 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf140b7030 con 0x561470aef1b0 2024-07-04T08:18:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.437+0000 7fdf1a000640 1 -- 172.21.15.39:0/3225394000 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf140a9d20 con 0x561470aef1b0 2024-07-04T08:18:15.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.437+0000 7fdf1a000640 1 -- 172.21.15.39:0/3225394000 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf140c0070 con 0x561470aef1b0 2024-07-04T08:18:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.437+0000 7fdf32721740 1 -- 172.21.15.39:0/3225394000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470aef1b0 msgr2=0x561470eb9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.437+0000 7fdf32721740 1 --2- 172.21.15.39:0/3225394000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470aef1b0 0x561470eb9940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf14004a40 tx=0x7fdf140a9950 comp rx=0 tx=0).stop 2024-07-04T08:18:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.438+0000 7fdf32721740 1 -- 172.21.15.39:0/3225394000 shutdown_connections 2024-07-04T08:18:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.438+0000 7fdf32721740 1 --2- 172.21.15.39:0/3225394000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470b18070 0x561470ec1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.438+0000 7fdf32721740 1 --2- 172.21.15.39:0/3225394000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470af7390 0x561470eba6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.438+0000 7fdf32721740 1 --2- 172.21.15.39:0/3225394000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470aef1b0 0x561470eb9940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.438+0000 7fdf32721740 1 -- 172.21.15.39:0/3225394000 >> 172.21.15.39:0/3225394000 conn(0x561470b0c460 msgr2=0x561470eb9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.438+0000 7fdf32721740 1 -- 172.21.15.39:0/3225394000 shutdown_connections 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.438+0000 7fdf32721740 1 -- 172.21.15.39:0/3225394000 wait complete. 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.438+0000 7fdf32721740 1 -- start start 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.438+0000 7fdf32721740 1 Processor -- start 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf32721740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470af7390 0x561470ed09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf32721740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470b18070 0x561470ed3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf32721740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aabda0 0x561470ed4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf32721740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241b3870 con 0x561470af7390 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf32721740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241b31a0 con 0x561470aabda0 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf32721740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf241b03e0 con 0x561470b18070 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf32721740 10 client.0 _ll_register_callbacks cb 0x7fdf14004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf29600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470af7390 0x561470ed09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf29600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470af7390 0x561470ed09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57292/0 (socket says 172.21.15.39:57292) 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf29600640 1 -- 172.21.15.39:0/519362844 learned_addr learned my addr 172.21.15.39:0/519362844 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.439+0000 7fdf28c00640 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470b18070 0x561470ed3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf2a000640 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aabda0 0x561470ed4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf29600640 1 -- 172.21.15.39:0/519362844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470b18070 msgr2=0x561470ed3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf29600640 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470b18070 0x561470ed3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf29600640 1 -- 172.21.15.39:0/519362844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aabda0 msgr2=0x561470ed4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf29600640 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aabda0 0x561470ed4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf29600640 1 -- 172.21.15.39:0/519362844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561470ed4960 con 0x561470af7390 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf2a000640 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aabda0 0x561470ed4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf29600640 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470af7390 0x561470ed09a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdf140a6f90 tx=0x7fdf140a9b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e7964b98ddcf35 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf0f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:15.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.440+0000 7fdf0f400640 1 -- 172.21.15.39:0/519362844 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf140b7030 con 0x561470af7390 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf0f400640 1 -- 172.21.15.39:0/519362844 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdf140a6310 con 0x561470af7390 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf32721740 1 -- 172.21.15.39:0/519362844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf140a9d20 con 0x561470af7390 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf0f400640 1 -- 172.21.15.39:0/519362844 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf140c0050 con 0x561470af7390 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf32721740 10 client.16077 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf32721740 1 -- 172.21.15.39:0/519362844 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561470f42b80 con 0x561470af7390 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf0f400640 1 -- 172.21.15.39:0/519362844 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf140a9d20 con 0x561470af7390 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf0f400640 1 -- 172.21.15.39:0/519362844 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdf14093c80 con 0x561470af7390 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf32721740 10 client.16077 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf32721740 10 client.16077 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf32721740 20 client.16077 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf32721740 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x561470f43000 0x561470f45420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf32721740 4 client.16077 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:15.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.441+0000 7fdf32721740 1 -- 172.21.15.39:0/519362844 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561470e308d0 con 0x561470f43000 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.442+0000 7fdf28c00640 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x561470f43000 0x561470f45420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf28c00640 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x561470f43000 0x561470f45420 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68d5ac0524cbfc39 server_cookie=654326c3a240ef20 in_seq=0 out_seq=0 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf0f400640 10 client.16077 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf0f400640 1 -- 172.21.15.39:0/519362844 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561470e308d0 con 0x561470f43000 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf0f400640 10 client.16077 handle_command_reply: tid=0 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf32721740 1 client.16077 shutdown 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf32721740 1 -- 172.21.15.39:0/519362844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470af7390 msgr2=0x561470ed09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf32721740 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470af7390 0x561470ed09a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdf140a6f90 tx=0x7fdf140a9b70 comp rx=0 tx=0).stop 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf32721740 1 -- 172.21.15.39:0/519362844 shutdown_connections 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf32721740 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x561470f43000 0x561470f45420 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf32721740 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561470aabda0 0x561470ed4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf32721740 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561470b18070 0x561470ed3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf32721740 1 --2- 172.21.15.39:0/519362844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561470af7390 0x561470ed09a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf32721740 1 -- 172.21.15.39:0/519362844 >> 172.21.15.39:0/519362844 conn(0x561470b0c460 msgr2=0x561470ec9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.443+0000 7fdf0f400640 0 client.16077 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:18:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.444+0000 7fdf32721740 1 -- 172.21.15.39:0/519362844 shutdown_connections 2024-07-04T08:18:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.444+0000 7fdf32721740 1 -- 172.21.15.39:0/519362844 wait complete. 2024-07-04T08:18:15.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.444+0000 7fdf32721740 20 client.16077 trim_cache size 0 max 16384 2024-07-04T08:18:15.449 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.444+0000 7fdf2c000640 1 -- 172.21.15.39:0/444491636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf080b28f0 msgr2=0x7fdf080b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.444+0000 7fdf2c000640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf080b28f0 0x7fdf080b4e00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdf1002a9e0 tx=0x7fdf10000bc0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.444+0000 7fdf2c000640 1 -- 172.21.15.39:0/444491636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf241030b0 msgr2=0x7fdf2419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.444+0000 7fdf2c000640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf241030b0 0x7fdf2419bc50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400e9f0 tx=0x7fdf1400edf0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.445+0000 7fdf2c000640 1 -- 172.21.15.39:0/444491636 shutdown_connections 2024-07-04T08:18:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.445+0000 7fdf2c000640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf080b28f0 0x7fdf080b4e00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.445+0000 7fdf2c000640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf241089f0 0x7fdf2419c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.445+0000 7fdf2c000640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf24109340 0x7fdf241a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.445+0000 7fdf2c000640 1 --2- 172.21.15.39:0/444491636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf241030b0 0x7fdf2419bc50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:15.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.445+0000 7fdf2c000640 1 -- 172.21.15.39:0/444491636 >> 172.21.15.39:0/444491636 conn(0x7fdf2407ada0 msgr2=0x7fdf24106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:15.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.445+0000 7fdf2c000640 1 -- 172.21.15.39:0/444491636 shutdown_connections 2024-07-04T08:18:15.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:15.445+0000 7fdf2c000640 1 -- 172.21.15.39:0/444491636 wait complete. 2024-07-04T08:18:16.386 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:18:16.881 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:17.222 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:16 smithi039 ceph-mon[32062]: pgmap v572: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 KiB/s wr, 28 op/s 2024-07-04T08:18:17.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:16 smithi133 ceph-mon[37050]: pgmap v572: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 KiB/s wr, 28 op/s 2024-07-04T08:18:17.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:16 smithi050 ceph-mon[36577]: pgmap v572: 129 pgs: 129 active+clean; 2.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 43 KiB/s wr, 28 op/s 2024-07-04T08:18:19.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.168+0000 7fcec5a00640 1 -- 172.21.15.39:0/3008644600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec010b3a0 msgr2=0x7fcec0111bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:19.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.168+0000 7fcec5a00640 1 --2- 172.21.15.39:0/3008644600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec010b3a0 0x7fcec0111bd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009ca0 tx=0x7fceb802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.170 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:18 smithi039 ceph-mon[32062]: pgmap v573: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-07-04T08:18:19.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.168+0000 7fcec5a00640 1 -- 172.21.15.39:0/3008644600 shutdown_connections 2024-07-04T08:18:19.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.168+0000 7fcec5a00640 1 --2- 172.21.15.39:0/3008644600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec010b3a0 0x7fcec0111bd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.168+0000 7fcec5a00640 1 --2- 172.21.15.39:0/3008644600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcec010aa50 0x7fcec010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.168+0000 7fcec5a00640 1 --2- 172.21.15.39:0/3008644600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcec0105110 0x7fcec0105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.168+0000 7fcec5a00640 1 -- 172.21.15.39:0/3008644600 >> 172.21.15.39:0/3008644600 conn(0x7fcec0100b60 msgr2=0x7fcec0102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:19.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.169+0000 7fcec5a00640 1 -- 172.21.15.39:0/3008644600 shutdown_connections 2024-07-04T08:18:19.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.169+0000 7fcec5a00640 1 -- 172.21.15.39:0/3008644600 wait complete. 2024-07-04T08:18:19.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.170+0000 7fcec5a00640 1 Processor -- start 2024-07-04T08:18:19.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.170+0000 7fcec5a00640 1 -- start start 2024-07-04T08:18:19.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcec5a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcec0105110 0x7fcec019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcec5a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcec010aa50 0x7fcec019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcec5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec010b3a0 0x7fcec01a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcec5a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0115050 con 0x7fcec010b3a0 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcec5a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0114ed0 con 0x7fcec0105110 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcec5a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01151d0 con 0x7fcec010aa50 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcebfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcec0105110 0x7fcec019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcebf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcec010aa50 0x7fcec019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcebfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcec0105110 0x7fcec019b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53654/0 (socket says 172.21.15.39:53654) 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcebfe00640 1 -- 172.21.15.39:0/41375639 learned_addr learned my addr 172.21.15.39:0/41375639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.171+0000 7fcec5000640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec010b3a0 0x7fcec01a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcebfe00640 1 -- 172.21.15.39:0/41375639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcec010aa50 msgr2=0x7fcec019bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcebfe00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcec010aa50 0x7fcec019bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcebfe00640 1 -- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec010b3a0 msgr2=0x7fcec01a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcebfe00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec010b3a0 0x7fcec01a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcebfe00640 1 -- 172.21.15.39:0/41375639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec01a1930 con 0x7fcec0105110 2024-07-04T08:18:19.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcebfe00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcec0105110 0x7fcec019b9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e9f0 tx=0x7fceb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcebcc00640 1 -- 172.21.15.39:0/41375639 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb000cd80 con 0x7fcec0105110 2024-07-04T08:18:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcec5a00640 1 -- 172.21.15.39:0/41375639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec01a1c20 con 0x7fcec0105110 2024-07-04T08:18:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcebcc00640 1 -- 172.21.15.39:0/41375639 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb00108e0 con 0x7fcec0105110 2024-07-04T08:18:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.172+0000 7fcebcc00640 1 -- 172.21.15.39:0/41375639 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0018980 con 0x7fcec0105110 2024-07-04T08:18:19.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.173+0000 7fcec5a00640 1 -- 172.21.15.39:0/41375639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec01a8380 con 0x7fcec0105110 2024-07-04T08:18:19.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.176+0000 7fcecd5ce740 1 Processor -- start 2024-07-04T08:18:19.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.176+0000 7fcecd5ce740 1 -- start start 2024-07-04T08:18:19.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.176+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63dc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.176+0000 7fcebcc00640 1 -- 172.21.15.39:0/41375639 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fceb0010430 con 0x7fcec0105110 2024-07-04T08:18:19.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.176+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 0x564cf63dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63dd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564cf6352f00 con 0x564cf6038070 2024-07-04T08:18:19.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564cf63424a0 con 0x564cf600f1b0 2024-07-04T08:18:19.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564cf63464e0 con 0x564cf6017390 2024-07-04T08:18:19.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcec5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63dc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcebfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 0x564cf63dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcec5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63dc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57330/0 (socket says 172.21.15.39:57330) 2024-07-04T08:18:19.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcec5000640 1 -- 172.21.15.39:0/25186433 learned_addr learned my addr 172.21.15.39:0/25186433 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:19.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcebf400640 1 --2- 172.21.15.39:0/25186433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63dd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.177+0000 7fcec5000640 1 -- 172.21.15.39:0/25186433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 msgr2=0x564cf63dd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:19.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.178+0000 7fcec5000640 1 --2- 172.21.15.39:0/25186433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63dd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.178+0000 7fcec5000640 1 -- 172.21.15.39:0/25186433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 msgr2=0x564cf63dceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:19.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.178+0000 7fcec5000640 1 --2- 172.21.15.39:0/25186433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 0x564cf63dceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.178+0000 7fcec5000640 1 -- 172.21.15.39:0/25186433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf62da670 con 0x564cf6038070 2024-07-04T08:18:19.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.178+0000 7fcec5000640 1 --2- 172.21.15.39:0/25186433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63dc970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fceb802f410 tx=0x7fceb802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a27caaa8c3ef9c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:19.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.178+0000 7fceb5600640 1 -- 172.21.15.39:0/25186433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8031cf0 con 0x564cf6038070 2024-07-04T08:18:19.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.178+0000 7fceb5600640 1 -- 172.21.15.39:0/25186433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb8036030 con 0x564cf6038070 2024-07-04T08:18:19.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.178+0000 7fceb5600640 1 -- 172.21.15.39:0/25186433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb80042a0 con 0x564cf6038070 2024-07-04T08:18:19.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.178+0000 7fcebcc00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcea40b2960 0x7fcea40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.179+0000 7fcebcc00640 1 -- 172.21.15.39:0/41375639 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fceb00e2c70 con 0x7fcec0105110 2024-07-04T08:18:19.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.179+0000 7fcebfe00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcea40b2960 0x7fcea40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcecd5ce740 1 -- 172.21.15.39:0/25186433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 msgr2=0x564cf63dc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/25186433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63dc970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fceb802f410 tx=0x7fceb802f440 comp rx=0 tx=0).stop 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcebfe00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcea40b2960 0x7fcea40b4e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fceb00e0140 tx=0x7fceb00e67b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcecd5ce740 1 -- 172.21.15.39:0/25186433 shutdown_connections 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/25186433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63dd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/25186433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 0x564cf63dceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/25186433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63dc970 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcecd5ce740 1 -- 172.21.15.39:0/25186433 >> 172.21.15.39:0/25186433 conn(0x564cf602c460 msgr2=0x564cf63da860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcecd5ce740 1 -- 172.21.15.39:0/25186433 shutdown_connections 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.180+0000 7fcecd5ce740 1 -- 172.21.15.39:0/25186433 wait complete. 2024-07-04T08:18:19.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.181+0000 7fcecd5ce740 1 -- start start 2024-07-04T08:18:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.181+0000 7fcecd5ce740 1 Processor -- start 2024-07-04T08:18:19.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.181+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 0x564cf63e8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.181+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.181+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63f4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.181+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564cf61a6700 con 0x564cf6038070 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.181+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564cf62e9bd0 con 0x564cf600f1b0 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.181+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564cf63eaf70 con 0x564cf6017390 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcecd5ce740 10 client.0 _ll_register_callbacks cb 0x564cf633b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcec5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 0x564cf63e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcec5000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 0x564cf63e8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53674/0 (socket says 172.21.15.39:53674) 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcec5000640 1 -- 172.21.15.39:0/2241719552 learned_addr learned my addr 172.21.15.39:0/2241719552 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcebf400640 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcec5000640 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcebf400640 1 -- 172.21.15.39:0/2241719552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 msgr2=0x564cf63e8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcebf400640 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 0x564cf63e8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcebf400640 1 -- 172.21.15.39:0/2241719552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 msgr2=0x564cf63f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcebf400640 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.182+0000 7fcebf400640 1 -- 172.21.15.39:0/2241719552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf60a3650 con 0x564cf6017390 2024-07-04T08:18:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.183+0000 7fcebf400640 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63e8a80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcec006abe0 tx=0x7fceac012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0be325c4ac7b75 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.183+0000 7fceaa000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:18:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.183+0000 7fceaa000640 1 -- 172.21.15.39:0/2241719552 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac004050 con 0x564cf6017390 2024-07-04T08:18:19.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.183+0000 7fcecd5ce740 1 -- 172.21.15.39:0/2241719552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb8036030 con 0x564cf6017390 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.183+0000 7fceaa000640 1 -- 172.21.15.39:0/2241719552 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fceac0041f0 con 0x564cf6017390 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fceaa000640 1 -- 172.21.15.39:0/2241719552 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fceac025050 con 0x564cf6017390 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fcecd5ce740 10 client.26167 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fcecd5ce740 1 -- 172.21.15.39:0/2241719552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564cf63f6fe0 con 0x564cf6017390 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fceaa000640 1 -- 172.21.15.39:0/2241719552 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb8036030 con 0x564cf6017390 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fceaa000640 1 -- 172.21.15.39:0/2241719552 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fceac003900 con 0x564cf6017390 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fcecd5ce740 10 client.26167 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fcecd5ce740 10 client.26167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fcecd5ce740 20 client.26167 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564cf5ff69f0 0x564cf63fa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fcecd5ce740 4 client.26167 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.184+0000 7fcecd5ce740 1 -- 172.21.15.39:0/2241719552 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cf6354e50 con 0x564cf5ff69f0 2024-07-04T08:18:19.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.185+0000 7fcec5000640 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564cf5ff69f0 0x564cf63fa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.185+0000 7fcec5000640 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564cf5ff69f0 0x564cf63fa4a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2052abdfa614ee6 server_cookie=a89e9cd6681dd3a8 in_seq=0 out_seq=0 2024-07-04T08:18:19.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.185+0000 7fceaa000640 10 client.26167 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.186+0000 7fceaa000640 1 -- 172.21.15.39:0/2241719552 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564cf6354e50 con 0x564cf5ff69f0 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.186+0000 7fceaa000640 10 client.26167 handle_command_reply: tid=0 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.186+0000 7fcecd5ce740 1 client.26167 shutdown 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 -- 172.21.15.39:0/2241719552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 msgr2=0x564cf63e8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63e8a80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcec006abe0 tx=0x7fceac012b60 comp rx=0 tx=0).stop 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 -- 172.21.15.39:0/2241719552 shutdown_connections 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564cf5ff69f0 0x564cf63fa4a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf6038070 0x564cf63f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63e8a80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/2241719552 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf600f1b0 0x564cf63e8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fceaa000640 0 client.26167 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:19.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 -- 172.21.15.39:0/2241719552 >> 172.21.15.39:0/2241719552 conn(0x564cf602c460 msgr2=0x564cf63dfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:19.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 -- 172.21.15.39:0/2241719552 shutdown_connections 2024-07-04T08:18:19.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 1 -- 172.21.15.39:0/2241719552 wait complete. 2024-07-04T08:18:19.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.187+0000 7fcecd5ce740 20 client.26167 trim_cache size 0 max 16384 2024-07-04T08:18:19.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.203+0000 7fcecd5ce740 1 Processor -- start 2024-07-04T08:18:19.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.203+0000 7fcecd5ce740 1 -- start start 2024-07-04T08:18:19.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.203+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.203+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.203+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 0x564cf63f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.203+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01b2200 con 0x564cf600f1b0 2024-07-04T08:18:19.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.203+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01b3d40 con 0x564cf6038070 2024-07-04T08:18:19.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.203+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01b28d0 con 0x564cf6017390 2024-07-04T08:18:19.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcebf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcec5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcec5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63e0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57352/0 (socket says 172.21.15.39:57352) 2024-07-04T08:18:19.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcec5000640 1 -- 172.21.15.39:0/3825508122 learned_addr learned my addr 172.21.15.39:0/3825508122 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:19.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcebf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63de0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53980/0 (socket says 172.21.15.39:53980) 2024-07-04T08:18:19.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcec5000640 1 --2- 172.21.15.39:0/3825508122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 0x564cf63f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcebf400640 1 -- 172.21.15.39:0/3825508122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 msgr2=0x564cf63f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:19.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcebf400640 1 --2- 172.21.15.39:0/3825508122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 0x564cf63f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcebf400640 1 -- 172.21.15.39:0/3825508122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 msgr2=0x564cf63e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcebf400640 1 --2- 172.21.15.39:0/3825508122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.204+0000 7fcebf400640 1 -- 172.21.15.39:0/3825508122 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf63dd780 con 0x564cf6017390 2024-07-04T08:18:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.205+0000 7fcebf400640 1 --2- 172.21.15.39:0/3825508122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63de0b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fceac003cd0 tx=0x7fceac0106c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=283374acf394ed8b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.205+0000 7fceb4c00640 1 -- 172.21.15.39:0/3825508122 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac02d020 con 0x564cf6017390 2024-07-04T08:18:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.205+0000 7fceb4c00640 1 -- 172.21.15.39:0/3825508122 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceac003aa0 con 0x564cf6017390 2024-07-04T08:18:19.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.205+0000 7fceb4c00640 1 -- 172.21.15.39:0/3825508122 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac00f8f0 con 0x564cf6017390 2024-07-04T08:18:19.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.206+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3825508122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 msgr2=0x564cf63de0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:19.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.206+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3825508122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63de0b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fceac003cd0 tx=0x7fceac0106c0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.206+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3825508122 shutdown_connections 2024-07-04T08:18:19.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.206+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3825508122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 0x564cf63f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.206+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3825508122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63de0b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.206+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3825508122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.206+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3825508122 >> 172.21.15.39:0/3825508122 conn(0x564cf602c460 msgr2=0x564cf63dd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:19.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.206+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3825508122 shutdown_connections 2024-07-04T08:18:19.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.206+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3825508122 wait complete. 2024-07-04T08:18:19.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.207+0000 7fcecd5ce740 1 -- start start 2024-07-04T08:18:19.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.207+0000 7fcecd5ce740 1 Processor -- start 2024-07-04T08:18:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.207+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.207+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63ebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.207+0000 7fcecd5ce740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 0x564cf63ec130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.207+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01b3670 con 0x564cf600f1b0 2024-07-04T08:18:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.207+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01b2fa0 con 0x564cf6038070 2024-07-04T08:18:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.207+0000 7fcecd5ce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcec01b01e0 con 0x564cf6017390 2024-07-04T08:18:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcecd5ce740 10 client.0 _ll_register_callbacks cb 0x7fceac0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:19.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcec5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcec5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63f06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57354/0 (socket says 172.21.15.39:57354) 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcec5000640 1 -- 172.21.15.39:0/3140799561 learned_addr learned my addr 172.21.15.39:0/3140799561 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcebf400640 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63ebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcec5000640 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 0x564cf63ec130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcec5000640 1 -- 172.21.15.39:0/3140799561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 msgr2=0x564cf63ebbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcec5000640 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63ebbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcec5000640 1 -- 172.21.15.39:0/3140799561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 msgr2=0x564cf63ec130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcec5000640 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 0x564cf63ec130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.208+0000 7fcec5000640 1 -- 172.21.15.39:0/3140799561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf63ec6c0 con 0x564cf600f1b0 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.209+0000 7fcec5000640 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63f06a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009bf0 tx=0x7fceb8058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37058c1dbe8b9ec server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.209+0000 7fceaa000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.209+0000 7fceaa000640 1 -- 172.21.15.39:0/3140799561 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8031d00 con 0x564cf600f1b0 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.209+0000 7fceaa000640 1 -- 172.21.15.39:0/3140799561 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fceb8031450 con 0x564cf600f1b0 2024-07-04T08:18:19.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.209+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3140799561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceac003aa0 con 0x564cf600f1b0 2024-07-04T08:18:19.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.209+0000 7fceaa000640 1 -- 172.21.15.39:0/3140799561 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fceb8031650 con 0x564cf600f1b0 2024-07-04T08:18:19.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.209+0000 7fcecd5ce740 10 client.16095 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:19.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.209+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3140799561 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564cf6462990 con 0x564cf600f1b0 2024-07-04T08:18:19.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.209+0000 7fceaa000640 1 -- 172.21.15.39:0/3140799561 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceac003aa0 con 0x564cf600f1b0 2024-07-04T08:18:19.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.210+0000 7fceaa000640 1 -- 172.21.15.39:0/3140799561 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fceb8043070 con 0x564cf600f1b0 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.210+0000 7fcecd5ce740 10 client.16095 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.210+0000 7fcecd5ce740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.210+0000 7fcecd5ce740 20 client.16095 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.210+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564cf5fcbda0 0x564cf6464d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.210+0000 7fcecd5ce740 4 client.16095 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.210+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3140799561 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564cf63508f0 con 0x564cf5fcbda0 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.210+0000 7fcebf400640 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564cf5fcbda0 0x564cf6464d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcebf400640 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564cf5fcbda0 0x564cf6464d90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4670bc938d174bfd server_cookie=f702f51dea4f531 in_seq=0 out_seq=0 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fceaa000640 10 client.16095 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fceaa000640 1 -- 172.21.15.39:0/3140799561 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564cf63508f0 con 0x564cf5fcbda0 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fceaa000640 10 client.16095 handle_command_reply: tid=0 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcecd5ce740 1 client.16095 shutdown 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3140799561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 msgr2=0x564cf63f06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63f06a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fceb8009bf0 tx=0x7fceb8058040 comp rx=0 tx=0).stop 2024-07-04T08:18:19.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3140799561 shutdown_connections 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564cf5fcbda0 0x564cf6464d90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564cf6038070 0x564cf63ec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cf6017390 0x564cf63ebbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcecd5ce740 1 --2- 172.21.15.39:0/3140799561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564cf600f1b0 0x564cf63f06a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.211+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3140799561 >> 172.21.15.39:0/3140799561 conn(0x564cf602c460 msgr2=0x564cf63e0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.212+0000 7fceaa000640 0 client.16095 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.212+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3140799561 shutdown_connections 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.212+0000 7fcecd5ce740 1 -- 172.21.15.39:0/3140799561 wait complete. 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.212+0000 7fcecd5ce740 20 client.16095 trim_cache size 0 max 16384 2024-07-04T08:18:19.214 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.212+0000 7fcec5a00640 1 -- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcea40b2960 msgr2=0x7fcea40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.212+0000 7fcec5a00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcea40b2960 0x7fcea40b4e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fceb00e0140 tx=0x7fceb00e67b0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.212+0000 7fcec5a00640 1 -- 172.21.15.39:0/41375639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcec0105110 msgr2=0x7fcec019b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.212+0000 7fcec5a00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcec0105110 0x7fcec019b9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e9f0 tx=0x7fceb000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.213+0000 7fcec5a00640 1 -- 172.21.15.39:0/41375639 shutdown_connections 2024-07-04T08:18:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.213+0000 7fcec5a00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcea40b2960 0x7fcea40b4e70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.213+0000 7fcec5a00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcec010b3a0 0x7fcec01a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.213+0000 7fcec5a00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcec010aa50 0x7fcec019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.213+0000 7fcec5a00640 1 --2- 172.21.15.39:0/41375639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcec0105110 0x7fcec019b9c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.213+0000 7fcec5a00640 1 -- 172.21.15.39:0/41375639 >> 172.21.15.39:0/41375639 conn(0x7fcec0100b60 msgr2=0x7fcec0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.213+0000 7fcec5a00640 1 -- 172.21.15.39:0/41375639 shutdown_connections 2024-07-04T08:18:19.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:19.213+0000 7fcec5a00640 1 -- 172.21.15.39:0/41375639 wait complete. 2024-07-04T08:18:19.318 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:18 smithi133 ceph-mon[37050]: pgmap v573: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-07-04T08:18:19.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:18 smithi050 ceph-mon[36577]: pgmap v573: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-07-04T08:18:21.153 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:18:21.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:18:21.238 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:20 smithi039 ceph-mon[32062]: pgmap v574: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-07-04T08:18:21.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:20 smithi133 ceph-mon[37050]: pgmap v574: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-07-04T08:18:21.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:20 smithi050 ceph-mon[36577]: pgmap v574: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-07-04T08:18:21.650 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:23.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:22 smithi133 ceph-mon[37050]: pgmap v575: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:23.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:22 smithi039 ceph-mon[32062]: pgmap v575: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:23.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:22 smithi050 ceph-mon[36577]: pgmap v575: 129 pgs: 129 active+clean; 2.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:23.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.940+0000 7f337f200640 1 -- 172.21.15.39:0/3097388772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3378103d90 msgr2=0x7f33781041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:23.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.940+0000 7f337f200640 1 --2- 172.21.15.39:0/3097388772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3378103d90 0x7f33781041a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3368009900 tx=0x7f336802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.940+0000 7f337f200640 1 -- 172.21.15.39:0/3097388772 shutdown_connections 2024-07-04T08:18:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.940+0000 7f337f200640 1 --2- 172.21.15.39:0/3097388772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33781047c0 0x7f337810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.940+0000 7f337f200640 1 --2- 172.21.15.39:0/3097388772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f337810d340 0x7f337810f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.940+0000 7f337f200640 1 --2- 172.21.15.39:0/3097388772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3378103d90 0x7f33781041a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.940+0000 7f337f200640 1 -- 172.21.15.39:0/3097388772 >> 172.21.15.39:0/3097388772 conn(0x7f3378078c80 msgr2=0x7f3378079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.940+0000 7f337f200640 1 -- 172.21.15.39:0/3097388772 shutdown_connections 2024-07-04T08:18:23.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.940+0000 7f337f200640 1 -- 172.21.15.39:0/3097388772 wait complete. 2024-07-04T08:18:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.942+0000 7f337f200640 1 Processor -- start 2024-07-04T08:18:23.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.942+0000 7f337f200640 1 -- start start 2024-07-04T08:18:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.943+0000 7f337f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378103d90 0x7f3378101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.943+0000 7f337f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33781047c0 0x7f33781021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.943+0000 7f337f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f337810d340 0x7f3378102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.943+0000 7f337f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3378112ff0 con 0x7f337810d340 2024-07-04T08:18:23.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.943+0000 7f337f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3378112e70 con 0x7f33781047c0 2024-07-04T08:18:23.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.943+0000 7f337f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3378113170 con 0x7f3378103d90 2024-07-04T08:18:23.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f337d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f337810d340 0x7f3378102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f3377e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378103d90 0x7f3378101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f3377e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378103d90 0x7f3378101ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:48248/0 (socket says 172.21.15.39:48248) 2024-07-04T08:18:23.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f3377e00640 1 -- 172.21.15.39:0/1591502402 learned_addr learned my addr 172.21.15.39:0/1591502402 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:23.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f336fe00640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33781047c0 0x7f33781021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f337d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f337810d340 0x7f3378102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57328/0 (socket says 172.21.15.39:57328) 2024-07-04T08:18:23.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f3377e00640 1 -- 172.21.15.39:0/1591502402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33781047c0 msgr2=0x7f33781021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:23.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f3377e00640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33781047c0 0x7f33781021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f3377e00640 1 -- 172.21.15.39:0/1591502402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f337810d340 msgr2=0x7f3378102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:23.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f3377e00640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f337810d340 0x7f3378102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.944+0000 7f3377e00640 1 -- 172.21.15.39:0/1591502402 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33781a7cc0 con 0x7f3378103d90 2024-07-04T08:18:23.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.945+0000 7f3377e00640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378103d90 0x7f3378101ca0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3368032fd0 tx=0x7f3368031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:23.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.945+0000 7f3375600640 1 -- 172.21.15.39:0/1591502402 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336803f030 con 0x7f3378103d90 2024-07-04T08:18:23.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.945+0000 7f3375600640 1 -- 172.21.15.39:0/1591502402 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3368004490 con 0x7f3378103d90 2024-07-04T08:18:23.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.945+0000 7f337f200640 1 -- 172.21.15.39:0/1591502402 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33781a7f50 con 0x7f3378103d90 2024-07-04T08:18:23.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.945+0000 7f3375600640 1 -- 172.21.15.39:0/1591502402 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3368035070 con 0x7f3378103d90 2024-07-04T08:18:23.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.945+0000 7f337f200640 1 -- 172.21.15.39:0/1591502402 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33781a83a0 con 0x7f3378103d90 2024-07-04T08:18:23.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.949+0000 7f33857f7740 1 Processor -- start 2024-07-04T08:18:23.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.950+0000 7f33857f7740 1 -- start start 2024-07-04T08:18:23.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.950+0000 7f33857f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730754640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.950+0000 7f33857f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 0x55c730754b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.950+0000 7f33857f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c7307550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.950+0000 7f33857f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c73067e8e0 con 0x55c73038f670 2024-07-04T08:18:23.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.950+0000 7f33857f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c7306caec0 con 0x55c7303b0350 2024-07-04T08:18:23.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.950+0000 7f33857f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c7306ba460 con 0x55c730387490 2024-07-04T08:18:23.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.950+0000 7f336fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c7307550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.950+0000 7f337d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730754640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f337d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730754640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50800/0 (socket says 172.21.15.39:50800) 2024-07-04T08:18:23.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f336fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c7307550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57332/0 (socket says 172.21.15.39:57332) 2024-07-04T08:18:23.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f337d200640 1 -- 172.21.15.39:0/2046517680 learned_addr learned my addr 172.21.15.39:0/2046517680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:23.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f3375600640 1 -- 172.21.15.39:0/1591502402 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3368042030 con 0x7f3378103d90 2024-07-04T08:18:23.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f337d200640 1 -- 172.21.15.39:0/2046517680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 msgr2=0x55c730754b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:23.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f337d200640 1 --2- 172.21.15.39:0/2046517680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 0x55c730754b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f337d200640 1 -- 172.21.15.39:0/2046517680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 msgr2=0x55c7307550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:23.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f337d200640 1 --2- 172.21.15.39:0/2046517680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c7307550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f337d200640 1 -- 172.21.15.39:0/2046517680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c730652280 con 0x55c7303b0350 2024-07-04T08:18:23.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.951+0000 7f337d200640 1 --2- 172.21.15.39:0/2046517680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730754640 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3370009fd0 tx=0x7f33700118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c60e4e059212b9bc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:23.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.952+0000 7f336d600640 1 -- 172.21.15.39:0/2046517680 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370004d20 con 0x55c7303b0350 2024-07-04T08:18:23.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.952+0000 7f336d600640 1 -- 172.21.15.39:0/2046517680 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3370004050 con 0x55c7303b0350 2024-07-04T08:18:23.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.952+0000 7f336d600640 1 -- 172.21.15.39:0/2046517680 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370005280 con 0x55c7303b0350 2024-07-04T08:18:23.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.952+0000 7f3375600640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f335c0b2960 0x7f335c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f33857f7740 1 -- 172.21.15.39:0/2046517680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 msgr2=0x55c730754640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:23.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f33857f7740 1 --2- 172.21.15.39:0/2046517680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730754640 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3370009fd0 tx=0x7f33700118d0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f33857f7740 1 -- 172.21.15.39:0/2046517680 shutdown_connections 2024-07-04T08:18:23.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f3375600640 1 -- 172.21.15.39:0/1591502402 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f3368102c80 con 0x7f3378103d90 2024-07-04T08:18:23.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f33857f7740 1 --2- 172.21.15.39:0/2046517680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c7307550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f33857f7740 1 --2- 172.21.15.39:0/2046517680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 0x55c730754b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f33857f7740 1 --2- 172.21.15.39:0/2046517680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730754640 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f33857f7740 1 -- 172.21.15.39:0/2046517680 >> 172.21.15.39:0/2046517680 conn(0x55c7303a4740 msgr2=0x55c730752530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:23.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f3377e00640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f335c0b2960 0x7f335c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f33857f7740 1 -- 172.21.15.39:0/2046517680 shutdown_connections 2024-07-04T08:18:23.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.953+0000 7f33857f7740 1 -- 172.21.15.39:0/2046517680 wait complete. 2024-07-04T08:18:23.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.954+0000 7f3377e00640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f335c0b2960 0x7f335c0b4e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c7307532c0 tx=0x7f336803b560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:23.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.954+0000 7f33857f7740 1 -- start start 2024-07-04T08:18:23.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.954+0000 7f33857f7740 1 Processor -- start 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f33857f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c730387490 0x55c730760160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f33857f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f33857f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7303b0350 0x55c73076bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f33857f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c730661760 con 0x55c7303b0350 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f33857f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c7306be4a0 con 0x55c730387490 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f33857f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c73051e890 con 0x55c73038f670 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f33857f7740 10 client.0 _ll_register_callbacks cb 0x55c7306622b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f337d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c730387490 0x55c730760160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f336fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f337d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7303b0350 0x55c73076bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f336fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:48260/0 (socket says 172.21.15.39:48260) 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.955+0000 7f336fe00640 1 -- 172.21.15.39:0/3666647828 learned_addr learned my addr 172.21.15.39:0/3666647828 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f336fe00640 1 -- 172.21.15.39:0/3666647828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c730387490 msgr2=0x55c730760160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f336fe00640 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c730387490 0x55c730760160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f336fe00640 1 -- 172.21.15.39:0/3666647828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7303b0350 msgr2=0x55c73076bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f336fe00640 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7303b0350 0x55c73076bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f336fe00640 1 -- 172.21.15.39:0/3666647828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73041b930 con 0x55c73038f670 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f336fe00640 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307606a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3360012700 tx=0x7f3360012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19bfaa7c85a5598d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f3366000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f3366000640 1 -- 172.21.15.39:0/3666647828 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336000f920 con 0x55c73038f670 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f3366000640 1 -- 172.21.15.39:0/3666647828 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f336000fac0 con 0x55c73038f670 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.956+0000 7f33857f7740 1 -- 172.21.15.39:0/3666647828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3370004050 con 0x55c73038f670 2024-07-04T08:18:23.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f3366000640 1 -- 172.21.15.39:0/3666647828 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f336000fcc0 con 0x55c73038f670 2024-07-04T08:18:23.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f33857f7740 10 client.26182 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:23.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f33857f7740 1 -- 172.21.15.39:0/3666647828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c73076eca0 con 0x55c73038f670 2024-07-04T08:18:23.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f3366000640 1 -- 172.21.15.39:0/3666647828 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3370004050 con 0x55c73038f670 2024-07-04T08:18:23.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f3366000640 1 -- 172.21.15.39:0/3666647828 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f336001c420 con 0x55c73038f670 2024-07-04T08:18:23.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f33857f7740 10 client.26182 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:23.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f33857f7740 10 client.26182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:23.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f33857f7740 20 client.26182 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f33857f7740 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55c73036ecd0 0x55c730772260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f33857f7740 4 client.26182 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.957+0000 7f33857f7740 1 -- 172.21.15.39:0/3666647828 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7306cce10 con 0x55c73036ecd0 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.958+0000 7f337d200640 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55c73036ecd0 0x55c730772260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.958+0000 7f337d200640 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55c73036ecd0 0x55c730772260 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9793bef3eb6e758e server_cookie=c58755a2d9f44393 in_seq=0 out_seq=0 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.958+0000 7f3366000640 10 client.26182 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.959+0000 7f3366000640 1 -- 172.21.15.39:0/3666647828 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c7306cce10 con 0x55c73036ecd0 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.959+0000 7f3366000640 10 client.26182 handle_command_reply: tid=0 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.959+0000 7f33857f7740 1 client.26182 shutdown 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 -- 172.21.15.39:0/3666647828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 msgr2=0x55c7307606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307606a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3360012700 tx=0x7f3360012bc0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 -- 172.21.15.39:0/3666647828 shutdown_connections 2024-07-04T08:18:23.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55c73036ecd0 0x55c730772260 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7303b0350 0x55c73076bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307606a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 --2- 172.21.15.39:0/3666647828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c730387490 0x55c730760160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f3366000640 0 client.26182 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:23.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 -- 172.21.15.39:0/3666647828 >> 172.21.15.39:0/3666647828 conn(0x55c7303a4740 msgr2=0x55c7307577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:23.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 -- 172.21.15.39:0/3666647828 shutdown_connections 2024-07-04T08:18:23.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 1 -- 172.21.15.39:0/3666647828 wait complete. 2024-07-04T08:18:23.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.960+0000 7f33857f7740 20 client.26182 trim_cache size 0 max 16384 2024-07-04T08:18:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.981+0000 7f33857f7740 1 Processor -- start 2024-07-04T08:18:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.981+0000 7f33857f7740 1 -- start start 2024-07-04T08:18:23.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.981+0000 7f33857f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c730387490 0x55c730761e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.981+0000 7f33857f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307524c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.981+0000 7f33857f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730755d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.981+0000 7f33857f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f33781b2440 con 0x55c730387490 2024-07-04T08:18:23.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.981+0000 7f33857f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f33781b3f80 con 0x55c7303b0350 2024-07-04T08:18:23.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.981+0000 7f33857f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f33781b2b10 con 0x55c73038f670 2024-07-04T08:18:23.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.982+0000 7f337d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c730387490 0x55c730761e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.982+0000 7f337d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c730387490 0x55c730761e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57342/0 (socket says 172.21.15.39:57342) 2024-07-04T08:18:23.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.982+0000 7f337d200640 1 -- 172.21.15.39:0/2924816083 learned_addr learned my addr 172.21.15.39:0/2924816083 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:23.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.982+0000 7f337d200640 1 --2- 172.21.15.39:0/2924816083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730755d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.982+0000 7f336fe00640 1 --2- 172.21.15.39:0/2924816083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.982+0000 7f336fe00640 1 -- 172.21.15.39:0/2924816083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 msgr2=0x55c730755d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.983+0000 7f336fe00640 1 --2- 172.21.15.39:0/2924816083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730755d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.983+0000 7f336fe00640 1 -- 172.21.15.39:0/2924816083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c730387490 msgr2=0x55c730761e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.983+0000 7f336fe00640 1 --2- 172.21.15.39:0/2924816083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c730387490 0x55c730761e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.983+0000 7f336fe00640 1 -- 172.21.15.39:0/2924816083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7307574a0 con 0x55c73038f670 2024-07-04T08:18:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.983+0000 7f336fe00640 1 --2- 172.21.15.39:0/2924816083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307524c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f336001c890 tx=0x7f33600096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a756678b960ed5d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.983+0000 7f336cc00640 1 -- 172.21.15.39:0/2924816083 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336000f9e0 con 0x55c73038f670 2024-07-04T08:18:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.983+0000 7f336cc00640 1 -- 172.21.15.39:0/2924816083 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336001c420 con 0x55c73038f670 2024-07-04T08:18:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.983+0000 7f336cc00640 1 -- 172.21.15.39:0/2924816083 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33600035f0 con 0x55c73038f670 2024-07-04T08:18:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.984+0000 7f33857f7740 1 -- 172.21.15.39:0/2924816083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 msgr2=0x55c7307524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.984+0000 7f33857f7740 1 --2- 172.21.15.39:0/2924816083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307524c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f336001c890 tx=0x7f33600096c0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.984+0000 7f33857f7740 1 -- 172.21.15.39:0/2924816083 shutdown_connections 2024-07-04T08:18:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.984+0000 7f33857f7740 1 --2- 172.21.15.39:0/2924816083 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c730755d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.984+0000 7f33857f7740 1 --2- 172.21.15.39:0/2924816083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c73038f670 0x55c7307524c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.984+0000 7f33857f7740 1 --2- 172.21.15.39:0/2924816083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c730387490 0x55c730761e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.984+0000 7f33857f7740 1 -- 172.21.15.39:0/2924816083 >> 172.21.15.39:0/2924816083 conn(0x55c7303a4740 msgr2=0x55c730755670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.984+0000 7f33857f7740 1 -- 172.21.15.39:0/2924816083 shutdown_connections 2024-07-04T08:18:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.984+0000 7f33857f7740 1 -- 172.21.15.39:0/2924816083 wait complete. 2024-07-04T08:18:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.985+0000 7f33857f7740 1 -- start start 2024-07-04T08:18:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.985+0000 7f33857f7740 1 Processor -- start 2024-07-04T08:18:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f33857f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 0x55c730764130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f33857f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c730764670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f33857f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c7307e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f33857f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f33781b38b0 con 0x55c73038f670 2024-07-04T08:18:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f33857f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f33781b31e0 con 0x55c7303b0350 2024-07-04T08:18:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f33857f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f33781b0420 con 0x55c730387490 2024-07-04T08:18:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f33857f7740 10 client.0 _ll_register_callbacks cb 0x7f3360009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f336fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c730764670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f337d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 0x55c730764130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f337d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 0x55c730764130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:48268/0 (socket says 172.21.15.39:48268) 2024-07-04T08:18:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f336fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c730764670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57350/0 (socket says 172.21.15.39:57350) 2024-07-04T08:18:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f337d200640 1 -- 172.21.15.39:0/1003288887 learned_addr learned my addr 172.21.15.39:0/1003288887 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.986+0000 7f337d200640 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c7307e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.987+0000 7f336fe00640 1 -- 172.21.15.39:0/1003288887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 msgr2=0x55c730764130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.987+0000 7f336fe00640 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 0x55c730764130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.987+0000 7f336fe00640 1 -- 172.21.15.39:0/1003288887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 msgr2=0x55c7307e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.987+0000 7f336fe00640 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c7307e2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.987+0000 7f336fe00640 1 -- 172.21.15.39:0/1003288887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c730764ef0 con 0x55c73038f670 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.987+0000 7f337d200640 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 0x55c730764130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.988+0000 7f336fe00640 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c730764670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f336001c930 tx=0x7f3360005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ad549e6268edbf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.988+0000 7f3366000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.988+0000 7f3366000640 1 -- 172.21.15.39:0/1003288887 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336001ca80 con 0x55c73038f670 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.988+0000 7f3366000640 1 -- 172.21.15.39:0/1003288887 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f336001cc20 con 0x55c73038f670 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.988+0000 7f33857f7740 1 -- 172.21.15.39:0/1003288887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f336001c420 con 0x55c73038f670 2024-07-04T08:18:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.988+0000 7f3366000640 1 -- 172.21.15.39:0/1003288887 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f336001ce20 con 0x55c73038f670 2024-07-04T08:18:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.988+0000 7f33857f7740 10 client.16113 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.988+0000 7f33857f7740 1 -- 172.21.15.39:0/1003288887 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c7307e37e0 con 0x55c73038f670 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.989+0000 7f3366000640 1 -- 172.21.15.39:0/1003288887 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336001c420 con 0x55c73038f670 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.989+0000 7f3366000640 1 -- 172.21.15.39:0/1003288887 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f336002a480 con 0x55c73038f670 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.989+0000 7f33857f7740 10 client.16113 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.989+0000 7f33857f7740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.989+0000 7f33857f7740 20 client.16113 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.989+0000 7f33857f7740 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55c730344080 0x55c7307e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.989+0000 7f33857f7740 4 client.16113 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.989+0000 7f33857f7740 1 -- 172.21.15.39:0/1003288887 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c7306c88b0 con 0x55c730344080 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.990+0000 7f337d200640 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55c730344080 0x55c7307e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.991+0000 7f3366000640 10 client.16113 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.991+0000 7f337d200640 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55c730344080 0x55c7307e5c50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27525ff184e8fbf1 server_cookie=3d419771c575df16 in_seq=0 out_seq=0 2024-07-04T08:18:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f3366000640 1 -- 172.21.15.39:0/1003288887 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c7306c88b0 con 0x55c730344080 2024-07-04T08:18:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f3366000640 10 client.16113 handle_command_reply: tid=0 2024-07-04T08:18:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f33857f7740 1 client.16113 shutdown 2024-07-04T08:18:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f33857f7740 1 -- 172.21.15.39:0/1003288887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 msgr2=0x55c730764670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f33857f7740 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c730764670 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f336001c930 tx=0x7f3360005760 comp rx=0 tx=0).stop 2024-07-04T08:18:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f33857f7740 1 -- 172.21.15.39:0/1003288887 shutdown_connections 2024-07-04T08:18:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f33857f7740 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55c730344080 0x55c7307e5c50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f33857f7740 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c73038f670 0x55c730764670 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f33857f7740 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7303b0350 0x55c7307e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f33857f7740 1 --2- 172.21.15.39:0/1003288887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c730387490 0x55c730764130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.992+0000 7f33857f7740 1 -- 172.21.15.39:0/1003288887 >> 172.21.15.39:0/1003288887 conn(0x55c7303a4740 msgr2=0x55c730761dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.993+0000 7f3366000640 0 client.16113 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.993+0000 7f33857f7740 1 -- 172.21.15.39:0/1003288887 shutdown_connections 2024-07-04T08:18:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.993+0000 7f33857f7740 1 -- 172.21.15.39:0/1003288887 wait complete. 2024-07-04T08:18:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.993+0000 7f33857f7740 20 client.16113 trim_cache size 0 max 16384 2024-07-04T08:18:23.994 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:18:23.994 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:18:23.994 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "a9af186f-b2a9-4cc1-88ba-f8b5b1eb40bb", 2024-07-04T08:18:23.995 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:18:23.995 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:18:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 -- 172.21.15.39:0/1591502402 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f335c0b2960 msgr2=0x7f335c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f335c0b2960 0x7f335c0b4e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c7307532c0 tx=0x7f336803b560 comp rx=0 tx=0).stop 2024-07-04T08:18:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 -- 172.21.15.39:0/1591502402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378103d90 msgr2=0x7f3378101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378103d90 0x7f3378101ca0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3368032fd0 tx=0x7f3368031a20 comp rx=0 tx=0).stop 2024-07-04T08:18:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 -- 172.21.15.39:0/1591502402 shutdown_connections 2024-07-04T08:18:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f335c0b2960 0x7f335c0b4e70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f337810d340 0x7f3378102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f33781047c0 0x7f33781021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 --2- 172.21.15.39:0/1591502402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378103d90 0x7f3378101ca0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 -- 172.21.15.39:0/1591502402 >> 172.21.15.39:0/1591502402 conn(0x7f3378078c80 msgr2=0x7f33780ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 -- 172.21.15.39:0/1591502402 shutdown_connections 2024-07-04T08:18:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:23.994+0000 7f337f200640 1 -- 172.21.15.39:0/1591502402 wait complete. 2024-07-04T08:18:24.902 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:18:25.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:24 smithi039 ceph-mon[32062]: pgmap v576: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:25.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:24 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:18:25.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:24 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:25.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:24 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:18:25.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:24 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:25.157 INFO:tasks.workunit.client.0.smithi039.stdout:1/600: open fd = 3 2024-07-04T08:18:25.157 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:2/600: open fd = 3 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:3/600: open fd = 3 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:4/600: open fd = 3 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.158 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:5/600: open fd = 3 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:6/600: open fd = 3 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:7/600: open fd = 3 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.159 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:8/600: open fd = 3 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:9/600: open fd = 3 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:10/600: open fd = 3 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:11/600: open fd = 3 2024-07-04T08:18:25.160 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:12/600: open fd = 3 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:13/600: open fd = 3 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:14/600: open fd = 3 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.161 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:15/600: open fd = 3 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:16/600: open fd = 3 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:17/600: open fd = 3 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.162 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:18/600: open fd = 3 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:19/600: open fd = 3 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:20/600: open fd = 3 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:21/600: open fd = 3 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.163 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:22/600: open fd = 3 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:23/600: open fd = 3 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:24/600: open fd = 3 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:25/600: open fd = 3 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.164 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:26/600: open fd = 3 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:27/600: open fd = 3 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:28/600: open fd = 3 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.165 INFO:tasks.workunit.client.0.smithi039.stdout:29/600: open fd = 3 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:30/600: open fd = 3 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:31/600: open fd = 3 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:32/600: open fd = 3 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.166 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:33/600: open fd = 3 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:34/600: open fd = 3 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:35/600: open fd = 3 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:36/600: open fd = 3 2024-07-04T08:18:25.167 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:37/600: open fd = 3 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:38/600: open fd = 3 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:39/600: open fd = 3 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.168 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:40/600: open fd = 3 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:41/600: open fd = 3 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:42/600: open fd = 3 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:43/600: open fd = 3 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.169 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:44/600: open fd = 3 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:45/600: open fd = 3 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:46/600: open fd = 3 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.170 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:47/600: open fd = 3 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:48/600: open fd = 3 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:49/600: open fd = 3 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:50/600: open fd = 3 2024-07-04T08:18:25.171 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:51/600: open fd = 3 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:52/600: open fd = 3 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:53/600: open fd = 3 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.172 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:54/600: open fd = 3 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:55/600: open fd = 3 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:56/600: open fd = 3 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.173 INFO:tasks.workunit.client.0.smithi039.stdout:57/600: open fd = 3 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:58/600: open fd = 3 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:59/600: open fd = 3 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.174 INFO:tasks.workunit.client.0.smithi039.stdout:60/600: open fd = 3 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:61/600: open fd = 3 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:62/600: open fd = 3 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.175 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.176 INFO:tasks.workunit.client.0.smithi039.stdout:63/600: open fd = 3 2024-07-04T08:18:25.176 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.176 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:18:25.176 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:18:25.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:24 smithi133 ceph-mon[37050]: pgmap v576: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:24 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:18:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:24 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:24 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:18:25.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:24 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:25.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:24 smithi050 ceph-mon[36577]: pgmap v576: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:25.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:24 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:18:25.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:24 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:25.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:24 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:18:25.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:24 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:27.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:26 smithi039 ceph-mon[32062]: pgmap v577: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:27.196 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:26 smithi050 ceph-mon[36577]: pgmap v577: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:27.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:26 smithi133 ceph-mon[37050]: pgmap v577: 129 pgs: 129 active+clean; 2.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.714+0000 7f1f05000640 1 -- 172.21.15.39:0/3146177179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f0010d170 msgr2=0x7f1f0010f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.714+0000 7f1f05000640 1 --2- 172.21.15.39:0/3146177179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f0010d170 0x7f1f0010f660 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009ca0 tx=0x7f1ef802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.715+0000 7f1f05000640 1 -- 172.21.15.39:0/3146177179 shutdown_connections 2024-07-04T08:18:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.715+0000 7f1f05000640 1 --2- 172.21.15.39:0/3146177179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f0010d170 0x7f1f0010f660 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.715+0000 7f1f05000640 1 --2- 172.21.15.39:0/3146177179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00104660 0x7f1f0010cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.715+0000 7f1f05000640 1 --2- 172.21.15.39:0/3146177179 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f00103d10 0x7f1f00104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.715+0000 7f1f05000640 1 -- 172.21.15.39:0/3146177179 >> 172.21.15.39:0/3146177179 conn(0x7f1f000fd810 msgr2=0x7f1f000ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.715+0000 7f1f05000640 1 -- 172.21.15.39:0/3146177179 shutdown_connections 2024-07-04T08:18:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.715+0000 7f1f05000640 1 -- 172.21.15.39:0/3146177179 wait complete. 2024-07-04T08:18:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.716+0000 7f1f05000640 1 Processor -- start 2024-07-04T08:18:27.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.716+0000 7f1f05000640 1 -- start start 2024-07-04T08:18:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1f05000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00103d10 0x7f1f001997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1f05000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f00104660 0x7f1f00199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1f05000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f0010d170 0x7f1f0019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1f05000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00112e40 con 0x7f1f0010d170 2024-07-04T08:18:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1f05000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00112cc0 con 0x7f1f00104660 2024-07-04T08:18:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1f05000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f00112fc0 con 0x7f1f00103d10 2024-07-04T08:18:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1eff400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00103d10 0x7f1f001997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1effe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f0010d170 0x7f1f0019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1efea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f00104660 0x7f1f00199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1efea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f00104660 0x7f1f00199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50836/0 (socket says 172.21.15.39:50836) 2024-07-04T08:18:27.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1eff400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00103d10 0x7f1f001997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:48286/0 (socket says 172.21.15.39:48286) 2024-07-04T08:18:27.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.717+0000 7f1efea00640 1 -- 172.21.15.39:0/2096187032 learned_addr learned my addr 172.21.15.39:0/2096187032 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1efea00640 1 -- 172.21.15.39:0/2096187032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00103d10 msgr2=0x7f1f001997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1efea00640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00103d10 0x7f1f001997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1efea00640 1 -- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f0010d170 msgr2=0x7f1f0019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1efea00640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f0010d170 0x7f1f0019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1efea00640 1 -- 172.21.15.39:0/2096187032 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0019f720 con 0x7f1f00104660 2024-07-04T08:18:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1effe00640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f0010d170 0x7f1f0019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1efea00640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f00104660 0x7f1f00199d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00e9c0 tx=0x7f1eec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1ef7e00640 1 -- 172.21.15.39:0/2096187032 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00ccc0 con 0x7f1f00104660 2024-07-04T08:18:27.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1f05000640 1 -- 172.21.15.39:0/2096187032 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0019fa10 con 0x7f1f00104660 2024-07-04T08:18:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1ef7e00640 1 -- 172.21.15.39:0/2096187032 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eec0054d0 con 0x7f1f00104660 2024-07-04T08:18:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.718+0000 7f1ef7e00640 1 -- 172.21.15.39:0/2096187032 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec010430 con 0x7f1f00104660 2024-07-04T08:18:27.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.719+0000 7f1f05000640 1 -- 172.21.15.39:0/2096187032 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f001a6170 con 0x7f1f00104660 2024-07-04T08:18:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.723+0000 7f1f0cc37740 1 Processor -- start 2024-07-04T08:18:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.723+0000 7f1ef7e00640 1 -- 172.21.15.39:0/2096187032 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1eec005020 con 0x7f1f00104660 2024-07-04T08:18:27.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.723+0000 7f1f0cc37740 1 -- start start 2024-07-04T08:18:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.723+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.723+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x56240023de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.723+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x56240023e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.723+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5624001b3ea0 con 0x5623ffe99070 2024-07-04T08:18:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.723+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5624001a3440 con 0x5623ffe701b0 2024-07-04T08:18:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.723+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5624001a7480 con 0x5623ffe78390 2024-07-04T08:18:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1effe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1effe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57374/0 (socket says 172.21.15.39:57374) 2024-07-04T08:18:27.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1effe00640 1 -- 172.21.15.39:0/577689686 learned_addr learned my addr 172.21.15.39:0/577689686 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:27.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1eff400640 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x56240023de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1efea00640 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x56240023e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1effe00640 1 -- 172.21.15.39:0/577689686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 msgr2=0x56240023e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1effe00640 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x56240023e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1effe00640 1 -- 172.21.15.39:0/577689686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 msgr2=0x56240023de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1effe00640 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x56240023de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1effe00640 1 -- 172.21.15.39:0/577689686 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240013b610 con 0x5623ffe99070 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1efea00640 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x56240023e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.724+0000 7f1eff400640 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x56240023de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.725+0000 7f1effe00640 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023d910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009ca0 tx=0x7f1ef8009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d5e24d62a48f25 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.725+0000 7f1ef7e00640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1ee40b29e0 0x7f1ee40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.725+0000 7f1ef7e00640 1 -- 172.21.15.39:0/2096187032 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1eec0e2c40 con 0x7f1f00104660 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.725+0000 7f1ef4c00640 1 -- 172.21.15.39:0/577689686 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef802f3d0 con 0x5623ffe99070 2024-07-04T08:18:27.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.725+0000 7f1ef4c00640 1 -- 172.21.15.39:0/577689686 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef8036040 con 0x5623ffe99070 2024-07-04T08:18:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.725+0000 7f1ef4c00640 1 -- 172.21.15.39:0/577689686 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8004400 con 0x5623ffe99070 2024-07-04T08:18:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.725+0000 7f1eff400640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1ee40b29e0 0x7f1ee40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.726+0000 7f1f0cc37740 1 -- 172.21.15.39:0/577689686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 msgr2=0x56240023d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.726+0000 7f1eff400640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1ee40b29e0 0x7f1ee40b4ef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56240023c590 tx=0x7f1ef0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.726+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023d910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009ca0 tx=0x7f1ef8009490 comp rx=0 tx=0).stop 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.726+0000 7f1f0cc37740 1 -- 172.21.15.39:0/577689686 shutdown_connections 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.726+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x56240023e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.726+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x56240023de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.726+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/577689686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023d910 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.726+0000 7f1f0cc37740 1 -- 172.21.15.39:0/577689686 >> 172.21.15.39:0/577689686 conn(0x5623ffe8d460 msgr2=0x56240023b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.727+0000 7f1f0cc37740 1 -- 172.21.15.39:0/577689686 shutdown_connections 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.727+0000 7f1f0cc37740 1 -- 172.21.15.39:0/577689686 wait complete. 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.727+0000 7f1f0cc37740 1 -- start start 2024-07-04T08:18:27.728 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.727+0000 7f1f0cc37740 1 Processor -- start 2024-07-04T08:18:27.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.728+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x562400249580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.728+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x562400249b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.728+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400259640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.728+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562400007700 con 0x5623ffe701b0 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.728+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56240014ab70 con 0x5623ffe78390 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.728+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56240024bf10 con 0x5623ffe99070 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.728+0000 7f1f0cc37740 10 client.0 _ll_register_callbacks cb 0x56240019c530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1eff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x562400249580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1efea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x562400249b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1efea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x562400249b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50842/0 (socket says 172.21.15.39:50842) 2024-07-04T08:18:27.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1eff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x562400249580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57384/0 (socket says 172.21.15.39:57384) 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1efea00640 1 -- 172.21.15.39:0/3785421739 learned_addr learned my addr 172.21.15.39:0/3785421739 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1effe00640 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400259640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1efea00640 1 -- 172.21.15.39:0/3785421739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 msgr2=0x562400259640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1efea00640 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400259640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1efea00640 1 -- 172.21.15.39:0/3785421739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 msgr2=0x562400249580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1efea00640 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x562400249580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1efea00640 1 -- 172.21.15.39:0/3785421739 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623fff04650 con 0x5623ffe78390 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1effe00640 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400259640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1eff400640 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x562400249580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.729+0000 7f1efea00640 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x562400249b00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00c6a0 tx=0x7f1eec00ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c27e1ef7c250f208 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:27.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.730+0000 7f1ee9600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.730+0000 7f1ee9600640 1 -- 172.21.15.39:0/3785421739 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0e0490 con 0x5623ffe78390 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.730+0000 7f1ee9600640 1 -- 172.21.15.39:0/3785421739 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1eec0b2560 con 0x5623ffe78390 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.730+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3785421739 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef8036040 con 0x5623ffe78390 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.730+0000 7f1ee9600640 1 -- 172.21.15.39:0/3785421739 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1eec005680 con 0x5623ffe78390 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.730+0000 7f1f0cc37740 10 client.26255 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.730+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3785421739 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56240025c400 con 0x5623ffe78390 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.730+0000 7f1ee9600640 1 -- 172.21.15.39:0/3785421739 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef8036040 con 0x5623ffe78390 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.731+0000 7f1ee9600640 1 -- 172.21.15.39:0/3785421739 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1eec0c4080 con 0x5623ffe78390 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.731+0000 7f1f0cc37740 10 client.26255 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.731+0000 7f1f0cc37740 10 client.26255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.731+0000 7f1f0cc37740 20 client.26255 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:27.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.731+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5623ffe579f0 0x56240025f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.731+0000 7f1f0cc37740 4 client.26255 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.731+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3785421739 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624001b5df0 con 0x5623ffe579f0 2024-07-04T08:18:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.731+0000 7f1eff400640 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5623ffe579f0 0x56240025f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.732+0000 7f1eff400640 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5623ffe579f0 0x56240025f8d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b6005888cc1a40 server_cookie=4fd8bc8d7d2441e7 in_seq=0 out_seq=0 2024-07-04T08:18:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.732+0000 7f1ee9600640 10 client.26255 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1ee9600640 1 -- 172.21.15.39:0/3785421739 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5624001b5df0 con 0x5623ffe579f0 2024-07-04T08:18:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1ee9600640 10 client.26255 handle_command_reply: tid=0 2024-07-04T08:18:27.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1f0cc37740 1 client.26255 shutdown 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3785421739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 msgr2=0x562400249b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x562400249b00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00c6a0 tx=0x7f1eec00ce60 comp rx=0 tx=0).stop 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3785421739 shutdown_connections 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5623ffe579f0 0x56240025f8d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400259640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1ee9600640 0 client.26255 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x562400249b00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3785421739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x562400249580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.733+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3785421739 >> 172.21.15.39:0/3785421739 conn(0x5623ffe8d460 msgr2=0x562400240ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:27.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.734+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3785421739 shutdown_connections 2024-07-04T08:18:27.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.734+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3785421739 wait complete. 2024-07-04T08:18:27.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.734+0000 7f1f0cc37740 20 client.26255 trim_cache size 0 max 16384 2024-07-04T08:18:27.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.754+0000 7f1f0cc37740 1 Processor -- start 2024-07-04T08:18:27.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.754+0000 7f1f0cc37740 1 -- start start 2024-07-04T08:18:27.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.754+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x562400243130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.754+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x5624002594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.754+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.754+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001add10 con 0x5623ffe99070 2024-07-04T08:18:27.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.754+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001af850 con 0x5623ffe701b0 2024-07-04T08:18:27.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.754+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001ae3e0 con 0x5623ffe78390 2024-07-04T08:18:27.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1efea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x5624002594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1eff400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x562400243130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1effe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1efea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x5624002594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:48316/0 (socket says 172.21.15.39:48316) 2024-07-04T08:18:27.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1efea00640 1 -- 172.21.15.39:0/874236616 learned_addr learned my addr 172.21.15.39:0/874236616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:27.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1eff400640 1 -- 172.21.15.39:0/874236616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 msgr2=0x5624002594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:27.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1eff400640 1 --2- 172.21.15.39:0/874236616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x5624002594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1eff400640 1 -- 172.21.15.39:0/874236616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 msgr2=0x56240023ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:27.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1eff400640 1 --2- 172.21.15.39:0/874236616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.755+0000 7f1eff400640 1 -- 172.21.15.39:0/874236616 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624002482e0 con 0x5623ffe701b0 2024-07-04T08:18:27.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.756+0000 7f1eff400640 1 --2- 172.21.15.39:0/874236616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x562400243130 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x56240025ff40 tx=0x7f1ef001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=856518e74af9bc25 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:27.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.756+0000 7f1eebe00640 1 -- 172.21.15.39:0/874236616 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef003a070 con 0x5623ffe701b0 2024-07-04T08:18:27.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.756+0000 7f1eebe00640 1 -- 172.21.15.39:0/874236616 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef001edc0 con 0x5623ffe701b0 2024-07-04T08:18:27.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.756+0000 7f1eebe00640 1 -- 172.21.15.39:0/874236616 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0035a60 con 0x5623ffe701b0 2024-07-04T08:18:27.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 -- 172.21.15.39:0/874236616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 msgr2=0x562400243130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:27.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/874236616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x562400243130 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x56240025ff40 tx=0x7f1ef001b000 comp rx=0 tx=0).stop 2024-07-04T08:18:27.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 -- 172.21.15.39:0/874236616 shutdown_connections 2024-07-04T08:18:27.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/874236616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe99070 0x56240023ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/874236616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe78390 0x5624002594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/874236616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe701b0 0x562400243130 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 -- 172.21.15.39:0/874236616 >> 172.21.15.39:0/874236616 conn(0x5623ffe8d460 msgr2=0x562400242a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:27.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 -- 172.21.15.39:0/874236616 shutdown_connections 2024-07-04T08:18:27.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 -- 172.21.15.39:0/874236616 wait complete. 2024-07-04T08:18:27.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.757+0000 7f1f0cc37740 1 -- start start 2024-07-04T08:18:27.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.758+0000 7f1f0cc37740 1 Processor -- start 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.758+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x56240024c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.758+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x56240024cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.758+0000 7f1f0cc37740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400251f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.758+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001af180 con 0x5623ffe701b0 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.758+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001aeab0 con 0x5623ffe78390 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.758+0000 7f1f0cc37740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f001abcf0 con 0x5623ffe99070 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.758+0000 7f1f0cc37740 10 client.0 _ll_register_callbacks cb 0x7f1eec004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1eff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x56240024c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1eff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x56240024c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:57414/0 (socket says 172.21.15.39:57414) 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1eff400640 1 -- 172.21.15.39:0/3426776163 learned_addr learned my addr 172.21.15.39:0/3426776163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1effe00640 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400251f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1efea00640 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x56240024cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1eff400640 1 -- 172.21.15.39:0/3426776163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 msgr2=0x562400251f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:27.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1eff400640 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400251f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1eff400640 1 -- 172.21.15.39:0/3426776163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 msgr2=0x56240024cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:27.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1eff400640 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x56240024cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1eff400640 1 -- 172.21.15.39:0/3426776163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562400252570 con 0x5623ffe701b0 2024-07-04T08:18:27.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.759+0000 7f1effe00640 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400251f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:27.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.760+0000 7f1eff400640 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x56240024c900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0005bc0 tx=0x7f1ef0035460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e76a622a569820 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:27.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.760+0000 7f1ee9600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:27.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.760+0000 7f1ee9600640 1 -- 172.21.15.39:0/3426776163 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0045060 con 0x5623ffe701b0 2024-07-04T08:18:27.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.760+0000 7f1ee9600640 1 -- 172.21.15.39:0/3426776163 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1ef0005250 con 0x5623ffe701b0 2024-07-04T08:18:27.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.760+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3426776163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef001edc0 con 0x5623ffe701b0 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1ee9600640 1 -- 172.21.15.39:0/3426776163 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1ef004c050 con 0x5623ffe701b0 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1f0cc37740 10 client.16131 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3426776163 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562400255950 con 0x5623ffe701b0 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1ee9600640 1 -- 172.21.15.39:0/3426776163 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef001edc0 con 0x5623ffe701b0 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1ee9600640 1 -- 172.21.15.39:0/3426776163 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1ef000f9a0 con 0x5623ffe701b0 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1f0cc37740 10 client.16131 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1f0cc37740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1f0cc37740 20 client.16131 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5623ffe2cda0 0x5624002ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1f0cc37740 4 client.16131 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.761+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3426776163 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624001b1890 con 0x5623ffe2cda0 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.762+0000 7f1efea00640 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5623ffe2cda0 0x5624002ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:27.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.762+0000 7f1efea00640 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5623ffe2cda0 0x5624002ce7a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e611ef4c2ea6ce19 server_cookie=e451e0b8e5ad1b2a in_seq=0 out_seq=0 2024-07-04T08:18:27.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.762+0000 7f1ee9600640 10 client.16131 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:27.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1ee9600640 1 -- 172.21.15.39:0/3426776163 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5624001b1890 con 0x5623ffe2cda0 2024-07-04T08:18:27.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1ee9600640 10 client.16131 handle_command_reply: tid=0 2024-07-04T08:18:27.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 client.16131 shutdown 2024-07-04T08:18:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3426776163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 msgr2=0x56240024c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x56240024c900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0005bc0 tx=0x7f1ef0035460 comp rx=0 tx=0).stop 2024-07-04T08:18:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3426776163 shutdown_connections 2024-07-04T08:18:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5623ffe2cda0 0x5624002ce7a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5623ffe99070 0x562400251f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623ffe78390 0x56240024cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 --2- 172.21.15.39:0/3426776163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5623ffe701b0 0x56240024c900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3426776163 >> 172.21.15.39:0/3426776163 conn(0x5623ffe8d460 msgr2=0x56240023e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1ee9600640 0 client.16131 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:27.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3426776163 shutdown_connections 2024-07-04T08:18:27.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 1 -- 172.21.15.39:0/3426776163 wait complete. 2024-07-04T08:18:27.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.763+0000 7f1f0cc37740 20 client.16131 trim_cache size 0 max 16384 2024-07-04T08:18:27.766 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:18:27.766 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:18:27.766 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:18:27.766 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:18:27.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 -- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1ee40b29e0 msgr2=0x7f1ee40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:27.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1ee40b29e0 0x7f1ee40b4ef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56240023c590 tx=0x7f1ef0005f90 comp rx=0 tx=0).stop 2024-07-04T08:18:27.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 -- 172.21.15.39:0/2096187032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f00104660 msgr2=0x7f1f00199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:27.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f00104660 0x7f1f00199d50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00e9c0 tx=0x7f1eec00ee80 comp rx=0 tx=0).stop 2024-07-04T08:18:27.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 -- 172.21.15.39:0/2096187032 shutdown_connections 2024-07-04T08:18:27.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1ee40b29e0 0x7f1ee40b4ef0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f0010d170 0x7f1f0019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f00104660 0x7f1f00199d50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 --2- 172.21.15.39:0/2096187032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f00103d10 0x7f1f001997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:27.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 -- 172.21.15.39:0/2096187032 >> 172.21.15.39:0/2096187032 conn(0x7f1f000fd810 msgr2=0x7f1f00106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:27.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 -- 172.21.15.39:0/2096187032 shutdown_connections 2024-07-04T08:18:27.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:27.764+0000 7f1f05000640 1 -- 172.21.15.39:0/2096187032 wait complete. 2024-07-04T08:18:28.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:27 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:18:28.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:27 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:18:28.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:27 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:18:28.726 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:18:28.727 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:18:29.226 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:29.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:28 smithi133 ceph-mon[37050]: pgmap v578: 129 pgs: 129 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-07-04T08:18:29.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:28 smithi039 ceph-mon[32062]: pgmap v578: 129 pgs: 129 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-07-04T08:18:29.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:28 smithi050 ceph-mon[36577]: pgmap v578: 129 pgs: 129 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-07-04T08:18:31.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:30 smithi039 ceph-mon[32062]: pgmap v579: 129 pgs: 129 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:31.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:30 smithi133 ceph-mon[37050]: pgmap v579: 129 pgs: 129 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:31.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:30 smithi050 ceph-mon[36577]: pgmap v579: 129 pgs: 129 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:31.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.588+0000 7f6c90600640 1 -- 172.21.15.39:0/358584118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c8810ac40 msgr2=0x7f6c8810b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:31.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.588+0000 7f6c90600640 1 --2- 172.21.15.39:0/358584118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c8810ac40 0x7f6c8810b050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009860 tx=0x7f6c7402a340 comp rx=0 tx=0).stop 2024-07-04T08:18:31.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.588+0000 7f6c90600640 1 -- 172.21.15.39:0/358584118 shutdown_connections 2024-07-04T08:18:31.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.588+0000 7f6c90600640 1 --2- 172.21.15.39:0/358584118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c8810b590 0x7f6c88111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.588+0000 7f6c90600640 1 --2- 172.21.15.39:0/358584118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c8810ac40 0x7f6c8810b050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.588+0000 7f6c90600640 1 --2- 172.21.15.39:0/358584118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c88105300 0x7f6c88105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.588+0000 7f6c90600640 1 -- 172.21.15.39:0/358584118 >> 172.21.15.39:0/358584118 conn(0x7f6c88100d10 msgr2=0x7f6c88103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:31.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.588+0000 7f6c90600640 1 -- 172.21.15.39:0/358584118 shutdown_connections 2024-07-04T08:18:31.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.588+0000 7f6c90600640 1 -- 172.21.15.39:0/358584118 wait complete. 2024-07-04T08:18:31.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.590+0000 7f6c90600640 1 Processor -- start 2024-07-04T08:18:31.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.590+0000 7f6c90600640 1 -- start start 2024-07-04T08:18:31.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c90600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c88105300 0x7f6c8819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:31.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c90600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8810ac40 0x7f6c8819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:31.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c90600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c8810b590 0x7f6c881a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:31.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c90600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88115220 con 0x7f6c88105300 2024-07-04T08:18:31.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c90600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881150a0 con 0x7f6c8810b590 2024-07-04T08:18:31.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c90600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c881153a0 con 0x7f6c8810ac40 2024-07-04T08:18:31.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c8dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c88105300 0x7f6c8819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:31.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c8dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c88105300 0x7f6c8819bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44974/0 (socket says 172.21.15.39:44974) 2024-07-04T08:18:31.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c8dc00640 1 -- 172.21.15.39:0/938315719 learned_addr learned my addr 172.21.15.39:0/938315719 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:31.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c8d200640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8810ac40 0x7f6c8819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:31.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.591+0000 7f6c8e600640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c8810b590 0x7f6c881a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:31.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.592+0000 7f6c8dc00640 1 -- 172.21.15.39:0/938315719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8810ac40 msgr2=0x7f6c8819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:31.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.592+0000 7f6c8dc00640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8810ac40 0x7f6c8819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.592+0000 7f6c8dc00640 1 -- 172.21.15.39:0/938315719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c8810b590 msgr2=0x7f6c881a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:31.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.592+0000 7f6c8dc00640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c8810b590 0x7f6c881a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.592+0000 7f6c8dc00640 1 -- 172.21.15.39:0/938315719 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c881a1be0 con 0x7f6c88105300 2024-07-04T08:18:31.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.592+0000 7f6c8e600640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c8810b590 0x7f6c881a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:31.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.592+0000 7f6c8dc00640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c88105300 0x7f6c8819bc70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800e9f0 tx=0x7f6c7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:31.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.593+0000 7f6c86000640 1 -- 172.21.15.39:0/938315719 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7800ccc0 con 0x7f6c88105300 2024-07-04T08:18:31.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.593+0000 7f6c86000640 1 -- 172.21.15.39:0/938315719 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c780054d0 con 0x7f6c88105300 2024-07-04T08:18:31.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.593+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c881a1ed0 con 0x7f6c88105300 2024-07-04T08:18:31.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.593+0000 7f6c86000640 1 -- 172.21.15.39:0/938315719 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78010430 con 0x7f6c88105300 2024-07-04T08:18:31.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.593+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c881a8630 con 0x7f6c88105300 2024-07-04T08:18:31.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.594+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c88105710 con 0x7f6c88105300 2024-07-04T08:18:31.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.597+0000 7f6c86000640 1 -- 172.21.15.39:0/938315719 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6c780046c0 con 0x7f6c88105300 2024-07-04T08:18:31.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.598+0000 7f6c86000640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c6c0b2960 0x7f6c6c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:31.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.598+0000 7f6c86000640 1 -- 172.21.15.39:0/938315719 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f6c780e2e20 con 0x7f6c88105300 2024-07-04T08:18:31.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.599+0000 7f6c8d200640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c6c0b2960 0x7f6c6c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:31.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.599+0000 7f6c8d200640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c6c0b2960 0x7f6c6c0b4e70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c74035040 tx=0x7f6c74030340 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:31.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.601+0000 7f6c86000640 1 -- 172.21.15.39:0/938315719 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f6c78055110 con 0x7f6c88105300 2024-07-04T08:18:31.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.860+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c8819cda0 con 0x7f6c88105300 2024-07-04T08:18:31.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.862+0000 7f6c86000640 1 -- 172.21.15.39:0/938315719 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f6c780aa470 con 0x7f6c88105300 2024-07-04T08:18:31.863 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:18:31.863 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:18:31.863 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:18:31.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c6c0b2960 msgr2=0x7f6c6c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:31.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c6c0b2960 0x7f6c6c0b4e70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6c74035040 tx=0x7f6c74030340 comp rx=0 tx=0).stop 2024-07-04T08:18:31.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c88105300 msgr2=0x7f6c8819bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:31.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c88105300 0x7f6c8819bc70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800e9f0 tx=0x7f6c7800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 shutdown_connections 2024-07-04T08:18:31.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6c6c0b2960 0x7f6c6c0b4e70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c8810b590 0x7f6c881a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c8810ac40 0x7f6c8819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 --2- 172.21.15.39:0/938315719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c88105300 0x7f6c8819bc70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:31.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.865+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 >> 172.21.15.39:0/938315719 conn(0x7f6c88100d10 msgr2=0x7f6c88109410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:31.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.866+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 shutdown_connections 2024-07-04T08:18:31.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:31.866+0000 7f6c90600640 1 -- 172.21.15.39:0/938315719 wait complete. 2024-07-04T08:18:32.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:31 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/938315719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:18:32.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:31 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/938315719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:18:32.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:31 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/938315719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:18:32.719 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:18:33.209 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:33.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:32 smithi133 ceph-mon[37050]: pgmap v580: 129 pgs: 129 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:33.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:32 smithi039 ceph-mon[32062]: pgmap v580: 129 pgs: 129 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:33.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:32 smithi050 ceph-mon[36577]: pgmap v580: 129 pgs: 129 active+clean; 2.8 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:35.260 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:35 smithi039 ceph-mon[32062]: pgmap v581: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:35.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:35 smithi133 ceph-mon[37050]: pgmap v581: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:35.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:35 smithi050 ceph-mon[36577]: pgmap v581: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.601+0000 7f7a81c00640 1 -- 172.21.15.39:0/1973325162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c074d00 msgr2=0x7f7a7c075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.601+0000 7f7a81c00640 1 --2- 172.21.15.39:0/1973325162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c074d00 0x7f7a7c075110 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009900 tx=0x7f7a6c02a400 comp rx=0 tx=0).stop 2024-07-04T08:18:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.601+0000 7f7a81c00640 1 -- 172.21.15.39:0/1973325162 shutdown_connections 2024-07-04T08:18:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.601+0000 7f7a81c00640 1 --2- 172.21.15.39:0/1973325162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a7c10f460 0x7f7a7c1118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.601+0000 7f7a81c00640 1 --2- 172.21.15.39:0/1973325162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a7c075650 0x7f7a7c10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.601+0000 7f7a81c00640 1 --2- 172.21.15.39:0/1973325162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c074d00 0x7f7a7c075110 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.601+0000 7f7a81c00640 1 -- 172.21.15.39:0/1973325162 >> 172.21.15.39:0/1973325162 conn(0x7f7a7c0fe990 msgr2=0x7f7a7c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:35.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.602+0000 7f7a81c00640 1 -- 172.21.15.39:0/1973325162 shutdown_connections 2024-07-04T08:18:35.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.602+0000 7f7a81c00640 1 -- 172.21.15.39:0/1973325162 wait complete. 2024-07-04T08:18:35.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.603+0000 7f7a81c00640 1 Processor -- start 2024-07-04T08:18:35.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.603+0000 7f7a81c00640 1 -- start start 2024-07-04T08:18:35.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a81c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a7c074d00 0x7f7a7c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a81c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c075650 0x7f7a7c19c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a81c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a7c10f460 0x7f7a7c1a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a81c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c114ff0 con 0x7f7a7c075650 2024-07-04T08:18:35.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a81c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c114e70 con 0x7f7a7c10f460 2024-07-04T08:18:35.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a81c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c115170 con 0x7f7a7c074d00 2024-07-04T08:18:35.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a73e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c075650 0x7f7a7c19c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a7be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a7c074d00 0x7f7a7c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a81200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a7c10f460 0x7f7a7c1a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a73e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c075650 0x7f7a7c19c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44988/0 (socket says 172.21.15.39:44988) 2024-07-04T08:18:35.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a7be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a7c074d00 0x7f7a7c19ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:35594/0 (socket says 172.21.15.39:35594) 2024-07-04T08:18:35.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.604+0000 7f7a73e00640 1 -- 172.21.15.39:0/2780871520 learned_addr learned my addr 172.21.15.39:0/2780871520 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:35.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a7be00640 1 -- 172.21.15.39:0/2780871520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a7c10f460 msgr2=0x7f7a7c1a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:35.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a7be00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a7c10f460 0x7f7a7c1a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a7be00640 1 -- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c075650 msgr2=0x7f7a7c19c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:35.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a7be00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c075650 0x7f7a7c19c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a7be00640 1 -- 172.21.15.39:0/2780871520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c1a19f0 con 0x7f7a7c074d00 2024-07-04T08:18:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a73e00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c075650 0x7f7a7c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a7be00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a7c074d00 0x7f7a7c19ba80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c02a920 tx=0x7f7a6c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a79600640 1 -- 172.21.15.39:0/2780871520 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c03f030 con 0x7f7a7c074d00 2024-07-04T08:18:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a81c00640 1 -- 172.21.15.39:0/2780871520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7c1a1c80 con 0x7f7a7c074d00 2024-07-04T08:18:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.605+0000 7f7a79600640 1 -- 172.21.15.39:0/2780871520 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a6c004490 con 0x7f7a7c074d00 2024-07-04T08:18:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.606+0000 7f7a79600640 1 -- 172.21.15.39:0/2780871520 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c035070 con 0x7f7a7c074d00 2024-07-04T08:18:35.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.606+0000 7f7a81c00640 1 -- 172.21.15.39:0/2780871520 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7c1a8440 con 0x7f7a7c074d00 2024-07-04T08:18:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.609+0000 7f7a897c9740 1 Processor -- start 2024-07-04T08:18:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.609+0000 7f7a897c9740 1 -- start start 2024-07-04T08:18:35.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011d9b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a79600640 1 -- 172.21.15.39:0/2780871520 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7a6c042030 con 0x7f7a7c074d00 2024-07-04T08:18:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011d9be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011d9c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a897c9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561011d11ee0 con 0x5610119ce1b0 2024-07-04T08:18:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a897c9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561011d01480 con 0x5610119f7070 2024-07-04T08:18:35.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a897c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561011d054c0 con 0x5610119d6390 2024-07-04T08:18:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a81200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011d9b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a7be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011d9be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a81200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011d9b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51758/0 (socket says 172.21.15.39:51758) 2024-07-04T08:18:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a81200640 1 -- 172.21.15.39:0/1946125307 learned_addr learned my addr 172.21.15.39:0/1946125307 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:35.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.610+0000 7f7a73e00640 1 --2- 172.21.15.39:0/1946125307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011d9c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.611+0000 7f7a81200640 1 -- 172.21.15.39:0/1946125307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 msgr2=0x561011d9c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.611+0000 7f7a81200640 1 --2- 172.21.15.39:0/1946125307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011d9c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.611+0000 7f7a81200640 1 -- 172.21.15.39:0/1946125307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 msgr2=0x561011d9be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.611+0000 7f7a81200640 1 --2- 172.21.15.39:0/1946125307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011d9be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.611+0000 7f7a81200640 1 -- 172.21.15.39:0/1946125307 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561011c99650 con 0x5610119f7070 2024-07-04T08:18:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.611+0000 7f7a81200640 1 --2- 172.21.15.39:0/1946125307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011d9b950 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009f50 tx=0x7f7a740118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9544f1616d750f57 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.611+0000 7f7a71600640 1 -- 172.21.15.39:0/1946125307 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74004d20 con 0x5610119f7070 2024-07-04T08:18:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.612+0000 7f7a71600640 1 -- 172.21.15.39:0/1946125307 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a74004050 con 0x5610119f7070 2024-07-04T08:18:35.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.612+0000 7f7a71600640 1 -- 172.21.15.39:0/1946125307 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74005280 con 0x5610119f7070 2024-07-04T08:18:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.612+0000 7f7a79600640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a600b2960 0x7f7a600b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.612+0000 7f7a79600640 1 -- 172.21.15.39:0/2780871520 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7a6c102bf0 con 0x7f7a7c074d00 2024-07-04T08:18:35.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.612+0000 7f7a7be00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a600b2960 0x7f7a600b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a897c9740 1 -- 172.21.15.39:0/1946125307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 msgr2=0x561011d9b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a897c9740 1 --2- 172.21.15.39:0/1946125307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011d9b950 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009f50 tx=0x7f7a740118d0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a897c9740 1 -- 172.21.15.39:0/1946125307 shutdown_connections 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a897c9740 1 --2- 172.21.15.39:0/1946125307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011d9be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a897c9740 1 --2- 172.21.15.39:0/1946125307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011d9c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a897c9740 1 --2- 172.21.15.39:0/1946125307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011d9b950 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a897c9740 1 -- 172.21.15.39:0/1946125307 >> 172.21.15.39:0/1946125307 conn(0x5610119eb460 msgr2=0x561011d99840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a897c9740 1 -- 172.21.15.39:0/1946125307 shutdown_connections 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a7be00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a600b2960 0x7f7a600b4e70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x561011d9a5d0 tx=0x7f7a6c1066a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.613+0000 7f7a897c9740 1 -- 172.21.15.39:0/1946125307 wait complete. 2024-07-04T08:18:35.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.614+0000 7f7a897c9740 1 -- start start 2024-07-04T08:18:35.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.614+0000 7f7a897c9740 1 Processor -- start 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011da7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119d6390 0x561011da7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119f7070 0x561011db3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a897c9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561011b65700 con 0x5610119ce1b0 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a897c9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561011ca8bb0 con 0x5610119d6390 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a897c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561011daa130 con 0x5610119f7070 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a897c9740 10 client.0 _ll_register_callbacks cb 0x561011cfa570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a81200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011da7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a73e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119d6390 0x561011da7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a81200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011da7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45004/0 (socket says 172.21.15.39:45004) 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a81200640 1 -- 172.21.15.39:0/2253072076 learned_addr learned my addr 172.21.15.39:0/2253072076 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:35.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.615+0000 7f7a81200640 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119f7070 0x561011db3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a73e00640 1 -- 172.21.15.39:0/2253072076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119f7070 msgr2=0x561011db3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a73e00640 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119f7070 0x561011db3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a73e00640 1 -- 172.21.15.39:0/2253072076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 msgr2=0x561011da7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a73e00640 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011da7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a73e00640 1 -- 172.21.15.39:0/2253072076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561011a62650 con 0x5610119d6390 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a81200640 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011da7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a73e00640 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119d6390 0x561011da7a70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c1118e0 tx=0x7f7a6400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c32b7342a68db168 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a6a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a6a000640 1 -- 172.21.15.39:0/2253072076 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64012110 con 0x5610119d6390 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a897c9740 1 -- 172.21.15.39:0/2253072076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a74004050 con 0x5610119d6390 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.616+0000 7f7a6a000640 1 -- 172.21.15.39:0/2253072076 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7a64004050 con 0x5610119d6390 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a6a000640 1 -- 172.21.15.39:0/2253072076 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a64025050 con 0x5610119d6390 2024-07-04T08:18:35.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a897c9740 10 client.26279 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:35.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a897c9740 1 -- 172.21.15.39:0/2253072076 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561011db6170 con 0x5610119d6390 2024-07-04T08:18:35.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a6a000640 1 -- 172.21.15.39:0/2253072076 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a74004050 con 0x5610119d6390 2024-07-04T08:18:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a6a000640 1 -- 172.21.15.39:0/2253072076 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7a64010430 con 0x5610119d6390 2024-07-04T08:18:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a897c9740 10 client.26279 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a897c9740 10 client.26279 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:18:35.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a897c9740 20 client.26279 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a897c9740 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5610119b59f0 0x561011db9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a897c9740 4 client.26279 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.617+0000 7f7a897c9740 1 -- 172.21.15.39:0/2253072076 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561011d13e30 con 0x5610119b59f0 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.618+0000 7f7a81200640 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5610119b59f0 0x561011db9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.619+0000 7f7a81200640 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5610119b59f0 0x561011db9620 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2fa1a29dfaf08fd4 server_cookie=83560264fa18b275 in_seq=0 out_seq=0 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.619+0000 7f7a6a000640 10 client.26279 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.619+0000 7f7a6a000640 1 -- 172.21.15.39:0/2253072076 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561011d13e30 con 0x5610119b59f0 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.619+0000 7f7a6a000640 10 client.26279 handle_command_reply: tid=0 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.619+0000 7f7a897c9740 1 client.26279 shutdown 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 -- 172.21.15.39:0/2253072076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119d6390 msgr2=0x561011da7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119d6390 0x561011da7a70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c1118e0 tx=0x7f7a6400f790 comp rx=0 tx=0).stop 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 -- 172.21.15.39:0/2253072076 shutdown_connections 2024-07-04T08:18:35.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5610119b59f0 0x561011db9620 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119f7070 0x561011db3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119d6390 0x561011da7a70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 --2- 172.21.15.39:0/2253072076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011da7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a6a000640 0 client.26279 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:18:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 -- 172.21.15.39:0/2253072076 >> 172.21.15.39:0/2253072076 conn(0x5610119eb460 msgr2=0x561011d9eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 -- 172.21.15.39:0/2253072076 shutdown_connections 2024-07-04T08:18:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 1 -- 172.21.15.39:0/2253072076 wait complete. 2024-07-04T08:18:35.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.620+0000 7f7a897c9740 20 client.26279 trim_cache size 0 max 16384 2024-07-04T08:18:35.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.636+0000 7f7a897c9740 1 Processor -- start 2024-07-04T08:18:35.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.636+0000 7f7a897c9740 1 -- start start 2024-07-04T08:18:35.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.636+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119ce1b0 0x561011d99770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.636+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011da0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.636+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119f7070 0x561011da10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.636+0000 7f7a897c9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c1b22f0 con 0x5610119f7070 2024-07-04T08:18:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.636+0000 7f7a897c9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c1b3e30 con 0x5610119ce1b0 2024-07-04T08:18:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.636+0000 7f7a897c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c1b29c0 con 0x5610119d6390 2024-07-04T08:18:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.636+0000 7f7a81200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119f7070 0x561011da10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a81200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119ce1b0 0x561011d99770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a73e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011da0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a81200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119ce1b0 0x561011d99770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51784/0 (socket says 172.21.15.39:51784) 2024-07-04T08:18:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a73e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011da0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:35606/0 (socket says 172.21.15.39:35606) 2024-07-04T08:18:35.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a81200640 1 -- 172.21.15.39:0/1391132228 learned_addr learned my addr 172.21.15.39:0/1391132228 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a81200640 1 -- 172.21.15.39:0/1391132228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 msgr2=0x561011da0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:35.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a81200640 1 --2- 172.21.15.39:0/1391132228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011da0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a81200640 1 -- 172.21.15.39:0/1391132228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119f7070 msgr2=0x561011da10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a81200640 1 --2- 172.21.15.39:0/1391132228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119f7070 0x561011da10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a81200640 1 -- 172.21.15.39:0/1391132228 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561011d9dc40 con 0x5610119ce1b0 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a73e00640 1 --2- 172.21.15.39:0/1391132228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011da0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.637+0000 7f7a81200640 1 --2- 172.21.15.39:0/1391132228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119ce1b0 0x561011d99770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7a740048d0 tx=0x7f7a74004900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb753f33af3eae82 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.638+0000 7f7a70c00640 1 -- 172.21.15.39:0/1391132228 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74004050 con 0x5610119ce1b0 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.638+0000 7f7a70c00640 1 -- 172.21.15.39:0/1391132228 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a74004e50 con 0x5610119ce1b0 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.638+0000 7f7a70c00640 1 -- 172.21.15.39:0/1391132228 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74005310 con 0x5610119ce1b0 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.638+0000 7f7a897c9740 1 -- 172.21.15.39:0/1391132228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119ce1b0 msgr2=0x561011d99770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.638+0000 7f7a897c9740 1 --2- 172.21.15.39:0/1391132228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119ce1b0 0x561011d99770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7a740048d0 tx=0x7f7a74004900 comp rx=0 tx=0).stop 2024-07-04T08:18:35.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.638+0000 7f7a897c9740 1 -- 172.21.15.39:0/1391132228 shutdown_connections 2024-07-04T08:18:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.639+0000 7f7a897c9740 1 --2- 172.21.15.39:0/1391132228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119f7070 0x561011da10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.639+0000 7f7a897c9740 1 --2- 172.21.15.39:0/1391132228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011da0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.639+0000 7f7a897c9740 1 --2- 172.21.15.39:0/1391132228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119ce1b0 0x561011d99770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.639+0000 7f7a897c9740 1 -- 172.21.15.39:0/1391132228 >> 172.21.15.39:0/1391132228 conn(0x5610119eb460 msgr2=0x561011d99060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.639+0000 7f7a897c9740 1 -- 172.21.15.39:0/1391132228 shutdown_connections 2024-07-04T08:18:35.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.639+0000 7f7a897c9740 1 -- 172.21.15.39:0/1391132228 wait complete. 2024-07-04T08:18:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.639+0000 7f7a897c9740 1 -- start start 2024-07-04T08:18:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.639+0000 7f7a897c9740 1 Processor -- start 2024-07-04T08:18:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011db2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011db2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a897c9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011dae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a897c9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c1b3760 con 0x5610119ce1b0 2024-07-04T08:18:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a897c9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c1b3090 con 0x5610119f7070 2024-07-04T08:18:35.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a897c9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c1b02d0 con 0x5610119d6390 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a897c9740 10 client.0 _ll_register_callbacks cb 0x7f7a64009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a81200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011db2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a81200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011db2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45026/0 (socket says 172.21.15.39:45026) 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a81200640 1 -- 172.21.15.39:0/3957004442 learned_addr learned my addr 172.21.15.39:0/3957004442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a81200640 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011dae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.640+0000 7f7a73e00640 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011db2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.641+0000 7f7a81200640 1 -- 172.21.15.39:0/3957004442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 msgr2=0x561011db2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.641+0000 7f7a81200640 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011db2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.641+0000 7f7a81200640 1 -- 172.21.15.39:0/3957004442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 msgr2=0x561011dae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.641+0000 7f7a81200640 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011dae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.641+0000 7f7a81200640 1 -- 172.21.15.39:0/3957004442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561011dae8f0 con 0x5610119ce1b0 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.641+0000 7f7a73e00640 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011db2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.641+0000 7f7a81200640 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011db2a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009d60 tx=0x7f7a740096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ade94ea2abf773 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.641+0000 7f7a6a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.642+0000 7f7a6a000640 1 -- 172.21.15.39:0/3957004442 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7403c060 con 0x5610119ce1b0 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.642+0000 7f7a6a000640 1 -- 172.21.15.39:0/3957004442 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7a74003ad0 con 0x5610119ce1b0 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.642+0000 7f7a897c9740 1 -- 172.21.15.39:0/3957004442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a74004e50 con 0x5610119ce1b0 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.642+0000 7f7a6a000640 1 -- 172.21.15.39:0/3957004442 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a74043050 con 0x5610119ce1b0 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.642+0000 7f7a897c9740 10 client.16155 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.642+0000 7f7a897c9740 1 -- 172.21.15.39:0/3957004442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561011dafb70 con 0x5610119ce1b0 2024-07-04T08:18:35.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.642+0000 7f7a6a000640 1 -- 172.21.15.39:0/3957004442 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a74004e50 con 0x5610119ce1b0 2024-07-04T08:18:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.643+0000 7f7a6a000640 1 -- 172.21.15.39:0/3957004442 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7a74028450 con 0x5610119ce1b0 2024-07-04T08:18:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.643+0000 7f7a897c9740 10 client.16155 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.643+0000 7f7a897c9740 10 client.16155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:18:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.643+0000 7f7a897c9740 20 client.16155 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.643+0000 7f7a897c9740 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56101198ada0 0x561011e23e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:35.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.643+0000 7f7a897c9740 4 client.16155 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.643+0000 7f7a897c9740 1 -- 172.21.15.39:0/3957004442 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561011d0f8d0 con 0x56101198ada0 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.643+0000 7f7a73e00640 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56101198ada0 0x561011e23e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a73e00640 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56101198ada0 0x561011e23e00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44fbca36494362f7 server_cookie=eb5f2109376aeb2b in_seq=0 out_seq=0 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a6a000640 10 client.16155 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a6a000640 1 -- 172.21.15.39:0/3957004442 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561011d0f8d0 con 0x56101198ada0 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a6a000640 10 client.16155 handle_command_reply: tid=0 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a897c9740 1 client.16155 shutdown 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a897c9740 1 -- 172.21.15.39:0/3957004442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 msgr2=0x561011db2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a897c9740 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011db2a20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009d60 tx=0x7f7a740096c0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a897c9740 1 -- 172.21.15.39:0/3957004442 shutdown_connections 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a897c9740 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56101198ada0 0x561011e23e00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a897c9740 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5610119f7070 0x561011dae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a897c9740 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5610119d6390 0x561011db2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a897c9740 1 --2- 172.21.15.39:0/3957004442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5610119ce1b0 0x561011db2a20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a6a000640 0 client.16155 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:18:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.644+0000 7f7a897c9740 1 -- 172.21.15.39:0/3957004442 >> 172.21.15.39:0/3957004442 conn(0x5610119eb460 msgr2=0x561011d97e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.645+0000 7f7a897c9740 1 -- 172.21.15.39:0/3957004442 shutdown_connections 2024-07-04T08:18:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.645+0000 7f7a897c9740 1 -- 172.21.15.39:0/3957004442 wait complete. 2024-07-04T08:18:35.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.645+0000 7f7a897c9740 20 client.16155 trim_cache size 0 max 16384 2024-07-04T08:18:35.649 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:35.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 -- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a600b2960 msgr2=0x7f7a600b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:35.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a600b2960 0x7f7a600b4e70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x561011d9a5d0 tx=0x7f7a6c1066a0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 -- 172.21.15.39:0/2780871520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a7c074d00 msgr2=0x7f7a7c19ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:35.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a7c074d00 0x7f7a7c19ba80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c02a920 tx=0x7f7a6c031a20 comp rx=0 tx=0).stop 2024-07-04T08:18:35.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 -- 172.21.15.39:0/2780871520 shutdown_connections 2024-07-04T08:18:35.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f7a600b2960 0x7f7a600b4e70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a7c075650 0x7f7a7c19c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7a7c10f460 0x7f7a7c1a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 --2- 172.21.15.39:0/2780871520 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a7c074d00 0x7f7a7c19ba80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:35.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 -- 172.21.15.39:0/2780871520 >> 172.21.15.39:0/2780871520 conn(0x7f7a7c0fe990 msgr2=0x7f7a7c108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:35.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 -- 172.21.15.39:0/2780871520 shutdown_connections 2024-07-04T08:18:35.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:35.646+0000 7f7a81c00640 1 -- 172.21.15.39:0/2780871520 wait complete. 2024-07-04T08:18:36.584 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:18:37.078 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:37.317 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:37 smithi133 ceph-mon[37050]: pgmap v582: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:37.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:37 smithi039 ceph-mon[32062]: pgmap v582: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:37.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:37 smithi050 ceph-mon[36577]: pgmap v582: 129 pgs: 129 active+clean; 2.9 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:39.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:39 smithi133 ceph-mon[37050]: pgmap v583: 129 pgs: 129 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-07-04T08:18:39.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:39 smithi039 ceph-mon[32062]: pgmap v583: 129 pgs: 129 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-07-04T08:18:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.329+0000 7fba06e00640 1 -- 172.21.15.39:0/3798539863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba001050f0 msgr2=0x7fba00105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.329+0000 7fba06e00640 1 --2- 172.21.15.39:0/3798539863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba001050f0 0x7fba00105500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009900 tx=0x7fb9f002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.329+0000 7fba06e00640 1 -- 172.21.15.39:0/3798539863 shutdown_connections 2024-07-04T08:18:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.329+0000 7fba06e00640 1 --2- 172.21.15.39:0/3798539863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba0010b380 0x7fba00111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.329+0000 7fba06e00640 1 --2- 172.21.15.39:0/3798539863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba0010aa30 0x7fba0010ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.329+0000 7fba06e00640 1 --2- 172.21.15.39:0/3798539863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba001050f0 0x7fba00105500 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.329+0000 7fba06e00640 1 -- 172.21.15.39:0/3798539863 >> 172.21.15.39:0/3798539863 conn(0x7fba00100b60 msgr2=0x7fba00102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.329+0000 7fba06e00640 1 -- 172.21.15.39:0/3798539863 shutdown_connections 2024-07-04T08:18:39.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.329+0000 7fba06e00640 1 -- 172.21.15.39:0/3798539863 wait complete. 2024-07-04T08:18:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.330+0000 7fba06e00640 1 Processor -- start 2024-07-04T08:18:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.331+0000 7fba06e00640 1 -- start start 2024-07-04T08:18:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.331+0000 7fba06e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba001050f0 0x7fba0019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.331+0000 7fba06e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba0010aa30 0x7fba0019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.331+0000 7fba06e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba0010b380 0x7fba001a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.331+0000 7fba06e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fba00115030 con 0x7fba0010b380 2024-07-04T08:18:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.331+0000 7fba06e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fba00114eb0 con 0x7fba0010aa30 2024-07-04T08:18:39.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.331+0000 7fba06e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fba001151b0 con 0x7fba001050f0 2024-07-04T08:18:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba06400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba0010b380 0x7fba001a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba05a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba001050f0 0x7fba0019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba06400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba0010b380 0x7fba001a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45040/0 (socket says 172.21.15.39:45040) 2024-07-04T08:18:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba05a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba001050f0 0x7fba0019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:35620/0 (socket says 172.21.15.39:35620) 2024-07-04T08:18:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba06400640 1 -- 172.21.15.39:0/2164555708 learned_addr learned my addr 172.21.15.39:0/2164555708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba05000640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba0010aa30 0x7fba0019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba05a00640 1 -- 172.21.15.39:0/2164555708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba0010aa30 msgr2=0x7fba0019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba05a00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba0010aa30 0x7fba0019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba05a00640 1 -- 172.21.15.39:0/2164555708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba0010b380 msgr2=0x7fba001a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba05a00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba0010b380 0x7fba001a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.332+0000 7fba05a00640 1 -- 172.21.15.39:0/2164555708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba001a19b0 con 0x7fba001050f0 2024-07-04T08:18:39.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.333+0000 7fba05a00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba001050f0 0x7fba0019ba40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb9f002a910 tx=0x7fb9f0031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.333+0000 7fb9fe000640 1 -- 172.21.15.39:0/2164555708 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f003f030 con 0x7fba001050f0 2024-07-04T08:18:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.333+0000 7fba06e00640 1 -- 172.21.15.39:0/2164555708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba001a1c40 con 0x7fba001050f0 2024-07-04T08:18:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.333+0000 7fba06e00640 1 -- 172.21.15.39:0/2164555708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001a8400 con 0x7fba001050f0 2024-07-04T08:18:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.333+0000 7fb9fe000640 1 -- 172.21.15.39:0/2164555708 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f0004490 con 0x7fba001050f0 2024-07-04T08:18:39.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.333+0000 7fb9fe000640 1 -- 172.21.15.39:0/2164555708 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0049060 con 0x7fba001050f0 2024-07-04T08:18:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.336+0000 7fba0eb23740 1 Processor -- start 2024-07-04T08:18:39.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.337+0000 7fba0eb23740 1 -- start start 2024-07-04T08:18:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.337+0000 7fb9fe000640 1 -- 172.21.15.39:0/2164555708 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb9f0042030 con 0x7fba001050f0 2024-07-04T08:18:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.337+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e57070 0x5619a51fb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.337+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e2e1b0 0x5619a51fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.337+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e36390 0x5619a51fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.337+0000 7fba0eb23740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5619a5171ee0 con 0x5619a4e2e1b0 2024-07-04T08:18:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.337+0000 7fba0eb23740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5619a5161480 con 0x5619a4e36390 2024-07-04T08:18:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.337+0000 7fba0eb23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5619a51654c0 con 0x5619a4e57070 2024-07-04T08:18:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.337+0000 7fba05a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e2e1b0 0x5619a51fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba06400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e57070 0x5619a51fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba06400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e57070 0x5619a51fb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:35630/0 (socket says 172.21.15.39:35630) 2024-07-04T08:18:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba06400640 1 -- 172.21.15.39:0/1791682236 learned_addr learned my addr 172.21.15.39:0/1791682236 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:39.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba05000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e36390 0x5619a51fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba06400640 1 -- 172.21.15.39:0/1791682236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e36390 msgr2=0x5619a51fc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:39.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba06400640 1 --2- 172.21.15.39:0/1791682236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e36390 0x5619a51fc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba06400640 1 -- 172.21.15.39:0/1791682236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e2e1b0 msgr2=0x5619a51fbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:39.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba06400640 1 --2- 172.21.15.39:0/1791682236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e2e1b0 0x5619a51fbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba06400640 1 -- 172.21.15.39:0/1791682236 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a50f9650 con 0x5619a4e57070 2024-07-04T08:18:39.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.338+0000 7fba06400640 1 --2- 172.21.15.39:0/1791682236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e57070 0x5619a51fb950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009d90 tx=0x7fb9f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e99590290e20df9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:39.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.339+0000 7fb9f6a00640 1 -- 172.21.15.39:0/1791682236 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f801e070 con 0x5619a4e57070 2024-07-04T08:18:39.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.339+0000 7fb9f6a00640 1 -- 172.21.15.39:0/1791682236 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f8019ae0 con 0x5619a4e57070 2024-07-04T08:18:39.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.339+0000 7fb9f6a00640 1 -- 172.21.15.39:0/1791682236 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800fb50 con 0x5619a4e57070 2024-07-04T08:18:39.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.339+0000 7fb9fe000640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9e40b29e0 0x7fb9e40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fba05a00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9e40b29e0 0x7fb9e40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fb9fe000640 1 -- 172.21.15.39:0/2164555708 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb9f0102540 con 0x7fba001050f0 2024-07-04T08:18:39.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fba0eb23740 1 -- 172.21.15.39:0/1791682236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e57070 msgr2=0x5619a51fb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:39.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fba0eb23740 1 --2- 172.21.15.39:0/1791682236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e57070 0x5619a51fb950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009d90 tx=0x7fb9f8011810 comp rx=0 tx=0).stop 2024-07-04T08:18:39.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fba0eb23740 1 -- 172.21.15.39:0/1791682236 shutdown_connections 2024-07-04T08:18:39.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fba0eb23740 1 --2- 172.21.15.39:0/1791682236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e2e1b0 0x5619a51fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fba0eb23740 1 --2- 172.21.15.39:0/1791682236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e36390 0x5619a51fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fba0eb23740 1 --2- 172.21.15.39:0/1791682236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e57070 0x5619a51fb950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fba0eb23740 1 -- 172.21.15.39:0/1791682236 >> 172.21.15.39:0/1791682236 conn(0x5619a4e4b460 msgr2=0x5619a51f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:39.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.340+0000 7fba0eb23740 1 -- 172.21.15.39:0/1791682236 shutdown_connections 2024-07-04T08:18:39.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.341+0000 7fba05a00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9e40b29e0 0x7fb9e40b4ef0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00ffa10 tx=0x7fb9f0000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:39.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.341+0000 7fba0eb23740 1 -- 172.21.15.39:0/1791682236 wait complete. 2024-07-04T08:18:39.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.341+0000 7fba0eb23740 1 -- start start 2024-07-04T08:18:39.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.341+0000 7fba0eb23740 1 Processor -- start 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e2e1b0 0x5619a5207550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e36390 0x5619a5207a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e57070 0x5619a5213480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba0eb23740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5619a4fc5700 con 0x5619a4e36390 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba0eb23740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5619a5108bb0 con 0x5619a4e57070 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba0eb23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5619a520a130 con 0x5619a4e2e1b0 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba0eb23740 10 client.0 _ll_register_callbacks cb 0x5619a515a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba05000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e36390 0x5619a5207a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba05000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e36390 0x5619a5207a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45058/0 (socket says 172.21.15.39:45058) 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba05000640 1 -- 172.21.15.39:0/3635017480 learned_addr learned my addr 172.21.15.39:0/3635017480 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:39.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.342+0000 7fba06400640 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e2e1b0 0x5619a5207550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.343+0000 7fba06400640 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e57070 0x5619a5213480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.343+0000 7fba05000640 1 -- 172.21.15.39:0/3635017480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e2e1b0 msgr2=0x5619a5207550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.343+0000 7fba05000640 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e2e1b0 0x5619a5207550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.343+0000 7fba05000640 1 -- 172.21.15.39:0/3635017480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e57070 msgr2=0x5619a5213480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.343+0000 7fba05000640 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e57070 0x5619a5213480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.343+0000 7fba05000640 1 -- 172.21.15.39:0/3635017480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a4ec2650 con 0x5619a4e36390 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.343+0000 7fba05000640 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e36390 0x5619a5207a90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec011d40 tx=0x7fb9ec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc3cc19cc0bb44b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.343+0000 7fb9eb400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.344+0000 7fb9eb400640 1 -- 172.21.15.39:0/3635017480 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00fe60 con 0x5619a4e36390 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.344+0000 7fb9eb400640 1 -- 172.21.15.39:0/3635017480 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb9ec004b20 con 0x5619a4e36390 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.344+0000 7fba0eb23740 1 -- 172.21.15.39:0/3635017480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f8019ae0 con 0x5619a4e36390 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.344+0000 7fb9eb400640 1 -- 172.21.15.39:0/3635017480 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9ec024050 con 0x5619a4e36390 2024-07-04T08:18:39.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.344+0000 7fba0eb23740 10 client.16167 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:39.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.344+0000 7fba0eb23740 1 -- 172.21.15.39:0/3635017480 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5619a5216230 con 0x5619a4e36390 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.344+0000 7fb9eb400640 1 -- 172.21.15.39:0/3635017480 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f8019ae0 con 0x5619a4e36390 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.345+0000 7fb9eb400640 1 -- 172.21.15.39:0/3635017480 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb9ec01b470 con 0x5619a4e36390 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.345+0000 7fba0eb23740 10 client.16167 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.345+0000 7fba0eb23740 10 client.16167 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.345+0000 7fba0eb23740 20 client.16167 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.345+0000 7fba0eb23740 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5619a4e159f0 0x5619a52197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.345+0000 7fba0eb23740 4 client.16167 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.345+0000 7fba0eb23740 1 -- 172.21.15.39:0/3635017480 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619a5173e30 con 0x5619a4e159f0 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.345+0000 7fba06400640 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5619a4e159f0 0x5619a52197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.346+0000 7fba06400640 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5619a4e159f0 0x5619a52197d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c6e2c9f18aee36c server_cookie=69ed587392e060c9 in_seq=0 out_seq=0 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.346+0000 7fb9eb400640 10 client.16167 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fb9eb400640 1 -- 172.21.15.39:0/3635017480 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5619a5173e30 con 0x5619a4e159f0 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fb9eb400640 10 client.16167 handle_command_reply: tid=0 2024-07-04T08:18:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 client.16167 shutdown 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 -- 172.21.15.39:0/3635017480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e36390 msgr2=0x5619a5207a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e36390 0x5619a5207a90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec011d40 tx=0x7fb9ec00f790 comp rx=0 tx=0).stop 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 -- 172.21.15.39:0/3635017480 shutdown_connections 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5619a4e159f0 0x5619a52197d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e36390 0x5619a5207a90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e57070 0x5619a5213480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 --2- 172.21.15.39:0/3635017480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e2e1b0 0x5619a5207550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 -- 172.21.15.39:0/3635017480 >> 172.21.15.39:0/3635017480 conn(0x5619a4e4b460 msgr2=0x5619a51feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fb9eb400640 0 client.16167 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 -- 172.21.15.39:0/3635017480 shutdown_connections 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.347+0000 7fba0eb23740 1 -- 172.21.15.39:0/3635017480 wait complete. 2024-07-04T08:18:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.348+0000 7fba0eb23740 20 client.16167 trim_cache size 0 max 16384 2024-07-04T08:18:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.363+0000 7fba0eb23740 1 Processor -- start 2024-07-04T08:18:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.363+0000 7fba0eb23740 1 -- start start 2024-07-04T08:18:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.363+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 0x5619a51ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.363+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a51fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.363+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a5213270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.363+0000 7fba0eb23740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e000fdc0 con 0x5619a4e57070 2024-07-04T08:18:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.363+0000 7fba0eb23740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0011900 con 0x5619a4e2e1b0 2024-07-04T08:18:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.363+0000 7fba0eb23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0010490 con 0x5619a4e36390 2024-07-04T08:18:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba06400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a5213270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba05000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a51fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba06400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a5213270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45072/0 (socket says 172.21.15.39:45072) 2024-07-04T08:18:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba05000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a51fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:35650/0 (socket says 172.21.15.39:35650) 2024-07-04T08:18:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba06400640 1 -- 172.21.15.39:0/2394734562 learned_addr learned my addr 172.21.15.39:0/2394734562 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba06400640 1 --2- 172.21.15.39:0/2394734562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 0x5619a51ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba06400640 1 -- 172.21.15.39:0/2394734562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 msgr2=0x5619a51fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba06400640 1 --2- 172.21.15.39:0/2394734562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a51fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba06400640 1 -- 172.21.15.39:0/2394734562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 msgr2=0x5619a51ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba06400640 1 --2- 172.21.15.39:0/2394734562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 0x5619a51ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba06400640 1 -- 172.21.15.39:0/2394734562 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a51fa130 con 0x5619a4e57070 2024-07-04T08:18:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.364+0000 7fba05000640 1 --2- 172.21.15.39:0/2394734562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a51fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.365+0000 7fba06400640 1 --2- 172.21.15.39:0/2394734562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a5213270 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800f7f0 tx=0x7fb9f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a371780c4f5316d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.365+0000 7fb9f6000640 1 -- 172.21.15.39:0/2394734562 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f801e040 con 0x5619a4e57070 2024-07-04T08:18:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.365+0000 7fb9f6000640 1 -- 172.21.15.39:0/2394734562 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f801f040 con 0x5619a4e57070 2024-07-04T08:18:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.365+0000 7fb9f6000640 1 -- 172.21.15.39:0/2394734562 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800fe10 con 0x5619a4e57070 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.366+0000 7fba0eb23740 1 -- 172.21.15.39:0/2394734562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 msgr2=0x5619a5213270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.366+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2394734562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a5213270 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800f7f0 tx=0x7fb9f8011810 comp rx=0 tx=0).stop 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.366+0000 7fba0eb23740 1 -- 172.21.15.39:0/2394734562 shutdown_connections 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.366+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2394734562 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a5213270 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.366+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2394734562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a51fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.366+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2394734562 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 0x5619a51ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.366+0000 7fba0eb23740 1 -- 172.21.15.39:0/2394734562 >> 172.21.15.39:0/2394734562 conn(0x5619a4e4b460 msgr2=0x5619a51fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.366+0000 7fba0eb23740 1 -- 172.21.15.39:0/2394734562 shutdown_connections 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.366+0000 7fba0eb23740 1 -- 172.21.15.39:0/2394734562 wait complete. 2024-07-04T08:18:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.367+0000 7fba0eb23740 1 -- start start 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.367+0000 7fba0eb23740 1 Processor -- start 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 0x5619a520f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a520fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba0eb23740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a520ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba0eb23740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0011230 con 0x5619a4e57070 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba0eb23740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0010b60 con 0x5619a4e2e1b0 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba0eb23740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e000dda0 con 0x5619a4e36390 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba0eb23740 10 client.0 _ll_register_callbacks cb 0x7fb9ec009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba06400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 0x5619a520f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba06400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a520ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba06400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 0x5619a520f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51836/0 (socket says 172.21.15.39:51836) 2024-07-04T08:18:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba06400640 1 -- 172.21.15.39:0/2056241250 learned_addr learned my addr 172.21.15.39:0/2056241250 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.368+0000 7fba05000640 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a520fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fba05000640 1 -- 172.21.15.39:0/2056241250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 msgr2=0x5619a520f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:39.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fba05000640 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 0x5619a520f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fba05000640 1 -- 172.21.15.39:0/2056241250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 msgr2=0x5619a520ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:39.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fba05000640 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a520ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fba05000640 1 -- 172.21.15.39:0/2056241250 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a520b4e0 con 0x5619a4e36390 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fba06400640 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a520ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fba05000640 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a520fcd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec006010 tx=0x7fb9ec00fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2fd40c4b55d0145 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fb9eb400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fb9eb400640 1 -- 172.21.15.39:0/2056241250 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec01ba80 con 0x5619a4e36390 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fba0eb23740 1 -- 172.21.15.39:0/2056241250 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f801f040 con 0x5619a4e36390 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.369+0000 7fb9eb400640 1 -- 172.21.15.39:0/2056241250 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb9ec017070 con 0x5619a4e36390 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fb9eb400640 1 -- 172.21.15.39:0/2056241250 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9ec039050 con 0x5619a4e36390 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fba0eb23740 10 client.26248 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fba0eb23740 1 -- 172.21.15.39:0/2056241250 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5619a5281b40 con 0x5619a4e36390 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fb9eb400640 1 -- 172.21.15.39:0/2056241250 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f801f040 con 0x5619a4e36390 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fb9eb400640 1 -- 172.21.15.39:0/2056241250 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb9ec031800 con 0x5619a4e36390 2024-07-04T08:18:39.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fba0eb23740 10 client.26248 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fba0eb23740 10 client.26248 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fba0eb23740 20 client.26248 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5619a4deada0 0x5619a5284330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fba0eb23740 4 client.26248 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.370+0000 7fba0eb23740 1 -- 172.21.15.39:0/2056241250 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619a516f8d0 con 0x5619a4deada0 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.371+0000 7fba06400640 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5619a4deada0 0x5619a5284330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.371+0000 7fba06400640 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5619a4deada0 0x5619a5284330 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=186c1cce61fedd4 server_cookie=8e0346b9ba25bb55 in_seq=0 out_seq=0 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.371+0000 7fb9eb400640 10 client.26248 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fb9eb400640 1 -- 172.21.15.39:0/2056241250 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619a516f8d0 con 0x5619a4deada0 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fb9eb400640 10 client.26248 handle_command_reply: tid=0 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 client.26248 shutdown 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 -- 172.21.15.39:0/2056241250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 msgr2=0x5619a520fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a520fcd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec006010 tx=0x7fb9ec00fc90 comp rx=0 tx=0).stop 2024-07-04T08:18:39.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 -- 172.21.15.39:0/2056241250 shutdown_connections 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5619a4deada0 0x5619a5284330 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5619a4e57070 0x5619a520ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619a4e36390 0x5619a520fcd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 --2- 172.21.15.39:0/2056241250 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5619a4e2e1b0 0x5619a520f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fb9eb400640 0 client.26248 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 -- 172.21.15.39:0/2056241250 >> 172.21.15.39:0/2056241250 conn(0x5619a4e4b460 msgr2=0x5619a51ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 -- 172.21.15.39:0/2056241250 shutdown_connections 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 1 -- 172.21.15.39:0/2056241250 wait complete. 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.372+0000 7fba0eb23740 20 client.26248 trim_cache size 0 max 16384 2024-07-04T08:18:39.376 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:39.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 -- 172.21.15.39:0/2164555708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9e40b29e0 msgr2=0x7fb9e40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:39.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9e40b29e0 0x7fb9e40b4ef0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00ffa10 tx=0x7fb9f0000bc0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 -- 172.21.15.39:0/2164555708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba001050f0 msgr2=0x7fba0019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:39.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba001050f0 0x7fba0019ba40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb9f002a910 tx=0x7fb9f0031a20 comp rx=0 tx=0).stop 2024-07-04T08:18:39.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 -- 172.21.15.39:0/2164555708 shutdown_connections 2024-07-04T08:18:39.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9e40b29e0 0x7fb9e40b4ef0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba0010b380 0x7fba001a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba0010aa30 0x7fba0019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 --2- 172.21.15.39:0/2164555708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba001050f0 0x7fba0019ba40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 -- 172.21.15.39:0/2164555708 >> 172.21.15.39:0/2164555708 conn(0x7fba00100b60 msgr2=0x7fba00102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 -- 172.21.15.39:0/2164555708 shutdown_connections 2024-07-04T08:18:39.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:39.373+0000 7fba06e00640 1 -- 172.21.15.39:0/2164555708 wait complete. 2024-07-04T08:18:39.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:39 smithi050 ceph-mon[36577]: pgmap v583: 129 pgs: 129 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-07-04T08:18:40.228 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:18:40.730 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:41.235 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:41 smithi039 ceph-mon[32062]: pgmap v584: 129 pgs: 129 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-07-04T08:18:41.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:41 smithi133 ceph-mon[37050]: pgmap v584: 129 pgs: 129 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-07-04T08:18:41.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:41 smithi050 ceph-mon[36577]: pgmap v584: 129 pgs: 129 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-07-04T08:18:43.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.168+0000 7f4a52600640 1 -- 172.21.15.39:0/4179288775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a4c10ee50 msgr2=0x7f4a4c10f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:43.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.168+0000 7f4a52600640 1 --2- 172.21.15.39:0/4179288775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a4c10ee50 0x7f4a4c10f260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009900 tx=0x7f4a3c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.168+0000 7f4a52600640 1 -- 172.21.15.39:0/4179288775 shutdown_connections 2024-07-04T08:18:43.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.168+0000 7f4a52600640 1 --2- 172.21.15.39:0/4179288775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a4c10ee50 0x7f4a4c10f260 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.168+0000 7f4a52600640 1 --2- 172.21.15.39:0/4179288775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c10f7a0 0x7f4a4c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.168+0000 7f4a52600640 1 --2- 172.21.15.39:0/4179288775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c109510 0x7f4a4c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.168+0000 7f4a52600640 1 -- 172.21.15.39:0/4179288775 >> 172.21.15.39:0/4179288775 conn(0x7f4a4c078b10 msgr2=0x7f4a4c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:43.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.168+0000 7f4a52600640 1 -- 172.21.15.39:0/4179288775 shutdown_connections 2024-07-04T08:18:43.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.169+0000 7f4a52600640 1 -- 172.21.15.39:0/4179288775 wait complete. 2024-07-04T08:18:43.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.170+0000 7f4a52600640 1 Processor -- start 2024-07-04T08:18:43.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.170+0000 7f4a52600640 1 -- start start 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.170+0000 7f4a52600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c109510 0x7f4a4c19fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.170+0000 7f4a52600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a4c10ee50 0x7f4a4c1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.170+0000 7f4a52600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c10f7a0 0x7f4a4c1a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.170+0000 7f4a52600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c119460 con 0x7f4a4c10ee50 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.170+0000 7f4a52600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1192e0 con 0x7f4a4c10f7a0 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.170+0000 7f4a52600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1195e0 con 0x7f4a4c109510 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a51200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c109510 0x7f4a4c19fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a4be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a4c10ee50 0x7f4a4c1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a51200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c109510 0x7f4a4c19fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:46918/0 (socket says 172.21.15.39:46918) 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a51200640 1 -- 172.21.15.39:0/2342310830 learned_addr learned my addr 172.21.15.39:0/2342310830 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:43.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a51c00640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c10f7a0 0x7f4a4c1a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a51200640 1 -- 172.21.15.39:0/2342310830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c10f7a0 msgr2=0x7f4a4c1a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:43.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a51200640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c10f7a0 0x7f4a4c1a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a51200640 1 -- 172.21.15.39:0/2342310830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a4c10ee50 msgr2=0x7f4a4c1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:43.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a51200640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a4c10ee50 0x7f4a4c1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.171+0000 7f4a51200640 1 -- 172.21.15.39:0/2342310830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c1a5de0 con 0x7f4a4c109510 2024-07-04T08:18:43.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.172+0000 7f4a51200640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c109510 0x7f4a4c19fe70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000e9f0 tx=0x7f4a4000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:43.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.172+0000 7f4a49600640 1 -- 172.21.15.39:0/2342310830 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000cd80 con 0x7f4a4c109510 2024-07-04T08:18:43.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.172+0000 7f4a49600640 1 -- 172.21.15.39:0/2342310830 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a400108e0 con 0x7f4a4c109510 2024-07-04T08:18:43.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.172+0000 7f4a52600640 1 -- 172.21.15.39:0/2342310830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c1a60d0 con 0x7f4a4c109510 2024-07-04T08:18:43.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.172+0000 7f4a49600640 1 -- 172.21.15.39:0/2342310830 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40018980 con 0x7f4a4c109510 2024-07-04T08:18:43.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.172+0000 7f4a52600640 1 -- 172.21.15.39:0/2342310830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c1ac830 con 0x7f4a4c109510 2024-07-04T08:18:43.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.175+0000 7f4a5a14c740 1 Processor -- start 2024-07-04T08:18:43.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.176+0000 7f4a5a14c740 1 -- start start 2024-07-04T08:18:43.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.176+0000 7f4a49600640 1 -- 172.21.15.39:0/2342310830 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4a40010430 con 0x7f4a4c109510 2024-07-04T08:18:43.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.176+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.176+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.176+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.176+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4447ee0 con 0x55cfb41041b0 2024-07-04T08:18:43.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.176+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cfb4437480 con 0x55cfb410c390 2024-07-04T08:18:43.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.176+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cfb443b4c0 con 0x55cfb412d070 2024-07-04T08:18:43.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.176+0000 7f4a51c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a51200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a51c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:46928/0 (socket says 172.21.15.39:46928) 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a51c00640 1 -- 172.21.15.39:0/745388073 learned_addr learned my addr 172.21.15.39:0/745388073 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a4be00640 1 --2- 172.21.15.39:0/745388073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a51c00640 1 -- 172.21.15.39:0/745388073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 msgr2=0x55cfb44d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a51c00640 1 --2- 172.21.15.39:0/745388073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a51c00640 1 -- 172.21.15.39:0/745388073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 msgr2=0x55cfb44d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a51c00640 1 --2- 172.21.15.39:0/745388073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44d1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a51c00640 1 -- 172.21.15.39:0/745388073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb43cf650 con 0x55cfb412d070 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.177+0000 7f4a4be00640 1 --2- 172.21.15.39:0/745388073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.178+0000 7f4a51c00640 1 --2- 172.21.15.39:0/745388073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44d1950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009fd0 tx=0x7f4a440118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcfdffe75b62e297 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.178+0000 7f4a49600640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4a340b29e0 0x7f4a340b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.178+0000 7f4a3a000640 1 -- 172.21.15.39:0/745388073 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44004d20 con 0x55cfb412d070 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.178+0000 7f4a3a000640 1 -- 172.21.15.39:0/745388073 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a44004050 con 0x55cfb412d070 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.178+0000 7f4a3a000640 1 -- 172.21.15.39:0/745388073 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44005260 con 0x55cfb412d070 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.178+0000 7f4a49600640 1 -- 172.21.15.39:0/2342310830 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4a400e2c70 con 0x7f4a4c109510 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.178+0000 7f4a51200640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4a340b29e0 0x7f4a340b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a5a14c740 1 -- 172.21.15.39:0/745388073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 msgr2=0x55cfb44d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/745388073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44d1950 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009fd0 tx=0x7f4a440118d0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a51200640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4a340b29e0 0x7f4a340b4ef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55cfb44d05d0 tx=0x7f4a400e6720 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a5a14c740 1 -- 172.21.15.39:0/745388073 shutdown_connections 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/745388073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/745388073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/745388073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44d1950 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a5a14c740 1 -- 172.21.15.39:0/745388073 >> 172.21.15.39:0/745388073 conn(0x55cfb4121460 msgr2=0x55cfb44cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a5a14c740 1 -- 172.21.15.39:0/745388073 shutdown_connections 2024-07-04T08:18:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.179+0000 7f4a5a14c740 1 -- 172.21.15.39:0/745388073 wait complete. 2024-07-04T08:18:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.180+0000 7f4a5a14c740 1 -- start start 2024-07-04T08:18:43.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.180+0000 7f4a5a14c740 1 Processor -- start 2024-07-04T08:18:43.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb410c390 0x55cfb44dd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb412d070 0x55cfb44e9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cfb429b700 con 0x55cfb41041b0 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cfb43debb0 con 0x55cfb412d070 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cfb44e0130 con 0x55cfb410c390 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a5a14c740 10 client.0 _ll_register_callbacks cb 0x55cfb4430570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a51c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a51c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43906/0 (socket says 172.21.15.39:43906) 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a51c00640 1 -- 172.21.15.39:0/1129292966 learned_addr learned my addr 172.21.15.39:0/1129292966 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a4be00640 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb410c390 0x55cfb44dd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.181+0000 7f4a51c00640 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb412d070 0x55cfb44e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.182+0000 7f4a51c00640 1 -- 172.21.15.39:0/1129292966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb410c390 msgr2=0x55cfb44dd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.182+0000 7f4a51c00640 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb410c390 0x55cfb44dd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.182+0000 7f4a51c00640 1 -- 172.21.15.39:0/1129292966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb412d070 msgr2=0x55cfb44e9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.182+0000 7f4a51c00640 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb412d070 0x55cfb44e9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.182+0000 7f4a51c00640 1 -- 172.21.15.39:0/1129292966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb4198650 con 0x55cfb41041b0 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.182+0000 7f4a51c00640 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dd4b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4a44011dd0 tx=0x7f4a44009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404dbb5fdd5b50a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.182+0000 7f4a2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.182+0000 7f4a2ea00640 1 -- 172.21.15.39:0/1129292966 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44004490 con 0x55cfb41041b0 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.183+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1129292966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a44004050 con 0x55cfb41041b0 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.183+0000 7f4a2ea00640 1 -- 172.21.15.39:0/1129292966 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4a44004630 con 0x55cfb41041b0 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.183+0000 7f4a2ea00640 1 -- 172.21.15.39:0/1129292966 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a44034050 con 0x55cfb41041b0 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.183+0000 7f4a5a14c740 10 client.16185 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:43.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.183+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1129292966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfb44ec0f0 con 0x55cfb41041b0 2024-07-04T08:18:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.183+0000 7f4a2ea00640 1 -- 172.21.15.39:0/1129292966 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a44004050 con 0x55cfb41041b0 2024-07-04T08:18:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.183+0000 7f4a2ea00640 1 -- 172.21.15.39:0/1129292966 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4a4400f550 con 0x55cfb41041b0 2024-07-04T08:18:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.184+0000 7f4a5a14c740 10 client.16185 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.184+0000 7f4a5a14c740 10 client.16185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:18:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.184+0000 7f4a5a14c740 20 client.16185 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.184+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55cfb40eb9f0 0x55cfb44ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.184+0000 7f4a5a14c740 4 client.16185 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.184+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1129292966 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfb4449e30 con 0x55cfb40eb9f0 2024-07-04T08:18:43.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.184+0000 7f4a4be00640 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55cfb40eb9f0 0x55cfb44ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.185+0000 7f4a4be00640 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55cfb40eb9f0 0x55cfb44ef4b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58e1c450b8b9942f server_cookie=8dd4785b3903b854 in_seq=0 out_seq=0 2024-07-04T08:18:43.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.185+0000 7f4a2ea00640 10 client.16185 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:18:43.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a2ea00640 1 -- 172.21.15.39:0/1129292966 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cfb4449e30 con 0x55cfb40eb9f0 2024-07-04T08:18:43.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a2ea00640 10 client.16185 handle_command_reply: tid=0 2024-07-04T08:18:43.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 client.16185 shutdown 2024-07-04T08:18:43.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1129292966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 msgr2=0x55cfb44dd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dd4b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4a44011dd0 tx=0x7f4a44009cc0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1129292966 shutdown_connections 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55cfb40eb9f0 0x55cfb44ef4b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb412d070 0x55cfb44e9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb410c390 0x55cfb44dd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1129292966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dd4b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1129292966 >> 172.21.15.39:0/1129292966 conn(0x55cfb4121460 msgr2=0x55cfb44d4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a2ea00640 0 client.16185 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1129292966 shutdown_connections 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.186+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1129292966 wait complete. 2024-07-04T08:18:43.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.187+0000 7f4a5a14c740 20 client.16185 trim_cache size 0 max 16384 2024-07-04T08:18:43.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.202+0000 7f4a5a14c740 1 Processor -- start 2024-07-04T08:18:43.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.202+0000 7f4a5a14c740 1 -- start start 2024-07-04T08:18:43.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:43 smithi039 ceph-mon[32062]: pgmap v585: 129 pgs: 129 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44e9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b6760 con 0x55cfb41041b0 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b82a0 con 0x55cfb410c390 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b6e30 con 0x55cfb412d070 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a4be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a51c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a4be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58234/0 (socket says 172.21.15.39:58234) 2024-07-04T08:18:43.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a4be00640 1 -- 172.21.15.39:0/1393691323 learned_addr learned my addr 172.21.15.39:0/1393691323 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:43.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.203+0000 7f4a51c00640 1 --2- 172.21.15.39:0/1393691323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44e9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.204+0000 7f4a4be00640 1 -- 172.21.15.39:0/1393691323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 msgr2=0x55cfb44e9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:43.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.204+0000 7f4a4be00640 1 --2- 172.21.15.39:0/1393691323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44e9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.204+0000 7f4a4be00640 1 -- 172.21.15.39:0/1393691323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 msgr2=0x55cfb44dc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:43.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.204+0000 7f4a4be00640 1 --2- 172.21.15.39:0/1393691323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.204+0000 7f4a4be00640 1 -- 172.21.15.39:0/1393691323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb44de570 con 0x55cfb410c390 2024-07-04T08:18:43.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.204+0000 7f4a4be00640 1 --2- 172.21.15.39:0/1393691323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44ce520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c005f70 tx=0x7f4a3c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f834473e817158 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:43.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.204+0000 7f4a39600640 1 -- 172.21.15.39:0/1393691323 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c035040 con 0x55cfb410c390 2024-07-04T08:18:43.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.204+0000 7f4a39600640 1 -- 172.21.15.39:0/1393691323 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a3c04b070 con 0x55cfb410c390 2024-07-04T08:18:43.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.205+0000 7f4a39600640 1 -- 172.21.15.39:0/1393691323 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0035f0 con 0x55cfb410c390 2024-07-04T08:18:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.205+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1393691323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 msgr2=0x55cfb44ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.205+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1393691323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44ce520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c005f70 tx=0x7f4a3c005a20 comp rx=0 tx=0).stop 2024-07-04T08:18:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.206+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1393691323 shutdown_connections 2024-07-04T08:18:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.206+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1393691323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb412d070 0x55cfb44e9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.206+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1393691323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44ce520 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.206+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/1393691323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb41041b0 0x55cfb44dc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.206+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1393691323 >> 172.21.15.39:0/1393691323 conn(0x55cfb4121460 msgr2=0x55cfb44cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:43.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.206+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1393691323 shutdown_connections 2024-07-04T08:18:43.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.206+0000 7f4a5a14c740 1 -- 172.21.15.39:0/1393691323 wait complete. 2024-07-04T08:18:43.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.206+0000 7f4a5a14c740 1 -- start start 2024-07-04T08:18:43.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.207+0000 7f4a5a14c740 1 Processor -- start 2024-07-04T08:18:43.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.207+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb41041b0 0x55cfb44e2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.207+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.207+0000 7f4a5a14c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb412d070 0x55cfb44e50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.207+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b7bd0 con 0x55cfb412d070 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.207+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b7500 con 0x55cfb410c390 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.207+0000 7f4a5a14c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c1b4740 con 0x55cfb41041b0 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.207+0000 7f4a5a14c740 10 client.0 _ll_register_callbacks cb 0x7f4a400e65f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a51c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb412d070 0x55cfb44e50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a4be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a4be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44e2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58250/0 (socket says 172.21.15.39:58250) 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a4be00640 1 -- 172.21.15.39:0/4008492706 learned_addr learned my addr 172.21.15.39:0/4008492706 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a51c00640 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb41041b0 0x55cfb44e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a4be00640 1 -- 172.21.15.39:0/4008492706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb41041b0 msgr2=0x55cfb44e2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:43.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a4be00640 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb41041b0 0x55cfb44e2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a4be00640 1 -- 172.21.15.39:0/4008492706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb412d070 msgr2=0x55cfb44e50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:43.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a4be00640 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb412d070 0x55cfb44e50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a4be00640 1 -- 172.21.15.39:0/4008492706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb44e5730 con 0x55cfb410c390 2024-07-04T08:18:43.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a51c00640 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb41041b0 0x55cfb44e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:43.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a4be00640 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44e2760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c02a910 tx=0x7f4a3c004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0affdf7a1eacc8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:43.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.208+0000 7f4a2ea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:43.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.209+0000 7f4a2ea00640 1 -- 172.21.15.39:0/4008492706 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c035050 con 0x55cfb410c390 2024-07-04T08:18:43.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.209+0000 7f4a5a14c740 1 -- 172.21.15.39:0/4008492706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a3c04b070 con 0x55cfb410c390 2024-07-04T08:18:43.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.209+0000 7f4a2ea00640 1 -- 172.21.15.39:0/4008492706 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4a3c005040 con 0x55cfb410c390 2024-07-04T08:18:43.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.209+0000 7f4a2ea00640 1 -- 172.21.15.39:0/4008492706 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a3c05e050 con 0x55cfb410c390 2024-07-04T08:18:43.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.209+0000 7f4a5a14c740 10 client.26321 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:43.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.209+0000 7f4a5a14c740 1 -- 172.21.15.39:0/4008492706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfb4557550 con 0x55cfb410c390 2024-07-04T08:18:43.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.210+0000 7f4a2ea00640 1 -- 172.21.15.39:0/4008492706 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a3c04b070 con 0x55cfb410c390 2024-07-04T08:18:43.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.210+0000 7f4a2ea00640 1 -- 172.21.15.39:0/4008492706 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4a3c003c40 con 0x55cfb410c390 2024-07-04T08:18:43.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.210+0000 7f4a5a14c740 10 client.26321 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:43.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.210+0000 7f4a5a14c740 10 client.26321 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.210+0000 7f4a5a14c740 20 client.26321 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.210+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55cfb40c0da0 0x55cfb4559c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.210+0000 7f4a5a14c740 4 client.26321 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.210+0000 7f4a5a14c740 1 -- 172.21.15.39:0/4008492706 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfb44d8e60 con 0x55cfb40c0da0 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.210+0000 7f4a51c00640 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55cfb40c0da0 0x55cfb4559c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.211+0000 7f4a51c00640 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55cfb40c0da0 0x55cfb4559c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63f9d6373ed7bfb1 server_cookie=20b1e5ab7bf38943 in_seq=0 out_seq=0 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.211+0000 7f4a2ea00640 10 client.26321 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.211+0000 7f4a2ea00640 1 -- 172.21.15.39:0/4008492706 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfb44d8e60 con 0x55cfb40c0da0 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.211+0000 7f4a2ea00640 10 client.26321 handle_command_reply: tid=0 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.211+0000 7f4a5a14c740 1 client.26321 shutdown 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 -- 172.21.15.39:0/4008492706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 msgr2=0x55cfb44e2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44e2760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c02a910 tx=0x7f4a3c004380 comp rx=0 tx=0).stop 2024-07-04T08:18:43.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 -- 172.21.15.39:0/4008492706 shutdown_connections 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55cfb40c0da0 0x55cfb4559c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a2ea00640 0 client.26321 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cfb412d070 0x55cfb44e50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cfb410c390 0x55cfb44e2760 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 --2- 172.21.15.39:0/4008492706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cfb41041b0 0x55cfb44e2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 -- 172.21.15.39:0/4008492706 >> 172.21.15.39:0/4008492706 conn(0x55cfb4121460 msgr2=0x55cfb44de740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 -- 172.21.15.39:0/4008492706 shutdown_connections 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 1 -- 172.21.15.39:0/4008492706 wait complete. 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.212+0000 7f4a5a14c740 20 client.26321 trim_cache size 0 max 16384 2024-07-04T08:18:43.215 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:43.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.213+0000 7f4a52600640 1 -- 172.21.15.39:0/2342310830 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4a340b29e0 msgr2=0x7f4a340b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:43.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.213+0000 7f4a52600640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4a340b29e0 0x7f4a340b4ef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55cfb44d05d0 tx=0x7f4a400e6720 comp rx=0 tx=0).stop 2024-07-04T08:18:43.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.213+0000 7f4a52600640 1 -- 172.21.15.39:0/2342310830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c109510 msgr2=0x7f4a4c19fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:43.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.213+0000 7f4a52600640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c109510 0x7f4a4c19fe70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000e9f0 tx=0x7f4a4000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.214+0000 7f4a52600640 1 -- 172.21.15.39:0/2342310830 shutdown_connections 2024-07-04T08:18:43.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.214+0000 7f4a52600640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4a340b29e0 0x7f4a340b4ef0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.214+0000 7f4a52600640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a4c10ee50 0x7f4a4c1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.214+0000 7f4a52600640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4a4c10f7a0 0x7f4a4c1a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.214+0000 7f4a52600640 1 --2- 172.21.15.39:0/2342310830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c109510 0x7f4a4c19fe70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:43.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.214+0000 7f4a52600640 1 -- 172.21.15.39:0/2342310830 >> 172.21.15.39:0/2342310830 conn(0x7f4a4c078b10 msgr2=0x7f4a4c0792e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:43.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.214+0000 7f4a52600640 1 -- 172.21.15.39:0/2342310830 shutdown_connections 2024-07-04T08:18:43.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:43.214+0000 7f4a52600640 1 -- 172.21.15.39:0/2342310830 wait complete. 2024-07-04T08:18:43.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:43 smithi133 ceph-mon[37050]: pgmap v585: 129 pgs: 129 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:43.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:43 smithi050 ceph-mon[36577]: pgmap v585: 129 pgs: 129 active+clean; 3.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:44.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:18:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:44.737 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:44 smithi039 ceph-mon[32062]: pgmap v586: 129 pgs: 129 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:44.815 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:44 smithi133 ceph-mon[37050]: pgmap v586: 129 pgs: 129 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:44.893 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:44 smithi050 ceph-mon[36577]: pgmap v586: 129 pgs: 129 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.966+0000 7f334be00640 1 -- 172.21.15.39:0/1272855498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c109190 msgr2=0x7f334c10f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.966+0000 7f334be00640 1 --2- 172.21.15.39:0/1272855498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c109190 0x7f334c10f9c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3344009900 tx=0x7f334402a410 comp rx=0 tx=0).stop 2024-07-04T08:18:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.966+0000 7f334be00640 1 -- 172.21.15.39:0/1272855498 shutdown_connections 2024-07-04T08:18:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.966+0000 7f334be00640 1 --2- 172.21.15.39:0/1272855498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c109190 0x7f334c10f9c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.966+0000 7f334be00640 1 --2- 172.21.15.39:0/1272855498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c108840 0x7f334c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.966+0000 7f334be00640 1 --2- 172.21.15.39:0/1272855498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f334c102f00 0x7f334c103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.966+0000 7f334be00640 1 -- 172.21.15.39:0/1272855498 >> 172.21.15.39:0/1272855498 conn(0x7f334c0fe990 msgr2=0x7f334c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.966+0000 7f334be00640 1 -- 172.21.15.39:0/1272855498 shutdown_connections 2024-07-04T08:18:46.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.966+0000 7f334be00640 1 -- 172.21.15.39:0/1272855498 wait complete. 2024-07-04T08:18:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.967+0000 7f334be00640 1 Processor -- start 2024-07-04T08:18:46.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.968+0000 7f334be00640 1 -- start start 2024-07-04T08:18:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.968+0000 7f334be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f334c102f00 0x7f334c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.968+0000 7f334be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c108840 0x7f334c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.968+0000 7f334be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c109190 0x7f334c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.968+0000 7f334be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f334c112ea0 con 0x7f334c102f00 2024-07-04T08:18:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.968+0000 7f334be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f334c112d20 con 0x7f334c108840 2024-07-04T08:18:46.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.968+0000 7f334be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f334c113020 con 0x7f334c109190 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f334c102f00 0x7f334c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c109190 0x7f334c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c108840 0x7f334c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c108840 0x7f334c199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58274/0 (socket says 172.21.15.39:58274) 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334a000640 1 -- 172.21.15.39:0/2357394729 learned_addr learned my addr 172.21.15.39:0/2357394729 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c109190 0x7f334c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:46968/0 (socket says 172.21.15.39:46968) 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334a000640 1 -- 172.21.15.39:0/2357394729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c109190 msgr2=0x7f334c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334a000640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c109190 0x7f334c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334a000640 1 -- 172.21.15.39:0/2357394729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f334c102f00 msgr2=0x7f334c199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:46.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334a000640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f334c102f00 0x7f334c199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.969+0000 7f334a000640 1 -- 172.21.15.39:0/2357394729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c19f7a0 con 0x7f334c108840 2024-07-04T08:18:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.970+0000 7f334a000640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c108840 0x7f334c199dd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f333400e9c0 tx=0x7f333400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.970+0000 7f3343400640 1 -- 172.21.15.39:0/2357394729 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333400ccc0 con 0x7f334c108840 2024-07-04T08:18:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.970+0000 7f3343400640 1 -- 172.21.15.39:0/2357394729 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33340054d0 con 0x7f334c108840 2024-07-04T08:18:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.970+0000 7f334be00640 1 -- 172.21.15.39:0/2357394729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c19fa90 con 0x7f334c108840 2024-07-04T08:18:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.970+0000 7f3343400640 1 -- 172.21.15.39:0/2357394729 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334010430 con 0x7f334c108840 2024-07-04T08:18:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.970+0000 7f334be00640 1 -- 172.21.15.39:0/2357394729 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c1a61f0 con 0x7f334c108840 2024-07-04T08:18:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.974+0000 7f3357d9d740 1 Processor -- start 2024-07-04T08:18:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.974+0000 7f3357d9d740 1 -- start start 2024-07-04T08:18:46.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.974+0000 7f3343400640 1 -- 172.21.15.39:0/2357394729 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3334005020 con 0x7f334c108840 2024-07-04T08:18:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.974+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a6a7070 0x556e6aa4b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.974+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a67e1b0 0x556e6aa4beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.974+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 0x556e6aa4c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.974+0000 7f3357d9d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556e6a9c1f00 con 0x556e6a686390 2024-07-04T08:18:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.974+0000 7f3357d9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556e6a9b14a0 con 0x556e6a6a7070 2024-07-04T08:18:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.974+0000 7f3357d9d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556e6a9b54e0 con 0x556e6a67e1b0 2024-07-04T08:18:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a67e1b0 0x556e6aa4beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a6a7070 0x556e6aa4b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a67e1b0 0x556e6aa4beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:46970/0 (socket says 172.21.15.39:46970) 2024-07-04T08:18:46.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334aa00640 1 -- 172.21.15.39:0/902653585 learned_addr learned my addr 172.21.15.39:0/902653585 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334a000640 1 --2- 172.21.15.39:0/902653585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 0x556e6aa4c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334b400640 1 -- 172.21.15.39:0/902653585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a67e1b0 msgr2=0x556e6aa4beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334b400640 1 --2- 172.21.15.39:0/902653585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a67e1b0 0x556e6aa4beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334b400640 1 -- 172.21.15.39:0/902653585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 msgr2=0x556e6aa4c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334b400640 1 --2- 172.21.15.39:0/902653585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 0x556e6aa4c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.975+0000 7f334b400640 1 -- 172.21.15.39:0/902653585 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6a949670 con 0x556e6a6a7070 2024-07-04T08:18:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.976+0000 7f334b400640 1 --2- 172.21.15.39:0/902653585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a6a7070 0x556e6aa4b970 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f334402a930 tx=0x7f3344005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cfba61866bd4e6f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.976+0000 7f3343400640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33300b2960 0x7f33300b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.976+0000 7f333be00640 1 -- 172.21.15.39:0/902653585 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344040070 con 0x556e6a6a7070 2024-07-04T08:18:46.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.976+0000 7f333be00640 1 -- 172.21.15.39:0/902653585 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33440044c0 con 0x556e6a6a7070 2024-07-04T08:18:46.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.976+0000 7f333be00640 1 -- 172.21.15.39:0/902653585 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344035040 con 0x556e6a6a7070 2024-07-04T08:18:46.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.976+0000 7f3343400640 1 -- 172.21.15.39:0/2357394729 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f33340e2c60 con 0x7f334c108840 2024-07-04T08:18:46.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.976+0000 7f334aa00640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33300b2960 0x7f33300b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f3357d9d740 1 -- 172.21.15.39:0/902653585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a6a7070 msgr2=0x556e6aa4b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f3357d9d740 1 --2- 172.21.15.39:0/902653585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a6a7070 0x556e6aa4b970 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f334402a930 tx=0x7f3344005f30 comp rx=0 tx=0).stop 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f3357d9d740 1 -- 172.21.15.39:0/902653585 shutdown_connections 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f3357d9d740 1 --2- 172.21.15.39:0/902653585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 0x556e6aa4c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f3357d9d740 1 --2- 172.21.15.39:0/902653585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a67e1b0 0x556e6aa4beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f3357d9d740 1 --2- 172.21.15.39:0/902653585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a6a7070 0x556e6aa4b970 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f334aa00640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33300b2960 0x7f33300b4e70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x556e6aa4a5f0 tx=0x7f333c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f3357d9d740 1 -- 172.21.15.39:0/902653585 >> 172.21.15.39:0/902653585 conn(0x556e6a69b460 msgr2=0x556e6aa49860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f3357d9d740 1 -- 172.21.15.39:0/902653585 shutdown_connections 2024-07-04T08:18:46.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.977+0000 7f3357d9d740 1 -- 172.21.15.39:0/902653585 wait complete. 2024-07-04T08:18:46.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.978+0000 7f3357d9d740 1 -- start start 2024-07-04T08:18:46.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.978+0000 7f3357d9d740 1 Processor -- start 2024-07-04T08:18:46.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 0x556e6aa5b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa67780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f3357d9d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556e6a815700 con 0x556e6a67e1b0 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f3357d9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556e6a958bd0 con 0x556e6a686390 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f3357d9d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556e6aa5e450 con 0x556e6a6a7070 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f3357d9d740 10 client.0 _ll_register_callbacks cb 0x556e6a9aa590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f334aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 0x556e6aa5b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f334a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f334a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58290/0 (socket says 172.21.15.39:58290) 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f334a000640 1 -- 172.21.15.39:0/2890410563 learned_addr learned my addr 172.21.15.39:0/2890410563 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.979+0000 7f334aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 0x556e6aa5b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43974/0 (socket says 172.21.15.39:43974) 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f334b400640 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa67780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f334a000640 1 -- 172.21.15.39:0/2890410563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 msgr2=0x556e6aa67780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f334a000640 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa67780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f334a000640 1 -- 172.21.15.39:0/2890410563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 msgr2=0x556e6aa5b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f334a000640 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 0x556e6aa5b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f334a000640 1 -- 172.21.15.39:0/2890410563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6a712650 con 0x556e6a686390 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f334b400640 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa67780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f334a000640 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5bfb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3334004650 tx=0x7f33340a9600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12360cf2ec7a1e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f3338c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.980+0000 7f3338c00640 1 -- 172.21.15.39:0/2890410563 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340a5380 con 0x556e6a686390 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3338c00640 1 -- 172.21.15.39:0/2890410563 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f33340a61e0 con 0x556e6a686390 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3357d9d740 1 -- 172.21.15.39:0/2890410563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e6a9c3e50 con 0x556e6a686390 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3338c00640 1 -- 172.21.15.39:0/2890410563 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3334005310 con 0x556e6a686390 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3357d9d740 10 client.26333 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3357d9d740 1 -- 172.21.15.39:0/2890410563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e6aa6a590 con 0x556e6a686390 2024-07-04T08:18:46.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3338c00640 1 -- 172.21.15.39:0/2890410563 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556e6a9c3e50 con 0x556e6a686390 2024-07-04T08:18:46.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3338c00640 1 -- 172.21.15.39:0/2890410563 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f33340e8070 con 0x556e6a686390 2024-07-04T08:18:46.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3357d9d740 10 client.26333 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:46.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3357d9d740 10 client.26333 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:18:46.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.981+0000 7f3357d9d740 20 client.26333 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:46.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.982+0000 7f3357d9d740 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556e6a6659f0 0x556e6aa6dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:46.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.982+0000 7f3357d9d740 4 client.26333 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:46.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.982+0000 7f3357d9d740 1 -- 172.21.15.39:0/2890410563 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e6a9bf8f0 con 0x556e6a6659f0 2024-07-04T08:18:46.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.982+0000 7f334aa00640 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556e6a6659f0 0x556e6aa6dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.983+0000 7f334aa00640 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556e6a6659f0 0x556e6aa6dad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=976993734afb20be server_cookie=b80691a50e7052a6 in_seq=0 out_seq=0 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.983+0000 7f3338c00640 10 client.26333 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.983+0000 7f3338c00640 1 -- 172.21.15.39:0/2890410563 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556e6a9bf8f0 con 0x556e6a6659f0 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.983+0000 7f3338c00640 10 client.26333 handle_command_reply: tid=0 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.983+0000 7f3357d9d740 1 client.26333 shutdown 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 -- 172.21.15.39:0/2890410563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 msgr2=0x556e6aa5bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5bfb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3334004650 tx=0x7f33340a9600 comp rx=0 tx=0).stop 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 -- 172.21.15.39:0/2890410563 shutdown_connections 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556e6a6659f0 0x556e6aa6dad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa67780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5bfb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3338c00640 0 client.26333 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:18:46.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 --2- 172.21.15.39:0/2890410563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 0x556e6aa5b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:46.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 -- 172.21.15.39:0/2890410563 >> 172.21.15.39:0/2890410563 conn(0x556e6a69b460 msgr2=0x556e6aa47e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:46.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 -- 172.21.15.39:0/2890410563 shutdown_connections 2024-07-04T08:18:46.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 1 -- 172.21.15.39:0/2890410563 wait complete. 2024-07-04T08:18:46.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:46.984+0000 7f3357d9d740 20 client.26333 trim_cache size 0 max 16384 2024-07-04T08:18:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.000+0000 7f3357d9d740 1 Processor -- start 2024-07-04T08:18:47.000 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:46 smithi039 ceph-mon[32062]: pgmap v587: 129 pgs: 129 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.000+0000 7f3357d9d740 1 -- start start 2024-07-04T08:18:47.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.000+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a67e1b0 0x556e6aa49890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:47.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.000+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 0x556e6aa67570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:47.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.000+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa47e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:47.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.000+0000 7f3357d9d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1adec0 con 0x556e6a686390 2024-07-04T08:18:47.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.000+0000 7f3357d9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1afa00 con 0x556e6a67e1b0 2024-07-04T08:18:47.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.000+0000 7f3357d9d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ae590 con 0x556e6a6a7070 2024-07-04T08:18:47.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa47e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:47.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a67e1b0 0x556e6aa49890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 0x556e6aa67570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa47e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:46996/0 (socket says 172.21.15.39:46996) 2024-07-04T08:18:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334b400640 1 -- 172.21.15.39:0/1947965332 learned_addr learned my addr 172.21.15.39:0/1947965332 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334b400640 1 -- 172.21.15.39:0/1947965332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a67e1b0 msgr2=0x556e6aa49890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334b400640 1 --2- 172.21.15.39:0/1947965332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a67e1b0 0x556e6aa49890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334b400640 1 -- 172.21.15.39:0/1947965332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 msgr2=0x556e6aa67570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334b400640 1 --2- 172.21.15.39:0/1947965332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 0x556e6aa67570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334b400640 1 -- 172.21.15.39:0/1947965332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e6aa5df30 con 0x556e6a6a7070 2024-07-04T08:18:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.001+0000 7f334b400640 1 --2- 172.21.15.39:0/1947965332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa47e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3344005ab0 tx=0x7f3344000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bfecb4619ad4b41 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.002+0000 7f333b400640 1 -- 172.21.15.39:0/1947965332 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344040070 con 0x556e6a6a7070 2024-07-04T08:18:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.002+0000 7f333b400640 1 -- 172.21.15.39:0/1947965332 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33440054d0 con 0x556e6a6a7070 2024-07-04T08:18:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.002+0000 7f333b400640 1 -- 172.21.15.39:0/1947965332 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344035040 con 0x556e6a6a7070 2024-07-04T08:18:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.003+0000 7f3357d9d740 1 -- 172.21.15.39:0/1947965332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 msgr2=0x556e6aa47e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.003+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1947965332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa47e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3344005ab0 tx=0x7f3344000bc0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.003+0000 7f3357d9d740 1 -- 172.21.15.39:0/1947965332 shutdown_connections 2024-07-04T08:18:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.003+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1947965332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a686390 0x556e6aa67570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.003+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1947965332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aa47e30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.003+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1947965332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a67e1b0 0x556e6aa49890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.003+0000 7f3357d9d740 1 -- 172.21.15.39:0/1947965332 >> 172.21.15.39:0/1947965332 conn(0x556e6a69b460 msgr2=0x556e6aa49180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.003+0000 7f3357d9d740 1 -- 172.21.15.39:0/1947965332 shutdown_connections 2024-07-04T08:18:47.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.003+0000 7f3357d9d740 1 -- 172.21.15.39:0/1947965332 wait complete. 2024-07-04T08:18:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.004+0000 7f3357d9d740 1 -- start start 2024-07-04T08:18:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.004+0000 7f3357d9d740 1 Processor -- start 2024-07-04T08:18:47.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 0x556e6aa5f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f3357d9d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aad5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f3357d9d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1af330 con 0x556e6a67e1b0 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f3357d9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1aec60 con 0x556e6a686390 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f3357d9d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1abea0 con 0x556e6a6a7070 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f3357d9d740 10 client.0 _ll_register_callbacks cb 0x7f33340096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f334a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f334aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 0x556e6aa5f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f334a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58298/0 (socket says 172.21.15.39:58298) 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f334a000640 1 -- 172.21.15.39:0/1572259461 learned_addr learned my addr 172.21.15.39:0/1572259461 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:47.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.005+0000 7f334b400640 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aad5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f334a000640 1 -- 172.21.15.39:0/1572259461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 msgr2=0x556e6aad5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f334a000640 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aad5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f334a000640 1 -- 172.21.15.39:0/1572259461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 msgr2=0x556e6aa5f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f334a000640 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 0x556e6aa5f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f334a000640 1 -- 172.21.15.39:0/1572259461 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e6aad57a0 con 0x556e6a686390 2024-07-04T08:18:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f334b400640 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aad5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f334a000640 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5fe80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f33340a6e60 tx=0x7f333400c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c399f3b1d2379eef server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f3338c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:47.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f3338c00640 1 -- 172.21.15.39:0/1572259461 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340a5380 con 0x556e6a686390 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f3338c00640 1 -- 172.21.15.39:0/1572259461 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f33340a61e0 con 0x556e6a686390 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.006+0000 7f3357d9d740 1 -- 172.21.15.39:0/1572259461 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33440054d0 con 0x556e6a686390 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3338c00640 1 -- 172.21.15.39:0/1572259461 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33340a99c0 con 0x556e6a686390 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3357d9d740 10 client.26339 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3357d9d740 1 -- 172.21.15.39:0/1572259461 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e6aad6870 con 0x556e6a686390 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3338c00640 1 -- 172.21.15.39:0/1572259461 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33440054d0 con 0x556e6a686390 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3338c00640 1 -- 172.21.15.39:0/1572259461 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f33340e8070 con 0x556e6a686390 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3357d9d740 10 client.26339 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3357d9d740 10 client.26339 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3357d9d740 20 client.26339 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556e6a63ada0 0x556e6aad9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3357d9d740 4 client.26339 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.007+0000 7f3357d9d740 1 -- 172.21.15.39:0/1572259461 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e6a9bd030 con 0x556e6a63ada0 2024-07-04T08:18:47.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.008+0000 7f334aa00640 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556e6a63ada0 0x556e6aad9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.008+0000 7f334aa00640 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556e6a63ada0 0x556e6aad9080 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3895bc24fa9115b server_cookie=f24b035e6c6c1f7c in_seq=0 out_seq=0 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.008+0000 7f3338c00640 10 client.26339 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3338c00640 1 -- 172.21.15.39:0/1572259461 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e6a9bd030 con 0x556e6a63ada0 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3338c00640 10 client.26339 handle_command_reply: tid=0 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 client.26339 shutdown 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 -- 172.21.15.39:0/1572259461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 msgr2=0x556e6aa5fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5fe80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f33340a6e60 tx=0x7f333400c840 comp rx=0 tx=0).stop 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 -- 172.21.15.39:0/1572259461 shutdown_connections 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556e6a63ada0 0x556e6aad9080 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556e6a6a7070 0x556e6aad5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556e6a686390 0x556e6aa5fe80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 --2- 172.21.15.39:0/1572259461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556e6a67e1b0 0x556e6aa5f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3338c00640 0 client.26339 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:18:47.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 -- 172.21.15.39:0/1572259461 >> 172.21.15.39:0/1572259461 conn(0x556e6a69b460 msgr2=0x556e6aa47e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:47.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 -- 172.21.15.39:0/1572259461 shutdown_connections 2024-07-04T08:18:47.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.009+0000 7f3357d9d740 1 -- 172.21.15.39:0/1572259461 wait complete. 2024-07-04T08:18:47.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f3357d9d740 20 client.26339 trim_cache size 0 max 16384 2024-07-04T08:18:47.012 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 -- 172.21.15.39:0/2357394729 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33300b2960 msgr2=0x7f33300b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33300b2960 0x7f33300b4e70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x556e6aa4a5f0 tx=0x7f333c009500 comp rx=0 tx=0).stop 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 -- 172.21.15.39:0/2357394729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c108840 msgr2=0x7f334c199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c108840 0x7f334c199dd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f333400e9c0 tx=0x7f333400ee80 comp rx=0 tx=0).stop 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 -- 172.21.15.39:0/2357394729 shutdown_connections 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f33300b2960 0x7f33300b4e70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f334c109190 0x7f334c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f334c108840 0x7f334c199dd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 --2- 172.21.15.39:0/2357394729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f334c102f00 0x7f334c199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.010+0000 7f334be00640 1 -- 172.21.15.39:0/2357394729 >> 172.21.15.39:0/2357394729 conn(0x7f334c0fe990 msgr2=0x7f334c107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.011+0000 7f334be00640 1 -- 172.21.15.39:0/2357394729 shutdown_connections 2024-07-04T08:18:47.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:47.011+0000 7f334be00640 1 -- 172.21.15.39:0/2357394729 wait complete. 2024-07-04T08:18:47.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:46 smithi050 ceph-mon[36577]: pgmap v587: 129 pgs: 129 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:47.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:46 smithi133 ceph-mon[37050]: pgmap v587: 129 pgs: 129 active+clean; 3.2 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:47.910 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:18:48.403 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:49.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:48 smithi050 ceph-mon[36577]: pgmap v588: 129 pgs: 129 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-07-04T08:18:49.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:48 smithi133 ceph-mon[37050]: pgmap v588: 129 pgs: 129 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-07-04T08:18:49.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:48 smithi039 ceph-mon[32062]: pgmap v588: 129 pgs: 129 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-07-04T08:18:50.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.743+0000 7fb09d000640 1 -- 172.21.15.39:0/1371187401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb09810f7a0 msgr2=0x7fb098115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:50.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.743+0000 7fb09d000640 1 --2- 172.21.15.39:0/1371187401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb09810f7a0 0x7fb098115fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb090009be0 tx=0x7fb09002a6c0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.743+0000 7fb09d000640 1 -- 172.21.15.39:0/1371187401 shutdown_connections 2024-07-04T08:18:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.743+0000 7fb09d000640 1 --2- 172.21.15.39:0/1371187401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb09810ee50 0x7fb09810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.743+0000 7fb09d000640 1 --2- 172.21.15.39:0/1371187401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb09810f7a0 0x7fb098115fd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.743+0000 7fb09d000640 1 --2- 172.21.15.39:0/1371187401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb098109510 0x7fb098109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.743+0000 7fb09d000640 1 -- 172.21.15.39:0/1371187401 >> 172.21.15.39:0/1371187401 conn(0x7fb098078b10 msgr2=0x7fb098107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.743+0000 7fb09d000640 1 -- 172.21.15.39:0/1371187401 shutdown_connections 2024-07-04T08:18:50.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.743+0000 7fb09d000640 1 -- 172.21.15.39:0/1371187401 wait complete. 2024-07-04T08:18:50.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.744+0000 7fb09d000640 1 Processor -- start 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.744+0000 7fb09d000640 1 -- start start 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb09d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb098109510 0x7fb09819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb09d000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb09810ee50 0x7fb0981a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb09d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb09810f7a0 0x7fb0981a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb09d000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb098119470 con 0x7fb09810f7a0 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb09d000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981192f0 con 0x7fb098109510 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb09d000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981195f0 con 0x7fb09810ee50 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb097400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb098109510 0x7fb09819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb097e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb09810f7a0 0x7fb0981a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb097400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb098109510 0x7fb09819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58326/0 (socket says 172.21.15.39:58326) 2024-07-04T08:18:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb097400640 1 -- 172.21.15.39:0/3555286442 learned_addr learned my addr 172.21.15.39:0/3555286442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.745+0000 7fb08fe00640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb09810ee50 0x7fb0981a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.746+0000 7fb097400640 1 -- 172.21.15.39:0/3555286442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb09810ee50 msgr2=0x7fb0981a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.746+0000 7fb097400640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb09810ee50 0x7fb0981a0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.746+0000 7fb097400640 1 -- 172.21.15.39:0/3555286442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb09810f7a0 msgr2=0x7fb0981a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.746+0000 7fb097400640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb09810f7a0 0x7fb0981a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.746+0000 7fb097400640 1 -- 172.21.15.39:0/3555286442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0981a5d30 con 0x7fb098109510 2024-07-04T08:18:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.746+0000 7fb08fe00640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb09810ee50 0x7fb0981a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:50.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.746+0000 7fb097400640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb098109510 0x7fb09819fdc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb08800e9f0 tx=0x7fb08800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:50.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.747+0000 7fb094c00640 1 -- 172.21.15.39:0/3555286442 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08800cd80 con 0x7fb098109510 2024-07-04T08:18:50.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.747+0000 7fb094c00640 1 -- 172.21.15.39:0/3555286442 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0880054d0 con 0x7fb098109510 2024-07-04T08:18:50.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.747+0000 7fb09d000640 1 -- 172.21.15.39:0/3555286442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0981a6020 con 0x7fb098109510 2024-07-04T08:18:50.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.747+0000 7fb094c00640 1 -- 172.21.15.39:0/3555286442 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088010690 con 0x7fb098109510 2024-07-04T08:18:50.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.747+0000 7fb09d000640 1 -- 172.21.15.39:0/3555286442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0981ac780 con 0x7fb098109510 2024-07-04T08:18:50.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.750+0000 7fb0a4b5d740 1 Processor -- start 2024-07-04T08:18:50.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.750+0000 7fb0a4b5d740 1 -- start start 2024-07-04T08:18:50.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb094c00640 1 -- 172.21.15.39:0/3555286442 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb088004e90 con 0x7fb098109510 2024-07-04T08:18:50.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c3962950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35951b0 0x55e4c3962e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c39633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e4c38d8ee0 con 0x55e4c35951b0 2024-07-04T08:18:50.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e4c38c8480 con 0x55e4c35be070 2024-07-04T08:18:50.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e4c38cc4c0 con 0x55e4c359d390 2024-07-04T08:18:50.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb097400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35951b0 0x55e4c3962e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb097e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c3962950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb097400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35951b0 0x55e4c3962e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44022/0 (socket says 172.21.15.39:44022) 2024-07-04T08:18:50.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb097e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c3962950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58342/0 (socket says 172.21.15.39:58342) 2024-07-04T08:18:50.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.751+0000 7fb097400640 1 -- 172.21.15.39:0/3022318572 learned_addr learned my addr 172.21.15.39:0/3022318572 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:50.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.752+0000 7fb08fe00640 1 --2- 172.21.15.39:0/3022318572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c39633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.752+0000 7fb097e00640 1 -- 172.21.15.39:0/3022318572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 msgr2=0x55e4c39633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.752+0000 7fb097e00640 1 --2- 172.21.15.39:0/3022318572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c39633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.752+0000 7fb097e00640 1 -- 172.21.15.39:0/3022318572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35951b0 msgr2=0x55e4c3962e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.752+0000 7fb097e00640 1 --2- 172.21.15.39:0/3022318572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35951b0 0x55e4c3962e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.752+0000 7fb097e00640 1 -- 172.21.15.39:0/3022318572 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4c3860650 con 0x55e4c35be070 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.752+0000 7fb08fe00640 1 --2- 172.21.15.39:0/3022318572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c39633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.752+0000 7fb097e00640 1 --2- 172.21.15.39:0/3022318572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c3962950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb09002abe0 tx=0x7fb090030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2caa35d782c187e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.753+0000 7fb08cc00640 1 -- 172.21.15.39:0/3022318572 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090041030 con 0x55e4c35be070 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.753+0000 7fb08cc00640 1 -- 172.21.15.39:0/3022318572 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb090003e20 con 0x55e4c35be070 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.753+0000 7fb08cc00640 1 -- 172.21.15.39:0/3022318572 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090036040 con 0x55e4c35be070 2024-07-04T08:18:50.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.753+0000 7fb094c00640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb07c0b2960 0x7fb07c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.753+0000 7fb094c00640 1 -- 172.21.15.39:0/3555286442 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb0880e2d40 con 0x7fb098109510 2024-07-04T08:18:50.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.753+0000 7fb097400640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb07c0b2960 0x7fb07c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/3022318572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 msgr2=0x55e4c3962950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:50.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/3022318572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c3962950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb09002abe0 tx=0x7fb090030420 comp rx=0 tx=0).stop 2024-07-04T08:18:50.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/3022318572 shutdown_connections 2024-07-04T08:18:50.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/3022318572 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35951b0 0x55e4c3962e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/3022318572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c39633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/3022318572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c3962950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/3022318572 >> 172.21.15.39:0/3022318572 conn(0x55e4c35b2460 msgr2=0x55e4c3960840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:50.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb097400640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb07c0b2960 0x7fb07c0b4e70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb08800c6a0 tx=0x7fb0880e6890 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:50.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/3022318572 shutdown_connections 2024-07-04T08:18:50.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.754+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/3022318572 wait complete. 2024-07-04T08:18:50.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.755+0000 7fb0a4b5d740 1 -- start start 2024-07-04T08:18:50.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.755+0000 7fb0a4b5d740 1 Processor -- start 2024-07-04T08:18:50.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 0x55e4c396e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c359d390 0x55e4c396ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c397a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e4c372c700 con 0x55e4c359d390 2024-07-04T08:18:50.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e4c386fbb0 con 0x55e4c35be070 2024-07-04T08:18:50.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e4c39710d0 con 0x55e4c35951b0 2024-07-04T08:18:50.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb0a4b5d740 10 client.0 _ll_register_callbacks cb 0x55e4c38c1570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c359d390 0x55e4c396ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb097e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 0x55e4c396e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c359d390 0x55e4c396ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44030/0 (socket says 172.21.15.39:44030) 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb08fe00640 1 -- 172.21.15.39:0/2229115726 learned_addr learned my addr 172.21.15.39:0/2229115726 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.756+0000 7fb097e00640 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c397a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb08fe00640 1 -- 172.21.15.39:0/2229115726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 msgr2=0x55e4c396e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb08fe00640 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 0x55e4c396e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb08fe00640 1 -- 172.21.15.39:0/2229115726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 msgr2=0x55e4c397a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb08fe00640 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c397a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb08fe00640 1 -- 172.21.15.39:0/2229115726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4c3629650 con 0x55e4c359d390 2024-07-04T08:18:50.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb08fe00640 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c359d390 0x55e4c396ea50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb080005fd0 tx=0x7fb080012de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e293dee129db581 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb085600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb085600640 1 -- 172.21.15.39:0/2229115726 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08000fa00 con 0x55e4c359d390 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb085600640 1 -- 172.21.15.39:0/2229115726 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb08000fba0 con 0x55e4c359d390 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.757+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2229115726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb090003e20 con 0x55e4c359d390 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.758+0000 7fb085600640 1 -- 172.21.15.39:0/2229115726 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb08000fda0 con 0x55e4c359d390 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.758+0000 7fb0a4b5d740 10 client.16221 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.758+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2229115726 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4c397d0f0 con 0x55e4c359d390 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.758+0000 7fb085600640 1 -- 172.21.15.39:0/2229115726 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb090003e20 con 0x55e4c359d390 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.758+0000 7fb085600640 1 -- 172.21.15.39:0/2229115726 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb08001c420 con 0x55e4c359d390 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.758+0000 7fb0a4b5d740 10 client.16221 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.758+0000 7fb0a4b5d740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.758+0000 7fb0a4b5d740 20 client.16221 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:50.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.759+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55e4c357c9f0 0x55e4c3980700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.759+0000 7fb0a4b5d740 4 client.16221 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:50.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.759+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2229115726 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4c38dae30 con 0x55e4c357c9f0 2024-07-04T08:18:50.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.759+0000 7fb097e00640 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55e4c357c9f0 0x55e4c3980700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.759+0000 7fb097e00640 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55e4c357c9f0 0x55e4c3980700 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66478ef42ed7dfb0 server_cookie=a0be32f94b12d90c in_seq=0 out_seq=0 2024-07-04T08:18:50.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.759+0000 7fb085600640 10 client.16221 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:50.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.760+0000 7fb085600640 1 -- 172.21.15.39:0/2229115726 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e4c38dae30 con 0x55e4c357c9f0 2024-07-04T08:18:50.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.760+0000 7fb085600640 10 client.16221 handle_command_reply: tid=0 2024-07-04T08:18:50.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.760+0000 7fb0a4b5d740 1 client.16221 shutdown 2024-07-04T08:18:50.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2229115726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c359d390 msgr2=0x55e4c396ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c359d390 0x55e4c396ea50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb080005fd0 tx=0x7fb080012de0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2229115726 shutdown_connections 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55e4c357c9f0 0x55e4c3980700 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c359d390 0x55e4c396ea50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35be070 0x55e4c397a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2229115726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 0x55e4c396e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb085600640 0 client.16221 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2229115726 >> 172.21.15.39:0/2229115726 conn(0x55e4c35b2460 msgr2=0x55e4c3965b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2229115726 shutdown_connections 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2229115726 wait complete. 2024-07-04T08:18:50.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.761+0000 7fb0a4b5d740 20 client.16221 trim_cache size 0 max 16384 2024-07-04T08:18:50.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.777+0000 7fb0a4b5d740 1 Processor -- start 2024-07-04T08:18:50.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.777+0000 7fb0a4b5d740 1 -- start start 2024-07-04T08:18:50.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.777+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35951b0 0x55e4c3966410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.777+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c3964090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.777+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 0x55e4c397a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.777+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b6510 con 0x55e4c35be070 2024-07-04T08:18:50.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.777+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b8050 con 0x55e4c35951b0 2024-07-04T08:18:50.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.777+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b6be0 con 0x55e4c359d390 2024-07-04T08:18:50.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c3964090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c3964090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:47058/0 (socket says 172.21.15.39:47058) 2024-07-04T08:18:50.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb08fe00640 1 -- 172.21.15.39:0/2287087589 learned_addr learned my addr 172.21.15.39:0/2287087589 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:50.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb097e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35951b0 0x55e4c3966410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb097e00640 1 --2- 172.21.15.39:0/2287087589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 0x55e4c397a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb08fe00640 1 -- 172.21.15.39:0/2287087589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35951b0 msgr2=0x55e4c3966410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb08fe00640 1 --2- 172.21.15.39:0/2287087589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35951b0 0x55e4c3966410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb08fe00640 1 -- 172.21.15.39:0/2287087589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 msgr2=0x55e4c397a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb08fe00640 1 --2- 172.21.15.39:0/2287087589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 0x55e4c397a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb08fe00640 1 -- 172.21.15.39:0/2287087589 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4c3964c40 con 0x55e4c359d390 2024-07-04T08:18:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.778+0000 7fb097e00640 1 --2- 172.21.15.39:0/2287087589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 0x55e4c397a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:18:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.779+0000 7fb08fe00640 1 --2- 172.21.15.39:0/2287087589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c3964090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb0800126a0 tx=0x7fb08001df50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a22fafd1c8c3ba9c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.779+0000 7fb087e00640 1 -- 172.21.15.39:0/2287087589 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08000fa00 con 0x55e4c359d390 2024-07-04T08:18:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.779+0000 7fb087e00640 1 -- 172.21.15.39:0/2287087589 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb08000fba0 con 0x55e4c359d390 2024-07-04T08:18:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.779+0000 7fb087e00640 1 -- 172.21.15.39:0/2287087589 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080004660 con 0x55e4c359d390 2024-07-04T08:18:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.780+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2287087589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 msgr2=0x55e4c3964090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.780+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2287087589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c3964090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb0800126a0 tx=0x7fb08001df50 comp rx=0 tx=0).stop 2024-07-04T08:18:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.780+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2287087589 shutdown_connections 2024-07-04T08:18:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.780+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2287087589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 0x55e4c397a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.780+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2287087589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c359d390 0x55e4c3964090 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.780+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2287087589 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c35951b0 0x55e4c3966410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.780+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2287087589 >> 172.21.15.39:0/2287087589 conn(0x55e4c35b2460 msgr2=0x55e4c3963980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.780+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2287087589 shutdown_connections 2024-07-04T08:18:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.780+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2287087589 wait complete. 2024-07-04T08:18:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.781+0000 7fb0a4b5d740 1 -- start start 2024-07-04T08:18:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.781+0000 7fb0a4b5d740 1 Processor -- start 2024-07-04T08:18:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 0x55e4c3972190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c359d390 0x55e4c39726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb0a4b5d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 0x55e4c3972c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b7980 con 0x55e4c35be070 2024-07-04T08:18:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b72b0 con 0x55e4c359d390 2024-07-04T08:18:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb0a4b5d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981b44f0 con 0x55e4c35951b0 2024-07-04T08:18:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb0a4b5d740 10 client.0 _ll_register_callbacks cb 0x7fb080012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c359d390 0x55e4c39726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb097e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 0x55e4c3972c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c359d390 0x55e4c39726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:58372/0 (socket says 172.21.15.39:58372) 2024-07-04T08:18:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb08fe00640 1 -- 172.21.15.39:0/2283625717 learned_addr learned my addr 172.21.15.39:0/2283625717 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.782+0000 7fb097e00640 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 0x55e4c3972190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb08fe00640 1 -- 172.21.15.39:0/2283625717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 msgr2=0x55e4c3972190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb08fe00640 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 0x55e4c3972190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb08fe00640 1 -- 172.21.15.39:0/2283625717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 msgr2=0x55e4c3972c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb08fe00640 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 0x55e4c3972c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb08fe00640 1 -- 172.21.15.39:0/2283625717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4c39e7e90 con 0x55e4c359d390 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb08fe00640 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c359d390 0x55e4c39726d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb080009fe0 tx=0x7fb0800128e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed814bae511df1b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb085600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb085600640 1 -- 172.21.15.39:0/2283625717 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080038060 con 0x55e4c359d390 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2283625717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb08000fba0 con 0x55e4c359d390 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.783+0000 7fb085600640 1 -- 172.21.15.39:0/2283625717 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb080010640 con 0x55e4c359d390 2024-07-04T08:18:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb085600640 1 -- 172.21.15.39:0/2283625717 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb08003f050 con 0x55e4c359d390 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb0a4b5d740 10 client.26357 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2283625717 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4c39e90b0 con 0x55e4c359d390 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb085600640 1 -- 172.21.15.39:0/2283625717 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb08000fba0 con 0x55e4c359d390 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb085600640 1 -- 172.21.15.39:0/2283625717 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb080033640 con 0x55e4c359d390 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb0a4b5d740 10 client.26357 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb0a4b5d740 10 client.26357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb0a4b5d740 20 client.26357 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55e4c3551da0 0x55e4c39eb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb0a4b5d740 4 client.26357 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.784+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2283625717 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4c38d68d0 con 0x55e4c3551da0 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.785+0000 7fb097e00640 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55e4c3551da0 0x55e4c39eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.785+0000 7fb097e00640 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55e4c3551da0 0x55e4c39eb770 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed05e44487c24d0 server_cookie=d7cb83aa05377afb in_seq=0 out_seq=0 2024-07-04T08:18:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.785+0000 7fb085600640 10 client.26357 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb085600640 1 -- 172.21.15.39:0/2283625717 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4c38d68d0 con 0x55e4c3551da0 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb085600640 10 client.26357 handle_command_reply: tid=0 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 client.26357 shutdown 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2283625717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c359d390 msgr2=0x55e4c39726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c359d390 0x55e4c39726d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb080009fe0 tx=0x7fb0800128e0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2283625717 shutdown_connections 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55e4c3551da0 0x55e4c39eb770 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e4c35be070 0x55e4c3972c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4c359d390 0x55e4c39726d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 --2- 172.21.15.39:0/2283625717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e4c35951b0 0x55e4c3972190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb085600640 0 client.26357 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2283625717 >> 172.21.15.39:0/2283625717 conn(0x55e4c35b2460 msgr2=0x55e4c39661e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2283625717 shutdown_connections 2024-07-04T08:18:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.786+0000 7fb0a4b5d740 1 -- 172.21.15.39:0/2283625717 wait complete. 2024-07-04T08:18:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.787+0000 7fb0a4b5d740 20 client.26357 trim_cache size 0 max 16384 2024-07-04T08:18:50.791 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.787+0000 7fb09d000640 1 -- 172.21.15.39:0/3555286442 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb07c0b2960 msgr2=0x7fb07c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.787+0000 7fb09d000640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb07c0b2960 0x7fb07c0b4e70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb08800c6a0 tx=0x7fb0880e6890 comp rx=0 tx=0).stop 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 -- 172.21.15.39:0/3555286442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb098109510 msgr2=0x7fb09819fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb098109510 0x7fb09819fdc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb08800e9f0 tx=0x7fb08800edf0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 -- 172.21.15.39:0/3555286442 shutdown_connections 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb07c0b2960 0x7fb07c0b4e70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb09810f7a0 0x7fb0981a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb09810ee50 0x7fb0981a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 --2- 172.21.15.39:0/3555286442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb098109510 0x7fb09819fdc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 -- 172.21.15.39:0/3555286442 >> 172.21.15.39:0/3555286442 conn(0x7fb098078b10 msgr2=0x7fb098079170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 -- 172.21.15.39:0/3555286442 shutdown_connections 2024-07-04T08:18:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:50.788+0000 7fb09d000640 1 -- 172.21.15.39:0/3555286442 wait complete. 2024-07-04T08:18:51.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:50 smithi039 ceph-mon[32062]: pgmap v589: 129 pgs: 129 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:51.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:50 smithi050 ceph-mon[36577]: pgmap v589: 129 pgs: 129 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:51.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:50 smithi133 ceph-mon[37050]: pgmap v589: 129 pgs: 129 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-07-04T08:18:52.692 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:18:52.692 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:18:53.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:52 smithi039 ceph-mon[32062]: pgmap v590: 129 pgs: 129 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:53.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:52 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:18:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:53.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:52 smithi133 ceph-mon[37050]: pgmap v590: 129 pgs: 129 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:53.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:52 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:18:53.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:52 smithi050 ceph-mon[36577]: pgmap v590: 129 pgs: 129 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:53.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:52 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:18:54.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:18:54.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:18:54.101 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:53 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:18:54.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:18:54.316 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:53 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:18:54.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:18:54.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:18:54.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:18:54.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:53 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:18:55.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:54 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-07-04T08:18:55.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:54 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-07-04T08:18:55.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:54 smithi039 ceph-mon[32062]: Unable to set osd_memory_target on smithi133 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-04T08:18:55.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:54 smithi039 ceph-mon[32062]: Unable to set osd_memory_target on smithi050 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-07-04T08:18:55.009 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:54 smithi039 ceph-mon[32062]: pgmap v591: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:55.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:54 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-07-04T08:18:55.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:54 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-07-04T08:18:55.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:54 smithi133 ceph-mon[37050]: Unable to set osd_memory_target on smithi133 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-04T08:18:55.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:54 smithi133 ceph-mon[37050]: Unable to set osd_memory_target on smithi050 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-07-04T08:18:55.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:54 smithi133 ceph-mon[37050]: pgmap v591: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:55.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:54 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-07-04T08:18:55.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:54 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-07-04T08:18:55.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:54 smithi050 ceph-mon[36577]: Unable to set osd_memory_target on smithi133 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-04T08:18:55.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:54 smithi050 ceph-mon[36577]: Unable to set osd_memory_target on smithi050 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-07-04T08:18:55.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:54 smithi050 ceph-mon[36577]: pgmap v591: 129 pgs: 129 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-07-04T08:18:55.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.590+0000 7f9a5e600640 1 -- 172.21.15.39:0/1623310223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a5810f7a0 msgr2=0x7f9a58115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:55.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.590+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1623310223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a5810f7a0 0x7f9a58115fd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009ca0 tx=0x7f9a5002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.590+0000 7f9a5e600640 1 -- 172.21.15.39:0/1623310223 shutdown_connections 2024-07-04T08:18:55.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.590+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1623310223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a5810f7a0 0x7f9a58115fd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.590+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1623310223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a5810ee50 0x7f9a5810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.590+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1623310223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a58109510 0x7f9a58109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.590+0000 7f9a5e600640 1 -- 172.21.15.39:0/1623310223 >> 172.21.15.39:0/1623310223 conn(0x7f9a58078b10 msgr2=0x7f9a58107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:55.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.591+0000 7f9a5e600640 1 -- 172.21.15.39:0/1623310223 shutdown_connections 2024-07-04T08:18:55.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.591+0000 7f9a5e600640 1 -- 172.21.15.39:0/1623310223 wait complete. 2024-07-04T08:18:55.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.592+0000 7f9a5e600640 1 Processor -- start 2024-07-04T08:18:55.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.593+0000 7f9a5e600640 1 -- start start 2024-07-04T08:18:55.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.593+0000 7f9a5e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a58109510 0x7f9a5819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a5e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a5810ee50 0x7f9a581a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a5e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a5810f7a0 0x7f9a581a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a5e600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a58119470 con 0x7f9a5810f7a0 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a5e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581192f0 con 0x7f9a58109510 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a5e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581195f0 con 0x7f9a5810ee50 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a5d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a58109510 0x7f9a5819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a5d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a58109510 0x7f9a5819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38752/0 (socket says 172.21.15.39:38752) 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a5d200640 1 -- 172.21.15.39:0/1633693242 learned_addr learned my addr 172.21.15.39:0/1633693242 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a5dc00640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a5810f7a0 0x7f9a581a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.594+0000 7f9a57e00640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a5810ee50 0x7f9a581a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.595+0000 7f9a5d200640 1 -- 172.21.15.39:0/1633693242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a5810ee50 msgr2=0x7f9a581a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.595+0000 7f9a5d200640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a5810ee50 0x7f9a581a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.595+0000 7f9a5d200640 1 -- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a5810f7a0 msgr2=0x7f9a581a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:55.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.595+0000 7f9a5d200640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a5810f7a0 0x7f9a581a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.595+0000 7f9a5d200640 1 -- 172.21.15.39:0/1633693242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a581a5d50 con 0x7f9a58109510 2024-07-04T08:18:55.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.595+0000 7f9a5d200640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a58109510 0x7f9a5819fde0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00d950 tx=0x7f9a4c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:55.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.595+0000 7f9a55600640 1 -- 172.21.15.39:0/1633693242 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c017070 con 0x7f9a58109510 2024-07-04T08:18:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.595+0000 7f9a55600640 1 -- 172.21.15.39:0/1633693242 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a4c009c30 con 0x7f9a58109510 2024-07-04T08:18:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.595+0000 7f9a55600640 1 -- 172.21.15.39:0/1633693242 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00f620 con 0x7f9a58109510 2024-07-04T08:18:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.596+0000 7f9a5e600640 1 -- 172.21.15.39:0/1633693242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a581a6040 con 0x7f9a58109510 2024-07-04T08:18:55.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.596+0000 7f9a5e600640 1 -- 172.21.15.39:0/1633693242 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a581ac7a0 con 0x7f9a58109510 2024-07-04T08:18:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.599+0000 7f9a6617d740 1 Processor -- start 2024-07-04T08:18:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.599+0000 7f9a6617d740 1 -- start start 2024-07-04T08:18:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b40350 0x5559c3ee47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b17490 0x5559c3ee4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a6617d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5559c3e0e8e0 con 0x5559c3b40350 2024-07-04T08:18:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a6617d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5559c3e5aec0 con 0x5559c3b1f670 2024-07-04T08:18:55.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a6617d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5559c3e4a460 con 0x5559c3b17490 2024-07-04T08:18:55.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a5dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b40350 0x5559c3ee47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a5dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b40350 0x5559c3ee47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41820/0 (socket says 172.21.15.39:41820) 2024-07-04T08:18:55.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a5dc00640 1 -- 172.21.15.39:0/3034710418 learned_addr learned my addr 172.21.15.39:0/3034710418 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.600+0000 7f9a57e00640 1 --2- 172.21.15.39:0/3034710418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.601+0000 7f9a5dc00640 1 -- 172.21.15.39:0/3034710418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b17490 msgr2=0x5559c3ee4d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:18:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.601+0000 7f9a5dc00640 1 --2- 172.21.15.39:0/3034710418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b17490 0x5559c3ee4d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.601+0000 7f9a5dc00640 1 -- 172.21.15.39:0/3034710418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 msgr2=0x5559c3ee5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.601+0000 7f9a5dc00640 1 --2- 172.21.15.39:0/3034710418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.601+0000 7f9a5dc00640 1 -- 172.21.15.39:0/3034710418 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c3de2280 con 0x5559c3b40350 2024-07-04T08:18:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.601+0000 7f9a57e00640 1 --2- 172.21.15.39:0/3034710418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.601+0000 7f9a55600640 1 -- 172.21.15.39:0/1633693242 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9a4c00f7c0 con 0x7f9a58109510 2024-07-04T08:18:55.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.601+0000 7f9a5dc00640 1 --2- 172.21.15.39:0/3034710418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b40350 0x5559c3ee47f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a5002f4c0 tx=0x7f9a50009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83c3f36f5c72b40 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:55.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.601+0000 7f9a4a000640 1 -- 172.21.15.39:0/3034710418 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50031cf0 con 0x5559c3b40350 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.602+0000 7f9a4a000640 1 -- 172.21.15.39:0/3034710418 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a50036030 con 0x5559c3b40350 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.602+0000 7f9a4a000640 1 -- 172.21.15.39:0/3034710418 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a500042a0 con 0x5559c3b40350 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.602+0000 7f9a6617d740 1 -- 172.21.15.39:0/3034710418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b40350 msgr2=0x5559c3ee47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.602+0000 7f9a6617d740 1 --2- 172.21.15.39:0/3034710418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b40350 0x5559c3ee47f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a5002f4c0 tx=0x7f9a50009800 comp rx=0 tx=0).stop 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a55600640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9a400b29e0 0x7f9a400b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a6617d740 1 -- 172.21.15.39:0/3034710418 shutdown_connections 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a6617d740 1 --2- 172.21.15.39:0/3034710418 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a6617d740 1 --2- 172.21.15.39:0/3034710418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b17490 0x5559c3ee4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a55600640 1 -- 172.21.15.39:0/1633693242 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f9a4c0e2570 con 0x7f9a58109510 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a6617d740 1 --2- 172.21.15.39:0/3034710418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b40350 0x5559c3ee47f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a6617d740 1 -- 172.21.15.39:0/3034710418 >> 172.21.15.39:0/3034710418 conn(0x5559c3b34740 msgr2=0x5559c3ee26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a5dc00640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9a400b29e0 0x7f9a400b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a6617d740 1 -- 172.21.15.39:0/3034710418 shutdown_connections 2024-07-04T08:18:55.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.603+0000 7f9a6617d740 1 -- 172.21.15.39:0/3034710418 wait complete. 2024-07-04T08:18:55.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.604+0000 7f9a6617d740 1 -- start start 2024-07-04T08:18:55.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.604+0000 7f9a5dc00640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9a400b29e0 0x7f9a400b4ef0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9a500094e0 tx=0x7f9a5002a8c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:55.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.604+0000 7f9a6617d740 1 Processor -- start 2024-07-04T08:18:55.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b1f670 0x5559c3ef08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b40350 0x5559c3efbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a6617d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5559c3df1760 con 0x5559c3b17490 2024-07-04T08:18:55.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a6617d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5559c3e4e4a0 con 0x5559c3b40350 2024-07-04T08:18:55.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a6617d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5559c3cae890 con 0x5559c3b1f670 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a6617d740 10 client.0 _ll_register_callbacks cb 0x5559c3df22b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a5d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a5d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41832/0 (socket says 172.21.15.39:41832) 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a5d200640 1 -- 172.21.15.39:0/1873199966 learned_addr learned my addr 172.21.15.39:0/1873199966 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a5dc00640 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b40350 0x5559c3efbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.605+0000 7f9a57e00640 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b1f670 0x5559c3ef08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a5d200640 1 -- 172.21.15.39:0/1873199966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b1f670 msgr2=0x5559c3ef08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a5d200640 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b1f670 0x5559c3ef08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a5d200640 1 -- 172.21.15.39:0/1873199966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b40350 msgr2=0x5559c3efbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:55.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a5d200640 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b40350 0x5559c3efbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a5d200640 1 -- 172.21.15.39:0/1873199966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c3bab930 con 0x5559c3b17490 2024-07-04T08:18:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a5d200640 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef0390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c000c00 tx=0x7f9a4c0dfeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cc08379d30cd5e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a3ea00640 1 -- 172.21.15.39:0/1873199966 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c020070 con 0x5559c3b17490 2024-07-04T08:18:55.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a3ea00640 1 -- 172.21.15.39:0/1873199966 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f9a4c0e2180 con 0x5559c3b17490 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.606+0000 7f9a6617d740 1 -- 172.21.15.39:0/1873199966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a50036030 con 0x5559c3b17490 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a3ea00640 1 -- 172.21.15.39:0/1873199966 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a4c0a9570 con 0x5559c3b17490 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a6617d740 10 client.16239 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a6617d740 1 -- 172.21.15.39:0/1873199966 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559c3efec80 con 0x5559c3b17490 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a3ea00640 1 -- 172.21.15.39:0/1873199966 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a50036030 con 0x5559c3b17490 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a3ea00640 1 -- 172.21.15.39:0/1873199966 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9a4c0a8240 con 0x5559c3b17490 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a6617d740 10 client.16239 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a6617d740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a6617d740 20 client.16239 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a6617d740 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5559c3afecd0 0x5559c3f02150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a6617d740 4 client.16239 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:55.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.607+0000 7f9a6617d740 1 -- 172.21.15.39:0/1873199966 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559c3e5ce10 con 0x5559c3afecd0 2024-07-04T08:18:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.608+0000 7f9a57e00640 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5559c3afecd0 0x5559c3f02150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.609+0000 7f9a57e00640 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5559c3afecd0 0x5559c3f02150 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc286d37939f856 server_cookie=8d237e7475fa877e in_seq=0 out_seq=0 2024-07-04T08:18:55.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.609+0000 7f9a3ea00640 10 client.16239 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.609+0000 7f9a3ea00640 1 -- 172.21.15.39:0/1873199966 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5559c3e5ce10 con 0x5559c3afecd0 2024-07-04T08:18:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.609+0000 7f9a3ea00640 10 client.16239 handle_command_reply: tid=0 2024-07-04T08:18:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.609+0000 7f9a6617d740 1 client.16239 shutdown 2024-07-04T08:18:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 -- 172.21.15.39:0/1873199966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 msgr2=0x5559c3ef0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef0390 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c000c00 tx=0x7f9a4c0dfeb0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 -- 172.21.15.39:0/1873199966 shutdown_connections 2024-07-04T08:18:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5559c3afecd0 0x5559c3f02150 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b40350 0x5559c3efbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b1f670 0x5559c3ef08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 --2- 172.21.15.39:0/1873199966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef0390 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a3ea00640 0 client.16239 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 -- 172.21.15.39:0/1873199966 >> 172.21.15.39:0/1873199966 conn(0x5559c3b34740 msgr2=0x5559c3ee79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 -- 172.21.15.39:0/1873199966 shutdown_connections 2024-07-04T08:18:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 1 -- 172.21.15.39:0/1873199966 wait complete. 2024-07-04T08:18:55.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.610+0000 7f9a6617d740 20 client.16239 trim_cache size 0 max 16384 2024-07-04T08:18:55.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.630+0000 7f9a6617d740 1 Processor -- start 2024-07-04T08:18:55.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.630+0000 7f9a6617d740 1 -- start start 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ee9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a6617d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b6530 con 0x5559c3b17490 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a6617d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b8070 con 0x5559c3b1f670 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a6617d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b6c00 con 0x5559c3b40350 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a5d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ee9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a5d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ee9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41848/0 (socket says 172.21.15.39:41848) 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a5d200640 1 -- 172.21.15.39:0/717140671 learned_addr learned my addr 172.21.15.39:0/717140671 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a57e00640 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.631+0000 7f9a5dc00640 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a5d200640 1 -- 172.21.15.39:0/717140671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 msgr2=0x5559c3ef1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a5d200640 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a5d200640 1 -- 172.21.15.39:0/717140671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 msgr2=0x5559c3ee61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a5d200640 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a5d200640 1 -- 172.21.15.39:0/717140671 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559c3eddb00 con 0x5559c3b17490 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a57e00640 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a5d200640 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ee9f70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0a8630 tx=0x7f9a4c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5124bc2b43e152 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a49600640 1 -- 172.21.15.39:0/717140671 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c020040 con 0x5559c3b17490 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a49600640 1 -- 172.21.15.39:0/717140671 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a4c0b2320 con 0x5559c3b17490 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.632+0000 7f9a49600640 1 -- 172.21.15.39:0/717140671 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0e2180 con 0x5559c3b17490 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.633+0000 7f9a6617d740 1 -- 172.21.15.39:0/717140671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 msgr2=0x5559c3ee9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.633+0000 7f9a6617d740 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ee9f70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0a8630 tx=0x7f9a4c010cb0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.633+0000 7f9a6617d740 1 -- 172.21.15.39:0/717140671 shutdown_connections 2024-07-04T08:18:55.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.633+0000 7f9a6617d740 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.633+0000 7f9a6617d740 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ee61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.633+0000 7f9a6617d740 1 --2- 172.21.15.39:0/717140671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ee9f70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.633+0000 7f9a6617d740 1 -- 172.21.15.39:0/717140671 >> 172.21.15.39:0/717140671 conn(0x5559c3b34740 msgr2=0x5559c3ef16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.633+0000 7f9a6617d740 1 -- 172.21.15.39:0/717140671 shutdown_connections 2024-07-04T08:18:55.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.634+0000 7f9a6617d740 1 -- 172.21.15.39:0/717140671 wait complete. 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.634+0000 7f9a6617d740 1 -- start start 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.634+0000 7f9a6617d740 1 Processor -- start 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ef4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a6617d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a6617d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b79a0 con 0x5559c3b17490 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a6617d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b72d0 con 0x5559c3b1f670 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a6617d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a581b4510 con 0x5559c3b40350 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a6617d740 10 client.0 _ll_register_callbacks cb 0x7f9a4c0a5a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a5d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a5dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a57e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ef4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a5dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34588/0 (socket says 172.21.15.39:34588) 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.635+0000 7f9a5dc00640 1 -- 172.21.15.39:0/2034199877 learned_addr learned my addr 172.21.15.39:0/2034199877 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a5d200640 1 -- 172.21.15.39:0/2034199877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 msgr2=0x5559c3ef4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a5d200640 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a5d200640 1 -- 172.21.15.39:0/2034199877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 msgr2=0x5559c3ef4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a5d200640 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ef4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a5d200640 1 -- 172.21.15.39:0/2034199877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559c3ef5750 con 0x5559c3b17490 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a5dc00640 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a57e00640 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ef4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:55.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a5d200640 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef85c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0a67e0 tx=0x7f9a4c0a89e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4376b6093fac527 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a3ea00640 1 -- 172.21.15.39:0/2034199877 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c087020 con 0x5559c3b17490 2024-07-04T08:18:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a3ea00640 1 -- 172.21.15.39:0/2034199877 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f9a4c0a4ee0 con 0x5559c3b17490 2024-07-04T08:18:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.636+0000 7f9a6617d740 1 -- 172.21.15.39:0/2034199877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a4c0b2320 con 0x5559c3b17490 2024-07-04T08:18:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a3ea00640 1 -- 172.21.15.39:0/2034199877 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9a4c020070 con 0x5559c3b17490 2024-07-04T08:18:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a6617d740 10 client.16245 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:55.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a6617d740 1 -- 172.21.15.39:0/2034199877 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559c3ef5750 con 0x5559c3b17490 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a3ea00640 1 -- 172.21.15.39:0/2034199877 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a4c0b2320 con 0x5559c3b17490 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a3ea00640 1 -- 172.21.15.39:0/2034199877 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9a4c0a50e0 con 0x5559c3b17490 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a6617d740 10 client.16245 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a6617d740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a6617d740 20 client.16245 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a6617d740 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5559c3ad4080 0x5559c3f6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a6617d740 4 client.16245 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.637+0000 7f9a6617d740 1 -- 172.21.15.39:0/2034199877 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5559c3e588b0 con 0x5559c3ad4080 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.638+0000 7f9a57e00640 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5559c3ad4080 0x5559c3f6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:55.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.638+0000 7f9a57e00640 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5559c3ad4080 0x5559c3f6da60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30afa289f2f24f21 server_cookie=eb97ae07b7e9e876 in_seq=0 out_seq=0 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.638+0000 7f9a3ea00640 10 client.16245 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.639+0000 7f9a3ea00640 1 -- 172.21.15.39:0/2034199877 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5559c3e588b0 con 0x5559c3ad4080 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.639+0000 7f9a3ea00640 10 client.16245 handle_command_reply: tid=0 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.639+0000 7f9a6617d740 1 client.16245 shutdown 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 -- 172.21.15.39:0/2034199877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 msgr2=0x5559c3ef85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef85c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0a67e0 tx=0x7f9a4c0a89e0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 -- 172.21.15.39:0/2034199877 shutdown_connections 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5559c3ad4080 0x5559c3f6da60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559c3b40350 0x5559c3ef4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559c3b1f670 0x5559c3ef4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 --2- 172.21.15.39:0/2034199877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559c3b17490 0x5559c3ef85c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a3ea00640 0 client.16245 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 -- 172.21.15.39:0/2034199877 >> 172.21.15.39:0/2034199877 conn(0x5559c3b34740 msgr2=0x5559c3ee5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 -- 172.21.15.39:0/2034199877 shutdown_connections 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 1 -- 172.21.15.39:0/2034199877 wait complete. 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.640+0000 7f9a6617d740 20 client.16245 trim_cache size 0 max 16384 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "d4da4c63-0332-45f6-88e1-284511563202", 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:18:55.642 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:18:55.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 -- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9a400b29e0 msgr2=0x7f9a400b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:55.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9a400b29e0 0x7f9a400b4ef0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9a500094e0 tx=0x7f9a5002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 -- 172.21.15.39:0/1633693242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a58109510 msgr2=0x7f9a5819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:55.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a58109510 0x7f9a5819fde0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00d950 tx=0x7f9a4c00de10 comp rx=0 tx=0).stop 2024-07-04T08:18:55.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 -- 172.21.15.39:0/1633693242 shutdown_connections 2024-07-04T08:18:55.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9a400b29e0 0x7f9a400b4ef0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a5810f7a0 0x7f9a581a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a5810ee50 0x7f9a581a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 --2- 172.21.15.39:0/1633693242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9a58109510 0x7f9a5819fde0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 -- 172.21.15.39:0/1633693242 >> 172.21.15.39:0/1633693242 conn(0x7f9a58078b10 msgr2=0x7f9a58079100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.641+0000 7f9a5e600640 1 -- 172.21.15.39:0/1633693242 shutdown_connections 2024-07-04T08:18:55.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:55.642+0000 7f9a5e600640 1 -- 172.21.15.39:0/1633693242 wait complete. 2024-07-04T08:18:56.586 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:18:57.057 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:57.195 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:56 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:18:57.195 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:56 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:57.195 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:56 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:18:57.195 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:56 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:57.196 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:56 smithi050 ceph-mon[36577]: pgmap v592: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:57.205 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:56 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:18:57.205 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:56 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:57.205 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:56 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:18:57.205 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:56 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:57.205 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:56 smithi039 ceph-mon[32062]: pgmap v592: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:57.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:56 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:18:57.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:56 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:57.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:56 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:18:57.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:56 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:18:57.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:56 smithi133 ceph-mon[37050]: pgmap v592: 129 pgs: 129 active+clean; 3.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-07-04T08:18:58.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:57 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:18:58.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:57 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:18:58.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:57 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:18:58.527 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 32.3 2024-07-04T08:18:58.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:18:59.211 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:18:58 smithi039 ceph-mon[32062]: pgmap v593: 129 pgs: 129 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-07-04T08:18:59.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:18:58 smithi133 ceph-mon[37050]: pgmap v593: 129 pgs: 129 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-07-04T08:18:59.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:18:58 smithi050 ceph-mon[36577]: pgmap v593: 129 pgs: 129 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-07-04T08:18:59.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.428+0000 7eff13400640 1 -- 172.21.15.39:0/1635470222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c10f750 msgr2=0x7eff0c111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.428+0000 7eff13400640 1 --2- 172.21.15.39:0/1635470222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c10f750 0x7eff0c111b70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7eff04009ca0 tx=0x7eff0402a950 comp rx=0 tx=0).stop 2024-07-04T08:18:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.429+0000 7eff13400640 1 -- 172.21.15.39:0/1635470222 shutdown_connections 2024-07-04T08:18:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.429+0000 7eff13400640 1 --2- 172.21.15.39:0/1635470222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c10f750 0x7eff0c111b70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.429+0000 7eff13400640 1 --2- 172.21.15.39:0/1635470222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff0c1024f0 0x7eff0c10f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.429+0000 7eff13400640 1 --2- 172.21.15.39:0/1635470222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0c101ba0 0x7eff0c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.429+0000 7eff13400640 1 -- 172.21.15.39:0/1635470222 >> 172.21.15.39:0/1635470222 conn(0x7eff0c078c80 msgr2=0x7eff0c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.429+0000 7eff13400640 1 -- 172.21.15.39:0/1635470222 shutdown_connections 2024-07-04T08:18:59.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.429+0000 7eff13400640 1 -- 172.21.15.39:0/1635470222 wait complete. 2024-07-04T08:18:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.430+0000 7eff13400640 1 Processor -- start 2024-07-04T08:18:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.430+0000 7eff13400640 1 -- start start 2024-07-04T08:18:59.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.430+0000 7eff13400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c101ba0 0x7eff0c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.430+0000 7eff13400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0c1024f0 0x7eff0c19c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.430+0000 7eff13400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff0c10f750 0x7eff0c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.430+0000 7eff13400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c115230 con 0x7eff0c101ba0 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.430+0000 7eff13400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1150b0 con 0x7eff0c10f750 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.430+0000 7eff13400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1153b0 con 0x7eff0c1024f0 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff10a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c101ba0 0x7eff0c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff10a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c101ba0 0x7eff0c19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41868/0 (socket says 172.21.15.39:41868) 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff10a00640 1 -- 172.21.15.39:0/599335207 learned_addr learned my addr 172.21.15.39:0/599335207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff11400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff0c10f750 0x7eff0c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff11400640 1 -- 172.21.15.39:0/599335207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0c1024f0 msgr2=0x7eff0c19c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff11400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0c1024f0 0x7eff0c19c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff11400640 1 -- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c101ba0 msgr2=0x7eff0c19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:59.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff11400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c101ba0 0x7eff0c19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff11400640 1 -- 172.21.15.39:0/599335207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c1a1b10 con 0x7eff0c10f750 2024-07-04T08:18:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.431+0000 7eff10a00640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c101ba0 0x7eff0c19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:18:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.432+0000 7eff11400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff0c10f750 0x7eff0c1a13e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7eff04005e30 tx=0x7eff04003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:59.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.432+0000 7eff09600640 1 -- 172.21.15.39:0/599335207 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04034040 con 0x7eff0c10f750 2024-07-04T08:18:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.432+0000 7eff09600640 1 -- 172.21.15.39:0/599335207 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff04003e50 con 0x7eff0c10f750 2024-07-04T08:18:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.432+0000 7eff13400640 1 -- 172.21.15.39:0/599335207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c1a1da0 con 0x7eff0c10f750 2024-07-04T08:18:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.432+0000 7eff09600640 1 -- 172.21.15.39:0/599335207 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff040045d0 con 0x7eff0c10f750 2024-07-04T08:18:59.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.433+0000 7eff13400640 1 -- 172.21.15.39:0/599335207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c1a8560 con 0x7eff0c10f750 2024-07-04T08:18:59.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.436+0000 7eff19968740 1 Processor -- start 2024-07-04T08:18:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.437+0000 7eff09600640 1 -- 172.21.15.39:0/599335207 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7eff04037020 con 0x7eff0c10f750 2024-07-04T08:18:59.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.437+0000 7eff19968740 1 -- start start 2024-07-04T08:18:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff19968740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b39b070 0x56438b747fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff19968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b3721b0 0x56438b7484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff19968740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b37a390 0x56438b748a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff19968740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56438b6b5ed0 con 0x56438b39b070 2024-07-04T08:18:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff19968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56438b6a5470 con 0x56438b3721b0 2024-07-04T08:18:59.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff19968740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56438b6a94b0 con 0x56438b37a390 2024-07-04T08:18:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff10a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b3721b0 0x56438b7484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff10a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b3721b0 0x56438b7484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38838/0 (socket says 172.21.15.39:38838) 2024-07-04T08:18:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff10a00640 1 -- 172.21.15.39:0/3598496462 learned_addr learned my addr 172.21.15.39:0/3598496462 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:59.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.438+0000 7eff0be00640 1 --2- 172.21.15.39:0/3598496462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b39b070 0x56438b747fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.439+0000 7eff10a00640 1 -- 172.21.15.39:0/3598496462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b37a390 msgr2=0x56438b748a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:18:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.439+0000 7eff10a00640 1 --2- 172.21.15.39:0/3598496462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b37a390 0x56438b748a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.439+0000 7eff10a00640 1 -- 172.21.15.39:0/3598496462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b39b070 msgr2=0x56438b747fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.439+0000 7eff10a00640 1 --2- 172.21.15.39:0/3598496462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b39b070 0x56438b747fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.439+0000 7eff10a00640 1 -- 172.21.15.39:0/3598496462 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438b63d640 con 0x56438b3721b0 2024-07-04T08:18:59.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.439+0000 7eff10a00640 1 --2- 172.21.15.39:0/3598496462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b3721b0 0x56438b7484f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7eff00009f20 tx=0x7eff0000e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b47d77c1e08faa68 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.440+0000 7eff09600640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efef40b29e0 0x7efef40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.440+0000 7efefe000640 1 -- 172.21.15.39:0/3598496462 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00017070 con 0x56438b3721b0 2024-07-04T08:18:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.440+0000 7efefe000640 1 -- 172.21.15.39:0/3598496462 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff00013920 con 0x56438b3721b0 2024-07-04T08:18:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.440+0000 7efefe000640 1 -- 172.21.15.39:0/3598496462 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff000129e0 con 0x56438b3721b0 2024-07-04T08:18:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.440+0000 7eff09600640 1 -- 172.21.15.39:0/599335207 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7eff04103670 con 0x7eff0c10f750 2024-07-04T08:18:59.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.440+0000 7eff0be00640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efef40b29e0 0x7efef40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff19968740 1 -- 172.21.15.39:0/3598496462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b3721b0 msgr2=0x56438b7484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff19968740 1 --2- 172.21.15.39:0/3598496462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b3721b0 0x56438b7484f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7eff00009f20 tx=0x7eff0000e660 comp rx=0 tx=0).stop 2024-07-04T08:18:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff0be00640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efef40b29e0 0x7efef40b4ef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff0c103520 tx=0x7efef8009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff19968740 1 -- 172.21.15.39:0/3598496462 shutdown_connections 2024-07-04T08:18:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff19968740 1 --2- 172.21.15.39:0/3598496462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b37a390 0x56438b748a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff19968740 1 --2- 172.21.15.39:0/3598496462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b3721b0 0x56438b7484f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff19968740 1 --2- 172.21.15.39:0/3598496462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b39b070 0x56438b747fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff19968740 1 -- 172.21.15.39:0/3598496462 >> 172.21.15.39:0/3598496462 conn(0x56438b38f460 msgr2=0x56438b745ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff19968740 1 -- 172.21.15.39:0/3598496462 shutdown_connections 2024-07-04T08:18:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.441+0000 7eff19968740 1 -- 172.21.15.39:0/3598496462 wait complete. 2024-07-04T08:18:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.442+0000 7eff19968740 1 -- start start 2024-07-04T08:18:59.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.442+0000 7eff19968740 1 Processor -- start 2024-07-04T08:18:59.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff19968740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b753b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff19968740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b754070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff19968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b75fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff19968740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56438b509700 con 0x56438b37a390 2024-07-04T08:18:59.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff19968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56438b64cba0 con 0x56438b39b070 2024-07-04T08:18:59.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff19968740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56438b756790 con 0x56438b3721b0 2024-07-04T08:18:59.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff19968740 10 client.0 _ll_register_callbacks cb 0x56438b69e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:59.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b754070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b754070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41892/0 (socket says 172.21.15.39:41892) 2024-07-04T08:18:59.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.443+0000 7eff0be00640 1 -- 172.21.15.39:0/3360330570 learned_addr learned my addr 172.21.15.39:0/3360330570 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:59.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.444+0000 7eff11400640 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b75fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.444+0000 7eff0be00640 1 -- 172.21.15.39:0/3360330570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 msgr2=0x56438b753b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:59.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.444+0000 7eff0be00640 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b753b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.444+0000 7eff0be00640 1 -- 172.21.15.39:0/3360330570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 msgr2=0x56438b75fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:59.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.444+0000 7eff0be00640 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b75fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.444+0000 7eff0be00640 1 -- 172.21.15.39:0/3360330570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438b406650 con 0x56438b37a390 2024-07-04T08:18:59.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.444+0000 7eff0be00640 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b754070 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efef800eba0 tx=0x7efef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27523bbe727fa87 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:59.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.444+0000 7efef2a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:18:59.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.445+0000 7efef2a00640 1 -- 172.21.15.39:0/3360330570 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef801cb20 con 0x56438b37a390 2024-07-04T08:18:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.445+0000 7eff19968740 1 -- 172.21.15.39:0/3360330570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff00013920 con 0x56438b37a390 2024-07-04T08:18:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.445+0000 7efef2a00640 1 -- 172.21.15.39:0/3360330570 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7efef801ccc0 con 0x56438b37a390 2024-07-04T08:18:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.445+0000 7efef2a00640 1 -- 172.21.15.39:0/3360330570 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efef8031050 con 0x56438b37a390 2024-07-04T08:18:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.445+0000 7eff19968740 10 client.16257 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.445+0000 7eff19968740 1 -- 172.21.15.39:0/3360330570 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56438b762810 con 0x56438b37a390 2024-07-04T08:18:59.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.446+0000 7efef2a00640 1 -- 172.21.15.39:0/3360330570 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff00013920 con 0x56438b37a390 2024-07-04T08:18:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.446+0000 7efef2a00640 1 -- 172.21.15.39:0/3360330570 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efef801d410 con 0x56438b37a390 2024-07-04T08:18:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.446+0000 7eff19968740 10 client.16257 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.446+0000 7eff19968740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.446+0000 7eff19968740 20 client.16257 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.446+0000 7eff19968740 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56438b3599f0 0x56438b765d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.446+0000 7eff19968740 4 client.16257 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:18:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.446+0000 7eff19968740 1 -- 172.21.15.39:0/3360330570 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56438b6b7e20 con 0x56438b3599f0 2024-07-04T08:18:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.447+0000 7eff10a00640 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56438b3599f0 0x56438b765d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.448+0000 7eff10a00640 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56438b3599f0 0x56438b765d80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1bea7bff0f7879b server_cookie=59d2188b794bdd8e in_seq=0 out_seq=0 2024-07-04T08:18:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.448+0000 7efef2a00640 10 client.16257 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.449+0000 7efef2a00640 1 -- 172.21.15.39:0/3360330570 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56438b6b7e20 con 0x56438b3599f0 2024-07-04T08:18:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.449+0000 7efef2a00640 10 client.16257 handle_command_reply: tid=0 2024-07-04T08:18:59.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.449+0000 7eff19968740 1 client.16257 shutdown 2024-07-04T08:18:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.449+0000 7eff19968740 1 -- 172.21.15.39:0/3360330570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 msgr2=0x56438b754070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.449+0000 7eff19968740 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b754070 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efef800eba0 tx=0x7efef80046c0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7eff19968740 1 -- 172.21.15.39:0/3360330570 shutdown_connections 2024-07-04T08:18:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7eff19968740 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56438b3599f0 0x56438b765d80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7eff19968740 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b754070 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7eff19968740 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b75fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7eff19968740 1 --2- 172.21.15.39:0/3360330570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b753b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7eff19968740 1 -- 172.21.15.39:0/3360330570 >> 172.21.15.39:0/3360330570 conn(0x56438b38f460 msgr2=0x56438b74b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7efef2a00640 0 client.16257 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7eff19968740 1 -- 172.21.15.39:0/3360330570 shutdown_connections 2024-07-04T08:18:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7eff19968740 1 -- 172.21.15.39:0/3360330570 wait complete. 2024-07-04T08:18:59.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.450+0000 7eff19968740 20 client.16257 trim_cache size 0 max 16384 2024-07-04T08:18:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:18:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.474+0000 7eff19968740 1 Processor -- start 2024-07-04T08:18:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.474+0000 7eff19968740 1 -- start start 2024-07-04T08:18:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff19968740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b73c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff19968740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b745e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff19968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b75f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff19968740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1adfd0 con 0x56438b37a390 2024-07-04T08:18:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff19968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1afb10 con 0x56438b39b070 2024-07-04T08:18:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff19968740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1ae6a0 con 0x56438b3721b0 2024-07-04T08:18:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b745e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b745e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41894/0 (socket says 172.21.15.39:41894) 2024-07-04T08:18:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff0be00640 1 -- 172.21.15.39:0/3109975398 learned_addr learned my addr 172.21.15.39:0/3109975398 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.475+0000 7eff11400640 1 --2- 172.21.15.39:0/3109975398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b75f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.476+0000 7eff0be00640 1 -- 172.21.15.39:0/3109975398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 msgr2=0x56438b73c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.476+0000 7eff0be00640 1 --2- 172.21.15.39:0/3109975398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b73c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.476+0000 7eff0be00640 1 -- 172.21.15.39:0/3109975398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 msgr2=0x56438b75f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.476+0000 7eff0be00640 1 --2- 172.21.15.39:0/3109975398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b75f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.476+0000 7eff0be00640 1 -- 172.21.15.39:0/3109975398 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438b746790 con 0x56438b37a390 2024-07-04T08:18:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.476+0000 7eff0be00640 1 --2- 172.21.15.39:0/3109975398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b745e00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efef8009b60 tx=0x7efef8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1804539828078ddf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.476+0000 7efefd600640 1 -- 172.21.15.39:0/3109975398 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef800e750 con 0x56438b37a390 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.476+0000 7efefd600640 1 -- 172.21.15.39:0/3109975398 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efef8009e80 con 0x56438b37a390 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.477+0000 7efefd600640 1 -- 172.21.15.39:0/3109975398 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef801c770 con 0x56438b37a390 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.477+0000 7eff19968740 1 -- 172.21.15.39:0/3109975398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 msgr2=0x56438b745e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.477+0000 7eff19968740 1 --2- 172.21.15.39:0/3109975398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b745e00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7efef8009b60 tx=0x7efef8009840 comp rx=0 tx=0).stop 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.478+0000 7eff19968740 1 -- 172.21.15.39:0/3109975398 shutdown_connections 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.478+0000 7eff19968740 1 --2- 172.21.15.39:0/3109975398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b745e00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.478+0000 7eff19968740 1 --2- 172.21.15.39:0/3109975398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b75f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.478+0000 7eff19968740 1 --2- 172.21.15.39:0/3109975398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b73c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.478+0000 7eff19968740 1 -- 172.21.15.39:0/3109975398 >> 172.21.15.39:0/3109975398 conn(0x56438b38f460 msgr2=0x56438b7456f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.478+0000 7eff19968740 1 -- 172.21.15.39:0/3109975398 shutdown_connections 2024-07-04T08:18:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.478+0000 7eff19968740 1 -- 172.21.15.39:0/3109975398 wait complete. 2024-07-04T08:18:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.478+0000 7eff19968740 1 -- start start 2024-07-04T08:18:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.479+0000 7eff19968740 1 Processor -- start 2024-07-04T08:18:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.479+0000 7eff19968740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b75c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.479+0000 7eff19968740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b7576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.479+0000 7eff19968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b757d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.479+0000 7eff19968740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1af440 con 0x56438b37a390 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.479+0000 7eff19968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1aed70 con 0x56438b39b070 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.479+0000 7eff19968740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c1abfd0 con 0x56438b3721b0 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.479+0000 7eff19968740 10 client.0 _ll_register_callbacks cb 0x7efef8009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff11400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b757d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b7576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff11400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b757d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38866/0 (socket says 172.21.15.39:38866) 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff11400640 1 -- 172.21.15.39:0/3663803448 learned_addr learned my addr 172.21.15.39:0/3663803448 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff10a00640 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b75c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff11400640 1 -- 172.21.15.39:0/3663803448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 msgr2=0x56438b75c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff11400640 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b75c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff11400640 1 -- 172.21.15.39:0/3663803448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 msgr2=0x56438b7576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:18:59.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff11400640 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b7576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff11400640 1 -- 172.21.15.39:0/3663803448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438b758470 con 0x56438b39b070 2024-07-04T08:18:59.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7eff11400640 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b757d00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7eff040ca6a0 tx=0x7eff0402a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=439e40fc87014a72 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:18:59.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.480+0000 7efef2a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:18:59.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.481+0000 7efef2a00640 1 -- 172.21.15.39:0/3663803448 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff040e3020 con 0x56438b39b070 2024-07-04T08:18:59.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.481+0000 7eff19968740 1 -- 172.21.15.39:0/3663803448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56438b6b38c0 con 0x56438b39b070 2024-07-04T08:18:59.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.481+0000 7efef2a00640 1 -- 172.21.15.39:0/3663803448 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7eff040b2070 con 0x56438b39b070 2024-07-04T08:18:59.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.481+0000 7efef2a00640 1 -- 172.21.15.39:0/3663803448 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff04030850 con 0x56438b39b070 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.481+0000 7eff19968740 10 client.26393 fetch_fsmap learned FSMap version 30 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.481+0000 7eff19968740 1 -- 172.21.15.39:0/3663803448 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56438b7ce600 con 0x56438b39b070 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.481+0000 7efef2a00640 1 -- 172.21.15.39:0/3663803448 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56438b6b38c0 con 0x56438b39b070 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.481+0000 7efef2a00640 1 -- 172.21.15.39:0/3663803448 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7eff040ca970 con 0x56438b39b070 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.482+0000 7eff19968740 10 client.26393 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.482+0000 7eff19968740 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.482+0000 7eff19968740 20 client.26393 populate_metadata read hostname 'smithi039' 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.482+0000 7eff19968740 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56438b32eda0 0x56438b7d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.482+0000 7eff19968740 4 client.26393 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.482+0000 7eff19968740 1 -- 172.21.15.39:0/3663803448 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56438b6b1000 con 0x56438b32eda0 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.482+0000 7eff10a00640 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56438b32eda0 0x56438b7d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff10a00640 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56438b32eda0 0x56438b7d0d90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0580f7869619259 server_cookie=6c9ad802a14d6fde in_seq=0 out_seq=0 2024-07-04T08:18:59.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7efef2a00640 10 client.26393 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7efef2a00640 1 -- 172.21.15.39:0/3663803448 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56438b6b1000 con 0x56438b32eda0 2024-07-04T08:18:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7efef2a00640 10 client.26393 handle_command_reply: tid=0 2024-07-04T08:18:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff19968740 1 client.26393 shutdown 2024-07-04T08:18:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff19968740 1 -- 172.21.15.39:0/3663803448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 msgr2=0x56438b757d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:59.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff19968740 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b757d00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7eff040ca6a0 tx=0x7eff0402a950 comp rx=0 tx=0).stop 2024-07-04T08:18:59.484 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:18:59.484 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:18:59.485 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:18:59.485 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:18:59.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff19968740 1 -- 172.21.15.39:0/3663803448 shutdown_connections 2024-07-04T08:18:59.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff19968740 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56438b32eda0 0x56438b7d0d90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff19968740 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56438b37a390 0x56438b7576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.485 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff19968740 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56438b39b070 0x56438b757d00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff19968740 1 --2- 172.21.15.39:0/3663803448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56438b3721b0 0x56438b75c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.483+0000 7eff19968740 1 -- 172.21.15.39:0/3663803448 >> 172.21.15.39:0/3663803448 conn(0x56438b38f460 msgr2=0x56438b748fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.484+0000 7efef2a00640 0 client.26393 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.484+0000 7eff19968740 1 -- 172.21.15.39:0/3663803448 shutdown_connections 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.484+0000 7eff19968740 1 -- 172.21.15.39:0/3663803448 wait complete. 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.484+0000 7eff19968740 20 client.26393 trim_cache size 0 max 16384 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 -- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efef40b29e0 msgr2=0x7efef40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efef40b29e0 0x7efef40b4ef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff0c103520 tx=0x7efef8009680 comp rx=0 tx=0).stop 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 -- 172.21.15.39:0/599335207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff0c10f750 msgr2=0x7eff0c1a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff0c10f750 0x7eff0c1a13e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7eff04005e30 tx=0x7eff04003ce0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 -- 172.21.15.39:0/599335207 shutdown_connections 2024-07-04T08:18:59.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7efef40b29e0 0x7efef40b4ef0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7eff0c10f750 0x7eff0c1a13e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0c1024f0 0x7eff0c19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 --2- 172.21.15.39:0/599335207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff0c101ba0 0x7eff0c19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:18:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 -- 172.21.15.39:0/599335207 >> 172.21.15.39:0/599335207 conn(0x7eff0c078c80 msgr2=0x7eff0c1081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:18:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 -- 172.21.15.39:0/599335207 shutdown_connections 2024-07-04T08:18:59.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:18:59.485+0000 7eff13400640 1 -- 172.21.15.39:0/599335207 wait complete. 2024-07-04T08:19:00.614 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:19:00.614 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:19:01.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:00 smithi039 ceph-mon[32062]: pgmap v594: 129 pgs: 129 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-07-04T08:19:01.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:00 smithi133 ceph-mon[37050]: pgmap v594: 129 pgs: 129 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-07-04T08:19:01.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:00 smithi050 ceph-mon[36577]: pgmap v594: 129 pgs: 129 active+clean; 3.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-07-04T08:19:01.729 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:02.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.625+0000 7f2cbbe00640 1 -- 172.21.15.39:0/3704515327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc109510 msgr2=0x7f2cbc109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:02.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.625+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/3704515327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc109510 0x7f2cbc109920 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009be0 tx=0x7f2cac02a6c0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.626+0000 7f2cbbe00640 1 -- 172.21.15.39:0/3704515327 shutdown_connections 2024-07-04T08:19:02.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.626+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/3704515327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc10ee50 0x7f2cbc10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.626+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/3704515327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f7a0 0x7f2cbc115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.626+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/3704515327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc109510 0x7f2cbc109920 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.626+0000 7f2cbbe00640 1 -- 172.21.15.39:0/3704515327 >> 172.21.15.39:0/3704515327 conn(0x7f2cbc078b10 msgr2=0x7f2cbc107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:02.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.626+0000 7f2cbbe00640 1 -- 172.21.15.39:0/3704515327 shutdown_connections 2024-07-04T08:19:02.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.626+0000 7f2cbbe00640 1 -- 172.21.15.39:0/3704515327 wait complete. 2024-07-04T08:19:02.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.627+0000 7f2cbbe00640 1 Processor -- start 2024-07-04T08:19:02.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.627+0000 7f2cbbe00640 1 -- start start 2024-07-04T08:19:02.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc109510 0x7f2cbc19fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:02.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc10ee50 0x7f2cbc1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:02.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f7a0 0x7f2cbc1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:02.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbbe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc119470 con 0x7f2cbc109510 2024-07-04T08:19:02.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbbe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc1192f0 con 0x7f2cbc10f7a0 2024-07-04T08:19:02.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbbe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc1195f0 con 0x7f2cbc10ee50 2024-07-04T08:19:02.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc109510 0x7f2cbc19fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:02.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbaa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc109510 0x7f2cbc19fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54762/0 (socket says 172.21.15.39:54762) 2024-07-04T08:19:02.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbaa00640 1 -- 172.21.15.39:0/2029450693 learned_addr learned my addr 172.21.15.39:0/2029450693 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:02.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cb2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc10ee50 0x7f2cbc1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:02.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.628+0000 7f2cbb400640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f7a0 0x7f2cbc1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:02.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cb2000640 1 -- 172.21.15.39:0/2029450693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f7a0 msgr2=0x7f2cbc1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:02.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cb2000640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f7a0 0x7f2cbc1a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cb2000640 1 -- 172.21.15.39:0/2029450693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc109510 msgr2=0x7f2cbc19fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:02.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cb2000640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc109510 0x7f2cbc19fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cb2000640 1 -- 172.21.15.39:0/2029450693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc1a5df0 con 0x7f2cbc10ee50 2024-07-04T08:19:02.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cb2000640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc10ee50 0x7f2cbc1a0420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc06bb00 tx=0x7f2ca400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:02.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2029450693 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400ccd0 con 0x7f2cbc10ee50 2024-07-04T08:19:02.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2029450693 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca40035f0 con 0x7f2cbc10ee50 2024-07-04T08:19:02.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc1a60e0 con 0x7f2cbc10ee50 2024-07-04T08:19:02.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.629+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2029450693 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4010690 con 0x7f2cbc10ee50 2024-07-04T08:19:02.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.630+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc1ac840 con 0x7f2cbc10ee50 2024-07-04T08:19:02.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.631+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc109920 con 0x7f2cbc10ee50 2024-07-04T08:19:02.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.633+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2029450693 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2ca4003a60 con 0x7f2cbc10ee50 2024-07-04T08:19:02.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.636+0000 7f2cb3e00640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b29e0 0x7f2ca00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:02.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.636+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2029450693 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2ca40e2b50 con 0x7f2cbc10ee50 2024-07-04T08:19:02.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.636+0000 7f2cbaa00640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b29e0 0x7f2ca00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:02.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.637+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2029450693 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f2ca4009510 con 0x7f2cbc10ee50 2024-07-04T08:19:02.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.637+0000 7f2cbaa00640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b29e0 0x7f2ca00b4ef0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009310 tx=0x7f2cac009280 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:02.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.893+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cbc1a0fb0 con 0x7f2cbc10ee50 2024-07-04T08:19:02.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.895+0000 7f2cb3e00640 1 -- 172.21.15.39:0/2029450693 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f2ca40aa1a0 con 0x7f2cbc10ee50 2024-07-04T08:19:02.894 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:19:02.894 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:19:02.895 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:19:02.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b29e0 msgr2=0x7f2ca00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:02.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b29e0 0x7f2ca00b4ef0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2cac009310 tx=0x7f2cac009280 comp rx=0 tx=0).stop 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc10ee50 msgr2=0x7f2cbc1a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc10ee50 0x7f2cbc1a0420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc06bb00 tx=0x7f2ca400ee50 comp rx=0 tx=0).stop 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 shutdown_connections 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2ca00b29e0 0x7f2ca00b4ef0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2cbc10f7a0 0x7f2cbc1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cbc10ee50 0x7f2cbc1a0420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 --2- 172.21.15.39:0/2029450693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2cbc109510 0x7f2cbc19fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 >> 172.21.15.39:0/2029450693 conn(0x7f2cbc078b10 msgr2=0x7f2cbc0794a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 shutdown_connections 2024-07-04T08:19:02.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:02.898+0000 7f2cbbe00640 1 -- 172.21.15.39:0/2029450693 wait complete. 2024-07-04T08:19:03.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:02 smithi133 ceph-mon[37050]: pgmap v595: 129 pgs: 129 active+clean; 3.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-07-04T08:19:03.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:02 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2029450693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:03.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:02 smithi039 ceph-mon[32062]: pgmap v595: 129 pgs: 129 active+clean; 3.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-07-04T08:19:03.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:02 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2029450693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:03.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:02 smithi050 ceph-mon[36577]: pgmap v595: 129 pgs: 129 active+clean; 3.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-07-04T08:19:03.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:02 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2029450693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:04.148 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 152.3 --await-for 45.0' 2024-07-04T08:19:04.148 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 152.3 --await-for 45.0 2024-07-04T08:19:04.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.704+0000 7fe35e000640 1 -- 172.21.15.39:0/3640265939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358109190 msgr2=0x7fe35810f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:04.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.704+0000 7fe35e000640 1 --2- 172.21.15.39:0/3640265939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358109190 0x7fe35810f9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe350009ca0 tx=0x7fe35002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:19:04.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.704+0000 7fe35e000640 1 -- 172.21.15.39:0/3640265939 shutdown_connections 2024-07-04T08:19:04.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.704+0000 7fe35e000640 1 --2- 172.21.15.39:0/3640265939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358109190 0x7fe35810f9c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:04.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.704+0000 7fe35e000640 1 --2- 172.21.15.39:0/3640265939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe358108840 0x7fe358108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:04.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.704+0000 7fe35e000640 1 --2- 172.21.15.39:0/3640265939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe358102f00 0x7fe358103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:04.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.704+0000 7fe35e000640 1 -- 172.21.15.39:0/3640265939 >> 172.21.15.39:0/3640265939 conn(0x7fe3580fe990 msgr2=0x7fe358100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:04.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.704+0000 7fe35e000640 1 -- 172.21.15.39:0/3640265939 shutdown_connections 2024-07-04T08:19:04.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.704+0000 7fe35e000640 1 -- 172.21.15.39:0/3640265939 wait complete. 2024-07-04T08:19:04.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.705+0000 7fe35e000640 1 Processor -- start 2024-07-04T08:19:04.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.706+0000 7fe35e000640 1 -- start start 2024-07-04T08:19:04.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.706+0000 7fe35e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358102f00 0x7fe358074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:04.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.706+0000 7fe35e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe358108840 0x7fe3580751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:04.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.706+0000 7fe35e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe358109190 0x7fe3580756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:04.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.706+0000 7fe35e000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe358112e50 con 0x7fe358102f00 2024-07-04T08:19:04.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.706+0000 7fe35e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe358112cd0 con 0x7fe358108840 2024-07-04T08:19:04.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.706+0000 7fe35e000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe358112fd0 con 0x7fe358109190 2024-07-04T08:19:04.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.706+0000 7fe357e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe358108840 0x7fe3580751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:04.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.707+0000 7fe35cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358102f00 0x7fe358074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:04.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.707+0000 7fe357e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe358108840 0x7fe3580751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37678/0 (socket says 172.21.15.39:37678) 2024-07-04T08:19:04.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.707+0000 7fe357e00640 1 -- 172.21.15.39:0/2437878553 learned_addr learned my addr 172.21.15.39:0/2437878553 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:04.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.707+0000 7fe35d600640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe358109190 0x7fe3580756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:04.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.707+0000 7fe35cc00640 1 -- 172.21.15.39:0/2437878553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe358109190 msgr2=0x7fe3580756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:04.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.707+0000 7fe35cc00640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe358109190 0x7fe3580756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:04.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.707+0000 7fe35cc00640 1 -- 172.21.15.39:0/2437878553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe358108840 msgr2=0x7fe3580751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:04.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.707+0000 7fe35cc00640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe358108840 0x7fe3580751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:04.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.707+0000 7fe35cc00640 1 -- 172.21.15.39:0/2437878553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3581a7aa0 con 0x7fe358102f00 2024-07-04T08:19:04.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.708+0000 7fe35d600640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe358109190 0x7fe3580756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:19:04.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.708+0000 7fe35cc00640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358102f00 0x7fe358074c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00e9f0 tx=0x7fe34c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:04.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.708+0000 7fe355600640 1 -- 172.21.15.39:0/2437878553 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00ccc0 con 0x7fe358102f00 2024-07-04T08:19:04.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.708+0000 7fe355600640 1 -- 172.21.15.39:0/2437878553 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe34c0108e0 con 0x7fe358102f00 2024-07-04T08:19:04.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.708+0000 7fe355600640 1 -- 172.21.15.39:0/2437878553 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c018880 con 0x7fe358102f00 2024-07-04T08:19:04.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.709+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3581a7d90 con 0x7fe358102f00 2024-07-04T08:19:04.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.709+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3581a82a0 con 0x7fe358102f00 2024-07-04T08:19:04.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.710+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe358104180 con 0x7fe358102f00 2024-07-04T08:19:04.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.713+0000 7fe355600640 1 -- 172.21.15.39:0/2437878553 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe34c010430 con 0x7fe358102f00 2024-07-04T08:19:04.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.715+0000 7fe355600640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3400b29e0 0x7fe3400b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:04.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.715+0000 7fe355600640 1 -- 172.21.15.39:0/2437878553 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fe34c0e2d10 con 0x7fe358102f00 2024-07-04T08:19:04.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.716+0000 7fe357e00640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3400b29e0 0x7fe3400b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:04.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.716+0000 7fe357e00640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3400b29e0 0x7fe3400b4ef0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe3581083d0 tx=0x7fe344005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:04.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:04.722+0000 7fe355600640 1 -- 172.21.15.39:0/2437878553 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe34c0e8020 con 0x7fe358102f00 2024-07-04T08:19:04.722 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:05.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.002+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe358103310 con 0x7fe358102f00 2024-07-04T08:19:05.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.003+0000 7fe355600640 1 -- 172.21.15.39:0/2437878553 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fe34c0aa360 con 0x7fe358102f00 2024-07-04T08:19:05.002 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:19:05.003 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:19:05.003 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:19:05.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3400b29e0 msgr2=0x7fe3400b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:05.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3400b29e0 0x7fe3400b4ef0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe3581083d0 tx=0x7fe344005f90 comp rx=0 tx=0).stop 2024-07-04T08:19:05.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358102f00 msgr2=0x7fe358074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:05.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358102f00 0x7fe358074c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00e9f0 tx=0x7fe34c00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:19:05.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 shutdown_connections 2024-07-04T08:19:05.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3400b29e0 0x7fe3400b4ef0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe358109190 0x7fe3580756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe358108840 0x7fe3580751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 --2- 172.21.15.39:0/2437878553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe358102f00 0x7fe358074c60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 >> 172.21.15.39:0/2437878553 conn(0x7fe3580fe990 msgr2=0x7fe358100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 shutdown_connections 2024-07-04T08:19:05.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:05.006+0000 7fe35e000640 1 -- 172.21.15.39:0/2437878553 wait complete. 2024-07-04T08:19:05.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:04 smithi133 ceph-mon[37050]: pgmap v596: 129 pgs: 129 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-07-04T08:19:05.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:04 smithi039 ceph-mon[32062]: pgmap v596: 129 pgs: 129 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-07-04T08:19:05.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:04 smithi050 ceph-mon[36577]: pgmap v596: 129 pgs: 129 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-07-04T08:19:06.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:05 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2437878553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:06.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:05 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2437878553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:06.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:19:06.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:05 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2437878553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:06.922 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:07.048 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:06 smithi039 ceph-mon[32062]: pgmap v597: 129 pgs: 129 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-07-04T08:19:07.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:06 smithi133 ceph-mon[37050]: pgmap v597: 129 pgs: 129 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-07-04T08:19:07.393 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:06 smithi050 ceph-mon[36577]: pgmap v597: 129 pgs: 129 active+clean; 3.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-07-04T08:19:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.794+0000 7ff8e7e00640 1 -- 172.21.15.39:0/3207685594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8e8104690 msgr2=0x7ff8e810cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:07.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.794+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/3207685594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8e8104690 0x7ff8e810cbe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009900 tx=0x7ff8d402a410 comp rx=0 tx=0).stop 2024-07-04T08:19:07.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.794+0000 7ff8e7e00640 1 -- 172.21.15.39:0/3207685594 shutdown_connections 2024-07-04T08:19:07.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.794+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/3207685594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8e8104690 0x7ff8e810cbe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.794+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/3207685594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8e810d1a0 0x7ff8e810f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.794+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/3207685594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8e8103d40 0x7ff8e8104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.794+0000 7ff8e7e00640 1 -- 172.21.15.39:0/3207685594 >> 172.21.15.39:0/3207685594 conn(0x7ff8e80fd880 msgr2=0x7ff8e80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:07.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.794+0000 7ff8e7e00640 1 -- 172.21.15.39:0/3207685594 shutdown_connections 2024-07-04T08:19:07.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.794+0000 7ff8e7e00640 1 -- 172.21.15.39:0/3207685594 wait complete. 2024-07-04T08:19:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.795+0000 7ff8e7e00640 1 Processor -- start 2024-07-04T08:19:07.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.795+0000 7ff8e7e00640 1 -- start start 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8e8103d40 0x7ff8e819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8e8104690 0x7ff8e819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8e810d1a0 0x7ff8e81a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e8112dc0 con 0x7ff8e8104690 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e7e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e8112c40 con 0x7ff8e8103d40 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e8112f40 con 0x7ff8e810d1a0 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8e8103d40 0x7ff8e819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8e8103d40 0x7ff8e819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37682/0 (socket says 172.21.15.39:37682) 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8e8104690 0x7ff8e819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e6a00640 1 -- 172.21.15.39:0/426669181 learned_addr learned my addr 172.21.15.39:0/426669181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.796+0000 7ff8e7400640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8e810d1a0 0x7ff8e81a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8e6a00640 1 -- 172.21.15.39:0/426669181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8e810d1a0 msgr2=0x7ff8e81a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8e6a00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8e810d1a0 0x7ff8e81a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8e6a00640 1 -- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8e8104690 msgr2=0x7ff8e819bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8e6a00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8e8104690 0x7ff8e819bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8e6a00640 1 -- 172.21.15.39:0/426669181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e81a1980 con 0x7ff8e8103d40 2024-07-04T08:19:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8e6000640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8e8104690 0x7ff8e819bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8e6a00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8e8103d40 0x7ff8e819ba10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800e9f0 tx=0x7ff8d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:07.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8df400640 1 -- 172.21.15.39:0/426669181 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800cd80 con 0x7ff8e8103d40 2024-07-04T08:19:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8e7e00640 1 -- 172.21.15.39:0/426669181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e81a1c70 con 0x7ff8e8103d40 2024-07-04T08:19:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8df400640 1 -- 172.21.15.39:0/426669181 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d80108e0 con 0x7ff8e8103d40 2024-07-04T08:19:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8df400640 1 -- 172.21.15.39:0/426669181 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8018980 con 0x7ff8e8103d40 2024-07-04T08:19:07.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.797+0000 7ff8e7e00640 1 -- 172.21.15.39:0/426669181 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e81a83d0 con 0x7ff8e8103d40 2024-07-04T08:19:07.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.801+0000 7ff8f4105740 1 Processor -- start 2024-07-04T08:19:07.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.801+0000 7ff8f4105740 1 -- start start 2024-07-04T08:19:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.801+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d664580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.801+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d664ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.801+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d665000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.802+0000 7ff8f4105740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56418d5d9ce0 con 0x56418d296980 2024-07-04T08:19:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.802+0000 7ff8f4105740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56418d5c9280 con 0x56418d2bf840 2024-07-04T08:19:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.802+0000 7ff8f4105740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56418d5cd2c0 con 0x56418d29eb60 2024-07-04T08:19:07.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.802+0000 7ff8df400640 1 -- 172.21.15.39:0/426669181 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff8d8010430 con 0x7ff8e8103d40 2024-07-04T08:19:07.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.802+0000 7ff8e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d664580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.802+0000 7ff8e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d664580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37692/0 (socket says 172.21.15.39:37692) 2024-07-04T08:19:07.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.802+0000 7ff8e7400640 1 -- 172.21.15.39:0/1572395384 learned_addr learned my addr 172.21.15.39:0/1572395384 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:07.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.802+0000 7ff8e6000640 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d665000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.802+0000 7ff8e6a00640 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d664ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8e7400640 1 -- 172.21.15.39:0/1572395384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 msgr2=0x56418d665000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8e7400640 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d665000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8e7400640 1 -- 172.21.15.39:0/1572395384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 msgr2=0x56418d664ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8e7400640 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d664ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8e7400640 1 -- 172.21.15.39:0/1572395384 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418d5606f0 con 0x56418d2bf840 2024-07-04T08:19:07.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8e6a00640 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d664ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8e7400640 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d664580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009fc0 tx=0x7ff8e0010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8017ac6b244c095 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8d3e00640 1 -- 172.21.15.39:0/1572395384 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0003b70 con 0x56418d2bf840 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8d3e00640 1 -- 172.21.15.39:0/1572395384 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e0003d10 con 0x56418d2bf840 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.803+0000 7ff8d3e00640 1 -- 172.21.15.39:0/1572395384 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0011970 con 0x56418d2bf840 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.804+0000 7ff8df400640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8cc0b29d0 0x7ff8cc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.804+0000 7ff8df400640 1 -- 172.21.15.39:0/426669181 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff8d80e2d20 con 0x7ff8e8103d40 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.804+0000 7ff8e6a00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8cc0b29d0 0x7ff8cc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.804+0000 7ff8f4105740 1 -- 172.21.15.39:0/1572395384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 msgr2=0x56418d664580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.804+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d664580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009fc0 tx=0x7ff8e0010860 comp rx=0 tx=0).stop 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.804+0000 7ff8f4105740 1 -- 172.21.15.39:0/1572395384 shutdown_connections 2024-07-04T08:19:07.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.804+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d664ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.804+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d665000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.804+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1572395384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d664580 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.805+0000 7ff8f4105740 1 -- 172.21.15.39:0/1572395384 >> 172.21.15.39:0/1572395384 conn(0x56418d2b3990 msgr2=0x56418d662470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:07.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.805+0000 7ff8f4105740 1 -- 172.21.15.39:0/1572395384 shutdown_connections 2024-07-04T08:19:07.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.805+0000 7ff8f4105740 1 -- 172.21.15.39:0/1572395384 wait complete. 2024-07-04T08:19:07.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.805+0000 7ff8e6a00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8cc0b29d0 0x7ff8cc0b4ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x56418d663200 tx=0x7ff8d8000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:07.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.805+0000 7ff8f4105740 1 -- start start 2024-07-04T08:19:07.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.806+0000 7ff8f4105740 1 Processor -- start 2024-07-04T08:19:07.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.806+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d296980 0x56418d678fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.806+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d29eb60 0x56418d6795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.806+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.806+0000 7ff8f4105740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56418d42d7a0 con 0x56418d2bf840 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.806+0000 7ff8f4105740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56418d5706d0 con 0x56418d29eb60 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.806+0000 7ff8f4105740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56418d672d60 con 0x56418d296980 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.806+0000 7ff8f4105740 10 client.0 _ll_register_callbacks cb 0x56418d2d7090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d296980 0x56418d678fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54812/0 (socket says 172.21.15.39:54812) 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d29eb60 0x56418d6795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e7400640 1 -- 172.21.15.39:0/1980911528 learned_addr learned my addr 172.21.15.39:0/1980911528 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:07.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e6000640 1 -- 172.21.15.39:0/1980911528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d296980 msgr2=0x56418d678fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e6000640 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d296980 0x56418d678fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e6000640 1 -- 172.21.15.39:0/1980911528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 msgr2=0x56418d67e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e6000640 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e6000640 1 -- 172.21.15.39:0/1980911528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418d5611d0 con 0x56418d29eb60 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.807+0000 7ff8e7400640 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.808+0000 7ff8e6000640 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d29eb60 0x56418d6795c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff8d402a930 tx=0x7ff8d40316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=218079d6ed7ca12a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.808+0000 7ff8d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.808+0000 7ff8d0c00640 1 -- 172.21.15.39:0/1980911528 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4035040 con 0x56418d29eb60 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.808+0000 7ff8f4105740 1 -- 172.21.15.39:0/1980911528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e0003d10 con 0x56418d29eb60 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.808+0000 7ff8d0c00640 1 -- 172.21.15.39:0/1980911528 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff8d4004240 con 0x56418d29eb60 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.808+0000 7ff8d0c00640 1 -- 172.21.15.39:0/1980911528 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8d404f050 con 0x56418d29eb60 2024-07-04T08:19:07.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.808+0000 7ff8f4105740 10 client.26417 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.808+0000 7ff8f4105740 1 -- 172.21.15.39:0/1980911528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56418d339b90 con 0x56418d29eb60 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.809+0000 7ff8d0c00640 1 -- 172.21.15.39:0/1980911528 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e0003d10 con 0x56418d29eb60 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.809+0000 7ff8d0c00640 1 -- 172.21.15.39:0/1980911528 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff8d4004890 con 0x56418d29eb60 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.809+0000 7ff8f4105740 10 client.26417 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.809+0000 7ff8f4105740 10 client.26417 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.809+0000 7ff8f4105740 20 client.26417 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.809+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56418d27e1c0 0x56418d68a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.809+0000 7ff8f4105740 4 client.26417 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.809+0000 7ff8f4105740 1 -- 172.21.15.39:0/1980911528 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56418d5dbc30 con 0x56418d27e1c0 2024-07-04T08:19:07.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.810+0000 7ff8e7400640 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56418d27e1c0 0x56418d68a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.810+0000 7ff8e7400640 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56418d27e1c0 0x56418d68a8a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784df8732caeaf89 server_cookie=36077248eda31ed2 in_seq=0 out_seq=0 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.810+0000 7ff8d0c00640 10 client.26417 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.811+0000 7ff8d0c00640 1 -- 172.21.15.39:0/1980911528 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56418d5dbc30 con 0x56418d27e1c0 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.811+0000 7ff8d0c00640 10 client.26417 handle_command_reply: tid=0 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 client.26417 shutdown 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 -- 172.21.15.39:0/1980911528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d29eb60 msgr2=0x56418d6795c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d29eb60 0x56418d6795c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff8d402a930 tx=0x7ff8d40316d0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 -- 172.21.15.39:0/1980911528 shutdown_connections 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56418d27e1c0 0x56418d68a8a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d29eb60 0x56418d6795c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 --2- 172.21.15.39:0/1980911528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d296980 0x56418d678fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8d0c00640 0 client.26417 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:07.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 -- 172.21.15.39:0/1980911528 >> 172.21.15.39:0/1980911528 conn(0x56418d2b3990 msgr2=0x56418d667760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:07.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 -- 172.21.15.39:0/1980911528 shutdown_connections 2024-07-04T08:19:07.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.812+0000 7ff8f4105740 1 -- 172.21.15.39:0/1980911528 wait complete. 2024-07-04T08:19:07.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.813+0000 7ff8f4105740 20 client.26417 trim_cache size 0 max 16384 2024-07-04T08:19:07.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.832+0000 7ff8f4105740 1 Processor -- start 2024-07-04T08:19:07.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.833+0000 7ff8f4105740 1 -- start start 2024-07-04T08:19:07.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.833+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d672080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.833+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d660af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.833+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d661030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.833+0000 7ff8f4105740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81ade70 con 0x56418d296980 2024-07-04T08:19:07.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.833+0000 7ff8f4105740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81af9b0 con 0x56418d2bf840 2024-07-04T08:19:07.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.833+0000 7ff8f4105740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81ae540 con 0x56418d29eb60 2024-07-04T08:19:07.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.833+0000 7ff8e7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d672080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d660af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d672080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54820/0 (socket says 172.21.15.39:54820) 2024-07-04T08:19:07.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e7400640 1 -- 172.21.15.39:0/43413008 learned_addr learned my addr 172.21.15.39:0/43413008 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:07.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e7400640 1 --2- 172.21.15.39:0/43413008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d661030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e6000640 1 -- 172.21.15.39:0/43413008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 msgr2=0x56418d661030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:07.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e6000640 1 --2- 172.21.15.39:0/43413008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d661030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e6000640 1 -- 172.21.15.39:0/43413008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 msgr2=0x56418d672080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:07.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e6000640 1 --2- 172.21.15.39:0/43413008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d672080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e6000640 1 -- 172.21.15.39:0/43413008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418d67a0a0 con 0x56418d29eb60 2024-07-04T08:19:07.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e7400640 1 --2- 172.21.15.39:0/43413008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d672080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:07.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.834+0000 7ff8e6000640 1 --2- 172.21.15.39:0/43413008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d660af0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff8d402a450 tx=0x7ff8d4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ffb99a65e92d71 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:07.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.835+0000 7ff8d3400640 1 -- 172.21.15.39:0/43413008 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4057070 con 0x56418d29eb60 2024-07-04T08:19:07.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.835+0000 7ff8d3400640 1 -- 172.21.15.39:0/43413008 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d4005d90 con 0x56418d29eb60 2024-07-04T08:19:07.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.835+0000 7ff8d3400640 1 -- 172.21.15.39:0/43413008 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4000e60 con 0x56418d29eb60 2024-07-04T08:19:07.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 -- 172.21.15.39:0/43413008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 msgr2=0x56418d660af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:07.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 --2- 172.21.15.39:0/43413008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d660af0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff8d402a450 tx=0x7ff8d4005fd0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 -- 172.21.15.39:0/43413008 shutdown_connections 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 --2- 172.21.15.39:0/43413008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d2bf840 0x56418d661030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 --2- 172.21.15.39:0/43413008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d660af0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 --2- 172.21.15.39:0/43413008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d296980 0x56418d672080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 -- 172.21.15.39:0/43413008 >> 172.21.15.39:0/43413008 conn(0x56418d2b3990 msgr2=0x56418d667680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 -- 172.21.15.39:0/43413008 shutdown_connections 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 -- 172.21.15.39:0/43413008 wait complete. 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.836+0000 7ff8f4105740 1 -- start start 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.837+0000 7ff8f4105740 1 Processor -- start 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.837+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d296980 0x56418d685480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.837+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d67f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.837+0000 7ff8f4105740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.837+0000 7ff8f4105740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81af2e0 con 0x56418d2bf840 2024-07-04T08:19:07.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.837+0000 7ff8f4105740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81aec10 con 0x56418d296980 2024-07-04T08:19:07.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.837+0000 7ff8f4105740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e81abeb0 con 0x56418d29eb60 2024-07-04T08:19:07.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.837+0000 7ff8f4105740 10 client.0 _ll_register_callbacks cb 0x7ff8d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:07.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d296980 0x56418d685480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d67f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d296980 0x56418d685480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37728/0 (socket says 172.21.15.39:37728) 2024-07-04T08:19:07.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e7400640 1 -- 172.21.15.39:0/888925769 learned_addr learned my addr 172.21.15.39:0/888925769 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e7400640 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e6000640 1 -- 172.21.15.39:0/888925769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d296980 msgr2=0x56418d685480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e6000640 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d296980 0x56418d685480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e6000640 1 -- 172.21.15.39:0/888925769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 msgr2=0x56418d67f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e6000640 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e6000640 1 -- 172.21.15.39:0/888925769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418d684200 con 0x56418d29eb60 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.838+0000 7ff8e7400640 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d296980 0x56418d685480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.839+0000 7ff8e6000640 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d67f1b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4004e00 tx=0x7ff8d40316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8dbd4c08d6d5faa server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.839+0000 7ff8d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:19:07.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.839+0000 7ff8d0c00640 1 -- 172.21.15.39:0/888925769 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4057070 con 0x56418d29eb60 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.839+0000 7ff8d0c00640 1 -- 172.21.15.39:0/888925769 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff8d4004980 con 0x56418d29eb60 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.839+0000 7ff8f4105740 1 -- 172.21.15.39:0/888925769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d4005d90 con 0x56418d29eb60 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.839+0000 7ff8d0c00640 1 -- 172.21.15.39:0/888925769 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8d406a050 con 0x56418d29eb60 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.839+0000 7ff8f4105740 10 client.26350 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.839+0000 7ff8f4105740 1 -- 172.21.15.39:0/888925769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56418d6f3180 con 0x56418d29eb60 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.840+0000 7ff8d0c00640 1 -- 172.21.15.39:0/888925769 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d4005d90 con 0x56418d29eb60 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.840+0000 7ff8d0c00640 1 -- 172.21.15.39:0/888925769 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff8d4003680 con 0x56418d29eb60 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.840+0000 7ff8f4105740 10 client.26350 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.840+0000 7ff8f4105740 10 client.26350 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.840+0000 7ff8f4105740 20 client.26350 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.840+0000 7ff8f4105740 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56418d253570 0x56418d6f5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.840+0000 7ff8f4105740 4 client.26350 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.3, "await_for": 45.0}] 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.840+0000 7ff8f4105740 1 -- 172.21.15.39:0/888925769 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.3, "await_for": 45.0}) -- 0x56418d5d76d0 con 0x56418d253570 2024-07-04T08:19:07.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.840+0000 7ff8e7400640 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56418d253570 0x56418d6f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:07.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.841+0000 7ff8e7400640 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56418d253570 0x56418d6f5890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16530a627251c086 server_cookie=a76de0959fb224a5 in_seq=0 out_seq=0 2024-07-04T08:19:07.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.841+0000 7ff8d0c00640 10 client.26350 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.852+0000 7ff8d0c00640 1 -- 172.21.15.39:0/888925769 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56418d5d76d0 con 0x56418d253570 2024-07-04T08:19:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.852+0000 7ff8d0c00640 10 client.26350 handle_command_reply: tid=0 2024-07-04T08:19:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.852+0000 7ff8f4105740 1 client.26350 shutdown 2024-07-04T08:19:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 -- 172.21.15.39:0/888925769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 msgr2=0x56418d67f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:07.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d67f1b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4004e00 tx=0x7ff8d40316d0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 -- 172.21.15.39:0/888925769 shutdown_connections 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56418d253570 0x56418d6f5890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56418d2bf840 0x56418d67f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56418d29eb60 0x56418d67f1b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 --2- 172.21.15.39:0/888925769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56418d296980 0x56418d685480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8d0c00640 0 client.26350 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 -- 172.21.15.39:0/888925769 >> 172.21.15.39:0/888925769 conn(0x56418d2b3990 msgr2=0x56418d660af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 -- 172.21.15.39:0/888925769 shutdown_connections 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 1 -- 172.21.15.39:0/888925769 wait complete. 2024-07-04T08:19:07.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.853+0000 7ff8f4105740 20 client.26350 trim_cache size 0 max 16384 2024-07-04T08:19:07.854 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:19:07.854 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:19:07.854 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:19:07.854 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 10, 2024-07-04T08:19:07.854 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:19:07.854 INFO:teuthology.orchestra.run.smithi039.stdout: "705d2328": { 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 10, 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 152.3, 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:19:07.855 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:19:07.856 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:19:07.856 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:19:07.856 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:19:07.856 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:07.856 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:07.856 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:07.856 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:07.856 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:07.856 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:19:07.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 -- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8cc0b29d0 msgr2=0x7ff8cc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:07.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8cc0b29d0 0x7ff8cc0b4ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x56418d663200 tx=0x7ff8d8000bc0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 -- 172.21.15.39:0/426669181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8e8103d40 msgr2=0x7ff8e819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:07.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8e8103d40 0x7ff8e819ba10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800e9f0 tx=0x7ff8d800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 -- 172.21.15.39:0/426669181 shutdown_connections 2024-07-04T08:19:07.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8cc0b29d0 0x7ff8cc0b4ee0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8e8104690 0x7ff8e819bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8e810d1a0 0x7ff8e81a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 --2- 172.21.15.39:0/426669181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8e8103d40 0x7ff8e819ba10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:07.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 -- 172.21.15.39:0/426669181 >> 172.21.15.39:0/426669181 conn(0x7ff8e80fd880 msgr2=0x7ff8e8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:07.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 -- 172.21.15.39:0/426669181 shutdown_connections 2024-07-04T08:19:07.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:07.854+0000 7ff8e7e00640 1 -- 172.21.15.39:0/426669181 wait complete. 2024-07-04T08:19:09.196 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:08 smithi039 ceph-mon[32062]: pgmap v598: 129 pgs: 129 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-07-04T08:19:09.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:08 smithi133 ceph-mon[37050]: pgmap v598: 129 pgs: 129 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-07-04T08:19:09.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:08 smithi050 ceph-mon[36577]: pgmap v598: 129 pgs: 129 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-07-04T08:19:09.615 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '705d2328', quiesce took 11.1 seconds. Will release after: 21.199999999999996 2024-07-04T08:19:10.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.271+0000 7f6760a00640 1 -- 172.21.15.39:0/3432594625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f675c105110 msgr2=0x7f675c105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:10.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.271+0000 7f6760a00640 1 --2- 172.21.15.39:0/3432594625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f675c105110 0x7f675c105520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f674c009900 tx=0x7f674c02a410 comp rx=0 tx=0).stop 2024-07-04T08:19:10.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.271+0000 7f6760a00640 1 -- 172.21.15.39:0/3432594625 shutdown_connections 2024-07-04T08:19:10.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.271+0000 7f6760a00640 1 --2- 172.21.15.39:0/3432594625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c10b3a0 0x7f675c111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.271+0000 7f6760a00640 1 --2- 172.21.15.39:0/3432594625 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f675c10aa50 0x7f675c10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.271+0000 7f6760a00640 1 --2- 172.21.15.39:0/3432594625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f675c105110 0x7f675c105520 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.271+0000 7f6760a00640 1 -- 172.21.15.39:0/3432594625 >> 172.21.15.39:0/3432594625 conn(0x7f675c100b80 msgr2=0x7f675c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.271+0000 7f6760a00640 1 -- 172.21.15.39:0/3432594625 shutdown_connections 2024-07-04T08:19:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.271+0000 7f6760a00640 1 -- 172.21.15.39:0/3432594625 wait complete. 2024-07-04T08:19:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.272+0000 7f6760a00640 1 Processor -- start 2024-07-04T08:19:10.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.273+0000 7f6760a00640 1 -- start start 2024-07-04T08:19:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.273+0000 7f6760a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f675c105110 0x7f675c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.273+0000 7f6760a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f675c10aa50 0x7f675c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.273+0000 7f6760a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c10b3a0 0x7f675c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.273+0000 7f6760a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1150a0 con 0x7f675c105110 2024-07-04T08:19:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.273+0000 7f6760a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f675c114f20 con 0x7f675c10aa50 2024-07-04T08:19:10.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.273+0000 7f6760a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f675c115220 con 0x7f675c10b3a0 2024-07-04T08:19:10.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f675c105110 0x7f675c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f675c10aa50 0x7f675c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f675c105110 0x7f675c19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54842/0 (socket says 172.21.15.39:54842) 2024-07-04T08:19:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675b400640 1 -- 172.21.15.39:0/2837359113 learned_addr learned my addr 172.21.15.39:0/2837359113 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675be00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c10b3a0 0x7f675c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675aa00640 1 -- 172.21.15.39:0/2837359113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c10b3a0 msgr2=0x7f675c1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675aa00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c10b3a0 0x7f675c1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675aa00640 1 -- 172.21.15.39:0/2837359113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f675c105110 msgr2=0x7f675c19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675aa00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f675c105110 0x7f675c19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675aa00640 1 -- 172.21.15.39:0/2837359113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c1a19c0 con 0x7f675c10aa50 2024-07-04T08:19:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.274+0000 7f675be00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c10b3a0 0x7f675c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:19:10.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.275+0000 7f675aa00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f675c10aa50 0x7f675c19bff0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f675c06ae80 tx=0x7f674800db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.275+0000 7f6753e00640 1 -- 172.21.15.39:0/2837359113 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748017070 con 0x7f675c10aa50 2024-07-04T08:19:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.275+0000 7f6760a00640 1 -- 172.21.15.39:0/2837359113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c1a1cb0 con 0x7f675c10aa50 2024-07-04T08:19:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.275+0000 7f6753e00640 1 -- 172.21.15.39:0/2837359113 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6748009820 con 0x7f675c10aa50 2024-07-04T08:19:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.275+0000 7f6753e00640 1 -- 172.21.15.39:0/2837359113 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748004b90 con 0x7f675c10aa50 2024-07-04T08:19:10.275 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.275+0000 7f6760a00640 1 -- 172.21.15.39:0/2837359113 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c1a8410 con 0x7f675c10aa50 2024-07-04T08:19:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.278+0000 7f67686ac740 1 Processor -- start 2024-07-04T08:19:10.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.279+0000 7f67686ac740 1 -- start start 2024-07-04T08:19:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.279+0000 7f67686ac740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134e49950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.279+0000 7f67686ac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a7c1b0 0x561134e49e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.279+0000 7f67686ac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a84390 0x561134e4a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.279+0000 7f67686ac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561134dbfee0 con 0x561134a84390 2024-07-04T08:19:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.279+0000 7f67686ac740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561134daf480 con 0x561134aa5070 2024-07-04T08:19:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.279+0000 7f67686ac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561134db34c0 con 0x561134a7c1b0 2024-07-04T08:19:10.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.279+0000 7f6753e00640 1 -- 172.21.15.39:0/2837359113 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6748004d30 con 0x7f675c10aa50 2024-07-04T08:19:10.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a7c1b0 0x561134e49e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134e49950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a7c1b0 0x561134e49e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56292/0 (socket says 172.21.15.39:56292) 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675b400640 1 -- 172.21.15.39:0/2504121157 learned_addr learned my addr 172.21.15.39:0/2504121157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675aa00640 1 --2- 172.21.15.39:0/2504121157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a84390 0x561134e4a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675b400640 1 -- 172.21.15.39:0/2504121157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 msgr2=0x561134e49950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675b400640 1 --2- 172.21.15.39:0/2504121157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134e49950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675b400640 1 -- 172.21.15.39:0/2504121157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a84390 msgr2=0x561134e4a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675b400640 1 --2- 172.21.15.39:0/2504121157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a84390 0x561134e4a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.280+0000 7f675b400640 1 -- 172.21.15.39:0/2504121157 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561134d47650 con 0x561134a7c1b0 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.281+0000 7f675b400640 1 --2- 172.21.15.39:0/2504121157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a7c1b0 0x561134e49e90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f674c006010 tx=0x7f674c004240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c4eefd5f3a89c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.281+0000 7f6750c00640 1 -- 172.21.15.39:0/2504121157 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c040070 con 0x561134a7c1b0 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.281+0000 7f6750c00640 1 -- 172.21.15.39:0/2504121157 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f674c0047a0 con 0x561134a7c1b0 2024-07-04T08:19:10.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.281+0000 7f6750c00640 1 -- 172.21.15.39:0/2504121157 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c035040 con 0x561134a7c1b0 2024-07-04T08:19:10.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.281+0000 7f6753e00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f67400b2960 0x7f67400b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f6753e00640 1 -- 172.21.15.39:0/2837359113 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f67480e2270 con 0x7f675c10aa50 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f675be00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f67400b2960 0x7f67400b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f67686ac740 1 -- 172.21.15.39:0/2504121157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a7c1b0 msgr2=0x561134e49e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f67686ac740 1 --2- 172.21.15.39:0/2504121157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a7c1b0 0x561134e49e90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f674c006010 tx=0x7f674c004240 comp rx=0 tx=0).stop 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f67686ac740 1 -- 172.21.15.39:0/2504121157 shutdown_connections 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f67686ac740 1 --2- 172.21.15.39:0/2504121157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a84390 0x561134e4a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f67686ac740 1 --2- 172.21.15.39:0/2504121157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a7c1b0 0x561134e49e90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f67686ac740 1 --2- 172.21.15.39:0/2504121157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134e49950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f67686ac740 1 -- 172.21.15.39:0/2504121157 >> 172.21.15.39:0/2504121157 conn(0x561134a99460 msgr2=0x561134e47840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.282+0000 7f67686ac740 1 -- 172.21.15.39:0/2504121157 shutdown_connections 2024-07-04T08:19:10.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.283+0000 7f67686ac740 1 -- 172.21.15.39:0/2504121157 wait complete. 2024-07-04T08:19:10.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.283+0000 7f675be00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f67400b2960 0x7f67400b4e70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6754005fd0 tx=0x7f6754009800 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.283+0000 7f67686ac740 1 -- start start 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.284+0000 7f67686ac740 1 Processor -- start 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.284+0000 7f67686ac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e55490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.284+0000 7f67686ac740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a84390 0x561134e559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.284+0000 7f67686ac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134aa5070 0x561134e613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.284+0000 7f67686ac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561134c13700 con 0x561134a7c1b0 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.284+0000 7f67686ac740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561134d56bb0 con 0x561134a84390 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.284+0000 7f67686ac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561134e580d0 con 0x561134aa5070 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.284+0000 7f67686ac740 10 client.0 _ll_register_callbacks cb 0x561134da8570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:10.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e55490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e55490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54862/0 (socket says 172.21.15.39:54862) 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675b400640 1 -- 172.21.15.39:0/2874891864 learned_addr learned my addr 172.21.15.39:0/2874891864 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675aa00640 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a84390 0x561134e559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675be00640 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134aa5070 0x561134e613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675b400640 1 -- 172.21.15.39:0/2874891864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134aa5070 msgr2=0x561134e613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675b400640 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134aa5070 0x561134e613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675b400640 1 -- 172.21.15.39:0/2874891864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a84390 msgr2=0x561134e559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675b400640 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a84390 0x561134e559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675b400640 1 -- 172.21.15.39:0/2874891864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561134b10650 con 0x561134a7c1b0 2024-07-04T08:19:10.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.285+0000 7f675aa00640 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a84390 0x561134e559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.286+0000 7f675b400640 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e55490 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f674c005ab0 tx=0x7f674c004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18011d63825cc389 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:10.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.286+0000 7f6745600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.286+0000 7f6745600640 1 -- 172.21.15.39:0/2874891864 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c035070 con 0x561134a7c1b0 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.286+0000 7f6745600640 1 -- 172.21.15.39:0/2874891864 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f674c005430 con 0x561134a7c1b0 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.286+0000 7f67686ac740 1 -- 172.21.15.39:0/2874891864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f674c0047a0 con 0x561134a7c1b0 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.286+0000 7f6745600640 1 -- 172.21.15.39:0/2874891864 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f674c040040 con 0x561134a7c1b0 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.286+0000 7f67686ac740 10 client.16305 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.286+0000 7f67686ac740 1 -- 172.21.15.39:0/2874891864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561134e64090 con 0x561134a7c1b0 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.287+0000 7f6745600640 1 -- 172.21.15.39:0/2874891864 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f674c0047a0 con 0x561134a7c1b0 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.287+0000 7f6745600640 1 -- 172.21.15.39:0/2874891864 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f674c0314a0 con 0x561134a7c1b0 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.287+0000 7f67686ac740 10 client.16305 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.287+0000 7f67686ac740 10 client.16305 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.287+0000 7f67686ac740 20 client.16305 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.287+0000 7f67686ac740 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x561134a639f0 0x561134e673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.287+0000 7f67686ac740 4 client.16305 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:10.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.287+0000 7f67686ac740 1 -- 172.21.15.39:0/2874891864 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561134dc1e30 con 0x561134a639f0 2024-07-04T08:19:10.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.288+0000 7f675aa00640 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x561134a639f0 0x561134e673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.288+0000 7f675aa00640 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x561134a639f0 0x561134e673f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a08fb0e1f13673d server_cookie=310f5bed8a7bd0c6 in_seq=0 out_seq=0 2024-07-04T08:19:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.288+0000 7f6745600640 10 client.16305 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:19:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.289+0000 7f6745600640 1 -- 172.21.15.39:0/2874891864 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561134dc1e30 con 0x561134a639f0 2024-07-04T08:19:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.289+0000 7f6745600640 10 client.16305 handle_command_reply: tid=0 2024-07-04T08:19:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.289+0000 7f67686ac740 1 client.16305 shutdown 2024-07-04T08:19:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 -- 172.21.15.39:0/2874891864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 msgr2=0x561134e55490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e55490 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f674c005ab0 tx=0x7f674c004b80 comp rx=0 tx=0).stop 2024-07-04T08:19:10.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 -- 172.21.15.39:0/2874891864 shutdown_connections 2024-07-04T08:19:10.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x561134a639f0 0x561134e673f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134aa5070 0x561134e613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a84390 0x561134e559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 --2- 172.21.15.39:0/2874891864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e55490 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f6745600640 0 client.16305 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:19:10.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 -- 172.21.15.39:0/2874891864 >> 172.21.15.39:0/2874891864 conn(0x561134a99460 msgr2=0x561134e4cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:10.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 -- 172.21.15.39:0/2874891864 shutdown_connections 2024-07-04T08:19:10.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 1 -- 172.21.15.39:0/2874891864 wait complete. 2024-07-04T08:19:10.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.290+0000 7f67686ac740 20 client.16305 trim_cache size 0 max 16384 2024-07-04T08:19:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.306+0000 7f67686ac740 1 Processor -- start 2024-07-04T08:19:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.306+0000 7f67686ac740 1 -- start start 2024-07-04T08:19:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f67686ac740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a7c1b0 0x561134e4f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f67686ac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 0x561134e61360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f67686ac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134aa5070 0x561134e618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f67686ac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b21f0 con 0x561134aa5070 2024-07-04T08:19:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f67686ac740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b3d30 con 0x561134a7c1b0 2024-07-04T08:19:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f67686ac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b28c0 con 0x561134a84390 2024-07-04T08:19:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f675be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134aa5070 0x561134e618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f675b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a7c1b0 0x561134e4f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f675be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134aa5070 0x561134e618a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54878/0 (socket says 172.21.15.39:54878) 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f675be00640 1 -- 172.21.15.39:0/3829522522 learned_addr learned my addr 172.21.15.39:0/3829522522 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.307+0000 7f675aa00640 1 --2- 172.21.15.39:0/3829522522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 0x561134e61360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f675b400640 1 -- 172.21.15.39:0/3829522522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 msgr2=0x561134e61360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f675b400640 1 --2- 172.21.15.39:0/3829522522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 0x561134e61360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f675b400640 1 -- 172.21.15.39:0/3829522522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134aa5070 msgr2=0x561134e618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f675b400640 1 --2- 172.21.15.39:0/3829522522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134aa5070 0x561134e618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f675b400640 1 -- 172.21.15.39:0/3829522522 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561134e4be70 con 0x561134a7c1b0 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f675be00640 1 --2- 172.21.15.39:0/3829522522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134aa5070 0x561134e618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:10.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f675b400640 1 --2- 172.21.15.39:0/3829522522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a7c1b0 0x561134e4f130 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f674c005940 tx=0x7f674c004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de360d4de474c78 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f6747e00640 1 -- 172.21.15.39:0/3829522522 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c035040 con 0x561134a7c1b0 2024-07-04T08:19:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f6747e00640 1 -- 172.21.15.39:0/3829522522 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f674c009520 con 0x561134a7c1b0 2024-07-04T08:19:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.308+0000 7f6747e00640 1 -- 172.21.15.39:0/3829522522 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0035f0 con 0x561134a7c1b0 2024-07-04T08:19:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.309+0000 7f67686ac740 1 -- 172.21.15.39:0/3829522522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a7c1b0 msgr2=0x561134e4f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:10.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.309+0000 7f67686ac740 1 --2- 172.21.15.39:0/3829522522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a7c1b0 0x561134e4f130 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f674c005940 tx=0x7f674c004590 comp rx=0 tx=0).stop 2024-07-04T08:19:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.309+0000 7f67686ac740 1 -- 172.21.15.39:0/3829522522 shutdown_connections 2024-07-04T08:19:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.309+0000 7f67686ac740 1 --2- 172.21.15.39:0/3829522522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134aa5070 0x561134e618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.309+0000 7f67686ac740 1 --2- 172.21.15.39:0/3829522522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 0x561134e61360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.309+0000 7f67686ac740 1 --2- 172.21.15.39:0/3829522522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134a7c1b0 0x561134e4f130 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.309+0000 7f67686ac740 1 -- 172.21.15.39:0/3829522522 >> 172.21.15.39:0/3829522522 conn(0x561134a99460 msgr2=0x561134e4ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:10.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.310+0000 7f67686ac740 1 -- 172.21.15.39:0/3829522522 shutdown_connections 2024-07-04T08:19:10.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.310+0000 7f67686ac740 1 -- 172.21.15.39:0/3829522522 wait complete. 2024-07-04T08:19:10.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.310+0000 7f67686ac740 1 -- start start 2024-07-04T08:19:10.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.311+0000 7f67686ac740 1 Processor -- start 2024-07-04T08:19:10.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.311+0000 7f67686ac740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e67d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.311+0000 7f67686ac740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 0x561134e682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.311+0000 7f67686ac740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134ec7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.311+0000 7f67686ac740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b3660 con 0x561134a7c1b0 2024-07-04T08:19:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.311+0000 7f67686ac740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b2f90 con 0x561134aa5070 2024-07-04T08:19:10.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.311+0000 7f67686ac740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b01d0 con 0x561134a84390 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.311+0000 7f67686ac740 10 client.0 _ll_register_callbacks cb 0x7f67540096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134ec7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 0x561134e682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e67d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134ec7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37782/0 (socket says 172.21.15.39:37782) 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675be00640 1 -- 172.21.15.39:0/1669212576 learned_addr learned my addr 172.21.15.39:0/1669212576 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675aa00640 1 -- 172.21.15.39:0/1669212576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 msgr2=0x561134ec7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675aa00640 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134ec7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675aa00640 1 -- 172.21.15.39:0/1669212576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 msgr2=0x561134e67d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675aa00640 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e67d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675aa00640 1 -- 172.21.15.39:0/1669212576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561134ec7fe0 con 0x561134a84390 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675be00640 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134ec7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f675aa00640 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 0x561134e682d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f67480df690 tx=0x7f674800df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd2ab24c079843ca server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:10.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.312+0000 7f6745600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:19:10.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.313+0000 7f6745600640 1 -- 172.21.15.39:0/1669212576 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748020040 con 0x561134a84390 2024-07-04T08:19:10.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.313+0000 7f6745600640 1 -- 172.21.15.39:0/1669212576 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f6748009c60 con 0x561134a84390 2024-07-04T08:19:10.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.313+0000 7f67686ac740 1 -- 172.21.15.39:0/1669212576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f674c009520 con 0x561134a84390 2024-07-04T08:19:10.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.313+0000 7f6745600640 1 -- 172.21.15.39:0/1669212576 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6748009e60 con 0x561134a84390 2024-07-04T08:19:10.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.313+0000 7f67686ac740 10 client.26368 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.313+0000 7f67686ac740 1 -- 172.21.15.39:0/1669212576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561134ecf430 con 0x561134a84390 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.313+0000 7f6745600640 1 -- 172.21.15.39:0/1669212576 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f674c009520 con 0x561134a84390 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.314+0000 7f6745600640 1 -- 172.21.15.39:0/1669212576 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f67480ad110 con 0x561134a84390 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.314+0000 7f67686ac740 10 client.26368 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.314+0000 7f67686ac740 10 client.26368 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.314+0000 7f67686ac740 20 client.26368 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.314+0000 7f67686ac740 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x561134a38da0 0x561134ed1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.314+0000 7f67686ac740 4 client.26368 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.314+0000 7f67686ac740 1 -- 172.21.15.39:0/1669212576 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561134dbd8d0 con 0x561134a38da0 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.314+0000 7f675b400640 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x561134a38da0 0x561134ed1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f675b400640 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x561134a38da0 0x561134ed1bc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c24f15b8a3202a8 server_cookie=f458964b7687e119 in_seq=0 out_seq=0 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f6745600640 10 client.26368 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:19:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f6745600640 1 -- 172.21.15.39:0/1669212576 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561134dbd8d0 con 0x561134a38da0 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f6745600640 10 client.26368 handle_command_reply: tid=0 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f67686ac740 1 client.26368 shutdown 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f67686ac740 1 -- 172.21.15.39:0/1669212576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 msgr2=0x561134e682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f67686ac740 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 0x561134e682d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f67480df690 tx=0x7f674800df90 comp rx=0 tx=0).stop 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f67686ac740 1 -- 172.21.15.39:0/1669212576 shutdown_connections 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f67686ac740 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x561134a38da0 0x561134ed1bc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f67686ac740 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561134aa5070 0x561134ec7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f67686ac740 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561134a84390 0x561134e682d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f6745600640 0 client.26368 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f67686ac740 1 --2- 172.21.15.39:0/1669212576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561134a7c1b0 0x561134e67d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.315+0000 7f67686ac740 1 -- 172.21.15.39:0/1669212576 >> 172.21.15.39:0/1669212576 conn(0x561134a99460 msgr2=0x561134e56740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.316+0000 7f67686ac740 1 -- 172.21.15.39:0/1669212576 shutdown_connections 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.316+0000 7f67686ac740 1 -- 172.21.15.39:0/1669212576 wait complete. 2024-07-04T08:19:10.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.316+0000 7f67686ac740 20 client.26368 trim_cache size 0 max 16384 2024-07-04T08:19:10.318 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:19:10.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.316+0000 7f6760a00640 1 -- 172.21.15.39:0/2837359113 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f67400b2960 msgr2=0x7f67400b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.316+0000 7f6760a00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f67400b2960 0x7f67400b4e70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6754005fd0 tx=0x7f6754009800 comp rx=0 tx=0).stop 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.316+0000 7f6760a00640 1 -- 172.21.15.39:0/2837359113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f675c10aa50 msgr2=0x7f675c19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.316+0000 7f6760a00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f675c10aa50 0x7f675c19bff0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f675c06ae80 tx=0x7f674800db30 comp rx=0 tx=0).stop 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.317+0000 7f6760a00640 1 -- 172.21.15.39:0/2837359113 shutdown_connections 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.317+0000 7f6760a00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f67400b2960 0x7f67400b4e70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.317+0000 7f6760a00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c10b3a0 0x7f675c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.317+0000 7f6760a00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f675c10aa50 0x7f675c19bff0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.317+0000 7f6760a00640 1 --2- 172.21.15.39:0/2837359113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f675c105110 0x7f675c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.317+0000 7f6760a00640 1 -- 172.21.15.39:0/2837359113 >> 172.21.15.39:0/2837359113 conn(0x7f675c100b80 msgr2=0x7f675c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.317+0000 7f6760a00640 1 -- 172.21.15.39:0/2837359113 shutdown_connections 2024-07-04T08:19:10.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:10.317+0000 7f6760a00640 1 -- 172.21.15.39:0/2837359113 wait complete. 2024-07-04T08:19:11.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:10 smithi039 ceph-mon[32062]: pgmap v599: 129 pgs: 129 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 28 op/s 2024-07-04T08:19:11.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:19:11.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:10 smithi133 ceph-mon[37050]: pgmap v599: 129 pgs: 129 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 28 op/s 2024-07-04T08:19:11.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:10 smithi050 ceph-mon[36577]: pgmap v599: 129 pgs: 129 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 28 op/s 2024-07-04T08:19:11.716 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:13.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:12 smithi039 ceph-mon[32062]: pgmap v600: 129 pgs: 129 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 29 op/s 2024-07-04T08:19:13.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:12 smithi133 ceph-mon[37050]: pgmap v600: 129 pgs: 129 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 29 op/s 2024-07-04T08:19:13.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:12 smithi050 ceph-mon[36577]: pgmap v600: 129 pgs: 129 active+clean; 3.8 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 29 op/s 2024-07-04T08:19:14.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.147+0000 7f8dfb200640 1 -- 172.21.15.39:0/3302675747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dfc1050f0 msgr2=0x7f8dfc105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:14.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.147+0000 7f8dfb200640 1 --2- 172.21.15.39:0/3302675747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dfc1050f0 0x7f8dfc105500 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8dec009ca0 tx=0x7f8dec02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.147+0000 7f8dfb200640 1 -- 172.21.15.39:0/3302675747 shutdown_connections 2024-07-04T08:19:14.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.147+0000 7f8dfb200640 1 --2- 172.21.15.39:0/3302675747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dfc10b380 0x7f8dfc111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.147+0000 7f8dfb200640 1 --2- 172.21.15.39:0/3302675747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dfc10aa30 0x7f8dfc10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.147+0000 7f8dfb200640 1 --2- 172.21.15.39:0/3302675747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dfc1050f0 0x7f8dfc105500 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.147+0000 7f8dfb200640 1 -- 172.21.15.39:0/3302675747 >> 172.21.15.39:0/3302675747 conn(0x7f8dfc100b60 msgr2=0x7f8dfc102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.147+0000 7f8dfb200640 1 -- 172.21.15.39:0/3302675747 shutdown_connections 2024-07-04T08:19:14.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.147+0000 7f8dfb200640 1 -- 172.21.15.39:0/3302675747 wait complete. 2024-07-04T08:19:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.148+0000 7f8dfb200640 1 Processor -- start 2024-07-04T08:19:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.149+0000 7f8dfb200640 1 -- start start 2024-07-04T08:19:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.149+0000 7f8dfb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dfc1050f0 0x7f8dfc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.149+0000 7f8dfb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dfc10aa30 0x7f8dfc19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.149+0000 7f8dfb200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dfc10b380 0x7f8dfc1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.149+0000 7f8dfb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc115050 con 0x7f8dfc1050f0 2024-07-04T08:19:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.149+0000 7f8dfb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc114ed0 con 0x7f8dfc10aa30 2024-07-04T08:19:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.149+0000 7f8dfb200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1151d0 con 0x7f8dfc10b380 2024-07-04T08:19:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.149+0000 7f8df9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dfc1050f0 0x7f8dfc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8df9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dfc1050f0 0x7f8dfc19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44930/0 (socket says 172.21.15.39:44930) 2024-07-04T08:19:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8df9e00640 1 -- 172.21.15.39:0/1253222531 learned_addr learned my addr 172.21.15.39:0/1253222531 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8dfa800640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dfc10b380 0x7f8dfc1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8df1400640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dfc10aa30 0x7f8dfc19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8df1400640 1 -- 172.21.15.39:0/1253222531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dfc10b380 msgr2=0x7f8dfc1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8df1400640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dfc10b380 0x7f8dfc1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8df1400640 1 -- 172.21.15.39:0/1253222531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dfc1050f0 msgr2=0x7f8dfc19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8df1400640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dfc1050f0 0x7f8dfc19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8df1400640 1 -- 172.21.15.39:0/1253222531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dfc1a19d0 con 0x7f8dfc10aa30 2024-07-04T08:19:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.150+0000 7f8df1400640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dfc10aa30 0x7f8dfc19c000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc06bd90 tx=0x7f8de400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.151+0000 7f8df3400640 1 -- 172.21.15.39:0/1253222531 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de400cc40 con 0x7f8dfc10aa30 2024-07-04T08:19:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.151+0000 7f8df3400640 1 -- 172.21.15.39:0/1253222531 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de400cde0 con 0x7f8dfc10aa30 2024-07-04T08:19:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.151+0000 7f8dfb200640 1 -- 172.21.15.39:0/1253222531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dfc1a1cc0 con 0x7f8dfc10aa30 2024-07-04T08:19:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.151+0000 7f8df3400640 1 -- 172.21.15.39:0/1253222531 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4018780 con 0x7f8dfc10aa30 2024-07-04T08:19:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.151+0000 7f8dfb200640 1 -- 172.21.15.39:0/1253222531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dfc1a8420 con 0x7f8dfc10aa30 2024-07-04T08:19:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.154+0000 7f8e071c0740 1 Processor -- start 2024-07-04T08:19:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.154+0000 7f8e071c0740 1 -- start start 2024-07-04T08:19:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 0x55b98c9eb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 0x55b98c9ec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8e071c0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b98c961ee0 con 0x55b98c61e1b0 2024-07-04T08:19:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8e071c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b98c951480 con 0x55b98c647070 2024-07-04T08:19:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8e071c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b98c9554c0 con 0x55b98c626390 2024-07-04T08:19:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8df3400640 1 -- 172.21.15.39:0/1253222531 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8de4010a90 con 0x7f8dfc10aa30 2024-07-04T08:19:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8df9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8dfa800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 0x55b98c9eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8df9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ebe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44944/0 (socket says 172.21.15.39:44944) 2024-07-04T08:19:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8df9e00640 1 -- 172.21.15.39:0/894745415 learned_addr learned my addr 172.21.15.39:0/894745415 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.155+0000 7f8df1400640 1 --2- 172.21.15.39:0/894745415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 0x55b98c9ec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.156+0000 7f8dfa800640 1 -- 172.21.15.39:0/894745415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 msgr2=0x55b98c9ec3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.156+0000 7f8dfa800640 1 --2- 172.21.15.39:0/894745415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 0x55b98c9ec3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.156+0000 7f8dfa800640 1 -- 172.21.15.39:0/894745415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 msgr2=0x55b98c9ebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.156+0000 7f8dfa800640 1 --2- 172.21.15.39:0/894745415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ebe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.156+0000 7f8dfa800640 1 -- 172.21.15.39:0/894745415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98c8e9650 con 0x55b98c647070 2024-07-04T08:19:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.156+0000 7f8df1400640 1 --2- 172.21.15.39:0/894745415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 0x55b98c9ec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:19:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.156+0000 7f8dfa800640 1 --2- 172.21.15.39:0/894745415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 0x55b98c9eb950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009b80 tx=0x7f8df4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac6da7e9f4b99d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.157+0000 7f8deb400640 1 -- 172.21.15.39:0/894745415 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004030 con 0x55b98c647070 2024-07-04T08:19:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.157+0000 7f8deb400640 1 -- 172.21.15.39:0/894745415 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df4005020 con 0x55b98c647070 2024-07-04T08:19:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.157+0000 7f8deb400640 1 -- 172.21.15.39:0/894745415 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4004ca0 con 0x55b98c647070 2024-07-04T08:19:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.157+0000 7f8df3400640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8de00b2960 0x7f8de00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8df3400640 1 -- 172.21.15.39:0/1253222531 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8de40e2c60 con 0x7f8dfc10aa30 2024-07-04T08:19:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8df9e00640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8de00b2960 0x7f8de00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8e071c0740 1 -- 172.21.15.39:0/894745415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 msgr2=0x55b98c9eb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8e071c0740 1 --2- 172.21.15.39:0/894745415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 0x55b98c9eb950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009b80 tx=0x7f8df4011810 comp rx=0 tx=0).stop 2024-07-04T08:19:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8e071c0740 1 -- 172.21.15.39:0/894745415 shutdown_connections 2024-07-04T08:19:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8e071c0740 1 --2- 172.21.15.39:0/894745415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8e071c0740 1 --2- 172.21.15.39:0/894745415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 0x55b98c9ec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8e071c0740 1 --2- 172.21.15.39:0/894745415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 0x55b98c9eb950 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8e071c0740 1 -- 172.21.15.39:0/894745415 >> 172.21.15.39:0/894745415 conn(0x55b98c63b460 msgr2=0x55b98c9e9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.158+0000 7f8e071c0740 1 -- 172.21.15.39:0/894745415 shutdown_connections 2024-07-04T08:19:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.159+0000 7f8e071c0740 1 -- 172.21.15.39:0/894745415 wait complete. 2024-07-04T08:19:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.159+0000 7f8df9e00640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8de00b2960 0x7f8de00b4e70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55b98c9ea5d0 tx=0x7f8dec0093e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.159+0000 7f8e071c0740 1 -- start start 2024-07-04T08:19:14.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.160+0000 7f8e071c0740 1 Processor -- start 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.160+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c61e1b0 0x55b98c9f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.160+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98c9f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.160+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c647070 0x55b98ca03450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.160+0000 7f8e071c0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b98c7b5700 con 0x55b98c647070 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.160+0000 7f8e071c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b98c8f8bb0 con 0x55b98c626390 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.160+0000 7f8e071c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b98c9fa130 con 0x55b98c61e1b0 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.160+0000 7f8e071c0740 10 client.0 _ll_register_callbacks cb 0x55b98c94a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.160+0000 7f8df1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98c9f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.161+0000 7f8df1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98c9f7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34474/0 (socket says 172.21.15.39:34474) 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.161+0000 7f8df1400640 1 -- 172.21.15.39:0/2733572082 learned_addr learned my addr 172.21.15.39:0/2733572082 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.161+0000 7f8df9e00640 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c61e1b0 0x55b98c9f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.161+0000 7f8dfa800640 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c647070 0x55b98ca03450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.161+0000 7f8df9e00640 1 -- 172.21.15.39:0/2733572082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 msgr2=0x55b98c9f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.161+0000 7f8df9e00640 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98c9f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.161+0000 7f8df9e00640 1 -- 172.21.15.39:0/2733572082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c647070 msgr2=0x55b98ca03450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.161+0000 7f8df9e00640 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c647070 0x55b98ca03450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.161+0000 7f8df9e00640 1 -- 172.21.15.39:0/2733572082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98c6b2650 con 0x55b98c61e1b0 2024-07-04T08:19:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.162+0000 7f8df9e00640 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c61e1b0 0x55b98c9f75c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8dec006010 tx=0x7f8dec02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52cb2786e3f4dd8c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.162+0000 7f8ddfe00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:19:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.162+0000 7f8ddfe00640 1 -- 172.21.15.39:0/2733572082 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec003d00 con 0x55b98c61e1b0 2024-07-04T08:19:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.162+0000 7f8ddfe00640 1 -- 172.21.15.39:0/2733572082 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8dec004040 con 0x55b98c61e1b0 2024-07-04T08:19:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.162+0000 7f8e071c0740 1 -- 172.21.15.39:0/2733572082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df4005020 con 0x55b98c61e1b0 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.162+0000 7f8ddfe00640 1 -- 172.21.15.39:0/2733572082 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8dec004240 con 0x55b98c61e1b0 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.162+0000 7f8e071c0740 10 client.26380 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.162+0000 7f8e071c0740 1 -- 172.21.15.39:0/2733572082 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b98ca061c0 con 0x55b98c61e1b0 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.163+0000 7f8ddfe00640 1 -- 172.21.15.39:0/2733572082 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8df4005020 con 0x55b98c61e1b0 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.163+0000 7f8ddfe00640 1 -- 172.21.15.39:0/2733572082 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8dec051070 con 0x55b98c61e1b0 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.163+0000 7f8e071c0740 10 client.26380 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.163+0000 7f8e071c0740 10 client.26380 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.163+0000 7f8e071c0740 20 client.26380 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.163+0000 7f8e071c0740 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b98c6059f0 0x55b98ca09760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.163+0000 7f8e071c0740 4 client.26380 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.163+0000 7f8e071c0740 1 -- 172.21.15.39:0/2733572082 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b98c963e30 con 0x55b98c6059f0 2024-07-04T08:19:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.164+0000 7f8df1400640 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b98c6059f0 0x55b98ca09760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.164+0000 7f8df1400640 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b98c6059f0 0x55b98ca09760 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=75adeade03773f20 server_cookie=3f10ed6764615a68 in_seq=0 out_seq=0 2024-07-04T08:19:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.164+0000 7f8ddfe00640 10 client.26380 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:19:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.165+0000 7f8ddfe00640 1 -- 172.21.15.39:0/2733572082 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b98c963e30 con 0x55b98c6059f0 2024-07-04T08:19:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.165+0000 7f8ddfe00640 10 client.26380 handle_command_reply: tid=0 2024-07-04T08:19:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.165+0000 7f8e071c0740 1 client.26380 shutdown 2024-07-04T08:19:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 -- 172.21.15.39:0/2733572082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c61e1b0 msgr2=0x55b98c9f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c61e1b0 0x55b98c9f75c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8dec006010 tx=0x7f8dec02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 -- 172.21.15.39:0/2733572082 shutdown_connections 2024-07-04T08:19:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b98c6059f0 0x55b98ca09760 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c647070 0x55b98ca03450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98c9f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8ddfe00640 0 client.26380 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:19:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 --2- 172.21.15.39:0/2733572082 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c61e1b0 0x55b98c9f75c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 -- 172.21.15.39:0/2733572082 >> 172.21.15.39:0/2733572082 conn(0x55b98c63b460 msgr2=0x55b98c9eeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 -- 172.21.15.39:0/2733572082 shutdown_connections 2024-07-04T08:19:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 1 -- 172.21.15.39:0/2733572082 wait complete. 2024-07-04T08:19:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.166+0000 7f8e071c0740 20 client.26380 trim_cache size 0 max 16384 2024-07-04T08:19:14.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.182+0000 7f8e071c0740 1 Processor -- start 2024-07-04T08:19:14.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.182+0000 7f8e071c0740 1 -- start start 2024-07-04T08:19:14.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.182+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.182+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 0x55b98c9ed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.182+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 0x55b98c9ef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.182+0000 7f8e071c0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b2220 con 0x55b98c61e1b0 2024-07-04T08:19:14.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.182+0000 7f8e071c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b3d60 con 0x55b98c647070 2024-07-04T08:19:14.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.182+0000 7f8e071c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b28f0 con 0x55b98c626390 2024-07-04T08:19:14.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8df9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8df9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44960/0 (socket says 172.21.15.39:44960) 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8df9e00640 1 -- 172.21.15.39:0/3068492055 learned_addr learned my addr 172.21.15.39:0/3068492055 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8df1400640 1 --2- 172.21.15.39:0/3068492055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 0x55b98c9ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8dfa800640 1 --2- 172.21.15.39:0/3068492055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 0x55b98c9ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8df9e00640 1 -- 172.21.15.39:0/3068492055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 msgr2=0x55b98c9ed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8df9e00640 1 --2- 172.21.15.39:0/3068492055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 0x55b98c9ed340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8df9e00640 1 -- 172.21.15.39:0/3068492055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 msgr2=0x55b98c9ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8df9e00640 1 --2- 172.21.15.39:0/3068492055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 0x55b98c9ef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.183+0000 7f8df9e00640 1 -- 172.21.15.39:0/3068492055 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98c9f7c80 con 0x55b98c61e1b0 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.184+0000 7f8df9e00640 1 --2- 172.21.15.39:0/3068492055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9f93b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8dec02fe60 tx=0x7f8dec02ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b86cb1eef13cbc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.184+0000 7f8deaa00640 1 -- 172.21.15.39:0/3068492055 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec005bf0 con 0x55b98c61e1b0 2024-07-04T08:19:14.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.184+0000 7f8deaa00640 1 -- 172.21.15.39:0/3068492055 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dec036050 con 0x55b98c61e1b0 2024-07-04T08:19:14.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.184+0000 7f8deaa00640 1 -- 172.21.15.39:0/3068492055 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec0307f0 con 0x55b98c61e1b0 2024-07-04T08:19:14.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.185+0000 7f8e071c0740 1 -- 172.21.15.39:0/3068492055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 msgr2=0x55b98c9f93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.185+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3068492055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9f93b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8dec02fe60 tx=0x7f8dec02ff80 comp rx=0 tx=0).stop 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.185+0000 7f8e071c0740 1 -- 172.21.15.39:0/3068492055 shutdown_connections 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.185+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3068492055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c647070 0x55b98c9ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.185+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3068492055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c626390 0x55b98c9ed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.185+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3068492055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9f93b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.185+0000 7f8e071c0740 1 -- 172.21.15.39:0/3068492055 >> 172.21.15.39:0/3068492055 conn(0x55b98c63b460 msgr2=0x55b98c9eea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.185+0000 7f8e071c0740 1 -- 172.21.15.39:0/3068492055 shutdown_connections 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.185+0000 7f8e071c0740 1 -- 172.21.15.39:0/3068492055 wait complete. 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.186+0000 7f8e071c0740 1 -- start start 2024-07-04T08:19:14.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.186+0000 7f8e071c0740 1 Processor -- start 2024-07-04T08:19:14.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98ca02c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8e071c0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c647070 0x55b98ca031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8e071c0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b3690 con 0x55b98c61e1b0 2024-07-04T08:19:14.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8e071c0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b2fc0 con 0x55b98c626390 2024-07-04T08:19:14.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8e071c0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8dfc1b0200 con 0x55b98c647070 2024-07-04T08:19:14.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8e071c0740 10 client.0 _ll_register_callbacks cb 0x7f8dec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8df9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8df1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98ca02c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8df9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44964/0 (socket says 172.21.15.39:44964) 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8df9e00640 1 -- 172.21.15.39:0/3349035274 learned_addr learned my addr 172.21.15.39:0/3349035274 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.187+0000 7f8dfa800640 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c647070 0x55b98ca031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.188+0000 7f8df9e00640 1 -- 172.21.15.39:0/3349035274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c647070 msgr2=0x55b98ca031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.188+0000 7f8df9e00640 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c647070 0x55b98ca031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.188+0000 7f8df9e00640 1 -- 172.21.15.39:0/3349035274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 msgr2=0x55b98ca02c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.188+0000 7f8df9e00640 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98ca02c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.188+0000 7f8df9e00640 1 -- 172.21.15.39:0/3349035274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98ca03840 con 0x55b98c61e1b0 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.188+0000 7f8df1400640 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98ca02c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.188+0000 7f8df9e00640 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ff9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8dec02ade0 tx=0x7f8dec0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1558b2de8fffb9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:14.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.188+0000 7f8ddfe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.189+0000 7f8ddfe00640 1 -- 172.21.15.39:0/3349035274 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec068070 con 0x55b98c61e1b0 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.189+0000 7f8ddfe00640 1 -- 172.21.15.39:0/3349035274 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8dec0043e0 con 0x55b98c61e1b0 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.189+0000 7f8e071c0740 1 -- 172.21.15.39:0/3349035274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dec036050 con 0x55b98c61e1b0 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.189+0000 7f8ddfe00640 1 -- 172.21.15.39:0/3349035274 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8dec0045e0 con 0x55b98c61e1b0 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.189+0000 7f8e071c0740 10 client.16329 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.189+0000 7f8e071c0740 1 -- 172.21.15.39:0/3349035274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b98ca71ef0 con 0x55b98c61e1b0 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.189+0000 7f8ddfe00640 1 -- 172.21.15.39:0/3349035274 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dec036050 con 0x55b98c61e1b0 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.190+0000 7f8ddfe00640 1 -- 172.21.15.39:0/3349035274 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8dec051040 con 0x55b98c61e1b0 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.190+0000 7f8e071c0740 10 client.16329 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:14.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.190+0000 7f8e071c0740 10 client.16329 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.190+0000 7f8e071c0740 20 client.16329 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.190+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b98c5dada0 0x55b98ca74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.190+0000 7f8e071c0740 4 client.16329 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.190+0000 7f8e071c0740 1 -- 172.21.15.39:0/3349035274 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b98c95f8d0 con 0x55b98c5dada0 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.190+0000 7f8df1400640 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b98c5dada0 0x55b98ca74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.191+0000 7f8df1400640 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b98c5dada0 0x55b98ca74130 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7a51b5f92be91ab server_cookie=34e83f94a3ac17fe in_seq=0 out_seq=0 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.191+0000 7f8ddfe00640 10 client.16329 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.191+0000 7f8ddfe00640 1 -- 172.21.15.39:0/3349035274 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b98c95f8d0 con 0x55b98c5dada0 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.191+0000 7f8ddfe00640 10 client.16329 handle_command_reply: tid=0 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.191+0000 7f8e071c0740 1 client.16329 shutdown 2024-07-04T08:19:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.191+0000 7f8e071c0740 1 -- 172.21.15.39:0/3349035274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 msgr2=0x55b98c9ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.191+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ff9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8dec02ade0 tx=0x7f8dec0040c0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8e071c0740 1 -- 172.21.15.39:0/3349035274 shutdown_connections 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b98c5dada0 0x55b98ca74130 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b98c647070 0x55b98ca031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b98c626390 0x55b98ca02c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8e071c0740 1 --2- 172.21.15.39:0/3349035274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b98c61e1b0 0x55b98c9ff9c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8e071c0740 1 -- 172.21.15.39:0/3349035274 >> 172.21.15.39:0/3349035274 conn(0x55b98c63b460 msgr2=0x55b98c9ed110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8ddfe00640 0 client.16329 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8e071c0740 1 -- 172.21.15.39:0/3349035274 shutdown_connections 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8e071c0740 1 -- 172.21.15.39:0/3349035274 wait complete. 2024-07-04T08:19:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.192+0000 7f8e071c0740 20 client.16329 trim_cache size 0 max 16384 2024-07-04T08:19:14.194 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:19:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 -- 172.21.15.39:0/1253222531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8de00b2960 msgr2=0x7f8de00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8de00b2960 0x7f8de00b4e70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55b98c9ea5d0 tx=0x7f8dec0093e0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 -- 172.21.15.39:0/1253222531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dfc10aa30 msgr2=0x7f8dfc19c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dfc10aa30 0x7f8dfc19c000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc06bd90 tx=0x7f8de400ee50 comp rx=0 tx=0).stop 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 -- 172.21.15.39:0/1253222531 shutdown_connections 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8de00b2960 0x7f8de00b4e70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dfc10b380 0x7f8dfc1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8dfc10aa30 0x7f8dfc19c000 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 --2- 172.21.15.39:0/1253222531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dfc1050f0 0x7f8dfc19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 -- 172.21.15.39:0/1253222531 >> 172.21.15.39:0/1253222531 conn(0x7f8dfc100b60 msgr2=0x7f8dfc102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 -- 172.21.15.39:0/1253222531 shutdown_connections 2024-07-04T08:19:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:14.193+0000 7f8dfb200640 1 -- 172.21.15.39:0/1253222531 wait complete. 2024-07-04T08:19:15.065 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:19:15.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:14 smithi133 ceph-mon[37050]: pgmap v601: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-07-04T08:19:15.350 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:14 smithi039 ceph-mon[32062]: pgmap v601: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-07-04T08:19:15.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:14 smithi050 ceph-mon[36577]: pgmap v601: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 50 KiB/s wr, 30 op/s 2024-07-04T08:19:15.547 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:17.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:17 smithi133 ceph-mon[37050]: pgmap v602: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 KiB/s wr, 27 op/s 2024-07-04T08:19:17.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:17 smithi039 ceph-mon[32062]: pgmap v602: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 KiB/s wr, 27 op/s 2024-07-04T08:19:17.394 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:17 smithi050 ceph-mon[36577]: pgmap v602: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 KiB/s wr, 27 op/s 2024-07-04T08:19:17.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.869+0000 7f44a7800640 1 -- 172.21.15.39:0/1923988335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a010f040 msgr2=0x7f44a010f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:17.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.869+0000 7f44a7800640 1 --2- 172.21.15.39:0/1923988335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a010f040 0x7f44a010f450 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f448c009900 tx=0x7f448c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.869+0000 7f44a7800640 1 -- 172.21.15.39:0/1923988335 shutdown_connections 2024-07-04T08:19:17.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.869+0000 7f44a7800640 1 --2- 172.21.15.39:0/1923988335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a010f040 0x7f44a010f450 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.869+0000 7f44a7800640 1 --2- 172.21.15.39:0/1923988335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44a010f990 0x7f44a01161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.869+0000 7f44a7800640 1 --2- 172.21.15.39:0/1923988335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44a0109700 0x7f44a0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.869+0000 7f44a7800640 1 -- 172.21.15.39:0/1923988335 >> 172.21.15.39:0/1923988335 conn(0x7f44a0077630 msgr2=0x7f44a0107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:17.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.869+0000 7f44a7800640 1 -- 172.21.15.39:0/1923988335 shutdown_connections 2024-07-04T08:19:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.869+0000 7f44a7800640 1 -- 172.21.15.39:0/1923988335 wait complete. 2024-07-04T08:19:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.870+0000 7f44a7800640 1 Processor -- start 2024-07-04T08:19:17.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.871+0000 7f44a7800640 1 -- start start 2024-07-04T08:19:17.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.871+0000 7f44a7800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a0109700 0x7f44a01a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.871+0000 7f44a7800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44a010f040 0x7f44a01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.871+0000 7f44a7800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44a010f990 0x7f44a01a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.871+0000 7f44a7800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0119600 con 0x7f44a0109700 2024-07-04T08:19:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.871+0000 7f44a7800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0119480 con 0x7f44a010f990 2024-07-04T08:19:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.871+0000 7f44a7800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0119780 con 0x7f44a010f040 2024-07-04T08:19:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a0109700 0x7f44a01a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a0109700 0x7f44a01a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44984/0 (socket says 172.21.15.39:44984) 2024-07-04T08:19:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a4e00640 1 -- 172.21.15.39:0/3125666708 learned_addr learned my addr 172.21.15.39:0/3125666708 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f449fe00640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44a010f040 0x7f44a01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a5800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44a010f990 0x7f44a01a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a5800640 1 -- 172.21.15.39:0/3125666708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44a010f040 msgr2=0x7f44a01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a5800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44a010f040 0x7f44a01a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a5800640 1 -- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a0109700 msgr2=0x7f44a01a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a5800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a0109700 0x7f44a01a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a5800640 1 -- 172.21.15.39:0/3125666708 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a01a6040 con 0x7f44a010f990 2024-07-04T08:19:17.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.872+0000 7f44a4e00640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a0109700 0x7f44a01a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.873+0000 7f44a5800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44a010f990 0x7f44a01a5910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4498009fd0 tx=0x7f449800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.873+0000 7f449d600640 1 -- 172.21.15.39:0/3125666708 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449800ce20 con 0x7f44a010f990 2024-07-04T08:19:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.873+0000 7f449d600640 1 -- 172.21.15.39:0/3125666708 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4498005020 con 0x7f44a010f990 2024-07-04T08:19:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.873+0000 7f44a7800640 1 -- 172.21.15.39:0/3125666708 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a01a6330 con 0x7f44a010f990 2024-07-04T08:19:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.873+0000 7f449d600640 1 -- 172.21.15.39:0/3125666708 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498010690 con 0x7f44a010f990 2024-07-04T08:19:17.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.873+0000 7f44a7800640 1 -- 172.21.15.39:0/3125666708 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a01aca90 con 0x7f44a010f990 2024-07-04T08:19:17.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.877+0000 7f44add8a740 1 Processor -- start 2024-07-04T08:19:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.877+0000 7f449d600640 1 -- 172.21.15.39:0/3125666708 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f44980054d0 con 0x7f44a010f990 2024-07-04T08:19:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.878+0000 7f44add8a740 1 -- start start 2024-07-04T08:19:17.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.878+0000 7f44add8a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb28070 0x55fc4fecc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.878+0000 7f44add8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.878+0000 7f44add8a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb07390 0x55fc4fecd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.878+0000 7f44add8a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fc4fe42f00 con 0x55fc4fb07390 2024-07-04T08:19:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.878+0000 7f44add8a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc4fe324a0 con 0x55fc4fb28070 2024-07-04T08:19:17.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.878+0000 7f44add8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc4fe364e0 con 0x55fc4faff1b0 2024-07-04T08:19:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.878+0000 7f449fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb28070 0x55fc4fecc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f44a4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f449fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb28070 0x55fc4fecc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34530/0 (socket says 172.21.15.39:34530) 2024-07-04T08:19:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f449fe00640 1 -- 172.21.15.39:0/214628151 learned_addr learned my addr 172.21.15.39:0/214628151 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f449fe00640 1 --2- 172.21.15.39:0/214628151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb07390 0x55fc4fecd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f44a4e00640 1 -- 172.21.15.39:0/214628151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb28070 msgr2=0x55fc4fecc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f44a4e00640 1 --2- 172.21.15.39:0/214628151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb28070 0x55fc4fecc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f44a4e00640 1 -- 172.21.15.39:0/214628151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb07390 msgr2=0x55fc4fecd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f44a4e00640 1 --2- 172.21.15.39:0/214628151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb07390 0x55fc4fecd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f44a4e00640 1 -- 172.21.15.39:0/214628151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4fdca670 con 0x55fc4faff1b0 2024-07-04T08:19:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.879+0000 7f449fe00640 1 --2- 172.21.15.39:0/214628151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb07390 0x55fc4fecd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:19:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.880+0000 7f44a4e00640 1 --2- 172.21.15.39:0/214628151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feccd10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f44940114f0 tx=0x7f44940119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddcbb1db58e82ff2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:17.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.880+0000 7f449d600640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44880b2960 0x7f44880b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.880+0000 7f449d600640 1 -- 172.21.15.39:0/3125666708 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f44980e2fb0 con 0x7f44a010f990 2024-07-04T08:19:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.880+0000 7f4492000640 1 -- 172.21.15.39:0/214628151 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449401e070 con 0x55fc4faff1b0 2024-07-04T08:19:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.880+0000 7f449fe00640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44880b2960 0x7f44880b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.880+0000 7f4492000640 1 -- 172.21.15.39:0/214628151 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44940035f0 con 0x55fc4faff1b0 2024-07-04T08:19:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.880+0000 7f4492000640 1 -- 172.21.15.39:0/214628151 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449400f820 con 0x55fc4faff1b0 2024-07-04T08:19:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f44add8a740 1 -- 172.21.15.39:0/214628151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 msgr2=0x55fc4feccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:17.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f44add8a740 1 --2- 172.21.15.39:0/214628151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feccd10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f44940114f0 tx=0x7f44940119b0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f449fe00640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44880b2960 0x7f44880b4e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f448c0096c0 tx=0x7f448c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:17.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f44add8a740 1 -- 172.21.15.39:0/214628151 shutdown_connections 2024-07-04T08:19:17.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f44add8a740 1 --2- 172.21.15.39:0/214628151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb07390 0x55fc4fecd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f44add8a740 1 --2- 172.21.15.39:0/214628151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feccd10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f44add8a740 1 --2- 172.21.15.39:0/214628151 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb28070 0x55fc4fecc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f44add8a740 1 -- 172.21.15.39:0/214628151 >> 172.21.15.39:0/214628151 conn(0x55fc4fb1c460 msgr2=0x55fc4feca6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:17.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f44add8a740 1 -- 172.21.15.39:0/214628151 shutdown_connections 2024-07-04T08:19:17.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.881+0000 7f44add8a740 1 -- 172.21.15.39:0/214628151 wait complete. 2024-07-04T08:19:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.882+0000 7f44add8a740 1 -- start start 2024-07-04T08:19:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.882+0000 7f44add8a740 1 Processor -- start 2024-07-04T08:19:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44add8a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4faff1b0 0x55fc4fed8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44add8a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fed8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44add8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb28070 0x55fc4fee42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44add8a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fc4fc96700 con 0x55fc4faff1b0 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44add8a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fc4fdd9bd0 con 0x55fc4fb07390 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44add8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fc4fedafb0 con 0x55fc4fb28070 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44add8a740 10 client.0 _ll_register_callbacks cb 0x55fc4fe2b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44a4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4faff1b0 0x55fc4fed8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44a4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4faff1b0 0x55fc4fed8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45000/0 (socket says 172.21.15.39:45000) 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44a4e00640 1 -- 172.21.15.39:0/929589668 learned_addr learned my addr 172.21.15.39:0/929589668 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f449fe00640 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fed8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.883+0000 7f44a5800640 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb28070 0x55fc4fee42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.884+0000 7f44a4e00640 1 -- 172.21.15.39:0/929589668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb28070 msgr2=0x55fc4fee42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.884+0000 7f44a4e00640 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb28070 0x55fc4fee42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.884+0000 7f44a4e00640 1 -- 172.21.15.39:0/929589668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 msgr2=0x55fc4fed8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.884+0000 7f44a4e00640 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fed8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.884+0000 7f44a4e00640 1 -- 172.21.15.39:0/929589668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4fb93650 con 0x55fc4faff1b0 2024-07-04T08:19:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.884+0000 7f449fe00640 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fed8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.884+0000 7f44a4e00640 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4faff1b0 0x55fc4fed8320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4494005e00 tx=0x7f44940041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66f93ee34f644b5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.884+0000 7f4486a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.885+0000 7f4486a00640 1 -- 172.21.15.39:0/929589668 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449401e030 con 0x55fc4faff1b0 2024-07-04T08:19:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.885+0000 7f4486a00640 1 -- 172.21.15.39:0/929589668 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4494005050 con 0x55fc4faff1b0 2024-07-04T08:19:17.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.885+0000 7f44add8a740 1 -- 172.21.15.39:0/929589668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44940035f0 con 0x55fc4faff1b0 2024-07-04T08:19:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.885+0000 7f4486a00640 1 -- 172.21.15.39:0/929589668 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4494031050 con 0x55fc4faff1b0 2024-07-04T08:19:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.885+0000 7f44add8a740 10 client.16341 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.885+0000 7f44add8a740 1 -- 172.21.15.39:0/929589668 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc4fee6fa0 con 0x55fc4faff1b0 2024-07-04T08:19:17.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.886+0000 7f4486a00640 1 -- 172.21.15.39:0/929589668 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44940035f0 con 0x55fc4faff1b0 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.886+0000 7f4486a00640 1 -- 172.21.15.39:0/929589668 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4494004c60 con 0x55fc4faff1b0 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.886+0000 7f44add8a740 10 client.16341 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.886+0000 7f44add8a740 10 client.16341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.886+0000 7f44add8a740 20 client.16341 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.886+0000 7f44add8a740 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55fc4fae69f0 0x55fc4feea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.886+0000 7f44add8a740 4 client.16341 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.886+0000 7f44add8a740 1 -- 172.21.15.39:0/929589668 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc4fe44e50 con 0x55fc4fae69f0 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.886+0000 7f449fe00640 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55fc4fae69f0 0x55fc4feea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.887+0000 7f449fe00640 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55fc4fae69f0 0x55fc4feea350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16737ca7962c1799 server_cookie=43c39587ac0edbc6 in_seq=0 out_seq=0 2024-07-04T08:19:17.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.887+0000 7f4486a00640 10 client.16341 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:19:17.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.888+0000 7f4486a00640 1 -- 172.21.15.39:0/929589668 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc4fe44e50 con 0x55fc4fae69f0 2024-07-04T08:19:17.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.888+0000 7f4486a00640 10 client.16341 handle_command_reply: tid=0 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.888+0000 7f44add8a740 1 client.16341 shutdown 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 -- 172.21.15.39:0/929589668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4faff1b0 msgr2=0x55fc4fed8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4faff1b0 0x55fc4fed8320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4494005e00 tx=0x7f44940041e0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 -- 172.21.15.39:0/929589668 shutdown_connections 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55fc4fae69f0 0x55fc4feea350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb28070 0x55fc4fee42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fed8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 --2- 172.21.15.39:0/929589668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4faff1b0 0x55fc4fed8320 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f4486a00640 0 client.16341 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 -- 172.21.15.39:0/929589668 >> 172.21.15.39:0/929589668 conn(0x55fc4fb1c460 msgr2=0x55fc4fecf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 -- 172.21.15.39:0/929589668 shutdown_connections 2024-07-04T08:19:17.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 1 -- 172.21.15.39:0/929589668 wait complete. 2024-07-04T08:19:17.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.889+0000 7f44add8a740 20 client.16341 trim_cache size 0 max 16384 2024-07-04T08:19:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.904+0000 7f44add8a740 1 Processor -- start 2024-07-04T08:19:17.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.905+0000 7f44add8a740 1 -- start start 2024-07-04T08:19:17.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.905+0000 7f44add8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feda000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.905+0000 7f44add8a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fecdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.905+0000 7f44add8a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 0x55fc4fee40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.905+0000 7f44add8a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b6900 con 0x55fc4fb28070 2024-07-04T08:19:17.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.905+0000 7f44add8a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b8440 con 0x55fc4fb07390 2024-07-04T08:19:17.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.905+0000 7f44add8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b6fd0 con 0x55fc4faff1b0 2024-07-04T08:19:17.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.905+0000 7f44a5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 0x55fc4fee40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.905+0000 7f449fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fecdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f449fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fecdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34544/0 (socket says 172.21.15.39:34544) 2024-07-04T08:19:17.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f44a4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feda000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f449fe00640 1 -- 172.21.15.39:0/187052970 learned_addr learned my addr 172.21.15.39:0/187052970 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:17.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f449fe00640 1 -- 172.21.15.39:0/187052970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 msgr2=0x55fc4feda000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f449fe00640 1 --2- 172.21.15.39:0/187052970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feda000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f449fe00640 1 -- 172.21.15.39:0/187052970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 msgr2=0x55fc4fee40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f449fe00640 1 --2- 172.21.15.39:0/187052970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 0x55fc4fee40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f449fe00640 1 -- 172.21.15.39:0/187052970 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4fed0850 con 0x55fc4fb07390 2024-07-04T08:19:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f44a4e00640 1 --2- 172.21.15.39:0/187052970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feda000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.906+0000 7f449fe00640 1 --2- 172.21.15.39:0/187052970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fecdf10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f448c009230 tx=0x7f448c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=977d5ac9efb40353 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.907+0000 7f4491600640 1 -- 172.21.15.39:0/187052970 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c05b070 con 0x55fc4fb07390 2024-07-04T08:19:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.907+0000 7f4491600640 1 -- 172.21.15.39:0/187052970 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f448c005610 con 0x55fc4fb07390 2024-07-04T08:19:17.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.907+0000 7f4491600640 1 -- 172.21.15.39:0/187052970 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c035040 con 0x55fc4fb07390 2024-07-04T08:19:17.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.908+0000 7f44add8a740 1 -- 172.21.15.39:0/187052970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 msgr2=0x55fc4fecdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:17.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.908+0000 7f44add8a740 1 --2- 172.21.15.39:0/187052970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fecdf10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f448c009230 tx=0x7f448c009260 comp rx=0 tx=0).stop 2024-07-04T08:19:17.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.908+0000 7f44add8a740 1 -- 172.21.15.39:0/187052970 shutdown_connections 2024-07-04T08:19:17.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.908+0000 7f44add8a740 1 --2- 172.21.15.39:0/187052970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 0x55fc4fee40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.908+0000 7f44add8a740 1 --2- 172.21.15.39:0/187052970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4fb07390 0x55fc4fecdf10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.908+0000 7f44add8a740 1 --2- 172.21.15.39:0/187052970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4faff1b0 0x55fc4feda000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.908+0000 7f44add8a740 1 -- 172.21.15.39:0/187052970 >> 172.21.15.39:0/187052970 conn(0x55fc4fb1c460 msgr2=0x55fc4fecd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:17.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.908+0000 7f44add8a740 1 -- 172.21.15.39:0/187052970 shutdown_connections 2024-07-04T08:19:17.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.908+0000 7f44add8a740 1 -- 172.21.15.39:0/187052970 wait complete. 2024-07-04T08:19:17.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.909+0000 7f44add8a740 1 -- start start 2024-07-04T08:19:17.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.909+0000 7f44add8a740 1 Processor -- start 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.909+0000 7f44add8a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4faff1b0 0x55fc4fee06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.909+0000 7f44add8a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb07390 0x55fc4fedbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.909+0000 7f44add8a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 0x55fc4fedc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.909+0000 7f44add8a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b7d70 con 0x55fc4fb28070 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.909+0000 7f44add8a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b76a0 con 0x55fc4faff1b0 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.909+0000 7f44add8a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01b48e0 con 0x55fc4fb07390 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44add8a740 10 client.0 _ll_register_callbacks cb 0x7f449401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44a4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4faff1b0 0x55fc4fee06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44a5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 0x55fc4fedc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44a4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4faff1b0 0x55fc4fee06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34556/0 (socket says 172.21.15.39:34556) 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f449fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb07390 0x55fc4fedbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44a4e00640 1 -- 172.21.15.39:0/1903446342 learned_addr learned my addr 172.21.15.39:0/1903446342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44a4e00640 1 -- 172.21.15.39:0/1903446342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb07390 msgr2=0x55fc4fedbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44a4e00640 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb07390 0x55fc4fedbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44a4e00640 1 -- 172.21.15.39:0/1903446342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 msgr2=0x55fc4fedc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44a4e00640 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 0x55fc4fedc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f44a4e00640 1 -- 172.21.15.39:0/1903446342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4fedc650 con 0x55fc4faff1b0 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.910+0000 7f449fe00640 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb07390 0x55fc4fedbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.911+0000 7f44a4e00640 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4faff1b0 0x55fc4fee06f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4494000c00 tx=0x7f449400ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab79fe3afe954bf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.911+0000 7f4486a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.911+0000 7f4486a00640 1 -- 172.21.15.39:0/1903446342 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449403a060 con 0x55fc4faff1b0 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.911+0000 7f44add8a740 1 -- 172.21.15.39:0/1903446342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f448c005610 con 0x55fc4faff1b0 2024-07-04T08:19:17.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.911+0000 7f4486a00640 1 -- 172.21.15.39:0/1903446342 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4494003af0 con 0x55fc4faff1b0 2024-07-04T08:19:17.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.911+0000 7f4486a00640 1 -- 172.21.15.39:0/1903446342 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4494041050 con 0x55fc4faff1b0 2024-07-04T08:19:17.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.911+0000 7f44add8a740 10 client.26477 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:17.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.911+0000 7f44add8a740 1 -- 172.21.15.39:0/1903446342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc4ff52710 con 0x55fc4faff1b0 2024-07-04T08:19:17.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.912+0000 7f4486a00640 1 -- 172.21.15.39:0/1903446342 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f448c005610 con 0x55fc4faff1b0 2024-07-04T08:19:17.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.912+0000 7f4486a00640 1 -- 172.21.15.39:0/1903446342 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f44940045e0 con 0x55fc4faff1b0 2024-07-04T08:19:17.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.912+0000 7f44add8a740 10 client.26477 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.912+0000 7f44add8a740 10 client.26477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.912+0000 7f44add8a740 20 client.26477 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.912+0000 7f44add8a740 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55fc4fabbda0 0x55fc4ff54e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.912+0000 7f44add8a740 4 client.26477 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.912+0000 7f44add8a740 1 -- 172.21.15.39:0/1903446342 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc4fe408f0 con 0x55fc4fabbda0 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.913+0000 7f449fe00640 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55fc4fabbda0 0x55fc4ff54e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.913+0000 7f449fe00640 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55fc4fabbda0 0x55fc4ff54e10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=553e78351b933c5b server_cookie=2a2f3c9a5dad91a in_seq=0 out_seq=0 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.913+0000 7f4486a00640 10 client.26477 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.913+0000 7f4486a00640 1 -- 172.21.15.39:0/1903446342 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc4fe408f0 con 0x55fc4fabbda0 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.913+0000 7f4486a00640 10 client.26477 handle_command_reply: tid=0 2024-07-04T08:19:17.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.913+0000 7f44add8a740 1 client.26477 shutdown 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 -- 172.21.15.39:0/1903446342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4faff1b0 msgr2=0x55fc4fee06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4faff1b0 0x55fc4fee06f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4494000c00 tx=0x7f449400ffd0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 -- 172.21.15.39:0/1903446342 shutdown_connections 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55fc4fabbda0 0x55fc4ff54e10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fc4fb28070 0x55fc4fedc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fc4fb07390 0x55fc4fedbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 --2- 172.21.15.39:0/1903446342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fc4faff1b0 0x55fc4fee06f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f4486a00640 0 client.26477 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 -- 172.21.15.39:0/1903446342 >> 172.21.15.39:0/1903446342 conn(0x55fc4fb1c460 msgr2=0x55fc4fece1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 -- 172.21.15.39:0/1903446342 shutdown_connections 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 1 -- 172.21.15.39:0/1903446342 wait complete. 2024-07-04T08:19:17.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.914+0000 7f44add8a740 20 client.26477 trim_cache size 0 max 16384 2024-07-04T08:19:17.916 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:19:17.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 -- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44880b2960 msgr2=0x7f44880b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:17.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44880b2960 0x7f44880b4e70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f448c0096c0 tx=0x7f448c031400 comp rx=0 tx=0).stop 2024-07-04T08:19:17.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 -- 172.21.15.39:0/3125666708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44a010f990 msgr2=0x7f44a01a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:17.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44a010f990 0x7f44a01a5910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4498009fd0 tx=0x7f449800ef10 comp rx=0 tx=0).stop 2024-07-04T08:19:17.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 -- 172.21.15.39:0/3125666708 shutdown_connections 2024-07-04T08:19:17.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44880b2960 0x7f44880b4e70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44a010f990 0x7f44a01a5910 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44a010f040 0x7f44a01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 --2- 172.21.15.39:0/3125666708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f44a0109700 0x7f44a01a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:17.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 -- 172.21.15.39:0/3125666708 >> 172.21.15.39:0/3125666708 conn(0x7f44a0077630 msgr2=0x7f44a010d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:17.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 -- 172.21.15.39:0/3125666708 shutdown_connections 2024-07-04T08:19:17.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:17.915+0000 7f44a7800640 1 -- 172.21.15.39:0/3125666708 wait complete. 2024-07-04T08:19:18.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:19:19.257 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:19.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:19 smithi133 ceph-mon[37050]: pgmap v603: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-07-04T08:19:19.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:19 smithi039 ceph-mon[32062]: pgmap v603: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-07-04T08:19:19.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:19 smithi050 ceph-mon[36577]: pgmap v603: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-07-04T08:19:21.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:21 smithi133 ceph-mon[37050]: pgmap v604: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-07-04T08:19:21.319 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:21 smithi039 ceph-mon[32062]: pgmap v604: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-07-04T08:19:21.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:21 smithi050 ceph-mon[36577]: pgmap v604: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-07-04T08:19:21.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.536+0000 7fcf39800640 1 -- 172.21.15.39:0/1126649232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3410f4e0 msgr2=0x7fcf34111900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:21.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.536+0000 7fcf39800640 1 --2- 172.21.15.39:0/1126649232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3410f4e0 0x7fcf34111900 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009ca0 tx=0x7fcf2c02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.536+0000 7fcf39800640 1 -- 172.21.15.39:0/1126649232 shutdown_connections 2024-07-04T08:19:21.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.536+0000 7fcf39800640 1 --2- 172.21.15.39:0/1126649232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3410f4e0 0x7fcf34111900 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.536+0000 7fcf39800640 1 --2- 172.21.15.39:0/1126649232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf340755d0 0x7fcf3410efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.536+0000 7fcf39800640 1 --2- 172.21.15.39:0/1126649232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf34074c80 0x7fcf34075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.536+0000 7fcf39800640 1 -- 172.21.15.39:0/1126649232 >> 172.21.15.39:0/1126649232 conn(0x7fcf340fe950 msgr2=0x7fcf34100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.536+0000 7fcf39800640 1 -- 172.21.15.39:0/1126649232 shutdown_connections 2024-07-04T08:19:21.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.536+0000 7fcf39800640 1 -- 172.21.15.39:0/1126649232 wait complete. 2024-07-04T08:19:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.537+0000 7fcf39800640 1 Processor -- start 2024-07-04T08:19:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.537+0000 7fcf39800640 1 -- start start 2024-07-04T08:19:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf39800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf34074c80 0x7fcf3419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf39800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf340755d0 0x7fcf3419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf39800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3410f4e0 0x7fcf341a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf39800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34115040 con 0x7fcf3410f4e0 2024-07-04T08:19:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf39800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34114ec0 con 0x7fcf340755d0 2024-07-04T08:19:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf39800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341151c0 con 0x7fcf34074c80 2024-07-04T08:19:21.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf33400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf340755d0 0x7fcf3419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf38e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3410f4e0 0x7fcf341a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf34074c80 0x7fcf3419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf33400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf340755d0 0x7fcf3419c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59004/0 (socket says 172.21.15.39:59004) 2024-07-04T08:19:21.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.538+0000 7fcf33400640 1 -- 172.21.15.39:0/3020308621 learned_addr learned my addr 172.21.15.39:0/3020308621 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:21.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.539+0000 7fcf33400640 1 -- 172.21.15.39:0/3020308621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf34074c80 msgr2=0x7fcf3419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:21.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.539+0000 7fcf33400640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf34074c80 0x7fcf3419bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.539+0000 7fcf33400640 1 -- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3410f4e0 msgr2=0x7fcf341a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.539+0000 7fcf33400640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3410f4e0 0x7fcf341a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.539+0000 7fcf33400640 1 -- 172.21.15.39:0/3020308621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf341a1a20 con 0x7fcf340755d0 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.539+0000 7fcf33e00640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf34074c80 0x7fcf3419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.539+0000 7fcf38e00640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3410f4e0 0x7fcf341a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.539+0000 7fcf33400640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf340755d0 0x7fcf3419c050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000e9f0 tx=0x7fcf2000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.540+0000 7fcf30c00640 1 -- 172.21.15.39:0/3020308621 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000ccc0 con 0x7fcf340755d0 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.540+0000 7fcf30c00640 1 -- 172.21.15.39:0/3020308621 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf200108e0 con 0x7fcf340755d0 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.540+0000 7fcf39800640 1 -- 172.21.15.39:0/3020308621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf341a1d10 con 0x7fcf340755d0 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.540+0000 7fcf30c00640 1 -- 172.21.15.39:0/3020308621 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20018860 con 0x7fcf340755d0 2024-07-04T08:19:21.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.540+0000 7fcf39800640 1 -- 172.21.15.39:0/3020308621 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf341a8470 con 0x7fcf340755d0 2024-07-04T08:19:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.543+0000 7fcf4141d740 1 Processor -- start 2024-07-04T08:19:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf4141d740 1 -- start start 2024-07-04T08:19:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40629070 0x55be409cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf4141d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55be40943ee0 con 0x55be406001b0 2024-07-04T08:19:21.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf4141d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55be40933480 con 0x55be40608390 2024-07-04T08:19:21.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf4141d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55be409374c0 con 0x55be40629070 2024-07-04T08:19:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf30c00640 1 -- 172.21.15.39:0/3020308621 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcf20010430 con 0x7fcf340755d0 2024-07-04T08:19:21.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409cde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36030/0 (socket says 172.21.15.39:36030) 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf33e00640 1 -- 172.21.15.39:0/3273305498 learned_addr learned my addr 172.21.15.39:0/3273305498 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.544+0000 7fcf38e00640 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40629070 0x55be409cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.545+0000 7fcf33400640 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.545+0000 7fcf33e00640 1 -- 172.21.15.39:0/3273305498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40629070 msgr2=0x55be409cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.545+0000 7fcf33e00640 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40629070 0x55be409cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.545+0000 7fcf33e00640 1 -- 172.21.15.39:0/3273305498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 msgr2=0x55be409ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.545+0000 7fcf33e00640 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.545+0000 7fcf33e00640 1 -- 172.21.15.39:0/3273305498 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be408cb650 con 0x55be406001b0 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.545+0000 7fcf33400640 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.545+0000 7fcf33e00640 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409cde90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf240115c0 tx=0x7fcf24011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf6f131dcd00678 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:21.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.546+0000 7fcf29600640 1 -- 172.21.15.39:0/3273305498 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400f640 con 0x55be406001b0 2024-07-04T08:19:21.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.546+0000 7fcf29600640 1 -- 172.21.15.39:0/3273305498 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf2400fc90 con 0x55be406001b0 2024-07-04T08:19:21.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.546+0000 7fcf29600640 1 -- 172.21.15.39:0/3273305498 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24019c60 con 0x55be406001b0 2024-07-04T08:19:21.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf30c00640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf180b2960 0x7fcf180b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273305498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 msgr2=0x55be409cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:21.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409cde90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf240115c0 tx=0x7fcf24011a80 comp rx=0 tx=0).stop 2024-07-04T08:19:21.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf30c00640 1 -- 172.21.15.39:0/3020308621 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fcf200e2bf0 con 0x7fcf340755d0 2024-07-04T08:19:21.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf38e00640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf180b2960 0x7fcf180b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273305498 shutdown_connections 2024-07-04T08:19:21.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409cde90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273305498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40629070 0x55be409cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273305498 >> 172.21.15.39:0/3273305498 conn(0x55be4061d460 msgr2=0x55be409cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273305498 shutdown_connections 2024-07-04T08:19:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.547+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273305498 wait complete. 2024-07-04T08:19:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.548+0000 7fcf38e00640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf180b2960 0x7fcf180b4e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006010 tx=0x7fcf2c032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.548+0000 7fcf4141d740 1 -- start start 2024-07-04T08:19:21.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.548+0000 7fcf4141d740 1 Processor -- start 2024-07-04T08:19:21.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409d94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40608390 0x55be409d9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 0x55be409e5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf4141d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55be40797700 con 0x55be40608390 2024-07-04T08:19:21.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf4141d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55be408dabb0 con 0x55be40629070 2024-07-04T08:19:21.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf4141d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55be409dc130 con 0x55be406001b0 2024-07-04T08:19:21.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf4141d740 10 client.0 _ll_register_callbacks cb 0x55be4092c570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:21.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf33400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40608390 0x55be409d9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf33400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40608390 0x55be409d9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36040/0 (socket says 172.21.15.39:36040) 2024-07-04T08:19:21.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf33400640 1 -- 172.21.15.39:0/2464033800 learned_addr learned my addr 172.21.15.39:0/2464033800 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:21.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf33e00640 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409d94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.549+0000 7fcf38e00640 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 0x55be409e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf33400640 1 -- 172.21.15.39:0/2464033800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 msgr2=0x55be409d94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:21.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf33400640 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409d94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf33400640 1 -- 172.21.15.39:0/2464033800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 msgr2=0x55be409e5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:21.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf33400640 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 0x55be409e5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf33400640 1 -- 172.21.15.39:0/2464033800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be40694650 con 0x55be40608390 2024-07-04T08:19:21.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf33400640 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40608390 0x55be409d9a00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf200a98f0 tx=0x7fcf200096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0293f16e737717a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:21.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf1e000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf1e000640 1 -- 172.21.15.39:0/2464033800 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200adef0 con 0x55be40608390 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf1e000640 1 -- 172.21.15.39:0/2464033800 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fcf200a61e0 con 0x55be40608390 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.550+0000 7fcf4141d740 1 -- 172.21.15.39:0/2464033800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2400fc90 con 0x55be40608390 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.551+0000 7fcf1e000640 1 -- 172.21.15.39:0/2464033800 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf200b29a0 con 0x55be40608390 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.551+0000 7fcf4141d740 10 client.16359 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.551+0000 7fcf4141d740 1 -- 172.21.15.39:0/2464033800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be409e8060 con 0x55be40608390 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.551+0000 7fcf1e000640 1 -- 172.21.15.39:0/2464033800 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf2400fc90 con 0x55be40608390 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.551+0000 7fcf1e000640 1 -- 172.21.15.39:0/2464033800 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcf200a88c0 con 0x55be40608390 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.551+0000 7fcf4141d740 10 client.16359 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.551+0000 7fcf4141d740 10 client.16359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.551+0000 7fcf4141d740 20 client.16359 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.551+0000 7fcf4141d740 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55be405e79f0 0x55be409eb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.552+0000 7fcf4141d740 4 client.16359 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.552+0000 7fcf4141d740 1 -- 172.21.15.39:0/2464033800 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be40945e30 con 0x55be405e79f0 2024-07-04T08:19:21.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.552+0000 7fcf33e00640 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55be405e79f0 0x55be409eb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.552+0000 7fcf33e00640 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55be405e79f0 0x55be409eb4f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc6d8eee87d22ae9 server_cookie=6b6d00bb54e3df23 in_seq=0 out_seq=0 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.553+0000 7fcf1e000640 10 client.16359 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.553+0000 7fcf1e000640 1 -- 172.21.15.39:0/2464033800 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55be40945e30 con 0x55be405e79f0 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.553+0000 7fcf1e000640 10 client.16359 handle_command_reply: tid=0 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.553+0000 7fcf4141d740 1 client.16359 shutdown 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 -- 172.21.15.39:0/2464033800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40608390 msgr2=0x55be409d9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40608390 0x55be409d9a00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf200a98f0 tx=0x7fcf200096a0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 -- 172.21.15.39:0/2464033800 shutdown_connections 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55be405e79f0 0x55be409eb4f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40608390 0x55be409d9a00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 0x55be409e5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 --2- 172.21.15.39:0/2464033800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409d94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf1e000640 0 client.16359 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 -- 172.21.15.39:0/2464033800 >> 172.21.15.39:0/2464033800 conn(0x55be4061d460 msgr2=0x55be409d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:21.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 -- 172.21.15.39:0/2464033800 shutdown_connections 2024-07-04T08:19:21.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 1 -- 172.21.15.39:0/2464033800 wait complete. 2024-07-04T08:19:21.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.554+0000 7fcf4141d740 20 client.16359 trim_cache size 0 max 16384 2024-07-04T08:19:21.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.570+0000 7fcf4141d740 1 Processor -- start 2024-07-04T08:19:21.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.570+0000 7fcf4141d740 1 -- start start 2024-07-04T08:19:21.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409daef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40629070 0x55be409e5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf4141d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341b2330 con 0x55be40629070 2024-07-04T08:19:21.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf4141d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341b3e70 con 0x55be40608390 2024-07-04T08:19:21.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf4141d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341b2a00 con 0x55be406001b0 2024-07-04T08:19:21.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50800/0 (socket says 172.21.15.39:50800) 2024-07-04T08:19:21.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf33e00640 1 -- 172.21.15.39:0/79013496 learned_addr learned my addr 172.21.15.39:0/79013496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:21.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf38e00640 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40629070 0x55be409e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.571+0000 7fcf33400640 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409daef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf33e00640 1 -- 172.21.15.39:0/79013496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 msgr2=0x55be409daef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:21.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf33e00640 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409daef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf33e00640 1 -- 172.21.15.39:0/79013496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40629070 msgr2=0x55be409e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:21.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf33e00640 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40629070 0x55be409e5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf33e00640 1 -- 172.21.15.39:0/79013496 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be409cfc40 con 0x55be406001b0 2024-07-04T08:19:21.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf33400640 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409daef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:21.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf33e00640 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409cf340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009d10 tx=0x7fcf24009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2491a1dd17ed89b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:21.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf28c00640 1 -- 172.21.15.39:0/79013496 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24019930 con 0x55be406001b0 2024-07-04T08:19:21.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf28c00640 1 -- 172.21.15.39:0/79013496 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf24028070 con 0x55be406001b0 2024-07-04T08:19:21.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.572+0000 7fcf28c00640 1 -- 172.21.15.39:0/79013496 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400f640 con 0x55be406001b0 2024-07-04T08:19:21.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.573+0000 7fcf4141d740 1 -- 172.21.15.39:0/79013496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 msgr2=0x55be409cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:21.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.573+0000 7fcf4141d740 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409cf340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009d10 tx=0x7fcf24009310 comp rx=0 tx=0).stop 2024-07-04T08:19:21.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.573+0000 7fcf4141d740 1 -- 172.21.15.39:0/79013496 shutdown_connections 2024-07-04T08:19:21.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.573+0000 7fcf4141d740 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be40629070 0x55be409e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.573+0000 7fcf4141d740 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40608390 0x55be409daef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.573+0000 7fcf4141d740 1 --2- 172.21.15.39:0/79013496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be406001b0 0x55be409cf340 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.573+0000 7fcf4141d740 1 -- 172.21.15.39:0/79013496 >> 172.21.15.39:0/79013496 conn(0x55be4061d460 msgr2=0x55be409da7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:21.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.574+0000 7fcf4141d740 1 -- 172.21.15.39:0/79013496 shutdown_connections 2024-07-04T08:19:21.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.574+0000 7fcf4141d740 1 -- 172.21.15.39:0/79013496 wait complete. 2024-07-04T08:19:21.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.574+0000 7fcf4141d740 1 -- start start 2024-07-04T08:19:21.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.574+0000 7fcf4141d740 1 Processor -- start 2024-07-04T08:19:21.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40608390 0x55be409dda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf4141d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 0x55be40a52c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf4141d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341b37a0 con 0x55be406001b0 2024-07-04T08:19:21.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf4141d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341b30d0 con 0x55be40629070 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf4141d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341b0310 con 0x55be40608390 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf4141d740 10 client.0 _ll_register_callbacks cb 0x7fcf200a5ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409dd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36058/0 (socket says 172.21.15.39:36058) 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.575+0000 7fcf33e00640 1 -- 172.21.15.39:0/3273541318 learned_addr learned my addr 172.21.15.39:0/3273541318 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.576+0000 7fcf33400640 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40608390 0x55be409dda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.576+0000 7fcf38e00640 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 0x55be40a52c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.576+0000 7fcf33e00640 1 -- 172.21.15.39:0/3273541318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40608390 msgr2=0x55be409dda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.576+0000 7fcf33e00640 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40608390 0x55be409dda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.576+0000 7fcf33e00640 1 -- 172.21.15.39:0/3273541318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 msgr2=0x55be40a52c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:21.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.576+0000 7fcf33e00640 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 0x55be40a52c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.576+0000 7fcf33e00640 1 -- 172.21.15.39:0/3273541318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be409de210 con 0x55be406001b0 2024-07-04T08:19:21.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.576+0000 7fcf33400640 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40608390 0x55be409dda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:21.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.577+0000 7fcf33e00640 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409dd4c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcf240058c0 tx=0x7fcf24005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f98b06ac46a21fe server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:21.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.577+0000 7fcf1e000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:21.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.577+0000 7fcf1e000640 1 -- 172.21.15.39:0/3273541318 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24005650 con 0x55be406001b0 2024-07-04T08:19:21.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.577+0000 7fcf1e000640 1 -- 172.21.15.39:0/3273541318 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fcf24019900 con 0x55be406001b0 2024-07-04T08:19:21.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.577+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273541318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf24028070 con 0x55be406001b0 2024-07-04T08:19:21.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.577+0000 7fcf1e000640 1 -- 172.21.15.39:0/3273541318 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf240050b0 con 0x55be406001b0 2024-07-04T08:19:21.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.577+0000 7fcf4141d740 10 client.16365 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:21.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.577+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273541318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55be40a54180 con 0x55be406001b0 2024-07-04T08:19:21.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.578+0000 7fcf1e000640 1 -- 172.21.15.39:0/3273541318 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf24028070 con 0x55be406001b0 2024-07-04T08:19:21.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.578+0000 7fcf1e000640 1 -- 172.21.15.39:0/3273541318 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcf2402d470 con 0x55be406001b0 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.578+0000 7fcf4141d740 10 client.16365 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.578+0000 7fcf4141d740 10 client.16365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.578+0000 7fcf4141d740 20 client.16365 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.578+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55be405bcda0 0x55be40a56710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.578+0000 7fcf4141d740 4 client.16365 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.578+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273541318 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be409418d0 con 0x55be405bcda0 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.578+0000 7fcf33400640 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55be405bcda0 0x55be40a56710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.579+0000 7fcf33400640 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55be405bcda0 0x55be40a56710 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b754e1416dd06aad server_cookie=a4e3307feb7ff9ad in_seq=0 out_seq=0 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.579+0000 7fcf1e000640 10 client.16365 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.579+0000 7fcf1e000640 1 -- 172.21.15.39:0/3273541318 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be409418d0 con 0x55be405bcda0 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.579+0000 7fcf1e000640 10 client.16365 handle_command_reply: tid=0 2024-07-04T08:19:21.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.579+0000 7fcf4141d740 1 client.16365 shutdown 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273541318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 msgr2=0x55be409dd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409dd4c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcf240058c0 tx=0x7fcf24005dc0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273541318 shutdown_connections 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55be405bcda0 0x55be40a56710 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55be40629070 0x55be40a52c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be40608390 0x55be409dda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf1e000640 0 client.16365 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 --2- 172.21.15.39:0/3273541318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55be406001b0 0x55be409dd4c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273541318 >> 172.21.15.39:0/3273541318 conn(0x55be4061d460 msgr2=0x55be409cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273541318 shutdown_connections 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 1 -- 172.21.15.39:0/3273541318 wait complete. 2024-07-04T08:19:21.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.580+0000 7fcf4141d740 20 client.16365 trim_cache size 0 max 16384 2024-07-04T08:19:21.582 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:19:21.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 -- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf180b2960 msgr2=0x7fcf180b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:21.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf180b2960 0x7fcf180b4e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006010 tx=0x7fcf2c032ed0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 -- 172.21.15.39:0/3020308621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf340755d0 msgr2=0x7fcf3419c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf340755d0 0x7fcf3419c050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000e9f0 tx=0x7fcf2000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 -- 172.21.15.39:0/3020308621 shutdown_connections 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcf180b2960 0x7fcf180b4e70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3410f4e0 0x7fcf341a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf340755d0 0x7fcf3419c050 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 --2- 172.21.15.39:0/3020308621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf34074c80 0x7fcf3419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.581+0000 7fcf39800640 1 -- 172.21.15.39:0/3020308621 >> 172.21.15.39:0/3020308621 conn(0x7fcf340fe950 msgr2=0x7fcf34108ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.582+0000 7fcf39800640 1 -- 172.21.15.39:0/3020308621 shutdown_connections 2024-07-04T08:19:21.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:21.582+0000 7fcf39800640 1 -- 172.21.15.39:0/3020308621 wait complete. 2024-07-04T08:19:22.487 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:19:22.975 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:23.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:23 smithi133 ceph-mon[37050]: pgmap v605: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-07-04T08:19:23.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:23 smithi039 ceph-mon[32062]: pgmap v605: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-07-04T08:19:23.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:23 smithi050 ceph-mon[36577]: pgmap v605: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-07-04T08:19:24.772 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:24 smithi039 ceph-mon[32062]: pgmap v606: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-07-04T08:19:24.813 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:24 smithi133 ceph-mon[37050]: pgmap v606: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-07-04T08:19:24.892 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:24 smithi050 ceph-mon[36577]: pgmap v606: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-07-04T08:19:25.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.352+0000 7fb04b200640 1 -- 172.21.15.39:0/255914936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb044107700 msgr2=0x7fb044107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:25.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.352+0000 7fb04b200640 1 --2- 172.21.15.39:0/255914936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb044107700 0x7fb044107b10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb034009860 tx=0x7fb03402a340 comp rx=0 tx=0).stop 2024-07-04T08:19:25.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.352+0000 7fb04b200640 1 -- 172.21.15.39:0/255914936 shutdown_connections 2024-07-04T08:19:25.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.352+0000 7fb04b200640 1 --2- 172.21.15.39:0/255914936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb044108050 0x7fb04410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.352+0000 7fb04b200640 1 --2- 172.21.15.39:0/255914936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb044107700 0x7fb044107b10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.352+0000 7fb04b200640 1 --2- 172.21.15.39:0/255914936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb044101dc0 0x7fb0441021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.352+0000 7fb04b200640 1 -- 172.21.15.39:0/255914936 >> 172.21.15.39:0/255914936 conn(0x7fb0440fd810 msgr2=0x7fb0440ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:25.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.352+0000 7fb04b200640 1 -- 172.21.15.39:0/255914936 shutdown_connections 2024-07-04T08:19:25.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.353+0000 7fb04b200640 1 -- 172.21.15.39:0/255914936 wait complete. 2024-07-04T08:19:25.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.354+0000 7fb04b200640 1 Processor -- start 2024-07-04T08:19:25.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.354+0000 7fb04b200640 1 -- start start 2024-07-04T08:19:25.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.354+0000 7fb04b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb044101dc0 0x7fb044197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.354+0000 7fb04b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb044107700 0x7fb044197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.354+0000 7fb04b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb044108050 0x7fb04419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.354+0000 7fb04b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb044110c10 con 0x7fb044101dc0 2024-07-04T08:19:25.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.354+0000 7fb04b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb044110a90 con 0x7fb044107700 2024-07-04T08:19:25.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.354+0000 7fb04b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb044110d90 con 0x7fb044108050 2024-07-04T08:19:25.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb044107700 0x7fb044197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb044107700 0x7fb044197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59056/0 (socket says 172.21.15.39:59056) 2024-07-04T08:19:25.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb044101dc0 0x7fb044197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049400640 1 -- 172.21.15.39:0/1965368419 learned_addr learned my addr 172.21.15.39:0/1965368419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:25.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb04a800640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb044108050 0x7fb04419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049400640 1 -- 172.21.15.39:0/1965368419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb044108050 msgr2=0x7fb04419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049400640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb044108050 0x7fb04419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049400640 1 -- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb044101dc0 msgr2=0x7fb044197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049400640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb044101dc0 0x7fb044197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049400640 1 -- 172.21.15.39:0/1965368419 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04419d570 con 0x7fb044107700 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.355+0000 7fb049e00640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb044101dc0 0x7fb044197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.356+0000 7fb049400640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb044107700 0x7fb044197ba0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb034035040 tx=0x7fb034032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.356+0000 7fb03ea00640 1 -- 172.21.15.39:0/1965368419 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034037040 con 0x7fb044107700 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.356+0000 7fb04b200640 1 -- 172.21.15.39:0/1965368419 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04419d800 con 0x7fb044107700 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.356+0000 7fb03ea00640 1 -- 172.21.15.39:0/1965368419 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb034004050 con 0x7fb044107700 2024-07-04T08:19:25.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.356+0000 7fb03ea00640 1 -- 172.21.15.39:0/1965368419 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0340035f0 con 0x7fb044107700 2024-07-04T08:19:25.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.356+0000 7fb04b200640 1 -- 172.21.15.39:0/1965368419 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0441a83d0 con 0x7fb044107700 2024-07-04T08:19:25.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.360+0000 7fb052d70740 1 Processor -- start 2024-07-04T08:19:25.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.360+0000 7fb052d70740 1 -- start start 2024-07-04T08:19:25.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.360+0000 7fb052d70740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b50070 0x564bb1ef4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.360+0000 7fb052d70740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b271b0 0x564bb1ef4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.360+0000 7fb052d70740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 0x564bb1ef53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.360+0000 7fb052d70740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564bb1e6af00 con 0x564bb1b50070 2024-07-04T08:19:25.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.360+0000 7fb052d70740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564bb1e5a4a0 con 0x564bb1b2f390 2024-07-04T08:19:25.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.360+0000 7fb052d70740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564bb1e5e4e0 con 0x564bb1b271b0 2024-07-04T08:19:25.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.360+0000 7fb03ea00640 1 -- 172.21.15.39:0/1965368419 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb034043020 con 0x7fb044107700 2024-07-04T08:19:25.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb04a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b50070 0x564bb1ef4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb049e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b271b0 0x564bb1ef4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb04a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b50070 0x564bb1ef4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36090/0 (socket says 172.21.15.39:36090) 2024-07-04T08:19:25.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb04a800640 1 -- 172.21.15.39:0/2701062395 learned_addr learned my addr 172.21.15.39:0/2701062395 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:25.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb049e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b271b0 0x564bb1ef4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50834/0 (socket says 172.21.15.39:50834) 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb049400640 1 --2- 172.21.15.39:0/2701062395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 0x564bb1ef53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb049e00640 1 -- 172.21.15.39:0/2701062395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 msgr2=0x564bb1ef53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb049e00640 1 --2- 172.21.15.39:0/2701062395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 0x564bb1ef53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb049e00640 1 -- 172.21.15.39:0/2701062395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b50070 msgr2=0x564bb1ef4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb049e00640 1 --2- 172.21.15.39:0/2701062395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b50070 0x564bb1ef4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.361+0000 7fb049e00640 1 -- 172.21.15.39:0/2701062395 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb1df2670 con 0x564bb1b271b0 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.362+0000 7fb049e00640 1 --2- 172.21.15.39:0/2701062395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b271b0 0x564bb1ef4eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb038011440 tx=0x7fb038011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e029743682773353 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.362+0000 7fb033400640 1 -- 172.21.15.39:0/2701062395 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038004310 con 0x564bb1b271b0 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.362+0000 7fb033400640 1 -- 172.21.15.39:0/2701062395 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0380044b0 con 0x564bb1b271b0 2024-07-04T08:19:25.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.362+0000 7fb033400640 1 -- 172.21.15.39:0/2701062395 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03800fa70 con 0x564bb1b271b0 2024-07-04T08:19:25.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.362+0000 7fb03ea00640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb02c0b2950 0x7fb02c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.363+0000 7fb03ea00640 1 -- 172.21.15.39:0/1965368419 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb034101b00 con 0x7fb044107700 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.363+0000 7fb04a800640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb02c0b2950 0x7fb02c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.363+0000 7fb052d70740 1 -- 172.21.15.39:0/2701062395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b271b0 msgr2=0x564bb1ef4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.363+0000 7fb052d70740 1 --2- 172.21.15.39:0/2701062395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b271b0 0x564bb1ef4eb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb038011440 tx=0x7fb038011900 comp rx=0 tx=0).stop 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.363+0000 7fb052d70740 1 -- 172.21.15.39:0/2701062395 shutdown_connections 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.363+0000 7fb052d70740 1 --2- 172.21.15.39:0/2701062395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 0x564bb1ef53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.363+0000 7fb052d70740 1 --2- 172.21.15.39:0/2701062395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b271b0 0x564bb1ef4eb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.363+0000 7fb052d70740 1 --2- 172.21.15.39:0/2701062395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b50070 0x564bb1ef4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.363+0000 7fb052d70740 1 -- 172.21.15.39:0/2701062395 >> 172.21.15.39:0/2701062395 conn(0x564bb1b44460 msgr2=0x564bb1ef2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.364+0000 7fb052d70740 1 -- 172.21.15.39:0/2701062395 shutdown_connections 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.364+0000 7fb052d70740 1 -- 172.21.15.39:0/2701062395 wait complete. 2024-07-04T08:19:25.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.364+0000 7fb04a800640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb02c0b2950 0x7fb02c0b4e60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb0400116b0 tx=0x7fb040009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:25.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.364+0000 7fb052d70740 1 -- start start 2024-07-04T08:19:25.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.364+0000 7fb052d70740 1 Processor -- start 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb052d70740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f00520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb052d70740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 0x564bb1f00a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb052d70740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b50070 0x564bb1f0c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb052d70740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564bb1cbe700 con 0x564bb1b271b0 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb052d70740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564bb1e01bd0 con 0x564bb1b2f390 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb052d70740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564bb1f03150 con 0x564bb1b50070 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb052d70740 10 client.0 _ll_register_callbacks cb 0x564bb1e53590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb049e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f00520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb049e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f00520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36106/0 (socket says 172.21.15.39:36106) 2024-07-04T08:19:25.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.365+0000 7fb049e00640 1 -- 172.21.15.39:0/580977740 learned_addr learned my addr 172.21.15.39:0/580977740 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb049400640 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 0x564bb1f00a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb04a800640 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b50070 0x564bb1f0c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb049e00640 1 -- 172.21.15.39:0/580977740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b50070 msgr2=0x564bb1f0c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb049e00640 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b50070 0x564bb1f0c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb049e00640 1 -- 172.21.15.39:0/580977740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 msgr2=0x564bb1f00a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb049e00640 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 0x564bb1f00a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb049e00640 1 -- 172.21.15.39:0/580977740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb1bbb650 con 0x564bb1b271b0 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb049400640 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 0x564bb1f00a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb049e00640 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f00520 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb03801b040 tx=0x7fb038009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7daf95e92ea2a2f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:25.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.366+0000 7fb027e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:25.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.367+0000 7fb027e00640 1 -- 172.21.15.39:0/580977740 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038029020 con 0x564bb1b271b0 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.367+0000 7fb027e00640 1 -- 172.21.15.39:0/580977740 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb03800f9d0 con 0x564bb1b271b0 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.367+0000 7fb052d70740 1 -- 172.21.15.39:0/580977740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0380044b0 con 0x564bb1b271b0 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.367+0000 7fb027e00640 1 -- 172.21.15.39:0/580977740 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb038034050 con 0x564bb1b271b0 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.367+0000 7fb052d70740 10 client.16377 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.367+0000 7fb052d70740 1 -- 172.21.15.39:0/580977740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564bb1f0f140 con 0x564bb1b271b0 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.368+0000 7fb027e00640 1 -- 172.21.15.39:0/580977740 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0380044b0 con 0x564bb1b271b0 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.368+0000 7fb027e00640 1 -- 172.21.15.39:0/580977740 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb0380035f0 con 0x564bb1b271b0 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.368+0000 7fb052d70740 10 client.16377 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.368+0000 7fb052d70740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.368+0000 7fb052d70740 20 client.16377 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.368+0000 7fb052d70740 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564bb1b0e9f0 0x564bb1f12550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.368+0000 7fb052d70740 4 client.16377 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:25.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.368+0000 7fb052d70740 1 -- 172.21.15.39:0/580977740 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bb1e6ce50 con 0x564bb1b0e9f0 2024-07-04T08:19:25.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.369+0000 7fb049400640 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564bb1b0e9f0 0x564bb1f12550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.369+0000 7fb049400640 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564bb1b0e9f0 0x564bb1f12550 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b286f6a8ab51863 server_cookie=db47adb990a19589 in_seq=0 out_seq=0 2024-07-04T08:19:25.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.369+0000 7fb027e00640 10 client.16377 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:25.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.370+0000 7fb027e00640 1 -- 172.21.15.39:0/580977740 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564bb1e6ce50 con 0x564bb1b0e9f0 2024-07-04T08:19:25.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.370+0000 7fb027e00640 10 client.16377 handle_command_reply: tid=0 2024-07-04T08:19:25.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 client.16377 shutdown 2024-07-04T08:19:25.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 -- 172.21.15.39:0/580977740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 msgr2=0x564bb1f00520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:25.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f00520 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb03801b040 tx=0x7fb038009600 comp rx=0 tx=0).stop 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 -- 172.21.15.39:0/580977740 shutdown_connections 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564bb1b0e9f0 0x564bb1f12550 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b50070 0x564bb1f0c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b2f390 0x564bb1f00a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 --2- 172.21.15.39:0/580977740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f00520 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 -- 172.21.15.39:0/580977740 >> 172.21.15.39:0/580977740 conn(0x564bb1b44460 msgr2=0x564bb1ef7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb027e00640 0 client.16377 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 -- 172.21.15.39:0/580977740 shutdown_connections 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 1 -- 172.21.15.39:0/580977740 wait complete. 2024-07-04T08:19:25.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.371+0000 7fb052d70740 20 client.16377 trim_cache size 0 max 16384 2024-07-04T08:19:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.387+0000 7fb052d70740 1 Processor -- start 2024-07-04T08:19:25.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.387+0000 7fb052d70740 1 -- start start 2024-07-04T08:19:25.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.387+0000 7fb052d70740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1ef6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.387+0000 7fb052d70740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f02200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.387+0000 7fb052d70740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 0x564bb1ef8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.387+0000 7fb052d70740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441b22a0 con 0x564bb1b271b0 2024-07-04T08:19:25.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.387+0000 7fb052d70740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441b3de0 con 0x564bb1b50070 2024-07-04T08:19:25.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.387+0000 7fb052d70740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441b2970 con 0x564bb1b2f390 2024-07-04T08:19:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1ef6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1ef6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36116/0 (socket says 172.21.15.39:36116) 2024-07-04T08:19:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049e00640 1 -- 172.21.15.39:0/3956454581 learned_addr learned my addr 172.21.15.39:0/3956454581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049400640 1 --2- 172.21.15.39:0/3956454581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f02200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb04a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 0x564bb1ef8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049e00640 1 -- 172.21.15.39:0/3956454581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 msgr2=0x564bb1f02200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049e00640 1 --2- 172.21.15.39:0/3956454581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f02200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049e00640 1 -- 172.21.15.39:0/3956454581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 msgr2=0x564bb1ef8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049e00640 1 --2- 172.21.15.39:0/3956454581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 0x564bb1ef8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049e00640 1 -- 172.21.15.39:0/3956454581 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb1ef77d0 con 0x564bb1b271b0 2024-07-04T08:19:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.388+0000 7fb049400640 1 --2- 172.21.15.39:0/3956454581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f02200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:25.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.389+0000 7fb049e00640 1 --2- 172.21.15.39:0/3956454581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1ef6470 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb038003a90 tx=0x7fb038009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8099ab9fcfd0a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.389+0000 7fb032a00640 1 -- 172.21.15.39:0/3956454581 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038029040 con 0x564bb1b271b0 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.389+0000 7fb032a00640 1 -- 172.21.15.39:0/3956454581 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03800f540 con 0x564bb1b271b0 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.389+0000 7fb032a00640 1 -- 172.21.15.39:0/3956454581 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038025470 con 0x564bb1b271b0 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.390+0000 7fb052d70740 1 -- 172.21.15.39:0/3956454581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 msgr2=0x564bb1ef6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.390+0000 7fb052d70740 1 --2- 172.21.15.39:0/3956454581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1ef6470 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb038003a90 tx=0x7fb038009d60 comp rx=0 tx=0).stop 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.390+0000 7fb052d70740 1 -- 172.21.15.39:0/3956454581 shutdown_connections 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.390+0000 7fb052d70740 1 --2- 172.21.15.39:0/3956454581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 0x564bb1ef8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.390+0000 7fb052d70740 1 --2- 172.21.15.39:0/3956454581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f02200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.390+0000 7fb052d70740 1 --2- 172.21.15.39:0/3956454581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1ef6470 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.390+0000 7fb052d70740 1 -- 172.21.15.39:0/3956454581 >> 172.21.15.39:0/3956454581 conn(0x564bb1b44460 msgr2=0x564bb1ef7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.391+0000 7fb052d70740 1 -- 172.21.15.39:0/3956454581 shutdown_connections 2024-07-04T08:19:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.391+0000 7fb052d70740 1 -- 172.21.15.39:0/3956454581 wait complete. 2024-07-04T08:19:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.391+0000 7fb052d70740 1 -- start start 2024-07-04T08:19:25.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.392+0000 7fb052d70740 1 Processor -- start 2024-07-04T08:19:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.392+0000 7fb052d70740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f0b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.392+0000 7fb052d70740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f0bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.392+0000 7fb052d70740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 0x564bb1f073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.392+0000 7fb052d70740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441b3710 con 0x564bb1b271b0 2024-07-04T08:19:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.392+0000 7fb052d70740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441b3040 con 0x564bb1b50070 2024-07-04T08:19:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.392+0000 7fb052d70740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb0441b0280 con 0x564bb1b2f390 2024-07-04T08:19:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb052d70740 10 client.0 _ll_register_callbacks cb 0x7fb040009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:25.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f0bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f0b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f0bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50858/0 (socket says 172.21.15.39:50858) 2024-07-04T08:19:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049400640 1 -- 172.21.15.39:0/2490445851 learned_addr learned my addr 172.21.15.39:0/2490445851 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:25.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb04a800640 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 0x564bb1f073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049400640 1 -- 172.21.15.39:0/2490445851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 msgr2=0x564bb1f073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049400640 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 0x564bb1f073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049400640 1 -- 172.21.15.39:0/2490445851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 msgr2=0x564bb1f0b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:25.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049400640 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f0b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049400640 1 -- 172.21.15.39:0/2490445851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb1f0c420 con 0x564bb1b2f390 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.393+0000 7fb049e00640 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f0b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.394+0000 7fb049400640 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f0bba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb0340c4be0 tx=0x7fb0340eb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcf326ac4b16e67e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.394+0000 7fb027e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.394+0000 7fb027e00640 1 -- 172.21.15.39:0/2490445851 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034109040 con 0x564bb1b2f390 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.394+0000 7fb027e00640 1 -- 172.21.15.39:0/2490445851 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fb0340ff140 con 0x564bb1b2f390 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.394+0000 7fb052d70740 1 -- 172.21.15.39:0/2490445851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb03800f540 con 0x564bb1b2f390 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.394+0000 7fb027e00640 1 -- 172.21.15.39:0/2490445851 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb0340c8930 con 0x564bb1b2f390 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.394+0000 7fb052d70740 10 client.26440 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.394+0000 7fb052d70740 1 -- 172.21.15.39:0/2490445851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564bb1f08920 con 0x564bb1b2f390 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.395+0000 7fb027e00640 1 -- 172.21.15.39:0/2490445851 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03800f540 con 0x564bb1b2f390 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.395+0000 7fb027e00640 1 -- 172.21.15.39:0/2490445851 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb0340d1530 con 0x564bb1b2f390 2024-07-04T08:19:25.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.395+0000 7fb052d70740 10 client.26440 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:25.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.395+0000 7fb052d70740 10 client.26440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:19:25.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.395+0000 7fb052d70740 20 client.26440 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:25.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.395+0000 7fb052d70740 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564bb1ae3da0 0x564bb1f7cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:25.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.395+0000 7fb052d70740 4 client.26440 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:19:25.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.395+0000 7fb052d70740 1 -- 172.21.15.39:0/2490445851 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bb1e688f0 con 0x564bb1ae3da0 2024-07-04T08:19:25.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.395+0000 7fb049e00640 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564bb1ae3da0 0x564bb1f7cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:25.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.396+0000 7fb049e00640 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564bb1ae3da0 0x564bb1f7cc30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39de181160ae74f server_cookie=d15c2f5ab0e466e6 in_seq=0 out_seq=0 2024-07-04T08:19:25.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.396+0000 7fb027e00640 10 client.26440 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:25.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.396+0000 7fb027e00640 1 -- 172.21.15.39:0/2490445851 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bb1e688f0 con 0x564bb1ae3da0 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.396+0000 7fb027e00640 10 client.26440 handle_command_reply: tid=0 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.396+0000 7fb052d70740 1 client.26440 shutdown 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 -- 172.21.15.39:0/2490445851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 msgr2=0x564bb1f0bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f0bba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb0340c4be0 tx=0x7fb0340eb040 comp rx=0 tx=0).stop 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 -- 172.21.15.39:0/2490445851 shutdown_connections 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564bb1ae3da0 0x564bb1f7cc30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564bb1b50070 0x564bb1f073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bb1b2f390 0x564bb1f0bba0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 --2- 172.21.15.39:0/2490445851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564bb1b271b0 0x564bb1f0b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb027e00640 0 client.26440 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 -- 172.21.15.39:0/2490445851 >> 172.21.15.39:0/2490445851 conn(0x564bb1b44460 msgr2=0x564bb1f015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 -- 172.21.15.39:0/2490445851 shutdown_connections 2024-07-04T08:19:25.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 1 -- 172.21.15.39:0/2490445851 wait complete. 2024-07-04T08:19:25.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.397+0000 7fb052d70740 20 client.26440 trim_cache size 0 max 16384 2024-07-04T08:19:25.399 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 -- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb02c0b2950 msgr2=0x7fb02c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb02c0b2950 0x7fb02c0b4e60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb0400116b0 tx=0x7fb040009500 comp rx=0 tx=0).stop 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 -- 172.21.15.39:0/1965368419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb044107700 msgr2=0x7fb044197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb044107700 0x7fb044197ba0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb034035040 tx=0x7fb034032d50 comp rx=0 tx=0).stop 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 -- 172.21.15.39:0/1965368419 shutdown_connections 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb02c0b2950 0x7fb02c0b4e60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb044108050 0x7fb04419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb044107700 0x7fb044197ba0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 --2- 172.21.15.39:0/1965368419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb044101dc0 0x7fb044197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 -- 172.21.15.39:0/1965368419 >> 172.21.15.39:0/1965368419 conn(0x7fb0440fd810 msgr2=0x7fb0440ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:25.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 -- 172.21.15.39:0/1965368419 shutdown_connections 2024-07-04T08:19:25.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:25.398+0000 7fb04b200640 1 -- 172.21.15.39:0/1965368419 wait complete. 2024-07-04T08:19:27.142 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:26 smithi050 ceph-mon[36577]: pgmap v607: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-07-04T08:19:27.283 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:19:27.284 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:19:27.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:26 smithi133 ceph-mon[37050]: pgmap v607: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-07-04T08:19:27.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:26 smithi039 ceph-mon[32062]: pgmap v607: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-07-04T08:19:27.778 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:29.078 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:28 smithi039 ceph-mon[32062]: pgmap v608: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-07-04T08:19:29.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:28 smithi133 ceph-mon[37050]: pgmap v608: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-07-04T08:19:29.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:28 smithi050 ceph-mon[36577]: pgmap v608: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.8 KiB/s wr, 11 op/s 2024-07-04T08:19:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.957+0000 7f2f25a00640 1 -- 172.21.15.39:0/3006670445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f20074d00 msgr2=0x7f2f20075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.957+0000 7f2f25a00640 1 --2- 172.21.15.39:0/3006670445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f20074d00 0x7f2f20075110 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009900 tx=0x7f2f1002a410 comp rx=0 tx=0).stop 2024-07-04T08:19:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.957+0000 7f2f25a00640 1 -- 172.21.15.39:0/3006670445 shutdown_connections 2024-07-04T08:19:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.957+0000 7f2f25a00640 1 --2- 172.21.15.39:0/3006670445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2010f460 0x7f2f201118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.957+0000 7f2f25a00640 1 --2- 172.21.15.39:0/3006670445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f20075650 0x7f2f2010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.957+0000 7f2f25a00640 1 --2- 172.21.15.39:0/3006670445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f20074d00 0x7f2f20075110 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.957+0000 7f2f25a00640 1 -- 172.21.15.39:0/3006670445 >> 172.21.15.39:0/3006670445 conn(0x7f2f200fe990 msgr2=0x7f2f20100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.957+0000 7f2f25a00640 1 -- 172.21.15.39:0/3006670445 shutdown_connections 2024-07-04T08:19:29.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.957+0000 7f2f25a00640 1 -- 172.21.15.39:0/3006670445 wait complete. 2024-07-04T08:19:29.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.958+0000 7f2f25a00640 1 Processor -- start 2024-07-04T08:19:29.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.958+0000 7f2f25a00640 1 -- start start 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f25a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f20074d00 0x7f2f2019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f25a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f20075650 0x7f2f2019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f25a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2010f460 0x7f2f201a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f25a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20115060 con 0x7f2f2010f460 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f25a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20114ee0 con 0x7f2f20074d00 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f25a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201151e0 con 0x7f2f20075650 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f25000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2010f460 0x7f2f201a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f25000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2010f460 0x7f2f201a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36150/0 (socket says 172.21.15.39:36150) 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f25000640 1 -- 172.21.15.39:0/36185930 learned_addr learned my addr 172.21.15.39:0/36185930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f1f400640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f20075650 0x7f2f2019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.959+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f20074d00 0x7f2f2019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.960+0000 7f2f25000640 1 -- 172.21.15.39:0/36185930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f20075650 msgr2=0x7f2f2019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:29.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.960+0000 7f2f25000640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f20075650 0x7f2f2019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.960+0000 7f2f25000640 1 -- 172.21.15.39:0/36185930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f20074d00 msgr2=0x7f2f2019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:29.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.960+0000 7f2f25000640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f20074d00 0x7f2f2019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.960+0000 7f2f25000640 1 -- 172.21.15.39:0/36185930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f201a19a0 con 0x7f2f2010f460 2024-07-04T08:19:29.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.960+0000 7f2f25000640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2010f460 0x7f2f201a1270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800e970 tx=0x7f2f1800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:29.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.960+0000 7f2f1cc00640 1 -- 172.21.15.39:0/36185930 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800ccc0 con 0x7f2f2010f460 2024-07-04T08:19:29.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.961+0000 7f2f1cc00640 1 -- 172.21.15.39:0/36185930 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f1800ce60 con 0x7f2f2010f460 2024-07-04T08:19:29.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.961+0000 7f2f25a00640 1 -- 172.21.15.39:0/36185930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f201a1c90 con 0x7f2f2010f460 2024-07-04T08:19:29.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.961+0000 7f2f1cc00640 1 -- 172.21.15.39:0/36185930 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18010430 con 0x7f2f2010f460 2024-07-04T08:19:29.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.961+0000 7f2f25a00640 1 -- 172.21.15.39:0/36185930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201a83f0 con 0x7f2f2010f460 2024-07-04T08:19:29.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.965+0000 7f2f2d6d1740 1 Processor -- start 2024-07-04T08:19:29.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.966+0000 7f2f1cc00640 1 -- 172.21.15.39:0/36185930 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2f18004ff0 con 0x7f2f2010f460 2024-07-04T08:19:29.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.966+0000 7f2f2d6d1740 1 -- start start 2024-07-04T08:19:29.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.966+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f80800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bb3490 0x5625f9f80d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 0x5625f9f81280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5625f9eaa8f0 con 0x5625f9bb3490 2024-07-04T08:19:29.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5625f9ef6ed0 con 0x5625f9bbb670 2024-07-04T08:19:29.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5625f9ee6470 con 0x5625f9bdc350 2024-07-04T08:19:29.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f1fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bb3490 0x5625f9f80d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f1f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f80800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f1f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f80800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50886/0 (socket says 172.21.15.39:50886) 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f1fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bb3490 0x5625f9f80d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36164/0 (socket says 172.21.15.39:36164) 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f1f400640 1 -- 172.21.15.39:0/3184030397 learned_addr learned my addr 172.21.15.39:0/3184030397 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f1f400640 1 --2- 172.21.15.39:0/3184030397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 0x5625f9f81280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f1f400640 1 -- 172.21.15.39:0/3184030397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 msgr2=0x5625f9f81280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.967+0000 7f2f1f400640 1 --2- 172.21.15.39:0/3184030397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 0x5625f9f81280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.968+0000 7f2f1f400640 1 -- 172.21.15.39:0/3184030397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bb3490 msgr2=0x5625f9f80d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.968+0000 7f2f1f400640 1 --2- 172.21.15.39:0/3184030397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bb3490 0x5625f9f80d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.968+0000 7f2f1f400640 1 -- 172.21.15.39:0/3184030397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f9e7e290 con 0x5625f9bdc350 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.968+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/3184030397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bb3490 0x5625f9f80d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:29.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.968+0000 7f2f1f400640 1 --2- 172.21.15.39:0/3184030397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f80800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f08011b40 tx=0x7f2f08005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14c33c78e861bced server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:29.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.968+0000 7f2f15600640 1 -- 172.21.15.39:0/3184030397 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08011050 con 0x5625f9bdc350 2024-07-04T08:19:29.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.968+0000 7f2f1cc00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2f000b2960 0x7f2f000b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.968+0000 7f2f15600640 1 -- 172.21.15.39:0/3184030397 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f08003e20 con 0x5625f9bdc350 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.968+0000 7f2f15600640 1 -- 172.21.15.39:0/3184030397 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08004030 con 0x5625f9bdc350 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.969+0000 7f2f1cc00640 1 -- 172.21.15.39:0/36185930 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2f180e2d90 con 0x7f2f2010f460 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.969+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2f000b2960 0x7f2f000b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.969+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3184030397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 msgr2=0x5625f9f80800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.969+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/3184030397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f80800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f08011b40 tx=0x7f2f08005bd0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.970+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3184030397 shutdown_connections 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.970+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/3184030397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bb3490 0x5625f9f80d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.970+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/3184030397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 0x5625f9f81280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.970+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/3184030397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f80800 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.970+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3184030397 >> 172.21.15.39:0/3184030397 conn(0x5625f9bd0740 msgr2=0x5625f9f7e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.970+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2f000b2960 0x7f2f000b4e70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005ab0 tx=0x7f2f10005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.970+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3184030397 shutdown_connections 2024-07-04T08:19:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.970+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3184030397 wait complete. 2024-07-04T08:19:29.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.970+0000 7f2f2d6d1740 1 -- start start 2024-07-04T08:19:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.971+0000 7f2f2d6d1740 1 Processor -- start 2024-07-04T08:19:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.971+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 0x5625f9f8c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.971+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f8c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.971+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.971+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5625f9e8d770 con 0x5625f9bbb670 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.971+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5625f9eea4b0 con 0x5625f9bb3490 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.971+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5625f9d4a890 con 0x5625f9bdc350 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.971+0000 7f2f2d6d1740 10 client.0 _ll_register_callbacks cb 0x5625f9e8e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f8c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 0x5625f9f8c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f8c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36178/0 (socket says 172.21.15.39:36178) 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1f400640 1 -- 172.21.15.39:0/1486715845 learned_addr learned my addr 172.21.15.39:0/1486715845 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f25000640 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1f400640 1 -- 172.21.15.39:0/1486715845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 msgr2=0x5625f9f98120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1f400640 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f98120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1f400640 1 -- 172.21.15.39:0/1486715845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 msgr2=0x5625f9f8c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:29.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1f400640 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 0x5625f9f8c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1f400640 1 -- 172.21.15.39:0/1486715845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f9c47930 con 0x5625f9bbb670 2024-07-04T08:19:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.972+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 0x5625f9f8c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.973+0000 7f2f1f400640 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f8c850 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2f08006010 tx=0x7f2f08011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9e199460cc1c23 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.973+0000 7f2f0e000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.973+0000 7f2f0e000640 1 -- 172.21.15.39:0/1486715845 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08003b80 con 0x5625f9bbb670 2024-07-04T08:19:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.973+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/1486715845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f08003e20 con 0x5625f9bbb670 2024-07-04T08:19:29.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.973+0000 7f2f0e000640 1 -- 172.21.15.39:0/1486715845 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2f08004a40 con 0x5625f9bbb670 2024-07-04T08:19:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.973+0000 7f2f0e000640 1 -- 172.21.15.39:0/1486715845 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2f08004c80 con 0x5625f9bbb670 2024-07-04T08:19:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.973+0000 7f2f2d6d1740 10 client.16395 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.973+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/1486715845 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625f9f9ad30 con 0x5625f9bbb670 2024-07-04T08:19:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.974+0000 7f2f0e000640 1 -- 172.21.15.39:0/1486715845 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f08003e20 con 0x5625f9bbb670 2024-07-04T08:19:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.974+0000 7f2f0e000640 1 -- 172.21.15.39:0/1486715845 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2f08005040 con 0x5625f9bbb670 2024-07-04T08:19:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.974+0000 7f2f2d6d1740 10 client.16395 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:29.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.974+0000 7f2f2d6d1740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.974+0000 7f2f2d6d1740 20 client.16395 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.974+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625f9b9acd0 0x5625f9f9e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.974+0000 7f2f2d6d1740 4 client.16395 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.974+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/1486715845 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625f9ef8e20 con 0x5625f9b9acd0 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.975+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625f9b9acd0 0x5625f9f9e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.975+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625f9b9acd0 0x5625f9f9e090 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d5c07df94bd33a server_cookie=f26e61f7900dc2da in_seq=0 out_seq=0 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.975+0000 7f2f0e000640 10 client.16395 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.976+0000 7f2f0e000640 1 -- 172.21.15.39:0/1486715845 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5625f9ef8e20 con 0x5625f9b9acd0 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.976+0000 7f2f0e000640 10 client.16395 handle_command_reply: tid=0 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.976+0000 7f2f2d6d1740 1 client.16395 shutdown 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.976+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/1486715845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 msgr2=0x5625f9f8c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.976+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f8c850 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2f08006010 tx=0x7f2f08011440 comp rx=0 tx=0).stop 2024-07-04T08:19:29.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/1486715845 shutdown_connections 2024-07-04T08:19:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625f9b9acd0 0x5625f9f9e090 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f8c850 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/1486715845 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 0x5625f9f8c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/1486715845 >> 172.21.15.39:0/1486715845 conn(0x5625f9bd0740 msgr2=0x5625f9f83990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f0e000640 0 client.16395 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/1486715845 shutdown_connections 2024-07-04T08:19:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/1486715845 wait complete. 2024-07-04T08:19:29.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.977+0000 7f2f2d6d1740 20 client.16395 trim_cache size 0 max 16384 2024-07-04T08:19:29.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.997+0000 7f2f2d6d1740 1 Processor -- start 2024-07-04T08:19:29.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.997+0000 7f2f2d6d1740 1 -- start start 2024-07-04T08:19:29.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 0x5625f9f86920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f8dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201b2290 con 0x5625f9bbb670 2024-07-04T08:19:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201b3dd0 con 0x5625f9bb3490 2024-07-04T08:19:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201b2960 con 0x5625f9bdc350 2024-07-04T08:19:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f821f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36190/0 (socket says 172.21.15.39:36190) 2024-07-04T08:19:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f1f400640 1 -- 172.21.15.39:0/3355390456 learned_addr learned my addr 172.21.15.39:0/3355390456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f25000640 1 --2- 172.21.15.39:0/3355390456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f8dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.998+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/3355390456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 0x5625f9f86920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:29.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.999+0000 7f2f1f400640 1 -- 172.21.15.39:0/3355390456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 msgr2=0x5625f9f8dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:29.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.999+0000 7f2f1f400640 1 --2- 172.21.15.39:0/3355390456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f8dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.999+0000 7f2f1f400640 1 -- 172.21.15.39:0/3355390456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 msgr2=0x5625f9f86920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:29.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.999+0000 7f2f1f400640 1 --2- 172.21.15.39:0/3355390456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 0x5625f9f86920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:29.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.999+0000 7f2f1f400640 1 -- 172.21.15.39:0/3355390456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f9f7efe0 con 0x5625f9bbb670 2024-07-04T08:19:30.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:29.999+0000 7f2f1f400640 1 --2- 172.21.15.39:0/3355390456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f821f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f08000f80 tx=0x7f2f08004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31116062138c850 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:30.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.000+0000 7f2f14c00640 1 -- 172.21.15.39:0/3355390456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0803b070 con 0x5625f9bbb670 2024-07-04T08:19:30.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.000+0000 7f2f14c00640 1 -- 172.21.15.39:0/3355390456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f08004e20 con 0x5625f9bbb670 2024-07-04T08:19:30.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.000+0000 7f2f14c00640 1 -- 172.21.15.39:0/3355390456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08005000 con 0x5625f9bbb670 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.000+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3355390456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 msgr2=0x5625f9f821f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.001+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/3355390456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f821f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f08000f80 tx=0x7f2f08004c10 comp rx=0 tx=0).stop 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.001+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3355390456 shutdown_connections 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.001+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/3355390456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bbb670 0x5625f9f821f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.001+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/3355390456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bdc350 0x5625f9f8dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.001+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/3355390456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bb3490 0x5625f9f86920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.001+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3355390456 >> 172.21.15.39:0/3355390456 conn(0x5625f9bd0740 msgr2=0x5625f9f8d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.001+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3355390456 shutdown_connections 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.001+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/3355390456 wait complete. 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.001+0000 7f2f2d6d1740 1 -- start start 2024-07-04T08:19:30.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.002+0000 7f2f2d6d1740 1 Processor -- start 2024-07-04T08:19:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.002+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bb3490 0x5625f9fa1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.002+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 0x5625f9f9dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.002+0000 7f2f2d6d1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bdc350 0x5625f9f9e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:30.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.002+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201b3700 con 0x5625f9bdc350 2024-07-04T08:19:30.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.002+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201b3030 con 0x5625f9bbb670 2024-07-04T08:19:30.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.002+0000 7f2f2d6d1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201b0270 con 0x5625f9bb3490 2024-07-04T08:19:30.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.002+0000 7f2f2d6d1740 10 client.0 _ll_register_callbacks cb 0x7f2f080093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:30.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bb3490 0x5625f9fa1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:30.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bb3490 0x5625f9fa1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50924/0 (socket says 172.21.15.39:50924) 2024-07-04T08:19:30.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1fe00640 1 -- 172.21.15.39:0/610660421 learned_addr learned my addr 172.21.15.39:0/610660421 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:30.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1f400640 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 0x5625f9f9dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:30.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f25000640 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bdc350 0x5625f9f9e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:30.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1f400640 1 -- 172.21.15.39:0/610660421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bb3490 msgr2=0x5625f9fa1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:30.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1f400640 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bb3490 0x5625f9fa1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1f400640 1 -- 172.21.15.39:0/610660421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bdc350 msgr2=0x5625f9f9e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:30.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1f400640 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bdc350 0x5625f9f9e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1f400640 1 -- 172.21.15.39:0/610660421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f9f9eb50 con 0x5625f9bbb670 2024-07-04T08:19:30.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.003+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bb3490 0x5625f9fa1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:30.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.004+0000 7f2f1f400640 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 0x5625f9f9dd90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009c60 tx=0x7f2f08005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62dba43d788a9fed server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.004+0000 7f2f0e000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.004+0000 7f2f0e000640 1 -- 172.21.15.39:0/610660421 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0803b070 con 0x5625f9bbb670 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.004+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/610660421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f08004e20 con 0x5625f9bbb670 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.004+0000 7f2f0e000640 1 -- 172.21.15.39:0/610660421 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2f08011240 con 0x5625f9bbb670 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.004+0000 7f2f0e000640 1 -- 172.21.15.39:0/610660421 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2f08049050 con 0x5625f9bbb670 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.004+0000 7f2f2d6d1740 10 client.26528 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.004+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/610660421 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625fa005650 con 0x5625f9bbb670 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.005+0000 7f2f0e000640 1 -- 172.21.15.39:0/610660421 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f08004e20 con 0x5625f9bbb670 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.005+0000 7f2f0e000640 1 -- 172.21.15.39:0/610660421 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2f08040c80 con 0x5625f9bbb670 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.005+0000 7f2f2d6d1740 10 client.26528 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.005+0000 7f2f2d6d1740 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:19:30.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.005+0000 7f2f2d6d1740 20 client.26528 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:30.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.005+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625f9b70080 0x5625fa007ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:30.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.005+0000 7f2f2d6d1740 4 client.26528 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:19:30.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.005+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/610660421 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5625f9ef48c0 con 0x5625f9b70080 2024-07-04T08:19:30.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.005+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625f9b70080 0x5625fa007ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:30.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.006+0000 7f2f1fe00640 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625f9b70080 0x5625fa007ce0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d581ae1713b8636 server_cookie=4c70b62145c8dd8b in_seq=0 out_seq=0 2024-07-04T08:19:30.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.006+0000 7f2f0e000640 10 client.26528 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:30.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.007+0000 7f2f0e000640 1 -- 172.21.15.39:0/610660421 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5625f9ef48c0 con 0x5625f9b70080 2024-07-04T08:19:30.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.007+0000 7f2f0e000640 10 client.26528 handle_command_reply: tid=0 2024-07-04T08:19:30.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.007+0000 7f2f2d6d1740 1 client.26528 shutdown 2024-07-04T08:19:30.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/610660421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 msgr2=0x5625f9f9dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:30.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 0x5625f9f9dd90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009c60 tx=0x7f2f08005280 comp rx=0 tx=0).stop 2024-07-04T08:19:30.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/610660421 shutdown_connections 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625f9b70080 0x5625fa007ce0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625f9bdc350 0x5625f9f9e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625f9bbb670 0x5625f9f9dd90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 --2- 172.21.15.39:0/610660421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625f9bb3490 0x5625f9fa1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/610660421 >> 172.21.15.39:0/610660421 conn(0x5625f9bd0740 msgr2=0x5625f9f8def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f0e000640 0 client.26528 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/610660421 shutdown_connections 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 1 -- 172.21.15.39:0/610660421 wait complete. 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.008+0000 7f2f2d6d1740 20 client.26528 trim_cache size 0 max 16384 2024-07-04T08:19:30.009 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:19:30.010 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:19:30.010 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "1f30ca74-e872-4694-82f3-fe34603ddc85", 2024-07-04T08:19:30.010 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:19:30.010 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:19:30.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 -- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2f000b2960 msgr2=0x7f2f000b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:30.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2f000b2960 0x7f2f000b4e70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005ab0 tx=0x7f2f10005a20 comp rx=0 tx=0).stop 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 -- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2010f460 msgr2=0x7f2f201a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2010f460 0x7f2f201a1270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800e970 tx=0x7f2f1800ed70 comp rx=0 tx=0).stop 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 -- 172.21.15.39:0/36185930 shutdown_connections 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2f000b2960 0x7f2f000b4e70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f2010f460 0x7f2f201a1270 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f20075650 0x7f2f2019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 --2- 172.21.15.39:0/36185930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2f20074d00 0x7f2f2019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.009+0000 7f2f25a00640 1 -- 172.21.15.39:0/36185930 >> 172.21.15.39:0/36185930 conn(0x7f2f200fe990 msgr2=0x7f2f20109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.010+0000 7f2f25a00640 1 -- 172.21.15.39:0/36185930 shutdown_connections 2024-07-04T08:19:30.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:30.010+0000 7f2f25a00640 1 -- 172.21.15.39:0/36185930 wait complete. 2024-07-04T08:19:30.816 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:19:31.001 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:19:31.311 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:31.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:30 smithi133 ceph-mon[37050]: pgmap v609: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-07-04T08:19:31.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:30 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:19:31.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:30 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:19:31.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:30 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:19:31.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:30 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:19:31.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:30 smithi039 ceph-mon[32062]: pgmap v609: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-07-04T08:19:31.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:30 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:19:31.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:30 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:19:31.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:30 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:19:31.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:30 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:19:31.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:30 smithi050 ceph-mon[36577]: pgmap v609: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-07-04T08:19:31.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:30 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:19:31.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:30 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:19:31.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:30 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:19:31.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:30 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:19:32.787 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:33.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:32 smithi039 ceph-mon[32062]: pgmap v610: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-07-04T08:19:33.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:32 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:19:33.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:32 smithi133 ceph-mon[37050]: pgmap v610: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-07-04T08:19:33.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:32 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:19:33.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:32 smithi050 ceph-mon[36577]: pgmap v610: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-07-04T08:19:33.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:32 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:19:33.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.758+0000 7f1523a00640 1 -- 172.21.15.39:0/1180767969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c101fd0 msgr2=0x7f151c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:33.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.758+0000 7f1523a00640 1 --2- 172.21.15.39:0/1180767969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c101fd0 0x7f151c1023e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1510009900 tx=0x7f151002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:19:33.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.758+0000 7f1523a00640 1 -- 172.21.15.39:0/1180767969 shutdown_connections 2024-07-04T08:19:33.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.758+0000 7f1523a00640 1 --2- 172.21.15.39:0/1180767969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f151c108260 0x7f151c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:33.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.758+0000 7f1523a00640 1 --2- 172.21.15.39:0/1180767969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f151c107910 0x7f151c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:33.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.758+0000 7f1523a00640 1 --2- 172.21.15.39:0/1180767969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c101fd0 0x7f151c1023e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:33.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.758+0000 7f1523a00640 1 -- 172.21.15.39:0/1180767969 >> 172.21.15.39:0/1180767969 conn(0x7f151c078c70 msgr2=0x7f151c079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:33.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.759+0000 7f1523a00640 1 -- 172.21.15.39:0/1180767969 shutdown_connections 2024-07-04T08:19:33.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.759+0000 7f1523a00640 1 -- 172.21.15.39:0/1180767969 wait complete. 2024-07-04T08:19:33.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.760+0000 7f1523a00640 1 Processor -- start 2024-07-04T08:19:33.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.760+0000 7f1523a00640 1 -- start start 2024-07-04T08:19:33.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f1523a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f151c101fd0 0x7f151c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:33.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f1523a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107910 0x7f151c19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:33.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f1523a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f151c108260 0x7f151c1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:33.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f1523a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f151c111f70 con 0x7f151c107910 2024-07-04T08:19:33.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f1523a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f151c111df0 con 0x7f151c101fd0 2024-07-04T08:19:33.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f1523a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f151c1120f0 con 0x7f151c108260 2024-07-04T08:19:33.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f1521000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f151c101fd0 0x7f151c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:33.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f151be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107910 0x7f151c19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:33.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f1521000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f151c101fd0 0x7f151c19bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50126/0 (socket says 172.21.15.39:50126) 2024-07-04T08:19:33.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.761+0000 7f1521000640 1 -- 172.21.15.39:0/3872714876 learned_addr learned my addr 172.21.15.39:0/3872714876 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:33.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.762+0000 7f1521a00640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f151c108260 0x7f151c1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:33.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.762+0000 7f1521000640 1 -- 172.21.15.39:0/3872714876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f151c108260 msgr2=0x7f151c1a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:33.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.762+0000 7f1521000640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f151c108260 0x7f151c1a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:33.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.762+0000 7f1521000640 1 -- 172.21.15.39:0/3872714876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107910 msgr2=0x7f151c19c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:33.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.762+0000 7f1521000640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107910 0x7f151c19c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.762+0000 7f1521000640 1 -- 172.21.15.39:0/3872714876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c1a1c10 con 0x7f151c101fd0 2024-07-04T08:19:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.762+0000 7f1521000640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f151c101fd0 0x7f151c19bca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1510005ab0 tx=0x7f1510003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.763+0000 7f1519600640 1 -- 172.21.15.39:0/3872714876 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151003f070 con 0x7f151c101fd0 2024-07-04T08:19:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.763+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c1a1ea0 con 0x7f151c101fd0 2024-07-04T08:19:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.763+0000 7f1519600640 1 -- 172.21.15.39:0/3872714876 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1510031b60 con 0x7f151c101fd0 2024-07-04T08:19:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.763+0000 7f1519600640 1 -- 172.21.15.39:0/3872714876 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510035030 con 0x7f151c101fd0 2024-07-04T08:19:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.763+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c1a8660 con 0x7f151c101fd0 2024-07-04T08:19:33.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.764+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151c1023e0 con 0x7f151c101fd0 2024-07-04T08:19:33.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.767+0000 7f1519600640 1 -- 172.21.15.39:0/3872714876 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1510042070 con 0x7f151c101fd0 2024-07-04T08:19:33.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.768+0000 7f1519600640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f15040b29e0 0x7f15040b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:33.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.768+0000 7f1519600640 1 -- 172.21.15.39:0/3872714876 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1510103690 con 0x7f151c101fd0 2024-07-04T08:19:33.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.768+0000 7f151be00640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f15040b29e0 0x7f15040b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:33.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.769+0000 7f151be00640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f15040b29e0 0x7f15040b4ef0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f151c104850 tx=0x7f1508005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:33.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:33.770+0000 7f1519600640 1 -- 172.21.15.39:0/3872714876 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f15100ca690 con 0x7f151c101fd0 2024-07-04T08:19:34.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.031+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f151c19cdd0 con 0x7f151c101fd0 2024-07-04T08:19:34.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.033+0000 7f1519600640 1 -- 172.21.15.39:0/3872714876 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f15100ce990 con 0x7f151c101fd0 2024-07-04T08:19:34.033 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:19:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:19:34.033 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f15040b29e0 msgr2=0x7f15040b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f15040b29e0 0x7f15040b4ef0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f151c104850 tx=0x7f1508005f90 comp rx=0 tx=0).stop 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f151c101fd0 msgr2=0x7f151c19bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f151c101fd0 0x7f151c19bca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1510005ab0 tx=0x7f1510003c10 comp rx=0 tx=0).stop 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 shutdown_connections 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f15040b29e0 0x7f15040b4ef0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c107910 0x7f151c19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f151c108260 0x7f151c1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 --2- 172.21.15.39:0/3872714876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f151c101fd0 0x7f151c19bca0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 >> 172.21.15.39:0/3872714876 conn(0x7f151c078c70 msgr2=0x7f151c105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:34.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 shutdown_connections 2024-07-04T08:19:34.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:34.036+0000 7f1523a00640 1 -- 172.21.15.39:0/3872714876 wait complete. 2024-07-04T08:19:35.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:34 smithi039 ceph-mon[32062]: pgmap v611: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:19:35.100 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:34 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3872714876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:35.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:34 smithi133 ceph-mon[37050]: pgmap v611: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:19:35.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:34 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3872714876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:35.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:34 smithi050 ceph-mon[36577]: pgmap v611: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:19:35.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:34 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3872714876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:35.443 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id 705d2328 --release --await' 2024-07-04T08:19:35.443 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id 705d2328 --release --await 2024-07-04T08:19:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.079+0000 7f4f25e00640 1 -- 172.21.15.39:0/722443967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f20074d00 msgr2=0x7f4f20075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.079+0000 7f4f25e00640 1 --2- 172.21.15.39:0/722443967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f20074d00 0x7f4f20075110 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009860 tx=0x7f4f1402a340 comp rx=0 tx=0).stop 2024-07-04T08:19:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.079+0000 7f4f25e00640 1 -- 172.21.15.39:0/722443967 shutdown_connections 2024-07-04T08:19:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.079+0000 7f4f25e00640 1 --2- 172.21.15.39:0/722443967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f2010f460 0x7f4f201118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.079+0000 7f4f25e00640 1 --2- 172.21.15.39:0/722443967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f20075650 0x7f4f2010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.079+0000 7f4f25e00640 1 --2- 172.21.15.39:0/722443967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f20074d00 0x7f4f20075110 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.079+0000 7f4f25e00640 1 -- 172.21.15.39:0/722443967 >> 172.21.15.39:0/722443967 conn(0x7f4f200fe990 msgr2=0x7f4f20100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:36.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.080+0000 7f4f25e00640 1 -- 172.21.15.39:0/722443967 shutdown_connections 2024-07-04T08:19:36.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.080+0000 7f4f25e00640 1 -- 172.21.15.39:0/722443967 wait complete. 2024-07-04T08:19:36.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.081+0000 7f4f25e00640 1 Processor -- start 2024-07-04T08:19:36.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.081+0000 7f4f25e00640 1 -- start start 2024-07-04T08:19:36.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.081+0000 7f4f25e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f20074d00 0x7f4f2019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.081+0000 7f4f25e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f20075650 0x7f4f2019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.081+0000 7f4f25e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f2010f460 0x7f4f201a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.081+0000 7f4f25e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f20115010 con 0x7f4f2010f460 2024-07-04T08:19:36.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.081+0000 7f4f25e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f20114e90 con 0x7f4f20075650 2024-07-04T08:19:36.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.081+0000 7f4f25e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f20115190 con 0x7f4f20074d00 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f24a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f20074d00 0x7f4f2019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f24a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f20074d00 0x7f4f2019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33510/0 (socket says 172.21.15.39:33510) 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f1fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f20075650 0x7f4f2019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f24a00640 1 -- 172.21.15.39:0/1085255035 learned_addr learned my addr 172.21.15.39:0/1085255035 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f25400640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f2010f460 0x7f4f201a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f24a00640 1 -- 172.21.15.39:0/1085255035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f20075650 msgr2=0x7f4f2019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f24a00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f20075650 0x7f4f2019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f24a00640 1 -- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f2010f460 msgr2=0x7f4f201a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f24a00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f2010f460 0x7f4f201a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f24a00640 1 -- 172.21.15.39:0/1085255035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f201a1970 con 0x7f4f20074d00 2024-07-04T08:19:36.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.082+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f20075650 0x7f4f2019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.083+0000 7f4f24a00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f20074d00 0x7f4f2019ba00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f140303f0 tx=0x7f4f14032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.083+0000 7f4f1d600640 1 -- 172.21.15.39:0/1085255035 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14040020 con 0x7f4f20074d00 2024-07-04T08:19:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.083+0000 7f4f1d600640 1 -- 172.21.15.39:0/1085255035 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f14030dd0 con 0x7f4f20074d00 2024-07-04T08:19:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.083+0000 7f4f25e00640 1 -- 172.21.15.39:0/1085255035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f201a1c00 con 0x7f4f20074d00 2024-07-04T08:19:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.083+0000 7f4f1d600640 1 -- 172.21.15.39:0/1085255035 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14037040 con 0x7f4f20074d00 2024-07-04T08:19:36.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.083+0000 7f4f25e00640 1 -- 172.21.15.39:0/1085255035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f201a83c0 con 0x7f4f20074d00 2024-07-04T08:19:36.084 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:36.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.087+0000 7f4f2d99d740 1 Processor -- start 2024-07-04T08:19:36.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.087+0000 7f4f2d99d740 1 -- start start 2024-07-04T08:19:36.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 0x55a200050940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 0x55a200050e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a2000513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a1fffc6ed0 con 0x55a1ffc831b0 2024-07-04T08:19:36.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a1fffb6470 con 0x55a1ffc8b390 2024-07-04T08:19:36.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a1fffba4b0 con 0x55a1ffcac070 2024-07-04T08:19:36.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f1fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a2000513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f1fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a2000513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50158/0 (socket says 172.21.15.39:50158) 2024-07-04T08:19:36.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f1fe00640 1 -- 172.21.15.39:0/523346209 learned_addr learned my addr 172.21.15.39:0/523346209 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:36.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f25400640 1 --2- 172.21.15.39:0/523346209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 0x55a200050940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f1fe00640 1 -- 172.21.15.39:0/523346209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 msgr2=0x55a200050940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:36.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/523346209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 0x55a200050940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f1fe00640 1 -- 172.21.15.39:0/523346209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 msgr2=0x55a200050e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:19:36.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/523346209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 0x55a200050e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.088+0000 7f4f1fe00640 1 -- 172.21.15.39:0/523346209 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1fff4e640 con 0x55a1ffc8b390 2024-07-04T08:19:36.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.089+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/523346209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a2000513c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c011470 tx=0x7f4f0c011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=844fd07196bea892 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:36.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.089+0000 7f4f12000640 1 -- 172.21.15.39:0/523346209 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c01e070 con 0x55a1ffc8b390 2024-07-04T08:19:36.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.089+0000 7f4f1d600640 1 -- 172.21.15.39:0/1085255035 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4f14043020 con 0x7f4f20074d00 2024-07-04T08:19:36.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.089+0000 7f4f12000640 1 -- 172.21.15.39:0/523346209 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c00f9f0 con 0x55a1ffc8b390 2024-07-04T08:19:36.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.089+0000 7f4f12000640 1 -- 172.21.15.39:0/523346209 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c019aa0 con 0x55a1ffc8b390 2024-07-04T08:19:36.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.090+0000 7f4f2d99d740 1 -- 172.21.15.39:0/523346209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 msgr2=0x55a2000513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:36.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.090+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/523346209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a2000513c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c011470 tx=0x7f4f0c011930 comp rx=0 tx=0).stop 2024-07-04T08:19:36.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.091+0000 7f4f2d99d740 1 -- 172.21.15.39:0/523346209 shutdown_connections 2024-07-04T08:19:36.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.091+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/523346209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 0x55a200050e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.091+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/523346209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a2000513c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.091+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/523346209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 0x55a200050940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.091+0000 7f4f2d99d740 1 -- 172.21.15.39:0/523346209 >> 172.21.15.39:0/523346209 conn(0x55a1ffca0460 msgr2=0x55a20004e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:36.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.091+0000 7f4f2d99d740 1 -- 172.21.15.39:0/523346209 shutdown_connections 2024-07-04T08:19:36.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.091+0000 7f4f2d99d740 1 -- 172.21.15.39:0/523346209 wait complete. 2024-07-04T08:19:36.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.092+0000 7f4f2d99d740 1 -- start start 2024-07-04T08:19:36.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.092+0000 7f4f1d600640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f080b2870 0x7f4f080b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.092+0000 7f4f1d600640 1 -- 172.21.15.39:0/1085255035 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4f14102520 con 0x7f4f20074d00 2024-07-04T08:19:36.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.092+0000 7f4f2d99d740 1 Processor -- start 2024-07-04T08:19:36.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.092+0000 7f4f25400640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f080b2870 0x7f4f080b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc831b0 0x55a20005c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc8b390 0x55a20005cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffcac070 0x55a200070c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a1ffe1a700 con 0x55a1ffc8b390 2024-07-04T08:19:36.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a1fff5dba0 con 0x55a1ffcac070 2024-07-04T08:19:36.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a20005f120 con 0x55a1ffc831b0 2024-07-04T08:19:36.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f2d99d740 10 client.0 _ll_register_callbacks cb 0x55a1fffaf560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:36.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f24a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc831b0 0x55a20005c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f24a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc831b0 0x55a20005c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33518/0 (socket says 172.21.15.39:33518) 2024-07-04T08:19:36.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f24a00640 1 -- 172.21.15.39:0/395473291 learned_addr learned my addr 172.21.15.39:0/395473291 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:36.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f25400640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f080b2870 0x7f4f080b4d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4f18005fd0 tx=0x7f4f18009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:36.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f25400640 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffcac070 0x55a200070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.093+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc8b390 0x55a20005cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.094+0000 7f4f24a00640 1 -- 172.21.15.39:0/395473291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffcac070 msgr2=0x55a200070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:36.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.094+0000 7f4f24a00640 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffcac070 0x55a200070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.094+0000 7f4f24a00640 1 -- 172.21.15.39:0/395473291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc8b390 msgr2=0x55a20005cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:36.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.094+0000 7f4f24a00640 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc8b390 0x55a20005cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.094+0000 7f4f24a00640 1 -- 172.21.15.39:0/395473291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ffd17650 con 0x55a1ffc831b0 2024-07-04T08:19:36.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.094+0000 7f4f25400640 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffcac070 0x55a200070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:36.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.094+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc8b390 0x55a20005cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.094+0000 7f4f24a00640 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc831b0 0x55a20005c5b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f140ffb00 tx=0x7f4f140c93b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2f4be93b8483ba9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.094+0000 7f4f06a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.095+0000 7f4f06a00640 1 -- 172.21.15.39:0/395473291 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140c90f0 con 0x55a1ffc831b0 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.095+0000 7f4f06a00640 1 -- 172.21.15.39:0/395473291 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4f141021d0 con 0x55a1ffc831b0 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.095+0000 7f4f2d99d740 1 -- 172.21.15.39:0/395473291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0c00f9f0 con 0x55a1ffc831b0 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.095+0000 7f4f06a00640 1 -- 172.21.15.39:0/395473291 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f140328c0 con 0x55a1ffc831b0 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.095+0000 7f4f2d99d740 10 client.26476 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.095+0000 7f4f2d99d740 1 -- 172.21.15.39:0/395473291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a2000739d0 con 0x55a1ffc831b0 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.095+0000 7f4f06a00640 1 -- 172.21.15.39:0/395473291 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c00f9f0 con 0x55a1ffc831b0 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.096+0000 7f4f06a00640 1 -- 172.21.15.39:0/395473291 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f1404d060 con 0x55a1ffc831b0 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.096+0000 7f4f2d99d740 10 client.26476 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.096+0000 7f4f2d99d740 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:19:36.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.096+0000 7f4f2d99d740 20 client.26476 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:36.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.096+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1ffc6a9f0 0x55a200076e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.096+0000 7f4f2d99d740 4 client.26476 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:36.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.096+0000 7f4f2d99d740 1 -- 172.21.15.39:0/395473291 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1fffc8e20 con 0x55a1ffc6a9f0 2024-07-04T08:19:36.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.096+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1ffc6a9f0 0x55a200076e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.097+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1ffc6a9f0 0x55a200076e10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f53084c6b3fdcfc server_cookie=f4dc143de5fd6139 in_seq=0 out_seq=0 2024-07-04T08:19:36.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.097+0000 7f4f06a00640 10 client.26476 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:36.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f06a00640 1 -- 172.21.15.39:0/395473291 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a1fffc8e20 con 0x55a1ffc6a9f0 2024-07-04T08:19:36.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f06a00640 10 client.26476 handle_command_reply: tid=0 2024-07-04T08:19:36.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 client.26476 shutdown 2024-07-04T08:19:36.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 -- 172.21.15.39:0/395473291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc831b0 msgr2=0x55a20005c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc831b0 0x55a20005c5b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f140ffb00 tx=0x7f4f140c93b0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 -- 172.21.15.39:0/395473291 shutdown_connections 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1ffc6a9f0 0x55a200076e10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc8b390 0x55a20005cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffcac070 0x55a200070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/395473291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc831b0 0x55a20005c5b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f06a00640 0 client.26476 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 -- 172.21.15.39:0/395473291 >> 172.21.15.39:0/395473291 conn(0x55a1ffca0460 msgr2=0x55a200053ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 -- 172.21.15.39:0/395473291 shutdown_connections 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.098+0000 7f4f2d99d740 1 -- 172.21.15.39:0/395473291 wait complete. 2024-07-04T08:19:36.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.099+0000 7f4f2d99d740 20 client.26476 trim_cache size 0 max 16384 2024-07-04T08:19:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.119+0000 7f4f2d99d740 1 Processor -- start 2024-07-04T08:19:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.119+0000 7f4f2d99d740 1 -- start start 2024-07-04T08:19:36.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.119+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 0x55a200056160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.119+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a20004e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.119+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 0x55a200070a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.119+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f201adda0 con 0x55a1ffc831b0 2024-07-04T08:19:36.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.119+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f201af8e0 con 0x55a1ffc8b390 2024-07-04T08:19:36.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.119+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f201ae470 con 0x55a1ffcac070 2024-07-04T08:19:36.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f24a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 0x55a200056160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f1fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a20004e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f24a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 0x55a200056160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50758/0 (socket says 172.21.15.39:50758) 2024-07-04T08:19:36.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f1fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a20004e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50172/0 (socket says 172.21.15.39:50172) 2024-07-04T08:19:36.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f24a00640 1 -- 172.21.15.39:0/4052539207 learned_addr learned my addr 172.21.15.39:0/4052539207 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:36.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f25400640 1 --2- 172.21.15.39:0/4052539207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 0x55a200070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f1fe00640 1 -- 172.21.15.39:0/4052539207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 msgr2=0x55a200070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/4052539207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 0x55a200070a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f1fe00640 1 -- 172.21.15.39:0/4052539207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 msgr2=0x55a200056160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/4052539207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 0x55a200056160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f1fe00640 1 -- 172.21.15.39:0/4052539207 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a20005b310 con 0x55a1ffc8b390 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.120+0000 7f4f24a00640 1 --2- 172.21.15.39:0/4052539207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 0x55a200056160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.121+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/4052539207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a20004e170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c019780 tx=0x7f4f0c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c187c6746c27c5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.121+0000 7f4f11600640 1 -- 172.21.15.39:0/4052539207 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c01e070 con 0x55a1ffc8b390 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.121+0000 7f4f11600640 1 -- 172.21.15.39:0/4052539207 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c01f040 con 0x55a1ffc8b390 2024-07-04T08:19:36.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.121+0000 7f4f11600640 1 -- 172.21.15.39:0/4052539207 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0054b0 con 0x55a1ffc8b390 2024-07-04T08:19:36.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.122+0000 7f4f2d99d740 1 -- 172.21.15.39:0/4052539207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 msgr2=0x55a20004e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:36.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.122+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/4052539207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a20004e170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c019780 tx=0x7f4f0c000bc0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.122+0000 7f4f2d99d740 1 -- 172.21.15.39:0/4052539207 shutdown_connections 2024-07-04T08:19:36.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.122+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/4052539207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffcac070 0x55a200070a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.122+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/4052539207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc8b390 0x55a20004e170 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.122+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/4052539207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffc831b0 0x55a200056160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.123+0000 7f4f2d99d740 1 -- 172.21.15.39:0/4052539207 >> 172.21.15.39:0/4052539207 conn(0x55a1ffca0460 msgr2=0x55a200055a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:36.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.123+0000 7f4f2d99d740 1 -- 172.21.15.39:0/4052539207 shutdown_connections 2024-07-04T08:19:36.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.123+0000 7f4f2d99d740 1 -- 172.21.15.39:0/4052539207 wait complete. 2024-07-04T08:19:36.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.123+0000 7f4f2d99d740 1 -- start start 2024-07-04T08:19:36.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.124+0000 7f4f2d99d740 1 Processor -- start 2024-07-04T08:19:36.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.124+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc831b0 0x55a200068b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc8b390 0x55a2000690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f2d99d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffcac070 0x55a2000695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f24a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc831b0 0x55a200068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f24a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc831b0 0x55a200068b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50188/0 (socket says 172.21.15.39:50188) 2024-07-04T08:19:36.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f201af210 con 0x55a1ffcac070 2024-07-04T08:19:36.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f201aeb40 con 0x55a1ffc831b0 2024-07-04T08:19:36.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f2d99d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f201abe00 con 0x55a1ffc8b390 2024-07-04T08:19:36.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f24a00640 1 -- 172.21.15.39:0/2032428969 learned_addr learned my addr 172.21.15.39:0/2032428969 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:36.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f2d99d740 10 client.0 _ll_register_callbacks cb 0x7f4f14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:36.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc8b390 0x55a2000690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.125+0000 7f4f25400640 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffcac070 0x55a2000695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.126+0000 7f4f1fe00640 1 -- 172.21.15.39:0/2032428969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc831b0 msgr2=0x55a200068b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:36.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.126+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc831b0 0x55a200068b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.126+0000 7f4f1fe00640 1 -- 172.21.15.39:0/2032428969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffcac070 msgr2=0x55a2000695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:36.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.126+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffcac070 0x55a2000695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.126+0000 7f4f1fe00640 1 -- 172.21.15.39:0/2032428969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2000de650 con 0x55a1ffc8b390 2024-07-04T08:19:36.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.126+0000 7f4f1fe00640 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc8b390 0x55a2000690a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c011e30 tx=0x7f4f0c0095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7236b76503ab6f98 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:36.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.126+0000 7f4f06a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:19:36.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.127+0000 7f4f06a00640 1 -- 172.21.15.39:0/2032428969 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c033020 con 0x55a1ffc8b390 2024-07-04T08:19:36.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.127+0000 7f4f2d99d740 1 -- 172.21.15.39:0/2032428969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0c01f040 con 0x55a1ffc8b390 2024-07-04T08:19:36.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.127+0000 7f4f06a00640 1 -- 172.21.15.39:0/2032428969 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4f0c005130 con 0x55a1ffc8b390 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.127+0000 7f4f06a00640 1 -- 172.21.15.39:0/2032428969 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f0c01e030 con 0x55a1ffc8b390 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.127+0000 7f4f2d99d740 10 client.26482 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.127+0000 7f4f2d99d740 1 -- 172.21.15.39:0/2032428969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a2000df8c0 con 0x55a1ffc8b390 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.128+0000 7f4f06a00640 1 -- 172.21.15.39:0/2032428969 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c01f040 con 0x55a1ffc8b390 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.128+0000 7f4f06a00640 1 -- 172.21.15.39:0/2032428969 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f0c025040 con 0x55a1ffc8b390 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.128+0000 7f4f2d99d740 10 client.26482 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.128+0000 7f4f2d99d740 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.128+0000 7f4f2d99d740 20 client.26482 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.128+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1ffc3fda0 0x55a2000e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.128+0000 7f4f2d99d740 4 client.26482 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:19:36.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.128+0000 7f4f2d99d740 1 -- 172.21.15.39:0/2032428969 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a1fffc48c0 con 0x55a1ffc3fda0 2024-07-04T08:19:36.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.129+0000 7f4f24a00640 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1ffc3fda0 0x55a2000e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:36.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.129+0000 7f4f24a00640 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1ffc3fda0 0x55a2000e1e90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8594339c7342554d server_cookie=f16f73afd26e4aaa in_seq=0 out_seq=0 2024-07-04T08:19:36.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.129+0000 7f4f06a00640 10 client.26482 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:36.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.130+0000 7f4f06a00640 1 -- 172.21.15.39:0/2032428969 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a1fffc48c0 con 0x55a1ffc3fda0 2024-07-04T08:19:36.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.130+0000 7f4f06a00640 10 client.26482 handle_command_reply: tid=0 2024-07-04T08:19:36.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.130+0000 7f4f2d99d740 1 client.26482 shutdown 2024-07-04T08:19:36.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.130+0000 7f4f2d99d740 1 -- 172.21.15.39:0/2032428969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc8b390 msgr2=0x55a2000690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.130+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc8b390 0x55a2000690a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c011e30 tx=0x7f4f0c0095e0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.130+0000 7f4f2d99d740 1 -- 172.21.15.39:0/2032428969 shutdown_connections 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.130+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1ffc3fda0 0x55a2000e1e90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.130+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1ffcac070 0x55a2000695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.130+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1ffc8b390 0x55a2000690a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.131+0000 7f4f06a00640 0 client.26482 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.131+0000 7f4f2d99d740 1 --2- 172.21.15.39:0/2032428969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1ffc831b0 0x55a200068b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.131+0000 7f4f2d99d740 1 -- 172.21.15.39:0/2032428969 >> 172.21.15.39:0/2032428969 conn(0x55a1ffca0460 msgr2=0x55a20005d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.131+0000 7f4f2d99d740 1 -- 172.21.15.39:0/2032428969 shutdown_connections 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.131+0000 7f4f2d99d740 1 -- 172.21.15.39:0/2032428969 wait complete. 2024-07-04T08:19:36.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.131+0000 7f4f2d99d740 20 client.26482 trim_cache size 0 max 16384 2024-07-04T08:19:36.132 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:19:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:19:36.132 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:19:36.132 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 -- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f080b2870 msgr2=0x7f4f080b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f080b2870 0x7f4f080b4d80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4f18005fd0 tx=0x7f4f18009500 comp rx=0 tx=0).stop 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 -- 172.21.15.39:0/1085255035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f20074d00 msgr2=0x7f4f2019ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f20074d00 0x7f4f2019ba00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f140303f0 tx=0x7f4f14032da0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 -- 172.21.15.39:0/1085255035 shutdown_connections 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f080b2870 0x7f4f080b4d80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f2010f460 0x7f4f201a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f20075650 0x7f4f2019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 --2- 172.21.15.39:0/1085255035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f20074d00 0x7f4f2019ba00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 -- 172.21.15.39:0/1085255035 >> 172.21.15.39:0/1085255035 conn(0x7f4f200fe990 msgr2=0x7f4f20100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.132+0000 7f4f25e00640 1 -- 172.21.15.39:0/1085255035 shutdown_connections 2024-07-04T08:19:36.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:36.133+0000 7f4f25e00640 1 -- 172.21.15.39:0/1085255035 wait complete. 2024-07-04T08:19:37.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:36 smithi039 ceph-mon[32062]: pgmap v612: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:19:37.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:36 smithi133 ceph-mon[37050]: pgmap v612: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:19:37.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:36 smithi050 ceph-mon[36577]: pgmap v612: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-07-04T08:19:37.749 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:19:37.749 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:19:39.153 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:39.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.249+0000 7faf61800640 1 -- 172.21.15.39:0/1240367495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c109350 msgr2=0x7faf5c10fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:39.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.249+0000 7faf61800640 1 --2- 172.21.15.39:0/1240367495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c109350 0x7faf5c10fb80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faf54009ca0 tx=0x7faf5402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.248 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:38 smithi039 ceph-mon[32062]: pgmap v613: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-07-04T08:19:39.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.249+0000 7faf61800640 1 -- 172.21.15.39:0/1240367495 shutdown_connections 2024-07-04T08:19:39.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.249+0000 7faf61800640 1 --2- 172.21.15.39:0/1240367495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf5c108a00 0x7faf5c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.249+0000 7faf61800640 1 --2- 172.21.15.39:0/1240367495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c109350 0x7faf5c10fb80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.249+0000 7faf61800640 1 --2- 172.21.15.39:0/1240367495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf5c1030c0 0x7faf5c1034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.249+0000 7faf61800640 1 -- 172.21.15.39:0/1240367495 >> 172.21.15.39:0/1240367495 conn(0x7faf5c0feb30 msgr2=0x7faf5c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.249+0000 7faf61800640 1 -- 172.21.15.39:0/1240367495 shutdown_connections 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.249+0000 7faf61800640 1 -- 172.21.15.39:0/1240367495 wait complete. 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.251+0000 7faf61800640 1 Processor -- start 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.251+0000 7faf61800640 1 -- start start 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.251+0000 7faf61800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c1030c0 0x7faf5c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.251+0000 7faf61800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf5c108a00 0x7faf5c19c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.251+0000 7faf61800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf5c109350 0x7faf5c1a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.251+0000 7faf61800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c113020 con 0x7faf5c108a00 2024-07-04T08:19:39.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.251+0000 7faf61800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c112ea0 con 0x7faf5c1030c0 2024-07-04T08:19:39.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.251+0000 7faf61800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1131a0 con 0x7faf5c109350 2024-07-04T08:19:39.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c1030c0 0x7faf5c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf53e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf5c108a00 0x7faf5c19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c1030c0 0x7faf5c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50212/0 (socket says 172.21.15.39:50212) 2024-07-04T08:19:39.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf5aa00640 1 -- 172.21.15.39:0/4126807856 learned_addr learned my addr 172.21.15.39:0/4126807856 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:39.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf5b400640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf5c109350 0x7faf5c1a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf53e00640 1 -- 172.21.15.39:0/4126807856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf5c109350 msgr2=0x7faf5c1a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:39.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf53e00640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf5c109350 0x7faf5c1a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf53e00640 1 -- 172.21.15.39:0/4126807856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c1030c0 msgr2=0x7faf5c19bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:39.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf53e00640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c1030c0 0x7faf5c19bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf53e00640 1 -- 172.21.15.39:0/4126807856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5c1a1b00 con 0x7faf5c108a00 2024-07-04T08:19:39.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.252+0000 7faf5aa00640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c1030c0 0x7faf5c19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:39.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.253+0000 7faf53e00640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf5c108a00 0x7faf5c19c130 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faf5c06b290 tx=0x7faf4400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:39.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.253+0000 7faf53400640 1 -- 172.21.15.39:0/4126807856 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4400cd00 con 0x7faf5c108a00 2024-07-04T08:19:39.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.253+0000 7faf53400640 1 -- 172.21.15.39:0/4126807856 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf440035f0 con 0x7faf5c108a00 2024-07-04T08:19:39.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.253+0000 7faf53400640 1 -- 172.21.15.39:0/4126807856 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44010690 con 0x7faf5c108a00 2024-07-04T08:19:39.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.253+0000 7faf61800640 1 -- 172.21.15.39:0/4126807856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5c1a1df0 con 0x7faf5c108a00 2024-07-04T08:19:39.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.253+0000 7faf61800640 1 -- 172.21.15.39:0/4126807856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5c1a8550 con 0x7faf5c108a00 2024-07-04T08:19:39.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.257+0000 7faf67e3e740 1 Processor -- start 2024-07-04T08:19:39.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.257+0000 7faf67e3e740 1 -- start start 2024-07-04T08:19:39.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.257+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec7462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.257+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec378660 0x55a4ec7467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.257+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec746d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.257+0000 7faf53400640 1 -- 172.21.15.39:0/4126807856 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7faf440037a0 con 0x7faf5c108a00 2024-07-04T08:19:39.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.257+0000 7faf67e3e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec660b00 con 0x55a4ec380840 2024-07-04T08:19:39.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.257+0000 7faf67e3e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec6bc290 con 0x55a4ec378660 2024-07-04T08:19:39.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.257+0000 7faf67e3e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec6ab830 con 0x55a4ec3a1520 2024-07-04T08:19:39.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.258+0000 7faf53e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec746d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.258+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec378660 0x55a4ec7467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.258+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec378660 0x55a4ec7467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50220/0 (socket says 172.21.15.39:50220) 2024-07-04T08:19:39.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.258+0000 7faf5aa00640 1 -- 172.21.15.39:0/909337656 learned_addr learned my addr 172.21.15.39:0/909337656 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:39.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.258+0000 7faf5aa00640 1 -- 172.21.15.39:0/909337656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 msgr2=0x55a4ec7462a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:19:39.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.258+0000 7faf5aa00640 1 --2- 172.21.15.39:0/909337656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec7462a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.258+0000 7faf5aa00640 1 -- 172.21.15.39:0/909337656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 msgr2=0x55a4ec746d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:39.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.258+0000 7faf5aa00640 1 --2- 172.21.15.39:0/909337656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec746d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.258+0000 7faf5aa00640 1 -- 172.21.15.39:0/909337656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ec665760 con 0x55a4ec378660 2024-07-04T08:19:39.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.259+0000 7faf5aa00640 1 --2- 172.21.15.39:0/909337656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec378660 0x55a4ec7467e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faf4c011630 tx=0x7faf4c011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d51d2d5f88b25a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:39.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.259+0000 7faf4be00640 1 -- 172.21.15.39:0/909337656 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c01e070 con 0x55a4ec378660 2024-07-04T08:19:39.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.259+0000 7faf4be00640 1 -- 172.21.15.39:0/909337656 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c00fba0 con 0x55a4ec378660 2024-07-04T08:19:39.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.259+0000 7faf4be00640 1 -- 172.21.15.39:0/909337656 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c019c80 con 0x55a4ec378660 2024-07-04T08:19:39.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.259+0000 7faf53400640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faf400b29e0 0x7faf400b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.260+0000 7faf53400640 1 -- 172.21.15.39:0/4126807856 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7faf440e2b70 con 0x7faf5c108a00 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.260+0000 7faf5b400640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faf400b29e0 0x7faf400b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.260+0000 7faf67e3e740 1 -- 172.21.15.39:0/909337656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec378660 msgr2=0x55a4ec7467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.260+0000 7faf67e3e740 1 --2- 172.21.15.39:0/909337656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec378660 0x55a4ec7467e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faf4c011630 tx=0x7faf4c011af0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.261+0000 7faf67e3e740 1 -- 172.21.15.39:0/909337656 shutdown_connections 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.261+0000 7faf67e3e740 1 --2- 172.21.15.39:0/909337656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec746d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.261+0000 7faf67e3e740 1 --2- 172.21.15.39:0/909337656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec378660 0x55a4ec7467e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.261+0000 7faf67e3e740 1 --2- 172.21.15.39:0/909337656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec7462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.261+0000 7faf67e3e740 1 -- 172.21.15.39:0/909337656 >> 172.21.15.39:0/909337656 conn(0x55a4ec395910 msgr2=0x55a4ec744190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:39.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.261+0000 7faf5b400640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faf400b29e0 0x7faf400b4ef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf54006010 tx=0x7faf540093e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:39.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.261+0000 7faf67e3e740 1 -- 172.21.15.39:0/909337656 shutdown_connections 2024-07-04T08:19:39.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.261+0000 7faf67e3e740 1 -- 172.21.15.39:0/909337656 wait complete. 2024-07-04T08:19:39.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.261+0000 7faf67e3e740 1 -- start start 2024-07-04T08:19:39.261 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.262+0000 7faf67e3e740 1 Processor -- start 2024-07-04T08:19:39.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.262+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec751910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.262+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec751e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.262+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec75dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.262+0000 7faf67e3e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec652d00 con 0x55a4ec378660 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.262+0000 7faf67e3e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec6af870 con 0x55a4ec380840 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.262+0000 7faf67e3e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a4ec50fbb0 con 0x55a4ec3a1520 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.262+0000 7faf67e3e740 10 client.0 _ll_register_callbacks cb 0x55a4ec6539c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec751910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf53e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec751e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec751910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50792/0 (socket says 172.21.15.39:50792) 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf5aa00640 1 -- 172.21.15.39:0/2222099358 learned_addr learned my addr 172.21.15.39:0/2222099358 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf53e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec751e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50232/0 (socket says 172.21.15.39:50232) 2024-07-04T08:19:39.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf5aa00640 1 -- 172.21.15.39:0/2222099358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 msgr2=0x55a4ec75dbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf5aa00640 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec75dbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf5aa00640 1 -- 172.21.15.39:0/2222099358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 msgr2=0x55a4ec751e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf5aa00640 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec751e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf5aa00640 1 -- 172.21.15.39:0/2222099358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ec642640 con 0x55a4ec378660 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf53e00640 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec751e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf5aa00640 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec751910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faf4c01c040 tx=0x7faf4c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546e216042eb6499 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.263+0000 7faf48c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.264+0000 7faf48c00640 1 -- 172.21.15.39:0/2222099358 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c026020 con 0x55a4ec378660 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.264+0000 7faf48c00640 1 -- 172.21.15.39:0/2222099358 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7faf4c00fd50 con 0x55a4ec378660 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.264+0000 7faf67e3e740 1 -- 172.21.15.39:0/2222099358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf4c00fba0 con 0x55a4ec378660 2024-07-04T08:19:39.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.264+0000 7faf48c00640 1 -- 172.21.15.39:0/2222099358 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf4c031050 con 0x55a4ec378660 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.264+0000 7faf67e3e740 10 client.16431 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.264+0000 7faf67e3e740 1 -- 172.21.15.39:0/2222099358 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4ec40cb00 con 0x55a4ec378660 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.265+0000 7faf48c00640 1 -- 172.21.15.39:0/2222099358 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c00fba0 con 0x55a4ec378660 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.265+0000 7faf48c00640 1 -- 172.21.15.39:0/2222099358 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7faf4c021050 con 0x55a4ec378660 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.265+0000 7faf67e3e740 10 client.16431 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.265+0000 7faf67e3e740 10 client.16431 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.265+0000 7faf67e3e740 20 client.16431 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.265+0000 7faf67e3e740 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a4ec35fea0 0x55a4ec763810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.265+0000 7faf67e3e740 4 client.16431 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.265+0000 7faf67e3e740 1 -- 172.21.15.39:0/2222099358 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4ec6be1e0 con 0x55a4ec35fea0 2024-07-04T08:19:39.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.266+0000 7faf53e00640 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a4ec35fea0 0x55a4ec763810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.266+0000 7faf53e00640 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a4ec35fea0 0x55a4ec763810 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd963f9b300990f server_cookie=e2adfcea0eba944b in_seq=0 out_seq=0 2024-07-04T08:19:39.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.266+0000 7faf48c00640 10 client.16431 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:39.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf48c00640 1 -- 172.21.15.39:0/2222099358 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7faf4408ae80 con 0x55a4ec35fea0 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf48c00640 10 client.16431 handle_command_reply: tid=0 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf67e3e740 1 client.16431 shutdown 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf67e3e740 1 -- 172.21.15.39:0/2222099358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 msgr2=0x55a4ec751910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf67e3e740 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec751910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faf4c01c040 tx=0x7faf4c009f10 comp rx=0 tx=0).stop 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf67e3e740 1 -- 172.21.15.39:0/2222099358 shutdown_connections 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf67e3e740 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a4ec35fea0 0x55a4ec763810 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf67e3e740 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec75dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf67e3e740 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec751e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf67e3e740 1 --2- 172.21.15.39:0/2222099358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec751910 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf67e3e740 1 -- 172.21.15.39:0/2222099358 >> 172.21.15.39:0/2222099358 conn(0x55a4ec395910 msgr2=0x55a4ec749430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.267+0000 7faf48c00640 0 client.16431 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.268+0000 7faf67e3e740 1 -- 172.21.15.39:0/2222099358 shutdown_connections 2024-07-04T08:19:39.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.268+0000 7faf67e3e740 1 -- 172.21.15.39:0/2222099358 wait complete. 2024-07-04T08:19:39.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.268+0000 7faf67e3e740 20 client.16431 trim_cache size 0 max 16384 2024-07-04T08:19:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.288+0000 7faf67e3e740 1 Processor -- start 2024-07-04T08:19:39.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.288+0000 7faf67e3e740 1 -- start start 2024-07-04T08:19:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec747940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec74b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec74b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf67e3e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1b2490 con 0x55a4ec378660 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf67e3e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1b3fd0 con 0x55a4ec380840 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf67e3e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1b2b60 con 0x55a4ec3a1520 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec747940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf5aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec747940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50808/0 (socket says 172.21.15.39:50808) 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf5aa00640 1 -- 172.21.15.39:0/1755212587 learned_addr learned my addr 172.21.15.39:0/1755212587 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf5b400640 1 --2- 172.21.15.39:0/1755212587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec74b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf53e00640 1 --2- 172.21.15.39:0/1755212587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec74b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.289+0000 7faf5aa00640 1 -- 172.21.15.39:0/1755212587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 msgr2=0x55a4ec74b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.290+0000 7faf5aa00640 1 --2- 172.21.15.39:0/1755212587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec74b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.290+0000 7faf5aa00640 1 -- 172.21.15.39:0/1755212587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 msgr2=0x55a4ec74b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.290+0000 7faf5aa00640 1 --2- 172.21.15.39:0/1755212587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec74b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.290+0000 7faf5aa00640 1 -- 172.21.15.39:0/1755212587 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4ec483960 con 0x55a4ec378660 2024-07-04T08:19:39.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.290+0000 7faf5aa00640 1 --2- 172.21.15.39:0/1755212587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec747940 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faf4c004990 tx=0x7faf4c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4392b79d9df981 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:39.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.290+0000 7faf4b400640 1 -- 172.21.15.39:0/1755212587 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c026070 con 0x55a4ec378660 2024-07-04T08:19:39.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.290+0000 7faf4b400640 1 -- 172.21.15.39:0/1755212587 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c004b80 con 0x55a4ec378660 2024-07-04T08:19:39.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.290+0000 7faf4b400640 1 -- 172.21.15.39:0/1755212587 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c01e040 con 0x55a4ec378660 2024-07-04T08:19:39.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.291+0000 7faf67e3e740 1 -- 172.21.15.39:0/1755212587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 msgr2=0x55a4ec747940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:39.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.291+0000 7faf67e3e740 1 --2- 172.21.15.39:0/1755212587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec747940 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faf4c004990 tx=0x7faf4c005f90 comp rx=0 tx=0).stop 2024-07-04T08:19:39.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.291+0000 7faf67e3e740 1 -- 172.21.15.39:0/1755212587 shutdown_connections 2024-07-04T08:19:39.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.291+0000 7faf67e3e740 1 --2- 172.21.15.39:0/1755212587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec3a1520 0x55a4ec74b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.291+0000 7faf67e3e740 1 --2- 172.21.15.39:0/1755212587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec380840 0x55a4ec74b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.291+0000 7faf67e3e740 1 --2- 172.21.15.39:0/1755212587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec378660 0x55a4ec747940 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.291+0000 7faf67e3e740 1 -- 172.21.15.39:0/1755212587 >> 172.21.15.39:0/1755212587 conn(0x55a4ec395910 msgr2=0x55a4ec7537f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:39.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.291+0000 7faf67e3e740 1 -- 172.21.15.39:0/1755212587 shutdown_connections 2024-07-04T08:19:39.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.291+0000 7faf67e3e740 1 -- 172.21.15.39:0/1755212587 wait complete. 2024-07-04T08:19:39.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.292+0000 7faf67e3e740 1 -- start start 2024-07-04T08:19:39.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.292+0000 7faf67e3e740 1 Processor -- start 2024-07-04T08:19:39.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.292+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec378660 0x55a4ec75a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.292+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec755840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.292+0000 7faf67e3e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec3a1520 0x55a4ec755ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf67e3e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1b3900 con 0x55a4ec380840 2024-07-04T08:19:39.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf67e3e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1b3230 con 0x55a4ec3a1520 2024-07-04T08:19:39.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf67e3e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c1b0470 con 0x55a4ec378660 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf67e3e740 10 client.0 _ll_register_callbacks cb 0x7faf4c0114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf53e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec755840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf53e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec755840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50810/0 (socket says 172.21.15.39:50810) 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf53e00640 1 -- 172.21.15.39:0/116172043 learned_addr learned my addr 172.21.15.39:0/116172043 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf5b400640 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec3a1520 0x55a4ec755ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf53e00640 1 -- 172.21.15.39:0/116172043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec378660 msgr2=0x55a4ec75a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf53e00640 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec378660 0x55a4ec75a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf53e00640 1 -- 172.21.15.39:0/116172043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec3a1520 msgr2=0x55a4ec755ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf53e00640 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec3a1520 0x55a4ec755ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.293+0000 7faf53e00640 1 -- 172.21.15.39:0/116172043 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4ec678880 con 0x55a4ec380840 2024-07-04T08:19:39.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.294+0000 7faf53e00640 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec755840 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faf440dff90 tx=0x7faf440e6220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e09d1e8f575fba server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.294+0000 7faf48c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.294+0000 7faf48c00640 1 -- 172.21.15.39:0/116172043 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44009460 con 0x55a4ec380840 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.294+0000 7faf48c00640 1 -- 172.21.15.39:0/116172043 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7faf44003ac0 con 0x55a4ec380840 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.294+0000 7faf67e3e740 1 -- 172.21.15.39:0/116172043 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf4c004b80 con 0x55a4ec380840 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.294+0000 7faf48c00640 1 -- 172.21.15.39:0/116172043 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf440b2260 con 0x55a4ec380840 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.294+0000 7faf67e3e740 10 client.16437 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.294+0000 7faf67e3e740 1 -- 172.21.15.39:0/116172043 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4ec674430 con 0x55a4ec380840 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.295+0000 7faf48c00640 1 -- 172.21.15.39:0/116172043 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c004b80 con 0x55a4ec380840 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.295+0000 7faf48c00640 1 -- 172.21.15.39:0/116172043 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7faf440a5d50 con 0x55a4ec380840 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.295+0000 7faf67e3e740 10 client.16437 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.295+0000 7faf67e3e740 10 client.16437 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.295+0000 7faf67e3e740 20 client.16437 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:39.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.295+0000 7faf67e3e740 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a4ec335250 0x55a4ec7ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:39.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.295+0000 7faf67e3e740 4 client.16437 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "705d2328", "release": true, "await": true}] 2024-07-04T08:19:39.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.295+0000 7faf67e3e740 1 -- 172.21.15.39:0/116172043 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "705d2328", "release": true, "await": true}) -- 0x55a4ec6b9c80 con 0x55a4ec335250 2024-07-04T08:19:39.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.295+0000 7faf5aa00640 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a4ec335250 0x55a4ec7ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:39.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.296+0000 7faf5aa00640 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a4ec335250 0x55a4ec7ce3c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cdb8f36b1d6b65 server_cookie=55e8958f59c508b3 in_seq=0 out_seq=0 2024-07-04T08:19:39.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.296+0000 7faf48c00640 10 client.16437 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:39.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf48c00640 1 -- 172.21.15.39:0/116172043 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a4ec6b9c80 con 0x55a4ec335250 2024-07-04T08:19:39.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf48c00640 10 client.16437 handle_command_reply: tid=0 2024-07-04T08:19:39.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 client.16437 shutdown 2024-07-04T08:19:39.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 -- 172.21.15.39:0/116172043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 msgr2=0x55a4ec755840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:39.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec755840 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faf440dff90 tx=0x7faf440e6220 comp rx=0 tx=0).stop 2024-07-04T08:19:39.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 -- 172.21.15.39:0/116172043 shutdown_connections 2024-07-04T08:19:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a4ec335250 0x55a4ec7ce3c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4ec380840 0x55a4ec755840 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4ec3a1520 0x55a4ec755ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 --2- 172.21.15.39:0/116172043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4ec378660 0x55a4ec75a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 -- 172.21.15.39:0/116172043 >> 172.21.15.39:0/116172043 conn(0x55a4ec395910 msgr2=0x55a4ec75c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf48c00640 0 client.16437 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:19:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 -- 172.21.15.39:0/116172043 shutdown_connections 2024-07-04T08:19:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 1 -- 172.21.15.39:0/116172043 wait complete. 2024-07-04T08:19:39.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.298+0000 7faf67e3e740 20 client.16437 trim_cache size 0 max 16384 2024-07-04T08:19:39.299 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:19:39.299 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:19:39.299 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:19:39.299 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 12, 2024-07-04T08:19:39.299 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:19:39.299 INFO:teuthology.orchestra.run.smithi039.stdout: "705d2328": { 2024-07-04T08:19:39.299 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 12, 2024-07-04T08:19:39.299 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:19:39.299 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 152.3, 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:39.300 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:39.301 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:39.301 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:19:39.301 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:19:39.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 -- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faf400b29e0 msgr2=0x7faf400b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:39.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faf400b29e0 0x7faf400b4ef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf54006010 tx=0x7faf540093e0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 -- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf5c108a00 msgr2=0x7faf5c19c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf5c108a00 0x7faf5c19c130 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faf5c06b290 tx=0x7faf4400ee50 comp rx=0 tx=0).stop 2024-07-04T08:19:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 -- 172.21.15.39:0/4126807856 shutdown_connections 2024-07-04T08:19:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7faf400b29e0 0x7faf400b4ef0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf5c108a00 0x7faf5c19c130 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf5c109350 0x7faf5c1a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 --2- 172.21.15.39:0/4126807856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faf5c1030c0 0x7faf5c19bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.299+0000 7faf61800640 1 -- 172.21.15.39:0/4126807856 >> 172.21.15.39:0/4126807856 conn(0x7faf5c0feb30 msgr2=0x7faf5c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.300+0000 7faf61800640 1 -- 172.21.15.39:0/4126807856 shutdown_connections 2024-07-04T08:19:39.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:39.300+0000 7faf61800640 1 -- 172.21.15.39:0/4126807856 wait complete. 2024-07-04T08:19:39.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:38 smithi133 ceph-mon[37050]: pgmap v613: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-07-04T08:19:39.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:38 smithi050 ceph-mon[36577]: pgmap v613: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-07-04T08:19:41.204 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:40 smithi039 ceph-mon[32062]: pgmap v614: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-07-04T08:19:41.207 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '705d2328', total seconds elapsed: 42.7 2024-07-04T08:19:41.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:40 smithi133 ceph-mon[37050]: pgmap v614: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-07-04T08:19:41.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:40 smithi050 ceph-mon[36577]: pgmap v614: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-07-04T08:19:41.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.788+0000 7f26e8e00640 1 -- 172.21.15.39:0/734928349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 msgr2=0x7f26e4105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:41.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.788+0000 7f26e8e00640 1 --2- 172.21.15.39:0/734928349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 0x7f26e4105580 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009900 tx=0x7f26cc02a410 comp rx=0 tx=0).stop 2024-07-04T08:19:41.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.789+0000 7f26e8e00640 1 -- 172.21.15.39:0/734928349 shutdown_connections 2024-07-04T08:19:41.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.789+0000 7f26e8e00640 1 --2- 172.21.15.39:0/734928349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e410b370 0x7f26e4111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:41.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.789+0000 7f26e8e00640 1 --2- 172.21.15.39:0/734928349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e410aa20 0x7f26e410ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:41.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.789+0000 7f26e8e00640 1 --2- 172.21.15.39:0/734928349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 0x7f26e4105580 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:41.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.789+0000 7f26e8e00640 1 -- 172.21.15.39:0/734928349 >> 172.21.15.39:0/734928349 conn(0x7f26e4100be0 msgr2=0x7f26e4103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:41.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.789+0000 7f26e8e00640 1 -- 172.21.15.39:0/734928349 shutdown_connections 2024-07-04T08:19:41.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.789+0000 7f26e8e00640 1 -- 172.21.15.39:0/734928349 wait complete. 2024-07-04T08:19:41.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.790+0000 7f26e8e00640 1 Processor -- start 2024-07-04T08:19:41.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.791+0000 7f26e8e00640 1 -- start start 2024-07-04T08:19:41.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.791+0000 7f26e8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 0x7f26e419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.791+0000 7f26e8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e410aa20 0x7f26e419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.791+0000 7f26e8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e410b370 0x7f26e41a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.791+0000 7f26e8e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4115070 con 0x7f26e410b370 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.791+0000 7f26e8e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26e4114ef0 con 0x7f26e4105170 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.791+0000 7f26e8e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f26e41151f0 con 0x7f26e410aa20 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e410b370 0x7f26e41a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 0x7f26e419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e410aa20 0x7f26e419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 0x7f26e419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38732/0 (socket says 172.21.15.39:38732) 2024-07-04T08:19:41.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3400640 1 -- 172.21.15.39:0/3574577895 learned_addr learned my addr 172.21.15.39:0/3574577895 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:41.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3400640 1 -- 172.21.15.39:0/3574577895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e410aa20 msgr2=0x7f26e419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:41.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3400640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e410aa20 0x7f26e419bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:41.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3400640 1 -- 172.21.15.39:0/3574577895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e410b370 msgr2=0x7f26e41a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:41.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3400640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e410b370 0x7f26e41a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3400640 1 -- 172.21.15.39:0/3574577895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e41a1950 con 0x7f26e4105170 2024-07-04T08:19:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.792+0000 7f26e3400640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 0x7f26e419b9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005ab0 tx=0x7f26cc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.793+0000 7f26dbe00640 1 -- 172.21.15.39:0/3574577895 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc03f070 con 0x7f26e4105170 2024-07-04T08:19:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.793+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e41a1be0 con 0x7f26e4105170 2024-07-04T08:19:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.793+0000 7f26dbe00640 1 -- 172.21.15.39:0/3574577895 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26cc031b60 con 0x7f26e4105170 2024-07-04T08:19:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.793+0000 7f26dbe00640 1 -- 172.21.15.39:0/3574577895 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc035030 con 0x7f26e4105170 2024-07-04T08:19:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.793+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e41a83a0 con 0x7f26e4105170 2024-07-04T08:19:41.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.794+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e4105580 con 0x7f26e4105170 2024-07-04T08:19:41.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.797+0000 7f26dbe00640 1 -- 172.21.15.39:0/3574577895 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f26cc042070 con 0x7f26e4105170 2024-07-04T08:19:41.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.798+0000 7f26dbe00640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26c80b2960 0x7f26c80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:41.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.799+0000 7f26dbe00640 1 -- 172.21.15.39:0/3574577895 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f26cc1036c0 con 0x7f26e4105170 2024-07-04T08:19:41.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.799+0000 7f26e2a00640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26c80b2960 0x7f26c80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:41.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.800+0000 7f26e2a00640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26c80b2960 0x7f26c80b4e70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f26e410a570 tx=0x7f26d4009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:41.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:41.800+0000 7f26dbe00640 1 -- 172.21.15.39:0/3574577895 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f26cc0ca6c0 con 0x7f26e4105170 2024-07-04T08:19:42.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.059+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26e419cb10 con 0x7f26e4105170 2024-07-04T08:19:42.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.060+0000 7f26dbe00640 1 -- 172.21.15.39:0/3574577895 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f26cc0ce9c0 con 0x7f26e4105170 2024-07-04T08:19:42.059 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:19:42.059 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:19:42.059 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:19:42.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26c80b2960 msgr2=0x7f26c80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:42.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26c80b2960 0x7f26c80b4e70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f26e410a570 tx=0x7f26d4009500 comp rx=0 tx=0).stop 2024-07-04T08:19:42.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 msgr2=0x7f26e419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:42.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 0x7f26e419b9e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f26cc005ab0 tx=0x7f26cc003c10 comp rx=0 tx=0).stop 2024-07-04T08:19:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 shutdown_connections 2024-07-04T08:19:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26c80b2960 0x7f26c80b4e70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e410b370 0x7f26e41a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26e410aa20 0x7f26e419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 --2- 172.21.15.39:0/3574577895 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26e4105170 0x7f26e419b9e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:42.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.064+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 >> 172.21.15.39:0/3574577895 conn(0x7f26e4100be0 msgr2=0x7f26e4103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:42.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.065+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 shutdown_connections 2024-07-04T08:19:42.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:42.065+0000 7f26e8e00640 1 -- 172.21.15.39:0/3574577895 wait complete. 2024-07-04T08:19:42.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:19:43.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:42 smithi133 ceph-mon[37050]: pgmap v615: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-04T08:19:43.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:42 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3574577895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:43.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:42 smithi039 ceph-mon[32062]: pgmap v615: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-04T08:19:43.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:42 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3574577895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:43.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:42 smithi050 ceph-mon[36577]: pgmap v615: 129 pgs: 129 active+clean; 3.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 10 op/s 2024-07-04T08:19:43.392 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:42 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3574577895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:19:43.414 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:45.203 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:44 smithi039 ceph-mon[32062]: pgmap v616: 129 pgs: 129 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-07-04T08:19:45.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:44 smithi133 ceph-mon[37050]: pgmap v616: 129 pgs: 129 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-07-04T08:19:45.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:44 smithi050 ceph-mon[36577]: pgmap v616: 129 pgs: 129 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-07-04T08:19:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.717+0000 7f1b9e800640 1 -- 172.21.15.39:0/1647670335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9810f460 msgr2=0x7f1b981118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.717+0000 7f1b9e800640 1 --2- 172.21.15.39:0/1647670335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9810f460 0x7f1b981118a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009ca0 tx=0x7f1b9002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.717+0000 7f1b9e800640 1 -- 172.21.15.39:0/1647670335 shutdown_connections 2024-07-04T08:19:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.717+0000 7f1b9e800640 1 --2- 172.21.15.39:0/1647670335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9810f460 0x7f1b981118a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.717+0000 7f1b9e800640 1 --2- 172.21.15.39:0/1647670335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98075650 0x7f1b9810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.717+0000 7f1b9e800640 1 --2- 172.21.15.39:0/1647670335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98074d00 0x7f1b98075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.717+0000 7f1b9e800640 1 -- 172.21.15.39:0/1647670335 >> 172.21.15.39:0/1647670335 conn(0x7f1b980fe990 msgr2=0x7f1b98100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:45.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.717+0000 7f1b9e800640 1 -- 172.21.15.39:0/1647670335 shutdown_connections 2024-07-04T08:19:45.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.717+0000 7f1b9e800640 1 -- 172.21.15.39:0/1647670335 wait complete. 2024-07-04T08:19:45.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.719+0000 7f1b9e800640 1 Processor -- start 2024-07-04T08:19:45.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.719+0000 7f1b9e800640 1 -- start start 2024-07-04T08:19:45.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.719+0000 7f1b9e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98074d00 0x7f1b9819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.719+0000 7f1b9e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98075650 0x7f1b9819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.719+0000 7f1b9e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9810f460 0x7f1b981a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.719+0000 7f1b9e800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98115040 con 0x7f1b9810f460 2024-07-04T08:19:45.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.719+0000 7f1b9e800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b98114ec0 con 0x7f1b98075650 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.719+0000 7f1b9e800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b981151c0 con 0x7f1b98074d00 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98075650 0x7f1b9819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98074d00 0x7f1b9819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98075650 0x7f1b9819c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38750/0 (socket says 172.21.15.39:38750) 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9ca00640 1 -- 172.21.15.39:0/3898201236 learned_addr learned my addr 172.21.15.39:0/3898201236 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9de00640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9810f460 0x7f1b981a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9ca00640 1 -- 172.21.15.39:0/3898201236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98074d00 msgr2=0x7f1b9819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98074d00 0x7f1b9819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9ca00640 1 -- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9810f460 msgr2=0x7f1b981a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9810f460 0x7f1b981a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9ca00640 1 -- 172.21.15.39:0/3898201236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b981a19e0 con 0x7f1b98075650 2024-07-04T08:19:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.720+0000 7f1b9d400640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98074d00 0x7f1b9819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.721+0000 7f1b9de00640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9810f460 0x7f1b981a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:19:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.721+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98075650 0x7f1b9819c010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400e9c0 tx=0x7f1b8400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.721+0000 7f1b96000640 1 -- 172.21.15.39:0/3898201236 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8400cd80 con 0x7f1b98075650 2024-07-04T08:19:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.721+0000 7f1b9e800640 1 -- 172.21.15.39:0/3898201236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b981a1cd0 con 0x7f1b98075650 2024-07-04T08:19:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.722+0000 7f1b9e800640 1 -- 172.21.15.39:0/3898201236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b981a8430 con 0x7f1b98075650 2024-07-04T08:19:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.722+0000 7f1b96000640 1 -- 172.21.15.39:0/3898201236 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b840035f0 con 0x7f1b98075650 2024-07-04T08:19:45.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.722+0000 7f1b96000640 1 -- 172.21.15.39:0/3898201236 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84010690 con 0x7f1b98075650 2024-07-04T08:19:45.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.725+0000 7f1ba63d6740 1 Processor -- start 2024-07-04T08:19:45.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.725+0000 7f1ba63d6740 1 -- start start 2024-07-04T08:19:45.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d7629070 0x55a4d79cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 0x55a4d79cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7608390 0x55a4d79ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a4d7943ee0 con 0x55a4d7608390 2024-07-04T08:19:45.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a4d7933480 con 0x55a4d7629070 2024-07-04T08:19:45.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a4d79374c0 con 0x55a4d76001b0 2024-07-04T08:19:45.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1b9de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d7629070 0x55a4d79cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 0x55a4d79cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1b9de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d7629070 0x55a4d79cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38760/0 (socket says 172.21.15.39:38760) 2024-07-04T08:19:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.726+0000 7f1b9de00640 1 -- 172.21.15.39:0/1477868867 learned_addr learned my addr 172.21.15.39:0/1477868867 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b9de00640 1 -- 172.21.15.39:0/1477868867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 msgr2=0x55a4d79cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b9de00640 1 --2- 172.21.15.39:0/1477868867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 0x55a4d79cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b9de00640 1 -- 172.21.15.39:0/1477868867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7608390 msgr2=0x55a4d79ce3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:19:45.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b9de00640 1 --2- 172.21.15.39:0/1477868867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7608390 0x55a4d79ce3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b9de00640 1 -- 172.21.15.39:0/1477868867 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d78cb650 con 0x55a4d7629070 2024-07-04T08:19:45.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b9d400640 1 --2- 172.21.15.39:0/1477868867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 0x55a4d79cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:45.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b9de00640 1 --2- 172.21.15.39:0/1477868867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d7629070 0x55a4d79cd950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009870 tx=0x7f1b900098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0253aaa57caa499 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:45.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b8aa00640 1 -- 172.21.15.39:0/1477868867 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90031d70 con 0x55a4d7629070 2024-07-04T08:19:45.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b8aa00640 1 -- 172.21.15.39:0/1477868867 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b90036040 con 0x55a4d7629070 2024-07-04T08:19:45.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.727+0000 7f1b8aa00640 1 -- 172.21.15.39:0/1477868867 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90004330 con 0x55a4d7629070 2024-07-04T08:19:45.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.728+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1477868867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d7629070 msgr2=0x55a4d79cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:45.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.728+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1477868867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d7629070 0x55a4d79cd950 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009870 tx=0x7f1b900098a0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.728+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1477868867 shutdown_connections 2024-07-04T08:19:45.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.729+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1477868867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7608390 0x55a4d79ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.729+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1477868867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 0x55a4d79cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.729+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1477868867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d7629070 0x55a4d79cd950 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.729+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1477868867 >> 172.21.15.39:0/1477868867 conn(0x55a4d761d460 msgr2=0x55a4d79cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:45.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.729+0000 7f1b96000640 1 -- 172.21.15.39:0/3898201236 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1b840037a0 con 0x7f1b98075650 2024-07-04T08:19:45.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.729+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1477868867 shutdown_connections 2024-07-04T08:19:45.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.729+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1477868867 wait complete. 2024-07-04T08:19:45.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.729+0000 7f1ba63d6740 1 -- start start 2024-07-04T08:19:45.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.730+0000 7f1ba63d6740 1 Processor -- start 2024-07-04T08:19:45.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.730+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 0x55a4d79d95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.730+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7629070 0x55a4d79d9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.730+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d75e79f0 0x55a4d79e5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.730+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a4d7797700 con 0x55a4d7629070 2024-07-04T08:19:45.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.730+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a4d78dabb0 con 0x55a4d75e79f0 2024-07-04T08:19:45.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.730+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a4d79dc0d0 con 0x55a4d76001b0 2024-07-04T08:19:45.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.731+0000 7f1b96000640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b800b28f0 0x7f1b800b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.731+0000 7f1ba63d6740 10 client.0 _ll_register_callbacks cb 0x55a4d792c570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:45.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.731+0000 7f1b96000640 1 -- 172.21.15.39:0/3898201236 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1b840e2ce0 con 0x7f1b98075650 2024-07-04T08:19:45.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.731+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7629070 0x55a4d79d9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.731+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7629070 0x55a4d79d9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56924/0 (socket says 172.21.15.39:56924) 2024-07-04T08:19:45.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.731+0000 7f1b9d400640 1 -- 172.21.15.39:0/1933450547 learned_addr learned my addr 172.21.15.39:0/1933450547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.731+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d75e79f0 0x55a4d79e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.731+0000 7f1b9de00640 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 0x55a4d79d95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.731+0000 7f1b9de00640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b800b28f0 0x7f1b800b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.732+0000 7f1b9d400640 1 -- 172.21.15.39:0/1933450547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 msgr2=0x55a4d79d95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.732+0000 7f1b9d400640 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 0x55a4d79d95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.732+0000 7f1b9d400640 1 -- 172.21.15.39:0/1933450547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d75e79f0 msgr2=0x55a4d79e5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.732+0000 7f1b9d400640 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d75e79f0 0x55a4d79e5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.732+0000 7f1b9d400640 1 -- 172.21.15.39:0/1933450547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d7694650 con 0x55a4d7629070 2024-07-04T08:19:45.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.732+0000 7f1b9d400640 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7629070 0x55a4d79d9bb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c012d20 tx=0x7f1b8c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70eedd52e081b3e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:45.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.732+0000 7f1b7f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:45.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.732+0000 7f1b9de00640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b800b28f0 0x7f1b800b4e00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1b9002f470 tx=0x7f1b90005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:45.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.733+0000 7f1b7f400640 1 -- 172.21.15.39:0/1933450547 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c0120d0 con 0x55a4d7629070 2024-07-04T08:19:45.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.733+0000 7f1b7f400640 1 -- 172.21.15.39:0/1933450547 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1b8c0047a0 con 0x55a4d7629070 2024-07-04T08:19:45.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.733+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1933450547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b90036040 con 0x55a4d7629070 2024-07-04T08:19:45.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.733+0000 7f1b7f400640 1 -- 172.21.15.39:0/1933450547 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1b8c025050 con 0x55a4d7629070 2024-07-04T08:19:45.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.733+0000 7f1ba63d6740 10 client.16452 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:45.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.733+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1933450547 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4d79e8250 con 0x55a4d7629070 2024-07-04T08:19:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.733+0000 7f1b7f400640 1 -- 172.21.15.39:0/1933450547 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b90036040 con 0x55a4d7629070 2024-07-04T08:19:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.734+0000 7f1b7f400640 1 -- 172.21.15.39:0/1933450547 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1b8c01a400 con 0x55a4d7629070 2024-07-04T08:19:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.734+0000 7f1ba63d6740 10 client.16452 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.734+0000 7f1ba63d6740 10 client.16452 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:19:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.734+0000 7f1ba63d6740 20 client.16452 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.734+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a4d75bcda0 0x55a4d79eb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.734+0000 7f1ba63d6740 4 client.16452 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.734+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1933450547 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4d7945e30 con 0x55a4d75bcda0 2024-07-04T08:19:45.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.735+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a4d75bcda0 0x55a4d79eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.736+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a4d75bcda0 0x55a4d79eb750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c7d6e7093c5bc113 server_cookie=f09de7fbdb5ecbcb in_seq=0 out_seq=0 2024-07-04T08:19:45.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.736+0000 7f1b7f400640 10 client.16452 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:19:45.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1b7f400640 1 -- 172.21.15.39:0/1933450547 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a4d7945e30 con 0x55a4d75bcda0 2024-07-04T08:19:45.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1b7f400640 10 client.16452 handle_command_reply: tid=0 2024-07-04T08:19:45.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1ba63d6740 1 client.16452 shutdown 2024-07-04T08:19:45.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1933450547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7629070 msgr2=0x55a4d79d9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7629070 0x55a4d79d9bb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c012d20 tx=0x7f1b8c00f790 comp rx=0 tx=0).stop 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1933450547 shutdown_connections 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a4d75bcda0 0x55a4d79eb750 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d7629070 0x55a4d79d9bb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d75e79f0 0x55a4d79e5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/1933450547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d76001b0 0x55a4d79d95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1b7f400640 0 client.16452 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.738+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1933450547 >> 172.21.15.39:0/1933450547 conn(0x55a4d761d460 msgr2=0x55a4d79d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.739+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1933450547 shutdown_connections 2024-07-04T08:19:45.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.739+0000 7f1ba63d6740 1 -- 172.21.15.39:0/1933450547 wait complete. 2024-07-04T08:19:45.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.739+0000 7f1ba63d6740 20 client.16452 trim_cache size 0 max 16384 2024-07-04T08:19:45.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.771+0000 7f1ba63d6740 1 Processor -- start 2024-07-04T08:19:45.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.772+0000 7f1ba63d6740 1 -- start start 2024-07-04T08:19:45.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.772+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79d9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.772+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 0x55a4d79f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.772+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 0x55a4d79f1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.772+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b981b23e0 con 0x55a4d75e79f0 2024-07-04T08:19:45.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.772+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b981b3f20 con 0x55a4d76001b0 2024-07-04T08:19:45.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.772+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b981b2ab0 con 0x55a4d7629070 2024-07-04T08:19:45.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.773+0000 7f1b9de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 0x55a4d79f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.773+0000 7f1b9de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 0x55a4d79f1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36536/0 (socket says 172.21.15.39:36536) 2024-07-04T08:19:45.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.773+0000 7f1b9de00640 1 -- 172.21.15.39:0/386586581 learned_addr learned my addr 172.21.15.39:0/386586581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:45.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.773+0000 7f1b9d400640 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79d9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.774+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 0x55a4d79f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.774+0000 7f1b9de00640 1 -- 172.21.15.39:0/386586581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 msgr2=0x55a4d79f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:45.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.774+0000 7f1b9de00640 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 0x55a4d79f0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.774+0000 7f1b9de00640 1 -- 172.21.15.39:0/386586581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 msgr2=0x55a4d79d9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:45.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.774+0000 7f1b9de00640 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79d9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.774+0000 7f1b9de00640 1 -- 172.21.15.39:0/386586581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d79cae70 con 0x55a4d7629070 2024-07-04T08:19:45.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.774+0000 7f1b9d400640 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79d9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:19:45.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.774+0000 7f1b9de00640 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 0x55a4d79f1450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b90000f80 tx=0x7f1b90009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f452ff6d18c9a4f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:45.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.774+0000 7f1b8a000640 1 -- 172.21.15.39:0/386586581 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90004430 con 0x55a4d7629070 2024-07-04T08:19:45.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.775+0000 7f1b8a000640 1 -- 172.21.15.39:0/386586581 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b90036070 con 0x55a4d7629070 2024-07-04T08:19:45.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.775+0000 7f1b8a000640 1 -- 172.21.15.39:0/386586581 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9002f950 con 0x55a4d7629070 2024-07-04T08:19:45.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.776+0000 7f1ba63d6740 1 -- 172.21.15.39:0/386586581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 msgr2=0x55a4d79f1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:45.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.776+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 0x55a4d79f1450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1b90000f80 tx=0x7f1b90009800 comp rx=0 tx=0).stop 2024-07-04T08:19:45.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.776+0000 7f1ba63d6740 1 -- 172.21.15.39:0/386586581 shutdown_connections 2024-07-04T08:19:45.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.776+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 0x55a4d79f1450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.776+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 0x55a4d79f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.776+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/386586581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79d9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.776+0000 7f1ba63d6740 1 -- 172.21.15.39:0/386586581 >> 172.21.15.39:0/386586581 conn(0x55a4d761d460 msgr2=0x55a4d79d95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:45.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.776+0000 7f1ba63d6740 1 -- 172.21.15.39:0/386586581 shutdown_connections 2024-07-04T08:19:45.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.776+0000 7f1ba63d6740 1 -- 172.21.15.39:0/386586581 wait complete. 2024-07-04T08:19:45.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.777+0000 7f1ba63d6740 1 -- start start 2024-07-04T08:19:45.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.777+0000 7f1ba63d6740 1 Processor -- start 2024-07-04T08:19:45.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79dd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 0x55a4d79dd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1ba63d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 0x55a4d79ddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b981b3850 con 0x55a4d75e79f0 2024-07-04T08:19:45.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b981b3180 con 0x55a4d76001b0 2024-07-04T08:19:45.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1ba63d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b981b03c0 con 0x55a4d7629070 2024-07-04T08:19:45.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1ba63d6740 10 client.0 _ll_register_callbacks cb 0x7f1b8c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:45.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79dd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56950/0 (socket says 172.21.15.39:56950) 2024-07-04T08:19:45.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1b9d400640 1 -- 172.21.15.39:0/2775264870 learned_addr learned my addr 172.21.15.39:0/2775264870 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:45.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.778+0000 7f1b9ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 0x55a4d79dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.779+0000 7f1b9d400640 1 -- 172.21.15.39:0/2775264870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 msgr2=0x55a4d79ddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:45.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.779+0000 7f1b9d400640 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 0x55a4d79ddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.779+0000 7f1b9d400640 1 -- 172.21.15.39:0/2775264870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 msgr2=0x55a4d79dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.779+0000 7f1b9d400640 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 0x55a4d79dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.779+0000 7f1b9d400640 1 -- 172.21.15.39:0/2775264870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d7a52ee0 con 0x55a4d75e79f0 2024-07-04T08:19:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.779+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 0x55a4d79dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.779+0000 7f1b9d400640 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79dd230 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00fd80 tx=0x7f1b8c012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e76342cab05a8c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:45.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.779+0000 7f1b7f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:45.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.780+0000 7f1b7f400640 1 -- 172.21.15.39:0/2775264870 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c01a580 con 0x55a4d75e79f0 2024-07-04T08:19:45.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.780+0000 7f1ba63d6740 1 -- 172.21.15.39:0/2775264870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b90036070 con 0x55a4d75e79f0 2024-07-04T08:19:45.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.780+0000 7f1b7f400640 1 -- 172.21.15.39:0/2775264870 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1b8c01abd0 con 0x55a4d75e79f0 2024-07-04T08:19:45.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.780+0000 7f1b7f400640 1 -- 172.21.15.39:0/2775264870 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1b8c031050 con 0x55a4d75e79f0 2024-07-04T08:19:45.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.780+0000 7f1ba63d6740 10 client.16455 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:45.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.780+0000 7f1ba63d6740 1 -- 172.21.15.39:0/2775264870 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4d7a54140 con 0x55a4d75e79f0 2024-07-04T08:19:45.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.781+0000 7f1b7f400640 1 -- 172.21.15.39:0/2775264870 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b90036070 con 0x55a4d75e79f0 2024-07-04T08:19:45.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.781+0000 7f1b7f400640 1 -- 172.21.15.39:0/2775264870 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1b8c028070 con 0x55a4d75e79f0 2024-07-04T08:19:45.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.781+0000 7f1ba63d6740 10 client.16455 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:45.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.781+0000 7f1ba63d6740 10 client.16455 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:19:45.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.781+0000 7f1ba63d6740 20 client.16455 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:45.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.781+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a4d7a548f0 0x55a4d7a56d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:45.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.781+0000 7f1ba63d6740 4 client.16455 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:19:45.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.781+0000 7f1ba63d6740 1 -- 172.21.15.39:0/2775264870 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4d79418d0 con 0x55a4d7a548f0 2024-07-04T08:19:45.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.782+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a4d7a548f0 0x55a4d7a56d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:45.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.782+0000 7f1b9ca00640 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a4d7a548f0 0x55a4d7a56d10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c088b72f1d455691 server_cookie=df83bbe6853efb89 in_seq=0 out_seq=0 2024-07-04T08:19:45.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.782+0000 7f1b7f400640 10 client.16455 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:19:45.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.782+0000 7f1b7f400640 1 -- 172.21.15.39:0/2775264870 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4d79418d0 con 0x55a4d7a548f0 2024-07-04T08:19:45.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1b7f400640 10 client.16455 handle_command_reply: tid=0 2024-07-04T08:19:45.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 client.16455 shutdown 2024-07-04T08:19:45.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 -- 172.21.15.39:0/2775264870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 msgr2=0x55a4d79dd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:45.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79dd230 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00fd80 tx=0x7f1b8c012910 comp rx=0 tx=0).stop 2024-07-04T08:19:45.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 -- 172.21.15.39:0/2775264870 shutdown_connections 2024-07-04T08:19:45.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a4d7a548f0 0x55a4d7a56d10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a4d7629070 0x55a4d79ddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a4d76001b0 0x55a4d79dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 --2- 172.21.15.39:0/2775264870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a4d75e79f0 0x55a4d79dd230 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1b7f400640 0 client.16455 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:19:45.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 -- 172.21.15.39:0/2775264870 >> 172.21.15.39:0/2775264870 conn(0x55a4d761d460 msgr2=0x55a4d79ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:45.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 -- 172.21.15.39:0/2775264870 shutdown_connections 2024-07-04T08:19:45.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.783+0000 7f1ba63d6740 1 -- 172.21.15.39:0/2775264870 wait complete. 2024-07-04T08:19:45.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.784+0000 7f1ba63d6740 20 client.16455 trim_cache size 0 max 16384 2024-07-04T08:19:45.786 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:19:45.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 -- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b800b28f0 msgr2=0x7f1b800b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:45.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b800b28f0 0x7f1b800b4e00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1b9002f470 tx=0x7f1b90005fd0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 -- 172.21.15.39:0/3898201236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98075650 msgr2=0x7f1b9819c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:45.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98075650 0x7f1b9819c010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400e9c0 tx=0x7f1b8400ee80 comp rx=0 tx=0).stop 2024-07-04T08:19:45.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 -- 172.21.15.39:0/3898201236 shutdown_connections 2024-07-04T08:19:45.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b800b28f0 0x7f1b800b4e00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b9810f460 0x7f1b981a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b98075650 0x7f1b9819c010 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 --2- 172.21.15.39:0/3898201236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98074d00 0x7f1b9819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:45.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 -- 172.21.15.39:0/3898201236 >> 172.21.15.39:0/3898201236 conn(0x7f1b980fe990 msgr2=0x7f1b98108f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:45.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 -- 172.21.15.39:0/3898201236 shutdown_connections 2024-07-04T08:19:45.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:45.785+0000 7f1b9e800640 1 -- 172.21.15.39:0/3898201236 wait complete. 2024-07-04T08:19:46.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:19:47.000 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:47.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:46 smithi133 ceph-mon[37050]: pgmap v617: 129 pgs: 129 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-07-04T08:19:47.326 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:46 smithi039 ceph-mon[32062]: pgmap v617: 129 pgs: 129 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-07-04T08:19:47.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:46 smithi050 ceph-mon[36577]: pgmap v617: 129 pgs: 129 active+clean; 4.0 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-07-04T08:19:49.099 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:48 smithi039 ceph-mon[32062]: pgmap v618: 129 pgs: 129 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-07-04T08:19:49.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:48 smithi133 ceph-mon[37050]: pgmap v618: 129 pgs: 129 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-07-04T08:19:49.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.366+0000 7f4d1c600640 1 -- 172.21.15.39:0/891549727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14109700 msgr2=0x7f4d14109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.366+0000 7f4d1c600640 1 --2- 172.21.15.39:0/891549727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14109700 0x7f4d14109b10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009900 tx=0x7f4d0802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.366+0000 7f4d1c600640 1 -- 172.21.15.39:0/891549727 shutdown_connections 2024-07-04T08:19:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.366+0000 7f4d1c600640 1 --2- 172.21.15.39:0/891549727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1410f990 0x7f4d141161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.366+0000 7f4d1c600640 1 --2- 172.21.15.39:0/891549727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d1410f040 0x7f4d1410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.366+0000 7f4d1c600640 1 --2- 172.21.15.39:0/891549727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14109700 0x7f4d14109b10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.366+0000 7f4d1c600640 1 -- 172.21.15.39:0/891549727 >> 172.21.15.39:0/891549727 conn(0x7f4d14077630 msgr2=0x7f4d14107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.366+0000 7f4d1c600640 1 -- 172.21.15.39:0/891549727 shutdown_connections 2024-07-04T08:19:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.366+0000 7f4d1c600640 1 -- 172.21.15.39:0/891549727 wait complete. 2024-07-04T08:19:49.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.367+0000 7f4d1c600640 1 Processor -- start 2024-07-04T08:19:49.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d1c600640 1 -- start start 2024-07-04T08:19:49.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d1c600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14109700 0x7f4d141a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d1c600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d1410f040 0x7f4d141a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d1c600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1410f990 0x7f4d141a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d1c600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d141196b0 con 0x7f4d14109700 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d1c600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d14119530 con 0x7f4d1410f040 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d1c600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d14119830 con 0x7f4d1410f990 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d19c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14109700 0x7f4d141a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d19c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14109700 0x7f4d141a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56968/0 (socket says 172.21.15.39:56968) 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.368+0000 7f4d19200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d1410f040 0x7f4d141a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.369+0000 7f4d19c00640 1 -- 172.21.15.39:0/882766911 learned_addr learned my addr 172.21.15.39:0/882766911 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.369+0000 7f4d19200640 1 -- 172.21.15.39:0/882766911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1410f990 msgr2=0x7f4d141a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.369+0000 7f4d19200640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1410f990 0x7f4d141a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.369+0000 7f4d19200640 1 -- 172.21.15.39:0/882766911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14109700 msgr2=0x7f4d141a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.369+0000 7f4d19200640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14109700 0x7f4d141a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.369+0000 7f4d19200640 1 -- 172.21.15.39:0/882766911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d141a5fb0 con 0x7f4d1410f040 2024-07-04T08:19:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.369+0000 7f4d19200640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d1410f040 0x7f4d141a05e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d1406bf70 tx=0x7f4d0400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.370+0000 7f4d12000640 1 -- 172.21.15.39:0/882766911 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0400cc40 con 0x7f4d1410f040 2024-07-04T08:19:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.370+0000 7f4d12000640 1 -- 172.21.15.39:0/882766911 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d0400cde0 con 0x7f4d1410f040 2024-07-04T08:19:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.370+0000 7f4d12000640 1 -- 172.21.15.39:0/882766911 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04018780 con 0x7f4d1410f040 2024-07-04T08:19:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.370+0000 7f4d1c600640 1 -- 172.21.15.39:0/882766911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d141a62a0 con 0x7f4d1410f040 2024-07-04T08:19:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.370+0000 7f4d1c600640 1 -- 172.21.15.39:0/882766911 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d141aca00 con 0x7f4d1410f040 2024-07-04T08:19:49.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.373+0000 7f4d22d09740 1 Processor -- start 2024-07-04T08:19:49.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.373+0000 7f4d22d09740 1 -- start start 2024-07-04T08:19:49.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b2034d070 0x563b206f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d12000640 1 -- 172.21.15.39:0/882766911 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4d04010a90 con 0x7f4d1410f040 2024-07-04T08:19:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b203241b0 0x563b206f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2032c390 0x563b206f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d22d09740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563b20667ee0 con 0x563b203241b0 2024-07-04T08:19:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d22d09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563b20657480 con 0x563b2034d070 2024-07-04T08:19:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d22d09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563b2065b4c0 con 0x563b2032c390 2024-07-04T08:19:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d19c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b203241b0 0x563b206f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d1a600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b2034d070 0x563b206f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d19c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b203241b0 0x563b206f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56970/0 (socket says 172.21.15.39:56970) 2024-07-04T08:19:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d19c00640 1 -- 172.21.15.39:0/1711980693 learned_addr learned my addr 172.21.15.39:0/1711980693 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.374+0000 7f4d19200640 1 --2- 172.21.15.39:0/1711980693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2032c390 0x563b206f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.375+0000 7f4d1a600640 1 -- 172.21.15.39:0/1711980693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2032c390 msgr2=0x563b206f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.375+0000 7f4d1a600640 1 --2- 172.21.15.39:0/1711980693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2032c390 0x563b206f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.375+0000 7f4d1a600640 1 -- 172.21.15.39:0/1711980693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b203241b0 msgr2=0x563b206f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.375+0000 7f4d1a600640 1 --2- 172.21.15.39:0/1711980693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b203241b0 0x563b206f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.375+0000 7f4d1a600640 1 -- 172.21.15.39:0/1711980693 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b205ef650 con 0x563b2034d070 2024-07-04T08:19:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.375+0000 7f4d1a600640 1 --2- 172.21.15.39:0/1711980693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b2034d070 0x563b206f17b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c005fd0 tx=0x7f4d0c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9abc053cf14040c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.375+0000 7f4d02a00640 1 -- 172.21.15.39:0/1711980693 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c004d20 con 0x563b2034d070 2024-07-04T08:19:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.376+0000 7f4d02a00640 1 -- 172.21.15.39:0/1711980693 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d0c00ee70 con 0x563b2034d070 2024-07-04T08:19:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.376+0000 7f4d02a00640 1 -- 172.21.15.39:0/1711980693 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c005020 con 0x563b2034d070 2024-07-04T08:19:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.376+0000 7f4d12000640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4cfc0b29e0 0x7f4cfc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.376+0000 7f4d12000640 1 -- 172.21.15.39:0/882766911 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4d040e2ca0 con 0x7f4d1410f040 2024-07-04T08:19:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.377+0000 7f4d19c00640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4cfc0b29e0 0x7f4cfc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.377+0000 7f4d22d09740 1 -- 172.21.15.39:0/1711980693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b2034d070 msgr2=0x563b206f17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.377+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1711980693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b2034d070 0x563b206f17b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c005fd0 tx=0x7f4d0c00e720 comp rx=0 tx=0).stop 2024-07-04T08:19:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.377+0000 7f4d22d09740 1 -- 172.21.15.39:0/1711980693 shutdown_connections 2024-07-04T08:19:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.377+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1711980693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b203241b0 0x563b206f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.377+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1711980693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2032c390 0x563b206f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.377+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1711980693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b2034d070 0x563b206f17b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.377+0000 7f4d22d09740 1 -- 172.21.15.39:0/1711980693 >> 172.21.15.39:0/1711980693 conn(0x563b20341460 msgr2=0x563b206ef6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.378+0000 7f4d22d09740 1 -- 172.21.15.39:0/1711980693 shutdown_connections 2024-07-04T08:19:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.378+0000 7f4d22d09740 1 -- 172.21.15.39:0/1711980693 wait complete. 2024-07-04T08:19:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.378+0000 7f4d19c00640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4cfc0b29e0 0x7f4cfc0b4ef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d080096c0 tx=0x7f4d08005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.378+0000 7f4d22d09740 1 -- start start 2024-07-04T08:19:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.379+0000 7f4d22d09740 1 Processor -- start 2024-07-04T08:19:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.379+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206fd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.379+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.379+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 0x563b207092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.379+0000 7f4d22d09740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563b204bb700 con 0x563b2032c390 2024-07-04T08:19:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.379+0000 7f4d22d09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563b205febb0 con 0x563b203241b0 2024-07-04T08:19:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.379+0000 7f4d22d09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563b206fff90 con 0x563b2034d070 2024-07-04T08:19:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.379+0000 7f4d22d09740 10 client.0 _ll_register_callbacks cb 0x563b20650570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206fd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38828/0 (socket says 172.21.15.39:38828) 2024-07-04T08:19:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19c00640 1 -- 172.21.15.39:0/1844957533 learned_addr learned my addr 172.21.15.39:0/1844957533 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19c00640 1 -- 172.21.15.39:0/1844957533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 msgr2=0x563b207092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19c00640 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 0x563b207092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19c00640 1 -- 172.21.15.39:0/1844957533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 msgr2=0x563b206fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19c00640 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206fd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19c00640 1 -- 172.21.15.39:0/1844957533 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b203b8650 con 0x563b203241b0 2024-07-04T08:19:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.380+0000 7f4d19200640 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.381+0000 7f4d19c00640 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206fd360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4d08005ab0 tx=0x7f4d08009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86e7bfa196432c2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.381+0000 7f4cf7400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:19:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.381+0000 7f4cf7400640 1 -- 172.21.15.39:0/1844957533 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08035040 con 0x563b203241b0 2024-07-04T08:19:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.381+0000 7f4d22d09740 1 -- 172.21.15.39:0/1844957533 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d0c00ee70 con 0x563b203241b0 2024-07-04T08:19:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.381+0000 7f4cf7400640 1 -- 172.21.15.39:0/1844957533 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4d080318b0 con 0x563b203241b0 2024-07-04T08:19:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.381+0000 7f4cf7400640 1 -- 172.21.15.39:0/1844957533 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d0805a050 con 0x563b203241b0 2024-07-04T08:19:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.381+0000 7f4d22d09740 10 client.26603 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:49.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.381+0000 7f4d22d09740 1 -- 172.21.15.39:0/1844957533 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b2070c000 con 0x563b203241b0 2024-07-04T08:19:49.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.382+0000 7f4cf7400640 1 -- 172.21.15.39:0/1844957533 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d0c00ee70 con 0x563b203241b0 2024-07-04T08:19:49.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.382+0000 7f4cf7400640 1 -- 172.21.15.39:0/1844957533 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4d08044030 con 0x563b203241b0 2024-07-04T08:19:49.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.382+0000 7f4d22d09740 10 client.26603 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:49.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.382+0000 7f4d22d09740 10 client.26603 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:19:49.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.382+0000 7f4d22d09740 20 client.26603 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:49.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.382+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563b2030b9f0 0x563b2070f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.382+0000 7f4d22d09740 4 client.26603 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:49.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.382+0000 7f4d22d09740 1 -- 172.21.15.39:0/1844957533 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b20669e30 con 0x563b2030b9f0 2024-07-04T08:19:49.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.383+0000 7f4d19200640 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563b2030b9f0 0x563b2070f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.383+0000 7f4d19200640 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563b2030b9f0 0x563b2070f5d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=80cd5336ccf140ea server_cookie=cf6ef2e03c5e3a03 in_seq=0 out_seq=0 2024-07-04T08:19:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.383+0000 7f4cf7400640 10 client.26603 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:19:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.384+0000 7f4cf7400640 1 -- 172.21.15.39:0/1844957533 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563b20669e30 con 0x563b2030b9f0 2024-07-04T08:19:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.384+0000 7f4cf7400640 10 client.26603 handle_command_reply: tid=0 2024-07-04T08:19:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.384+0000 7f4d22d09740 1 client.26603 shutdown 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.384+0000 7f4d22d09740 1 -- 172.21.15.39:0/1844957533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 msgr2=0x563b206fd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.384+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206fd360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4d08005ab0 tx=0x7f4d08009260 comp rx=0 tx=0).stop 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.384+0000 7f4d22d09740 1 -- 172.21.15.39:0/1844957533 shutdown_connections 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.384+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563b2030b9f0 0x563b2070f5d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.385+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.385+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 0x563b207092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.385+0000 7f4cf7400640 0 client.26603 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.385+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1844957533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206fd360 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.385+0000 7f4d22d09740 1 -- 172.21.15.39:0/1844957533 >> 172.21.15.39:0/1844957533 conn(0x563b20341460 msgr2=0x563b206f4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.385+0000 7f4d22d09740 1 -- 172.21.15.39:0/1844957533 shutdown_connections 2024-07-04T08:19:49.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.385+0000 7f4d22d09740 1 -- 172.21.15.39:0/1844957533 wait complete. 2024-07-04T08:19:49.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.385+0000 7f4d22d09740 20 client.26603 trim_cache size 0 max 16384 2024-07-04T08:19:49.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:48 smithi050 ceph-mon[36577]: pgmap v618: 129 pgs: 129 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-07-04T08:19:49.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.403+0000 7f4d22d09740 1 Processor -- start 2024-07-04T08:19:49.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.403+0000 7f4d22d09740 1 -- start start 2024-07-04T08:19:49.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 0x563b207090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d22d09740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d141b6740 con 0x563b2032c390 2024-07-04T08:19:49.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d22d09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d141b8280 con 0x563b203241b0 2024-07-04T08:19:49.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d22d09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d141b6e10 con 0x563b2034d070 2024-07-04T08:19:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d19200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d1a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 0x563b207090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d19c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d19200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56988/0 (socket says 172.21.15.39:56988) 2024-07-04T08:19:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d1a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 0x563b207090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36584/0 (socket says 172.21.15.39:36584) 2024-07-04T08:19:49.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.404+0000 7f4d19200640 1 -- 172.21.15.39:0/1915131688 learned_addr learned my addr 172.21.15.39:0/1915131688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.405+0000 7f4d1a600640 1 -- 172.21.15.39:0/1915131688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 msgr2=0x563b206f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.405+0000 7f4d1a600640 1 --2- 172.21.15.39:0/1915131688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.405+0000 7f4d1a600640 1 -- 172.21.15.39:0/1915131688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 msgr2=0x563b206f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.405+0000 7f4d1a600640 1 --2- 172.21.15.39:0/1915131688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.405+0000 7f4d1a600640 1 -- 172.21.15.39:0/1915131688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b206ff740 con 0x563b2034d070 2024-07-04T08:19:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.405+0000 7f4d19200640 1 --2- 172.21.15.39:0/1915131688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:49.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.406+0000 7f4d1a600640 1 --2- 172.21.15.39:0/1915131688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 0x563b207090d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009270 tx=0x7f4d0c009a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8ddccc6effccb6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.406+0000 7f4d02000640 1 -- 172.21.15.39:0/1915131688 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c004d20 con 0x563b2034d070 2024-07-04T08:19:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.406+0000 7f4d02000640 1 -- 172.21.15.39:0/1915131688 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d0c003e20 con 0x563b2034d070 2024-07-04T08:19:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.406+0000 7f4d02000640 1 -- 172.21.15.39:0/1915131688 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c005020 con 0x563b2034d070 2024-07-04T08:19:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.407+0000 7f4d22d09740 1 -- 172.21.15.39:0/1915131688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 msgr2=0x563b207090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.407+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1915131688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 0x563b207090d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009270 tx=0x7f4d0c009a90 comp rx=0 tx=0).stop 2024-07-04T08:19:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.407+0000 7f4d22d09740 1 -- 172.21.15.39:0/1915131688 shutdown_connections 2024-07-04T08:19:49.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.407+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1915131688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2032c390 0x563b206f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.407+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1915131688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2034d070 0x563b207090d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.407+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1915131688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b206f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.407+0000 7f4d22d09740 1 -- 172.21.15.39:0/1915131688 >> 172.21.15.39:0/1915131688 conn(0x563b20341460 msgr2=0x563b206f8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:49.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.407+0000 7f4d22d09740 1 -- 172.21.15.39:0/1915131688 shutdown_connections 2024-07-04T08:19:49.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.407+0000 7f4d22d09740 1 -- 172.21.15.39:0/1915131688 wait complete. 2024-07-04T08:19:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.408+0000 7f4d22d09740 1 -- start start 2024-07-04T08:19:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.408+0000 7f4d22d09740 1 Processor -- start 2024-07-04T08:19:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b20705b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2032c390 0x563b20700fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d22d09740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2034d070 0x563b207016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d22d09740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d141b7bb0 con 0x563b2034d070 2024-07-04T08:19:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d22d09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d141b74e0 con 0x563b203241b0 2024-07-04T08:19:49.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d22d09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d141b4720 con 0x563b2032c390 2024-07-04T08:19:49.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d22d09740 10 client.0 _ll_register_callbacks cb 0x7f4d08009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:49.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d1a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2034d070 0x563b207016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d1a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2034d070 0x563b207016e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56998/0 (socket says 172.21.15.39:56998) 2024-07-04T08:19:49.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d19c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b20705b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.409+0000 7f4d1a600640 1 -- 172.21.15.39:0/1242713022 learned_addr learned my addr 172.21.15.39:0/1242713022 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:49.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.410+0000 7f4d19c00640 1 -- 172.21.15.39:0/1242713022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2032c390 msgr2=0x563b20700fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:49.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.410+0000 7f4d19c00640 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2032c390 0x563b20700fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.410+0000 7f4d19c00640 1 -- 172.21.15.39:0/1242713022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2034d070 msgr2=0x563b207016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.410+0000 7f4d19c00640 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2034d070 0x563b207016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.410+0000 7f4d19c00640 1 -- 172.21.15.39:0/1242713022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b20776d60 con 0x563b203241b0 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.410+0000 7f4d1a600640 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2034d070 0x563b207016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.410+0000 7f4d19c00640 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b20705b00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d08051d10 tx=0x7f4d08031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=652387deeacdd1cd server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.410+0000 7f4cf7400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.411+0000 7f4cf7400640 1 -- 172.21.15.39:0/1242713022 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080316b0 con 0x563b203241b0 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.411+0000 7f4d22d09740 1 -- 172.21.15.39:0/1242713022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d0c003e20 con 0x563b203241b0 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.411+0000 7f4cf7400640 1 -- 172.21.15.39:0/1242713022 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4d08031850 con 0x563b203241b0 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.411+0000 7f4cf7400640 1 -- 172.21.15.39:0/1242713022 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d08031a50 con 0x563b203241b0 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.411+0000 7f4d22d09740 10 client.26609 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.411+0000 7f4d22d09740 1 -- 172.21.15.39:0/1242713022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b20777e90 con 0x563b203241b0 2024-07-04T08:19:49.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.412+0000 7f4cf7400640 1 -- 172.21.15.39:0/1242713022 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d0c003e20 con 0x563b203241b0 2024-07-04T08:19:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.412+0000 7f4cf7400640 1 -- 172.21.15.39:0/1242713022 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4d08044040 con 0x563b203241b0 2024-07-04T08:19:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.412+0000 7f4d22d09740 10 client.26609 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.412+0000 7f4d22d09740 10 client.26609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:19:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.412+0000 7f4d22d09740 20 client.26609 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.412+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563b202e0da0 0x563b2077a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.412+0000 7f4d22d09740 4 client.26609 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:19:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.412+0000 7f4d22d09740 1 -- 172.21.15.39:0/1242713022 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b206658d0 con 0x563b202e0da0 2024-07-04T08:19:49.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.412+0000 7f4d19200640 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563b202e0da0 0x563b2077a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.413+0000 7f4d19200640 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563b202e0da0 0x563b2077a640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20d8394aa3d25406 server_cookie=bead9f007690086c in_seq=0 out_seq=0 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.413+0000 7f4cf7400640 10 client.26609 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.413+0000 7f4cf7400640 1 -- 172.21.15.39:0/1242713022 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b206658d0 con 0x563b202e0da0 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.413+0000 7f4cf7400640 10 client.26609 handle_command_reply: tid=0 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.413+0000 7f4d22d09740 1 client.26609 shutdown 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.413+0000 7f4d22d09740 1 -- 172.21.15.39:0/1242713022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 msgr2=0x563b20705b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.413+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b20705b00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d08051d10 tx=0x7f4d08031400 comp rx=0 tx=0).stop 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4d22d09740 1 -- 172.21.15.39:0/1242713022 shutdown_connections 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563b202e0da0 0x563b2077a640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563b2034d070 0x563b207016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b2032c390 0x563b20700fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4d22d09740 1 --2- 172.21.15.39:0/1242713022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563b203241b0 0x563b20705b00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4d22d09740 1 -- 172.21.15.39:0/1242713022 >> 172.21.15.39:0/1242713022 conn(0x563b20341460 msgr2=0x563b206f27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:49.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4cf7400640 0 client.26609 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:19:49.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4d22d09740 1 -- 172.21.15.39:0/1242713022 shutdown_connections 2024-07-04T08:19:49.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4d22d09740 1 -- 172.21.15.39:0/1242713022 wait complete. 2024-07-04T08:19:49.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.414+0000 7f4d22d09740 20 client.26609 trim_cache size 0 max 16384 2024-07-04T08:19:49.415 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:19:49.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 -- 172.21.15.39:0/882766911 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4cfc0b29e0 msgr2=0x7f4cfc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4cfc0b29e0 0x7f4cfc0b4ef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d080096c0 tx=0x7f4d08005a20 comp rx=0 tx=0).stop 2024-07-04T08:19:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 -- 172.21.15.39:0/882766911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d1410f040 msgr2=0x7f4d141a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d1410f040 0x7f4d141a05e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d1406bf70 tx=0x7f4d0400ee50 comp rx=0 tx=0).stop 2024-07-04T08:19:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 -- 172.21.15.39:0/882766911 shutdown_connections 2024-07-04T08:19:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4cfc0b29e0 0x7f4cfc0b4ef0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d1410f990 0x7f4d141a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d1410f040 0x7f4d141a05e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 --2- 172.21.15.39:0/882766911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d14109700 0x7f4d141a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:49.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 -- 172.21.15.39:0/882766911 >> 172.21.15.39:0/882766911 conn(0x7f4d14077630 msgr2=0x7f4d14107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.415+0000 7f4d1c600640 1 -- 172.21.15.39:0/882766911 shutdown_connections 2024-07-04T08:19:49.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:49.416+0000 7f4d1c600640 1 -- 172.21.15.39:0/882766911 wait complete. 2024-07-04T08:19:50.356 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:19:50.852 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:51.230 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:50 smithi039 ceph-mon[32062]: pgmap v619: 129 pgs: 129 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-07-04T08:19:51.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:50 smithi133 ceph-mon[37050]: pgmap v619: 129 pgs: 129 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-07-04T08:19:51.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:50 smithi050 ceph-mon[36577]: pgmap v619: 129 pgs: 129 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-07-04T08:19:53.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.150+0000 7f0166000640 1 -- 172.21.15.39:0/1266851544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 msgr2=0x7f0160105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:53.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.150+0000 7f0166000640 1 --2- 172.21.15.39:0/1266851544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 0x7f0160105520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0154009900 tx=0x7f015402a410 comp rx=0 tx=0).stop 2024-07-04T08:19:53.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.150+0000 7f0166000640 1 -- 172.21.15.39:0/1266851544 shutdown_connections 2024-07-04T08:19:53.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.150+0000 7f0166000640 1 --2- 172.21.15.39:0/1266851544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f016010b3a0 0x7f0160111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.150+0000 7f0166000640 1 --2- 172.21.15.39:0/1266851544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f016010aa50 0x7f016010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.150+0000 7f0166000640 1 --2- 172.21.15.39:0/1266851544 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 0x7f0160105520 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.150+0000 7f0166000640 1 -- 172.21.15.39:0/1266851544 >> 172.21.15.39:0/1266851544 conn(0x7f0160100b80 msgr2=0x7f0160102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:53.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.150+0000 7f0166000640 1 -- 172.21.15.39:0/1266851544 shutdown_connections 2024-07-04T08:19:53.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.150+0000 7f0166000640 1 -- 172.21.15.39:0/1266851544 wait complete. 2024-07-04T08:19:53.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.151+0000 7f0166000640 1 Processor -- start 2024-07-04T08:19:53.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.151+0000 7f0166000640 1 -- start start 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.152+0000 7f0166000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 0x7f016019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.152+0000 7f0166000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f016010aa50 0x7f016019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.152+0000 7f0166000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f016010b3a0 0x7f01601a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.152+0000 7f0166000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f01601150f0 con 0x7f0160105110 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.152+0000 7f0166000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0160114f70 con 0x7f016010b3a0 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.152+0000 7f0166000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0160115270 con 0x7f016010aa50 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.152+0000 7f0164c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 0x7f016019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.152+0000 7f0164c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 0x7f016019bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52460/0 (socket says 172.21.15.39:52460) 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.152+0000 7f0164c00640 1 -- 172.21.15.39:0/3267057276 learned_addr learned my addr 172.21.15.39:0/3267057276 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.153+0000 7f015fe00640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f016010aa50 0x7f016019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.153+0000 7f0165600640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f016010b3a0 0x7f01601a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.153+0000 7f0164c00640 1 -- 172.21.15.39:0/3267057276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f016010aa50 msgr2=0x7f016019c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:53.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.153+0000 7f0164c00640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f016010aa50 0x7f016019c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.153+0000 7f0164c00640 1 -- 172.21.15.39:0/3267057276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f016010b3a0 msgr2=0x7f01601a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:53.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.153+0000 7f0164c00640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f016010b3a0 0x7f01601a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.153+0000 7f0164c00640 1 -- 172.21.15.39:0/3267057276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01601a1ab0 con 0x7f0160105110 2024-07-04T08:19:53.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.153+0000 7f0164c00640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 0x7f016019bb40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0154005ab0 tx=0x7f0154003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:53.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.154+0000 7f015d600640 1 -- 172.21.15.39:0/3267057276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015403f070 con 0x7f0160105110 2024-07-04T08:19:53.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.154+0000 7f015d600640 1 -- 172.21.15.39:0/3267057276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0154031d60 con 0x7f0160105110 2024-07-04T08:19:53.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.154+0000 7f0166000640 1 -- 172.21.15.39:0/3267057276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01601a1d40 con 0x7f0160105110 2024-07-04T08:19:53.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.154+0000 7f015d600640 1 -- 172.21.15.39:0/3267057276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154035030 con 0x7f0160105110 2024-07-04T08:19:53.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.154+0000 7f0166000640 1 -- 172.21.15.39:0/3267057276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01601a8500 con 0x7f0160105110 2024-07-04T08:19:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.157+0000 7f016dbcc740 1 Processor -- start 2024-07-04T08:19:53.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.158+0000 7f016dbcc740 1 -- start start 2024-07-04T08:19:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.158+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6c9f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.158+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 0x55b4d6c9feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.158+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6ca03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.158+0000 7f016dbcc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b4d6c15f00 con 0x55b4d68fb070 2024-07-04T08:19:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.158+0000 7f016dbcc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b4d6c054a0 con 0x55b4d68d21b0 2024-07-04T08:19:53.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.158+0000 7f016dbcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b4d6c094e0 con 0x55b4d68da390 2024-07-04T08:19:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.158+0000 7f015d600640 1 -- 172.21.15.39:0/3267057276 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0154042070 con 0x7f0160105110 2024-07-04T08:19:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.158+0000 7f0165600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6c9f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.159+0000 7f0165600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6c9f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52474/0 (socket says 172.21.15.39:52474) 2024-07-04T08:19:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.159+0000 7f0165600640 1 -- 172.21.15.39:0/1617414885 learned_addr learned my addr 172.21.15.39:0/1617414885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.159+0000 7f015fe00640 1 --2- 172.21.15.39:0/1617414885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6ca03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.159+0000 7f0164c00640 1 --2- 172.21.15.39:0/1617414885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 0x55b4d6c9feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.159+0000 7f0165600640 1 -- 172.21.15.39:0/1617414885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 msgr2=0x55b4d6ca03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.159+0000 7f0165600640 1 --2- 172.21.15.39:0/1617414885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6ca03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.159+0000 7f0165600640 1 -- 172.21.15.39:0/1617414885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 msgr2=0x55b4d6c9feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.159+0000 7f0165600640 1 --2- 172.21.15.39:0/1617414885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 0x55b4d6c9feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.159+0000 7f0165600640 1 -- 172.21.15.39:0/1617414885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d6b9d670 con 0x55b4d68fb070 2024-07-04T08:19:53.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.160+0000 7f0165600640 1 --2- 172.21.15.39:0/1617414885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6c9f970 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01580039f0 tx=0x7f0158003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b9c09973440a23 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.160+0000 7f0152000640 1 -- 172.21.15.39:0/1617414885 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015801a470 con 0x55b4d68fb070 2024-07-04T08:19:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.160+0000 7f0152000640 1 -- 172.21.15.39:0/1617414885 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0158015070 con 0x55b4d68fb070 2024-07-04T08:19:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.160+0000 7f0152000640 1 -- 172.21.15.39:0/1617414885 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01580107e0 con 0x55b4d68fb070 2024-07-04T08:19:53.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f015d600640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01480b29e0 0x7f01480b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f015d600640 1 -- 172.21.15.39:0/3267057276 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0154102d70 con 0x7f0160105110 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f0164c00640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01480b29e0 0x7f01480b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f016dbcc740 1 -- 172.21.15.39:0/1617414885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 msgr2=0x55b4d6c9f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f016dbcc740 1 --2- 172.21.15.39:0/1617414885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6c9f970 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01580039f0 tx=0x7f0158003eb0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f016dbcc740 1 -- 172.21.15.39:0/1617414885 shutdown_connections 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f016dbcc740 1 --2- 172.21.15.39:0/1617414885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6ca03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f016dbcc740 1 --2- 172.21.15.39:0/1617414885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 0x55b4d6c9feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f016dbcc740 1 --2- 172.21.15.39:0/1617414885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6c9f970 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f016dbcc740 1 -- 172.21.15.39:0/1617414885 >> 172.21.15.39:0/1617414885 conn(0x55b4d68ef460 msgr2=0x55b4d6c9d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.161+0000 7f016dbcc740 1 -- 172.21.15.39:0/1617414885 shutdown_connections 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.162+0000 7f016dbcc740 1 -- 172.21.15.39:0/1617414885 wait complete. 2024-07-04T08:19:53.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.162+0000 7f0164c00640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01480b29e0 0x7f01480b4ef0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0154100240 tx=0x7f0154000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.162+0000 7f016dbcc740 1 -- start start 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.163+0000 7f016dbcc740 1 Processor -- start 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.163+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68d21b0 0x55b4d6cab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.163+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6caba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.163+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68fb070 0x55b4d6cb7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.163+0000 7f016dbcc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b4d6a69700 con 0x55b4d68d21b0 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.163+0000 7f016dbcc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b4d6bacbd0 con 0x55b4d68fb070 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.163+0000 7f016dbcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b4d6cae150 con 0x55b4d68da390 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.163+0000 7f016dbcc740 10 client.0 _ll_register_callbacks cb 0x55b4d6bfe590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f0165600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68d21b0 0x55b4d6cab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f0165600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68d21b0 0x55b4d6cab540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52476/0 (socket says 172.21.15.39:52476) 2024-07-04T08:19:53.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f0165600640 1 -- 172.21.15.39:0/30559477 learned_addr learned my addr 172.21.15.39:0/30559477 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:53.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f015fe00640 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6caba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f0165600640 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68fb070 0x55b4d6cb7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f015fe00640 1 -- 172.21.15.39:0/30559477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68fb070 msgr2=0x55b4d6cb7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:53.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f015fe00640 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68fb070 0x55b4d6cb7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f015fe00640 1 -- 172.21.15.39:0/30559477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68d21b0 msgr2=0x55b4d6cab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:53.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f015fe00640 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68d21b0 0x55b4d6cab540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.164+0000 7f015fe00640 1 -- 172.21.15.39:0/30559477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d6966650 con 0x55b4d68da390 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.165+0000 7f015fe00640 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6caba80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f016006ac50 tx=0x7f014c0119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a589ffb77aab84 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.165+0000 7f0146a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.165+0000 7f0146a00640 1 -- 172.21.15.39:0/30559477 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c013670 con 0x55b4d68da390 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.165+0000 7f0146a00640 1 -- 172.21.15.39:0/30559477 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f014c013810 con 0x55b4d68da390 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.165+0000 7f016dbcc740 1 -- 172.21.15.39:0/30559477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0158015070 con 0x55b4d68da390 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.165+0000 7f0146a00640 1 -- 172.21.15.39:0/30559477 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f014c026050 con 0x55b4d68da390 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.165+0000 7f016dbcc740 10 client.26530 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.166+0000 7f016dbcc740 1 -- 172.21.15.39:0/30559477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4d6cba170 con 0x55b4d68da390 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.166+0000 7f0146a00640 1 -- 172.21.15.39:0/30559477 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0158015070 con 0x55b4d68da390 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.166+0000 7f0146a00640 1 -- 172.21.15.39:0/30559477 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f014c014b80 con 0x55b4d68da390 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.166+0000 7f016dbcc740 10 client.26530 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:53.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.166+0000 7f016dbcc740 10 client.26530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:19:53.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.166+0000 7f016dbcc740 20 client.26530 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:53.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.166+0000 7f016dbcc740 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4d68b99f0 0x55b4d6cbd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.166+0000 7f016dbcc740 4 client.26530 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:53.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.166+0000 7f016dbcc740 1 -- 172.21.15.39:0/30559477 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4d6c17e50 con 0x55b4d68b99f0 2024-07-04T08:19:53.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.167+0000 7f0165600640 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4d68b99f0 0x55b4d6cbd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.167+0000 7f0165600640 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4d68b99f0 0x55b4d6cbd6d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4b5ca900337a84d server_cookie=7f505d95ed209a32 in_seq=0 out_seq=0 2024-07-04T08:19:53.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.167+0000 7f0146a00640 10 client.26530 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:19:53.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f0146a00640 1 -- 172.21.15.39:0/30559477 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b4d6c17e50 con 0x55b4d68b99f0 2024-07-04T08:19:53.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f0146a00640 10 client.26530 handle_command_reply: tid=0 2024-07-04T08:19:53.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f016dbcc740 1 client.26530 shutdown 2024-07-04T08:19:53.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f016dbcc740 1 -- 172.21.15.39:0/30559477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 msgr2=0x55b4d6caba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:53.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f016dbcc740 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6caba80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f016006ac50 tx=0x7f014c0119d0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f016dbcc740 1 -- 172.21.15.39:0/30559477 shutdown_connections 2024-07-04T08:19:53.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f016dbcc740 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4d68b99f0 0x55b4d6cbd6d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f016dbcc740 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68fb070 0x55b4d6cb7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f016dbcc740 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6caba80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f016dbcc740 1 --2- 172.21.15.39:0/30559477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68d21b0 0x55b4d6cab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f0146a00640 0 client.26530 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:19:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.168+0000 7f016dbcc740 1 -- 172.21.15.39:0/30559477 >> 172.21.15.39:0/30559477 conn(0x55b4d68ef460 msgr2=0x55b4d6ca2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.169+0000 7f016dbcc740 1 -- 172.21.15.39:0/30559477 shutdown_connections 2024-07-04T08:19:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.169+0000 7f016dbcc740 1 -- 172.21.15.39:0/30559477 wait complete. 2024-07-04T08:19:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.169+0000 7f016dbcc740 20 client.26530 trim_cache size 0 max 16384 2024-07-04T08:19:53.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.184+0000 7f016dbcc740 1 Processor -- start 2024-07-04T08:19:53.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.184+0000 7f016dbcc740 1 -- start start 2024-07-04T08:19:53.182 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:52 smithi039 ceph-mon[32062]: pgmap v620: 129 pgs: 129 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-07-04T08:19:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68d21b0 0x55b4d6ca10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68da390 0x55b4d6cb7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6cb7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f016dbcc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b2370 con 0x55b4d68fb070 2024-07-04T08:19:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f016dbcc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b3eb0 con 0x55b4d68da390 2024-07-04T08:19:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f016dbcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b2a40 con 0x55b4d68d21b0 2024-07-04T08:19:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f015fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68da390 0x55b4d6cb7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f0165600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6cb7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f015fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68da390 0x55b4d6cb7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35438/0 (socket says 172.21.15.39:35438) 2024-07-04T08:19:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f015fe00640 1 -- 172.21.15.39:0/1378314116 learned_addr learned my addr 172.21.15.39:0/1378314116 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.185+0000 7f0165600640 1 --2- 172.21.15.39:0/1378314116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68d21b0 0x55b4d6ca10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.186+0000 7f015fe00640 1 -- 172.21.15.39:0/1378314116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68d21b0 msgr2=0x55b4d6ca10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.186+0000 7f015fe00640 1 --2- 172.21.15.39:0/1378314116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68d21b0 0x55b4d6ca10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.186+0000 7f015fe00640 1 -- 172.21.15.39:0/1378314116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 msgr2=0x55b4d6cb7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.186+0000 7f015fe00640 1 --2- 172.21.15.39:0/1378314116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6cb7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.186+0000 7f015fe00640 1 -- 172.21.15.39:0/1378314116 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4d6ca21c0 con 0x55b4d68da390 2024-07-04T08:19:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.186+0000 7f015fe00640 1 --2- 172.21.15.39:0/1378314116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68da390 0x55b4d6cb7230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f014c000c00 tx=0x7f014c00afd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4fd5503b3bf0a1f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.187+0000 7f0151600640 1 -- 172.21.15.39:0/1378314116 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c02f070 con 0x55b4d68da390 2024-07-04T08:19:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.187+0000 7f0151600640 1 -- 172.21.15.39:0/1378314116 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f014c013d20 con 0x55b4d68da390 2024-07-04T08:19:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.187+0000 7f0151600640 1 -- 172.21.15.39:0/1378314116 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c014440 con 0x55b4d68da390 2024-07-04T08:19:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.188+0000 7f016dbcc740 1 -- 172.21.15.39:0/1378314116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68da390 msgr2=0x55b4d6cb7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.188+0000 7f016dbcc740 1 --2- 172.21.15.39:0/1378314116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68da390 0x55b4d6cb7230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f014c000c00 tx=0x7f014c00afd0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.188+0000 7f016dbcc740 1 -- 172.21.15.39:0/1378314116 shutdown_connections 2024-07-04T08:19:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.188+0000 7f016dbcc740 1 --2- 172.21.15.39:0/1378314116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6cb7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.188+0000 7f016dbcc740 1 --2- 172.21.15.39:0/1378314116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68da390 0x55b4d6cb7230 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.188+0000 7f016dbcc740 1 --2- 172.21.15.39:0/1378314116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68d21b0 0x55b4d6ca10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.188+0000 7f016dbcc740 1 -- 172.21.15.39:0/1378314116 >> 172.21.15.39:0/1378314116 conn(0x55b4d68ef460 msgr2=0x55b4d6ca09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:53.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.188+0000 7f016dbcc740 1 -- 172.21.15.39:0/1378314116 shutdown_connections 2024-07-04T08:19:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.188+0000 7f016dbcc740 1 -- 172.21.15.39:0/1378314116 wait complete. 2024-07-04T08:19:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.189+0000 7f016dbcc740 1 -- start start 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.189+0000 7f016dbcc740 1 Processor -- start 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.189+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 0x55b4d6cb3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.189+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6caef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.190+0000 7f016dbcc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6caf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.190+0000 7f016dbcc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b37e0 con 0x55b4d68fb070 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.190+0000 7f016dbcc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b3110 con 0x55b4d68d21b0 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.190+0000 7f016dbcc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f01601b0350 con 0x55b4d68da390 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.190+0000 7f016dbcc740 10 client.0 _ll_register_callbacks cb 0x7f014c0094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.190+0000 7f0165600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 0x55b4d6cb3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.190+0000 7f0165600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 0x55b4d6cb3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35440/0 (socket says 172.21.15.39:35440) 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.190+0000 7f0165600640 1 -- 172.21.15.39:0/798998290 learned_addr learned my addr 172.21.15.39:0/798998290 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.190+0000 7f0165600640 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6caf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.191+0000 7f0165600640 1 -- 172.21.15.39:0/798998290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 msgr2=0x55b4d6caef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:53.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.191+0000 7f0165600640 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6caef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.191+0000 7f0165600640 1 -- 172.21.15.39:0/798998290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 msgr2=0x55b4d6cb3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.191+0000 7f0165600640 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 0x55b4d6cb3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.191+0000 7f0165600640 1 -- 172.21.15.39:0/798998290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4d6cafbf0 con 0x55b4d68fb070 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.191+0000 7f0165600640 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6caf460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f01580039f0 tx=0x7f0158011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b406bbfdfc281746 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.191+0000 7f0146a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.191+0000 7f0146a00640 1 -- 172.21.15.39:0/798998290 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158011a80 con 0x55b4d68fb070 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.192+0000 7f0146a00640 1 -- 172.21.15.39:0/798998290 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f015801a470 con 0x55b4d68fb070 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.192+0000 7f016dbcc740 1 -- 172.21.15.39:0/798998290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f014c013d20 con 0x55b4d68fb070 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.192+0000 7f0146a00640 1 -- 172.21.15.39:0/798998290 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f015801a690 con 0x55b4d68fb070 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.192+0000 7f016dbcc740 10 client.16491 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.192+0000 7f016dbcc740 1 -- 172.21.15.39:0/798998290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4d6d25df0 con 0x55b4d68fb070 2024-07-04T08:19:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.192+0000 7f0146a00640 1 -- 172.21.15.39:0/798998290 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f014c013d20 con 0x55b4d68fb070 2024-07-04T08:19:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.193+0000 7f0146a00640 1 -- 172.21.15.39:0/798998290 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0158032050 con 0x55b4d68fb070 2024-07-04T08:19:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.193+0000 7f016dbcc740 10 client.16491 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.193+0000 7f016dbcc740 10 client.16491 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:19:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.193+0000 7f016dbcc740 20 client.16491 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.193+0000 7f016dbcc740 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4d688eda0 0x55b4d6d282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:53.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.193+0000 7f016dbcc740 4 client.16491 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:19:53.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.193+0000 7f016dbcc740 1 -- 172.21.15.39:0/798998290 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4d6c138f0 con 0x55b4d688eda0 2024-07-04T08:19:53.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.193+0000 7f015fe00640 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4d688eda0 0x55b4d6d282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.194+0000 7f015fe00640 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4d688eda0 0x55b4d6d282b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca3dc22f903bd02b server_cookie=a23843442eb1f13e in_seq=0 out_seq=0 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.194+0000 7f0146a00640 10 client.16491 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.194+0000 7f0146a00640 1 -- 172.21.15.39:0/798998290 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4d6c138f0 con 0x55b4d688eda0 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.194+0000 7f0146a00640 10 client.16491 handle_command_reply: tid=0 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.194+0000 7f016dbcc740 1 client.16491 shutdown 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 -- 172.21.15.39:0/798998290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 msgr2=0x55b4d6caf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6caf460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f01580039f0 tx=0x7f0158011f90 comp rx=0 tx=0).stop 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 -- 172.21.15.39:0/798998290 shutdown_connections 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4d688eda0 0x55b4d6d282b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4d68fb070 0x55b4d6caf460 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4d68da390 0x55b4d6caef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 --2- 172.21.15.39:0/798998290 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4d68d21b0 0x55b4d6cb3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 -- 172.21.15.39:0/798998290 >> 172.21.15.39:0/798998290 conn(0x55b4d68ef460 msgr2=0x55b4d6ca09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f0146a00640 0 client.16491 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:19:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 -- 172.21.15.39:0/798998290 shutdown_connections 2024-07-04T08:19:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 1 -- 172.21.15.39:0/798998290 wait complete. 2024-07-04T08:19:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.195+0000 7f016dbcc740 20 client.16491 trim_cache size 0 max 16384 2024-07-04T08:19:53.194 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:19:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 -- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01480b29e0 msgr2=0x7f01480b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01480b29e0 0x7f01480b4ef0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0154100240 tx=0x7f0154000bc0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 -- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 msgr2=0x7f016019bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 0x7f016019bb40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0154005ab0 tx=0x7f0154003cd0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 -- 172.21.15.39:0/3267057276 shutdown_connections 2024-07-04T08:19:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01480b29e0 0x7f01480b4ef0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f016010b3a0 0x7f01601a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f016010aa50 0x7f016019c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 --2- 172.21.15.39:0/3267057276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0160105110 0x7f016019bb40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 -- 172.21.15.39:0/3267057276 >> 172.21.15.39:0/3267057276 conn(0x7f0160100b80 msgr2=0x7f0160102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 -- 172.21.15.39:0/3267057276 shutdown_connections 2024-07-04T08:19:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:53.196+0000 7f0166000640 1 -- 172.21.15.39:0/3267057276 wait complete. 2024-07-04T08:19:53.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:52 smithi133 ceph-mon[37050]: pgmap v620: 129 pgs: 129 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-07-04T08:19:53.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:52 smithi050 ceph-mon[36577]: pgmap v620: 129 pgs: 129 active+clean; 4.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-07-04T08:19:53.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:19:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:55.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:54 smithi133 ceph-mon[37050]: pgmap v621: 129 pgs: 129 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-07-04T08:19:55.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:19:55.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:19:55.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:19:55.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:54 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:19:55.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:54 smithi039 ceph-mon[32062]: pgmap v621: 129 pgs: 129 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-07-04T08:19:55.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:19:55.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:19:55.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:19:55.349 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:54 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:19:55.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:54 smithi050 ceph-mon[36577]: pgmap v621: 129 pgs: 129 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-07-04T08:19:55.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:19:55.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:19:55.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:19:55.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:54 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:19:56.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.671+0000 7f98a3e00640 1 -- 172.21.15.39:0/4175598161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a410f7a0 msgr2=0x7f98a4115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:56.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.671+0000 7f98a3e00640 1 --2- 172.21.15.39:0/4175598161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a410f7a0 0x7f98a4115fd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f989c009ca0 tx=0x7f989c02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.671+0000 7f98a3e00640 1 -- 172.21.15.39:0/4175598161 shutdown_connections 2024-07-04T08:19:56.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.671+0000 7f98a3e00640 1 --2- 172.21.15.39:0/4175598161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98a410ee50 0x7f98a410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.671+0000 7f98a3e00640 1 --2- 172.21.15.39:0/4175598161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a410f7a0 0x7f98a4115fd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.671+0000 7f98a3e00640 1 --2- 172.21.15.39:0/4175598161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98a4109510 0x7f98a4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.671+0000 7f98a3e00640 1 -- 172.21.15.39:0/4175598161 >> 172.21.15.39:0/4175598161 conn(0x7f98a4078b10 msgr2=0x7f98a4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:56.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.671+0000 7f98a3e00640 1 -- 172.21.15.39:0/4175598161 shutdown_connections 2024-07-04T08:19:56.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.672+0000 7f98a3e00640 1 -- 172.21.15.39:0/4175598161 wait complete. 2024-07-04T08:19:56.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.673+0000 7f98a3e00640 1 Processor -- start 2024-07-04T08:19:56.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.673+0000 7f98a3e00640 1 -- start start 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.673+0000 7f98a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98a4109510 0x7f98a419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.673+0000 7f98a3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a410ee50 0x7f98a41a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.673+0000 7f98a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98a410f7a0 0x7f98a41a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.673+0000 7f98a3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4119450 con 0x7f98a4109510 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.673+0000 7f98a3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41192d0 con 0x7f98a410ee50 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.673+0000 7f98a3e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41195d0 con 0x7f98a410f7a0 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f98a2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98a4109510 0x7f98a419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f98a2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98a4109510 0x7f98a419fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52510/0 (socket says 172.21.15.39:52510) 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f98a2a00640 1 -- 172.21.15.39:0/3146736559 learned_addr learned my addr 172.21.15.39:0/3146736559 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f989be00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a410ee50 0x7f98a41a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f98a3400640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98a410f7a0 0x7f98a41a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f98a2a00640 1 -- 172.21.15.39:0/3146736559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98a410f7a0 msgr2=0x7f98a41a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:56.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f98a2a00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98a410f7a0 0x7f98a41a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f98a2a00640 1 -- 172.21.15.39:0/3146736559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a410ee50 msgr2=0x7f98a41a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f98a2a00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a410ee50 0x7f98a41a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f98a2a00640 1 -- 172.21.15.39:0/3146736559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a41a5d90 con 0x7f98a4109510 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.674+0000 7f989be00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a410ee50 0x7f98a41a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.675+0000 7f98a2a00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98a4109510 0x7f98a419fe20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9f0 tx=0x7f989400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.675+0000 7f989b400640 1 -- 172.21.15.39:0/3146736559 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989400cd80 con 0x7f98a4109510 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.675+0000 7f98a3e00640 1 -- 172.21.15.39:0/3146736559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a41a6080 con 0x7f98a4109510 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.675+0000 7f989b400640 1 -- 172.21.15.39:0/3146736559 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98940108e0 con 0x7f98a4109510 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.675+0000 7f989b400640 1 -- 172.21.15.39:0/3146736559 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894018980 con 0x7f98a4109510 2024-07-04T08:19:56.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.675+0000 7f98a3e00640 1 -- 172.21.15.39:0/3146736559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41ac7e0 con 0x7f98a4109510 2024-07-04T08:19:56.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.679+0000 7f98afe4e740 1 Processor -- start 2024-07-04T08:19:56.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.679+0000 7f98afe4e740 1 -- start start 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29709b070 0x55d29743f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 0x55d29743feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98afe4e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d2973b5f00 con 0x55d2970721b0 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98afe4e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d2973a54a0 con 0x55d29709b070 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98afe4e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d2973a94e0 con 0x55d29707a390 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f989b400640 1 -- 172.21.15.39:0/3146736559 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9894010430 con 0x7f98a4109510 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98a3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29709b070 0x55d29743f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98a3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29709b070 0x55d29743f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35486/0 (socket says 172.21.15.39:35486) 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98a3400640 1 -- 172.21.15.39:0/2297428655 learned_addr learned my addr 172.21.15.39:0/2297428655 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:56.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f989be00640 1 --2- 172.21.15.39:0/2297428655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98a2a00640 1 --2- 172.21.15.39:0/2297428655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 0x55d29743feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98a3400640 1 -- 172.21.15.39:0/2297428655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 msgr2=0x55d2974403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:56.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98a3400640 1 --2- 172.21.15.39:0/2297428655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98a3400640 1 -- 172.21.15.39:0/2297428655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 msgr2=0x55d29743feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:56.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.680+0000 7f98a3400640 1 --2- 172.21.15.39:0/2297428655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 0x55d29743feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.681+0000 7f98a3400640 1 -- 172.21.15.39:0/2297428655 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d29733d670 con 0x55d29709b070 2024-07-04T08:19:56.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.681+0000 7f98a3400640 1 --2- 172.21.15.39:0/2297428655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29709b070 0x55d29743f970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f989c02f410 tx=0x7f989c02f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae1c7e56e8e5c9ef server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:56.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.681+0000 7f9893e00640 1 -- 172.21.15.39:0/2297428655 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c031cf0 con 0x55d29709b070 2024-07-04T08:19:56.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.681+0000 7f9893e00640 1 -- 172.21.15.39:0/2297428655 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f989c036030 con 0x55d29709b070 2024-07-04T08:19:56.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.681+0000 7f9893e00640 1 -- 172.21.15.39:0/2297428655 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c0042a0 con 0x55d29709b070 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.682+0000 7f989b400640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f98840b2960 0x7f98840b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.682+0000 7f98a2a00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f98840b2960 0x7f98840b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.682+0000 7f989b400640 1 -- 172.21.15.39:0/3146736559 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f98940e2cd0 con 0x7f98a4109510 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.682+0000 7f98afe4e740 1 -- 172.21.15.39:0/2297428655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29709b070 msgr2=0x55d29743f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.682+0000 7f98afe4e740 1 --2- 172.21.15.39:0/2297428655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29709b070 0x55d29743f970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f989c02f410 tx=0x7f989c02f440 comp rx=0 tx=0).stop 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.683+0000 7f98afe4e740 1 -- 172.21.15.39:0/2297428655 shutdown_connections 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.683+0000 7f98afe4e740 1 --2- 172.21.15.39:0/2297428655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 0x55d29743feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.683+0000 7f98afe4e740 1 --2- 172.21.15.39:0/2297428655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.683+0000 7f98afe4e740 1 --2- 172.21.15.39:0/2297428655 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29709b070 0x55d29743f970 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.683+0000 7f98afe4e740 1 -- 172.21.15.39:0/2297428655 >> 172.21.15.39:0/2297428655 conn(0x55d29708f460 msgr2=0x55d29743d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.683+0000 7f98afe4e740 1 -- 172.21.15.39:0/2297428655 shutdown_connections 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.683+0000 7f98a2a00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f98840b2960 0x7f98840b4e70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55d29743e5f0 tx=0x7f9894000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:56.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.683+0000 7f98afe4e740 1 -- 172.21.15.39:0/2297428655 wait complete. 2024-07-04T08:19:56.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.683+0000 7f98afe4e740 1 -- start start 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.684+0000 7f98afe4e740 1 Processor -- start 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.684+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 0x55d29744b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.684+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.684+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29709b070 0x55d297457260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.684+0000 7f98afe4e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d297209700 con 0x55d2970721b0 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.684+0000 7f98afe4e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d29734cbd0 con 0x55d29707a390 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.684+0000 7f98afe4e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d29744df70 con 0x55d29709b070 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f98afe4e740 10 client.0 _ll_register_callbacks cb 0x55d29739e590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f98a3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 0x55d29744b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f989be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f989be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35502/0 (socket says 172.21.15.39:35502) 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f989be00640 1 -- 172.21.15.39:0/3896739794 learned_addr learned my addr 172.21.15.39:0/3896739794 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f98a3400640 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29709b070 0x55d297457260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f989be00640 1 -- 172.21.15.39:0/3896739794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29709b070 msgr2=0x55d297457260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f989be00640 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29709b070 0x55d297457260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f989be00640 1 -- 172.21.15.39:0/3896739794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 msgr2=0x55d29744b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f989be00640 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 0x55d29744b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f989be00640 1 -- 172.21.15.39:0/3896739794 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d297106650 con 0x55d29707a390 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.685+0000 7f98a3400640 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 0x55d29744b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.686+0000 7f989be00640 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ba70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f988c012d20 tx=0x7f988c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78c64ab5e18e7ebd server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.686+0000 7f9890c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:19:56.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.686+0000 7f9890c00640 1 -- 172.21.15.39:0/3896739794 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c012110 con 0x55d29707a390 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.686+0000 7f9890c00640 1 -- 172.21.15.39:0/3896739794 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f988c004030 con 0x55d29707a390 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.686+0000 7f98afe4e740 1 -- 172.21.15.39:0/3896739794 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f989c036030 con 0x55d29707a390 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.686+0000 7f9890c00640 1 -- 172.21.15.39:0/3896739794 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f988c025050 con 0x55d29707a390 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.686+0000 7f98afe4e740 10 client.26639 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.687+0000 7f98afe4e740 1 -- 172.21.15.39:0/3896739794 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d297459fe0 con 0x55d29707a390 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.687+0000 7f9890c00640 1 -- 172.21.15.39:0/3896739794 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f989c036030 con 0x55d29707a390 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.687+0000 7f9890c00640 1 -- 172.21.15.39:0/3896739794 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f988c003bf0 con 0x55d29707a390 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.687+0000 7f98afe4e740 10 client.26639 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.687+0000 7f98afe4e740 10 client.26639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.687+0000 7f98afe4e740 20 client.26639 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.687+0000 7f98afe4e740 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d2970599f0 0x55d29745d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.687+0000 7f98afe4e740 4 client.26639 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.687+0000 7f98afe4e740 1 -- 172.21.15.39:0/3896739794 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2973b7e50 con 0x55d2970599f0 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.688+0000 7f98a3400640 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d2970599f0 0x55d29745d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.688+0000 7f98a3400640 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d2970599f0 0x55d29745d540 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a1dc3a9c7bae995 server_cookie=63c3f9501f3bafb4 in_seq=0 out_seq=0 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.688+0000 7f9890c00640 10 client.26639 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.689+0000 7f9890c00640 1 -- 172.21.15.39:0/3896739794 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d2973b7e50 con 0x55d2970599f0 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.689+0000 7f9890c00640 10 client.26639 handle_command_reply: tid=0 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.689+0000 7f98afe4e740 1 client.26639 shutdown 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.689+0000 7f98afe4e740 1 -- 172.21.15.39:0/3896739794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 msgr2=0x55d29744ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.689+0000 7f98afe4e740 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ba70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f988c012d20 tx=0x7f988c00f790 comp rx=0 tx=0).stop 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f98afe4e740 1 -- 172.21.15.39:0/3896739794 shutdown_connections 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f98afe4e740 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d2970599f0 0x55d29745d540 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f98afe4e740 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29709b070 0x55d297457260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f98afe4e740 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ba70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f98afe4e740 1 --2- 172.21.15.39:0/3896739794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d2970721b0 0x55d29744b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f9890c00640 0 client.26639 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:19:56.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f98afe4e740 1 -- 172.21.15.39:0/3896739794 >> 172.21.15.39:0/3896739794 conn(0x55d29708f460 msgr2=0x55d297442b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:56.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f98afe4e740 1 -- 172.21.15.39:0/3896739794 shutdown_connections 2024-07-04T08:19:56.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f98afe4e740 1 -- 172.21.15.39:0/3896739794 wait complete. 2024-07-04T08:19:56.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.690+0000 7f98afe4e740 20 client.26639 trim_cache size 0 max 16384 2024-07-04T08:19:56.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.706+0000 7f98afe4e740 1 Processor -- start 2024-07-04T08:19:56.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.706+0000 7f98afe4e740 1 -- start start 2024-07-04T08:19:56.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d2970721b0 0x55d297443430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 0x55d29743ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f98afe4e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b6660 con 0x55d29709b070 2024-07-04T08:19:56.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f98afe4e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b81a0 con 0x55d2970721b0 2024-07-04T08:19:56.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f98afe4e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b6d30 con 0x55d29707a390 2024-07-04T08:19:56.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f989be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f989be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60912/0 (socket says 172.21.15.39:60912) 2024-07-04T08:19:56.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f989be00640 1 -- 172.21.15.39:0/4199000297 learned_addr learned my addr 172.21.15.39:0/4199000297 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:56.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f98a3400640 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 0x55d29743ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.707+0000 7f98a3400640 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d2970721b0 0x55d297443430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.708+0000 7f989be00640 1 -- 172.21.15.39:0/4199000297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d2970721b0 msgr2=0x55d297443430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:56.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.708+0000 7f989be00640 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d2970721b0 0x55d297443430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.708+0000 7f989be00640 1 -- 172.21.15.39:0/4199000297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 msgr2=0x55d29743ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:56.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.708+0000 7f989be00640 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 0x55d29743ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.708+0000 7f989be00640 1 -- 172.21.15.39:0/4199000297 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d297441e90 con 0x55d29707a390 2024-07-04T08:19:56.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.708+0000 7f98a3400640 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d2970721b0 0x55d297443430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:19:56.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.708+0000 7f989be00640 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974410b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f988c010480 tx=0x7f988c012320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf505530e8d38498 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:56.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.708+0000 7f9893400640 1 -- 172.21.15.39:0/4199000297 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c02e070 con 0x55d29707a390 2024-07-04T08:19:56.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.709+0000 7f9893400640 1 -- 172.21.15.39:0/4199000297 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f988c00fe80 con 0x55d29707a390 2024-07-04T08:19:56.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.709+0000 7f9893400640 1 -- 172.21.15.39:0/4199000297 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c022040 con 0x55d29707a390 2024-07-04T08:19:56.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.710+0000 7f98afe4e740 1 -- 172.21.15.39:0/4199000297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 msgr2=0x55d2974410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:56.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.710+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974410b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f988c010480 tx=0x7f988c012320 comp rx=0 tx=0).stop 2024-07-04T08:19:56.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.710+0000 7f98afe4e740 1 -- 172.21.15.39:0/4199000297 shutdown_connections 2024-07-04T08:19:56.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.710+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 0x55d29743ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.710+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d29707a390 0x55d2974410b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.710+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4199000297 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d2970721b0 0x55d297443430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.710+0000 7f98afe4e740 1 -- 172.21.15.39:0/4199000297 >> 172.21.15.39:0/4199000297 conn(0x55d29708f460 msgr2=0x55d2974409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:56.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.710+0000 7f98afe4e740 1 -- 172.21.15.39:0/4199000297 shutdown_connections 2024-07-04T08:19:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.710+0000 7f98afe4e740 1 -- 172.21.15.39:0/4199000297 wait complete. 2024-07-04T08:19:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.711+0000 7f98afe4e740 1 -- start start 2024-07-04T08:19:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.711+0000 7f98afe4e740 1 Processor -- start 2024-07-04T08:19:56.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2970721b0 0x55d2974539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f98afe4e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 0x55d29744f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f98afe4e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b7ad0 con 0x55d29709b070 2024-07-04T08:19:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f98afe4e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b7400 con 0x55d29707a390 2024-07-04T08:19:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f98afe4e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b4640 con 0x55d2970721b0 2024-07-04T08:19:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f98afe4e740 10 client.0 _ll_register_callbacks cb 0x7f988c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:19:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f989be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f98a3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2970721b0 0x55d2974539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f989be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35528/0 (socket says 172.21.15.39:35528) 2024-07-04T08:19:56.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f989be00640 1 -- 172.21.15.39:0/4246961129 learned_addr learned my addr 172.21.15.39:0/4246961129 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:19:56.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.712+0000 7f98a3400640 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 0x55d29744f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.713+0000 7f989be00640 1 -- 172.21.15.39:0/4246961129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2970721b0 msgr2=0x55d2974539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:56.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.713+0000 7f989be00640 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2970721b0 0x55d2974539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.713+0000 7f989be00640 1 -- 172.21.15.39:0/4246961129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 msgr2=0x55d29744f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:19:56.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.713+0000 7f989be00640 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 0x55d29744f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.713+0000 7f989be00640 1 -- 172.21.15.39:0/4246961129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d29744fc40 con 0x55d29707a390 2024-07-04T08:19:56.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.713+0000 7f989be00640 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ee60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f988c0059f0 tx=0x7f988c012a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=947384cb065aa77f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:19:56.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.713+0000 7f9890c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:19:56.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.714+0000 7f9890c00640 1 -- 172.21.15.39:0/4246961129 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c02e070 con 0x55d29707a390 2024-07-04T08:19:56.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.714+0000 7f9890c00640 1 -- 172.21.15.39:0/4246961129 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f988c0053e0 con 0x55d29707a390 2024-07-04T08:19:56.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.714+0000 7f98afe4e740 1 -- 172.21.15.39:0/4246961129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d2973b38f0 con 0x55d29707a390 2024-07-04T08:19:56.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.714+0000 7f9890c00640 1 -- 172.21.15.39:0/4246961129 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f988c03c050 con 0x55d29707a390 2024-07-04T08:19:56.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.714+0000 7f98afe4e740 10 client.26645 fetch_fsmap learned FSMap version 30 2024-07-04T08:19:56.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.714+0000 7f98afe4e740 1 -- 172.21.15.39:0/4246961129 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d2974c5d20 con 0x55d29707a390 2024-07-04T08:19:56.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.715+0000 7f9890c00640 1 -- 172.21.15.39:0/4246961129 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d2973b38f0 con 0x55d29707a390 2024-07-04T08:19:56.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.715+0000 7f9890c00640 1 -- 172.21.15.39:0/4246961129 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f988c0039b0 con 0x55d29707a390 2024-07-04T08:19:56.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.715+0000 7f98afe4e740 10 client.26645 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:19:56.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.715+0000 7f98afe4e740 10 client.26645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:19:56.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.715+0000 7f98afe4e740 20 client.26645 populate_metadata read hostname 'smithi039' 2024-07-04T08:19:56.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.715+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d29702eda0 0x55d2974c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:19:56.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.715+0000 7f98afe4e740 4 client.26645 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:19:56.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.715+0000 7f98afe4e740 1 -- 172.21.15.39:0/4246961129 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2973b1030 con 0x55d29702eda0 2024-07-04T08:19:56.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.716+0000 7f98a3400640 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d29702eda0 0x55d2974c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:19:56.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.716+0000 7f98a3400640 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d29702eda0 0x55d2974c8440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eee49f9e0bc7c4da server_cookie=f1fd68cc9fe4baf8 in_seq=0 out_seq=0 2024-07-04T08:19:56.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.716+0000 7f9890c00640 10 client.26645 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:19:56.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f9890c00640 1 -- 172.21.15.39:0/4246961129 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2973b1030 con 0x55d29702eda0 2024-07-04T08:19:56.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f9890c00640 10 client.26645 handle_command_reply: tid=0 2024-07-04T08:19:56.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f98afe4e740 1 client.26645 shutdown 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f98afe4e740 1 -- 172.21.15.39:0/4246961129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 msgr2=0x55d29744ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ee60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f988c0059f0 tx=0x7f988c012a00 comp rx=0 tx=0).stop 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f98afe4e740 1 -- 172.21.15.39:0/4246961129 shutdown_connections 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d29702eda0 0x55d2974c8440 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d29709b070 0x55d29744f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d29707a390 0x55d29744ee60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f98afe4e740 1 --2- 172.21.15.39:0/4246961129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d2970721b0 0x55d2974539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f9890c00640 0 client.26645 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.717+0000 7f98afe4e740 1 -- 172.21.15.39:0/4246961129 >> 172.21.15.39:0/4246961129 conn(0x55d29708f460 msgr2=0x55d297443200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.718+0000 7f98afe4e740 1 -- 172.21.15.39:0/4246961129 shutdown_connections 2024-07-04T08:19:56.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.718+0000 7f98afe4e740 1 -- 172.21.15.39:0/4246961129 wait complete. 2024-07-04T08:19:56.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.718+0000 7f98afe4e740 20 client.26645 trim_cache size 0 max 16384 2024-07-04T08:19:56.718 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:19:56.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 -- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f98840b2960 msgr2=0x7f98840b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:56.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f98840b2960 0x7f98840b4e70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55d29743e5f0 tx=0x7f9894000bc0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 -- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98a4109510 msgr2=0x7f98a419fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:19:56.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98a4109510 0x7f98a419fe20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9f0 tx=0x7f989400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 -- 172.21.15.39:0/3146736559 shutdown_connections 2024-07-04T08:19:56.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f98840b2960 0x7f98840b4e70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f98a410f7a0 0x7f98a41a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98a410ee50 0x7f98a41a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 --2- 172.21.15.39:0/3146736559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98a4109510 0x7f98a419fe20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:19:56.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.719+0000 7f98a3e00640 1 -- 172.21.15.39:0/3146736559 >> 172.21.15.39:0/3146736559 conn(0x7f98a4078b10 msgr2=0x7f98a41078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:19:56.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.720+0000 7f98a3e00640 1 -- 172.21.15.39:0/3146736559 shutdown_connections 2024-07-04T08:19:56.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:19:56.720+0000 7f98a3e00640 1 -- 172.21.15.39:0/3146736559 wait complete. 2024-07-04T08:19:57.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:57 smithi133 ceph-mon[37050]: pgmap v622: 129 pgs: 129 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-07-04T08:19:57.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:57 smithi039 ceph-mon[32062]: pgmap v622: 129 pgs: 129 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-07-04T08:19:57.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:57 smithi050 ceph-mon[36577]: pgmap v622: 129 pgs: 129 active+clean; 4.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-07-04T08:19:57.495 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:19:57.978 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:19:59.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:19:59 smithi133 ceph-mon[37050]: pgmap v623: 129 pgs: 129 active+clean; 4.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 16 op/s 2024-07-04T08:19:59.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:19:59 smithi039 ceph-mon[32062]: pgmap v623: 129 pgs: 129 active+clean; 4.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 16 op/s 2024-07-04T08:19:59.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:19:59 smithi050 ceph-mon[36577]: pgmap v623: 129 pgs: 129 active+clean; 4.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 16 op/s 2024-07-04T08:19:59.483 INFO:tasks.workunit.client.0.smithi039.stdout:64/600: open fd = 3 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:65/600: open fd = 3 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:66/600: open fd = 3 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:67/600: open fd = 3 2024-07-04T08:19:59.484 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:68/600: open fd = 3 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:69/600: open fd = 3 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:70/600: open fd = 3 2024-07-04T08:19:59.485 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.486 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.486 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.486 INFO:tasks.workunit.client.0.smithi039.stdout:71/600: open fd = 3 2024-07-04T08:19:59.486 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.486 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.486 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.486 INFO:tasks.workunit.client.0.smithi039.stdout:72/600: open fd = 3 2024-07-04T08:19:59.486 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.486 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:73/600: open fd = 3 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:74/600: open fd = 3 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:75/600: open fd = 3 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.487 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:76/600: open fd = 3 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:77/600: open fd = 3 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:78/600: open fd = 3 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.488 INFO:tasks.workunit.client.0.smithi039.stdout:79/600: open fd = 3 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:80/600: open fd = 3 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:81/600: open fd = 3 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:82/600: open fd = 3 2024-07-04T08:19:59.489 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:83/600: open fd = 3 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:84/600: open fd = 3 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:85/600: open fd = 3 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.490 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:86/600: open fd = 3 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:87/600: open fd = 3 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:88/600: open fd = 3 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.491 INFO:tasks.workunit.client.0.smithi039.stdout:89/600: open fd = 3 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:90/600: open fd = 3 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:91/600: open fd = 3 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:92/600: open fd = 3 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.492 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:93/600: open fd = 3 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:94/600: open fd = 3 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.493 INFO:tasks.workunit.client.0.smithi039.stdout:95/600: open fd = 3 2024-07-04T08:19:59.494 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.494 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.494 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.494 INFO:tasks.workunit.client.0.smithi039.stdout:96/600: open fd = 3 2024-07-04T08:19:59.494 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.494 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.494 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.494 INFO:tasks.workunit.client.0.smithi039.stdout:97/600: open fd = 3 2024-07-04T08:19:59.494 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:98/600: open fd = 3 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:99/600: open fd = 3 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:100/600: open fd = 3 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:101/600: open fd = 3 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:102/600: open fd = 3 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:103/600: open fd = 3 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:104/600: open fd = 3 2024-07-04T08:19:59.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.497 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.497 INFO:tasks.workunit.client.0.smithi039.stdout:105/600: open fd = 3 2024-07-04T08:19:59.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.497 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.497 INFO:tasks.workunit.client.0.smithi039.stdout:106/600: open fd = 3 2024-07-04T08:19:59.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:107/600: open fd = 3 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:108/600: open fd = 3 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:109/600: open fd = 3 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.498 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:110/600: open fd = 3 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:111/600: open fd = 3 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:112/600: open fd = 3 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:113/600: open fd = 3 2024-07-04T08:19:59.499 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:114/600: open fd = 3 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:115/600: open fd = 3 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:116/600: open fd = 3 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.500 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:117/600: open fd = 3 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:118/600: open fd = 3 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:119/600: open fd = 3 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:120/600: open fd = 3 2024-07-04T08:19:59.501 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:121/600: open fd = 3 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:122/600: open fd = 3 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:123/600: open fd = 3 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.502 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:124/600: open fd = 3 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:125/600: open fd = 3 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:126/600: open fd = 3 2024-07-04T08:19:59.503 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:20:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.306+0000 7f1da7200640 1 -- 172.21.15.39:0/975364767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da8109510 msgr2=0x7f1da8109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.306+0000 7f1da7200640 1 --2- 172.21.15.39:0/975364767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da8109510 0x7f1da8109920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d9802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.306+0000 7f1da7200640 1 -- 172.21.15.39:0/975364767 shutdown_connections 2024-07-04T08:20:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.306+0000 7f1da7200640 1 --2- 172.21.15.39:0/975364767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da810ee50 0x7f1da810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.306+0000 7f1da7200640 1 --2- 172.21.15.39:0/975364767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da810f7a0 0x7f1da8115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.306+0000 7f1da7200640 1 --2- 172.21.15.39:0/975364767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da8109510 0x7f1da8109920 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.306+0000 7f1da7200640 1 -- 172.21.15.39:0/975364767 >> 172.21.15.39:0/975364767 conn(0x7f1da8078b10 msgr2=0x7f1da8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:00.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.306+0000 7f1da7200640 1 -- 172.21.15.39:0/975364767 shutdown_connections 2024-07-04T08:20:00.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.306+0000 7f1da7200640 1 -- 172.21.15.39:0/975364767 wait complete. 2024-07-04T08:20:00.305 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:00 smithi039 ceph-mon[32062]: overall HEALTH_OK 2024-07-04T08:20:00.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.307+0000 7f1da7200640 1 Processor -- start 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.307+0000 7f1da7200640 1 -- start start 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1da7200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da8109510 0x7f1da819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1da7200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da810ee50 0x7f1da81a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1da7200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da810f7a0 0x7f1da81a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1da7200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8119450 con 0x7f1da810f7a0 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1da7200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81192d0 con 0x7f1da8109510 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1da7200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81195d0 con 0x7f1da810ee50 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1d9d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da810ee50 0x7f1da81a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1da6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da810f7a0 0x7f1da81a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1d9d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da810ee50 0x7f1da81a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60936/0 (socket says 172.21.15.39:60936) 2024-07-04T08:20:00.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1d9d400640 1 -- 172.21.15.39:0/2549386696 learned_addr learned my addr 172.21.15.39:0/2549386696 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:00.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1da6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da810f7a0 0x7f1da81a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52576/0 (socket says 172.21.15.39:52576) 2024-07-04T08:20:00.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.308+0000 7f1da5e00640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da8109510 0x7f1da819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.309+0000 7f1d9d400640 1 -- 172.21.15.39:0/2549386696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da8109510 msgr2=0x7f1da819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:00.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.309+0000 7f1d9d400640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da8109510 0x7f1da819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.309+0000 7f1d9d400640 1 -- 172.21.15.39:0/2549386696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da810f7a0 msgr2=0x7f1da81a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:00.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.309+0000 7f1d9d400640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da810f7a0 0x7f1da81a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.309+0000 7f1d9d400640 1 -- 172.21.15.39:0/2549386696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da81a5db0 con 0x7f1da810ee50 2024-07-04T08:20:00.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.309+0000 7f1d9d400640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da810ee50 0x7f1da81a03e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000e9f0 tx=0x7f1d9000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:00.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.310+0000 7f1d9f400640 1 -- 172.21.15.39:0/2549386696 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9000ccc0 con 0x7f1da810ee50 2024-07-04T08:20:00.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.310+0000 7f1da7200640 1 -- 172.21.15.39:0/2549386696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da81a60a0 con 0x7f1da810ee50 2024-07-04T08:20:00.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.310+0000 7f1d9f400640 1 -- 172.21.15.39:0/2549386696 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d900108e0 con 0x7f1da810ee50 2024-07-04T08:20:00.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.310+0000 7f1d9f400640 1 -- 172.21.15.39:0/2549386696 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90018880 con 0x7f1da810ee50 2024-07-04T08:20:00.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.310+0000 7f1da7200640 1 -- 172.21.15.39:0/2549386696 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da81ac800 con 0x7f1da810ee50 2024-07-04T08:20:00.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.313+0000 7f1db3006740 1 Processor -- start 2024-07-04T08:20:00.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.313+0000 7f1db3006740 1 -- start start 2024-07-04T08:20:00.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1db3006740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 0x559e4e78d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1db3006740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e78de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1db3006740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e78e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1db3006740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559e4e703ee0 con 0x559e4e3c8390 2024-07-04T08:20:00.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1db3006740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559e4e6f3480 con 0x559e4e3c01b0 2024-07-04T08:20:00.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1db3006740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559e4e6f74c0 con 0x559e4e3e9070 2024-07-04T08:20:00.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1d9f400640 1 -- 172.21.15.39:0/2549386696 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1d90010430 con 0x7f1da810ee50 2024-07-04T08:20:00.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1da5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e78de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1da5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e78de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35550/0 (socket says 172.21.15.39:35550) 2024-07-04T08:20:00.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1da5e00640 1 -- 172.21.15.39:0/49415842 learned_addr learned my addr 172.21.15.39:0/49415842 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:00.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.314+0000 7f1da6800640 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 0x559e4e78d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.315+0000 7f1d9d400640 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e78e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.315+0000 7f1da6800640 1 -- 172.21.15.39:0/49415842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 msgr2=0x559e4e78de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.315+0000 7f1da6800640 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e78de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.315+0000 7f1da6800640 1 -- 172.21.15.39:0/49415842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 msgr2=0x559e4e78e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.315+0000 7f1da6800640 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e78e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.315+0000 7f1da6800640 1 -- 172.21.15.39:0/49415842 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e4e68b650 con 0x559e4e3e9070 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.315+0000 7f1d9d400640 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e78e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.315+0000 7f1da6800640 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 0x559e4e78d950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009f10 tx=0x7f1da0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a337d68cc0f09eb9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.316+0000 7f1d97400640 1 -- 172.21.15.39:0/49415842 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0004d20 con 0x559e4e3e9070 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.316+0000 7f1d97400640 1 -- 172.21.15.39:0/49415842 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da0004030 con 0x559e4e3e9070 2024-07-04T08:20:00.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.316+0000 7f1d97400640 1 -- 172.21.15.39:0/49415842 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0005220 con 0x559e4e3e9070 2024-07-04T08:20:00.315 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:00 smithi133 ceph-mon[37050]: overall HEALTH_OK 2024-07-04T08:20:00.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.316+0000 7f1d9f400640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d8c0b2960 0x7f1d8c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1d9f400640 1 -- 172.21.15.39:0/2549386696 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1d900e2d10 con 0x7f1da810ee50 2024-07-04T08:20:00.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1da5e00640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d8c0b2960 0x7f1d8c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1db3006740 1 -- 172.21.15.39:0/49415842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 msgr2=0x559e4e78d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1db3006740 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 0x559e4e78d950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009f10 tx=0x7f1da0011810 comp rx=0 tx=0).stop 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1db3006740 1 -- 172.21.15.39:0/49415842 shutdown_connections 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1db3006740 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e78e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1db3006740 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e78de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1db3006740 1 --2- 172.21.15.39:0/49415842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 0x559e4e78d950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1db3006740 1 -- 172.21.15.39:0/49415842 >> 172.21.15.39:0/49415842 conn(0x559e4e3dd460 msgr2=0x559e4e78b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1db3006740 1 -- 172.21.15.39:0/49415842 shutdown_connections 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.317+0000 7f1db3006740 1 -- 172.21.15.39:0/49415842 wait complete. 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.318+0000 7f1da5e00640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d8c0b2960 0x7f1d8c0b4e70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005f70 tx=0x7f1d98009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.318+0000 7f1db3006740 1 -- start start 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.318+0000 7f1db3006740 1 Processor -- start 2024-07-04T08:20:00.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1db3006740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e799460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1db3006740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e7999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1db3006740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 0x559e4e7a5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1db3006740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559e4e557700 con 0x559e4e3c8390 2024-07-04T08:20:00.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1db3006740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559e4e69abb0 con 0x559e4e3c01b0 2024-07-04T08:20:00.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1db3006740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559e4e79c130 con 0x559e4e3e9070 2024-07-04T08:20:00.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1db3006740 10 client.0 _ll_register_callbacks cb 0x559e4e6ec570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:00.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1da5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e799460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1d9d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e7999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1da5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e799460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35552/0 (socket says 172.21.15.39:35552) 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1da5e00640 1 -- 172.21.15.39:0/1465264445 learned_addr learned my addr 172.21.15.39:0/1465264445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.319+0000 7f1da6800640 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 0x559e4e7a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1d9d400640 1 -- 172.21.15.39:0/1465264445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 msgr2=0x559e4e7a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1d9d400640 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 0x559e4e7a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1d9d400640 1 -- 172.21.15.39:0/1465264445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 msgr2=0x559e4e799460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1d9d400640 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e799460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1d9d400640 1 -- 172.21.15.39:0/1465264445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e4e454650 con 0x559e4e3c8390 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1da5e00640 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e799460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1d9d400640 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e7999a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1d900e00d0 tx=0x7f1d900a9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268af1d4bdea4fc2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1d8be00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1d8be00640 1 -- 172.21.15.39:0/1465264445 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900e0560 con 0x559e4e3c8390 2024-07-04T08:20:00.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1d8be00640 1 -- 172.21.15.39:0/1465264445 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1d900a8ec0 con 0x559e4e3c8390 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.320+0000 7f1db3006740 1 -- 172.21.15.39:0/1465264445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da0004030 con 0x559e4e3c8390 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1d8be00640 1 -- 172.21.15.39:0/1465264445 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d900a9770 con 0x559e4e3c8390 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1db3006740 10 client.16521 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1db3006740 1 -- 172.21.15.39:0/1465264445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e4e7a80c0 con 0x559e4e3c8390 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1d8be00640 1 -- 172.21.15.39:0/1465264445 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da0004030 con 0x559e4e3c8390 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1d8be00640 1 -- 172.21.15.39:0/1465264445 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d900e7040 con 0x559e4e3c8390 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1db3006740 10 client.16521 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1db3006740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1db3006740 20 client.16521 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1db3006740 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559e4e3a79f0 0x559e4e7ab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1db3006740 4 client.16521 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:00.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.321+0000 7f1db3006740 1 -- 172.21.15.39:0/1465264445 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e4e705e30 con 0x559e4e3a79f0 2024-07-04T08:20:00.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.322+0000 7f1da5e00640 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559e4e3a79f0 0x559e4e7ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.323+0000 7f1da5e00640 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559e4e3a79f0 0x559e4e7ab560 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73580b70654f9e94 server_cookie=97a56406f800e61e in_seq=0 out_seq=0 2024-07-04T08:20:00.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.323+0000 7f1d8be00640 10 client.16521 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:00.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.323+0000 7f1d8be00640 1 -- 172.21.15.39:0/1465264445 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559e4e705e30 con 0x559e4e3a79f0 2024-07-04T08:20:00.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.323+0000 7f1d8be00640 10 client.16521 handle_command_reply: tid=0 2024-07-04T08:20:00.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.323+0000 7f1db3006740 1 client.16521 shutdown 2024-07-04T08:20:00.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 -- 172.21.15.39:0/1465264445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 msgr2=0x559e4e7999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:00.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e7999a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1d900e00d0 tx=0x7f1d900a9b60 comp rx=0 tx=0).stop 2024-07-04T08:20:00.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 -- 172.21.15.39:0/1465264445 shutdown_connections 2024-07-04T08:20:00.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559e4e3a79f0 0x559e4e7ab560 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e7999a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3e9070 0x559e4e7a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 --2- 172.21.15.39:0/1465264445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c01b0 0x559e4e799460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 -- 172.21.15.39:0/1465264445 >> 172.21.15.39:0/1465264445 conn(0x559e4e3dd460 msgr2=0x559e4e790ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:00.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1d8be00640 0 client.16521 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:00.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 -- 172.21.15.39:0/1465264445 shutdown_connections 2024-07-04T08:20:00.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 1 -- 172.21.15.39:0/1465264445 wait complete. 2024-07-04T08:20:00.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.324+0000 7f1db3006740 20 client.16521 trim_cache size 0 max 16384 2024-07-04T08:20:00.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.340+0000 7f1db3006740 1 Processor -- start 2024-07-04T08:20:00.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.340+0000 7f1db3006740 1 -- start start 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1db3006740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e78f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1db3006740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e79b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1db3006740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3e9070 0x559e4e7981f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1db3006740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b66b0 con 0x559e4e3c8390 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1db3006740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b81f0 con 0x559e4e3e9070 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1db3006740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b6d80 con 0x559e4e3c01b0 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1d9d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e79b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1d9d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e79b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52588/0 (socket says 172.21.15.39:52588) 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1d9d400640 1 -- 172.21.15.39:0/520297404 learned_addr learned my addr 172.21.15.39:0/520297404 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1da5e00640 1 --2- 172.21.15.39:0/520297404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e78f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.341+0000 7f1da6800640 1 --2- 172.21.15.39:0/520297404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3e9070 0x559e4e7981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.342+0000 7f1d9d400640 1 -- 172.21.15.39:0/520297404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 msgr2=0x559e4e78f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.342+0000 7f1d9d400640 1 --2- 172.21.15.39:0/520297404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e78f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.342+0000 7f1d9d400640 1 -- 172.21.15.39:0/520297404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3e9070 msgr2=0x559e4e7981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.342+0000 7f1d9d400640 1 --2- 172.21.15.39:0/520297404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3e9070 0x559e4e7981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.342+0000 7f1d9d400640 1 -- 172.21.15.39:0/520297404 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e4e7901e0 con 0x559e4e3c8390 2024-07-04T08:20:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.342+0000 7f1d9d400640 1 --2- 172.21.15.39:0/520297404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e79b140 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1d900a9960 tx=0x7f1d900a54e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47824565552c0e24 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.342+0000 7f1d96a00640 1 -- 172.21.15.39:0/520297404 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900e5070 con 0x559e4e3c8390 2024-07-04T08:20:00.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.342+0000 7f1d96a00640 1 -- 172.21.15.39:0/520297404 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d90091040 con 0x559e4e3c8390 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.342+0000 7f1d96a00640 1 -- 172.21.15.39:0/520297404 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900ae010 con 0x559e4e3c8390 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.343+0000 7f1db3006740 1 -- 172.21.15.39:0/520297404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 msgr2=0x559e4e79b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.343+0000 7f1db3006740 1 --2- 172.21.15.39:0/520297404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e79b140 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1d900a9960 tx=0x7f1d900a54e0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.343+0000 7f1db3006740 1 -- 172.21.15.39:0/520297404 shutdown_connections 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.343+0000 7f1db3006740 1 --2- 172.21.15.39:0/520297404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3c8390 0x559e4e79b140 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.343+0000 7f1db3006740 1 --2- 172.21.15.39:0/520297404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3e9070 0x559e4e7981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.343+0000 7f1db3006740 1 --2- 172.21.15.39:0/520297404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e78f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.343+0000 7f1db3006740 1 -- 172.21.15.39:0/520297404 >> 172.21.15.39:0/520297404 conn(0x559e4e3dd460 msgr2=0x559e4e797ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.344+0000 7f1db3006740 1 -- 172.21.15.39:0/520297404 shutdown_connections 2024-07-04T08:20:00.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.344+0000 7f1db3006740 1 -- 172.21.15.39:0/520297404 wait complete. 2024-07-04T08:20:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.344+0000 7f1db3006740 1 -- start start 2024-07-04T08:20:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.344+0000 7f1db3006740 1 Processor -- start 2024-07-04T08:20:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1db3006740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e79f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1db3006740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c8390 0x559e4e79f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1db3006740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3e9070 0x559e4e79fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1db3006740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b7b20 con 0x559e4e3e9070 2024-07-04T08:20:00.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1db3006740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b7450 con 0x559e4e3c8390 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1db3006740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b4690 con 0x559e4e3c01b0 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1db3006740 10 client.0 _ll_register_callbacks cb 0x7f1d9000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1da6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3e9070 0x559e4e79fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1da5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e79f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1da6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3e9070 0x559e4e79fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52592/0 (socket says 172.21.15.39:52592) 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1da6800640 1 -- 172.21.15.39:0/3835859938 learned_addr learned my addr 172.21.15.39:0/3835859938 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1da5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e79f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60978/0 (socket says 172.21.15.39:60978) 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.345+0000 7f1d9d400640 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c8390 0x559e4e79f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.346+0000 7f1da5e00640 1 -- 172.21.15.39:0/3835859938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c8390 msgr2=0x559e4e79f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.346+0000 7f1da5e00640 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c8390 0x559e4e79f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.346+0000 7f1da5e00640 1 -- 172.21.15.39:0/3835859938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3e9070 msgr2=0x559e4e79fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:00.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.346+0000 7f1da5e00640 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3e9070 0x559e4e79fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.346+0000 7f1da5e00640 1 -- 172.21.15.39:0/3835859938 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e4e812d00 con 0x559e4e3c01b0 2024-07-04T08:20:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.346+0000 7f1da5e00640 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e79f210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009230 tx=0x7f1d98009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e31254c804ba04b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.346+0000 7f1d8be00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.347+0000 7f1d8be00640 1 -- 172.21.15.39:0/3835859938 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98035040 con 0x559e4e3c01b0 2024-07-04T08:20:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.347+0000 7f1db3006740 1 -- 172.21.15.39:0/3835859938 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d90091040 con 0x559e4e3c01b0 2024-07-04T08:20:00.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.347+0000 7f1d8be00640 1 -- 172.21.15.39:0/3835859938 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1d98005450 con 0x559e4e3c01b0 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.347+0000 7f1d8be00640 1 -- 172.21.15.39:0/3835859938 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d98005650 con 0x559e4e3c01b0 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.347+0000 7f1db3006740 10 client.26569 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.347+0000 7f1db3006740 1 -- 172.21.15.39:0/3835859938 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e4e813f50 con 0x559e4e3c01b0 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.347+0000 7f1d8be00640 1 -- 172.21.15.39:0/3835859938 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d90091040 con 0x559e4e3c01b0 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.348+0000 7f1d8be00640 1 -- 172.21.15.39:0/3835859938 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d980037d0 con 0x559e4e3c01b0 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.348+0000 7f1db3006740 10 client.26569 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.348+0000 7f1db3006740 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.348+0000 7f1db3006740 20 client.26569 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.348+0000 7f1db3006740 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559e4e37cda0 0x559e4e8166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.348+0000 7f1db3006740 4 client.26569 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:20:00.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.348+0000 7f1db3006740 1 -- 172.21.15.39:0/3835859938 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e4e7018d0 con 0x559e4e37cda0 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.348+0000 7f1d9d400640 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559e4e37cda0 0x559e4e8166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.349+0000 7f1d9d400640 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559e4e37cda0 0x559e4e8166f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75ff6737cdd10c3 server_cookie=50800128c599aa06 in_seq=0 out_seq=0 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.349+0000 7f1d8be00640 10 client.26569 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.349+0000 7f1d8be00640 1 -- 172.21.15.39:0/3835859938 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e4e7018d0 con 0x559e4e37cda0 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.349+0000 7f1d8be00640 10 client.26569 handle_command_reply: tid=0 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.349+0000 7f1db3006740 1 client.26569 shutdown 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.349+0000 7f1db3006740 1 -- 172.21.15.39:0/3835859938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 msgr2=0x559e4e79f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.349+0000 7f1db3006740 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e79f210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009230 tx=0x7f1d98009260 comp rx=0 tx=0).stop 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1db3006740 1 -- 172.21.15.39:0/3835859938 shutdown_connections 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1db3006740 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559e4e37cda0 0x559e4e8166f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1db3006740 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559e4e3e9070 0x559e4e79fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1db3006740 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559e4e3c8390 0x559e4e79f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1db3006740 1 --2- 172.21.15.39:0/3835859938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559e4e3c01b0 0x559e4e79f210 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1db3006740 1 -- 172.21.15.39:0/3835859938 >> 172.21.15.39:0/3835859938 conn(0x559e4e3dd460 msgr2=0x559e4e79aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:00.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1d8be00640 0 client.26569 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:00.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1db3006740 1 -- 172.21.15.39:0/3835859938 shutdown_connections 2024-07-04T08:20:00.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1db3006740 1 -- 172.21.15.39:0/3835859938 wait complete. 2024-07-04T08:20:00.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.350+0000 7f1db3006740 20 client.26569 trim_cache size 0 max 16384 2024-07-04T08:20:00.350 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:20:00.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 -- 172.21.15.39:0/2549386696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d8c0b2960 msgr2=0x7f1d8c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:00.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d8c0b2960 0x7f1d8c0b4e70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1d98005f70 tx=0x7f1d98009560 comp rx=0 tx=0).stop 2024-07-04T08:20:00.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 -- 172.21.15.39:0/2549386696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da810ee50 msgr2=0x7f1da81a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:00.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da810ee50 0x7f1da81a03e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000e9f0 tx=0x7f1d9000edf0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 -- 172.21.15.39:0/2549386696 shutdown_connections 2024-07-04T08:20:00.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d8c0b2960 0x7f1d8c0b4e70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da810f7a0 0x7f1da81a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da810ee50 0x7f1da81a03e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 --2- 172.21.15.39:0/2549386696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da8109510 0x7f1da819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:00.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 -- 172.21.15.39:0/2549386696 >> 172.21.15.39:0/2549386696 conn(0x7f1da8078b10 msgr2=0x7f1da80793d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:00.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 -- 172.21.15.39:0/2549386696 shutdown_connections 2024-07-04T08:20:00.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:00.351+0000 7f1da7200640 1 -- 172.21.15.39:0/2549386696 wait complete. 2024-07-04T08:20:00.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:00 smithi050 ceph-mon[36577]: overall HEALTH_OK 2024-07-04T08:20:01.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:01 smithi133 ceph-mon[37050]: pgmap v624: 129 pgs: 129 active+clean; 4.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-07-04T08:20:01.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:01 smithi039 ceph-mon[32062]: pgmap v624: 129 pgs: 129 active+clean; 4.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-07-04T08:20:01.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:01 smithi050 ceph-mon[36577]: pgmap v624: 129 pgs: 129 active+clean; 4.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-07-04T08:20:02.160 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:20:02.160 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:20:02.648 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:03.098 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:03 smithi039 ceph-mon[32062]: pgmap v625: 129 pgs: 129 active+clean; 4.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-07-04T08:20:03.312 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:03 smithi133 ceph-mon[37050]: pgmap v625: 129 pgs: 129 active+clean; 4.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-07-04T08:20:03.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:03 smithi050 ceph-mon[36577]: pgmap v625: 129 pgs: 129 active+clean; 4.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-07-04T08:20:04.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.884+0000 7f88f0a00640 1 -- 172.21.15.39:0/3409271190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88ec109340 msgr2=0x7f88ec10fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:04.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.884+0000 7f88f0a00640 1 --2- 172.21.15.39:0/3409271190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88ec109340 0x7f88ec10fb70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009ca0 tx=0x7f88e402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.884+0000 7f88f0a00640 1 -- 172.21.15.39:0/3409271190 shutdown_connections 2024-07-04T08:20:04.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.884+0000 7f88f0a00640 1 --2- 172.21.15.39:0/3409271190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88ec109340 0x7f88ec10fb70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.884+0000 7f88f0a00640 1 --2- 172.21.15.39:0/3409271190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88ec1089f0 0x7f88ec108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.884+0000 7f88f0a00640 1 --2- 172.21.15.39:0/3409271190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec1030b0 0x7f88ec1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.884+0000 7f88f0a00640 1 -- 172.21.15.39:0/3409271190 >> 172.21.15.39:0/3409271190 conn(0x7f88ec07ada0 msgr2=0x7f88ec07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:04.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.884+0000 7f88f0a00640 1 -- 172.21.15.39:0/3409271190 shutdown_connections 2024-07-04T08:20:04.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.884+0000 7f88f0a00640 1 -- 172.21.15.39:0/3409271190 wait complete. 2024-07-04T08:20:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.885+0000 7f88f0a00640 1 Processor -- start 2024-07-04T08:20:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.885+0000 7f88f0a00640 1 -- start start 2024-07-04T08:20:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88f0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88ec1030b0 0x7f88ec19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88f0a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88ec1089f0 0x7f88ec19c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88f0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec109340 0x7f88ec1a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88f0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec112ff0 con 0x7f88ec1030b0 2024-07-04T08:20:04.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88f0a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec112e70 con 0x7f88ec109340 2024-07-04T08:20:04.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88f0a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec113170 con 0x7f88ec1089f0 2024-07-04T08:20:04.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88ea800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec109340 0x7f88ec1a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88ec1030b0 0x7f88ec19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88e9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88ec1089f0 0x7f88ec19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88ea800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec109340 0x7f88ec1a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:59328/0 (socket says 172.21.15.39:59328) 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88e9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88ec1089f0 0x7f88ec19c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34382/0 (socket says 172.21.15.39:34382) 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.886+0000 7f88ea800640 1 -- 172.21.15.39:0/1414194872 learned_addr learned my addr 172.21.15.39:0/1414194872 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.887+0000 7f88e9400640 1 -- 172.21.15.39:0/1414194872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec109340 msgr2=0x7f88ec1a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.887+0000 7f88e9400640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec109340 0x7f88ec1a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.887+0000 7f88e9400640 1 -- 172.21.15.39:0/1414194872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88ec1030b0 msgr2=0x7f88ec19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.887+0000 7f88e9400640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88ec1030b0 0x7f88ec19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.887+0000 7f88e9400640 1 -- 172.21.15.39:0/1414194872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec1a1b30 con 0x7f88ec1089f0 2024-07-04T08:20:04.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.887+0000 7f88e9400640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88ec1089f0 0x7f88ec19c160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f88ec06bf40 tx=0x7f88d800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:04.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.888+0000 7f88e2a00640 1 -- 172.21.15.39:0/1414194872 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d800cc40 con 0x7f88ec1089f0 2024-07-04T08:20:04.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.888+0000 7f88f0a00640 1 -- 172.21.15.39:0/1414194872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec1a1e20 con 0x7f88ec1089f0 2024-07-04T08:20:04.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.888+0000 7f88e2a00640 1 -- 172.21.15.39:0/1414194872 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d800cde0 con 0x7f88ec1089f0 2024-07-04T08:20:04.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.888+0000 7f88e2a00640 1 -- 172.21.15.39:0/1414194872 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8018760 con 0x7f88ec1089f0 2024-07-04T08:20:04.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.888+0000 7f88f0a00640 1 -- 172.21.15.39:0/1414194872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec1a8580 con 0x7f88ec1089f0 2024-07-04T08:20:04.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.892+0000 7f88f708a740 1 Processor -- start 2024-07-04T08:20:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.892+0000 7f88f708a740 1 -- start start 2024-07-04T08:20:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88f708a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a23013640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88f708a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23013b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88f708a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c4e670 0x558a230140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88f708a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a22f3d8e0 con 0x558a22c4e670 2024-07-04T08:20:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88f708a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558a22f89ec0 con 0x558a22c6f350 2024-07-04T08:20:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88f708a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a22f79460 con 0x558a22c46490 2024-07-04T08:20:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23013b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88ea800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a23013640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23013b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34388/0 (socket says 172.21.15.39:34388) 2024-07-04T08:20:04.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.893+0000 7f88e9e00640 1 -- 172.21.15.39:0/3959842600 learned_addr learned my addr 172.21.15.39:0/3959842600 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:04.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.894+0000 7f88e9e00640 1 -- 172.21.15.39:0/3959842600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 msgr2=0x558a23013640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.894+0000 7f88e9e00640 1 --2- 172.21.15.39:0/3959842600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a23013640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.894+0000 7f88e9e00640 1 -- 172.21.15.39:0/3959842600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c4e670 msgr2=0x558a230140c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.894+0000 7f88e9e00640 1 --2- 172.21.15.39:0/3959842600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c4e670 0x558a230140c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.894+0000 7f88e9e00640 1 -- 172.21.15.39:0/3959842600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a22f11280 con 0x558a22c46490 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.894+0000 7f88e2a00640 1 -- 172.21.15.39:0/1414194872 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f88d8010a90 con 0x7f88ec1089f0 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.894+0000 7f88e9e00640 1 --2- 172.21.15.39:0/3959842600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23013b80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009cf0 tx=0x7f88dc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e15c72c673783888 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.894+0000 7f88d7400640 1 -- 172.21.15.39:0/3959842600 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc004230 con 0x558a22c46490 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.894+0000 7f88d7400640 1 -- 172.21.15.39:0/3959842600 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc0043d0 con 0x558a22c46490 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.895+0000 7f88d7400640 1 -- 172.21.15.39:0/3959842600 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc019820 con 0x558a22c46490 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.895+0000 7f88f708a740 1 -- 172.21.15.39:0/3959842600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 msgr2=0x558a23013b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:04.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.895+0000 7f88f708a740 1 --2- 172.21.15.39:0/3959842600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23013b80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009cf0 tx=0x7f88dc011810 comp rx=0 tx=0).stop 2024-07-04T08:20:04.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88f708a740 1 -- 172.21.15.39:0/3959842600 shutdown_connections 2024-07-04T08:20:04.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88f708a740 1 --2- 172.21.15.39:0/3959842600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c4e670 0x558a230140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88f708a740 1 --2- 172.21.15.39:0/3959842600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23013b80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88f708a740 1 --2- 172.21.15.39:0/3959842600 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a23013640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88f708a740 1 -- 172.21.15.39:0/3959842600 >> 172.21.15.39:0/3959842600 conn(0x558a22c63740 msgr2=0x558a23011530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88f708a740 1 -- 172.21.15.39:0/3959842600 shutdown_connections 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88f708a740 1 -- 172.21.15.39:0/3959842600 wait complete. 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88e2a00640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88d00b2870 0x7f88d00b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88e2a00640 1 -- 172.21.15.39:0/1414194872 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f88d80e2c60 con 0x7f88ec1089f0 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88e9e00640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88d00b2870 0x7f88d00b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.896+0000 7f88f708a740 1 -- start start 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.897+0000 7f88f708a740 1 Processor -- start 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.897+0000 7f88e9e00640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88d00b2870 0x7f88d00b4d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f88dc003eb0 tx=0x7f88dc000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.897+0000 7f88f708a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c46490 0x558a2301f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.897+0000 7f88f708a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c4e670 0x558a2301f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.897+0000 7f88f708a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a2302af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.897+0000 7f88f708a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a22f20760 con 0x558a22c46490 2024-07-04T08:20:04.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.897+0000 7f88f708a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558a22f7d4a0 con 0x558a22c6f350 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.897+0000 7f88f708a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a22ddd890 con 0x558a22c4e670 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.897+0000 7f88f708a740 10 client.0 _ll_register_callbacks cb 0x558a22f212b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c46490 0x558a2301f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88e9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c4e670 0x558a2301f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88e9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c4e670 0x558a2301f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34404/0 (socket says 172.21.15.39:34404) 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88ea800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a2302af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88e9400640 1 -- 172.21.15.39:0/3274768525 learned_addr learned my addr 172.21.15.39:0/3274768525 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88e9400640 1 -- 172.21.15.39:0/3274768525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 msgr2=0x558a2302af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88e9400640 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a2302af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88e9400640 1 -- 172.21.15.39:0/3274768525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c46490 msgr2=0x558a2301f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88e9400640 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c46490 0x558a2301f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.898+0000 7f88e9400640 1 -- 172.21.15.39:0/3274768525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a22cda930 con 0x558a22c4e670 2024-07-04T08:20:04.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.899+0000 7f88e9400640 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c4e670 0x558a2301f6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f88d80a5ce0 tx=0x7f88d80a61a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f27c46cdb02f01e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.899+0000 7f88cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.899+0000 7f88cbe00640 1 -- 172.21.15.39:0/3274768525 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d80cae60 con 0x558a22c4e670 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.899+0000 7f88cbe00640 1 -- 172.21.15.39:0/3274768525 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f88d80a90d0 con 0x558a22c4e670 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.899+0000 7f88f708a740 1 -- 172.21.15.39:0/3274768525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88dc0043d0 con 0x558a22c4e670 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.899+0000 7f88cbe00640 1 -- 172.21.15.39:0/3274768525 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88d80a9c60 con 0x558a22c4e670 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.899+0000 7f88f708a740 10 client.26581 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.899+0000 7f88f708a740 1 -- 172.21.15.39:0/3274768525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a2302dc60 con 0x558a22c4e670 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.900+0000 7f88cbe00640 1 -- 172.21.15.39:0/3274768525 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc0043d0 con 0x558a22c4e670 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.900+0000 7f88cbe00640 1 -- 172.21.15.39:0/3274768525 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f88d80a8930 con 0x558a22c4e670 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.900+0000 7f88f708a740 10 client.26581 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.900+0000 7f88f708a740 10 client.26581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:04.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.900+0000 7f88f708a740 20 client.26581 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.900+0000 7f88f708a740 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a22c2dcd0 0x558a230310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.900+0000 7f88f708a740 4 client.26581 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.900+0000 7f88f708a740 1 -- 172.21.15.39:0/3274768525 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a22f8be10 con 0x558a22c2dcd0 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.900+0000 7f88e9e00640 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a22c2dcd0 0x558a230310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.901+0000 7f88e9e00640 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a22c2dcd0 0x558a230310d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e18449ebe21a3e5 server_cookie=f1f499d71b714778 in_seq=0 out_seq=0 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.901+0000 7f88cbe00640 10 client.26581 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88cbe00640 1 -- 172.21.15.39:0/3274768525 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558a22f8be10 con 0x558a22c2dcd0 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88cbe00640 10 client.26581 handle_command_reply: tid=0 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88f708a740 1 client.26581 shutdown 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88f708a740 1 -- 172.21.15.39:0/3274768525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c4e670 msgr2=0x558a2301f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88f708a740 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c4e670 0x558a2301f6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f88d80a5ce0 tx=0x7f88d80a61a0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88f708a740 1 -- 172.21.15.39:0/3274768525 shutdown_connections 2024-07-04T08:20:04.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88f708a740 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a22c2dcd0 0x558a230310d0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88f708a740 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a2302af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88f708a740 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c4e670 0x558a2301f6b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88f708a740 1 --2- 172.21.15.39:0/3274768525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c46490 0x558a2301f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88cbe00640 0 client.26581 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:04.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.902+0000 7f88f708a740 1 -- 172.21.15.39:0/3274768525 >> 172.21.15.39:0/3274768525 conn(0x558a22c63740 msgr2=0x558a23016820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:04.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.903+0000 7f88f708a740 1 -- 172.21.15.39:0/3274768525 shutdown_connections 2024-07-04T08:20:04.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.903+0000 7f88f708a740 1 -- 172.21.15.39:0/3274768525 wait complete. 2024-07-04T08:20:04.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.903+0000 7f88f708a740 20 client.26581 trim_cache size 0 max 16384 2024-07-04T08:20:04.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.923+0000 7f88f708a740 1 Processor -- start 2024-07-04T08:20:04.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.923+0000 7f88f708a740 1 -- start start 2024-07-04T08:20:04.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88f708a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23020ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88f708a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c4e670 0x558a23036840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88f708a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c6f350 0x558a23036d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88f708a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1b24e0 con 0x558a22c6f350 2024-07-04T08:20:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88f708a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1b4020 con 0x558a22c4e670 2024-07-04T08:20:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88f708a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1b2bb0 con 0x558a22c46490 2024-07-04T08:20:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88ea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c6f350 0x558a23036d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23020ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88ea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c6f350 0x558a23036d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48050/0 (socket says 172.21.15.39:48050) 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23020ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34416/0 (socket says 172.21.15.39:34416) 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88ea800640 1 -- 172.21.15.39:0/2533950740 learned_addr learned my addr 172.21.15.39:0/2533950740 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.924+0000 7f88e9400640 1 --2- 172.21.15.39:0/2533950740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c4e670 0x558a23036840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.925+0000 7f88ea800640 1 -- 172.21.15.39:0/2533950740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 msgr2=0x558a23020ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.925+0000 7f88ea800640 1 --2- 172.21.15.39:0/2533950740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23020ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.925+0000 7f88ea800640 1 -- 172.21.15.39:0/2533950740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c4e670 msgr2=0x558a23036840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.925+0000 7f88ea800640 1 --2- 172.21.15.39:0/2533950740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c4e670 0x558a23036840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.925+0000 7f88ea800640 1 -- 172.21.15.39:0/2533950740 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a2301fa20 con 0x558a22c6f350 2024-07-04T08:20:04.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.925+0000 7f88ea800640 1 --2- 172.21.15.39:0/2533950740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c6f350 0x558a23036d80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f88e402f3f0 tx=0x7f88e4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf635cb66b57f5e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.926+0000 7f88d6a00640 1 -- 172.21.15.39:0/2533950740 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4031e30 con 0x558a22c6f350 2024-07-04T08:20:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.926+0000 7f88d6a00640 1 -- 172.21.15.39:0/2533950740 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e404b070 con 0x558a22c6f350 2024-07-04T08:20:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.926+0000 7f88d6a00640 1 -- 172.21.15.39:0/2533950740 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e40042a0 con 0x558a22c6f350 2024-07-04T08:20:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 -- 172.21.15.39:0/2533950740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c6f350 msgr2=0x558a23036d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 --2- 172.21.15.39:0/2533950740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c6f350 0x558a23036d80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f88e402f3f0 tx=0x7f88e4005c50 comp rx=0 tx=0).stop 2024-07-04T08:20:04.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 -- 172.21.15.39:0/2533950740 shutdown_connections 2024-07-04T08:20:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 --2- 172.21.15.39:0/2533950740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c6f350 0x558a23036d80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 --2- 172.21.15.39:0/2533950740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c4e670 0x558a23036840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 --2- 172.21.15.39:0/2533950740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23020ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 -- 172.21.15.39:0/2533950740 >> 172.21.15.39:0/2533950740 conn(0x558a22c63740 msgr2=0x558a23020490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 -- 172.21.15.39:0/2533950740 shutdown_connections 2024-07-04T08:20:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 -- 172.21.15.39:0/2533950740 wait complete. 2024-07-04T08:20:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.927+0000 7f88f708a740 1 -- start start 2024-07-04T08:20:04.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.928+0000 7f88f708a740 1 Processor -- start 2024-07-04T08:20:04.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.928+0000 7f88f708a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23027590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.928+0000 7f88f708a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c4e670 0x558a23022a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.928+0000 7f88f708a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a23022fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.928+0000 7f88f708a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1b3950 con 0x558a22c4e670 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.928+0000 7f88f708a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1b3280 con 0x558a22c6f350 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.928+0000 7f88f708a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec1b04c0 con 0x558a22c46490 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.928+0000 7f88f708a740 10 client.0 _ll_register_callbacks cb 0x7f88d80e04d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23027590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88e9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c4e670 0x558a23022a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88e9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23027590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34420/0 (socket says 172.21.15.39:34420) 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88e9e00640 1 -- 172.21.15.39:0/2275425685 learned_addr learned my addr 172.21.15.39:0/2275425685 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88ea800640 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a23022fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88e9e00640 1 -- 172.21.15.39:0/2275425685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 msgr2=0x558a23022fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:04.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88e9e00640 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a23022fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88e9e00640 1 -- 172.21.15.39:0/2275425685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c4e670 msgr2=0x558a23022a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:04.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88e9e00640 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c4e670 0x558a23022a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.929+0000 7f88e9e00640 1 -- 172.21.15.39:0/2275425685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a23023740 con 0x558a22c46490 2024-07-04T08:20:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.930+0000 7f88e9e00640 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23027590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f88dc004060 tx=0x7f88dc019420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=609e0184c8756c8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.930+0000 7f88cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.930+0000 7f88cbe00640 1 -- 172.21.15.39:0/2275425685 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc003a80 con 0x558a22c46490 2024-07-04T08:20:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.930+0000 7f88f708a740 1 -- 172.21.15.39:0/2275425685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88e404b070 con 0x558a22c46490 2024-07-04T08:20:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.930+0000 7f88cbe00640 1 -- 172.21.15.39:0/2275425685 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f88dc004640 con 0x558a22c46490 2024-07-04T08:20:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88cbe00640 1 -- 172.21.15.39:0/2275425685 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88dc048050 con 0x558a22c46490 2024-07-04T08:20:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88f708a740 10 client.26587 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:04.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88f708a740 1 -- 172.21.15.39:0/2275425685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a230a1f60 con 0x558a22c46490 2024-07-04T08:20:04.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88cbe00640 1 -- 172.21.15.39:0/2275425685 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e404b070 con 0x558a22c46490 2024-07-04T08:20:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88cbe00640 1 -- 172.21.15.39:0/2275425685 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f88dc019750 con 0x558a22c46490 2024-07-04T08:20:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88f708a740 10 client.26587 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:04.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88f708a740 10 client.26587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88f708a740 20 client.26587 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88f708a740 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a22c03080 0x558a230a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88f708a740 4 client.26587 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:20:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.931+0000 7f88f708a740 1 -- 172.21.15.39:0/2275425685 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558a22f878b0 con 0x558a22c03080 2024-07-04T08:20:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.932+0000 7f88e9400640 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a22c03080 0x558a230a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.932+0000 7f88e9400640 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a22c03080 0x558a230a45b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f195e2c3887c8fb server_cookie=7e4f2db7874a28ff in_seq=0 out_seq=0 2024-07-04T08:20:04.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.932+0000 7f88cbe00640 10 client.26587 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88cbe00640 1 -- 172.21.15.39:0/2275425685 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558a22f878b0 con 0x558a22c03080 2024-07-04T08:20:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88cbe00640 10 client.26587 handle_command_reply: tid=0 2024-07-04T08:20:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88f708a740 1 client.26587 shutdown 2024-07-04T08:20:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88f708a740 1 -- 172.21.15.39:0/2275425685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 msgr2=0x558a23027590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:04.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88f708a740 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23027590 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f88dc004060 tx=0x7f88dc019420 comp rx=0 tx=0).stop 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88f708a740 1 -- 172.21.15.39:0/2275425685 shutdown_connections 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88f708a740 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a22c03080 0x558a230a45b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88f708a740 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a22c4e670 0x558a23022a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88f708a740 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a22c6f350 0x558a23022fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88f708a740 1 --2- 172.21.15.39:0/2275425685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a22c46490 0x558a23027590 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88cbe00640 0 client.26587 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.934+0000 7f88f708a740 1 -- 172.21.15.39:0/2275425685 >> 172.21.15.39:0/2275425685 conn(0x558a22c63740 msgr2=0x558a2301d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.935+0000 7f88f708a740 1 -- 172.21.15.39:0/2275425685 shutdown_connections 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.935+0000 7f88f708a740 1 -- 172.21.15.39:0/2275425685 wait complete. 2024-07-04T08:20:04.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.935+0000 7f88f708a740 20 client.26587 trim_cache size 0 max 16384 2024-07-04T08:20:04.936 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:20:04.936 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:20:04.936 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "bcf9dff3-66ef-4312-87f2-2ce5bf91f822", 2024-07-04T08:20:04.936 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:20:04.936 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:20:04.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 -- 172.21.15.39:0/1414194872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88d00b2870 msgr2=0x7f88d00b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88d00b2870 0x7f88d00b4d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f88dc003eb0 tx=0x7f88dc000bc0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 -- 172.21.15.39:0/1414194872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88ec1089f0 msgr2=0x7f88ec19c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88ec1089f0 0x7f88ec19c160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f88ec06bf40 tx=0x7f88d800ee50 comp rx=0 tx=0).stop 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 -- 172.21.15.39:0/1414194872 shutdown_connections 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88d00b2870 0x7f88d00b4d80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f88ec109340 0x7f88ec1a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88ec1089f0 0x7f88ec19c160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 --2- 172.21.15.39:0/1414194872 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88ec1030b0 0x7f88ec19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 -- 172.21.15.39:0/1414194872 >> 172.21.15.39:0/1414194872 conn(0x7f88ec07ada0 msgr2=0x7f88ec106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 -- 172.21.15.39:0/1414194872 shutdown_connections 2024-07-04T08:20:04.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:04.936+0000 7f88f0a00640 1 -- 172.21.15.39:0/1414194872 wait complete. 2024-07-04T08:20:05.313 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:05 smithi039 ceph-mon[32062]: pgmap v626: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-07-04T08:20:05.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:05 smithi133 ceph-mon[37050]: pgmap v626: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-07-04T08:20:05.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:05 smithi050 ceph-mon[36577]: pgmap v626: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-07-04T08:20:05.828 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:20:06.312 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:06 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:20:06.312 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:06 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:06.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:06 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:20:06.313 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:06 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:06.317 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:06.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:06 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:20:06.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:06 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:06.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:06 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:20:06.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:06 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:06.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:06 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:20:06.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:06 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:06.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:06 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:20:06.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:06 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:07.312 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:07 smithi133 ceph-mon[37050]: pgmap v627: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-07-04T08:20:07.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:07 smithi039 ceph-mon[32062]: pgmap v627: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-07-04T08:20:07.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:07 smithi050 ceph-mon[36577]: pgmap v627: 129 pgs: 129 active+clean; 4.6 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-07-04T08:20:08.330 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:08 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:20:08.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:08 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:20:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.549+0000 7fa429400640 1 -- 172.21.15.39:0/1945697684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa42410b590 msgr2=0x7fa424111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.549+0000 7fa429400640 1 --2- 172.21.15.39:0/1945697684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa42410b590 0x7fa424111dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009be0 tx=0x7fa41c02a6d0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.549+0000 7fa429400640 1 -- 172.21.15.39:0/1945697684 shutdown_connections 2024-07-04T08:20:08.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.549+0000 7fa429400640 1 --2- 172.21.15.39:0/1945697684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa42410b590 0x7fa424111dc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.549+0000 7fa429400640 1 --2- 172.21.15.39:0/1945697684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42410ac40 0x7fa42410b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.549+0000 7fa429400640 1 --2- 172.21.15.39:0/1945697684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa424105300 0x7fa424105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.549+0000 7fa429400640 1 -- 172.21.15.39:0/1945697684 >> 172.21.15.39:0/1945697684 conn(0x7fa424100d30 msgr2=0x7fa424103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.549+0000 7fa429400640 1 -- 172.21.15.39:0/1945697684 shutdown_connections 2024-07-04T08:20:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.549+0000 7fa429400640 1 -- 172.21.15.39:0/1945697684 wait complete. 2024-07-04T08:20:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.551+0000 7fa429400640 1 Processor -- start 2024-07-04T08:20:08.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.551+0000 7fa429400640 1 -- start start 2024-07-04T08:20:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.551+0000 7fa429400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa424105300 0x7fa42419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.551+0000 7fa429400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42410ac40 0x7fa42419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.551+0000 7fa429400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa42410b590 0x7fa4241a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.551+0000 7fa429400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa424115240 con 0x7fa42410b590 2024-07-04T08:20:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.551+0000 7fa429400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241150c0 con 0x7fa42410ac40 2024-07-04T08:20:08.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.551+0000 7fa429400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241153c0 con 0x7fa424105300 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa422800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa424105300 0x7fa42419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa422800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa424105300 0x7fa42419bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34440/0 (socket says 172.21.15.39:34440) 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa422800640 1 -- 172.21.15.39:0/1936470412 learned_addr learned my addr 172.21.15.39:0/1936470412 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa41be00640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42410ac40 0x7fa42419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa423200640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa42410b590 0x7fa4241a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa422800640 1 -- 172.21.15.39:0/1936470412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42410ac40 msgr2=0x7fa42419c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa422800640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42410ac40 0x7fa42419c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa422800640 1 -- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa42410b590 msgr2=0x7fa4241a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa422800640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa42410b590 0x7fa4241a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.552+0000 7fa422800640 1 -- 172.21.15.39:0/1936470412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4241a1b20 con 0x7fa424105300 2024-07-04T08:20:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.553+0000 7fa422800640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa424105300 0x7fa42419bbb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e9c0 tx=0x7fa41400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.553+0000 7fa41b400640 1 -- 172.21.15.39:0/1936470412 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41400ccc0 con 0x7fa424105300 2024-07-04T08:20:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.553+0000 7fa41b400640 1 -- 172.21.15.39:0/1936470412 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4140054d0 con 0x7fa424105300 2024-07-04T08:20:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.553+0000 7fa429400640 1 -- 172.21.15.39:0/1936470412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4241a1e10 con 0x7fa424105300 2024-07-04T08:20:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.553+0000 7fa41b400640 1 -- 172.21.15.39:0/1936470412 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414010430 con 0x7fa424105300 2024-07-04T08:20:08.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.553+0000 7fa429400640 1 -- 172.21.15.39:0/1936470412 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4241a8570 con 0x7fa424105300 2024-07-04T08:20:08.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.557+0000 7fa42fa31740 1 Processor -- start 2024-07-04T08:20:08.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.557+0000 7fa41b400640 1 -- 172.21.15.39:0/1936470412 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa414004f80 con 0x7fa424105300 2024-07-04T08:20:08.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.557+0000 7fa42fa31740 1 -- start start 2024-07-04T08:20:08.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.557+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb8a070 0x55de0ff2e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.557+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.557+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 0x55de0ff2f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.557+0000 7fa42fa31740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55de0fea4ea0 con 0x55de0fb8a070 2024-07-04T08:20:08.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.557+0000 7fa42fa31740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55de0fe94440 con 0x55de0fb69390 2024-07-04T08:20:08.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.557+0000 7fa42fa31740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55de0fe98480 con 0x55de0fb611b0 2024-07-04T08:20:08.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa423200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb8a070 0x55de0ff2e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa422800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa423200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb8a070 0x55de0ff2e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48086/0 (socket says 172.21.15.39:48086) 2024-07-04T08:20:08.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa423200640 1 -- 172.21.15.39:0/1232835097 learned_addr learned my addr 172.21.15.39:0/1232835097 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:08.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa41be00640 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 0x55de0ff2f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa422800640 1 -- 172.21.15.39:0/1232835097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 msgr2=0x55de0ff2f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:08.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa422800640 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 0x55de0ff2f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa422800640 1 -- 172.21.15.39:0/1232835097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb8a070 msgr2=0x55de0ff2e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:08.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa422800640 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb8a070 0x55de0ff2e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa422800640 1 -- 172.21.15.39:0/1232835097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de0fe2c610 con 0x55de0fb611b0 2024-07-04T08:20:08.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa423200640 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb8a070 0x55de0ff2e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:08.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.558+0000 7fa41be00640 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 0x55de0ff2f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:08.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.559+0000 7fa422800640 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2ecb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4140e0170 tx=0x7fa4140e97f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ca55a4ea5297f0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:08.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.559+0000 7fa41b400640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa4080b2960 0x7fa4080b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.559+0000 7fa413e00640 1 -- 172.21.15.39:0/1232835097 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a9650 con 0x55de0fb611b0 2024-07-04T08:20:08.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.559+0000 7fa413e00640 1 -- 172.21.15.39:0/1232835097 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4140aa400 con 0x55de0fb611b0 2024-07-04T08:20:08.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.559+0000 7fa413e00640 1 -- 172.21.15.39:0/1232835097 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a9840 con 0x55de0fb611b0 2024-07-04T08:20:08.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.559+0000 7fa423200640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa4080b2960 0x7fa4080b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.559+0000 7fa41b400640 1 -- 172.21.15.39:0/1936470412 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa4140e2db0 con 0x7fa424105300 2024-07-04T08:20:08.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.560+0000 7fa42fa31740 1 -- 172.21.15.39:0/1232835097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 msgr2=0x55de0ff2ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:08.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.560+0000 7fa42fa31740 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2ecb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa4140e0170 tx=0x7fa4140e97f0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.560+0000 7fa423200640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa4080b2960 0x7fa4080b4e70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa41c006010 tx=0x7fa41c004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:08.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.561+0000 7fa42fa31740 1 -- 172.21.15.39:0/1232835097 shutdown_connections 2024-07-04T08:20:08.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.561+0000 7fa42fa31740 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 0x55de0ff2f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.561+0000 7fa42fa31740 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2ecb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.561+0000 7fa42fa31740 1 --2- 172.21.15.39:0/1232835097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb8a070 0x55de0ff2e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.561+0000 7fa42fa31740 1 -- 172.21.15.39:0/1232835097 >> 172.21.15.39:0/1232835097 conn(0x55de0fb7e460 msgr2=0x55de0ff2c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:08.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.561+0000 7fa42fa31740 1 -- 172.21.15.39:0/1232835097 shutdown_connections 2024-07-04T08:20:08.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.561+0000 7fa42fa31740 1 -- 172.21.15.39:0/1232835097 wait complete. 2024-07-04T08:20:08.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.562+0000 7fa42fa31740 1 -- start start 2024-07-04T08:20:08.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.562+0000 7fa42fa31740 1 Processor -- start 2024-07-04T08:20:08.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.562+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff3a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.562+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 0x55de0ff3a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb8a070 0x55de0ff46240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa42fa31740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55de0fcf8700 con 0x55de0fb611b0 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa42fa31740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55de0fe3bb70 con 0x55de0fb69390 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa42fa31740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55de0ff3cf50 con 0x55de0fb8a070 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa42fa31740 10 client.0 _ll_register_callbacks cb 0x55de0fe8d530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa422800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff3a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa422800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff3a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48100/0 (socket says 172.21.15.39:48100) 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa422800640 1 -- 172.21.15.39:0/4088816234 learned_addr learned my addr 172.21.15.39:0/4088816234 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa423200640 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb8a070 0x55de0ff46240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa41be00640 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 0x55de0ff3a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa422800640 1 -- 172.21.15.39:0/4088816234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb8a070 msgr2=0x55de0ff46240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa422800640 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb8a070 0x55de0ff46240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa422800640 1 -- 172.21.15.39:0/4088816234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 msgr2=0x55de0ff3a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:08.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa422800640 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 0x55de0ff3a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.563+0000 7fa422800640 1 -- 172.21.15.39:0/4088816234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de0fbf5650 con 0x55de0fb611b0 2024-07-04T08:20:08.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.564+0000 7fa422800640 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff3a230 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa4140a9c90 tx=0x7fa4140e6630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28501a2ea4c4a8c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:08.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.564+0000 7fa410c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:20:08.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.564+0000 7fa410c00640 1 -- 172.21.15.39:0/4088816234 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a9ad0 con 0x55de0fb611b0 2024-07-04T08:20:08.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.564+0000 7fa410c00640 1 -- 172.21.15.39:0/4088816234 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa414005270 con 0x55de0fb611b0 2024-07-04T08:20:08.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.564+0000 7fa42fa31740 1 -- 172.21.15.39:0/4088816234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4140aa400 con 0x55de0fb611b0 2024-07-04T08:20:08.564 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:08 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:20:08.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.564+0000 7fa410c00640 1 -- 172.21.15.39:0/4088816234 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4140a5490 con 0x55de0fb611b0 2024-07-04T08:20:08.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.564+0000 7fa42fa31740 10 client.16554 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:08.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.564+0000 7fa42fa31740 1 -- 172.21.15.39:0/4088816234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55de0ff48e70 con 0x55de0fb611b0 2024-07-04T08:20:08.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.565+0000 7fa410c00640 1 -- 172.21.15.39:0/4088816234 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4140aa400 con 0x55de0fb611b0 2024-07-04T08:20:08.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.565+0000 7fa410c00640 1 -- 172.21.15.39:0/4088816234 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa4140e90d0 con 0x55de0fb611b0 2024-07-04T08:20:08.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.565+0000 7fa42fa31740 10 client.16554 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:08.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.565+0000 7fa42fa31740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.565+0000 7fa42fa31740 20 client.16554 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.565+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55de0fb489f0 0x55de0ff4c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.565+0000 7fa42fa31740 4 client.16554 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.565+0000 7fa42fa31740 1 -- 172.21.15.39:0/4088816234 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de0fea6df0 con 0x55de0fb489f0 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.566+0000 7fa41be00640 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55de0fb489f0 0x55de0ff4c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.566+0000 7fa41be00640 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55de0fb489f0 0x55de0ff4c000 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a27176090590a6 server_cookie=3993326ef75c1f8 in_seq=0 out_seq=0 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.566+0000 7fa410c00640 10 client.16554 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa410c00640 1 -- 172.21.15.39:0/4088816234 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55de0fea6df0 con 0x55de0fb489f0 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa410c00640 10 client.16554 handle_command_reply: tid=0 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa42fa31740 1 client.16554 shutdown 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa42fa31740 1 -- 172.21.15.39:0/4088816234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 msgr2=0x55de0ff3a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff3a230 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa4140a9c90 tx=0x7fa4140e6630 comp rx=0 tx=0).stop 2024-07-04T08:20:08.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa42fa31740 1 -- 172.21.15.39:0/4088816234 shutdown_connections 2024-07-04T08:20:08.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55de0fb489f0 0x55de0ff4c000 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb8a070 0x55de0ff46240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb69390 0x55de0ff3a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4088816234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff3a230 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa410c00640 0 client.16554 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:08.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.567+0000 7fa42fa31740 1 -- 172.21.15.39:0/4088816234 >> 172.21.15.39:0/4088816234 conn(0x55de0fb7e460 msgr2=0x55de0ff31950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:08.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.568+0000 7fa42fa31740 1 -- 172.21.15.39:0/4088816234 shutdown_connections 2024-07-04T08:20:08.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.568+0000 7fa42fa31740 1 -- 172.21.15.39:0/4088816234 wait complete. 2024-07-04T08:20:08.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.568+0000 7fa42fa31740 20 client.16554 trim_cache size 0 max 16384 2024-07-04T08:20:08.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.589+0000 7fa42fa31740 1 Processor -- start 2024-07-04T08:20:08.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.589+0000 7fa42fa31740 1 -- start start 2024-07-04T08:20:08.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.589+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb69390 0x55de0ff2c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff461b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa42fa31740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241b23c0 con 0x55de0fb69390 2024-07-04T08:20:08.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa42fa31740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241b3f00 con 0x55de0fb8a070 2024-07-04T08:20:08.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa42fa31740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241b2a90 con 0x55de0fb611b0 2024-07-04T08:20:08.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa422800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa422800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34476/0 (socket says 172.21.15.39:34476) 2024-07-04T08:20:08.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa422800640 1 -- 172.21.15.39:0/3481149190 learned_addr learned my addr 172.21.15.39:0/3481149190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:08.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa41be00640 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb69390 0x55de0ff2c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.590+0000 7fa423200640 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa422800640 1 -- 172.21.15.39:0/3481149190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 msgr2=0x55de0ff461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa422800640 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff461b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa422800640 1 -- 172.21.15.39:0/3481149190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb69390 msgr2=0x55de0ff2c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa422800640 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb69390 0x55de0ff2c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa422800640 1 -- 172.21.15.39:0/3481149190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de0ff27620 con 0x55de0fb611b0 2024-07-04T08:20:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa423200640 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa422800640 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2b900 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa4140a6f70 tx=0x7fa4140e97f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935ab9a8053d0541 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa413400640 1 -- 172.21.15.39:0/3481149190 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a5670 con 0x55de0fb611b0 2024-07-04T08:20:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa413400640 1 -- 172.21.15.39:0/3481149190 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa41409c030 con 0x55de0fb611b0 2024-07-04T08:20:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.591+0000 7fa413400640 1 -- 172.21.15.39:0/3481149190 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a59f0 con 0x55de0fb611b0 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.592+0000 7fa42fa31740 1 -- 172.21.15.39:0/3481149190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 msgr2=0x55de0ff2b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.592+0000 7fa42fa31740 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2b900 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa4140a6f70 tx=0x7fa4140e97f0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.592+0000 7fa42fa31740 1 -- 172.21.15.39:0/3481149190 shutdown_connections 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.592+0000 7fa42fa31740 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb69390 0x55de0ff2c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.592+0000 7fa42fa31740 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.592+0000 7fa42fa31740 1 --2- 172.21.15.39:0/3481149190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb611b0 0x55de0ff2b900 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.592+0000 7fa42fa31740 1 -- 172.21.15.39:0/3481149190 >> 172.21.15.39:0/3481149190 conn(0x55de0fb7e460 msgr2=0x55de0ff2bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.593+0000 7fa42fa31740 1 -- 172.21.15.39:0/3481149190 shutdown_connections 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.593+0000 7fa42fa31740 1 -- 172.21.15.39:0/3481149190 wait complete. 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.593+0000 7fa42fa31740 1 -- start start 2024-07-04T08:20:08.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.593+0000 7fa42fa31740 1 Processor -- start 2024-07-04T08:20:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.594+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff4f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.594+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb69390 0x55de0ff4acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.594+0000 7fa42fa31740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff4b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.594+0000 7fa42fa31740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241b3830 con 0x55de0fb611b0 2024-07-04T08:20:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.594+0000 7fa42fa31740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241b3160 con 0x55de0fb8a070 2024-07-04T08:20:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.594+0000 7fa42fa31740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241b03a0 con 0x55de0fb69390 2024-07-04T08:20:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.594+0000 7fa42fa31740 10 client.0 _ll_register_callbacks cb 0x7fa40c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:08.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.594+0000 7fa422800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff4f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa422800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff4f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48128/0 (socket says 172.21.15.39:48128) 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa422800640 1 -- 172.21.15.39:0/4262431017 learned_addr learned my addr 172.21.15.39:0/4262431017 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa41be00640 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb69390 0x55de0ff4acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa423200640 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff4b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa422800640 1 -- 172.21.15.39:0/4262431017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb69390 msgr2=0x55de0ff4acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa422800640 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb69390 0x55de0ff4acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa422800640 1 -- 172.21.15.39:0/4262431017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 msgr2=0x55de0ff4b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa422800640 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff4b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa422800640 1 -- 172.21.15.39:0/4262431017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de0ff4baa0 con 0x55de0fb611b0 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.595+0000 7fa423200640 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff4b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.596+0000 7fa422800640 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff4f810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa414000c00 tx=0x7fa4140a9ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d45d5b4328c89f3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:08.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.596+0000 7fa410c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:20:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.596+0000 7fa410c00640 1 -- 172.21.15.39:0/4262431017 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a5bc0 con 0x55de0fb611b0 2024-07-04T08:20:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.596+0000 7fa410c00640 1 -- 172.21.15.39:0/4262431017 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa414005270 con 0x55de0fb611b0 2024-07-04T08:20:08.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.596+0000 7fa42fa31740 1 -- 172.21.15.39:0/4262431017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa41409c030 con 0x55de0fb611b0 2024-07-04T08:20:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.596+0000 7fa410c00640 1 -- 172.21.15.39:0/4262431017 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4140a6350 con 0x55de0fb611b0 2024-07-04T08:20:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.596+0000 7fa42fa31740 10 client.16560 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.596+0000 7fa42fa31740 1 -- 172.21.15.39:0/4262431017 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55de0ff4baa0 con 0x55de0fb611b0 2024-07-04T08:20:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.597+0000 7fa410c00640 1 -- 172.21.15.39:0/4262431017 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa41409c030 con 0x55de0fb611b0 2024-07-04T08:20:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.597+0000 7fa410c00640 1 -- 172.21.15.39:0/4262431017 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa4140c4240 con 0x55de0fb611b0 2024-07-04T08:20:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.597+0000 7fa42fa31740 10 client.16560 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.597+0000 7fa42fa31740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.597+0000 7fa42fa31740 20 client.16560 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:08.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.597+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55de0fb1dda0 0x55de0ffb6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.597+0000 7fa42fa31740 4 client.16560 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:20:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.597+0000 7fa42fa31740 1 -- 172.21.15.39:0/4262431017 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55de0fea2890 con 0x55de0fb1dda0 2024-07-04T08:20:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.598+0000 7fa41be00640 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55de0fb1dda0 0x55de0ffb6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.598+0000 7fa41be00640 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55de0fb1dda0 0x55de0ffb6800 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca242a1a70031aff server_cookie=2ed3a04dd434f3e1 in_seq=0 out_seq=0 2024-07-04T08:20:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.598+0000 7fa410c00640 10 client.16560 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa410c00640 1 -- 172.21.15.39:0/4262431017 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55de0fea2890 con 0x55de0fb1dda0 2024-07-04T08:20:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa410c00640 10 client.16560 handle_command_reply: tid=0 2024-07-04T08:20:08.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 client.16560 shutdown 2024-07-04T08:20:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 -- 172.21.15.39:0/4262431017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 msgr2=0x55de0ff4f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff4f810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa414000c00 tx=0x7fa4140a9ad0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 -- 172.21.15.39:0/4262431017 shutdown_connections 2024-07-04T08:20:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55de0fb1dda0 0x55de0ffb6800 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55de0fb8a070 0x55de0ff4b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55de0fb69390 0x55de0ff4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 --2- 172.21.15.39:0/4262431017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55de0fb611b0 0x55de0ff4f810 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 -- 172.21.15.39:0/4262431017 >> 172.21.15.39:0/4262431017 conn(0x55de0fb7e460 msgr2=0x55de0ff2f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa410c00640 0 client.16560 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.599+0000 7fa42fa31740 1 -- 172.21.15.39:0/4262431017 shutdown_connections 2024-07-04T08:20:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.600+0000 7fa42fa31740 1 -- 172.21.15.39:0/4262431017 wait complete. 2024-07-04T08:20:08.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.600+0000 7fa42fa31740 20 client.16560 trim_cache size 0 max 16384 2024-07-04T08:20:08.599 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:20:08.599 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:20:08.599 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:20:08.599 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:20:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.600+0000 7fa429400640 1 -- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa4080b2960 msgr2=0x7fa4080b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.600+0000 7fa429400640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa4080b2960 0x7fa4080b4e70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa41c006010 tx=0x7fa41c004a40 comp rx=0 tx=0).stop 2024-07-04T08:20:08.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.600+0000 7fa429400640 1 -- 172.21.15.39:0/1936470412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa424105300 msgr2=0x7fa42419bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.600+0000 7fa429400640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa424105300 0x7fa42419bbb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e9c0 tx=0x7fa41400ee80 comp rx=0 tx=0).stop 2024-07-04T08:20:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.601+0000 7fa429400640 1 -- 172.21.15.39:0/1936470412 shutdown_connections 2024-07-04T08:20:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.601+0000 7fa429400640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa4080b2960 0x7fa4080b4e70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.601+0000 7fa429400640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa42410b590 0x7fa4241a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.601+0000 7fa429400640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa42410ac40 0x7fa42419c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.601+0000 7fa429400640 1 --2- 172.21.15.39:0/1936470412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa424105300 0x7fa42419bbb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.601+0000 7fa429400640 1 -- 172.21.15.39:0/1936470412 >> 172.21.15.39:0/1936470412 conn(0x7fa424100d30 msgr2=0x7fa42410a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.601+0000 7fa429400640 1 -- 172.21.15.39:0/1936470412 shutdown_connections 2024-07-04T08:20:08.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:08.601+0000 7fa429400640 1 -- 172.21.15.39:0/1936470412 wait complete. 2024-07-04T08:20:09.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:09 smithi039 ceph-mon[32062]: pgmap v628: 129 pgs: 129 active+clean; 4.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 64 KiB/s wr, 32 op/s 2024-07-04T08:20:09.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:09 smithi050 ceph-mon[36577]: pgmap v628: 129 pgs: 129 active+clean; 4.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 64 KiB/s wr, 32 op/s 2024-07-04T08:20:09.473 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:20:09.473 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:20:09.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:09 smithi133 ceph-mon[37050]: pgmap v628: 129 pgs: 129 active+clean; 4.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 64 KiB/s wr, 32 op/s 2024-07-04T08:20:09.962 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:11.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:11 smithi039 ceph-mon[32062]: pgmap v629: 129 pgs: 129 active+clean; 4.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 52 KiB/s wr, 28 op/s 2024-07-04T08:20:11.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:11 smithi050 ceph-mon[36577]: pgmap v629: 129 pgs: 129 active+clean; 4.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 52 KiB/s wr, 28 op/s 2024-07-04T08:20:11.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:11 smithi133 ceph-mon[37050]: pgmap v629: 129 pgs: 129 active+clean; 4.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 52 KiB/s wr, 28 op/s 2024-07-04T08:20:12.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.266+0000 7f23d5c00640 1 -- 172.21.15.39:0/4285090113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d010ee50 msgr2=0x7f23d010f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:12.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.266+0000 7f23d5c00640 1 --2- 172.21.15.39:0/4285090113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d010ee50 0x7f23d010f260 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009900 tx=0x7f23bc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.266+0000 7f23d5c00640 1 -- 172.21.15.39:0/4285090113 shutdown_connections 2024-07-04T08:20:12.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.266+0000 7f23d5c00640 1 --2- 172.21.15.39:0/4285090113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d010ee50 0x7f23d010f260 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.266+0000 7f23d5c00640 1 --2- 172.21.15.39:0/4285090113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23d010f7a0 0x7f23d0115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.266+0000 7f23d5c00640 1 --2- 172.21.15.39:0/4285090113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d0109510 0x7f23d0109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.266+0000 7f23d5c00640 1 -- 172.21.15.39:0/4285090113 >> 172.21.15.39:0/4285090113 conn(0x7f23d0078b10 msgr2=0x7f23d0107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:12.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.266+0000 7f23d5c00640 1 -- 172.21.15.39:0/4285090113 shutdown_connections 2024-07-04T08:20:12.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.266+0000 7f23d5c00640 1 -- 172.21.15.39:0/4285090113 wait complete. 2024-07-04T08:20:12.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.267+0000 7f23d5c00640 1 Processor -- start 2024-07-04T08:20:12.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.268+0000 7f23d5c00640 1 -- start start 2024-07-04T08:20:12.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.268+0000 7f23d5c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23d0109510 0x7f23d019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:12.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.268+0000 7f23d5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d010ee50 0x7f23d01a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:12.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.268+0000 7f23d5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d010f7a0 0x7f23d01a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:12.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.268+0000 7f23d5c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f23d0119460 con 0x7f23d010f7a0 2024-07-04T08:20:12.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.268+0000 7f23d5c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f23d01192e0 con 0x7f23d0109510 2024-07-04T08:20:12.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.268+0000 7f23d5c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23d01195e0 con 0x7f23d010ee50 2024-07-04T08:20:12.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d010ee50 0x7f23d01a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:12.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23d0109510 0x7f23d019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:12.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d010ee50 0x7f23d01a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33574/0 (socket says 172.21.15.39:33574) 2024-07-04T08:20:12.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cf400640 1 -- 172.21.15.39:0/1567054687 learned_addr learned my addr 172.21.15.39:0/1567054687 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:12.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23d5200640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d010f7a0 0x7f23d01a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:12.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cfe00640 1 -- 172.21.15.39:0/1567054687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d010ee50 msgr2=0x7f23d01a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:12.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cfe00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d010ee50 0x7f23d01a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cfe00640 1 -- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d010f7a0 msgr2=0x7f23d01a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:12.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cfe00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d010f7a0 0x7f23d01a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cfe00640 1 -- 172.21.15.39:0/1567054687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d01a5de0 con 0x7f23d0109510 2024-07-04T08:20:12.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.269+0000 7f23cf400640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d010ee50 0x7f23d01a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:12.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.270+0000 7f23cfe00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23d0109510 0x7f23d019fe70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f23c000e9f0 tx=0x7f23c000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:12.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.270+0000 7f23ccc00640 1 -- 172.21.15.39:0/1567054687 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c000ccc0 con 0x7f23d0109510 2024-07-04T08:20:12.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.270+0000 7f23ccc00640 1 -- 172.21.15.39:0/1567054687 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23c00108e0 con 0x7f23d0109510 2024-07-04T08:20:12.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.270+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d01a60d0 con 0x7f23d0109510 2024-07-04T08:20:12.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.270+0000 7f23ccc00640 1 -- 172.21.15.39:0/1567054687 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0018880 con 0x7f23d0109510 2024-07-04T08:20:12.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.270+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d01ac830 con 0x7f23d0109510 2024-07-04T08:20:12.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.271+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d0109920 con 0x7f23d0109510 2024-07-04T08:20:12.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.274+0000 7f23ccc00640 1 -- 172.21.15.39:0/1567054687 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f23c0010430 con 0x7f23d0109510 2024-07-04T08:20:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.275+0000 7f23ccc00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23b40b2960 0x7f23b40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.276+0000 7f23ccc00640 1 -- 172.21.15.39:0/1567054687 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f23c00e2cd0 con 0x7f23d0109510 2024-07-04T08:20:12.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.276+0000 7f23cf400640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23b40b2960 0x7f23b40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:12.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.277+0000 7f23cf400640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23b40b2960 0x7f23b40b4e70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f23bc006010 tx=0x7f23bc005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:12.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.280+0000 7f23ccc00640 1 -- 172.21.15.39:0/1567054687 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f23c00a9cd0 con 0x7f23d0109510 2024-07-04T08:20:12.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.541+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23d01a0fa0 con 0x7f23d0109510 2024-07-04T08:20:12.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.543+0000 7f23ccc00640 1 -- 172.21.15.39:0/1567054687 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f23c00adfd0 con 0x7f23d0109510 2024-07-04T08:20:12.541 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:20:12.541 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:20:12.541 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:20:12.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23b40b2960 msgr2=0x7f23b40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:12.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23b40b2960 0x7f23b40b4e70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f23bc006010 tx=0x7f23bc005a20 comp rx=0 tx=0).stop 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23d0109510 msgr2=0x7f23d019fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23d0109510 0x7f23d019fe70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f23c000e9f0 tx=0x7f23c000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 shutdown_connections 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23b40b2960 0x7f23b40b4e70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d010f7a0 0x7f23d01a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23d010ee50 0x7f23d01a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 --2- 172.21.15.39:0/1567054687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23d0109510 0x7f23d019fe70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 >> 172.21.15.39:0/1567054687 conn(0x7f23d0078b10 msgr2=0x7f23d00792e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.546+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 shutdown_connections 2024-07-04T08:20:12.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:12.547+0000 7f23d5c00640 1 -- 172.21.15.39:0/1567054687 wait complete. 2024-07-04T08:20:13.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:13 smithi039 ceph-mon[32062]: pgmap v630: 129 pgs: 129 active+clean; 4.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-07-04T08:20:13.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:13 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1567054687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:20:13.365 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:13 smithi133 ceph-mon[37050]: pgmap v630: 129 pgs: 129 active+clean; 4.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-07-04T08:20:13.365 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:13 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1567054687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:20:13.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:13 smithi050 ceph-mon[36577]: pgmap v630: 129 pgs: 129 active+clean; 4.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 52 KiB/s wr, 30 op/s 2024-07-04T08:20:13.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:13 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1567054687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:20:13.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:20:13.994 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:15.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:15 smithi050 ceph-mon[36577]: pgmap v631: 129 pgs: 129 active+clean; 4.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 69 KiB/s wr, 32 op/s 2024-07-04T08:20:15.426 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:15 smithi039 ceph-mon[32062]: pgmap v631: 129 pgs: 129 active+clean; 4.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 69 KiB/s wr, 32 op/s 2024-07-04T08:20:15.563 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:15 smithi133 ceph-mon[37050]: pgmap v631: 129 pgs: 129 active+clean; 4.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 69 KiB/s wr, 32 op/s 2024-07-04T08:20:16.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.295+0000 7f70a0a00640 1 -- 172.21.15.39:0/1577031196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 msgr2=0x7f709c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:16.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.295+0000 7f70a0a00640 1 --2- 172.21.15.39:0/1577031196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 0x7f709c107b10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f708c009900 tx=0x7f708c02a410 comp rx=0 tx=0).stop 2024-07-04T08:20:16.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.295+0000 7f70a0a00640 1 -- 172.21.15.39:0/1577031196 shutdown_connections 2024-07-04T08:20:16.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.295+0000 7f70a0a00640 1 --2- 172.21.15.39:0/1577031196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 0x7f709c107b10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.295+0000 7f70a0a00640 1 --2- 172.21.15.39:0/1577031196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f709c108050 0x7f709c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.295+0000 7f70a0a00640 1 --2- 172.21.15.39:0/1577031196 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f709c101dc0 0x7f709c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.295+0000 7f70a0a00640 1 -- 172.21.15.39:0/1577031196 >> 172.21.15.39:0/1577031196 conn(0x7f709c0fd810 msgr2=0x7f709c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:16.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.296+0000 7f70a0a00640 1 -- 172.21.15.39:0/1577031196 shutdown_connections 2024-07-04T08:20:16.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.296+0000 7f70a0a00640 1 -- 172.21.15.39:0/1577031196 wait complete. 2024-07-04T08:20:16.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.297+0000 7f70a0a00640 1 Processor -- start 2024-07-04T08:20:16.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.297+0000 7f70a0a00640 1 -- start start 2024-07-04T08:20:16.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f70a0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f709c101dc0 0x7f709c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f70a0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 0x7f709c19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f70a0a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f709c108050 0x7f709c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f70a0a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f709c111d10 con 0x7f709c107700 2024-07-04T08:20:16.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f70a0a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f709c111b90 con 0x7f709c101dc0 2024-07-04T08:20:16.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f70a0a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f709c111e90 con 0x7f709c108050 2024-07-04T08:20:16.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f709b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f709c101dc0 0x7f709c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f709aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 0x7f709c19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f709be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f709c108050 0x7f709c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f709b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f709c101dc0 0x7f709c19ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52606/0 (socket says 172.21.15.39:52606) 2024-07-04T08:20:16.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f709b400640 1 -- 172.21.15.39:0/926782355 learned_addr learned my addr 172.21.15.39:0/926782355 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:16.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.298+0000 7f709aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 0x7f709c19bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39044/0 (socket says 172.21.15.39:39044) 2024-07-04T08:20:16.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.299+0000 7f709be00640 1 -- 172.21.15.39:0/926782355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f709c101dc0 msgr2=0x7f709c19ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:16.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.299+0000 7f709be00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f709c101dc0 0x7f709c19ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.299+0000 7f709be00640 1 -- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 msgr2=0x7f709c19bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:16.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.299+0000 7f709be00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 0x7f709c19bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.299+0000 7f709be00640 1 -- 172.21.15.39:0/926782355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c1a19a0 con 0x7f709c108050 2024-07-04T08:20:16.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.299+0000 7f709aa00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 0x7f709c19bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:16.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.299+0000 7f709be00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f709c108050 0x7f709c1a1270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7094009fd0 tx=0x7f709400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:16.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.300+0000 7f7093e00640 1 -- 172.21.15.39:0/926782355 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709400ce20 con 0x7f709c108050 2024-07-04T08:20:16.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.300+0000 7f70a0a00640 1 -- 172.21.15.39:0/926782355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709c1a1c90 con 0x7f709c108050 2024-07-04T08:20:16.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.300+0000 7f7093e00640 1 -- 172.21.15.39:0/926782355 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7094005020 con 0x7f709c108050 2024-07-04T08:20:16.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.300+0000 7f7093e00640 1 -- 172.21.15.39:0/926782355 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7094010690 con 0x7f709c108050 2024-07-04T08:20:16.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.300+0000 7f70a0a00640 1 -- 172.21.15.39:0/926782355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709c1a83f0 con 0x7f709c108050 2024-07-04T08:20:16.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.303+0000 7f70a8542740 1 Processor -- start 2024-07-04T08:20:16.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.304+0000 7f7093e00640 1 -- 172.21.15.39:0/926782355 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f70940054d0 con 0x7f709c108050 2024-07-04T08:20:16.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.304+0000 7f70a8542740 1 -- start start 2024-07-04T08:20:16.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.304+0000 7f70a8542740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f74070 0x558548318970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.304+0000 7f70a8542740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548318eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.304+0000 7f70a8542740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 0x5585483193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.304+0000 7f70a8542740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55854828ef00 con 0x558547f4b1b0 2024-07-04T08:20:16.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.304+0000 7f70a8542740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55854827e4a0 con 0x558547f74070 2024-07-04T08:20:16.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.304+0000 7f70a8542740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5585482824e0 con 0x558547f53390 2024-07-04T08:20:16.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f74070 0x558548318970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548318eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f74070 0x558548318970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52618/0 (socket says 172.21.15.39:52618) 2024-07-04T08:20:16.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709aa00640 1 -- 172.21.15.39:0/587528656 learned_addr learned my addr 172.21.15.39:0/587528656 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:16.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709aa00640 1 --2- 172.21.15.39:0/587528656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 0x5585483193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709aa00640 1 -- 172.21.15.39:0/587528656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 msgr2=0x5585483193f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:16.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709aa00640 1 --2- 172.21.15.39:0/587528656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 0x5585483193f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709aa00640 1 -- 172.21.15.39:0/587528656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 msgr2=0x558548318eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:16.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709aa00640 1 --2- 172.21.15.39:0/587528656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548318eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.305+0000 7f709aa00640 1 -- 172.21.15.39:0/587528656 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558548216670 con 0x558547f74070 2024-07-04T08:20:16.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.306+0000 7f709aa00640 1 --2- 172.21.15.39:0/587528656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f74070 0x558548318970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f708c0057c0 tx=0x7f708c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68b36fa8960333d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:16.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.306+0000 7f7093e00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70800b2960 0x7f70800b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.306+0000 7f7090c00640 1 -- 172.21.15.39:0/587528656 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c035070 con 0x558547f74070 2024-07-04T08:20:16.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.306+0000 7f7093e00640 1 -- 172.21.15.39:0/926782355 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f70940e2fe0 con 0x7f709c108050 2024-07-04T08:20:16.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.306+0000 7f7090c00640 1 -- 172.21.15.39:0/587528656 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f708c0054d0 con 0x558547f74070 2024-07-04T08:20:16.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.306+0000 7f709b400640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70800b2960 0x7f70800b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.306+0000 7f7090c00640 1 -- 172.21.15.39:0/587528656 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c004b70 con 0x558547f74070 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f70a8542740 1 -- 172.21.15.39:0/587528656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f74070 msgr2=0x558548318970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f70a8542740 1 --2- 172.21.15.39:0/587528656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f74070 0x558548318970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f708c0057c0 tx=0x7f708c000bc0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f709b400640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70800b2960 0x7f70800b4e70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5585483175f0 tx=0x7f7084005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f70a8542740 1 -- 172.21.15.39:0/587528656 shutdown_connections 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f70a8542740 1 --2- 172.21.15.39:0/587528656 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548318eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f70a8542740 1 --2- 172.21.15.39:0/587528656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 0x5585483193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f70a8542740 1 --2- 172.21.15.39:0/587528656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f74070 0x558548318970 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f70a8542740 1 -- 172.21.15.39:0/587528656 >> 172.21.15.39:0/587528656 conn(0x558547f68460 msgr2=0x558548316860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f70a8542740 1 -- 172.21.15.39:0/587528656 shutdown_connections 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.307+0000 7f70a8542740 1 -- 172.21.15.39:0/587528656 wait complete. 2024-07-04T08:20:16.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.308+0000 7f70a8542740 1 -- start start 2024-07-04T08:20:16.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.308+0000 7f70a8542740 1 Processor -- start 2024-07-04T08:20:16.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f70a8542740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548324590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f70a8542740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 0x558548324ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f70a8542740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548330410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f70a8542740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5585480e2700 con 0x558547f4b1b0 2024-07-04T08:20:16.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f70a8542740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558548225bd0 con 0x558547f53390 2024-07-04T08:20:16.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f70a8542740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5585483270f0 con 0x558547f74070 2024-07-04T08:20:16.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f70a8542740 10 client.0 _ll_register_callbacks cb 0x558548277590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:16.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f709be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548330410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f709b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548324590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f709aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 0x558548324ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f709be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548330410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33628/0 (socket says 172.21.15.39:33628) 2024-07-04T08:20:16.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.309+0000 7f709be00640 1 -- 172.21.15.39:0/1321232760 learned_addr learned my addr 172.21.15.39:0/1321232760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:16.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f709be00640 1 -- 172.21.15.39:0/1321232760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 msgr2=0x558548324ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f709be00640 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 0x558548324ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f709be00640 1 -- 172.21.15.39:0/1321232760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 msgr2=0x558548324590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f709be00640 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548324590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f709be00640 1 -- 172.21.15.39:0/1321232760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558547fdf650 con 0x558547f74070 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f709b400640 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548324590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f709be00640 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548330410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f70940a9f20 tx=0x7f70940aa750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ac08ca743b8c9d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f7089600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f7089600640 1 -- 172.21.15.39:0/1321232760 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70940b2df0 con 0x558547f74070 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f7089600640 1 -- 172.21.15.39:0/1321232760 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f70940b2960 con 0x558547f74070 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.310+0000 7f70a8542740 1 -- 172.21.15.39:0/1321232760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f708c0054d0 con 0x558547f74070 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f7089600640 1 -- 172.21.15.39:0/1321232760 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70940b2b60 con 0x558547f74070 2024-07-04T08:20:16.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f70a8542740 10 client.26623 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f70a8542740 1 -- 172.21.15.39:0/1321232760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558548333110 con 0x558547f74070 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f7089600640 1 -- 172.21.15.39:0/1321232760 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f708c0054d0 con 0x558547f74070 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f7089600640 1 -- 172.21.15.39:0/1321232760 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f70940a5990 con 0x558547f74070 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f70a8542740 10 client.26623 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f70a8542740 10 client.26623 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f70a8542740 20 client.26623 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f70a8542740 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x558547f329f0 0x558548336670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f70a8542740 4 client.26623 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.311+0000 7f70a8542740 1 -- 172.21.15.39:0/1321232760 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558548290e50 con 0x558547f329f0 2024-07-04T08:20:16.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.312+0000 7f709b400640 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x558547f329f0 0x558548336670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.312+0000 7f709b400640 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x558547f329f0 0x558548336670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2ab38c3e38ab766 server_cookie=4b7bee8b3ce56934 in_seq=0 out_seq=0 2024-07-04T08:20:16.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.312+0000 7f7089600640 10 client.26623 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:20:16.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.313+0000 7f7089600640 1 -- 172.21.15.39:0/1321232760 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558548290e50 con 0x558547f329f0 2024-07-04T08:20:16.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.313+0000 7f7089600640 10 client.26623 handle_command_reply: tid=0 2024-07-04T08:20:16.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.313+0000 7f70a8542740 1 client.26623 shutdown 2024-07-04T08:20:16.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 -- 172.21.15.39:0/1321232760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 msgr2=0x558548330410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:16.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548330410 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f70940a9f20 tx=0x7f70940aa750 comp rx=0 tx=0).stop 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 -- 172.21.15.39:0/1321232760 shutdown_connections 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x558547f329f0 0x558548336670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548330410 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 0x558548324ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 --2- 172.21.15.39:0/1321232760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548324590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f7089600640 0 client.26623 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 -- 172.21.15.39:0/1321232760 >> 172.21.15.39:0/1321232760 conn(0x558547f68460 msgr2=0x55854831bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 -- 172.21.15.39:0/1321232760 shutdown_connections 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 1 -- 172.21.15.39:0/1321232760 wait complete. 2024-07-04T08:20:16.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.314+0000 7f70a8542740 20 client.26623 trim_cache size 0 max 16384 2024-07-04T08:20:16.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.329+0000 7f70a8542740 1 Processor -- start 2024-07-04T08:20:16.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.329+0000 7f70a8542740 1 -- start start 2024-07-04T08:20:16.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f70a8542740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f4b1b0 0x558548325fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f70a8542740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 0x55854833be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f70a8542740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f74070 0x55854833c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f70a8542740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1b2160 con 0x558547f74070 2024-07-04T08:20:16.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f70a8542740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1b3ca0 con 0x558547f4b1b0 2024-07-04T08:20:16.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f70a8542740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1b2830 con 0x558547f53390 2024-07-04T08:20:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f709b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f4b1b0 0x558548325fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f709aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 0x55854833be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f709b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f4b1b0 0x558548325fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52632/0 (socket says 172.21.15.39:52632) 2024-07-04T08:20:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f709b400640 1 -- 172.21.15.39:0/1055625268 learned_addr learned my addr 172.21.15.39:0/1055625268 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.330+0000 7f709be00640 1 --2- 172.21.15.39:0/1055625268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f74070 0x55854833c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.331+0000 7f709b400640 1 -- 172.21.15.39:0/1055625268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 msgr2=0x55854833be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.331+0000 7f709b400640 1 --2- 172.21.15.39:0/1055625268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 0x55854833be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.331+0000 7f709b400640 1 -- 172.21.15.39:0/1055625268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f74070 msgr2=0x55854833c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.331+0000 7f709b400640 1 --2- 172.21.15.39:0/1055625268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f74070 0x55854833c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.331+0000 7f709b400640 1 -- 172.21.15.39:0/1055625268 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558548325650 con 0x558547f4b1b0 2024-07-04T08:20:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.331+0000 7f709b400640 1 --2- 172.21.15.39:0/1055625268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f4b1b0 0x558548325fc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x558548336dd0 tx=0x7f708401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e948da1a36ae1af1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.332+0000 7f708be00640 1 -- 172.21.15.39:0/1055625268 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084005020 con 0x558547f4b1b0 2024-07-04T08:20:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.332+0000 7f708be00640 1 -- 172.21.15.39:0/1055625268 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70840047a0 con 0x558547f4b1b0 2024-07-04T08:20:16.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.332+0000 7f708be00640 1 -- 172.21.15.39:0/1055625268 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708402b740 con 0x558547f4b1b0 2024-07-04T08:20:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.333+0000 7f70a8542740 1 -- 172.21.15.39:0/1055625268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f4b1b0 msgr2=0x558548325fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.333+0000 7f70a8542740 1 --2- 172.21.15.39:0/1055625268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f4b1b0 0x558548325fc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x558548336dd0 tx=0x7f708401b000 comp rx=0 tx=0).stop 2024-07-04T08:20:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.333+0000 7f70a8542740 1 -- 172.21.15.39:0/1055625268 shutdown_connections 2024-07-04T08:20:16.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.333+0000 7f70a8542740 1 --2- 172.21.15.39:0/1055625268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f74070 0x55854833c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.333+0000 7f70a8542740 1 --2- 172.21.15.39:0/1055625268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f53390 0x55854833be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.333+0000 7f70a8542740 1 --2- 172.21.15.39:0/1055625268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f4b1b0 0x558548325fc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.333+0000 7f70a8542740 1 -- 172.21.15.39:0/1055625268 >> 172.21.15.39:0/1055625268 conn(0x558547f68460 msgr2=0x5585483258b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.333+0000 7f70a8542740 1 -- 172.21.15.39:0/1055625268 shutdown_connections 2024-07-04T08:20:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.333+0000 7f70a8542740 1 -- 172.21.15.39:0/1055625268 wait complete. 2024-07-04T08:20:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.334+0000 7f70a8542740 1 -- start start 2024-07-04T08:20:16.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.334+0000 7f70a8542740 1 Processor -- start 2024-07-04T08:20:16.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f70a8542740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548328440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f70a8542740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 0x558548328980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f70a8542740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548328ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f70a8542740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1b35d0 con 0x558547f4b1b0 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f70a8542740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1b2f00 con 0x558547f53390 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f70a8542740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f709c1b0140 con 0x558547f74070 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f70a8542740 10 client.0 _ll_register_callbacks cb 0x7f70940a5e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f709b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548328440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f709b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548328440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39074/0 (socket says 172.21.15.39:39074) 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.335+0000 7f709b400640 1 -- 172.21.15.39:0/2648164433 learned_addr learned my addr 172.21.15.39:0/2648164433 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f709aa00640 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 0x558548328980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f709be00640 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548328ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f709b400640 1 -- 172.21.15.39:0/2648164433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 msgr2=0x558548328ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f709b400640 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548328ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f709b400640 1 -- 172.21.15.39:0/2648164433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 msgr2=0x558548328980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f709b400640 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 0x558548328980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f709b400640 1 -- 172.21.15.39:0/2648164433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55854839e000 con 0x558547f4b1b0 2024-07-04T08:20:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f709aa00640 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 0x558548328980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f709b400640 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548328440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f70840044b0 tx=0x7f70840044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7aa975602e781b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:16.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.336+0000 7f7089600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.337+0000 7f7089600640 1 -- 172.21.15.39:0/2648164433 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708402bb60 con 0x558547f4b1b0 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.337+0000 7f70a8542740 1 -- 172.21.15.39:0/2648164433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70840047a0 con 0x558547f4b1b0 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.337+0000 7f7089600640 1 -- 172.21.15.39:0/2648164433 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f708401f4f0 con 0x558547f4b1b0 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.337+0000 7f7089600640 1 -- 172.21.15.39:0/2648164433 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7084049050 con 0x558547f4b1b0 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.337+0000 7f70a8542740 10 client.16584 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.337+0000 7f70a8542740 1 -- 172.21.15.39:0/2648164433 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55854839f280 con 0x558547f4b1b0 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.338+0000 7f7089600640 1 -- 172.21.15.39:0/2648164433 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70840047a0 con 0x558547f4b1b0 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.338+0000 7f7089600640 1 -- 172.21.15.39:0/2648164433 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f70840407a0 con 0x558547f4b1b0 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.338+0000 7f70a8542740 10 client.16584 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.338+0000 7f70a8542740 10 client.16584 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.338+0000 7f70a8542740 20 client.16584 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.338+0000 7f70a8542740 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x558547f07da0 0x5585483a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:16.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.338+0000 7f70a8542740 4 client.16584 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:20:16.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.338+0000 7f70a8542740 1 -- 172.21.15.39:0/2648164433 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55854828c8f0 con 0x558547f07da0 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.338+0000 7f709aa00640 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x558547f07da0 0x5585483a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.339+0000 7f7089600640 10 client.16584 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.339+0000 7f709aa00640 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x558547f07da0 0x5585483a19f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=af8c01b0e358d16c server_cookie=e0283585df068127 in_seq=0 out_seq=0 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.339+0000 7f7089600640 1 -- 172.21.15.39:0/2648164433 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55854828c8f0 con 0x558547f07da0 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.339+0000 7f7089600640 10 client.16584 handle_command_reply: tid=0 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.339+0000 7f70a8542740 1 client.16584 shutdown 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 -- 172.21.15.39:0/2648164433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 msgr2=0x558548328440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548328440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f70840044b0 tx=0x7f70840044e0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 -- 172.21.15.39:0/2648164433 shutdown_connections 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x558547f07da0 0x5585483a19f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558547f74070 0x558548328ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558547f53390 0x558548328980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 --2- 172.21.15.39:0/2648164433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558547f4b1b0 0x558548328440 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f7089600640 0 client.16584 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:20:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 -- 172.21.15.39:0/2648164433 >> 172.21.15.39:0/2648164433 conn(0x558547f68460 msgr2=0x558548325820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 -- 172.21.15.39:0/2648164433 shutdown_connections 2024-07-04T08:20:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 1 -- 172.21.15.39:0/2648164433 wait complete. 2024-07-04T08:20:16.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.340+0000 7f70a8542740 20 client.16584 trim_cache size 0 max 16384 2024-07-04T08:20:16.340 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 -- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70800b2960 msgr2=0x7f70800b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70800b2960 0x7f70800b4e70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x5585483175f0 tx=0x7f7084005f90 comp rx=0 tx=0).stop 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 -- 172.21.15.39:0/926782355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f709c108050 msgr2=0x7f709c1a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f709c108050 0x7f709c1a1270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7094009fd0 tx=0x7f709400ef10 comp rx=0 tx=0).stop 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 -- 172.21.15.39:0/926782355 shutdown_connections 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70800b2960 0x7f70800b4e70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f709c107700 0x7f709c19bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f709c108050 0x7f709c1a1270 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 --2- 172.21.15.39:0/926782355 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f709c101dc0 0x7f709c19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.341+0000 7f70a0a00640 1 -- 172.21.15.39:0/926782355 >> 172.21.15.39:0/926782355 conn(0x7f709c0fd810 msgr2=0x7f709c104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:16.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.342+0000 7f70a0a00640 1 -- 172.21.15.39:0/926782355 shutdown_connections 2024-07-04T08:20:16.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:16.342+0000 7f70a0a00640 1 -- 172.21.15.39:0/926782355 wait complete. 2024-07-04T08:20:17.292 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:20:17.366 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:17 smithi039 ceph-mon[32062]: pgmap v632: 129 pgs: 129 active+clean; 4.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-07-04T08:20:17.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:17 smithi050 ceph-mon[36577]: pgmap v632: 129 pgs: 129 active+clean; 4.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-07-04T08:20:17.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:17 smithi133 ceph-mon[37050]: pgmap v632: 129 pgs: 129 active+clean; 4.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-07-04T08:20:17.762 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:19.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:19 smithi050 ceph-mon[36577]: pgmap v633: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-07-04T08:20:19.532 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:19 smithi039 ceph-mon[32062]: pgmap v633: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-07-04T08:20:19.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:19 smithi133 ceph-mon[37050]: pgmap v633: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-07-04T08:20:20.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.113+0000 7f57e9c00640 1 -- 172.21.15.39:0/6498745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57e410aa50 msgr2=0x7f57e410ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.113+0000 7f57e9c00640 1 --2- 172.21.15.39:0/6498745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57e410aa50 0x7f57e410ae60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009900 tx=0x7f57d002a400 comp rx=0 tx=0).stop 2024-07-04T08:20:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.113+0000 7f57e9c00640 1 -- 172.21.15.39:0/6498745 shutdown_connections 2024-07-04T08:20:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.113+0000 7f57e9c00640 1 --2- 172.21.15.39:0/6498745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57e410b3a0 0x7f57e4111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.113+0000 7f57e9c00640 1 --2- 172.21.15.39:0/6498745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57e410aa50 0x7f57e410ae60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.113+0000 7f57e9c00640 1 --2- 172.21.15.39:0/6498745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57e4105110 0x7f57e4105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.113+0000 7f57e9c00640 1 -- 172.21.15.39:0/6498745 >> 172.21.15.39:0/6498745 conn(0x7f57e4100b80 msgr2=0x7f57e4102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.113+0000 7f57e9c00640 1 -- 172.21.15.39:0/6498745 shutdown_connections 2024-07-04T08:20:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.113+0000 7f57e9c00640 1 -- 172.21.15.39:0/6498745 wait complete. 2024-07-04T08:20:20.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.115+0000 7f57e9c00640 1 Processor -- start 2024-07-04T08:20:20.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.115+0000 7f57e9c00640 1 -- start start 2024-07-04T08:20:20.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e9c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57e4105110 0x7f57e419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e9c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57e410aa50 0x7f57e419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e9c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57e410b3a0 0x7f57e41a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e9c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4115010 con 0x7f57e410b3a0 2024-07-04T08:20:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e9c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4114e90 con 0x7f57e4105110 2024-07-04T08:20:20.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e9c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4115190 con 0x7f57e410aa50 2024-07-04T08:20:20.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57e410b3a0 0x7f57e41a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57e4105110 0x7f57e419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57e410aa50 0x7f57e419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57e410b3a0 0x7f57e41a12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39086/0 (socket says 172.21.15.39:39086) 2024-07-04T08:20:20.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.116+0000 7f57e9200640 1 -- 172.21.15.39:0/1714503098 learned_addr learned my addr 172.21.15.39:0/1714503098 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:20.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e3e00640 1 -- 172.21.15.39:0/1714503098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57e410aa50 msgr2=0x7f57e419c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:20.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e3e00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57e410aa50 0x7f57e419c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e3e00640 1 -- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57e410b3a0 msgr2=0x7f57e41a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:20.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e3e00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57e410b3a0 0x7f57e41a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e3e00640 1 -- 172.21.15.39:0/1714503098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e41a1a10 con 0x7f57e4105110 2024-07-04T08:20:20.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e9200640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57e410b3a0 0x7f57e41a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:20.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e3e00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57e4105110 0x7f57e419baa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f57d400e9f0 tx=0x7f57d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:20.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e0c00640 1 -- 172.21.15.39:0/1714503098 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d400ccc0 con 0x7f57e4105110 2024-07-04T08:20:20.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e0c00640 1 -- 172.21.15.39:0/1714503098 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d40108e0 con 0x7f57e4105110 2024-07-04T08:20:20.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.117+0000 7f57e9c00640 1 -- 172.21.15.39:0/1714503098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e41a1d00 con 0x7f57e4105110 2024-07-04T08:20:20.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.118+0000 7f57e0c00640 1 -- 172.21.15.39:0/1714503098 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4018880 con 0x7f57e4105110 2024-07-04T08:20:20.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.118+0000 7f57e9c00640 1 -- 172.21.15.39:0/1714503098 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e41a8460 con 0x7f57e4105110 2024-07-04T08:20:20.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.121+0000 7f57f1791740 1 Processor -- start 2024-07-04T08:20:20.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.121+0000 7f57f1791740 1 -- start start 2024-07-04T08:20:20.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57f1791740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644077070 0x55764441b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57f1791740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55764404e1b0 0x55764441beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57f1791740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644056390 0x55764441c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57f1791740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557644391f00 con 0x557644077070 2024-07-04T08:20:20.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57f1791740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5576443814a0 con 0x55764404e1b0 2024-07-04T08:20:20.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57f1791740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5576443854e0 con 0x557644056390 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57e9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644077070 0x55764441b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57e9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644077070 0x55764441b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39092/0 (socket says 172.21.15.39:39092) 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57e9200640 1 -- 172.21.15.39:0/4054192445 learned_addr learned my addr 172.21.15.39:0/4054192445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.122+0000 7f57e3400640 1 --2- 172.21.15.39:0/4054192445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644056390 0x55764441c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57e9200640 1 -- 172.21.15.39:0/4054192445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644056390 msgr2=0x55764441c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57e9200640 1 --2- 172.21.15.39:0/4054192445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644056390 0x55764441c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57e9200640 1 -- 172.21.15.39:0/4054192445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55764404e1b0 msgr2=0x55764441beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57e9200640 1 --2- 172.21.15.39:0/4054192445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55764404e1b0 0x55764441beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57e9200640 1 -- 172.21.15.39:0/4054192445 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557644319670 con 0x557644077070 2024-07-04T08:20:20.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57e3400640 1 --2- 172.21.15.39:0/4054192445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644056390 0x55764441c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:20.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57e9200640 1 --2- 172.21.15.39:0/4054192445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644077070 0x55764441b970 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e830 tx=0x7f57dc00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d13e0227371c722 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:20.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57d9600640 1 -- 172.21.15.39:0/4054192445 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc004d20 con 0x557644077070 2024-07-04T08:20:20.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57d9600640 1 -- 172.21.15.39:0/4054192445 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57dc005020 con 0x557644077070 2024-07-04T08:20:20.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.123+0000 7f57d9600640 1 -- 172.21.15.39:0/4054192445 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc01a5a0 con 0x557644077070 2024-07-04T08:20:20.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.124+0000 7f57f1791740 1 -- 172.21.15.39:0/4054192445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644077070 msgr2=0x55764441b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:20.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.124+0000 7f57f1791740 1 --2- 172.21.15.39:0/4054192445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644077070 0x55764441b970 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e830 tx=0x7f57dc00ecf0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.125+0000 7f57f1791740 1 -- 172.21.15.39:0/4054192445 shutdown_connections 2024-07-04T08:20:20.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.125+0000 7f57f1791740 1 --2- 172.21.15.39:0/4054192445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644056390 0x55764441c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.125+0000 7f57f1791740 1 --2- 172.21.15.39:0/4054192445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55764404e1b0 0x55764441beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.125+0000 7f57f1791740 1 --2- 172.21.15.39:0/4054192445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644077070 0x55764441b970 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.125+0000 7f57f1791740 1 -- 172.21.15.39:0/4054192445 >> 172.21.15.39:0/4054192445 conn(0x55764406b460 msgr2=0x557644419860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:20.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.125+0000 7f57e0c00640 1 -- 172.21.15.39:0/1714503098 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f57d4010430 con 0x7f57e4105110 2024-07-04T08:20:20.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.125+0000 7f57f1791740 1 -- 172.21.15.39:0/4054192445 shutdown_connections 2024-07-04T08:20:20.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.125+0000 7f57f1791740 1 -- 172.21.15.39:0/4054192445 wait complete. 2024-07-04T08:20:20.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.126+0000 7f57f1791740 1 -- start start 2024-07-04T08:20:20.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.126+0000 7f57f1791740 1 Processor -- start 2024-07-04T08:20:20.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.127+0000 7f57f1791740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644056390 0x557644427550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.127+0000 7f57f1791740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x557644427a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.127+0000 7f57f1791740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576440359f0 0x557644433470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.127+0000 7f57f1791740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5576441e5700 con 0x557644056390 2024-07-04T08:20:20.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.127+0000 7f57f1791740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557644328bd0 con 0x5576440359f0 2024-07-04T08:20:20.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.127+0000 7f57f1791740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55764442a150 con 0x557644077070 2024-07-04T08:20:20.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.127+0000 7f57f1791740 10 client.0 _ll_register_callbacks cb 0x55764437a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.128+0000 7f57e9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644056390 0x557644427550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.128+0000 7f57e0c00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c80b2870 0x7f57c80b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.128+0000 7f57e3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576440359f0 0x557644433470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.128+0000 7f57e9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644056390 0x557644427550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39096/0 (socket says 172.21.15.39:39096) 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.128+0000 7f57e9200640 1 -- 172.21.15.39:0/142635906 learned_addr learned my addr 172.21.15.39:0/142635906 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.128+0000 7f57e3e00640 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x557644427a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.128+0000 7f57e9200640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c80b2870 0x7f57c80b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.128+0000 7f57e0c00640 1 -- 172.21.15.39:0/1714503098 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f57d40e4e50 con 0x7f57e4105110 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.129+0000 7f57e3400640 1 -- 172.21.15.39:0/142635906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 msgr2=0x557644427a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.129+0000 7f57e3400640 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x557644427a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.129+0000 7f57e3400640 1 -- 172.21.15.39:0/142635906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644056390 msgr2=0x557644427550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:20.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.129+0000 7f57e3400640 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644056390 0x557644427550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.129+0000 7f57e3400640 1 -- 172.21.15.39:0/142635906 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576440e2650 con 0x5576440359f0 2024-07-04T08:20:20.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.129+0000 7f57e3400640 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576440359f0 0x557644433470 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f57d002a920 tx=0x7f57d0032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=921a9a52d8ab7667 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:20.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.129+0000 7f57ce000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:20.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.129+0000 7f57e9200640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c80b2870 0x7f57c80b4d80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0059a0 tx=0x7f57dc009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:20.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.129+0000 7f57ce000640 1 -- 172.21.15.39:0/142635906 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0035040 con 0x5576440359f0 2024-07-04T08:20:20.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.130+0000 7f57ce000640 1 -- 172.21.15.39:0/142635906 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f57d0031760 con 0x5576440359f0 2024-07-04T08:20:20.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.130+0000 7f57f1791740 1 -- 172.21.15.39:0/142635906 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57dc005020 con 0x5576440359f0 2024-07-04T08:20:20.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.130+0000 7f57ce000640 1 -- 172.21.15.39:0/142635906 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57d0004040 con 0x5576440359f0 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.130+0000 7f57f1791740 10 client.26732 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.130+0000 7f57f1791740 1 -- 172.21.15.39:0/142635906 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557644436160 con 0x5576440359f0 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.130+0000 7f57ce000640 1 -- 172.21.15.39:0/142635906 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57dc005020 con 0x5576440359f0 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.131+0000 7f57ce000640 1 -- 172.21.15.39:0/142635906 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f57d0004750 con 0x5576440359f0 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.131+0000 7f57f1791740 10 client.26732 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.131+0000 7f57f1791740 10 client.26732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.131+0000 7f57f1791740 20 client.26732 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.131+0000 7f57f1791740 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55764400ada0 0x557644439660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.131+0000 7f57f1791740 4 client.26732 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.131+0000 7f57f1791740 1 -- 172.21.15.39:0/142635906 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557644393e50 con 0x55764400ada0 2024-07-04T08:20:20.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.131+0000 7f57e3e00640 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55764400ada0 0x557644439660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.132+0000 7f57e3e00640 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55764400ada0 0x557644439660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=950a6a7769805177 server_cookie=b44e25c59bedc1ba in_seq=0 out_seq=0 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.132+0000 7f57ce000640 10 client.26732 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57ce000640 1 -- 172.21.15.39:0/142635906 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557644393e50 con 0x55764400ada0 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57ce000640 10 client.26732 handle_command_reply: tid=0 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 client.26732 shutdown 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 -- 172.21.15.39:0/142635906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576440359f0 msgr2=0x557644433470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576440359f0 0x557644433470 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f57d002a920 tx=0x7f57d0032e90 comp rx=0 tx=0).stop 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 -- 172.21.15.39:0/142635906 shutdown_connections 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55764400ada0 0x557644439660 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576440359f0 0x557644433470 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x557644427a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57ce000640 0 client.26732 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 --2- 172.21.15.39:0/142635906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557644056390 0x557644427550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 -- 172.21.15.39:0/142635906 >> 172.21.15.39:0/142635906 conn(0x55764406b460 msgr2=0x55764441eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:20.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 -- 172.21.15.39:0/142635906 shutdown_connections 2024-07-04T08:20:20.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.133+0000 7f57f1791740 1 -- 172.21.15.39:0/142635906 wait complete. 2024-07-04T08:20:20.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.134+0000 7f57f1791740 20 client.26732 trim_cache size 0 max 16384 2024-07-04T08:20:20.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.149+0000 7f57f1791740 1 Processor -- start 2024-07-04T08:20:20.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.149+0000 7f57f1791740 1 -- start start 2024-07-04T08:20:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57f1791740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 0x557644427c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57f1791740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x557644433260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57f1791740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x5576444337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57f1791740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57e41b23d0 con 0x5576440359f0 2024-07-04T08:20:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57f1791740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57e41b3f10 con 0x557644056390 2024-07-04T08:20:20.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57f1791740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57e41b2aa0 con 0x557644077070 2024-07-04T08:20:20.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57e3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x557644433260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57e3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x557644433260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52706/0 (socket says 172.21.15.39:52706) 2024-07-04T08:20:20.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57e3400640 1 -- 172.21.15.39:0/1952626510 learned_addr learned my addr 172.21.15.39:0/1952626510 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:20.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57e9200640 1 --2- 172.21.15.39:0/1952626510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x5576444337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.150+0000 7f57e3e00640 1 --2- 172.21.15.39:0/1952626510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 0x557644427c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.151+0000 7f57e3400640 1 -- 172.21.15.39:0/1952626510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 msgr2=0x5576444337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:20.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.151+0000 7f57e3400640 1 --2- 172.21.15.39:0/1952626510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x5576444337a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.151+0000 7f57e3400640 1 -- 172.21.15.39:0/1952626510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 msgr2=0x557644427c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:20.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.151+0000 7f57e3400640 1 --2- 172.21.15.39:0/1952626510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 0x557644427c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.151+0000 7f57e3400640 1 -- 172.21.15.39:0/1952626510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55764441df80 con 0x557644056390 2024-07-04T08:20:20.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.151+0000 7f57e3400640 1 --2- 172.21.15.39:0/1952626510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x557644433260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f57d002a440 tx=0x7f57d0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ae14fb251115e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.151+0000 7f57d8c00640 1 -- 172.21.15.39:0/1952626510 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0035030 con 0x557644056390 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.152+0000 7f57d8c00640 1 -- 172.21.15.39:0/1952626510 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d00048c0 con 0x557644056390 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.152+0000 7f57d8c00640 1 -- 172.21.15.39:0/1952626510 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0003730 con 0x557644056390 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.153+0000 7f57f1791740 1 -- 172.21.15.39:0/1952626510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 msgr2=0x557644433260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.153+0000 7f57f1791740 1 --2- 172.21.15.39:0/1952626510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x557644433260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f57d002a440 tx=0x7f57d0005850 comp rx=0 tx=0).stop 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.153+0000 7f57f1791740 1 -- 172.21.15.39:0/1952626510 shutdown_connections 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.153+0000 7f57f1791740 1 --2- 172.21.15.39:0/1952626510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x5576444337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.153+0000 7f57f1791740 1 --2- 172.21.15.39:0/1952626510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x557644433260 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.153+0000 7f57f1791740 1 --2- 172.21.15.39:0/1952626510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 0x557644427c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.153+0000 7f57f1791740 1 -- 172.21.15.39:0/1952626510 >> 172.21.15.39:0/1952626510 conn(0x55764406b460 msgr2=0x557644427530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.153+0000 7f57f1791740 1 -- 172.21.15.39:0/1952626510 shutdown_connections 2024-07-04T08:20:20.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.153+0000 7f57f1791740 1 -- 172.21.15.39:0/1952626510 wait complete. 2024-07-04T08:20:20.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.154+0000 7f57f1791740 1 -- start start 2024-07-04T08:20:20.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.154+0000 7f57f1791740 1 Processor -- start 2024-07-04T08:20:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.154+0000 7f57f1791740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 0x55764442fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57f1791740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x55764442b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57f1791740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x55764442b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57f1791740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57e41b3840 con 0x5576440359f0 2024-07-04T08:20:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57f1791740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57e41b3170 con 0x557644056390 2024-07-04T08:20:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57f1791740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57e41b03b0 con 0x557644077070 2024-07-04T08:20:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57f1791740 10 client.0 _ll_register_callbacks cb 0x7f57d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:20.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57e9200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x55764442b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 0x55764442fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57e9200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x55764442b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33700/0 (socket says 172.21.15.39:33700) 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57e9200640 1 -- 172.21.15.39:0/3934541604 learned_addr learned my addr 172.21.15.39:0/3934541604 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.155+0000 7f57e3400640 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x55764442b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57e3e00640 1 -- 172.21.15.39:0/3934541604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 msgr2=0x55764442b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57e3e00640 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x55764442b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57e3e00640 1 -- 172.21.15.39:0/3934541604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 msgr2=0x55764442b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57e3e00640 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x55764442b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57e3e00640 1 -- 172.21.15.39:0/3934541604 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576444a0d90 con 0x5576440359f0 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57e3e00640 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 0x55764442fd00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f57d40e2210 tx=0x7f57d401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8228d3c2846372e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57ce000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57ce000640 1 -- 172.21.15.39:0/3934541604 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d40e6040 con 0x5576440359f0 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57ce000640 1 -- 172.21.15.39:0/3934541604 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f57d40a80e0 con 0x5576440359f0 2024-07-04T08:20:20.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.156+0000 7f57f1791740 1 -- 172.21.15.39:0/3934541604 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d00048c0 con 0x5576440359f0 2024-07-04T08:20:20.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57ce000640 1 -- 172.21.15.39:0/3934541604 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57d40e2490 con 0x5576440359f0 2024-07-04T08:20:20.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57f1791740 10 client.16602 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:20.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57f1791740 1 -- 172.21.15.39:0/3934541604 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5576444a1f20 con 0x5576440359f0 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57ce000640 1 -- 172.21.15.39:0/3934541604 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d00048c0 con 0x5576440359f0 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57ce000640 1 -- 172.21.15.39:0/3934541604 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f57d40aab20 con 0x5576440359f0 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57f1791740 10 client.16602 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57f1791740 10 client.16602 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57f1791740 20 client.16602 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57f1791740 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5576444a25f0 0x5576444a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57f1791740 4 client.16602 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.157+0000 7f57f1791740 1 -- 172.21.15.39:0/3934541604 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55764438f8f0 con 0x5576444a25f0 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.158+0000 7f57e3400640 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5576444a25f0 0x5576444a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:20.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57ce000640 10 client.16602 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:20:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57e3400640 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5576444a25f0 0x5576444a4a10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=59cb0aeddb4a5580 server_cookie=ce534749c1909bdf in_seq=0 out_seq=0 2024-07-04T08:20:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57ce000640 1 -- 172.21.15.39:0/3934541604 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55764438f8f0 con 0x5576444a25f0 2024-07-04T08:20:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57ce000640 10 client.16602 handle_command_reply: tid=0 2024-07-04T08:20:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57f1791740 1 client.16602 shutdown 2024-07-04T08:20:20.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57f1791740 1 -- 172.21.15.39:0/3934541604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 msgr2=0x55764442fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57f1791740 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 0x55764442fd00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f57d40e2210 tx=0x7f57d401f000 comp rx=0 tx=0).stop 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57f1791740 1 -- 172.21.15.39:0/3934541604 shutdown_connections 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57f1791740 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5576444a25f0 0x5576444a4a10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57f1791740 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557644077070 0x55764442b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57f1791740 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557644056390 0x55764442b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57f1791740 1 --2- 172.21.15.39:0/3934541604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576440359f0 0x55764442fd00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57ce000640 0 client.16602 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.159+0000 7f57f1791740 1 -- 172.21.15.39:0/3934541604 >> 172.21.15.39:0/3934541604 conn(0x55764406b460 msgr2=0x55764441c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.160+0000 7f57f1791740 1 -- 172.21.15.39:0/3934541604 shutdown_connections 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.160+0000 7f57f1791740 1 -- 172.21.15.39:0/3934541604 wait complete. 2024-07-04T08:20:20.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.160+0000 7f57f1791740 20 client.16602 trim_cache size 0 max 16384 2024-07-04T08:20:20.160 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:20:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.160+0000 7f57e9c00640 1 -- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c80b2870 msgr2=0x7f57c80b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.160+0000 7f57e9c00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c80b2870 0x7f57c80b4d80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0059a0 tx=0x7f57dc009500 comp rx=0 tx=0).stop 2024-07-04T08:20:20.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 -- 172.21.15.39:0/1714503098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57e4105110 msgr2=0x7f57e419baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57e4105110 0x7f57e419baa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f57d400e9f0 tx=0x7f57d400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 -- 172.21.15.39:0/1714503098 shutdown_connections 2024-07-04T08:20:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c80b2870 0x7f57c80b4d80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57e410b3a0 0x7f57e41a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57e410aa50 0x7f57e419c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 --2- 172.21.15.39:0/1714503098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57e4105110 0x7f57e419baa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 -- 172.21.15.39:0/1714503098 >> 172.21.15.39:0/1714503098 conn(0x7f57e4100b80 msgr2=0x7f57e4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 -- 172.21.15.39:0/1714503098 shutdown_connections 2024-07-04T08:20:20.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:20.161+0000 7f57e9c00640 1 -- 172.21.15.39:0/1714503098 wait complete. 2024-07-04T08:20:21.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:20:21.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:21 smithi039 ceph-mon[32062]: pgmap v634: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:20:21.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:21 smithi050 ceph-mon[36577]: pgmap v634: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:20:21.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:21 smithi133 ceph-mon[37050]: pgmap v634: 129 pgs: 129 active+clean; 5.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:20:21.565 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:23.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:23 smithi039 ceph-mon[32062]: pgmap v635: 129 pgs: 129 active+clean; 5.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-07-04T08:20:23.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:23 smithi050 ceph-mon[36577]: pgmap v635: 129 pgs: 129 active+clean; 5.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-07-04T08:20:23.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:23 smithi133 ceph-mon[37050]: pgmap v635: 129 pgs: 129 active+clean; 5.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-07-04T08:20:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.978+0000 7fdec9a00640 1 -- 172.21.15.39:0/363937744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdec410f990 msgr2=0x7fdec41161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.978+0000 7fdec9a00640 1 --2- 172.21.15.39:0/363937744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdec410f990 0x7fdec41161c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009ca0 tx=0x7fdebc02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.978+0000 7fdec9a00640 1 -- 172.21.15.39:0/363937744 shutdown_connections 2024-07-04T08:20:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.978+0000 7fdec9a00640 1 --2- 172.21.15.39:0/363937744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdec410f990 0x7fdec41161c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.978+0000 7fdec9a00640 1 --2- 172.21.15.39:0/363937744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec410f040 0x7fdec410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.978+0000 7fdec9a00640 1 --2- 172.21.15.39:0/363937744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec4109700 0x7fdec4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.978+0000 7fdec9a00640 1 -- 172.21.15.39:0/363937744 >> 172.21.15.39:0/363937744 conn(0x7fdec4077630 msgr2=0x7fdec4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.978+0000 7fdec9a00640 1 -- 172.21.15.39:0/363937744 shutdown_connections 2024-07-04T08:20:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.978+0000 7fdec9a00640 1 -- 172.21.15.39:0/363937744 wait complete. 2024-07-04T08:20:23.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.980+0000 7fdec9a00640 1 Processor -- start 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.980+0000 7fdec9a00640 1 -- start start 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.980+0000 7fdec9a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec4109700 0x7fdec419ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.980+0000 7fdec9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdec410f040 0x7fdec41a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.980+0000 7fdec9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec410f990 0x7fdec41a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.980+0000 7fdec9a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4119660 con 0x7fdec410f990 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.980+0000 7fdec9a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41194e0 con 0x7fdec410f040 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.980+0000 7fdec9a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41197e0 con 0x7fdec4109700 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec4109700 0x7fdec419ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdec3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec410f990 0x7fdec41a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec4109700 0x7fdec419ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33616/0 (socket says 172.21.15.39:33616) 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdec2a00640 1 -- 172.21.15.39:0/286098286 learned_addr learned my addr 172.21.15.39:0/286098286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:23.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdebbe00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdec410f040 0x7fdec41a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdec2a00640 1 -- 172.21.15.39:0/286098286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdec410f040 msgr2=0x7fdec41a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:23.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdec2a00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdec410f040 0x7fdec41a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdec2a00640 1 -- 172.21.15.39:0/286098286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec410f990 msgr2=0x7fdec41a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdec2a00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec410f990 0x7fdec41a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.981+0000 7fdec2a00640 1 -- 172.21.15.39:0/286098286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec41a5f20 con 0x7fdec4109700 2024-07-04T08:20:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.982+0000 7fdec2a00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec4109700 0x7fdec419ffb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400d950 tx=0x7fdeb400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.982+0000 7fdebb400640 1 -- 172.21.15.39:0/286098286 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4017030 con 0x7fdec4109700 2024-07-04T08:20:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.982+0000 7fdec9a00640 1 -- 172.21.15.39:0/286098286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec41a6210 con 0x7fdec4109700 2024-07-04T08:20:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.982+0000 7fdebb400640 1 -- 172.21.15.39:0/286098286 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb4004020 con 0x7fdec4109700 2024-07-04T08:20:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.982+0000 7fdebb400640 1 -- 172.21.15.39:0/286098286 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40047a0 con 0x7fdec4109700 2024-07-04T08:20:23.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.982+0000 7fdec9a00640 1 -- 172.21.15.39:0/286098286 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec41ac970 con 0x7fdec4109700 2024-07-04T08:20:23.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.986+0000 7fded00b3740 1 Processor -- start 2024-07-04T08:20:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.986+0000 7fded00b3740 1 -- start start 2024-07-04T08:20:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.986+0000 7fded00b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c9792d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.986+0000 7fded00b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c975601b0 0x562c9792dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.986+0000 7fded00b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c97568390 0x562c9792e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.986+0000 7fded00b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562c978a3ee0 con 0x562c97589070 2024-07-04T08:20:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.986+0000 7fded00b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562c97893480 con 0x562c975601b0 2024-07-04T08:20:23.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.986+0000 7fded00b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562c978974c0 con 0x562c97568390 2024-07-04T08:20:23.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdebb400640 1 -- 172.21.15.39:0/286098286 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdeb4009a30 con 0x7fdec4109700 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdec3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c9792d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdec3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c9792d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51544/0 (socket says 172.21.15.39:51544) 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdec3400640 1 -- 172.21.15.39:0/183074705 learned_addr learned my addr 172.21.15.39:0/183074705 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c97568390 0x562c9792e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdec2a00640 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c975601b0 0x562c9792dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdebbe00640 1 -- 172.21.15.39:0/183074705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c975601b0 msgr2=0x562c9792dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdebbe00640 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c975601b0 0x562c9792dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdebbe00640 1 -- 172.21.15.39:0/183074705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 msgr2=0x562c9792d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdebbe00640 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c9792d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdebbe00640 1 -- 172.21.15.39:0/183074705 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c9782b650 con 0x562c97568390 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdec3400640 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c9792d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.987+0000 7fdec2a00640 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c975601b0 0x562c9792dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:23.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.988+0000 7fdebbe00640 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c97568390 0x562c9792e230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdec406bce0 tx=0x7fdeac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fa8b967fda83f60 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:23.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.988+0000 7fdeb3e00640 1 -- 172.21.15.39:0/183074705 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac004050 con 0x562c97568390 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.988+0000 7fdeb3e00640 1 -- 172.21.15.39:0/183074705 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac0054d0 con 0x562c97568390 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.988+0000 7fdeb3e00640 1 -- 172.21.15.39:0/183074705 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac004c60 con 0x562c97568390 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.989+0000 7fdebb400640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdea80b29e0 0x7fdea80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.989+0000 7fdebb400640 1 -- 172.21.15.39:0/286098286 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdeb40e2350 con 0x7fdec4109700 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.989+0000 7fdec3400640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdea80b29e0 0x7fdea80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.989+0000 7fded00b3740 1 -- 172.21.15.39:0/183074705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c97568390 msgr2=0x562c9792e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.989+0000 7fded00b3740 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c97568390 0x562c9792e230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdec406bce0 tx=0x7fdeac011810 comp rx=0 tx=0).stop 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.990+0000 7fded00b3740 1 -- 172.21.15.39:0/183074705 shutdown_connections 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.990+0000 7fded00b3740 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c97568390 0x562c9792e230 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.990+0000 7fded00b3740 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c975601b0 0x562c9792dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.990+0000 7fded00b3740 1 --2- 172.21.15.39:0/183074705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c9792d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.990+0000 7fded00b3740 1 -- 172.21.15.39:0/183074705 >> 172.21.15.39:0/183074705 conn(0x562c9757d460 msgr2=0x562c9792b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:23.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.990+0000 7fded00b3740 1 -- 172.21.15.39:0/183074705 shutdown_connections 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.990+0000 7fded00b3740 1 -- 172.21.15.39:0/183074705 wait complete. 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.990+0000 7fdec3400640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdea80b29e0 0x7fdea80b4ef0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdebc02f500 tx=0x7fdebc02f490 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.991+0000 7fded00b3740 1 -- start start 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.991+0000 7fded00b3740 1 Processor -- start 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fded00b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c979393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fded00b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 0x562c97939900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fded00b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fded00b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562c976f7700 con 0x562c97589070 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fded00b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562c9783abb0 con 0x562c97568390 2024-07-04T08:20:23.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fded00b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562c9793bf90 con 0x562c975601b0 2024-07-04T08:20:23.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fded00b3740 10 client.0 _ll_register_callbacks cb 0x562c9788c570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fdec3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c979393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c979393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33634/0 (socket says 172.21.15.39:33634) 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fdec2a00640 1 -- 172.21.15.39:0/1362399005 learned_addr learned my addr 172.21.15.39:0/1362399005 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.992+0000 7fdebbe00640 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 0x562c97939900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdec2a00640 1 -- 172.21.15.39:0/1362399005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 msgr2=0x562c97939900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdec2a00640 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 0x562c97939900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdec2a00640 1 -- 172.21.15.39:0/1362399005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 msgr2=0x562c97945280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdec2a00640 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdec2a00640 1 -- 172.21.15.39:0/1362399005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c975f4650 con 0x562c975601b0 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdebbe00640 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 0x562c97939900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdec2a00640 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c979393c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40df770 tx=0x7fdeb40a8cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b13ac1074bc565a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdeb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdeb0c00640 1 -- 172.21.15.39:0/1362399005 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4014070 con 0x562c975601b0 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdeb0c00640 1 -- 172.21.15.39:0/1362399005 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdeb40a5530 con 0x562c975601b0 2024-07-04T08:20:23.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fded00b3740 1 -- 172.21.15.39:0/1362399005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeac0054d0 con 0x562c975601b0 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fdeb0c00640 1 -- 172.21.15.39:0/1362399005 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdeb40dfe60 con 0x562c975601b0 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fded00b3740 10 client.26659 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.993+0000 7fded00b3740 1 -- 172.21.15.39:0/1362399005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c97948050 con 0x562c975601b0 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.994+0000 7fdeb0c00640 1 -- 172.21.15.39:0/1362399005 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac0054d0 con 0x562c975601b0 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.994+0000 7fdeb0c00640 1 -- 172.21.15.39:0/1362399005 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdeb40df990 con 0x562c975601b0 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.994+0000 7fded00b3740 10 client.26659 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.994+0000 7fded00b3740 10 client.26659 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.994+0000 7fded00b3740 20 client.26659 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.994+0000 7fded00b3740 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x562c975479f0 0x562c9794b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.994+0000 7fded00b3740 4 client.26659 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.994+0000 7fded00b3740 1 -- 172.21.15.39:0/1362399005 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c978a5e30 con 0x562c975479f0 2024-07-04T08:20:23.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.994+0000 7fdebbe00640 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x562c975479f0 0x562c9794b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.995+0000 7fdebbe00640 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x562c975479f0 0x562c9794b4b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47158c0112e04acd server_cookie=21f061fcd1a12e99 in_seq=0 out_seq=0 2024-07-04T08:20:23.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.995+0000 7fdeb0c00640 10 client.26659 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fdeb0c00640 1 -- 172.21.15.39:0/1362399005 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562c978a5e30 con 0x562c975479f0 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fdeb0c00640 10 client.26659 handle_command_reply: tid=0 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fded00b3740 1 client.26659 shutdown 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fded00b3740 1 -- 172.21.15.39:0/1362399005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 msgr2=0x562c979393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fded00b3740 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c979393c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40df770 tx=0x7fdeb40a8cb0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fded00b3740 1 -- 172.21.15.39:0/1362399005 shutdown_connections 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fded00b3740 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x562c975479f0 0x562c9794b4b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fded00b3740 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fded00b3740 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 0x562c97939900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fded00b3740 1 --2- 172.21.15.39:0/1362399005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c979393c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fdeb0c00640 0 client.26659 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.996+0000 7fded00b3740 1 -- 172.21.15.39:0/1362399005 >> 172.21.15.39:0/1362399005 conn(0x562c9757d460 msgr2=0x562c97930990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:23.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.997+0000 7fded00b3740 1 -- 172.21.15.39:0/1362399005 shutdown_connections 2024-07-04T08:20:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.997+0000 7fded00b3740 1 -- 172.21.15.39:0/1362399005 wait complete. 2024-07-04T08:20:23.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:23.997+0000 7fded00b3740 20 client.26659 trim_cache size 0 max 16384 2024-07-04T08:20:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.012+0000 7fded00b3740 1 Processor -- start 2024-07-04T08:20:24.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.012+0000 7fded00b3740 1 -- start start 2024-07-04T08:20:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fded00b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c9792a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fded00b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 0x562c97930fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fded00b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fded00b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b67b0 con 0x562c97589070 2024-07-04T08:20:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fded00b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b82f0 con 0x562c97568390 2024-07-04T08:20:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fded00b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b6e80 con 0x562c975601b0 2024-07-04T08:20:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fdec3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:24.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fdec3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51566/0 (socket says 172.21.15.39:51566) 2024-07-04T08:20:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fdec3400640 1 -- 172.21.15.39:0/3430789291 learned_addr learned my addr 172.21.15.39:0/3430789291 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fdec2a00640 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c9792a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:24.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fdebbe00640 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 0x562c97930fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fdec2a00640 1 -- 172.21.15.39:0/3430789291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 msgr2=0x562c97930fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fdec2a00640 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 0x562c97930fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fdec2a00640 1 -- 172.21.15.39:0/3430789291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 msgr2=0x562c97945070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.013+0000 7fdec2a00640 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.014+0000 7fdec2a00640 1 -- 172.21.15.39:0/3430789291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c97930040 con 0x562c975601b0 2024-07-04T08:20:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.014+0000 7fdec3400640 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:24.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.014+0000 7fdec2a00640 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c9792a940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40a8fa0 tx=0x7fdeb40a8d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad5df3dfb0f69c20 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.014+0000 7fdeb3400640 1 -- 172.21.15.39:0/3430789291 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4014070 con 0x562c975601b0 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.014+0000 7fdeb3400640 1 -- 172.21.15.39:0/3430789291 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb40b2100 con 0x562c975601b0 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.014+0000 7fdeb3400640 1 -- 172.21.15.39:0/3430789291 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40df990 con 0x562c975601b0 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.015+0000 7fded00b3740 1 -- 172.21.15.39:0/3430789291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 msgr2=0x562c9792a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.015+0000 7fded00b3740 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c9792a940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40a8fa0 tx=0x7fdeb40a8d20 comp rx=0 tx=0).stop 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.015+0000 7fded00b3740 1 -- 172.21.15.39:0/3430789291 shutdown_connections 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.015+0000 7fded00b3740 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97589070 0x562c97945070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.015+0000 7fded00b3740 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97568390 0x562c97930fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.015+0000 7fded00b3740 1 --2- 172.21.15.39:0/3430789291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c9792a940 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.015+0000 7fded00b3740 1 -- 172.21.15.39:0/3430789291 >> 172.21.15.39:0/3430789291 conn(0x562c9757d460 msgr2=0x562c979308b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.015+0000 7fded00b3740 1 -- 172.21.15.39:0/3430789291 shutdown_connections 2024-07-04T08:20:24.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.015+0000 7fded00b3740 1 -- 172.21.15.39:0/3430789291 wait complete. 2024-07-04T08:20:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.016+0000 7fded00b3740 1 -- start start 2024-07-04T08:20:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.016+0000 7fded00b3740 1 Processor -- start 2024-07-04T08:20:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.016+0000 7fded00b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c97941670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.016+0000 7fded00b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97568390 0x562c97950470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.016+0000 7fded00b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97589070 0x562c979509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.016+0000 7fded00b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b7c20 con 0x562c97568390 2024-07-04T08:20:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.016+0000 7fded00b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b7550 con 0x562c97589070 2024-07-04T08:20:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.016+0000 7fded00b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41b4790 con 0x562c975601b0 2024-07-04T08:20:24.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.016+0000 7fded00b3740 10 client.0 _ll_register_callbacks cb 0x7fdeb40a9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdebbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97568390 0x562c97950470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c97941670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdec3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97589070 0x562c979509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdec2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c97941670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33646/0 (socket says 172.21.15.39:33646) 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdec2a00640 1 -- 172.21.15.39:0/591083657 learned_addr learned my addr 172.21.15.39:0/591083657 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdec2a00640 1 -- 172.21.15.39:0/591083657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97589070 msgr2=0x562c979509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdec2a00640 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97589070 0x562c979509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdec2a00640 1 -- 172.21.15.39:0/591083657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97568390 msgr2=0x562c97950470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdec2a00640 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97568390 0x562c97950470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.017+0000 7fdec2a00640 1 -- 172.21.15.39:0/591083657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c97950ef0 con 0x562c975601b0 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fdec2a00640 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c97941670 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40e5c30 tx=0x7fdeb40a9120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2611ac8c0ca63dc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fdeb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:24.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fdeb0c00640 1 -- 172.21.15.39:0/591083657 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4014070 con 0x562c975601b0 2024-07-04T08:20:24.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fdeb0c00640 1 -- 172.21.15.39:0/591083657 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fdeb40ad3a0 con 0x562c975601b0 2024-07-04T08:20:24.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fded00b3740 1 -- 172.21.15.39:0/591083657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb40b2100 con 0x562c975601b0 2024-07-04T08:20:24.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fdeb0c00640 1 -- 172.21.15.39:0/591083657 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdeb40ca330 con 0x562c975601b0 2024-07-04T08:20:24.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fded00b3740 10 client.26665 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:24.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fded00b3740 1 -- 172.21.15.39:0/591083657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562c97950ef0 con 0x562c975601b0 2024-07-04T08:20:24.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fdeb0c00640 1 -- 172.21.15.39:0/591083657 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb40b2100 con 0x562c975601b0 2024-07-04T08:20:24.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.018+0000 7fdeb0c00640 1 -- 172.21.15.39:0/591083657 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdeb40a4c10 con 0x562c975601b0 2024-07-04T08:20:24.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.019+0000 7fded00b3740 10 client.26665 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.019+0000 7fded00b3740 10 client.26665 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.019+0000 7fded00b3740 20 client.26665 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.019+0000 7fded00b3740 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x562c9751cda0 0x562c979b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.019+0000 7fded00b3740 4 client.26665 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.019+0000 7fded00b3740 1 -- 172.21.15.39:0/591083657 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c978a18d0 con 0x562c9751cda0 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.019+0000 7fdebbe00640 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x562c9751cda0 0x562c979b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fdebbe00640 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x562c9751cda0 0x562c979b5ca0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9c94f85b7e55a7f server_cookie=fc74ded8c7d54ad2 in_seq=0 out_seq=0 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fdeb0c00640 10 client.26665 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fdeb0c00640 1 -- 172.21.15.39:0/591083657 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c978a18d0 con 0x562c9751cda0 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fdeb0c00640 10 client.26665 handle_command_reply: tid=0 2024-07-04T08:20:24.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 client.26665 shutdown 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 -- 172.21.15.39:0/591083657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 msgr2=0x562c97941670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c97941670 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40e5c30 tx=0x7fdeb40a9120 comp rx=0 tx=0).stop 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 -- 172.21.15.39:0/591083657 shutdown_connections 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x562c9751cda0 0x562c979b5ca0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562c97568390 0x562c97950470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562c97589070 0x562c979509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 --2- 172.21.15.39:0/591083657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562c975601b0 0x562c97941670 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fdeb0c00640 0 client.26665 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 -- 172.21.15.39:0/591083657 >> 172.21.15.39:0/591083657 conn(0x562c9757d460 msgr2=0x562c9792af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 -- 172.21.15.39:0/591083657 shutdown_connections 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.020+0000 7fded00b3740 1 -- 172.21.15.39:0/591083657 wait complete. 2024-07-04T08:20:24.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fded00b3740 20 client.26665 trim_cache size 0 max 16384 2024-07-04T08:20:24.022 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:20:24.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 -- 172.21.15.39:0/286098286 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdea80b29e0 msgr2=0x7fdea80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdea80b29e0 0x7fdea80b4ef0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdebc02f500 tx=0x7fdebc02f490 comp rx=0 tx=0).stop 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 -- 172.21.15.39:0/286098286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec4109700 msgr2=0x7fdec419ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec4109700 0x7fdec419ffb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400d950 tx=0x7fdeb400de10 comp rx=0 tx=0).stop 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 -- 172.21.15.39:0/286098286 shutdown_connections 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdea80b29e0 0x7fdea80b4ef0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec410f990 0x7fdec41a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdec410f040 0x7fdec41a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 --2- 172.21.15.39:0/286098286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdec4109700 0x7fdec419ffb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.021+0000 7fdec9a00640 1 -- 172.21.15.39:0/286098286 >> 172.21.15.39:0/286098286 conn(0x7fdec4077630 msgr2=0x7fdec4077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.022+0000 7fdec9a00640 1 -- 172.21.15.39:0/286098286 shutdown_connections 2024-07-04T08:20:24.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:24.022+0000 7fdec9a00640 1 -- 172.21.15.39:0/286098286 wait complete. 2024-07-04T08:20:24.961 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:20:25.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:25 smithi039 ceph-mon[32062]: pgmap v636: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-07-04T08:20:25.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:25 smithi050 ceph-mon[36577]: pgmap v636: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-07-04T08:20:25.462 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:25.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:25 smithi133 ceph-mon[37050]: pgmap v636: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-07-04T08:20:27.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:27 smithi039 ceph-mon[32062]: pgmap v637: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-07-04T08:20:27.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:27 smithi133 ceph-mon[37050]: pgmap v637: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-07-04T08:20:27.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:27 smithi050 ceph-mon[36577]: pgmap v637: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-07-04T08:20:27.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.913+0000 7f8251800640 1 -- 172.21.15.39:0/3006539320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c10ee50 msgr2=0x7f824c10f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:27.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.913+0000 7f8251800640 1 --2- 172.21.15.39:0/3006539320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c10ee50 0x7f824c10f260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8234009be0 tx=0x7f823402a6c0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.913+0000 7f8251800640 1 -- 172.21.15.39:0/3006539320 shutdown_connections 2024-07-04T08:20:27.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.913+0000 7f8251800640 1 --2- 172.21.15.39:0/3006539320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f824c10f7a0 0x7f824c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.913+0000 7f8251800640 1 --2- 172.21.15.39:0/3006539320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c10ee50 0x7f824c10f260 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.913+0000 7f8251800640 1 --2- 172.21.15.39:0/3006539320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824c109510 0x7f824c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.913+0000 7f8251800640 1 -- 172.21.15.39:0/3006539320 >> 172.21.15.39:0/3006539320 conn(0x7f824c078b10 msgr2=0x7f824c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:27.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.913+0000 7f8251800640 1 -- 172.21.15.39:0/3006539320 shutdown_connections 2024-07-04T08:20:27.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.913+0000 7f8251800640 1 -- 172.21.15.39:0/3006539320 wait complete. 2024-07-04T08:20:27.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.915+0000 7f8251800640 1 Processor -- start 2024-07-04T08:20:27.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.915+0000 7f8251800640 1 -- start start 2024-07-04T08:20:27.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f8251800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c109510 0x7f824c19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f8251800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f824c10ee50 0x7f824c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f8251800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824c10f7a0 0x7f824c1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f8251800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f824c119410 con 0x7f824c10f7a0 2024-07-04T08:20:27.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f8251800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f824c119290 con 0x7f824c10ee50 2024-07-04T08:20:27.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f8251800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f824c119590 con 0x7f824c109510 2024-07-04T08:20:27.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f8250e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824c10f7a0 0x7f824c1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f824be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c109510 0x7f824c19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f824be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c109510 0x7f824c19fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33672/0 (socket says 172.21.15.39:33672) 2024-07-04T08:20:27.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f824be00640 1 -- 172.21.15.39:0/577624926 learned_addr learned my addr 172.21.15.39:0/577624926 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:27.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.916+0000 7f8243400640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f824c10ee50 0x7f824c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f824be00640 1 -- 172.21.15.39:0/577624926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f824c10ee50 msgr2=0x7f824c1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f824be00640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f824c10ee50 0x7f824c1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f824be00640 1 -- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824c10f7a0 msgr2=0x7f824c1a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f824be00640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824c10f7a0 0x7f824c1a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f824be00640 1 -- 172.21.15.39:0/577624926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c1a5e30 con 0x7f824c109510 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f8250e00640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824c10f7a0 0x7f824c1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f824be00640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c109510 0x7f824c19fec0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f823c00e9c0 tx=0x7f823c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f8249600640 1 -- 172.21.15.39:0/577624926 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c00ccc0 con 0x7f824c109510 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f8249600640 1 -- 172.21.15.39:0/577624926 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823c0054d0 con 0x7f824c109510 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f8249600640 1 -- 172.21.15.39:0/577624926 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c010430 con 0x7f824c109510 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f8251800640 1 -- 172.21.15.39:0/577624926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c1a6120 con 0x7f824c109510 2024-07-04T08:20:27.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.917+0000 7f8251800640 1 -- 172.21.15.39:0/577624926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c1ac880 con 0x7f824c109510 2024-07-04T08:20:27.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.921+0000 7f82593fa740 1 Processor -- start 2024-07-04T08:20:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.921+0000 7f82593fa740 1 -- start start 2024-07-04T08:20:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.921+0000 7f82593fa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07951950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.921+0000 7f82593fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae07951e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.921+0000 7f82593fa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae0758c390 0x55ae079523d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.921+0000 7f82593fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae078c7ee0 con 0x55ae075841b0 2024-07-04T08:20:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.921+0000 7f82593fa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ae078b7480 con 0x55ae075ad070 2024-07-04T08:20:27.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.921+0000 7f82593fa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ae078bb4c0 con 0x55ae0758c390 2024-07-04T08:20:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8250e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07951950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8250e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07951950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45126/0 (socket says 172.21.15.39:45126) 2024-07-04T08:20:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8250e00640 1 -- 172.21.15.39:0/3418266469 learned_addr learned my addr 172.21.15.39:0/3418266469 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8243400640 1 --2- 172.21.15.39:0/3418266469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae0758c390 0x55ae079523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8249600640 1 -- 172.21.15.39:0/577624926 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f823c004f80 con 0x7f824c109510 2024-07-04T08:20:27.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8250e00640 1 -- 172.21.15.39:0/3418266469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae0758c390 msgr2=0x55ae079523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:27.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8250e00640 1 --2- 172.21.15.39:0/3418266469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae0758c390 0x55ae079523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8250e00640 1 -- 172.21.15.39:0/3418266469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 msgr2=0x55ae07951e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:27.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8250e00640 1 --2- 172.21.15.39:0/3418266469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae07951e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.922+0000 7f8250e00640 1 -- 172.21.15.39:0/3418266469 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae0784f650 con 0x55ae075ad070 2024-07-04T08:20:27.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.923+0000 7f8250e00640 1 --2- 172.21.15.39:0/3418266469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07951950 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8244011440 tx=0x7f8244011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0919b9cccf7374 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.923+0000 7f8241600640 1 -- 172.21.15.39:0/3418266469 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244004020 con 0x55ae075ad070 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.923+0000 7f8241600640 1 -- 172.21.15.39:0/3418266469 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82440041c0 con 0x55ae075ad070 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.923+0000 7f8241600640 1 -- 172.21.15.39:0/3418266469 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400f7c0 con 0x55ae075ad070 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f82593fa740 1 -- 172.21.15.39:0/3418266469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 msgr2=0x55ae07951950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f82593fa740 1 --2- 172.21.15.39:0/3418266469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07951950 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8244011440 tx=0x7f8244011900 comp rx=0 tx=0).stop 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f8249600640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82300b29e0 0x7f82300b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f82593fa740 1 -- 172.21.15.39:0/3418266469 shutdown_connections 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f82593fa740 1 --2- 172.21.15.39:0/3418266469 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae07951e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f82593fa740 1 --2- 172.21.15.39:0/3418266469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae0758c390 0x55ae079523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f82593fa740 1 --2- 172.21.15.39:0/3418266469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07951950 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f82593fa740 1 -- 172.21.15.39:0/3418266469 >> 172.21.15.39:0/3418266469 conn(0x55ae075a1460 msgr2=0x55ae0794f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:27.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f8249600640 1 -- 172.21.15.39:0/577624926 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f823c0e2ce0 con 0x7f824c109510 2024-07-04T08:20:27.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f824be00640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82300b29e0 0x7f82300b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.924+0000 7f82593fa740 1 -- 172.21.15.39:0/3418266469 shutdown_connections 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.925+0000 7f82593fa740 1 -- 172.21.15.39:0/3418266469 wait complete. 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.925+0000 7f824be00640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82300b29e0 0x7f82300b4ef0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55ae079505d0 tx=0x7f823c00efb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.925+0000 7f82593fa740 1 -- start start 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.925+0000 7f82593fa740 1 Processor -- start 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.926+0000 7f82593fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae0795d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.926+0000 7f82593fa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.926+0000 7f82593fa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 0x55ae07969420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.926+0000 7f82593fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae0771b700 con 0x55ae075841b0 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.926+0000 7f82593fa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ae0785ebb0 con 0x55ae0758c390 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.926+0000 7f82593fa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ae07960130 con 0x55ae075ad070 2024-07-04T08:20:27.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.926+0000 7f82593fa740 10 client.0 _ll_register_callbacks cb 0x55ae078b0570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.926+0000 7f8250e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae0795d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8250e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae0795d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51618/0 (socket says 172.21.15.39:51618) 2024-07-04T08:20:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8250e00640 1 -- 172.21.15.39:0/1499823453 learned_addr learned my addr 172.21.15.39:0/1499823453 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8243400640 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8250e00640 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 0x55ae07969420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8243400640 1 -- 172.21.15.39:0/1499823453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 msgr2=0x55ae07969420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8243400640 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 0x55ae07969420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8243400640 1 -- 172.21.15.39:0/1499823453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 msgr2=0x55ae0795d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8243400640 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae0795d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8243400640 1 -- 172.21.15.39:0/1499823453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae07618650 con 0x55ae0758c390 2024-07-04T08:20:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.927+0000 7f8243400640 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795da30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8234034040 tx=0x7f8234030710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c36e3e1767a71099 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:27.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.928+0000 7f823a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.928+0000 7f823a000640 1 -- 172.21.15.39:0/1499823453 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234036040 con 0x55ae0758c390 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.928+0000 7f82593fa740 1 -- 172.21.15.39:0/1499823453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82440041c0 con 0x55ae0758c390 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.928+0000 7f823a000640 1 -- 172.21.15.39:0/1499823453 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8234003b70 con 0x55ae0758c390 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.928+0000 7f823a000640 1 -- 172.21.15.39:0/1499823453 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f823404f050 con 0x55ae0758c390 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.928+0000 7f82593fa740 10 client.26768 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.928+0000 7f82593fa740 1 -- 172.21.15.39:0/1499823453 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ae0796c110 con 0x55ae0758c390 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.929+0000 7f823a000640 1 -- 172.21.15.39:0/1499823453 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82440041c0 con 0x55ae0758c390 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.929+0000 7f823a000640 1 -- 172.21.15.39:0/1499823453 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f82340044d0 con 0x55ae0758c390 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.929+0000 7f82593fa740 10 client.26768 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.929+0000 7f82593fa740 10 client.26768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:20:27.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.929+0000 7f82593fa740 20 client.26768 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:27.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.929+0000 7f82593fa740 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55ae0756b9f0 0x55ae0796f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.929+0000 7f82593fa740 4 client.26768 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:27.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.929+0000 7f82593fa740 1 -- 172.21.15.39:0/1499823453 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae078c9e30 con 0x55ae0756b9f0 2024-07-04T08:20:27.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.929+0000 7f8250e00640 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55ae0756b9f0 0x55ae0796f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.930+0000 7f8250e00640 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55ae0756b9f0 0x55ae0796f610 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72edbb26df7ecc68 server_cookie=47323fb14b1f4bca in_seq=0 out_seq=0 2024-07-04T08:20:27.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.930+0000 7f823a000640 10 client.26768 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:20:27.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.931+0000 7f823a000640 1 -- 172.21.15.39:0/1499823453 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ae078c9e30 con 0x55ae0756b9f0 2024-07-04T08:20:27.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.931+0000 7f823a000640 10 client.26768 handle_command_reply: tid=0 2024-07-04T08:20:27.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.931+0000 7f82593fa740 1 client.26768 shutdown 2024-07-04T08:20:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.931+0000 7f82593fa740 1 -- 172.21.15.39:0/1499823453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 msgr2=0x55ae0795da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.931+0000 7f82593fa740 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795da30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8234034040 tx=0x7f8234030710 comp rx=0 tx=0).stop 2024-07-04T08:20:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f82593fa740 1 -- 172.21.15.39:0/1499823453 shutdown_connections 2024-07-04T08:20:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f82593fa740 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55ae0756b9f0 0x55ae0796f610 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f82593fa740 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 0x55ae07969420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f82593fa740 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795da30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f82593fa740 1 --2- 172.21.15.39:0/1499823453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae0795d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f823a000640 0 client.26768 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:20:27.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f82593fa740 1 -- 172.21.15.39:0/1499823453 >> 172.21.15.39:0/1499823453 conn(0x55ae075a1460 msgr2=0x55ae07954b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f82593fa740 1 -- 172.21.15.39:0/1499823453 shutdown_connections 2024-07-04T08:20:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f82593fa740 1 -- 172.21.15.39:0/1499823453 wait complete. 2024-07-04T08:20:27.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.932+0000 7f82593fa740 20 client.26768 trim_cache size 0 max 16384 2024-07-04T08:20:27.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.947+0000 7f82593fa740 1 Processor -- start 2024-07-04T08:20:27.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f82593fa740 1 -- start start 2024-07-04T08:20:27.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f82593fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae07953340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f82593fa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f82593fa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 0x55ae0794ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f82593fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b66f0 con 0x55ae075841b0 2024-07-04T08:20:27.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f82593fa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b8230 con 0x55ae0758c390 2024-07-04T08:20:27.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f82593fa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b6dc0 con 0x55ae075ad070 2024-07-04T08:20:27.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f8250e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae07953340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f8250e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae07953340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51634/0 (socket says 172.21.15.39:51634) 2024-07-04T08:20:27.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f8250e00640 1 -- 172.21.15.39:0/996833998 learned_addr learned my addr 172.21.15.39:0/996833998 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:27.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.948+0000 7f8243400640 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.949+0000 7f8250e00640 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 0x55ae0794ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.949+0000 7f8243400640 1 -- 172.21.15.39:0/996833998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 msgr2=0x55ae0794ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:27.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.949+0000 7f8243400640 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 0x55ae0794ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.949+0000 7f8243400640 1 -- 172.21.15.39:0/996833998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 msgr2=0x55ae07953340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:27.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.949+0000 7f8243400640 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae07953340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.949+0000 7f8243400640 1 -- 172.21.15.39:0/996833998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae07953c40 con 0x55ae0758c390 2024-07-04T08:20:27.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.949+0000 7f8250e00640 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae07953340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:27.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.949+0000 7f8243400640 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795ef20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f82340099a0 tx=0x7f823402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bc07fbe07048bb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:27.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.950+0000 7f8240c00640 1 -- 172.21.15.39:0/996833998 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234036040 con 0x55ae0758c390 2024-07-04T08:20:27.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.950+0000 7f8240c00640 1 -- 172.21.15.39:0/996833998 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8234030970 con 0x55ae0758c390 2024-07-04T08:20:27.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.950+0000 7f8240c00640 1 -- 172.21.15.39:0/996833998 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823403eb00 con 0x55ae0758c390 2024-07-04T08:20:27.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.951+0000 7f82593fa740 1 -- 172.21.15.39:0/996833998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 msgr2=0x55ae0795ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:27.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.951+0000 7f82593fa740 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795ef20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f82340099a0 tx=0x7f823402a6c0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.951+0000 7f82593fa740 1 -- 172.21.15.39:0/996833998 shutdown_connections 2024-07-04T08:20:27.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.951+0000 7f82593fa740 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075ad070 0x55ae0794ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.951+0000 7f82593fa740 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae0758c390 0x55ae0795ef20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.951+0000 7f82593fa740 1 --2- 172.21.15.39:0/996833998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae075841b0 0x55ae07953340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.951+0000 7f82593fa740 1 -- 172.21.15.39:0/996833998 >> 172.21.15.39:0/996833998 conn(0x55ae075a1460 msgr2=0x55ae0795e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:27.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.951+0000 7f82593fa740 1 -- 172.21.15.39:0/996833998 shutdown_connections 2024-07-04T08:20:27.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.951+0000 7f82593fa740 1 -- 172.21.15.39:0/996833998 wait complete. 2024-07-04T08:20:27.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.952+0000 7f82593fa740 1 -- start start 2024-07-04T08:20:27.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.952+0000 7f82593fa740 1 Processor -- start 2024-07-04T08:20:27.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.952+0000 7f82593fa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075841b0 0x55ae07961460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.952+0000 7f82593fa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae0758c390 0x55ae079619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.952+0000 7f82593fa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07961f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.952+0000 7f82593fa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b7b60 con 0x55ae0758c390 2024-07-04T08:20:27.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.952+0000 7f82593fa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b7490 con 0x55ae075ad070 2024-07-04T08:20:27.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.952+0000 7f82593fa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f824c1b46d0 con 0x55ae075841b0 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f82593fa740 10 client.0 _ll_register_callbacks cb 0x7f8244005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8250e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075841b0 0x55ae07961460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8250e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075841b0 0x55ae07961460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:33706/0 (socket says 172.21.15.39:33706) 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8250e00640 1 -- 172.21.15.39:0/1503498788 learned_addr learned my addr 172.21.15.39:0/1503498788 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8243400640 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae0758c390 0x55ae079619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8250e00640 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07961f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8250e00640 1 -- 172.21.15.39:0/1503498788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 msgr2=0x55ae07961f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8250e00640 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07961f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8250e00640 1 -- 172.21.15.39:0/1503498788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae0758c390 msgr2=0x55ae079619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8250e00640 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae0758c390 0x55ae079619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.953+0000 7f8250e00640 1 -- 172.21.15.39:0/1503498788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae079d7010 con 0x55ae075841b0 2024-07-04T08:20:27.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.954+0000 7f8250e00640 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075841b0 0x55ae07961460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8244003e30 tx=0x7f8244003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7798ffbedd1a6d22 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:27.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.954+0000 7f823a000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:27.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.954+0000 7f823a000640 1 -- 172.21.15.39:0/1503498788 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824401d040 con 0x55ae075841b0 2024-07-04T08:20:27.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.954+0000 7f82593fa740 1 -- 172.21.15.39:0/1503498788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8234030970 con 0x55ae075841b0 2024-07-04T08:20:27.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.954+0000 7f823a000640 1 -- 172.21.15.39:0/1503498788 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8244016040 con 0x55ae075841b0 2024-07-04T08:20:27.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.954+0000 7f823a000640 1 -- 172.21.15.39:0/1503498788 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8244003820 con 0x55ae075841b0 2024-07-04T08:20:27.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.954+0000 7f82593fa740 10 client.26683 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:27.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.954+0000 7f82593fa740 1 -- 172.21.15.39:0/1503498788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ae079d8290 con 0x55ae075841b0 2024-07-04T08:20:27.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.955+0000 7f823a000640 1 -- 172.21.15.39:0/1503498788 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8234030970 con 0x55ae075841b0 2024-07-04T08:20:27.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.955+0000 7f823a000640 1 -- 172.21.15.39:0/1503498788 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8244004870 con 0x55ae075841b0 2024-07-04T08:20:27.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.955+0000 7f82593fa740 10 client.26683 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:27.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.955+0000 7f82593fa740 10 client.26683 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:20:27.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.955+0000 7f82593fa740 20 client.26683 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:27.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.955+0000 7f82593fa740 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55ae07540da0 0x55ae079da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:27.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.955+0000 7f82593fa740 4 client.26683 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:20:27.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.955+0000 7f82593fa740 1 -- 172.21.15.39:0/1503498788 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae078c58d0 con 0x55ae07540da0 2024-07-04T08:20:27.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.955+0000 7f8243400640 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55ae07540da0 0x55ae079da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:27.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.956+0000 7f823a000640 10 client.26683 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:20:27.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.956+0000 7f8243400640 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55ae07540da0 0x55ae079da810 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5adc4539fb1a1490 server_cookie=bb251c77164e2e33 in_seq=0 out_seq=0 2024-07-04T08:20:27.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f823a000640 1 -- 172.21.15.39:0/1503498788 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae078c58d0 con 0x55ae07540da0 2024-07-04T08:20:27.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f823a000640 10 client.26683 handle_command_reply: tid=0 2024-07-04T08:20:27.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 client.26683 shutdown 2024-07-04T08:20:27.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 -- 172.21.15.39:0/1503498788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075841b0 msgr2=0x55ae07961460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:27.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075841b0 0x55ae07961460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8244003e30 tx=0x7f8244003ec0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 -- 172.21.15.39:0/1503498788 shutdown_connections 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55ae07540da0 0x55ae079da810 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae0758c390 0x55ae079619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae075ad070 0x55ae07961f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 --2- 172.21.15.39:0/1503498788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae075841b0 0x55ae07961460 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 -- 172.21.15.39:0/1503498788 >> 172.21.15.39:0/1503498788 conn(0x55ae075a1460 msgr2=0x55ae079551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f823a000640 0 client.26683 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 -- 172.21.15.39:0/1503498788 shutdown_connections 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 1 -- 172.21.15.39:0/1503498788 wait complete. 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.957+0000 7f82593fa740 20 client.26683 trim_cache size 0 max 16384 2024-07-04T08:20:27.957 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:20:27.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.958+0000 7f8251800640 1 -- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82300b29e0 msgr2=0x7f82300b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:27.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.958+0000 7f8251800640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82300b29e0 0x7f82300b4ef0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55ae079505d0 tx=0x7f823c00efb0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.958+0000 7f8251800640 1 -- 172.21.15.39:0/577624926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c109510 msgr2=0x7f824c19fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:27.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.958+0000 7f8251800640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c109510 0x7f824c19fec0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f823c00e9c0 tx=0x7f823c00ee80 comp rx=0 tx=0).stop 2024-07-04T08:20:27.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.959+0000 7f8251800640 1 -- 172.21.15.39:0/577624926 shutdown_connections 2024-07-04T08:20:27.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.959+0000 7f8251800640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82300b29e0 0x7f82300b4ef0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.959+0000 7f8251800640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f824c10f7a0 0x7f824c1a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.959+0000 7f8251800640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f824c10ee50 0x7f824c1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.959+0000 7f8251800640 1 --2- 172.21.15.39:0/577624926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f824c109510 0x7f824c19fec0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:27.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.959+0000 7f8251800640 1 -- 172.21.15.39:0/577624926 >> 172.21.15.39:0/577624926 conn(0x7f824c078b10 msgr2=0x7f824c079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:27.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.959+0000 7f8251800640 1 -- 172.21.15.39:0/577624926 shutdown_connections 2024-07-04T08:20:27.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:27.959+0000 7f8251800640 1 -- 172.21.15.39:0/577624926 wait complete. 2024-07-04T08:20:28.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:20:29.419 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:29.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:29 smithi133 ceph-mon[37050]: pgmap v638: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-07-04T08:20:29.578 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:29 smithi039 ceph-mon[32062]: pgmap v638: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-07-04T08:20:29.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:29 smithi050 ceph-mon[36577]: pgmap v638: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-07-04T08:20:31.245 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:31 smithi039 ceph-mon[32062]: pgmap v639: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-07-04T08:20:31.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:31 smithi133 ceph-mon[37050]: pgmap v639: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-07-04T08:20:31.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:31 smithi050 ceph-mon[36577]: pgmap v639: 129 pgs: 129 active+clean; 5.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-07-04T08:20:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.837+0000 7f28c9600640 1 -- 172.21.15.39:0/1981513221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28c40777e0 msgr2=0x7f28c4113590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.837+0000 7f28c9600640 1 --2- 172.21.15.39:0/1981513221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28c40777e0 0x7f28c4113590 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009900 tx=0x7f28b002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.837+0000 7f28c9600640 1 -- 172.21.15.39:0/1981513221 shutdown_connections 2024-07-04T08:20:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.837+0000 7f28c9600640 1 --2- 172.21.15.39:0/1981513221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c4113ad0 0x7f28c4115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.837+0000 7f28c9600640 1 --2- 172.21.15.39:0/1981513221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28c40777e0 0x7f28c4113590 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.837+0000 7f28c9600640 1 --2- 172.21.15.39:0/1981513221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28c4076e90 0x7f28c40772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.837+0000 7f28c9600640 1 -- 172.21.15.39:0/1981513221 >> 172.21.15.39:0/1981513221 conn(0x7f28c4100b80 msgr2=0x7f28c4102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.837+0000 7f28c9600640 1 -- 172.21.15.39:0/1981513221 shutdown_connections 2024-07-04T08:20:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.837+0000 7f28c9600640 1 -- 172.21.15.39:0/1981513221 wait complete. 2024-07-04T08:20:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.839+0000 7f28c9600640 1 Processor -- start 2024-07-04T08:20:31.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.839+0000 7f28c9600640 1 -- start start 2024-07-04T08:20:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.840+0000 7f28c9600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c4076e90 0x7f28c419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.840+0000 7f28c9600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28c40777e0 0x7f28c41a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.840+0000 7f28c9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28c4113ad0 0x7f28c41a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.840+0000 7f28c9600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4119420 con 0x7f28c4113ad0 2024-07-04T08:20:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.840+0000 7f28c9600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41192a0 con 0x7f28c40777e0 2024-07-04T08:20:31.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.840+0000 7f28c9600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41195a0 con 0x7f28c4076e90 2024-07-04T08:20:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c4076e90 0x7f28c419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28c40777e0 0x7f28c41a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28c4113ad0 0x7f28c41a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c4076e90 0x7f28c419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57396/0 (socket says 172.21.15.39:57396) 2024-07-04T08:20:31.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c3e00640 1 -- 172.21.15.39:0/2644346636 learned_addr learned my addr 172.21.15.39:0/2644346636 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c3e00640 1 -- 172.21.15.39:0/2644346636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28c40777e0 msgr2=0x7f28c41a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c3e00640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28c40777e0 0x7f28c41a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c3e00640 1 -- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28c4113ad0 msgr2=0x7f28c41a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c3e00640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28c4113ad0 0x7f28c41a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c3e00640 1 -- 172.21.15.39:0/2644346636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c41a5d80 con 0x7f28c4076e90 2024-07-04T08:20:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.841+0000 7f28c8c00640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28c4113ad0 0x7f28c41a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:31.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.842+0000 7f28c3e00640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c4076e90 0x7f28c419fe10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e9f0 tx=0x7f28b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.842+0000 7f28c0c00640 1 -- 172.21.15.39:0/2644346636 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b400cd80 con 0x7f28c4076e90 2024-07-04T08:20:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.842+0000 7f28c0c00640 1 -- 172.21.15.39:0/2644346636 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b40108e0 con 0x7f28c4076e90 2024-07-04T08:20:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.842+0000 7f28c9600640 1 -- 172.21.15.39:0/2644346636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c41a6070 con 0x7f28c4076e90 2024-07-04T08:20:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.842+0000 7f28c0c00640 1 -- 172.21.15.39:0/2644346636 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4018980 con 0x7f28c4076e90 2024-07-04T08:20:31.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.842+0000 7f28c9600640 1 -- 172.21.15.39:0/2644346636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c41ac7d0 con 0x7f28c4076e90 2024-07-04T08:20:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.846+0000 7f28d1220740 1 Processor -- start 2024-07-04T08:20:31.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.846+0000 7f28d1220740 1 -- start start 2024-07-04T08:20:31.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.846+0000 7f28c0c00640 1 -- 172.21.15.39:0/2644346636 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f28b4010430 con 0x7f28c4076e90 2024-07-04T08:20:31.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.846+0000 7f28d1220740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b483070 0x55607b827970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.846+0000 7f28d1220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 0x55607b827eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.846+0000 7f28d1220740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b8283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.846+0000 7f28d1220740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55607b79df00 con 0x55607b483070 2024-07-04T08:20:31.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.846+0000 7f28d1220740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55607b78d4a0 con 0x55607b462390 2024-07-04T08:20:31.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.846+0000 7f28d1220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55607b7914e0 con 0x55607b45a1b0 2024-07-04T08:20:31.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b483070 0x55607b827970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 0x55607b827eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b483070 0x55607b827970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38330/0 (socket says 172.21.15.39:38330) 2024-07-04T08:20:31.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c8c00640 1 -- 172.21.15.39:0/2198271276 learned_addr learned my addr 172.21.15.39:0/2198271276 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:31.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b8283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c8c00640 1 -- 172.21.15.39:0/2198271276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 msgr2=0x55607b827eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:31.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c8c00640 1 --2- 172.21.15.39:0/2198271276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 0x55607b827eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c8c00640 1 -- 172.21.15.39:0/2198271276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 msgr2=0x55607b8283f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:31.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c8c00640 1 --2- 172.21.15.39:0/2198271276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b8283f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c8c00640 1 -- 172.21.15.39:0/2198271276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55607b725670 con 0x55607b483070 2024-07-04T08:20:31.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.847+0000 7f28c3e00640 1 --2- 172.21.15.39:0/2198271276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 0x55607b827eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:31.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.848+0000 7f28c8c00640 1 --2- 172.21.15.39:0/2198271276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b483070 0x55607b827970 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0117e0 tx=0x7f28bc011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c728a86160b29e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:31.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.848+0000 7f28b9600640 1 -- 172.21.15.39:0/2198271276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc011310 con 0x55607b483070 2024-07-04T08:20:31.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.848+0000 7f28b9600640 1 -- 172.21.15.39:0/2198271276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28bc003e20 con 0x55607b483070 2024-07-04T08:20:31.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.848+0000 7f28b9600640 1 -- 172.21.15.39:0/2198271276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0044d0 con 0x55607b483070 2024-07-04T08:20:31.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.848+0000 7f28c0c00640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28a80b2960 0x7f28a80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28c0c00640 1 -- 172.21.15.39:0/2644346636 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f28b40e2cd0 con 0x7f28c4076e90 2024-07-04T08:20:31.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28c3e00640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28a80b2960 0x7f28a80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28d1220740 1 -- 172.21.15.39:0/2198271276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b483070 msgr2=0x55607b827970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:31.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28d1220740 1 --2- 172.21.15.39:0/2198271276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b483070 0x55607b827970 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0117e0 tx=0x7f28bc011ca0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28d1220740 1 -- 172.21.15.39:0/2198271276 shutdown_connections 2024-07-04T08:20:31.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28d1220740 1 --2- 172.21.15.39:0/2198271276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b8283f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28d1220740 1 --2- 172.21.15.39:0/2198271276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 0x55607b827eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28d1220740 1 --2- 172.21.15.39:0/2198271276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b483070 0x55607b827970 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28d1220740 1 -- 172.21.15.39:0/2198271276 >> 172.21.15.39:0/2198271276 conn(0x55607b477460 msgr2=0x55607b825860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:31.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28d1220740 1 -- 172.21.15.39:0/2198271276 shutdown_connections 2024-07-04T08:20:31.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.849+0000 7f28c3e00640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28a80b2960 0x7f28a80b4e70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55607b8265f0 tx=0x7f28b40e6780 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:31.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.850+0000 7f28d1220740 1 -- 172.21.15.39:0/2198271276 wait complete. 2024-07-04T08:20:31.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.850+0000 7f28d1220740 1 -- start start 2024-07-04T08:20:31.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.850+0000 7f28d1220740 1 Processor -- start 2024-07-04T08:20:31.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28d1220740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b833560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28d1220740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b833aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28d1220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 0x55607b83f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28d1220740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55607b5f1700 con 0x55607b45a1b0 2024-07-04T08:20:31.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28d1220740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55607b734bd0 con 0x55607b462390 2024-07-04T08:20:31.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28d1220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55607b836150 con 0x55607b483070 2024-07-04T08:20:31.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28d1220740 10 client.0 _ll_register_callbacks cb 0x55607b786590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:31.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28c8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b833560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28c8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b833560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38346/0 (socket says 172.21.15.39:38346) 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28c8c00640 1 -- 172.21.15.39:0/3050842990 learned_addr learned my addr 172.21.15.39:0/3050842990 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.851+0000 7f28c3400640 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b833aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.852+0000 7f28c8c00640 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 0x55607b83f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.852+0000 7f28c3400640 1 -- 172.21.15.39:0/3050842990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 msgr2=0x55607b83f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.852+0000 7f28c3400640 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 0x55607b83f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.852+0000 7f28c3400640 1 -- 172.21.15.39:0/3050842990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 msgr2=0x55607b833560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.852+0000 7f28c3400640 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b833560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.852+0000 7f28c3400640 1 -- 172.21.15.39:0/3050842990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55607b4ee650 con 0x55607b462390 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.852+0000 7f28c8c00640 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 0x55607b83f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.852+0000 7f28c3400640 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b833aa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f28b002a910 tx=0x7f28b0032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550429ccb4995a10 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.852+0000 7f28ae000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:31.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.853+0000 7f28ae000640 1 -- 172.21.15.39:0/3050842990 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0035040 con 0x55607b462390 2024-07-04T08:20:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.853+0000 7f28d1220740 1 -- 172.21.15.39:0/3050842990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28bc003e20 con 0x55607b462390 2024-07-04T08:20:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.853+0000 7f28ae000640 1 -- 172.21.15.39:0/3050842990 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f28b0031760 con 0x55607b462390 2024-07-04T08:20:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.853+0000 7f28ae000640 1 -- 172.21.15.39:0/3050842990 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f28b0004040 con 0x55607b462390 2024-07-04T08:20:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.853+0000 7f28d1220740 10 client.26786 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:31.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.853+0000 7f28d1220740 1 -- 172.21.15.39:0/3050842990 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55607b8421b0 con 0x55607b462390 2024-07-04T08:20:31.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.854+0000 7f28ae000640 1 -- 172.21.15.39:0/3050842990 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28bc003e20 con 0x55607b462390 2024-07-04T08:20:31.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.854+0000 7f28ae000640 1 -- 172.21.15.39:0/3050842990 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f28b0004750 con 0x55607b462390 2024-07-04T08:20:31.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.854+0000 7f28d1220740 10 client.26786 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.854+0000 7f28d1220740 10 client.26786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.854+0000 7f28d1220740 20 client.26786 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.854+0000 7f28d1220740 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607b4419f0 0x55607b8456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.854+0000 7f28d1220740 4 client.26786 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.854+0000 7f28d1220740 1 -- 172.21.15.39:0/3050842990 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55607b79fe50 con 0x55607b4419f0 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.854+0000 7f28c8c00640 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607b4419f0 0x55607b8456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.855+0000 7f28c8c00640 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607b4419f0 0x55607b8456c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ce4351c615d574 server_cookie=d3217e672b60a7d4 in_seq=0 out_seq=0 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.855+0000 7f28ae000640 10 client.26786 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28ae000640 1 -- 172.21.15.39:0/3050842990 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55607b79fe50 con 0x55607b4419f0 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28ae000640 10 client.26786 handle_command_reply: tid=0 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 client.26786 shutdown 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 -- 172.21.15.39:0/3050842990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 msgr2=0x55607b833aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:31.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b833aa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f28b002a910 tx=0x7f28b0032e90 comp rx=0 tx=0).stop 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 -- 172.21.15.39:0/3050842990 shutdown_connections 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607b4419f0 0x55607b8456c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 0x55607b83f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b833aa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 --2- 172.21.15.39:0/3050842990 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b833560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28ae000640 0 client.26786 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 -- 172.21.15.39:0/3050842990 >> 172.21.15.39:0/3050842990 conn(0x55607b477460 msgr2=0x55607b82ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 -- 172.21.15.39:0/3050842990 shutdown_connections 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.856+0000 7f28d1220740 1 -- 172.21.15.39:0/3050842990 wait complete. 2024-07-04T08:20:31.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.857+0000 7f28d1220740 20 client.26786 trim_cache size 0 max 16384 2024-07-04T08:20:31.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.872+0000 7f28d1220740 1 Processor -- start 2024-07-04T08:20:31.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.872+0000 7f28d1220740 1 -- start start 2024-07-04T08:20:31.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28d1220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 0x55607b8290b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28d1220740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b462390 0x55607b83f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28d1220740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b483070 0x55607b83f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28d1220740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b65f0 con 0x55607b462390 2024-07-04T08:20:31.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28d1220740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b8130 con 0x55607b483070 2024-07-04T08:20:31.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28d1220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b6cc0 con 0x55607b45a1b0 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28c3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b462390 0x55607b83f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28c3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b462390 0x55607b83f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38354/0 (socket says 172.21.15.39:38354) 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28c3400640 1 -- 172.21.15.39:0/340574958 learned_addr learned my addr 172.21.15.39:0/340574958 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28c8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 0x55607b8290b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.873+0000 7f28c8c00640 1 --2- 172.21.15.39:0/340574958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b483070 0x55607b83f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.874+0000 7f28c3400640 1 -- 172.21.15.39:0/340574958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 msgr2=0x55607b8290b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.874+0000 7f28c3400640 1 --2- 172.21.15.39:0/340574958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 0x55607b8290b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.874+0000 7f28c3400640 1 -- 172.21.15.39:0/340574958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b483070 msgr2=0x55607b83f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.874+0000 7f28c3400640 1 --2- 172.21.15.39:0/340574958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b483070 0x55607b83f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.874+0000 7f28c3400640 1 -- 172.21.15.39:0/340574958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55607b82a200 con 0x55607b462390 2024-07-04T08:20:31.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.874+0000 7f28c3400640 1 --2- 172.21.15.39:0/340574958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b462390 0x55607b83f260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006010 tx=0x7f28b0005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6852665c52345dd8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:31.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.874+0000 7f28b8c00640 1 -- 172.21.15.39:0/340574958 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0035070 con 0x55607b462390 2024-07-04T08:20:31.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.875+0000 7f28b8c00640 1 -- 172.21.15.39:0/340574958 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b00048c0 con 0x55607b462390 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.875+0000 7f28b8c00640 1 -- 172.21.15.39:0/340574958 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0004d60 con 0x55607b462390 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 -- 172.21.15.39:0/340574958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b462390 msgr2=0x55607b83f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 --2- 172.21.15.39:0/340574958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b462390 0x55607b83f260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006010 tx=0x7f28b0005ec0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 -- 172.21.15.39:0/340574958 shutdown_connections 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 --2- 172.21.15.39:0/340574958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b462390 0x55607b83f260 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 --2- 172.21.15.39:0/340574958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b483070 0x55607b83f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 --2- 172.21.15.39:0/340574958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b45a1b0 0x55607b8290b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 -- 172.21.15.39:0/340574958 >> 172.21.15.39:0/340574958 conn(0x55607b477460 msgr2=0x55607b8289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 -- 172.21.15.39:0/340574958 shutdown_connections 2024-07-04T08:20:31.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 -- 172.21.15.39:0/340574958 wait complete. 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.876+0000 7f28d1220740 1 -- start start 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.877+0000 7f28d1220740 1 Processor -- start 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.877+0000 7f28d1220740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b83ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.877+0000 7f28d1220740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b836f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.877+0000 7f28d1220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 0x55607b8374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.877+0000 7f28d1220740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b7a60 con 0x55607b45a1b0 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.877+0000 7f28d1220740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b7390 con 0x55607b462390 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.877+0000 7f28d1220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41b45d0 con 0x55607b483070 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.877+0000 7f28d1220740 10 client.0 _ll_register_callbacks cb 0x7f28b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b83ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b836f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b83ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38368/0 (socket says 172.21.15.39:38368) 2024-07-04T08:20:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c8c00640 1 -- 172.21.15.39:0/3391846229 learned_addr learned my addr 172.21.15.39:0/3391846229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:31.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c8c00640 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 0x55607b8374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c3400640 1 -- 172.21.15.39:0/3391846229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 msgr2=0x55607b8374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c3400640 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 0x55607b8374d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c3400640 1 -- 172.21.15.39:0/3391846229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 msgr2=0x55607b83ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c3400640 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b83ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.878+0000 7f28c3400640 1 -- 172.21.15.39:0/3391846229 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55607b837c40 con 0x55607b462390 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.879+0000 7f28c3400640 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b836f90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f28b0005f70 tx=0x7f28b0003640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e9b0f6371b320c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.879+0000 7f28ae000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.879+0000 7f28ae000640 1 -- 172.21.15.39:0/3391846229 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0035070 con 0x55607b462390 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.879+0000 7f28ae000640 1 -- 172.21.15.39:0/3391846229 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f28b00046c0 con 0x55607b462390 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.879+0000 7f28d1220740 1 -- 172.21.15.39:0/3391846229 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b00048c0 con 0x55607b462390 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28ae000640 1 -- 172.21.15.39:0/3391846229 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f28b006a050 con 0x55607b462390 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28d1220740 10 client.26792 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28d1220740 1 -- 172.21.15.39:0/3391846229 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55607b8addc0 con 0x55607b462390 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28ae000640 1 -- 172.21.15.39:0/3391846229 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b00048c0 con 0x55607b462390 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28ae000640 1 -- 172.21.15.39:0/3391846229 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f28b0065030 con 0x55607b462390 2024-07-04T08:20:31.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28d1220740 10 client.26792 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28d1220740 10 client.26792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28d1220740 20 client.26792 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28d1220740 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607b416da0 0x55607b8b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28d1220740 4 client.26792 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:20:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.880+0000 7f28d1220740 1 -- 172.21.15.39:0/3391846229 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55607b79b8f0 con 0x55607b416da0 2024-07-04T08:20:31.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.881+0000 7f28c8c00640 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607b416da0 0x55607b8b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.881+0000 7f28c8c00640 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607b416da0 0x55607b8b04e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec85a8d4c40b4fac server_cookie=b710cf5684f7fb79 in_seq=0 out_seq=0 2024-07-04T08:20:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.881+0000 7f28ae000640 10 client.26792 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28ae000640 1 -- 172.21.15.39:0/3391846229 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55607b79b8f0 con 0x55607b416da0 2024-07-04T08:20:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28ae000640 10 client.26792 handle_command_reply: tid=0 2024-07-04T08:20:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 client.26792 shutdown 2024-07-04T08:20:31.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 -- 172.21.15.39:0/3391846229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 msgr2=0x55607b836f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b836f90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f28b0005f70 tx=0x7f28b0003640 comp rx=0 tx=0).stop 2024-07-04T08:20:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 -- 172.21.15.39:0/3391846229 shutdown_connections 2024-07-04T08:20:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607b416da0 0x55607b8b04e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607b483070 0x55607b8374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607b462390 0x55607b836f90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 --2- 172.21.15.39:0/3391846229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607b45a1b0 0x55607b83ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28ae000640 0 client.26792 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 -- 172.21.15.39:0/3391846229 >> 172.21.15.39:0/3391846229 conn(0x55607b477460 msgr2=0x55607b834790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:31.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 -- 172.21.15.39:0/3391846229 shutdown_connections 2024-07-04T08:20:31.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 1 -- 172.21.15.39:0/3391846229 wait complete. 2024-07-04T08:20:31.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.882+0000 7f28d1220740 20 client.26792 trim_cache size 0 max 16384 2024-07-04T08:20:31.882 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:20:31.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 -- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28a80b2960 msgr2=0x7f28a80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28a80b2960 0x7f28a80b4e70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55607b8265f0 tx=0x7f28b40e6780 comp rx=0 tx=0).stop 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 -- 172.21.15.39:0/2644346636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c4076e90 msgr2=0x7f28c419fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c4076e90 0x7f28c419fe10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f28b400e9f0 tx=0x7f28b400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 -- 172.21.15.39:0/2644346636 shutdown_connections 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28a80b2960 0x7f28a80b4e70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28c4113ad0 0x7f28c41a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28c40777e0 0x7f28c41a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 --2- 172.21.15.39:0/2644346636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c4076e90 0x7f28c419fe10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.883+0000 7f28c9600640 1 -- 172.21.15.39:0/2644346636 >> 172.21.15.39:0/2644346636 conn(0x7f28c4100b80 msgr2=0x7f28c4115bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.884+0000 7f28c9600640 1 -- 172.21.15.39:0/2644346636 shutdown_connections 2024-07-04T08:20:31.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:31.884+0000 7f28c9600640 1 -- 172.21.15.39:0/2644346636 wait complete. 2024-07-04T08:20:33.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:33 smithi133 ceph-mon[37050]: pgmap v640: 129 pgs: 129 active+clean; 5.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-07-04T08:20:33.597 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:33 smithi039 ceph-mon[32062]: pgmap v640: 129 pgs: 129 active+clean; 5.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-07-04T08:20:33.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:33 smithi050 ceph-mon[36577]: pgmap v640: 129 pgs: 129 active+clean; 5.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-07-04T08:20:33.813 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:20:33.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:20:34.303 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:35.494 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:35 smithi039 ceph-mon[32062]: pgmap v641: 129 pgs: 129 active+clean; 5.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-07-04T08:20:35.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:35 smithi133 ceph-mon[37050]: pgmap v641: 129 pgs: 129 active+clean; 5.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-07-04T08:20:35.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:35 smithi050 ceph-mon[36577]: pgmap v641: 129 pgs: 129 active+clean; 5.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-07-04T08:20:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.667+0000 7f250f200640 1 -- 172.21.15.39:0/2587802977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2510102f00 msgr2=0x7f2510103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.667+0000 7f250f200640 1 --2- 172.21.15.39:0/2587802977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2510102f00 0x7f2510103310 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2500009860 tx=0x7f250002a340 comp rx=0 tx=0).stop 2024-07-04T08:20:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.667+0000 7f250f200640 1 -- 172.21.15.39:0/2587802977 shutdown_connections 2024-07-04T08:20:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.667+0000 7f250f200640 1 --2- 172.21.15.39:0/2587802977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2510109190 0x7f251010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.667+0000 7f250f200640 1 --2- 172.21.15.39:0/2587802977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2510108840 0x7f2510108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.667+0000 7f250f200640 1 --2- 172.21.15.39:0/2587802977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2510102f00 0x7f2510103310 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.667+0000 7f250f200640 1 -- 172.21.15.39:0/2587802977 >> 172.21.15.39:0/2587802977 conn(0x7f25100fe950 msgr2=0x7f2510100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.668+0000 7f250f200640 1 -- 172.21.15.39:0/2587802977 shutdown_connections 2024-07-04T08:20:36.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.668+0000 7f250f200640 1 -- 172.21.15.39:0/2587802977 wait complete. 2024-07-04T08:20:36.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.669+0000 7f250f200640 1 Processor -- start 2024-07-04T08:20:36.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.669+0000 7f250f200640 1 -- start start 2024-07-04T08:20:36.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2510102f00 0x7f251019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2510108840 0x7f251019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2510109190 0x7f25101a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2510112e50 con 0x7f2510108840 2024-07-04T08:20:36.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2510112cd0 con 0x7f2510102f00 2024-07-04T08:20:36.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2510112fd0 con 0x7f2510109190 2024-07-04T08:20:36.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2510108840 0x7f251019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2510102f00 0x7f251019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2510102f00 0x7f251019baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48810/0 (socket says 172.21.15.39:48810) 2024-07-04T08:20:36.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250de00640 1 -- 172.21.15.39:0/1888220802 learned_addr learned my addr 172.21.15.39:0/1888220802 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:36.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.670+0000 7f250e800640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2510109190 0x7f25101a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.671+0000 7f250de00640 1 -- 172.21.15.39:0/1888220802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2510109190 msgr2=0x7f25101a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:36.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.671+0000 7f250de00640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2510109190 0x7f25101a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.671+0000 7f250de00640 1 -- 172.21.15.39:0/1888220802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2510108840 msgr2=0x7f251019c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:36.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.671+0000 7f250de00640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2510108840 0x7f251019c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.671+0000 7f250de00640 1 -- 172.21.15.39:0/1888220802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25101a1a10 con 0x7f2510102f00 2024-07-04T08:20:36.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.671+0000 7f250e800640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2510109190 0x7f25101a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:36.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.671+0000 7f250de00640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2510102f00 0x7f251019baa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2500035040 tx=0x7f2500032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.672+0000 7f2506a00640 1 -- 172.21.15.39:0/1888220802 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500037040 con 0x7f2510102f00 2024-07-04T08:20:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.672+0000 7f2506a00640 1 -- 172.21.15.39:0/1888220802 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2500041070 con 0x7f2510102f00 2024-07-04T08:20:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.672+0000 7f250f200640 1 -- 172.21.15.39:0/1888220802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25101a1ca0 con 0x7f2510102f00 2024-07-04T08:20:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.672+0000 7f2506a00640 1 -- 172.21.15.39:0/1888220802 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500003850 con 0x7f2510102f00 2024-07-04T08:20:36.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.672+0000 7f250f200640 1 -- 172.21.15.39:0/1888220802 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25101a8460 con 0x7f2510102f00 2024-07-04T08:20:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.675+0000 7f251b347740 1 Processor -- start 2024-07-04T08:20:36.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.675+0000 7f251b347740 1 -- start start 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f2506a00640 1 -- 172.21.15.39:0/1888220802 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f25000039f0 con 0x7f2510102f00 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f251b347740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8c9ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f251b347740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f251b347740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c62a670 0x559d8c9f0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f251b347740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559d8c9198e0 con 0x559d8c64b350 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f251b347740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559d8c965ec0 con 0x559d8c62a670 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f251b347740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559d8c955460 con 0x559d8c622490 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f250e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8c9ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f250e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8c9ef7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38392/0 (socket says 172.21.15.39:38392) 2024-07-04T08:20:36.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f250e800640 1 -- 172.21.15.39:0/2323667401 learned_addr learned my addr 172.21.15.39:0/2323667401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:36.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.676+0000 7f250de00640 1 --2- 172.21.15.39:0/2323667401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.677+0000 7f250d400640 1 --2- 172.21.15.39:0/2323667401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c62a670 0x559d8c9f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.677+0000 7f250e800640 1 -- 172.21.15.39:0/2323667401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 msgr2=0x559d8c9efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.677+0000 7f250e800640 1 --2- 172.21.15.39:0/2323667401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9efd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.677+0000 7f250e800640 1 -- 172.21.15.39:0/2323667401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c62a670 msgr2=0x559d8c9f0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.677+0000 7f250e800640 1 --2- 172.21.15.39:0/2323667401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c62a670 0x559d8c9f0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.677+0000 7f250e800640 1 -- 172.21.15.39:0/2323667401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d8c8ed280 con 0x559d8c64b350 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.677+0000 7f250e800640 1 --2- 172.21.15.39:0/2323667401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8c9ef7f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f250800e8f0 tx=0x7f25080095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e909e3bb6c6804a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.678+0000 7f24fb400640 1 -- 172.21.15.39:0/2323667401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508017420 con 0x559d8c64b350 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.678+0000 7f24fb400640 1 -- 172.21.15.39:0/2323667401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250801c070 con 0x559d8c64b350 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.678+0000 7f24fb400640 1 -- 172.21.15.39:0/2323667401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508016660 con 0x559d8c64b350 2024-07-04T08:20:36.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.678+0000 7f2506a00640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24f40b29e0 0x7f24f40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.678+0000 7f2506a00640 1 -- 172.21.15.39:0/1888220802 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f25001022f0 con 0x7f2510102f00 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.678+0000 7f250de00640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24f40b29e0 0x7f24f40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f251b347740 1 -- 172.21.15.39:0/2323667401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 msgr2=0x559d8c9ef7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f251b347740 1 --2- 172.21.15.39:0/2323667401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8c9ef7f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f250800e8f0 tx=0x7f25080095c0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f251b347740 1 -- 172.21.15.39:0/2323667401 shutdown_connections 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f251b347740 1 --2- 172.21.15.39:0/2323667401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c62a670 0x559d8c9f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f251b347740 1 --2- 172.21.15.39:0/2323667401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f251b347740 1 --2- 172.21.15.39:0/2323667401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8c9ef7f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f251b347740 1 -- 172.21.15.39:0/2323667401 >> 172.21.15.39:0/2323667401 conn(0x559d8c63f740 msgr2=0x559d8c9ed6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f251b347740 1 -- 172.21.15.39:0/2323667401 shutdown_connections 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f251b347740 1 -- 172.21.15.39:0/2323667401 wait complete. 2024-07-04T08:20:36.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.679+0000 7f250de00640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24f40b29e0 0x7f24f40b4ef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2500105fd0 tx=0x7f2500105da0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:36.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.680+0000 7f251b347740 1 -- start start 2024-07-04T08:20:36.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.680+0000 7f251b347740 1 Processor -- start 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f251b347740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c622490 0x559d8c9fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f251b347740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f251b347740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8ca07140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f251b347740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559d8c8fc760 con 0x559d8c64b350 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f251b347740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559d8c9594a0 con 0x559d8c622490 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f251b347740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559d8c7b9890 con 0x559d8c62a670 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f251b347740 10 client.0 _ll_register_callbacks cb 0x559d8c8fd2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f250e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8ca07140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f250e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8ca07140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38396/0 (socket says 172.21.15.39:38396) 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f250e800640 1 -- 172.21.15.39:0/3641036690 learned_addr learned my addr 172.21.15.39:0/3641036690 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f250d400640 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.681+0000 7f250e800640 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c622490 0x559d8c9fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.682+0000 7f250d400640 1 -- 172.21.15.39:0/3641036690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c622490 msgr2=0x559d8c9fb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.682+0000 7f250d400640 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c622490 0x559d8c9fb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.682+0000 7f250d400640 1 -- 172.21.15.39:0/3641036690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 msgr2=0x559d8ca07140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.682+0000 7f250d400640 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8ca07140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.682+0000 7f250d400640 1 -- 172.21.15.39:0/3641036690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d8c6b6930 con 0x559d8c62a670 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.682+0000 7f250e800640 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8ca07140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.682+0000 7f250d400640 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fb820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f251006ac50 tx=0x7f24fc012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7adfc286053a9304 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.682+0000 7f24efe00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.683+0000 7f24efe00640 1 -- 172.21.15.39:0/3641036690 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc004020 con 0x559d8c62a670 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.683+0000 7f24efe00640 1 -- 172.21.15.39:0/3641036690 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f24fc0041c0 con 0x559d8c62a670 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.683+0000 7f251b347740 1 -- 172.21.15.39:0/3641036690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f250801c070 con 0x559d8c62a670 2024-07-04T08:20:36.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.683+0000 7f24efe00640 1 -- 172.21.15.39:0/3641036690 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24fc025050 con 0x559d8c62a670 2024-07-04T08:20:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.683+0000 7f251b347740 10 client.26713 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.683+0000 7f251b347740 1 -- 172.21.15.39:0/3641036690 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d8ca09e40 con 0x559d8c62a670 2024-07-04T08:20:36.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.683+0000 7f24efe00640 1 -- 172.21.15.39:0/3641036690 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250801c070 con 0x559d8c62a670 2024-07-04T08:20:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.684+0000 7f24efe00640 1 -- 172.21.15.39:0/3641036690 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f24fc0039c0 con 0x559d8c62a670 2024-07-04T08:20:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.684+0000 7f251b347740 10 client.26713 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.684+0000 7f251b347740 10 client.26713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:36.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.684+0000 7f251b347740 20 client.26713 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.684+0000 7f251b347740 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559d8c609cd0 0x559d8ca0d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.684+0000 7f251b347740 4 client.26713 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.684+0000 7f251b347740 1 -- 172.21.15.39:0/3641036690 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d8c967e10 con 0x559d8c609cd0 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.684+0000 7f250e800640 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559d8c609cd0 0x559d8ca0d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.685+0000 7f250e800640 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559d8c609cd0 0x559d8ca0d3b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf43f1610e3a1b server_cookie=169d8680a55cfb7e in_seq=0 out_seq=0 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.685+0000 7f24efe00640 10 client.26713 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f24efe00640 1 -- 172.21.15.39:0/3641036690 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559d8c967e10 con 0x559d8c609cd0 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f24efe00640 10 client.26713 handle_command_reply: tid=0 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 client.26713 shutdown 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 -- 172.21.15.39:0/3641036690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 msgr2=0x559d8c9fb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fb820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f251006ac50 tx=0x7f24fc012b60 comp rx=0 tx=0).stop 2024-07-04T08:20:36.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 -- 172.21.15.39:0/3641036690 shutdown_connections 2024-07-04T08:20:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559d8c609cd0 0x559d8ca0d3b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c64b350 0x559d8ca07140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fb820 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 --2- 172.21.15.39:0/3641036690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c622490 0x559d8c9fb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f24efe00640 0 client.26713 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 -- 172.21.15.39:0/3641036690 >> 172.21.15.39:0/3641036690 conn(0x559d8c63f740 msgr2=0x559d8c9f29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 -- 172.21.15.39:0/3641036690 shutdown_connections 2024-07-04T08:20:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 1 -- 172.21.15.39:0/3641036690 wait complete. 2024-07-04T08:20:36.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.686+0000 7f251b347740 20 client.26713 trim_cache size 0 max 16384 2024-07-04T08:20:36.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.707+0000 7f251b347740 1 Processor -- start 2024-07-04T08:20:36.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.708+0000 7f251b347740 1 -- start start 2024-07-04T08:20:36.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.708+0000 7f251b347740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9f5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.708+0000 7f251b347740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c62a670 0x559d8ca06f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.708+0000 7f251b347740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 0x559d8ca07470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.708+0000 7f251b347740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b2280 con 0x559d8c62a670 2024-07-04T08:20:36.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.708+0000 7f251b347740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b3dc0 con 0x559d8c64b350 2024-07-04T08:20:36.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.708+0000 7f251b347740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b2950 con 0x559d8c622490 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.708+0000 7f250e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57490/0 (socket says 172.21.15.39:57490) 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250e800640 1 -- 172.21.15.39:0/2109151205 learned_addr learned my addr 172.21.15.39:0/2109151205 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250e800640 1 --2- 172.21.15.39:0/2109151205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 0x559d8ca07470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250d400640 1 --2- 172.21.15.39:0/2109151205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c62a670 0x559d8ca06f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250e800640 1 -- 172.21.15.39:0/2109151205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 msgr2=0x559d8ca07470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250e800640 1 --2- 172.21.15.39:0/2109151205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 0x559d8ca07470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250e800640 1 -- 172.21.15.39:0/2109151205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c62a670 msgr2=0x559d8ca06f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250e800640 1 --2- 172.21.15.39:0/2109151205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c62a670 0x559d8ca06f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250e800640 1 -- 172.21.15.39:0/2109151205 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d8c9ee4b0 con 0x559d8c622490 2024-07-04T08:20:36.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.709+0000 7f250e800640 1 --2- 172.21.15.39:0/2109151205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9f5910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f250800d7b0 tx=0x7f250800e500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2f47215510cbf78 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.710+0000 7f24faa00640 1 -- 172.21.15.39:0/2109151205 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508003970 con 0x559d8c622490 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.710+0000 7f24faa00640 1 -- 172.21.15.39:0/2109151205 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250801c040 con 0x559d8c622490 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.710+0000 7f24faa00640 1 -- 172.21.15.39:0/2109151205 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25080046b0 con 0x559d8c622490 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.710+0000 7f251b347740 1 -- 172.21.15.39:0/2109151205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 msgr2=0x559d8c9f5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.710+0000 7f251b347740 1 --2- 172.21.15.39:0/2109151205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9f5910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f250800d7b0 tx=0x7f250800e500 comp rx=0 tx=0).stop 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.711+0000 7f251b347740 1 -- 172.21.15.39:0/2109151205 shutdown_connections 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.711+0000 7f251b347740 1 --2- 172.21.15.39:0/2109151205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c62a670 0x559d8ca06f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.711+0000 7f251b347740 1 --2- 172.21.15.39:0/2109151205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 0x559d8ca07470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.711+0000 7f251b347740 1 --2- 172.21.15.39:0/2109151205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c622490 0x559d8c9f5910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.711+0000 7f251b347740 1 -- 172.21.15.39:0/2109151205 >> 172.21.15.39:0/2109151205 conn(0x559d8c63f740 msgr2=0x559d8c9f11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.711+0000 7f251b347740 1 -- 172.21.15.39:0/2109151205 shutdown_connections 2024-07-04T08:20:36.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.711+0000 7f251b347740 1 -- 172.21.15.39:0/2109151205 wait complete. 2024-07-04T08:20:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.711+0000 7f251b347740 1 -- start start 2024-07-04T08:20:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.712+0000 7f251b347740 1 Processor -- start 2024-07-04T08:20:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.712+0000 7f251b347740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c622490 0x559d8c9fe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.712+0000 7f251b347740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.712+0000 7f251b347740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 0x559d8ca76410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.712+0000 7f251b347740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b36f0 con 0x559d8c622490 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.712+0000 7f251b347740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b3020 con 0x559d8c64b350 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.712+0000 7f251b347740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b0260 con 0x559d8c62a670 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.712+0000 7f251b347740 10 client.0 _ll_register_callbacks cb 0x7f24fc009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c622490 0x559d8c9fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c622490 0x559d8c9fe990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38410/0 (socket says 172.21.15.39:38410) 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250e800640 1 -- 172.21.15.39:0/1074620224 learned_addr learned my addr 172.21.15.39:0/1074620224 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:36.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250e800640 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 0x559d8ca76410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250d400640 1 -- 172.21.15.39:0/1074620224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 msgr2=0x559d8ca76410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:36.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250d400640 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 0x559d8ca76410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250d400640 1 -- 172.21.15.39:0/1074620224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c622490 msgr2=0x559d8c9fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250d400640 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c622490 0x559d8c9fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.713+0000 7f250d400640 1 -- 172.21.15.39:0/1074620224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d8ca76a80 con 0x559d8c62a670 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.714+0000 7f250d400640 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fef70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f24fc003d60 tx=0x7f24fc003900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6292beb116015f85 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.714+0000 7f24efe00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.714+0000 7f24efe00640 1 -- 172.21.15.39:0/1074620224 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc02e060 con 0x559d8c62a670 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.714+0000 7f251b347740 1 -- 172.21.15.39:0/1074620224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f250801c040 con 0x559d8c62a670 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.714+0000 7f24efe00640 1 -- 172.21.15.39:0/1074620224 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f24fc018070 con 0x559d8c62a670 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f24efe00640 1 -- 172.21.15.39:0/1074620224 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24fc00f8f0 con 0x559d8c62a670 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f251b347740 10 client.26719 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:36.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f251b347740 1 -- 172.21.15.39:0/1074620224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d8ca7de00 con 0x559d8c62a670 2024-07-04T08:20:36.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f24efe00640 1 -- 172.21.15.39:0/1074620224 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250801c040 con 0x559d8c62a670 2024-07-04T08:20:36.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f24efe00640 1 -- 172.21.15.39:0/1074620224 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f24fc01c470 con 0x559d8c62a670 2024-07-04T08:20:36.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f251b347740 10 client.26719 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:36.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f251b347740 10 client.26719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f251b347740 20 client.26719 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f251b347740 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559d8c5df080 0x559d8ca805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f251b347740 4 client.26719 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:20:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.715+0000 7f251b347740 1 -- 172.21.15.39:0/1074620224 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559d8c9638b0 con 0x559d8c5df080 2024-07-04T08:20:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.716+0000 7f250e800640 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559d8c5df080 0x559d8ca805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.717+0000 7f250e800640 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559d8c5df080 0x559d8ca805f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb145fa400d7a03 server_cookie=5d1a68876d20f3c4 in_seq=0 out_seq=0 2024-07-04T08:20:36.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.717+0000 7f24efe00640 10 client.26719 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:36.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.718+0000 7f24efe00640 1 -- 172.21.15.39:0/1074620224 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559d8c9638b0 con 0x559d8c5df080 2024-07-04T08:20:36.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.718+0000 7f24efe00640 10 client.26719 handle_command_reply: tid=0 2024-07-04T08:20:36.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.718+0000 7f251b347740 1 client.26719 shutdown 2024-07-04T08:20:36.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.718+0000 7f251b347740 1 -- 172.21.15.39:0/1074620224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 msgr2=0x559d8c9fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:36.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.718+0000 7f251b347740 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fef70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f24fc003d60 tx=0x7f24fc003900 comp rx=0 tx=0).stop 2024-07-04T08:20:36.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.718+0000 7f251b347740 1 -- 172.21.15.39:0/1074620224 shutdown_connections 2024-07-04T08:20:36.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.718+0000 7f251b347740 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559d8c5df080 0x559d8ca805f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.718+0000 7f251b347740 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559d8c64b350 0x559d8ca76410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.719+0000 7f24efe00640 0 client.26719 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:36.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.719+0000 7f251b347740 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559d8c62a670 0x559d8c9fef70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.719+0000 7f251b347740 1 --2- 172.21.15.39:0/1074620224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559d8c622490 0x559d8c9fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.719+0000 7f251b347740 1 -- 172.21.15.39:0/1074620224 >> 172.21.15.39:0/1074620224 conn(0x559d8c63f740 msgr2=0x559d8c9f0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:36.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.719+0000 7f251b347740 1 -- 172.21.15.39:0/1074620224 shutdown_connections 2024-07-04T08:20:36.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.719+0000 7f251b347740 1 -- 172.21.15.39:0/1074620224 wait complete. 2024-07-04T08:20:36.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.719+0000 7f251b347740 20 client.26719 trim_cache size 0 max 16384 2024-07-04T08:20:36.717 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:20:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:20:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "ee83dce0-ea7c-45bf-98e1-975d5dfae2e0", 2024-07-04T08:20:36.717 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:20:36.717 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:20:36.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 -- 172.21.15.39:0/1888220802 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24f40b29e0 msgr2=0x7f24f40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:36.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24f40b29e0 0x7f24f40b4ef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2500105fd0 tx=0x7f2500105da0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 -- 172.21.15.39:0/1888220802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2510102f00 msgr2=0x7f251019baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:36.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2510102f00 0x7f251019baa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2500035040 tx=0x7f2500032d50 comp rx=0 tx=0).stop 2024-07-04T08:20:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 -- 172.21.15.39:0/1888220802 shutdown_connections 2024-07-04T08:20:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24f40b29e0 0x7f24f40b4ef0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2510108840 0x7f251019c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2510109190 0x7f25101a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 --2- 172.21.15.39:0/1888220802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2510102f00 0x7f251019baa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 -- 172.21.15.39:0/1888220802 >> 172.21.15.39:0/1888220802 conn(0x7f25100fe950 msgr2=0x7f2510107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 -- 172.21.15.39:0/1888220802 shutdown_connections 2024-07-04T08:20:36.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:36.720+0000 7f250f200640 1 -- 172.21.15.39:0/1888220802 wait complete. 2024-07-04T08:20:37.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:37 smithi133 ceph-mon[37050]: pgmap v642: 129 pgs: 129 active+clean; 5.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-07-04T08:20:37.570 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:37 smithi039 ceph-mon[32062]: pgmap v642: 129 pgs: 129 active+clean; 5.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-07-04T08:20:37.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:37 smithi050 ceph-mon[36577]: pgmap v642: 129 pgs: 129 active+clean; 5.7 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-07-04T08:20:37.650 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:20:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:38.470 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:38 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:20:38.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:38 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:38.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:38 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:20:38.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:38 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:38.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:38 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:20:38.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:38 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:20:38.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:38 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:38.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:38 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:20:38.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:38 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:38.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:38 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:20:38.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:38 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:20:38.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:38 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:38.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:38 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:20:38.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:38 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:20:38.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:38 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:20:39.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:39 smithi039 ceph-mon[32062]: pgmap v643: 129 pgs: 129 active+clean; 5.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-07-04T08:20:39.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:39 smithi133 ceph-mon[37050]: pgmap v643: 129 pgs: 129 active+clean; 5.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-07-04T08:20:39.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:39 smithi050 ceph-mon[36577]: pgmap v643: 129 pgs: 129 active+clean; 5.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-07-04T08:20:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.323+0000 7ff34b200640 1 -- 172.21.15.39:0/930550479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c104660 msgr2=0x7ff34c10cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.323+0000 7ff34b200640 1 --2- 172.21.15.39:0/930550479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c104660 0x7ff34c10cbb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff338009ca0 tx=0x7ff33802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.323+0000 7ff34b200640 1 -- 172.21.15.39:0/930550479 shutdown_connections 2024-07-04T08:20:40.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.323+0000 7ff34b200640 1 --2- 172.21.15.39:0/930550479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c104660 0x7ff34c10cbb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.323+0000 7ff34b200640 1 --2- 172.21.15.39:0/930550479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34c10d170 0x7ff34c10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.323+0000 7ff34b200640 1 --2- 172.21.15.39:0/930550479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff34c103d10 0x7ff34c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.323+0000 7ff34b200640 1 -- 172.21.15.39:0/930550479 >> 172.21.15.39:0/930550479 conn(0x7ff34c0fd810 msgr2=0x7ff34c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:40.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.323+0000 7ff34b200640 1 -- 172.21.15.39:0/930550479 shutdown_connections 2024-07-04T08:20:40.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.323+0000 7ff34b200640 1 -- 172.21.15.39:0/930550479 wait complete. 2024-07-04T08:20:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.325+0000 7ff34b200640 1 Processor -- start 2024-07-04T08:20:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.325+0000 7ff34b200640 1 -- start start 2024-07-04T08:20:40.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.325+0000 7ff34b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34c103d10 0x7ff34c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.325+0000 7ff34b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff34c104660 0x7ff34c19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.325+0000 7ff34b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10d170 0x7ff34c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.325+0000 7ff34b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c112d90 con 0x7ff34c10d170 2024-07-04T08:20:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.325+0000 7ff34b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c112c10 con 0x7ff34c104660 2024-07-04T08:20:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.325+0000 7ff34b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c112f10 con 0x7ff34c103d10 2024-07-04T08:20:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff349e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34c103d10 0x7ff34c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff349e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34c103d10 0x7ff34c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57506/0 (socket says 172.21.15.39:57506) 2024-07-04T08:20:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff349e00640 1 -- 172.21.15.39:0/4278489351 learned_addr learned my addr 172.21.15.39:0/4278489351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:40.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff34a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10d170 0x7ff34c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff349400640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff34c104660 0x7ff34c19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff34a800640 1 -- 172.21.15.39:0/4278489351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34c103d10 msgr2=0x7ff34c19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff34a800640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34c103d10 0x7ff34c19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff34a800640 1 -- 172.21.15.39:0/4278489351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff34c104660 msgr2=0x7ff34c19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff34a800640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff34c104660 0x7ff34c19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff34a800640 1 -- 172.21.15.39:0/4278489351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c1a1990 con 0x7ff34c10d170 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.326+0000 7ff349e00640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34c103d10 0x7ff34c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.327+0000 7ff34a800640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10d170 0x7ff34c1a1260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff34400e970 tx=0x7ff34400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.327+0000 7ff342a00640 1 -- 172.21.15.39:0/4278489351 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34400ccc0 con 0x7ff34c10d170 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.327+0000 7ff342a00640 1 -- 172.21.15.39:0/4278489351 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff344005020 con 0x7ff34c10d170 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.327+0000 7ff34b200640 1 -- 172.21.15.39:0/4278489351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c1a1c80 con 0x7ff34c10d170 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.327+0000 7ff342a00640 1 -- 172.21.15.39:0/4278489351 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344004a00 con 0x7ff34c10d170 2024-07-04T08:20:40.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.327+0000 7ff34b200640 1 -- 172.21.15.39:0/4278489351 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c1a83e0 con 0x7ff34c10d170 2024-07-04T08:20:40.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.331+0000 7ff357092740 1 Processor -- start 2024-07-04T08:20:40.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.331+0000 7ff342a00640 1 -- 172.21.15.39:0/4278489351 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff344004ba0 con 0x7ff34c10d170 2024-07-04T08:20:40.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.332+0000 7ff357092740 1 -- start start 2024-07-04T08:20:40.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.332+0000 7ff357092740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 0x5625328a9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.332+0000 7ff357092740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324dc1b0 0x5625328a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.332+0000 7ff357092740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324e4390 0x5625328aa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.332+0000 7ff357092740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56253281fea0 con 0x562532505070 2024-07-04T08:20:40.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.332+0000 7ff357092740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56253280f440 con 0x5625324e4390 2024-07-04T08:20:40.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.332+0000 7ff357092740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562532813480 con 0x5625324dc1b0 2024-07-04T08:20:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.332+0000 7ff349400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 0x5625328a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.333+0000 7ff349400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 0x5625328a9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38434/0 (socket says 172.21.15.39:38434) 2024-07-04T08:20:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.333+0000 7ff349400640 1 -- 172.21.15.39:0/481604910 learned_addr learned my addr 172.21.15.39:0/481604910 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.333+0000 7ff349e00640 1 --2- 172.21.15.39:0/481604910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324dc1b0 0x5625328a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.333+0000 7ff349400640 1 --2- 172.21.15.39:0/481604910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324e4390 0x5625328aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.333+0000 7ff349e00640 1 -- 172.21.15.39:0/481604910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324e4390 msgr2=0x5625328aa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.333+0000 7ff349e00640 1 --2- 172.21.15.39:0/481604910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324e4390 0x5625328aa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.333+0000 7ff349e00640 1 -- 172.21.15.39:0/481604910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 msgr2=0x5625328a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:40.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.333+0000 7ff349e00640 1 --2- 172.21.15.39:0/481604910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 0x5625328a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.333+0000 7ff349e00640 1 -- 172.21.15.39:0/481604910 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625327a7610 con 0x5625324dc1b0 2024-07-04T08:20:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.334+0000 7ff349e00640 1 --2- 172.21.15.39:0/481604910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324dc1b0 0x5625328a9e50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff33c011510 tx=0x7ff33c0119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=283cbc5835f02602 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:40.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.334+0000 7ff342a00640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3300b2960 0x7ff3300b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.334+0000 7ff337400640 1 -- 172.21.15.39:0/481604910 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c005020 con 0x5625324dc1b0 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.334+0000 7ff342a00640 1 -- 172.21.15.39:0/4278489351 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff3440e2910 con 0x7ff34c10d170 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.334+0000 7ff337400640 1 -- 172.21.15.39:0/481604910 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff33c0047a0 con 0x5625324dc1b0 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.334+0000 7ff337400640 1 -- 172.21.15.39:0/481604910 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c00f8e0 con 0x5625324dc1b0 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.334+0000 7ff349400640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3300b2960 0x7ff3300b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff357092740 1 -- 172.21.15.39:0/481604910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324dc1b0 msgr2=0x5625328a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff349400640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3300b2960 0x7ff3300b4e70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff338009a60 tx=0x7ff3380093d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff357092740 1 --2- 172.21.15.39:0/481604910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324dc1b0 0x5625328a9e50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff33c011510 tx=0x7ff33c0119d0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff357092740 1 -- 172.21.15.39:0/481604910 shutdown_connections 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff357092740 1 --2- 172.21.15.39:0/481604910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324e4390 0x5625328aa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff357092740 1 --2- 172.21.15.39:0/481604910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324dc1b0 0x5625328a9e50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff357092740 1 --2- 172.21.15.39:0/481604910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 0x5625328a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff357092740 1 -- 172.21.15.39:0/481604910 >> 172.21.15.39:0/481604910 conn(0x5625324f9460 msgr2=0x5625328a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff357092740 1 -- 172.21.15.39:0/481604910 shutdown_connections 2024-07-04T08:20:40.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.335+0000 7ff357092740 1 -- 172.21.15.39:0/481604910 wait complete. 2024-07-04T08:20:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.336+0000 7ff357092740 1 -- start start 2024-07-04T08:20:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.336+0000 7ff357092740 1 Processor -- start 2024-07-04T08:20:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff357092740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff357092740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff357092740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 0x5625328c1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff357092740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562532673700 con 0x5625324e4390 2024-07-04T08:20:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff357092740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5625327b6b70 con 0x5625324dc1b0 2024-07-04T08:20:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff357092740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5625328b80f0 con 0x562532505070 2024-07-04T08:20:40.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff357092740 10 client.0 _ll_register_callbacks cb 0x562532808530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:40.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff349e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff349400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff349e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48884/0 (socket says 172.21.15.39:48884) 2024-07-04T08:20:40.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff349e00640 1 -- 172.21.15.39:0/2356141797 learned_addr learned my addr 172.21.15.39:0/2356141797 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:40.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.337+0000 7ff34a800640 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 0x5625328c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.338+0000 7ff349e00640 1 -- 172.21.15.39:0/2356141797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 msgr2=0x5625328c1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:40.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.338+0000 7ff349e00640 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 0x5625328c1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.338+0000 7ff349e00640 1 -- 172.21.15.39:0/2356141797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 msgr2=0x5625328b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.338+0000 7ff349e00640 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.338+0000 7ff349e00640 1 -- 172.21.15.39:0/2356141797 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562532570650 con 0x5625324dc1b0 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.338+0000 7ff349e00640 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328b5410 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96bc6055f7c24105 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.338+0000 7ff32be00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.339+0000 7ff32be00640 1 -- 172.21.15.39:0/2356141797 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c027070 con 0x5625324dc1b0 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.339+0000 7ff357092740 1 -- 172.21.15.39:0/2356141797 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff33c0047a0 con 0x5625324dc1b0 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.339+0000 7ff32be00640 1 -- 172.21.15.39:0/2356141797 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff33c0050d0 con 0x5625324dc1b0 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.339+0000 7ff32be00640 1 -- 172.21.15.39:0/2356141797 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff33c033050 con 0x5625324dc1b0 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.339+0000 7ff357092740 10 client.26822 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:40.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.339+0000 7ff357092740 1 -- 172.21.15.39:0/2356141797 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5625328c4010 con 0x5625324dc1b0 2024-07-04T08:20:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.339+0000 7ff32be00640 1 -- 172.21.15.39:0/2356141797 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff33c0047a0 con 0x5625324dc1b0 2024-07-04T08:20:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.340+0000 7ff32be00640 1 -- 172.21.15.39:0/2356141797 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff33c0035f0 con 0x5625324dc1b0 2024-07-04T08:20:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.340+0000 7ff357092740 10 client.26822 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.340+0000 7ff357092740 10 client.26822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.340+0000 7ff357092740 20 client.26822 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.340+0000 7ff357092740 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625324c39f0 0x5625328c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.340+0000 7ff357092740 4 client.26822 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.340+0000 7ff357092740 1 -- 172.21.15.39:0/2356141797 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562532821df0 con 0x5625324c39f0 2024-07-04T08:20:40.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.340+0000 7ff349400640 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625324c39f0 0x5625328c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.341+0000 7ff349400640 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625324c39f0 0x5625328c73d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484963d4734de9ac server_cookie=fa42e6689341e68c in_seq=0 out_seq=0 2024-07-04T08:20:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.341+0000 7ff32be00640 10 client.26822 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:40.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff32be00640 1 -- 172.21.15.39:0/2356141797 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562532821df0 con 0x5625324c39f0 2024-07-04T08:20:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff32be00640 10 client.26822 handle_command_reply: tid=0 2024-07-04T08:20:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 client.26822 shutdown 2024-07-04T08:20:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 -- 172.21.15.39:0/2356141797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 msgr2=0x5625328b5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328b5410 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000c00 tx=0x7ff33c0042f0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 -- 172.21.15.39:0/2356141797 shutdown_connections 2024-07-04T08:20:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5625324c39f0 0x5625328c73d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 0x5625328c1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 --2- 172.21.15.39:0/2356141797 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328b5410 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 -- 172.21.15.39:0/2356141797 >> 172.21.15.39:0/2356141797 conn(0x5625324f9460 msgr2=0x5625328acaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff32be00640 0 client.26822 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.342+0000 7ff357092740 1 -- 172.21.15.39:0/2356141797 shutdown_connections 2024-07-04T08:20:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.343+0000 7ff357092740 1 -- 172.21.15.39:0/2356141797 wait complete. 2024-07-04T08:20:40.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.343+0000 7ff357092740 20 client.26822 trim_cache size 0 max 16384 2024-07-04T08:20:40.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.363+0000 7ff357092740 1 Processor -- start 2024-07-04T08:20:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.363+0000 7ff357092740 1 -- start start 2024-07-04T08:20:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.363+0000 7ff357092740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328ab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.363+0000 7ff357092740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324e4390 0x5625328b6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff357092740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 0x5625328a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff357092740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1b22b0 con 0x562532505070 2024-07-04T08:20:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff357092740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1b3df0 con 0x5625324dc1b0 2024-07-04T08:20:40.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff357092740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1b2980 con 0x5625324e4390 2024-07-04T08:20:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff34a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 0x5625328a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324e4390 0x5625328b6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328ab300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48900/0 (socket says 172.21.15.39:48900) 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349e00640 1 -- 172.21.15.39:0/2321204485 learned_addr learned my addr 172.21.15.39:0/2321204485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349400640 1 -- 172.21.15.39:0/2321204485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 msgr2=0x5625328ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349400640 1 --2- 172.21.15.39:0/2321204485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328ab300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349400640 1 -- 172.21.15.39:0/2321204485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 msgr2=0x5625328a6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349400640 1 --2- 172.21.15.39:0/2321204485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 0x5625328a6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349400640 1 -- 172.21.15.39:0/2321204485 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625328b0aa0 con 0x5625324e4390 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.364+0000 7ff349e00640 1 --2- 172.21.15.39:0/2321204485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.365+0000 7ff349400640 1 --2- 172.21.15.39:0/2321204485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324e4390 0x5625328b6e40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff33802ade0 tx=0x7ff3380095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d53595630cf242ce server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:40.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.365+0000 7ff336a00640 1 -- 172.21.15.39:0/2321204485 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33805c070 con 0x5625324e4390 2024-07-04T08:20:40.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.365+0000 7ff336a00640 1 -- 172.21.15.39:0/2321204485 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff338036040 con 0x5625324e4390 2024-07-04T08:20:40.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.365+0000 7ff336a00640 1 -- 172.21.15.39:0/2321204485 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338056470 con 0x5625324e4390 2024-07-04T08:20:40.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.366+0000 7ff357092740 1 -- 172.21.15.39:0/2321204485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324e4390 msgr2=0x5625328b6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.366+0000 7ff357092740 1 --2- 172.21.15.39:0/2321204485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324e4390 0x5625328b6e40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff33802ade0 tx=0x7ff3380095c0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.366+0000 7ff357092740 1 -- 172.21.15.39:0/2321204485 shutdown_connections 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.366+0000 7ff357092740 1 --2- 172.21.15.39:0/2321204485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562532505070 0x5625328a6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.366+0000 7ff357092740 1 --2- 172.21.15.39:0/2321204485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5625324e4390 0x5625328b6e40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.366+0000 7ff357092740 1 --2- 172.21.15.39:0/2321204485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.366+0000 7ff357092740 1 -- 172.21.15.39:0/2321204485 >> 172.21.15.39:0/2321204485 conn(0x5625324f9460 msgr2=0x5625328b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.366+0000 7ff357092740 1 -- 172.21.15.39:0/2321204485 shutdown_connections 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.366+0000 7ff357092740 1 -- 172.21.15.39:0/2321204485 wait complete. 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.367+0000 7ff357092740 1 -- start start 2024-07-04T08:20:40.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.367+0000 7ff357092740 1 Processor -- start 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff357092740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff357092740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff357092740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 0x5625328b9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff357092740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1b3720 con 0x5625324e4390 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff357092740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1b3050 con 0x5625324dc1b0 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff357092740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1b0290 con 0x562532505070 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff357092740 10 client.0 _ll_register_callbacks cb 0x7ff344005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff349400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff349400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38462/0 (socket says 172.21.15.39:38462) 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff349400640 1 -- 172.21.15.39:0/3606260748 learned_addr learned my addr 172.21.15.39:0/3606260748 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.368+0000 7ff349e00640 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328bd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.369+0000 7ff34a800640 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 0x5625328b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.369+0000 7ff349e00640 1 -- 172.21.15.39:0/3606260748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 msgr2=0x5625328b9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.369+0000 7ff349e00640 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 0x5625328b9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.369+0000 7ff349e00640 1 -- 172.21.15.39:0/3606260748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 msgr2=0x5625328b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.369+0000 7ff349e00640 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.369+0000 7ff349e00640 1 -- 172.21.15.39:0/3606260748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625328b9870 con 0x5625324dc1b0 2024-07-04T08:20:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.369+0000 7ff349400640 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.369+0000 7ff349e00640 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328bd890 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff33c011ed0 tx=0x7ff33c0094e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0e12128f548587 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:40.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.369+0000 7ff32be00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:40.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.370+0000 7ff32be00640 1 -- 172.21.15.39:0/3606260748 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c027040 con 0x5625324dc1b0 2024-07-04T08:20:40.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.370+0000 7ff357092740 1 -- 172.21.15.39:0/3606260748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff338036040 con 0x5625324dc1b0 2024-07-04T08:20:40.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.370+0000 7ff32be00640 1 -- 172.21.15.39:0/3606260748 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff33c004360 con 0x5625324dc1b0 2024-07-04T08:20:40.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.370+0000 7ff32be00640 1 -- 172.21.15.39:0/3606260748 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff33c0035f0 con 0x5625324dc1b0 2024-07-04T08:20:40.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.370+0000 7ff357092740 10 client.26828 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.370+0000 7ff357092740 1 -- 172.21.15.39:0/3606260748 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562532938040 con 0x5625324dc1b0 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.370+0000 7ff32be00640 1 -- 172.21.15.39:0/3606260748 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff338036040 con 0x5625324dc1b0 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.371+0000 7ff32be00640 1 -- 172.21.15.39:0/3606260748 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff33c00f680 con 0x5625324dc1b0 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.371+0000 7ff357092740 10 client.26828 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.371+0000 7ff357092740 10 client.26828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.371+0000 7ff357092740 20 client.26828 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.371+0000 7ff357092740 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562532498da0 0x56253293a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.371+0000 7ff357092740 4 client.26828 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.371+0000 7ff357092740 1 -- 172.21.15.39:0/3606260748 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56253281d890 con 0x562532498da0 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.371+0000 7ff349400640 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562532498da0 0x56253293a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff349400640 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562532498da0 0x56253293a7a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143068149b40e9c9 server_cookie=8b4285a54ba06b15 in_seq=0 out_seq=0 2024-07-04T08:20:40.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff32be00640 10 client.26828 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff32be00640 1 -- 172.21.15.39:0/3606260748 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56253281d890 con 0x562532498da0 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff32be00640 10 client.26828 handle_command_reply: tid=0 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff357092740 1 client.26828 shutdown 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff357092740 1 -- 172.21.15.39:0/3606260748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 msgr2=0x5625328bd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff357092740 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328bd890 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff33c011ed0 tx=0x7ff33c0094e0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff357092740 1 -- 172.21.15.39:0/3606260748 shutdown_connections 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff357092740 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562532498da0 0x56253293a7a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff357092740 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5625324e4390 0x5625328b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.372+0000 7ff357092740 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562532505070 0x5625328b9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.373+0000 7ff357092740 1 --2- 172.21.15.39:0/3606260748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5625324dc1b0 0x5625328bd890 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.373+0000 7ff357092740 1 -- 172.21.15.39:0/3606260748 >> 172.21.15.39:0/3606260748 conn(0x5625324f9460 msgr2=0x5625328b7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.373+0000 7ff32be00640 0 client.26828 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:40.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.373+0000 7ff357092740 1 -- 172.21.15.39:0/3606260748 shutdown_connections 2024-07-04T08:20:40.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.373+0000 7ff357092740 1 -- 172.21.15.39:0/3606260748 wait complete. 2024-07-04T08:20:40.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.373+0000 7ff357092740 20 client.26828 trim_cache size 0 max 16384 2024-07-04T08:20:40.372 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:20:40.372 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:20:40.372 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:20:40.372 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:20:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 -- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3300b2960 msgr2=0x7ff3300b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3300b2960 0x7ff3300b4e70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff338009a60 tx=0x7ff3380093d0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 -- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10d170 msgr2=0x7ff34c1a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10d170 0x7ff34c1a1260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff34400e970 tx=0x7ff34400ee30 comp rx=0 tx=0).stop 2024-07-04T08:20:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 -- 172.21.15.39:0/4278489351 shutdown_connections 2024-07-04T08:20:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff3300b2960 0x7ff3300b4e70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff34c10d170 0x7ff34c1a1260 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff34c104660 0x7ff34c19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 --2- 172.21.15.39:0/4278489351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34c103d10 0x7ff34c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 -- 172.21.15.39:0/4278489351 >> 172.21.15.39:0/4278489351 conn(0x7ff34c0fd810 msgr2=0x7ff34c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 -- 172.21.15.39:0/4278489351 shutdown_connections 2024-07-04T08:20:40.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:40.374+0000 7ff34b200640 1 -- 172.21.15.39:0/4278489351 wait complete. 2024-07-04T08:20:41.195 INFO:tasks.quiescer.fs.[cephfs]:Total duration (47.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-04T08:20:41.195 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.95 2024-07-04T08:20:41.195 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:20:41.314 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:20:41.314 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:20:41.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:41 smithi133 ceph-mon[37050]: pgmap v644: 129 pgs: 129 active+clean; 5.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-07-04T08:20:41.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:41 smithi039 ceph-mon[32062]: pgmap v644: 129 pgs: 129 active+clean; 5.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-07-04T08:20:41.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:41 smithi050 ceph-mon[36577]: pgmap v644: 129 pgs: 129 active+clean; 5.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-07-04T08:20:41.684 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:42.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:42 smithi039 ceph-mon[32062]: pgmap v645: 129 pgs: 129 active+clean; 5.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-07-04T08:20:42.532 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:42.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:42 smithi133 ceph-mon[37050]: pgmap v645: 129 pgs: 129 active+clean; 5.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-07-04T08:20:42.640 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:42 smithi050 ceph-mon[36577]: pgmap v645: 129 pgs: 129 active+clean; 5.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-07-04T08:20:45.133 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:44 smithi039 ceph-mon[32062]: pgmap v646: 129 pgs: 129 active+clean; 5.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-07-04T08:20:45.264 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:44 smithi133 ceph-mon[37050]: pgmap v646: 129 pgs: 129 active+clean; 5.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-07-04T08:20:45.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.379+0000 7fe7fb600640 1 -- 172.21.15.39:0/733157587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f41030b0 msgr2=0x7fe7f41034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.379+0000 7fe7fb600640 1 --2- 172.21.15.39:0/733157587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f41030b0 0x7fe7f41034c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009900 tx=0x7fe7e802a410 comp rx=0 tx=0).stop 2024-07-04T08:20:45.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.379+0000 7fe7fb600640 1 -- 172.21.15.39:0/733157587 shutdown_connections 2024-07-04T08:20:45.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.379+0000 7fe7fb600640 1 --2- 172.21.15.39:0/733157587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7f4109340 0x7fe7f410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.379+0000 7fe7fb600640 1 --2- 172.21.15.39:0/733157587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f41089f0 0x7fe7f4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.379+0000 7fe7fb600640 1 --2- 172.21.15.39:0/733157587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f41030b0 0x7fe7f41034c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.379+0000 7fe7fb600640 1 -- 172.21.15.39:0/733157587 >> 172.21.15.39:0/733157587 conn(0x7fe7f407ada0 msgr2=0x7fe7f407b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:45.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.379+0000 7fe7fb600640 1 -- 172.21.15.39:0/733157587 shutdown_connections 2024-07-04T08:20:45.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.379+0000 7fe7fb600640 1 -- 172.21.15.39:0/733157587 wait complete. 2024-07-04T08:20:45.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.380+0000 7fe7fb600640 1 Processor -- start 2024-07-04T08:20:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.381+0000 7fe7fb600640 1 -- start start 2024-07-04T08:20:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.381+0000 7fe7fb600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7f41030b0 0x7fe7f4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.381+0000 7fe7fb600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f41089f0 0x7fe7f4199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.381+0000 7fe7fb600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f4109340 0x7fe7f419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.381+0000 7fe7fb600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4112ff0 con 0x7fe7f4109340 2024-07-04T08:20:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.381+0000 7fe7fb600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4112e70 con 0x7fe7f41030b0 2024-07-04T08:20:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.381+0000 7fe7fb600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4113170 con 0x7fe7f41089f0 2024-07-04T08:20:45.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.381+0000 7fe7f9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f4109340 0x7fe7f419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.381+0000 7fe7f9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f4109340 0x7fe7f419f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50720/0 (socket says 172.21.15.39:50720) 2024-07-04T08:20:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f9600640 1 -- 172.21.15.39:0/3187706368 learned_addr learned my addr 172.21.15.39:0/3187706368 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f3e00640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f41089f0 0x7fe7f4199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f8c00640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7f41030b0 0x7fe7f4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:45.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f9600640 1 -- 172.21.15.39:0/3187706368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f41089f0 msgr2=0x7fe7f4199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f9600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f41089f0 0x7fe7f4199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f9600640 1 -- 172.21.15.39:0/3187706368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7f41030b0 msgr2=0x7fe7f4199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f9600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7f41030b0 0x7fe7f4199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f9600640 1 -- 172.21.15.39:0/3187706368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f419f970 con 0x7fe7f4109340 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f8c00640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7f41030b0 0x7fe7f4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.382+0000 7fe7f9600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f4109340 0x7fe7f419f240 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00eb10 tx=0x7fe7ec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.383+0000 7fe7f1600640 1 -- 172.21.15.39:0/3187706368 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec004b20 con 0x7fe7f4109340 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.383+0000 7fe7f1600640 1 -- 172.21.15.39:0/3187706368 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ec004cc0 con 0x7fe7f4109340 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.383+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f419fc60 con 0x7fe7f4109340 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.383+0000 7fe7f1600640 1 -- 172.21.15.39:0/3187706368 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec010670 con 0x7fe7f4109340 2024-07-04T08:20:45.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.383+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f41a63c0 con 0x7fe7f4109340 2024-07-04T08:20:45.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.384+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f41034c0 con 0x7fe7f4109340 2024-07-04T08:20:45.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.386+0000 7fe7f1600640 1 -- 172.21.15.39:0/3187706368 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe7ec005020 con 0x7fe7f4109340 2024-07-04T08:20:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.389+0000 7fe7f1600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7dc0b2960 0x7fe7dc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.389+0000 7fe7f1600640 1 -- 172.21.15.39:0/3187706368 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fe7ec0e2f40 con 0x7fe7f4109340 2024-07-04T08:20:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.389+0000 7fe7f8c00640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7dc0b2960 0x7fe7dc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.390+0000 7fe7f8c00640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7dc0b2960 0x7fe7dc0b4e70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8031440 tx=0x7fe7e8032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.393+0000 7fe7f1600640 1 -- 172.21.15.39:0/3187706368 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fe7ec0aa590 con 0x7fe7f4109340 2024-07-04T08:20:45.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:44 smithi050 ceph-mon[36577]: pgmap v646: 129 pgs: 129 active+clean; 5.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-07-04T08:20:45.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.619+0000 7f039dc00640 1 -- 172.21.15.39:0/4241450971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0398104870 msgr2=0x7f039810cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.619+0000 7f039dc00640 1 --2- 172.21.15.39:0/4241450971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0398104870 0x7f039810cdc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0384009860 tx=0x7f038402a340 comp rx=0 tx=0).stop 2024-07-04T08:20:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.619+0000 7f039dc00640 1 -- 172.21.15.39:0/4241450971 shutdown_connections 2024-07-04T08:20:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.619+0000 7f039dc00640 1 --2- 172.21.15.39:0/4241450971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f039810d380 0x7f039810f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.619+0000 7f039dc00640 1 --2- 172.21.15.39:0/4241450971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0398104870 0x7f039810cdc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.619+0000 7f039dc00640 1 --2- 172.21.15.39:0/4241450971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0398103f20 0x7f0398104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.619+0000 7f039dc00640 1 -- 172.21.15.39:0/4241450971 >> 172.21.15.39:0/4241450971 conn(0x7f0398078c90 msgr2=0x7f03980790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.619+0000 7f039dc00640 1 -- 172.21.15.39:0/4241450971 shutdown_connections 2024-07-04T08:20:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.619+0000 7f039dc00640 1 -- 172.21.15.39:0/4241450971 wait complete. 2024-07-04T08:20:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.620+0000 7f039dc00640 1 Processor -- start 2024-07-04T08:20:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.621+0000 7f039dc00640 1 -- start start 2024-07-04T08:20:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.621+0000 7f039dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0398103f20 0x7f0398199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.621+0000 7f039dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398104870 0x7f0398199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.621+0000 7f039dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039810d380 0x7f039819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.621+0000 7f039dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0398112fc0 con 0x7f039810d380 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.621+0000 7f039dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0398112e40 con 0x7f0398103f20 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.621+0000 7f039dc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0398113140 con 0x7f0398104870 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0396a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0398103f20 0x7f0398199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0397400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039810d380 0x7f039819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0396a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0398103f20 0x7f0398199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57762/0 (socket says 172.21.15.39:57762) 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0396a00640 1 -- 172.21.15.39:0/97980713 learned_addr learned my addr 172.21.15.39:0/97980713 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0396000640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398104870 0x7f0398199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0396000640 1 -- 172.21.15.39:0/97980713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0398103f20 msgr2=0x7f0398199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0396000640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0398103f20 0x7f0398199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0396000640 1 -- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039810d380 msgr2=0x7f039819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0396000640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039810d380 0x7f039819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0396000640 1 -- 172.21.15.39:0/97980713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039819f980 con 0x7f0398104870 2024-07-04T08:20:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.622+0000 7f0397400640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039810d380 0x7f039819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.623+0000 7f0396000640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398104870 0x7f0398199fb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f038402a860 tx=0x7f0384032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:45.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.623+0000 7f038f400640 1 -- 172.21.15.39:0/97980713 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384040020 con 0x7f0398104870 2024-07-04T08:20:45.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.623+0000 7f038f400640 1 -- 172.21.15.39:0/97980713 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0384030e10 con 0x7f0398104870 2024-07-04T08:20:45.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.623+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039819fc10 con 0x7f0398104870 2024-07-04T08:20:45.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.623+0000 7f038f400640 1 -- 172.21.15.39:0/97980713 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384037040 con 0x7f0398104870 2024-07-04T08:20:45.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.623+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03981a63d0 con 0x7f0398104870 2024-07-04T08:20:45.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.625+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039806b790 con 0x7f0398104870 2024-07-04T08:20:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.627+0000 7f038f400640 1 -- 172.21.15.39:0/97980713 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0384043020 con 0x7f0398104870 2024-07-04T08:20:45.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.629+0000 7f038f400640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f037c0b2960 0x7f037c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:45.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.629+0000 7f038f400640 1 -- 172.21.15.39:0/97980713 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0384102bd0 con 0x7f0398104870 2024-07-04T08:20:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.629+0000 7f0396a00640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f037c0b2960 0x7f037c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.630+0000 7f0396a00640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f037c0b2960 0x7f037c0b4e70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f03880096c0 tx=0x7f0388009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.631+0000 7f038f400640 1 -- 172.21.15.39:0/97980713 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f038404a020 con 0x7f0398104870 2024-07-04T08:20:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.672+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7f402de10 con 0x7fe7f4109340 2024-07-04T08:20:45.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.673+0000 7fe7f1600640 1 -- 172.21.15.39:0/3187706368 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fe7ec0a9e90 con 0x7fe7f4109340 2024-07-04T08:20:45.670 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:20:45.670 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:20:45.670 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:20:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.676+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7dc0b2960 msgr2=0x7fe7dc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.676+0000 7fe7fb600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7dc0b2960 0x7fe7dc0b4e70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8031440 tx=0x7fe7e8032ed0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.676+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f4109340 msgr2=0x7fe7f419f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.676+0000 7fe7fb600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f4109340 0x7fe7f419f240 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00eb10 tx=0x7fe7ec00c2d0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.677+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 shutdown_connections 2024-07-04T08:20:45.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.677+0000 7fe7fb600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7dc0b2960 0x7fe7dc0b4e70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.677+0000 7fe7fb600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7f4109340 0x7fe7f419f240 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.677+0000 7fe7fb600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7f41089f0 0x7fe7f4199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.677+0000 7fe7fb600640 1 --2- 172.21.15.39:0/3187706368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe7f41030b0 0x7fe7f4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.677+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 >> 172.21.15.39:0/3187706368 conn(0x7fe7f407ada0 msgr2=0x7fe7f4106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:45.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.677+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 shutdown_connections 2024-07-04T08:20:45.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.677+0000 7fe7fb600640 1 -- 172.21.15.39:0/3187706368 wait complete. 2024-07-04T08:20:45.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.882+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0398104330 con 0x7f0398104870 2024-07-04T08:20:45.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.883+0000 7f038f400640 1 -- 172.21.15.39:0/97980713 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f03840ca3b0 con 0x7f0398104870 2024-07-04T08:20:45.880 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:20:45.880 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:20:45.881 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:20:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f037c0b2960 msgr2=0x7f037c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f037c0b2960 0x7f037c0b4e70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f03880096c0 tx=0x7f0388009260 comp rx=0 tx=0).stop 2024-07-04T08:20:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398104870 msgr2=0x7f0398199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398104870 0x7f0398199fb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f038402a860 tx=0x7f0384032d50 comp rx=0 tx=0).stop 2024-07-04T08:20:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 shutdown_connections 2024-07-04T08:20:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f037c0b2960 0x7f037c0b4e70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f039810d380 0x7f039819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0398104870 0x7f0398199fb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 --2- 172.21.15.39:0/97980713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0398103f20 0x7f0398199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 >> 172.21.15.39:0/97980713 conn(0x7f0398078c90 msgr2=0x7f0398105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.887+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 shutdown_connections 2024-07-04T08:20:45.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:45.888+0000 7f039dc00640 1 -- 172.21.15.39:0/97980713 wait complete. 2024-07-04T08:20:46.287 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:45 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3187706368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:20:46.287 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:45 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/97980713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:20:46.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:45 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3187706368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:20:46.312 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:45 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/97980713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:20:46.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:45 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3187706368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:20:46.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:45 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/97980713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:20:46.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:20:47.180 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:46 smithi039 ceph-mon[32062]: pgmap v647: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-07-04T08:20:47.258 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 143.95 --await-for 45.0' 2024-07-04T08:20:47.258 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 143.95 --await-for 45.0 2024-07-04T08:20:47.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:46 smithi133 ceph-mon[37050]: pgmap v647: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-07-04T08:20:47.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:46 smithi050 ceph-mon[36577]: pgmap v647: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-07-04T08:20:47.469 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:48.378 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:49.223 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:48 smithi039 ceph-mon[32062]: pgmap v648: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-07-04T08:20:49.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:48 smithi133 ceph-mon[37050]: pgmap v648: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-07-04T08:20:49.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:48 smithi050 ceph-mon[36577]: pgmap v648: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-07-04T08:20:50.778 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:20:50.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.354+0000 7f2b1f200640 1 -- 172.21.15.39:0/654519900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b180777e0 msgr2=0x7f2b18113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.354+0000 7f2b1f200640 1 --2- 172.21.15.39:0/654519900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b180777e0 0x7f2b18113780 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009860 tx=0x7f2b0402a340 comp rx=0 tx=0).stop 2024-07-04T08:20:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.354+0000 7f2b1f200640 1 -- 172.21.15.39:0/654519900 shutdown_connections 2024-07-04T08:20:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.354+0000 7f2b1f200640 1 --2- 172.21.15.39:0/654519900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b18113cc0 0x7f2b181160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.354+0000 7f2b1f200640 1 --2- 172.21.15.39:0/654519900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b180777e0 0x7f2b18113780 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.354+0000 7f2b1f200640 1 --2- 172.21.15.39:0/654519900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b18076e90 0x7f2b180772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.354+0000 7f2b1f200640 1 -- 172.21.15.39:0/654519900 >> 172.21.15.39:0/654519900 conn(0x7f2b18100d10 msgr2=0x7f2b18103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.355+0000 7f2b1f200640 1 -- 172.21.15.39:0/654519900 shutdown_connections 2024-07-04T08:20:50.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.355+0000 7f2b1f200640 1 -- 172.21.15.39:0/654519900 wait complete. 2024-07-04T08:20:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.356+0000 7f2b1f200640 1 Processor -- start 2024-07-04T08:20:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.356+0000 7f2b1f200640 1 -- start start 2024-07-04T08:20:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.356+0000 7f2b1f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b18076e90 0x7f2b181a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.356+0000 7f2b1f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b180777e0 0x7f2b181a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b1f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b18113cc0 0x7f2b181a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b1f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181195f0 con 0x7f2b180777e0 2024-07-04T08:20:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b1f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18119470 con 0x7f2b18076e90 2024-07-04T08:20:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b1f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18119770 con 0x7f2b18113cc0 2024-07-04T08:20:50.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b17400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b180777e0 0x7f2b181a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b17e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b18076e90 0x7f2b181a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b17e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b18076e90 0x7f2b181a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57786/0 (socket says 172.21.15.39:57786) 2024-07-04T08:20:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b17e00640 1 -- 172.21.15.39:0/1263753530 learned_addr learned my addr 172.21.15.39:0/1263753530 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:50.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b1d200640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b18113cc0 0x7f2b181a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b17e00640 1 -- 172.21.15.39:0/1263753530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b18113cc0 msgr2=0x7f2b181a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b17e00640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b18113cc0 0x7f2b181a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b17e00640 1 -- 172.21.15.39:0/1263753530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b180777e0 msgr2=0x7f2b181a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b17e00640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b180777e0 0x7f2b181a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.357+0000 7f2b17e00640 1 -- 172.21.15.39:0/1263753530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b181a5ff0 con 0x7f2b18076e90 2024-07-04T08:20:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.358+0000 7f2b17e00640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b18076e90 0x7f2b181a0080 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e9c0 tx=0x7f2b0800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.358+0000 7f2b14c00640 1 -- 172.21.15.39:0/1263753530 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800ccc0 con 0x7f2b18076e90 2024-07-04T08:20:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.358+0000 7f2b1f200640 1 -- 172.21.15.39:0/1263753530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b181a62e0 con 0x7f2b18076e90 2024-07-04T08:20:50.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.358+0000 7f2b14c00640 1 -- 172.21.15.39:0/1263753530 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b080054d0 con 0x7f2b18076e90 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.358+0000 7f2b14c00640 1 -- 172.21.15.39:0/1263753530 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08010430 con 0x7f2b18076e90 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.358+0000 7f2b1f200640 1 -- 172.21.15.39:0/1263753530 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b181aca40 con 0x7f2b18076e90 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.362+0000 7f2b2586d740 1 Processor -- start 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.362+0000 7f2b2586d740 1 -- start start 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.362+0000 7f2b14c00640 1 -- 172.21.15.39:0/1263753530 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2b08004f80 con 0x7f2b18076e90 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300fc070 0x5562304a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300db390 0x5562304a1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b2586d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556230416f00 con 0x5562300d31b0 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b2586d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5562304064a0 con 0x5562300fc070 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b2586d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55623040a4e0 con 0x5562300db390 2024-07-04T08:20:50.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300fc070 0x5562304a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300fc070 0x5562304a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57788/0 (socket says 172.21.15.39:57788) 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b1d200640 1 -- 172.21.15.39:0/2525184091 learned_addr learned my addr 172.21.15.39:0/2525184091 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b17e00640 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b17400640 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300db390 0x5562304a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b1d200640 1 -- 172.21.15.39:0/2525184091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300db390 msgr2=0x5562304a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b1d200640 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300db390 0x5562304a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b1d200640 1 -- 172.21.15.39:0/2525184091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 msgr2=0x5562304a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b1d200640 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b1d200640 1 -- 172.21.15.39:0/2525184091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623039e670 con 0x5562300fc070 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.363+0000 7f2b17400640 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300db390 0x5562304a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:50.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.364+0000 7f2b1d200640 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300fc070 0x5562304a07d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009f50 tx=0x7f2b100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4c7836379eb3f99 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.364+0000 7f2b0d600640 1 -- 172.21.15.39:0/2525184091 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10004d20 con 0x5562300fc070 2024-07-04T08:20:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.364+0000 7f2b0d600640 1 -- 172.21.15.39:0/2525184091 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b10004050 con 0x5562300fc070 2024-07-04T08:20:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.364+0000 7f2b0d600640 1 -- 172.21.15.39:0/2525184091 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10005280 con 0x5562300fc070 2024-07-04T08:20:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.364+0000 7f2b14c00640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2afc0b2960 0x7f2afc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.364+0000 7f2b14c00640 1 -- 172.21.15.39:0/1263753530 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2b080e2d70 con 0x7f2b18076e90 2024-07-04T08:20:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b17e00640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2afc0b2960 0x7f2afc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b2586d740 1 -- 172.21.15.39:0/2525184091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300fc070 msgr2=0x5562304a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b2586d740 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300fc070 0x5562304a07d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009f50 tx=0x7f2b100118d0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b2586d740 1 -- 172.21.15.39:0/2525184091 shutdown_connections 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b2586d740 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b2586d740 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300db390 0x5562304a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b2586d740 1 --2- 172.21.15.39:0/2525184091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300fc070 0x5562304a07d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b2586d740 1 -- 172.21.15.39:0/2525184091 >> 172.21.15.39:0/2525184091 conn(0x5562300f0460 msgr2=0x55623049e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b2586d740 1 -- 172.21.15.39:0/2525184091 shutdown_connections 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b17e00640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2afc0b2960 0x7f2afc0b4e70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55623049f450 tx=0x7f2b080e6820 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.365+0000 7f2b2586d740 1 -- 172.21.15.39:0/2525184091 wait complete. 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.366+0000 7f2b2586d740 1 -- start start 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.366+0000 7f2b2586d740 1 Processor -- start 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 0x5562304ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b2586d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55623026a700 con 0x5562300d31b0 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b2586d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5562303adbd0 con 0x5562300db390 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b2586d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5562304aefb0 con 0x5562300fc070 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b2586d740 10 client.0 _ll_register_callbacks cb 0x5562303ff590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50794/0 (socket says 172.21.15.39:50794) 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b1d200640 1 -- 172.21.15.39:0/3515397504 learned_addr learned my addr 172.21.15.39:0/3515397504 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.367+0000 7f2b17400640 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 0x5562304ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.368+0000 7f2b1d200640 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.368+0000 7f2b17400640 1 -- 172.21.15.39:0/3515397504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 msgr2=0x5562304b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:50.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.368+0000 7f2b17400640 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.368+0000 7f2b17400640 1 -- 172.21.15.39:0/3515397504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 msgr2=0x5562304ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.368+0000 7f2b17400640 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ac390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.368+0000 7f2b17400640 1 -- 172.21.15.39:0/3515397504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556230167650 con 0x5562300db390 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.368+0000 7f2b17400640 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 0x5562304ac8d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009780 tx=0x7f2b04030f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65fabf4e4a61e765 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.368+0000 7f2b02000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.369+0000 7f2b02000640 1 -- 172.21.15.39:0/3515397504 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04037040 con 0x5562300db390 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.369+0000 7f2b2586d740 1 -- 172.21.15.39:0/3515397504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b10004050 con 0x5562300db390 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.369+0000 7f2b02000640 1 -- 172.21.15.39:0/3515397504 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2b04030450 con 0x5562300db390 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.369+0000 7f2b02000640 1 -- 172.21.15.39:0/3515397504 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2b0404c050 con 0x5562300db390 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b2586d740 10 client.26852 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b2586d740 1 -- 172.21.15.39:0/3515397504 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562304bb020 con 0x5562300db390 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b02000640 1 -- 172.21.15.39:0/3515397504 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b10004050 con 0x5562300db390 2024-07-04T08:20:50.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b02000640 1 -- 172.21.15.39:0/3515397504 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2b04032440 con 0x5562300db390 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b2586d740 10 client.26852 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b2586d740 10 client.26852 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b2586d740 20 client.26852 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b2586d740 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562300ba9f0 0x5562304be540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b2586d740 4 client.26852 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.370+0000 7f2b2586d740 1 -- 172.21.15.39:0/3515397504 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556230418e50 con 0x5562300ba9f0 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.371+0000 7f2b1d200640 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562300ba9f0 0x5562304be540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.371+0000 7f2b1d200640 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562300ba9f0 0x5562304be540 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe34fb88980ba77e server_cookie=d5cb97247b34c9d7 in_seq=0 out_seq=0 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.371+0000 7f2b02000640 10 client.26852 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.372+0000 7f2b02000640 1 -- 172.21.15.39:0/3515397504 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556230418e50 con 0x5562300ba9f0 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.372+0000 7f2b02000640 10 client.26852 handle_command_reply: tid=0 2024-07-04T08:20:50.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.372+0000 7f2b2586d740 1 client.26852 shutdown 2024-07-04T08:20:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.372+0000 7f2b2586d740 1 -- 172.21.15.39:0/3515397504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 msgr2=0x5562304ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.372+0000 7f2b2586d740 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 0x5562304ac8d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009780 tx=0x7f2b04030f90 comp rx=0 tx=0).stop 2024-07-04T08:20:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b2586d740 1 -- 172.21.15.39:0/3515397504 shutdown_connections 2024-07-04T08:20:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b2586d740 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562300ba9f0 0x5562304be540 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b2586d740 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b2586d740 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 0x5562304ac8d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b2586d740 1 --2- 172.21.15.39:0/3515397504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b02000640 0 client.26852 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:20:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b2586d740 1 -- 172.21.15.39:0/3515397504 >> 172.21.15.39:0/3515397504 conn(0x5562300f0460 msgr2=0x5562304a3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b2586d740 1 -- 172.21.15.39:0/3515397504 shutdown_connections 2024-07-04T08:20:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b2586d740 1 -- 172.21.15.39:0/3515397504 wait complete. 2024-07-04T08:20:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.373+0000 7f2b2586d740 20 client.26852 trim_cache size 0 max 16384 2024-07-04T08:20:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.389+0000 7f2b2586d740 1 Processor -- start 2024-07-04T08:20:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.389+0000 7f2b2586d740 1 -- start start 2024-07-04T08:20:50.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 0x5562304a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304a5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b2586d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b6720 con 0x5562300d31b0 2024-07-04T08:20:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b2586d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b8260 con 0x5562300db390 2024-07-04T08:20:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b2586d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b6df0 con 0x5562300fc070 2024-07-04T08:20:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:50808/0 (socket says 172.21.15.39:50808) 2024-07-04T08:20:50.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b1d200640 1 -- 172.21.15.39:0/305640541 learned_addr learned my addr 172.21.15.39:0/305640541 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:50.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b17400640 1 --2- 172.21.15.39:0/305640541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 0x5562304a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b1d200640 1 --2- 172.21.15.39:0/305640541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b1d200640 1 -- 172.21.15.39:0/305640541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 msgr2=0x5562304a5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:50.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b1d200640 1 --2- 172.21.15.39:0/305640541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304a5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b1d200640 1 -- 172.21.15.39:0/305640541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 msgr2=0x5562304a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:50.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b1d200640 1 --2- 172.21.15.39:0/305640541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 0x5562304a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.390+0000 7f2b1d200640 1 -- 172.21.15.39:0/305640541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562304a3060 con 0x5562300d31b0 2024-07-04T08:20:50.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.391+0000 7f2b1d200640 1 --2- 172.21.15.39:0/305640541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ae070 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2b10012fd0 tx=0x7f2b100096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77491830e2bd86b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:50.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.391+0000 7f2b0cc00640 1 -- 172.21.15.39:0/305640541 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10004480 con 0x5562300d31b0 2024-07-04T08:20:50.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.391+0000 7f2b0cc00640 1 -- 172.21.15.39:0/305640541 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b10004da0 con 0x5562300d31b0 2024-07-04T08:20:50.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.391+0000 7f2b0cc00640 1 -- 172.21.15.39:0/305640541 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100057a0 con 0x5562300d31b0 2024-07-04T08:20:50.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.392+0000 7f2b2586d740 1 -- 172.21.15.39:0/305640541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 msgr2=0x5562304ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:50.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.392+0000 7f2b2586d740 1 --2- 172.21.15.39:0/305640541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ae070 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2b10012fd0 tx=0x7f2b100096c0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.392+0000 7f2b2586d740 1 -- 172.21.15.39:0/305640541 shutdown_connections 2024-07-04T08:20:50.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.392+0000 7f2b2586d740 1 --2- 172.21.15.39:0/305640541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304a5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.392+0000 7f2b2586d740 1 --2- 172.21.15.39:0/305640541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300db390 0x5562304a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.392+0000 7f2b2586d740 1 --2- 172.21.15.39:0/305640541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300d31b0 0x5562304ae070 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.392+0000 7f2b2586d740 1 -- 172.21.15.39:0/305640541 >> 172.21.15.39:0/305640541 conn(0x5562300f0460 msgr2=0x5562304a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:50.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.392+0000 7f2b2586d740 1 -- 172.21.15.39:0/305640541 shutdown_connections 2024-07-04T08:20:50.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.392+0000 7f2b2586d740 1 -- 172.21.15.39:0/305640541 wait complete. 2024-07-04T08:20:50.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.393+0000 7f2b2586d740 1 -- start start 2024-07-04T08:20:50.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.393+0000 7f2b2586d740 1 Processor -- start 2024-07-04T08:20:50.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.393+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300d31b0 0x5562304b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.393+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300db390 0x5562304afe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.393+0000 7f2b2586d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b2586d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b7b90 con 0x5562300db390 2024-07-04T08:20:50.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b2586d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b74c0 con 0x5562300d31b0 2024-07-04T08:20:50.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b2586d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b4700 con 0x5562300fc070 2024-07-04T08:20:50.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b2586d740 10 client.0 _ll_register_callbacks cb 0x7f2b100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:50.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300d31b0 0x5562304b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b1d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300d31b0 0x5562304b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57804/0 (socket says 172.21.15.39:57804) 2024-07-04T08:20:50.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b1d200640 1 -- 172.21.15.39:0/1412445882 learned_addr learned my addr 172.21.15.39:0/1412445882 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:50.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b17400640 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300db390 0x5562304afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b1d200640 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b1d200640 1 -- 172.21.15.39:0/1412445882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 msgr2=0x5562304b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:50.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b1d200640 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b1d200640 1 -- 172.21.15.39:0/1412445882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300db390 msgr2=0x5562304afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b1d200640 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300db390 0x5562304afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b1d200640 1 -- 172.21.15.39:0/1412445882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562304b0b70 con 0x5562300d31b0 2024-07-04T08:20:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.394+0000 7f2b17400640 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300db390 0x5562304afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.395+0000 7f2b1d200640 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300d31b0 0x5562304b4990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009270 tx=0x7f2b10005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26012c549606b835 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.395+0000 7f2b02000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:50.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.395+0000 7f2b02000640 1 -- 172.21.15.39:0/1412445882 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10004af0 con 0x5562300d31b0 2024-07-04T08:20:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.395+0000 7f2b02000640 1 -- 172.21.15.39:0/1412445882 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f2b10004190 con 0x5562300d31b0 2024-07-04T08:20:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.395+0000 7f2b2586d740 1 -- 172.21.15.39:0/1412445882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b10004da0 con 0x5562300d31b0 2024-07-04T08:20:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.396+0000 7f2b02000640 1 -- 172.21.15.39:0/1412445882 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2b100043b0 con 0x5562300d31b0 2024-07-04T08:20:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.396+0000 7f2b2586d740 10 client.26858 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.396+0000 7f2b2586d740 1 -- 172.21.15.39:0/1412445882 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556230526cd0 con 0x5562300d31b0 2024-07-04T08:20:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.396+0000 7f2b02000640 1 -- 172.21.15.39:0/1412445882 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b10004da0 con 0x5562300d31b0 2024-07-04T08:20:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.396+0000 7f2b02000640 1 -- 172.21.15.39:0/1412445882 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2b1000f720 con 0x5562300d31b0 2024-07-04T08:20:50.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.397+0000 7f2b2586d740 10 client.26858 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:50.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.397+0000 7f2b2586d740 10 client.26858 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:20:50.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.397+0000 7f2b2586d740 20 client.26858 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:50.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.397+0000 7f2b2586d740 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55623008fda0 0x5562305291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:50.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.397+0000 7f2b2586d740 4 client.26858 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:20:50.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.397+0000 7f2b2586d740 1 -- 172.21.15.39:0/1412445882 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562304148f0 con 0x55623008fda0 2024-07-04T08:20:50.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.397+0000 7f2b17400640 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55623008fda0 0x5562305291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:50.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b17400640 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55623008fda0 0x5562305291c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32c119787bdabdb1 server_cookie=14a2053cdf8559c in_seq=0 out_seq=0 2024-07-04T08:20:50.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b02000640 10 client.26858 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:20:50.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b02000640 1 -- 172.21.15.39:0/1412445882 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562304148f0 con 0x55623008fda0 2024-07-04T08:20:50.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b02000640 10 client.26858 handle_command_reply: tid=0 2024-07-04T08:20:50.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b2586d740 1 client.26858 shutdown 2024-07-04T08:20:50.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b2586d740 1 -- 172.21.15.39:0/1412445882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300d31b0 msgr2=0x5562304b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:50.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b2586d740 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300d31b0 0x5562304b4990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009270 tx=0x7f2b10005bd0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b2586d740 1 -- 172.21.15.39:0/1412445882 shutdown_connections 2024-07-04T08:20:50.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b2586d740 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55623008fda0 0x5562305291c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b2586d740 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562300db390 0x5562304afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b2586d740 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562300fc070 0x5562304b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b02000640 0 client.26858 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:20:50.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b2586d740 1 --2- 172.21.15.39:0/1412445882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562300d31b0 0x5562304b4990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.398+0000 7f2b2586d740 1 -- 172.21.15.39:0/1412445882 >> 172.21.15.39:0/1412445882 conn(0x5562300f0460 msgr2=0x5562304ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:50.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.399+0000 7f2b2586d740 1 -- 172.21.15.39:0/1412445882 shutdown_connections 2024-07-04T08:20:50.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.399+0000 7f2b2586d740 1 -- 172.21.15.39:0/1412445882 wait complete. 2024-07-04T08:20:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.399+0000 7f2b2586d740 20 client.26858 trim_cache size 0 max 16384 2024-07-04T08:20:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.399+0000 7f2b1f200640 1 -- 172.21.15.39:0/1263753530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2afc0b2960 msgr2=0x7f2afc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.399+0000 7f2b1f200640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2afc0b2960 0x7f2afc0b4e70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55623049f450 tx=0x7f2b080e6820 comp rx=0 tx=0).stop 2024-07-04T08:20:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.399+0000 7f2b1f200640 1 -- 172.21.15.39:0/1263753530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b18076e90 msgr2=0x7f2b181a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.399+0000 7f2b1f200640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b18076e90 0x7f2b181a0080 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e9c0 tx=0x7f2b0800ee80 comp rx=0 tx=0).stop 2024-07-04T08:20:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.400+0000 7f2b1f200640 1 -- 172.21.15.39:0/1263753530 shutdown_connections 2024-07-04T08:20:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.400+0000 7f2b1f200640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2afc0b2960 0x7f2afc0b4e70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.400+0000 7f2b1f200640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b180777e0 0x7f2b181a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.400+0000 7f2b1f200640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b18113cc0 0x7f2b181a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.400+0000 7f2b1f200640 1 --2- 172.21.15.39:0/1263753530 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2b18076e90 0x7f2b181a0080 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:50.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.400+0000 7f2b1f200640 1 -- 172.21.15.39:0/1263753530 >> 172.21.15.39:0/1263753530 conn(0x7f2b18100d10 msgr2=0x7f2b18103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:50.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.400+0000 7f2b1f200640 1 -- 172.21.15.39:0/1263753530 shutdown_connections 2024-07-04T08:20:50.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:50.400+0000 7f2b1f200640 1 -- 172.21.15.39:0/1263753530 wait complete. 2024-07-04T08:20:51.091 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:50 smithi039 ceph-mon[32062]: pgmap v649: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:20:51.312 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:50 smithi133 ceph-mon[37050]: pgmap v649: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:20:51.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:50 smithi050 ceph-mon[36577]: pgmap v649: 129 pgs: 129 active+clean; 6.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:20:51.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:20:52.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.230+0000 7f24c9a00640 1 -- 172.21.15.39:0/1061484985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24c4101fd0 msgr2=0x7f24c41023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:52.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.230+0000 7f24c9a00640 1 --2- 172.21.15.39:0/1061484985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24c4101fd0 0x7f24c41023e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009900 tx=0x7f24b402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.231+0000 7f24c9a00640 1 -- 172.21.15.39:0/1061484985 shutdown_connections 2024-07-04T08:20:52.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.231+0000 7f24c9a00640 1 --2- 172.21.15.39:0/1061484985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c4108260 0x7f24c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.231+0000 7f24c9a00640 1 --2- 172.21.15.39:0/1061484985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c4107910 0x7f24c4107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.231+0000 7f24c9a00640 1 --2- 172.21.15.39:0/1061484985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24c4101fd0 0x7f24c41023e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.231+0000 7f24c9a00640 1 -- 172.21.15.39:0/1061484985 >> 172.21.15.39:0/1061484985 conn(0x7f24c4078cb0 msgr2=0x7f24c40790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:52.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.231+0000 7f24c9a00640 1 -- 172.21.15.39:0/1061484985 shutdown_connections 2024-07-04T08:20:52.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.231+0000 7f24c9a00640 1 -- 172.21.15.39:0/1061484985 wait complete. 2024-07-04T08:20:52.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.232+0000 7f24c9a00640 1 Processor -- start 2024-07-04T08:20:52.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.232+0000 7f24c9a00640 1 -- start start 2024-07-04T08:20:52.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.233+0000 7f24c9a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c4101fd0 0x7f24c41037c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.233+0000 7f24c9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24c4107910 0x7f24c4103d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.233+0000 7f24c9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c4108260 0x7f24c4104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.233+0000 7f24c9a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4111f20 con 0x7f24c4107910 2024-07-04T08:20:52.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.233+0000 7f24c9a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4111da0 con 0x7f24c4108260 2024-07-04T08:20:52.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.233+0000 7f24c9a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41120a0 con 0x7f24c4101fd0 2024-07-04T08:20:52.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.233+0000 7f24c2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c4101fd0 0x7f24c41037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.233+0000 7f24c2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c4101fd0 0x7f24c41037c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42778/0 (socket says 172.21.15.39:42778) 2024-07-04T08:20:52.230 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.233+0000 7f24c2a00640 1 -- 172.21.15.39:0/3403556224 learned_addr learned my addr 172.21.15.39:0/3403556224 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:52.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.234+0000 7f24c3400640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c4108260 0x7f24c4104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.234+0000 7f24c2000640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24c4107910 0x7f24c4103d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.234+0000 7f24c2a00640 1 -- 172.21.15.39:0/3403556224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c4108260 msgr2=0x7f24c4104240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:52.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.234+0000 7f24c2a00640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c4108260 0x7f24c4104240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.234+0000 7f24c2a00640 1 -- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24c4107910 msgr2=0x7f24c4103d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:52.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.234+0000 7f24c2a00640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24c4107910 0x7f24c4103d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.234+0000 7f24c2a00640 1 -- 172.21.15.39:0/3403556224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c41a7c40 con 0x7f24c4101fd0 2024-07-04T08:20:52.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.234+0000 7f24c3400640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c4108260 0x7f24c4104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:52.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.235+0000 7f24c2a00640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c4101fd0 0x7f24c41037c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f24b4005ab0 tx=0x7f24b4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:52.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.235+0000 7f24bb400640 1 -- 172.21.15.39:0/3403556224 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b403f070 con 0x7f24c4101fd0 2024-07-04T08:20:52.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.235+0000 7f24bb400640 1 -- 172.21.15.39:0/3403556224 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b4031b60 con 0x7f24c4101fd0 2024-07-04T08:20:52.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.235+0000 7f24bb400640 1 -- 172.21.15.39:0/3403556224 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4035030 con 0x7f24c4101fd0 2024-07-04T08:20:52.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.235+0000 7f24c9a00640 1 -- 172.21.15.39:0/3403556224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c41a7ed0 con 0x7f24c4101fd0 2024-07-04T08:20:52.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.235+0000 7f24c9a00640 1 -- 172.21.15.39:0/3403556224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41a8270 con 0x7f24c4101fd0 2024-07-04T08:20:52.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.239+0000 7f24bb400640 1 -- 172.21.15.39:0/3403556224 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f24b4042070 con 0x7f24c4101fd0 2024-07-04T08:20:52.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.240+0000 7f24d00a7740 1 Processor -- start 2024-07-04T08:20:52.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24d00a7740 1 -- start start 2024-07-04T08:20:52.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24bb400640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24a80b2960 0x7f24a80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24bb400640 1 -- 172.21.15.39:0/3403556224 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f24b4102de0 con 0x7f24c4101fd0 2024-07-04T08:20:52.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6347c850 0x562f63821780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f63453990 0x562f63821cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6345bb70 0x562f63822200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24d00a7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562f637e7140 con 0x562f6345bb70 2024-07-04T08:20:52.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24d00a7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562f63708380 con 0x562f6347c850 2024-07-04T08:20:52.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24d00a7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562f63796c30 con 0x562f63453990 2024-07-04T08:20:52.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.241+0000 7f24c3400640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24a80b2960 0x7f24a80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.242+0000 7f24c2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6347c850 0x562f63821780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.242+0000 7f24c2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6347c850 0x562f63821780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51850/0 (socket says 172.21.15.39:51850) 2024-07-04T08:20:52.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.242+0000 7f24c2a00640 1 -- 172.21.15.39:0/3747455099 learned_addr learned my addr 172.21.15.39:0/3747455099 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:52.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.242+0000 7f24c2000640 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f63453990 0x562f63821cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.242+0000 7f24c3400640 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6345bb70 0x562f63822200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.242+0000 7f24c2000640 1 -- 172.21.15.39:0/3747455099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6347c850 msgr2=0x562f63821780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:52.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24c2000640 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6347c850 0x562f63821780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24c2000640 1 -- 172.21.15.39:0/3747455099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6345bb70 msgr2=0x562f63822200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:52.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24c2000640 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6345bb70 0x562f63822200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24c2000640 1 -- 172.21.15.39:0/3747455099 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f634f6ba0 con 0x562f63453990 2024-07-04T08:20:52.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24c2a00640 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6347c850 0x562f63821780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:52.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24c2000640 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f63453990 0x562f63821cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24c4069de0 tx=0x7f24b00113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd74bc6f802f5cd9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:52.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24c3400640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24a80b2960 0x7f24a80b4e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005fd0 tx=0x7f24bc00feb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:52.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24afe00640 1 -- 172.21.15.39:0/3747455099 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0011de0 con 0x562f63453990 2024-07-04T08:20:52.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24afe00640 1 -- 172.21.15.39:0/3747455099 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b0012900 con 0x562f63453990 2024-07-04T08:20:52.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.243+0000 7f24afe00640 1 -- 172.21.15.39:0/3747455099 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b000f910 con 0x562f63453990 2024-07-04T08:20:52.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.244+0000 7f24d00a7740 1 -- 172.21.15.39:0/3747455099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f63453990 msgr2=0x562f63821cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:52.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.244+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f63453990 0x562f63821cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24c4069de0 tx=0x7f24b00113e0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.245+0000 7f24d00a7740 1 -- 172.21.15.39:0/3747455099 shutdown_connections 2024-07-04T08:20:52.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.245+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6345bb70 0x562f63822200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.245+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f63453990 0x562f63821cc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.245+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3747455099 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6347c850 0x562f63821780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.245+0000 7f24d00a7740 1 -- 172.21.15.39:0/3747455099 >> 172.21.15.39:0/3747455099 conn(0x562f63470c40 msgr2=0x562f6381f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:52.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.245+0000 7f24d00a7740 1 -- 172.21.15.39:0/3747455099 shutdown_connections 2024-07-04T08:20:52.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.245+0000 7f24d00a7740 1 -- 172.21.15.39:0/3747455099 wait complete. 2024-07-04T08:20:52.242 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:52.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.246+0000 7f24d00a7740 1 -- start start 2024-07-04T08:20:52.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.246+0000 7f24d00a7740 1 Processor -- start 2024-07-04T08:20:52.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.246+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f6382d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.246+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f6382d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.246+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 0x562f638390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.246+0000 7f24d00a7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562f6378a210 con 0x562f6347c850 2024-07-04T08:20:52.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.246+0000 7f24d00a7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562f637861d0 con 0x562f63453990 2024-07-04T08:20:52.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.246+0000 7f24d00a7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562f6372d5d0 con 0x562f6345bb70 2024-07-04T08:20:52.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24d00a7740 10 client.0 _ll_register_callbacks cb 0x562f634940a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:52.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f6382d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f6382d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51856/0 (socket says 172.21.15.39:51856) 2024-07-04T08:20:52.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f6382d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c2a00640 1 -- 172.21.15.39:0/385809611 learned_addr learned my addr 172.21.15.39:0/385809611 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:52.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c3400640 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 0x562f638390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c2a00640 1 -- 172.21.15.39:0/385809611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 msgr2=0x562f6382d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:52.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c2a00640 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f6382d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c2a00640 1 -- 172.21.15.39:0/385809611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 msgr2=0x562f638390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:52.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c2a00640 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 0x562f638390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.247+0000 7f24c2a00640 1 -- 172.21.15.39:0/385809611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f6371d960 con 0x562f63453990 2024-07-04T08:20:52.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.248+0000 7f24c2000640 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f6382d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:52.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.248+0000 7f24c2a00640 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f6382d3a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f24b40c9cc0 tx=0x7f24b40ca430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c9909a9d42f8a99 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:52.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.248+0000 7f24acc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:52.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.248+0000 7f24acc00640 1 -- 172.21.15.39:0/385809611 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4049040 con 0x562f63453990 2024-07-04T08:20:52.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.248+0000 7f24acc00640 1 -- 172.21.15.39:0/385809611 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f24b40c5fc0 con 0x562f63453990 2024-07-04T08:20:52.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.248+0000 7f24d00a7740 1 -- 172.21.15.39:0/385809611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b0012900 con 0x562f63453990 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.248+0000 7f24acc00640 1 -- 172.21.15.39:0/385809611 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24b40d2b90 con 0x562f63453990 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24d00a7740 10 client.26870 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24d00a7740 1 -- 172.21.15.39:0/385809611 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f6383c200 con 0x562f63453990 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24acc00640 1 -- 172.21.15.39:0/385809611 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b0012900 con 0x562f63453990 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24acc00640 1 -- 172.21.15.39:0/385809611 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f24b404c060 con 0x562f63453990 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24d00a7740 10 client.26870 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24d00a7740 10 client.26870 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24d00a7740 20 client.26870 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24d00a7740 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f6343b1d0 0x562f6383f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24d00a7740 4 client.26870 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:52.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.249+0000 7f24d00a7740 1 -- 172.21.15.39:0/385809611 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f63798b80 con 0x562f6343b1d0 2024-07-04T08:20:52.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.250+0000 7f24c2000640 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f6343b1d0 0x562f6383f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.250+0000 7f24c2000640 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f6343b1d0 0x562f6383f780 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ebf94ebecae0ac server_cookie=21ddbb3b40e7f617 in_seq=0 out_seq=0 2024-07-04T08:20:52.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.250+0000 7f24acc00640 10 client.26870 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:52.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.251+0000 7f24acc00640 1 -- 172.21.15.39:0/385809611 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562f63798b80 con 0x562f6343b1d0 2024-07-04T08:20:52.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.251+0000 7f24acc00640 10 client.26870 handle_command_reply: tid=0 2024-07-04T08:20:52.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.251+0000 7f24d00a7740 1 client.26870 shutdown 2024-07-04T08:20:52.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 -- 172.21.15.39:0/385809611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 msgr2=0x562f6382d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:52.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f6382d3a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f24b40c9cc0 tx=0x7f24b40ca430 comp rx=0 tx=0).stop 2024-07-04T08:20:52.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 -- 172.21.15.39:0/385809611 shutdown_connections 2024-07-04T08:20:52.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f6343b1d0 0x562f6383f780 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 0x562f638390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f6382d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 --2- 172.21.15.39:0/385809611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f6382d3a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24acc00640 0 client.26870 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:52.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 -- 172.21.15.39:0/385809611 >> 172.21.15.39:0/385809611 conn(0x562f63470c40 msgr2=0x562f63824960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:52.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 -- 172.21.15.39:0/385809611 shutdown_connections 2024-07-04T08:20:52.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 1 -- 172.21.15.39:0/385809611 wait complete. 2024-07-04T08:20:52.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.252+0000 7f24d00a7740 20 client.26870 trim_cache size 0 max 16384 2024-07-04T08:20:52.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.273+0000 7f24d00a7740 1 Processor -- start 2024-07-04T08:20:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.273+0000 7f24d00a7740 1 -- start start 2024-07-04T08:20:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.273+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f63453990 0x562f6382f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.273+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6345bb70 0x562f63828350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.273+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6347c850 0x562f63838ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.273+0000 7f24d00a7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b3f50 con 0x562f63453990 2024-07-04T08:20:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.273+0000 7f24d00a7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b2ae0 con 0x562f6345bb70 2024-07-04T08:20:52.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.273+0000 7f24d00a7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b31b0 con 0x562f6347c850 2024-07-04T08:20:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6345bb70 0x562f63828350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6345bb70 0x562f63828350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51872/0 (socket says 172.21.15.39:51872) 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c2000640 1 -- 172.21.15.39:0/4014811335 learned_addr learned my addr 172.21.15.39:0/4014811335 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c2a00640 1 --2- 172.21.15.39:0/4014811335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f63453990 0x562f6382f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c3400640 1 --2- 172.21.15.39:0/4014811335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6347c850 0x562f63838ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c2000640 1 -- 172.21.15.39:0/4014811335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6347c850 msgr2=0x562f63838ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c2000640 1 --2- 172.21.15.39:0/4014811335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6347c850 0x562f63838ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c2000640 1 -- 172.21.15.39:0/4014811335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f63453990 msgr2=0x562f6382f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c2000640 1 --2- 172.21.15.39:0/4014811335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f63453990 0x562f6382f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.274+0000 7f24c2000640 1 -- 172.21.15.39:0/4014811335 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6381e330 con 0x562f6345bb70 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.275+0000 7f24c2000640 1 --2- 172.21.15.39:0/4014811335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6345bb70 0x562f63828350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009900 tx=0x7f24b0009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4030922f69c0e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.275+0000 7f24af400640 1 -- 172.21.15.39:0/4014811335 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b00052c0 con 0x562f6345bb70 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.275+0000 7f24af400640 1 -- 172.21.15.39:0/4014811335 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b0005460 con 0x562f6345bb70 2024-07-04T08:20:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.275+0000 7f24af400640 1 -- 172.21.15.39:0/4014811335 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0003850 con 0x562f6345bb70 2024-07-04T08:20:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.276+0000 7f24d00a7740 1 -- 172.21.15.39:0/4014811335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6345bb70 msgr2=0x562f63828350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.276+0000 7f24d00a7740 1 --2- 172.21.15.39:0/4014811335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6345bb70 0x562f63828350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009900 tx=0x7f24b0009530 comp rx=0 tx=0).stop 2024-07-04T08:20:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.276+0000 7f24d00a7740 1 -- 172.21.15.39:0/4014811335 shutdown_connections 2024-07-04T08:20:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.276+0000 7f24d00a7740 1 --2- 172.21.15.39:0/4014811335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6347c850 0x562f63838ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.276+0000 7f24d00a7740 1 --2- 172.21.15.39:0/4014811335 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f6345bb70 0x562f63828350 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.276+0000 7f24d00a7740 1 --2- 172.21.15.39:0/4014811335 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f63453990 0x562f6382f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.276+0000 7f24d00a7740 1 -- 172.21.15.39:0/4014811335 >> 172.21.15.39:0/4014811335 conn(0x562f63470c40 msgr2=0x562f6381eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.277+0000 7f24d00a7740 1 -- 172.21.15.39:0/4014811335 shutdown_connections 2024-07-04T08:20:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.277+0000 7f24d00a7740 1 -- 172.21.15.39:0/4014811335 wait complete. 2024-07-04T08:20:52.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.277+0000 7f24d00a7740 1 -- start start 2024-07-04T08:20:52.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.278+0000 7f24d00a7740 1 Processor -- start 2024-07-04T08:20:52.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.278+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f63844c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.278+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f638451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.279 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24d00a7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 0x562f63845710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24d00a7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b03f0 con 0x562f6347c850 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24d00a7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b3880 con 0x562f63453990 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24d00a7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b0a60 con 0x562f6345bb70 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24d00a7740 10 client.0 _ll_register_callbacks cb 0x7f24b4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24c2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f63844c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24c2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f63844c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51886/0 (socket says 172.21.15.39:51886) 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24c2a00640 1 -- 172.21.15.39:0/3324184664 learned_addr learned my addr 172.21.15.39:0/3324184664 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24c2000640 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f638451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.279+0000 7f24c3400640 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 0x562f63845710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24c2000640 1 -- 172.21.15.39:0/3324184664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 msgr2=0x562f63844c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24c2000640 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f63844c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24c2000640 1 -- 172.21.15.39:0/3324184664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 msgr2=0x562f63845710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24c2000640 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 0x562f63845710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24c2000640 1 -- 172.21.15.39:0/3324184664 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f638345f0 con 0x562f6345bb70 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24c3400640 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 0x562f63845710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24c2000640 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f638451d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24b000a010 tx=0x7f24b00056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5e23dd7d90f36ca server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24acc00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24acc00640 1 -- 172.21.15.39:0/3324184664 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0012870 con 0x562f6345bb70 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.280+0000 7f24acc00640 1 -- 172.21.15.39:0/3324184664 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f24b0012a10 con 0x562f6345bb70 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.281+0000 7f24d00a7740 1 -- 172.21.15.39:0/3324184664 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b0005460 con 0x562f6345bb70 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.281+0000 7f24acc00640 1 -- 172.21.15.39:0/3324184664 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24b003e050 con 0x562f6345bb70 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.281+0000 7f24d00a7740 10 client.26785 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.281+0000 7f24d00a7740 1 -- 172.21.15.39:0/3324184664 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f638a8160 con 0x562f6345bb70 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.281+0000 7f24acc00640 1 -- 172.21.15.39:0/3324184664 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b0005460 con 0x562f6345bb70 2024-07-04T08:20:52.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.282+0000 7f24acc00640 1 -- 172.21.15.39:0/3324184664 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f24b002c070 con 0x562f6345bb70 2024-07-04T08:20:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.282+0000 7f24d00a7740 10 client.26785 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.282+0000 7f24d00a7740 10 client.26785 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:20:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.282+0000 7f24d00a7740 20 client.26785 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.282+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f63410580 0x562f638aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.282+0000 7f24d00a7740 4 client.26785 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.95, "await_for": 45.0}] 2024-07-04T08:20:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.282+0000 7f24d00a7740 1 -- 172.21.15.39:0/3324184664 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.95, "await_for": 45.0}) -- 0x562f63794620 con 0x562f63410580 2024-07-04T08:20:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.282+0000 7f24c2a00640 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f63410580 0x562f638aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.283+0000 7f24c2a00640 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f63410580 0x562f638aa6b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4ee02695818276 server_cookie=e03aeaff774c22d9 in_seq=0 out_seq=0 2024-07-04T08:20:52.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.283+0000 7f24acc00640 10 client.26785 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:52.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24acc00640 1 -- 172.21.15.39:0/3324184664 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562f63794620 con 0x562f63410580 2024-07-04T08:20:52.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24acc00640 10 client.26785 handle_command_reply: tid=0 2024-07-04T08:20:52.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24d00a7740 1 client.26785 shutdown 2024-07-04T08:20:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24d00a7740 1 -- 172.21.15.39:0/3324184664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 msgr2=0x562f638451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f638451d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24b000a010 tx=0x7f24b00056a0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24d00a7740 1 -- 172.21.15.39:0/3324184664 shutdown_connections 2024-07-04T08:20:52.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f63410580 0x562f638aa6b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f6347c850 0x562f63845710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f6345bb70 0x562f638451d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24d00a7740 1 --2- 172.21.15.39:0/3324184664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f63453990 0x562f63844c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24acc00640 0 client.26785 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:20:52.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.293+0000 7f24d00a7740 1 -- 172.21.15.39:0/3324184664 >> 172.21.15.39:0/3324184664 conn(0x562f63470c40 msgr2=0x562f6381eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:52.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.294+0000 7f24d00a7740 1 -- 172.21.15.39:0/3324184664 shutdown_connections 2024-07-04T08:20:52.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.294+0000 7f24d00a7740 1 -- 172.21.15.39:0/3324184664 wait complete. 2024-07-04T08:20:52.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.294+0000 7f24d00a7740 20 client.26785 trim_cache size 0 max 16384 2024-07-04T08:20:52.292 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:20:52.292 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:20:52.292 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:20:52.293 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 14, 2024-07-04T08:20:52.293 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:20:52.293 INFO:teuthology.orchestra.run.smithi039.stdout: "899c97a4": { 2024-07-04T08:20:52.293 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 14, 2024-07-04T08:20:52.293 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:20:52.293 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:20:52.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:20:52.293 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:20:52.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:20:52.294 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:20:52.294 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 143.9, 2024-07-04T08:20:52.294 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:20:52.294 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:20:52.294 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:20:52.295 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:20:52.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:20:52.295 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:20:52.295 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:20:52.295 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:20:52.295 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:20:52.295 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:20:52.295 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:20:52.295 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:20:52.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.295+0000 7f24c9a00640 1 -- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24a80b2960 msgr2=0x7f24a80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:52.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.295+0000 7f24c9a00640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24a80b2960 0x7f24a80b4e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005fd0 tx=0x7f24bc00feb0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.295+0000 7f24c9a00640 1 -- 172.21.15.39:0/3403556224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c4101fd0 msgr2=0x7f24c41037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:52.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.295+0000 7f24c9a00640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c4101fd0 0x7f24c41037c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f24b4005ab0 tx=0x7f24b4003c10 comp rx=0 tx=0).stop 2024-07-04T08:20:52.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.295+0000 7f24c9a00640 1 -- 172.21.15.39:0/3403556224 shutdown_connections 2024-07-04T08:20:52.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.296+0000 7f24c9a00640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f24a80b2960 0x7f24a80b4e70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.296+0000 7f24c9a00640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24c4107910 0x7f24c4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.296+0000 7f24c9a00640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f24c4108260 0x7f24c4104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.296+0000 7f24c9a00640 1 --2- 172.21.15.39:0/3403556224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24c4101fd0 0x7f24c41037c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:52.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.296+0000 7f24c9a00640 1 -- 172.21.15.39:0/3403556224 >> 172.21.15.39:0/3403556224 conn(0x7f24c4078cb0 msgr2=0x7f24c4105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:52.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.296+0000 7f24c9a00640 1 -- 172.21.15.39:0/3403556224 shutdown_connections 2024-07-04T08:20:52.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:52.296+0000 7f24c9a00640 1 -- 172.21.15.39:0/3403556224 wait complete. 2024-07-04T08:20:53.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:52 smithi133 ceph-mon[37050]: pgmap v650: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-07-04T08:20:53.348 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:52 smithi039 ceph-mon[32062]: pgmap v650: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-07-04T08:20:53.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:52 smithi050 ceph-mon[36577]: pgmap v650: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-07-04T08:20:53.833 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '899c97a4', quiesce took 12.6 seconds. Will release after: 11.35 2024-07-04T08:20:54.997 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:54 smithi039 ceph-mon[32062]: pgmap v651: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-07-04T08:20:55.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:54 smithi133 ceph-mon[37050]: pgmap v651: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-07-04T08:20:55.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.349+0000 7ff08de00640 1 -- 172.21.15.39:0/1400098591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff088102f00 msgr2=0x7ff088103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:55.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.349+0000 7ff08de00640 1 --2- 172.21.15.39:0/1400098591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff088102f00 0x7ff088103310 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009900 tx=0x7ff07c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.349+0000 7ff08de00640 1 -- 172.21.15.39:0/1400098591 shutdown_connections 2024-07-04T08:20:55.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.349+0000 7ff08de00640 1 --2- 172.21.15.39:0/1400098591 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff088108840 0x7ff088108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.349+0000 7ff08de00640 1 --2- 172.21.15.39:0/1400098591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff088109190 0x7ff08810f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.349+0000 7ff08de00640 1 --2- 172.21.15.39:0/1400098591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff088102f00 0x7ff088103310 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.349+0000 7ff08de00640 1 -- 172.21.15.39:0/1400098591 >> 172.21.15.39:0/1400098591 conn(0x7ff0880fe990 msgr2=0x7ff088100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:55.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.350+0000 7ff08de00640 1 -- 172.21.15.39:0/1400098591 shutdown_connections 2024-07-04T08:20:55.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.350+0000 7ff08de00640 1 -- 172.21.15.39:0/1400098591 wait complete. 2024-07-04T08:20:55.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.351+0000 7ff08de00640 1 Processor -- start 2024-07-04T08:20:55.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.351+0000 7ff08de00640 1 -- start start 2024-07-04T08:20:55.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.351+0000 7ff08de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff088102f00 0x7ff08819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.351+0000 7ff08de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff088108840 0x7ff08819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.351+0000 7ff08de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff088109190 0x7ff0881a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.351+0000 7ff08de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff088112ea0 con 0x7ff088108840 2024-07-04T08:20:55.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08de00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff088112d20 con 0x7ff088102f00 2024-07-04T08:20:55.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08de00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff088113020 con 0x7ff088109190 2024-07-04T08:20:55.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff088102f00 0x7ff08819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff088102f00 0x7ff08819ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51904/0 (socket says 172.21.15.39:51904) 2024-07-04T08:20:55.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08ca00640 1 -- 172.21.15.39:0/3485982765 learned_addr learned my addr 172.21.15.39:0/3485982765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:55.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff087e00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff088108840 0x7ff08819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08d400640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff088109190 0x7ff0881a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08d400640 1 -- 172.21.15.39:0/3485982765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff088102f00 msgr2=0x7ff08819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:55.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08d400640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff088102f00 0x7ff08819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08d400640 1 -- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff088108840 msgr2=0x7ff08819c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:55.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08d400640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff088108840 0x7ff08819c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08d400640 1 -- 172.21.15.39:0/3485982765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0881a19e0 con 0x7ff088109190 2024-07-04T08:20:55.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.352+0000 7ff08ca00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff088102f00 0x7ff08819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:55.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.353+0000 7ff08d400640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff088109190 0x7ff0881a12b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff08000ec50 tx=0x7ff08000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:55.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.353+0000 7ff085600640 1 -- 172.21.15.39:0/3485982765 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080003b70 con 0x7ff088109190 2024-07-04T08:20:55.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.353+0000 7ff085600640 1 -- 172.21.15.39:0/3485982765 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff080003d10 con 0x7ff088109190 2024-07-04T08:20:55.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.353+0000 7ff08de00640 1 -- 172.21.15.39:0/3485982765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0881a1cd0 con 0x7ff088109190 2024-07-04T08:20:55.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.353+0000 7ff08de00640 1 -- 172.21.15.39:0/3485982765 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0881a8430 con 0x7ff088109190 2024-07-04T08:20:55.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.353+0000 7ff085600640 1 -- 172.21.15.39:0/3485982765 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080010830 con 0x7ff088109190 2024-07-04T08:20:55.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.357+0000 7ff095a9f740 1 Processor -- start 2024-07-04T08:20:55.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.358+0000 7ff085600640 1 -- 172.21.15.39:0/3485982765 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff0800109d0 con 0x7ff088109190 2024-07-04T08:20:55.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.358+0000 7ff095a9f740 1 -- start start 2024-07-04T08:20:55.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.358+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7f0070 0x55ececb94970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.358+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7c71b0 0x55ececb94eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.358+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececb953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.359+0000 7ff095a9f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ececb0af00 con 0x55ecec7c71b0 2024-07-04T08:20:55.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.359+0000 7ff095a9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ececafa4a0 con 0x55ecec7f0070 2024-07-04T08:20:55.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.359+0000 7ff095a9f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ececafe4e0 con 0x55ecec7cf390 2024-07-04T08:20:55.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.359+0000 7ff087e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7f0070 0x55ececb94970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.359+0000 7ff087e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7f0070 0x55ececb94970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51906/0 (socket says 172.21.15.39:51906) 2024-07-04T08:20:55.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.359+0000 7ff087e00640 1 -- 172.21.15.39:0/3731818420 learned_addr learned my addr 172.21.15.39:0/3731818420 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:55.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.359+0000 7ff087e00640 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececb953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff087e00640 1 -- 172.21.15.39:0/3731818420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 msgr2=0x55ececb953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:55.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff087e00640 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececb953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff08ca00640 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7c71b0 0x55ececb94eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff087e00640 1 -- 172.21.15.39:0/3731818420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7c71b0 msgr2=0x55ececb94eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:55.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff087e00640 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7c71b0 0x55ececb94eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff085600640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff0700b29e0 0x7ff0700b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff087e00640 1 -- 172.21.15.39:0/3731818420 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ececa92670 con 0x55ecec7f0070 2024-07-04T08:20:55.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff08ca00640 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7c71b0 0x55ececb94eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:20:55.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff085600640 1 -- 172.21.15.39:0/3485982765 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff0800e2c70 con 0x7ff088109190 2024-07-04T08:20:55.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff087e00640 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7f0070 0x55ececb94970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff074013780 tx=0x7ff0740096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d3f6e8dea2648b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:55.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.360+0000 7ff087e00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff0700b29e0 0x7ff0700b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.361+0000 7ff07a000640 1 -- 172.21.15.39:0/3731818420 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074013dc0 con 0x55ecec7f0070 2024-07-04T08:20:55.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.361+0000 7ff07a000640 1 -- 172.21.15.39:0/3731818420 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff074013180 con 0x55ecec7f0070 2024-07-04T08:20:55.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.361+0000 7ff07a000640 1 -- 172.21.15.39:0/3731818420 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074003850 con 0x55ecec7f0070 2024-07-04T08:20:55.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff095a9f740 1 -- 172.21.15.39:0/3731818420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7f0070 msgr2=0x55ececb94970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:55.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7f0070 0x55ececb94970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff074013780 tx=0x7ff0740096c0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff087e00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff0700b29e0 0x7ff0700b4ef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff074013430 tx=0x7ff07400ff50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:55.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff095a9f740 1 -- 172.21.15.39:0/3731818420 shutdown_connections 2024-07-04T08:20:55.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7c71b0 0x55ececb94eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececb953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3731818420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7f0070 0x55ececb94970 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff095a9f740 1 -- 172.21.15.39:0/3731818420 >> 172.21.15.39:0/3731818420 conn(0x55ecec7e4460 msgr2=0x55ececb92860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff095a9f740 1 -- 172.21.15.39:0/3731818420 shutdown_connections 2024-07-04T08:20:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.362+0000 7ff095a9f740 1 -- 172.21.15.39:0/3731818420 wait complete. 2024-07-04T08:20:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.363+0000 7ff095a9f740 1 -- start start 2024-07-04T08:20:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.363+0000 7ff095a9f740 1 Processor -- start 2024-07-04T08:20:55.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7c71b0 0x55ececba04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7cf390 0x55ececba09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececbac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff095a9f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ecec95e700 con 0x55ecec7f0070 2024-07-04T08:20:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff095a9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ececaa1bd0 con 0x55ecec7cf390 2024-07-04T08:20:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff095a9f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ececba3150 con 0x55ecec7c71b0 2024-07-04T08:20:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff095a9f740 10 client.0 _ll_register_callbacks cb 0x55ececaf3590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff087e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7cf390 0x55ececba09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff08ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7c71b0 0x55ececba04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff08d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececbac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff087e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7cf390 0x55ececba09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51910/0 (socket says 172.21.15.39:51910) 2024-07-04T08:20:55.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.364+0000 7ff087e00640 1 -- 172.21.15.39:0/1133387485 learned_addr learned my addr 172.21.15.39:0/1133387485 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:55.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.365+0000 7ff087e00640 1 -- 172.21.15.39:0/1133387485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7c71b0 msgr2=0x55ececba04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:55.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.365+0000 7ff087e00640 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7c71b0 0x55ececba04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.365+0000 7ff087e00640 1 -- 172.21.15.39:0/1133387485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 msgr2=0x55ececbac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:55.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.365+0000 7ff087e00640 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececbac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.365+0000 7ff087e00640 1 -- 172.21.15.39:0/1133387485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecec85b650 con 0x55ecec7cf390 2024-07-04T08:20:55.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.365+0000 7ff087e00640 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7cf390 0x55ececba09f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff07400fa70 tx=0x7ff074012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26a54989c09d57e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:55.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.365+0000 7ff06ea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:55.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.366+0000 7ff06ea00640 1 -- 172.21.15.39:0/1133387485 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074003810 con 0x55ecec7cf390 2024-07-04T08:20:55.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.366+0000 7ff06ea00640 1 -- 172.21.15.39:0/1133387485 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff07400f7c0 con 0x55ecec7cf390 2024-07-04T08:20:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.366+0000 7ff095a9f740 1 -- 172.21.15.39:0/1133387485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff074013180 con 0x55ecec7cf390 2024-07-04T08:20:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.366+0000 7ff06ea00640 1 -- 172.21.15.39:0/1133387485 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff074005450 con 0x55ecec7cf390 2024-07-04T08:20:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.366+0000 7ff095a9f740 10 client.26888 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.366+0000 7ff095a9f740 1 -- 172.21.15.39:0/1133387485 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ececbaf090 con 0x55ecec7cf390 2024-07-04T08:20:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.366+0000 7ff06ea00640 1 -- 172.21.15.39:0/1133387485 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff074013180 con 0x55ecec7cf390 2024-07-04T08:20:55.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.367+0000 7ff06ea00640 1 -- 172.21.15.39:0/1133387485 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff074004680 con 0x55ecec7cf390 2024-07-04T08:20:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.367+0000 7ff095a9f740 10 client.26888 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.367+0000 7ff095a9f740 10 client.26888 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:20:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.367+0000 7ff095a9f740 20 client.26888 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.367+0000 7ff095a9f740 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55ecec7ae9f0 0x55ececbb23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.367+0000 7ff095a9f740 4 client.26888 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:55.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.367+0000 7ff095a9f740 1 -- 172.21.15.39:0/1133387485 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ececb0ce50 con 0x55ecec7ae9f0 2024-07-04T08:20:55.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.367+0000 7ff08ca00640 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55ecec7ae9f0 0x55ececbb23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.368+0000 7ff08ca00640 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55ecec7ae9f0 0x55ececbb23f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4b9c35a0cacf0b5 server_cookie=7c3634e9a4c0d14c in_seq=0 out_seq=0 2024-07-04T08:20:55.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.368+0000 7ff06ea00640 10 client.26888 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:20:55.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff06ea00640 1 -- 172.21.15.39:0/1133387485 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ececb0ce50 con 0x55ecec7ae9f0 2024-07-04T08:20:55.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff06ea00640 10 client.26888 handle_command_reply: tid=0 2024-07-04T08:20:55.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff095a9f740 1 client.26888 shutdown 2024-07-04T08:20:55.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff095a9f740 1 -- 172.21.15.39:0/1133387485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7cf390 msgr2=0x55ececba09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:55.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff095a9f740 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7cf390 0x55ececba09f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff07400fa70 tx=0x7ff074012000 comp rx=0 tx=0).stop 2024-07-04T08:20:55.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff095a9f740 1 -- 172.21.15.39:0/1133387485 shutdown_connections 2024-07-04T08:20:55.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff095a9f740 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55ecec7ae9f0 0x55ececbb23f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff095a9f740 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececbac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff095a9f740 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7cf390 0x55ececba09f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff095a9f740 1 --2- 172.21.15.39:0/1133387485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7c71b0 0x55ececba04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.369+0000 7ff095a9f740 1 -- 172.21.15.39:0/1133387485 >> 172.21.15.39:0/1133387485 conn(0x55ecec7e4460 msgr2=0x55ececb97b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:55.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.370+0000 7ff06ea00640 0 client.26888 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:20:55.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.370+0000 7ff095a9f740 1 -- 172.21.15.39:0/1133387485 shutdown_connections 2024-07-04T08:20:55.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.370+0000 7ff095a9f740 1 -- 172.21.15.39:0/1133387485 wait complete. 2024-07-04T08:20:55.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.370+0000 7ff095a9f740 20 client.26888 trim_cache size 0 max 16384 2024-07-04T08:20:55.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.386+0000 7ff095a9f740 1 Processor -- start 2024-07-04T08:20:55.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.386+0000 7ff095a9f740 1 -- start start 2024-07-04T08:20:55.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.386+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececb917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.386+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececb98180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.386+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececbac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.386+0000 7ff095a9f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff06800fdd0 con 0x55ecec7f0070 2024-07-04T08:20:55.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.386+0000 7ff095a9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff068011910 con 0x55ecec7c71b0 2024-07-04T08:20:55.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.386+0000 7ff095a9f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff0680104a0 con 0x55ecec7cf390 2024-07-04T08:20:55.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececb917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececb917f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51912/0 (socket says 172.21.15.39:51912) 2024-07-04T08:20:55.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08ca00640 1 -- 172.21.15.39:0/2714489651 learned_addr learned my addr 172.21.15.39:0/2714489651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:55.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff087e00640 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececb98180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08ca00640 1 -- 172.21.15.39:0/2714489651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 msgr2=0x55ececb98180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:55.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08d400640 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececbac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08ca00640 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececb98180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08ca00640 1 -- 172.21.15.39:0/2714489651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 msgr2=0x55ececbac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:55.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08ca00640 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececbac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08ca00640 1 -- 172.21.15.39:0/2714489651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecec7aec20 con 0x55ecec7c71b0 2024-07-04T08:20:55.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff087e00640 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececb98180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:20:55.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08d400640 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececbac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:20:55.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.387+0000 7ff08ca00640 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececb917f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff07c005ab0 tx=0x7ff07c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9692a375100a84d0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:55.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.388+0000 7ff079600640 1 -- 172.21.15.39:0/2714489651 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c049030 con 0x55ecec7c71b0 2024-07-04T08:20:55.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.388+0000 7ff079600640 1 -- 172.21.15.39:0/2714489651 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07c003e70 con 0x55ecec7c71b0 2024-07-04T08:20:55.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.388+0000 7ff079600640 1 -- 172.21.15.39:0/2714489651 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c035040 con 0x55ecec7c71b0 2024-07-04T08:20:55.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.389+0000 7ff095a9f740 1 -- 172.21.15.39:0/2714489651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 msgr2=0x55ececb917f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.389+0000 7ff095a9f740 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececb917f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff07c005ab0 tx=0x7ff07c005a20 comp rx=0 tx=0).stop 2024-07-04T08:20:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.389+0000 7ff095a9f740 1 -- 172.21.15.39:0/2714489651 shutdown_connections 2024-07-04T08:20:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.389+0000 7ff095a9f740 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececbac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.389+0000 7ff095a9f740 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececb98180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.389+0000 7ff095a9f740 1 --2- 172.21.15.39:0/2714489651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececb917f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.389+0000 7ff095a9f740 1 -- 172.21.15.39:0/2714489651 >> 172.21.15.39:0/2714489651 conn(0x55ecec7e4460 msgr2=0x55ececb97a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:55.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.389+0000 7ff095a9f740 1 -- 172.21.15.39:0/2714489651 shutdown_connections 2024-07-04T08:20:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.389+0000 7ff095a9f740 1 -- 172.21.15.39:0/2714489651 wait complete. 2024-07-04T08:20:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.390+0000 7ff095a9f740 1 -- start start 2024-07-04T08:20:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.390+0000 7ff095a9f740 1 Processor -- start 2024-07-04T08:20:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.390+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececba3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.390+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececba4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.390+0000 7ff095a9f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececba8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.390+0000 7ff095a9f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff068011240 con 0x55ecec7f0070 2024-07-04T08:20:55.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff095a9f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff068010b70 con 0x55ecec7c71b0 2024-07-04T08:20:55.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff095a9f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff06800ddb0 con 0x55ecec7cf390 2024-07-04T08:20:55.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff095a9f740 10 client.0 _ll_register_callbacks cb 0x7ff074009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:55.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff08ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececba3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff08ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececba3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51924/0 (socket says 172.21.15.39:51924) 2024-07-04T08:20:55.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff08ca00640 1 -- 172.21.15.39:0/3755716253 learned_addr learned my addr 172.21.15.39:0/3755716253 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:55.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff087e00640 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececba4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff08d400640 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececba8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff08ca00640 1 -- 172.21.15.39:0/3755716253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 msgr2=0x55ececba4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:55.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff08ca00640 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececba4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff08ca00640 1 -- 172.21.15.39:0/3755716253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 msgr2=0x55ececba8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:55.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff08ca00640 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececba8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff08ca00640 1 -- 172.21.15.39:0/3755716253 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ececba8790 con 0x55ecec7c71b0 2024-07-04T08:20:55.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.391+0000 7ff087e00640 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececba4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:55.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.392+0000 7ff08ca00640 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececba3cc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff07c005620 tx=0x7ff07c005650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e18522525135d0d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:55.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.392+0000 7ff06ea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:20:55.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.392+0000 7ff06ea00640 1 -- 172.21.15.39:0/3755716253 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c052070 con 0x55ecec7c71b0 2024-07-04T08:20:55.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.392+0000 7ff06ea00640 1 -- 172.21.15.39:0/3755716253 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7ff07c045340 con 0x55ecec7c71b0 2024-07-04T08:20:55.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.392+0000 7ff095a9f740 1 -- 172.21.15.39:0/3755716253 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff07c003e70 con 0x55ecec7c71b0 2024-07-04T08:20:55.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.392+0000 7ff06ea00640 1 -- 172.21.15.39:0/3755716253 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff07c035040 con 0x55ecec7c71b0 2024-07-04T08:20:55.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.392+0000 7ff095a9f740 10 client.26894 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:55.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.392+0000 7ff095a9f740 1 -- 172.21.15.39:0/3755716253 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ececc1a440 con 0x55ecec7c71b0 2024-07-04T08:20:55.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.393+0000 7ff06ea00640 1 -- 172.21.15.39:0/3755716253 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07c003e70 con 0x55ecec7c71b0 2024-07-04T08:20:55.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.393+0000 7ff06ea00640 1 -- 172.21.15.39:0/3755716253 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff07c031980 con 0x55ecec7c71b0 2024-07-04T08:20:55.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.393+0000 7ff095a9f740 10 client.26894 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:55.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.393+0000 7ff095a9f740 10 client.26894 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:20:55.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.393+0000 7ff095a9f740 20 client.26894 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:55.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.393+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55ecec783da0 0x55ececc1cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:55.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.393+0000 7ff095a9f740 4 client.26894 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:20:55.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.393+0000 7ff095a9f740 1 -- 172.21.15.39:0/3755716253 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ececb088f0 con 0x55ecec783da0 2024-07-04T08:20:55.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.394+0000 7ff087e00640 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55ecec783da0 0x55ececc1cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:55.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.394+0000 7ff087e00640 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55ecec783da0 0x55ececc1cb40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf62e5d10e33c6a1 server_cookie=586c2762e76c6cea in_seq=0 out_seq=0 2024-07-04T08:20:55.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.394+0000 7ff06ea00640 10 client.26894 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:20:55.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.394+0000 7ff06ea00640 1 -- 172.21.15.39:0/3755716253 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ececb088f0 con 0x55ecec783da0 2024-07-04T08:20:55.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.394+0000 7ff06ea00640 10 client.26894 handle_command_reply: tid=0 2024-07-04T08:20:55.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.394+0000 7ff095a9f740 1 client.26894 shutdown 2024-07-04T08:20:55.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 -- 172.21.15.39:0/3755716253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 msgr2=0x55ececba3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:55.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececba3cc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff07c005620 tx=0x7ff07c005650 comp rx=0 tx=0).stop 2024-07-04T08:20:55.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 -- 172.21.15.39:0/3755716253 shutdown_connections 2024-07-04T08:20:55.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55ecec783da0 0x55ececc1cb40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ecec7f0070 0x55ececba8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ecec7cf390 0x55ececba4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 --2- 172.21.15.39:0/3755716253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ecec7c71b0 0x55ececba3cc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff06ea00640 0 client.26894 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:20:55.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 -- 172.21.15.39:0/3755716253 >> 172.21.15.39:0/3755716253 conn(0x55ecec7e4460 msgr2=0x55ececb915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:55.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 -- 172.21.15.39:0/3755716253 shutdown_connections 2024-07-04T08:20:55.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 1 -- 172.21.15.39:0/3755716253 wait complete. 2024-07-04T08:20:55.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.395+0000 7ff095a9f740 20 client.26894 trim_cache size 0 max 16384 2024-07-04T08:20:55.403 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:20:55.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 -- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff0700b29e0 msgr2=0x7ff0700b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:55.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff0700b29e0 0x7ff0700b4ef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff074013430 tx=0x7ff07400ff50 comp rx=0 tx=0).stop 2024-07-04T08:20:55.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 -- 172.21.15.39:0/3485982765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff088109190 msgr2=0x7ff0881a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff088109190 0x7ff0881a12b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff08000ec50 tx=0x7ff08000c6a0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 -- 172.21.15.39:0/3485982765 shutdown_connections 2024-07-04T08:20:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff0700b29e0 0x7ff0700b4ef0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff088108840 0x7ff08819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff088109190 0x7ff0881a12b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 --2- 172.21.15.39:0/3485982765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff088102f00 0x7ff08819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.396+0000 7ff08de00640 1 -- 172.21.15.39:0/3485982765 >> 172.21.15.39:0/3485982765 conn(0x7ff0880fe990 msgr2=0x7ff08810dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.397+0000 7ff08de00640 1 -- 172.21.15.39:0/3485982765 shutdown_connections 2024-07-04T08:20:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:55.397+0000 7ff08de00640 1 -- 172.21.15.39:0/3485982765 wait complete. 2024-07-04T08:20:55.406 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:54 smithi050 ceph-mon[36577]: pgmap v651: 129 pgs: 129 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-07-04T08:20:56.096 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:55 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:20:56.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:55 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:20:56.097 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:55 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:20:56.251 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:20:56.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:55 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:20:56.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:55 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:20:56.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:55 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:20:56.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:55 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:20:56.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:55 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:20:56.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:55 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:20:56.715 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:20:57.124 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:56 smithi039 ceph-mon[32062]: pgmap v652: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-07-04T08:20:57.125 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:56 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:20:57.125 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:56 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:20:57.125 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:56 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:20:57.291 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:56 smithi050 ceph-mon[36577]: pgmap v652: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-07-04T08:20:57.291 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:56 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:20:57.291 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:56 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:20:57.291 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:56 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:20:57.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:56 smithi133 ceph-mon[37050]: pgmap v652: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-07-04T08:20:57.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:56 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:20:57.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:56 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:20:57.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:56 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:20:59.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.077+0000 7fa930400640 1 -- 172.21.15.39:0/1741581477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa928105300 msgr2=0x7fa928105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:59.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.077+0000 7fa930400640 1 --2- 172.21.15.39:0/1741581477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa928105300 0x7fa928105710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009900 tx=0x7fa91c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.077+0000 7fa930400640 1 -- 172.21.15.39:0/1741581477 shutdown_connections 2024-07-04T08:20:59.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.077+0000 7fa930400640 1 --2- 172.21.15.39:0/1741581477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa92810b590 0x7fa928111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.077+0000 7fa930400640 1 --2- 172.21.15.39:0/1741581477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa92810ac40 0x7fa92810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.077+0000 7fa930400640 1 --2- 172.21.15.39:0/1741581477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa928105300 0x7fa928105710 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.078+0000 7fa930400640 1 -- 172.21.15.39:0/1741581477 >> 172.21.15.39:0/1741581477 conn(0x7fa928100d30 msgr2=0x7fa928103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:59.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.078+0000 7fa930400640 1 -- 172.21.15.39:0/1741581477 shutdown_connections 2024-07-04T08:20:59.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.078+0000 7fa930400640 1 -- 172.21.15.39:0/1741581477 wait complete. 2024-07-04T08:20:59.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.079+0000 7fa930400640 1 Processor -- start 2024-07-04T08:20:59.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.079+0000 7fa930400640 1 -- start start 2024-07-04T08:20:59.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa930400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa928105300 0x7fa92819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa930400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa92810ac40 0x7fa92819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa930400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa92810b590 0x7fa9281a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa930400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa928115260 con 0x7fa92810ac40 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa930400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281150e0 con 0x7fa928105300 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa930400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281153e0 con 0x7fa92810b590 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa92da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa928105300 0x7fa92819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa92d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa92810ac40 0x7fa92819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa92da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa928105300 0x7fa92819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51928/0 (socket says 172.21.15.39:51928) 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa92da00640 1 -- 172.21.15.39:0/1697952803 learned_addr learned my addr 172.21.15.39:0/1697952803 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.080+0000 7fa92e400640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa92810b590 0x7fa9281a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa92da00640 1 -- 172.21.15.39:0/1697952803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa92810b590 msgr2=0x7fa9281a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:59.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa92da00640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa92810b590 0x7fa9281a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa92da00640 1 -- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa92810ac40 msgr2=0x7fa92819c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:59.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa92da00640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa92810ac40 0x7fa92819c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa92da00640 1 -- 172.21.15.39:0/1697952803 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9281a1ba0 con 0x7fa928105300 2024-07-04T08:20:59.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa92d000640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa92810ac40 0x7fa92819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:59.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa92da00640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa928105300 0x7fa92819bc30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005940 tx=0x7fa91c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:59.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa926000640 1 -- 172.21.15.39:0/1697952803 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c03f030 con 0x7fa928105300 2024-07-04T08:20:59.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa926000640 1 -- 172.21.15.39:0/1697952803 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91c0044c0 con 0x7fa928105300 2024-07-04T08:20:59.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.081+0000 7fa930400640 1 -- 172.21.15.39:0/1697952803 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9281a1e30 con 0x7fa928105300 2024-07-04T08:20:59.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.082+0000 7fa926000640 1 -- 172.21.15.39:0/1697952803 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c035070 con 0x7fa928105300 2024-07-04T08:20:59.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.082+0000 7fa930400640 1 -- 172.21.15.39:0/1697952803 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9281a85f0 con 0x7fa928105300 2024-07-04T08:20:59.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.085+0000 7fa936a95740 1 Processor -- start 2024-07-04T08:20:59.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.085+0000 7fa926000640 1 -- 172.21.15.39:0/1697952803 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa91c042030 con 0x7fa928105300 2024-07-04T08:20:59.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.085+0000 7fa936a95740 1 -- start start 2024-07-04T08:20:59.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.086+0000 7fa936a95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b6f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.086+0000 7fa936a95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b6fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.086+0000 7fa936a95740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b70250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.086+0000 7fa936a95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558062ae5f00 con 0x5580627a21b0 2024-07-04T08:20:59.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.086+0000 7fa936a95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558062ad54a0 con 0x5580627cb070 2024-07-04T08:20:59.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.086+0000 7fa936a95740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558062ad94e0 con 0x5580627aa390 2024-07-04T08:20:59.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.086+0000 7fa92e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b6f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b6f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51942/0 (socket says 172.21.15.39:51942) 2024-07-04T08:20:59.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92e400640 1 -- 172.21.15.39:0/150438506 learned_addr learned my addr 172.21.15.39:0/150438506 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:59.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92d000640 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b70250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92da00640 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b6fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92d000640 1 -- 172.21.15.39:0/150438506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 msgr2=0x558062b6f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:59.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92d000640 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b6f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92d000640 1 -- 172.21.15.39:0/150438506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 msgr2=0x558062b6fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:59.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92d000640 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b6fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92d000640 1 -- 172.21.15.39:0/150438506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558062a6d670 con 0x5580627aa390 2024-07-04T08:20:59.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.087+0000 7fa92e400640 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b6f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:59.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.088+0000 7fa92d000640 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b70250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa92806bce0 tx=0x7fa9140118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01c45a1bfd17d33 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:59.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.088+0000 7fa926000640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9100b29e0 0x7fa9100b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.088+0000 7fa91aa00640 1 -- 172.21.15.39:0/150438506 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914004030 con 0x5580627aa390 2024-07-04T08:20:59.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.088+0000 7fa91aa00640 1 -- 172.21.15.39:0/150438506 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9140054d0 con 0x5580627aa390 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.088+0000 7fa91aa00640 1 -- 172.21.15.39:0/150438506 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914004c40 con 0x5580627aa390 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.088+0000 7fa926000640 1 -- 172.21.15.39:0/1697952803 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa91c102900 con 0x7fa928105300 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.088+0000 7fa92e400640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9100b29e0 0x7fa9100b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa936a95740 1 -- 172.21.15.39:0/150438506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 msgr2=0x558062b70250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa936a95740 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b70250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa92806bce0 tx=0x7fa9140118d0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa92e400640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9100b29e0 0x7fa9100b4ef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa920011890 tx=0x7fa920009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa936a95740 1 -- 172.21.15.39:0/150438506 shutdown_connections 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa936a95740 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b6fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa936a95740 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b70250 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa936a95740 1 --2- 172.21.15.39:0/150438506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b6f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa936a95740 1 -- 172.21.15.39:0/150438506 >> 172.21.15.39:0/150438506 conn(0x5580627bf460 msgr2=0x558062b6d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:59.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa936a95740 1 -- 172.21.15.39:0/150438506 shutdown_connections 2024-07-04T08:20:59.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.089+0000 7fa936a95740 1 -- 172.21.15.39:0/150438506 wait complete. 2024-07-04T08:20:59.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.090+0000 7fa936a95740 1 -- start start 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.090+0000 7fa936a95740 1 Processor -- start 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa936a95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627a21b0 0x558062b7b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa936a95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627aa390 0x558062b7b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa936a95740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 0x558062b87330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa936a95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558062939700 con 0x5580627aa390 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa936a95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558062a7cbd0 con 0x5580627a21b0 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa936a95740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558062b7dfb0 con 0x5580627cb070 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa936a95740 10 client.0 _ll_register_callbacks cb 0x558062ace590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa92da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627a21b0 0x558062b7b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa92d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627aa390 0x558062b7b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa92da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627a21b0 0x558062b7b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51956/0 (socket says 172.21.15.39:51956) 2024-07-04T08:20:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa92da00640 1 -- 172.21.15.39:0/1241670444 learned_addr learned my addr 172.21.15.39:0/1241670444 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.091+0000 7fa92e400640 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 0x558062b87330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.092+0000 7fa92d000640 1 -- 172.21.15.39:0/1241670444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 msgr2=0x558062b87330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.092+0000 7fa92d000640 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 0x558062b87330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.092+0000 7fa92d000640 1 -- 172.21.15.39:0/1241670444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627a21b0 msgr2=0x558062b7b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.092+0000 7fa92d000640 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627a21b0 0x558062b7b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.092+0000 7fa92d000640 1 -- 172.21.15.39:0/1241670444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558062836650 con 0x5580627aa390 2024-07-04T08:20:59.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.092+0000 7fa92da00640 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627a21b0 0x558062b7b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:59.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.092+0000 7fa92d000640 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627aa390 0x558062b7b860 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa914009700 tx=0x7fa91400fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c062d0102d0baa74 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:59.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.092+0000 7fa90f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:20:59.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.092+0000 7fa90f400640 1 -- 172.21.15.39:0/1241670444 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914028020 con 0x5580627aa390 2024-07-04T08:20:59.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.093+0000 7fa90f400640 1 -- 172.21.15.39:0/1241670444 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa91400fe30 con 0x5580627aa390 2024-07-04T08:20:59.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.093+0000 7fa936a95740 1 -- 172.21.15.39:0/1241670444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9140054d0 con 0x5580627aa390 2024-07-04T08:20:59.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.093+0000 7fa90f400640 1 -- 172.21.15.39:0/1241670444 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa91401d040 con 0x5580627aa390 2024-07-04T08:20:59.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.093+0000 7fa936a95740 10 client.16761 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.093+0000 7fa936a95740 1 -- 172.21.15.39:0/1241670444 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558062b89f80 con 0x5580627aa390 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.093+0000 7fa90f400640 1 -- 172.21.15.39:0/1241670444 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9140054d0 con 0x5580627aa390 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.094+0000 7fa90f400640 1 -- 172.21.15.39:0/1241670444 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa914004940 con 0x5580627aa390 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.094+0000 7fa936a95740 10 client.16761 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.094+0000 7fa936a95740 10 client.16761 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.094+0000 7fa936a95740 20 client.16761 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.094+0000 7fa936a95740 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5580627899f0 0x558062b8d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.094+0000 7fa936a95740 4 client.16761 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.094+0000 7fa936a95740 1 -- 172.21.15.39:0/1241670444 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558062ae7e50 con 0x5580627899f0 2024-07-04T08:20:59.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.094+0000 7fa92da00640 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5580627899f0 0x558062b8d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.095+0000 7fa92da00640 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5580627899f0 0x558062b8d2e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=145940fc0e0e135e server_cookie=47c0df8e7027dae4 in_seq=0 out_seq=0 2024-07-04T08:20:59.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.095+0000 7fa90f400640 10 client.16761 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa90f400640 1 -- 172.21.15.39:0/1241670444 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558062ae7e50 con 0x5580627899f0 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa90f400640 10 client.16761 handle_command_reply: tid=0 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 client.16761 shutdown 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 -- 172.21.15.39:0/1241670444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627aa390 msgr2=0x558062b7b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627aa390 0x558062b7b860 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa914009700 tx=0x7fa91400fc40 comp rx=0 tx=0).stop 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 -- 172.21.15.39:0/1241670444 shutdown_connections 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5580627899f0 0x558062b8d2e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627aa390 0x558062b7b860 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 0x558062b87330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 --2- 172.21.15.39:0/1241670444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627a21b0 0x558062b7b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 -- 172.21.15.39:0/1241670444 >> 172.21.15.39:0/1241670444 conn(0x5580627bf460 msgr2=0x558062b72960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:59.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa90f400640 0 client.16761 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:20:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 -- 172.21.15.39:0/1241670444 shutdown_connections 2024-07-04T08:20:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.096+0000 7fa936a95740 1 -- 172.21.15.39:0/1241670444 wait complete. 2024-07-04T08:20:59.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.097+0000 7fa936a95740 20 client.16761 trim_cache size 0 max 16384 2024-07-04T08:20:59.108 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:20:58 smithi039 ceph-mon[32062]: pgmap v653: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-07-04T08:20:59.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.112+0000 7fa936a95740 1 Processor -- start 2024-07-04T08:20:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.112+0000 7fa936a95740 1 -- start start 2024-07-04T08:20:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.112+0000 7fa936a95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b70f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.112+0000 7fa936a95740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b74860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.112+0000 7fa936a95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b74da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.112+0000 7fa936a95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b24e0 con 0x5580627a21b0 2024-07-04T08:20:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.112+0000 7fa936a95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b4020 con 0x5580627cb070 2024-07-04T08:20:59.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.112+0000 7fa936a95740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b2bb0 con 0x5580627aa390 2024-07-04T08:20:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.113+0000 7fa92e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b74da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.113+0000 7fa92da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b70f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.113+0000 7fa92e400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b74da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51964/0 (socket says 172.21.15.39:51964) 2024-07-04T08:20:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.113+0000 7fa92e400640 1 -- 172.21.15.39:0/2979546201 learned_addr learned my addr 172.21.15.39:0/2979546201 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.113+0000 7fa92d000640 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b74860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.113+0000 7fa92d000640 1 -- 172.21.15.39:0/2979546201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 msgr2=0x558062b74da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.113+0000 7fa92d000640 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b74da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.113+0000 7fa92d000640 1 -- 172.21.15.39:0/2979546201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 msgr2=0x558062b70f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:59.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.113+0000 7fa92d000640 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b70f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.114+0000 7fa92d000640 1 -- 172.21.15.39:0/2979546201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558062b75e10 con 0x5580627aa390 2024-07-04T08:20:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.114+0000 7fa92e400640 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b74da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.114+0000 7fa92da00640 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b70f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.114+0000 7fa92d000640 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b74860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa914004de0 tx=0x7fa914005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89025d5f5078f28a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.114+0000 7fa91a000640 1 -- 172.21.15.39:0/2979546201 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914028040 con 0x5580627aa390 2024-07-04T08:20:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.115+0000 7fa91a000640 1 -- 172.21.15.39:0/2979546201 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91400fcb0 con 0x5580627aa390 2024-07-04T08:20:59.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.115+0000 7fa91a000640 1 -- 172.21.15.39:0/2979546201 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914003d00 con 0x5580627aa390 2024-07-04T08:20:59.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.116+0000 7fa936a95740 1 -- 172.21.15.39:0/2979546201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 msgr2=0x558062b74860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:59.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.116+0000 7fa936a95740 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b74860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa914004de0 tx=0x7fa914005970 comp rx=0 tx=0).stop 2024-07-04T08:20:59.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.116+0000 7fa936a95740 1 -- 172.21.15.39:0/2979546201 shutdown_connections 2024-07-04T08:20:59.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.116+0000 7fa936a95740 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627cb070 0x558062b74da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.116+0000 7fa936a95740 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627aa390 0x558062b74860 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.116+0000 7fa936a95740 1 --2- 172.21.15.39:0/2979546201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b70f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.116+0000 7fa936a95740 1 -- 172.21.15.39:0/2979546201 >> 172.21.15.39:0/2979546201 conn(0x5580627bf460 msgr2=0x558062b70800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.116+0000 7fa936a95740 1 -- 172.21.15.39:0/2979546201 shutdown_connections 2024-07-04T08:20:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.116+0000 7fa936a95740 1 -- 172.21.15.39:0/2979546201 wait complete. 2024-07-04T08:20:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.117+0000 7fa936a95740 1 -- start start 2024-07-04T08:20:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.117+0000 7fa936a95740 1 Processor -- start 2024-07-04T08:20:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa936a95740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b7e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa936a95740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627aa390 0x558062b7e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa936a95740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 0x558062b7ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa936a95740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b3950 con 0x5580627a21b0 2024-07-04T08:20:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa936a95740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b3280 con 0x5580627aa390 2024-07-04T08:20:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa936a95740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa9281b04c0 con 0x5580627cb070 2024-07-04T08:20:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa936a95740 10 client.0 _ll_register_callbacks cb 0x7fa91c108400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:20:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa92da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b7e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa92d000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627aa390 0x558062b7e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa92da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b7e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60204/0 (socket says 172.21.15.39:60204) 2024-07-04T08:20:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa92da00640 1 -- 172.21.15.39:0/1434433831 learned_addr learned my addr 172.21.15.39:0/1434433831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:20:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa92d000640 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627aa390 0x558062b7e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51968/0 (socket says 172.21.15.39:51968) 2024-07-04T08:20:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.118+0000 7fa92e400640 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 0x558062b7ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa92da00640 1 -- 172.21.15.39:0/1434433831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 msgr2=0x558062b7ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa92da00640 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 0x558062b7ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa92da00640 1 -- 172.21.15.39:0/1434433831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627aa390 msgr2=0x558062b7e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa92da00640 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627aa390 0x558062b7e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa92da00640 1 -- 172.21.15.39:0/1434433831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558062bf3950 con 0x5580627a21b0 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa92d000640 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627aa390 0x558062b7e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa92da00640 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b7e250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0c9600 tx=0x7fa91c0ec040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210480b25928ee9e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa90f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa90f400640 1 -- 172.21.15.39:0/1434433831 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c049040 con 0x5580627a21b0 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa90f400640 1 -- 172.21.15.39:0/1434433831 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fa91c0c5b90 con 0x5580627a21b0 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.119+0000 7fa936a95740 1 -- 172.21.15.39:0/1434433831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa91400fcb0 con 0x5580627a21b0 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa90f400640 1 -- 172.21.15.39:0/1434433831 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa91c0c9070 con 0x5580627a21b0 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa936a95740 10 client.16767 fetch_fsmap learned FSMap version 30 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa936a95740 1 -- 172.21.15.39:0/1434433831 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558062bf4c30 con 0x5580627a21b0 2024-07-04T08:20:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa90f400640 1 -- 172.21.15.39:0/1434433831 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91400fcb0 con 0x5580627a21b0 2024-07-04T08:20:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa90f400640 1 -- 172.21.15.39:0/1434433831 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa91c0fff40 con 0x5580627a21b0 2024-07-04T08:20:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa936a95740 10 client.16767 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:20:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa936a95740 10 client.16767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:20:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa936a95740 20 client.16767 populate_metadata read hostname 'smithi039' 2024-07-04T08:20:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa936a95740 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55806275eda0 0x558062bf7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:20:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa936a95740 4 client.16767 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:20:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.120+0000 7fa936a95740 1 -- 172.21.15.39:0/1434433831 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558062ae38f0 con 0x55806275eda0 2024-07-04T08:20:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.121+0000 7fa92d000640 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55806275eda0 0x558062bf7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:20:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.121+0000 7fa92d000640 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55806275eda0 0x558062bf7180 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61adb0a91d9d384a server_cookie=caf170fc369efd80 in_seq=0 out_seq=0 2024-07-04T08:20:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.121+0000 7fa90f400640 10 client.16767 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa90f400640 1 -- 172.21.15.39:0/1434433831 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558062ae38f0 con 0x55806275eda0 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa90f400640 10 client.16767 handle_command_reply: tid=0 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 client.16767 shutdown 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 -- 172.21.15.39:0/1434433831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 msgr2=0x558062b7e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b7e250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0c9600 tx=0x7fa91c0ec040 comp rx=0 tx=0).stop 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 -- 172.21.15.39:0/1434433831 shutdown_connections 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55806275eda0 0x558062bf7180 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5580627cb070 0x558062b7ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580627aa390 0x558062b7e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 --2- 172.21.15.39:0/1434433831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5580627a21b0 0x558062b7e250 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa90f400640 0 client.16767 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:20:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 -- 172.21.15.39:0/1434433831 >> 172.21.15.39:0/1434433831 conn(0x5580627bf460 msgr2=0x558062b7c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 -- 172.21.15.39:0/1434433831 shutdown_connections 2024-07-04T08:20:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 1 -- 172.21.15.39:0/1434433831 wait complete. 2024-07-04T08:20:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.122+0000 7fa936a95740 20 client.16767 trim_cache size 0 max 16384 2024-07-04T08:20:59.121 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 -- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9100b29e0 msgr2=0x7fa9100b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9100b29e0 0x7fa9100b4ef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa920011890 tx=0x7fa920009500 comp rx=0 tx=0).stop 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 -- 172.21.15.39:0/1697952803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa928105300 msgr2=0x7fa92819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa928105300 0x7fa92819bc30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa91c005940 tx=0x7fa91c0040c0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 -- 172.21.15.39:0/1697952803 shutdown_connections 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa9100b29e0 0x7fa9100b4ef0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa92810ac40 0x7fa92819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa92810b590 0x7fa9281a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 --2- 172.21.15.39:0/1697952803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa928105300 0x7fa92819bc30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.123+0000 7fa930400640 1 -- 172.21.15.39:0/1697952803 >> 172.21.15.39:0/1697952803 conn(0x7fa928100d30 msgr2=0x7fa928103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:20:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.124+0000 7fa930400640 1 -- 172.21.15.39:0/1697952803 shutdown_connections 2024-07-04T08:20:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:20:59.124+0000 7fa930400640 1 -- 172.21.15.39:0/1697952803 wait complete. 2024-07-04T08:20:59.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:20:58 smithi133 ceph-mon[37050]: pgmap v653: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-07-04T08:20:59.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:20:58 smithi050 ceph-mon[36577]: pgmap v653: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-07-04T08:21:00.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:21:00.517 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:01.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:01 smithi133 ceph-mon[37050]: pgmap v654: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-07-04T08:21:01.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:01 smithi039 ceph-mon[32062]: pgmap v654: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-07-04T08:21:01.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:01 smithi050 ceph-mon[36577]: pgmap v654: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-07-04T08:21:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.767+0000 7f491b000640 1 -- 172.21.15.39:0/223958372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4914109350 msgr2=0x7f491410fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.767+0000 7f491b000640 1 --2- 172.21.15.39:0/223958372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4914109350 0x7f491410fb80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f490c009860 tx=0x7f490c02a340 comp rx=0 tx=0).stop 2024-07-04T08:21:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.767+0000 7f491b000640 1 -- 172.21.15.39:0/223958372 shutdown_connections 2024-07-04T08:21:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.767+0000 7f491b000640 1 --2- 172.21.15.39:0/223958372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4914109350 0x7f491410fb80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.767+0000 7f491b000640 1 --2- 172.21.15.39:0/223958372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4914108a00 0x7f4914108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.767+0000 7f491b000640 1 --2- 172.21.15.39:0/223958372 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141030c0 0x7f49141034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.767+0000 7f491b000640 1 -- 172.21.15.39:0/223958372 >> 172.21.15.39:0/223958372 conn(0x7f49140feb30 msgr2=0x7f4914100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:02.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.767+0000 7f491b000640 1 -- 172.21.15.39:0/223958372 shutdown_connections 2024-07-04T08:21:02.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.767+0000 7f491b000640 1 -- 172.21.15.39:0/223958372 wait complete. 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.769+0000 7f491b000640 1 Processor -- start 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.769+0000 7f491b000640 1 -- start start 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.769+0000 7f491b000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141030c0 0x7f4914074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.769+0000 7f491b000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4914108a00 0x7f49140751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.769+0000 7f491b000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4914109350 0x7f49140756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.769+0000 7f491b000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4914113020 con 0x7f49141030c0 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.769+0000 7f491b000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4914112ea0 con 0x7f4914108a00 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.769+0000 7f491b000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f49141131a0 con 0x7f4914109350 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4913e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141030c0 0x7f4914074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4913e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141030c0 0x7f4914074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36120/0 (socket says 172.21.15.39:36120) 2024-07-04T08:21:02.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4919000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4914109350 0x7f49140756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4913e00640 1 -- 172.21.15.39:0/691757019 learned_addr learned my addr 172.21.15.39:0/691757019 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:02.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4913400640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4914108a00 0x7f49140751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4919000640 1 -- 172.21.15.39:0/691757019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4914108a00 msgr2=0x7f49140751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4919000640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4914108a00 0x7f49140751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4919000640 1 -- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141030c0 msgr2=0x7f4914074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4919000640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141030c0 0x7f4914074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4919000640 1 -- 172.21.15.39:0/691757019 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49141a7c50 con 0x7f4914109350 2024-07-04T08:21:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.770+0000 7f4913e00640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141030c0 0x7f4914074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.771+0000 7f4919000640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4914109350 0x7f49140756e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f490c02a930 tx=0x7f490c032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.771+0000 7f4910c00640 1 -- 172.21.15.39:0/691757019 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c040020 con 0x7f4914109350 2024-07-04T08:21:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.771+0000 7f4910c00640 1 -- 172.21.15.39:0/691757019 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f490c030d50 con 0x7f4914109350 2024-07-04T08:21:02.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.771+0000 7f491b000640 1 -- 172.21.15.39:0/691757019 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49141a7ee0 con 0x7f4914109350 2024-07-04T08:21:02.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.771+0000 7f4910c00640 1 -- 172.21.15.39:0/691757019 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c037040 con 0x7f4914109350 2024-07-04T08:21:02.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.771+0000 7f491b000640 1 -- 172.21.15.39:0/691757019 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49141a8330 con 0x7f4914109350 2024-07-04T08:21:02.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.775+0000 7f492169b740 1 Processor -- start 2024-07-04T08:21:02.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.775+0000 7f4910c00640 1 -- 172.21.15.39:0/691757019 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f490c043020 con 0x7f4914109350 2024-07-04T08:21:02.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.775+0000 7f492169b740 1 -- start start 2024-07-04T08:21:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f492169b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 0x560d1b8ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f492169b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f492169b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 0x560d1b8cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f492169b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560d1b844ee0 con 0x560d1b509390 2024-07-04T08:21:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f492169b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560d1b834480 con 0x560d1b5011b0 2024-07-04T08:21:02.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f492169b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d1b8384c0 con 0x560d1b52a070 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f4913e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f4913400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 0x560d1b8ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f4913e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:46082/0 (socket says 172.21.15.39:46082) 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f4913e00640 1 -- 172.21.15.39:0/4110362580 learned_addr learned my addr 172.21.15.39:0/4110362580 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.776+0000 7f4913400640 1 --2- 172.21.15.39:0/4110362580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 0x560d1b8cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4913e00640 1 -- 172.21.15.39:0/4110362580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 msgr2=0x560d1b8ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4913e00640 1 --2- 172.21.15.39:0/4110362580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 0x560d1b8ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4913e00640 1 -- 172.21.15.39:0/4110362580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 msgr2=0x560d1b8cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4913e00640 1 --2- 172.21.15.39:0/4110362580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 0x560d1b8cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4913e00640 1 -- 172.21.15.39:0/4110362580 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1b7cc650 con 0x560d1b5011b0 2024-07-04T08:21:02.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4913400640 1 --2- 172.21.15.39:0/4110362580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 0x560d1b8cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:02.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4910c00640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f48f40b2960 0x7f48f40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4913e00640 1 --2- 172.21.15.39:0/4110362580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8cecf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f49040115f0 tx=0x7f4904009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c7a5e5159bd3d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4910c00640 1 -- 172.21.15.39:0/691757019 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f490c1023f0 con 0x7f4914109350 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4909600640 1 -- 172.21.15.39:0/4110362580 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490401e070 con 0x560d1b5011b0 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4913400640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f48f40b2960 0x7f48f40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.777+0000 7f4909600640 1 -- 172.21.15.39:0/4110362580 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f490400fbb0 con 0x560d1b5011b0 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.778+0000 7f4909600640 1 -- 172.21.15.39:0/4110362580 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904019bd0 con 0x560d1b5011b0 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.778+0000 7f492169b740 1 -- 172.21.15.39:0/4110362580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 msgr2=0x560d1b8cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.778+0000 7f492169b740 1 --2- 172.21.15.39:0/4110362580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8cecf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f49040115f0 tx=0x7f4904009f90 comp rx=0 tx=0).stop 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.778+0000 7f4913400640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f48f40b2960 0x7f48f40b4e70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f49141084a0 tx=0x7f48fc013040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.779+0000 7f492169b740 1 -- 172.21.15.39:0/4110362580 shutdown_connections 2024-07-04T08:21:02.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.779+0000 7f492169b740 1 --2- 172.21.15.39:0/4110362580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 0x560d1b8cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.779+0000 7f492169b740 1 --2- 172.21.15.39:0/4110362580 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8cecf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.779+0000 7f492169b740 1 --2- 172.21.15.39:0/4110362580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 0x560d1b8ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.779+0000 7f492169b740 1 -- 172.21.15.39:0/4110362580 >> 172.21.15.39:0/4110362580 conn(0x560d1b51e460 msgr2=0x560d1b8cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:02.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.779+0000 7f492169b740 1 -- 172.21.15.39:0/4110362580 shutdown_connections 2024-07-04T08:21:02.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.779+0000 7f492169b740 1 -- 172.21.15.39:0/4110362580 wait complete. 2024-07-04T08:21:02.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.779+0000 7f492169b740 1 -- start start 2024-07-04T08:21:02.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.780+0000 7f492169b740 1 Processor -- start 2024-07-04T08:21:02.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.780+0000 7f492169b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8da380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.780+0000 7f492169b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b509390 0x560d1b8da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.780+0000 7f492169b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b52a070 0x560d1b8e6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.780+0000 7f492169b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560d1b698700 con 0x560d1b52a070 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.780+0000 7f492169b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560d1b7dbbb0 con 0x560d1b5011b0 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.780+0000 7f492169b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d1b8dcf90 con 0x560d1b509390 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.780+0000 7f492169b740 10 client.0 _ll_register_callbacks cb 0x560d1b82d570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4913e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8da380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4913e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8da380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:46086/0 (socket says 172.21.15.39:46086) 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4913e00640 1 -- 172.21.15.39:0/665871939 learned_addr learned my addr 172.21.15.39:0/665871939 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4919000640 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b52a070 0x560d1b8e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4913400640 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b509390 0x560d1b8da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4913400640 1 -- 172.21.15.39:0/665871939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 msgr2=0x560d1b8da380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4913400640 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8da380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4913400640 1 -- 172.21.15.39:0/665871939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b52a070 msgr2=0x560d1b8e6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4913400640 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b52a070 0x560d1b8e6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4913400640 1 -- 172.21.15.39:0/665871939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1b595650 con 0x560d1b509390 2024-07-04T08:21:02.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.781+0000 7f4919000640 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b52a070 0x560d1b8e6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.782+0000 7f4913400640 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b509390 0x560d1b8da8c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f491406b290 tx=0x7f48fc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80e3e3a0df5d3da5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.782+0000 7f4902000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:21:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.782+0000 7f4902000640 1 -- 172.21.15.39:0/665871939 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc02f060 con 0x560d1b509390 2024-07-04T08:21:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.782+0000 7f492169b740 1 -- 172.21.15.39:0/665871939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490400fbb0 con 0x560d1b509390 2024-07-04T08:21:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.782+0000 7f4902000640 1 -- 172.21.15.39:0/665871939 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f48fc01a030 con 0x560d1b509390 2024-07-04T08:21:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.782+0000 7f4902000640 1 -- 172.21.15.39:0/665871939 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f48fc036050 con 0x560d1b509390 2024-07-04T08:21:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.782+0000 7f492169b740 10 client.26833 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.782+0000 7f492169b740 1 -- 172.21.15.39:0/665871939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d1b8e9060 con 0x560d1b509390 2024-07-04T08:21:02.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.783+0000 7f4902000640 1 -- 172.21.15.39:0/665871939 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f490400fbb0 con 0x560d1b509390 2024-07-04T08:21:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.783+0000 7f4902000640 1 -- 172.21.15.39:0/665871939 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f48fc00e3b0 con 0x560d1b509390 2024-07-04T08:21:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.783+0000 7f492169b740 10 client.26833 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.783+0000 7f492169b740 10 client.26833 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:21:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.783+0000 7f492169b740 20 client.26833 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.783+0000 7f492169b740 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x560d1b4e89f0 0x560d1b8ec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.783+0000 7f492169b740 4 client.26833 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.783+0000 7f492169b740 1 -- 172.21.15.39:0/665871939 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d1b846e30 con 0x560d1b4e89f0 2024-07-04T08:21:02.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.784+0000 7f4913e00640 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x560d1b4e89f0 0x560d1b8ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.784+0000 7f4902000640 10 client.26833 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:21:02.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.784+0000 7f4913e00640 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x560d1b4e89f0 0x560d1b8ec540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92f6df74cfb7d91d server_cookie=b58ca6d5f40a8346 in_seq=0 out_seq=0 2024-07-04T08:21:02.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.785+0000 7f4902000640 1 -- 172.21.15.39:0/665871939 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d1b846e30 con 0x560d1b4e89f0 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.785+0000 7f4902000640 10 client.26833 handle_command_reply: tid=0 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.785+0000 7f492169b740 1 client.26833 shutdown 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 -- 172.21.15.39:0/665871939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b509390 msgr2=0x560d1b8da8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b509390 0x560d1b8da8c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f491406b290 tx=0x7f48fc004ab0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 -- 172.21.15.39:0/665871939 shutdown_connections 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x560d1b4e89f0 0x560d1b8ec540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b52a070 0x560d1b8e6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b509390 0x560d1b8da8c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 --2- 172.21.15.39:0/665871939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b5011b0 0x560d1b8da380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f4902000640 0 client.26833 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 -- 172.21.15.39:0/665871939 >> 172.21.15.39:0/665871939 conn(0x560d1b51e460 msgr2=0x560d1b8d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 -- 172.21.15.39:0/665871939 shutdown_connections 2024-07-04T08:21:02.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 1 -- 172.21.15.39:0/665871939 wait complete. 2024-07-04T08:21:02.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.786+0000 7f492169b740 20 client.26833 trim_cache size 0 max 16384 2024-07-04T08:21:02.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.801+0000 7f492169b740 1 Processor -- start 2024-07-04T08:21:02.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.802+0000 7f492169b740 1 -- start start 2024-07-04T08:21:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.802+0000 7f492169b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b5011b0 0x560d1b8dc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.802+0000 7f492169b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b509390 0x560d1b8cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.802+0000 7f492169b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 0x560d1b8d3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.802+0000 7f492169b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f49141b24c0 con 0x560d1b5011b0 2024-07-04T08:21:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.802+0000 7f492169b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49141b4000 con 0x560d1b509390 2024-07-04T08:21:02.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.802+0000 7f492169b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f49141b2b90 con 0x560d1b52a070 2024-07-04T08:21:02.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.802+0000 7f4913e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b5011b0 0x560d1b8dc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b509390 0x560d1b8cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b509390 0x560d1b8cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:46102/0 (socket says 172.21.15.39:46102) 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913400640 1 -- 172.21.15.39:0/2890040525 learned_addr learned my addr 172.21.15.39:0/2890040525 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4919000640 1 --2- 172.21.15.39:0/2890040525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 0x560d1b8d3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913400640 1 -- 172.21.15.39:0/2890040525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 msgr2=0x560d1b8d3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913400640 1 --2- 172.21.15.39:0/2890040525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 0x560d1b8d3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913400640 1 -- 172.21.15.39:0/2890040525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b5011b0 msgr2=0x560d1b8dc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913400640 1 --2- 172.21.15.39:0/2890040525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b5011b0 0x560d1b8dc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913400640 1 -- 172.21.15.39:0/2890040525 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1b8d1040 con 0x560d1b509390 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913e00640 1 --2- 172.21.15.39:0/2890040525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b5011b0 0x560d1b8dc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.803+0000 7f4913400640 1 --2- 172.21.15.39:0/2890040525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b509390 0x560d1b8cfef0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009e90 tx=0x7f48fc0095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f440e761d19ff3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:02.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.804+0000 7f4908c00640 1 -- 172.21.15.39:0/2890040525 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00ee40 con 0x560d1b509390 2024-07-04T08:21:02.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.804+0000 7f4908c00640 1 -- 172.21.15.39:0/2890040525 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48fc00e3b0 con 0x560d1b509390 2024-07-04T08:21:02.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.804+0000 7f4908c00640 1 -- 172.21.15.39:0/2890040525 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc004070 con 0x560d1b509390 2024-07-04T08:21:02.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.805+0000 7f492169b740 1 -- 172.21.15.39:0/2890040525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b509390 msgr2=0x560d1b8cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:02.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.805+0000 7f492169b740 1 --2- 172.21.15.39:0/2890040525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b509390 0x560d1b8cfef0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009e90 tx=0x7f48fc0095f0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.805+0000 7f492169b740 1 -- 172.21.15.39:0/2890040525 shutdown_connections 2024-07-04T08:21:02.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.805+0000 7f492169b740 1 --2- 172.21.15.39:0/2890040525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b52a070 0x560d1b8d3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.805+0000 7f492169b740 1 --2- 172.21.15.39:0/2890040525 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b509390 0x560d1b8cfef0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.805+0000 7f492169b740 1 --2- 172.21.15.39:0/2890040525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b5011b0 0x560d1b8dc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.805+0000 7f492169b740 1 -- 172.21.15.39:0/2890040525 >> 172.21.15.39:0/2890040525 conn(0x560d1b51e460 msgr2=0x560d1b8cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:02.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.805+0000 7f492169b740 1 -- 172.21.15.39:0/2890040525 shutdown_connections 2024-07-04T08:21:02.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.805+0000 7f492169b740 1 -- 172.21.15.39:0/2890040525 wait complete. 2024-07-04T08:21:02.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.806+0000 7f492169b740 1 -- start start 2024-07-04T08:21:02.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.806+0000 7f492169b740 1 Processor -- start 2024-07-04T08:21:02.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f492169b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b5011b0 0x560d1b8e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f492169b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 0x560d1b8e5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f492169b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b52a070 0x560d1b8e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f492169b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f49141b3930 con 0x560d1b509390 2024-07-04T08:21:02.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f492169b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f49141b3260 con 0x560d1b52a070 2024-07-04T08:21:02.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f492169b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f49141b04a0 con 0x560d1b5011b0 2024-07-04T08:21:02.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f492169b740 10 client.0 _ll_register_callbacks cb 0x7f48fc00fc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:02.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f4913400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 0x560d1b8e5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f4919000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b52a070 0x560d1b8e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f4913e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b5011b0 0x560d1b8e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f4919000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b52a070 0x560d1b8e6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:46106/0 (socket says 172.21.15.39:46106) 2024-07-04T08:21:02.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.807+0000 7f4919000640 1 -- 172.21.15.39:0/4256450287 learned_addr learned my addr 172.21.15.39:0/4256450287 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f4919000640 1 -- 172.21.15.39:0/4256450287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b5011b0 msgr2=0x560d1b8e2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f4919000640 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b5011b0 0x560d1b8e2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f4919000640 1 -- 172.21.15.39:0/4256450287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 msgr2=0x560d1b8e5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:02.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f4919000640 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 0x560d1b8e5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f4919000640 1 -- 172.21.15.39:0/4256450287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1b8e67d0 con 0x560d1b52a070 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f4919000640 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b52a070 0x560d1b8e6090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f490c0c93e0 tx=0x7f490c009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75c04a476f99d032 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f4902000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f4902000640 1 -- 172.21.15.39:0/4256450287 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c0c8bc0 con 0x560d1b52a070 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f492169b740 1 -- 172.21.15.39:0/4256450287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48fc00e3b0 con 0x560d1b52a070 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.808+0000 7f4902000640 1 -- 172.21.15.39:0/4256450287 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f490c0d2030 con 0x560d1b52a070 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f4902000640 1 -- 172.21.15.39:0/4256450287 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f490c0c8f60 con 0x560d1b52a070 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f492169b740 10 client.26930 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f492169b740 1 -- 172.21.15.39:0/4256450287 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560d1b954d10 con 0x560d1b52a070 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f4902000640 1 -- 172.21.15.39:0/4256450287 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48fc00e3b0 con 0x560d1b52a070 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f4902000640 1 -- 172.21.15.39:0/4256450287 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f490c0e9020 con 0x560d1b52a070 2024-07-04T08:21:02.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f492169b740 10 client.26930 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:02.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f492169b740 10 client.26930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:21:02.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f492169b740 20 client.26930 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:02.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f492169b740 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x560d1b4bdda0 0x560d1b9573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:02.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f492169b740 4 client.26930 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:21:02.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.809+0000 7f492169b740 1 -- 172.21.15.39:0/4256450287 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d1b8dac50 con 0x560d1b4bdda0 2024-07-04T08:21:02.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.810+0000 7f4913e00640 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x560d1b4bdda0 0x560d1b9573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:02.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.810+0000 7f4913e00640 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x560d1b4bdda0 0x560d1b9573d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf97ee21c0c95f9c server_cookie=7525cd743511c196 in_seq=0 out_seq=0 2024-07-04T08:21:02.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.810+0000 7f4902000640 10 client.26930 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:21:02.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f4902000640 1 -- 172.21.15.39:0/4256450287 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d1b8dac50 con 0x560d1b4bdda0 2024-07-04T08:21:02.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f4902000640 10 client.26930 handle_command_reply: tid=0 2024-07-04T08:21:02.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 client.26930 shutdown 2024-07-04T08:21:02.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 -- 172.21.15.39:0/4256450287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b52a070 msgr2=0x560d1b8e6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b52a070 0x560d1b8e6090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f490c0c93e0 tx=0x7f490c009520 comp rx=0 tx=0).stop 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 -- 172.21.15.39:0/4256450287 shutdown_connections 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x560d1b4bdda0 0x560d1b9573d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560d1b509390 0x560d1b8e5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f4902000640 0 client.26930 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560d1b52a070 0x560d1b8e6090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 --2- 172.21.15.39:0/4256450287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1b5011b0 0x560d1b8e2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 -- 172.21.15.39:0/4256450287 >> 172.21.15.39:0/4256450287 conn(0x560d1b51e460 msgr2=0x560d1b8dbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.811+0000 7f492169b740 1 -- 172.21.15.39:0/4256450287 shutdown_connections 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.812+0000 7f492169b740 1 -- 172.21.15.39:0/4256450287 wait complete. 2024-07-04T08:21:02.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.812+0000 7f492169b740 20 client.26930 trim_cache size 0 max 16384 2024-07-04T08:21:02.810 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:21:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 -- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f48f40b2960 msgr2=0x7f48f40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f48f40b2960 0x7f48f40b4e70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f49141084a0 tx=0x7f48fc013040 comp rx=0 tx=0).stop 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 -- 172.21.15.39:0/691757019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4914109350 msgr2=0x7f49140756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4914109350 0x7f49140756e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f490c02a930 tx=0x7f490c032d50 comp rx=0 tx=0).stop 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 -- 172.21.15.39:0/691757019 shutdown_connections 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f48f40b2960 0x7f48f40b4e70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4914109350 0x7f49140756e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4914108a00 0x7f49140751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 --2- 172.21.15.39:0/691757019 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49141030c0 0x7f4914074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 -- 172.21.15.39:0/691757019 >> 172.21.15.39:0/691757019 conn(0x7f49140feb30 msgr2=0x7f4914100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 -- 172.21.15.39:0/691757019 shutdown_connections 2024-07-04T08:21:02.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:02.813+0000 7f491b000640 1 -- 172.21.15.39:0/691757019 wait complete. 2024-07-04T08:21:03.096 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:03 smithi039 ceph-mon[32062]: pgmap v655: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-07-04T08:21:03.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:03 smithi133 ceph-mon[37050]: pgmap v655: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-07-04T08:21:03.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:03 smithi050 ceph-mon[36577]: pgmap v655: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-07-04T08:21:03.623 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:21:04.094 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:05.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:21:05.269 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:05 smithi039 ceph-mon[32062]: pgmap v656: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-07-04T08:21:05.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:05 smithi133 ceph-mon[37050]: pgmap v656: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-07-04T08:21:05.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:05 smithi050 ceph-mon[36577]: pgmap v656: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.7 KiB/s wr, 11 op/s 2024-07-04T08:21:05.669 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:06.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.507+0000 7f0932200640 1 -- 172.21.15.39:0/1888334664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f092c104800 msgr2=0x7f092c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:06.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.507+0000 7f0932200640 1 --2- 172.21.15.39:0/1888334664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f092c104800 0x7f092c10ce40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0918009900 tx=0x7f091802a400 comp rx=0 tx=0).stop 2024-07-04T08:21:06.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.507+0000 7f0932200640 1 -- 172.21.15.39:0/1888334664 shutdown_connections 2024-07-04T08:21:06.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.507+0000 7f0932200640 1 --2- 172.21.15.39:0/1888334664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f092c104800 0x7f092c10ce40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.507+0000 7f0932200640 1 --2- 172.21.15.39:0/1888334664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f092c10d380 0x7f092c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.507+0000 7f0932200640 1 --2- 172.21.15.39:0/1888334664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c103dd0 0x7f092c1041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.508+0000 7f0932200640 1 -- 172.21.15.39:0/1888334664 >> 172.21.15.39:0/1888334664 conn(0x7f092c078cb0 msgr2=0x7f092c0790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:06.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.508+0000 7f0932200640 1 -- 172.21.15.39:0/1888334664 shutdown_connections 2024-07-04T08:21:06.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.508+0000 7f0932200640 1 -- 172.21.15.39:0/1888334664 wait complete. 2024-07-04T08:21:06.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.509+0000 7f0932200640 1 Processor -- start 2024-07-04T08:21:06.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.509+0000 7f0932200640 1 -- start start 2024-07-04T08:21:06.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.509+0000 7f0932200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f092c103dd0 0x7f092c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f0932200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f092c104800 0x7f092c19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f0932200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c10d380 0x7f092c1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f0932200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f092c113030 con 0x7f092c103dd0 2024-07-04T08:21:06.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f0932200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f092c112eb0 con 0x7f092c104800 2024-07-04T08:21:06.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f0932200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f092c1131b0 con 0x7f092c10d380 2024-07-04T08:21:06.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f092c104800 0x7f092c19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f092c104800 0x7f092c19c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:46116/0 (socket says 172.21.15.39:46116) 2024-07-04T08:21:06.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092aa00640 1 -- 172.21.15.39:0/4015128669 learned_addr learned my addr 172.21.15.39:0/4015128669 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:06.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092b400640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f092c103dd0 0x7f092c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092be00640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c10d380 0x7f092c1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092aa00640 1 -- 172.21.15.39:0/4015128669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c10d380 msgr2=0x7f092c1a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:06.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092aa00640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c10d380 0x7f092c1a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092aa00640 1 -- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f092c103dd0 msgr2=0x7f092c19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:06.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092aa00640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f092c103dd0 0x7f092c19bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092aa00640 1 -- 172.21.15.39:0/4015128669 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c1a1c10 con 0x7f092c104800 2024-07-04T08:21:06.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.510+0000 7f092be00640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c10d380 0x7f092c1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:06.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.511+0000 7f092aa00640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f092c104800 0x7f092c19c240 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0918005ab0 tx=0x7f0918003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:06.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.511+0000 7f0923e00640 1 -- 172.21.15.39:0/4015128669 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091803f070 con 0x7f092c104800 2024-07-04T08:21:06.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.511+0000 7f0923e00640 1 -- 172.21.15.39:0/4015128669 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0918031b50 con 0x7f092c104800 2024-07-04T08:21:06.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.511+0000 7f0923e00640 1 -- 172.21.15.39:0/4015128669 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918035030 con 0x7f092c104800 2024-07-04T08:21:06.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.511+0000 7f0932200640 1 -- 172.21.15.39:0/4015128669 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092c1a1ea0 con 0x7f092c104800 2024-07-04T08:21:06.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.512+0000 7f0932200640 1 -- 172.21.15.39:0/4015128669 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092c1a8660 con 0x7f092c104800 2024-07-04T08:21:06.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.515+0000 7f0938877740 1 Processor -- start 2024-07-04T08:21:06.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.515+0000 7f0938877740 1 -- start start 2024-07-04T08:21:06.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f0938877740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbd0c070 0x5555bc0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f0938877740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f0938877740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbceb390 0x5555bc0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f0938877740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5555bc026f00 con 0x5555bbce31b0 2024-07-04T08:21:06.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f0938877740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5555bc0164a0 con 0x5555bbd0c070 2024-07-04T08:21:06.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f0938877740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5555bc01a4e0 con 0x5555bbceb390 2024-07-04T08:21:06.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f092b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f092b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36178/0 (socket says 172.21.15.39:36178) 2024-07-04T08:21:06.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f092b400640 1 -- 172.21.15.39:0/1788673309 learned_addr learned my addr 172.21.15.39:0/1788673309 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:06.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f092be00640 1 --2- 172.21.15.39:0/1788673309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbd0c070 0x5555bc0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.516+0000 7f0923e00640 1 -- 172.21.15.39:0/4015128669 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0918042070 con 0x7f092c104800 2024-07-04T08:21:06.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f092b400640 1 -- 172.21.15.39:0/1788673309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbceb390 msgr2=0x5555bc0b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:06.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f092b400640 1 --2- 172.21.15.39:0/1788673309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbceb390 0x5555bc0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f092b400640 1 -- 172.21.15.39:0/1788673309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbd0c070 msgr2=0x5555bc0b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:06.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f092b400640 1 --2- 172.21.15.39:0/1788673309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbd0c070 0x5555bc0b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f092b400640 1 -- 172.21.15.39:0/1788673309 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555bbfae670 con 0x5555bbce31b0 2024-07-04T08:21:06.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f092be00640 1 --2- 172.21.15.39:0/1788673309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbd0c070 0x5555bc0b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:06.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f092b400640 1 --2- 172.21.15.39:0/1788673309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0b0d10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f091c011470 tx=0x7f091c011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8685ea616b0854 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:06.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f0920c00640 1 -- 172.21.15.39:0/1788673309 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c003970 con 0x5555bbce31b0 2024-07-04T08:21:06.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f0920c00640 1 -- 172.21.15.39:0/1788673309 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f091c004020 con 0x5555bbce31b0 2024-07-04T08:21:06.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.517+0000 7f0920c00640 1 -- 172.21.15.39:0/1788673309 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c00f8d0 con 0x5555bbce31b0 2024-07-04T08:21:06.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.518+0000 7f0938877740 1 -- 172.21.15.39:0/1788673309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 msgr2=0x5555bc0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:06.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.518+0000 7f0938877740 1 --2- 172.21.15.39:0/1788673309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0b0d10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f091c011470 tx=0x7f091c011930 comp rx=0 tx=0).stop 2024-07-04T08:21:06.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.518+0000 7f0938877740 1 -- 172.21.15.39:0/1788673309 shutdown_connections 2024-07-04T08:21:06.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.518+0000 7f0938877740 1 --2- 172.21.15.39:0/1788673309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0b0d10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.518+0000 7f0938877740 1 --2- 172.21.15.39:0/1788673309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbceb390 0x5555bc0b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.518+0000 7f0938877740 1 --2- 172.21.15.39:0/1788673309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbd0c070 0x5555bc0b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.518+0000 7f0923e00640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09100b2a60 0x7f09100b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.518+0000 7f0938877740 1 -- 172.21.15.39:0/1788673309 >> 172.21.15.39:0/1788673309 conn(0x5555bbd00460 msgr2=0x5555bc0ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.519+0000 7f0923e00640 1 -- 172.21.15.39:0/4015128669 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f0918102bc0 con 0x7f092c104800 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.519+0000 7f0938877740 1 -- 172.21.15.39:0/1788673309 shutdown_connections 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.519+0000 7f092be00640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09100b2a60 0x7f09100b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.519+0000 7f0938877740 1 -- 172.21.15.39:0/1788673309 wait complete. 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.520+0000 7f0938877740 1 -- start start 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.520+0000 7f092be00640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09100b2a60 0x7f09100b4f70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0924009c70 tx=0x7f0924005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:06.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.520+0000 7f0938877740 1 Processor -- start 2024-07-04T08:21:06.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.520+0000 7f0938877740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.520+0000 7f0938877740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 0x5555bc0bc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f0938877740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 0x5555bc0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f0938877740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5555bbe7a700 con 0x5555bbce31b0 2024-07-04T08:21:06.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f0938877740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5555bbfbdbd0 con 0x5555bbceb390 2024-07-04T08:21:06.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f0938877740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5555bc0befb0 con 0x5555bbd0c070 2024-07-04T08:21:06.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f0938877740 10 client.0 _ll_register_callbacks cb 0x5555bc00f590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:06.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 0x5555bc0bc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0bc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36192/0 (socket says 172.21.15.39:36192) 2024-07-04T08:21:06.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092b400640 1 -- 172.21.15.39:0/21629445 learned_addr learned my addr 172.21.15.39:0/21629445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:06.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092be00640 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 0x5555bc0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092aa00640 1 -- 172.21.15.39:0/21629445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 msgr2=0x5555bc0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092aa00640 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 0x5555bc0c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092aa00640 1 -- 172.21.15.39:0/21629445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 msgr2=0x5555bc0bc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092aa00640 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0bc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.521+0000 7f092aa00640 1 -- 172.21.15.39:0/21629445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555bbd77650 con 0x5555bbceb390 2024-07-04T08:21:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.522+0000 7f092aa00640 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 0x5555bc0bc8e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09180fff80 tx=0x7f09180c9460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba25851d01f91d83 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.522+0000 7f0915600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:21:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.522+0000 7f0915600640 1 -- 172.21.15.39:0/21629445 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918049040 con 0x5555bbceb390 2024-07-04T08:21:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.522+0000 7f0915600640 1 -- 172.21.15.39:0/21629445 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f09180c5da0 con 0x5555bbceb390 2024-07-04T08:21:06.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.522+0000 7f0938877740 1 -- 172.21.15.39:0/21629445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f091c004020 con 0x5555bbceb390 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0915600640 1 -- 172.21.15.39:0/21629445 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f09180c9620 con 0x5555bbceb390 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0938877740 10 client.26942 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0938877740 1 -- 172.21.15.39:0/21629445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555bc0cafa0 con 0x5555bbceb390 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0915600640 1 -- 172.21.15.39:0/21629445 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f091c004020 con 0x5555bbceb390 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0915600640 1 -- 172.21.15.39:0/21629445 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f091804c060 con 0x5555bbceb390 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0938877740 10 client.26942 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0938877740 10 client.26942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0938877740 20 client.26942 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0938877740 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5555bbcca9f0 0x5555bc0ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0938877740 4 client.26942 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:06.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.523+0000 7f0938877740 1 -- 172.21.15.39:0/21629445 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555bc028e50 con 0x5555bbcca9f0 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.524+0000 7f092b400640 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5555bbcca9f0 0x5555bc0ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.524+0000 7f0915600640 10 client.26942 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.524+0000 7f092b400640 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5555bbcca9f0 0x5555bc0ce3c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990070baf3d2af8a server_cookie=17db01c3f737f46 in_seq=0 out_seq=0 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.525+0000 7f0915600640 1 -- 172.21.15.39:0/21629445 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5555bc028e50 con 0x5555bbcca9f0 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.525+0000 7f0915600640 10 client.26942 handle_command_reply: tid=0 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.525+0000 7f0938877740 1 client.26942 shutdown 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 -- 172.21.15.39:0/21629445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 msgr2=0x5555bc0bc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 0x5555bc0bc8e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09180fff80 tx=0x7f09180c9460 comp rx=0 tx=0).stop 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 -- 172.21.15.39:0/21629445 shutdown_connections 2024-07-04T08:21:06.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5555bbcca9f0 0x5555bc0ce3c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 0x5555bc0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 0x5555bc0bc8e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0915600640 0 client.26942 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 --2- 172.21.15.39:0/21629445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0bc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 -- 172.21.15.39:0/21629445 >> 172.21.15.39:0/21629445 conn(0x5555bbd00460 msgr2=0x5555bc0b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 -- 172.21.15.39:0/21629445 shutdown_connections 2024-07-04T08:21:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 1 -- 172.21.15.39:0/21629445 wait complete. 2024-07-04T08:21:06.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.526+0000 7f0938877740 20 client.26942 trim_cache size 0 max 16384 2024-07-04T08:21:06.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.542+0000 7f0938877740 1 Processor -- start 2024-07-04T08:21:06.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.542+0000 7f0938877740 1 -- start start 2024-07-04T08:21:06.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.542+0000 7f0938877740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.542+0000 7f0938877740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 0x5555bc0ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.542+0000 7f0938877740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 0x5555bc0c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.542+0000 7f0938877740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f092c1b2470 con 0x5555bbce31b0 2024-07-04T08:21:06.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.542+0000 7f0938877740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f092c1b3fb0 con 0x5555bbceb390 2024-07-04T08:21:06.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.542+0000 7f0938877740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f092c1b2b40 con 0x5555bbd0c070 2024-07-04T08:21:06.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36204/0 (socket says 172.21.15.39:36204) 2024-07-04T08:21:06.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092b400640 1 -- 172.21.15.39:0/2137975171 learned_addr learned my addr 172.21.15.39:0/2137975171 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:06.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092be00640 1 --2- 172.21.15.39:0/2137975171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 0x5555bc0c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092aa00640 1 --2- 172.21.15.39:0/2137975171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 0x5555bc0ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092b400640 1 -- 172.21.15.39:0/2137975171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 msgr2=0x5555bc0c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092b400640 1 --2- 172.21.15.39:0/2137975171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 0x5555bc0c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092b400640 1 -- 172.21.15.39:0/2137975171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 msgr2=0x5555bc0ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092b400640 1 --2- 172.21.15.39:0/2137975171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 0x5555bc0ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.543+0000 7f092b400640 1 -- 172.21.15.39:0/2137975171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555bc0bee90 con 0x5555bbce31b0 2024-07-04T08:21:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.544+0000 7f092b400640 1 --2- 172.21.15.39:0/2137975171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0ad960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x5555bc0af200 tx=0x7f091c004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451980680618b76 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.544+0000 7f0917e00640 1 -- 172.21.15.39:0/2137975171 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c032070 con 0x5555bbce31b0 2024-07-04T08:21:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.544+0000 7f0917e00640 1 -- 172.21.15.39:0/2137975171 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f091c00fab0 con 0x5555bbce31b0 2024-07-04T08:21:06.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.544+0000 7f0917e00640 1 -- 172.21.15.39:0/2137975171 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c0055b0 con 0x5555bbce31b0 2024-07-04T08:21:06.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.545+0000 7f0938877740 1 -- 172.21.15.39:0/2137975171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 msgr2=0x5555bc0ad960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:06.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.545+0000 7f0938877740 1 --2- 172.21.15.39:0/2137975171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0ad960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x5555bc0af200 tx=0x7f091c004b60 comp rx=0 tx=0).stop 2024-07-04T08:21:06.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.545+0000 7f0938877740 1 -- 172.21.15.39:0/2137975171 shutdown_connections 2024-07-04T08:21:06.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.545+0000 7f0938877740 1 --2- 172.21.15.39:0/2137975171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbd0c070 0x5555bc0c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.545+0000 7f0938877740 1 --2- 172.21.15.39:0/2137975171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbceb390 0x5555bc0ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.545+0000 7f0938877740 1 --2- 172.21.15.39:0/2137975171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbce31b0 0x5555bc0ad960 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.545+0000 7f0938877740 1 -- 172.21.15.39:0/2137975171 >> 172.21.15.39:0/2137975171 conn(0x5555bbd00460 msgr2=0x5555bc0adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:06.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.545+0000 7f0938877740 1 -- 172.21.15.39:0/2137975171 shutdown_connections 2024-07-04T08:21:06.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.545+0000 7f0938877740 1 -- 172.21.15.39:0/2137975171 wait complete. 2024-07-04T08:21:06.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.546+0000 7f0938877740 1 -- start start 2024-07-04T08:21:06.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.546+0000 7f0938877740 1 Processor -- start 2024-07-04T08:21:06.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f0938877740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbce31b0 0x5555bc0c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f0938877740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbceb390 0x5555bc0c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f0938877740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbd0c070 0x5555bc0bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f0938877740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f092c1b38e0 con 0x5555bbd0c070 2024-07-04T08:21:06.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f0938877740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f092c1b3210 con 0x5555bbce31b0 2024-07-04T08:21:06.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f0938877740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f092c1b0450 con 0x5555bbceb390 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f0938877740 10 client.0 _ll_register_callbacks cb 0x7f09180c59d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbce31b0 0x5555bc0c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbd0c070 0x5555bc0bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbce31b0 0x5555bc0c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:46134/0 (socket says 172.21.15.39:46134) 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092b400640 1 -- 172.21.15.39:0/2694365059 learned_addr learned my addr 172.21.15.39:0/2694365059 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092aa00640 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbceb390 0x5555bc0c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092b400640 1 -- 172.21.15.39:0/2694365059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbceb390 msgr2=0x5555bc0c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092b400640 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbceb390 0x5555bc0c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092b400640 1 -- 172.21.15.39:0/2694365059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbd0c070 msgr2=0x5555bc0bf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092b400640 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbd0c070 0x5555bc0bf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.547+0000 7f092b400640 1 -- 172.21.15.39:0/2694365059 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555bc0c4ec0 con 0x5555bbce31b0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.548+0000 7f092b400640 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbce31b0 0x5555bc0c4240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f091c009730 tx=0x7f091c00fe90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ea26685b5a957 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.548+0000 7f0915600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.548+0000 7f0915600640 1 -- 172.21.15.39:0/2694365059 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c03d060 con 0x5555bbce31b0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.548+0000 7f0938877740 1 -- 172.21.15.39:0/2694365059 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f091c00fab0 con 0x5555bbce31b0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.548+0000 7f0915600640 1 -- 172.21.15.39:0/2694365059 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f091c00f810 con 0x5555bbce31b0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0915600640 1 -- 172.21.15.39:0/2694365059 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f091c032070 con 0x5555bbce31b0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0938877740 10 client.26948 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0938877740 1 -- 172.21.15.39:0/2694365059 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555bc137110 con 0x5555bbce31b0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0915600640 1 -- 172.21.15.39:0/2694365059 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f091c00fab0 con 0x5555bbce31b0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0915600640 1 -- 172.21.15.39:0/2694365059 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f091c02c030 con 0x5555bbce31b0 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0938877740 10 client.26948 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0938877740 10 client.26948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:06.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0938877740 20 client.26948 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0938877740 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5555bbc9fda0 0x5555bc139710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0938877740 4 client.26948 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.549+0000 7f0938877740 1 -- 172.21.15.39:0/2694365059 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555bc0248f0 con 0x5555bbc9fda0 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.550+0000 7f092aa00640 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5555bbc9fda0 0x5555bc139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.550+0000 7f092aa00640 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5555bbc9fda0 0x5555bc139710 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=968bdce7167a75bd server_cookie=8b5d55ee8bf63cf0 in_seq=0 out_seq=0 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.550+0000 7f0915600640 10 client.26948 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.550+0000 7f0915600640 1 -- 172.21.15.39:0/2694365059 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555bc0248f0 con 0x5555bbc9fda0 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.550+0000 7f0915600640 10 client.26948 handle_command_reply: tid=0 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.550+0000 7f0938877740 1 client.26948 shutdown 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 -- 172.21.15.39:0/2694365059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbce31b0 msgr2=0x5555bc0c4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbce31b0 0x5555bc0c4240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f091c009730 tx=0x7f091c00fe90 comp rx=0 tx=0).stop 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 -- 172.21.15.39:0/2694365059 shutdown_connections 2024-07-04T08:21:06.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5555bbc9fda0 0x5555bc139710 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5555bbd0c070 0x5555bc0bf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5555bbceb390 0x5555bc0c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 --2- 172.21.15.39:0/2694365059 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5555bbce31b0 0x5555bc0c4240 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0915600640 0 client.26948 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 -- 172.21.15.39:0/2694365059 >> 172.21.15.39:0/2694365059 conn(0x5555bbd00460 msgr2=0x5555bc0b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 -- 172.21.15.39:0/2694365059 shutdown_connections 2024-07-04T08:21:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 1 -- 172.21.15.39:0/2694365059 wait complete. 2024-07-04T08:21:06.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.551+0000 7f0938877740 20 client.26948 trim_cache size 0 max 16384 2024-07-04T08:21:06.552 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:21:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 -- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09100b2a60 msgr2=0x7f09100b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09100b2a60 0x7f09100b4f70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0924009c70 tx=0x7f0924005f90 comp rx=0 tx=0).stop 2024-07-04T08:21:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 -- 172.21.15.39:0/4015128669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f092c104800 msgr2=0x7f092c19c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f092c104800 0x7f092c19c240 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0918005ab0 tx=0x7f0918003b90 comp rx=0 tx=0).stop 2024-07-04T08:21:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 -- 172.21.15.39:0/4015128669 shutdown_connections 2024-07-04T08:21:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09100b2a60 0x7f09100b4f70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f092c10d380 0x7f092c1a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f092c104800 0x7f092c19c240 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 --2- 172.21.15.39:0/4015128669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f092c103dd0 0x7f092c19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 -- 172.21.15.39:0/4015128669 >> 172.21.15.39:0/4015128669 conn(0x7f092c078cb0 msgr2=0x7f092c10c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 -- 172.21.15.39:0/4015128669 shutdown_connections 2024-07-04T08:21:06.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:06.552+0000 7f0932200640 1 -- 172.21.15.39:0/4015128669 wait complete. 2024-07-04T08:21:07.276 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:07 smithi039 ceph-mon[32062]: pgmap v657: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-07-04T08:21:07.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:07 smithi133 ceph-mon[37050]: pgmap v657: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-07-04T08:21:07.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:07 smithi050 ceph-mon[36577]: pgmap v657: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 KiB/s wr, 26 op/s 2024-07-04T08:21:08.689 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:21:08.689 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:21:09.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.110+0000 7f8ff7e00640 1 -- 172.21.15.39:0/3906817489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ff01089f0 msgr2=0x7f8ff0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:09.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.110+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/3906817489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ff01089f0 0x7f8ff0108e00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009900 tx=0x7f8fdc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.110+0000 7f8ff7e00640 1 -- 172.21.15.39:0/3906817489 shutdown_connections 2024-07-04T08:21:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.110+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/3906817489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ff01089f0 0x7f8ff0108e00 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.110+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/3906817489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff0109340 0x7f8ff010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.110+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/3906817489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ff01030b0 0x7f8ff01034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.110+0000 7f8ff7e00640 1 -- 172.21.15.39:0/3906817489 >> 172.21.15.39:0/3906817489 conn(0x7f8ff007ada0 msgr2=0x7f8ff007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.110+0000 7f8ff7e00640 1 -- 172.21.15.39:0/3906817489 shutdown_connections 2024-07-04T08:21:09.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.110+0000 7f8ff7e00640 1 -- 172.21.15.39:0/3906817489 wait complete. 2024-07-04T08:21:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.112+0000 7f8ff7e00640 1 Processor -- start 2024-07-04T08:21:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.112+0000 7f8ff7e00640 1 -- start start 2024-07-04T08:21:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.112+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ff01030b0 0x7f8ff019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ff01089f0 0x7f8ff019c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:09.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff0109340 0x7f8ff01a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:09.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0112fd0 con 0x7f8ff01030b0 2024-07-04T08:21:09.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0112e50 con 0x7f8ff01089f0 2024-07-04T08:21:09.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0113150 con 0x7f8ff0109340 2024-07-04T08:21:09.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ff01030b0 0x7f8ff019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ff01089f0 0x7f8ff019c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff0109340 0x7f8ff01a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ff01089f0 0x7f8ff019c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:46148/0 (socket says 172.21.15.39:46148) 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.113+0000 7f8ff4a00640 1 -- 172.21.15.39:0/523553134 learned_addr learned my addr 172.21.15.39:0/523553134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8ff5e00640 1 -- 172.21.15.39:0/523553134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ff01089f0 msgr2=0x7f8ff019c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8ff5e00640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ff01089f0 0x7f8ff019c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8ff5e00640 1 -- 172.21.15.39:0/523553134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ff01030b0 msgr2=0x7f8ff019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8ff5e00640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ff01030b0 0x7f8ff019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8ff5e00640 1 -- 172.21.15.39:0/523553134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff01a1b90 con 0x7f8ff0109340 2024-07-04T08:21:09.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8ff5e00640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff0109340 0x7f8ff01a1460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009fd0 tx=0x7f8fe800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8fee000640 1 -- 172.21.15.39:0/523553134 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800ce20 con 0x7f8ff0109340 2024-07-04T08:21:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8fee000640 1 -- 172.21.15.39:0/523553134 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe8005020 con 0x7f8ff0109340 2024-07-04T08:21:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff01a1e80 con 0x7f8ff0109340 2024-07-04T08:21:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.114+0000 7f8fee000640 1 -- 172.21.15.39:0/523553134 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8010690 con 0x7f8ff0109340 2024-07-04T08:21:09.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.115+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01a85e0 con 0x7f8ff0109340 2024-07-04T08:21:09.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.116+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff01034c0 con 0x7f8ff0109340 2024-07-04T08:21:09.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.120+0000 7f8fee000640 1 -- 172.21.15.39:0/523553134 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8fe80054d0 con 0x7f8ff0109340 2024-07-04T08:21:09.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.121+0000 7f8fee000640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8fd80b2960 0x7f8fd80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.122+0000 7f8fee000640 1 -- 172.21.15.39:0/523553134 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8fe80e32d0 con 0x7f8ff0109340 2024-07-04T08:21:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.122+0000 7f8ff5400640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8fd80b2960 0x7f8fd80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.122+0000 7f8fee000640 1 -- 172.21.15.39:0/523553134 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f8fe80aa920 con 0x7f8ff0109340 2024-07-04T08:21:09.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.123+0000 7f8ff5400640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8fd80b2960 0x7f8fd80b4e70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009bd0 tx=0x7f8fe4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:09.181 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:09.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:09 smithi133 ceph-mon[37050]: pgmap v658: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-07-04T08:21:09.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:09 smithi039 ceph-mon[32062]: pgmap v658: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-07-04T08:21:09.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:09 smithi050 ceph-mon[36577]: pgmap v658: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-07-04T08:21:09.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.424+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ff019cd50 con 0x7f8ff0109340 2024-07-04T08:21:09.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.425+0000 7f8fee000640 1 -- 172.21.15.39:0/523553134 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f8ff019cd50 con 0x7f8ff0109340 2024-07-04T08:21:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:21:09.423 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:21:09.423 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:21:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.428+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8fd80b2960 msgr2=0x7f8fd80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.428+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8fd80b2960 0x7f8fd80b4e70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009bd0 tx=0x7f8fe4005f90 comp rx=0 tx=0).stop 2024-07-04T08:21:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.428+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff0109340 msgr2=0x7f8ff01a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:09.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.428+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff0109340 0x7f8ff01a1460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009fd0 tx=0x7f8fe800ef10 comp rx=0 tx=0).stop 2024-07-04T08:21:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.428+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 shutdown_connections 2024-07-04T08:21:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.428+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8fd80b2960 0x7f8fd80b4e70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.428+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ff0109340 0x7f8ff01a1460 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.428+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ff01089f0 0x7f8ff019c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.428+0000 7f8ff7e00640 1 --2- 172.21.15.39:0/523553134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ff01030b0 0x7f8ff019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.429+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 >> 172.21.15.39:0/523553134 conn(0x7f8ff007ada0 msgr2=0x7f8ff0106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.429+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 shutdown_connections 2024-07-04T08:21:09.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:09.429+0000 7f8ff7e00640 1 -- 172.21.15.39:0/523553134 wait complete. 2024-07-04T08:21:10.274 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:10 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/523553134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:10.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:10 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/523553134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:10.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:10 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/523553134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:11.165 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:11 smithi039 ceph-mon[32062]: pgmap v659: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-07-04T08:21:11.212 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id 899c97a4 --release --await' 2024-07-04T08:21:11.212 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id 899c97a4 --release --await 2024-07-04T08:21:11.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:11 smithi133 ceph-mon[37050]: pgmap v659: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-07-04T08:21:11.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:11 smithi050 ceph-mon[36577]: pgmap v659: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-07-04T08:21:11.795 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.818+0000 7f09b9600640 1 -- 172.21.15.39:0/1571402460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 msgr2=0x7f09b410fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.818+0000 7f09b9600640 1 --2- 172.21.15.39:0/1571402460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 0x7f09b410fb80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009be0 tx=0x7f09ac02a6c0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.818+0000 7f09b9600640 1 -- 172.21.15.39:0/1571402460 shutdown_connections 2024-07-04T08:21:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.818+0000 7f09b9600640 1 --2- 172.21.15.39:0/1571402460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b4108a00 0x7f09b4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.818+0000 7f09b9600640 1 --2- 172.21.15.39:0/1571402460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 0x7f09b410fb80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.818+0000 7f09b9600640 1 --2- 172.21.15.39:0/1571402460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09b4103110 0x7f09b4103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.818+0000 7f09b9600640 1 -- 172.21.15.39:0/1571402460 >> 172.21.15.39:0/1571402460 conn(0x7f09b407ae60 msgr2=0x7f09b407b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.818+0000 7f09b9600640 1 -- 172.21.15.39:0/1571402460 shutdown_connections 2024-07-04T08:21:11.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.818+0000 7f09b9600640 1 -- 172.21.15.39:0/1571402460 wait complete. 2024-07-04T08:21:11.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.819+0000 7f09b9600640 1 Processor -- start 2024-07-04T08:21:11.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.820+0000 7f09b9600640 1 -- start start 2024-07-04T08:21:11.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.820+0000 7f09b9600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09b4103110 0x7f09b4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.820+0000 7f09b9600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b4108a00 0x7f09b40751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.820+0000 7f09b9600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 0x7f09b40756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.820+0000 7f09b9600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09b4113000 con 0x7f09b4108a00 2024-07-04T08:21:11.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.820+0000 7f09b9600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09b4112e80 con 0x7f09b4109350 2024-07-04T08:21:11.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.820+0000 7f09b9600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09b4113180 con 0x7f09b4103110 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 0x7f09b40756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09b4103110 0x7f09b4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 0x7f09b40756e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35862/0 (socket says 172.21.15.39:35862) 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b3400640 1 -- 172.21.15.39:0/3752762227 learned_addr learned my addr 172.21.15.39:0/3752762227 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b2000640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b4108a00 0x7f09b40751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b3400640 1 -- 172.21.15.39:0/3752762227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09b4103110 msgr2=0x7f09b4074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b3400640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09b4103110 0x7f09b4074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b3400640 1 -- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b4108a00 msgr2=0x7f09b40751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b3400640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b4108a00 0x7f09b40751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.821+0000 7f09b3400640 1 -- 172.21.15.39:0/3752762227 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b41a7c30 con 0x7f09b4109350 2024-07-04T08:21:11.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.822+0000 7f09b3400640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 0x7f09b40756e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09ac004af0 tx=0x7f09ac003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.822+0000 7f09ab400640 1 -- 172.21.15.39:0/3752762227 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac041020 con 0x7f09b4109350 2024-07-04T08:21:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.822+0000 7f09b9600640 1 -- 172.21.15.39:0/3752762227 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b41a7ec0 con 0x7f09b4109350 2024-07-04T08:21:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.822+0000 7f09ab400640 1 -- 172.21.15.39:0/3752762227 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09ac003e10 con 0x7f09b4109350 2024-07-04T08:21:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.822+0000 7f09ab400640 1 -- 172.21.15.39:0/3752762227 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ac036040 con 0x7f09b4109350 2024-07-04T08:21:11.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.823+0000 7f09b9600640 1 -- 172.21.15.39:0/3752762227 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b41a8370 con 0x7f09b4109350 2024-07-04T08:21:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.826+0000 7f09bfc6e740 1 Processor -- start 2024-07-04T08:21:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.827+0000 7f09ab400640 1 -- 172.21.15.39:0/3752762227 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f09ac044020 con 0x7f09b4109350 2024-07-04T08:21:11.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.827+0000 7f09bfc6e740 1 -- start start 2024-07-04T08:21:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.827+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b28f350 0x56262b633640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.827+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b266490 0x56262b633b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.827+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 0x56262b6340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.827+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56262b55d8e0 con 0x56262b266490 2024-07-04T08:21:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.827+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56262b5a9ec0 con 0x56262b26e670 2024-07-04T08:21:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.827+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56262b599460 con 0x56262b28f350 2024-07-04T08:21:11.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.828+0000 7f09b2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b28f350 0x56262b633640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.828+0000 7f09b2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b28f350 0x56262b633640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36182/0 (socket says 172.21.15.39:36182) 2024-07-04T08:21:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.828+0000 7f09b2000640 1 -- 172.21.15.39:0/266662026 learned_addr learned my addr 172.21.15.39:0/266662026 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.828+0000 7f09b2a00640 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b266490 0x56262b633b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.828+0000 7f09b2000640 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 0x56262b6340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.828+0000 7f09b2000640 1 -- 172.21.15.39:0/266662026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 msgr2=0x56262b6340c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.828+0000 7f09b2000640 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 0x56262b6340c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09b2000640 1 -- 172.21.15.39:0/266662026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b266490 msgr2=0x56262b633b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09b2000640 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b266490 0x56262b633b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09b2000640 1 -- 172.21.15.39:0/266662026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262b531280 con 0x56262b28f350 2024-07-04T08:21:11.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09b2a00640 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b266490 0x56262b633b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:11.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09ab400640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09980b29e0 0x7f09980b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09b2000640 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b28f350 0x56262b633640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f099c012720 tx=0x7f099c009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cc3e7ab906d2ec server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:11.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09ab400640 1 -- 172.21.15.39:0/3752762227 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f09ac103500 con 0x7f09b4109350 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09b2a00640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09980b29e0 0x7f09980b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09a3e00640 1 -- 172.21.15.39:0/266662026 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c012ca0 con 0x56262b28f350 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.829+0000 7f09a3e00640 1 -- 172.21.15.39:0/266662026 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099c012e40 con 0x56262b28f350 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.830+0000 7f09a3e00640 1 -- 172.21.15.39:0/266662026 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0035f0 con 0x56262b28f350 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.830+0000 7f09bfc6e740 1 -- 172.21.15.39:0/266662026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b28f350 msgr2=0x56262b633640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.830+0000 7f09b2a00640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09980b29e0 0x7f09980b4ef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x56262b6322c0 tx=0x7f09a4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.830+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b28f350 0x56262b633640 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f099c012720 tx=0x7f099c009680 comp rx=0 tx=0).stop 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.831+0000 7f09bfc6e740 1 -- 172.21.15.39:0/266662026 shutdown_connections 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.831+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b266490 0x56262b633b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.831+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 0x56262b6340c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.831+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/266662026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b28f350 0x56262b633640 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.831+0000 7f09bfc6e740 1 -- 172.21.15.39:0/266662026 >> 172.21.15.39:0/266662026 conn(0x56262b283740 msgr2=0x56262b631530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.831+0000 7f09bfc6e740 1 -- 172.21.15.39:0/266662026 shutdown_connections 2024-07-04T08:21:11.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.831+0000 7f09bfc6e740 1 -- 172.21.15.39:0/266662026 wait complete. 2024-07-04T08:21:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.832+0000 7f09bfc6e740 1 -- start start 2024-07-04T08:21:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.832+0000 7f09bfc6e740 1 Processor -- start 2024-07-04T08:21:11.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.832+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b63f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b63f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56262b540760 con 0x56262b26e670 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56262b59d4a0 con 0x56262b28f350 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56262b3fd890 con 0x56262b266490 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09bfc6e740 10 client.0 _ll_register_callbacks cb 0x56262b5412b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09b2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b63f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09b2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b63f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45466/0 (socket says 172.21.15.39:45466) 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09b2000640 1 -- 172.21.15.39:0/3257425841 learned_addr learned my addr 172.21.15.39:0/3257425841 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09b2a00640 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b63f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09b3400640 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09b2a00640 1 -- 172.21.15.39:0/3257425841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 msgr2=0x56262b64af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.833+0000 7f09b2a00640 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.834+0000 7f09b2a00640 1 -- 172.21.15.39:0/3257425841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 msgr2=0x56262b63f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.834+0000 7f09b2a00640 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b63f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.834+0000 7f09b2a00640 1 -- 172.21.15.39:0/3257425841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262b2fa930 con 0x56262b266490 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.834+0000 7f09b3400640 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.834+0000 7f09b2000640 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b63f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.834+0000 7f09b2a00640 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b63f1a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009fd0 tx=0x7f09a4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=190c5e5a91dbf101 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.834+0000 7f09a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.834+0000 7f09a0c00640 1 -- 172.21.15.39:0/3257425841 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a402e060 con 0x56262b266490 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.834+0000 7f09a0c00640 1 -- 172.21.15.39:0/3257425841 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f09a4018030 con 0x56262b266490 2024-07-04T08:21:11.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.835+0000 7f09bfc6e740 1 -- 172.21.15.39:0/3257425841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56262b5abe10 con 0x56262b266490 2024-07-04T08:21:11.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.835+0000 7f09a0c00640 1 -- 172.21.15.39:0/3257425841 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f09a4035050 con 0x56262b266490 2024-07-04T08:21:11.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.835+0000 7f09bfc6e740 10 client.26872 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.835+0000 7f09bfc6e740 1 -- 172.21.15.39:0/3257425841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56262b64dc30 con 0x56262b266490 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.836+0000 7f09a0c00640 1 -- 172.21.15.39:0/3257425841 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56262b5abe10 con 0x56262b266490 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.836+0000 7f09a0c00640 1 -- 172.21.15.39:0/3257425841 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f09a401c3a0 con 0x56262b266490 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.836+0000 7f09bfc6e740 10 client.26872 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.836+0000 7f09bfc6e740 10 client.26872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.836+0000 7f09bfc6e740 20 client.26872 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.836+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56262b24dcd0 0x56262b651140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.836+0000 7f09bfc6e740 4 client.26872 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.836+0000 7f09bfc6e740 1 -- 172.21.15.39:0/3257425841 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56262b5a78b0 con 0x56262b24dcd0 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.837+0000 7f09b2000640 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56262b24dcd0 0x56262b651140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.837+0000 7f09b2000640 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56262b24dcd0 0x56262b651140 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445f3ce3ffb192a3 server_cookie=b1286e356617fee0 in_seq=0 out_seq=0 2024-07-04T08:21:11.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.837+0000 7f09a0c00640 10 client.26872 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.838+0000 7f09a0c00640 1 -- 172.21.15.39:0/3257425841 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56262b5a78b0 con 0x56262b24dcd0 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.838+0000 7f09a0c00640 10 client.26872 handle_command_reply: tid=0 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.838+0000 7f09bfc6e740 1 client.26872 shutdown 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 -- 172.21.15.39:0/3257425841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 msgr2=0x56262b63f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b63f1a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009fd0 tx=0x7f09a4004730 comp rx=0 tx=0).stop 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 -- 172.21.15.39:0/3257425841 shutdown_connections 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56262b24dcd0 0x56262b651140 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b63f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/3257425841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b63f1a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 -- 172.21.15.39:0/3257425841 >> 172.21.15.39:0/3257425841 conn(0x56262b283740 msgr2=0x56262b6367d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:11.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09a0c00640 0 client.26872 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:11.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 -- 172.21.15.39:0/3257425841 shutdown_connections 2024-07-04T08:21:11.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 1 -- 172.21.15.39:0/3257425841 wait complete. 2024-07-04T08:21:11.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.839+0000 7f09bfc6e740 20 client.26872 trim_cache size 0 max 16384 2024-07-04T08:21:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.861+0000 7f09bfc6e740 1 Processor -- start 2024-07-04T08:21:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.861+0000 7f09bfc6e740 1 -- start start 2024-07-04T08:21:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.861+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b640e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.861+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b634d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.861+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09b41b2420 con 0x56262b26e670 2024-07-04T08:21:11.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09b41b3f60 con 0x56262b28f350 2024-07-04T08:21:11.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09b41b2af0 con 0x56262b266490 2024-07-04T08:21:11.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b634d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b640e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b640e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36194/0 (socket says 172.21.15.39:36194) 2024-07-04T08:21:11.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35886/0 (socket says 172.21.15.39:35886) 2024-07-04T08:21:11.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2a00640 1 -- 172.21.15.39:0/2250494599 learned_addr learned my addr 172.21.15.39:0/2250494599 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:11.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b634d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45476/0 (socket says 172.21.15.39:45476) 2024-07-04T08:21:11.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2a00640 1 -- 172.21.15.39:0/2250494599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 msgr2=0x56262b64ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:11.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2a00640 1 --2- 172.21.15.39:0/2250494599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2a00640 1 -- 172.21.15.39:0/2250494599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 msgr2=0x56262b634d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:11.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2a00640 1 --2- 172.21.15.39:0/2250494599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b634d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b2a00640 1 -- 172.21.15.39:0/2250494599 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262b641950 con 0x56262b266490 2024-07-04T08:21:11.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.862+0000 7f09b3400640 1 --2- 172.21.15.39:0/2250494599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:11.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.863+0000 7f09b2a00640 1 --2- 172.21.15.39:0/2250494599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b640e80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f09a4019040 tx=0x7f09a4009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10e765c74bf8d529 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:11.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.863+0000 7f09a3400640 1 -- 172.21.15.39:0/2250494599 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a401c3a0 con 0x56262b266490 2024-07-04T08:21:11.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.863+0000 7f09a3400640 1 -- 172.21.15.39:0/2250494599 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a403e070 con 0x56262b266490 2024-07-04T08:21:11.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.863+0000 7f09a3400640 1 -- 172.21.15.39:0/2250494599 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4039d00 con 0x56262b266490 2024-07-04T08:21:11.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.864+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2250494599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 msgr2=0x56262b640e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:11.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.864+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2250494599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b640e80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f09a4019040 tx=0x7f09a4009950 comp rx=0 tx=0).stop 2024-07-04T08:21:11.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.864+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2250494599 shutdown_connections 2024-07-04T08:21:11.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.864+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2250494599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b26e670 0x56262b634d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.864+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2250494599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b28f350 0x56262b64ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.864+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2250494599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b640e80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.864+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2250494599 >> 172.21.15.39:0/2250494599 conn(0x56262b283740 msgr2=0x56262b634670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:11.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.864+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2250494599 shutdown_connections 2024-07-04T08:21:11.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.864+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2250494599 wait complete. 2024-07-04T08:21:11.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.865+0000 7f09bfc6e740 1 -- start start 2024-07-04T08:21:11.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.865+0000 7f09bfc6e740 1 Processor -- start 2024-07-04T08:21:11.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b643150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 0x56262b643690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09bfc6e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b28f350 0x56262b6b8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09b41b3890 con 0x56262b28f350 2024-07-04T08:21:11.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09b41b31c0 con 0x56262b26e670 2024-07-04T08:21:11.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09bfc6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09b41b0400 con 0x56262b266490 2024-07-04T08:21:11.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09bfc6e740 10 client.0 _ll_register_callbacks cb 0x7f09a401e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:11.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09b3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b28f350 0x56262b6b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09b2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b643150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09b3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b28f350 0x56262b6b8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45488/0 (socket says 172.21.15.39:45488) 2024-07-04T08:21:11.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09b2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b643150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36198/0 (socket says 172.21.15.39:36198) 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09b3400640 1 -- 172.21.15.39:0/2237560214 learned_addr learned my addr 172.21.15.39:0/2237560214 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.866+0000 7f09b2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 0x56262b643690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.867+0000 7f09b2a00640 1 -- 172.21.15.39:0/2237560214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 msgr2=0x56262b643690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.867+0000 7f09b2a00640 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 0x56262b643690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.867+0000 7f09b2a00640 1 -- 172.21.15.39:0/2237560214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b28f350 msgr2=0x56262b6b8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.867+0000 7f09b2a00640 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b28f350 0x56262b6b8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.867+0000 7f09b2a00640 1 -- 172.21.15.39:0/2237560214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262b6b8e60 con 0x56262b266490 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.867+0000 7f09b2a00640 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b643150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f09a401c890 tx=0x7f09a4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d75a3c776f61df server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.867+0000 7f09a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.868+0000 7f09a0c00640 1 -- 172.21.15.39:0/2237560214 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4047060 con 0x56262b266490 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.868+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2237560214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a403e070 con 0x56262b266490 2024-07-04T08:21:11.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.868+0000 7f09a0c00640 1 -- 172.21.15.39:0/2237560214 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f09a4039430 con 0x56262b266490 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.868+0000 7f09a0c00640 1 -- 172.21.15.39:0/2237560214 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f09a404e050 con 0x56262b266490 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.868+0000 7f09bfc6e740 10 client.26878 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.868+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2237560214 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56262b6b9fc0 con 0x56262b266490 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.869+0000 7f09a0c00640 1 -- 172.21.15.39:0/2237560214 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a403e070 con 0x56262b266490 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.869+0000 7f09a0c00640 1 -- 172.21.15.39:0/2237560214 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f09a40051f0 con 0x56262b266490 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.869+0000 7f09bfc6e740 10 client.26878 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.869+0000 7f09bfc6e740 10 client.26878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.869+0000 7f09bfc6e740 20 client.26878 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.869+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56262b223080 0x56262b6bc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.869+0000 7f09bfc6e740 4 client.26878 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:21:11.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.869+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2237560214 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56262b5a4ff0 con 0x56262b223080 2024-07-04T08:21:11.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.869+0000 7f09b2000640 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56262b223080 0x56262b6bc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:11.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.871+0000 7f09b2000640 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56262b223080 0x56262b6bc6f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1df0d246e35eba2 server_cookie=364b1a70a03df102 in_seq=0 out_seq=0 2024-07-04T08:21:11.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.871+0000 7f09a0c00640 10 client.26878 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.872+0000 7f09a0c00640 1 -- 172.21.15.39:0/2237560214 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56262b5a4ff0 con 0x56262b223080 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.872+0000 7f09a0c00640 10 client.26878 handle_command_reply: tid=0 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.872+0000 7f09bfc6e740 1 client.26878 shutdown 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2237560214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 msgr2=0x56262b643150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b643150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f09a401c890 tx=0x7f09a4017000 comp rx=0 tx=0).stop 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2237560214 shutdown_connections 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56262b223080 0x56262b6bc6f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56262b28f350 0x56262b6b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56262b26e670 0x56262b643690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 --2- 172.21.15.39:0/2237560214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56262b266490 0x56262b643150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09a0c00640 0 client.26878 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:11.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2237560214 >> 172.21.15.39:0/2237560214 conn(0x56262b283740 msgr2=0x56262b634e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:11.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2237560214 shutdown_connections 2024-07-04T08:21:11.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 1 -- 172.21.15.39:0/2237560214 wait complete. 2024-07-04T08:21:11.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.873+0000 7f09bfc6e740 20 client.26878 trim_cache size 0 max 16384 2024-07-04T08:21:11.872 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:21:11.872 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:21:11.872 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "bc05246c-6b9c-4f84-ac46-547224d1f02f", 2024-07-04T08:21:11.872 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:21:11.872 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:21:11.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 -- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09980b29e0 msgr2=0x7f09980b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:11.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09980b29e0 0x7f09980b4ef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x56262b6322c0 tx=0x7f09a4005f90 comp rx=0 tx=0).stop 2024-07-04T08:21:11.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 -- 172.21.15.39:0/3752762227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 msgr2=0x7f09b40756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:11.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 0x7f09b40756e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09ac004af0 tx=0x7f09ac003c20 comp rx=0 tx=0).stop 2024-07-04T08:21:11.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 -- 172.21.15.39:0/3752762227 shutdown_connections 2024-07-04T08:21:11.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09980b29e0 0x7f09980b4ef0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09b4108a00 0x7f09b40751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09b4109350 0x7f09b40756e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 --2- 172.21.15.39:0/3752762227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09b4103110 0x7f09b4074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:11.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.874+0000 7f09b9600640 1 -- 172.21.15.39:0/3752762227 >> 172.21.15.39:0/3752762227 conn(0x7f09b407ae60 msgr2=0x7f09b41069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:11.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.875+0000 7f09b9600640 1 -- 172.21.15.39:0/3752762227 shutdown_connections 2024-07-04T08:21:11.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:11.875+0000 7f09b9600640 1 -- 172.21.15.39:0/3752762227 wait complete. 2024-07-04T08:21:13.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:13 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:21:13.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:13 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:13.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:13 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:21:13.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:13 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:13.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:13 smithi133 ceph-mon[37050]: pgmap v660: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 26 op/s 2024-07-04T08:21:13.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:13 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:21:13.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:13 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:21:13.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:13 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:13.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:13 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:21:13.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:13 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:13.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:13 smithi039 ceph-mon[32062]: pgmap v660: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 26 op/s 2024-07-04T08:21:13.347 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:13 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:21:13.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:13 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:21:13.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:13 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:13.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:13 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:21:13.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:13 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:13.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:13 smithi050 ceph-mon[36577]: pgmap v660: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 26 op/s 2024-07-04T08:21:13.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:13 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:21:13.449 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:21:14.026 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:15.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:15 smithi039 ceph-mon[32062]: pgmap v661: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:21:15.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:15 smithi133 ceph-mon[37050]: pgmap v661: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:21:15.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:15 smithi050 ceph-mon[36577]: pgmap v661: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:21:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.633+0000 7f8d0b200640 1 -- 172.21.15.39:0/763946775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c105600 msgr2=0x7f8d0c105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.633+0000 7f8d0b200640 1 --2- 172.21.15.39:0/763946775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c105600 0x7f8d0c105a10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009900 tx=0x7f8cf802a410 comp rx=0 tx=0).stop 2024-07-04T08:21:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.633+0000 7f8d0b200640 1 -- 172.21.15.39:0/763946775 shutdown_connections 2024-07-04T08:21:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.633+0000 7f8d0b200640 1 --2- 172.21.15.39:0/763946775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c105600 0x7f8d0c105a10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.633+0000 7f8d0b200640 1 --2- 172.21.15.39:0/763946775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d0c105f50 0x7f8d0c10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.633+0000 7f8d0b200640 1 --2- 172.21.15.39:0/763946775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c101320 0x7f8d0c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.633+0000 7f8d0b200640 1 -- 172.21.15.39:0/763946775 >> 172.21.15.39:0/763946775 conn(0x7f8d0c076ff0 msgr2=0x7f8d0c077430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:15.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.633+0000 7f8d0b200640 1 -- 172.21.15.39:0/763946775 shutdown_connections 2024-07-04T08:21:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.634+0000 7f8d0b200640 1 -- 172.21.15.39:0/763946775 wait complete. 2024-07-04T08:21:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.635+0000 7f8d0b200640 1 Processor -- start 2024-07-04T08:21:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.635+0000 7f8d0b200640 1 -- start start 2024-07-04T08:21:15.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d0b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c101320 0x7f8d0c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d0b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d0c105600 0x7f8d0c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d0b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c105f50 0x7f8d0c1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d0b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c112df0 con 0x7f8d0c105f50 2024-07-04T08:21:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d0b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c112c70 con 0x7f8d0c105600 2024-07-04T08:21:15.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d0b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c112f70 con 0x7f8d0c101320 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d09400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d0c105600 0x7f8d0c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d09400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d0c105600 0x7f8d0c19bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35932/0 (socket says 172.21.15.39:35932) 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d09400640 1 -- 172.21.15.39:0/1138380276 learned_addr learned my addr 172.21.15.39:0/1138380276 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d0a800640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c105f50 0x7f8d0c1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.636+0000 7f8d09e00640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c101320 0x7f8d0c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.637+0000 7f8d09e00640 1 -- 172.21.15.39:0/1138380276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d0c105600 msgr2=0x7f8d0c19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.637+0000 7f8d09e00640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d0c105600 0x7f8d0c19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.637+0000 7f8d09e00640 1 -- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c105f50 msgr2=0x7f8d0c1a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.637+0000 7f8d09e00640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c105f50 0x7f8d0c1a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.637+0000 7f8d09e00640 1 -- 172.21.15.39:0/1138380276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c1a1970 con 0x7f8d0c101320 2024-07-04T08:21:15.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.637+0000 7f8d0a800640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c105f50 0x7f8d0c1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.637+0000 7f8d09400640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d0c105600 0x7f8d0c19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.637+0000 7f8d09e00640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c101320 0x7f8d0c19ba00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00e9f0 tx=0x7f8cfc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.638+0000 7f8d02a00640 1 -- 172.21.15.39:0/1138380276 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc00ccc0 con 0x7f8d0c101320 2024-07-04T08:21:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.638+0000 7f8d0b200640 1 -- 172.21.15.39:0/1138380276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c1a1c60 con 0x7f8d0c101320 2024-07-04T08:21:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.638+0000 7f8d02a00640 1 -- 172.21.15.39:0/1138380276 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cfc0108e0 con 0x7f8d0c101320 2024-07-04T08:21:15.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.638+0000 7f8d02a00640 1 -- 172.21.15.39:0/1138380276 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc018880 con 0x7f8d0c101320 2024-07-04T08:21:15.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.638+0000 7f8d0b200640 1 -- 172.21.15.39:0/1138380276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c1a83c0 con 0x7f8d0c101320 2024-07-04T08:21:15.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.642+0000 7f8d171f1740 1 Processor -- start 2024-07-04T08:21:15.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.643+0000 7f8d171f1740 1 -- start start 2024-07-04T08:21:15.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.643+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00292490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.643+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffec4660 0x55cf002929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.643+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffecc840 0x55cf00292f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.643+0000 7f8d171f1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cf001acb20 con 0x55ceffecc840 2024-07-04T08:21:15.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.643+0000 7f8d171f1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cf002082b0 con 0x55ceffec4660 2024-07-04T08:21:15.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.643+0000 7f8d171f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf001f7850 con 0x55ceffeed520 2024-07-04T08:21:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00292490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00292490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36220/0 (socket says 172.21.15.39:36220) 2024-07-04T08:21:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d0a800640 1 -- 172.21.15.39:0/1755426448 learned_addr learned my addr 172.21.15.39:0/1755426448 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:15.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d09400640 1 --2- 172.21.15.39:0/1755426448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffecc840 0x55cf00292f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d0a800640 1 -- 172.21.15.39:0/1755426448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffec4660 msgr2=0x55cf002929d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:21:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d0a800640 1 --2- 172.21.15.39:0/1755426448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffec4660 0x55cf002929d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d0a800640 1 -- 172.21.15.39:0/1755426448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffecc840 msgr2=0x55cf00292f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d0a800640 1 --2- 172.21.15.39:0/1755426448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffecc840 0x55cf00292f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d0a800640 1 -- 172.21.15.39:0/1755426448 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf001b1780 con 0x55ceffeed520 2024-07-04T08:21:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d09400640 1 --2- 172.21.15.39:0/1755426448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffecc840 0x55cf00292f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.644+0000 7f8d02a00640 1 -- 172.21.15.39:0/1138380276 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8cfc010430 con 0x7f8d0c101320 2024-07-04T08:21:15.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.645+0000 7f8d0a800640 1 --2- 172.21.15.39:0/1755426448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00292490 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8d040117e0 tx=0x7f8d04011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b57244edec7464f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.645+0000 7f8cf7400640 1 -- 172.21.15.39:0/1755426448 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04011310 con 0x55ceffeed520 2024-07-04T08:21:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.645+0000 7f8cf7400640 1 -- 172.21.15.39:0/1755426448 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d04003e20 con 0x55ceffeed520 2024-07-04T08:21:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.645+0000 7f8cf7400640 1 -- 172.21.15.39:0/1755426448 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040044f0 con 0x55ceffeed520 2024-07-04T08:21:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.646+0000 7f8d171f1740 1 -- 172.21.15.39:0/1755426448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 msgr2=0x55cf00292490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.646+0000 7f8d02a00640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8cf00b29e0 0x7f8cf00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.646+0000 7f8d171f1740 1 --2- 172.21.15.39:0/1755426448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00292490 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8d040117e0 tx=0x7f8d04011ca0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.646+0000 7f8d171f1740 1 -- 172.21.15.39:0/1755426448 shutdown_connections 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.646+0000 7f8d171f1740 1 --2- 172.21.15.39:0/1755426448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffecc840 0x55cf00292f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.646+0000 7f8d171f1740 1 --2- 172.21.15.39:0/1755426448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffec4660 0x55cf002929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.646+0000 7f8d171f1740 1 --2- 172.21.15.39:0/1755426448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00292490 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.646+0000 7f8d171f1740 1 -- 172.21.15.39:0/1755426448 >> 172.21.15.39:0/1755426448 conn(0x55ceffee1910 msgr2=0x55cf00290380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.647+0000 7f8d09e00640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8cf00b29e0 0x7f8cf00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.647+0000 7f8d171f1740 1 -- 172.21.15.39:0/1755426448 shutdown_connections 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.647+0000 7f8d171f1740 1 -- 172.21.15.39:0/1755426448 wait complete. 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.647+0000 7f8d02a00640 1 -- 172.21.15.39:0/1138380276 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8cfc0e2c60 con 0x7f8d0c101320 2024-07-04T08:21:15.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.647+0000 7f8d171f1740 1 -- start start 2024-07-04T08:21:15.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.648+0000 7f8d171f1740 1 Processor -- start 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.648+0000 7f8d09e00640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8cf00b29e0 0x7f8cf00b4ef0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55cf00291110 tx=0x7f8cfc009700 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.648+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf0029db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.648+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf0029e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.648+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf002a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.648+0000 7f8d171f1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cf0019ed20 con 0x55ceffec4660 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.648+0000 7f8d171f1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cf001fb890 con 0x55ceffecc840 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.648+0000 7f8d171f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cf0005bbb0 con 0x55ceffeed520 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.648+0000 7f8d171f1740 10 client.0 _ll_register_callbacks cb 0x55cf0019f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf0029db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d09400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf0029e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf0029db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45524/0 (socket says 172.21.15.39:45524) 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d0a800640 1 -- 172.21.15.39:0/2260647736 learned_addr learned my addr 172.21.15.39:0/2260647736 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:15.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d0a800640 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf002a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d09400640 1 -- 172.21.15.39:0/2260647736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 msgr2=0x55cf002a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:15.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d09400640 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf002a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d09400640 1 -- 172.21.15.39:0/2260647736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 msgr2=0x55cf0029db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d09400640 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf0029db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.649+0000 7f8d09400640 1 -- 172.21.15.39:0/2260647736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf0018e660 con 0x55ceffecc840 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.650+0000 7f8d09400640 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf0029e0d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8031440 tx=0x7f8cf80316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df0e8c44ec12cd9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.650+0000 7f8cebe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.650+0000 7f8cebe00640 1 -- 172.21.15.39:0/2260647736 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8035040 con 0x55ceffecc840 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.650+0000 7f8d171f1740 1 -- 172.21.15.39:0/2260647736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d04003e20 con 0x55ceffecc840 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.650+0000 7f8cebe00640 1 -- 172.21.15.39:0/2260647736 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8cf8004240 con 0x55ceffecc840 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.651+0000 7f8cebe00640 1 -- 172.21.15.39:0/2260647736 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8cf8004440 con 0x55ceffecc840 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.651+0000 7f8d171f1740 10 client.26981 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:15.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.651+0000 7f8d171f1740 1 -- 172.21.15.39:0/2260647736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cefff58b00 con 0x55ceffecc840 2024-07-04T08:21:15.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.651+0000 7f8cebe00640 1 -- 172.21.15.39:0/2260647736 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d04003e20 con 0x55ceffecc840 2024-07-04T08:21:15.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.652+0000 7f8cebe00640 1 -- 172.21.15.39:0/2260647736 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8cf8004c70 con 0x55ceffecc840 2024-07-04T08:21:15.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.652+0000 7f8d171f1740 10 client.26981 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:15.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.652+0000 7f8d171f1740 10 client.26981 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:21:15.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.652+0000 7f8d171f1740 20 client.26981 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:15.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.652+0000 7f8d171f1740 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ceffeabea0 0x55cf002afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.652+0000 7f8d171f1740 4 client.26981 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:15.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.652+0000 7f8d171f1740 1 -- 172.21.15.39:0/2260647736 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf0020a200 con 0x55ceffeabea0 2024-07-04T08:21:15.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.652+0000 7f8d0a800640 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ceffeabea0 0x55cf002afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.653+0000 7f8d0a800640 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ceffeabea0 0x55cf002afb70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642151877e5b969f server_cookie=9b9668b6f1125509 in_seq=0 out_seq=0 2024-07-04T08:21:15.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.653+0000 7f8cebe00640 10 client.26981 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:15.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.653+0000 7f8cebe00640 1 -- 172.21.15.39:0/2260647736 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cf0020a200 con 0x55ceffeabea0 2024-07-04T08:21:15.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.653+0000 7f8cebe00640 10 client.26981 handle_command_reply: tid=0 2024-07-04T08:21:15.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 client.26981 shutdown 2024-07-04T08:21:15.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 -- 172.21.15.39:0/2260647736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 msgr2=0x55cf0029e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:15.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf0029e0d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8031440 tx=0x7f8cf80316d0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 -- 172.21.15.39:0/2260647736 shutdown_connections 2024-07-04T08:21:15.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ceffeabea0 0x55cf002afb70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf002a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf0029e0d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 --2- 172.21.15.39:0/2260647736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf0029db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 -- 172.21.15.39:0/2260647736 >> 172.21.15.39:0/2260647736 conn(0x55ceffee1910 msgr2=0x55cf00295620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:15.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8cebe00640 0 client.26981 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:15.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 -- 172.21.15.39:0/2260647736 shutdown_connections 2024-07-04T08:21:15.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 1 -- 172.21.15.39:0/2260647736 wait complete. 2024-07-04T08:21:15.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.654+0000 7f8d171f1740 20 client.26981 trim_cache size 0 max 16384 2024-07-04T08:21:15.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.674+0000 7f8d171f1740 1 Processor -- start 2024-07-04T08:21:15.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.674+0000 7f8d171f1740 1 -- start start 2024-07-04T08:21:15.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.671 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffecc840 0x55cf0029fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffeed520 0x55cf002a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d171f1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1b23e0 con 0x55ceffec4660 2024-07-04T08:21:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d171f1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1b3f20 con 0x55ceffeed520 2024-07-04T08:21:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d171f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1b2ab0 con 0x55ceffecc840 2024-07-04T08:21:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45530/0 (socket says 172.21.15.39:45530) 2024-07-04T08:21:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d09400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffecc840 0x55cf0029fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d0a800640 1 -- 172.21.15.39:0/1327346496 learned_addr learned my addr 172.21.15.39:0/1327346496 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.675+0000 7f8d0a800640 1 --2- 172.21.15.39:0/1327346496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffeed520 0x55cf002a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.676+0000 7f8d09400640 1 -- 172.21.15.39:0/1327346496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffeed520 msgr2=0x55cf002a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.676+0000 7f8d09400640 1 --2- 172.21.15.39:0/1327346496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffeed520 0x55cf002a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.676+0000 7f8d09400640 1 -- 172.21.15.39:0/1327346496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 msgr2=0x55cf002993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.676+0000 7f8d09400640 1 --2- 172.21.15.39:0/1327346496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.676+0000 7f8d09400640 1 -- 172.21.15.39:0/1327346496 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cefffcf960 con 0x55ceffecc840 2024-07-04T08:21:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.676+0000 7f8d09400640 1 --2- 172.21.15.39:0/1327346496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffecc840 0x55cf0029fa70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8cf802a930 tx=0x7f8cf8031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaafc2e9c78e9044 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.677+0000 7f8cf6a00640 1 -- 172.21.15.39:0/1327346496 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8058070 con 0x55ceffecc840 2024-07-04T08:21:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.677+0000 7f8cf6a00640 1 -- 172.21.15.39:0/1327346496 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cf8003bd0 con 0x55ceffecc840 2024-07-04T08:21:15.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.677+0000 7f8cf6a00640 1 -- 172.21.15.39:0/1327346496 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8035070 con 0x55ceffecc840 2024-07-04T08:21:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.678+0000 7f8d171f1740 1 -- 172.21.15.39:0/1327346496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffecc840 msgr2=0x55cf0029fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.678+0000 7f8d171f1740 1 --2- 172.21.15.39:0/1327346496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffecc840 0x55cf0029fa70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8cf802a930 tx=0x7f8cf8031a50 comp rx=0 tx=0).stop 2024-07-04T08:21:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.678+0000 7f8d171f1740 1 -- 172.21.15.39:0/1327346496 shutdown_connections 2024-07-04T08:21:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.678+0000 7f8d171f1740 1 --2- 172.21.15.39:0/1327346496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffeed520 0x55cf002a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.678+0000 7f8d171f1740 1 --2- 172.21.15.39:0/1327346496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffecc840 0x55cf0029fa70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.678+0000 7f8d171f1740 1 --2- 172.21.15.39:0/1327346496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.678+0000 7f8d171f1740 1 -- 172.21.15.39:0/1327346496 >> 172.21.15.39:0/1327346496 conn(0x55ceffee1910 msgr2=0x55cf002934c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.678+0000 7f8d171f1740 1 -- 172.21.15.39:0/1327346496 shutdown_connections 2024-07-04T08:21:15.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.678+0000 7f8d171f1740 1 -- 172.21.15.39:0/1327346496 wait complete. 2024-07-04T08:21:15.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.679+0000 7f8d171f1740 1 -- start start 2024-07-04T08:21:15.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.679+0000 7f8d171f1740 1 Processor -- start 2024-07-04T08:21:15.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.680+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.680+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf00298720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.680+0000 7f8d171f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00298e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.680+0000 7f8d171f1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1b3850 con 0x55ceffec4660 2024-07-04T08:21:15.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.680+0000 7f8d171f1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1b3180 con 0x55ceffecc840 2024-07-04T08:21:15.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.680+0000 7f8d171f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1b03c0 con 0x55ceffeed520 2024-07-04T08:21:15.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.680+0000 7f8d171f1740 10 client.0 _ll_register_callbacks cb 0x7f8cf8005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:15.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.680+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d09400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf00298720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d09400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf00298720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35956/0 (socket says 172.21.15.39:35956) 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002a6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45534/0 (socket says 172.21.15.39:45534) 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d09400640 1 -- 172.21.15.39:0/945856072 learned_addr learned my addr 172.21.15.39:0/945856072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d0a800640 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00298e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d09400640 1 -- 172.21.15.39:0/945856072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 msgr2=0x55cf00298e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d09400640 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00298e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d09400640 1 -- 172.21.15.39:0/945856072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 msgr2=0x55cf002a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d09400640 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d09400640 1 -- 172.21.15.39:0/945856072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf001c48a0 con 0x55ceffecc840 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d0a800640 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:15.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8d09400640 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf00298720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8056040 tx=0x7f8cf8005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763f1f27fb323b19 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:15.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.681+0000 7f8cebe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:21:15.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.682+0000 7f8cebe00640 1 -- 172.21.15.39:0/945856072 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8035050 con 0x55ceffecc840 2024-07-04T08:21:15.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.682+0000 7f8d171f1740 1 -- 172.21.15.39:0/945856072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf8003bd0 con 0x55ceffecc840 2024-07-04T08:21:15.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.682+0000 7f8cebe00640 1 -- 172.21.15.39:0/945856072 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8cf8005250 con 0x55ceffecc840 2024-07-04T08:21:15.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.682+0000 7f8cebe00640 1 -- 172.21.15.39:0/945856072 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8cf8058070 con 0x55ceffecc840 2024-07-04T08:21:15.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.682+0000 7f8d171f1740 10 client.26987 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:15.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.682+0000 7f8d171f1740 1 -- 172.21.15.39:0/945856072 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf001c0450 con 0x55ceffecc840 2024-07-04T08:21:15.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.683+0000 7f8cebe00640 1 -- 172.21.15.39:0/945856072 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cf8003bd0 con 0x55ceffecc840 2024-07-04T08:21:15.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.683+0000 7f8cebe00640 1 -- 172.21.15.39:0/945856072 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8cf803c440 con 0x55ceffecc840 2024-07-04T08:21:15.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.683+0000 7f8d171f1740 10 client.26987 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:15.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.683+0000 7f8d171f1740 10 client.26987 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:21:15.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.683+0000 7f8d171f1740 20 client.26987 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:15.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.683+0000 7f8d171f1740 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ceffe81250 0x55cf0031a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:15.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.683+0000 7f8d171f1740 4 client.26987 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "899c97a4", "release": true, "await": true}] 2024-07-04T08:21:15.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.683+0000 7f8d171f1740 1 -- 172.21.15.39:0/945856072 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "899c97a4", "release": true, "await": true}) -- 0x55cf00205ca0 con 0x55ceffe81250 2024-07-04T08:21:15.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.684+0000 7f8d0a800640 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ceffe81250 0x55cf0031a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:15.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.684+0000 7f8d0a800640 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ceffe81250 0x55cf0031a620 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b34c1de3c8bc41 server_cookie=73bda5c380a6b3fa in_seq=0 out_seq=0 2024-07-04T08:21:15.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.684+0000 7f8cebe00640 10 client.26987 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:15.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.686+0000 7f8cebe00640 1 -- 172.21.15.39:0/945856072 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cf00205ca0 con 0x55ceffe81250 2024-07-04T08:21:15.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.686+0000 7f8cebe00640 10 client.26987 handle_command_reply: tid=0 2024-07-04T08:21:15.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.686+0000 7f8d171f1740 1 client.26987 shutdown 2024-07-04T08:21:15.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.686+0000 7f8d171f1740 1 -- 172.21.15.39:0/945856072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 msgr2=0x55cf00298720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.686+0000 7f8d171f1740 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf00298720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8056040 tx=0x7f8cf8005990 comp rx=0 tx=0).stop 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8d171f1740 1 -- 172.21.15.39:0/945856072 shutdown_connections 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8d171f1740 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ceffe81250 0x55cf0031a620 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8d171f1740 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ceffeed520 0x55cf00298e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8d171f1740 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ceffecc840 0x55cf00298720 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8d171f1740 1 --2- 172.21.15.39:0/945856072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ceffec4660 0x55cf002a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8cebe00640 0 client.26987 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8d171f1740 1 -- 172.21.15.39:0/945856072 >> 172.21.15.39:0/945856072 conn(0x55ceffee1910 msgr2=0x55cf00295590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8d171f1740 1 -- 172.21.15.39:0/945856072 shutdown_connections 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8d171f1740 1 -- 172.21.15.39:0/945856072 wait complete. 2024-07-04T08:21:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.687+0000 7f8d171f1740 20 client.26987 trim_cache size 0 max 16384 2024-07-04T08:21:15.685 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:21:15.685 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:21:15.685 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:21:15.685 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 16, 2024-07-04T08:21:15.685 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:21:15.685 INFO:teuthology.orchestra.run.smithi039.stdout: "899c97a4": { 2024-07-04T08:21:15.685 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 16, 2024-07-04T08:21:15.685 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:21:15.685 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 143.9, 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:15.686 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:15.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:15.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:15.687 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:15.687 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:21:15.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 -- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8cf00b29e0 msgr2=0x7f8cf00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:15.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8cf00b29e0 0x7f8cf00b4ef0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55cf00291110 tx=0x7f8cfc009700 comp rx=0 tx=0).stop 2024-07-04T08:21:15.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 -- 172.21.15.39:0/1138380276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c101320 msgr2=0x7f8d0c19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c101320 0x7f8d0c19ba00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00e9f0 tx=0x7f8cfc00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 -- 172.21.15.39:0/1138380276 shutdown_connections 2024-07-04T08:21:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8cf00b29e0 0x7f8cf00b4ef0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d0c105f50 0x7f8d0c1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d0c105600 0x7f8d0c19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 --2- 172.21.15.39:0/1138380276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c101320 0x7f8d0c19ba00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 -- 172.21.15.39:0/1138380276 >> 172.21.15.39:0/1138380276 conn(0x7f8d0c076ff0 msgr2=0x7f8d0c106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.688+0000 7f8d0b200640 1 -- 172.21.15.39:0/1138380276 shutdown_connections 2024-07-04T08:21:15.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:15.689+0000 7f8d0b200640 1 -- 172.21.15.39:0/1138380276 wait complete. 2024-07-04T08:21:16.788 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '899c97a4', total seconds elapsed: 35.6 2024-07-04T08:21:16.788 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.95 2024-07-04T08:21:16.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:21:17.179 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:17 smithi039 ceph-mon[32062]: pgmap v662: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:21:17.390 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:17 smithi050 ceph-mon[36577]: pgmap v662: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:21:17.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.396+0000 7f8a26200640 1 -- 172.21.15.39:0/2057556802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a200777e0 msgr2=0x7f8a20113750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:17.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.396+0000 7f8a26200640 1 --2- 172.21.15.39:0/2057556802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a200777e0 0x7f8a20113750 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009860 tx=0x7f8a0802a340 comp rx=0 tx=0).stop 2024-07-04T08:21:17.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.397+0000 7f8a26200640 1 -- 172.21.15.39:0/2057556802 shutdown_connections 2024-07-04T08:21:17.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.397+0000 7f8a26200640 1 --2- 172.21.15.39:0/2057556802 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a20113c90 0x7f8a201160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.397+0000 7f8a26200640 1 --2- 172.21.15.39:0/2057556802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a200777e0 0x7f8a20113750 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.397+0000 7f8a26200640 1 --2- 172.21.15.39:0/2057556802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a20076e90 0x7f8a200772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.397+0000 7f8a26200640 1 -- 172.21.15.39:0/2057556802 >> 172.21.15.39:0/2057556802 conn(0x7f8a20100d60 msgr2=0x7f8a201031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:17.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.397+0000 7f8a26200640 1 -- 172.21.15.39:0/2057556802 shutdown_connections 2024-07-04T08:21:17.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.397+0000 7f8a26200640 1 -- 172.21.15.39:0/2057556802 wait complete. 2024-07-04T08:21:17.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.398+0000 7f8a26200640 1 Processor -- start 2024-07-04T08:21:17.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.398+0000 7f8a26200640 1 -- start start 2024-07-04T08:21:17.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a26200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a20076e90 0x7f8a201a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a26200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a200777e0 0x7f8a201a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a26200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a20113c90 0x7f8a201a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a26200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201195c0 con 0x7f8a20076e90 2024-07-04T08:21:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a26200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20119440 con 0x7f8a20113c90 2024-07-04T08:21:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a26200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20119740 con 0x7f8a200777e0 2024-07-04T08:21:17.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a20076e90 0x7f8a201a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a200777e0 0x7f8a201a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a1f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a20076e90 0x7f8a201a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45564/0 (socket says 172.21.15.39:45564) 2024-07-04T08:21:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a1f400640 1 -- 172.21.15.39:0/3862424458 learned_addr learned my addr 172.21.15.39:0/3862424458 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.399+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a20113c90 0x7f8a201a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.400+0000 7f8a17e00640 1 -- 172.21.15.39:0/3862424458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a20113c90 msgr2=0x7f8a201a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.400+0000 7f8a17e00640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a20113c90 0x7f8a201a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.400+0000 7f8a17e00640 1 -- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a20076e90 msgr2=0x7f8a201a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:17.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.400+0000 7f8a17e00640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a20076e90 0x7f8a201a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.400+0000 7f8a17e00640 1 -- 172.21.15.39:0/3862424458 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a201a5fa0 con 0x7f8a200777e0 2024-07-04T08:21:17.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.400+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a20113c90 0x7f8a201a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:17.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.400+0000 7f8a1f400640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a20076e90 0x7f8a201a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:17.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.400+0000 7f8a17e00640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a200777e0 0x7f8a201a05d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802a860 tx=0x7f8a08032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:17.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.400+0000 7f8a1cc00640 1 -- 172.21.15.39:0/3862424458 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08040020 con 0x7f8a200777e0 2024-07-04T08:21:17.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.401+0000 7f8a26200640 1 -- 172.21.15.39:0/3862424458 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a201a6230 con 0x7f8a200777e0 2024-07-04T08:21:17.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.401+0000 7f8a1cc00640 1 -- 172.21.15.39:0/3862424458 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a08030e10 con 0x7f8a200777e0 2024-07-04T08:21:17.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.401+0000 7f8a1cc00640 1 -- 172.21.15.39:0/3862424458 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08037040 con 0x7f8a200777e0 2024-07-04T08:21:17.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.401+0000 7f8a26200640 1 -- 172.21.15.39:0/3862424458 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a201ac9f0 con 0x7f8a200777e0 2024-07-04T08:21:17.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.405+0000 7f8a1cc00640 1 -- 172.21.15.39:0/3862424458 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8a08043020 con 0x7f8a200777e0 2024-07-04T08:21:17.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.406+0000 7f8a2c812740 1 Processor -- start 2024-07-04T08:21:17.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.406+0000 7f8a2c812740 1 -- start start 2024-07-04T08:21:17.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.406+0000 7f8a1cc00640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8a040b2960 0x7f8a040b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a1cc00640 1 -- 172.21.15.39:0/3862424458 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8a08102420 con 0x7f8a200777e0 2024-07-04T08:21:17.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd75070 0x555ade119770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd4c1b0 0x555ade119cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 0x555ade11a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a2c812740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555ade08fea0 con 0x555addd4c1b0 2024-07-04T08:21:17.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a2c812740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555ade07f440 con 0x555addd54390 2024-07-04T08:21:17.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a2c812740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555ade083480 con 0x555addd75070 2024-07-04T08:21:17.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8a040b2960 0x7f8a040b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd4c1b0 0x555ade119cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a1f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd75070 0x555ade119770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd4c1b0 0x555ade119cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45578/0 (socket says 172.21.15.39:45578) 2024-07-04T08:21:17.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a1f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd75070 0x555ade119770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36276/0 (socket says 172.21.15.39:36276) 2024-07-04T08:21:17.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.407+0000 7f8a17e00640 1 -- 172.21.15.39:0/1120112212 learned_addr learned my addr 172.21.15.39:0/1120112212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:17.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.408+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/1120112212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 0x555ade11a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.408+0000 7f8a17e00640 1 -- 172.21.15.39:0/1120112212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd75070 msgr2=0x555ade119770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:17.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.408+0000 7f8a17e00640 1 --2- 172.21.15.39:0/1120112212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd75070 0x555ade119770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.408+0000 7f8a17e00640 1 -- 172.21.15.39:0/1120112212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 msgr2=0x555ade11a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:17.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.408+0000 7f8a17e00640 1 --2- 172.21.15.39:0/1120112212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 0x555ade11a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.408+0000 7f8a17e00640 1 -- 172.21.15.39:0/1120112212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ade017610 con 0x555addd4c1b0 2024-07-04T08:21:17.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.408+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/1120112212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 0x555ade11a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:17.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.408+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8a040b2960 0x7f8a040b4e70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009fd0 tx=0x7f8a1800ef30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:17.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.409+0000 7f8a17e00640 1 --2- 172.21.15.39:0/1120112212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd4c1b0 0x555ade119cb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a080c5500 tx=0x7f8a08005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79849ff57e32a25b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:17.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.409+0000 7f8a14c00640 1 -- 172.21.15.39:0/1120112212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080c9480 con 0x555addd4c1b0 2024-07-04T08:21:17.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.409+0000 7f8a14c00640 1 -- 172.21.15.39:0/1120112212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080cd830 con 0x555addd4c1b0 2024-07-04T08:21:17.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.409+0000 7f8a14c00640 1 -- 172.21.15.39:0/1120112212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080ffbf0 con 0x555addd4c1b0 2024-07-04T08:21:17.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.410+0000 7f8a2c812740 1 -- 172.21.15.39:0/1120112212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd4c1b0 msgr2=0x555ade119cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:17.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.410+0000 7f8a2c812740 1 --2- 172.21.15.39:0/1120112212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd4c1b0 0x555ade119cb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8a080c5500 tx=0x7f8a08005c90 comp rx=0 tx=0).stop 2024-07-04T08:21:17.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.410+0000 7f8a2c812740 1 -- 172.21.15.39:0/1120112212 shutdown_connections 2024-07-04T08:21:17.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.410+0000 7f8a2c812740 1 --2- 172.21.15.39:0/1120112212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd4c1b0 0x555ade119cb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.410+0000 7f8a2c812740 1 --2- 172.21.15.39:0/1120112212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 0x555ade11a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.410+0000 7f8a2c812740 1 --2- 172.21.15.39:0/1120112212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd75070 0x555ade119770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.410+0000 7f8a2c812740 1 -- 172.21.15.39:0/1120112212 >> 172.21.15.39:0/1120112212 conn(0x555addd69460 msgr2=0x555ade117660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:17.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.410+0000 7f8a2c812740 1 -- 172.21.15.39:0/1120112212 shutdown_connections 2024-07-04T08:21:17.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.410+0000 7f8a2c812740 1 -- 172.21.15.39:0/1120112212 wait complete. 2024-07-04T08:21:17.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.411+0000 7f8a2c812740 1 -- start start 2024-07-04T08:21:17.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.411+0000 7f8a2c812740 1 Processor -- start 2024-07-04T08:21:17.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.412+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd4c1b0 0x555ade1252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.412+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 0x555ade125820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.412+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade130f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.412+0000 7f8a2c812740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555addee3700 con 0x555addd75070 2024-07-04T08:21:17.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.412+0000 7f8a2c812740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555ade026b70 con 0x555addd54390 2024-07-04T08:21:17.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.412+0000 7f8a2c812740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555ade127d10 con 0x555addd4c1b0 2024-07-04T08:21:17.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.412+0000 7f8a2c812740 10 client.0 _ll_register_callbacks cb 0x555ade078530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd4c1b0 0x555ade1252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd4c1b0 0x555ade1252e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36282/0 (socket says 172.21.15.39:36282) 2024-07-04T08:21:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1f400640 1 -- 172.21.15.39:0/2859676676 learned_addr learned my addr 172.21.15.39:0/2859676676 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a17e00640 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 0x555ade125820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade130f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1f400640 1 -- 172.21.15.39:0/2859676676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 msgr2=0x555ade125820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:17.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1f400640 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 0x555ade125820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1f400640 1 -- 172.21.15.39:0/2859676676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 msgr2=0x555ade130f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:17.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1f400640 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade130f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1f400640 1 -- 172.21.15.39:0/2859676676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555addde0650 con 0x555addd4c1b0 2024-07-04T08:21:17.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.413+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade130f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:17.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.414+0000 7f8a1f400640 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd4c1b0 0x555ade1252e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8a10012d20 tx=0x7f8a1000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6183bbc69967825 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:17.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.414+0000 7f8a0d600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:21:17.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.414+0000 7f8a0d600640 1 -- 172.21.15.39:0/2859676676 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10004050 con 0x555addd4c1b0 2024-07-04T08:21:17.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.414+0000 7f8a0d600640 1 -- 172.21.15.39:0/2859676676 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8a100041f0 con 0x555addd4c1b0 2024-07-04T08:21:17.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.414+0000 7f8a2c812740 1 -- 172.21.15.39:0/2859676676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a080cd830 con 0x555addd4c1b0 2024-07-04T08:21:17.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.415+0000 7f8a0d600640 1 -- 172.21.15.39:0/2859676676 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8a100043f0 con 0x555addd4c1b0 2024-07-04T08:21:17.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.415+0000 7f8a2c812740 10 client.26908 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:17.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.415+0000 7f8a2c812740 1 -- 172.21.15.39:0/2859676676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ade133ca0 con 0x555addd4c1b0 2024-07-04T08:21:17.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.415+0000 7f8a0d600640 1 -- 172.21.15.39:0/2859676676 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080cd830 con 0x555addd4c1b0 2024-07-04T08:21:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.415+0000 7f8a0d600640 1 -- 172.21.15.39:0/2859676676 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8a100104e0 con 0x555addd4c1b0 2024-07-04T08:21:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.415+0000 7f8a2c812740 10 client.26908 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.416+0000 7f8a2c812740 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.416+0000 7f8a2c812740 20 client.26908 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.416+0000 7f8a2c812740 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555addd339f0 0x555ade137220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.416+0000 7f8a2c812740 4 client.26908 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.416+0000 7f8a2c812740 1 -- 172.21.15.39:0/2859676676 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ade091df0 con 0x555addd339f0 2024-07-04T08:21:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.416+0000 7f8a17e00640 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555addd339f0 0x555ade137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.417+0000 7f8a17e00640 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555addd339f0 0x555ade137220 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=768cfc4a69dccee2 server_cookie=df20de925168c336 in_seq=0 out_seq=0 2024-07-04T08:21:17.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.417+0000 7f8a0d600640 10 client.26908 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:17.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.417+0000 7f8a0d600640 1 -- 172.21.15.39:0/2859676676 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ade091df0 con 0x555addd339f0 2024-07-04T08:21:17.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.417+0000 7f8a0d600640 10 client.26908 handle_command_reply: tid=0 2024-07-04T08:21:17.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.417+0000 7f8a2c812740 1 client.26908 shutdown 2024-07-04T08:21:17.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 -- 172.21.15.39:0/2859676676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd4c1b0 msgr2=0x555ade1252e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:17.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd4c1b0 0x555ade1252e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8a10012d20 tx=0x7f8a1000f790 comp rx=0 tx=0).stop 2024-07-04T08:21:17.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 -- 172.21.15.39:0/2859676676 shutdown_connections 2024-07-04T08:21:17.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555addd339f0 0x555ade137220 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade130f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd54390 0x555ade125820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 --2- 172.21.15.39:0/2859676676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd4c1b0 0x555ade1252e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a0d600640 0 client.26908 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:17.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 -- 172.21.15.39:0/2859676676 >> 172.21.15.39:0/2859676676 conn(0x555addd69460 msgr2=0x555ade11c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:17.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 -- 172.21.15.39:0/2859676676 shutdown_connections 2024-07-04T08:21:17.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 1 -- 172.21.15.39:0/2859676676 wait complete. 2024-07-04T08:21:17.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.418+0000 7f8a2c812740 20 client.26908 trim_cache size 0 max 16384 2024-07-04T08:21:17.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.440+0000 7f8a2c812740 1 Processor -- start 2024-07-04T08:21:17.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.440+0000 7f8a2c812740 1 -- start start 2024-07-04T08:21:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.441+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 0x555ade11e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.441+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade11ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.441+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade13ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.441+0000 7f8a2c812740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201b6790 con 0x555addd75070 2024-07-04T08:21:17.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.441+0000 7f8a2c812740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201b82d0 con 0x555addd4c1b0 2024-07-04T08:21:17.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.441+0000 7f8a2c812740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201b6e60 con 0x555addd54390 2024-07-04T08:21:17.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.441+0000 7f8a1fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade13ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.441+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade11ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a1fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade13ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45592/0 (socket says 172.21.15.39:45592) 2024-07-04T08:21:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade11ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36296/0 (socket says 172.21.15.39:36296) 2024-07-04T08:21:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a1fe00640 1 -- 172.21.15.39:0/209936004 learned_addr learned my addr 172.21.15.39:0/209936004 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a1f400640 1 --2- 172.21.15.39:0/209936004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 0x555ade11e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a1fe00640 1 -- 172.21.15.39:0/209936004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 msgr2=0x555ade11ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/209936004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade11ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a1fe00640 1 -- 172.21.15.39:0/209936004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 msgr2=0x555ade11e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/209936004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 0x555ade11e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a1fe00640 1 -- 172.21.15.39:0/209936004 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ade11f280 con 0x555addd75070 2024-07-04T08:21:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.442+0000 7f8a1f400640 1 --2- 172.21.15.39:0/209936004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 0x555ade11e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.443+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/209936004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade13ca20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800cd00 tx=0x7f8a1801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd733a4f7a28fd1f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.443+0000 7f8a0fe00640 1 -- 172.21.15.39:0/209936004 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18017040 con 0x555addd75070 2024-07-04T08:21:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.443+0000 7f8a0fe00640 1 -- 172.21.15.39:0/209936004 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a18003e20 con 0x555addd75070 2024-07-04T08:21:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.443+0000 7f8a0fe00640 1 -- 172.21.15.39:0/209936004 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1801f430 con 0x555addd75070 2024-07-04T08:21:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.444+0000 7f8a2c812740 1 -- 172.21.15.39:0/209936004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 msgr2=0x555ade13ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.444+0000 7f8a2c812740 1 --2- 172.21.15.39:0/209936004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade13ca20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800cd00 tx=0x7f8a1801c000 comp rx=0 tx=0).stop 2024-07-04T08:21:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.445+0000 7f8a2c812740 1 -- 172.21.15.39:0/209936004 shutdown_connections 2024-07-04T08:21:17.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.445+0000 7f8a2c812740 1 --2- 172.21.15.39:0/209936004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade13ca20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.445+0000 7f8a2c812740 1 --2- 172.21.15.39:0/209936004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade11ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.445+0000 7f8a2c812740 1 --2- 172.21.15.39:0/209936004 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 0x555ade11e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.445+0000 7f8a2c812740 1 -- 172.21.15.39:0/209936004 >> 172.21.15.39:0/209936004 conn(0x555addd69460 msgr2=0x555ade123960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:17.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.445+0000 7f8a2c812740 1 -- 172.21.15.39:0/209936004 shutdown_connections 2024-07-04T08:21:17.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.445+0000 7f8a2c812740 1 -- 172.21.15.39:0/209936004 wait complete. 2024-07-04T08:21:17.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.446+0000 7f8a2c812740 1 -- start start 2024-07-04T08:21:17.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.446+0000 7f8a2c812740 1 Processor -- start 2024-07-04T08:21:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.446+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 0x555ade12d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.446+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade128c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.446+0000 7f8a2c812740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade129370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.446+0000 7f8a2c812740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201b7c00 con 0x555addd75070 2024-07-04T08:21:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.446+0000 7f8a2c812740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201b7530 con 0x555addd4c1b0 2024-07-04T08:21:17.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.446+0000 7f8a2c812740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a201b4770 con 0x555addd54390 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a2c812740 10 client.0 _ll_register_callbacks cb 0x7f8a10009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade128c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade128c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36312/0 (socket says 172.21.15.39:36312) 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a17e00640 1 -- 172.21.15.39:0/386410861 learned_addr learned my addr 172.21.15.39:0/386410861 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a1fe00640 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade129370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a17e00640 1 -- 172.21.15.39:0/386410861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 msgr2=0x555ade12d790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a17e00640 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 0x555ade12d790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a17e00640 1 -- 172.21.15.39:0/386410861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 msgr2=0x555ade129370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a17e00640 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade129370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.447+0000 7f8a17e00640 1 -- 172.21.15.39:0/386410861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ade19e9f0 con 0x555addd54390 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.448+0000 7f8a17e00640 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade128c60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8a080307c0 tx=0x7f8a080c9570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de85af72bb7004fc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:17.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.448+0000 7f8a0d600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:21:17.448 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.448+0000 7f8a0d600640 1 -- 172.21.15.39:0/386410861 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a080d22e0 con 0x555addd54390 2024-07-04T08:21:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.448+0000 7f8a0d600640 1 -- 172.21.15.39:0/386410861 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8a080c5710 con 0x555addd54390 2024-07-04T08:21:17.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.448+0000 7f8a2c812740 1 -- 172.21.15.39:0/386410861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a18003e20 con 0x555addd54390 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.448+0000 7f8a0d600640 1 -- 172.21.15.39:0/386410861 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8a080cd830 con 0x555addd54390 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a2c812740 10 client.26914 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a2c812740 1 -- 172.21.15.39:0/386410861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ade19fb30 con 0x555addd54390 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a0d600640 1 -- 172.21.15.39:0/386410861 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a18003e20 con 0x555addd54390 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a0d600640 1 -- 172.21.15.39:0/386410861 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8a080d0bb0 con 0x555addd54390 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a2c812740 10 client.26914 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a2c812740 10 client.26914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a2c812740 20 client.26914 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a2c812740 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555addd08da0 0x555ade1a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a2c812740 4 client.26914 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.449+0000 7f8a2c812740 1 -- 172.21.15.39:0/386410861 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ade08d890 con 0x555addd08da0 2024-07-04T08:21:17.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.450+0000 7f8a1f400640 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555addd08da0 0x555ade1a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.450+0000 7f8a1f400640 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555addd08da0 0x555ade1a2060 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6bf8308d36c443f server_cookie=cf4fcda443500431 in_seq=0 out_seq=0 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.450+0000 7f8a0d600640 10 client.26914 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a0d600640 1 -- 172.21.15.39:0/386410861 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ade08d890 con 0x555addd08da0 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a0d600640 10 client.26914 handle_command_reply: tid=0 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 client.26914 shutdown 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 -- 172.21.15.39:0/386410861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 msgr2=0x555ade128c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade128c60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8a080307c0 tx=0x7f8a080c9570 comp rx=0 tx=0).stop 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 -- 172.21.15.39:0/386410861 shutdown_connections 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555addd08da0 0x555ade1a2060 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555addd75070 0x555ade129370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555addd54390 0x555ade128c60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 --2- 172.21.15.39:0/386410861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555addd4c1b0 0x555ade12d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 -- 172.21.15.39:0/386410861 >> 172.21.15.39:0/386410861 conn(0x555addd69460 msgr2=0x555ade11a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:17.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a0d600640 0 client.26914 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:17.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.451+0000 7f8a2c812740 1 -- 172.21.15.39:0/386410861 shutdown_connections 2024-07-04T08:21:17.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.452+0000 7f8a2c812740 1 -- 172.21.15.39:0/386410861 wait complete. 2024-07-04T08:21:17.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.452+0000 7f8a2c812740 20 client.26914 trim_cache size 0 max 16384 2024-07-04T08:21:17.452 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:21:17.452 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:21:17.452 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:21:17.452 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:21:17.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 -- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8a040b2960 msgr2=0x7f8a040b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:17.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8a040b2960 0x7f8a040b4e70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009fd0 tx=0x7f8a1800ef30 comp rx=0 tx=0).stop 2024-07-04T08:21:17.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 -- 172.21.15.39:0/3862424458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a200777e0 msgr2=0x7f8a201a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:17.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a200777e0 0x7f8a201a05d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802a860 tx=0x7f8a08032d50 comp rx=0 tx=0).stop 2024-07-04T08:21:17.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 -- 172.21.15.39:0/3862424458 shutdown_connections 2024-07-04T08:21:17.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8a040b2960 0x7f8a040b4e70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a20113c90 0x7f8a201a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a200777e0 0x7f8a201a05d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 --2- 172.21.15.39:0/3862424458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a20076e90 0x7f8a201a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 -- 172.21.15.39:0/3862424458 >> 172.21.15.39:0/3862424458 conn(0x7f8a20100d60 msgr2=0x7f8a20115da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 -- 172.21.15.39:0/3862424458 shutdown_connections 2024-07-04T08:21:17.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:17.453+0000 7f8a26200640 1 -- 172.21.15.39:0/3862424458 wait complete. 2024-07-04T08:21:17.561 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:17 smithi133 ceph-mon[37050]: pgmap v662: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-07-04T08:21:18.552 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:21:18.552 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:21:19.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:19 smithi039 ceph-mon[32062]: pgmap v663: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-07-04T08:21:19.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:19 smithi050 ceph-mon[36577]: pgmap v663: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-07-04T08:21:19.561 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:19 smithi133 ceph-mon[37050]: pgmap v663: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-07-04T08:21:19.694 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:21.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:21 smithi039 ceph-mon[32062]: pgmap v664: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-07-04T08:21:21.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:21 smithi050 ceph-mon[36577]: pgmap v664: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-07-04T08:21:21.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.508+0000 7f31efe00640 1 -- 172.21.15.39:0/3554076540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f01022e0 msgr2=0x7f31f010f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:21.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.508+0000 7f31efe00640 1 --2- 172.21.15.39:0/3554076540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f01022e0 0x7f31f010f000 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009860 tx=0x7f31dc02a3d0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.508+0000 7f31efe00640 1 -- 172.21.15.39:0/3554076540 shutdown_connections 2024-07-04T08:21:21.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.508+0000 7f31efe00640 1 --2- 172.21.15.39:0/3554076540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31f010f540 0x7f31f0111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.508+0000 7f31efe00640 1 --2- 172.21.15.39:0/3554076540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f01022e0 0x7f31f010f000 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.508+0000 7f31efe00640 1 --2- 172.21.15.39:0/3554076540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f0101990 0x7f31f0101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.508+0000 7f31efe00640 1 -- 172.21.15.39:0/3554076540 >> 172.21.15.39:0/3554076540 conn(0x7f31f00fd810 msgr2=0x7f31f00ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:21.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.508+0000 7f31efe00640 1 -- 172.21.15.39:0/3554076540 shutdown_connections 2024-07-04T08:21:21.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.509+0000 7f31efe00640 1 -- 172.21.15.39:0/3554076540 wait complete. 2024-07-04T08:21:21.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.510+0000 7f31efe00640 1 Processor -- start 2024-07-04T08:21:21.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.510+0000 7f31efe00640 1 -- start start 2024-07-04T08:21:21.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.510+0000 7f31efe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f0101990 0x7f31f019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:21.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.510+0000 7f31efe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31f01022e0 0x7f31f019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:21.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.510+0000 7f31efe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f010f540 0x7f31f01a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:21.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.510+0000 7f31efe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f31f0115000 con 0x7f31f010f540 2024-07-04T08:21:21.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.510+0000 7f31efe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f31f0114e80 con 0x7f31f0101990 2024-07-04T08:21:21.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.510+0000 7f31efe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f31f0115180 con 0x7f31f01022e0 2024-07-04T08:21:21.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31eea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f0101990 0x7f31f019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:21.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31eea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f0101990 0x7f31f019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51562/0 (socket says 172.21.15.39:51562) 2024-07-04T08:21:21.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31ef400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f010f540 0x7f31f01a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:21.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31eea00640 1 -- 172.21.15.39:0/1665786707 learned_addr learned my addr 172.21.15.39:0/1665786707 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:21.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31ee000640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31f01022e0 0x7f31f019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:21.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31eea00640 1 -- 172.21.15.39:0/1665786707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31f01022e0 msgr2=0x7f31f019bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:21.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31eea00640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31f01022e0 0x7f31f019bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31eea00640 1 -- 172.21.15.39:0/1665786707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f010f540 msgr2=0x7f31f01a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:21.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31eea00640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f010f540 0x7f31f01a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.511+0000 7f31eea00640 1 -- 172.21.15.39:0/1665786707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f01a1960 con 0x7f31f0101990 2024-07-04T08:21:21.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.512+0000 7f31eea00640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f0101990 0x7f31f019b9f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f31e000e9c0 tx=0x7f31e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:21.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.512+0000 7f31e7400640 1 -- 172.21.15.39:0/1665786707 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e000cde0 con 0x7f31f0101990 2024-07-04T08:21:21.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.512+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f01a1c50 con 0x7f31f0101990 2024-07-04T08:21:21.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.512+0000 7f31e7400640 1 -- 172.21.15.39:0/1665786707 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31e00035f0 con 0x7f31f0101990 2024-07-04T08:21:21.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.512+0000 7f31e7400640 1 -- 172.21.15.39:0/1665786707 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0010690 con 0x7f31f0101990 2024-07-04T08:21:21.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.512+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f01a83b0 con 0x7f31f0101990 2024-07-04T08:21:21.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.514+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f0101da0 con 0x7f31f0101990 2024-07-04T08:21:21.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.516+0000 7f31e7400640 1 -- 172.21.15.39:0/1665786707 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f31e0004000 con 0x7f31f0101990 2024-07-04T08:21:21.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.517+0000 7f31e7400640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f31d40b2960 0x7f31d40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:21.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.518+0000 7f31e7400640 1 -- 172.21.15.39:0/1665786707 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f31e00e2ca0 con 0x7f31f0101990 2024-07-04T08:21:21.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.518+0000 7f31ee000640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f31d40b2960 0x7f31d40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:21.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.519+0000 7f31ee000640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f31d40b2960 0x7f31d40b4e70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f31f0110890 tx=0x7f31dc005c90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:21.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.522+0000 7f31e7400640 1 -- 172.21.15.39:0/1665786707 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f31e0009510 con 0x7f31f0101990 2024-07-04T08:21:21.561 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:21 smithi133 ceph-mon[37050]: pgmap v664: 129 pgs: 129 active+clean; 6.3 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-07-04T08:21:21.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.786+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31f01a86f0 con 0x7f31f0101990 2024-07-04T08:21:21.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.787+0000 7f31e7400640 1 -- 172.21.15.39:0/1665786707 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f31e00aa2f0 con 0x7f31f0101990 2024-07-04T08:21:21.783 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:21:21.784 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:21:21.784 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:21:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f31d40b2960 msgr2=0x7f31d40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f31d40b2960 0x7f31d40b4e70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f31f0110890 tx=0x7f31dc005c90 comp rx=0 tx=0).stop 2024-07-04T08:21:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f0101990 msgr2=0x7f31f019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f0101990 0x7f31f019b9f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f31e000e9c0 tx=0x7f31e000ee80 comp rx=0 tx=0).stop 2024-07-04T08:21:21.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 shutdown_connections 2024-07-04T08:21:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f31d40b2960 0x7f31d40b4e70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f31f010f540 0x7f31f01a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31f01022e0 0x7f31f019bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 --2- 172.21.15.39:0/1665786707 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31f0101990 0x7f31f019b9f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.790+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 >> 172.21.15.39:0/1665786707 conn(0x7f31f00fd810 msgr2=0x7f31f01005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.791+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 shutdown_connections 2024-07-04T08:21:21.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:21.791+0000 7f31efe00640 1 -- 172.21.15.39:0/1665786707 wait complete. 2024-07-04T08:21:22.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.097+0000 7f5ac6c00640 1 -- 172.21.15.39:0/3842751482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 msgr2=0x7f5ac01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.097+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/3842751482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 0x7f5ac01021d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009900 tx=0x7f5ab402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.097+0000 7f5ac6c00640 1 -- 172.21.15.39:0/3842751482 shutdown_connections 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.097+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/3842751482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac0107700 0x7f5ac0107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.097+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/3842751482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ac0108050 0x7f5ac010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.097+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/3842751482 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 0x7f5ac01021d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.097+0000 7f5ac6c00640 1 -- 172.21.15.39:0/3842751482 >> 172.21.15.39:0/3842751482 conn(0x7f5ac00fd810 msgr2=0x7f5ac00ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.097+0000 7f5ac6c00640 1 -- 172.21.15.39:0/3842751482 shutdown_connections 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.098+0000 7f5ac6c00640 1 -- 172.21.15.39:0/3842751482 wait complete. 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.099+0000 7f5ac6c00640 1 Processor -- start 2024-07-04T08:21:22.264 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.099+0000 7f5ac6c00640 1 -- start start 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.100+0000 7f5ac6c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 0x7f5ac01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.100+0000 7f5ac6c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac0107700 0x7f5ac0199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.100+0000 7f5ac6c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ac0108050 0x7f5ac019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.100+0000 7f5ac6c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac0111d60 con 0x7f5ac0107700 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.100+0000 7f5ac6c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac0111be0 con 0x7f5ac0101dc0 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.100+0000 7f5ac6c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac0111ee0 con 0x7f5ac0108050 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.100+0000 7f5ac5800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 0x7f5ac01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.101+0000 7f5ac5800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 0x7f5ac01998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51582/0 (socket says 172.21.15.39:51582) 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.101+0000 7f5ac5800640 1 -- 172.21.15.39:0/2437717263 learned_addr learned my addr 172.21.15.39:0/2437717263 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.101+0000 7f5ac6200640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ac0108050 0x7f5ac019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:22.265 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.101+0000 7f5ac4e00640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac0107700 0x7f5ac0199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.101+0000 7f5ac5800640 1 -- 172.21.15.39:0/2437717263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ac0108050 msgr2=0x7f5ac019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.101+0000 7f5ac5800640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ac0108050 0x7f5ac019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.101+0000 7f5ac5800640 1 -- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac0107700 msgr2=0x7f5ac0199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.101+0000 7f5ac5800640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac0107700 0x7f5ac0199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.101+0000 7f5ac5800640 1 -- 172.21.15.39:0/2437717263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac019f810 con 0x7f5ac0101dc0 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.102+0000 7f5ac5800640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 0x7f5ac01998a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4005ab0 tx=0x7f5ab4003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.102+0000 7f5abe000640 1 -- 172.21.15.39:0/2437717263 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab403f070 con 0x7f5ac0101dc0 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.102+0000 7f5abe000640 1 -- 172.21.15.39:0/2437717263 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab4031d60 con 0x7f5ac0101dc0 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.102+0000 7f5abe000640 1 -- 172.21.15.39:0/2437717263 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4035030 con 0x7f5ac0101dc0 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.102+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac019faa0 con 0x7f5ac0101dc0 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.102+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac01a6260 con 0x7f5ac0101dc0 2024-07-04T08:21:22.266 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.104+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac01021d0 con 0x7f5ac0101dc0 2024-07-04T08:21:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.106+0000 7f5abe000640 1 -- 172.21.15.39:0/2437717263 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5ab4042070 con 0x7f5ac0101dc0 2024-07-04T08:21:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.108+0000 7f5abe000640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5aa80b29e0 0x7f5aa80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.108+0000 7f5abe000640 1 -- 172.21.15.39:0/2437717263 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f5ab4103170 con 0x7f5ac0101dc0 2024-07-04T08:21:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.108+0000 7f5ac4e00640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5aa80b29e0 0x7f5aa80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.109+0000 7f5ac4e00640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5aa80b29e0 0x7f5aa80b4ef0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ac01074c0 tx=0x7f5aac005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:22.267 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.112+0000 7f5abe000640 1 -- 172.21.15.39:0/2437717263 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5ab40ca7c0 con 0x7f5ac0101dc0 2024-07-04T08:21:22.346 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:22 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1665786707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:22.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.373+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ac01a6490 con 0x7f5ac0101dc0 2024-07-04T08:21:22.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.375+0000 7f5abe000640 1 -- 172.21.15.39:0/2437717263 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f5ab40ca0c0 con 0x7f5ac0101dc0 2024-07-04T08:21:22.371 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:21:22.371 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:21:22.371 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:21:22.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5aa80b29e0 msgr2=0x7f5aa80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:22.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5aa80b29e0 0x7f5aa80b4ef0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ac01074c0 tx=0x7f5aac005f90 comp rx=0 tx=0).stop 2024-07-04T08:21:22.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 msgr2=0x7f5ac01998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:22.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 0x7f5ac01998a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4005ab0 tx=0x7f5ab4003c80 comp rx=0 tx=0).stop 2024-07-04T08:21:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 shutdown_connections 2024-07-04T08:21:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5aa80b29e0 0x7f5aa80b4ef0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ac0107700 0x7f5ac0199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ac0108050 0x7f5ac019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 --2- 172.21.15.39:0/2437717263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ac0101dc0 0x7f5ac01998a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.378+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 >> 172.21.15.39:0/2437717263 conn(0x7f5ac00fd810 msgr2=0x7f5ac01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.379+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 shutdown_connections 2024-07-04T08:21:22.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:22.379+0000 7f5ac6c00640 1 -- 172.21.15.39:0/2437717263 wait complete. 2024-07-04T08:21:22.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:22 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1665786707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:22.561 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:22 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1665786707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:23.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:21:23.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:23 smithi050 ceph-mon[36577]: pgmap v665: 129 pgs: 129 active+clean; 6.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-07-04T08:21:23.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:23 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2437717263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:23.561 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:23 smithi133 ceph-mon[37050]: pgmap v665: 129 pgs: 129 active+clean; 6.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-07-04T08:21:23.561 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:23 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2437717263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:23.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:23 smithi039 ceph-mon[32062]: pgmap v665: 129 pgs: 129 active+clean; 6.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-07-04T08:21:23.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:23 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2437717263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:23.691 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 143.95 --await-for 45.0' 2024-07-04T08:21:23.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 143.95 --await-for 45.0 2024-07-04T08:21:23.790 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:24.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:24 smithi039 ceph-mon[32062]: pgmap v666: 129 pgs: 129 active+clean; 6.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-07-04T08:21:24.766 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.766 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.766 INFO:tasks.workunit.client.0.smithi039.stdout:127/600: open fd = 3 2024-07-04T08:21:24.766 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.766 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.766 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.767 INFO:tasks.workunit.client.0.smithi039.stdout:128/600: open fd = 3 2024-07-04T08:21:24.767 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.767 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.767 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.767 INFO:tasks.workunit.client.0.smithi039.stdout:129/600: open fd = 3 2024-07-04T08:21:24.767 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.767 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.767 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.767 INFO:tasks.workunit.client.0.smithi039.stdout:130/600: open fd = 3 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:131/600: open fd = 3 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:132/600: open fd = 3 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.768 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.769 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.769 INFO:tasks.workunit.client.0.smithi039.stdout:133/600: open fd = 3 2024-07-04T08:21:24.769 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.769 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.769 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.769 INFO:tasks.workunit.client.0.smithi039.stdout:134/600: open fd = 3 2024-07-04T08:21:24.769 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.769 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.769 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.770 INFO:tasks.workunit.client.0.smithi039.stdout:135/600: open fd = 3 2024-07-04T08:21:24.770 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.770 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.770 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.770 INFO:tasks.workunit.client.0.smithi039.stdout:136/600: open fd = 3 2024-07-04T08:21:24.770 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.770 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.770 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.771 INFO:tasks.workunit.client.0.smithi039.stdout:137/600: open fd = 3 2024-07-04T08:21:24.771 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.771 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.771 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.771 INFO:tasks.workunit.client.0.smithi039.stdout:138/600: open fd = 3 2024-07-04T08:21:24.771 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.771 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.771 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.771 INFO:tasks.workunit.client.0.smithi039.stdout:139/600: open fd = 3 2024-07-04T08:21:24.772 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.772 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.772 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.772 INFO:tasks.workunit.client.0.smithi039.stdout:140/600: open fd = 3 2024-07-04T08:21:24.772 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.772 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.773 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.773 INFO:tasks.workunit.client.0.smithi039.stdout:141/600: open fd = 3 2024-07-04T08:21:24.773 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.773 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.773 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.773 INFO:tasks.workunit.client.0.smithi039.stdout:142/600: open fd = 3 2024-07-04T08:21:24.773 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:143/600: open fd = 3 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:144/600: open fd = 3 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.774 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:145/600: open fd = 3 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:146/600: open fd = 3 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:147/600: open fd = 3 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.775 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:148/600: open fd = 3 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:149/600: open fd = 3 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:150/600: open fd = 3 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.776 INFO:tasks.workunit.client.0.smithi039.stdout:151/600: open fd = 3 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:152/600: open fd = 3 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:153/600: open fd = 3 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.777 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.778 INFO:tasks.workunit.client.0.smithi039.stdout:154/600: open fd = 3 2024-07-04T08:21:24.778 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.778 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.778 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.778 INFO:tasks.workunit.client.0.smithi039.stdout:155/600: open fd = 3 2024-07-04T08:21:24.778 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.778 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.778 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.779 INFO:tasks.workunit.client.0.smithi039.stdout:156/600: open fd = 3 2024-07-04T08:21:24.779 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.779 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.779 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.779 INFO:tasks.workunit.client.0.smithi039.stdout:157/600: open fd = 3 2024-07-04T08:21:24.779 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.779 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.779 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.780 INFO:tasks.workunit.client.0.smithi039.stdout:158/600: open fd = 3 2024-07-04T08:21:24.780 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.780 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.780 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.780 INFO:tasks.workunit.client.0.smithi039.stdout:159/600: open fd = 3 2024-07-04T08:21:24.780 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.780 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.781 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.781 INFO:tasks.workunit.client.0.smithi039.stdout:160/600: open fd = 3 2024-07-04T08:21:24.781 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.781 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.781 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.781 INFO:tasks.workunit.client.0.smithi039.stdout:161/600: open fd = 3 2024-07-04T08:21:24.781 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.781 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.782 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.782 INFO:tasks.workunit.client.0.smithi039.stdout:162/600: open fd = 3 2024-07-04T08:21:24.782 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.782 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.782 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.782 INFO:tasks.workunit.client.0.smithi039.stdout:163/600: open fd = 3 2024-07-04T08:21:24.782 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.782 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.783 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.783 INFO:tasks.workunit.client.0.smithi039.stdout:164/600: open fd = 3 2024-07-04T08:21:24.783 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.783 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.783 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.783 INFO:tasks.workunit.client.0.smithi039.stdout:165/600: open fd = 3 2024-07-04T08:21:24.785 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.785 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.785 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.785 INFO:tasks.workunit.client.0.smithi039.stdout:166/600: open fd = 3 2024-07-04T08:21:24.785 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.785 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.785 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.785 INFO:tasks.workunit.client.0.smithi039.stdout:167/600: open fd = 3 2024-07-04T08:21:24.785 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:168/600: open fd = 3 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:169/600: open fd = 3 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:170/600: open fd = 3 2024-07-04T08:21:24.786 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:171/600: open fd = 3 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:172/600: open fd = 3 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.787 INFO:tasks.workunit.client.0.smithi039.stdout:173/600: open fd = 3 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:174/600: open fd = 3 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:175/600: open fd = 3 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:176/600: open fd = 3 2024-07-04T08:21:24.788 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:177/600: open fd = 3 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:178/600: open fd = 3 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:179/600: open fd = 3 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.789 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:180/600: open fd = 3 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:181/600: open fd = 3 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:182/600: open fd = 3 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.790 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:183/600: open fd = 3 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:184/600: open fd = 3 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:185/600: open fd = 3 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.791 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:186/600: open fd = 3 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:187/600: open fd = 3 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:188/600: open fd = 3 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.792 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:21:24.811 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:24 smithi133 ceph-mon[37050]: pgmap v666: 129 pgs: 129 active+clean; 6.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-07-04T08:21:24.889 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:24 smithi050 ceph-mon[36577]: pgmap v666: 129 pgs: 129 active+clean; 6.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-07-04T08:21:25.156 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.192+0000 7f90dfc00640 1 -- 172.21.15.39:0/680749209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d8109360 msgr2=0x7f90d810fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.192+0000 7f90dfc00640 1 --2- 172.21.15.39:0/680749209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d8109360 0x7f90d810fb90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009ca0 tx=0x7f90d002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.192+0000 7f90dfc00640 1 -- 172.21.15.39:0/680749209 shutdown_connections 2024-07-04T08:21:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.192+0000 7f90dfc00640 1 --2- 172.21.15.39:0/680749209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d8109360 0x7f90d810fb90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.192+0000 7f90dfc00640 1 --2- 172.21.15.39:0/680749209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90d8108a10 0x7f90d8108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.192+0000 7f90dfc00640 1 --2- 172.21.15.39:0/680749209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90d81030d0 0x7f90d81034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.192+0000 7f90dfc00640 1 -- 172.21.15.39:0/680749209 >> 172.21.15.39:0/680749209 conn(0x7f90d80feb20 msgr2=0x7f90d8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.192+0000 7f90dfc00640 1 -- 172.21.15.39:0/680749209 shutdown_connections 2024-07-04T08:21:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.193+0000 7f90dfc00640 1 -- 172.21.15.39:0/680749209 wait complete. 2024-07-04T08:21:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.194+0000 7f90dfc00640 1 Processor -- start 2024-07-04T08:21:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.194+0000 7f90dfc00640 1 -- start start 2024-07-04T08:21:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.194+0000 7f90dfc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d81030d0 0x7f90d81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.194+0000 7f90dfc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90d8108a10 0x7f90d8199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.194+0000 7f90dfc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90d8109360 0x7f90d819f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.194+0000 7f90dfc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8113030 con 0x7f90d81030d0 2024-07-04T08:21:27.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.194+0000 7f90dfc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8112eb0 con 0x7f90d8109360 2024-07-04T08:21:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.194+0000 7f90dfc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81131b0 con 0x7f90d8108a10 2024-07-04T08:21:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90dd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d81030d0 0x7f90d81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90dd200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d81030d0 0x7f90d81999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41458/0 (socket says 172.21.15.39:41458) 2024-07-04T08:21:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90dd200640 1 -- 172.21.15.39:0/1240364706 learned_addr learned my addr 172.21.15.39:0/1240364706 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:27.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90ddc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90d8109360 0x7f90d819f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90d7e00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90d8108a10 0x7f90d8199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90dd200640 1 -- 172.21.15.39:0/1240364706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90d8108a10 msgr2=0x7f90d8199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90dd200640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90d8108a10 0x7f90d8199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90dd200640 1 -- 172.21.15.39:0/1240364706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90d8109360 msgr2=0x7f90d819f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90dd200640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90d8109360 0x7f90d819f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90dd200640 1 -- 172.21.15.39:0/1240364706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d819f930 con 0x7f90d81030d0 2024-07-04T08:21:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.195+0000 7f90ddc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90d8109360 0x7f90d819f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.196+0000 7f90dd200640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d81030d0 0x7f90d81999c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00e9f0 tx=0x7f90cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:27.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.196+0000 7f90d5600640 1 -- 172.21.15.39:0/1240364706 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc00cd80 con 0x7f90d81030d0 2024-07-04T08:21:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.196+0000 7f90d5600640 1 -- 172.21.15.39:0/1240364706 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90cc0054d0 con 0x7f90d81030d0 2024-07-04T08:21:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.196+0000 7f90d5600640 1 -- 172.21.15.39:0/1240364706 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc010690 con 0x7f90d81030d0 2024-07-04T08:21:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.196+0000 7f90dfc00640 1 -- 172.21.15.39:0/1240364706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d819fc20 con 0x7f90d81030d0 2024-07-04T08:21:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.196+0000 7f90dfc00640 1 -- 172.21.15.39:0/1240364706 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d81a6380 con 0x7f90d81030d0 2024-07-04T08:21:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90e62fe740 1 Processor -- start 2024-07-04T08:21:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90e62fe740 1 -- start start 2024-07-04T08:21:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x557087056bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x557087057100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087057640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90e62fe740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557086fc8ee0 con 0x557086cae070 2024-07-04T08:21:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90e62fe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557086fb8480 con 0x557086c8d390 2024-07-04T08:21:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90e62fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557086fbc4c0 con 0x557086c851b0 2024-07-04T08:21:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90ddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x557087056bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.200+0000 7f90ddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x557087056bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41472/0 (socket says 172.21.15.39:41472) 2024-07-04T08:21:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.201+0000 7f90ddc00640 1 -- 172.21.15.39:0/1284318989 learned_addr learned my addr 172.21.15.39:0/1284318989 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.201+0000 7f90d5600640 1 -- 172.21.15.39:0/1240364706 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f90cc004e90 con 0x7f90d81030d0 2024-07-04T08:21:27.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.201+0000 7f90d7e00640 1 --2- 172.21.15.39:0/1284318989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087057640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.201+0000 7f90d7e00640 1 -- 172.21.15.39:0/1284318989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 msgr2=0x557087057100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.201+0000 7f90d7e00640 1 --2- 172.21.15.39:0/1284318989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x557087057100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.201+0000 7f90d7e00640 1 -- 172.21.15.39:0/1284318989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 msgr2=0x557087056bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.201+0000 7f90d7e00640 1 --2- 172.21.15.39:0/1284318989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x557087056bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.201+0000 7f90d7e00640 1 -- 172.21.15.39:0/1284318989 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557086f50650 con 0x557086c8d390 2024-07-04T08:21:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.201+0000 7f90ddc00640 1 --2- 172.21.15.39:0/1284318989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x557087056bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.202+0000 7f90d7e00640 1 --2- 172.21.15.39:0/1284318989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087057640 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f90d806bcd0 tx=0x7f90c4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa4fedeb4214fc4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:27.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.202+0000 7f90ca000640 1 -- 172.21.15.39:0/1284318989 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4004050 con 0x557086c8d390 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.202+0000 7f90ca000640 1 -- 172.21.15.39:0/1284318989 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c40054d0 con 0x557086c8d390 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.202+0000 7f90ca000640 1 -- 172.21.15.39:0/1284318989 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4004c80 con 0x557086c8d390 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.202+0000 7f90d5600640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90c00b2960 0x7f90c00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90d5600640 1 -- 172.21.15.39:0/1240364706 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f90cc0e2d60 con 0x7f90d81030d0 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90e62fe740 1 -- 172.21.15.39:0/1284318989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 msgr2=0x557087057640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90e62fe740 1 --2- 172.21.15.39:0/1284318989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087057640 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f90d806bcd0 tx=0x7f90c4011810 comp rx=0 tx=0).stop 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90ddc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90c00b2960 0x7f90c00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90e62fe740 1 -- 172.21.15.39:0/1284318989 shutdown_connections 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90e62fe740 1 --2- 172.21.15.39:0/1284318989 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087057640 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90e62fe740 1 --2- 172.21.15.39:0/1284318989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x557087057100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90e62fe740 1 --2- 172.21.15.39:0/1284318989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x557087056bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90e62fe740 1 -- 172.21.15.39:0/1284318989 >> 172.21.15.39:0/1284318989 conn(0x557086ca2460 msgr2=0x557087054ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90e62fe740 1 -- 172.21.15.39:0/1284318989 shutdown_connections 2024-07-04T08:21:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.203+0000 7f90e62fe740 1 -- 172.21.15.39:0/1284318989 wait complete. 2024-07-04T08:21:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.204+0000 7f90ddc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90c00b2960 0x7f90c00b4e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f90d0035000 tx=0x7f90d0032eb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.204+0000 7f90e62fe740 1 -- start start 2024-07-04T08:21:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.204+0000 7f90e62fe740 1 Processor -- start 2024-07-04T08:21:27.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x5570870627f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087062d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90e62fe740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557086e1c700 con 0x557086cae070 2024-07-04T08:21:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90e62fe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557086f5fbb0 con 0x557086c8d390 2024-07-04T08:21:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90e62fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5570870653a0 con 0x557086c851b0 2024-07-04T08:21:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90e62fe740 10 client.0 _ll_register_callbacks cb 0x557086fb1570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90ddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90ddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41482/0 (socket says 172.21.15.39:41482) 2024-07-04T08:21:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90ddc00640 1 -- 172.21.15.39:0/419418766 learned_addr learned my addr 172.21.15.39:0/419418766 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90d7e00640 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087062d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.205+0000 7f90dd200640 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x5570870627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90ddc00640 1 -- 172.21.15.39:0/419418766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 msgr2=0x5570870627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90ddc00640 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x5570870627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90ddc00640 1 -- 172.21.15.39:0/419418766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 msgr2=0x557087062d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90ddc00640 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087062d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90ddc00640 1 -- 172.21.15.39:0/419418766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557086d19650 con 0x557086cae070 2024-07-04T08:21:27.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90d7e00640 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087062d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90ddc00640 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e6f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f90d0031a00 tx=0x7f90d002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a73ec9c57b9241 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90bea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90bea00640 1 -- 172.21.15.39:0/419418766 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0004250 con 0x557086cae070 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90bea00640 1 -- 172.21.15.39:0/419418766 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f90d00043f0 con 0x557086cae070 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.206+0000 7f90e62fe740 1 -- 172.21.15.39:0/419418766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c40054d0 con 0x557086cae070 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90bea00640 1 -- 172.21.15.39:0/419418766 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90d00045f0 con 0x557086cae070 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90e62fe740 10 client.16887 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90e62fe740 1 -- 172.21.15.39:0/419418766 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557087071420 con 0x557086cae070 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90bea00640 1 -- 172.21.15.39:0/419418766 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c40054d0 con 0x557086cae070 2024-07-04T08:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90bea00640 1 -- 172.21.15.39:0/419418766 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f90d0052070 con 0x557086cae070 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90e62fe740 10 client.16887 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90e62fe740 10 client.16887 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90e62fe740 20 client.16887 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90e62fe740 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x557086c6c9f0 0x557087074910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.207+0000 7f90e62fe740 4 client.16887 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.208+0000 7f90e62fe740 1 -- 172.21.15.39:0/419418766 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557086fcae30 con 0x557086c6c9f0 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.208+0000 7f90dd200640 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x557086c6c9f0 0x557087074910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.209+0000 7f90dd200640 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x557086c6c9f0 0x557087074910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f254c8b76f3bb640 server_cookie=293492a9d042a14 in_seq=0 out_seq=0 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.209+0000 7f90bea00640 10 client.16887 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.209+0000 7f90bea00640 1 -- 172.21.15.39:0/419418766 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557086fcae30 con 0x557086c6c9f0 2024-07-04T08:21:27.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.209+0000 7f90bea00640 10 client.16887 handle_command_reply: tid=0 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.209+0000 7f90e62fe740 1 client.16887 shutdown 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 -- 172.21.15.39:0/419418766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 msgr2=0x55708706e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e6f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f90d0031a00 tx=0x7f90d002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 -- 172.21.15.39:0/419418766 shutdown_connections 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x557086c6c9f0 0x557087074910 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e6f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087062d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 --2- 172.21.15.39:0/419418766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x5570870627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90bea00640 0 client.16887 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:21:27.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 -- 172.21.15.39:0/419418766 >> 172.21.15.39:0/419418766 conn(0x557086ca2460 msgr2=0x557087059da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:27.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 -- 172.21.15.39:0/419418766 shutdown_connections 2024-07-04T08:21:27.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 1 -- 172.21.15.39:0/419418766 wait complete. 2024-07-04T08:21:27.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.210+0000 7f90e62fe740 20 client.16887 trim_cache size 0 max 16384 2024-07-04T08:21:27.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.226+0000 7f90e62fe740 1 Processor -- start 2024-07-04T08:21:27.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.226+0000 7f90e62fe740 1 -- start start 2024-07-04T08:21:27.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.226+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x55708704f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.226+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087054ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.226+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.226+0000 7f90e62fe740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81adfe0 con 0x557086cae070 2024-07-04T08:21:27.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.226+0000 7f90e62fe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81afb20 con 0x557086c8d390 2024-07-04T08:21:27.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.226+0000 7f90e62fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81ae6b0 con 0x557086c851b0 2024-07-04T08:21:27.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90ddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90ddc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41484/0 (socket says 172.21.15.39:41484) 2024-07-04T08:21:27.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90ddc00640 1 -- 172.21.15.39:0/2758316720 learned_addr learned my addr 172.21.15.39:0/2758316720 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:27.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90dd200640 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x55708704f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90d7e00640 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087054ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90dd200640 1 -- 172.21.15.39:0/2758316720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 msgr2=0x557087054ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90dd200640 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087054ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90dd200640 1 -- 172.21.15.39:0/2758316720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 msgr2=0x55708706e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90dd200640 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90dd200640 1 -- 172.21.15.39:0/2758316720 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570870638b0 con 0x557086c851b0 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90ddc00640 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.227+0000 7f90d7e00640 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087054ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.228+0000 7f90dd200640 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x55708704f940 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0e0160 tx=0x7f90cc005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aab4bbd5d463a49 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.228+0000 7f90c9600640 1 -- 172.21.15.39:0/2758316720 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0c9070 con 0x557086c851b0 2024-07-04T08:21:27.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.228+0000 7f90c9600640 1 -- 172.21.15.39:0/2758316720 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90cc0a9800 con 0x557086c851b0 2024-07-04T08:21:27.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.228+0000 7f90c9600640 1 -- 172.21.15.39:0/2758316720 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0a5f80 con 0x557086c851b0 2024-07-04T08:21:27.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.229+0000 7f90e62fe740 1 -- 172.21.15.39:0/2758316720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 msgr2=0x55708704f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:27.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.229+0000 7f90e62fe740 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x55708704f940 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0e0160 tx=0x7f90cc005710 comp rx=0 tx=0).stop 2024-07-04T08:21:27.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.229+0000 7f90e62fe740 1 -- 172.21.15.39:0/2758316720 shutdown_connections 2024-07-04T08:21:27.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.229+0000 7f90e62fe740 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086cae070 0x55708706e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.229+0000 7f90e62fe740 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c8d390 0x557087054ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.229+0000 7f90e62fe740 1 --2- 172.21.15.39:0/2758316720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086c851b0 0x55708704f940 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.229+0000 7f90e62fe740 1 -- 172.21.15.39:0/2758316720 >> 172.21.15.39:0/2758316720 conn(0x557086ca2460 msgr2=0x5570870543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:27.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.229+0000 7f90e62fe740 1 -- 172.21.15.39:0/2758316720 shutdown_connections 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.229+0000 7f90e62fe740 1 -- 172.21.15.39:0/2758316720 wait complete. 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.230+0000 7f90e62fe740 1 -- start start 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.230+0000 7f90e62fe740 1 Processor -- start 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c851b0 0x557087079d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086c8d390 0x55708707a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90e62fe740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086cae070 0x55708707a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90e62fe740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81af450 con 0x557086c8d390 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90e62fe740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81aed80 con 0x557086c851b0 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90e62fe740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81abfc0 con 0x557086cae070 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90e62fe740 10 client.0 _ll_register_callbacks cb 0x7f90d0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:27.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90d7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086c8d390 0x55708707a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90dd200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c851b0 0x557087079d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90d7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086c8d390 0x55708707a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41500/0 (socket says 172.21.15.39:41500) 2024-07-04T08:21:27.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90d7e00640 1 -- 172.21.15.39:0/3050149652 learned_addr learned my addr 172.21.15.39:0/3050149652 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:27.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90ddc00640 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086cae070 0x55708707a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90d7e00640 1 -- 172.21.15.39:0/3050149652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086cae070 msgr2=0x55708707a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:27.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90d7e00640 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086cae070 0x55708707a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90d7e00640 1 -- 172.21.15.39:0/3050149652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c851b0 msgr2=0x557087079d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:27.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.231+0000 7f90d7e00640 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c851b0 0x557087079d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.232+0000 7f90d7e00640 1 -- 172.21.15.39:0/3050149652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557087069bd0 con 0x557086c8d390 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.232+0000 7f90dd200640 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c851b0 0x557087079d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.232+0000 7f90d7e00640 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086c8d390 0x55708707a270 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009230 tx=0x7f90c4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ce2efd4da6bfe1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.232+0000 7f90bea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.232+0000 7f90bea00640 1 -- 172.21.15.39:0/3050149652 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c401d040 con 0x557086c8d390 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.232+0000 7f90bea00640 1 -- 172.21.15.39:0/3050149652 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f90c4003a00 con 0x557086c8d390 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.232+0000 7f90e62fe740 1 -- 172.21.15.39:0/3050149652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90cc0a9800 con 0x557086c8d390 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90bea00640 1 -- 172.21.15.39:0/3050149652 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90c4037050 con 0x557086c8d390 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90e62fe740 10 client.16893 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:27.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90e62fe740 1 -- 172.21.15.39:0/3050149652 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5570870dd2a0 con 0x557086c8d390 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90bea00640 1 -- 172.21.15.39:0/3050149652 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90cc0a9800 con 0x557086c8d390 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90bea00640 1 -- 172.21.15.39:0/3050149652 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f90c4024bb0 con 0x557086c8d390 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90e62fe740 10 client.16893 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90e62fe740 10 client.16893 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90e62fe740 20 client.16893 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90e62fe740 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x557086c41da0 0x5570870df8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90e62fe740 4 client.16893 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.233+0000 7f90e62fe740 1 -- 172.21.15.39:0/3050149652 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557086fc68d0 con 0x557086c41da0 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.234+0000 7f90dd200640 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x557086c41da0 0x5570870df8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:27.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.234+0000 7f90bea00640 10 client.16893 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:21:27.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.234+0000 7f90dd200640 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x557086c41da0 0x5570870df8f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89dde9e468d7e29 server_cookie=93b1125b4d9683fd in_seq=0 out_seq=0 2024-07-04T08:21:27.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90bea00640 1 -- 172.21.15.39:0/3050149652 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557086fc68d0 con 0x557086c41da0 2024-07-04T08:21:27.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90bea00640 10 client.16893 handle_command_reply: tid=0 2024-07-04T08:21:27.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 client.16893 shutdown 2024-07-04T08:21:27.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 -- 172.21.15.39:0/3050149652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086c8d390 msgr2=0x55708707a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:27.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086c8d390 0x55708707a270 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009230 tx=0x7f90c4009830 comp rx=0 tx=0).stop 2024-07-04T08:21:27.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 -- 172.21.15.39:0/3050149652 shutdown_connections 2024-07-04T08:21:27.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x557086c41da0 0x5570870df8f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557086c8d390 0x55708707a270 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557086cae070 0x55708707a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 --2- 172.21.15.39:0/3050149652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557086c851b0 0x557087079d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90bea00640 0 client.16893 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:21:27.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 -- 172.21.15.39:0/3050149652 >> 172.21.15.39:0/3050149652 conn(0x557086ca2460 msgr2=0x557087063a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:27.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 -- 172.21.15.39:0/3050149652 shutdown_connections 2024-07-04T08:21:27.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 1 -- 172.21.15.39:0/3050149652 wait complete. 2024-07-04T08:21:27.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.235+0000 7f90e62fe740 20 client.16893 trim_cache size 0 max 16384 2024-07-04T08:21:27.024 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:21:27.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 -- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90c00b2960 msgr2=0x7f90c00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:27.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90c00b2960 0x7f90c00b4e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f90d0035000 tx=0x7f90d0032eb0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 -- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d81030d0 msgr2=0x7f90d81999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:27.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d81030d0 0x7f90d81999c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00e9f0 tx=0x7f90cc00edf0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 -- 172.21.15.39:0/1240364706 shutdown_connections 2024-07-04T08:21:27.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90c00b2960 0x7f90c00b4e70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f90d8109360 0x7f90d819f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90d8108a10 0x7f90d8199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 --2- 172.21.15.39:0/1240364706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f90d81030d0 0x7f90d81999c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:27.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 -- 172.21.15.39:0/1240364706 >> 172.21.15.39:0/1240364706 conn(0x7f90d80feb20 msgr2=0x7f90d8100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:27.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 -- 172.21.15.39:0/1240364706 shutdown_connections 2024-07-04T08:21:27.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:26.236+0000 7f90dfc00640 1 -- 172.21.15.39:0/1240364706 wait complete. 2024-07-04T08:21:27.122 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:26 smithi039 ceph-mon[32062]: pgmap v667: 129 pgs: 129 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:21:27.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:26 smithi133 ceph-mon[37050]: pgmap v667: 129 pgs: 129 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:21:27.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:26 smithi050 ceph-mon[36577]: pgmap v667: 129 pgs: 129 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:21:28.280 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.284+0000 7f3469800640 1 -- 172.21.15.39:0/769528735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3464104850 msgr2=0x7f346410cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:28.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.284+0000 7f3469800640 1 --2- 172.21.15.39:0/769528735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3464104850 0x7f346410cda0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3450009900 tx=0x7f345002a400 comp rx=0 tx=0).stop 2024-07-04T08:21:28.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.285+0000 7f3469800640 1 -- 172.21.15.39:0/769528735 shutdown_connections 2024-07-04T08:21:28.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.285+0000 7f3469800640 1 --2- 172.21.15.39:0/769528735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3464104850 0x7f346410cda0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.285+0000 7f3469800640 1 --2- 172.21.15.39:0/769528735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f346410d360 0x7f346410f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.285+0000 7f3469800640 1 --2- 172.21.15.39:0/769528735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3464103f00 0x7f3464104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.285+0000 7f3469800640 1 -- 172.21.15.39:0/769528735 >> 172.21.15.39:0/769528735 conn(0x7f3464078c20 msgr2=0x7f3464079030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:28.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.285+0000 7f3469800640 1 -- 172.21.15.39:0/769528735 shutdown_connections 2024-07-04T08:21:28.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.285+0000 7f3469800640 1 -- 172.21.15.39:0/769528735 wait complete. 2024-07-04T08:21:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.286+0000 7f3469800640 1 Processor -- start 2024-07-04T08:21:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.286+0000 7f3469800640 1 -- start start 2024-07-04T08:21:28.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.286+0000 7f3469800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3464103f00 0x7f34641999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.286+0000 7f3469800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464104850 0x7f3464199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.287+0000 7f3469800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f346410d360 0x7f346419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.287+0000 7f3469800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3464112f90 con 0x7f3464103f00 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.287+0000 7f3469800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3464112e10 con 0x7f346410d360 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.287+0000 7f3469800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3464113110 con 0x7f3464104850 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.287+0000 7f3462000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464104850 0x7f3464199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.287+0000 7f3463400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f346410d360 0x7f346419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.287+0000 7f3462a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3464103f00 0x7f34641999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.287+0000 7f3462000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464104850 0x7f3464199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43090/0 (socket says 172.21.15.39:43090) 2024-07-04T08:21:28.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.287+0000 7f3462000640 1 -- 172.21.15.39:0/2559710582 learned_addr learned my addr 172.21.15.39:0/2559710582 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.288+0000 7f3462a00640 1 -- 172.21.15.39:0/2559710582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464104850 msgr2=0x7f3464199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.288+0000 7f3462a00640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464104850 0x7f3464199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.288+0000 7f3462a00640 1 -- 172.21.15.39:0/2559710582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f346410d360 msgr2=0x7f346419f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.288+0000 7f3462a00640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f346410d360 0x7f346419f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.288+0000 7f3462a00640 1 -- 172.21.15.39:0/2559710582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346419f910 con 0x7f3464103f00 2024-07-04T08:21:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.288+0000 7f3462a00640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3464103f00 0x7f34641999a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f345400e9f0 tx=0x7f345400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.288+0000 7f345b400640 1 -- 172.21.15.39:0/2559710582 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345400ccc0 con 0x7f3464103f00 2024-07-04T08:21:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.288+0000 7f345b400640 1 -- 172.21.15.39:0/2559710582 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34540108e0 con 0x7f3464103f00 2024-07-04T08:21:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.288+0000 7f345b400640 1 -- 172.21.15.39:0/2559710582 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454018880 con 0x7f3464103f00 2024-07-04T08:21:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.289+0000 7f3469800640 1 -- 172.21.15.39:0/2559710582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346419fc00 con 0x7f3464103f00 2024-07-04T08:21:28.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.289+0000 7f3469800640 1 -- 172.21.15.39:0/2559710582 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34641a6360 con 0x7f3464103f00 2024-07-04T08:21:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.293+0000 7f346fde3740 1 Processor -- start 2024-07-04T08:21:28.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.293+0000 7f346fde3740 1 -- start start 2024-07-04T08:21:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.293+0000 7f345b400640 1 -- 172.21.15.39:0/2559710582 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3454010430 con 0x7f3464103f00 2024-07-04T08:21:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.293+0000 7f346fde3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba912850 0x557dbacb7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.293+0000 7f346fde3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba8e9990 0x557dbacb7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.293+0000 7f346fde3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.293+0000 7f346fde3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557dbac7d140 con 0x557dba8e9990 2024-07-04T08:21:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.293+0000 7f346fde3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557dbab9e380 con 0x557dba8f1b70 2024-07-04T08:21:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.293+0000 7f346fde3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557dbac2cc30 con 0x557dba912850 2024-07-04T08:21:28.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3462a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba8e9990 0x557dbacb7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3463400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba912850 0x557dbacb7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3462000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3463400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba912850 0x557dbacb7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43096/0 (socket says 172.21.15.39:43096) 2024-07-04T08:21:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3462000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51650/0 (socket says 172.21.15.39:51650) 2024-07-04T08:21:28.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3463400640 1 -- 172.21.15.39:0/3473451928 learned_addr learned my addr 172.21.15.39:0/3473451928 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3463400640 1 -- 172.21.15.39:0/3473451928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 msgr2=0x557dbacb8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3463400640 1 --2- 172.21.15.39:0/3473451928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3463400640 1 -- 172.21.15.39:0/3473451928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba8e9990 msgr2=0x557dbacb7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3463400640 1 --2- 172.21.15.39:0/3473451928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba8e9990 0x557dbacb7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.294+0000 7f3463400640 1 -- 172.21.15.39:0/3473451928 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dba98cba0 con 0x557dba912850 2024-07-04T08:21:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.295+0000 7f3463400640 1 --2- 172.21.15.39:0/3473451928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba912850 0x557dbacb7780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f345c010370 tx=0x7f345c010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a43cae2086a7ac8d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.295+0000 7f344fe00640 1 -- 172.21.15.39:0/3473451928 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c016070 con 0x557dba912850 2024-07-04T08:21:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.295+0000 7f344fe00640 1 -- 172.21.15.39:0/3473451928 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f345c004d20 con 0x557dba912850 2024-07-04T08:21:28.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.295+0000 7f344fe00640 1 -- 172.21.15.39:0/3473451928 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c005420 con 0x557dba912850 2024-07-04T08:21:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.296+0000 7f345b400640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34480b2960 0x7f34480b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.296+0000 7f346fde3740 1 -- 172.21.15.39:0/3473451928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba912850 msgr2=0x557dbacb7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.296+0000 7f346fde3740 1 --2- 172.21.15.39:0/3473451928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba912850 0x557dbacb7780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f345c010370 tx=0x7f345c010830 comp rx=0 tx=0).stop 2024-07-04T08:21:28.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.296+0000 7f3462a00640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34480b2960 0x7f34480b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:21:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.296+0000 7f346fde3740 1 -- 172.21.15.39:0/3473451928 shutdown_connections 2024-07-04T08:21:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.296+0000 7f346fde3740 1 --2- 172.21.15.39:0/3473451928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba8e9990 0x557dbacb7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.296+0000 7f345b400640 1 -- 172.21.15.39:0/2559710582 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f34540e2cd0 con 0x7f3464103f00 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.296+0000 7f346fde3740 1 --2- 172.21.15.39:0/3473451928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.297+0000 7f346fde3740 1 --2- 172.21.15.39:0/3473451928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba912850 0x557dbacb7780 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.297+0000 7f346fde3740 1 -- 172.21.15.39:0/3473451928 >> 172.21.15.39:0/3473451928 conn(0x557dba906c40 msgr2=0x557dbacb5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.297+0000 7f346fde3740 1 -- 172.21.15.39:0/3473451928 shutdown_connections 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.297+0000 7f346fde3740 1 -- 172.21.15.39:0/3473451928 wait complete. 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.297+0000 7f3462a00640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34480b2960 0x7f34480b4e70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x557dbacb6400 tx=0x7f34540e6780 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.298+0000 7f346fde3740 1 -- start start 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.298+0000 7f346fde3740 1 Processor -- start 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.298+0000 7f346fde3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 0x557dbacc3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.298+0000 7f346fde3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacc3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.298+0000 7f346fde3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbaccf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.298+0000 7f346fde3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557dbac20210 con 0x557dba912850 2024-07-04T08:21:28.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.298+0000 7f346fde3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557dbac1c1d0 con 0x557dba8e9990 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.298+0000 7f346fde3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557dbabc35d0 con 0x557dba8f1b70 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.298+0000 7f346fde3740 10 client.0 _ll_register_callbacks cb 0x557dba92a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.299+0000 7f3463400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 0x557dbacc3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.299+0000 7f3463400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbaccf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.299+0000 7f3463400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbaccf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41528/0 (socket says 172.21.15.39:41528) 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.299+0000 7f3463400640 1 -- 172.21.15.39:0/2449327042 learned_addr learned my addr 172.21.15.39:0/2449327042 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.299+0000 7f3462000640 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacc3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f3463400640 1 -- 172.21.15.39:0/2449327042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 msgr2=0x557dbacc3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f3463400640 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacc3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f3463400640 1 -- 172.21.15.39:0/2449327042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 msgr2=0x557dbacc3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f3463400640 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 0x557dbacc3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f3463400640 1 -- 172.21.15.39:0/2449327042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbabb3960 con 0x557dba912850 2024-07-04T08:21:28.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f3462000640 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacc3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f3463400640 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbaccf130 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x557dbacb61b0 tx=0x7f345c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1093e2f67950ba server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:28.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f344cc00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f344cc00640 1 -- 172.21.15.39:0/2449327042 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c016040 con 0x557dba912850 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f344cc00640 1 -- 172.21.15.39:0/2449327042 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f345c005850 con 0x557dba912850 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.300+0000 7f346fde3740 1 -- 172.21.15.39:0/2449327042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345c004d20 con 0x557dba912850 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.301+0000 7f344cc00640 1 -- 172.21.15.39:0/2449327042 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f345c033050 con 0x557dba912850 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.301+0000 7f346fde3740 10 client.16905 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.301+0000 7f346fde3740 1 -- 172.21.15.39:0/2449327042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dbacd2160 con 0x557dba912850 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.301+0000 7f344cc00640 1 -- 172.21.15.39:0/2449327042 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f345c004d20 con 0x557dba912850 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.301+0000 7f344cc00640 1 -- 172.21.15.39:0/2449327042 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f345c028040 con 0x557dba912850 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.302+0000 7f346fde3740 10 client.16905 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:28.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.302+0000 7f346fde3740 10 client.16905 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:21:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.302+0000 7f346fde3740 20 client.16905 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.302+0000 7f346fde3740 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557dba8d11d0 0x557dbacd55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.302+0000 7f346fde3740 4 client.16905 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.302+0000 7f346fde3740 1 -- 172.21.15.39:0/2449327042 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dbac2eb80 con 0x557dba8d11d0 2024-07-04T08:21:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.302+0000 7f3462000640 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557dba8d11d0 0x557dbacd55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.303+0000 7f3462000640 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557dba8d11d0 0x557dbacd55d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835270832569e11d server_cookie=89a30cf438c4f341 in_seq=0 out_seq=0 2024-07-04T08:21:28.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.303+0000 7f344cc00640 10 client.16905 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.304+0000 7f344cc00640 1 -- 172.21.15.39:0/2449327042 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557dbac2eb80 con 0x557dba8d11d0 2024-07-04T08:21:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.304+0000 7f344cc00640 10 client.16905 handle_command_reply: tid=0 2024-07-04T08:21:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.304+0000 7f346fde3740 1 client.16905 shutdown 2024-07-04T08:21:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.304+0000 7f346fde3740 1 -- 172.21.15.39:0/2449327042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 msgr2=0x557dbaccf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.304+0000 7f346fde3740 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbaccf130 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x557dbacb61b0 tx=0x7f345c000f40 comp rx=0 tx=0).stop 2024-07-04T08:21:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.304+0000 7f346fde3740 1 -- 172.21.15.39:0/2449327042 shutdown_connections 2024-07-04T08:21:28.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.304+0000 7f346fde3740 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557dba8d11d0 0x557dbacd55d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.305+0000 7f346fde3740 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbaccf130 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.305+0000 7f346fde3740 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacc3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.305+0000 7f346fde3740 1 --2- 172.21.15.39:0/2449327042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 0x557dbacc3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.305+0000 7f346fde3740 1 -- 172.21.15.39:0/2449327042 >> 172.21.15.39:0/2449327042 conn(0x557dba906c40 msgr2=0x557dbacba910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.305+0000 7f344cc00640 0 client.16905 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.305+0000 7f346fde3740 1 -- 172.21.15.39:0/2449327042 shutdown_connections 2024-07-04T08:21:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.305+0000 7f346fde3740 1 -- 172.21.15.39:0/2449327042 wait complete. 2024-07-04T08:21:28.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.305+0000 7f346fde3740 20 client.16905 trim_cache size 0 max 16384 2024-07-04T08:21:28.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.327+0000 7f346fde3740 1 Processor -- start 2024-07-04T08:21:28.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.327+0000 7f346fde3740 1 -- start start 2024-07-04T08:21:28.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.327+0000 7f346fde3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8e9990 0x557dbaccf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.327+0000 7f346fde3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f346fde3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbacd85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f346fde3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34641b1e00 con 0x557dba912850 2024-07-04T08:21:28.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f346fde3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34641b0990 con 0x557dba8f1b70 2024-07-04T08:21:28.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f346fde3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34641b1060 con 0x557dba8e9990 2024-07-04T08:21:28.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f3462000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f3462000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:51682/0 (socket says 172.21.15.39:51682) 2024-07-04T08:21:28.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f3462000640 1 -- 172.21.15.39:0/328442971 learned_addr learned my addr 172.21.15.39:0/328442971 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:28.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f3463400640 1 --2- 172.21.15.39:0/328442971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8e9990 0x557dbaccf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f3463400640 1 --2- 172.21.15.39:0/328442971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbacd85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f3462000640 1 -- 172.21.15.39:0/328442971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8e9990 msgr2=0x557dbaccf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:28.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.328+0000 7f3462000640 1 --2- 172.21.15.39:0/328442971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8e9990 0x557dbaccf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.329+0000 7f3462000640 1 -- 172.21.15.39:0/328442971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 msgr2=0x557dbacd85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:28.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.329+0000 7f3462000640 1 --2- 172.21.15.39:0/328442971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbacd85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.329+0000 7f3462000640 1 -- 172.21.15.39:0/328442971 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbacb0da0 con 0x557dba8f1b70 2024-07-04T08:21:28.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.329+0000 7f3462000640 1 --2- 172.21.15.39:0/328442971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb4360 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3450009230 tx=0x7f3450005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22282fe47052f0b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:28.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.329+0000 7f344f400640 1 -- 172.21.15.39:0/328442971 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345004a020 con 0x557dba8f1b70 2024-07-04T08:21:28.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.329+0000 7f344f400640 1 -- 172.21.15.39:0/328442971 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3450031950 con 0x557dba8f1b70 2024-07-04T08:21:28.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.329+0000 7f344f400640 1 -- 172.21.15.39:0/328442971 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450035040 con 0x557dba8f1b70 2024-07-04T08:21:28.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.330+0000 7f346fde3740 1 -- 172.21.15.39:0/328442971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 msgr2=0x557dbacb4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:28.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.330+0000 7f346fde3740 1 --2- 172.21.15.39:0/328442971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb4360 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3450009230 tx=0x7f3450005890 comp rx=0 tx=0).stop 2024-07-04T08:21:28.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.330+0000 7f346fde3740 1 -- 172.21.15.39:0/328442971 shutdown_connections 2024-07-04T08:21:28.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.330+0000 7f346fde3740 1 --2- 172.21.15.39:0/328442971 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbacd85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.330+0000 7f346fde3740 1 --2- 172.21.15.39:0/328442971 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8f1b70 0x557dbacb4360 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.330+0000 7f346fde3740 1 --2- 172.21.15.39:0/328442971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8e9990 0x557dbaccf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.330+0000 7f346fde3740 1 -- 172.21.15.39:0/328442971 >> 172.21.15.39:0/328442971 conn(0x557dba906c40 msgr2=0x557dbacb3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:28.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.330+0000 7f346fde3740 1 -- 172.21.15.39:0/328442971 shutdown_connections 2024-07-04T08:21:28.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.331+0000 7f346fde3740 1 -- 172.21.15.39:0/328442971 wait complete. 2024-07-04T08:21:28.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.331+0000 7f346fde3740 1 -- start start 2024-07-04T08:21:28.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.332+0000 7f346fde3740 1 Processor -- start 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.332+0000 7f346fde3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 0x557dbacd6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.332+0000 7f346fde3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacd6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.332+0000 7f346fde3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbad38f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.332+0000 7f346fde3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34641ae2a0 con 0x557dba912850 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.332+0000 7f346fde3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34641b1730 con 0x557dba8e9990 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.332+0000 7f346fde3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34641ae910 con 0x557dba8f1b70 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.332+0000 7f346fde3740 10 client.0 _ll_register_callbacks cb 0x7f34540e6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3463400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 0x557dbacd6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3462000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacd6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3462000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacd6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43132/0 (socket says 172.21.15.39:43132) 2024-07-04T08:21:28.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3462000640 1 -- 172.21.15.39:0/219644131 learned_addr learned my addr 172.21.15.39:0/219644131 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:28.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3463400640 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbad38f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3462000640 1 -- 172.21.15.39:0/219644131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 msgr2=0x557dbacd6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:28.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3462000640 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 0x557dbacd6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3462000640 1 -- 172.21.15.39:0/219644131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 msgr2=0x557dbad38f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:28.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3462000640 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbad38f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3462000640 1 -- 172.21.15.39:0/219644131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbad395c0 con 0x557dba8f1b70 2024-07-04T08:21:28.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.333+0000 7f3463400640 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbad38f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:28.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.334+0000 7f3462000640 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacd6610 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3450003700 tx=0x7f3450000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f1ada703b8b1794 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:28.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.334+0000 7f344cc00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:21:28.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.334+0000 7f344cc00640 1 -- 172.21.15.39:0/219644131 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345004a030 con 0x557dba8f1b70 2024-07-04T08:21:28.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.334+0000 7f346fde3740 1 -- 172.21.15.39:0/219644131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557dbac2a620 con 0x557dba8f1b70 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.334+0000 7f344cc00640 1 -- 172.21.15.39:0/219644131 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f3450004180 con 0x557dba8f1b70 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f344cc00640 1 -- 172.21.15.39:0/219644131 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3450060050 con 0x557dba8f1b70 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f346fde3740 10 client.26959 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f346fde3740 1 -- 172.21.15.39:0/219644131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dbad41ab0 con 0x557dba8f1b70 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f344cc00640 1 -- 172.21.15.39:0/219644131 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557dbac2a620 con 0x557dba8f1b70 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f344cc00640 1 -- 172.21.15.39:0/219644131 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f34500467c0 con 0x557dba8f1b70 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f346fde3740 10 client.26959 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f346fde3740 10 client.26959 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f346fde3740 20 client.26959 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f346fde3740 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557dba8a6580 0x557dbad442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f346fde3740 4 client.26959 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.95, "await_for": 45.0}] 2024-07-04T08:21:28.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.335+0000 7f346fde3740 1 -- 172.21.15.39:0/219644131 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.95, "await_for": 45.0}) -- 0x557dbac27d60 con 0x557dba8a6580 2024-07-04T08:21:28.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.336+0000 7f3463400640 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557dba8a6580 0x557dbad442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:28.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.336+0000 7f3463400640 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557dba8a6580 0x557dbad442e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f1454bae35ff42 server_cookie=b4d8927999914a50 in_seq=0 out_seq=0 2024-07-04T08:21:28.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.336+0000 7f344cc00640 10 client.26959 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:28.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.346+0000 7f344cc00640 1 -- 172.21.15.39:0/219644131 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557dbac27d60 con 0x557dba8a6580 2024-07-04T08:21:28.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.346+0000 7f344cc00640 10 client.26959 handle_command_reply: tid=0 2024-07-04T08:21:28.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.346+0000 7f346fde3740 1 client.26959 shutdown 2024-07-04T08:21:28.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 -- 172.21.15.39:0/219644131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 msgr2=0x557dbacd6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:28.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacd6610 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3450003700 tx=0x7f3450000bc0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 -- 172.21.15.39:0/219644131 shutdown_connections 2024-07-04T08:21:28.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557dba8a6580 0x557dbad442e0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557dba912850 0x557dbad38f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557dba8f1b70 0x557dbacd6610 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 --2- 172.21.15.39:0/219644131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557dba8e9990 0x557dbacd6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f344cc00640 0 client.26959 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:28.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 -- 172.21.15.39:0/219644131 >> 172.21.15.39:0/219644131 conn(0x557dba906c40 msgr2=0x557dbacb5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:28.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 -- 172.21.15.39:0/219644131 shutdown_connections 2024-07-04T08:21:28.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 1 -- 172.21.15.39:0/219644131 wait complete. 2024-07-04T08:21:28.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.347+0000 7f346fde3740 20 client.26959 trim_cache size 0 max 16384 2024-07-04T08:21:28.344 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:21:28.344 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:21:28.344 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:21:28.344 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 18, 2024-07-04T08:21:28.344 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:21:28.344 INFO:teuthology.orchestra.run.smithi039.stdout: "90f50498": { 2024-07-04T08:21:28.344 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 18, 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 143.9, 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:21:28.345 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:21:28.346 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:21:28.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:28.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:28.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:28.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:28.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:28.346 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 -- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34480b2960 msgr2=0x7f34480b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34480b2960 0x7f34480b4e70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x557dbacb6400 tx=0x7f34540e6780 comp rx=0 tx=0).stop 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 -- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3464103f00 msgr2=0x7f34641999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3464103f00 0x7f34641999a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f345400e9f0 tx=0x7f345400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 -- 172.21.15.39:0/2559710582 shutdown_connections 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34480b2960 0x7f34480b4e70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f346410d360 0x7f346419f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464104850 0x7f3464199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 --2- 172.21.15.39:0/2559710582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3464103f00 0x7f34641999a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 -- 172.21.15.39:0/2559710582 >> 172.21.15.39:0/2559710582 conn(0x7f3464078c20 msgr2=0x7f3464106e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:28.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.348+0000 7f3469800640 1 -- 172.21.15.39:0/2559710582 shutdown_connections 2024-07-04T08:21:28.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:28.349+0000 7f3469800640 1 -- 172.21.15.39:0/2559710582 wait complete. 2024-07-04T08:21:29.096 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:28 smithi039 ceph-mon[32062]: pgmap v668: 129 pgs: 129 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:21:29.242 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '90f50498', quiesce took 12.5 seconds. Will release after: 11.45 2024-07-04T08:21:29.264 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:29.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:28 smithi133 ceph-mon[37050]: pgmap v668: 129 pgs: 129 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:21:29.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:28 smithi050 ceph-mon[36577]: pgmap v668: 129 pgs: 129 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-07-04T08:21:31.130 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:30 smithi039 ceph-mon[32062]: pgmap v669: 129 pgs: 129 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-07-04T08:21:31.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:30 smithi133 ceph-mon[37050]: pgmap v669: 129 pgs: 129 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-07-04T08:21:31.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:30 smithi050 ceph-mon[36577]: pgmap v669: 129 pgs: 129 active+clean; 6.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 13 op/s 2024-07-04T08:21:31.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.465+0000 7f4423e00640 1 -- 172.21.15.39:0/1370127411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 msgr2=0x7f4424104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:31.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.465+0000 7f4423e00640 1 --2- 172.21.15.39:0/1370127411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 0x7f4424104170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4414009900 tx=0x7f441402a400 comp rx=0 tx=0).stop 2024-07-04T08:21:31.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.465+0000 7f4423e00640 1 -- 172.21.15.39:0/1370127411 shutdown_connections 2024-07-04T08:21:31.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.465+0000 7f4423e00640 1 --2- 172.21.15.39:0/1370127411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f442410d1c0 0x7f442410f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.465+0000 7f4423e00640 1 --2- 172.21.15.39:0/1370127411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44241046b0 0x7f442410cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.465+0000 7f4423e00640 1 --2- 172.21.15.39:0/1370127411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 0x7f4424104170 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.465+0000 7f4423e00640 1 -- 172.21.15.39:0/1370127411 >> 172.21.15.39:0/1370127411 conn(0x7f44240fd840 msgr2=0x7f44240ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:31.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.465+0000 7f4423e00640 1 -- 172.21.15.39:0/1370127411 shutdown_connections 2024-07-04T08:21:31.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.465+0000 7f4423e00640 1 -- 172.21.15.39:0/1370127411 wait complete. 2024-07-04T08:21:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.466+0000 7f4423e00640 1 Processor -- start 2024-07-04T08:21:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.466+0000 7f4423e00640 1 -- start start 2024-07-04T08:21:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4423e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 0x7f442419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4423e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44241046b0 0x7f442419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4423e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f442410d1c0 0x7f44241a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4423e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4424112e90 con 0x7f4424103d60 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4423e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4424112d10 con 0x7f44241046b0 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4423e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4424113010 con 0x7f442410d1c0 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4422a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 0x7f442419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4422a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 0x7f442419ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44342/0 (socket says 172.21.15.39:44342) 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4422a00640 1 -- 172.21.15.39:0/699416249 learned_addr learned my addr 172.21.15.39:0/699416249 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.467+0000 7f4422000640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44241046b0 0x7f442419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.468+0000 7f4423400640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f442410d1c0 0x7f44241a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.468+0000 7f4422a00640 1 -- 172.21.15.39:0/699416249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f442410d1c0 msgr2=0x7f44241a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.468+0000 7f4422a00640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f442410d1c0 0x7f44241a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.468+0000 7f4422a00640 1 -- 172.21.15.39:0/699416249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44241046b0 msgr2=0x7f442419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.468+0000 7f4422a00640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44241046b0 0x7f442419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.468+0000 7f4422a00640 1 -- 172.21.15.39:0/699416249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44241a19f0 con 0x7f4424103d60 2024-07-04T08:21:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.468+0000 7f4422000640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44241046b0 0x7f442419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.468+0000 7f4422a00640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 0x7f442419ba80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4414005ab0 tx=0x7f4414003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.469+0000 7f441b400640 1 -- 172.21.15.39:0/699416249 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441403f070 con 0x7f4424103d60 2024-07-04T08:21:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.469+0000 7f441b400640 1 -- 172.21.15.39:0/699416249 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4414031b60 con 0x7f4424103d60 2024-07-04T08:21:31.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.469+0000 7f441b400640 1 -- 172.21.15.39:0/699416249 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414035030 con 0x7f4424103d60 2024-07-04T08:21:31.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.469+0000 7f4423e00640 1 -- 172.21.15.39:0/699416249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44241a1c80 con 0x7f4424103d60 2024-07-04T08:21:31.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.469+0000 7f4423e00640 1 -- 172.21.15.39:0/699416249 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44241a8440 con 0x7f4424103d60 2024-07-04T08:21:31.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.472+0000 7f4430154740 1 Processor -- start 2024-07-04T08:21:31.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.472+0000 7f4430154740 1 -- start start 2024-07-04T08:21:31.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4430154740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 0x55c7ba9c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4430154740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4430154740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f441b400640 1 -- 172.21.15.39:0/699416249 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4414042070 con 0x7f4424103d60 2024-07-04T08:21:31.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4430154740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba939ee0 con 0x55c7ba5f61b0 2024-07-04T08:21:31.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4430154740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba929480 con 0x55c7ba5fe390 2024-07-04T08:21:31.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4430154740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba92d4c0 con 0x55c7ba61f070 2024-07-04T08:21:31.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4422000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4422a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4422000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41038/0 (socket says 172.21.15.39:41038) 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4422000640 1 -- 172.21.15.39:0/3729061205 learned_addr learned my addr 172.21.15.39:0/3729061205 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.473+0000 7f4423400640 1 --2- 172.21.15.39:0/3729061205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 0x55c7ba9c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.474+0000 7f4422000640 1 -- 172.21.15.39:0/3729061205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 msgr2=0x55c7ba9c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.474+0000 7f4422000640 1 --2- 172.21.15.39:0/3729061205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 0x55c7ba9c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.474+0000 7f4422000640 1 -- 172.21.15.39:0/3729061205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 msgr2=0x55c7ba9c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.474+0000 7f4422000640 1 --2- 172.21.15.39:0/3729061205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.474+0000 7f4422000640 1 -- 172.21.15.39:0/3729061205 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ba8c1650 con 0x55c7ba5fe390 2024-07-04T08:21:31.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.474+0000 7f4422000640 1 --2- 172.21.15.39:0/3729061205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c43d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4410011470 tx=0x7f4410011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a255506694da652 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:31.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.475+0000 7f441b400640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44080b2960 0x7f44080b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.475+0000 7f440fe00640 1 -- 172.21.15.39:0/3729061205 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441001e070 con 0x55c7ba5fe390 2024-07-04T08:21:31.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.475+0000 7f440fe00640 1 -- 172.21.15.39:0/3729061205 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4410004310 con 0x55c7ba5fe390 2024-07-04T08:21:31.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.475+0000 7f440fe00640 1 -- 172.21.15.39:0/3729061205 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000f900 con 0x55c7ba5fe390 2024-07-04T08:21:31.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.475+0000 7f441b400640 1 -- 172.21.15.39:0/699416249 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f4414102ce0 con 0x7f4424103d60 2024-07-04T08:21:31.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.475+0000 7f4423400640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44080b2960 0x7f44080b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4430154740 1 -- 172.21.15.39:0/3729061205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 msgr2=0x55c7ba9c43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4430154740 1 --2- 172.21.15.39:0/3729061205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c43d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4410011470 tx=0x7f4410011930 comp rx=0 tx=0).stop 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4423400640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44080b2960 0x7f44080b4e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f441c005fd0 tx=0x7f441c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4430154740 1 -- 172.21.15.39:0/3729061205 shutdown_connections 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4430154740 1 --2- 172.21.15.39:0/3729061205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4430154740 1 --2- 172.21.15.39:0/3729061205 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c43d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4430154740 1 --2- 172.21.15.39:0/3729061205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 0x55c7ba9c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4430154740 1 -- 172.21.15.39:0/3729061205 >> 172.21.15.39:0/3729061205 conn(0x55c7ba613460 msgr2=0x55c7ba9c1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4430154740 1 -- 172.21.15.39:0/3729061205 shutdown_connections 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.476+0000 7f4430154740 1 -- 172.21.15.39:0/3729061205 wait complete. 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.477+0000 7f4430154740 1 -- start start 2024-07-04T08:21:31.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.477+0000 7f4430154740 1 Processor -- start 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4430154740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4430154740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4430154740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 0x55c7ba9db420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4430154740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba78d700 con 0x55c7ba5fe390 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4430154740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba8d0bb0 con 0x55c7ba61f070 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4430154740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c7ba9d2130 con 0x55c7ba5f61b0 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4430154740 10 client.0 _ll_register_callbacks cb 0x55c7ba922570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4422a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4422000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4422a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56252/0 (socket says 172.21.15.39:56252) 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4422a00640 1 -- 172.21.15.39:0/2098876021 learned_addr learned my addr 172.21.15.39:0/2098876021 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4423400640 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 0x55c7ba9db420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4422a00640 1 -- 172.21.15.39:0/2098876021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 msgr2=0x55c7ba9db420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4422a00640 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 0x55c7ba9db420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4422a00640 1 -- 172.21.15.39:0/2098876021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 msgr2=0x55c7ba9cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4422a00640 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.478+0000 7f4422a00640 1 -- 172.21.15.39:0/2098876021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ba68a650 con 0x55c7ba5f61b0 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.479+0000 7f4423400640 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 0x55c7ba9db420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.479+0000 7f4422a00640 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9cf4c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f44140c9c70 tx=0x7f4414009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cca0c8ee6d4f82d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.479+0000 7f440cc00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.479+0000 7f440cc00640 1 -- 172.21.15.39:0/2098876021 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414049070 con 0x55c7ba5f61b0 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.479+0000 7f440cc00640 1 -- 172.21.15.39:0/2098876021 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f441403b460 con 0x55c7ba5f61b0 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.479+0000 7f4430154740 1 -- 172.21.15.39:0/2098876021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4410004310 con 0x55c7ba5f61b0 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.479+0000 7f440cc00640 1 -- 172.21.15.39:0/2098876021 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44140c9450 con 0x55c7ba5f61b0 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.479+0000 7f4430154740 10 client.26971 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:31.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.479+0000 7f4430154740 1 -- 172.21.15.39:0/2098876021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c7ba9de0f0 con 0x55c7ba5f61b0 2024-07-04T08:21:31.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.480+0000 7f440cc00640 1 -- 172.21.15.39:0/2098876021 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4410004310 con 0x55c7ba5f61b0 2024-07-04T08:21:31.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.480+0000 7f440cc00640 1 -- 172.21.15.39:0/2098876021 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f441404c060 con 0x55c7ba5f61b0 2024-07-04T08:21:31.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.480+0000 7f4430154740 10 client.26971 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:31.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.480+0000 7f4430154740 10 client.26971 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:21:31.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.480+0000 7f4430154740 20 client.26971 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:31.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.480+0000 7f4430154740 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55c7ba5dd9f0 0x55c7ba9e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.480+0000 7f4430154740 4 client.26971 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:31.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.480+0000 7f4430154740 1 -- 172.21.15.39:0/2098876021 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7ba93be30 con 0x55c7ba5dd9f0 2024-07-04T08:21:31.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.481+0000 7f4422000640 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55c7ba5dd9f0 0x55c7ba9e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.481+0000 7f4422000640 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55c7ba5dd9f0 0x55c7ba9e1530 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=947b1893692af8fc server_cookie=2a28402fc87b8ac5 in_seq=0 out_seq=0 2024-07-04T08:21:31.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.481+0000 7f440cc00640 10 client.26971 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:21:31.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.482+0000 7f440cc00640 1 -- 172.21.15.39:0/2098876021 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c7ba93be30 con 0x55c7ba5dd9f0 2024-07-04T08:21:31.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.482+0000 7f440cc00640 10 client.26971 handle_command_reply: tid=0 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.482+0000 7f4430154740 1 client.26971 shutdown 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.482+0000 7f4430154740 1 -- 172.21.15.39:0/2098876021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 msgr2=0x55c7ba9cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.482+0000 7f4430154740 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9cf4c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f44140c9c70 tx=0x7f4414009620 comp rx=0 tx=0).stop 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.482+0000 7f4430154740 1 -- 172.21.15.39:0/2098876021 shutdown_connections 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.483+0000 7f4430154740 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55c7ba5dd9f0 0x55c7ba9e1530 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.483+0000 7f4430154740 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.483+0000 7f4430154740 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 0x55c7ba9db420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.483+0000 7f440cc00640 0 client.26971 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.483+0000 7f4430154740 1 --2- 172.21.15.39:0/2098876021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9cf4c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.483+0000 7f4430154740 1 -- 172.21.15.39:0/2098876021 >> 172.21.15.39:0/2098876021 conn(0x55c7ba613460 msgr2=0x55c7ba9c6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.483+0000 7f4430154740 1 -- 172.21.15.39:0/2098876021 shutdown_connections 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.483+0000 7f4430154740 1 -- 172.21.15.39:0/2098876021 wait complete. 2024-07-04T08:21:31.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.483+0000 7f4430154740 20 client.26971 trim_cache size 0 max 16384 2024-07-04T08:21:31.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.498+0000 7f4430154740 1 Processor -- start 2024-07-04T08:21:31.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.498+0000 7f4430154740 1 -- start start 2024-07-04T08:21:31.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4430154740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4430154740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4430154740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 0x55c7ba9c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4430154740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44241b21f0 con 0x55c7ba5fe390 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4430154740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44241b3d30 con 0x55c7ba61f070 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4430154740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44241b28c0 con 0x55c7ba5f61b0 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4422000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4422a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4422000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44380/0 (socket says 172.21.15.39:44380) 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4422000640 1 -- 172.21.15.39:0/1203493666 learned_addr learned my addr 172.21.15.39:0/1203493666 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:31.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.499+0000 7f4423400640 1 --2- 172.21.15.39:0/1203493666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 0x55c7ba9c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.500+0000 7f4422000640 1 -- 172.21.15.39:0/1203493666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 msgr2=0x55c7ba9c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:31.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.500+0000 7f4422000640 1 --2- 172.21.15.39:0/1203493666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9c08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.500+0000 7f4422000640 1 -- 172.21.15.39:0/1203493666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 msgr2=0x55c7ba9c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:31.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.500+0000 7f4422000640 1 --2- 172.21.15.39:0/1203493666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 0x55c7ba9c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.500+0000 7f4422000640 1 -- 172.21.15.39:0/1203493666 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ba9c2520 con 0x55c7ba5fe390 2024-07-04T08:21:31.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.500+0000 7f4422a00640 1 --2- 172.21.15.39:0/1203493666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:31.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.500+0000 7f4422000640 1 --2- 172.21.15.39:0/1203493666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c7410 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4410003f50 tx=0x7f4410005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ed74f6b1815a93 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:31.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.501+0000 7f440f400640 1 -- 172.21.15.39:0/1203493666 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441001e040 con 0x55c7ba5fe390 2024-07-04T08:21:31.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.501+0000 7f440f400640 1 -- 172.21.15.39:0/1203493666 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f441001f040 con 0x55c7ba5fe390 2024-07-04T08:21:31.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.501+0000 7f440f400640 1 -- 172.21.15.39:0/1203493666 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44100045e0 con 0x55c7ba5fe390 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 -- 172.21.15.39:0/1203493666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 msgr2=0x55c7ba9c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 --2- 172.21.15.39:0/1203493666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c7410 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4410003f50 tx=0x7f4410005d90 comp rx=0 tx=0).stop 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 -- 172.21.15.39:0/1203493666 shutdown_connections 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 --2- 172.21.15.39:0/1203493666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5fe390 0x55c7ba9c7410 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 --2- 172.21.15.39:0/1203493666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba61f070 0x55c7ba9c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 --2- 172.21.15.39:0/1203493666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 -- 172.21.15.39:0/1203493666 >> 172.21.15.39:0/1203493666 conn(0x55c7ba613460 msgr2=0x55c7ba9c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 -- 172.21.15.39:0/1203493666 shutdown_connections 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 -- 172.21.15.39:0/1203493666 wait complete. 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.502+0000 7f4430154740 1 -- start start 2024-07-04T08:21:31.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.503+0000 7f4430154740 1 Processor -- start 2024-07-04T08:21:31.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.503+0000 7f4430154740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9d7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.503+0000 7f4430154740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.503+0000 7f4430154740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 0x55c7ba9d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.503+0000 7f4430154740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44241b3660 con 0x55c7ba5f61b0 2024-07-04T08:21:31.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.503+0000 7f4430154740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44241b2f90 con 0x55c7ba5fe390 2024-07-04T08:21:31.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.503+0000 7f4430154740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44241b01d0 con 0x55c7ba61f070 2024-07-04T08:21:31.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.503+0000 7f4430154740 10 client.0 _ll_register_callbacks cb 0x7f4414106480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:31.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4422a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9d7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4422a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9d7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44396/0 (socket says 172.21.15.39:44396) 2024-07-04T08:21:31.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4422a00640 1 -- 172.21.15.39:0/3750367727 learned_addr learned my addr 172.21.15.39:0/3750367727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:31.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4422000640 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4423400640 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 0x55c7ba9d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4422000640 1 -- 172.21.15.39:0/3750367727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 msgr2=0x55c7ba9d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:31.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4422000640 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 0x55c7ba9d37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4422000640 1 -- 172.21.15.39:0/3750367727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 msgr2=0x55c7ba9d7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:31.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4422000640 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9d7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.504+0000 7f4422000640 1 -- 172.21.15.39:0/3750367727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7baa48d00 con 0x55c7ba5fe390 2024-07-04T08:21:31.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.505+0000 7f4422000640 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9d30d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4410011440 tx=0x7f4410005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=533396d6f31afb17 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:31.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.505+0000 7f440cc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:21:31.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.505+0000 7f440cc00640 1 -- 172.21.15.39:0/3750367727 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441003d060 con 0x55c7ba5fe390 2024-07-04T08:21:31.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.505+0000 7f440cc00640 1 -- 172.21.15.39:0/3750367727 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f44100037f0 con 0x55c7ba5fe390 2024-07-04T08:21:31.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.505+0000 7f4430154740 1 -- 172.21.15.39:0/3750367727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f441001f040 con 0x55c7ba5fe390 2024-07-04T08:21:31.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.505+0000 7f440cc00640 1 -- 172.21.15.39:0/3750367727 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f441001e040 con 0x55c7ba5fe390 2024-07-04T08:21:31.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.505+0000 7f4430154740 10 client.27068 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:31.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.505+0000 7f4430154740 1 -- 172.21.15.39:0/3750367727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c7baa49e20 con 0x55c7ba5fe390 2024-07-04T08:21:31.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.506+0000 7f440cc00640 1 -- 172.21.15.39:0/3750367727 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f441001f040 con 0x55c7ba5fe390 2024-07-04T08:21:31.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.506+0000 7f440cc00640 1 -- 172.21.15.39:0/3750367727 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f44100049d0 con 0x55c7ba5fe390 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.506+0000 7f4430154740 10 client.27068 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.506+0000 7f4430154740 10 client.27068 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.506+0000 7f4430154740 20 client.27068 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.506+0000 7f4430154740 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55c7ba5b2da0 0x55c7baa4c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.506+0000 7f4430154740 4 client.27068 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.506+0000 7f4430154740 1 -- 172.21.15.39:0/3750367727 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7ba9378d0 con 0x55c7ba5b2da0 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.507+0000 7f4422a00640 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55c7ba5b2da0 0x55c7baa4c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.507+0000 7f4422a00640 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55c7ba5b2da0 0x55c7baa4c350 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2a1c9809ce117dbd server_cookie=39c52db89965a109 in_seq=0 out_seq=0 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.507+0000 7f440cc00640 10 client.27068 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:21:31.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f440cc00640 1 -- 172.21.15.39:0/3750367727 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7ba9378d0 con 0x55c7ba5b2da0 2024-07-04T08:21:31.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f440cc00640 10 client.27068 handle_command_reply: tid=0 2024-07-04T08:21:31.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 client.27068 shutdown 2024-07-04T08:21:31.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 -- 172.21.15.39:0/3750367727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 msgr2=0x55c7ba9d30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:31.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9d30d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4410011440 tx=0x7f4410005270 comp rx=0 tx=0).stop 2024-07-04T08:21:31.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 -- 172.21.15.39:0/3750367727 shutdown_connections 2024-07-04T08:21:31.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55c7ba5b2da0 0x55c7baa4c350 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7ba61f070 0x55c7ba9d37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f440cc00640 0 client.27068 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:21:31.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c7ba5fe390 0x55c7ba9d30d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 --2- 172.21.15.39:0/3750367727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c7ba5f61b0 0x55c7ba9d7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 -- 172.21.15.39:0/3750367727 >> 172.21.15.39:0/3750367727 conn(0x55c7ba613460 msgr2=0x55c7ba9c71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:31.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 -- 172.21.15.39:0/3750367727 shutdown_connections 2024-07-04T08:21:31.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.508+0000 7f4430154740 1 -- 172.21.15.39:0/3750367727 wait complete. 2024-07-04T08:21:31.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.509+0000 7f4430154740 20 client.27068 trim_cache size 0 max 16384 2024-07-04T08:21:31.507 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:21:31.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.509+0000 7f4423e00640 1 -- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44080b2960 msgr2=0x7f44080b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:31.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.509+0000 7f4423e00640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44080b2960 0x7f44080b4e70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f441c005fd0 tx=0x7f441c009500 comp rx=0 tx=0).stop 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.509+0000 7f4423e00640 1 -- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 msgr2=0x7f442419ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.509+0000 7f4423e00640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 0x7f442419ba80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4414005ab0 tx=0x7f4414003c10 comp rx=0 tx=0).stop 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.510+0000 7f4423e00640 1 -- 172.21.15.39:0/699416249 shutdown_connections 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.510+0000 7f4423e00640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f44080b2960 0x7f44080b4e70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.510+0000 7f4423e00640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f442410d1c0 0x7f44241a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.510+0000 7f4423e00640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f44241046b0 0x7f442419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.510+0000 7f4423e00640 1 --2- 172.21.15.39:0/699416249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4424103d60 0x7f442419ba80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.510+0000 7f4423e00640 1 -- 172.21.15.39:0/699416249 >> 172.21.15.39:0/699416249 conn(0x7f44240fd840 msgr2=0x7f4424101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.510+0000 7f4423e00640 1 -- 172.21.15.39:0/699416249 shutdown_connections 2024-07-04T08:21:31.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:31.510+0000 7f4423e00640 1 -- 172.21.15.39:0/699416249 wait complete. 2024-07-04T08:21:32.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:21:32.844 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:33.231 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:32 smithi039 ceph-mon[32062]: pgmap v670: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-07-04T08:21:33.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:32 smithi133 ceph-mon[37050]: pgmap v670: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-07-04T08:21:33.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:32 smithi050 ceph-mon[36577]: pgmap v670: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.993+0000 7fe3c5200640 1 -- 172.21.15.39:0/3210797573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c0102490 msgr2=0x7fe3c010f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.993+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3210797573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c0102490 0x7fe3c010f1b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009900 tx=0x7fe3a802a480 comp rx=0 tx=0).stop 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.993+0000 7fe3c5200640 1 -- 172.21.15.39:0/3210797573 shutdown_connections 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.993+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3210797573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3c010f6f0 0x7fe3c0111b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.993+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3210797573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c0102490 0x7fe3c010f1b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.993+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3210797573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3c0101b40 0x7fe3c0101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.993+0000 7fe3c5200640 1 -- 172.21.15.39:0/3210797573 >> 172.21.15.39:0/3210797573 conn(0x7fe3c0078c70 msgr2=0x7fe3c0079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.993+0000 7fe3c5200640 1 -- 172.21.15.39:0/3210797573 shutdown_connections 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.993+0000 7fe3c5200640 1 -- 172.21.15.39:0/3210797573 wait complete. 2024-07-04T08:21:34.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.994+0000 7fe3c5200640 1 Processor -- start 2024-07-04T08:21:34.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.995+0000 7fe3c5200640 1 -- start start 2024-07-04T08:21:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.995+0000 7fe3c5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3c0101b40 0x7fe3c0107b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.995+0000 7fe3c5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c0102490 0x7fe3c0103e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.995+0000 7fe3c5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3c010f6f0 0x7fe3c0104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.995+0000 7fe3c5200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0115200 con 0x7fe3c0101b40 2024-07-04T08:21:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.995+0000 7fe3c5200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0115080 con 0x7fe3c0102490 2024-07-04T08:21:34.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.995+0000 7fe3c5200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0115380 con 0x7fe3c010f6f0 2024-07-04T08:21:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.995+0000 7fe3bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3c0101b40 0x7fe3c0107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.995+0000 7fe3bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3c0101b40 0x7fe3c0107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44418/0 (socket says 172.21.15.39:44418) 2024-07-04T08:21:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3bde00640 1 -- 172.21.15.39:0/3991139720 learned_addr learned my addr 172.21.15.39:0/3991139720 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3b5400640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c0102490 0x7fe3c0103e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3be800640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3c010f6f0 0x7fe3c0104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3b5400640 1 -- 172.21.15.39:0/3991139720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3c010f6f0 msgr2=0x7fe3c0104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3b5400640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3c010f6f0 0x7fe3c0104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3b5400640 1 -- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3c0101b40 msgr2=0x7fe3c0107b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:34.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3b5400640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3c0101b40 0x7fe3c0107b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:34.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3b5400640 1 -- 172.21.15.39:0/3991139720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c01ac0c0 con 0x7fe3c0102490 2024-07-04T08:21:34.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3bde00640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3c0101b40 0x7fe3c0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:34.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.996+0000 7fe3b5400640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c0102490 0x7fe3c0103e70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe3a802aa70 tx=0x7fe3a8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:34.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.997+0000 7fe3b7400640 1 -- 172.21.15.39:0/3991139720 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a803f070 con 0x7fe3c0102490 2024-07-04T08:21:34.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.997+0000 7fe3b7400640 1 -- 172.21.15.39:0/3991139720 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a8031b50 con 0x7fe3c0102490 2024-07-04T08:21:34.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.997+0000 7fe3b7400640 1 -- 172.21.15.39:0/3991139720 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8035030 con 0x7fe3c0102490 2024-07-04T08:21:34.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.997+0000 7fe3c5200640 1 -- 172.21.15.39:0/3991139720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c01ac290 con 0x7fe3c0102490 2024-07-04T08:21:34.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:34.997+0000 7fe3c5200640 1 -- 172.21.15.39:0/3991139720 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c01ac750 con 0x7fe3c0102490 2024-07-04T08:21:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.002+0000 7fe3cb758740 1 Processor -- start 2024-07-04T08:21:34.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.002+0000 7fe3cb758740 1 -- start start 2024-07-04T08:21:34.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3b7400640 1 -- 172.21.15.39:0/3991139720 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe3a8042070 con 0x7fe3c0102490 2024-07-04T08:21:34.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2f61b0 0x55b4da6c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 0x55b4da6c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3cb758740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b4da639f00 con 0x55b4da31f070 2024-07-04T08:21:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3cb758740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b4da6294a0 con 0x55b4da2f61b0 2024-07-04T08:21:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3cb758740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b4da62d4e0 con 0x55b4da2fe390 2024-07-04T08:21:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3be800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3be800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44420/0 (socket says 172.21.15.39:44420) 2024-07-04T08:21:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3be800640 1 -- 172.21.15.39:0/4267613701 learned_addr learned my addr 172.21.15.39:0/4267613701 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:34.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3bde00640 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2f61b0 0x55b4da6c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.003+0000 7fe3b5400640 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 0x55b4da6c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.004+0000 7fe3bde00640 1 -- 172.21.15.39:0/4267613701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 msgr2=0x55b4da6c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.004+0000 7fe3bde00640 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 0x55b4da6c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.004+0000 7fe3bde00640 1 -- 172.21.15.39:0/4267613701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 msgr2=0x55b4da6c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.004+0000 7fe3bde00640 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.004+0000 7fe3bde00640 1 -- 172.21.15.39:0/4267613701 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4da5c1670 con 0x55b4da2f61b0 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.004+0000 7fe3b5400640 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 0x55b4da6c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.004+0000 7fe3be800640 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.004+0000 7fe3bde00640 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2f61b0 0x55b4da6c3d10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe3c01078e0 tx=0x7fe3b00119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98eb7ba8345b609c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.005+0000 7fe3af400640 1 -- 172.21.15.39:0/4267613701 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0004030 con 0x55b4da2f61b0 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.005+0000 7fe3b7400640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3a00b2a10 0x7fe3a00b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.005+0000 7fe3af400640 1 -- 172.21.15.39:0/4267613701 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b0005020 con 0x55b4da2f61b0 2024-07-04T08:21:35.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.005+0000 7fe3af400640 1 -- 172.21.15.39:0/4267613701 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0004cd0 con 0x55b4da2f61b0 2024-07-04T08:21:35.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.005+0000 7fe3b7400640 1 -- 172.21.15.39:0/3991139720 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fe3a8102ea0 con 0x7fe3c0102490 2024-07-04T08:21:35.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.005+0000 7fe3be800640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3a00b2a10 0x7fe3a00b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3cb758740 1 -- 172.21.15.39:0/4267613701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2f61b0 msgr2=0x55b4da6c3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:35.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3cb758740 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2f61b0 0x55b4da6c3d10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe3c01078e0 tx=0x7fe3b00119a0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3be800640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3a00b2a10 0x7fe3a00b4f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80125f0 tx=0x7fe3b8009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:35.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3cb758740 1 -- 172.21.15.39:0/4267613701 shutdown_connections 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3cb758740 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 0x55b4da6c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3cb758740 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2f61b0 0x55b4da6c3d10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3cb758740 1 --2- 172.21.15.39:0/4267613701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3cb758740 1 -- 172.21.15.39:0/4267613701 >> 172.21.15.39:0/4267613701 conn(0x55b4da313460 msgr2=0x55b4da6c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3cb758740 1 -- 172.21.15.39:0/4267613701 shutdown_connections 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.006+0000 7fe3cb758740 1 -- 172.21.15.39:0/4267613701 wait complete. 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.007+0000 7fe3cb758740 1 -- start start 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.007+0000 7fe3cb758740 1 Processor -- start 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 0x55b4da6cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2fe390 0x55b4da6cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3cb758740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b4da48d700 con 0x55b4da31f070 2024-07-04T08:21:35.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3cb758740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b4da5d0bd0 con 0x55b4da2fe390 2024-07-04T08:21:35.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3cb758740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b4da6d1fb0 con 0x55b4da2f61b0 2024-07-04T08:21:35.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3cb758740 10 client.0 _ll_register_callbacks cb 0x55b4da622590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:35.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3be800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3be800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6db2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44432/0 (socket says 172.21.15.39:44432) 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3be800640 1 -- 172.21.15.39:0/3228098750 learned_addr learned my addr 172.21.15.39:0/3228098750 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.008+0000 7fe3b5400640 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2fe390 0x55b4da6cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3bde00640 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 0x55b4da6cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3be800640 1 -- 172.21.15.39:0/3228098750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 msgr2=0x55b4da6cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3be800640 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 0x55b4da6cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3be800640 1 -- 172.21.15.39:0/3228098750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2fe390 msgr2=0x55b4da6cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3be800640 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2fe390 0x55b4da6cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3be800640 1 -- 172.21.15.39:0/3228098750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4da38a650 con 0x55b4da31f070 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3b5400640 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2fe390 0x55b4da6cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:35.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3bde00640 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 0x55b4da6cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3be800640 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6db2d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800fba0 tx=0x7fe3b8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5242d921fca4d52 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.009+0000 7fe3a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.010+0000 7fe3a7e00640 1 -- 172.21.15.39:0/3228098750 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800fd50 con 0x55b4da31f070 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.010+0000 7fe3a7e00640 1 -- 172.21.15.39:0/3228098750 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fe3b8018030 con 0x55b4da31f070 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.010+0000 7fe3cb758740 1 -- 172.21.15.39:0/3228098750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b0005020 con 0x55b4da31f070 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.010+0000 7fe3a7e00640 1 -- 172.21.15.39:0/3228098750 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3b8034050 con 0x55b4da31f070 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.010+0000 7fe3cb758740 10 client.16941 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.010+0000 7fe3cb758740 1 -- 172.21.15.39:0/3228098750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4da6ddfd0 con 0x55b4da31f070 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.010+0000 7fe3a7e00640 1 -- 172.21.15.39:0/3228098750 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b0005020 con 0x55b4da31f070 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.011+0000 7fe3a7e00640 1 -- 172.21.15.39:0/3228098750 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe3b8029560 con 0x55b4da31f070 2024-07-04T08:21:35.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.011+0000 7fe3cb758740 10 client.16941 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.011+0000 7fe3cb758740 10 client.16941 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:21:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.011+0000 7fe3cb758740 20 client.16941 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.011+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4da2dd9f0 0x55b4da6e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.011+0000 7fe3cb758740 4 client.16941 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.011+0000 7fe3cb758740 1 -- 172.21.15.39:0/3228098750 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4da63be50 con 0x55b4da2dd9f0 2024-07-04T08:21:35.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.011+0000 7fe3bde00640 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4da2dd9f0 0x55b4da6e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.012+0000 7fe3bde00640 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4da2dd9f0 0x55b4da6e15a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=323370b6d58cf0b1 server_cookie=bfa4c1d134d10cd4 in_seq=0 out_seq=0 2024-07-04T08:21:35.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.012+0000 7fe3a7e00640 10 client.16941 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:21:35.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.013+0000 7fe3a7e00640 1 -- 172.21.15.39:0/3228098750 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b4da63be50 con 0x55b4da2dd9f0 2024-07-04T08:21:35.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.013+0000 7fe3a7e00640 10 client.16941 handle_command_reply: tid=0 2024-07-04T08:21:35.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.013+0000 7fe3cb758740 1 client.16941 shutdown 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.013+0000 7fe3cb758740 1 -- 172.21.15.39:0/3228098750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 msgr2=0x55b4da6db2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.013+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6db2d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800fba0 tx=0x7fe3b8004ab0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3cb758740 1 -- 172.21.15.39:0/3228098750 shutdown_connections 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4da2dd9f0 0x55b4da6e15a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da31f070 0x55b4da6db2d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da2fe390 0x55b4da6cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3228098750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 0x55b4da6cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3cb758740 1 -- 172.21.15.39:0/3228098750 >> 172.21.15.39:0/3228098750 conn(0x55b4da313460 msgr2=0x55b4da6c69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3a7e00640 0 client.16941 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3cb758740 1 -- 172.21.15.39:0/3228098750 shutdown_connections 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3cb758740 1 -- 172.21.15.39:0/3228098750 wait complete. 2024-07-04T08:21:35.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.014+0000 7fe3cb758740 20 client.16941 trim_cache size 0 max 16384 2024-07-04T08:21:35.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.029+0000 7fe3cb758740 1 Processor -- start 2024-07-04T08:21:35.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.029+0000 7fe3cb758740 1 -- start start 2024-07-04T08:21:35.025 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:34 smithi039 ceph-mon[32062]: pgmap v671: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-07-04T08:21:35.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 0x55b4da6d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2fe390 0x55b4da6db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da6db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3cb758740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01b67b0 con 0x55b4da2fe390 2024-07-04T08:21:35.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3cb758740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01b82f0 con 0x55b4da31f070 2024-07-04T08:21:35.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3cb758740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01b6e80 con 0x55b4da2f61b0 2024-07-04T08:21:35.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3b5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2fe390 0x55b4da6db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da6db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3be800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da6db600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41108/0 (socket says 172.21.15.39:41108) 2024-07-04T08:21:35.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3b5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2fe390 0x55b4da6db0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44442/0 (socket says 172.21.15.39:44442) 2024-07-04T08:21:35.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3be800640 1 -- 172.21.15.39:0/3826594159 learned_addr learned my addr 172.21.15.39:0/3826594159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:35.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.030+0000 7fe3bde00640 1 --2- 172.21.15.39:0/3826594159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 0x55b4da6d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.031+0000 7fe3be800640 1 -- 172.21.15.39:0/3826594159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 msgr2=0x55b4da6d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:35.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.031+0000 7fe3be800640 1 --2- 172.21.15.39:0/3826594159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 0x55b4da6d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.031+0000 7fe3be800640 1 -- 172.21.15.39:0/3826594159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2fe390 msgr2=0x55b4da6db0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:35.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.031+0000 7fe3be800640 1 --2- 172.21.15.39:0/3826594159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2fe390 0x55b4da6db0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.031+0000 7fe3be800640 1 -- 172.21.15.39:0/3826594159 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4da6c6630 con 0x55b4da31f070 2024-07-04T08:21:35.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.031+0000 7fe3be800640 1 --2- 172.21.15.39:0/3826594159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da6db600 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8013010 tx=0x7fe3b800f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=974b87ed626639e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:35.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.031+0000 7fe3aea00640 1 -- 172.21.15.39:0/3826594159 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8029bc0 con 0x55b4da31f070 2024-07-04T08:21:35.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.032+0000 7fe3aea00640 1 -- 172.21.15.39:0/3826594159 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b8029d60 con 0x55b4da31f070 2024-07-04T08:21:35.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.032+0000 7fe3aea00640 1 -- 172.21.15.39:0/3826594159 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b801ec80 con 0x55b4da31f070 2024-07-04T08:21:35.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.032+0000 7fe3cb758740 1 -- 172.21.15.39:0/3826594159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 msgr2=0x55b4da6db600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:35.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.032+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3826594159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da6db600 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8013010 tx=0x7fe3b800f3c0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.032+0000 7fe3cb758740 1 -- 172.21.15.39:0/3826594159 shutdown_connections 2024-07-04T08:21:35.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.032+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3826594159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2fe390 0x55b4da6db0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.032+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3826594159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da6db600 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.032+0000 7fe3cb758740 1 --2- 172.21.15.39:0/3826594159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2f61b0 0x55b4da6d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.032+0000 7fe3cb758740 1 -- 172.21.15.39:0/3826594159 >> 172.21.15.39:0/3826594159 conn(0x55b4da313460 msgr2=0x55b4da6d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:35.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.033+0000 7fe3cb758740 1 -- 172.21.15.39:0/3826594159 shutdown_connections 2024-07-04T08:21:35.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.033+0000 7fe3cb758740 1 -- 172.21.15.39:0/3826594159 wait complete. 2024-07-04T08:21:35.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.033+0000 7fe3cb758740 1 -- start start 2024-07-04T08:21:35.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.033+0000 7fe3cb758740 1 Processor -- start 2024-07-04T08:21:35.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.034+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2f61b0 0x55b4da6d2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.034+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 0x55b4da6d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.034+0000 7fe3cb758740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da747d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.034+0000 7fe3cb758740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01b7c20 con 0x55b4da2f61b0 2024-07-04T08:21:35.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.034+0000 7fe3cb758740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01b7550 con 0x55b4da31f070 2024-07-04T08:21:35.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.034+0000 7fe3cb758740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01b4790 con 0x55b4da2fe390 2024-07-04T08:21:35.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.034+0000 7fe3cb758740 10 client.0 _ll_register_callbacks cb 0x7fe3b8009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:35.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.034+0000 7fe3bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2f61b0 0x55b4da6d2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.034+0000 7fe3bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2f61b0 0x55b4da6d2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44456/0 (socket says 172.21.15.39:44456) 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3bde00640 1 -- 172.21.15.39:0/2478220268 learned_addr learned my addr 172.21.15.39:0/2478220268 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3b5400640 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 0x55b4da6d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3be800640 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da747d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3bde00640 1 -- 172.21.15.39:0/2478220268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 msgr2=0x55b4da6d2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3bde00640 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 0x55b4da6d2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3bde00640 1 -- 172.21.15.39:0/2478220268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 msgr2=0x55b4da747d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3bde00640 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da747d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3bde00640 1 -- 172.21.15.39:0/2478220268 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4da7482a0 con 0x55b4da2f61b0 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3bde00640 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2f61b0 0x55b4da6d2600 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009380 tx=0x7fe3b0009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ea38109066fb2d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.035+0000 7fe3a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:35.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.036+0000 7fe3a7e00640 1 -- 172.21.15.39:0/2478220268 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0031060 con 0x55b4da2f61b0 2024-07-04T08:21:35.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.036+0000 7fe3a7e00640 1 -- 172.21.15.39:0/2478220268 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fe3b0011070 con 0x55b4da2f61b0 2024-07-04T08:21:35.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.036+0000 7fe3cb758740 1 -- 172.21.15.39:0/2478220268 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b8029d60 con 0x55b4da2f61b0 2024-07-04T08:21:35.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.036+0000 7fe3a7e00640 1 -- 172.21.15.39:0/2478220268 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3b0038050 con 0x55b4da2f61b0 2024-07-04T08:21:35.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.036+0000 7fe3cb758740 10 client.16947 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:35.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.036+0000 7fe3cb758740 1 -- 172.21.15.39:0/2478220268 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4da749370 con 0x55b4da2f61b0 2024-07-04T08:21:35.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.037+0000 7fe3a7e00640 1 -- 172.21.15.39:0/2478220268 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b8029d60 con 0x55b4da2f61b0 2024-07-04T08:21:35.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.037+0000 7fe3a7e00640 1 -- 172.21.15.39:0/2478220268 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe3b0003ab0 con 0x55b4da2f61b0 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.037+0000 7fe3cb758740 10 client.16947 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.037+0000 7fe3cb758740 10 client.16947 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.037+0000 7fe3cb758740 20 client.16947 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.037+0000 7fe3cb758740 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4da2b2da0 0x55b4da74b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.037+0000 7fe3cb758740 4 client.16947 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.037+0000 7fe3cb758740 1 -- 172.21.15.39:0/2478220268 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4da6378f0 con 0x55b4da2b2da0 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.038+0000 7fe3b5400640 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4da2b2da0 0x55b4da74b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.038+0000 7fe3b5400640 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4da2b2da0 0x55b4da74b940 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd81a5fc3f6d5221 server_cookie=ce3183b0f0b70828 in_seq=0 out_seq=0 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.038+0000 7fe3a7e00640 10 client.16947 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.038+0000 7fe3a7e00640 1 -- 172.21.15.39:0/2478220268 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4da6378f0 con 0x55b4da2b2da0 2024-07-04T08:21:35.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.038+0000 7fe3a7e00640 10 client.16947 handle_command_reply: tid=0 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.038+0000 7fe3cb758740 1 client.16947 shutdown 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 -- 172.21.15.39:0/2478220268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2f61b0 msgr2=0x55b4da6d2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2f61b0 0x55b4da6d2600 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009380 tx=0x7fe3b0009d90 comp rx=0 tx=0).stop 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 -- 172.21.15.39:0/2478220268 shutdown_connections 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b4da2b2da0 0x55b4da74b940 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b4da31f070 0x55b4da747d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4da2fe390 0x55b4da6d2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 --2- 172.21.15.39:0/2478220268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b4da2f61b0 0x55b4da6d2600 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 -- 172.21.15.39:0/2478220268 >> 172.21.15.39:0/2478220268 conn(0x55b4da313460 msgr2=0x55b4da6c5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3a7e00640 0 client.16947 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 -- 172.21.15.39:0/2478220268 shutdown_connections 2024-07-04T08:21:35.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 1 -- 172.21.15.39:0/2478220268 wait complete. 2024-07-04T08:21:35.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.039+0000 7fe3cb758740 20 client.16947 trim_cache size 0 max 16384 2024-07-04T08:21:35.038 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:21:35.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 -- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3a00b2a10 msgr2=0x7fe3a00b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3a00b2a10 0x7fe3a00b4f20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80125f0 tx=0x7fe3b8009500 comp rx=0 tx=0).stop 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 -- 172.21.15.39:0/3991139720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c0102490 msgr2=0x7fe3c0103e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c0102490 0x7fe3c0103e70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe3a802aa70 tx=0x7fe3a8003b20 comp rx=0 tx=0).stop 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 -- 172.21.15.39:0/3991139720 shutdown_connections 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe3a00b2a10 0x7fe3a00b4f20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3c010f6f0 0x7fe3c0104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe3c0102490 0x7fe3c0103e70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 --2- 172.21.15.39:0/3991139720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3c0101b40 0x7fe3c0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.040+0000 7fe3c5200640 1 -- 172.21.15.39:0/3991139720 >> 172.21.15.39:0/3991139720 conn(0x7fe3c0078c70 msgr2=0x7fe3c0108160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.041+0000 7fe3c5200640 1 -- 172.21.15.39:0/3991139720 shutdown_connections 2024-07-04T08:21:35.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:35.041+0000 7fe3c5200640 1 -- 172.21.15.39:0/3991139720 wait complete. 2024-07-04T08:21:35.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:34 smithi133 ceph-mon[37050]: pgmap v671: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-07-04T08:21:35.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:34 smithi050 ceph-mon[36577]: pgmap v671: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-07-04T08:21:35.932 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:21:36.436 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:37.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:36 smithi133 ceph-mon[37050]: pgmap v672: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-07-04T08:21:37.345 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:36 smithi039 ceph-mon[32062]: pgmap v672: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-07-04T08:21:37.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:36 smithi050 ceph-mon[36577]: pgmap v672: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-07-04T08:21:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.819+0000 7f7797400640 1 -- 172.21.15.39:0/2616487691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f779010f990 msgr2=0x7f77901161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.819+0000 7f7797400640 1 --2- 172.21.15.39:0/2616487691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f779010f990 0x7f77901161c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7788009ca0 tx=0x7f778802a8b0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.820+0000 7f7797400640 1 -- 172.21.15.39:0/2616487691 shutdown_connections 2024-07-04T08:21:38.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.820+0000 7f7797400640 1 --2- 172.21.15.39:0/2616487691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f779010f990 0x7f77901161c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.820+0000 7f7797400640 1 --2- 172.21.15.39:0/2616487691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779010f040 0x7f779010f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.820+0000 7f7797400640 1 --2- 172.21.15.39:0/2616487691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7790109700 0x7f7790109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.820+0000 7f7797400640 1 -- 172.21.15.39:0/2616487691 >> 172.21.15.39:0/2616487691 conn(0x7f7790077630 msgr2=0x7f7790107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:38.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.820+0000 7f7797400640 1 -- 172.21.15.39:0/2616487691 shutdown_connections 2024-07-04T08:21:38.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.820+0000 7f7797400640 1 -- 172.21.15.39:0/2616487691 wait complete. 2024-07-04T08:21:38.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.822+0000 7f7797400640 1 Processor -- start 2024-07-04T08:21:38.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.822+0000 7f7797400640 1 -- start start 2024-07-04T08:21:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.823+0000 7f7797400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7790109700 0x7f779019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.823+0000 7f7797400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f779010f040 0x7f77901a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.823+0000 7f7797400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779010f990 0x7f77901a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.823+0000 7f7797400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7790119690 con 0x7f779010f990 2024-07-04T08:21:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.823+0000 7f7797400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7790119510 con 0x7f779010f040 2024-07-04T08:21:38.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.823+0000 7f7797400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7790119810 con 0x7f7790109700 2024-07-04T08:21:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.823+0000 7f7794a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7790109700 0x7f779019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.823+0000 7f7795400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779010f990 0x7f77901a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.824+0000 7f7794a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7790109700 0x7f779019ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56340/0 (socket says 172.21.15.39:56340) 2024-07-04T08:21:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.824+0000 7f7794a00640 1 -- 172.21.15.39:0/2574659021 learned_addr learned my addr 172.21.15.39:0/2574659021 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.824+0000 7f778fe00640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f779010f040 0x7f77901a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.824+0000 7f7794a00640 1 -- 172.21.15.39:0/2574659021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f779010f040 msgr2=0x7f77901a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.824+0000 7f7794a00640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f779010f040 0x7f77901a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.824+0000 7f7794a00640 1 -- 172.21.15.39:0/2574659021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779010f990 msgr2=0x7f77901a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.824+0000 7f7794a00640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779010f990 0x7f77901a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.824+0000 7f7794a00640 1 -- 172.21.15.39:0/2574659021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77901a5ed0 con 0x7f7790109700 2024-07-04T08:21:38.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.824+0000 7f7794a00640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7790109700 0x7f779019ff60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:38.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.825+0000 7f778d600640 1 -- 172.21.15.39:0/2574659021 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778400cd80 con 0x7f7790109700 2024-07-04T08:21:38.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.825+0000 7f7797400640 1 -- 172.21.15.39:0/2574659021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77901a61c0 con 0x7f7790109700 2024-07-04T08:21:38.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.825+0000 7f778d600640 1 -- 172.21.15.39:0/2574659021 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77840108e0 con 0x7f7790109700 2024-07-04T08:21:38.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.825+0000 7f778d600640 1 -- 172.21.15.39:0/2574659021 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784018980 con 0x7f7790109700 2024-07-04T08:21:38.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.825+0000 7f7797400640 1 -- 172.21.15.39:0/2574659021 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77901ac920 con 0x7f7790109700 2024-07-04T08:21:38.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.828+0000 7f779da73740 1 Processor -- start 2024-07-04T08:21:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f779da73740 1 -- start start 2024-07-04T08:21:38.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f779da73740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4ca57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f779da73740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48d81b0 0x5613a4ca5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f779da73740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48e0390 0x5613a4ca6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f779da73740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5613a4c1bf00 con 0x5613a48d81b0 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f779da73740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5613a4c0b4a0 con 0x5613a4901070 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f779da73740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5613a4c0f4e0 con 0x5613a48e0390 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f778fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48e0390 0x5613a4ca6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f778fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48e0390 0x5613a4ca6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56348/0 (socket says 172.21.15.39:56348) 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f778fe00640 1 -- 172.21.15.39:0/3757426680 learned_addr learned my addr 172.21.15.39:0/3757426680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.829+0000 7f778d600640 1 -- 172.21.15.39:0/2574659021 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7784010430 con 0x7f7790109700 2024-07-04T08:21:38.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.830+0000 7f7795400640 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4ca57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.830+0000 7f778fe00640 1 -- 172.21.15.39:0/3757426680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 msgr2=0x5613a4ca57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:38.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.830+0000 7f778fe00640 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4ca57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.830+0000 7f778fe00640 1 -- 172.21.15.39:0/3757426680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48d81b0 msgr2=0x5613a4ca5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.830+0000 7f7794a00640 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48d81b0 0x5613a4ca5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.830+0000 7f778fe00640 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48d81b0 0x5613a4ca5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.830+0000 7f778fe00640 1 -- 172.21.15.39:0/3757426680 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613a4ba3670 con 0x5613a48e0390 2024-07-04T08:21:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.830+0000 7f7794a00640 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48d81b0 0x5613a4ca5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:21:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.831+0000 7f778fe00640 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48e0390 0x5613a4ca6250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7780009c20 tx=0x7f7780011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d55b758b7999869 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.831+0000 7f777e000640 1 -- 172.21.15.39:0/3757426680 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780004030 con 0x5613a48e0390 2024-07-04T08:21:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.831+0000 7f777e000640 1 -- 172.21.15.39:0/3757426680 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7780005020 con 0x5613a48e0390 2024-07-04T08:21:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.831+0000 7f777e000640 1 -- 172.21.15.39:0/3757426680 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780004d80 con 0x5613a48e0390 2024-07-04T08:21:38.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.831+0000 7f778d600640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2960 0x7f77780b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f778d600640 1 -- 172.21.15.39:0/2574659021 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f77840e2cd0 con 0x7f7790109700 2024-07-04T08:21:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f779da73740 1 -- 172.21.15.39:0/3757426680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48e0390 msgr2=0x5613a4ca6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:38.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f779da73740 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48e0390 0x5613a4ca6250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7780009c20 tx=0x7f7780011810 comp rx=0 tx=0).stop 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f7795400640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2960 0x7f77780b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f779da73740 1 -- 172.21.15.39:0/3757426680 shutdown_connections 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f779da73740 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48d81b0 0x5613a4ca5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f779da73740 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48e0390 0x5613a4ca6250 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f779da73740 1 --2- 172.21.15.39:0/3757426680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4ca57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f779da73740 1 -- 172.21.15.39:0/3757426680 >> 172.21.15.39:0/3757426680 conn(0x5613a48f5460 msgr2=0x5613a4ca36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f779da73740 1 -- 172.21.15.39:0/3757426680 shutdown_connections 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.832+0000 7f779da73740 1 -- 172.21.15.39:0/3757426680 wait complete. 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.833+0000 7f7795400640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2960 0x7f77780b4e70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7788006010 tx=0x7f7788005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.833+0000 7f779da73740 1 -- start start 2024-07-04T08:21:38.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.833+0000 7f779da73740 1 Processor -- start 2024-07-04T08:21:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f779da73740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 0x5613a4cb1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f779da73740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f779da73740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4cbd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f779da73740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5613a4a6f700 con 0x5613a48e0390 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f779da73740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5613a4bb2bd0 con 0x5613a4901070 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f779da73740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5613a4cb3fb0 con 0x5613a48d81b0 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f779da73740 10 client.0 _ll_register_callbacks cb 0x5613a4c04590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f778fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f7794a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 0x5613a4cb1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f778fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44492/0 (socket says 172.21.15.39:44492) 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f778fe00640 1 -- 172.21.15.39:0/3981339060 learned_addr learned my addr 172.21.15.39:0/3981339060 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.834+0000 7f7795400640 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4cbd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.835+0000 7f7794a00640 1 -- 172.21.15.39:0/3981339060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 msgr2=0x5613a4cbd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.835+0000 7f7794a00640 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4cbd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.835+0000 7f7794a00640 1 -- 172.21.15.39:0/3981339060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 msgr2=0x5613a4cb1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.835+0000 7f7794a00640 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.835+0000 7f7794a00640 1 -- 172.21.15.39:0/3981339060 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613a496c650 con 0x5613a48d81b0 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.835+0000 7f7794a00640 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 0x5613a4cb1330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77840e0090 tx=0x7f77840a9500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12513845db466aad server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.835+0000 7f7772a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.836+0000 7f7772a00640 1 -- 172.21.15.39:0/3981339060 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840e0520 con 0x5613a48d81b0 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.836+0000 7f7772a00640 1 -- 172.21.15.39:0/3981339060 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f77840a8f30 con 0x5613a48d81b0 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.836+0000 7f779da73740 1 -- 172.21.15.39:0/3981339060 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7780005020 con 0x5613a48d81b0 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.836+0000 7f7772a00640 1 -- 172.21.15.39:0/3981339060 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77840adfd0 con 0x5613a48d81b0 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.836+0000 7f779da73740 10 client.27007 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.836+0000 7f779da73740 1 -- 172.21.15.39:0/3981339060 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5613a4cbff30 con 0x5613a48d81b0 2024-07-04T08:21:38.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.836+0000 7f7772a00640 1 -- 172.21.15.39:0/3981339060 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7780005020 con 0x5613a48d81b0 2024-07-04T08:21:38.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.836+0000 7f7772a00640 1 -- 172.21.15.39:0/3981339060 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f77840a5640 con 0x5613a48d81b0 2024-07-04T08:21:38.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.837+0000 7f779da73740 10 client.27007 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:38.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.837+0000 7f779da73740 10 client.27007 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:21:38.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.837+0000 7f779da73740 20 client.27007 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:38.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.837+0000 7f779da73740 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5613a48bf9f0 0x5613a4cc3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.837+0000 7f779da73740 4 client.27007 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.837+0000 7f779da73740 1 -- 172.21.15.39:0/3981339060 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613a4c1de50 con 0x5613a48bf9f0 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.837+0000 7f778fe00640 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5613a48bf9f0 0x5613a4cc3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.838+0000 7f778fe00640 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5613a48bf9f0 0x5613a4cc3440 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=741a7e6f9e5d5a11 server_cookie=8bac9231aeb3f19f in_seq=0 out_seq=0 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.838+0000 7f7772a00640 10 client.27007 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.838+0000 7f7772a00640 1 -- 172.21.15.39:0/3981339060 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5613a4c1de50 con 0x5613a48bf9f0 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.838+0000 7f7772a00640 10 client.27007 handle_command_reply: tid=0 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 client.27007 shutdown 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 -- 172.21.15.39:0/3981339060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 msgr2=0x5613a4cb1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 0x5613a4cb1330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77840e0090 tx=0x7f77840a9500 comp rx=0 tx=0).stop 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 -- 172.21.15.39:0/3981339060 shutdown_connections 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5613a48bf9f0 0x5613a4cc3440 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4cbd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 --2- 172.21.15.39:0/3981339060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 0x5613a4cb1330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 -- 172.21.15.39:0/3981339060 >> 172.21.15.39:0/3981339060 conn(0x5613a48f5460 msgr2=0x5613a4ca89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:38.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f7772a00640 0 client.27007 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:21:38.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 -- 172.21.15.39:0/3981339060 shutdown_connections 2024-07-04T08:21:38.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 1 -- 172.21.15.39:0/3981339060 wait complete. 2024-07-04T08:21:38.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.839+0000 7f779da73740 20 client.27007 trim_cache size 0 max 16384 2024-07-04T08:21:38.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.855+0000 7f779da73740 1 Processor -- start 2024-07-04T08:21:38.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.855+0000 7f779da73740 1 -- start start 2024-07-04T08:21:38.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f779da73740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a48d81b0 0x5613a4ca8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f779da73740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4caa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f779da73740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a4901070 0x5613a4caadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f779da73740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b6730 con 0x5613a48e0390 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f779da73740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b8270 con 0x5613a48d81b0 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f779da73740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b6e00 con 0x5613a4901070 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f7794a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a48d81b0 0x5613a4ca8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f7794a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a48d81b0 0x5613a4ca8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41172/0 (socket says 172.21.15.39:41172) 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f7794a00640 1 -- 172.21.15.39:0/2430321975 learned_addr learned my addr 172.21.15.39:0/2430321975 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f778fe00640 1 --2- 172.21.15.39:0/2430321975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4caa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.856+0000 7f7795400640 1 --2- 172.21.15.39:0/2430321975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a4901070 0x5613a4caadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.857+0000 7f7794a00640 1 -- 172.21.15.39:0/2430321975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a4901070 msgr2=0x5613a4caadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:38.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.857+0000 7f7794a00640 1 --2- 172.21.15.39:0/2430321975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a4901070 0x5613a4caadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.857+0000 7f7794a00640 1 -- 172.21.15.39:0/2430321975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 msgr2=0x5613a4caa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:38.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.857+0000 7f7794a00640 1 --2- 172.21.15.39:0/2430321975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4caa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.857+0000 7f7794a00640 1 -- 172.21.15.39:0/2430321975 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613a4cb23f0 con 0x5613a48d81b0 2024-07-04T08:21:38.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.857+0000 7f7794a00640 1 --2- 172.21.15.39:0/2430321975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a48d81b0 0x5613a4ca8fe0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f77840ae190 tx=0x7f77840a9570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8509bf7c71d0637f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:38.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.857+0000 7f777d600640 1 -- 172.21.15.39:0/2430321975 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840a9620 con 0x5613a48d81b0 2024-07-04T08:21:38.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.858+0000 7f777d600640 1 -- 172.21.15.39:0/2430321975 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77840a89a0 con 0x5613a48d81b0 2024-07-04T08:21:38.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.858+0000 7f777d600640 1 -- 172.21.15.39:0/2430321975 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784018b20 con 0x5613a48d81b0 2024-07-04T08:21:38.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.858+0000 7f779da73740 1 -- 172.21.15.39:0/2430321975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a48d81b0 msgr2=0x5613a4ca8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.858+0000 7f779da73740 1 --2- 172.21.15.39:0/2430321975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a48d81b0 0x5613a4ca8fe0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f77840ae190 tx=0x7f77840a9570 comp rx=0 tx=0).stop 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.859+0000 7f779da73740 1 -- 172.21.15.39:0/2430321975 shutdown_connections 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.859+0000 7f779da73740 1 --2- 172.21.15.39:0/2430321975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4caa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.859+0000 7f779da73740 1 --2- 172.21.15.39:0/2430321975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a4901070 0x5613a4caadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.859+0000 7f779da73740 1 --2- 172.21.15.39:0/2430321975 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a48d81b0 0x5613a4ca8fe0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.859+0000 7f779da73740 1 -- 172.21.15.39:0/2430321975 >> 172.21.15.39:0/2430321975 conn(0x5613a48f5460 msgr2=0x5613a4ca88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.859+0000 7f779da73740 1 -- 172.21.15.39:0/2430321975 shutdown_connections 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.859+0000 7f779da73740 1 -- 172.21.15.39:0/2430321975 wait complete. 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.859+0000 7f779da73740 1 -- start start 2024-07-04T08:21:38.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.860+0000 7f779da73740 1 Processor -- start 2024-07-04T08:21:38.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.860+0000 7f779da73740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 0x5613a4cb5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.860+0000 7f779da73740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.860+0000 7f779da73740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4cb5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.860+0000 7f779da73740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b7ba0 con 0x5613a48e0390 2024-07-04T08:21:38.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.860+0000 7f779da73740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b74d0 con 0x5613a4901070 2024-07-04T08:21:38.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.860+0000 7f779da73740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77901b4710 con 0x5613a48d81b0 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.860+0000 7f779da73740 10 client.0 _ll_register_callbacks cb 0x7f778400c7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.860+0000 7f778fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.861+0000 7f778fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44514/0 (socket says 172.21.15.39:44514) 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.861+0000 7f778fe00640 1 -- 172.21.15.39:0/958494636 learned_addr learned my addr 172.21.15.39:0/958494636 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.861+0000 7f7795400640 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4cb5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.861+0000 7f7794a00640 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 0x5613a4cb5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.861+0000 7f778fe00640 1 -- 172.21.15.39:0/958494636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 msgr2=0x5613a4cb5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.861+0000 7f778fe00640 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 0x5613a4cb5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.861+0000 7f778fe00640 1 -- 172.21.15.39:0/958494636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 msgr2=0x5613a4cb5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.861+0000 7f778fe00640 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4cb5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.861+0000 7f778fe00640 1 -- 172.21.15.39:0/958494636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613a4d2abf0 con 0x5613a48e0390 2024-07-04T08:21:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.862+0000 7f778fe00640 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb5580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7780005b90 tx=0x7f7780009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613a67f27e14cf93 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.862+0000 7f7772a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:38.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.862+0000 7f7772a00640 1 -- 172.21.15.39:0/958494636 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780034060 con 0x5613a48e0390 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.862+0000 7f779da73740 1 -- 172.21.15.39:0/958494636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77840a89a0 con 0x5613a48e0390 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.862+0000 7f7772a00640 1 -- 172.21.15.39:0/958494636 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7780004450 con 0x5613a48e0390 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.862+0000 7f7772a00640 1 -- 172.21.15.39:0/958494636 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f778003b050 con 0x5613a48e0390 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.862+0000 7f779da73740 10 client.16962 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.862+0000 7f779da73740 1 -- 172.21.15.39:0/958494636 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5613a4d2bde0 con 0x5613a48e0390 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.863+0000 7f7772a00640 1 -- 172.21.15.39:0/958494636 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77840a89a0 con 0x5613a48e0390 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.863+0000 7f7772a00640 1 -- 172.21.15.39:0/958494636 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7780018070 con 0x5613a48e0390 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.863+0000 7f779da73740 10 client.16962 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.863+0000 7f779da73740 10 client.16962 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.863+0000 7f779da73740 20 client.16962 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.863+0000 7f779da73740 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5613a4894da0 0x5613a4d2e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.863+0000 7f779da73740 4 client.16962 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:21:38.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.863+0000 7f779da73740 1 -- 172.21.15.39:0/958494636 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613a4c198f0 con 0x5613a4894da0 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.864+0000 7f7794a00640 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5613a4894da0 0x5613a4d2e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.864+0000 7f7794a00640 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5613a4894da0 0x5613a4d2e3b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff381b767c496bae server_cookie=e4f065e431a4dce0 in_seq=0 out_seq=0 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.864+0000 7f7772a00640 10 client.16962 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f7772a00640 1 -- 172.21.15.39:0/958494636 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613a4c198f0 con 0x5613a4894da0 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f7772a00640 10 client.16962 handle_command_reply: tid=0 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 client.16962 shutdown 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 -- 172.21.15.39:0/958494636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 msgr2=0x5613a4cb5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb5580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7780005b90 tx=0x7f7780009ef0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 -- 172.21.15.39:0/958494636 shutdown_connections 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5613a4894da0 0x5613a4d2e3b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5613a48e0390 0x5613a4cb5580 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5613a4901070 0x5613a4cb5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f7772a00640 0 client.16962 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:21:38.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 --2- 172.21.15.39:0/958494636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5613a48d81b0 0x5613a4cb5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 -- 172.21.15.39:0/958494636 >> 172.21.15.39:0/958494636 conn(0x5613a48f5460 msgr2=0x5613a4cb25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:38.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 -- 172.21.15.39:0/958494636 shutdown_connections 2024-07-04T08:21:38.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.865+0000 7f779da73740 1 -- 172.21.15.39:0/958494636 wait complete. 2024-07-04T08:21:38.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.866+0000 7f779da73740 20 client.16962 trim_cache size 0 max 16384 2024-07-04T08:21:38.862 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:21:38.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.866+0000 7f7797400640 1 -- 172.21.15.39:0/2574659021 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2960 msgr2=0x7f77780b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:38.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.866+0000 7f7797400640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2960 0x7f77780b4e70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7788006010 tx=0x7f7788005d80 comp rx=0 tx=0).stop 2024-07-04T08:21:38.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.866+0000 7f7797400640 1 -- 172.21.15.39:0/2574659021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7790109700 msgr2=0x7f779019ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:38.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.866+0000 7f7797400640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7790109700 0x7f779019ff60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.867+0000 7f7797400640 1 -- 172.21.15.39:0/2574659021 shutdown_connections 2024-07-04T08:21:38.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.867+0000 7f7797400640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f77780b2960 0x7f77780b4e70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.867+0000 7f7797400640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f779010f990 0x7f77901a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.867+0000 7f7797400640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f779010f040 0x7f77901a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.867+0000 7f7797400640 1 --2- 172.21.15.39:0/2574659021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7790109700 0x7f779019ff60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:38.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.867+0000 7f7797400640 1 -- 172.21.15.39:0/2574659021 >> 172.21.15.39:0/2574659021 conn(0x7f7790077630 msgr2=0x7f7790077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:38.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.867+0000 7f7797400640 1 -- 172.21.15.39:0/2574659021 shutdown_connections 2024-07-04T08:21:38.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:38.867+0000 7f7797400640 1 -- 172.21.15.39:0/2574659021 wait complete. 2024-07-04T08:21:39.261 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:38 smithi039 ceph-mon[32062]: pgmap v673: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:21:39.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:38 smithi133 ceph-mon[37050]: pgmap v673: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:21:39.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:38 smithi050 ceph-mon[36577]: pgmap v673: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:21:39.650 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:21:40.141 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:40.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:21:41.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:41 smithi133 ceph-mon[37050]: pgmap v674: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-07-04T08:21:41.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:41 smithi039 ceph-mon[32062]: pgmap v674: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-07-04T08:21:41.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:41 smithi050 ceph-mon[36577]: pgmap v674: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-07-04T08:21:41.606 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.068+0000 7f60e1e00640 1 -- 172.21.15.39:0/4194329540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60dc10b590 msgr2=0x7f60dc111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.068+0000 7f60e1e00640 1 --2- 172.21.15.39:0/4194329540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60dc10b590 0x7f60dc111dc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009900 tx=0x7f60d402a410 comp rx=0 tx=0).stop 2024-07-04T08:21:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.068+0000 7f60e1e00640 1 -- 172.21.15.39:0/4194329540 shutdown_connections 2024-07-04T08:21:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.068+0000 7f60e1e00640 1 --2- 172.21.15.39:0/4194329540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60dc10b590 0x7f60dc111dc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.068+0000 7f60e1e00640 1 --2- 172.21.15.39:0/4194329540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60dc10ac40 0x7f60dc10b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.068+0000 7f60e1e00640 1 --2- 172.21.15.39:0/4194329540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc105300 0x7f60dc105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.068+0000 7f60e1e00640 1 -- 172.21.15.39:0/4194329540 >> 172.21.15.39:0/4194329540 conn(0x7f60dc100d10 msgr2=0x7f60dc103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.069+0000 7f60e1e00640 1 -- 172.21.15.39:0/4194329540 shutdown_connections 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.069+0000 7f60e1e00640 1 -- 172.21.15.39:0/4194329540 wait complete. 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.070+0000 7f60e1e00640 1 Processor -- start 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.070+0000 7f60e1e00640 1 -- start start 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.070+0000 7f60e1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc105300 0x7f60dc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.070+0000 7f60e1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60dc10ac40 0x7f60dc19c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.070+0000 7f60e1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60dc10b590 0x7f60dc1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.070+0000 7f60e1e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc115290 con 0x7f60dc10b590 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.070+0000 7f60e1e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc115110 con 0x7f60dc10ac40 2024-07-04T08:21:43.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.070+0000 7f60e1e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc115410 con 0x7f60dc105300 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60daa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc105300 0x7f60dc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60dc10b590 0x7f60dc1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60da000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60dc10ac40 0x7f60dc19c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60daa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc105300 0x7f60dc19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36540/0 (socket says 172.21.15.39:36540) 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60daa00640 1 -- 172.21.15.39:0/899801951 learned_addr learned my addr 172.21.15.39:0/899801951 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60db400640 1 -- 172.21.15.39:0/899801951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc105300 msgr2=0x7f60dc19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60db400640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc105300 0x7f60dc19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60db400640 1 -- 172.21.15.39:0/899801951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60dc10ac40 msgr2=0x7f60dc19c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60db400640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60dc10ac40 0x7f60dc19c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60db400640 1 -- 172.21.15.39:0/899801951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc1a1b10 con 0x7f60dc10b590 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.071+0000 7f60daa00640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc105300 0x7f60dc19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.072+0000 7f60db400640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60dc10b590 0x7f60dc1a13e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f60d402a930 tx=0x7f60d4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.072+0000 7f60e0a00640 1 -- 172.21.15.39:0/899801951 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d403f070 con 0x7f60dc10b590 2024-07-04T08:21:43.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.072+0000 7f60e0a00640 1 -- 172.21.15.39:0/899801951 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60d4031b50 con 0x7f60dc10b590 2024-07-04T08:21:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.072+0000 7f60e1e00640 1 -- 172.21.15.39:0/899801951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc1a1da0 con 0x7f60dc10b590 2024-07-04T08:21:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.072+0000 7f60e0a00640 1 -- 172.21.15.39:0/899801951 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4035030 con 0x7f60dc10b590 2024-07-04T08:21:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.072+0000 7f60e1e00640 1 -- 172.21.15.39:0/899801951 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc1a8560 con 0x7f60dc10b590 2024-07-04T08:21:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.076+0000 7f60e84da740 1 Processor -- start 2024-07-04T08:21:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.076+0000 7f60e0a00640 1 -- 172.21.15.39:0/899801951 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f60d4042070 con 0x7f60dc10b590 2024-07-04T08:21:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.077+0000 7f60e84da740 1 -- start start 2024-07-04T08:21:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.077+0000 7f60e84da740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326ae57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.077+0000 7f60e84da740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573267181b0 0x557326ae5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.077+0000 7f60e84da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326ae6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.077+0000 7f60e84da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557326a5bee0 con 0x557326720390 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.077+0000 7f60e84da740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557326a4b480 con 0x557326741070 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.077+0000 7f60e84da740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557326a4f4c0 con 0x5573267181b0 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60daa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573267181b0 0x557326ae5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60daa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573267181b0 0x557326ae5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36554/0 (socket says 172.21.15.39:36554) 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60daa00640 1 -- 172.21.15.39:0/1192321047 learned_addr learned my addr 172.21.15.39:0/1192321047 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60da000640 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326ae57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60da000640 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326ae6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60daa00640 1 -- 172.21.15.39:0/1192321047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 msgr2=0x557326ae57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60daa00640 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326ae57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60daa00640 1 -- 172.21.15.39:0/1192321047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 msgr2=0x557326ae6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60daa00640 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326ae6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60daa00640 1 -- 172.21.15.39:0/1192321047 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573269e3650 con 0x5573267181b0 2024-07-04T08:21:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.078+0000 7f60da000640 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326ae6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.079+0000 7f60daa00640 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573267181b0 0x557326ae5cf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00f8d0 tx=0x7f60cc00fd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b3093d5b7b19b67 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:43.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.079+0000 7f60e0a00640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60c00b29e0 0x7f60c00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.290 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:43 smithi039 ceph-mon[32062]: pgmap v675: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:21:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.079+0000 7f60d0c00640 1 -- 172.21.15.39:0/1192321047 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc011c20 con 0x5573267181b0 2024-07-04T08:21:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.079+0000 7f60e0a00640 1 -- 172.21.15.39:0/899801951 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f60d4102e70 con 0x7f60dc10b590 2024-07-04T08:21:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.079+0000 7f60d0c00640 1 -- 172.21.15.39:0/1192321047 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc011dc0 con 0x5573267181b0 2024-07-04T08:21:43.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.079+0000 7f60da000640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60c00b29e0 0x7f60c00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.079+0000 7f60d0c00640 1 -- 172.21.15.39:0/1192321047 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0186a0 con 0x5573267181b0 2024-07-04T08:21:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60e84da740 1 -- 172.21.15.39:0/1192321047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573267181b0 msgr2=0x557326ae5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60da000640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60c00b29e0 0x7f60c00b4ef0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009730 tx=0x7f60c8013040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60e84da740 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573267181b0 0x557326ae5cf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00f8d0 tx=0x7f60cc00fd90 comp rx=0 tx=0).stop 2024-07-04T08:21:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60e84da740 1 -- 172.21.15.39:0/1192321047 shutdown_connections 2024-07-04T08:21:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60e84da740 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326ae6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60e84da740 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5573267181b0 0x557326ae5cf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60e84da740 1 --2- 172.21.15.39:0/1192321047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326ae57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60e84da740 1 -- 172.21.15.39:0/1192321047 >> 172.21.15.39:0/1192321047 conn(0x557326735460 msgr2=0x557326ae36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60e84da740 1 -- 172.21.15.39:0/1192321047 shutdown_connections 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.080+0000 7f60e84da740 1 -- 172.21.15.39:0/1192321047 wait complete. 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.081+0000 7f60e84da740 1 -- start start 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.081+0000 7f60e84da740 1 Processor -- start 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60e84da740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5573267181b0 0x557326af1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60e84da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326af1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60e84da740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326741070 0x557326afd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60e84da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5573268af700 con 0x557326720390 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60e84da740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5573269f2bb0 con 0x5573267181b0 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60e84da740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557326af3f90 con 0x557326741070 2024-07-04T08:21:43.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60e84da740 10 client.0 _ll_register_callbacks cb 0x557326a44570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60da000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326af1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60db400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326741070 0x557326afd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60da000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326af1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33810/0 (socket says 172.21.15.39:33810) 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60da000640 1 -- 172.21.15.39:0/2266637139 learned_addr learned my addr 172.21.15.39:0/2266637139 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.082+0000 7f60daa00640 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5573267181b0 0x557326af1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60db400640 1 -- 172.21.15.39:0/2266637139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5573267181b0 msgr2=0x557326af1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60db400640 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5573267181b0 0x557326af1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60db400640 1 -- 172.21.15.39:0/2266637139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 msgr2=0x557326af1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60db400640 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326af1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60db400640 1 -- 172.21.15.39:0/2266637139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573267ac650 con 0x557326741070 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60db400640 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326741070 0x557326afd2e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f60d4100230 tx=0x7f60d40c97c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=230f05c8524dab12 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60c5600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:21:43.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60c5600640 1 -- 172.21.15.39:0/2266637139 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4049070 con 0x557326741070 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60c5600640 1 -- 172.21.15.39:0/2266637139 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f60d41004b0 con 0x557326741070 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.083+0000 7f60e84da740 1 -- 172.21.15.39:0/2266637139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60cc011dc0 con 0x557326741070 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60c5600640 1 -- 172.21.15.39:0/2266637139 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60d40ce170 con 0x557326741070 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60e84da740 10 client.27025 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60e84da740 1 -- 172.21.15.39:0/2266637139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557326affeb0 con 0x557326741070 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60c5600640 1 -- 172.21.15.39:0/2266637139 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc011dc0 con 0x557326741070 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60c5600640 1 -- 172.21.15.39:0/2266637139 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f60d40ca0d0 con 0x557326741070 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60e84da740 10 client.27025 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60e84da740 10 client.27025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60e84da740 20 client.27025 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:43.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60e84da740 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5573266ff9f0 0x557326b03450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60e84da740 4 client.27025 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.084+0000 7f60e84da740 1 -- 172.21.15.39:0/2266637139 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557326a5de30 con 0x5573266ff9f0 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.085+0000 7f60daa00640 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5573266ff9f0 0x557326b03450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.086+0000 7f60daa00640 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5573266ff9f0 0x557326b03450 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ed33da6619e63ad server_cookie=e710e63470b327d4 in_seq=0 out_seq=0 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.086+0000 7f60c5600640 10 client.27025 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60c5600640 1 -- 172.21.15.39:0/2266637139 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557326a5de30 con 0x5573266ff9f0 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60c5600640 10 client.27025 handle_command_reply: tid=0 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 client.27025 shutdown 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 -- 172.21.15.39:0/2266637139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326741070 msgr2=0x557326afd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326741070 0x557326afd2e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f60d4100230 tx=0x7f60d40c97c0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 -- 172.21.15.39:0/2266637139 shutdown_connections 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5573266ff9f0 0x557326b03450 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557326720390 0x557326af1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326741070 0x557326afd2e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60c5600640 0 client.27025 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 --2- 172.21.15.39:0/2266637139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5573267181b0 0x557326af1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 -- 172.21.15.39:0/2266637139 >> 172.21.15.39:0/2266637139 conn(0x557326735460 msgr2=0x557326ae8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 -- 172.21.15.39:0/2266637139 shutdown_connections 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.087+0000 7f60e84da740 1 -- 172.21.15.39:0/2266637139 wait complete. 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.088+0000 7f60e84da740 20 client.27025 trim_cache size 0 max 16384 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.103+0000 7f60e84da740 1 Processor -- start 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60e84da740 1 -- start start 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60e84da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326ae2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60e84da740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 0x557326ae36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60e84da740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326aea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60e84da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b23e0 con 0x5573267181b0 2024-07-04T08:21:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60e84da740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b3f20 con 0x557326741070 2024-07-04T08:21:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60e84da740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b2ab0 con 0x557326720390 2024-07-04T08:21:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326ae2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326ae2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33818/0 (socket says 172.21.15.39:33818) 2024-07-04T08:21:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.104+0000 7f60daa00640 1 -- 172.21.15.39:0/958867234 learned_addr learned my addr 172.21.15.39:0/958867234 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.105+0000 7f60db400640 1 --2- 172.21.15.39:0/958867234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326aea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.105+0000 7f60da000640 1 --2- 172.21.15.39:0/958867234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 0x557326ae36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.105+0000 7f60daa00640 1 -- 172.21.15.39:0/958867234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 msgr2=0x557326ae36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.105+0000 7f60daa00640 1 --2- 172.21.15.39:0/958867234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 0x557326ae36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.105+0000 7f60daa00640 1 -- 172.21.15.39:0/958867234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 msgr2=0x557326aea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.105+0000 7f60daa00640 1 --2- 172.21.15.39:0/958867234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326aea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.105+0000 7f60daa00640 1 -- 172.21.15.39:0/958867234 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557326af1970 con 0x5573267181b0 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.105+0000 7f60daa00640 1 --2- 172.21.15.39:0/958867234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326ae2940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f60cc019f60 tx=0x7f60cc019ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cccc5247c1aaa5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.106+0000 7f60c7e00640 1 -- 172.21.15.39:0/958867234 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc003b70 con 0x5573267181b0 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.106+0000 7f60c7e00640 1 -- 172.21.15.39:0/958867234 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc003d10 con 0x5573267181b0 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.106+0000 7f60c7e00640 1 -- 172.21.15.39:0/958867234 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0198b0 con 0x5573267181b0 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.106+0000 7f60e84da740 1 -- 172.21.15.39:0/958867234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 msgr2=0x557326ae2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.106+0000 7f60e84da740 1 --2- 172.21.15.39:0/958867234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326ae2940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f60cc019f60 tx=0x7f60cc019ef0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.107+0000 7f60e84da740 1 -- 172.21.15.39:0/958867234 shutdown_connections 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.107+0000 7f60e84da740 1 --2- 172.21.15.39:0/958867234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326aea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.107+0000 7f60e84da740 1 --2- 172.21.15.39:0/958867234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 0x557326ae36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.107+0000 7f60e84da740 1 --2- 172.21.15.39:0/958867234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326ae2940 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.107+0000 7f60e84da740 1 -- 172.21.15.39:0/958867234 >> 172.21.15.39:0/958867234 conn(0x557326735460 msgr2=0x557326ae2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.107+0000 7f60e84da740 1 -- 172.21.15.39:0/958867234 shutdown_connections 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.107+0000 7f60e84da740 1 -- 172.21.15.39:0/958867234 wait complete. 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.107+0000 7f60e84da740 1 -- start start 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60e84da740 1 Processor -- start 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60e84da740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326af94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60e84da740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 0x557326af9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60e84da740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326b08310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60e84da740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b3850 con 0x5573267181b0 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60e84da740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b3180 con 0x557326741070 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60e84da740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc1b03c0 con 0x557326720390 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60e84da740 10 client.0 _ll_register_callbacks cb 0x7f60d4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60da000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 0x557326af9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.108+0000 7f60db400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326b08310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60db400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326b08310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52116/0 (socket says 172.21.15.39:52116) 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60da000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 0x557326af9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36580/0 (socket says 172.21.15.39:36580) 2024-07-04T08:21:43.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60db400640 1 -- 172.21.15.39:0/2146609765 learned_addr learned my addr 172.21.15.39:0/2146609765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:43.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60daa00640 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326af94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60db400640 1 -- 172.21.15.39:0/2146609765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 msgr2=0x557326af9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:43.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60db400640 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 0x557326af9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60db400640 1 -- 172.21.15.39:0/2146609765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 msgr2=0x557326af94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:43.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60db400640 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326af94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60db400640 1 -- 172.21.15.39:0/2146609765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557326b08a80 con 0x557326741070 2024-07-04T08:21:43.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60db400640 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326b08310 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f60d40c9b70 tx=0x7f60d40c98f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a9a1df06de112bc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:43.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60c5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:21:43.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.109+0000 7f60c5600640 1 -- 172.21.15.39:0/2146609765 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4049040 con 0x557326741070 2024-07-04T08:21:43.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60e84da740 1 -- 172.21.15.39:0/2146609765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60cc003d10 con 0x557326741070 2024-07-04T08:21:43.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60c5600640 1 -- 172.21.15.39:0/2146609765 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f60d40d2790 con 0x557326741070 2024-07-04T08:21:43.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60c5600640 1 -- 172.21.15.39:0/2146609765 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60d40c95e0 con 0x557326741070 2024-07-04T08:21:43.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60e84da740 10 client.27122 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:43.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60e84da740 1 -- 172.21.15.39:0/2146609765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557326af8d40 con 0x557326741070 2024-07-04T08:21:43.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60c5600640 1 -- 172.21.15.39:0/2146609765 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60cc003d10 con 0x557326741070 2024-07-04T08:21:43.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60c5600640 1 -- 172.21.15.39:0/2146609765 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f60d40c90d0 con 0x557326741070 2024-07-04T08:21:43.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60e84da740 10 client.27122 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:43.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60e84da740 10 client.27122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:43.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60e84da740 20 client.27122 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:43.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60e84da740 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5573266d4da0 0x557326b6dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:43.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60e84da740 4 client.27122 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:21:43.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.110+0000 7f60e84da740 1 -- 172.21.15.39:0/2146609765 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557326a598d0 con 0x5573266d4da0 2024-07-04T08:21:43.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.111+0000 7f60daa00640 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5573266d4da0 0x557326b6dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:43.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.111+0000 7f60daa00640 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5573266d4da0 0x557326b6dcd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b1edd32c21d77a0 server_cookie=7a105e02a8d87b52 in_seq=0 out_seq=0 2024-07-04T08:21:43.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.111+0000 7f60c5600640 10 client.27122 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:43.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60c5600640 1 -- 172.21.15.39:0/2146609765 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557326a598d0 con 0x5573266d4da0 2024-07-04T08:21:43.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60c5600640 10 client.27122 handle_command_reply: tid=0 2024-07-04T08:21:43.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 client.27122 shutdown 2024-07-04T08:21:43.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 -- 172.21.15.39:0/2146609765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 msgr2=0x557326b08310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:43.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326b08310 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f60d40c9b70 tx=0x7f60d40c98f0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 -- 172.21.15.39:0/2146609765 shutdown_connections 2024-07-04T08:21:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5573266d4da0 0x557326b6dcd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557326741070 0x557326b08310 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557326720390 0x557326af9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 --2- 172.21.15.39:0/2146609765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5573267181b0 0x557326af94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60c5600640 0 client.27122 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:43.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 -- 172.21.15.39:0/2146609765 >> 172.21.15.39:0/2146609765 conn(0x557326735460 msgr2=0x557326ae67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:43.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 -- 172.21.15.39:0/2146609765 shutdown_connections 2024-07-04T08:21:43.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 1 -- 172.21.15.39:0/2146609765 wait complete. 2024-07-04T08:21:43.312 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:21:43.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.112+0000 7f60e84da740 20 client.27122 trim_cache size 0 max 16384 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 -- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60c00b29e0 msgr2=0x7f60c00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60c00b29e0 0x7f60c00b4ef0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009730 tx=0x7f60c8013040 comp rx=0 tx=0).stop 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 -- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60dc10b590 msgr2=0x7f60dc1a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60dc10b590 0x7f60dc1a13e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f60d402a930 tx=0x7f60d4003b90 comp rx=0 tx=0).stop 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 -- 172.21.15.39:0/899801951 shutdown_connections 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f60c00b29e0 0x7f60c00b4ef0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f60dc10b590 0x7f60dc1a13e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f60dc10ac40 0x7f60dc19c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 --2- 172.21.15.39:0/899801951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60dc105300 0x7f60dc19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.113+0000 7f60e1e00640 1 -- 172.21.15.39:0/899801951 >> 172.21.15.39:0/899801951 conn(0x7f60dc100d10 msgr2=0x7f60dc108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.114+0000 7f60e1e00640 1 -- 172.21.15.39:0/899801951 shutdown_connections 2024-07-04T08:21:43.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:43.114+0000 7f60e1e00640 1 -- 172.21.15.39:0/899801951 wait complete. 2024-07-04T08:21:43.314 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:43 smithi133 ceph-mon[37050]: pgmap v675: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:21:43.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:43 smithi050 ceph-mon[36577]: pgmap v675: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:21:44.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.826+0000 7f8305800640 1 -- 172.21.15.39:0/2860568074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 msgr2=0x7f83001034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:44.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.826+0000 7f8305800640 1 --2- 172.21.15.39:0/2860568074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 0x7f83001034e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009860 tx=0x7f82f002a340 comp rx=0 tx=0).stop 2024-07-04T08:21:44.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.826+0000 7f8305800640 1 -- 172.21.15.39:0/2860568074 shutdown_connections 2024-07-04T08:21:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.826+0000 7f8305800640 1 --2- 172.21.15.39:0/2860568074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8300108a10 0x7f8300108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.826+0000 7f8305800640 1 --2- 172.21.15.39:0/2860568074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8300109360 0x7f830010fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.826+0000 7f8305800640 1 --2- 172.21.15.39:0/2860568074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 0x7f83001034e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.826+0000 7f8305800640 1 -- 172.21.15.39:0/2860568074 >> 172.21.15.39:0/2860568074 conn(0x7f83000feae0 msgr2=0x7f8300100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.826+0000 7f8305800640 1 -- 172.21.15.39:0/2860568074 shutdown_connections 2024-07-04T08:21:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.826+0000 7f8305800640 1 -- 172.21.15.39:0/2860568074 wait complete. 2024-07-04T08:21:44.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.827+0000 7f8305800640 1 Processor -- start 2024-07-04T08:21:44.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.828+0000 7f8305800640 1 -- start start 2024-07-04T08:21:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.828+0000 7f8305800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 0x7f8300074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.828+0000 7f8305800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8300108a10 0x7f83000751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.828+0000 7f8305800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8300109360 0x7f83000756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.828+0000 7f8305800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8300113010 con 0x7f8300109360 2024-07-04T08:21:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.828+0000 7f8305800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8300112e90 con 0x7f83001030d0 2024-07-04T08:21:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.828+0000 7f8305800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8300113190 con 0x7f8300108a10 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82fea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 0x7f8300074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82fea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 0x7f8300074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52126/0 (socket says 172.21.15.39:52126) 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82fea00640 1 -- 172.21.15.39:0/1941190166 learned_addr learned my addr 172.21.15.39:0/1941190166 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82f7e00640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8300108a10 0x7f83000751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82ff400640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8300109360 0x7f83000756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82fea00640 1 -- 172.21.15.39:0/1941190166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8300108a10 msgr2=0x7f83000751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82fea00640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8300108a10 0x7f83000751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82fea00640 1 -- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8300109360 msgr2=0x7f83000756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82fea00640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8300109360 0x7f83000756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.829+0000 7f82fea00640 1 -- 172.21.15.39:0/1941190166 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83001a7de0 con 0x7f83001030d0 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.830+0000 7f82fea00640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 0x7f8300074c60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82f00303f0 tx=0x7f82f0032eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.830+0000 7f82f7400640 1 -- 172.21.15.39:0/1941190166 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0040020 con 0x7f83001030d0 2024-07-04T08:21:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.830+0000 7f82f7400640 1 -- 172.21.15.39:0/1941190166 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82f0030450 con 0x7f83001030d0 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.830+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83001a8070 con 0x7f83001030d0 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.830+0000 7f82f7400640 1 -- 172.21.15.39:0/1941190166 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0037040 con 0x7f83001030d0 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.830+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83001a85a0 con 0x7f83001030d0 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.832+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83001034e0 con 0x7f83001030d0 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.834+0000 7f82f7400640 1 -- 172.21.15.39:0/1941190166 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f82f0043020 con 0x7f83001030d0 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.836+0000 7f82f7400640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82e40b29e0 0x7f82e40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.836+0000 7f82f7400640 1 -- 172.21.15.39:0/1941190166 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f82f0102760 con 0x7f83001030d0 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.837+0000 7f82f7e00640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82e40b29e0 0x7f82e40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.837+0000 7f82f7e00640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82e40b29e0 0x7f82e40b4ef0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8300102350 tx=0x7f82e8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:44.840+0000 7f82f7400640 1 -- 172.21.15.39:0/1941190166 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f82f004a030 con 0x7f83001030d0 2024-07-04T08:21:45.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.098+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f830010b110 con 0x7f83001030d0 2024-07-04T08:21:45.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.100+0000 7f82f7400640 1 -- 172.21.15.39:0/1941190166 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f82f00c9ec0 con 0x7f83001030d0 2024-07-04T08:21:45.095 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:21:45.095 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:21:45.096 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:21:45.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.102+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82e40b29e0 msgr2=0x7f82e40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:45.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.102+0000 7f8305800640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82e40b29e0 0x7f82e40b4ef0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8300102350 tx=0x7f82e8009260 comp rx=0 tx=0).stop 2024-07-04T08:21:45.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.102+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 msgr2=0x7f8300074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:45.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.102+0000 7f8305800640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 0x7f8300074c60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f82f00303f0 tx=0x7f82f0032eb0 comp rx=0 tx=0).stop 2024-07-04T08:21:45.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.103+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 shutdown_connections 2024-07-04T08:21:45.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.103+0000 7f8305800640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82e40b29e0 0x7f82e40b4ef0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:45.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.103+0000 7f8305800640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8300109360 0x7f83000756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:45.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.103+0000 7f8305800640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8300108a10 0x7f83000751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:45.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.103+0000 7f8305800640 1 --2- 172.21.15.39:0/1941190166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83001030d0 0x7f8300074c60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:45.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.103+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 >> 172.21.15.39:0/1941190166 conn(0x7f83000feae0 msgr2=0x7f8300100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:45.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.103+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 shutdown_connections 2024-07-04T08:21:45.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:45.103+0000 7f8305800640 1 -- 172.21.15.39:0/1941190166 wait complete. 2024-07-04T08:21:45.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:45 smithi133 ceph-mon[37050]: pgmap v676: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-07-04T08:21:45.345 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:45 smithi039 ceph-mon[32062]: pgmap v676: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-07-04T08:21:45.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:45 smithi050 ceph-mon[36577]: pgmap v676: 129 pgs: 129 active+clean; 6.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-07-04T08:21:45.502 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:21:45.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:21:46.109 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id 90f50498 --release --await' 2024-07-04T08:21:46.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id 90f50498 --release --await 2024-07-04T08:21:46.161 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:46.309 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:46 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1941190166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:46.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:46 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1941190166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:46.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:46 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1941190166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:47.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:47 smithi133 ceph-mon[37050]: pgmap v677: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-07-04T08:21:47.345 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:47 smithi039 ceph-mon[32062]: pgmap v677: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-07-04T08:21:47.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:47 smithi050 ceph-mon[36577]: pgmap v677: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 11 op/s 2024-07-04T08:21:47.561 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:48.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.940+0000 7fd453a00640 1 -- 172.21.15.39:0/3672568217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd44c1030d0 msgr2=0x7fd44c1034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:48.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.940+0000 7fd453a00640 1 --2- 172.21.15.39:0/3672568217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd44c1030d0 0x7fd44c1034e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd440009900 tx=0x7fd44002a410 comp rx=0 tx=0).stop 2024-07-04T08:21:48.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.941+0000 7fd453a00640 1 -- 172.21.15.39:0/3672568217 shutdown_connections 2024-07-04T08:21:48.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.941+0000 7fd453a00640 1 --2- 172.21.15.39:0/3672568217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd44c109360 0x7fd44c10fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.941+0000 7fd453a00640 1 --2- 172.21.15.39:0/3672568217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c108a10 0x7fd44c108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.941+0000 7fd453a00640 1 --2- 172.21.15.39:0/3672568217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd44c1030d0 0x7fd44c1034e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.941+0000 7fd453a00640 1 -- 172.21.15.39:0/3672568217 >> 172.21.15.39:0/3672568217 conn(0x7fd44c0feb20 msgr2=0x7fd44c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:48.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.941+0000 7fd453a00640 1 -- 172.21.15.39:0/3672568217 shutdown_connections 2024-07-04T08:21:48.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.941+0000 7fd453a00640 1 -- 172.21.15.39:0/3672568217 wait complete. 2024-07-04T08:21:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.942+0000 7fd453a00640 1 Processor -- start 2024-07-04T08:21:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.942+0000 7fd453a00640 1 -- start start 2024-07-04T08:21:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.942+0000 7fd453a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c1030d0 0x7fd44c19bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.942+0000 7fd453a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd44c108a10 0x7fd44c19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.942+0000 7fd453a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd44c109360 0x7fd44c1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.942+0000 7fd453a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c113030 con 0x7fd44c109360 2024-07-04T08:21:48.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.942+0000 7fd453a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c112eb0 con 0x7fd44c1030d0 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.942+0000 7fd453a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1131b0 con 0x7fd44c108a10 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd451000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c1030d0 0x7fd44c19bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd451000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c1030d0 0x7fd44c19bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52138/0 (socket says 172.21.15.39:52138) 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd451000640 1 -- 172.21.15.39:0/2366754908 learned_addr learned my addr 172.21.15.39:0/2366754908 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd44be00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd44c108a10 0x7fd44c19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd451a00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd44c109360 0x7fd44c1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd451000640 1 -- 172.21.15.39:0/2366754908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd44c108a10 msgr2=0x7fd44c19c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd451000640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd44c108a10 0x7fd44c19c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd451000640 1 -- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd44c109360 msgr2=0x7fd44c1a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd451000640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd44c109360 0x7fd44c1a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.943+0000 7fd451000640 1 -- 172.21.15.39:0/2366754908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c1a1b90 con 0x7fd44c1030d0 2024-07-04T08:21:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.944+0000 7fd451000640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c1030d0 0x7fd44c19bc20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd440005940 tx=0x7fd4400040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.944+0000 7fd449600640 1 -- 172.21.15.39:0/2366754908 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44003f030 con 0x7fd44c1030d0 2024-07-04T08:21:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.944+0000 7fd453a00640 1 -- 172.21.15.39:0/2366754908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c1a1e20 con 0x7fd44c1030d0 2024-07-04T08:21:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.944+0000 7fd449600640 1 -- 172.21.15.39:0/2366754908 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4400044c0 con 0x7fd44c1030d0 2024-07-04T08:21:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.944+0000 7fd449600640 1 -- 172.21.15.39:0/2366754908 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440035070 con 0x7fd44c1030d0 2024-07-04T08:21:48.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.944+0000 7fd453a00640 1 -- 172.21.15.39:0/2366754908 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c1a85e0 con 0x7fd44c1030d0 2024-07-04T08:21:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.947+0000 7fd45a059740 1 Processor -- start 2024-07-04T08:21:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.948+0000 7fd45a059740 1 -- start start 2024-07-04T08:21:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.948+0000 7fd45a059740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd0c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.948+0000 7fd45a059740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 0x557f9fd0cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.948+0000 7fd45a059740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd0d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.948+0000 7fd45a059740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557f9fc368f0 con 0x557f9f968350 2024-07-04T08:21:48.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.948+0000 7fd45a059740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557f9fc82ed0 con 0x557f9f947670 2024-07-04T08:21:48.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.948+0000 7fd45a059740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557f9fc72470 con 0x557f9f93f490 2024-07-04T08:21:48.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.948+0000 7fd449600640 1 -- 172.21.15.39:0/2366754908 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd440042030 con 0x7fd44c1030d0 2024-07-04T08:21:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd451a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd0c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd44be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd0d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd44be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd0d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52140/0 (socket says 172.21.15.39:52140) 2024-07-04T08:21:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd44be00640 1 -- 172.21.15.39:0/369071814 learned_addr learned my addr 172.21.15.39:0/369071814 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd451000640 1 --2- 172.21.15.39:0/369071814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 0x557f9fd0cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd44be00640 1 -- 172.21.15.39:0/369071814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 msgr2=0x557f9fd0cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd44be00640 1 --2- 172.21.15.39:0/369071814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 0x557f9fd0cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd44be00640 1 -- 172.21.15.39:0/369071814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 msgr2=0x557f9fd0c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:48.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd44be00640 1 --2- 172.21.15.39:0/369071814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd0c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd44be00640 1 -- 172.21.15.39:0/369071814 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f9fc0a290 con 0x557f9f947670 2024-07-04T08:21:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.949+0000 7fd44be00640 1 --2- 172.21.15.39:0/369071814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd0d0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd438009c20 tx=0x7fd438011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ab6736796024ea server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.950+0000 7fd43e000640 1 -- 172.21.15.39:0/369071814 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438004230 con 0x557f9f947670 2024-07-04T08:21:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.950+0000 7fd43e000640 1 -- 172.21.15.39:0/369071814 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4380043d0 con 0x557f9f947670 2024-07-04T08:21:48.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.950+0000 7fd43e000640 1 -- 172.21.15.39:0/369071814 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438004d50 con 0x557f9f947670 2024-07-04T08:21:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.950+0000 7fd449600640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd4340b2960 0x7fd4340b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd449600640 1 -- 172.21.15.39:0/2366754908 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fd440102ca0 con 0x7fd44c1030d0 2024-07-04T08:21:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd451a00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd4340b2960 0x7fd4340b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd45a059740 1 -- 172.21.15.39:0/369071814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 msgr2=0x557f9fd0d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd45a059740 1 --2- 172.21.15.39:0/369071814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd0d0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd438009c20 tx=0x7fd438011810 comp rx=0 tx=0).stop 2024-07-04T08:21:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd45a059740 1 -- 172.21.15.39:0/369071814 shutdown_connections 2024-07-04T08:21:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd45a059740 1 --2- 172.21.15.39:0/369071814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd0d0d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd45a059740 1 --2- 172.21.15.39:0/369071814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 0x557f9fd0cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd45a059740 1 --2- 172.21.15.39:0/369071814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd0c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd45a059740 1 -- 172.21.15.39:0/369071814 >> 172.21.15.39:0/369071814 conn(0x557f9f95c740 msgr2=0x557f9fd0a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd45a059740 1 -- 172.21.15.39:0/369071814 shutdown_connections 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.951+0000 7fd45a059740 1 -- 172.21.15.39:0/369071814 wait complete. 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.952+0000 7fd451a00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd4340b2960 0x7fd4340b4e70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd444005fd0 tx=0x7fd444009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.952+0000 7fd45a059740 1 -- start start 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.952+0000 7fd45a059740 1 Processor -- start 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd45a059740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f93f490 0x557f9fd181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd45a059740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f947670 0x557f9fd186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd45a059740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 0x557f9fd23fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd45a059740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557f9fc19770 con 0x557f9f947670 2024-07-04T08:21:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd45a059740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557f9fc764b0 con 0x557f9f93f490 2024-07-04T08:21:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd45a059740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557f9fad6890 con 0x557f9f968350 2024-07-04T08:21:48.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd45a059740 10 client.0 _ll_register_callbacks cb 0x557f9fc1a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:48.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd44be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f947670 0x557f9fd186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd44be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f947670 0x557f9fd186f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33876/0 (socket says 172.21.15.39:33876) 2024-07-04T08:21:48.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd44be00640 1 -- 172.21.15.39:0/2378780924 learned_addr learned my addr 172.21.15.39:0/2378780924 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:48.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd451000640 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f93f490 0x557f9fd181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.953+0000 7fd451a00640 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 0x557f9fd23fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.954+0000 7fd44be00640 1 -- 172.21.15.39:0/2378780924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 msgr2=0x557f9fd23fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.954+0000 7fd44be00640 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 0x557f9fd23fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.954+0000 7fd44be00640 1 -- 172.21.15.39:0/2378780924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f93f490 msgr2=0x557f9fd181b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.954+0000 7fd44be00640 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f93f490 0x557f9fd181b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.954+0000 7fd44be00640 1 -- 172.21.15.39:0/2378780924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f9f9d3930 con 0x557f9f947670 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.954+0000 7fd451a00640 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 0x557f9fd23fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.954+0000 7fd44be00640 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f947670 0x557f9fd186f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd438019b00 tx=0x7fd438019b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fff414e1e34d123 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.954+0000 7fd432a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.955+0000 7fd432a00640 1 -- 172.21.15.39:0/2378780924 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438028020 con 0x557f9f947670 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.955+0000 7fd432a00640 1 -- 172.21.15.39:0/2378780924 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fd438024620 con 0x557f9f947670 2024-07-04T08:21:48.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.955+0000 7fd45a059740 1 -- 172.21.15.39:0/2378780924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4380043d0 con 0x557f9f947670 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.955+0000 7fd432a00640 1 -- 172.21.15.39:0/2378780924 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd438033050 con 0x557f9f947670 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.955+0000 7fd45a059740 10 client.16998 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.955+0000 7fd45a059740 1 -- 172.21.15.39:0/2378780924 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f9fd26c40 con 0x557f9f947670 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.956+0000 7fd432a00640 1 -- 172.21.15.39:0/2378780924 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4380043d0 con 0x557f9f947670 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.956+0000 7fd432a00640 1 -- 172.21.15.39:0/2378780924 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd438018070 con 0x557f9f947670 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.956+0000 7fd45a059740 10 client.16998 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.956+0000 7fd45a059740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.956+0000 7fd45a059740 20 client.16998 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.956+0000 7fd45a059740 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f9f926cd0 0x557f9fd2a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.956+0000 7fd45a059740 4 client.16998 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:48.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.956+0000 7fd45a059740 1 -- 172.21.15.39:0/2378780924 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f9fc84e20 con 0x557f9f926cd0 2024-07-04T08:21:48.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.956+0000 7fd451000640 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f9f926cd0 0x557f9fd2a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.957+0000 7fd451000640 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f9f926cd0 0x557f9fd2a050 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8462d548d7e11b9 server_cookie=99355365131932e4 in_seq=0 out_seq=0 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.957+0000 7fd432a00640 10 client.16998 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd432a00640 1 -- 172.21.15.39:0/2378780924 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f9fc84e20 con 0x557f9f926cd0 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd432a00640 10 client.16998 handle_command_reply: tid=0 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd45a059740 1 client.16998 shutdown 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd45a059740 1 -- 172.21.15.39:0/2378780924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f947670 msgr2=0x557f9fd186f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd45a059740 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f947670 0x557f9fd186f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd438019b00 tx=0x7fd438019b30 comp rx=0 tx=0).stop 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd45a059740 1 -- 172.21.15.39:0/2378780924 shutdown_connections 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd45a059740 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f9f926cd0 0x557f9fd2a050 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd45a059740 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f947670 0x557f9fd186f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd45a059740 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 0x557f9fd23fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd45a059740 1 --2- 172.21.15.39:0/2378780924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f93f490 0x557f9fd181b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd45a059740 1 -- 172.21.15.39:0/2378780924 >> 172.21.15.39:0/2378780924 conn(0x557f9f95c740 msgr2=0x557f9fd0f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.958+0000 7fd432a00640 0 client.16998 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.959+0000 7fd45a059740 1 -- 172.21.15.39:0/2378780924 shutdown_connections 2024-07-04T08:21:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.959+0000 7fd45a059740 1 -- 172.21.15.39:0/2378780924 wait complete. 2024-07-04T08:21:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.959+0000 7fd45a059740 20 client.16998 trim_cache size 0 max 16384 2024-07-04T08:21:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.979+0000 7fd45a059740 1 Processor -- start 2024-07-04T08:21:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.979+0000 7fd45a059740 1 -- start start 2024-07-04T08:21:48.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd45a059740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f93f490 0x557f9fd0dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd45a059740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd11730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd45a059740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 0x557f9fd11c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd45a059740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b2420 con 0x557f9f93f490 2024-07-04T08:21:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd45a059740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b3f60 con 0x557f9f947670 2024-07-04T08:21:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd45a059740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b2af0 con 0x557f9f968350 2024-07-04T08:21:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd44be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd11730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd44be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd11730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52152/0 (socket says 172.21.15.39:52152) 2024-07-04T08:21:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd44be00640 1 -- 172.21.15.39:0/523133211 learned_addr learned my addr 172.21.15.39:0/523133211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd451a00640 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 0x557f9fd11c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.980+0000 7fd451000640 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f93f490 0x557f9fd0dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.981+0000 7fd44be00640 1 -- 172.21.15.39:0/523133211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 msgr2=0x557f9fd11c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.981+0000 7fd44be00640 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 0x557f9fd11c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.981+0000 7fd44be00640 1 -- 172.21.15.39:0/523133211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f93f490 msgr2=0x557f9fd0dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.981+0000 7fd44be00640 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f93f490 0x557f9fd0dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.981+0000 7fd44be00640 1 -- 172.21.15.39:0/523133211 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f9fd0aa80 con 0x557f9f947670 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.981+0000 7fd451000640 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f93f490 0x557f9fd0dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.981+0000 7fd44be00640 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd11730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd438012fd0 tx=0x7fd438005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29f712070fa5795e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.981+0000 7fd43d600640 1 -- 172.21.15.39:0/523133211 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438028040 con 0x557f9f947670 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.981+0000 7fd43d600640 1 -- 172.21.15.39:0/523133211 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43801f040 con 0x557f9f947670 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.982+0000 7fd43d600640 1 -- 172.21.15.39:0/523133211 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438004330 con 0x557f9f947670 2024-07-04T08:21:48.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.982+0000 7fd45a059740 1 -- 172.21.15.39:0/523133211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 msgr2=0x557f9fd11730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:48.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.982+0000 7fd45a059740 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd11730 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd438012fd0 tx=0x7fd438005f70 comp rx=0 tx=0).stop 2024-07-04T08:21:48.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.982+0000 7fd45a059740 1 -- 172.21.15.39:0/523133211 shutdown_connections 2024-07-04T08:21:48.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.982+0000 7fd45a059740 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f968350 0x557f9fd11c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.982+0000 7fd45a059740 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd11730 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.982+0000 7fd45a059740 1 --2- 172.21.15.39:0/523133211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f93f490 0x557f9fd0dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.982+0000 7fd45a059740 1 -- 172.21.15.39:0/523133211 >> 172.21.15.39:0/523133211 conn(0x557f9f95c740 msgr2=0x557f9fd0d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:48.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.983+0000 7fd45a059740 1 -- 172.21.15.39:0/523133211 shutdown_connections 2024-07-04T08:21:48.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.983+0000 7fd45a059740 1 -- 172.21.15.39:0/523133211 wait complete. 2024-07-04T08:21:48.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.983+0000 7fd45a059740 1 -- start start 2024-07-04T08:21:48.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.983+0000 7fd45a059740 1 Processor -- start 2024-07-04T08:21:48.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd45a059740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 0x557f9fd235a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd45a059740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd23ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd45a059740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd1ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd45a059740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b3890 con 0x557f9f968350 2024-07-04T08:21:48.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd45a059740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b31c0 con 0x557f9f947670 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd45a059740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c1b0400 con 0x557f9f93f490 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd45a059740 10 client.0 _ll_register_callbacks cb 0x7fd438019790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd451a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd1ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd451a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd1ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33888/0 (socket says 172.21.15.39:33888) 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd451a00640 1 -- 172.21.15.39:0/1964830217 learned_addr learned my addr 172.21.15.39:0/1964830217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd451000640 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 0x557f9fd235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.984+0000 7fd44be00640 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd23ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.985+0000 7fd451a00640 1 -- 172.21.15.39:0/1964830217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 msgr2=0x557f9fd235a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.985+0000 7fd451a00640 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 0x557f9fd235a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.985+0000 7fd451a00640 1 -- 172.21.15.39:0/1964830217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 msgr2=0x557f9fd23ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.985+0000 7fd451a00640 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd23ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.985+0000 7fd451a00640 1 -- 172.21.15.39:0/1964830217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f9fd1f4f0 con 0x557f9f968350 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.985+0000 7fd451a00640 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd1ef30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd444004650 tx=0x7fd444004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb3fce85a7472a1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.985+0000 7fd432a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.986+0000 7fd432a00640 1 -- 172.21.15.39:0/1964830217 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44402e060 con 0x557f9f968350 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.986+0000 7fd432a00640 1 -- 172.21.15.39:0/1964830217 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7fd44401e430 con 0x557f9f968350 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.986+0000 7fd45a059740 1 -- 172.21.15.39:0/1964830217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd43801f040 con 0x557f9f968350 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.986+0000 7fd432a00640 1 -- 172.21.15.39:0/1964830217 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd444035050 con 0x557f9f968350 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.986+0000 7fd45a059740 10 client.17004 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.986+0000 7fd45a059740 1 -- 172.21.15.39:0/1964830217 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f9fd20810 con 0x557f9f968350 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.986+0000 7fd432a00640 1 -- 172.21.15.39:0/1964830217 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43801f040 con 0x557f9f968350 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.987+0000 7fd432a00640 1 -- 172.21.15.39:0/1964830217 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd44401d650 con 0x557f9f968350 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.987+0000 7fd45a059740 10 client.17004 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:48.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.987+0000 7fd45a059740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:48.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.987+0000 7fd45a059740 20 client.17004 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:48.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.987+0000 7fd45a059740 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f9f8fc080 0x557f9fd949a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:48.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.987+0000 7fd45a059740 4 client.17004 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:21:48.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.987+0000 7fd45a059740 1 -- 172.21.15.39:0/1964830217 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557f9fc808c0 con 0x557f9f8fc080 2024-07-04T08:21:48.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.987+0000 7fd451000640 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f9f8fc080 0x557f9fd949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:48.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.988+0000 7fd451000640 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f9f8fc080 0x557f9fd949a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc5c26f15a9448d5 server_cookie=b5b5413ec6cae0ca in_seq=0 out_seq=0 2024-07-04T08:21:48.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.988+0000 7fd432a00640 10 client.17004 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:48.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd432a00640 1 -- 172.21.15.39:0/1964830217 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557f9fc808c0 con 0x557f9f8fc080 2024-07-04T08:21:48.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd432a00640 10 client.17004 handle_command_reply: tid=0 2024-07-04T08:21:48.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 client.17004 shutdown 2024-07-04T08:21:48.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 -- 172.21.15.39:0/1964830217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 msgr2=0x557f9fd1ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd1ef30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd444004650 tx=0x7fd444004a40 comp rx=0 tx=0).stop 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 -- 172.21.15.39:0/1964830217 shutdown_connections 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f9f8fc080 0x557f9fd949a0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f9f968350 0x557f9fd1ef30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f9f947670 0x557f9fd23ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd432a00640 0 client.17004 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 --2- 172.21.15.39:0/1964830217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f9f93f490 0x557f9fd235a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 -- 172.21.15.39:0/1964830217 >> 172.21.15.39:0/1964830217 conn(0x557f9f95c740 msgr2=0x557f9fd0e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.989+0000 7fd45a059740 1 -- 172.21.15.39:0/1964830217 shutdown_connections 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.990+0000 7fd45a059740 1 -- 172.21.15.39:0/1964830217 wait complete. 2024-07-04T08:21:48.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.990+0000 7fd45a059740 20 client.17004 trim_cache size 0 max 16384 2024-07-04T08:21:48.988 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:21:48.988 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:21:48.988 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "aebf7655-ce8d-457e-a0c3-75598e1b8777", 2024-07-04T08:21:48.988 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:21:48.988 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:21:48.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.990+0000 7fd453a00640 1 -- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd4340b2960 msgr2=0x7fd4340b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.990+0000 7fd453a00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd4340b2960 0x7fd4340b4e70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd444005fd0 tx=0x7fd444009500 comp rx=0 tx=0).stop 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.990+0000 7fd453a00640 1 -- 172.21.15.39:0/2366754908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c1030d0 msgr2=0x7fd44c19bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.990+0000 7fd453a00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c1030d0 0x7fd44c19bc20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd440005940 tx=0x7fd4400040c0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.991+0000 7fd453a00640 1 -- 172.21.15.39:0/2366754908 shutdown_connections 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.991+0000 7fd453a00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd4340b2960 0x7fd4340b4e70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.991+0000 7fd453a00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd44c109360 0x7fd44c1a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.991+0000 7fd453a00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd44c108a10 0x7fd44c19c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.991+0000 7fd453a00640 1 --2- 172.21.15.39:0/2366754908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd44c1030d0 0x7fd44c19bc20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.991+0000 7fd453a00640 1 -- 172.21.15.39:0/2366754908 >> 172.21.15.39:0/2366754908 conn(0x7fd44c0feb20 msgr2=0x7fd44c108140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:48.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.991+0000 7fd453a00640 1 -- 172.21.15.39:0/2366754908 shutdown_connections 2024-07-04T08:21:48.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:48.991+0000 7fd453a00640 1 -- 172.21.15.39:0/2366754908 wait complete. 2024-07-04T08:21:49.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:49 smithi133 ceph-mon[37050]: pgmap v678: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-07-04T08:21:49.345 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:49 smithi039 ceph-mon[32062]: pgmap v678: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-07-04T08:21:49.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:49 smithi050 ceph-mon[36577]: pgmap v678: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-07-04T08:21:50.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:50 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:21:50.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:50 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:50.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:50 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:21:50.292 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:50 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:50.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:50 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:21:50.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:50 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:50.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:50 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:21:50.311 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:50 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:50.362 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:21:50.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:50 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:21:50.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:50 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:50.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:50 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:21:50.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:50 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:21:50.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.922+0000 7f7021400640 1 -- 172.21.15.39:0/1090815502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c1050f0 msgr2=0x7f701c105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:50.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.922+0000 7f7021400640 1 --2- 172.21.15.39:0/1090815502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c1050f0 0x7f701c105500 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f700c009900 tx=0x7f700c02a410 comp rx=0 tx=0).stop 2024-07-04T08:21:50.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.923+0000 7f7021400640 1 -- 172.21.15.39:0/1090815502 shutdown_connections 2024-07-04T08:21:50.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.923+0000 7f7021400640 1 --2- 172.21.15.39:0/1090815502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c10b380 0x7f701c111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.923+0000 7f7021400640 1 --2- 172.21.15.39:0/1090815502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c10aa30 0x7f701c10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.923+0000 7f7021400640 1 --2- 172.21.15.39:0/1090815502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c1050f0 0x7f701c105500 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.923+0000 7f7021400640 1 -- 172.21.15.39:0/1090815502 >> 172.21.15.39:0/1090815502 conn(0x7f701c100b60 msgr2=0x7f701c102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:50.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.923+0000 7f7021400640 1 -- 172.21.15.39:0/1090815502 shutdown_connections 2024-07-04T08:21:50.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.923+0000 7f7021400640 1 -- 172.21.15.39:0/1090815502 wait complete. 2024-07-04T08:21:50.919 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:50.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.924+0000 7f7021400640 1 Processor -- start 2024-07-04T08:21:50.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.924+0000 7f7021400640 1 -- start start 2024-07-04T08:21:50.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f7021400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c1050f0 0x7f701c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f7021400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c10aa30 0x7f701c19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f7021400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c10b380 0x7f701c1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f7021400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f701c115050 con 0x7f701c10b380 2024-07-04T08:21:50.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f7021400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f701c114ed0 con 0x7f701c10aa30 2024-07-04T08:21:50.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f7021400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1151d0 con 0x7f701c1050f0 2024-07-04T08:21:50.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f701be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c1050f0 0x7f701c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f7020a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c10b380 0x7f701c1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f701be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c1050f0 0x7f701c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36652/0 (socket says 172.21.15.39:36652) 2024-07-04T08:21:50.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f701be00640 1 -- 172.21.15.39:0/2765945396 learned_addr learned my addr 172.21.15.39:0/2765945396 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:50.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.925+0000 7f7013e00640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c10aa30 0x7f701c19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.926+0000 7f701be00640 1 -- 172.21.15.39:0/2765945396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c10aa30 msgr2=0x7f701c19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:50.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.926+0000 7f701be00640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c10aa30 0x7f701c19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.926+0000 7f701be00640 1 -- 172.21.15.39:0/2765945396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c10b380 msgr2=0x7f701c1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:50.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.926+0000 7f701be00640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c10b380 0x7f701c1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.926+0000 7f701be00640 1 -- 172.21.15.39:0/2765945396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c1a1990 con 0x7f701c1050f0 2024-07-04T08:21:50.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.926+0000 7f701be00640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c1050f0 0x7f701c19ba20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f700c005940 tx=0x7f700c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:50.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.927+0000 7f7019600640 1 -- 172.21.15.39:0/2765945396 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c03f030 con 0x7f701c1050f0 2024-07-04T08:21:50.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.927+0000 7f7021400640 1 -- 172.21.15.39:0/2765945396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c1a1c20 con 0x7f701c1050f0 2024-07-04T08:21:50.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.927+0000 7f7019600640 1 -- 172.21.15.39:0/2765945396 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f700c0044c0 con 0x7f701c1050f0 2024-07-04T08:21:50.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.927+0000 7f7019600640 1 -- 172.21.15.39:0/2765945396 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c035070 con 0x7f701c1050f0 2024-07-04T08:21:50.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.927+0000 7f7021400640 1 -- 172.21.15.39:0/2765945396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c1a83e0 con 0x7f701c1050f0 2024-07-04T08:21:50.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.931+0000 7f702901d740 1 Processor -- start 2024-07-04T08:21:50.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.931+0000 7f702901d740 1 -- start start 2024-07-04T08:21:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.932+0000 7f702901d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7b9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.932+0000 7f702901d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.932+0000 7f702901d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3f3840 0x5583fb7b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.932+0000 7f702901d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5583fb6d3b00 con 0x5583fb414520 2024-07-04T08:21:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.932+0000 7f702901d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5583fb72f290 con 0x5583fb3f3840 2024-07-04T08:21:50.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.932+0000 7f702901d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5583fb71e830 con 0x5583fb3eb660 2024-07-04T08:21:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.932+0000 7f7020a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.932+0000 7f7019600640 1 -- 172.21.15.39:0/2765945396 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f700c042030 con 0x7f701c1050f0 2024-07-04T08:21:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f7020a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7b9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33924/0 (socket says 172.21.15.39:33924) 2024-07-04T08:21:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f7020a00640 1 -- 172.21.15.39:0/511621046 learned_addr learned my addr 172.21.15.39:0/511621046 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f7013e00640 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3f3840 0x5583fb7b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f701be00640 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f7013e00640 1 -- 172.21.15.39:0/511621046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 msgr2=0x5583fb7b99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:50.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f7013e00640 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7b99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f7013e00640 1 -- 172.21.15.39:0/511621046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 msgr2=0x5583fb7b9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:50.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f7013e00640 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7b9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f7013e00640 1 -- 172.21.15.39:0/511621046 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fb6d8760 con 0x5583fb3f3840 2024-07-04T08:21:50.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.933+0000 7f7020a00640 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7b9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.934+0000 7f7013e00640 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3f3840 0x5583fb7b9ef0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f701c06bbd0 tx=0x7f7004011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26832c054171f33f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.934+0000 7f7011600640 1 -- 172.21.15.39:0/511621046 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004009d70 con 0x5583fb3f3840 2024-07-04T08:21:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.934+0000 7f7011600640 1 -- 172.21.15.39:0/511621046 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70040198c0 con 0x5583fb3f3840 2024-07-04T08:21:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.934+0000 7f7011600640 1 -- 172.21.15.39:0/511621046 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040188b0 con 0x5583fb3f3840 2024-07-04T08:21:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.934+0000 7f7019600640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70000b2960 0x7f70000b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.935+0000 7f7019600640 1 -- 172.21.15.39:0/2765945396 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f700c102990 con 0x7f701c1050f0 2024-07-04T08:21:50.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.935+0000 7f7020a00640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70000b2960 0x7f70000b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.935+0000 7f702901d740 1 -- 172.21.15.39:0/511621046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3f3840 msgr2=0x5583fb7b9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:50.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.935+0000 7f702901d740 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3f3840 0x5583fb7b9ef0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f701c06bbd0 tx=0x7f7004011810 comp rx=0 tx=0).stop 2024-07-04T08:21:50.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.935+0000 7f702901d740 1 -- 172.21.15.39:0/511621046 shutdown_connections 2024-07-04T08:21:50.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.936+0000 7f702901d740 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3f3840 0x5583fb7b9ef0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.936+0000 7f702901d740 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.936+0000 7f702901d740 1 --2- 172.21.15.39:0/511621046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7b9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.936+0000 7f702901d740 1 -- 172.21.15.39:0/511621046 >> 172.21.15.39:0/511621046 conn(0x5583fb408910 msgr2=0x5583fb7b7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:50.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.936+0000 7f702901d740 1 -- 172.21.15.39:0/511621046 shutdown_connections 2024-07-04T08:21:50.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.936+0000 7f7020a00640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70000b2960 0x7f70000b4e70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7014011890 tx=0x7f7014009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:50.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.936+0000 7f702901d740 1 -- 172.21.15.39:0/511621046 wait complete. 2024-07-04T08:21:50.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.936+0000 7f702901d740 1 -- start start 2024-07-04T08:21:50.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.937+0000 7f702901d740 1 Processor -- start 2024-07-04T08:21:50.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.937+0000 7f702901d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.937+0000 7f702901d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 0x5583fb7c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.937+0000 7f702901d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.937+0000 7f702901d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5583fb6c5d00 con 0x5583fb3f3840 2024-07-04T08:21:50.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.937+0000 7f702901d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5583fb722870 con 0x5583fb414520 2024-07-04T08:21:50.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.937+0000 7f702901d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5583fb582bb0 con 0x5583fb3eb660 2024-07-04T08:21:50.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.937+0000 7f702901d740 10 client.0 _ll_register_callbacks cb 0x5583fb6c69c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f701be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f7013e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 0x5583fb7c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f7020a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f701be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36670/0 (socket says 172.21.15.39:36670) 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f701be00640 1 -- 172.21.15.39:0/1130786100 learned_addr learned my addr 172.21.15.39:0/1130786100 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f7020a00640 1 -- 172.21.15.39:0/1130786100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 msgr2=0x5583fb7c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f7020a00640 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f7020a00640 1 -- 172.21.15.39:0/1130786100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 msgr2=0x5583fb7c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f7020a00640 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 0x5583fb7c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.938+0000 7f7020a00640 1 -- 172.21.15.39:0/1130786100 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fb6b5640 con 0x5583fb414520 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.939+0000 7f7020a00640 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7d0e20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7014009fd0 tx=0x7f7014004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7e1b88508e4ffa server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.939+0000 7f700a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.939+0000 7f700a000640 1 -- 172.21.15.39:0/1130786100 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140043d0 con 0x5583fb414520 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.939+0000 7f700a000640 1 -- 172.21.15.39:0/1130786100 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7014004570 con 0x5583fb414520 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.939+0000 7f702901d740 1 -- 172.21.15.39:0/1130786100 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70040198c0 con 0x5583fb414520 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.939+0000 7f700a000640 1 -- 172.21.15.39:0/1130786100 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7014004770 con 0x5583fb414520 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.939+0000 7f702901d740 10 client.27158 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.939+0000 7f702901d740 1 -- 172.21.15.39:0/1130786100 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5583fb47fb00 con 0x5583fb414520 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.940+0000 7f700a000640 1 -- 172.21.15.39:0/1130786100 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70040198c0 con 0x5583fb414520 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.940+0000 7f700a000640 1 -- 172.21.15.39:0/1130786100 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7014026070 con 0x5583fb414520 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.940+0000 7f702901d740 10 client.27158 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:50.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.940+0000 7f702901d740 10 client.27158 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:21:50.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.940+0000 7f702901d740 20 client.27158 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:50.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.940+0000 7f702901d740 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5583fb3d2ea0 0x5583fb7d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.940+0000 7f702901d740 4 client.27158 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:50.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.940+0000 7f702901d740 1 -- 172.21.15.39:0/1130786100 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583fb7311e0 con 0x5583fb3d2ea0 2024-07-04T08:21:50.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.941+0000 7f701be00640 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5583fb3d2ea0 0x5583fb7d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.941+0000 7f701be00640 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5583fb3d2ea0 0x5583fb7d6c50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c023579e2010833f server_cookie=7bfd9d6f9ee199ce in_seq=0 out_seq=0 2024-07-04T08:21:50.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.941+0000 7f700a000640 10 client.27158 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:50.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.942+0000 7f700a000640 1 -- 172.21.15.39:0/1130786100 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5583fb7311e0 con 0x5583fb3d2ea0 2024-07-04T08:21:50.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.942+0000 7f700a000640 10 client.27158 handle_command_reply: tid=0 2024-07-04T08:21:50.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.942+0000 7f702901d740 1 client.27158 shutdown 2024-07-04T08:21:50.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.942+0000 7f702901d740 1 -- 172.21.15.39:0/1130786100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 msgr2=0x5583fb7d0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.942+0000 7f702901d740 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7d0e20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7014009fd0 tx=0x7f7014004ab0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f702901d740 1 -- 172.21.15.39:0/1130786100 shutdown_connections 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f702901d740 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5583fb3d2ea0 0x5583fb7d6c50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f702901d740 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 0x5583fb7c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f702901d740 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7d0e20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f702901d740 1 --2- 172.21.15.39:0/1130786100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f700a000640 0 client.27158 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f702901d740 1 -- 172.21.15.39:0/1130786100 >> 172.21.15.39:0/1130786100 conn(0x5583fb408910 msgr2=0x5583fb7bc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f702901d740 1 -- 172.21.15.39:0/1130786100 shutdown_connections 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f702901d740 1 -- 172.21.15.39:0/1130786100 wait complete. 2024-07-04T08:21:50.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.943+0000 7f702901d740 20 client.27158 trim_cache size 0 max 16384 2024-07-04T08:21:50.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.963+0000 7f702901d740 1 Processor -- start 2024-07-04T08:21:50.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.963+0000 7f702901d740 1 -- start start 2024-07-04T08:21:50.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f702901d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3eb660 0x5583fb7c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f702901d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3f3840 0x5583fb7d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f702901d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f702901d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1b2260 con 0x5583fb414520 2024-07-04T08:21:50.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f702901d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1b3da0 con 0x5583fb3eb660 2024-07-04T08:21:50.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f702901d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1b2930 con 0x5583fb3f3840 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f7013e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3f3840 0x5583fb7d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f701be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3eb660 0x5583fb7c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f7013e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3f3840 0x5583fb7d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36686/0 (socket says 172.21.15.39:36686) 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f7013e00640 1 -- 172.21.15.39:0/3963590288 learned_addr learned my addr 172.21.15.39:0/3963590288 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.964+0000 7f7020a00640 1 --2- 172.21.15.39:0/3963590288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.965+0000 7f7013e00640 1 -- 172.21.15.39:0/3963590288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3eb660 msgr2=0x5583fb7c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.965+0000 7f7013e00640 1 --2- 172.21.15.39:0/3963590288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3eb660 0x5583fb7c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.965+0000 7f7013e00640 1 -- 172.21.15.39:0/3963590288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 msgr2=0x5583fb7d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.965+0000 7f7013e00640 1 --2- 172.21.15.39:0/3963590288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7d1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.965+0000 7f7013e00640 1 -- 172.21.15.39:0/3963590288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583fb4f6960 con 0x5583fb3f3840 2024-07-04T08:21:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.965+0000 7f7013e00640 1 --2- 172.21.15.39:0/3963590288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3f3840 0x5583fb7d0c10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7004018560 tx=0x7f70040184f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5144ec6142e392a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.965+0000 7f7010c00640 1 -- 172.21.15.39:0/3963590288 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004009df0 con 0x5583fb3f3840 2024-07-04T08:21:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.966+0000 7f7010c00640 1 -- 172.21.15.39:0/3963590288 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7004005550 con 0x5583fb3f3840 2024-07-04T08:21:50.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.966+0000 7f7010c00640 1 -- 172.21.15.39:0/3963590288 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040035f0 con 0x5583fb3f3840 2024-07-04T08:21:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.966+0000 7f702901d740 1 -- 172.21.15.39:0/3963590288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3f3840 msgr2=0x5583fb7d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.966+0000 7f702901d740 1 --2- 172.21.15.39:0/3963590288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3f3840 0x5583fb7d0c10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7004018560 tx=0x7f70040184f0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.967+0000 7f702901d740 1 -- 172.21.15.39:0/3963590288 shutdown_connections 2024-07-04T08:21:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.967+0000 7f702901d740 1 --2- 172.21.15.39:0/3963590288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb414520 0x5583fb7d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.967+0000 7f702901d740 1 --2- 172.21.15.39:0/3963590288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3f3840 0x5583fb7d0c10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.967+0000 7f702901d740 1 --2- 172.21.15.39:0/3963590288 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb3eb660 0x5583fb7c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.967+0000 7f702901d740 1 -- 172.21.15.39:0/3963590288 >> 172.21.15.39:0/3963590288 conn(0x5583fb408910 msgr2=0x5583fb7b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.967+0000 7f702901d740 1 -- 172.21.15.39:0/3963590288 shutdown_connections 2024-07-04T08:21:50.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.967+0000 7f702901d740 1 -- 172.21.15.39:0/3963590288 wait complete. 2024-07-04T08:21:50.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.968+0000 7f702901d740 1 -- start start 2024-07-04T08:21:50.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.968+0000 7f702901d740 1 Processor -- start 2024-07-04T08:21:50.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.968+0000 7f702901d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.968+0000 7f702901d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 0x5583fb7c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.968+0000 7f702901d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.968+0000 7f702901d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1b36d0 con 0x5583fb3f3840 2024-07-04T08:21:50.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.968+0000 7f702901d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1b3000 con 0x5583fb414520 2024-07-04T08:21:50.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.968+0000 7f702901d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1b0240 con 0x5583fb3eb660 2024-07-04T08:21:50.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f702901d740 10 client.0 _ll_register_callbacks cb 0x7f7014003ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:50.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f701be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f7013e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 0x5583fb7c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f701be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36688/0 (socket says 172.21.15.39:36688) 2024-07-04T08:21:50.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f7020a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f701be00640 1 -- 172.21.15.39:0/2385907089 learned_addr learned my addr 172.21.15.39:0/2385907089 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:50.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f7020a00640 1 -- 172.21.15.39:0/2385907089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 msgr2=0x5583fb7cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:50.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f7020a00640 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f7020a00640 1 -- 172.21.15.39:0/2385907089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 msgr2=0x5583fb7c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:50.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f7020a00640 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 0x5583fb7c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.969+0000 7f7020a00640 1 -- 172.21.15.39:0/2385907089 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583fb6eb880 con 0x5583fb414520 2024-07-04T08:21:50.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.970+0000 7f7020a00640 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7c9c40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f701401e6f0 tx=0x7f7014028440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95aaed053ab4b928 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:50.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.970+0000 7f700a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:21:50.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.970+0000 7f700a000640 1 -- 172.21.15.39:0/2385907089 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701403a020 con 0x5583fb414520 2024-07-04T08:21:50.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.970+0000 7f702901d740 1 -- 172.21.15.39:0/2385907089 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7004005550 con 0x5583fb414520 2024-07-04T08:21:50.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.970+0000 7f700a000640 1 -- 172.21.15.39:0/2385907089 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f701401eb80 con 0x5583fb414520 2024-07-04T08:21:50.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.970+0000 7f700a000640 1 -- 172.21.15.39:0/2385907089 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f701401ed80 con 0x5583fb414520 2024-07-04T08:21:50.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f702901d740 10 client.27164 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:50.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f702901d740 1 -- 172.21.15.39:0/2385907089 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5583fb6e7430 con 0x5583fb414520 2024-07-04T08:21:50.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f700a000640 1 -- 172.21.15.39:0/2385907089 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7004005550 con 0x5583fb414520 2024-07-04T08:21:50.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f700a000640 1 -- 172.21.15.39:0/2385907089 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7014026040 con 0x5583fb414520 2024-07-04T08:21:50.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f702901d740 10 client.27164 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:50.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f702901d740 10 client.27164 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:21:50.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f702901d740 20 client.27164 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:50.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f702901d740 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5583fb3a8250 0x5583fb84ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:50.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f702901d740 4 client.27164 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "90f50498", "release": true, "await": true}] 2024-07-04T08:21:50.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f702901d740 1 -- 172.21.15.39:0/2385907089 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "90f50498", "release": true, "await": true}) -- 0x5583fb7c43d0 con 0x5583fb3a8250 2024-07-04T08:21:50.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.971+0000 7f701be00640 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5583fb3a8250 0x5583fb84ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:50.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.972+0000 7f701be00640 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5583fb3a8250 0x5583fb84ad70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a94a593ffb3582b server_cookie=1a4d062a846be46a in_seq=0 out_seq=0 2024-07-04T08:21:50.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.972+0000 7f700a000640 10 client.27164 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:50.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.975+0000 7f700a000640 1 -- 172.21.15.39:0/2385907089 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5583fb7c43d0 con 0x5583fb3a8250 2024-07-04T08:21:50.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.975+0000 7f700a000640 10 client.27164 handle_command_reply: tid=0 2024-07-04T08:21:50.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.975+0000 7f702901d740 1 client.27164 shutdown 2024-07-04T08:21:50.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.975+0000 7f702901d740 1 -- 172.21.15.39:0/2385907089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 msgr2=0x5583fb7c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:50.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.975+0000 7f702901d740 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7c9c40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f701401e6f0 tx=0x7f7014028440 comp rx=0 tx=0).stop 2024-07-04T08:21:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f702901d740 1 -- 172.21.15.39:0/2385907089 shutdown_connections 2024-07-04T08:21:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f702901d740 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5583fb3a8250 0x5583fb84ad70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f702901d740 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5583fb3f3840 0x5583fb7c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f702901d740 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583fb414520 0x5583fb7c9c40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f702901d740 1 --2- 172.21.15.39:0/2385907089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5583fb3eb660 0x5583fb7cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f702901d740 1 -- 172.21.15.39:0/2385907089 >> 172.21.15.39:0/2385907089 conn(0x5583fb408910 msgr2=0x5583fb7b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:50.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f700a000640 0 client.27164 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f702901d740 1 -- 172.21.15.39:0/2385907089 shutdown_connections 2024-07-04T08:21:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f702901d740 1 -- 172.21.15.39:0/2385907089 wait complete. 2024-07-04T08:21:50.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.976+0000 7f702901d740 20 client.27164 trim_cache size 0 max 16384 2024-07-04T08:21:50.972 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:21:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:21:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:21:50.972 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 20, 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "90f50498": { 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 20, 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 143.9, 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:21:50.973 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:21:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:21:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:21:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:21:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:50.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:21:50.974 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:21:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 -- 172.21.15.39:0/2765945396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70000b2960 msgr2=0x7f70000b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70000b2960 0x7f70000b4e70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7014011890 tx=0x7f7014009500 comp rx=0 tx=0).stop 2024-07-04T08:21:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 -- 172.21.15.39:0/2765945396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c1050f0 msgr2=0x7f701c19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c1050f0 0x7f701c19ba20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f700c005940 tx=0x7f700c0040c0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 -- 172.21.15.39:0/2765945396 shutdown_connections 2024-07-04T08:21:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f70000b2960 0x7f70000b4e70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f701c10b380 0x7f701c1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f701c10aa30 0x7f701c19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 --2- 172.21.15.39:0/2765945396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c1050f0 0x7f701c19ba20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 -- 172.21.15.39:0/2765945396 >> 172.21.15.39:0/2765945396 conn(0x7f701c100b60 msgr2=0x7f701c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.977+0000 7f7021400640 1 -- 172.21.15.39:0/2765945396 shutdown_connections 2024-07-04T08:21:50.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:50.978+0000 7f7021400640 1 -- 172.21.15.39:0/2765945396 wait complete. 2024-07-04T08:21:51.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:51 smithi133 ceph-mon[37050]: pgmap v679: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-04T08:21:51.345 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:51 smithi039 ceph-mon[32062]: pgmap v679: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-04T08:21:51.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:51 smithi050 ceph-mon[36577]: pgmap v679: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-07-04T08:21:52.988 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '90f50498', total seconds elapsed: 36.2 2024-07-04T08:21:53.238 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:53 smithi039 ceph-mon[32062]: pgmap v680: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-07-04T08:21:53.238 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:53 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:21:53.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:53 smithi133 ceph-mon[37050]: pgmap v680: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-07-04T08:21:53.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:53 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:21:53.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:53 smithi050 ceph-mon[36577]: pgmap v680: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-07-04T08:21:53.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:53 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:21:53.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.634+0000 7f805fe00640 1 -- 172.21.15.39:0/4245839149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f805810b590 msgr2=0x7f8058111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.634+0000 7f805fe00640 1 --2- 172.21.15.39:0/4245839149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f805810b590 0x7f8058111dc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8050009ca0 tx=0x7f805002a8b0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.634+0000 7f805fe00640 1 -- 172.21.15.39:0/4245839149 shutdown_connections 2024-07-04T08:21:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.634+0000 7f805fe00640 1 --2- 172.21.15.39:0/4245839149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f805810b590 0x7f8058111dc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.634+0000 7f805fe00640 1 --2- 172.21.15.39:0/4245839149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f805810ac40 0x7f805810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.634+0000 7f805fe00640 1 --2- 172.21.15.39:0/4245839149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8058105300 0x7f8058105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.634+0000 7f805fe00640 1 -- 172.21.15.39:0/4245839149 >> 172.21.15.39:0/4245839149 conn(0x7f8058100d10 msgr2=0x7f8058103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.634+0000 7f805fe00640 1 -- 172.21.15.39:0/4245839149 shutdown_connections 2024-07-04T08:21:53.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.634+0000 7f805fe00640 1 -- 172.21.15.39:0/4245839149 wait complete. 2024-07-04T08:21:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.635+0000 7f805fe00640 1 Processor -- start 2024-07-04T08:21:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.636+0000 7f805fe00640 1 -- start start 2024-07-04T08:21:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.636+0000 7f805fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8058105300 0x7f805819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.636+0000 7f805fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f805810ac40 0x7f805819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.636+0000 7f805fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f805810b590 0x7f80581a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.636+0000 7f805fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8058115290 con 0x7f8058105300 2024-07-04T08:21:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.636+0000 7f805fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8058115110 con 0x7f805810b590 2024-07-04T08:21:53.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.636+0000 7f805fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8058115410 con 0x7f805810ac40 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8058105300 0x7f805819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f805810ac40 0x7f805819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8058105300 0x7f805819bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35576/0 (socket says 172.21.15.39:35576) 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805d400640 1 -- 172.21.15.39:0/3987387821 learned_addr learned my addr 172.21.15.39:0/3987387821 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805de00640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f805810b590 0x7f80581a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805d400640 1 -- 172.21.15.39:0/3987387821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f805810ac40 msgr2=0x7f805819c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805d400640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f805810ac40 0x7f805819c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805d400640 1 -- 172.21.15.39:0/3987387821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f805810b590 msgr2=0x7f80581a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805d400640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f805810b590 0x7f80581a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.637+0000 7f805d400640 1 -- 172.21.15.39:0/3987387821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80581a1b90 con 0x7f8058105300 2024-07-04T08:21:53.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.638+0000 7f805d400640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8058105300 0x7f805819bc20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f804c00d950 tx=0x7f804c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:53.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.638+0000 7f8056000640 1 -- 172.21.15.39:0/3987387821 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c017030 con 0x7f8058105300 2024-07-04T08:21:53.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.638+0000 7f805fe00640 1 -- 172.21.15.39:0/3987387821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80581a1e80 con 0x7f8058105300 2024-07-04T08:21:53.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.638+0000 7f8056000640 1 -- 172.21.15.39:0/3987387821 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f804c004020 con 0x7f8058105300 2024-07-04T08:21:53.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.638+0000 7f8056000640 1 -- 172.21.15.39:0/3987387821 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c0047a0 con 0x7f8058105300 2024-07-04T08:21:53.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.638+0000 7f805fe00640 1 -- 172.21.15.39:0/3987387821 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80581a85e0 con 0x7f8058105300 2024-07-04T08:21:53.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.641+0000 7f8066427740 1 Processor -- start 2024-07-04T08:21:53.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.642+0000 7f8066427740 1 -- start start 2024-07-04T08:21:53.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.642+0000 7f8066427740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a174c6070 0x558a1786a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.642+0000 7f8066427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a1749d1b0 0x558a1786ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.642+0000 7f8066427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a1786b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.642+0000 7f8066427740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a177e0ed0 con 0x558a174c6070 2024-07-04T08:21:53.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.642+0000 7f8066427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558a177d0470 con 0x558a1749d1b0 2024-07-04T08:21:53.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.642+0000 7f8066427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a177d44b0 con 0x558a174a5390 2024-07-04T08:21:53.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.642+0000 7f805de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a174c6070 0x558a1786a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f805de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a174c6070 0x558a1786a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35590/0 (socket says 172.21.15.39:35590) 2024-07-04T08:21:53.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f805de00640 1 -- 172.21.15.39:0/1152205171 learned_addr learned my addr 172.21.15.39:0/1152205171 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:53.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f805ca00640 1 --2- 172.21.15.39:0/1152205171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a1786b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f8056000640 1 -- 172.21.15.39:0/3987387821 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f804c009a30 con 0x7f8058105300 2024-07-04T08:21:53.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f805de00640 1 -- 172.21.15.39:0/1152205171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 msgr2=0x558a1786b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:53.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f805de00640 1 --2- 172.21.15.39:0/1152205171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a1786b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f805de00640 1 -- 172.21.15.39:0/1152205171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a1749d1b0 msgr2=0x558a1786ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:53.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f805de00640 1 --2- 172.21.15.39:0/1152205171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a1749d1b0 0x558a1786ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f805de00640 1 -- 172.21.15.39:0/1152205171 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a17768640 con 0x558a174c6070 2024-07-04T08:21:53.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.643+0000 7f805ca00640 1 --2- 172.21.15.39:0/1152205171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a1786b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:53.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.644+0000 7f805de00640 1 --2- 172.21.15.39:0/1152205171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a174c6070 0x558a1786a7a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8050009ca0 tx=0x7f8050009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10b848012259663 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:53.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.644+0000 7f804aa00640 1 -- 172.21.15.39:0/1152205171 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050030800 con 0x558a174c6070 2024-07-04T08:21:53.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.644+0000 7f804aa00640 1 -- 172.21.15.39:0/1152205171 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8050036030 con 0x558a174c6070 2024-07-04T08:21:53.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.644+0000 7f804aa00640 1 -- 172.21.15.39:0/1152205171 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050004500 con 0x558a174c6070 2024-07-04T08:21:53.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f8056000640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f80400b2960 0x7f80400b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f8066427740 1 -- 172.21.15.39:0/1152205171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a174c6070 msgr2=0x558a1786a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f8066427740 1 --2- 172.21.15.39:0/1152205171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a174c6070 0x558a1786a7a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8050009ca0 tx=0x7f8050009950 comp rx=0 tx=0).stop 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f8056000640 1 -- 172.21.15.39:0/3987387821 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f804c0e2380 con 0x7f8058105300 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f805d400640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f80400b2960 0x7f80400b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f8066427740 1 -- 172.21.15.39:0/1152205171 shutdown_connections 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f8066427740 1 --2- 172.21.15.39:0/1152205171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a1786b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f8066427740 1 --2- 172.21.15.39:0/1152205171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a1749d1b0 0x558a1786ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f8066427740 1 --2- 172.21.15.39:0/1152205171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a174c6070 0x558a1786a7a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.645+0000 7f8066427740 1 -- 172.21.15.39:0/1152205171 >> 172.21.15.39:0/1152205171 conn(0x558a174ba460 msgr2=0x558a17868690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.646+0000 7f8066427740 1 -- 172.21.15.39:0/1152205171 shutdown_connections 2024-07-04T08:21:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.646+0000 7f8066427740 1 -- 172.21.15.39:0/1152205171 wait complete. 2024-07-04T08:21:53.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.646+0000 7f8066427740 1 -- start start 2024-07-04T08:21:53.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.646+0000 7f805d400640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f80400b2960 0x7f80400b4e70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x558a17869420 tx=0x7f804c0046c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:53.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.647+0000 7f8066427740 1 Processor -- start 2024-07-04T08:21:53.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.647+0000 7f8066427740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.647+0000 7f8066427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a178768e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.647+0000 7f8066427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 0x558a17882240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.647+0000 7f8066427740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558a17634700 con 0x558a1749d1b0 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.647+0000 7f8066427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558a17777ba0 con 0x558a174a5390 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.647+0000 7f8066427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558a17878f20 con 0x558a174c6070 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.647+0000 7f8066427740 10 client.0 _ll_register_callbacks cb 0x558a177c9560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.647+0000 7f805de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178763a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35592/0 (socket says 172.21.15.39:35592) 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805de00640 1 -- 172.21.15.39:0/3345808864 learned_addr learned my addr 172.21.15.39:0/3345808864 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805ca00640 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a178768e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805de00640 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 0x558a17882240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805de00640 1 -- 172.21.15.39:0/3345808864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 msgr2=0x558a17882240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805de00640 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 0x558a17882240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805de00640 1 -- 172.21.15.39:0/3345808864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 msgr2=0x558a178768e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:53.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805de00640 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a178768e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805de00640 1 -- 172.21.15.39:0/3345808864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a17531650 con 0x558a1749d1b0 2024-07-04T08:21:53.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f805de00640 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178763a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f805002add0 tx=0x7f80500040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1924bc6b9109d504 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:53.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.648+0000 7f803f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:53.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.649+0000 7f803f400640 1 -- 172.21.15.39:0/3345808864 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050058060 con 0x558a1749d1b0 2024-07-04T08:21:53.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.649+0000 7f803f400640 1 -- 172.21.15.39:0/3345808864 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f8050030a10 con 0x558a1749d1b0 2024-07-04T08:21:53.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.649+0000 7f8066427740 1 -- 172.21.15.39:0/3345808864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8050036030 con 0x558a1749d1b0 2024-07-04T08:21:53.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.649+0000 7f803f400640 1 -- 172.21.15.39:0/3345808864 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f805005f050 con 0x558a1749d1b0 2024-07-04T08:21:53.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.649+0000 7f8066427740 10 client.17034 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.649+0000 7f8066427740 1 -- 172.21.15.39:0/3345808864 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a17884f60 con 0x558a1749d1b0 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.649+0000 7f803f400640 1 -- 172.21.15.39:0/3345808864 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8050036030 con 0x558a1749d1b0 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.650+0000 7f803f400640 1 -- 172.21.15.39:0/3345808864 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8050050070 con 0x558a1749d1b0 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.650+0000 7f8066427740 10 client.17034 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.650+0000 7f8066427740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.650+0000 7f8066427740 20 client.17034 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.650+0000 7f8066427740 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a174849f0 0x558a178882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.650+0000 7f8066427740 4 client.17034 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.650+0000 7f8066427740 1 -- 172.21.15.39:0/3345808864 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a177e2e20 con 0x558a174849f0 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.650+0000 7f805ca00640 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a174849f0 0x558a178882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.651+0000 7f805ca00640 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a174849f0 0x558a178882c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd4b9576e557e8c2 server_cookie=680ba0b30699bfb6 in_seq=0 out_seq=0 2024-07-04T08:21:53.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.651+0000 7f803f400640 10 client.17034 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:53.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.652+0000 7f803f400640 1 -- 172.21.15.39:0/3345808864 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558a177e2e20 con 0x558a174849f0 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.652+0000 7f803f400640 10 client.17034 handle_command_reply: tid=0 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.652+0000 7f8066427740 1 client.17034 shutdown 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.652+0000 7f8066427740 1 -- 172.21.15.39:0/3345808864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 msgr2=0x558a178763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.652+0000 7f8066427740 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178763a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f805002add0 tx=0x7f80500040e0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.652+0000 7f8066427740 1 -- 172.21.15.39:0/3345808864 shutdown_connections 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.653+0000 7f8066427740 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a174849f0 0x558a178882c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.653+0000 7f8066427740 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 0x558a17882240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.653+0000 7f8066427740 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a178768e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.653+0000 7f8066427740 1 --2- 172.21.15.39:0/3345808864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178763a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.653+0000 7f803f400640 0 client.17034 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.653+0000 7f8066427740 1 -- 172.21.15.39:0/3345808864 >> 172.21.15.39:0/3345808864 conn(0x558a174ba460 msgr2=0x558a1786d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.653+0000 7f8066427740 1 -- 172.21.15.39:0/3345808864 shutdown_connections 2024-07-04T08:21:53.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.653+0000 7f8066427740 1 -- 172.21.15.39:0/3345808864 wait complete. 2024-07-04T08:21:53.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.653+0000 7f8066427740 20 client.17034 trim_cache size 0 max 16384 2024-07-04T08:21:53.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.673+0000 7f8066427740 1 Processor -- start 2024-07-04T08:21:53.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.673+0000 7f8066427740 1 -- start start 2024-07-04T08:21:53.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.673+0000 7f8066427740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a17878080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.673+0000 7f8066427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a1786bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.673+0000 7f8066427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 0x558a178821b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.673+0000 7f8066427740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f80581b2480 con 0x558a1749d1b0 2024-07-04T08:21:53.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.673+0000 7f8066427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80581b3fc0 con 0x558a174a5390 2024-07-04T08:21:53.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.673+0000 7f8066427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80581b2b50 con 0x558a174c6070 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a17878080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a1786bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a1786bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52854/0 (socket says 172.21.15.39:52854) 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805ca00640 1 -- 172.21.15.39:0/851436154 learned_addr learned my addr 172.21.15.39:0/851436154 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805de00640 1 --2- 172.21.15.39:0/851436154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a17878080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35594/0 (socket says 172.21.15.39:35594) 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805de00640 1 --2- 172.21.15.39:0/851436154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 0x558a178821b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805ca00640 1 -- 172.21.15.39:0/851436154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 msgr2=0x558a178821b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805ca00640 1 --2- 172.21.15.39:0/851436154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 0x558a178821b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805ca00640 1 -- 172.21.15.39:0/851436154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 msgr2=0x558a17878080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805ca00640 1 --2- 172.21.15.39:0/851436154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a17878080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.674+0000 7f805ca00640 1 -- 172.21.15.39:0/851436154 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a178770f0 con 0x558a174a5390 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.675+0000 7f805ca00640 1 --2- 172.21.15.39:0/851436154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a1786bee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8058103400 tx=0x7f8044005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eac31dffb4d7275 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.675+0000 7f804a000640 1 -- 172.21.15.39:0/851436154 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044022030 con 0x558a174a5390 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.675+0000 7f804a000640 1 -- 172.21.15.39:0/851436154 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80440035f0 con 0x558a174a5390 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.675+0000 7f804a000640 1 -- 172.21.15.39:0/851436154 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804401e640 con 0x558a174a5390 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.676+0000 7f8066427740 1 -- 172.21.15.39:0/851436154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 msgr2=0x558a1786bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.676+0000 7f8066427740 1 --2- 172.21.15.39:0/851436154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a1786bee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8058103400 tx=0x7f8044005f90 comp rx=0 tx=0).stop 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.676+0000 7f8066427740 1 -- 172.21.15.39:0/851436154 shutdown_connections 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.676+0000 7f8066427740 1 --2- 172.21.15.39:0/851436154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174c6070 0x558a178821b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.676+0000 7f8066427740 1 --2- 172.21.15.39:0/851436154 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174a5390 0x558a1786bee0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.676+0000 7f8066427740 1 --2- 172.21.15.39:0/851436154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a17878080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.676+0000 7f8066427740 1 -- 172.21.15.39:0/851436154 >> 172.21.15.39:0/851436154 conn(0x558a174ba460 msgr2=0x558a1786b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:53.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.676+0000 7f8066427740 1 -- 172.21.15.39:0/851436154 shutdown_connections 2024-07-04T08:21:53.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.676+0000 7f8066427740 1 -- 172.21.15.39:0/851436154 wait complete. 2024-07-04T08:21:53.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.677+0000 7f8066427740 1 -- start start 2024-07-04T08:21:53.673 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.677+0000 7f8066427740 1 Processor -- start 2024-07-04T08:21:53.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f8066427740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178873c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f8066427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a17887900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f8066427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174c6070 0x558a17887e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f8066427740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f80581b38f0 con 0x558a1749d1b0 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f8066427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f80581b3220 con 0x558a174c6070 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f8066427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80581b0460 con 0x558a174a5390 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f8066427740 10 client.0 _ll_register_callbacks cb 0x7f80440097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f805de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178873c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f805ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a17887900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f805de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178873c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35610/0 (socket says 172.21.15.39:35610) 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f805de00640 1 -- 172.21.15.39:0/4161904918 learned_addr learned my addr 172.21.15.39:0/4161904918 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:53.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.678+0000 7f805de00640 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174c6070 0x558a17887e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f805de00640 1 -- 172.21.15.39:0/4161904918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 msgr2=0x558a17887900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f805de00640 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a17887900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f805de00640 1 -- 172.21.15.39:0/4161904918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174c6070 msgr2=0x558a17887e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f805de00640 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174c6070 0x558a17887e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f805de00640 1 -- 172.21.15.39:0/4161904918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a178efa00 con 0x558a1749d1b0 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f805ca00640 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a17887900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f805de00640 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178873c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8050004af0 tx=0x7f8050000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebb386f3001cd18 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f803f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f803f400640 1 -- 172.21.15.39:0/4161904918 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050058040 con 0x558a1749d1b0 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f803f400640 1 -- 172.21.15.39:0/4161904918 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f805004a040 con 0x558a1749d1b0 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.679+0000 7f8066427740 1 -- 172.21.15.39:0/4161904918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80440035f0 con 0x558a1749d1b0 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f803f400640 1 -- 172.21.15.39:0/4161904918 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f805006f050 con 0x558a1749d1b0 2024-07-04T08:21:53.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f8066427740 10 client.17040 fetch_fsmap learned FSMap version 30 2024-07-04T08:21:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f8066427740 1 -- 172.21.15.39:0/4161904918 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a178f0c90 con 0x558a1749d1b0 2024-07-04T08:21:53.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f803f400640 1 -- 172.21.15.39:0/4161904918 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80440035f0 con 0x558a1749d1b0 2024-07-04T08:21:53.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f803f400640 1 -- 172.21.15.39:0/4161904918 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8050050040 con 0x558a1749d1b0 2024-07-04T08:21:53.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f8066427740 10 client.17040 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:21:53.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f8066427740 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:21:53.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f8066427740 20 client.17040 populate_metadata read hostname 'smithi039' 2024-07-04T08:21:53.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f8066427740 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a17459da0 0x558a178f3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f8066427740 4 client.17040 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.680+0000 7f8066427740 1 -- 172.21.15.39:0/4161904918 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a177de8c0 con 0x558a17459da0 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.681+0000 7f805ca00640 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a17459da0 0x558a178f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.681+0000 7f805ca00640 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a17459da0 0x558a178f3420 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d108295699869d68 server_cookie=4231b17f9032d7da in_seq=0 out_seq=0 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.681+0000 7f803f400640 10 client.17040 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f803f400640 1 -- 172.21.15.39:0/4161904918 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558a177de8c0 con 0x558a17459da0 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f803f400640 10 client.17040 handle_command_reply: tid=0 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 client.17040 shutdown 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 -- 172.21.15.39:0/4161904918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 msgr2=0x558a178873c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178873c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8050004af0 tx=0x7f8050000f40 comp rx=0 tx=0).stop 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 -- 172.21.15.39:0/4161904918 shutdown_connections 2024-07-04T08:21:53.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558a17459da0 0x558a178f3420 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558a174c6070 0x558a17887e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558a174a5390 0x558a17887900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 --2- 172.21.15.39:0/4161904918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558a1749d1b0 0x558a178873c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 -- 172.21.15.39:0/4161904918 >> 172.21.15.39:0/4161904918 conn(0x558a174ba460 msgr2=0x558a17877de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f803f400640 0 client.17040 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 -- 172.21.15.39:0/4161904918 shutdown_connections 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.682+0000 7f8066427740 1 -- 172.21.15.39:0/4161904918 wait complete. 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f8066427740 20 client.17040 trim_cache size 0 max 16384 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:21:53.680 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:21:53.681 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:21:53.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 -- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f80400b2960 msgr2=0x7f80400b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:53.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f80400b2960 0x7f80400b4e70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x558a17869420 tx=0x7f804c0046c0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 -- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8058105300 msgr2=0x7f805819bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8058105300 0x7f805819bc20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f804c00d950 tx=0x7f804c00de10 comp rx=0 tx=0).stop 2024-07-04T08:21:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 -- 172.21.15.39:0/3987387821 shutdown_connections 2024-07-04T08:21:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f80400b2960 0x7f80400b4e70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f805810b590 0x7f80581a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f805810ac40 0x7f805819c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 --2- 172.21.15.39:0/3987387821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8058105300 0x7f805819bc20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.683+0000 7f805fe00640 1 -- 172.21.15.39:0/3987387821 >> 172.21.15.39:0/3987387821 conn(0x7f8058100d10 msgr2=0x7f805810a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.684+0000 7f805fe00640 1 -- 172.21.15.39:0/3987387821 shutdown_connections 2024-07-04T08:21:53.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:53.684+0000 7f805fe00640 1 -- 172.21.15.39:0/3987387821 wait complete. 2024-07-04T08:21:54.630 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:21:54.630 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:21:55.112 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:21:55.345 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:55 smithi039 ceph-mon[32062]: pgmap v681: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-04T08:21:55.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:55 smithi050 ceph-mon[36577]: pgmap v681: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-04T08:21:55.560 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:55 smithi133 ceph-mon[37050]: pgmap v681: 129 pgs: 129 active+clean; 6.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-04T08:21:57.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:57 smithi039 ceph-mon[32062]: pgmap v682: 129 pgs: 129 active+clean; 7.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-07-04T08:21:57.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:57 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:21:57.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:57 smithi050 ceph-mon[36577]: pgmap v682: 129 pgs: 129 active+clean; 7.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-07-04T08:21:57.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:57 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:21:57.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.445+0000 7f1b34600640 1 -- 172.21.15.39:0/2685142819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c104870 msgr2=0x7f1b2c10cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:57.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.445+0000 7f1b34600640 1 --2- 172.21.15.39:0/2685142819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c104870 0x7f1b2c10cdc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009900 tx=0x7f1b1802a400 comp rx=0 tx=0).stop 2024-07-04T08:21:57.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.445+0000 7f1b34600640 1 -- 172.21.15.39:0/2685142819 shutdown_connections 2024-07-04T08:21:57.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.445+0000 7f1b34600640 1 --2- 172.21.15.39:0/2685142819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c10d380 0x7f1b2c10f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.445+0000 7f1b34600640 1 --2- 172.21.15.39:0/2685142819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c104870 0x7f1b2c10cdc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.445+0000 7f1b34600640 1 --2- 172.21.15.39:0/2685142819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b2c103f20 0x7f1b2c104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.445+0000 7f1b34600640 1 -- 172.21.15.39:0/2685142819 >> 172.21.15.39:0/2685142819 conn(0x7f1b2c078c90 msgr2=0x7f1b2c0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:57.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.445+0000 7f1b34600640 1 -- 172.21.15.39:0/2685142819 shutdown_connections 2024-07-04T08:21:57.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.445+0000 7f1b34600640 1 -- 172.21.15.39:0/2685142819 wait complete. 2024-07-04T08:21:57.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.447+0000 7f1b34600640 1 Processor -- start 2024-07-04T08:21:57.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.447+0000 7f1b34600640 1 -- start start 2024-07-04T08:21:57.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b34600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c103f20 0x7f1b2c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:57.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b34600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b2c104870 0x7f1b2c1a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:57.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b34600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c10d380 0x7f1b2c1abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:57.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b34600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c112ff0 con 0x7f1b2c104870 2024-07-04T08:21:57.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b34600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c112e70 con 0x7f1b2c103f20 2024-07-04T08:21:57.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b34600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c113170 con 0x7f1b2c10d380 2024-07-04T08:21:57.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b31c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c103f20 0x7f1b2c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:57.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b31c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c103f20 0x7f1b2c1a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52884/0 (socket says 172.21.15.39:52884) 2024-07-04T08:21:57.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b31c00640 1 -- 172.21.15.39:0/1298551959 learned_addr learned my addr 172.21.15.39:0/1298551959 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:21:57.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.448+0000 7f1b32600640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c10d380 0x7f1b2c1abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:57.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.449+0000 7f1b31200640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b2c104870 0x7f1b2c1a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:57.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.449+0000 7f1b31c00640 1 -- 172.21.15.39:0/1298551959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c10d380 msgr2=0x7f1b2c1abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:57.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.449+0000 7f1b31c00640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c10d380 0x7f1b2c1abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.449+0000 7f1b31c00640 1 -- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b2c104870 msgr2=0x7f1b2c1a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:57.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.449+0000 7f1b31c00640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b2c104870 0x7f1b2c1a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.449+0000 7f1b31c00640 1 -- 172.21.15.39:0/1298551959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c1a1ac0 con 0x7f1b2c103f20 2024-07-04T08:21:57.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.449+0000 7f1b31200640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b2c104870 0x7f1b2c1a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:21:57.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.449+0000 7f1b31c00640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c103f20 0x7f1b2c1a0cd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000e9c0 tx=0x7f1b2000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:57.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.450+0000 7f1b2a000640 1 -- 172.21.15.39:0/1298551959 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2000cd80 con 0x7f1b2c103f20 2024-07-04T08:21:57.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.450+0000 7f1b2a000640 1 -- 172.21.15.39:0/1298551959 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b200035f0 con 0x7f1b2c103f20 2024-07-04T08:21:57.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.450+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c1ac3b0 con 0x7f1b2c103f20 2024-07-04T08:21:57.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.450+0000 7f1b2a000640 1 -- 172.21.15.39:0/1298551959 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20010690 con 0x7f1b2c103f20 2024-07-04T08:21:57.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.450+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c1ac8c0 con 0x7f1b2c103f20 2024-07-04T08:21:57.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.451+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c06b790 con 0x7f1b2c103f20 2024-07-04T08:21:57.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.454+0000 7f1b2a000640 1 -- 172.21.15.39:0/1298551959 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1b200037a0 con 0x7f1b2c103f20 2024-07-04T08:21:57.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.455+0000 7f1b2a000640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b140b2960 0x7f1b140b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:21:57.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.455+0000 7f1b2a000640 1 -- 172.21.15.39:0/1298551959 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f1b200e2e70 con 0x7f1b2c103f20 2024-07-04T08:21:57.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.455+0000 7f1b31200640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b140b2960 0x7f1b140b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:21:57.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.456+0000 7f1b31200640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b140b2960 0x7f1b140b4e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1b18005ab0 tx=0x7f1b18005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:21:57.454 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.459+0000 7f1b2a000640 1 -- 172.21.15.39:0/1298551959 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f1b200aa4c0 con 0x7f1b2c103f20 2024-07-04T08:21:57.560 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:57 smithi133 ceph-mon[37050]: pgmap v682: 129 pgs: 129 active+clean; 7.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-07-04T08:21:57.560 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:57 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:21:57.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.750+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b2c10c7c0 con 0x7f1b2c103f20 2024-07-04T08:21:57.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.751+0000 7f1b2a000640 1 -- 172.21.15.39:0/1298551959 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f1b2c10c7c0 con 0x7f1b2c103f20 2024-07-04T08:21:57.747 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:21:57.747 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:21:57.748 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:21:57.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.754+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b140b2960 msgr2=0x7f1b140b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:57.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.754+0000 7f1b34600640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b140b2960 0x7f1b140b4e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1b18005ab0 tx=0x7f1b18005fd0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.754+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c103f20 msgr2=0x7f1b2c1a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:21:57.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.754+0000 7f1b34600640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c103f20 0x7f1b2c1a0cd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000e9c0 tx=0x7f1b2000ee80 comp rx=0 tx=0).stop 2024-07-04T08:21:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.754+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 shutdown_connections 2024-07-04T08:21:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.754+0000 7f1b34600640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1b140b2960 0x7f1b140b4e70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.754+0000 7f1b34600640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b2c104870 0x7f1b2c1a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.754+0000 7f1b34600640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c10d380 0x7f1b2c1abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.754+0000 7f1b34600640 1 --2- 172.21.15.39:0/1298551959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1b2c103f20 0x7f1b2c1a0cd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:21:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.755+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 >> 172.21.15.39:0/1298551959 conn(0x7f1b2c078c90 msgr2=0x7f1b2c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:21:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.755+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 shutdown_connections 2024-07-04T08:21:57.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:21:57.755+0000 7f1b34600640 1 -- 172.21.15.39:0/1298551959 wait complete. 2024-07-04T08:21:58.376 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.638 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.638 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.639 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.639 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:58 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.639 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:58 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1298551959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:58.639 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:58 smithi050 ceph-mon[36577]: pgmap v683: 129 pgs: 129 active+clean; 7.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-07-04T08:21:58.759 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.760 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.760 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:58 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.760 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:58 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1298551959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:58.760 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:58 smithi039 ceph-mon[32062]: pgmap v683: 129 pgs: 129 active+clean; 7.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-07-04T08:21:58.810 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.810 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.810 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.810 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:58 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:21:58.810 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:58 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1298551959' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:21:58.811 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:58 smithi133 ceph-mon[37050]: pgmap v683: 129 pgs: 129 active+clean; 7.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-07-04T08:21:58.824 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:21:59.412 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:00.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:21:59 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:21:59 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:00.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:21:59 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:01.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:00 smithi050 ceph-mon[36577]: pgmap v684: 129 pgs: 129 active+clean; 7.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-07-04T08:22:01.178 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:00 smithi039 ceph-mon[32062]: pgmap v684: 129 pgs: 129 active+clean; 7.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-07-04T08:22:01.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:00 smithi133 ceph-mon[37050]: pgmap v684: 129 pgs: 129 active+clean; 7.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-07-04T08:22:01.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.734+0000 7f784aa00640 1 -- 172.21.15.39:0/2596208521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78440755d0 msgr2=0x7f784410f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:01.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.734+0000 7f784aa00640 1 --2- 172.21.15.39:0/2596208521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78440755d0 0x7f784410f170 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f782c009900 tx=0x7f782c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.735+0000 7f784aa00640 1 -- 172.21.15.39:0/2596208521 shutdown_connections 2024-07-04T08:22:01.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.735+0000 7f784aa00640 1 --2- 172.21.15.39:0/2596208521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f784410f6b0 0x7f7844111ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.735+0000 7f784aa00640 1 --2- 172.21.15.39:0/2596208521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78440755d0 0x7f784410f170 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.735+0000 7f784aa00640 1 --2- 172.21.15.39:0/2596208521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7844074c80 0x7f7844075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.735+0000 7f784aa00640 1 -- 172.21.15.39:0/2596208521 >> 172.21.15.39:0/2596208521 conn(0x7f78440feb20 msgr2=0x7f7844100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:01.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.735+0000 7f784aa00640 1 -- 172.21.15.39:0/2596208521 shutdown_connections 2024-07-04T08:22:01.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.735+0000 7f784aa00640 1 -- 172.21.15.39:0/2596208521 wait complete. 2024-07-04T08:22:01.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.737+0000 7f784aa00640 1 Processor -- start 2024-07-04T08:22:01.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.737+0000 7f784aa00640 1 -- start start 2024-07-04T08:22:01.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.737+0000 7f784aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7844074c80 0x7f784419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.737+0000 7f784aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78440755d0 0x7f784419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.737+0000 7f784aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f784410f6b0 0x7f78441a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f784aa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7844115220 con 0x7f78440755d0 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f784aa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78441150a0 con 0x7f784410f6b0 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f784aa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78441153a0 con 0x7f7844074c80 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f783b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78440755d0 0x7f784419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f7843e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7844074c80 0x7f784419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f7843e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7844074c80 0x7f784419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39802/0 (socket says 172.21.15.39:39802) 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f783b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78440755d0 0x7f784419c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59078/0 (socket says 172.21.15.39:59078) 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f7843e00640 1 -- 172.21.15.39:0/1993792157 learned_addr learned my addr 172.21.15.39:0/1993792157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:01.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f7848a00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f784410f6b0 0x7f78441a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f783b400640 1 -- 172.21.15.39:0/1993792157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7844074c80 msgr2=0x7f784419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:01.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f783b400640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7844074c80 0x7f784419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f783b400640 1 -- 172.21.15.39:0/1993792157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f784410f6b0 msgr2=0x7f78441a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f783b400640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f784410f6b0 0x7f78441a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.738+0000 7f783b400640 1 -- 172.21.15.39:0/1993792157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78441a1b80 con 0x7f78440755d0 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.739+0000 7f7848a00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f784410f6b0 0x7f78441a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.739+0000 7f7843e00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7844074c80 0x7f784419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.739+0000 7f783b400640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78440755d0 0x7f784419c1b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f782c009230 tx=0x7f782c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.739+0000 7f7841600640 1 -- 172.21.15.39:0/1993792157 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c03f070 con 0x7f78440755d0 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.739+0000 7f7841600640 1 -- 172.21.15.39:0/1993792157 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f782c031b50 con 0x7f78440755d0 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.739+0000 7f784aa00640 1 -- 172.21.15.39:0/1993792157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78441a1e10 con 0x7f78440755d0 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.739+0000 7f784aa00640 1 -- 172.21.15.39:0/1993792157 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441a85d0 con 0x7f78440755d0 2024-07-04T08:22:01.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.739+0000 7f7841600640 1 -- 172.21.15.39:0/1993792157 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c035030 con 0x7f78440755d0 2024-07-04T08:22:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.743+0000 7f7851020740 1 Processor -- start 2024-07-04T08:22:01.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.744+0000 7f7851020740 1 -- start start 2024-07-04T08:22:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.744+0000 7f7841600640 1 -- 172.21.15.39:0/1993792157 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f782c042070 con 0x7f78440755d0 2024-07-04T08:22:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.744+0000 7f7851020740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.744+0000 7f7851020740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 0x562fcfc60cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.744+0000 7f7851020740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf89b390 0x562fcfc61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.744+0000 7f7851020740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562fcfbd6ee0 con 0x562fcf8931b0 2024-07-04T08:22:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.744+0000 7f7851020740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562fcfbc6480 con 0x562fcf89b390 2024-07-04T08:22:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.744+0000 7f7851020740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562fcfbca4c0 con 0x562fcf8bc070 2024-07-04T08:22:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.745+0000 7f7848a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.745+0000 7f7848a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39812/0 (socket says 172.21.15.39:39812) 2024-07-04T08:22:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.745+0000 7f7848a00640 1 -- 172.21.15.39:0/2447245058 learned_addr learned my addr 172.21.15.39:0/2447245058 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.745+0000 7f7848a00640 1 -- 172.21.15.39:0/2447245058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf89b390 msgr2=0x562fcfc61230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:22:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.745+0000 7f7848a00640 1 --2- 172.21.15.39:0/2447245058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf89b390 0x562fcfc61230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.745+0000 7f7848a00640 1 -- 172.21.15.39:0/2447245058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 msgr2=0x562fcfc60cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.745+0000 7f7848a00640 1 --2- 172.21.15.39:0/2447245058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 0x562fcfc60cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.745+0000 7f7848a00640 1 -- 172.21.15.39:0/2447245058 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fcfb5e650 con 0x562fcf8bc070 2024-07-04T08:22:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.746+0000 7f7841600640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78280b2960 0x7f78280b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.746+0000 7f7841600640 1 -- 172.21.15.39:0/1993792157 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f782c102dd0 con 0x7f78440755d0 2024-07-04T08:22:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.746+0000 7f7843e00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78280b2960 0x7f78280b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.747+0000 7f7848a00640 1 --2- 172.21.15.39:0/2447245058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc607b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f783c011540 tx=0x7f783c011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a23c8f85e30a464 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.747+0000 7f7839600640 1 -- 172.21.15.39:0/2447245058 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c00f5c0 con 0x562fcf8bc070 2024-07-04T08:22:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.747+0000 7f7839600640 1 -- 172.21.15.39:0/2447245058 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c00fc10 con 0x562fcf8bc070 2024-07-04T08:22:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.747+0000 7f7839600640 1 -- 172.21.15.39:0/2447245058 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c019bf0 con 0x562fcf8bc070 2024-07-04T08:22:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7851020740 1 -- 172.21.15.39:0/2447245058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 msgr2=0x562fcfc607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7851020740 1 --2- 172.21.15.39:0/2447245058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc607b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f783c011540 tx=0x7f783c011a00 comp rx=0 tx=0).stop 2024-07-04T08:22:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7843e00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78280b2960 0x7f78280b4e70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7834009aa0 tx=0x7f7834009a30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7851020740 1 -- 172.21.15.39:0/2447245058 shutdown_connections 2024-07-04T08:22:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7851020740 1 --2- 172.21.15.39:0/2447245058 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 0x562fcfc60cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7851020740 1 --2- 172.21.15.39:0/2447245058 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf89b390 0x562fcfc61230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7851020740 1 --2- 172.21.15.39:0/2447245058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc607b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7851020740 1 -- 172.21.15.39:0/2447245058 >> 172.21.15.39:0/2447245058 conn(0x562fcf8b0460 msgr2=0x562fcfc5e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:01.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7851020740 1 -- 172.21.15.39:0/2447245058 shutdown_connections 2024-07-04T08:22:01.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.748+0000 7f7851020740 1 -- 172.21.15.39:0/2447245058 wait complete. 2024-07-04T08:22:01.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.749+0000 7f7851020740 1 -- start start 2024-07-04T08:22:01.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.749+0000 7f7851020740 1 Processor -- start 2024-07-04T08:22:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.750+0000 7f7851020740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc6c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.750+0000 7f7851020740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 0x562fcfc6c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.750+0000 7f7851020740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc78280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.750+0000 7f7851020740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562fcfa2a700 con 0x562fcf89b390 2024-07-04T08:22:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.750+0000 7f7851020740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562fcfb6dbb0 con 0x562fcf8931b0 2024-07-04T08:22:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.750+0000 7f7851020740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562fcfc6ef90 con 0x562fcf8bc070 2024-07-04T08:22:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.750+0000 7f7851020740 10 client.0 _ll_register_callbacks cb 0x562fcfbbf570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f7848a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc78280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f783b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 0x562fcfc6c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f7843e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc6c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f7848a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc78280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39816/0 (socket says 172.21.15.39:39816) 2024-07-04T08:22:01.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f7848a00640 1 -- 172.21.15.39:0/197296149 learned_addr learned my addr 172.21.15.39:0/197296149 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:01.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f7848a00640 1 -- 172.21.15.39:0/197296149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 msgr2=0x562fcfc6c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:01.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f7848a00640 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc6c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f7848a00640 1 -- 172.21.15.39:0/197296149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 msgr2=0x562fcfc6c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:01.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f7848a00640 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 0x562fcfc6c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.751+0000 7f7848a00640 1 -- 172.21.15.39:0/197296149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fcf927650 con 0x562fcf8bc070 2024-07-04T08:22:01.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.752+0000 7f7848a00640 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc78280 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f783c009ce0 tx=0x7f783c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b5fc679cc549e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:01.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.752+0000 7f7832000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.752+0000 7f7832000640 1 -- 172.21.15.39:0/197296149 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c027060 con 0x562fcf8bc070 2024-07-04T08:22:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.752+0000 7f7851020740 1 -- 172.21.15.39:0/197296149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783c00fc10 con 0x562fcf8bc070 2024-07-04T08:22:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.752+0000 7f7832000640 1 -- 172.21.15.39:0/197296149 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f783c00f5c0 con 0x562fcf8bc070 2024-07-04T08:22:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.752+0000 7f7832000640 1 -- 172.21.15.39:0/197296149 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f783c02e050 con 0x562fcf8bc070 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.752+0000 7f7851020740 10 client.27109 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.752+0000 7f7851020740 1 -- 172.21.15.39:0/197296149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562fcfc7aff0 con 0x562fcf8bc070 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.753+0000 7f7832000640 1 -- 172.21.15.39:0/197296149 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c00fc10 con 0x562fcf8bc070 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.753+0000 7f7832000640 1 -- 172.21.15.39:0/197296149 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f783c005300 con 0x562fcf8bc070 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.753+0000 7f7851020740 10 client.27109 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.753+0000 7f7851020740 10 client.27109 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.753+0000 7f7851020740 20 client.27109 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.753+0000 7f7851020740 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x562fcf87a9f0 0x562fcfc7e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.753+0000 7f7851020740 4 client.27109 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.753+0000 7f7851020740 1 -- 172.21.15.39:0/197296149 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fcfbd8e30 con 0x562fcf87a9f0 2024-07-04T08:22:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.754+0000 7f7843e00640 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x562fcf87a9f0 0x562fcfc7e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.755+0000 7f7843e00640 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x562fcf87a9f0 0x562fcfc7e410 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=75f1ce99d4a047be server_cookie=1af222e6b1daa911 in_seq=0 out_seq=0 2024-07-04T08:22:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.755+0000 7f7832000640 10 client.27109 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.755+0000 7f7832000640 1 -- 172.21.15.39:0/197296149 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562fcfbd8e30 con 0x562fcf87a9f0 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.755+0000 7f7832000640 10 client.27109 handle_command_reply: tid=0 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.755+0000 7f7851020740 1 client.27109 shutdown 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 -- 172.21.15.39:0/197296149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 msgr2=0x562fcfc78280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc78280 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f783c009ce0 tx=0x7f783c0040c0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 -- 172.21.15.39:0/197296149 shutdown_connections 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x562fcf87a9f0 0x562fcfc7e410 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 0x562fcfc6c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc78280 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7832000640 0 client.27109 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 --2- 172.21.15.39:0/197296149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc6c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 -- 172.21.15.39:0/197296149 >> 172.21.15.39:0/197296149 conn(0x562fcf8b0460 msgr2=0x562fcfc63940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 -- 172.21.15.39:0/197296149 shutdown_connections 2024-07-04T08:22:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 1 -- 172.21.15.39:0/197296149 wait complete. 2024-07-04T08:22:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.756+0000 7f7851020740 20 client.27109 trim_cache size 0 max 16384 2024-07-04T08:22:01.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.772+0000 7f7851020740 1 Processor -- start 2024-07-04T08:22:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.772+0000 7f7851020740 1 -- start start 2024-07-04T08:22:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.772+0000 7f7851020740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 0x562fcfc621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.772+0000 7f7851020740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf89b390 0x562fcfc6ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.772+0000 7f7851020740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc65840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.772+0000 7f7851020740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78441b2460 con 0x562fcf8931b0 2024-07-04T08:22:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.772+0000 7f7851020740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78441b3fa0 con 0x562fcf89b390 2024-07-04T08:22:01.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.772+0000 7f7851020740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78441b2b30 con 0x562fcf8bc070 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.773+0000 7f7843e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 0x562fcfc621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.773+0000 7f7843e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 0x562fcfc621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59112/0 (socket says 172.21.15.39:59112) 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.773+0000 7f7843e00640 1 -- 172.21.15.39:0/3553032694 learned_addr learned my addr 172.21.15.39:0/3553032694 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.773+0000 7f7848a00640 1 --2- 172.21.15.39:0/3553032694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc65840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.773+0000 7f7848a00640 1 -- 172.21.15.39:0/3553032694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf89b390 msgr2=0x562fcfc6ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.773+0000 7f7848a00640 1 --2- 172.21.15.39:0/3553032694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf89b390 0x562fcfc6ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.773+0000 7f7848a00640 1 -- 172.21.15.39:0/3553032694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 msgr2=0x562fcfc621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.773+0000 7f7848a00640 1 --2- 172.21.15.39:0/3553032694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 0x562fcfc621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.773+0000 7f7848a00640 1 -- 172.21.15.39:0/3553032694 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fcfc6cc90 con 0x562fcf8bc070 2024-07-04T08:22:01.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.774+0000 7f7848a00640 1 --2- 172.21.15.39:0/3553032694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc65840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f783c000c00 tx=0x7f783c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae2461eff4a4df31 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.774+0000 7f7838c00640 1 -- 172.21.15.39:0/3553032694 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c037070 con 0x562fcf8bc070 2024-07-04T08:22:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.774+0000 7f7838c00640 1 -- 172.21.15.39:0/3553032694 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c00fc70 con 0x562fcf8bc070 2024-07-04T08:22:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.774+0000 7f7838c00640 1 -- 172.21.15.39:0/3553032694 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c027040 con 0x562fcf8bc070 2024-07-04T08:22:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.775+0000 7f7851020740 1 -- 172.21.15.39:0/3553032694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 msgr2=0x562fcfc65840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.775+0000 7f7851020740 1 --2- 172.21.15.39:0/3553032694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc65840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f783c000c00 tx=0x7f783c009b30 comp rx=0 tx=0).stop 2024-07-04T08:22:01.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.775+0000 7f7851020740 1 -- 172.21.15.39:0/3553032694 shutdown_connections 2024-07-04T08:22:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.775+0000 7f7851020740 1 --2- 172.21.15.39:0/3553032694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc65840 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.775+0000 7f7851020740 1 --2- 172.21.15.39:0/3553032694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf89b390 0x562fcfc6ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.775+0000 7f7851020740 1 --2- 172.21.15.39:0/3553032694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf8931b0 0x562fcfc621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.775+0000 7f7851020740 1 -- 172.21.15.39:0/3553032694 >> 172.21.15.39:0/3553032694 conn(0x562fcf8b0460 msgr2=0x562fcfc6d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.776+0000 7f7851020740 1 -- 172.21.15.39:0/3553032694 shutdown_connections 2024-07-04T08:22:01.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.776+0000 7f7851020740 1 -- 172.21.15.39:0/3553032694 wait complete. 2024-07-04T08:22:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.776+0000 7f7851020740 1 -- start start 2024-07-04T08:22:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.777+0000 7f7851020740 1 Processor -- start 2024-07-04T08:22:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.777+0000 7f7851020740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc74700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.777+0000 7f7851020740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 0x562fcfc77ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.777+0000 7f7851020740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc77ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.777+0000 7f7851020740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78441b38d0 con 0x562fcf89b390 2024-07-04T08:22:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.777+0000 7f7851020740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78441b3200 con 0x562fcf8931b0 2024-07-04T08:22:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.777+0000 7f7851020740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78441b0440 con 0x562fcf8bc070 2024-07-04T08:22:01.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.777+0000 7f7851020740 10 client.0 _ll_register_callbacks cb 0x7f783c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:01.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7843e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc74700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7843e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc74700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56348/0 (socket says 172.21.15.39:56348) 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7843e00640 1 -- 172.21.15.39:0/2545363134 learned_addr learned my addr 172.21.15.39:0/2545363134 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7848a00640 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc77ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7843e00640 1 -- 172.21.15.39:0/2545363134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 msgr2=0x562fcfc77ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7843e00640 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc77ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7843e00640 1 -- 172.21.15.39:0/2545363134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 msgr2=0x562fcfc77ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f783b400640 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 0x562fcfc77ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7843e00640 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 0x562fcfc77ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7843e00640 1 -- 172.21.15.39:0/2545363134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fcfc785e0 con 0x562fcf8931b0 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f7848a00640 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc77ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:22:01.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.778+0000 7f783b400640 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 0x562fcfc77ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:22:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.779+0000 7f7843e00640 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc74700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x562fcfc5f430 tx=0x7f7834009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d6dfc2d89d6eff6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.779+0000 7f7832000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.779+0000 7f7832000640 1 -- 172.21.15.39:0/2545363134 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834030470 con 0x562fcf8931b0 2024-07-04T08:22:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.779+0000 7f7832000640 1 -- 172.21.15.39:0/2545363134 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9336+0+0 (secure 0 0 0) 0x7f7834030610 con 0x562fcf8931b0 2024-07-04T08:22:01.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.779+0000 7f7851020740 1 -- 172.21.15.39:0/2545363134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562fcfbd48d0 con 0x562fcf8931b0 2024-07-04T08:22:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7832000640 1 -- 172.21.15.39:0/2545363134 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7834039050 con 0x562fcf8931b0 2024-07-04T08:22:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7851020740 10 client.27194 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7851020740 1 -- 172.21.15.39:0/2545363134 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562fcfce6a40 con 0x562fcf8931b0 2024-07-04T08:22:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7832000640 1 -- 172.21.15.39:0/2545363134 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562fcfbd48d0 con 0x562fcf8931b0 2024-07-04T08:22:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7832000640 1 -- 172.21.15.39:0/2545363134 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f783401d4d0 con 0x562fcf8931b0 2024-07-04T08:22:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7851020740 10 client.27194 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:01.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7851020740 10 client.27194 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:22:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7851020740 20 client.27194 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7851020740 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x562fcf84fda0 0x562fcfce9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7851020740 4 client.27194 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.780+0000 7f7851020740 1 -- 172.21.15.39:0/2545363134 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fcfbd2010 con 0x562fcf84fda0 2024-07-04T08:22:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.781+0000 7f783b400640 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x562fcf84fda0 0x562fcfce9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.781+0000 7f783b400640 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x562fcf84fda0 0x562fcfce9070 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e63a7038f45559c5 server_cookie=9e8be4f56a9c3da3 in_seq=0 out_seq=0 2024-07-04T08:22:01.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.781+0000 7f7832000640 10 client.27194 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:22:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7832000640 1 -- 172.21.15.39:0/2545363134 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fcfbd2010 con 0x562fcf84fda0 2024-07-04T08:22:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7832000640 10 client.27194 handle_command_reply: tid=0 2024-07-04T08:22:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 client.27194 shutdown 2024-07-04T08:22:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 -- 172.21.15.39:0/2545363134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 msgr2=0x562fcfc74700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:01.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc74700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x562fcfc5f430 tx=0x7f7834009500 comp rx=0 tx=0).stop 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 -- 172.21.15.39:0/2545363134 shutdown_connections 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x562fcf84fda0 0x562fcfce9070 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fcf89b390 0x562fcfc77ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fcf8bc070 0x562fcfc77ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 --2- 172.21.15.39:0/2545363134 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fcf8931b0 0x562fcfc74700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 -- 172.21.15.39:0/2545363134 >> 172.21.15.39:0/2545363134 conn(0x562fcf8b0460 msgr2=0x562fcfc6dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7832000640 0 client.27194 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 -- 172.21.15.39:0/2545363134 shutdown_connections 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.782+0000 7f7851020740 1 -- 172.21.15.39:0/2545363134 wait complete. 2024-07-04T08:22:01.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.783+0000 7f7851020740 20 client.27194 trim_cache size 0 max 16384 2024-07-04T08:22:01.781 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.783+0000 7f784aa00640 1 -- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78280b2960 msgr2=0x7f78280b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.783+0000 7f784aa00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78280b2960 0x7f78280b4e70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7834009aa0 tx=0x7f7834009a30 comp rx=0 tx=0).stop 2024-07-04T08:22:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.783+0000 7f784aa00640 1 -- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78440755d0 msgr2=0x7f784419c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.783+0000 7f784aa00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78440755d0 0x7f784419c1b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f782c009230 tx=0x7f782c003b90 comp rx=0 tx=0).stop 2024-07-04T08:22:01.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.784+0000 7f784aa00640 1 -- 172.21.15.39:0/1993792157 shutdown_connections 2024-07-04T08:22:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.784+0000 7f784aa00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78280b2960 0x7f78280b4e70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.784+0000 7f784aa00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78440755d0 0x7f784419c1b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.784+0000 7f784aa00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f784410f6b0 0x7f78441a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.784+0000 7f784aa00640 1 --2- 172.21.15.39:0/1993792157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7844074c80 0x7f784419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.784+0000 7f784aa00640 1 -- 172.21.15.39:0/1993792157 >> 172.21.15.39:0/1993792157 conn(0x7f78440feb20 msgr2=0x7f7844109250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.784+0000 7f784aa00640 1 -- 172.21.15.39:0/1993792157 shutdown_connections 2024-07-04T08:22:01.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:01.784+0000 7f784aa00640 1 -- 172.21.15.39:0/1993792157 wait complete. 2024-07-04T08:22:02.702 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:22:03.060 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:03.060 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:02 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:03.060 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:02 smithi133 ceph-mon[37050]: pgmap v685: 129 pgs: 129 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-07-04T08:22:03.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:03.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:02 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:03.096 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:02 smithi039 ceph-mon[32062]: pgmap v685: 129 pgs: 129 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-07-04T08:22:03.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:03.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:02 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:03.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:02 smithi050 ceph-mon[36577]: pgmap v685: 129 pgs: 129 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-07-04T08:22:03.181 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:04.060 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:03 smithi133 ceph-mon[37050]: osdmap e93: 12 total, 12 up, 12 in 2024-07-04T08:22:04.060 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:22:04.061 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:22:04.061 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:04.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:03 smithi039 ceph-mon[32062]: osdmap e93: 12 total, 12 up, 12 in 2024-07-04T08:22:04.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:22:04.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:22:04.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:04.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:03 smithi050 ceph-mon[36577]: osdmap e93: 12 total, 12 up, 12 in 2024-07-04T08:22:04.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:22:04.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:22:04.139 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:22:05.050 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:04 smithi039 ceph-mon[32062]: pgmap v687: 129 pgs: 129 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-07-04T08:22:05.060 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:04 smithi133 ceph-mon[37050]: pgmap v687: 129 pgs: 129 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-07-04T08:22:05.138 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:04 smithi050 ceph-mon[36577]: pgmap v687: 129 pgs: 129 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-07-04T08:22:05.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.647+0000 7f55b1a00640 1 -- 172.21.15.39:0/3071036040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55ac105100 msgr2=0x7f55ac105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:05.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.647+0000 7f55b1a00640 1 --2- 172.21.15.39:0/3071036040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55ac105100 0x7f55ac105510 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f559c009900 tx=0x7f559c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.648+0000 7f55b1a00640 1 -- 172.21.15.39:0/3071036040 shutdown_connections 2024-07-04T08:22:05.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.648+0000 7f55b1a00640 1 --2- 172.21.15.39:0/3071036040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac10b390 0x7f55ac111bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.648+0000 7f55b1a00640 1 --2- 172.21.15.39:0/3071036040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac10aa40 0x7f55ac10ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.648+0000 7f55b1a00640 1 --2- 172.21.15.39:0/3071036040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55ac105100 0x7f55ac105510 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.648+0000 7f55b1a00640 1 -- 172.21.15.39:0/3071036040 >> 172.21.15.39:0/3071036040 conn(0x7f55ac100bd0 msgr2=0x7f55ac103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:05.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.648+0000 7f55b1a00640 1 -- 172.21.15.39:0/3071036040 shutdown_connections 2024-07-04T08:22:05.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.648+0000 7f55b1a00640 1 -- 172.21.15.39:0/3071036040 wait complete. 2024-07-04T08:22:05.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.649+0000 7f55b1a00640 1 Processor -- start 2024-07-04T08:22:05.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.650+0000 7f55b1a00640 1 -- start start 2024-07-04T08:22:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.650+0000 7f55b1a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac105100 0x7f55ac19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.650+0000 7f55b1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac10aa40 0x7f55ac19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.650+0000 7f55b1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55ac10b390 0x7f55ac1a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.650+0000 7f55b1a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac115040 con 0x7f55ac10b390 2024-07-04T08:22:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.650+0000 7f55b1a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac114ec0 con 0x7f55ac105100 2024-07-04T08:22:05.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.650+0000 7f55b1a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac1151c0 con 0x7f55ac10aa40 2024-07-04T08:22:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55b1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55ac10b390 0x7f55ac1a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac10aa40 0x7f55ac19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55abe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac105100 0x7f55ac19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac10aa40 0x7f55ac19c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39848/0 (socket says 172.21.15.39:39848) 2024-07-04T08:22:05.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55a3e00640 1 -- 172.21.15.39:0/354550424 learned_addr learned my addr 172.21.15.39:0/354550424 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55b1000640 1 -- 172.21.15.39:0/354550424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac10aa40 msgr2=0x7f55ac19c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55b1000640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac10aa40 0x7f55ac19c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55b1000640 1 -- 172.21.15.39:0/354550424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac105100 msgr2=0x7f55ac19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55b1000640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac105100 0x7f55ac19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55b1000640 1 -- 172.21.15.39:0/354550424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ac1a19e0 con 0x7f55ac10b390 2024-07-04T08:22:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.651+0000 7f55a3e00640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac10aa40 0x7f55ac19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:05.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.652+0000 7f55b1000640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55ac10b390 0x7f55ac1a12b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009fd0 tx=0x7f55a400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:05.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.652+0000 7f55a9600640 1 -- 172.21.15.39:0/354550424 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a400ce20 con 0x7f55ac10b390 2024-07-04T08:22:05.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.652+0000 7f55a9600640 1 -- 172.21.15.39:0/354550424 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a4005020 con 0x7f55ac10b390 2024-07-04T08:22:05.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.652+0000 7f55b1a00640 1 -- 172.21.15.39:0/354550424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ac1a1cd0 con 0x7f55ac10b390 2024-07-04T08:22:05.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.652+0000 7f55a9600640 1 -- 172.21.15.39:0/354550424 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4010690 con 0x7f55ac10b390 2024-07-04T08:22:05.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.652+0000 7f55b1a00640 1 -- 172.21.15.39:0/354550424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ac1a8430 con 0x7f55ac10b390 2024-07-04T08:22:05.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.656+0000 7f55b9663740 1 Processor -- start 2024-07-04T08:22:05.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.656+0000 7f55a9600640 1 -- 172.21.15.39:0/354550424 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f55a40054d0 con 0x7f55ac10b390 2024-07-04T08:22:05.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.656+0000 7f55b9663740 1 -- start start 2024-07-04T08:22:05.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55b9663740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa1b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55b9663740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a64e1b0 0x563c2aa1beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55b9663740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a656390 0x563c2aa1c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55b9663740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563c2a991f00 con 0x563c2a677070 2024-07-04T08:22:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55b9663740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563c2a9814a0 con 0x563c2a656390 2024-07-04T08:22:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55b9663740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c2a9854e0 con 0x563c2a64e1b0 2024-07-04T08:22:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa1b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa1b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59140/0 (socket says 172.21.15.39:59140) 2024-07-04T08:22:05.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55a3e00640 1 -- 172.21.15.39:0/4050493247 learned_addr learned my addr 172.21.15.39:0/4050493247 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:05.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55abe00640 1 --2- 172.21.15.39:0/4050493247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a64e1b0 0x563c2aa1beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.657+0000 7f55a3e00640 1 --2- 172.21.15.39:0/4050493247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a656390 0x563c2aa1c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.658+0000 7f55abe00640 1 -- 172.21.15.39:0/4050493247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a656390 msgr2=0x563c2aa1c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:05.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.658+0000 7f55abe00640 1 --2- 172.21.15.39:0/4050493247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a656390 0x563c2aa1c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.658+0000 7f55abe00640 1 -- 172.21.15.39:0/4050493247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 msgr2=0x563c2aa1b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:05.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.658+0000 7f55abe00640 1 --2- 172.21.15.39:0/4050493247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa1b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.658+0000 7f55abe00640 1 -- 172.21.15.39:0/4050493247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2a919670 con 0x563c2a64e1b0 2024-07-04T08:22:05.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.658+0000 7f55a9600640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55900b2960 0x7f55900b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.658+0000 7f55abe00640 1 --2- 172.21.15.39:0/4050493247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a64e1b0 0x563c2aa1beb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f559c009230 tx=0x7f559c032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ad59d7d71581223 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:05.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.658+0000 7f55a9600640 1 -- 172.21.15.39:0/354550424 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9512+0+0 (secure 0 0 0) 0x7f55a40e30f0 con 0x7f55ac10b390 2024-07-04T08:22:05.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.659+0000 7f55a3e00640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55900b2960 0x7f55900b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.659+0000 7f55a1600640 1 -- 172.21.15.39:0/4050493247 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c040070 con 0x563c2a64e1b0 2024-07-04T08:22:05.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.659+0000 7f55a1600640 1 -- 172.21.15.39:0/4050493247 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c004030 con 0x563c2a64e1b0 2024-07-04T08:22:05.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.659+0000 7f55a1600640 1 -- 172.21.15.39:0/4050493247 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c035040 con 0x563c2a64e1b0 2024-07-04T08:22:05.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55b9663740 1 -- 172.21.15.39:0/4050493247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a64e1b0 msgr2=0x563c2aa1beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:05.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55b9663740 1 --2- 172.21.15.39:0/4050493247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a64e1b0 0x563c2aa1beb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f559c009230 tx=0x7f559c032b90 comp rx=0 tx=0).stop 2024-07-04T08:22:05.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55a3e00640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55900b2960 0x7f55900b4e70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f55940058c0 tx=0x7f5594009bb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:05.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55b9663740 1 -- 172.21.15.39:0/4050493247 shutdown_connections 2024-07-04T08:22:05.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55b9663740 1 --2- 172.21.15.39:0/4050493247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a656390 0x563c2aa1c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55b9663740 1 --2- 172.21.15.39:0/4050493247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a64e1b0 0x563c2aa1beb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55b9663740 1 --2- 172.21.15.39:0/4050493247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa1b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55b9663740 1 -- 172.21.15.39:0/4050493247 >> 172.21.15.39:0/4050493247 conn(0x563c2a66b460 msgr2=0x563c2aa19860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55b9663740 1 -- 172.21.15.39:0/4050493247 shutdown_connections 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.660+0000 7f55b9663740 1 -- 172.21.15.39:0/4050493247 wait complete. 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.661+0000 7f55b9663740 1 -- start start 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.661+0000 7f55b9663740 1 Processor -- start 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.661+0000 7f55b9663740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 0x563c2aa27520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.661+0000 7f55b9663740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a656390 0x563c2aa27a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55b9663740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a677070 0x563c2aa33410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55b9663740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563c2a7e5700 con 0x563c2a656390 2024-07-04T08:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55b9663740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563c2a928bd0 con 0x563c2a64e1b0 2024-07-04T08:22:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55b9663740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c2aa2a0f0 con 0x563c2a677070 2024-07-04T08:22:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55b9663740 10 client.0 _ll_register_callbacks cb 0x563c2a97a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a656390 0x563c2aa27a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a656390 0x563c2aa27a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59144/0 (socket says 172.21.15.39:59144) 2024-07-04T08:22:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55a3e00640 1 -- 172.21.15.39:0/1415200430 learned_addr learned my addr 172.21.15.39:0/1415200430 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55abe00640 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 0x563c2aa27520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55b1000640 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a677070 0x563c2aa33410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55a3e00640 1 -- 172.21.15.39:0/1415200430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a677070 msgr2=0x563c2aa33410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55a3e00640 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a677070 0x563c2aa33410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55a3e00640 1 -- 172.21.15.39:0/1415200430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 msgr2=0x563c2aa27520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.662+0000 7f55a3e00640 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 0x563c2aa27520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.663+0000 7f55a3e00640 1 -- 172.21.15.39:0/1415200430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2a6e2650 con 0x563c2a656390 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.663+0000 7f55a3e00640 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a656390 0x563c2aa27a60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5594004300 tx=0x7f5594005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6cbf95517f182 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.663+0000 7f559a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.663+0000 7f559a000640 1 -- 172.21.15.39:0/1415200430 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559402c440 con 0x563c2a656390 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.663+0000 7f559a000640 1 -- 172.21.15.39:0/1415200430 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9512+0+0 (secure 0 0 0) 0x7f5594009040 con 0x563c2a656390 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.663+0000 7f55b9663740 1 -- 172.21.15.39:0/1415200430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559c004030 con 0x563c2a656390 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f559a000640 1 -- 172.21.15.39:0/1415200430 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f559402c5e0 con 0x563c2a656390 2024-07-04T08:22:05.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f55b9663740 10 client.17067 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f55b9663740 1 -- 172.21.15.39:0/1415200430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c2aa360e0 con 0x563c2a656390 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f559a000640 1 -- 172.21.15.39:0/1415200430 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c004030 con 0x563c2a656390 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f559a000640 1 -- 172.21.15.39:0/1415200430 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f559402a020 con 0x563c2a656390 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f55b9663740 10 client.17067 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f55b9663740 10 client.17067 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f55b9663740 20 client.17067 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f55b9663740 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563c2a6359f0 0x563c2aa39720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f55b9663740 4 client.17067 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.664+0000 7f55b9663740 1 -- 172.21.15.39:0/1415200430 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c2a993e50 con 0x563c2a6359f0 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.665+0000 7f55abe00640 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563c2a6359f0 0x563c2aa39720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.665+0000 7f55abe00640 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563c2a6359f0 0x563c2aa39720 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=92615af7a1b353bc server_cookie=ce7982d95d1883a7 in_seq=0 out_seq=0 2024-07-04T08:22:05.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.665+0000 7f559a000640 10 client.17067 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.666+0000 7f559a000640 1 -- 172.21.15.39:0/1415200430 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563c2a993e50 con 0x563c2a6359f0 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.666+0000 7f559a000640 10 client.17067 handle_command_reply: tid=0 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.666+0000 7f55b9663740 1 client.17067 shutdown 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 -- 172.21.15.39:0/1415200430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a656390 msgr2=0x563c2aa27a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a656390 0x563c2aa27a60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5594004300 tx=0x7f5594005f90 comp rx=0 tx=0).stop 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 -- 172.21.15.39:0/1415200430 shutdown_connections 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563c2a6359f0 0x563c2aa39720 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a656390 0x563c2aa27a60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a677070 0x563c2aa33410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 --2- 172.21.15.39:0/1415200430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 0x563c2aa27520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f559a000640 0 client.17067 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:22:05.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 -- 172.21.15.39:0/1415200430 >> 172.21.15.39:0/1415200430 conn(0x563c2a66b460 msgr2=0x563c2aa1eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:05.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 -- 172.21.15.39:0/1415200430 shutdown_connections 2024-07-04T08:22:05.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 1 -- 172.21.15.39:0/1415200430 wait complete. 2024-07-04T08:22:05.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.667+0000 7f55b9663740 20 client.17067 trim_cache size 0 max 16384 2024-07-04T08:22:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.683+0000 7f55b9663740 1 Processor -- start 2024-07-04T08:22:05.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.683+0000 7f55b9663740 1 -- start start 2024-07-04T08:22:05.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.683+0000 7f55b9663740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 0x563c2aa1d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.683+0000 7f55b9663740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa20bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.683+0000 7f55b9663740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa21130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.683+0000 7f55b9663740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac1b24d0 con 0x563c2a677070 2024-07-04T08:22:05.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.683+0000 7f55b9663740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac1b4010 con 0x563c2a64e1b0 2024-07-04T08:22:05.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.683+0000 7f55b9663740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac1b2ba0 con 0x563c2a656390 2024-07-04T08:22:05.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa20bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55abe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 0x563c2aa1d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa20bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39882/0 (socket says 172.21.15.39:39882) 2024-07-04T08:22:05.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55a3e00640 1 -- 172.21.15.39:0/2271673445 learned_addr learned my addr 172.21.15.39:0/2271673445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:05.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55b1000640 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa21130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55abe00640 1 -- 172.21.15.39:0/2271673445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 msgr2=0x563c2aa20bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:05.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55abe00640 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa20bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55abe00640 1 -- 172.21.15.39:0/2271673445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 msgr2=0x563c2aa21130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:05.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55abe00640 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa21130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55abe00640 1 -- 172.21.15.39:0/2271673445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2aa285e0 con 0x563c2a64e1b0 2024-07-04T08:22:05.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55b1000640 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa21130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:22:05.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.684+0000 7f55a3e00640 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa20bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:05.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.685+0000 7f55abe00640 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 0x563c2aa1d0b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f559c031440 tx=0x7f559c0050e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7977f4be7eebff68 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:05.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.685+0000 7f55a0c00640 1 -- 172.21.15.39:0/2271673445 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c059070 con 0x563c2a64e1b0 2024-07-04T08:22:05.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.685+0000 7f55a0c00640 1 -- 172.21.15.39:0/2271673445 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c031d10 con 0x563c2a64e1b0 2024-07-04T08:22:05.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.685+0000 7f55a0c00640 1 -- 172.21.15.39:0/2271673445 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c035040 con 0x563c2a64e1b0 2024-07-04T08:22:05.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.686+0000 7f55b9663740 1 -- 172.21.15.39:0/2271673445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 msgr2=0x563c2aa1d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:05.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.686+0000 7f55b9663740 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 0x563c2aa1d0b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f559c031440 tx=0x7f559c0050e0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.686+0000 7f55b9663740 1 -- 172.21.15.39:0/2271673445 shutdown_connections 2024-07-04T08:22:05.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.686+0000 7f55b9663740 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a677070 0x563c2aa21130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.686+0000 7f55b9663740 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa20bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.686+0000 7f55b9663740 1 --2- 172.21.15.39:0/2271673445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a64e1b0 0x563c2aa1d0b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.686+0000 7f55b9663740 1 -- 172.21.15.39:0/2271673445 >> 172.21.15.39:0/2271673445 conn(0x563c2a66b460 msgr2=0x563c2aa1c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:05.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.686+0000 7f55b9663740 1 -- 172.21.15.39:0/2271673445 shutdown_connections 2024-07-04T08:22:05.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.687+0000 7f55b9663740 1 -- 172.21.15.39:0/2271673445 wait complete. 2024-07-04T08:22:05.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.687+0000 7f55b9663740 1 -- start start 2024-07-04T08:22:05.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.687+0000 7f55b9663740 1 Processor -- start 2024-07-04T08:22:05.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.688+0000 7f55b9663740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a64e1b0 0x563c2aa2fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.688+0000 7f55b9663740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa32e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.688+0000 7f55b9663740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a677070 0x563c2aa33580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.688+0000 7f55b9663740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac1b3940 con 0x563c2a64e1b0 2024-07-04T08:22:05.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.688+0000 7f55b9663740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac1b3270 con 0x563c2a677070 2024-07-04T08:22:05.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.688+0000 7f55b9663740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac1b04d0 con 0x563c2a656390 2024-07-04T08:22:05.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.688+0000 7f55b9663740 10 client.0 _ll_register_callbacks cb 0x7f5594013e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa32e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa32e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39886/0 (socket says 172.21.15.39:39886) 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55a3e00640 1 -- 172.21.15.39:0/3860089321 learned_addr learned my addr 172.21.15.39:0/3860089321 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55abe00640 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a64e1b0 0x563c2aa2fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55b1000640 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a677070 0x563c2aa33580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55b1000640 1 -- 172.21.15.39:0/3860089321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 msgr2=0x563c2aa32e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55b1000640 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa32e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55b1000640 1 -- 172.21.15.39:0/3860089321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a64e1b0 msgr2=0x563c2aa2fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55b1000640 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a64e1b0 0x563c2aa2fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55b1000640 1 -- 172.21.15.39:0/3860089321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2aa2e830 con 0x563c2a677070 2024-07-04T08:22:05.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.689+0000 7f55abe00640 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a64e1b0 0x563c2aa2fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:05.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.690+0000 7f55b1000640 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a677070 0x563c2aa33580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f55a40e0400 tx=0x7f55a40a9b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdc7e893a7d0cd3c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:05.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.690+0000 7f559a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:05.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.690+0000 7f559a000640 1 -- 172.21.15.39:0/3860089321 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40ae340 con 0x563c2a677070 2024-07-04T08:22:05.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.690+0000 7f559a000640 1 -- 172.21.15.39:0/3860089321 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9512+0+0 (secure 0 0 0) 0x7f55a40b2b70 con 0x563c2a677070 2024-07-04T08:22:05.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.690+0000 7f55b9663740 1 -- 172.21.15.39:0/3860089321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559c031d10 con 0x563c2a677070 2024-07-04T08:22:05.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.690+0000 7f559a000640 1 -- 172.21.15.39:0/3860089321 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f55a40b2df0 con 0x563c2a677070 2024-07-04T08:22:05.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.690+0000 7f55b9663740 10 client.27212 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:05.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.690+0000 7f55b9663740 1 -- 172.21.15.39:0/3860089321 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c2aaaa810 con 0x563c2a677070 2024-07-04T08:22:05.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.691+0000 7f559a000640 1 -- 172.21.15.39:0/3860089321 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c031d10 con 0x563c2a677070 2024-07-04T08:22:05.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.691+0000 7f559a000640 1 -- 172.21.15.39:0/3860089321 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f55a40a8c60 con 0x563c2a677070 2024-07-04T08:22:05.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.691+0000 7f55b9663740 10 client.27212 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.691+0000 7f55b9663740 10 client.27212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.691+0000 7f55b9663740 20 client.27212 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.691+0000 7f55b9663740 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563c2a60ada0 0x563c2aaad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.691+0000 7f55b9663740 4 client.27212 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.691+0000 7f55b9663740 1 -- 172.21.15.39:0/3860089321 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c2a98f8f0 con 0x563c2a60ada0 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.692+0000 7f55abe00640 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563c2a60ada0 0x563c2aaad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.692+0000 7f55abe00640 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563c2a60ada0 0x563c2aaad020 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3f424f8e37e1aee1 server_cookie=41fe574aba31a553 in_seq=0 out_seq=0 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.692+0000 7f559a000640 10 client.27212 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.692+0000 7f559a000640 1 -- 172.21.15.39:0/3860089321 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c2a98f8f0 con 0x563c2a60ada0 2024-07-04T08:22:05.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f559a000640 10 client.27212 handle_command_reply: tid=0 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 client.27212 shutdown 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 -- 172.21.15.39:0/3860089321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a677070 msgr2=0x563c2aa33580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a677070 0x563c2aa33580 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f55a40e0400 tx=0x7f55a40a9b10 comp rx=0 tx=0).stop 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 -- 172.21.15.39:0/3860089321 shutdown_connections 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x563c2a60ada0 0x563c2aaad020 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c2a677070 0x563c2aa33580 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c2a656390 0x563c2aa32e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 --2- 172.21.15.39:0/3860089321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c2a64e1b0 0x563c2aa2fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 -- 172.21.15.39:0/3860089321 >> 172.21.15.39:0/3860089321 conn(0x563c2a66b460 msgr2=0x563c2aa1c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f559a000640 0 client.27212 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 -- 172.21.15.39:0/3860089321 shutdown_connections 2024-07-04T08:22:05.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 1 -- 172.21.15.39:0/3860089321 wait complete. 2024-07-04T08:22:05.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.693+0000 7f55b9663740 20 client.27212 trim_cache size 0 max 16384 2024-07-04T08:22:05.692 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.694+0000 7f55b1a00640 1 -- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55900b2960 msgr2=0x7f55900b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.694+0000 7f55b1a00640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55900b2960 0x7f55900b4e70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f55940058c0 tx=0x7f5594009bb0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.694+0000 7f55b1a00640 1 -- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55ac10b390 msgr2=0x7f55ac1a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.694+0000 7f55b1a00640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55ac10b390 0x7f55ac1a12b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009fd0 tx=0x7f55a400ef10 comp rx=0 tx=0).stop 2024-07-04T08:22:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.695+0000 7f55b1a00640 1 -- 172.21.15.39:0/354550424 shutdown_connections 2024-07-04T08:22:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.695+0000 7f55b1a00640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55900b2960 0x7f55900b4e70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.695+0000 7f55b1a00640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55ac10b390 0x7f55ac1a12b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.695+0000 7f55b1a00640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55ac10aa40 0x7f55ac19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.695+0000 7f55b1a00640 1 --2- 172.21.15.39:0/354550424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55ac105100 0x7f55ac19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:05.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.695+0000 7f55b1a00640 1 -- 172.21.15.39:0/354550424 >> 172.21.15.39:0/354550424 conn(0x7f55ac100bd0 msgr2=0x7f55ac102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:05.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.695+0000 7f55b1a00640 1 -- 172.21.15.39:0/354550424 shutdown_connections 2024-07-04T08:22:05.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:05.695+0000 7f55b1a00640 1 -- 172.21.15.39:0/354550424 wait complete. 2024-07-04T08:22:06.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:22:07.040 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:07.228 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:06 smithi039 ceph-mon[32062]: pgmap v688: 129 pgs: 129 active+clean; 7.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-07-04T08:22:07.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:06 smithi133 ceph-mon[37050]: pgmap v688: 129 pgs: 129 active+clean; 7.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-07-04T08:22:07.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:06 smithi050 ceph-mon[36577]: pgmap v688: 129 pgs: 129 active+clean; 7.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-07-04T08:22:09.095 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:08 smithi039 ceph-mon[32062]: pgmap v689: 129 pgs: 129 active+clean; 7.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-07-04T08:22:09.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:08 smithi133 ceph-mon[37050]: pgmap v689: 129 pgs: 129 active+clean; 7.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-07-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.340+0000 7f9da7800640 1 -- 172.21.15.39:0/614869027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da010d360 msgr2=0x7f9da010f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.340+0000 7f9da7800640 1 --2- 172.21.15.39:0/614869027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da010d360 0x7f9da010f850 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009ca0 tx=0x7f9d9802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.340+0000 7f9da7800640 1 -- 172.21.15.39:0/614869027 shutdown_connections 2024-07-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.340+0000 7f9da7800640 1 --2- 172.21.15.39:0/614869027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da0104850 0x7f9da010cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.340+0000 7f9da7800640 1 --2- 172.21.15.39:0/614869027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da010d360 0x7f9da010f850 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.340+0000 7f9da7800640 1 --2- 172.21.15.39:0/614869027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9da0103f00 0x7f9da0104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.340+0000 7f9da7800640 1 -- 172.21.15.39:0/614869027 >> 172.21.15.39:0/614869027 conn(0x7f9da0078c80 msgr2=0x7f9da0079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:09.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.340+0000 7f9da7800640 1 -- 172.21.15.39:0/614869027 shutdown_connections 2024-07-04T08:22:09.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.340+0000 7f9da7800640 1 -- 172.21.15.39:0/614869027 wait complete. 2024-07-04T08:22:09.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.341+0000 7f9da7800640 1 Processor -- start 2024-07-04T08:22:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.341+0000 7f9da7800640 1 -- start start 2024-07-04T08:22:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.341+0000 7f9da7800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da0103f00 0x7f9da01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da7800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9da0104850 0x7f9da0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da7800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da010d360 0x7f9da019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da7800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0112fe0 con 0x7f9da010d360 2024-07-04T08:22:09.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da7800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0112e60 con 0x7f9da0103f00 2024-07-04T08:22:09.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da7800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0113160 con 0x7f9da0104850 2024-07-04T08:22:09.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da0103f00 0x7f9da01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da010d360 0x7f9da019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da0103f00 0x7f9da01999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56438/0 (socket says 172.21.15.39:56438) 2024-07-04T08:22:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da4e00640 1 -- 172.21.15.39:0/1478531890 learned_addr learned my addr 172.21.15.39:0/1478531890 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da010d360 0x7f9da019f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59202/0 (socket says 172.21.15.39:59202) 2024-07-04T08:22:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9da0104850 0x7f9da0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da4e00640 1 -- 172.21.15.39:0/1478531890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9da0104850 msgr2=0x7f9da0199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da4e00640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9da0104850 0x7f9da0199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da4e00640 1 -- 172.21.15.39:0/1478531890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da010d360 msgr2=0x7f9da019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:09.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.342+0000 7f9da4e00640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da010d360 0x7f9da019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.343+0000 7f9da4e00640 1 -- 172.21.15.39:0/1478531890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da019f960 con 0x7f9da0103f00 2024-07-04T08:22:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.343+0000 7f9da4e00640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da0103f00 0x7f9da01999f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400e9c0 tx=0x7f9d9400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.343+0000 7f9d9d600640 1 -- 172.21.15.39:0/1478531890 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400cd80 con 0x7f9da0103f00 2024-07-04T08:22:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.343+0000 7f9d9d600640 1 -- 172.21.15.39:0/1478531890 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d940035f0 con 0x7f9da0103f00 2024-07-04T08:22:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.343+0000 7f9da7800640 1 -- 172.21.15.39:0/1478531890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da019fc50 con 0x7f9da0103f00 2024-07-04T08:22:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.343+0000 7f9d9d600640 1 -- 172.21.15.39:0/1478531890 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94010690 con 0x7f9da0103f00 2024-07-04T08:22:09.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.344+0000 7f9da7800640 1 -- 172.21.15.39:0/1478531890 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da01a63b0 con 0x7f9da0103f00 2024-07-04T08:22:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.347+0000 7f9dade2f740 1 Processor -- start 2024-07-04T08:22:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.347+0000 7f9dade2f740 1 -- start start 2024-07-04T08:22:09.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.347+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.347+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 0x561a73756cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.347+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 0x561a73757230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.347+0000 7f9dade2f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561a736ccee0 con 0x561a733b2070 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.347+0000 7f9dade2f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561a736bc480 con 0x561a73391390 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9dade2f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561a736c04c0 con 0x561a733891b0 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9da5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9da5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59212/0 (socket says 172.21.15.39:59212) 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9da5800640 1 -- 172.21.15.39:0/98670475 learned_addr learned my addr 172.21.15.39:0/98670475 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9d9d600640 1 -- 172.21.15.39:0/1478531890 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9d94004000 con 0x7f9da0103f00 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/98670475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 0x561a73757230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9da5800640 1 -- 172.21.15.39:0/98670475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 msgr2=0x561a73756cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:09.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9da5800640 1 --2- 172.21.15.39:0/98670475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 0x561a73756cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9da5800640 1 -- 172.21.15.39:0/98670475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 msgr2=0x561a73757230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9da5800640 1 --2- 172.21.15.39:0/98670475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 0x561a73757230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.348+0000 7f9da5800640 1 -- 172.21.15.39:0/98670475 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a73654650 con 0x561a733b2070 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.349+0000 7f9da5800640 1 --2- 172.21.15.39:0/98670475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737567b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9d9802f410 tx=0x7f9d9802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00d84ac4ed37b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.349+0000 7f9d92000640 1 -- 172.21.15.39:0/98670475 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98031cf0 con 0x561a733b2070 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.349+0000 7f9d92000640 1 -- 172.21.15.39:0/98670475 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d98036030 con 0x561a733b2070 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.349+0000 7f9d92000640 1 -- 172.21.15.39:0/98670475 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980042a0 con 0x561a733b2070 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9d9d600640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9d880b2960 0x7f9d880b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9d9d600640 1 -- 172.21.15.39:0/1478531890 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9512+0+0 (secure 0 0 0) 0x7f9d940e2f50 con 0x7f9da0103f00 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9dade2f740 1 -- 172.21.15.39:0/98670475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 msgr2=0x561a737567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9dade2f740 1 --2- 172.21.15.39:0/98670475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737567b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9d9802f410 tx=0x7f9d9802f440 comp rx=0 tx=0).stop 2024-07-04T08:22:09.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9da4e00640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9d880b2960 0x7f9d880b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9dade2f740 1 -- 172.21.15.39:0/98670475 shutdown_connections 2024-07-04T08:22:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9dade2f740 1 --2- 172.21.15.39:0/98670475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 0x561a73757230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9dade2f740 1 --2- 172.21.15.39:0/98670475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 0x561a73756cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9dade2f740 1 --2- 172.21.15.39:0/98670475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737567b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.350+0000 7f9dade2f740 1 -- 172.21.15.39:0/98670475 >> 172.21.15.39:0/98670475 conn(0x561a733a6460 msgr2=0x561a737546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.351+0000 7f9dade2f740 1 -- 172.21.15.39:0/98670475 shutdown_connections 2024-07-04T08:22:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.351+0000 7f9dade2f740 1 -- 172.21.15.39:0/98670475 wait complete. 2024-07-04T08:22:09.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.351+0000 7f9da4e00640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9d880b2960 0x7f9d880b4e70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x561a73755430 tx=0x7f9d94003d70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.351+0000 7f9dade2f740 1 -- start start 2024-07-04T08:22:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.352+0000 7f9dade2f740 1 Processor -- start 2024-07-04T08:22:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.352+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 0x561a73762310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.352+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 0x561a73762850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.352+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.352+0000 7f9dade2f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561a73520700 con 0x561a733b2070 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.352+0000 7f9dade2f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561a73663bb0 con 0x561a73391390 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.352+0000 7f9dade2f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561a73764db0 con 0x561a733891b0 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.352+0000 7f9dade2f740 10 client.0 _ll_register_callbacks cb 0x561a736b5570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9da5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 0x561a73762310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 0x561a73762850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9da5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 0x561a73762310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39928/0 (socket says 172.21.15.39:39928) 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9da5800640 1 -- 172.21.15.39:0/2690934606 learned_addr learned my addr 172.21.15.39:0/2690934606 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9da5800640 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9d9fe00640 1 -- 172.21.15.39:0/2690934606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 msgr2=0x561a73762310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 0x561a73762310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9d9fe00640 1 -- 172.21.15.39:0/2690934606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 msgr2=0x561a737768b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737768b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.353+0000 7f9d9fe00640 1 -- 172.21.15.39:0/2690934606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7341d650 con 0x561a73391390 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.354+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 0x561a73762850 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9da0069ff0 tx=0x7f9d8c012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b8e66fa15dcb44d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.354+0000 7f9d86a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.354+0000 7f9d86a00640 1 -- 172.21.15.39:0/2690934606 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00f890 con 0x561a73391390 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.354+0000 7f9dade2f740 1 -- 172.21.15.39:0/2690934606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d98036030 con 0x561a73391390 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.354+0000 7f9d86a00640 1 -- 172.21.15.39:0/2690934606 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9512+0+0 (secure 0 0 0) 0x7f9d8c00fa30 con 0x561a73391390 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.354+0000 7f9d86a00640 1 -- 172.21.15.39:0/2690934606 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c00fc30 con 0x561a73391390 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.354+0000 7f9dade2f740 10 client.27224 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.354+0000 7f9dade2f740 1 -- 172.21.15.39:0/2690934606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a73779630 con 0x561a73391390 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.355+0000 7f9d86a00640 1 -- 172.21.15.39:0/2690934606 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d98036030 con 0x561a73391390 2024-07-04T08:22:09.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.355+0000 7f9d86a00640 1 -- 172.21.15.39:0/2690934606 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9d8c01c420 con 0x561a73391390 2024-07-04T08:22:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.355+0000 7f9dade2f740 10 client.27224 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.355+0000 7f9dade2f740 10 client.27224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:22:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.355+0000 7f9dade2f740 20 client.27224 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.355+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x561a733709f0 0x561a7377cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.355+0000 7f9dade2f740 4 client.27224 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.355+0000 7f9dade2f740 1 -- 172.21.15.39:0/2690934606 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a736cee30 con 0x561a733709f0 2024-07-04T08:22:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.356+0000 7f9da5800640 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x561a733709f0 0x561a7377cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.356+0000 7f9da5800640 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x561a733709f0 0x561a7377cc60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e690b42b3209d411 server_cookie=cf5df14383a0ad35 in_seq=0 out_seq=0 2024-07-04T08:22:09.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.356+0000 7f9d86a00640 10 client.27224 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:22:09.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.357+0000 7f9d86a00640 1 -- 172.21.15.39:0/2690934606 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561a736cee30 con 0x561a733709f0 2024-07-04T08:22:09.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.357+0000 7f9d86a00640 10 client.27224 handle_command_reply: tid=0 2024-07-04T08:22:09.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.357+0000 7f9dade2f740 1 client.27224 shutdown 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.357+0000 7f9dade2f740 1 -- 172.21.15.39:0/2690934606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 msgr2=0x561a73762850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.357+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 0x561a73762850 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9da0069ff0 tx=0x7f9d8c012b60 comp rx=0 tx=0).stop 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9dade2f740 1 -- 172.21.15.39:0/2690934606 shutdown_connections 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x561a733709f0 0x561a7377cc60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733b2070 0x561a737768b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a73391390 0x561a73762850 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2690934606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733891b0 0x561a73762310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9d86a00640 0 client.27224 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9dade2f740 1 -- 172.21.15.39:0/2690934606 >> 172.21.15.39:0/2690934606 conn(0x561a733a6460 msgr2=0x561a73759990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9dade2f740 1 -- 172.21.15.39:0/2690934606 shutdown_connections 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9dade2f740 1 -- 172.21.15.39:0/2690934606 wait complete. 2024-07-04T08:22:09.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.358+0000 7f9dade2f740 20 client.27224 trim_cache size 0 max 16384 2024-07-04T08:22:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.374+0000 7f9dade2f740 1 Processor -- start 2024-07-04T08:22:09.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.374+0000 7f9dade2f740 1 -- start start 2024-07-04T08:22:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.374+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733891b0 0x561a73753940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.374+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a73391390 0x561a73757ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.374+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733b2070 0x561a7375b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.374+0000 7f9dade2f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01ae210 con 0x561a733891b0 2024-07-04T08:22:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.374+0000 7f9dade2f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01afd50 con 0x561a733b2070 2024-07-04T08:22:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.374+0000 7f9dade2f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01ae8e0 con 0x561a73391390 2024-07-04T08:22:09.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9da5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733891b0 0x561a73753940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9da5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733891b0 0x561a73753940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59224/0 (socket says 172.21.15.39:59224) 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9da5800640 1 -- 172.21.15.39:0/2040052570 learned_addr learned my addr 172.21.15.39:0/2040052570 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/2040052570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a73391390 0x561a73757ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9da5800640 1 --2- 172.21.15.39:0/2040052570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733b2070 0x561a7375b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9d9fe00640 1 -- 172.21.15.39:0/2040052570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733b2070 msgr2=0x561a7375b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/2040052570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733b2070 0x561a7375b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9d9fe00640 1 -- 172.21.15.39:0/2040052570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733891b0 msgr2=0x561a73753940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/2040052570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733891b0 0x561a73753940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9d9fe00640 1 -- 172.21.15.39:0/2040052570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a73755010 con 0x561a73391390 2024-07-04T08:22:09.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.375+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/2040052570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a73391390 0x561a73757ef0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c012420 tx=0x7f9d8c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e1f6754072763c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.376+0000 7f9d91600640 1 -- 172.21.15.39:0/2040052570 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00fe50 con 0x561a73391390 2024-07-04T08:22:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.376+0000 7f9d91600640 1 -- 172.21.15.39:0/2040052570 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d8c01cb20 con 0x561a73391390 2024-07-04T08:22:09.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.376+0000 7f9d91600640 1 -- 172.21.15.39:0/2040052570 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c005070 con 0x561a73391390 2024-07-04T08:22:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.377+0000 7f9dade2f740 1 -- 172.21.15.39:0/2040052570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a73391390 msgr2=0x561a73757ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.377+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2040052570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a73391390 0x561a73757ef0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c012420 tx=0x7f9d8c005850 comp rx=0 tx=0).stop 2024-07-04T08:22:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.377+0000 7f9dade2f740 1 -- 172.21.15.39:0/2040052570 shutdown_connections 2024-07-04T08:22:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.377+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2040052570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733b2070 0x561a7375b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.377+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2040052570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a73391390 0x561a73757ef0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.377+0000 7f9dade2f740 1 --2- 172.21.15.39:0/2040052570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a733891b0 0x561a73753940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.377+0000 7f9dade2f740 1 -- 172.21.15.39:0/2040052570 >> 172.21.15.39:0/2040052570 conn(0x561a733a6460 msgr2=0x561a737577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.377+0000 7f9dade2f740 1 -- 172.21.15.39:0/2040052570 shutdown_connections 2024-07-04T08:22:09.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.377+0000 7f9dade2f740 1 -- 172.21.15.39:0/2040052570 wait complete. 2024-07-04T08:22:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.378+0000 7f9dade2f740 1 -- start start 2024-07-04T08:22:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.378+0000 7f9dade2f740 1 Processor -- start 2024-07-04T08:22:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.378+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733891b0 0x561a73781c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.378+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a73391390 0x561a73782140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.378+0000 7f9dade2f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733b2070 0x561a73771890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9dade2f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01af680 con 0x561a73391390 2024-07-04T08:22:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9dade2f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01aefb0 con 0x561a733891b0 2024-07-04T08:22:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9dade2f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01ac210 con 0x561a733b2070 2024-07-04T08:22:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9dade2f740 10 client.0 _ll_register_callbacks cb 0x7f9d8c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a73391390 0x561a73782140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9d9fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a73391390 0x561a73782140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59230/0 (socket says 172.21.15.39:59230) 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9d9fe00640 1 -- 172.21.15.39:0/4114629453 learned_addr learned my addr 172.21.15.39:0/4114629453 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9da5800640 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733891b0 0x561a73781c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9da5800640 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733b2070 0x561a73771890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9d9fe00640 1 -- 172.21.15.39:0/4114629453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733b2070 msgr2=0x561a73771890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733b2070 0x561a73771890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9d9fe00640 1 -- 172.21.15.39:0/4114629453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733891b0 msgr2=0x561a73781c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733891b0 0x561a73781c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.379+0000 7f9d9fe00640 1 -- 172.21.15.39:0/4114629453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a73771e20 con 0x561a73391390 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.380+0000 7f9d9fe00640 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a73391390 0x561a73782140 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c01caa0 tx=0x7f9d8c00fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f92a324e9f18e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.380+0000 7f9d86a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.380+0000 7f9d86a00640 1 -- 172.21.15.39:0/4114629453 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c037060 con 0x561a73391390 2024-07-04T08:22:09.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.380+0000 7f9d86a00640 1 -- 172.21.15.39:0/4114629453 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9512+0+0 (secure 0 0 0) 0x7f9d8c0052d0 con 0x561a73391390 2024-07-04T08:22:09.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.380+0000 7f9dade2f740 1 -- 172.21.15.39:0/4114629453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c01cb20 con 0x561a73391390 2024-07-04T08:22:09.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9d86a00640 1 -- 172.21.15.39:0/4114629453 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c03e050 con 0x561a73391390 2024-07-04T08:22:09.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9dade2f740 10 client.17091 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:09.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9dade2f740 1 -- 172.21.15.39:0/4114629453 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561a737730a0 con 0x561a73391390 2024-07-04T08:22:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9d86a00640 1 -- 172.21.15.39:0/4114629453 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d8c01cb20 con 0x561a73391390 2024-07-04T08:22:09.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9d86a00640 1 -- 172.21.15.39:0/4114629453 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9d8c031030 con 0x561a73391390 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9dade2f740 10 client.17091 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9dade2f740 10 client.17091 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9dade2f740 20 client.17091 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9dade2f740 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x561a73345da0 0x561a737e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9dade2f740 4 client.17091 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.381+0000 7f9dade2f740 1 -- 172.21.15.39:0/4114629453 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a736ca8d0 con 0x561a73345da0 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.382+0000 7f9da5800640 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x561a73345da0 0x561a737e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.382+0000 7f9da5800640 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x561a73345da0 0x561a737e74d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da9cd51472628628 server_cookie=6c41c1bd7127b091 in_seq=0 out_seq=0 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.382+0000 7f9d86a00640 10 client.17091 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9d86a00640 1 -- 172.21.15.39:0/4114629453 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a736ca8d0 con 0x561a73345da0 2024-07-04T08:22:09.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9d86a00640 10 client.17091 handle_command_reply: tid=0 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 client.17091 shutdown 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 -- 172.21.15.39:0/4114629453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a73391390 msgr2=0x561a73782140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a73391390 0x561a73782140 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c01caa0 tx=0x7f9d8c00fb20 comp rx=0 tx=0).stop 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 -- 172.21.15.39:0/4114629453 shutdown_connections 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x561a73345da0 0x561a737e74d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561a73391390 0x561a73782140 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a733b2070 0x561a73771890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 --2- 172.21.15.39:0/4114629453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561a733891b0 0x561a73781c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 -- 172.21.15.39:0/4114629453 >> 172.21.15.39:0/4114629453 conn(0x561a733a6460 msgr2=0x561a737634a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9d86a00640 0 client.17091 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 -- 172.21.15.39:0/4114629453 shutdown_connections 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.383+0000 7f9dade2f740 1 -- 172.21.15.39:0/4114629453 wait complete. 2024-07-04T08:22:09.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9dade2f740 20 client.17091 trim_cache size 0 max 16384 2024-07-04T08:22:09.382 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 -- 172.21.15.39:0/1478531890 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9d880b2960 msgr2=0x7f9d880b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9d880b2960 0x7f9d880b4e70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x561a73755430 tx=0x7f9d94003d70 comp rx=0 tx=0).stop 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 -- 172.21.15.39:0/1478531890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da0103f00 msgr2=0x7f9da01999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da0103f00 0x7f9da01999f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400e9c0 tx=0x7f9d9400ee80 comp rx=0 tx=0).stop 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 -- 172.21.15.39:0/1478531890 shutdown_connections 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9d880b2960 0x7f9d880b4e70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da010d360 0x7f9da019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9da0104850 0x7f9da0199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 --2- 172.21.15.39:0/1478531890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9da0103f00 0x7f9da01999f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.384+0000 7f9da7800640 1 -- 172.21.15.39:0/1478531890 >> 172.21.15.39:0/1478531890 conn(0x7f9da0078c80 msgr2=0x7f9da00ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:09.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.385+0000 7f9da7800640 1 -- 172.21.15.39:0/1478531890 shutdown_connections 2024-07-04T08:22:09.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:09.385+0000 7f9da7800640 1 -- 172.21.15.39:0/1478531890 wait complete. 2024-07-04T08:22:09.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:08 smithi050 ceph-mon[36577]: pgmap v689: 129 pgs: 129 active+clean; 7.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-07-04T08:22:10.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:22:10.745 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:11.249 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:11 smithi039 ceph-mon[32062]: pgmap v690: 129 pgs: 129 active+clean; 7.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-07-04T08:22:11.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:11 smithi133 ceph-mon[37050]: pgmap v690: 129 pgs: 129 active+clean; 7.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-07-04T08:22:11.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:11 smithi050 ceph-mon[36577]: pgmap v690: 129 pgs: 129 active+clean; 7.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 91 KiB/s wr, 35 op/s 2024-07-04T08:22:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.020+0000 7f0e1a400640 1 -- 172.21.15.39:0/3715012881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14109340 msgr2=0x7f0e1410fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.020+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3715012881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14109340 0x7f0e1410fb70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009be0 tx=0x7f0e0c02a6d0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.021+0000 7f0e1a400640 1 -- 172.21.15.39:0/3715012881 shutdown_connections 2024-07-04T08:22:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.021+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3715012881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e14109340 0x7f0e1410fb70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.021+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3715012881 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e141089f0 0x7f0e14108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.021+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3715012881 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e141030b0 0x7f0e141034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.021+0000 7f0e1a400640 1 -- 172.21.15.39:0/3715012881 >> 172.21.15.39:0/3715012881 conn(0x7f0e1407ada0 msgr2=0x7f0e1407b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.021+0000 7f0e1a400640 1 -- 172.21.15.39:0/3715012881 shutdown_connections 2024-07-04T08:22:13.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.021+0000 7f0e1a400640 1 -- 172.21.15.39:0/3715012881 wait complete. 2024-07-04T08:22:13.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.022+0000 7f0e1a400640 1 Processor -- start 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.022+0000 7f0e1a400640 1 -- start start 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e1a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e141030b0 0x7f0e1419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e1a400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e141089f0 0x7f0e1419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e1a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e14109340 0x7f0e141a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e1a400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14112ff0 con 0x7f0e141030b0 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e1a400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14112e70 con 0x7f0e14109340 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e1a400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e14113170 con 0x7f0e141089f0 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e13400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e141030b0 0x7f0e1419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e141089f0 0x7f0e1419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e13400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e141030b0 0x7f0e1419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43518/0 (socket says 172.21.15.39:43518) 2024-07-04T08:22:13.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e13400640 1 -- 172.21.15.39:0/3473482604 learned_addr learned my addr 172.21.15.39:0/3473482604 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:13.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.023+0000 7f0e13e00640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e14109340 0x7f0e141a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e0be00640 1 -- 172.21.15.39:0/3473482604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e14109340 msgr2=0x7f0e141a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e0be00640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e14109340 0x7f0e141a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e0be00640 1 -- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e141030b0 msgr2=0x7f0e1419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e0be00640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e141030b0 0x7f0e1419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e0be00640 1 -- 172.21.15.39:0/3473482604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e141a1b50 con 0x7f0e141089f0 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e13400640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e141030b0 0x7f0e1419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e0be00640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e141089f0 0x7f0e1419c180 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0e1406bcb0 tx=0x7f0dfc00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e10c00640 1 -- 172.21.15.39:0/3473482604 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc00ccd0 con 0x7f0e141089f0 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e10c00640 1 -- 172.21.15.39:0/3473482604 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dfc0035f0 con 0x7f0e141089f0 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e1a400640 1 -- 172.21.15.39:0/3473482604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e141a1e40 con 0x7f0e141089f0 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.024+0000 7f0e10c00640 1 -- 172.21.15.39:0/3473482604 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc010690 con 0x7f0e141089f0 2024-07-04T08:22:13.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.025+0000 7f0e1a400640 1 -- 172.21.15.39:0/3473482604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e141a85a0 con 0x7f0e141089f0 2024-07-04T08:22:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.028+0000 7f0e10c00640 1 -- 172.21.15.39:0/3473482604 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0dfc0037a0 con 0x7f0e141089f0 2024-07-04T08:22:13.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.028+0000 7f0e20b07740 1 Processor -- start 2024-07-04T08:22:13.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.028+0000 7f0e20b07740 1 -- start start 2024-07-04T08:22:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e82eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e82ec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e20b07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5626e8261ee0 con 0x5626e7f1e1b0 2024-07-04T08:22:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e20b07740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5626e8251480 con 0x5626e7f47070 2024-07-04T08:22:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e20b07740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5626e82554c0 con 0x5626e7f26390 2024-07-04T08:22:13.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e13400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e13400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82ebd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43524/0 (socket says 172.21.15.39:43524) 2024-07-04T08:22:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e13400640 1 -- 172.21.15.39:0/3427722185 learned_addr learned my addr 172.21.15.39:0/3427722185 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.029+0000 7f0e13e00640 1 --2- 172.21.15.39:0/3427722185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e82eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.030+0000 7f0e0be00640 1 --2- 172.21.15.39:0/3427722185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e82ec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.030+0000 7f0e13e00640 1 -- 172.21.15.39:0/3427722185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 msgr2=0x5626e82ec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.030+0000 7f0e13e00640 1 --2- 172.21.15.39:0/3427722185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e82ec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.030+0000 7f0e13e00640 1 -- 172.21.15.39:0/3427722185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 msgr2=0x5626e82ebd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.030+0000 7f0e13e00640 1 --2- 172.21.15.39:0/3427722185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82ebd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.030+0000 7f0e13e00640 1 -- 172.21.15.39:0/3427722185 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e81e9650 con 0x5626e7f47070 2024-07-04T08:22:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.030+0000 7f0e10c00640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0df80b29e0 0x7f0df80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.030+0000 7f0e13e00640 1 --2- 172.21.15.39:0/3427722185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e82eb7d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009b30 tx=0x7f0e0c030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b444a356bb6f8240 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.031+0000 7f0e10c00640 1 -- 172.21.15.39:0/3473482604 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9512+0+0 (secure 0 0 0) 0x7f0dfc0e2b00 con 0x7f0e141089f0 2024-07-04T08:22:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.031+0000 7f0e13400640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0df80b29e0 0x7f0df80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.031+0000 7f0e08c00640 1 -- 172.21.15.39:0/3427722185 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c041030 con 0x5626e7f47070 2024-07-04T08:22:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.031+0000 7f0e08c00640 1 -- 172.21.15.39:0/3427722185 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e0c003e20 con 0x5626e7f47070 2024-07-04T08:22:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.031+0000 7f0e08c00640 1 -- 172.21.15.39:0/3427722185 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c036040 con 0x5626e7f47070 2024-07-04T08:22:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e20b07740 1 -- 172.21.15.39:0/3427722185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 msgr2=0x5626e82eb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3427722185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e82eb7d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009b30 tx=0x7f0e0c030430 comp rx=0 tx=0).stop 2024-07-04T08:22:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e13400640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0df80b29e0 0x7f0df80b4ef0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5626e82ea450 tx=0x7f0e04005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e20b07740 1 -- 172.21.15.39:0/3427722185 shutdown_connections 2024-07-04T08:22:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3427722185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82ebd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3427722185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e82ec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3427722185 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e82eb7d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e20b07740 1 -- 172.21.15.39:0/3427722185 >> 172.21.15.39:0/3427722185 conn(0x5626e7f3b460 msgr2=0x5626e82e96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e20b07740 1 -- 172.21.15.39:0/3427722185 shutdown_connections 2024-07-04T08:22:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.032+0000 7f0e20b07740 1 -- 172.21.15.39:0/3427722185 wait complete. 2024-07-04T08:22:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.033+0000 7f0e20b07740 1 -- start start 2024-07-04T08:22:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.033+0000 7f0e20b07740 1 Processor -- start 2024-07-04T08:22:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e82f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f26390 0x5626e82f7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f47070 0x5626e8303240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e20b07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5626e80b5700 con 0x5626e7f26390 2024-07-04T08:22:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e20b07740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5626e81f8bb0 con 0x5626e7f1e1b0 2024-07-04T08:22:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e20b07740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5626e82f9f50 con 0x5626e7f47070 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e20b07740 10 client.0 _ll_register_callbacks cb 0x5626e824a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f47070 0x5626e8303240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f26390 0x5626e82f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f47070 0x5626e8303240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59570/0 (socket says 172.21.15.39:59570) 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e13e00640 1 -- 172.21.15.39:0/936163435 learned_addr learned my addr 172.21.15.39:0/936163435 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.034+0000 7f0e13400640 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e82f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.035+0000 7f0e13e00640 1 -- 172.21.15.39:0/936163435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 msgr2=0x5626e82f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.035+0000 7f0e13e00640 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e82f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.035+0000 7f0e13e00640 1 -- 172.21.15.39:0/936163435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f26390 msgr2=0x5626e82f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.035+0000 7f0e13e00640 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f26390 0x5626e82f7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.035+0000 7f0e13e00640 1 -- 172.21.15.39:0/936163435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e7fb2650 con 0x5626e7f47070 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.035+0000 7f0e13e00640 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f47070 0x5626e8303240 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c02acc0 tx=0x7f0e0c009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f855c822ae3993fc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.035+0000 7f0e01600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:13.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.036+0000 7f0e01600640 1 -- 172.21.15.39:0/936163435 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c036050 con 0x5626e7f47070 2024-07-04T08:22:13.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.036+0000 7f0e20b07740 1 -- 172.21.15.39:0/936163435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e0c003e20 con 0x5626e7f47070 2024-07-04T08:22:13.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.036+0000 7f0e01600640 1 -- 172.21.15.39:0/936163435 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9512+0+0 (secure 0 0 0) 0x7f0e0c030970 con 0x5626e7f47070 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.036+0000 7f0e01600640 1 -- 172.21.15.39:0/936163435 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e0c041040 con 0x5626e7f47070 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.036+0000 7f0e20b07740 10 client.27157 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.036+0000 7f0e20b07740 1 -- 172.21.15.39:0/936163435 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626e8305fb0 con 0x5626e7f47070 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.037+0000 7f0e01600640 1 -- 172.21.15.39:0/936163435 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e0c003e20 con 0x5626e7f47070 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.037+0000 7f0e01600640 1 -- 172.21.15.39:0/936163435 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0e0c0045e0 con 0x5626e7f47070 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.037+0000 7f0e20b07740 10 client.27157 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.037+0000 7f0e20b07740 10 client.27157 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.037+0000 7f0e20b07740 20 client.27157 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.037+0000 7f0e20b07740 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5626e7f059f0 0x5626e8309310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.037+0000 7f0e20b07740 4 client.27157 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.037+0000 7f0e20b07740 1 -- 172.21.15.39:0/936163435 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626e8263e30 con 0x5626e7f059f0 2024-07-04T08:22:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.037+0000 7f0e13400640 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5626e7f059f0 0x5626e8309310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.038+0000 7f0e13400640 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5626e7f059f0 0x5626e8309310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab3a7bbfec3dc254 server_cookie=ab269631b8057b8e in_seq=0 out_seq=0 2024-07-04T08:22:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.038+0000 7f0e01600640 10 client.27157 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e01600640 1 -- 172.21.15.39:0/936163435 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5626e8263e30 con 0x5626e7f059f0 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e01600640 10 client.27157 handle_command_reply: tid=0 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e20b07740 1 client.27157 shutdown 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e20b07740 1 -- 172.21.15.39:0/936163435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f47070 msgr2=0x5626e8303240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e20b07740 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f47070 0x5626e8303240 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c02acc0 tx=0x7f0e0c009880 comp rx=0 tx=0).stop 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e20b07740 1 -- 172.21.15.39:0/936163435 shutdown_connections 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e20b07740 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5626e7f059f0 0x5626e8309310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e20b07740 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f26390 0x5626e82f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e20b07740 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f47070 0x5626e8303240 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e20b07740 1 --2- 172.21.15.39:0/936163435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e82f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e01600640 0 client.27157 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:22:13.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.039+0000 7f0e20b07740 1 -- 172.21.15.39:0/936163435 >> 172.21.15.39:0/936163435 conn(0x5626e7f3b460 msgr2=0x5626e82ee9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:13.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.040+0000 7f0e20b07740 1 -- 172.21.15.39:0/936163435 shutdown_connections 2024-07-04T08:22:13.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.040+0000 7f0e20b07740 1 -- 172.21.15.39:0/936163435 wait complete. 2024-07-04T08:22:13.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.040+0000 7f0e20b07740 20 client.27157 trim_cache size 0 max 16384 2024-07-04T08:22:13.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.055+0000 7f0e20b07740 1 Processor -- start 2024-07-04T08:22:13.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.055+0000 7f0e20b07740 1 -- start start 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e82f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e83031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e20b07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141b2630 con 0x5626e7f1e1b0 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e20b07740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141b4170 con 0x5626e7f47070 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e20b07740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141b2d00 con 0x5626e7f26390 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e13400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e13e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e83031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e13400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82e89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43544/0 (socket says 172.21.15.39:43544) 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e13400640 1 -- 172.21.15.39:0/852018680 learned_addr learned my addr 172.21.15.39:0/852018680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.056+0000 7f0e0be00640 1 --2- 172.21.15.39:0/852018680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e82f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e13e00640 1 -- 172.21.15.39:0/852018680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 msgr2=0x5626e82f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e13e00640 1 --2- 172.21.15.39:0/852018680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e82f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e13e00640 1 -- 172.21.15.39:0/852018680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 msgr2=0x5626e82e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e13e00640 1 --2- 172.21.15.39:0/852018680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82e89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e13e00640 1 -- 172.21.15.39:0/852018680 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e82ee630 con 0x5626e7f47070 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e13400640 1 --2- 172.21.15.39:0/852018680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e13e00640 1 --2- 172.21.15.39:0/852018680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e83031b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c02a710 tx=0x7f0e0c02ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c9ece776999a74 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e03e00640 1 -- 172.21.15.39:0/852018680 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c036070 con 0x5626e7f47070 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e03e00640 1 -- 172.21.15.39:0/852018680 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e0c03d5a0 con 0x5626e7f47070 2024-07-04T08:22:13.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.057+0000 7f0e03e00640 1 -- 172.21.15.39:0/852018680 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0316a0 con 0x5626e7f47070 2024-07-04T08:22:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.058+0000 7f0e20b07740 1 -- 172.21.15.39:0/852018680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 msgr2=0x5626e83031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.058+0000 7f0e20b07740 1 --2- 172.21.15.39:0/852018680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e83031b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c02a710 tx=0x7f0e0c02ae50 comp rx=0 tx=0).stop 2024-07-04T08:22:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.058+0000 7f0e20b07740 1 -- 172.21.15.39:0/852018680 shutdown_connections 2024-07-04T08:22:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.058+0000 7f0e20b07740 1 --2- 172.21.15.39:0/852018680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f47070 0x5626e83031b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.058+0000 7f0e20b07740 1 --2- 172.21.15.39:0/852018680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e82f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.058+0000 7f0e20b07740 1 --2- 172.21.15.39:0/852018680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f1e1b0 0x5626e82e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.058+0000 7f0e20b07740 1 -- 172.21.15.39:0/852018680 >> 172.21.15.39:0/852018680 conn(0x5626e7f3b460 msgr2=0x5626e82f86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.058+0000 7f0e20b07740 1 -- 172.21.15.39:0/852018680 shutdown_connections 2024-07-04T08:22:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.058+0000 7f0e20b07740 1 -- 172.21.15.39:0/852018680 wait complete. 2024-07-04T08:22:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.059+0000 7f0e20b07740 1 -- start start 2024-07-04T08:22:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.059+0000 7f0e20b07740 1 Processor -- start 2024-07-04T08:22:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e8307ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e8308240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e20b07740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f47070 0x5626e83698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e20b07740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141b3aa0 con 0x5626e7f47070 2024-07-04T08:22:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e20b07740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141b33d0 con 0x5626e7f1e1b0 2024-07-04T08:22:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e20b07740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e141b0630 con 0x5626e7f26390 2024-07-04T08:22:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e20b07740 10 client.0 _ll_register_callbacks cb 0x7f0e04009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e13400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e8307ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e13400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e8307ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:57930/0 (socket says 172.21.15.39:57930) 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e13400640 1 -- 172.21.15.39:0/3616265796 learned_addr learned my addr 172.21.15.39:0/3616265796 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e13e00640 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f47070 0x5626e83698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e8308240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e13400640 1 -- 172.21.15.39:0/3616265796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 msgr2=0x5626e8308240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e13400640 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e8308240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e13400640 1 -- 172.21.15.39:0/3616265796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f47070 msgr2=0x5626e83698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e13400640 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f47070 0x5626e83698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.060+0000 7f0e13400640 1 -- 172.21.15.39:0/3616265796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e8369f60 con 0x5626e7f1e1b0 2024-07-04T08:22:13.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.061+0000 7f0e13400640 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e8307ca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400fca0 tx=0x7f0e04017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2c66f7c78928dd server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:13.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.061+0000 7f0e01600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:13.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.061+0000 7f0e01600640 1 -- 172.21.15.39:0/3616265796 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0401d890 con 0x5626e7f1e1b0 2024-07-04T08:22:13.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.061+0000 7f0e01600640 1 -- 172.21.15.39:0/3616265796 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9512+0+0 (secure 0 0 0) 0x7f0e0401da30 con 0x5626e7f1e1b0 2024-07-04T08:22:13.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.061+0000 7f0e20b07740 1 -- 172.21.15.39:0/3616265796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e0c03d5a0 con 0x5626e7f1e1b0 2024-07-04T08:22:13.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e01600640 1 -- 172.21.15.39:0/3616265796 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e0401dc30 con 0x5626e7f1e1b0 2024-07-04T08:22:13.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e20b07740 10 client.27245 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:13.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e20b07740 1 -- 172.21.15.39:0/3616265796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626e83713b0 con 0x5626e7f1e1b0 2024-07-04T08:22:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e01600640 1 -- 172.21.15.39:0/3616265796 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e0c03d5a0 con 0x5626e7f1e1b0 2024-07-04T08:22:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e01600640 1 -- 172.21.15.39:0/3616265796 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0e04005460 con 0x5626e7f1e1b0 2024-07-04T08:22:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e20b07740 10 client.27245 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e20b07740 10 client.27245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:22:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e20b07740 20 client.27245 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5626e7edada0 0x5626e8373b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e20b07740 4 client.27245 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.062+0000 7f0e20b07740 1 -- 172.21.15.39:0/3616265796 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626e825f8d0 con 0x5626e7edada0 2024-07-04T08:22:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.063+0000 7f0e0be00640 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5626e7edada0 0x5626e8373b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.063+0000 7f0e01600640 10 client.27245 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:22:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.063+0000 7f0e0be00640 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5626e7edada0 0x5626e8373b80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a43a49d7cca55470 server_cookie=b2324e45c09b3636 in_seq=0 out_seq=0 2024-07-04T08:22:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e01600640 1 -- 172.21.15.39:0/3616265796 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626e825f8d0 con 0x5626e7edada0 2024-07-04T08:22:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e01600640 10 client.27245 handle_command_reply: tid=0 2024-07-04T08:22:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 client.27245 shutdown 2024-07-04T08:22:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 -- 172.21.15.39:0/3616265796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 msgr2=0x5626e8307ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e8307ca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400fca0 tx=0x7f0e04017000 comp rx=0 tx=0).stop 2024-07-04T08:22:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 -- 172.21.15.39:0/3616265796 shutdown_connections 2024-07-04T08:22:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5626e7edada0 0x5626e8373b80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5626e7f47070 0x5626e83698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626e7f26390 0x5626e8308240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 --2- 172.21.15.39:0/3616265796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5626e7f1e1b0 0x5626e8307ca0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 -- 172.21.15.39:0/3616265796 >> 172.21.15.39:0/3616265796 conn(0x5626e7f3b460 msgr2=0x5626e82ed080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e01600640 0 client.27245 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:22:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 -- 172.21.15.39:0/3616265796 shutdown_connections 2024-07-04T08:22:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 1 -- 172.21.15.39:0/3616265796 wait complete. 2024-07-04T08:22:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.064+0000 7f0e20b07740 20 client.27245 trim_cache size 0 max 16384 2024-07-04T08:22:13.063 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 -- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0df80b29e0 msgr2=0x7f0df80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0df80b29e0 0x7f0df80b4ef0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5626e82ea450 tx=0x7f0e04005f90 comp rx=0 tx=0).stop 2024-07-04T08:22:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 -- 172.21.15.39:0/3473482604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e141089f0 msgr2=0x7f0e1419c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e141089f0 0x7f0e1419c180 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0e1406bcb0 tx=0x7f0dfc00ee50 comp rx=0 tx=0).stop 2024-07-04T08:22:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 -- 172.21.15.39:0/3473482604 shutdown_connections 2024-07-04T08:22:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0df80b29e0 0x7f0df80b4ef0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e14109340 0x7f0e141a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e141089f0 0x7f0e1419c180 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 --2- 172.21.15.39:0/3473482604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e141030b0 0x7f0e1419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 -- 172.21.15.39:0/3473482604 >> 172.21.15.39:0/3473482604 conn(0x7f0e1407ada0 msgr2=0x7f0e14106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 -- 172.21.15.39:0/3473482604 shutdown_connections 2024-07-04T08:22:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:13.065+0000 7f0e1a400640 1 -- 172.21.15.39:0/3473482604 wait complete. 2024-07-04T08:22:13.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:13 smithi133 ceph-mon[37050]: pgmap v691: 129 pgs: 129 active+clean; 7.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-07-04T08:22:13.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:13 smithi039 ceph-mon[32062]: pgmap v691: 129 pgs: 129 active+clean; 7.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-07-04T08:22:13.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:13 smithi050 ceph-mon[36577]: pgmap v691: 129 pgs: 129 active+clean; 7.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-07-04T08:22:13.998 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:22:14.462 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:14.800 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:14 smithi039 ceph-mon[32062]: osdmap e94: 12 total, 12 up, 12 in 2024-07-04T08:22:14.801 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:14 smithi039 ceph-mon[32062]: pgmap v693: 129 pgs: 129 active+clean; 7.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-07-04T08:22:14.810 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:14 smithi133 ceph-mon[37050]: osdmap e94: 12 total, 12 up, 12 in 2024-07-04T08:22:14.810 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:14 smithi133 ceph-mon[37050]: pgmap v693: 129 pgs: 129 active+clean; 7.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-07-04T08:22:14.888 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:14 smithi050 ceph-mon[36577]: osdmap e94: 12 total, 12 up, 12 in 2024-07-04T08:22:14.888 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:14 smithi050 ceph-mon[36577]: pgmap v693: 129 pgs: 129 active+clean; 7.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-07-04T08:22:16.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.980+0000 7fed34800640 1 -- 172.21.15.39:0/1462244454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c10b590 msgr2=0x7fed2c111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:16.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.980+0000 7fed34800640 1 --2- 172.21.15.39:0/1462244454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c10b590 0x7fed2c111dc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fed28009ca0 tx=0x7fed2802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.980+0000 7fed34800640 1 -- 172.21.15.39:0/1462244454 shutdown_connections 2024-07-04T08:22:16.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.980+0000 7fed34800640 1 --2- 172.21.15.39:0/1462244454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c10b590 0x7fed2c111dc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.980+0000 7fed34800640 1 --2- 172.21.15.39:0/1462244454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed2c10ac40 0x7fed2c10b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.980+0000 7fed34800640 1 --2- 172.21.15.39:0/1462244454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c105300 0x7fed2c105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.980+0000 7fed34800640 1 -- 172.21.15.39:0/1462244454 >> 172.21.15.39:0/1462244454 conn(0x7fed2c100d10 msgr2=0x7fed2c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:16.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.980+0000 7fed34800640 1 -- 172.21.15.39:0/1462244454 shutdown_connections 2024-07-04T08:22:16.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.980+0000 7fed34800640 1 -- 172.21.15.39:0/1462244454 wait complete. 2024-07-04T08:22:16.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.981+0000 7fed34800640 1 Processor -- start 2024-07-04T08:22:16.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.982+0000 7fed34800640 1 -- start start 2024-07-04T08:22:16.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.982+0000 7fed34800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c105300 0x7fed2c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.982+0000 7fed34800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c10ac40 0x7fed2c19c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.982+0000 7fed34800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed2c10b590 0x7fed2c1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.982+0000 7fed34800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c115240 con 0x7fed2c10ac40 2024-07-04T08:22:16.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.982+0000 7fed34800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1150c0 con 0x7fed2c10b590 2024-07-04T08:22:16.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.982+0000 7fed34800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1153c0 con 0x7fed2c105300 2024-07-04T08:22:16.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed31e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c105300 0x7fed2c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed31e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c105300 0x7fed2c19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59610/0 (socket says 172.21.15.39:59610) 2024-07-04T08:22:16.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed31e00640 1 -- 172.21.15.39:0/3180234092 learned_addr learned my addr 172.21.15.39:0/3180234092 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:16.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed31400640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c10ac40 0x7fed2c19c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed32800640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed2c10b590 0x7fed2c1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed31e00640 1 -- 172.21.15.39:0/3180234092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed2c10b590 msgr2=0x7fed2c1a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:16.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed31e00640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed2c10b590 0x7fed2c1a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed31e00640 1 -- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c10ac40 msgr2=0x7fed2c19c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:16.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed31e00640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c10ac40 0x7fed2c19c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.983+0000 7fed31e00640 1 -- 172.21.15.39:0/3180234092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c1a1b20 con 0x7fed2c105300 2024-07-04T08:22:16.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.984+0000 7fed31e00640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c105300 0x7fed2c19bbb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fed2000e9f0 tx=0x7fed2000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:16.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.987+0000 7fed26a00640 1 -- 172.21.15.39:0/3180234092 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2000cd80 con 0x7fed2c105300 2024-07-04T08:22:16.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.987+0000 7fed26a00640 1 -- 172.21.15.39:0/3180234092 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed200108e0 con 0x7fed2c105300 2024-07-04T08:22:16.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.987+0000 7fed34800640 1 -- 172.21.15.39:0/3180234092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c1a1e10 con 0x7fed2c105300 2024-07-04T08:22:16.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.987+0000 7fed26a00640 1 -- 172.21.15.39:0/3180234092 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20018860 con 0x7fed2c105300 2024-07-04T08:22:16.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.987+0000 7fed34800640 1 -- 172.21.15.39:0/3180234092 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1a8570 con 0x7fed2c105300 2024-07-04T08:22:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.991+0000 7fed3adb7740 1 Processor -- start 2024-07-04T08:22:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.991+0000 7fed26a00640 1 -- 172.21.15.39:0/3180234092 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fed20018a00 con 0x7fed2c105300 2024-07-04T08:22:16.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.991+0000 7fed3adb7740 1 -- start start 2024-07-04T08:22:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.991+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e56070 0x560f251fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.991+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f251facf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.991+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 0x560f251fb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.991+0000 7fed3adb7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560f25170ee0 con 0x560f24e2d1b0 2024-07-04T08:22:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.991+0000 7fed3adb7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560f25160480 con 0x560f24e35390 2024-07-04T08:22:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.991+0000 7fed3adb7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560f251644c0 con 0x560f24e56070 2024-07-04T08:22:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed32800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e56070 0x560f251fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed32800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e56070 0x560f251fa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59614/0 (socket says 172.21.15.39:59614) 2024-07-04T08:22:16.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed32800640 1 -- 172.21.15.39:0/1051485014 learned_addr learned my addr 172.21.15.39:0/1051485014 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed31e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f251facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed31400640 1 --2- 172.21.15.39:0/1051485014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 0x560f251fb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed32800640 1 -- 172.21.15.39:0/1051485014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 msgr2=0x560f251fb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed32800640 1 --2- 172.21.15.39:0/1051485014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 0x560f251fb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed32800640 1 -- 172.21.15.39:0/1051485014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 msgr2=0x560f251facf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed32800640 1 --2- 172.21.15.39:0/1051485014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f251facf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed32800640 1 -- 172.21.15.39:0/1051485014 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f250f8650 con 0x560f24e56070 2024-07-04T08:22:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.992+0000 7fed31e00640 1 --2- 172.21.15.39:0/1051485014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f251facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:22:16.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.993+0000 7fed32800640 1 --2- 172.21.15.39:0/1051485014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e56070 0x560f251fa7b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed2802f410 tx=0x7fed2802f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca40e2ab2ac26661 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.993+0000 7fed26a00640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed140b2ab0 0x7fed140b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.993+0000 7fed1b400640 1 -- 172.21.15.39:0/1051485014 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28031cf0 con 0x560f24e56070 2024-07-04T08:22:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.993+0000 7fed26a00640 1 -- 172.21.15.39:0/3180234092 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7fed200e2c20 con 0x7fed2c105300 2024-07-04T08:22:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.993+0000 7fed1b400640 1 -- 172.21.15.39:0/1051485014 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed28036030 con 0x560f24e56070 2024-07-04T08:22:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.993+0000 7fed31e00640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed140b2ab0 0x7fed140b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.993+0000 7fed1b400640 1 -- 172.21.15.39:0/1051485014 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed280042a0 con 0x560f24e56070 2024-07-04T08:22:16.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.994+0000 7fed3adb7740 1 -- 172.21.15.39:0/1051485014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e56070 msgr2=0x560f251fa7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:16.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.994+0000 7fed31e00640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed140b2ab0 0x7fed140b4fc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x560f251f9430 tx=0x7fed20000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:16.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.994+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1051485014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e56070 0x560f251fa7b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed2802f410 tx=0x7fed2802f440 comp rx=0 tx=0).stop 2024-07-04T08:22:16.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.994+0000 7fed3adb7740 1 -- 172.21.15.39:0/1051485014 shutdown_connections 2024-07-04T08:22:16.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.994+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1051485014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f251facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.994+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1051485014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 0x560f251fb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.994+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1051485014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e56070 0x560f251fa7b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.994+0000 7fed3adb7740 1 -- 172.21.15.39:0/1051485014 >> 172.21.15.39:0/1051485014 conn(0x560f24e4a460 msgr2=0x560f251f86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:16.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.995+0000 7fed3adb7740 1 -- 172.21.15.39:0/1051485014 shutdown_connections 2024-07-04T08:22:16.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.995+0000 7fed3adb7740 1 -- 172.21.15.39:0/1051485014 wait complete. 2024-07-04T08:22:16.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.995+0000 7fed3adb7740 1 -- start start 2024-07-04T08:22:16.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.996+0000 7fed3adb7740 1 Processor -- start 2024-07-04T08:22:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.996+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f25206350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.996+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e35390 0x560f25206890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.996+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 0x560f252120d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.996+0000 7fed3adb7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560f24fc4700 con 0x560f24e2d1b0 2024-07-04T08:22:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.996+0000 7fed3adb7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560f25107bb0 con 0x560f24e56070 2024-07-04T08:22:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.996+0000 7fed3adb7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560f25208db0 con 0x560f24e35390 2024-07-04T08:22:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.996+0000 7fed3adb7740 10 client.0 _ll_register_callbacks cb 0x560f25159570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed32800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f25206350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed31400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e35390 0x560f25206890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed32800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f25206350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43564/0 (socket says 172.21.15.39:43564) 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed31400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e35390 0x560f25206890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59628/0 (socket says 172.21.15.39:59628) 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed32800640 1 -- 172.21.15.39:0/2118001124 learned_addr learned my addr 172.21.15.39:0/2118001124 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed32800640 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 0x560f252120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed31400640 1 -- 172.21.15.39:0/2118001124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 msgr2=0x560f252120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed31400640 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 0x560f252120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed31400640 1 -- 172.21.15.39:0/2118001124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 msgr2=0x560f25206350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed31400640 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f25206350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed31400640 1 -- 172.21.15.39:0/2118001124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f24ec1650 con 0x560f24e35390 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.997+0000 7fed32800640 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f25206350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:22:16.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.998+0000 7fed31400640 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e35390 0x560f25206890 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fed2c06bc30 tx=0x7fed1c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1327abaed68a28a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.998+0000 7fed0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.998+0000 7fed0fe00640 1 -- 172.21.15.39:0/2118001124 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00f890 con 0x560f24e35390 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.998+0000 7fed3adb7740 1 -- 172.21.15.39:0/2118001124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed28036030 con 0x560f24e35390 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.998+0000 7fed0fe00640 1 -- 172.21.15.39:0/2118001124 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7fed1c00fa30 con 0x560f24e35390 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.998+0000 7fed0fe00640 1 -- 172.21.15.39:0/2118001124 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed1c00fc30 con 0x560f24e35390 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.998+0000 7fed3adb7740 10 client.27175 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.998+0000 7fed3adb7740 1 -- 172.21.15.39:0/2118001124 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f25214e20 con 0x560f24e35390 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.999+0000 7fed0fe00640 1 -- 172.21.15.39:0/2118001124 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed28036030 con 0x560f24e35390 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.999+0000 7fed0fe00640 1 -- 172.21.15.39:0/2118001124 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fed1c01c420 con 0x560f24e35390 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.999+0000 7fed3adb7740 10 client.27175 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.999+0000 7fed3adb7740 10 client.27175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.999+0000 7fed3adb7740 20 client.27175 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:16.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.999+0000 7fed3adb7740 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560f24e149f0 0x560f252183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:16.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.999+0000 7fed3adb7740 4 client.27175 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:16.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:16.999+0000 7fed3adb7740 1 -- 172.21.15.39:0/2118001124 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f25172e30 con 0x560f24e149f0 2024-07-04T08:22:16.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.000+0000 7fed32800640 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560f24e149f0 0x560f252183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.000+0000 7fed32800640 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560f24e149f0 0x560f252183f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b70b75cdc6a5827 server_cookie=64f93a52b8e4c32a in_seq=0 out_seq=0 2024-07-04T08:22:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.000+0000 7fed0fe00640 10 client.27175 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.001+0000 7fed0fe00640 1 -- 172.21.15.39:0/2118001124 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560f25172e30 con 0x560f24e149f0 2024-07-04T08:22:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.001+0000 7fed0fe00640 10 client.27175 handle_command_reply: tid=0 2024-07-04T08:22:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.001+0000 7fed3adb7740 1 client.27175 shutdown 2024-07-04T08:22:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 -- 172.21.15.39:0/2118001124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e35390 msgr2=0x560f25206890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e35390 0x560f25206890 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fed2c06bc30 tx=0x7fed1c012b60 comp rx=0 tx=0).stop 2024-07-04T08:22:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 -- 172.21.15.39:0/2118001124 shutdown_connections 2024-07-04T08:22:16.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560f24e149f0 0x560f252183f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 0x560f252120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e35390 0x560f25206890 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 --2- 172.21.15.39:0/2118001124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e2d1b0 0x560f25206350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:16.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 -- 172.21.15.39:0/2118001124 >> 172.21.15.39:0/2118001124 conn(0x560f24e4a460 msgr2=0x560f251fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:16.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed0fe00640 0 client.27175 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:16.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 -- 172.21.15.39:0/2118001124 shutdown_connections 2024-07-04T08:22:16.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 1 -- 172.21.15.39:0/2118001124 wait complete. 2024-07-04T08:22:16.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.002+0000 7fed3adb7740 20 client.27175 trim_cache size 0 max 16384 2024-07-04T08:22:17.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.017+0000 7fed3adb7740 1 Processor -- start 2024-07-04T08:22:17.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed3adb7740 1 -- start start 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 0x560f251fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e35390 0x560f251ffa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 0x560f251fff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed3adb7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b2480 con 0x560f24e35390 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed3adb7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b3fc0 con 0x560f24e56070 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed3adb7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b2b50 con 0x560f24e2d1b0 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed31400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e35390 0x560f251ffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed32800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 0x560f251fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed31400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e35390 0x560f251ffa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43574/0 (socket says 172.21.15.39:43574) 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.018+0000 7fed31400640 1 -- 172.21.15.39:0/452277102 learned_addr learned my addr 172.21.15.39:0/452277102 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:17.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.019+0000 7fed32800640 1 --2- 172.21.15.39:0/452277102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 0x560f251fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:17.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.019+0000 7fed31400640 1 -- 172.21.15.39:0/452277102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 msgr2=0x560f251fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:17.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.019+0000 7fed31400640 1 --2- 172.21.15.39:0/452277102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 0x560f251fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.019+0000 7fed31400640 1 -- 172.21.15.39:0/452277102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 msgr2=0x560f251fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:17.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.019+0000 7fed31400640 1 --2- 172.21.15.39:0/452277102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 0x560f251fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.019+0000 7fed31400640 1 -- 172.21.15.39:0/452277102 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f25200df0 con 0x560f24e35390 2024-07-04T08:22:17.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.019+0000 7fed31400640 1 --2- 172.21.15.39:0/452277102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e35390 0x560f251ffa30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fed1c000c00 tx=0x7fed1c01df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc6801a6ef26d9b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:17.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.020+0000 7fed1aa00640 1 -- 172.21.15.39:0/452277102 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c01cad0 con 0x560f24e35390 2024-07-04T08:22:17.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.020+0000 7fed1aa00640 1 -- 172.21.15.39:0/452277102 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1c01cc70 con 0x560f24e35390 2024-07-04T08:22:17.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.020+0000 7fed1aa00640 1 -- 172.21.15.39:0/452277102 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c004660 con 0x560f24e35390 2024-07-04T08:22:17.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 -- 172.21.15.39:0/452277102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e35390 msgr2=0x560f251ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:17.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 --2- 172.21.15.39:0/452277102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e35390 0x560f251ffa30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fed1c000c00 tx=0x7fed1c01df30 comp rx=0 tx=0).stop 2024-07-04T08:22:17.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 -- 172.21.15.39:0/452277102 shutdown_connections 2024-07-04T08:22:17.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 --2- 172.21.15.39:0/452277102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e35390 0x560f251ffa30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 --2- 172.21.15.39:0/452277102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e56070 0x560f251fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 --2- 172.21.15.39:0/452277102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 0x560f251fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 -- 172.21.15.39:0/452277102 >> 172.21.15.39:0/452277102 conn(0x560f24e4a460 msgr2=0x560f251fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:17.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 -- 172.21.15.39:0/452277102 shutdown_connections 2024-07-04T08:22:17.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 -- 172.21.15.39:0/452277102 wait complete. 2024-07-04T08:22:17.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.021+0000 7fed3adb7740 1 -- start start 2024-07-04T08:22:17.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.022+0000 7fed3adb7740 1 Processor -- start 2024-07-04T08:22:17.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.022+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 0x560f2520e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:17.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.022+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 0x560f25209dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:17.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.022+0000 7fed3adb7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e56070 0x560f2520a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:17.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.022+0000 7fed3adb7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b38f0 con 0x560f24e56070 2024-07-04T08:22:17.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.022+0000 7fed3adb7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b3220 con 0x560f24e35390 2024-07-04T08:22:17.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.022+0000 7fed3adb7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c1b0460 con 0x560f24e2d1b0 2024-07-04T08:22:17.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.022+0000 7fed3adb7740 10 client.0 _ll_register_callbacks cb 0x7fed1c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:17.018 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:16 smithi039 ceph-mon[32062]: pgmap v694: 129 pgs: 129 active+clean; 7.8 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-07-04T08:22:17.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed32800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 0x560f2520e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:17.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed32800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 0x560f2520e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59646/0 (socket says 172.21.15.39:59646) 2024-07-04T08:22:17.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed32800640 1 -- 172.21.15.39:0/1861729309 learned_addr learned my addr 172.21.15.39:0/1861729309 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed31400640 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 0x560f25209dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed32800640 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e56070 0x560f2520a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed31400640 1 -- 172.21.15.39:0/1861729309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 msgr2=0x560f2520e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed31400640 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 0x560f2520e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed31400640 1 -- 172.21.15.39:0/1861729309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e56070 msgr2=0x560f2520a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed31400640 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e56070 0x560f2520a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.023+0000 7fed31400640 1 -- 172.21.15.39:0/1861729309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f2527fb80 con 0x560f24e35390 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.024+0000 7fed31400640 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 0x560f25209dc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fed1c01ce70 tx=0x7fed1c00faa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44805c157c3aeb94 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.024+0000 7fed0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.024+0000 7fed0fe00640 1 -- 172.21.15.39:0/1861729309 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c036020 con 0x560f24e35390 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.024+0000 7fed0fe00640 1 -- 172.21.15.39:0/1861729309 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7fed1c005250 con 0x560f24e35390 2024-07-04T08:22:17.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.024+0000 7fed3adb7740 1 -- 172.21.15.39:0/1861729309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed1c01cc70 con 0x560f24e35390 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.024+0000 7fed0fe00640 1 -- 172.21.15.39:0/1861729309 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed1c01ca80 con 0x560f24e35390 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.024+0000 7fed3adb7740 10 client.27263 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.024+0000 7fed3adb7740 1 -- 172.21.15.39:0/1861729309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560f25280cb0 con 0x560f24e35390 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.025+0000 7fed0fe00640 1 -- 172.21.15.39:0/1861729309 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1c01cc70 con 0x560f24e35390 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.025+0000 7fed0fe00640 1 -- 172.21.15.39:0/1861729309 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fed1c010430 con 0x560f24e35390 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.025+0000 7fed3adb7740 10 client.27263 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.025+0000 7fed3adb7740 10 client.27263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.025+0000 7fed3adb7740 20 client.27263 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.025+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560f24de9da0 0x560f25283180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.025+0000 7fed3adb7740 4 client.27263 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:17.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.025+0000 7fed3adb7740 1 -- 172.21.15.39:0/1861729309 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f2516e8d0 con 0x560f24de9da0 2024-07-04T08:22:17.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.026+0000 7fed32800640 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560f24de9da0 0x560f25283180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.026+0000 7fed32800640 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560f24de9da0 0x560f25283180 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b09180437ebfea8 server_cookie=a9825e73b60722f3 in_seq=0 out_seq=0 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.026+0000 7fed0fe00640 10 client.27263 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.026+0000 7fed0fe00640 1 -- 172.21.15.39:0/1861729309 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f2516e8d0 con 0x560f24de9da0 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.026+0000 7fed0fe00640 10 client.27263 handle_command_reply: tid=0 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.026+0000 7fed3adb7740 1 client.27263 shutdown 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 -- 172.21.15.39:0/1861729309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 msgr2=0x560f25209dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 0x560f25209dc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fed1c01ce70 tx=0x7fed1c00faa0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 -- 172.21.15.39:0/1861729309 shutdown_connections 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560f24de9da0 0x560f25283180 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560f24e56070 0x560f2520a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560f24e35390 0x560f25209dc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 --2- 172.21.15.39:0/1861729309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f24e2d1b0 0x560f2520e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 -- 172.21.15.39:0/1861729309 >> 172.21.15.39:0/1861729309 conn(0x560f24e4a460 msgr2=0x560f251fb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:17.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed0fe00640 0 client.27263 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:17.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 -- 172.21.15.39:0/1861729309 shutdown_connections 2024-07-04T08:22:17.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 1 -- 172.21.15.39:0/1861729309 wait complete. 2024-07-04T08:22:17.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.027+0000 7fed3adb7740 20 client.27263 trim_cache size 0 max 16384 2024-07-04T08:22:17.024 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:17.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 -- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed140b2ab0 msgr2=0x7fed140b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:17.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed140b2ab0 0x7fed140b4fc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x560f251f9430 tx=0x7fed20000bc0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 -- 172.21.15.39:0/3180234092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c105300 msgr2=0x7fed2c19bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:17.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c105300 0x7fed2c19bbb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fed2000e9f0 tx=0x7fed2000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 -- 172.21.15.39:0/3180234092 shutdown_connections 2024-07-04T08:22:17.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fed140b2ab0 0x7fed140b4fc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed2c10ac40 0x7fed2c19c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fed2c10b590 0x7fed2c1a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 --2- 172.21.15.39:0/3180234092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c105300 0x7fed2c19bbb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:17.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 -- 172.21.15.39:0/3180234092 >> 172.21.15.39:0/3180234092 conn(0x7fed2c100d10 msgr2=0x7fed2c10a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:17.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 -- 172.21.15.39:0/3180234092 shutdown_connections 2024-07-04T08:22:17.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:17.028+0000 7fed34800640 1 -- 172.21.15.39:0/3180234092 wait complete. 2024-07-04T08:22:17.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:16 smithi133 ceph-mon[37050]: pgmap v694: 129 pgs: 129 active+clean; 7.8 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-07-04T08:22:17.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:16 smithi050 ceph-mon[36577]: pgmap v694: 129 pgs: 129 active+clean; 7.8 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-07-04T08:22:18.926 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:22:18.927 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:22:19.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:18 smithi133 ceph-mon[37050]: pgmap v695: 129 pgs: 129 active+clean; 7.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-07-04T08:22:19.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:18 smithi039 ceph-mon[32062]: pgmap v695: 129 pgs: 129 active+clean; 7.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-07-04T08:22:19.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:18 smithi050 ceph-mon[36577]: pgmap v695: 129 pgs: 129 active+clean; 7.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-07-04T08:22:19.440 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:21.094 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:20 smithi039 ceph-mon[32062]: pgmap v696: 129 pgs: 129 active+clean; 7.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-07-04T08:22:21.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:20 smithi133 ceph-mon[37050]: pgmap v696: 129 pgs: 129 active+clean; 7.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-07-04T08:22:21.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:20 smithi050 ceph-mon[36577]: pgmap v696: 129 pgs: 129 active+clean; 7.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-07-04T08:22:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.706+0000 7f977fe00640 1 -- 172.21.15.39:0/2138458051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97781089f0 msgr2=0x7f9778108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.706+0000 7f977fe00640 1 --2- 172.21.15.39:0/2138458051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97781089f0 0x7f9778108e00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9764009ca0 tx=0x7f976402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.706+0000 7f977fe00640 1 -- 172.21.15.39:0/2138458051 shutdown_connections 2024-07-04T08:22:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.706+0000 7f977fe00640 1 --2- 172.21.15.39:0/2138458051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97781089f0 0x7f9778108e00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.706+0000 7f977fe00640 1 --2- 172.21.15.39:0/2138458051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9778109340 0x7f977810fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.706+0000 7f977fe00640 1 --2- 172.21.15.39:0/2138458051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97781030b0 0x7f97781034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.706+0000 7f977fe00640 1 -- 172.21.15.39:0/2138458051 >> 172.21.15.39:0/2138458051 conn(0x7f977807ada0 msgr2=0x7f977807b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:21.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.706+0000 7f977fe00640 1 -- 172.21.15.39:0/2138458051 shutdown_connections 2024-07-04T08:22:21.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.706+0000 7f977fe00640 1 -- 172.21.15.39:0/2138458051 wait complete. 2024-07-04T08:22:21.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.708+0000 7f977fe00640 1 Processor -- start 2024-07-04T08:22:21.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.708+0000 7f977fe00640 1 -- start start 2024-07-04T08:22:21.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.708+0000 7f977fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97781030b0 0x7f9778074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.708+0000 7f977fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97781089f0 0x7f97780751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.708+0000 7f977fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9778109340 0x7f97780756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9778112fb0 con 0x7f97781030b0 2024-07-04T08:22:21.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9778112e30 con 0x7f97781089f0 2024-07-04T08:22:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9778113130 con 0x7f9778109340 2024-07-04T08:22:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97781030b0 0x7f9778074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97781089f0 0x7f97780751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97781030b0 0x7f9778074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51796/0 (socket says 172.21.15.39:51796) 2024-07-04T08:22:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97781089f0 0x7f97780751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50788/0 (socket says 172.21.15.39:50788) 2024-07-04T08:22:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977d400640 1 -- 172.21.15.39:0/199256127 learned_addr learned my addr 172.21.15.39:0/199256127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:21.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977de00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9778109340 0x7f97780756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977ca00640 1 -- 172.21.15.39:0/199256127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9778109340 msgr2=0x7f97780756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.709+0000 7f977ca00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9778109340 0x7f97780756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.710+0000 7f977ca00640 1 -- 172.21.15.39:0/199256127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97781030b0 msgr2=0x7f9778074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.710+0000 7f977ca00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97781030b0 0x7f9778074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.710+0000 7f977ca00640 1 -- 172.21.15.39:0/199256127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97781a7d60 con 0x7f97781089f0 2024-07-04T08:22:21.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.710+0000 7f977ca00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97781089f0 0x7f97780751a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f976402ade0 tx=0x7f9764031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:21.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.710+0000 7f9776000640 1 -- 172.21.15.39:0/199256127 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764003b70 con 0x7f97781089f0 2024-07-04T08:22:21.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.710+0000 7f9776000640 1 -- 172.21.15.39:0/199256127 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9764031910 con 0x7f97781089f0 2024-07-04T08:22:21.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.710+0000 7f977fe00640 1 -- 172.21.15.39:0/199256127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97781a7ff0 con 0x7f97781089f0 2024-07-04T08:22:21.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.710+0000 7f9776000640 1 -- 172.21.15.39:0/199256127 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976402f950 con 0x7f97781089f0 2024-07-04T08:22:21.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.710+0000 7f977fe00640 1 -- 172.21.15.39:0/199256127 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97781a84a0 con 0x7f97781089f0 2024-07-04T08:22:21.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.714+0000 7f97863b1740 1 Processor -- start 2024-07-04T08:22:21.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.714+0000 7f97863b1740 1 -- start start 2024-07-04T08:22:21.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f9776000640 1 -- 172.21.15.39:0/199256127 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f976402faf0 con 0x7f97781089f0 2024-07-04T08:22:21.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f97863b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d45a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f97863b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d45ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f97863b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 0x557f3d45b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f97863b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557f3d3848e0 con 0x557f3d08d490 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f97863b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557f3d3d0ec0 con 0x557f3d095670 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f97863b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557f3d3c0460 con 0x557f3d0b6350 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f977d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d45ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f977de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d45a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f977d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d45ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51800/0 (socket says 172.21.15.39:51800) 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f977d400640 1 -- 172.21.15.39:0/3733765072 learned_addr learned my addr 172.21.15.39:0/3733765072 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:21.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.715+0000 7f977ca00640 1 --2- 172.21.15.39:0/3733765072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 0x557f3d45b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.716+0000 7f977d400640 1 -- 172.21.15.39:0/3733765072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 msgr2=0x557f3d45a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.716+0000 7f977d400640 1 --2- 172.21.15.39:0/3733765072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d45a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.716+0000 7f977d400640 1 -- 172.21.15.39:0/3733765072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 msgr2=0x557f3d45b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.716+0000 7f977d400640 1 --2- 172.21.15.39:0/3733765072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 0x557f3d45b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.716+0000 7f977d400640 1 -- 172.21.15.39:0/3733765072 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3d358280 con 0x557f3d08d490 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.716+0000 7f977de00640 1 --2- 172.21.15.39:0/3733765072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d45a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.716+0000 7f977d400640 1 --2- 172.21.15.39:0/3733765072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d45ab80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9768009d70 tx=0x7f9768011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1580d24922d87bcf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.717+0000 7f976ea00640 1 -- 172.21.15.39:0/3733765072 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976801e070 con 0x557f3d08d490 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.717+0000 7f976ea00640 1 -- 172.21.15.39:0/3733765072 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9768019ae0 con 0x557f3d08d490 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.717+0000 7f976ea00640 1 -- 172.21.15.39:0/3733765072 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976800eb40 con 0x557f3d08d490 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.717+0000 7f9776000640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f975c0b29e0 0x7f975c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.717+0000 7f9776000640 1 -- 172.21.15.39:0/199256127 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f9764103980 con 0x7f97781089f0 2024-07-04T08:22:21.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.717+0000 7f977de00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f975c0b29e0 0x7f975c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f97863b1740 1 -- 172.21.15.39:0/3733765072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 msgr2=0x557f3d45ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:21.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f97863b1740 1 --2- 172.21.15.39:0/3733765072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d45ab80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9768009d70 tx=0x7f9768011810 comp rx=0 tx=0).stop 2024-07-04T08:22:21.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f97863b1740 1 -- 172.21.15.39:0/3733765072 shutdown_connections 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f97863b1740 1 --2- 172.21.15.39:0/3733765072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d45ab80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f97863b1740 1 --2- 172.21.15.39:0/3733765072 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 0x557f3d45b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f97863b1740 1 --2- 172.21.15.39:0/3733765072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d45a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f97863b1740 1 -- 172.21.15.39:0/3733765072 >> 172.21.15.39:0/3733765072 conn(0x557f3d0aa740 msgr2=0x557f3d458530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f97863b1740 1 -- 172.21.15.39:0/3733765072 shutdown_connections 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f97863b1740 1 -- 172.21.15.39:0/3733765072 wait complete. 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.718+0000 7f977de00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f975c0b29e0 0x7f975c0b4ef0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9770009e10 tx=0x7f9770005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.719+0000 7f97863b1740 1 -- start start 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.719+0000 7f97863b1740 1 Processor -- start 2024-07-04T08:22:21.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f97863b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 0x557f3d465ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f97863b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d095670 0x557f3d466420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f97863b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d0b6350 0x557f3d471fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f97863b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557f3d367760 con 0x557f3d0b6350 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f97863b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557f3d3c44a0 con 0x557f3d08d490 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f97863b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557f3d224890 con 0x557f3d095670 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f97863b1740 10 client.0 _ll_register_callbacks cb 0x557f3d3682b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f977ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d095670 0x557f3d466420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f977d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 0x557f3d465ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f977ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d095670 0x557f3d466420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57934/0 (socket says 172.21.15.39:57934) 2024-07-04T08:22:21.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f977d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 0x557f3d465ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50806/0 (socket says 172.21.15.39:50806) 2024-07-04T08:22:21.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f977ca00640 1 -- 172.21.15.39:0/4117720210 learned_addr learned my addr 172.21.15.39:0/4117720210 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:21.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.720+0000 7f977de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d0b6350 0x557f3d471fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f977ca00640 1 -- 172.21.15.39:0/4117720210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 msgr2=0x557f3d465ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f977ca00640 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 0x557f3d465ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f977ca00640 1 -- 172.21.15.39:0/4117720210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d0b6350 msgr2=0x557f3d471fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f977ca00640 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d0b6350 0x557f3d471fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f977ca00640 1 -- 172.21.15.39:0/4117720210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3d121930 con 0x557f3d095670 2024-07-04T08:22:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f977ca00640 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d095670 0x557f3d466420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f97640ca6f0 tx=0x7f97640c6970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e67c65cd481112d0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f9763400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f9763400640 1 -- 172.21.15.39:0/4117720210 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640ca160 con 0x557f3d095670 2024-07-04T08:22:21.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f9763400640 1 -- 172.21.15.39:0/4117720210 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f9764100fd0 con 0x557f3d095670 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.721+0000 7f97863b1740 1 -- 172.21.15.39:0/4117720210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9768019ae0 con 0x557f3d095670 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.722+0000 7f9763400640 1 -- 172.21.15.39:0/4117720210 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97641011d0 con 0x557f3d095670 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.722+0000 7f97863b1740 10 client.27193 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.722+0000 7f97863b1740 1 -- 172.21.15.39:0/4117720210 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f3d474b30 con 0x557f3d095670 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.722+0000 7f9763400640 1 -- 172.21.15.39:0/4117720210 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9768019ae0 con 0x557f3d095670 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.722+0000 7f9763400640 1 -- 172.21.15.39:0/4117720210 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f97640ed0d0 con 0x557f3d095670 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.722+0000 7f97863b1740 10 client.27193 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.722+0000 7f97863b1740 10 client.27193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.722+0000 7f97863b1740 20 client.27193 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:21.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.722+0000 7f97863b1740 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f3d074cd0 0x557f3d477f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.723+0000 7f97863b1740 4 client.27193 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.723+0000 7f97863b1740 1 -- 172.21.15.39:0/4117720210 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f3d3d2e10 con 0x557f3d074cd0 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.723+0000 7f977d400640 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f3d074cd0 0x557f3d477f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.724+0000 7f977d400640 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f3d074cd0 0x557f3d477f50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2535336cfc82f9 server_cookie=9e7be16ab42b580a in_seq=0 out_seq=0 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.724+0000 7f9763400640 10 client.27193 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.724+0000 7f9763400640 1 -- 172.21.15.39:0/4117720210 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f3d3d2e10 con 0x557f3d074cd0 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.724+0000 7f9763400640 10 client.27193 handle_command_reply: tid=0 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.724+0000 7f97863b1740 1 client.27193 shutdown 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 -- 172.21.15.39:0/4117720210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d095670 msgr2=0x557f3d466420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:21.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d095670 0x557f3d466420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f97640ca6f0 tx=0x7f97640c6970 comp rx=0 tx=0).stop 2024-07-04T08:22:21.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 -- 172.21.15.39:0/4117720210 shutdown_connections 2024-07-04T08:22:21.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f3d074cd0 0x557f3d477f50 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d0b6350 0x557f3d471fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d095670 0x557f3d466420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 --2- 172.21.15.39:0/4117720210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 0x557f3d465ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 -- 172.21.15.39:0/4117720210 >> 172.21.15.39:0/4117720210 conn(0x557f3d0aa740 msgr2=0x557f3d45d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:21.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f9763400640 0 client.27193 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:21.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 -- 172.21.15.39:0/4117720210 shutdown_connections 2024-07-04T08:22:21.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 1 -- 172.21.15.39:0/4117720210 wait complete. 2024-07-04T08:22:21.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.725+0000 7f97863b1740 20 client.27193 trim_cache size 0 max 16384 2024-07-04T08:22:21.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.746+0000 7f97863b1740 1 Processor -- start 2024-07-04T08:22:21.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.746+0000 7f97863b1740 1 -- start start 2024-07-04T08:22:21.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.746+0000 7f97863b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 0x557f3d45de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.746+0000 7f97863b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d095670 0x557f3d471db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.747+0000 7f97863b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d4722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.747+0000 7f97863b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b2540 con 0x557f3d095670 2024-07-04T08:22:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.747+0000 7f97863b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b4080 con 0x557f3d08d490 2024-07-04T08:22:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.747+0000 7f97863b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b2c10 con 0x557f3d0b6350 2024-07-04T08:22:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.747+0000 7f977ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d095670 0x557f3d471db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.747+0000 7f977ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d095670 0x557f3d471db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51832/0 (socket says 172.21.15.39:51832) 2024-07-04T08:22:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.747+0000 7f977ca00640 1 -- 172.21.15.39:0/4270457296 learned_addr learned my addr 172.21.15.39:0/4270457296 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.747+0000 7f977d400640 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 0x557f3d45de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.747+0000 7f977de00640 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d4722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f977ca00640 1 -- 172.21.15.39:0/4270457296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 msgr2=0x557f3d4722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:21.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f977ca00640 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d4722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f977ca00640 1 -- 172.21.15.39:0/4270457296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 msgr2=0x557f3d45de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:21.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f977ca00640 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 0x557f3d45de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f977ca00640 1 -- 172.21.15.39:0/4270457296 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3d45d4a0 con 0x557f3d095670 2024-07-04T08:22:21.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f977de00640 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d4722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:21.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f977ca00640 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d095670 0x557f3d471db0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9764000f80 tx=0x7f9764004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4b5b7e739c98e7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:21.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f976e000640 1 -- 172.21.15.39:0/4270457296 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640e4030 con 0x557f3d095670 2024-07-04T08:22:21.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f976e000640 1 -- 172.21.15.39:0/4270457296 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9764036070 con 0x557f3d095670 2024-07-04T08:22:21.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.748+0000 7f976e000640 1 -- 172.21.15.39:0/4270457296 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640ed0d0 con 0x557f3d095670 2024-07-04T08:22:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.750+0000 7f97863b1740 1 -- 172.21.15.39:0/4270457296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d095670 msgr2=0x557f3d471db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.750+0000 7f97863b1740 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d095670 0x557f3d471db0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9764000f80 tx=0x7f9764004820 comp rx=0 tx=0).stop 2024-07-04T08:22:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.750+0000 7f97863b1740 1 -- 172.21.15.39:0/4270457296 shutdown_connections 2024-07-04T08:22:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.750+0000 7f97863b1740 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d095670 0x557f3d471db0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.750+0000 7f97863b1740 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d4722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.750+0000 7f97863b1740 1 --2- 172.21.15.39:0/4270457296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d08d490 0x557f3d45de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.750+0000 7f97863b1740 1 -- 172.21.15.39:0/4270457296 >> 172.21.15.39:0/4270457296 conn(0x557f3d0aa740 msgr2=0x557f3d45d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:21.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.751+0000 7f97863b1740 1 -- 172.21.15.39:0/4270457296 shutdown_connections 2024-07-04T08:22:21.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.751+0000 7f97863b1740 1 -- 172.21.15.39:0/4270457296 wait complete. 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.751+0000 7f97863b1740 1 -- start start 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.751+0000 7f97863b1740 1 Processor -- start 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f97863b1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d46e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f97863b1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 0x557f3d469aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f97863b1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d469fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f97863b1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b39b0 con 0x557f3d08d490 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f97863b1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b32e0 con 0x557f3d095670 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f97863b1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97781b0520 con 0x557f3d0b6350 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f97863b1740 10 client.0 _ll_register_callbacks cb 0x7f97640095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:21.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f977d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d46e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f977d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d46e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51834/0 (socket says 172.21.15.39:51834) 2024-07-04T08:22:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.752+0000 7f977d400640 1 -- 172.21.15.39:0/1616092772 learned_addr learned my addr 172.21.15.39:0/1616092772 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977de00640 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d469fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977ca00640 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 0x557f3d469aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977d400640 1 -- 172.21.15.39:0/1616092772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 msgr2=0x557f3d469fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:21.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977d400640 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d469fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977d400640 1 -- 172.21.15.39:0/1616092772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 msgr2=0x557f3d469aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:21.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977d400640 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 0x557f3d469aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977d400640 1 -- 172.21.15.39:0/1616092772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3d46a770 con 0x557f3d08d490 2024-07-04T08:22:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977de00640 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d469fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977ca00640 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 0x557f3d469aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f977d400640 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d46e5f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x557f3d4592c0 tx=0x7f9768005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ddfa9a9169e988 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.753+0000 7f9763400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:22:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.754+0000 7f9763400640 1 -- 172.21.15.39:0/1616092772 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976801e040 con 0x557f3d08d490 2024-07-04T08:22:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.754+0000 7f97863b1740 1 -- 172.21.15.39:0/1616092772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9764036070 con 0x557f3d08d490 2024-07-04T08:22:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.754+0000 7f9763400640 1 -- 172.21.15.39:0/1616092772 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f97680050c0 con 0x557f3d08d490 2024-07-04T08:22:21.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.754+0000 7f9763400640 1 -- 172.21.15.39:0/1616092772 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97680052c0 con 0x557f3d08d490 2024-07-04T08:22:21.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.754+0000 7f97863b1740 10 client.17145 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:21.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.754+0000 7f97863b1740 1 -- 172.21.15.39:0/1616092772 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f3d4e06a0 con 0x557f3d08d490 2024-07-04T08:22:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.755+0000 7f9763400640 1 -- 172.21.15.39:0/1616092772 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9764036070 con 0x557f3d08d490 2024-07-04T08:22:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.755+0000 7f9763400640 1 -- 172.21.15.39:0/1616092772 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f976800e750 con 0x557f3d08d490 2024-07-04T08:22:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.755+0000 7f97863b1740 10 client.17145 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.755+0000 7f97863b1740 10 client.17145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.755+0000 7f97863b1740 20 client.17145 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.755+0000 7f97863b1740 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f3d04a080 0x557f3d4e2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.755+0000 7f97863b1740 4 client.17145 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:22:21.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.755+0000 7f97863b1740 1 -- 172.21.15.39:0/1616092772 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557f3d3ce8b0 con 0x557f3d04a080 2024-07-04T08:22:21.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.755+0000 7f977ca00640 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f3d04a080 0x557f3d4e2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:21.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.756+0000 7f9763400640 10 client.17145 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:21.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.756+0000 7f977ca00640 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f3d04a080 0x557f3d4e2d00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb2df622a077c3b server_cookie=e03628d6869149a0 in_seq=0 out_seq=0 2024-07-04T08:22:21.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.757+0000 7f9763400640 1 -- 172.21.15.39:0/1616092772 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557f3d3ce8b0 con 0x557f3d04a080 2024-07-04T08:22:21.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.757+0000 7f9763400640 10 client.17145 handle_command_reply: tid=0 2024-07-04T08:22:21.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f97863b1740 1 client.17145 shutdown 2024-07-04T08:22:21.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f97863b1740 1 -- 172.21.15.39:0/1616092772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 msgr2=0x557f3d46e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:21.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f97863b1740 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d46e5f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x557f3d4592c0 tx=0x7f9768005da0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f97863b1740 1 -- 172.21.15.39:0/1616092772 shutdown_connections 2024-07-04T08:22:21.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f97863b1740 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557f3d04a080 0x557f3d4e2d00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f97863b1740 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f3d0b6350 0x557f3d469fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f97863b1740 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f3d095670 0x557f3d469aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f97863b1740 1 --2- 172.21.15.39:0/1616092772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f3d08d490 0x557f3d46e5f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f9763400640 0 client.17145 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.758+0000 7f97863b1740 1 -- 172.21.15.39:0/1616092772 >> 172.21.15.39:0/1616092772 conn(0x557f3d0aa740 msgr2=0x557f3d467270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.759+0000 7f97863b1740 1 -- 172.21.15.39:0/1616092772 shutdown_connections 2024-07-04T08:22:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.759+0000 7f97863b1740 1 -- 172.21.15.39:0/1616092772 wait complete. 2024-07-04T08:22:21.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.759+0000 7f97863b1740 20 client.17145 trim_cache size 0 max 16384 2024-07-04T08:22:21.755 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:22:21.755 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:22:21.756 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "bd7a1c16-fd1d-43c6-8484-afa46b351cf3", 2024-07-04T08:22:21.756 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:22:21.756 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:22:21.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.760+0000 7f977fe00640 1 -- 172.21.15.39:0/199256127 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f975c0b29e0 msgr2=0x7f975c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:21.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.760+0000 7f977fe00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f975c0b29e0 0x7f975c0b4ef0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9770009e10 tx=0x7f9770005f90 comp rx=0 tx=0).stop 2024-07-04T08:22:21.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.760+0000 7f977fe00640 1 -- 172.21.15.39:0/199256127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97781089f0 msgr2=0x7f97780751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:21.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.760+0000 7f977fe00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97781089f0 0x7f97780751a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f976402ade0 tx=0x7f9764031660 comp rx=0 tx=0).stop 2024-07-04T08:22:21.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.761+0000 7f977fe00640 1 -- 172.21.15.39:0/199256127 shutdown_connections 2024-07-04T08:22:21.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.761+0000 7f977fe00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f975c0b29e0 0x7f975c0b4ef0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.761+0000 7f977fe00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9778109340 0x7f97780756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.761+0000 7f977fe00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f97781089f0 0x7f97780751a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.761+0000 7f977fe00640 1 --2- 172.21.15.39:0/199256127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97781030b0 0x7f9778074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.761+0000 7f977fe00640 1 -- 172.21.15.39:0/199256127 >> 172.21.15.39:0/199256127 conn(0x7f977807ada0 msgr2=0x7f9778106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.761+0000 7f977fe00640 1 -- 172.21.15.39:0/199256127 shutdown_connections 2024-07-04T08:22:21.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:21.761+0000 7f977fe00640 1 -- 172.21.15.39:0/199256127 wait complete. 2024-07-04T08:22:22.628 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:22:23.107 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:23.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:23 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:22:23.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:23 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:23.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:23 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:22:23.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:23 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:23.255 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:23 smithi039 ceph-mon[32062]: pgmap v697: 129 pgs: 129 active+clean; 8.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-07-04T08:22:23.256 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:23 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:22:23.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:23 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:22:23.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:23 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:23.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:23 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:22:23.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:23 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:23.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:23 smithi133 ceph-mon[37050]: pgmap v697: 129 pgs: 129 active+clean; 8.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-07-04T08:22:23.310 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:23 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:22:23.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:23 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:22:23.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:23 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:23.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:23 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:22:23.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:23 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:23.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:23 smithi050 ceph-mon[36577]: pgmap v697: 129 pgs: 129 active+clean; 8.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 15 op/s 2024-07-04T08:22:23.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:23 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:22:25.155 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:25 smithi039 ceph-mon[32062]: pgmap v698: 129 pgs: 129 active+clean; 8.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-07-04T08:22:25.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:25 smithi133 ceph-mon[37050]: pgmap v698: 129 pgs: 129 active+clean; 8.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-07-04T08:22:25.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:25 smithi050 ceph-mon[36577]: pgmap v698: 129 pgs: 129 active+clean; 8.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-07-04T08:22:25.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.506+0000 7f2843600640 1 -- 172.21.15.39:0/1012688200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f283c1078f0 msgr2=0x7f283c107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.506+0000 7f2843600640 1 --2- 172.21.15.39:0/1012688200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f283c1078f0 0x7f283c107d00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2828009860 tx=0x7f282802a340 comp rx=0 tx=0).stop 2024-07-04T08:22:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.507+0000 7f2843600640 1 -- 172.21.15.39:0/1012688200 shutdown_connections 2024-07-04T08:22:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.507+0000 7f2843600640 1 --2- 172.21.15.39:0/1012688200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f283c108240 0x7f283c10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.507+0000 7f2843600640 1 --2- 172.21.15.39:0/1012688200 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f283c1078f0 0x7f283c107d00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.507+0000 7f2843600640 1 --2- 172.21.15.39:0/1012688200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c101fb0 0x7f283c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.507+0000 7f2843600640 1 -- 172.21.15.39:0/1012688200 >> 172.21.15.39:0/1012688200 conn(0x7f283c078c80 msgr2=0x7f283c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:25.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.507+0000 7f2843600640 1 -- 172.21.15.39:0/1012688200 shutdown_connections 2024-07-04T08:22:25.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.507+0000 7f2843600640 1 -- 172.21.15.39:0/1012688200 wait complete. 2024-07-04T08:22:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.508+0000 7f2843600640 1 Processor -- start 2024-07-04T08:22:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.508+0000 7f2843600640 1 -- start start 2024-07-04T08:22:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2843600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f283c101fb0 0x7f283c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2843600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f283c1078f0 0x7f283c199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2843600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c108240 0x7f283c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2843600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f283c111eb0 con 0x7f283c101fb0 2024-07-04T08:22:25.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2843600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f283c111d30 con 0x7f283c1078f0 2024-07-04T08:22:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2843600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f283c112030 con 0x7f283c108240 2024-07-04T08:22:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2840c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f283c101fb0 0x7f283c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2840c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f283c101fb0 0x7f283c199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51842/0 (socket says 172.21.15.39:51842) 2024-07-04T08:22:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f283be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f283c1078f0 0x7f283c199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2840c00640 1 -- 172.21.15.39:0/189096730 learned_addr learned my addr 172.21.15.39:0/189096730 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.509+0000 7f2841600640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c108240 0x7f283c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.510+0000 7f2840c00640 1 -- 172.21.15.39:0/189096730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c108240 msgr2=0x7f283c19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.510+0000 7f2840c00640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c108240 0x7f283c19f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.510+0000 7f2840c00640 1 -- 172.21.15.39:0/189096730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f283c1078f0 msgr2=0x7f283c199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.510+0000 7f2840c00640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f283c1078f0 0x7f283c199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.510+0000 7f2840c00640 1 -- 172.21.15.39:0/189096730 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c19f900 con 0x7f283c101fb0 2024-07-04T08:22:25.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.510+0000 7f2840c00640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f283c101fb0 0x7f283c199990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f283000e9c0 tx=0x7f283000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:25.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.510+0000 7f2839600640 1 -- 172.21.15.39:0/189096730 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283000ccc0 con 0x7f283c101fb0 2024-07-04T08:22:25.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.511+0000 7f2839600640 1 -- 172.21.15.39:0/189096730 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28300054d0 con 0x7f283c101fb0 2024-07-04T08:22:25.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.511+0000 7f2839600640 1 -- 172.21.15.39:0/189096730 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830010430 con 0x7f283c101fb0 2024-07-04T08:22:25.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.511+0000 7f2843600640 1 -- 172.21.15.39:0/189096730 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c19fbf0 con 0x7f283c101fb0 2024-07-04T08:22:25.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.511+0000 7f2843600640 1 -- 172.21.15.39:0/189096730 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c1a6350 con 0x7f283c101fb0 2024-07-04T08:22:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.514+0000 7f2849c3b740 1 Processor -- start 2024-07-04T08:22:25.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.514+0000 7f2849c3b740 1 -- start start 2024-07-04T08:22:25.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 0x563240b21770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5632407541b0 0x563240b21cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f2849c3b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563240a97ea0 con 0x56324075c390 2024-07-04T08:22:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f2849c3b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563240a87440 con 0x5632407541b0 2024-07-04T08:22:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f2849c3b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563240a8b480 con 0x56324077d070 2024-07-04T08:22:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f283be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f283be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b221f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51848/0 (socket says 172.21.15.39:51848) 2024-07-04T08:22:25.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f283be00640 1 -- 172.21.15.39:0/1117074998 learned_addr learned my addr 172.21.15.39:0/1117074998 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:25.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.515+0000 7f2841600640 1 --2- 172.21.15.39:0/1117074998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 0x563240b21770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.516+0000 7f283be00640 1 -- 172.21.15.39:0/1117074998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 msgr2=0x563240b21770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:25.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.516+0000 7f283be00640 1 --2- 172.21.15.39:0/1117074998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 0x563240b21770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.516+0000 7f283be00640 1 -- 172.21.15.39:0/1117074998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5632407541b0 msgr2=0x563240b21cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:22:25.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.516+0000 7f283be00640 1 --2- 172.21.15.39:0/1117074998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5632407541b0 0x563240b21cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.516+0000 7f283be00640 1 -- 172.21.15.39:0/1117074998 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563240a1f610 con 0x56324075c390 2024-07-04T08:22:25.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.516+0000 7f2841600640 1 --2- 172.21.15.39:0/1117074998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 0x563240b21770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:25.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.516+0000 7f283be00640 1 --2- 172.21.15.39:0/1117074998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b221f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f283c069ff0 tx=0x7f2828005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0378c649d0f9e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.516+0000 7f2839600640 1 -- 172.21.15.39:0/189096730 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2830004f80 con 0x7f283c101fb0 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.517+0000 7f282e000640 1 -- 172.21.15.39:0/1117074998 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828037040 con 0x56324075c390 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.517+0000 7f282e000640 1 -- 172.21.15.39:0/1117074998 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2828030450 con 0x56324075c390 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.517+0000 7f282e000640 1 -- 172.21.15.39:0/1117074998 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828003950 con 0x56324075c390 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2849c3b740 1 -- 172.21.15.39:0/1117074998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 msgr2=0x563240b221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2849c3b740 1 --2- 172.21.15.39:0/1117074998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b221f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f283c069ff0 tx=0x7f2828005d30 comp rx=0 tx=0).stop 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2849c3b740 1 -- 172.21.15.39:0/1117074998 shutdown_connections 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2849c3b740 1 --2- 172.21.15.39:0/1117074998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b221f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2849c3b740 1 --2- 172.21.15.39:0/1117074998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5632407541b0 0x563240b21cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2849c3b740 1 --2- 172.21.15.39:0/1117074998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 0x563240b21770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2849c3b740 1 -- 172.21.15.39:0/1117074998 >> 172.21.15.39:0/1117074998 conn(0x563240771460 msgr2=0x563240b1f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2849c3b740 1 -- 172.21.15.39:0/1117074998 shutdown_connections 2024-07-04T08:22:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2849c3b740 1 -- 172.21.15.39:0/1117074998 wait complete. 2024-07-04T08:22:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.518+0000 7f2839600640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28240b2870 0x7f28240b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.519+0000 7f2839600640 1 -- 172.21.15.39:0/189096730 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f28300e2ca0 con 0x7f283c101fb0 2024-07-04T08:22:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.519+0000 7f283be00640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28240b2870 0x7f28240b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.519+0000 7f2849c3b740 1 -- start start 2024-07-04T08:22:25.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.519+0000 7f2849c3b740 1 Processor -- start 2024-07-04T08:22:25.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f283be00640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28240b2870 0x7f28240b4d80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2828006010 tx=0x7f2828000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:25.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b31730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324075c390 0x563240b31c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 0x563240b3d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f2849c3b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5632408eb700 con 0x5632407541b0 2024-07-04T08:22:25.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f2849c3b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563240a2eb70 con 0x56324077d070 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f2849c3b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563240b34250 con 0x56324075c390 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f2849c3b740 10 client.0 _ll_register_callbacks cb 0x563240a80530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f283be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324075c390 0x563240b31c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f283be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324075c390 0x563240b31c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57996/0 (socket says 172.21.15.39:57996) 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.520+0000 7f283be00640 1 -- 172.21.15.39:0/2852899347 learned_addr learned my addr 172.21.15.39:0/2852899347 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.521+0000 7f2841600640 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 0x563240b3d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.521+0000 7f2840c00640 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b31730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.521+0000 7f283be00640 1 -- 172.21.15.39:0/2852899347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 msgr2=0x563240b3d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.521+0000 7f283be00640 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 0x563240b3d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.521+0000 7f283be00640 1 -- 172.21.15.39:0/2852899347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 msgr2=0x563240b31730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:25.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.521+0000 7f283be00640 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b31730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.521+0000 7f283be00640 1 -- 172.21.15.39:0/2852899347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632407e8650 con 0x56324075c390 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.521+0000 7f283be00640 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324075c390 0x563240b31c70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f282802a930 tx=0x7f28280035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26d6dd4b2d5a03d1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.521+0000 7f2822a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.522+0000 7f2822a00640 1 -- 172.21.15.39:0/2852899347 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828037040 con 0x56324075c390 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.522+0000 7f2849c3b740 1 -- 172.21.15.39:0/2852899347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2828030450 con 0x56324075c390 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.522+0000 7f2822a00640 1 -- 172.21.15.39:0/2852899347 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f2828005050 con 0x56324075c390 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.522+0000 7f2822a00640 1 -- 172.21.15.39:0/2852899347 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2828061050 con 0x56324075c390 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.522+0000 7f2849c3b740 10 client.27211 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.522+0000 7f2849c3b740 1 -- 172.21.15.39:0/2852899347 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563240b402a0 con 0x56324075c390 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.523+0000 7f2822a00640 1 -- 172.21.15.39:0/2852899347 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2828030450 con 0x56324075c390 2024-07-04T08:22:25.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.523+0000 7f2822a00640 1 -- 172.21.15.39:0/2852899347 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2828053460 con 0x56324075c390 2024-07-04T08:22:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.523+0000 7f2849c3b740 10 client.27211 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.523+0000 7f2849c3b740 10 client.27211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.523+0000 7f2849c3b740 20 client.27211 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.523+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56324073b9f0 0x563240b43600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.523+0000 7f2849c3b740 4 client.27211 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.523+0000 7f2849c3b740 1 -- 172.21.15.39:0/2852899347 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563240a99df0 con 0x56324073b9f0 2024-07-04T08:22:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.523+0000 7f2840c00640 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56324073b9f0 0x563240b43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.524+0000 7f2840c00640 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56324073b9f0 0x563240b43600 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e75cb3cd50ec098 server_cookie=900acd512a322bb5 in_seq=0 out_seq=0 2024-07-04T08:22:25.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.524+0000 7f2822a00640 10 client.27211 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:25.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2822a00640 1 -- 172.21.15.39:0/2852899347 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563240a99df0 con 0x56324073b9f0 2024-07-04T08:22:25.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2822a00640 10 client.27211 handle_command_reply: tid=0 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 client.27211 shutdown 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 -- 172.21.15.39:0/2852899347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324075c390 msgr2=0x563240b31c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324075c390 0x563240b31c70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f282802a930 tx=0x7f28280035f0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 -- 172.21.15.39:0/2852899347 shutdown_connections 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56324073b9f0 0x563240b43600 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 0x563240b3d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324075c390 0x563240b31c70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2852899347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b31730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2822a00640 0 client.27211 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 -- 172.21.15.39:0/2852899347 >> 172.21.15.39:0/2852899347 conn(0x563240771460 msgr2=0x563240b23160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:25.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.525+0000 7f2849c3b740 1 -- 172.21.15.39:0/2852899347 shutdown_connections 2024-07-04T08:22:25.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.526+0000 7f2849c3b740 1 -- 172.21.15.39:0/2852899347 wait complete. 2024-07-04T08:22:25.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.526+0000 7f2849c3b740 20 client.27211 trim_cache size 0 max 16384 2024-07-04T08:22:25.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.546+0000 7f2849c3b740 1 Processor -- start 2024-07-04T08:22:25.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.546+0000 7f2849c3b740 1 -- start start 2024-07-04T08:22:25.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.546+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632407541b0 0x563240b33410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.546+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b22eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.546+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 0x563240b3d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.546+0000 7f2849c3b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1ae0f0 con 0x56324075c390 2024-07-04T08:22:25.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.546+0000 7f2849c3b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1afc30 con 0x56324077d070 2024-07-04T08:22:25.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.546+0000 7f2849c3b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1ae7c0 con 0x5632407541b0 2024-07-04T08:22:25.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2840c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632407541b0 0x563240b33410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2840c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632407541b0 0x563240b33410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58008/0 (socket says 172.21.15.39:58008) 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2840c00640 1 -- 172.21.15.39:0/408558218 learned_addr learned my addr 172.21.15.39:0/408558218 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2841600640 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 0x563240b3d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f283be00640 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b22eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2840c00640 1 -- 172.21.15.39:0/408558218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 msgr2=0x563240b3d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2840c00640 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 0x563240b3d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2840c00640 1 -- 172.21.15.39:0/408558218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 msgr2=0x563240b22eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2840c00640 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b22eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2840c00640 1 -- 172.21.15.39:0/408558218 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563240b24090 con 0x5632407541b0 2024-07-04T08:22:25.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f283be00640 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b22eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:22:25.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2841600640 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 0x563240b3d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:22:25.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.547+0000 7f2840c00640 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632407541b0 0x563240b33410 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f28300a9b90 tx=0x7f2830005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17abd1ac43f63b4c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:25.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.548+0000 7f282d600640 1 -- 172.21.15.39:0/408558218 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283008c070 con 0x5632407541b0 2024-07-04T08:22:25.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.548+0000 7f282d600640 1 -- 172.21.15.39:0/408558218 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28300a9710 con 0x5632407541b0 2024-07-04T08:22:25.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.548+0000 7f282d600640 1 -- 172.21.15.39:0/408558218 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300046c0 con 0x5632407541b0 2024-07-04T08:22:25.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 -- 172.21.15.39:0/408558218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632407541b0 msgr2=0x563240b33410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:25.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632407541b0 0x563240b33410 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f28300a9b90 tx=0x7f2830005880 comp rx=0 tx=0).stop 2024-07-04T08:22:25.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 -- 172.21.15.39:0/408558218 shutdown_connections 2024-07-04T08:22:25.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56324075c390 0x563240b22eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324077d070 0x563240b3d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 --2- 172.21.15.39:0/408558218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632407541b0 0x563240b33410 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 -- 172.21.15.39:0/408558218 >> 172.21.15.39:0/408558218 conn(0x563240771460 msgr2=0x563240b227a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 -- 172.21.15.39:0/408558218 shutdown_connections 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 -- 172.21.15.39:0/408558218 wait complete. 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.549+0000 7f2849c3b740 1 -- start start 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2849c3b740 1 Processor -- start 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b44490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324075c390 0x563240b449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2849c3b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 0x563240b44f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2849c3b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1af560 con 0x5632407541b0 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2849c3b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1aee90 con 0x56324075c390 2024-07-04T08:22:25.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2849c3b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1ac0d0 con 0x56324077d070 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2849c3b740 10 client.0 _ll_register_callbacks cb 0x7f2834009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2840c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b44490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2840c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b44490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51876/0 (socket says 172.21.15.39:51876) 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.550+0000 7f2840c00640 1 -- 172.21.15.39:0/2559184837 learned_addr learned my addr 172.21.15.39:0/2559184837 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.551+0000 7f2841600640 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 0x563240b44f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.551+0000 7f283be00640 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324075c390 0x563240b449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.551+0000 7f2840c00640 1 -- 172.21.15.39:0/2559184837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 msgr2=0x563240b44f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.551+0000 7f2840c00640 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 0x563240b44f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.551+0000 7f2840c00640 1 -- 172.21.15.39:0/2559184837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324075c390 msgr2=0x563240b449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:25.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.551+0000 7f2840c00640 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324075c390 0x563240b449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.551+0000 7f2840c00640 1 -- 172.21.15.39:0/2559184837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563240baab50 con 0x5632407541b0 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.551+0000 7f2840c00640 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b44490 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f283000c6a0 tx=0x7f2830009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4abeabe52d46f25 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.551+0000 7f2822a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.552+0000 7f2822a00640 1 -- 172.21.15.39:0/2559184837 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28300c8030 con 0x5632407541b0 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.552+0000 7f2822a00640 1 -- 172.21.15.39:0/2559184837 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f28300c4290 con 0x5632407541b0 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.552+0000 7f2849c3b740 1 -- 172.21.15.39:0/2559184837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28300a9710 con 0x5632407541b0 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.552+0000 7f2822a00640 1 -- 172.21.15.39:0/2559184837 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f283008c040 con 0x5632407541b0 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.552+0000 7f2849c3b740 10 client.17160 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.552+0000 7f2849c3b740 1 -- 172.21.15.39:0/2559184837 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563240babcc0 con 0x5632407541b0 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.552+0000 7f2822a00640 1 -- 172.21.15.39:0/2559184837 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28300a9710 con 0x5632407541b0 2024-07-04T08:22:25.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.553+0000 7f2822a00640 1 -- 172.21.15.39:0/2559184837 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f28300a5570 con 0x5632407541b0 2024-07-04T08:22:25.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.553+0000 7f2849c3b740 10 client.17160 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:25.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.553+0000 7f2849c3b740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:25.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.553+0000 7f2849c3b740 20 client.17160 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:25.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.553+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563240710da0 0x563240bae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:25.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.553+0000 7f2849c3b740 4 client.17160 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:22:25.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.553+0000 7f2849c3b740 1 -- 172.21.15.39:0/2559184837 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563240a95890 con 0x563240710da0 2024-07-04T08:22:25.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.553+0000 7f283be00640 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563240710da0 0x563240bae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:25.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f283be00640 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563240710da0 0x563240bae260 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d898b036be528390 server_cookie=3f5b1e9370fd156c in_seq=0 out_seq=0 2024-07-04T08:22:25.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2822a00640 10 client.17160 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2822a00640 1 -- 172.21.15.39:0/2559184837 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563240a95890 con 0x563240710da0 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2822a00640 10 client.17160 handle_command_reply: tid=0 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2849c3b740 1 client.17160 shutdown 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2849c3b740 1 -- 172.21.15.39:0/2559184837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 msgr2=0x563240b44490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b44490 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f283000c6a0 tx=0x7f2830009f90 comp rx=0 tx=0).stop 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2849c3b740 1 -- 172.21.15.39:0/2559184837 shutdown_connections 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563240710da0 0x563240bae260 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56324077d070 0x563240b44f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56324075c390 0x563240b449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2849c3b740 1 --2- 172.21.15.39:0/2559184837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5632407541b0 0x563240b44490 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.554+0000 7f2849c3b740 1 -- 172.21.15.39:0/2559184837 >> 172.21.15.39:0/2559184837 conn(0x563240771460 msgr2=0x563240b1e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:25.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.555+0000 7f2822a00640 0 client.17160 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:25.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.555+0000 7f2849c3b740 1 -- 172.21.15.39:0/2559184837 shutdown_connections 2024-07-04T08:22:25.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.555+0000 7f2849c3b740 1 -- 172.21.15.39:0/2559184837 wait complete. 2024-07-04T08:22:25.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.555+0000 7f2849c3b740 20 client.17160 trim_cache size 0 max 16384 2024-07-04T08:22:25.555 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:22:25.556 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:22:25.556 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:22:25.556 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:22:25.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.555+0000 7f2843600640 1 -- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28240b2870 msgr2=0x7f28240b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:25.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.555+0000 7f2843600640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28240b2870 0x7f28240b4d80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2828006010 tx=0x7f2828000bc0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.555+0000 7f2843600640 1 -- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f283c101fb0 msgr2=0x7f283c199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:25.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.555+0000 7f2843600640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f283c101fb0 0x7f283c199990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f283000e9c0 tx=0x7f283000ee80 comp rx=0 tx=0).stop 2024-07-04T08:22:25.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.556+0000 7f2843600640 1 -- 172.21.15.39:0/189096730 shutdown_connections 2024-07-04T08:22:25.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.556+0000 7f2843600640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28240b2870 0x7f28240b4d80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.556+0000 7f2843600640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f283c108240 0x7f283c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.556+0000 7f2843600640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f283c1078f0 0x7f283c199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.556+0000 7f2843600640 1 --2- 172.21.15.39:0/189096730 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f283c101fb0 0x7f283c199990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:25.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.556+0000 7f2843600640 1 -- 172.21.15.39:0/189096730 >> 172.21.15.39:0/189096730 conn(0x7f283c078c80 msgr2=0x7f283c06a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:25.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.556+0000 7f2843600640 1 -- 172.21.15.39:0/189096730 shutdown_connections 2024-07-04T08:22:25.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:25.556+0000 7f2843600640 1 -- 172.21.15.39:0/189096730 wait complete. 2024-07-04T08:22:26.439 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:22:26.440 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:22:26.922 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:27.303 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:27 smithi039 ceph-mon[32062]: pgmap v699: 129 pgs: 129 active+clean; 8.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-07-04T08:22:27.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:27 smithi133 ceph-mon[37050]: pgmap v699: 129 pgs: 129 active+clean; 8.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-07-04T08:22:27.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:27 smithi050 ceph-mon[36577]: pgmap v699: 129 pgs: 129 active+clean; 8.3 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 17 op/s 2024-07-04T08:22:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.242+0000 7fbb3c000640 1 -- 172.21.15.39:0/2270941224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3410b590 msgr2=0x7fbb34111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.242+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2270941224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3410b590 0x7fbb34111dc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009ca0 tx=0x7fbb2c02a8b0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.242+0000 7fbb3c000640 1 -- 172.21.15.39:0/2270941224 shutdown_connections 2024-07-04T08:22:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.242+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2270941224 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3410b590 0x7fbb34111dc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.242+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2270941224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb3410ac40 0x7fbb3410b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.242+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2270941224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb34105300 0x7fbb34105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.242+0000 7fbb3c000640 1 -- 172.21.15.39:0/2270941224 >> 172.21.15.39:0/2270941224 conn(0x7fbb34100d30 msgr2=0x7fbb34103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:29.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.242+0000 7fbb3c000640 1 -- 172.21.15.39:0/2270941224 shutdown_connections 2024-07-04T08:22:29.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.242+0000 7fbb3c000640 1 -- 172.21.15.39:0/2270941224 wait complete. 2024-07-04T08:22:29.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.244+0000 7fbb3c000640 1 Processor -- start 2024-07-04T08:22:29.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.244+0000 7fbb3c000640 1 -- start start 2024-07-04T08:22:29.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb3c000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb34105300 0x7fbb3419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:29.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb3c000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb3410ac40 0x7fbb3419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:29.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb3c000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3410b590 0x7fbb341a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:29.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb3c000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34115290 con 0x7fbb3410b590 2024-07-04T08:22:29.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb3c000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34115110 con 0x7fbb34105300 2024-07-04T08:22:29.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb3c000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbb34115410 con 0x7fbb3410ac40 2024-07-04T08:22:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb39600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb34105300 0x7fbb3419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb39600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb34105300 0x7fbb3419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50888/0 (socket says 172.21.15.39:50888) 2024-07-04T08:22:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb39600640 1 -- 172.21.15.39:0/2672289142 learned_addr learned my addr 172.21.15.39:0/2672289142 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb38c00640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb3410ac40 0x7fbb3419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:29.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb3a000640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3410b590 0x7fbb341a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb39600640 1 -- 172.21.15.39:0/2672289142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb3410ac40 msgr2=0x7fbb3419c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb39600640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb3410ac40 0x7fbb3419c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb39600640 1 -- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3410b590 msgr2=0x7fbb341a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.245+0000 7fbb39600640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3410b590 0x7fbb341a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.246+0000 7fbb39600640 1 -- 172.21.15.39:0/2672289142 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb341a1b50 con 0x7fbb34105300 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.246+0000 7fbb39600640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb34105300 0x7fbb3419bbe0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800e9f0 tx=0x7fbb2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.246+0000 7fbb32000640 1 -- 172.21.15.39:0/2672289142 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800cd80 con 0x7fbb34105300 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.246+0000 7fbb32000640 1 -- 172.21.15.39:0/2672289142 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb280108e0 con 0x7fbb34105300 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.246+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb341a1e40 con 0x7fbb34105300 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.246+0000 7fbb32000640 1 -- 172.21.15.39:0/2672289142 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28018980 con 0x7fbb34105300 2024-07-04T08:22:29.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.246+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb341a85a0 con 0x7fbb34105300 2024-07-04T08:22:29.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.248+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb34105710 con 0x7fbb34105300 2024-07-04T08:22:29.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.250+0000 7fbb32000640 1 -- 172.21.15.39:0/2672289142 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbb28010430 con 0x7fbb34105300 2024-07-04T08:22:29.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.251+0000 7fbb32000640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbb1c0b29e0 0x7fbb1c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:29.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.252+0000 7fbb32000640 1 -- 172.21.15.39:0/2672289142 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7fbb280e2f80 con 0x7fbb34105300 2024-07-04T08:22:29.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.252+0000 7fbb38c00640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbb1c0b29e0 0x7fbb1c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:29.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.253+0000 7fbb38c00640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbb1c0b29e0 0x7fbb1c0b4ef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbb34104430 tx=0x7fbb20005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:29.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.253+0000 7fbb32000640 1 -- 172.21.15.39:0/2672289142 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fbb280aa5a0 con 0x7fbb34105300 2024-07-04T08:22:29.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:29 smithi133 ceph-mon[37050]: pgmap v700: 129 pgs: 129 active+clean; 8.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-07-04T08:22:29.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:29 smithi039 ceph-mon[32062]: pgmap v700: 129 pgs: 129 active+clean; 8.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-07-04T08:22:29.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:29 smithi050 ceph-mon[36577]: pgmap v700: 129 pgs: 129 active+clean; 8.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-07-04T08:22:29.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.506+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb3402de10 con 0x7fbb34105300 2024-07-04T08:22:29.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.507+0000 7fbb32000640 1 -- 172.21.15.39:0/2672289142 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fbb280a9ea0 con 0x7fbb34105300 2024-07-04T08:22:29.502 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:22:29.502 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:22:29.502 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:22:29.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbb1c0b29e0 msgr2=0x7fbb1c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:29.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbb1c0b29e0 0x7fbb1c0b4ef0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbb34104430 tx=0x7fbb20005f90 comp rx=0 tx=0).stop 2024-07-04T08:22:29.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb34105300 msgr2=0x7fbb3419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:29.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb34105300 0x7fbb3419bbe0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800e9f0 tx=0x7fbb2800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 shutdown_connections 2024-07-04T08:22:29.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbb1c0b29e0 0x7fbb1c0b4ef0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3410b590 0x7fbb341a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbb3410ac40 0x7fbb3419c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 --2- 172.21.15.39:0/2672289142 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb34105300 0x7fbb3419bbe0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:29.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.510+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 >> 172.21.15.39:0/2672289142 conn(0x7fbb34100d30 msgr2=0x7fbb341091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:29.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.511+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 shutdown_connections 2024-07-04T08:22:29.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:29.511+0000 7fbb3c000640 1 -- 172.21.15.39:0/2672289142 wait complete. 2024-07-04T08:22:30.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:30 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2672289142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:22:30.321 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:30 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2672289142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:22:30.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:22:30.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:30 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2672289142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:22:30.865 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:31.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:31 smithi133 ceph-mon[37050]: pgmap v701: 129 pgs: 129 active+clean; 8.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-07-04T08:22:31.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:31 smithi039 ceph-mon[32062]: pgmap v701: 129 pgs: 129 active+clean; 8.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-07-04T08:22:31.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:31 smithi050 ceph-mon[36577]: pgmap v701: 129 pgs: 129 active+clean; 8.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-07-04T08:22:33.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.073+0000 7f400da00640 1 -- 172.21.15.39:0/2493773893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f400810d360 msgr2=0x7f400810f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:33.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.073+0000 7f400da00640 1 --2- 172.21.15.39:0/2493773893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f400810d360 0x7f400810f850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4000009ca0 tx=0x7f400002a8b0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.073+0000 7f400da00640 1 -- 172.21.15.39:0/2493773893 shutdown_connections 2024-07-04T08:22:33.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.073+0000 7f400da00640 1 --2- 172.21.15.39:0/2493773893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f400810d360 0x7f400810f850 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.073+0000 7f400da00640 1 --2- 172.21.15.39:0/2493773893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008104850 0x7f400810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.073+0000 7f400da00640 1 --2- 172.21.15.39:0/2493773893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4008103f00 0x7f4008104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.073+0000 7f400da00640 1 -- 172.21.15.39:0/2493773893 >> 172.21.15.39:0/2493773893 conn(0x7f4008078c80 msgr2=0x7f4008079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:33.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.074+0000 7f400da00640 1 -- 172.21.15.39:0/2493773893 shutdown_connections 2024-07-04T08:22:33.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.074+0000 7f400da00640 1 -- 172.21.15.39:0/2493773893 wait complete. 2024-07-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.075+0000 7f400da00640 1 Processor -- start 2024-07-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.075+0000 7f400da00640 1 -- start start 2024-07-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f400da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4008103f00 0x7f400819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f400da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008104850 0x7f400819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f400da00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400810d360 0x7f40081a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f400da00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4008112fc0 con 0x7f400810d360 2024-07-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f400da00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4008112e40 con 0x7f4008103f00 2024-07-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f400da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4008113140 con 0x7f4008104850 2024-07-04T08:22:33.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f4006a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4008103f00 0x7f400819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f4006000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008104850 0x7f400819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f4006000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008104850 0x7f400819c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:48760/0 (socket says 172.21.15.39:48760) 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f4006a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4008103f00 0x7f400819bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47222/0 (socket says 172.21.15.39:47222) 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f4006000640 1 -- 172.21.15.39:0/838842930 learned_addr learned my addr 172.21.15.39:0/838842930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f4007400640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400810d360 0x7f40081a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.076+0000 7f4006000640 1 -- 172.21.15.39:0/838842930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4008103f00 msgr2=0x7f400819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f4006000640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4008103f00 0x7f400819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f4006000640 1 -- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400810d360 msgr2=0x7f40081a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f4006000640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400810d360 0x7f40081a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f4006000640 1 -- 172.21.15.39:0/838842930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40081a1b20 con 0x7f4008104850 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f4006000640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008104850 0x7f400819c150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000e970 tx=0x7f3ff000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f3fff400640 1 -- 172.21.15.39:0/838842930 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000cd80 con 0x7f4008104850 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f3fff400640 1 -- 172.21.15.39:0/838842930 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff00035f0 con 0x7f4008104850 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f400da00640 1 -- 172.21.15.39:0/838842930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40081a1e10 con 0x7f4008104850 2024-07-04T08:22:33.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f3fff400640 1 -- 172.21.15.39:0/838842930 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0010690 con 0x7f4008104850 2024-07-04T08:22:33.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.077+0000 7f400da00640 1 -- 172.21.15.39:0/838842930 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40081a8570 con 0x7f4008104850 2024-07-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.081+0000 7f3fff400640 1 -- 172.21.15.39:0/838842930 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3ff0003aa0 con 0x7f4008104850 2024-07-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.081+0000 7f40140d4740 1 Processor -- start 2024-07-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.081+0000 7f40140d4740 1 -- start start 2024-07-04T08:22:33.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f40140d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f40140d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdc5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f40140d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa00390 0x56165fdc6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f40140d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56165fd3bf00 con 0x56165f9f81b0 2024-07-04T08:22:33.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f40140d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56165fd2b4a0 con 0x56165fa21070 2024-07-04T08:22:33.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f40140d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56165fd2f4e0 con 0x56165fa00390 2024-07-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f4007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f4007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47238/0 (socket says 172.21.15.39:47238) 2024-07-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f4007400640 1 -- 172.21.15.39:0/3210757460 learned_addr learned my addr 172.21.15.39:0/3210757460 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f4006a00640 1 --2- 172.21.15.39:0/3210757460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdc5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.082+0000 7f4006000640 1 --2- 172.21.15.39:0/3210757460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa00390 0x56165fdc6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.083+0000 7f4007400640 1 -- 172.21.15.39:0/3210757460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa00390 msgr2=0x56165fdc6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.083+0000 7f4007400640 1 --2- 172.21.15.39:0/3210757460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa00390 0x56165fdc6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.083+0000 7f4007400640 1 -- 172.21.15.39:0/3210757460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 msgr2=0x56165fdc5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.083+0000 7f4007400640 1 --2- 172.21.15.39:0/3210757460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdc5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.083+0000 7f4007400640 1 -- 172.21.15.39:0/3210757460 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165fcc3670 con 0x56165fa21070 2024-07-04T08:22:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.083+0000 7f4007400640 1 --2- 172.21.15.39:0/3210757460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc57d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4000009800 tx=0x7f4000009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63dd1b3802d934de server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.084+0000 7f3ff7e00640 1 -- 172.21.15.39:0/3210757460 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40000305a0 con 0x56165fa21070 2024-07-04T08:22:33.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.084+0000 7f3fff400640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3fec0b29e0 0x7f3fec0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.084+0000 7f3ff7e00640 1 -- 172.21.15.39:0/3210757460 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4000036030 con 0x56165fa21070 2024-07-04T08:22:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.084+0000 7f3ff7e00640 1 -- 172.21.15.39:0/3210757460 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40000042a0 con 0x56165fa21070 2024-07-04T08:22:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.084+0000 7f3fff400640 1 -- 172.21.15.39:0/838842930 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f3ff00e2bb0 con 0x7f4008104850 2024-07-04T08:22:33.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.084+0000 7f4006a00640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3fec0b29e0 0x7f3fec0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f40140d4740 1 -- 172.21.15.39:0/3210757460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 msgr2=0x56165fdc57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f4006a00640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3fec0b29e0 0x7f3fec0b4ef0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x56165fdc4450 tx=0x7f3ff8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f40140d4740 1 --2- 172.21.15.39:0/3210757460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc57d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4000009800 tx=0x7f4000009830 comp rx=0 tx=0).stop 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f40140d4740 1 -- 172.21.15.39:0/3210757460 shutdown_connections 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f40140d4740 1 --2- 172.21.15.39:0/3210757460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdc5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f40140d4740 1 --2- 172.21.15.39:0/3210757460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa00390 0x56165fdc6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f40140d4740 1 --2- 172.21.15.39:0/3210757460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc57d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f40140d4740 1 -- 172.21.15.39:0/3210757460 >> 172.21.15.39:0/3210757460 conn(0x56165fa15460 msgr2=0x56165fdc36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f40140d4740 1 -- 172.21.15.39:0/3210757460 shutdown_connections 2024-07-04T08:22:33.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.085+0000 7f40140d4740 1 -- 172.21.15.39:0/3210757460 wait complete. 2024-07-04T08:22:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.086+0000 7f40140d4740 1 -- start start 2024-07-04T08:22:33.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.086+0000 7f40140d4740 1 Processor -- start 2024-07-04T08:22:33.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f40140d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdd1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f40140d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fdd1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f40140d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 0x56165fddd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f40140d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56165fb8f700 con 0x56165f9f81b0 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f40140d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56165fcd2bd0 con 0x56165fa00390 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f40140d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56165fdd3f50 con 0x56165fa21070 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f40140d4740 10 client.0 _ll_register_callbacks cb 0x56165fd24590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f4006a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdd1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f4006000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fdd1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f4006000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fdd1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47244/0 (socket says 172.21.15.39:47244) 2024-07-04T08:22:33.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f4006000640 1 -- 172.21.15.39:0/3692225936 learned_addr learned my addr 172.21.15.39:0/3692225936 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f4006a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdd1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40466/0 (socket says 172.21.15.39:40466) 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.087+0000 7f4007400640 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 0x56165fddd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f4006000640 1 -- 172.21.15.39:0/3692225936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 msgr2=0x56165fddd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f4006000640 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 0x56165fddd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f4006000640 1 -- 172.21.15.39:0/3692225936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 msgr2=0x56165fdd1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f4006000640 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdd1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f4006000640 1 -- 172.21.15.39:0/3692225936 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165fa8c650 con 0x56165fa00390 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f4007400640 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 0x56165fddd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f4006000640 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fdd1990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00a9a60 tx=0x7f3ff00a94a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b381d09051430097 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f3ff4c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3692225936 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00e03d0 con 0x56165fa00390 2024-07-04T08:22:33.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f40140d4740 1 -- 172.21.15.39:0/3692225936 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4000036030 con 0x56165fa00390 2024-07-04T08:22:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.088+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3692225936 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f3ff00a60c0 con 0x56165fa00390 2024-07-04T08:22:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3692225936 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ff00aa1d0 con 0x56165fa00390 2024-07-04T08:22:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f40140d4740 10 client.27314 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:33.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f40140d4740 1 -- 172.21.15.39:0/3692225936 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56165fddff90 con 0x56165fa00390 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3692225936 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4000036030 con 0x56165fa00390 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3692225936 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3ff00b24a0 con 0x56165fa00390 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f40140d4740 10 client.27314 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f40140d4740 10 client.27314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f40140d4740 20 client.27314 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f40140d4740 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56165f9df9f0 0x56165fde3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f40140d4740 4 client.27314 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.089+0000 7f40140d4740 1 -- 172.21.15.39:0/3692225936 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56165fd3de50 con 0x56165f9df9f0 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.090+0000 7f4006a00640 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56165f9df9f0 0x56165fde3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.091+0000 7f4006a00640 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56165f9df9f0 0x56165fde3450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7007f64a75570c32 server_cookie=2837d0e5b0087f64 in_seq=0 out_seq=0 2024-07-04T08:22:33.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.091+0000 7f3ff4c00640 10 client.27314 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.091+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3692225936 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56165fd3de50 con 0x56165f9df9f0 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.091+0000 7f3ff4c00640 10 client.27314 handle_command_reply: tid=0 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.091+0000 7f40140d4740 1 client.27314 shutdown 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 -- 172.21.15.39:0/3692225936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 msgr2=0x56165fdd1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fdd1990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00a9a60 tx=0x7f3ff00a94a0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 -- 172.21.15.39:0/3692225936 shutdown_connections 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56165f9df9f0 0x56165fde3450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 0x56165fddd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fdd1990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 --2- 172.21.15.39:0/3692225936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdd1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f3ff4c00640 0 client.27314 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:22:33.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 -- 172.21.15.39:0/3692225936 >> 172.21.15.39:0/3692225936 conn(0x56165fa15460 msgr2=0x56165fdc8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:33.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 -- 172.21.15.39:0/3692225936 shutdown_connections 2024-07-04T08:22:33.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 1 -- 172.21.15.39:0/3692225936 wait complete. 2024-07-04T08:22:33.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.092+0000 7f40140d4740 20 client.27314 trim_cache size 0 max 16384 2024-07-04T08:22:33.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.108+0000 7f40140d4740 1 Processor -- start 2024-07-04T08:22:33.102 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:33 smithi039 ceph-mon[32062]: pgmap v702: 129 pgs: 129 active+clean; 8.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-07-04T08:22:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.108+0000 7f40140d4740 1 -- start start 2024-07-04T08:22:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f40140d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdc2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f40140d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fddd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f40140d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 0x56165fdd2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f40140d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40081b25d0 con 0x56165f9f81b0 2024-07-04T08:22:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f40140d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40081b4110 con 0x56165fa00390 2024-07-04T08:22:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f40140d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40081b2ca0 con 0x56165fa21070 2024-07-04T08:22:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f4006a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdc2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f4006a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdc2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40474/0 (socket says 172.21.15.39:40474) 2024-07-04T08:22:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f4006a00640 1 -- 172.21.15.39:0/3552063163 learned_addr learned my addr 172.21.15.39:0/3552063163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f4006000640 1 --2- 172.21.15.39:0/3552063163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fddd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.109+0000 7f4007400640 1 --2- 172.21.15.39:0/3552063163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 0x56165fdd2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.110+0000 7f4006000640 1 -- 172.21.15.39:0/3552063163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 msgr2=0x56165fdd2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.110+0000 7f4006000640 1 --2- 172.21.15.39:0/3552063163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 0x56165fdd2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.110+0000 7f4006000640 1 -- 172.21.15.39:0/3552063163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 msgr2=0x56165fdc2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.110+0000 7f4006000640 1 --2- 172.21.15.39:0/3552063163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdc2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.110+0000 7f4006000640 1 -- 172.21.15.39:0/3552063163 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165fdc9850 con 0x56165fa00390 2024-07-04T08:22:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.110+0000 7f4006000640 1 --2- 172.21.15.39:0/3552063163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fddd060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00a97d0 tx=0x7f3ff00094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d26b8553dbccce server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.110+0000 7f3ff7400640 1 -- 172.21.15.39:0/3552063163 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00e03d0 con 0x56165fa00390 2024-07-04T08:22:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.110+0000 7f3ff7400640 1 -- 172.21.15.39:0/3552063163 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff00c4120 con 0x56165fa00390 2024-07-04T08:22:33.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.110+0000 7f3ff7400640 1 -- 172.21.15.39:0/3552063163 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00ade80 con 0x56165fa00390 2024-07-04T08:22:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.111+0000 7f40140d4740 1 -- 172.21.15.39:0/3552063163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 msgr2=0x56165fddd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.111+0000 7f40140d4740 1 --2- 172.21.15.39:0/3552063163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fddd060 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00a97d0 tx=0x7f3ff00094b0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.111+0000 7f40140d4740 1 -- 172.21.15.39:0/3552063163 shutdown_connections 2024-07-04T08:22:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.111+0000 7f40140d4740 1 --2- 172.21.15.39:0/3552063163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165fa21070 0x56165fdd2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.111+0000 7f40140d4740 1 --2- 172.21.15.39:0/3552063163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa00390 0x56165fddd060 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.111+0000 7f40140d4740 1 --2- 172.21.15.39:0/3552063163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165f9f81b0 0x56165fdc2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.111+0000 7f40140d4740 1 -- 172.21.15.39:0/3552063163 >> 172.21.15.39:0/3552063163 conn(0x56165fa15460 msgr2=0x56165fdd2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.112+0000 7f40140d4740 1 -- 172.21.15.39:0/3552063163 shutdown_connections 2024-07-04T08:22:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.112+0000 7f40140d4740 1 -- 172.21.15.39:0/3552063163 wait complete. 2024-07-04T08:22:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.112+0000 7f40140d4740 1 -- start start 2024-07-04T08:22:33.107 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.113+0000 7f40140d4740 1 Processor -- start 2024-07-04T08:22:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.113+0000 7f40140d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165f9f81b0 0x56165fdd9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.113+0000 7f40140d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165fa00390 0x56165fdc26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.113+0000 7f40140d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.113+0000 7f40140d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40081b3a40 con 0x56165fa00390 2024-07-04T08:22:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.113+0000 7f40140d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40081b3370 con 0x56165fa21070 2024-07-04T08:22:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.113+0000 7f40140d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40081b05b0 con 0x56165f9f81b0 2024-07-04T08:22:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.113+0000 7f40140d4740 10 client.0 _ll_register_callbacks cb 0x7f3ff00a98a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:33.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4006000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165fa00390 0x56165fdc26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4006000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165fa00390 0x56165fdc26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40488/0 (socket says 172.21.15.39:40488) 2024-07-04T08:22:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4006000640 1 -- 172.21.15.39:0/3010007560 learned_addr learned my addr 172.21.15.39:0/3010007560 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4006a00640 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165f9f81b0 0x56165fdd9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4007400640 1 -- 172.21.15.39:0/3010007560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165f9f81b0 msgr2=0x56165fdd9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4007400640 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165f9f81b0 0x56165fdd9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4007400640 1 -- 172.21.15.39:0/3010007560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165fa00390 msgr2=0x56165fdc26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4007400640 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165fa00390 0x56165fdc26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4007400640 1 -- 172.21.15.39:0/3010007560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165fdc3140 con 0x56165fa21070 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4006000640 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165fa00390 0x56165fdc26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.114+0000 7f4006a00640 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165f9f81b0 0x56165fdd9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.115+0000 7f4007400640 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc2c00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4000006010 tx=0x7f4000005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10777a21e609065f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.115+0000 7f3ff4c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.115+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3010007560 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000030a10 con 0x56165fa21070 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.115+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3010007560 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f4000030bb0 con 0x56165fa21070 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.115+0000 7f40140d4740 1 -- 172.21.15.39:0/3010007560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56165fd398f0 con 0x56165fa21070 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.115+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3010007560 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4000003970 con 0x56165fa21070 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f40140d4740 10 client.27320 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f40140d4740 1 -- 172.21.15.39:0/3010007560 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56165fdd7e50 con 0x56165fa21070 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3010007560 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56165fd398f0 con 0x56165fa21070 2024-07-04T08:22:33.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3010007560 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4000043070 con 0x56165fa21070 2024-07-04T08:22:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f40140d4740 10 client.27320 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f40140d4740 10 client.27320 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:22:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f40140d4740 20 client.27320 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f40140d4740 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56165f9b4da0 0x56165fe564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f40140d4740 4 client.27320 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.116+0000 7f40140d4740 1 -- 172.21.15.39:0/3010007560 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56165fd37030 con 0x56165f9b4da0 2024-07-04T08:22:33.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.117+0000 7f4006a00640 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56165f9b4da0 0x56165fe564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.117+0000 7f4006a00640 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56165f9b4da0 0x56165fe564f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd7ecb16a8974fcf server_cookie=7ad40427ff6b71cc in_seq=0 out_seq=0 2024-07-04T08:22:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.117+0000 7f3ff4c00640 10 client.27320 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:22:33.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f3ff4c00640 1 -- 172.21.15.39:0/3010007560 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56165fd37030 con 0x56165f9b4da0 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f3ff4c00640 10 client.27320 handle_command_reply: tid=0 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 client.27320 shutdown 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 -- 172.21.15.39:0/3010007560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 msgr2=0x56165fdc2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc2c00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4000006010 tx=0x7f4000005c80 comp rx=0 tx=0).stop 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 -- 172.21.15.39:0/3010007560 shutdown_connections 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56165f9b4da0 0x56165fe564f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56165fa00390 0x56165fdc26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56165fa21070 0x56165fdc2c00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 --2- 172.21.15.39:0/3010007560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56165f9f81b0 0x56165fdd9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 -- 172.21.15.39:0/3010007560 >> 172.21.15.39:0/3010007560 conn(0x56165fa15460 msgr2=0x56165fdc6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:33.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f3ff4c00640 0 client.27320 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:22:33.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 -- 172.21.15.39:0/3010007560 shutdown_connections 2024-07-04T08:22:33.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.118+0000 7f40140d4740 1 -- 172.21.15.39:0/3010007560 wait complete. 2024-07-04T08:22:33.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.119+0000 7f40140d4740 20 client.27320 trim_cache size 0 max 16384 2024-07-04T08:22:33.115 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.119+0000 7f400da00640 1 -- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3fec0b29e0 msgr2=0x7f3fec0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.119+0000 7f400da00640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3fec0b29e0 0x7f3fec0b4ef0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x56165fdc4450 tx=0x7f3ff8005f90 comp rx=0 tx=0).stop 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.119+0000 7f400da00640 1 -- 172.21.15.39:0/838842930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008104850 msgr2=0x7f400819c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.119+0000 7f400da00640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008104850 0x7f400819c150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000e970 tx=0x7f3ff000ee30 comp rx=0 tx=0).stop 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.120+0000 7f400da00640 1 -- 172.21.15.39:0/838842930 shutdown_connections 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.120+0000 7f400da00640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3fec0b29e0 0x7f3fec0b4ef0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.120+0000 7f400da00640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f400810d360 0x7f40081a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.120+0000 7f400da00640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008104850 0x7f400819c150 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.120+0000 7f400da00640 1 --2- 172.21.15.39:0/838842930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4008103f00 0x7f400819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:33.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.120+0000 7f400da00640 1 -- 172.21.15.39:0/838842930 >> 172.21.15.39:0/838842930 conn(0x7f4008078c80 msgr2=0x7f400810c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:33.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.120+0000 7f400da00640 1 -- 172.21.15.39:0/838842930 shutdown_connections 2024-07-04T08:22:33.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:33.120+0000 7f400da00640 1 -- 172.21.15.39:0/838842930 wait complete. 2024-07-04T08:22:33.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:33 smithi133 ceph-mon[37050]: pgmap v702: 129 pgs: 129 active+clean; 8.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-07-04T08:22:33.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:33 smithi050 ceph-mon[36577]: pgmap v702: 129 pgs: 129 active+clean; 8.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-07-04T08:22:34.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:22:34.513 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:35.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:35 smithi133 ceph-mon[37050]: pgmap v703: 129 pgs: 129 active+clean; 8.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-07-04T08:22:35.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:35 smithi039 ceph-mon[32062]: pgmap v703: 129 pgs: 129 active+clean; 8.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-07-04T08:22:35.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:35 smithi050 ceph-mon[36577]: pgmap v703: 129 pgs: 129 active+clean; 8.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-07-04T08:22:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.900+0000 7f6fef200640 1 -- 172.21.15.39:0/2264407695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff010f4a0 msgr2=0x7f6ff01118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.900+0000 7f6fef200640 1 --2- 172.21.15.39:0/2264407695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff010f4a0 0x7f6ff01118c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009be0 tx=0x7f6fe802a6d0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.901+0000 7f6fef200640 1 -- 172.21.15.39:0/2264407695 shutdown_connections 2024-07-04T08:22:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.901+0000 7f6fef200640 1 --2- 172.21.15.39:0/2264407695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff010f4a0 0x7f6ff01118c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.901+0000 7f6fef200640 1 --2- 172.21.15.39:0/2264407695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff00755f0 0x7f6ff010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.901+0000 7f6fef200640 1 --2- 172.21.15.39:0/2264407695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff0074ca0 0x7f6ff00750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.901+0000 7f6fef200640 1 -- 172.21.15.39:0/2264407695 >> 172.21.15.39:0/2264407695 conn(0x7f6ff00fe930 msgr2=0x7f6ff0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.901+0000 7f6fef200640 1 -- 172.21.15.39:0/2264407695 shutdown_connections 2024-07-04T08:22:36.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.901+0000 7f6fef200640 1 -- 172.21.15.39:0/2264407695 wait complete. 2024-07-04T08:22:36.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.902+0000 7f6fef200640 1 Processor -- start 2024-07-04T08:22:36.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.902+0000 7f6fef200640 1 -- start start 2024-07-04T08:22:36.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.902+0000 7f6fef200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0074ca0 0x7f6ff019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.902+0000 7f6fef200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff00755f0 0x7f6ff019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.902+0000 7f6fef200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff010f4a0 0x7f6ff01a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.902+0000 7f6fef200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0115000 con 0x7f6ff00755f0 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.902+0000 7f6fef200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0114e80 con 0x7f6ff010f4a0 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.902+0000 7f6fef200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0115180 con 0x7f6ff0074ca0 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fe5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff00755f0 0x7f6ff019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fede00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0074ca0 0x7f6ff019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fee800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff010f4a0 0x7f6ff01a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fede00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0074ca0 0x7f6ff019ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:48810/0 (socket says 172.21.15.39:48810) 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fede00640 1 -- 172.21.15.39:0/2092665690 learned_addr learned my addr 172.21.15.39:0/2092665690 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fee800640 1 -- 172.21.15.39:0/2092665690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0074ca0 msgr2=0x7f6ff019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fee800640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0074ca0 0x7f6ff019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fee800640 1 -- 172.21.15.39:0/2092665690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff00755f0 msgr2=0x7f6ff019bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:36.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fee800640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff00755f0 0x7f6ff019bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.903+0000 7f6fee800640 1 -- 172.21.15.39:0/2092665690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff01a1980 con 0x7f6ff010f4a0 2024-07-04T08:22:36.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.904+0000 7f6fee800640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff010f4a0 0x7f6ff01a1250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009b30 tx=0x7f6fe8003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:36.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.904+0000 7f6fe7400640 1 -- 172.21.15.39:0/2092665690 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8041020 con 0x7f6ff010f4a0 2024-07-04T08:22:36.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.904+0000 7f6fef200640 1 -- 172.21.15.39:0/2092665690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff01a1c10 con 0x7f6ff010f4a0 2024-07-04T08:22:36.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.904+0000 7f6fe7400640 1 -- 172.21.15.39:0/2092665690 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe80318f0 con 0x7f6ff010f4a0 2024-07-04T08:22:36.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.904+0000 7f6fe7400640 1 -- 172.21.15.39:0/2092665690 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8036040 con 0x7f6ff010f4a0 2024-07-04T08:22:36.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.904+0000 7f6fef200640 1 -- 172.21.15.39:0/2092665690 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff01a83d0 con 0x7f6ff010f4a0 2024-07-04T08:22:36.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.908+0000 7f6ffb7b3740 1 Processor -- start 2024-07-04T08:22:36.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.908+0000 7f6fe7400640 1 -- 172.21.15.39:0/2092665690 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6fe8044020 con 0x7f6ff010f4a0 2024-07-04T08:22:36.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6ffb7b3740 1 -- start start 2024-07-04T08:22:36.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff15970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb481b0 0x55706ff15eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff163f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55706fe8bf00 con 0x55706fb71070 2024-07-04T08:22:36.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55706fe7b4a0 con 0x55706fb50390 2024-07-04T08:22:36.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55706fe7f4e0 con 0x55706fb481b0 2024-07-04T08:22:36.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6fe5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff15970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6fe5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff15970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40508/0 (socket says 172.21.15.39:40508) 2024-07-04T08:22:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6fe5400640 1 -- 172.21.15.39:0/2274158261 learned_addr learned my addr 172.21.15.39:0/2274158261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.909+0000 7f6fede00640 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb481b0 0x55706ff15eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.910+0000 7f6fe5400640 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.910+0000 7f6fede00640 1 -- 172.21.15.39:0/2274158261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 msgr2=0x55706ff163f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.910+0000 7f6fede00640 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff163f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.910+0000 7f6fede00640 1 -- 172.21.15.39:0/2274158261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 msgr2=0x55706ff15970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.910+0000 7f6fede00640 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff15970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.910+0000 7f6fede00640 1 -- 172.21.15.39:0/2274158261 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706fe13670 con 0x55706fb481b0 2024-07-04T08:22:36.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.910+0000 7f6fe5400640 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:22:36.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.910+0000 7f6fe7400640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b2960 0x7f6fd40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.910+0000 7f6fede00640 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb481b0 0x55706ff15eb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009cc0 tx=0x7f6fe0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c451e49227a62211 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.911+0000 7f6fe7400640 1 -- 172.21.15.39:0/2092665690 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f6fe81034b0 con 0x7f6ff010f4a0 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.911+0000 7f6fe5400640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b2960 0x7f6fd40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.911+0000 7f6fdf400640 1 -- 172.21.15.39:0/2274158261 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0004230 con 0x55706fb481b0 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.911+0000 7f6fdf400640 1 -- 172.21.15.39:0/2274158261 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe00043d0 con 0x55706fb481b0 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.911+0000 7f6fdf400640 1 -- 172.21.15.39:0/2274158261 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0004c90 con 0x55706fb481b0 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2274158261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb481b0 msgr2=0x55706ff15eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb481b0 0x55706ff15eb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009cc0 tx=0x7f6fe0011810 comp rx=0 tx=0).stop 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6fe5400640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b2960 0x7f6fd40b4e70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8010660 tx=0x7f6fd8015040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2274158261 shutdown_connections 2024-07-04T08:22:36.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff163f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb481b0 0x55706ff15eb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2274158261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff15970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2274158261 >> 172.21.15.39:0/2274158261 conn(0x55706fb65460 msgr2=0x55706ff13860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:36.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2274158261 shutdown_connections 2024-07-04T08:22:36.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.912+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2274158261 wait complete. 2024-07-04T08:22:36.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.913+0000 7f6ffb7b3740 1 -- start start 2024-07-04T08:22:36.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.913+0000 7f6ffb7b3740 1 Processor -- start 2024-07-04T08:22:36.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb50390 0x55706ff21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff2d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55706fcdf700 con 0x55706fb71070 2024-07-04T08:22:36.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55706fe22bd0 con 0x55706fb481b0 2024-07-04T08:22:36.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55706ff24150 con 0x55706fb50390 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6ffb7b3740 10 client.0 _ll_register_callbacks cb 0x55706fe74590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6fede00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6fede00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff214d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47302/0 (socket says 172.21.15.39:47302) 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6fede00640 1 -- 172.21.15.39:0/2771527974 learned_addr learned my addr 172.21.15.39:0/2771527974 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.914+0000 7f6fe5400640 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb50390 0x55706ff21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fee800640 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff2d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fede00640 1 -- 172.21.15.39:0/2771527974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb50390 msgr2=0x55706ff21a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fede00640 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb50390 0x55706ff21a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fede00640 1 -- 172.21.15.39:0/2771527974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 msgr2=0x55706ff2d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fede00640 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff2d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fede00640 1 -- 172.21.15.39:0/2771527974 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706fbdc650 con 0x55706fb481b0 2024-07-04T08:22:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fee800640 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff2d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:22:36.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fede00640 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff214d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009880 tx=0x7f6fe0019b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6fcf70d97b88c2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:36.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:36.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.915+0000 7f6fd3e00640 1 -- 172.21.15.39:0/2771527974 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0028020 con 0x55706fb481b0 2024-07-04T08:22:36.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.916+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2771527974 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe00043d0 con 0x55706fb481b0 2024-07-04T08:22:36.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.916+0000 7f6fd3e00640 1 -- 172.21.15.39:0/2771527974 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f6fe0024620 con 0x55706fb481b0 2024-07-04T08:22:36.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.916+0000 7f6fd3e00640 1 -- 172.21.15.39:0/2771527974 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fe0033050 con 0x55706fb481b0 2024-07-04T08:22:36.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.916+0000 7f6ffb7b3740 10 client.27332 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:36.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.916+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2771527974 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55706ff300e0 con 0x55706fb481b0 2024-07-04T08:22:36.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.916+0000 7f6fd3e00640 1 -- 172.21.15.39:0/2771527974 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe00043d0 con 0x55706fb481b0 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.917+0000 7f6fd3e00640 1 -- 172.21.15.39:0/2771527974 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6fe0018070 con 0x55706fb481b0 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.917+0000 7f6ffb7b3740 10 client.27332 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.917+0000 7f6ffb7b3740 10 client.27332 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.917+0000 7f6ffb7b3740 20 client.27332 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.917+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55706fb2f9f0 0x55706ff334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.917+0000 7f6ffb7b3740 4 client.27332 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.917+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2771527974 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55706fe8de50 con 0x55706fb2f9f0 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.917+0000 7f6fe5400640 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55706fb2f9f0 0x55706ff334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.918+0000 7f6fe5400640 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55706fb2f9f0 0x55706ff334a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=be7fbc346e3e1545 server_cookie=1ccc5a0fe52897d1 in_seq=0 out_seq=0 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.918+0000 7f6fd3e00640 10 client.27332 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:22:36.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6fd3e00640 1 -- 172.21.15.39:0/2771527974 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55706fe8de50 con 0x55706fb2f9f0 2024-07-04T08:22:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6fd3e00640 10 client.27332 handle_command_reply: tid=0 2024-07-04T08:22:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 client.27332 shutdown 2024-07-04T08:22:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2771527974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 msgr2=0x55706ff214d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff214d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009880 tx=0x7f6fe0019b50 comp rx=0 tx=0).stop 2024-07-04T08:22:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2771527974 shutdown_connections 2024-07-04T08:22:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55706fb2f9f0 0x55706ff334a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb71070 0x55706ff2d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb50390 0x55706ff21a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/2771527974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff214d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2771527974 >> 172.21.15.39:0/2771527974 conn(0x55706fb65460 msgr2=0x55706ff18b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6fd3e00640 0 client.27332 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:22:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2771527974 shutdown_connections 2024-07-04T08:22:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/2771527974 wait complete. 2024-07-04T08:22:36.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.919+0000 7f6ffb7b3740 20 client.27332 trim_cache size 0 max 16384 2024-07-04T08:22:36.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.935+0000 7f6ffb7b3740 1 Processor -- start 2024-07-04T08:22:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.935+0000 7f6ffb7b3740 1 -- start start 2024-07-04T08:22:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb50390 0x55706ff18a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 0x55706ff170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b2300 con 0x55706fb50390 2024-07-04T08:22:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b3e40 con 0x55706fb481b0 2024-07-04T08:22:36.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b29d0 con 0x55706fb71070 2024-07-04T08:22:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6fede00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6fede00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff231b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47308/0 (socket says 172.21.15.39:47308) 2024-07-04T08:22:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6fede00640 1 -- 172.21.15.39:0/1073745494 learned_addr learned my addr 172.21.15.39:0/1073745494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.936+0000 7f6fe5400640 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb50390 0x55706ff18a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.937+0000 7f6fee800640 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 0x55706ff170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.937+0000 7f6fede00640 1 -- 172.21.15.39:0/1073745494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 msgr2=0x55706ff170b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:36.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.937+0000 7f6fede00640 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 0x55706ff170b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.937+0000 7f6fede00640 1 -- 172.21.15.39:0/1073745494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb50390 msgr2=0x55706ff18a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:36.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.937+0000 7f6fede00640 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb50390 0x55706ff18a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.937+0000 7f6fede00640 1 -- 172.21.15.39:0/1073745494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706ff17c60 con 0x55706fb481b0 2024-07-04T08:22:36.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.937+0000 7f6fee800640 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 0x55706ff170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:22:36.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.937+0000 7f6fede00640 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff231b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0011850 tx=0x7f6fe0019bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84bb7474cac99ade server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.937+0000 7f6fdea00640 1 -- 172.21.15.39:0/1073745494 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0028070 con 0x55706fb481b0 2024-07-04T08:22:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.938+0000 7f6fdea00640 1 -- 172.21.15.39:0/1073745494 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe001f040 con 0x55706fb481b0 2024-07-04T08:22:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.938+0000 7f6fdea00640 1 -- 172.21.15.39:0/1073745494 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00042f0 con 0x55706fb481b0 2024-07-04T08:22:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/1073745494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 msgr2=0x55706ff231b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff231b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0011850 tx=0x7f6fe0019bc0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/1073745494 shutdown_connections 2024-07-04T08:22:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb50390 0x55706ff18a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 0x55706ff170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/1073745494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb481b0 0x55706ff231b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/1073745494 >> 172.21.15.39:0/1073745494 conn(0x55706fb65460 msgr2=0x55706ff169a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:36.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/1073745494 shutdown_connections 2024-07-04T08:22:36.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/1073745494 wait complete. 2024-07-04T08:22:36.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.939+0000 7f6ffb7b3740 1 -- start start 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.940+0000 7f6ffb7b3740 1 Processor -- start 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.940+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb481b0 0x55706ff247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.940+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff24d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.940+0000 7f6ffb7b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 0x55706ff99b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.940+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b3770 con 0x55706fb481b0 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.940+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b30a0 con 0x55706fb50390 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.940+0000 7f6ffb7b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b02e0 con 0x55706fb71070 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.940+0000 7f6ffb7b3740 10 client.0 _ll_register_callbacks cb 0x7f6fd8009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fe5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff24d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fe5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff24d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47310/0 (socket says 172.21.15.39:47310) 2024-07-04T08:22:36.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fe5400640 1 -- 172.21.15.39:0/3040199440 learned_addr learned my addr 172.21.15.39:0/3040199440 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:36.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fede00640 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb481b0 0x55706ff247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fee800640 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 0x55706ff99b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fe5400640 1 -- 172.21.15.39:0/3040199440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 msgr2=0x55706ff99b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:36.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fe5400640 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 0x55706ff99b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fe5400640 1 -- 172.21.15.39:0/3040199440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb481b0 msgr2=0x55706ff247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fe5400640 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb481b0 0x55706ff247a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fe5400640 1 -- 172.21.15.39:0/3040199440 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706ff9a070 con 0x55706fb50390 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.941+0000 7f6fede00640 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb481b0 0x55706ff247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.942+0000 7f6fe5400640 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff24d20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009460 tx=0x7f6fd8009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f0ed072cf2d4f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.942+0000 7f6fd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.942+0000 7f6fd3e00640 1 -- 172.21.15.39:0/3040199440 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8033070 con 0x55706fb50390 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.942+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/3040199440 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe001f040 con 0x55706fb50390 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.942+0000 7f6fd3e00640 1 -- 172.21.15.39:0/3040199440 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f6fd8004220 con 0x55706fb50390 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.942+0000 7f6fd3e00640 1 -- 172.21.15.39:0/3040199440 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fd803d050 con 0x55706fb50390 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.942+0000 7f6ffb7b3740 10 client.27338 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.942+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/3040199440 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55706ff9b270 con 0x55706fb50390 2024-07-04T08:22:36.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.943+0000 7f6fd3e00640 1 -- 172.21.15.39:0/3040199440 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe001f040 con 0x55706fb50390 2024-07-04T08:22:36.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.943+0000 7f6fd3e00640 1 -- 172.21.15.39:0/3040199440 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6fd8003a40 con 0x55706fb50390 2024-07-04T08:22:36.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.943+0000 7f6ffb7b3740 10 client.27338 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:36.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.943+0000 7f6ffb7b3740 10 client.27338 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:22:36.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.943+0000 7f6ffb7b3740 20 client.27338 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:36.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.943+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55706fb04da0 0x55706ff9da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:36.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.943+0000 7f6ffb7b3740 4 client.27338 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:36.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.943+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/3040199440 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55706fe898f0 con 0x55706fb04da0 2024-07-04T08:22:36.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.944+0000 7f6fede00640 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55706fb04da0 0x55706ff9da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:36.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.944+0000 7f6fede00640 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55706fb04da0 0x55706ff9da60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b814adffcb198cc server_cookie=22a7c73cdf7433d5 in_seq=0 out_seq=0 2024-07-04T08:22:36.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.944+0000 7f6fd3e00640 10 client.27338 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:22:36.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.944+0000 7f6fd3e00640 1 -- 172.21.15.39:0/3040199440 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55706fe898f0 con 0x55706fb04da0 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.944+0000 7f6fd3e00640 10 client.27338 handle_command_reply: tid=0 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.944+0000 7f6ffb7b3740 1 client.27338 shutdown 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/3040199440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 msgr2=0x55706ff24d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff24d20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009460 tx=0x7f6fd8009570 comp rx=0 tx=0).stop 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/3040199440 shutdown_connections 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55706fb04da0 0x55706ff9da60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55706fb71070 0x55706ff99b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55706fb50390 0x55706ff24d20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 --2- 172.21.15.39:0/3040199440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55706fb481b0 0x55706ff247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/3040199440 >> 172.21.15.39:0/3040199440 conn(0x55706fb65460 msgr2=0x55706ff23010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6fd3e00640 0 client.27338 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/3040199440 shutdown_connections 2024-07-04T08:22:36.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 1 -- 172.21.15.39:0/3040199440 wait complete. 2024-07-04T08:22:36.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.945+0000 7f6ffb7b3740 20 client.27338 trim_cache size 0 max 16384 2024-07-04T08:22:36.942 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 -- 172.21.15.39:0/2092665690 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b2960 msgr2=0x7f6fd40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b2960 0x7f6fd40b4e70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8010660 tx=0x7f6fd8015040 comp rx=0 tx=0).stop 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 -- 172.21.15.39:0/2092665690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff010f4a0 msgr2=0x7f6ff01a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff010f4a0 0x7f6ff01a1250 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009b30 tx=0x7f6fe8003b80 comp rx=0 tx=0).stop 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 -- 172.21.15.39:0/2092665690 shutdown_connections 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b2960 0x7f6fd40b4e70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff00755f0 0x7f6ff019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff010f4a0 0x7f6ff01a1250 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 --2- 172.21.15.39:0/2092665690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0074ca0 0x7f6ff019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 -- 172.21.15.39:0/2092665690 >> 172.21.15.39:0/2092665690 conn(0x7f6ff00fe930 msgr2=0x7f6ff0109040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 -- 172.21.15.39:0/2092665690 shutdown_connections 2024-07-04T08:22:36.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:36.946+0000 7f6fef200640 1 -- 172.21.15.39:0/2092665690 wait complete. 2024-07-04T08:22:37.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:37 smithi133 ceph-mon[37050]: pgmap v704: 129 pgs: 129 active+clean; 8.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-07-04T08:22:37.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:37 smithi039 ceph-mon[32062]: pgmap v704: 129 pgs: 129 active+clean; 8.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-07-04T08:22:37.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:37 smithi050 ceph-mon[36577]: pgmap v704: 129 pgs: 129 active+clean; 8.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-07-04T08:22:37.847 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:22:38.343 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:39.299 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:39 smithi039 ceph-mon[32062]: pgmap v705: 129 pgs: 129 active+clean; 8.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-07-04T08:22:39.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:39 smithi050 ceph-mon[36577]: pgmap v705: 129 pgs: 129 active+clean; 8.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-07-04T08:22:39.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:39 smithi133 ceph-mon[37050]: pgmap v705: 129 pgs: 129 active+clean; 8.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-07-04T08:22:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.647+0000 7f700e000640 1 -- 172.21.15.39:0/3999046470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081052d0 msgr2=0x7f70081056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.647+0000 7f700e000640 1 --2- 172.21.15.39:0/3999046470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081052d0 0x7f70081056e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009900 tx=0x7f6ff802a400 comp rx=0 tx=0).stop 2024-07-04T08:22:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.647+0000 7f700e000640 1 -- 172.21.15.39:0/3999046470 shutdown_connections 2024-07-04T08:22:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.647+0000 7f700e000640 1 --2- 172.21.15.39:0/3999046470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f700810b560 0x7f7008111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.647+0000 7f700e000640 1 --2- 172.21.15.39:0/3999046470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700810ac10 0x7f700810b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.647+0000 7f700e000640 1 --2- 172.21.15.39:0/3999046470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081052d0 0x7f70081056e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.647+0000 7f700e000640 1 -- 172.21.15.39:0/3999046470 >> 172.21.15.39:0/3999046470 conn(0x7f7008100d60 msgr2=0x7f70081031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.648+0000 7f700e000640 1 -- 172.21.15.39:0/3999046470 shutdown_connections 2024-07-04T08:22:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.648+0000 7f700e000640 1 -- 172.21.15.39:0/3999046470 wait complete. 2024-07-04T08:22:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.649+0000 7f700e000640 1 Processor -- start 2024-07-04T08:22:40.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.649+0000 7f700e000640 1 -- start start 2024-07-04T08:22:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f700e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081052d0 0x7f700819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f700e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700810ac10 0x7f700819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f700e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f700810b560 0x7f70081a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f700e000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7008115280 con 0x7f700810b560 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f700e000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7008115100 con 0x7f70081052d0 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f700e000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7008115400 con 0x7f700810ac10 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f7007e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f700810b560 0x7f70081a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f7007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081052d0 0x7f700819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f7007e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f700810b560 0x7f70081a1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40548/0 (socket says 172.21.15.39:40548) 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f7007e00640 1 -- 172.21.15.39:0/3038343608 learned_addr learned my addr 172.21.15.39:0/3038343608 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.650+0000 7f7006a00640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700810ac10 0x7f700819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.651+0000 7f7007e00640 1 -- 172.21.15.39:0/3038343608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700810ac10 msgr2=0x7f700819c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.651+0000 7f7007e00640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700810ac10 0x7f700819c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.651+0000 7f7007e00640 1 -- 172.21.15.39:0/3038343608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081052d0 msgr2=0x7f700819bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.651+0000 7f7007e00640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081052d0 0x7f700819bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.651+0000 7f7007e00640 1 -- 172.21.15.39:0/3038343608 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70081a1ba0 con 0x7f700810b560 2024-07-04T08:22:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.651+0000 7f7007e00640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f700810b560 0x7f70081a1470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f700000e9b0 tx=0x7f700000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.652+0000 7f6fffe00640 1 -- 172.21.15.39:0/3038343608 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700000cd80 con 0x7f700810b560 2024-07-04T08:22:40.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.652+0000 7f6fffe00640 1 -- 172.21.15.39:0/3038343608 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70000035f0 con 0x7f700810b560 2024-07-04T08:22:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.652+0000 7f700e000640 1 -- 172.21.15.39:0/3038343608 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70081a1e90 con 0x7f700810b560 2024-07-04T08:22:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.652+0000 7f6fffe00640 1 -- 172.21.15.39:0/3038343608 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000010690 con 0x7f700810b560 2024-07-04T08:22:40.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.652+0000 7f700e000640 1 -- 172.21.15.39:0/3038343608 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70081a85f0 con 0x7f700810b560 2024-07-04T08:22:40.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.656+0000 7f70146aa740 1 Processor -- start 2024-07-04T08:22:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.657+0000 7f6fffe00640 1 -- 172.21.15.39:0/3038343608 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7000004000 con 0x7f700810b560 2024-07-04T08:22:40.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.657+0000 7f70146aa740 1 -- start start 2024-07-04T08:22:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f70146aa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 0x5637390ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f70146aa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d321b0 0x5637390ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f70146aa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 0x563739100230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f70146aa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563739075ee0 con 0x563738d321b0 2024-07-04T08:22:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f70146aa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563739065480 con 0x563738d5b070 2024-07-04T08:22:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f70146aa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5637390694c0 con 0x563738d3a390 2024-07-04T08:22:40.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f7007400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d321b0 0x5637390ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f7007400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d321b0 0x5637390ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40552/0 (socket says 172.21.15.39:40552) 2024-07-04T08:22:40.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f7007400640 1 -- 172.21.15.39:0/4041272046 learned_addr learned my addr 172.21.15.39:0/4041272046 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:40.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f7006a00640 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 0x5637390ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.658+0000 7f7006a00640 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 0x563739100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.659+0000 7f7007400640 1 -- 172.21.15.39:0/4041272046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 msgr2=0x563739100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.659+0000 7f7007400640 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 0x563739100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.659+0000 7f7007400640 1 -- 172.21.15.39:0/4041272046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 msgr2=0x5637390ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.659+0000 7f7007400640 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 0x5637390ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.659+0000 7f7007400640 1 -- 172.21.15.39:0/4041272046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563738ffd650 con 0x563738d321b0 2024-07-04T08:22:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.659+0000 7f7006a00640 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 0x5637390ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.659+0000 7f7007400640 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d321b0 0x5637390ffcf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009230 tx=0x7f6ff8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e36ae3eb709a71 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.659+0000 7f6fffe00640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fe80b2960 0x7f6fe80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.660+0000 7f6ffcc00640 1 -- 172.21.15.39:0/4041272046 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8035040 con 0x563738d321b0 2024-07-04T08:22:40.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.660+0000 7f6ffcc00640 1 -- 172.21.15.39:0/4041272046 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff80054f0 con 0x563738d321b0 2024-07-04T08:22:40.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.660+0000 7f6ffcc00640 1 -- 172.21.15.39:0/4041272046 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff80035f0 con 0x563738d321b0 2024-07-04T08:22:40.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.660+0000 7f6fffe00640 1 -- 172.21.15.39:0/3038343608 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f70000e2ce0 con 0x7f700810b560 2024-07-04T08:22:40.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.660+0000 7f7006a00640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fe80b2960 0x7f6fe80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f70146aa740 1 -- 172.21.15.39:0/4041272046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d321b0 msgr2=0x5637390ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f7006a00640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fe80b2960 0x7f6fe80b4e70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f700810a790 tx=0x7f6ff0004ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f70146aa740 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d321b0 0x5637390ffcf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009230 tx=0x7f6ff8004040 comp rx=0 tx=0).stop 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f70146aa740 1 -- 172.21.15.39:0/4041272046 shutdown_connections 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f70146aa740 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d321b0 0x5637390ffcf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f70146aa740 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 0x563739100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f70146aa740 1 --2- 172.21.15.39:0/4041272046 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 0x5637390ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f70146aa740 1 -- 172.21.15.39:0/4041272046 >> 172.21.15.39:0/4041272046 conn(0x563738d4f460 msgr2=0x5637390fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f70146aa740 1 -- 172.21.15.39:0/4041272046 shutdown_connections 2024-07-04T08:22:40.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.661+0000 7f70146aa740 1 -- 172.21.15.39:0/4041272046 wait complete. 2024-07-04T08:22:40.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.662+0000 7f70146aa740 1 -- start start 2024-07-04T08:22:40.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.662+0000 7f70146aa740 1 Processor -- start 2024-07-04T08:22:40.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.662+0000 7f70146aa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 0x56373910b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f70146aa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d3a390 0x56373910b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f70146aa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x5637391172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f70146aa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563738ec9700 con 0x563738d5b070 2024-07-04T08:22:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f70146aa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56373900cbb0 con 0x563738d3a390 2024-07-04T08:22:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f70146aa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56373910df30 con 0x563738d321b0 2024-07-04T08:22:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f70146aa740 10 client.0 _ll_register_callbacks cb 0x56373905e570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7006a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d3a390 0x56373910b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7006a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d3a390 0x56373910b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47352/0 (socket says 172.21.15.39:47352) 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7006a00640 1 -- 172.21.15.39:0/2900753404 learned_addr learned my addr 172.21.15.39:0/2900753404 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7007400640 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 0x56373910b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7007e00640 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x5637391172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7006a00640 1 -- 172.21.15.39:0/2900753404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 msgr2=0x56373910b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7006a00640 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 0x56373910b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7006a00640 1 -- 172.21.15.39:0/2900753404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 msgr2=0x5637391172b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7006a00640 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x5637391172b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.663+0000 7f7006a00640 1 -- 172.21.15.39:0/2900753404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563738dc6650 con 0x563738d3a390 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.664+0000 7f7006a00640 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d3a390 0x56373910b830 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00115a0 tx=0x7f6ff0009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a8a3197c384655 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:40.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.664+0000 7f6ff5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.664+0000 7f6ff5600640 1 -- 172.21.15.39:0/2900753404 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff002f060 con 0x563738d3a390 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.664+0000 7f70146aa740 1 -- 172.21.15.39:0/2900753404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff80054f0 con 0x563738d3a390 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.664+0000 7f6ff5600640 1 -- 172.21.15.39:0/2900753404 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f6ff001a050 con 0x563738d3a390 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.665+0000 7f6ff5600640 1 -- 172.21.15.39:0/2900753404 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ff0036050 con 0x563738d3a390 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.665+0000 7f70146aa740 10 client.27350 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.665+0000 7f70146aa740 1 -- 172.21.15.39:0/2900753404 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563739119fb0 con 0x563738d3a390 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.665+0000 7f6ff5600640 1 -- 172.21.15.39:0/2900753404 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff80054f0 con 0x563738d3a390 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.665+0000 7f6ff5600640 1 -- 172.21.15.39:0/2900753404 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6ff000e3b0 con 0x563738d3a390 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.665+0000 7f70146aa740 10 client.27350 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.665+0000 7f70146aa740 10 client.27350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.665+0000 7f70146aa740 20 client.27350 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:40.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.665+0000 7f70146aa740 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563738d199f0 0x56373911d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.666+0000 7f70146aa740 4 client.27350 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:40.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.666+0000 7f70146aa740 1 -- 172.21.15.39:0/2900753404 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563739077e30 con 0x563738d199f0 2024-07-04T08:22:40.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.666+0000 7f7007400640 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563738d199f0 0x56373911d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.667+0000 7f7007400640 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563738d199f0 0x56373911d490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c8138f33d802450 server_cookie=43f6e540fcafd41a in_seq=0 out_seq=0 2024-07-04T08:22:40.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.667+0000 7f6ff5600640 10 client.27350 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:22:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.667+0000 7f6ff5600640 1 -- 172.21.15.39:0/2900753404 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563739077e30 con 0x563738d199f0 2024-07-04T08:22:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.667+0000 7f6ff5600640 10 client.27350 handle_command_reply: tid=0 2024-07-04T08:22:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 client.27350 shutdown 2024-07-04T08:22:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 -- 172.21.15.39:0/2900753404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d3a390 msgr2=0x56373910b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d3a390 0x56373910b830 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00115a0 tx=0x7f6ff0009500 comp rx=0 tx=0).stop 2024-07-04T08:22:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 -- 172.21.15.39:0/2900753404 shutdown_connections 2024-07-04T08:22:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563738d199f0 0x56373911d490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x5637391172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d3a390 0x56373910b830 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 --2- 172.21.15.39:0/2900753404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 0x56373910b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f6ff5600640 0 client.27350 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:22:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 -- 172.21.15.39:0/2900753404 >> 172.21.15.39:0/2900753404 conn(0x563738d4f460 msgr2=0x563739102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 -- 172.21.15.39:0/2900753404 shutdown_connections 2024-07-04T08:22:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 1 -- 172.21.15.39:0/2900753404 wait complete. 2024-07-04T08:22:40.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.668+0000 7f70146aa740 20 client.27350 trim_cache size 0 max 16384 2024-07-04T08:22:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.684+0000 7f70146aa740 1 Processor -- start 2024-07-04T08:22:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.684+0000 7f70146aa740 1 -- start start 2024-07-04T08:22:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.684+0000 7f70146aa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d321b0 0x563739102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.684+0000 7f70146aa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 0x5637390fc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.684+0000 7f70146aa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x563739104840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.684+0000 7f70146aa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b2500 con 0x563738d5b070 2024-07-04T08:22:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.684+0000 7f70146aa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b4040 con 0x563738d321b0 2024-07-04T08:22:40.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.684+0000 7f70146aa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b2bd0 con 0x563738d3a390 2024-07-04T08:22:40.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x563739104840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d321b0 0x563739102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7006a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 0x5637390fc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x563739104840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40574/0 (socket says 172.21.15.39:40574) 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007e00640 1 -- 172.21.15.39:0/367098211 learned_addr learned my addr 172.21.15.39:0/367098211 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007e00640 1 -- 172.21.15.39:0/367098211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 msgr2=0x5637390fc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007e00640 1 --2- 172.21.15.39:0/367098211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 0x5637390fc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007e00640 1 -- 172.21.15.39:0/367098211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d321b0 msgr2=0x563739102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007e00640 1 --2- 172.21.15.39:0/367098211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d321b0 0x563739102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007e00640 1 -- 172.21.15.39:0/367098211 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373910dac0 con 0x563738d5b070 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7006a00640 1 --2- 172.21.15.39:0/367098211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 0x5637390fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007400640 1 --2- 172.21.15.39:0/367098211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d321b0 0x563739102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:40.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.685+0000 7f7007e00640 1 --2- 172.21.15.39:0/367098211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x563739104840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f70000a9b90 tx=0x7f70000a9710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fbbbc483da2866 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:40.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.686+0000 7f6ff7e00640 1 -- 172.21.15.39:0/367098211 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70000a9c00 con 0x563738d5b070 2024-07-04T08:22:40.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.686+0000 7f6ff7e00640 1 -- 172.21.15.39:0/367098211 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70000adfb0 con 0x563738d5b070 2024-07-04T08:22:40.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.686+0000 7f6ff7e00640 1 -- 172.21.15.39:0/367098211 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70000a5570 con 0x563738d5b070 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 -- 172.21.15.39:0/367098211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 msgr2=0x563739104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 --2- 172.21.15.39:0/367098211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x563739104840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f70000a9b90 tx=0x7f70000a9710 comp rx=0 tx=0).stop 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 -- 172.21.15.39:0/367098211 shutdown_connections 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 --2- 172.21.15.39:0/367098211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d5b070 0x563739104840 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 --2- 172.21.15.39:0/367098211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d3a390 0x5637390fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 --2- 172.21.15.39:0/367098211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d321b0 0x563739102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 -- 172.21.15.39:0/367098211 >> 172.21.15.39:0/367098211 conn(0x563738d4f460 msgr2=0x5637391028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 -- 172.21.15.39:0/367098211 shutdown_connections 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 -- 172.21.15.39:0/367098211 wait complete. 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.687+0000 7f70146aa740 1 -- start start 2024-07-04T08:22:40.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.688+0000 7f70146aa740 1 Processor -- start 2024-07-04T08:22:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.688+0000 7f70146aa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 0x563739113730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.688+0000 7f70146aa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d3a390 0x563739122530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.688+0000 7f70146aa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 0x563739122a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.688+0000 7f70146aa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b3970 con 0x563738d3a390 2024-07-04T08:22:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.688+0000 7f70146aa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b32a0 con 0x563738d5b070 2024-07-04T08:22:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.688+0000 7f70146aa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f70081b04e0 con 0x563738d321b0 2024-07-04T08:22:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.688+0000 7f70146aa740 10 client.0 _ll_register_callbacks cb 0x7f6ff0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:40.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7006a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d3a390 0x563739122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7007400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 0x563739113730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7006a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d3a390 0x563739122530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40578/0 (socket says 172.21.15.39:40578) 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7006a00640 1 -- 172.21.15.39:0/904951293 learned_addr learned my addr 172.21.15.39:0/904951293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7007e00640 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 0x563739122a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7006a00640 1 -- 172.21.15.39:0/904951293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 msgr2=0x563739113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7006a00640 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 0x563739113730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7006a00640 1 -- 172.21.15.39:0/904951293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 msgr2=0x563739122a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7006a00640 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 0x563739122a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7006a00640 1 -- 172.21.15.39:0/904951293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563739123060 con 0x563738d3a390 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7007400640 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 0x563739113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f7006a00640 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d3a390 0x563739122530 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000f8c0 tx=0x7f6ff0005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85af0deab8bb59f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.689+0000 7f6ff5600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:22:40.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.690+0000 7f6ff5600640 1 -- 172.21.15.39:0/904951293 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000fcc0 con 0x563738d3a390 2024-07-04T08:22:40.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.690+0000 7f6ff5600640 1 -- 172.21.15.39:0/904951293 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f6ff000e9f0 con 0x563738d3a390 2024-07-04T08:22:40.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.690+0000 7f70146aa740 1 -- 172.21.15.39:0/904951293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70000adfb0 con 0x563738d3a390 2024-07-04T08:22:40.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.690+0000 7f6ff5600640 1 -- 172.21.15.39:0/904951293 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ff0009e70 con 0x563738d3a390 2024-07-04T08:22:40.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.690+0000 7f70146aa740 10 client.17217 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:40.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.690+0000 7f70146aa740 1 -- 172.21.15.39:0/904951293 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563739110fc0 con 0x563738d3a390 2024-07-04T08:22:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.690+0000 7f6ff5600640 1 -- 172.21.15.39:0/904951293 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70000adfb0 con 0x563738d3a390 2024-07-04T08:22:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.691+0000 7f6ff5600640 1 -- 172.21.15.39:0/904951293 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6ff0004320 con 0x563738d3a390 2024-07-04T08:22:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.691+0000 7f70146aa740 10 client.17217 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:40.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.691+0000 7f70146aa740 10 client.17217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.691+0000 7f70146aa740 20 client.17217 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.691+0000 7f70146aa740 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563738ceeda0 0x5637391880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.691+0000 7f70146aa740 4 client.17217 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.691+0000 7f70146aa740 1 -- 172.21.15.39:0/904951293 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637390738d0 con 0x563738ceeda0 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.691+0000 7f7007400640 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563738ceeda0 0x5637391880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.692+0000 7f7007400640 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563738ceeda0 0x5637391880d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1ce0eef173805c8 server_cookie=34910c7eeb5352ee in_seq=0 out_seq=0 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.692+0000 7f6ff5600640 10 client.17217 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.692+0000 7f6ff5600640 1 -- 172.21.15.39:0/904951293 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637390738d0 con 0x563738ceeda0 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.692+0000 7f6ff5600640 10 client.17217 handle_command_reply: tid=0 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.692+0000 7f70146aa740 1 client.17217 shutdown 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.692+0000 7f70146aa740 1 -- 172.21.15.39:0/904951293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d3a390 msgr2=0x563739122530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.692+0000 7f70146aa740 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d3a390 0x563739122530 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000f8c0 tx=0x7f6ff0005e30 comp rx=0 tx=0).stop 2024-07-04T08:22:40.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f70146aa740 1 -- 172.21.15.39:0/904951293 shutdown_connections 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f70146aa740 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x563738ceeda0 0x5637391880d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f70146aa740 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563738d3a390 0x563739122530 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f70146aa740 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563738d5b070 0x563739122a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f70146aa740 1 --2- 172.21.15.39:0/904951293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563738d321b0 0x563739113730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f6ff5600640 0 client.17217 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f70146aa740 1 -- 172.21.15.39:0/904951293 >> 172.21.15.39:0/904951293 conn(0x563738d4f460 msgr2=0x5637390fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f70146aa740 1 -- 172.21.15.39:0/904951293 shutdown_connections 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f70146aa740 1 -- 172.21.15.39:0/904951293 wait complete. 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.693+0000 7f70146aa740 20 client.17217 trim_cache size 0 max 16384 2024-07-04T08:22:40.689 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:40.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 -- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fe80b2960 msgr2=0x7f6fe80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:40.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fe80b2960 0x7f6fe80b4e70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f700810a790 tx=0x7f6ff0004ab0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 -- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f700810b560 msgr2=0x7f70081a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:40.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f700810b560 0x7f70081a1470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f700000e9b0 tx=0x7f700000ee70 comp rx=0 tx=0).stop 2024-07-04T08:22:40.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 -- 172.21.15.39:0/3038343608 shutdown_connections 2024-07-04T08:22:40.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fe80b2960 0x7f6fe80b4e70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f700810b560 0x7f70081a1470 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f700810ac10 0x7f700819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 --2- 172.21.15.39:0/3038343608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f70081052d0 0x7f700819bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:40.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 -- 172.21.15.39:0/3038343608 >> 172.21.15.39:0/3038343608 conn(0x7f7008100d60 msgr2=0x7f7008103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:40.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 -- 172.21.15.39:0/3038343608 shutdown_connections 2024-07-04T08:22:40.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:40.694+0000 7f700e000640 1 -- 172.21.15.39:0/3038343608 wait complete. 2024-07-04T08:22:41.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:41 smithi039 ceph-mon[32062]: pgmap v706: 129 pgs: 129 active+clean; 8.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-07-04T08:22:41.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:41 smithi050 ceph-mon[36577]: pgmap v706: 129 pgs: 129 active+clean; 8.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-07-04T08:22:41.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:41 smithi133 ceph-mon[37050]: pgmap v706: 129 pgs: 129 active+clean; 8.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-07-04T08:22:41.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:22:42.129 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:43.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:43 smithi039 ceph-mon[32062]: pgmap v707: 129 pgs: 129 active+clean; 8.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-07-04T08:22:43.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:43 smithi050 ceph-mon[36577]: pgmap v707: 129 pgs: 129 active+clean; 8.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-07-04T08:22:43.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:43 smithi133 ceph-mon[37050]: pgmap v707: 129 pgs: 129 active+clean; 8.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 17 op/s 2024-07-04T08:22:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.538+0000 7fa911600640 1 -- 172.21.15.39:0/1572184949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90c1089f0 msgr2=0x7fa90c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.538+0000 7fa911600640 1 --2- 172.21.15.39:0/1572184949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90c1089f0 0x7fa90c108e00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009900 tx=0x7fa8f802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.538+0000 7fa911600640 1 -- 172.21.15.39:0/1572184949 shutdown_connections 2024-07-04T08:22:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.538+0000 7fa911600640 1 --2- 172.21.15.39:0/1572184949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90c109340 0x7fa90c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.538+0000 7fa911600640 1 --2- 172.21.15.39:0/1572184949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90c1089f0 0x7fa90c108e00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.538+0000 7fa911600640 1 --2- 172.21.15.39:0/1572184949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa90c1030b0 0x7fa90c1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.538+0000 7fa911600640 1 -- 172.21.15.39:0/1572184949 >> 172.21.15.39:0/1572184949 conn(0x7fa90c07ada0 msgr2=0x7fa90c07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.538+0000 7fa911600640 1 -- 172.21.15.39:0/1572184949 shutdown_connections 2024-07-04T08:22:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.538+0000 7fa911600640 1 -- 172.21.15.39:0/1572184949 wait complete. 2024-07-04T08:22:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.540+0000 7fa911600640 1 Processor -- start 2024-07-04T08:22:44.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.540+0000 7fa911600640 1 -- start start 2024-07-04T08:22:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.541+0000 7fa911600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90c1030b0 0x7fa90c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.541+0000 7fa911600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90c1089f0 0x7fa90c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.541+0000 7fa911600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa90c109340 0x7fa90c1a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.541+0000 7fa911600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c112fd0 con 0x7fa90c1030b0 2024-07-04T08:22:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.541+0000 7fa911600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c112e50 con 0x7fa90c1089f0 2024-07-04T08:22:44.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.541+0000 7fa911600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c113150 con 0x7fa90c109340 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.541+0000 7fa90aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90c1030b0 0x7fa90c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90c1030b0 0x7fa90c19bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48574/0 (socket says 172.21.15.39:48574) 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90aa00640 1 -- 172.21.15.39:0/3495391155 learned_addr learned my addr 172.21.15.39:0/3495391155 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90a000640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90c1089f0 0x7fa90c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90b400640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa90c109340 0x7fa90c1a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90aa00640 1 -- 172.21.15.39:0/3495391155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa90c109340 msgr2=0x7fa90c1a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90aa00640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa90c109340 0x7fa90c1a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90aa00640 1 -- 172.21.15.39:0/3495391155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90c1089f0 msgr2=0x7fa90c19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90aa00640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90c1089f0 0x7fa90c19c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90aa00640 1 -- 172.21.15.39:0/3495391155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90c1a1b70 con 0x7fa90c1030b0 2024-07-04T08:22:44.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90a000640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90c1089f0 0x7fa90c19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:44.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.542+0000 7fa90b400640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa90c109340 0x7fa90c1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:22:44.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.543+0000 7fa90aa00640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90c1030b0 0x7fa90c19bc00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e9f0 tx=0x7fa8fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:44.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.543+0000 7fa903400640 1 -- 172.21.15.39:0/3495391155 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00cd80 con 0x7fa90c1030b0 2024-07-04T08:22:44.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.543+0000 7fa903400640 1 -- 172.21.15.39:0/3495391155 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8fc0054d0 con 0x7fa90c1030b0 2024-07-04T08:22:44.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.543+0000 7fa911600640 1 -- 172.21.15.39:0/3495391155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90c1a1e60 con 0x7fa90c1030b0 2024-07-04T08:22:44.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.543+0000 7fa903400640 1 -- 172.21.15.39:0/3495391155 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc010690 con 0x7fa90c1030b0 2024-07-04T08:22:44.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.543+0000 7fa911600640 1 -- 172.21.15.39:0/3495391155 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90c1a85c0 con 0x7fa90c1030b0 2024-07-04T08:22:44.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.547+0000 7fa917c83740 1 Processor -- start 2024-07-04T08:22:44.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.547+0000 7fa917c83740 1 -- start start 2024-07-04T08:22:44.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.547+0000 7fa917c83740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db6803b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.547+0000 7fa917c83740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c6e1b0 0x55db6803bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.547+0000 7fa903400640 1 -- 172.21.15.39:0/3495391155 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa8fc004e90 con 0x7fa90c1030b0 2024-07-04T08:22:44.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.548+0000 7fa917c83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c76390 0x55db6803c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.548+0000 7fa917c83740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55db67fb1ee0 con 0x55db67c6e1b0 2024-07-04T08:22:44.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.548+0000 7fa917c83740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55db67fa1480 con 0x55db67c97070 2024-07-04T08:22:44.542 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.548+0000 7fa917c83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55db67fa54c0 con 0x55db67c76390 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.548+0000 7fa90aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c6e1b0 0x55db6803bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.548+0000 7fa90b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db6803b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.548+0000 7fa90b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db6803b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:42270/0 (socket says 172.21.15.39:42270) 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.548+0000 7fa90b400640 1 -- 172.21.15.39:0/3308320348 learned_addr learned my addr 172.21.15.39:0/3308320348 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.548+0000 7fa90a000640 1 --2- 172.21.15.39:0/3308320348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c76390 0x55db6803c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa90a000640 1 -- 172.21.15.39:0/3308320348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 msgr2=0x55db6803b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa90a000640 1 --2- 172.21.15.39:0/3308320348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db6803b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa90a000640 1 -- 172.21.15.39:0/3308320348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c6e1b0 msgr2=0x55db6803bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa90a000640 1 --2- 172.21.15.39:0/3308320348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c6e1b0 0x55db6803bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.543 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa90a000640 1 -- 172.21.15.39:0/3308320348 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db67f39650 con 0x55db67c76390 2024-07-04T08:22:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa90aa00640 1 --2- 172.21.15.39:0/3308320348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c6e1b0 0x55db6803bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:44.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa90a000640 1 --2- 172.21.15.39:0/3308320348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c76390 0x55db6803c230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa8f802a910 tx=0x7fa8f8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=280c74bef3582897 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa8f7e00640 1 -- 172.21.15.39:0/3308320348 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8040070 con 0x55db67c76390 2024-07-04T08:22:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa8f7e00640 1 -- 172.21.15.39:0/3308320348 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8f8031720 con 0x55db67c76390 2024-07-04T08:22:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.549+0000 7fa8f7e00640 1 -- 172.21.15.39:0/3308320348 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8035040 con 0x55db67c76390 2024-07-04T08:22:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.550+0000 7fa903400640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8f00b29e0 0x7fa8f00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.550+0000 7fa903400640 1 -- 172.21.15.39:0/3495391155 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7fa8fc0e2e00 con 0x7fa90c1030b0 2024-07-04T08:22:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.550+0000 7fa90b400640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8f00b29e0 0x7fa8f00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa917c83740 1 -- 172.21.15.39:0/3308320348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c76390 msgr2=0x55db6803c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa917c83740 1 --2- 172.21.15.39:0/3308320348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c76390 0x55db6803c230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa8f802a910 tx=0x7fa8f8000bc0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa90b400640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8f00b29e0 0x7fa8f00b4ef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa904005c40 tx=0x7fa904009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa917c83740 1 -- 172.21.15.39:0/3308320348 shutdown_connections 2024-07-04T08:22:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa917c83740 1 --2- 172.21.15.39:0/3308320348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c6e1b0 0x55db6803bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa917c83740 1 --2- 172.21.15.39:0/3308320348 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c76390 0x55db6803c230 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa917c83740 1 --2- 172.21.15.39:0/3308320348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db6803b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa917c83740 1 -- 172.21.15.39:0/3308320348 >> 172.21.15.39:0/3308320348 conn(0x55db67c8b460 msgr2=0x55db680396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa917c83740 1 -- 172.21.15.39:0/3308320348 shutdown_connections 2024-07-04T08:22:44.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.551+0000 7fa917c83740 1 -- 172.21.15.39:0/3308320348 wait complete. 2024-07-04T08:22:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.552+0000 7fa917c83740 1 -- start start 2024-07-04T08:22:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.552+0000 7fa917c83740 1 Processor -- start 2024-07-04T08:22:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa917c83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db68047320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa917c83740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68047860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa917c83740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db68053250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa917c83740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55db67e05700 con 0x55db67c76390 2024-07-04T08:22:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa917c83740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55db67f48bb0 con 0x55db67c97070 2024-07-04T08:22:44.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa917c83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55db68049f30 con 0x55db67c6e1b0 2024-07-04T08:22:44.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa917c83740 10 client.0 _ll_register_callbacks cb 0x55db67f9a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:44.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa90aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db68047320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa90a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68047860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa90aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db68047320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39176/0 (socket says 172.21.15.39:39176) 2024-07-04T08:22:44.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa90aa00640 1 -- 172.21.15.39:0/4189212161 learned_addr learned my addr 172.21.15.39:0/4189212161 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:44.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.553+0000 7fa90b400640 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db68053250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa90aa00640 1 -- 172.21.15.39:0/4189212161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 msgr2=0x55db68053250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:44.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa90aa00640 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db68053250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa90aa00640 1 -- 172.21.15.39:0/4189212161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 msgr2=0x55db68047860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:44.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa90aa00640 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68047860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa90aa00640 1 -- 172.21.15.39:0/4189212161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db67d02650 con 0x55db67c6e1b0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa90aa00640 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db68047320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e9c0 tx=0x7fa8fc0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=468232a98190b06b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa8f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa8f4c00640 1 -- 172.21.15.39:0/4189212161 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0a9490 con 0x55db67c6e1b0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa8f4c00640 1 -- 172.21.15.39:0/4189212161 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7fa8fc0b26f0 con 0x55db67c6e1b0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.554+0000 7fa917c83740 1 -- 172.21.15.39:0/4189212161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f8031720 con 0x55db67c6e1b0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa8f4c00640 1 -- 172.21.15.39:0/4189212161 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8fc0e0620 con 0x55db67c6e1b0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa917c83740 10 client.27289 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa917c83740 1 -- 172.21.15.39:0/4189212161 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55db68055f20 con 0x55db67c6e1b0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa8f4c00640 1 -- 172.21.15.39:0/4189212161 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8f8031720 con 0x55db67c6e1b0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa8f4c00640 1 -- 172.21.15.39:0/4189212161 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa8fc0a8f80 con 0x55db67c6e1b0 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa917c83740 10 client.27289 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:44.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa917c83740 10 client.27289 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:22:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa917c83740 20 client.27289 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa917c83740 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55db67c559f0 0x55db68059450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa917c83740 4 client.27289 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:44.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.555+0000 7fa917c83740 1 -- 172.21.15.39:0/4189212161 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db67fb3e30 con 0x55db67c559f0 2024-07-04T08:22:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.556+0000 7fa90a000640 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55db67c559f0 0x55db68059450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.557+0000 7fa90a000640 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55db67c559f0 0x55db68059450 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea6b948c5020570f server_cookie=56c983fd34cc3f66 in_seq=0 out_seq=0 2024-07-04T08:22:44.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.557+0000 7fa8f4c00640 10 client.27289 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa8f4c00640 1 -- 172.21.15.39:0/4189212161 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db67fb3e30 con 0x55db67c559f0 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa8f4c00640 10 client.27289 handle_command_reply: tid=0 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 client.27289 shutdown 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 -- 172.21.15.39:0/4189212161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 msgr2=0x55db68047320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db68047320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e9c0 tx=0x7fa8fc0048f0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 -- 172.21.15.39:0/4189212161 shutdown_connections 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55db67c559f0 0x55db68059450 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68047860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db68053250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 --2- 172.21.15.39:0/4189212161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db68047320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa8f4c00640 0 client.27289 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:22:44.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 -- 172.21.15.39:0/4189212161 >> 172.21.15.39:0/4189212161 conn(0x55db67c8b460 msgr2=0x55db6803e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:44.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 -- 172.21.15.39:0/4189212161 shutdown_connections 2024-07-04T08:22:44.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.558+0000 7fa917c83740 1 -- 172.21.15.39:0/4189212161 wait complete. 2024-07-04T08:22:44.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.559+0000 7fa917c83740 20 client.27289 trim_cache size 0 max 16384 2024-07-04T08:22:44.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.574+0000 7fa917c83740 1 Processor -- start 2024-07-04T08:22:44.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.574+0000 7fa917c83740 1 -- start start 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa917c83740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c6e1b0 0x55db6803cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa917c83740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68053040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa917c83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c97070 0x55db68053580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa917c83740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b24a0 con 0x55db67c76390 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa917c83740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b3fe0 con 0x55db67c6e1b0 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa917c83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b2b70 con 0x55db67c97070 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa90a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68053040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa90a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68053040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48604/0 (socket says 172.21.15.39:48604) 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa90aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c6e1b0 0x55db6803cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa90a000640 1 -- 172.21.15.39:0/3363112412 learned_addr learned my addr 172.21.15.39:0/3363112412 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:44.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.575+0000 7fa90b400640 1 --2- 172.21.15.39:0/3363112412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c97070 0x55db68053580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.576+0000 7fa90aa00640 1 -- 172.21.15.39:0/3363112412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c97070 msgr2=0x55db68053580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:44.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.576+0000 7fa90aa00640 1 --2- 172.21.15.39:0/3363112412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c97070 0x55db68053580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.576+0000 7fa90aa00640 1 -- 172.21.15.39:0/3363112412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 msgr2=0x55db68053040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:44.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.576+0000 7fa90aa00640 1 --2- 172.21.15.39:0/3363112412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68053040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.576+0000 7fa90aa00640 1 -- 172.21.15.39:0/3363112412 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db68049700 con 0x55db67c6e1b0 2024-07-04T08:22:44.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.576+0000 7fa90a000640 1 --2- 172.21.15.39:0/3363112412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68053040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:44.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.576+0000 7fa90aa00640 1 --2- 172.21.15.39:0/3363112412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c6e1b0 0x55db6803cef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0a6f90 tx=0x7fa8fc0a5d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1137a71274bfa7d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:44.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.576+0000 7fa8f7400640 1 -- 172.21.15.39:0/3363112412 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0c0070 con 0x55db67c6e1b0 2024-07-04T08:22:44.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.577+0000 7fa8f7400640 1 -- 172.21.15.39:0/3363112412 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8fc0c8070 con 0x55db67c6e1b0 2024-07-04T08:22:44.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.577+0000 7fa8f7400640 1 -- 172.21.15.39:0/3363112412 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc005180 con 0x55db67c6e1b0 2024-07-04T08:22:44.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.577+0000 7fa917c83740 1 -- 172.21.15.39:0/3363112412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c6e1b0 msgr2=0x55db6803cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:44.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.577+0000 7fa917c83740 1 --2- 172.21.15.39:0/3363112412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c6e1b0 0x55db6803cef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0a6f90 tx=0x7fa8fc0a5d10 comp rx=0 tx=0).stop 2024-07-04T08:22:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.578+0000 7fa917c83740 1 -- 172.21.15.39:0/3363112412 shutdown_connections 2024-07-04T08:22:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.578+0000 7fa917c83740 1 --2- 172.21.15.39:0/3363112412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db68053040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.578+0000 7fa917c83740 1 --2- 172.21.15.39:0/3363112412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c97070 0x55db68053580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.578+0000 7fa917c83740 1 --2- 172.21.15.39:0/3363112412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c6e1b0 0x55db6803cef0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.578+0000 7fa917c83740 1 -- 172.21.15.39:0/3363112412 >> 172.21.15.39:0/3363112412 conn(0x55db67c8b460 msgr2=0x55db6803c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.578+0000 7fa917c83740 1 -- 172.21.15.39:0/3363112412 shutdown_connections 2024-07-04T08:22:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.578+0000 7fa917c83740 1 -- 172.21.15.39:0/3363112412 wait complete. 2024-07-04T08:22:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.578+0000 7fa917c83740 1 -- start start 2024-07-04T08:22:44.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.579+0000 7fa917c83740 1 Processor -- start 2024-07-04T08:22:44.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.579+0000 7fa917c83740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db6804f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.579+0000 7fa917c83740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db6804ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.579+0000 7fa917c83740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db6804b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.579+0000 7fa917c83740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b3910 con 0x55db67c76390 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.579+0000 7fa917c83740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b3240 con 0x55db67c97070 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.579+0000 7fa917c83740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa90c1b0480 con 0x55db67c6e1b0 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.579+0000 7fa917c83740 10 client.0 _ll_register_callbacks cb 0x7fa8fc0e65a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.579+0000 7fa90a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db6804ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db6804ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48612/0 (socket says 172.21.15.39:48612) 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90a000640 1 -- 172.21.15.39:0/1342078596 learned_addr learned my addr 172.21.15.39:0/1342078596 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90aa00640 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db6804f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90b400640 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db6804b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90aa00640 1 -- 172.21.15.39:0/1342078596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 msgr2=0x55db6804b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:44.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90aa00640 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db6804b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90aa00640 1 -- 172.21.15.39:0/1342078596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 msgr2=0x55db6804ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90aa00640 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db6804ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90aa00640 1 -- 172.21.15.39:0/1342078596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db6804bc80 con 0x55db67c6e1b0 2024-07-04T08:22:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa90aa00640 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db6804f9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0a6f90 tx=0x7fa8fc0c4320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1808e0417d7dc6b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.580+0000 7fa8f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.581+0000 7fa8f4c00640 1 -- 172.21.15.39:0/1342078596 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0c0070 con 0x55db67c6e1b0 2024-07-04T08:22:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.581+0000 7fa8f4c00640 1 -- 172.21.15.39:0/1342078596 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7fa8fc0b26f0 con 0x55db67c6e1b0 2024-07-04T08:22:44.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.581+0000 7fa917c83740 1 -- 172.21.15.39:0/1342078596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8fc0c8070 con 0x55db67c6e1b0 2024-07-04T08:22:44.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.581+0000 7fa8f4c00640 1 -- 172.21.15.39:0/1342078596 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8fc005180 con 0x55db67c6e1b0 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.581+0000 7fa917c83740 10 client.27295 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.581+0000 7fa917c83740 1 -- 172.21.15.39:0/1342078596 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55db680c1cb0 con 0x55db67c6e1b0 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.581+0000 7fa8f4c00640 1 -- 172.21.15.39:0/1342078596 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8fc0c8070 con 0x55db67c6e1b0 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.582+0000 7fa8f4c00640 1 -- 172.21.15.39:0/1342078596 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa8fc0af260 con 0x55db67c6e1b0 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.582+0000 7fa917c83740 10 client.27295 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.582+0000 7fa917c83740 10 client.27295 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.582+0000 7fa917c83740 20 client.27295 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.582+0000 7fa917c83740 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55db67c2ada0 0x55db680c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.582+0000 7fa917c83740 4 client.27295 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.582+0000 7fa917c83740 1 -- 172.21.15.39:0/1342078596 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db67faf8d0 con 0x55db67c2ada0 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.582+0000 7fa90a000640 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55db67c2ada0 0x55db680c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:44.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.583+0000 7fa90a000640 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55db67c2ada0 0x55db680c3fb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87fe2b33df962e6c server_cookie=6fd4efec4a551ad2 in_seq=0 out_seq=0 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.583+0000 7fa8f4c00640 10 client.27295 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.583+0000 7fa8f4c00640 1 -- 172.21.15.39:0/1342078596 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db67faf8d0 con 0x55db67c2ada0 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.583+0000 7fa8f4c00640 10 client.27295 handle_command_reply: tid=0 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.583+0000 7fa917c83740 1 client.27295 shutdown 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.583+0000 7fa917c83740 1 -- 172.21.15.39:0/1342078596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 msgr2=0x55db6804f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.583+0000 7fa917c83740 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db6804f9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0a6f90 tx=0x7fa8fc0c4320 comp rx=0 tx=0).stop 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa917c83740 1 -- 172.21.15.39:0/1342078596 shutdown_connections 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa917c83740 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55db67c2ada0 0x55db680c3fb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa917c83740 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55db67c76390 0x55db6804ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa917c83740 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55db67c97070 0x55db6804b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa917c83740 1 --2- 172.21.15.39:0/1342078596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db67c6e1b0 0x55db6804f9c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa917c83740 1 -- 172.21.15.39:0/1342078596 >> 172.21.15.39:0/1342078596 conn(0x55db67c8b460 msgr2=0x55db6803c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa8f4c00640 0 client.27295 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:22:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa917c83740 1 -- 172.21.15.39:0/1342078596 shutdown_connections 2024-07-04T08:22:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa917c83740 1 -- 172.21.15.39:0/1342078596 wait complete. 2024-07-04T08:22:44.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.584+0000 7fa917c83740 20 client.27295 trim_cache size 0 max 16384 2024-07-04T08:22:44.580 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 -- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8f00b29e0 msgr2=0x7fa8f00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8f00b29e0 0x7fa8f00b4ef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa904005c40 tx=0x7fa904009500 comp rx=0 tx=0).stop 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 -- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90c1030b0 msgr2=0x7fa90c19bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90c1030b0 0x7fa90c19bc00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00e9f0 tx=0x7fa8fc00edf0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 -- 172.21.15.39:0/3495391155 shutdown_connections 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8f00b29e0 0x7fa8f00b4ef0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa90c109340 0x7fa90c1a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa90c1089f0 0x7fa90c19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 --2- 172.21.15.39:0/3495391155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90c1030b0 0x7fa90c19bc00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:44.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 -- 172.21.15.39:0/3495391155 >> 172.21.15.39:0/3495391155 conn(0x7fa90c07ada0 msgr2=0x7fa90c104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:44.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 -- 172.21.15.39:0/3495391155 shutdown_connections 2024-07-04T08:22:44.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:44.585+0000 7fa911600640 1 -- 172.21.15.39:0/3495391155 wait complete. 2024-07-04T08:22:45.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:45 smithi039 ceph-mon[32062]: pgmap v708: 129 pgs: 129 active+clean; 8.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-07-04T08:22:45.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:45 smithi050 ceph-mon[36577]: pgmap v708: 129 pgs: 129 active+clean; 8.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-07-04T08:22:45.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:22:45.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:45 smithi133 ceph-mon[37050]: pgmap v708: 129 pgs: 129 active+clean; 8.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-07-04T08:22:46.000 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:47.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:47 smithi039 ceph-mon[32062]: pgmap v709: 129 pgs: 129 active+clean; 9.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-07-04T08:22:47.360 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:47 smithi133 ceph-mon[37050]: pgmap v709: 129 pgs: 129 active+clean; 9.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-07-04T08:22:47.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:47 smithi050 ceph-mon[36577]: pgmap v709: 129 pgs: 129 active+clean; 9.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 17 op/s 2024-07-04T08:22:48.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.570+0000 7f634be00640 1 -- 172.21.15.39:0/3699673694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c10f4a0 msgr2=0x7f634c1118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:48.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.570+0000 7f634be00640 1 --2- 172.21.15.39:0/3699673694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c10f4a0 0x7f634c1118c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6344009ca0 tx=0x7f634402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.570+0000 7f634be00640 1 -- 172.21.15.39:0/3699673694 shutdown_connections 2024-07-04T08:22:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.570+0000 7f634be00640 1 --2- 172.21.15.39:0/3699673694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c10f4a0 0x7f634c1118c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.570+0000 7f634be00640 1 --2- 172.21.15.39:0/3699673694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c0755f0 0x7f634c10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.570+0000 7f634be00640 1 --2- 172.21.15.39:0/3699673694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634c074ca0 0x7f634c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.570+0000 7f634be00640 1 -- 172.21.15.39:0/3699673694 >> 172.21.15.39:0/3699673694 conn(0x7f634c0fe930 msgr2=0x7f634c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.570+0000 7f634be00640 1 -- 172.21.15.39:0/3699673694 shutdown_connections 2024-07-04T08:22:48.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.571+0000 7f634be00640 1 -- 172.21.15.39:0/3699673694 wait complete. 2024-07-04T08:22:48.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.572+0000 7f634be00640 1 Processor -- start 2024-07-04T08:22:48.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.572+0000 7f634be00640 1 -- start start 2024-07-04T08:22:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c074ca0 0x7f634c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634c0755f0 0x7f634c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c10f4a0 0x7f634c1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f634c115050 con 0x7f634c0755f0 2024-07-04T08:22:48.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f634c114ed0 con 0x7f634c10f4a0 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1151d0 con 0x7f634c074ca0 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c074ca0 0x7f634c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634c0755f0 0x7f634c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c074ca0 0x7f634c19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39208/0 (socket says 172.21.15.39:39208) 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634aa00640 1 -- 172.21.15.39:0/3181869862 learned_addr learned my addr 172.21.15.39:0/3181869862 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634b400640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c10f4a0 0x7f634c1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634aa00640 1 -- 172.21.15.39:0/3181869862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c10f4a0 msgr2=0x7f634c1a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634aa00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c10f4a0 0x7f634c1a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634aa00640 1 -- 172.21.15.39:0/3181869862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634c0755f0 msgr2=0x7f634c19bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634aa00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634c0755f0 0x7f634c19bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.573+0000 7f634aa00640 1 -- 172.21.15.39:0/3181869862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c1a18f0 con 0x7f634c074ca0 2024-07-04T08:22:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.574+0000 7f634aa00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c074ca0 0x7f634c19b980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e9f0 tx=0x7f633c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.574+0000 7f6350a00640 1 -- 172.21.15.39:0/3181869862 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c00cd80 con 0x7f634c074ca0 2024-07-04T08:22:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.574+0000 7f634be00640 1 -- 172.21.15.39:0/3181869862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c1a1be0 con 0x7f634c074ca0 2024-07-04T08:22:48.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.574+0000 7f6350a00640 1 -- 172.21.15.39:0/3181869862 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633c0108e0 con 0x7f634c074ca0 2024-07-04T08:22:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.574+0000 7f6350a00640 1 -- 172.21.15.39:0/3181869862 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c018980 con 0x7f634c074ca0 2024-07-04T08:22:48.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.574+0000 7f634be00640 1 -- 172.21.15.39:0/3181869862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c1a8340 con 0x7f634c074ca0 2024-07-04T08:22:48.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.578+0000 7f6358440740 1 Processor -- start 2024-07-04T08:22:48.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.578+0000 7f6358440740 1 -- start start 2024-07-04T08:22:48.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.578+0000 7f6358440740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 0x564b5606c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.578+0000 7f6358440740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 0x564b5606ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.578+0000 7f6358440740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b5606d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.578+0000 7f6358440740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564b55fe2ee0 con 0x564b55c9f1b0 2024-07-04T08:22:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.578+0000 7f6358440740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564b55fd2480 con 0x564b55cc8070 2024-07-04T08:22:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.578+0000 7f6358440740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b55fd64c0 con 0x564b55ca7390 2024-07-04T08:22:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.578+0000 7f6350a00640 1 -- 172.21.15.39:0/3181869862 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f633c010430 con 0x7f634c074ca0 2024-07-04T08:22:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b5606d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b5606d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39220/0 (socket says 172.21.15.39:39220) 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634a000640 1 -- 172.21.15.39:0/192772107 learned_addr learned my addr 172.21.15.39:0/192772107 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 0x564b5606c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634aa00640 1 --2- 172.21.15.39:0/192772107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 0x564b5606ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634a000640 1 -- 172.21.15.39:0/192772107 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 msgr2=0x564b5606c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634a000640 1 --2- 172.21.15.39:0/192772107 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 0x564b5606c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634a000640 1 -- 172.21.15.39:0/192772107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 msgr2=0x564b5606ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634a000640 1 --2- 172.21.15.39:0/192772107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 0x564b5606ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.579+0000 7f634a000640 1 -- 172.21.15.39:0/192772107 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b55f6a650 con 0x564b55ca7390 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.580+0000 7f634b400640 1 --2- 172.21.15.39:0/192772107 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 0x564b5606c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.580+0000 7f634a000640 1 --2- 172.21.15.39:0/192772107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b5606d3d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6338009c20 tx=0x7f6338011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=260ddd22982b3dce server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:48.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.580+0000 7f6340c00640 1 -- 172.21.15.39:0/192772107 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338004230 con 0x564b55ca7390 2024-07-04T08:22:48.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.580+0000 7f6340c00640 1 -- 172.21.15.39:0/192772107 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63380043d0 con 0x564b55ca7390 2024-07-04T08:22:48.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.580+0000 7f6340c00640 1 -- 172.21.15.39:0/192772107 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338004d80 con 0x564b55ca7390 2024-07-04T08:22:48.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.580+0000 7f6350a00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63300b2960 0x7f63300b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f6350a00640 1 -- 172.21.15.39:0/3181869862 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f633c0e2db0 con 0x7f634c074ca0 2024-07-04T08:22:48.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f634b400640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63300b2960 0x7f63300b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f6358440740 1 -- 172.21.15.39:0/192772107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 msgr2=0x564b5606d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:48.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f6358440740 1 --2- 172.21.15.39:0/192772107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b5606d3d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6338009c20 tx=0x7f6338011810 comp rx=0 tx=0).stop 2024-07-04T08:22:48.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f6358440740 1 -- 172.21.15.39:0/192772107 shutdown_connections 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f6358440740 1 --2- 172.21.15.39:0/192772107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 0x564b5606ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f6358440740 1 --2- 172.21.15.39:0/192772107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b5606d3d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f6358440740 1 --2- 172.21.15.39:0/192772107 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 0x564b5606c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f6358440740 1 -- 172.21.15.39:0/192772107 >> 172.21.15.39:0/192772107 conn(0x564b55cbc460 msgr2=0x564b5606a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.581+0000 7f6358440740 1 -- 172.21.15.39:0/192772107 shutdown_connections 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.582+0000 7f6358440740 1 -- 172.21.15.39:0/192772107 wait complete. 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.582+0000 7f634b400640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63300b2960 0x7f63300b4e70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6344004af0 tx=0x7f6344005da0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.582+0000 7f6358440740 1 -- start start 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.582+0000 7f6358440740 1 Processor -- start 2024-07-04T08:22:48.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.583+0000 7f6358440740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 0x564b56078510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.583+0000 7f6358440740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b56078a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.583+0000 7f6358440740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 0x564b56084420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.583+0000 7f6358440740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564b55e36700 con 0x564b55c9f1b0 2024-07-04T08:22:48.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.583+0000 7f6358440740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564b55f79bb0 con 0x564b55cc8070 2024-07-04T08:22:48.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.583+0000 7f6358440740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b5607b130 con 0x564b55ca7390 2024-07-04T08:22:48.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.583+0000 7f6358440740 10 client.0 _ll_register_callbacks cb 0x564b55fcb570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:48.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 0x564b56078510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b56078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 0x564b56078510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48638/0 (socket says 172.21.15.39:48638) 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634aa00640 1 -- 172.21.15.39:0/3239994006 learned_addr learned my addr 172.21.15.39:0/3239994006 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 0x564b56084420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b56078a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39222/0 (socket says 172.21.15.39:39222) 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634a000640 1 -- 172.21.15.39:0/3239994006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 msgr2=0x564b56084420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634a000640 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 0x564b56084420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634a000640 1 -- 172.21.15.39:0/3239994006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 msgr2=0x564b56078510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634a000640 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 0x564b56078510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.584+0000 7f634a000640 1 -- 172.21.15.39:0/3239994006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b55d33650 con 0x564b55ca7390 2024-07-04T08:22:48.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.585+0000 7f634a000640 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b56078a50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6338004920 tx=0x7f6338019b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624f10ca6073e579 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.585+0000 7f6335600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.585+0000 7f6335600640 1 -- 172.21.15.39:0/3239994006 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338028020 con 0x564b55ca7390 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.585+0000 7f6358440740 1 -- 172.21.15.39:0/3239994006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63380043d0 con 0x564b55ca7390 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.585+0000 7f6335600640 1 -- 172.21.15.39:0/3239994006 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f6338024620 con 0x564b55ca7390 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.585+0000 7f6335600640 1 -- 172.21.15.39:0/3239994006 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6338033050 con 0x564b55ca7390 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.585+0000 7f6358440740 10 client.27307 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.585+0000 7f6358440740 1 -- 172.21.15.39:0/3239994006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b560870f0 con 0x564b55ca7390 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.586+0000 7f6335600640 1 -- 172.21.15.39:0/3239994006 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63380043d0 con 0x564b55ca7390 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.586+0000 7f6335600640 1 -- 172.21.15.39:0/3239994006 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6338018070 con 0x564b55ca7390 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.586+0000 7f6358440740 10 client.27307 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.586+0000 7f6358440740 10 client.27307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:48.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.586+0000 7f6358440740 20 client.27307 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:48.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.586+0000 7f6358440740 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b55c869f0 0x564b5608a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.586+0000 7f6358440740 4 client.27307 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:48.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.586+0000 7f6358440740 1 -- 172.21.15.39:0/3239994006 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b55fe4e30 con 0x564b55c869f0 2024-07-04T08:22:48.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.587+0000 7f634aa00640 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b55c869f0 0x564b5608a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.587+0000 7f634aa00640 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b55c869f0 0x564b5608a450 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e5ebdd5428da8a server_cookie=e8cf729c73a0f7ae in_seq=0 out_seq=0 2024-07-04T08:22:48.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.587+0000 7f6335600640 10 client.27307 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:48.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.588+0000 7f6335600640 1 -- 172.21.15.39:0/3239994006 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564b55fe4e30 con 0x564b55c869f0 2024-07-04T08:22:48.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.588+0000 7f6335600640 10 client.27307 handle_command_reply: tid=0 2024-07-04T08:22:48.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.588+0000 7f6358440740 1 client.27307 shutdown 2024-07-04T08:22:48.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.588+0000 7f6358440740 1 -- 172.21.15.39:0/3239994006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 msgr2=0x564b56078a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:48.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.588+0000 7f6358440740 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b56078a50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6338004920 tx=0x7f6338019b90 comp rx=0 tx=0).stop 2024-07-04T08:22:48.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6358440740 1 -- 172.21.15.39:0/3239994006 shutdown_connections 2024-07-04T08:22:48.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6358440740 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b55c869f0 0x564b5608a450 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6358440740 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55cc8070 0x564b56084420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6358440740 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55ca7390 0x564b56078a50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6358440740 1 --2- 172.21.15.39:0/3239994006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55c9f1b0 0x564b56078510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6335600640 0 client.27307 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:48.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6358440740 1 -- 172.21.15.39:0/3239994006 >> 172.21.15.39:0/3239994006 conn(0x564b55cbc460 msgr2=0x564b5606fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:48.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6358440740 1 -- 172.21.15.39:0/3239994006 shutdown_connections 2024-07-04T08:22:48.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6358440740 1 -- 172.21.15.39:0/3239994006 wait complete. 2024-07-04T08:22:48.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.589+0000 7f6358440740 20 client.27307 trim_cache size 0 max 16384 2024-07-04T08:22:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.605+0000 7f6358440740 1 Processor -- start 2024-07-04T08:22:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.605+0000 7f6358440740 1 -- start start 2024-07-04T08:22:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.605+0000 7f6358440740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55c9f1b0 0x564b5606a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.605+0000 7f6358440740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55ca7390 0x564b56084210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.605+0000 7f6358440740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55cc8070 0x564b5608fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.605+0000 7f6358440740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b23c0 con 0x564b55cc8070 2024-07-04T08:22:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.605+0000 7f6358440740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b3f00 con 0x564b55ca7390 2024-07-04T08:22:48.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.605+0000 7f6358440740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b2a90 con 0x564b55c9f1b0 2024-07-04T08:22:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55ca7390 0x564b56084210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55cc8070 0x564b5608fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634a000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55ca7390 0x564b56084210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:42328/0 (socket says 172.21.15.39:42328) 2024-07-04T08:22:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634a000640 1 -- 172.21.15.39:0/1032799740 learned_addr learned my addr 172.21.15.39:0/1032799740 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:48.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634aa00640 1 --2- 172.21.15.39:0/1032799740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55c9f1b0 0x564b5606a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634a000640 1 -- 172.21.15.39:0/1032799740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55c9f1b0 msgr2=0x564b5606a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634a000640 1 --2- 172.21.15.39:0/1032799740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55c9f1b0 0x564b5606a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634a000640 1 -- 172.21.15.39:0/1032799740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55cc8070 msgr2=0x564b5608fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634a000640 1 --2- 172.21.15.39:0/1032799740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55cc8070 0x564b5608fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634a000640 1 -- 172.21.15.39:0/1032799740 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b5606ec40 con 0x564b55ca7390 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634aa00640 1 --2- 172.21.15.39:0/1032799740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55c9f1b0 0x564b5606a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.606+0000 7f634a000640 1 --2- 172.21.15.39:0/1032799740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55ca7390 0x564b56084210 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6338018380 tx=0x7f6338011e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa64d42679b8d99b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.607+0000 7f6337e00640 1 -- 172.21.15.39:0/1032799740 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633803c070 con 0x564b55ca7390 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.607+0000 7f6337e00640 1 -- 172.21.15.39:0/1032799740 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633801f040 con 0x564b55ca7390 2024-07-04T08:22:48.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.607+0000 7f6337e00640 1 -- 172.21.15.39:0/1032799740 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338028040 con 0x564b55ca7390 2024-07-04T08:22:48.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 -- 172.21.15.39:0/1032799740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55ca7390 msgr2=0x564b56084210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:48.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 --2- 172.21.15.39:0/1032799740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55ca7390 0x564b56084210 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6338018380 tx=0x7f6338011e40 comp rx=0 tx=0).stop 2024-07-04T08:22:48.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 -- 172.21.15.39:0/1032799740 shutdown_connections 2024-07-04T08:22:48.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 --2- 172.21.15.39:0/1032799740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55cc8070 0x564b5608fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 --2- 172.21.15.39:0/1032799740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55ca7390 0x564b56084210 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 --2- 172.21.15.39:0/1032799740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55c9f1b0 0x564b5606a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 -- 172.21.15.39:0/1032799740 >> 172.21.15.39:0/1032799740 conn(0x564b55cbc460 msgr2=0x564b5606a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:48.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 -- 172.21.15.39:0/1032799740 shutdown_connections 2024-07-04T08:22:48.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 -- 172.21.15.39:0/1032799740 wait complete. 2024-07-04T08:22:48.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.608+0000 7f6358440740 1 -- start start 2024-07-04T08:22:48.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.609+0000 7f6358440740 1 Processor -- start 2024-07-04T08:22:48.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.609+0000 7f6358440740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55c9f1b0 0x564b56080700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.609+0000 7f6358440740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55ca7390 0x564b5607bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.609+0000 7f6358440740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55cc8070 0x564b5607c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.609+0000 7f6358440740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b3830 con 0x564b55ca7390 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.609+0000 7f6358440740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b3160 con 0x564b55c9f1b0 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.609+0000 7f6358440740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b03a0 con 0x564b55cc8070 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.609+0000 7f6358440740 10 client.0 _ll_register_callbacks cb 0x7f633c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55ca7390 0x564b5607bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55ca7390 0x564b5607bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48646/0 (socket says 172.21.15.39:48646) 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55c9f1b0 0x564b56080700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634a000640 1 -- 172.21.15.39:0/1609739408 learned_addr learned my addr 172.21.15.39:0/1609739408 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634b400640 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55cc8070 0x564b5607c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634aa00640 1 -- 172.21.15.39:0/1609739408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55cc8070 msgr2=0x564b5607c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634aa00640 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55cc8070 0x564b5607c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634aa00640 1 -- 172.21.15.39:0/1609739408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55ca7390 msgr2=0x564b5607bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634aa00640 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55ca7390 0x564b5607bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634aa00640 1 -- 172.21.15.39:0/1609739408 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b5607c680 con 0x564b55c9f1b0 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634a000640 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55ca7390 0x564b5607bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.610+0000 7f634b400640 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55cc8070 0x564b5607c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.611+0000 7f634aa00640 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55c9f1b0 0x564b56080700 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e9c0 tx=0x7f633c018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8108f278e26731 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.611+0000 7f6335600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.611+0000 7f6335600640 1 -- 172.21.15.39:0/1609739408 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c0b2b30 con 0x564b55c9f1b0 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.611+0000 7f6335600640 1 -- 172.21.15.39:0/1609739408 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f633c0a5f60 con 0x564b55c9f1b0 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.611+0000 7f6358440740 1 -- 172.21.15.39:0/1609739408 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f633801f040 con 0x564b55c9f1b0 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.611+0000 7f6335600640 1 -- 172.21.15.39:0/1609739408 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f633c0e03c0 con 0x564b55c9f1b0 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.611+0000 7f6358440740 10 client.27392 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:48.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.611+0000 7f6358440740 1 -- 172.21.15.39:0/1609739408 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b560f2730 con 0x564b55c9f1b0 2024-07-04T08:22:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.611+0000 7f6335600640 1 -- 172.21.15.39:0/1609739408 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f633c0c1070 con 0x564b55c9f1b0 2024-07-04T08:22:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.612+0000 7f6335600640 1 -- 172.21.15.39:0/1609739408 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f633c018b20 con 0x564b55c9f1b0 2024-07-04T08:22:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.612+0000 7f6358440740 10 client.27392 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.612+0000 7f6358440740 10 client.27392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.612+0000 7f6358440740 20 client.27392 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.612+0000 7f6358440740 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b55c5bda0 0x564b560f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.612+0000 7f6358440740 4 client.27392 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:22:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.612+0000 7f6358440740 1 -- 172.21.15.39:0/1609739408 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b55fe08d0 con 0x564b55c5bda0 2024-07-04T08:22:48.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.612+0000 7f634a000640 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b55c5bda0 0x564b560f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:48.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f634a000640 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b55c5bda0 0x564b560f4f40 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b4eb17d699346e9 server_cookie=7954a65663981d34 in_seq=0 out_seq=0 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6335600640 10 client.27392 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6335600640 1 -- 172.21.15.39:0/1609739408 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b55fe08d0 con 0x564b55c5bda0 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6335600640 10 client.27392 handle_command_reply: tid=0 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6358440740 1 client.27392 shutdown 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6358440740 1 -- 172.21.15.39:0/1609739408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55c9f1b0 msgr2=0x564b56080700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6358440740 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55c9f1b0 0x564b56080700 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e9c0 tx=0x7f633c018420 comp rx=0 tx=0).stop 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6358440740 1 -- 172.21.15.39:0/1609739408 shutdown_connections 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6358440740 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b55c5bda0 0x564b560f4f40 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6358440740 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b55ca7390 0x564b5607bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6358440740 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b55cc8070 0x564b5607c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6358440740 1 --2- 172.21.15.39:0/1609739408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b55c9f1b0 0x564b56080700 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6335600640 0 client.27392 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:48.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.613+0000 7f6358440740 1 -- 172.21.15.39:0/1609739408 >> 172.21.15.39:0/1609739408 conn(0x564b55cbc460 msgr2=0x564b56068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.614+0000 7f6358440740 1 -- 172.21.15.39:0/1609739408 shutdown_connections 2024-07-04T08:22:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.614+0000 7f6358440740 1 -- 172.21.15.39:0/1609739408 wait complete. 2024-07-04T08:22:48.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.614+0000 7f6358440740 20 client.27392 trim_cache size 0 max 16384 2024-07-04T08:22:48.613 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:22:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.614+0000 7f634be00640 1 -- 172.21.15.39:0/3181869862 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63300b2960 msgr2=0x7f63300b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.614+0000 7f634be00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63300b2960 0x7f63300b4e70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6344004af0 tx=0x7f6344005da0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.614+0000 7f634be00640 1 -- 172.21.15.39:0/3181869862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c074ca0 msgr2=0x7f634c19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.614+0000 7f634be00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c074ca0 0x7f634c19b980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f633c00e9f0 tx=0x7f633c00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.615+0000 7f634be00640 1 -- 172.21.15.39:0/3181869862 shutdown_connections 2024-07-04T08:22:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.615+0000 7f634be00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f63300b2960 0x7f63300b4e70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.615+0000 7f634be00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634c0755f0 0x7f634c19bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.615+0000 7f634be00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f634c10f4a0 0x7f634c1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.615+0000 7f634be00640 1 --2- 172.21.15.39:0/3181869862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f634c074ca0 0x7f634c19b980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.615+0000 7f634be00640 1 -- 172.21.15.39:0/3181869862 >> 172.21.15.39:0/3181869862 conn(0x7f634c0fe930 msgr2=0x7f634c108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.615+0000 7f634be00640 1 -- 172.21.15.39:0/3181869862 shutdown_connections 2024-07-04T08:22:48.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:48.615+0000 7f634be00640 1 -- 172.21.15.39:0/3181869862 wait complete. 2024-07-04T08:22:49.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:49 smithi039 ceph-mon[32062]: pgmap v710: 129 pgs: 129 active+clean; 9.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-07-04T08:22:49.388 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:49 smithi050 ceph-mon[36577]: pgmap v710: 129 pgs: 129 active+clean; 9.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-07-04T08:22:49.494 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.495 INFO:tasks.workunit.client.0.smithi039.stdout:189/600: open fd = 3 2024-07-04T08:22:49.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.495 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.495 INFO:tasks.workunit.client.0.smithi039.stdout:190/600: open fd = 3 2024-07-04T08:22:49.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.495 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.495 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:191/600: open fd = 3 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:192/600: open fd = 3 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:193/600: open fd = 3 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.496 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:194/600: open fd = 3 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:195/600: open fd = 3 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:196/600: open fd = 3 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.497 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.498 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.502 INFO:tasks.workunit.client.0.smithi039.stdout:197/600: open fd = 3 2024-07-04T08:22:49.504 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.504 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.504 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.505 INFO:tasks.workunit.client.0.smithi039.stdout:198/600: open fd = 3 2024-07-04T08:22:49.506 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.514 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.514 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.514 INFO:tasks.workunit.client.0.smithi039.stdout:199/600: open fd = 3 2024-07-04T08:22:49.514 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:200/600: open fd = 3 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:201/600: open fd = 3 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:202/600: open fd = 3 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.515 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:203/600: open fd = 3 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:204/600: open fd = 3 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:205/600: open fd = 3 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.516 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:206/600: open fd = 3 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:207/600: open fd = 3 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:208/600: open fd = 3 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:209/600: open fd = 3 2024-07-04T08:22:49.517 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:210/600: open fd = 3 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:211/600: open fd = 3 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:212/600: open fd = 3 2024-07-04T08:22:49.518 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:213/600: open fd = 3 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:214/600: open fd = 3 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.519 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:215/600: open fd = 3 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:216/600: open fd = 3 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:217/600: open fd = 3 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.520 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:218/600: open fd = 3 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:219/600: open fd = 3 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:220/600: open fd = 3 2024-07-04T08:22:49.521 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:221/600: open fd = 3 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:222/600: open fd = 3 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:223/600: open fd = 3 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.522 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:224/600: open fd = 3 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:225/600: open fd = 3 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:226/600: open fd = 3 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.523 INFO:tasks.workunit.client.0.smithi039.stdout:227/600: open fd = 3 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:228/600: open fd = 3 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:229/600: open fd = 3 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:230/600: open fd = 3 2024-07-04T08:22:49.524 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:231/600: open fd = 3 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:232/600: open fd = 3 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:233/600: open fd = 3 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.525 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:234/600: open fd = 3 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:235/600: open fd = 3 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:236/600: open fd = 3 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.526 INFO:tasks.workunit.client.0.smithi039.stdout:237/600: open fd = 3 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:238/600: open fd = 3 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:239/600: open fd = 3 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:240/600: open fd = 3 2024-07-04T08:22:49.527 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:241/600: open fd = 3 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:242/600: open fd = 3 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:243/600: open fd = 3 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.528 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:244/600: open fd = 3 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:245/600: open fd = 3 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:246/600: open fd = 3 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.529 INFO:tasks.workunit.client.0.smithi039.stdout:247/600: open fd = 3 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:248/600: open fd = 3 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:249/600: open fd = 3 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:250/600: open fd = 3 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.530 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:22:49.560 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:49 smithi133 ceph-mon[37050]: pgmap v710: 129 pgs: 129 active+clean; 9.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-07-04T08:22:50.532 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:22:50.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:22:51.030 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:51.374 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:51 smithi039 ceph-mon[32062]: pgmap v711: 129 pgs: 129 active+clean; 9.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-07-04T08:22:51.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:51 smithi050 ceph-mon[36577]: pgmap v711: 129 pgs: 129 active+clean; 9.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-07-04T08:22:51.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:51 smithi133 ceph-mon[37050]: pgmap v711: 129 pgs: 129 active+clean; 9.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-07-04T08:22:53.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.330+0000 7f89d5c00640 1 -- 172.21.15.39:0/2667006177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 msgr2=0x7f89d0102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:53.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.330+0000 7f89d5c00640 1 --2- 172.21.15.39:0/2667006177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 0x7f89d0102230 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009900 tx=0x7f89c002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.330+0000 7f89d5c00640 1 -- 172.21.15.39:0/2667006177 shutdown_connections 2024-07-04T08:22:53.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.330+0000 7f89d5c00640 1 --2- 172.21.15.39:0/2667006177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89d01080b0 0x7f89d010e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.330+0000 7f89d5c00640 1 --2- 172.21.15.39:0/2667006177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d0107760 0x7f89d0107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.330+0000 7f89d5c00640 1 --2- 172.21.15.39:0/2667006177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 0x7f89d0102230 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.330+0000 7f89d5c00640 1 -- 172.21.15.39:0/2667006177 >> 172.21.15.39:0/2667006177 conn(0x7f89d00fd8d0 msgr2=0x7f89d00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:53.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.330+0000 7f89d5c00640 1 -- 172.21.15.39:0/2667006177 shutdown_connections 2024-07-04T08:22:53.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.330+0000 7f89d5c00640 1 -- 172.21.15.39:0/2667006177 wait complete. 2024-07-04T08:22:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.331+0000 7f89d5c00640 1 Processor -- start 2024-07-04T08:22:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.331+0000 7f89d5c00640 1 -- start start 2024-07-04T08:22:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89d5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 0x7f89d00fde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89d5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89d0107760 0x7f89d00fe340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89d5c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d01080b0 0x7f89d00fe880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89d5c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0111d70 con 0x7f89d0101e20 2024-07-04T08:22:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89d5c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0111bf0 con 0x7f89d01080b0 2024-07-04T08:22:53.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89d5c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89d0111ef0 con 0x7f89d0107760 2024-07-04T08:22:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89cfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 0x7f89d00fde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89cfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 0x7f89d00fde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58024/0 (socket says 172.21.15.39:58024) 2024-07-04T08:22:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89cfe00640 1 -- 172.21.15.39:0/1242413052 learned_addr learned my addr 172.21.15.39:0/1242413052 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89d5200640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d01080b0 0x7f89d00fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.332+0000 7f89cf400640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89d0107760 0x7f89d00fe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.333+0000 7f89cfe00640 1 -- 172.21.15.39:0/1242413052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89d0107760 msgr2=0x7f89d00fe340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.333+0000 7f89cfe00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89d0107760 0x7f89d00fe340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.333+0000 7f89cfe00640 1 -- 172.21.15.39:0/1242413052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d01080b0 msgr2=0x7f89d00fe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.333+0000 7f89cfe00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d01080b0 0x7f89d00fe880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.333+0000 7f89cfe00640 1 -- 172.21.15.39:0/1242413052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d01a7b10 con 0x7f89d0101e20 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.333+0000 7f89d5200640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d01080b0 0x7f89d00fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.333+0000 7f89cfe00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 0x7f89d00fde00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005940 tx=0x7f89c00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.334+0000 7f89ccc00640 1 -- 172.21.15.39:0/1242413052 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c003f030 con 0x7f89d0101e20 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.334+0000 7f89ccc00640 1 -- 172.21.15.39:0/1242413052 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c00044c0 con 0x7f89d0101e20 2024-07-04T08:22:53.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.334+0000 7f89d5c00640 1 -- 172.21.15.39:0/1242413052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d01a7da0 con 0x7f89d0101e20 2024-07-04T08:22:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.334+0000 7f89ccc00640 1 -- 172.21.15.39:0/1242413052 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0035070 con 0x7f89d0101e20 2024-07-04T08:22:53.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.334+0000 7f89d5c00640 1 -- 172.21.15.39:0/1242413052 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d01a8270 con 0x7f89d0101e20 2024-07-04T08:22:53.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.337+0000 7f89dd7dd740 1 Processor -- start 2024-07-04T08:22:53.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89dd7dd740 1 -- start start 2024-07-04T08:22:53.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89ccc00640 1 -- 172.21.15.39:0/1242413052 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f89c0042030 con 0x7f89d0101e20 2024-07-04T08:22:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b392e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3561490 0x5560b392ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3569670 0x5560b392f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5560b38588f0 con 0x5560b358a350 2024-07-04T08:22:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5560b38a4ed0 con 0x5560b3569670 2024-07-04T08:22:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5560b3894470 con 0x5560b3561490 2024-07-04T08:22:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89d5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b392e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.338+0000 7f89d5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b392e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58040/0 (socket says 172.21.15.39:58040) 2024-07-04T08:22:53.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.339+0000 7f89d5200640 1 -- 172.21.15.39:0/801175054 learned_addr learned my addr 172.21.15.39:0/801175054 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.339+0000 7f89cfe00640 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3561490 0x5560b392ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.339+0000 7f89cf400640 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3569670 0x5560b392f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.339+0000 7f89d5200640 1 -- 172.21.15.39:0/801175054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3561490 msgr2=0x5560b392ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.339+0000 7f89d5200640 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3561490 0x5560b392ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.339+0000 7f89d5200640 1 -- 172.21.15.39:0/801175054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3569670 msgr2=0x5560b392f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.339+0000 7f89d5200640 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3569670 0x5560b392f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.339+0000 7f89d5200640 1 -- 172.21.15.39:0/801175054 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b382c290 con 0x5560b358a350 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.339+0000 7f89cf400640 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3569670 0x5560b392f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.340+0000 7f89d5200640 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b392e800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f89c80117e0 tx=0x7f89c8011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff7b5639f70218b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.340+0000 7f89c5600640 1 -- 172.21.15.39:0/801175054 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8011310 con 0x5560b358a350 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.340+0000 7f89c5600640 1 -- 172.21.15.39:0/801175054 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c8003e20 con 0x5560b358a350 2024-07-04T08:22:53.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.340+0000 7f89c5600640 1 -- 172.21.15.39:0/801175054 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8004490 con 0x5560b358a350 2024-07-04T08:22:53.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.340+0000 7f89ccc00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89b40b29e0 0x7f89b40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.340+0000 7f89ccc00640 1 -- 172.21.15.39:0/1242413052 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f89c0102ca0 con 0x7f89d0101e20 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89cfe00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89b40b29e0 0x7f89b40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89dd7dd740 1 -- 172.21.15.39:0/801175054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 msgr2=0x5560b392e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b392e800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f89c80117e0 tx=0x7f89c8011ca0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89dd7dd740 1 -- 172.21.15.39:0/801175054 shutdown_connections 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3569670 0x5560b392f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3561490 0x5560b392ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/801175054 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b392e800 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89dd7dd740 1 -- 172.21.15.39:0/801175054 >> 172.21.15.39:0/801175054 conn(0x5560b357e740 msgr2=0x5560b392c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89dd7dd740 1 -- 172.21.15.39:0/801175054 shutdown_connections 2024-07-04T08:22:53.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.341+0000 7f89dd7dd740 1 -- 172.21.15.39:0/801175054 wait complete. 2024-07-04T08:22:53.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.342+0000 7f89cfe00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89b40b29e0 0x7f89b40b4ef0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x5560b392d480 tx=0x7f89c0106750 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:53.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.342+0000 7f89dd7dd740 1 -- start start 2024-07-04T08:22:53.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.342+0000 7f89dd7dd740 1 Processor -- start 2024-07-04T08:22:53.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3561490 0x5560b393a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b358a350 0x5560b3946180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5560b383b770 con 0x5560b3561490 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5560b38984b0 con 0x5560b358a350 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5560b36f8890 con 0x5560b3569670 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89dd7dd740 10 client.0 _ll_register_callbacks cb 0x5560b383c2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89d5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3561490 0x5560b393a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89d5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3561490 0x5560b393a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58056/0 (socket says 172.21.15.39:58056) 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89d5200640 1 -- 172.21.15.39:0/1158114984 learned_addr learned my addr 172.21.15.39:0/1158114984 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.343+0000 7f89cf400640 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.344+0000 7f89d5200640 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b358a350 0x5560b3946180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.344+0000 7f89cf400640 1 -- 172.21.15.39:0/1158114984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b358a350 msgr2=0x5560b3946180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.344+0000 7f89cf400640 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b358a350 0x5560b3946180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.344+0000 7f89cf400640 1 -- 172.21.15.39:0/1158114984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3561490 msgr2=0x5560b393a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.344+0000 7f89cf400640 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3561490 0x5560b393a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.344+0000 7f89cf400640 1 -- 172.21.15.39:0/1158114984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b35f5930 con 0x5560b3569670 2024-07-04T08:22:53.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.344+0000 7f89d5200640 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3561490 0x5560b393a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:53.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.344+0000 7f89cf400640 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393a910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f89d0069c40 tx=0x7f89bc012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f963e208b38dc38d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:53.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.344+0000 7f89ba000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:53.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.345+0000 7f89ba000640 1 -- 172.21.15.39:0/1158114984 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc004030 con 0x5560b3569670 2024-07-04T08:22:53.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.345+0000 7f89ba000640 1 -- 172.21.15.39:0/1158114984 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f89bc0041d0 con 0x5560b3569670 2024-07-04T08:22:53.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.345+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1158114984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c8003e20 con 0x5560b3569670 2024-07-04T08:22:53.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.345+0000 7f89ba000640 1 -- 172.21.15.39:0/1158114984 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f89bc025050 con 0x5560b3569670 2024-07-04T08:22:53.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.345+0000 7f89dd7dd740 10 client.27322 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:53.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.345+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1158114984 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5560b3948eb0 con 0x5560b3569670 2024-07-04T08:22:53.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.346+0000 7f89ba000640 1 -- 172.21.15.39:0/1158114984 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89c8003e20 con 0x5560b3569670 2024-07-04T08:22:53.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.346+0000 7f89ba000640 1 -- 172.21.15.39:0/1158114984 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f89bc003930 con 0x5560b3569670 2024-07-04T08:22:53.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.346+0000 7f89dd7dd740 10 client.27322 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:53.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.346+0000 7f89dd7dd740 10 client.27322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:53.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.346+0000 7f89dd7dd740 20 client.27322 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:53.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.346+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5560b3548cd0 0x5560b394c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.346+0000 7f89dd7dd740 4 client.27322 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:53.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.346+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1158114984 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560b38a6e20 con 0x5560b3548cd0 2024-07-04T08:22:53.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.346+0000 7f89d5200640 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5560b3548cd0 0x5560b394c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.347+0000 7f89d5200640 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5560b3548cd0 0x5560b394c3c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b022d7bebe7f733 server_cookie=5dd1650ec53f4a2b in_seq=0 out_seq=0 2024-07-04T08:22:53.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.347+0000 7f89ba000640 10 client.27322 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:53.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.348+0000 7f89ba000640 1 -- 172.21.15.39:0/1158114984 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5560b38a6e20 con 0x5560b3548cd0 2024-07-04T08:22:53.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.348+0000 7f89ba000640 10 client.27322 handle_command_reply: tid=0 2024-07-04T08:22:53.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.348+0000 7f89dd7dd740 1 client.27322 shutdown 2024-07-04T08:22:53.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.348+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1158114984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 msgr2=0x5560b393a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:53.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.348+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393a910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f89d0069c40 tx=0x7f89bc012b60 comp rx=0 tx=0).stop 2024-07-04T08:22:53.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1158114984 shutdown_connections 2024-07-04T08:22:53.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5560b3548cd0 0x5560b394c3c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b358a350 0x5560b3946180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393a910 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89ba000640 0 client.27322 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:53.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1158114984 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3561490 0x5560b393a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1158114984 >> 172.21.15.39:0/1158114984 conn(0x5560b357e740 msgr2=0x5560b39319e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:53.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1158114984 shutdown_connections 2024-07-04T08:22:53.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1158114984 wait complete. 2024-07-04T08:22:53.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.349+0000 7f89dd7dd740 20 client.27322 trim_cache size 0 max 16384 2024-07-04T08:22:53.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.369+0000 7f89dd7dd740 1 Processor -- start 2024-07-04T08:22:53.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.369+0000 7f89dd7dd740 1 -- start start 2024-07-04T08:22:53.364 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:53 smithi039 ceph-mon[32062]: pgmap v712: 129 pgs: 129 active+clean; 9.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 18 op/s 2024-07-04T08:22:53.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b39301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b3945f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01b2430 con 0x5560b358a350 2024-07-04T08:22:53.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01b3f70 con 0x5560b3561490 2024-07-04T08:22:53.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01b2b00 con 0x5560b3569670 2024-07-04T08:22:53.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89d5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b39301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89cf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89d5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b39301f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44144/0 (socket says 172.21.15.39:44144) 2024-07-04T08:22:53.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89d5200640 1 -- 172.21.15.39:0/1720878652 learned_addr learned my addr 172.21.15.39:0/1720878652 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:53.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.370+0000 7f89d5200640 1 --2- 172.21.15.39:0/1720878652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b3945f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.371+0000 7f89cf400640 1 -- 172.21.15.39:0/1720878652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 msgr2=0x5560b39301f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:53.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.371+0000 7f89cf400640 1 --2- 172.21.15.39:0/1720878652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b39301f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.371+0000 7f89cf400640 1 -- 172.21.15.39:0/1720878652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 msgr2=0x5560b3945f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:53.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.371+0000 7f89cf400640 1 --2- 172.21.15.39:0/1720878652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b3945f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.371+0000 7f89cf400640 1 -- 172.21.15.39:0/1720878652 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560b3931660 con 0x5560b3569670 2024-07-04T08:22:53.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.371+0000 7f89cf400640 1 --2- 172.21.15.39:0/1720878652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393be00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f89bc012420 tx=0x7f89bc0105c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd7b28b24b70f13e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:53.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.371+0000 7f89c4c00640 1 -- 172.21.15.39:0/1720878652 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc02e070 con 0x5560b3569670 2024-07-04T08:22:53.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.372+0000 7f89c4c00640 1 -- 172.21.15.39:0/1720878652 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89bc00fda0 con 0x5560b3569670 2024-07-04T08:22:53.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.372+0000 7f89c4c00640 1 -- 172.21.15.39:0/1720878652 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc022040 con 0x5560b3569670 2024-07-04T08:22:53.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.372+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1720878652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 msgr2=0x5560b393be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:53.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.372+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1720878652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393be00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f89bc012420 tx=0x7f89bc0105c0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.373+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1720878652 shutdown_connections 2024-07-04T08:22:53.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.373+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1720878652 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b358a350 0x5560b3945f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.373+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1720878652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b3569670 0x5560b393be00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.373+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/1720878652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b39301f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.373+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1720878652 >> 172.21.15.39:0/1720878652 conn(0x5560b357e740 msgr2=0x5560b393b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:53.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.373+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1720878652 shutdown_connections 2024-07-04T08:22:53.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.373+0000 7f89dd7dd740 1 -- 172.21.15.39:0/1720878652 wait complete. 2024-07-04T08:22:53.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.373+0000 7f89dd7dd740 1 -- start start 2024-07-04T08:22:53.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.374+0000 7f89dd7dd740 1 Processor -- start 2024-07-04T08:22:53.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.374+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b393e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.374+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3569670 0x5560b393ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.374+0000 7f89dd7dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b358a350 0x5560b39adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.374+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01b38a0 con 0x5560b3569670 2024-07-04T08:22:53.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.374+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01b31d0 con 0x5560b3561490 2024-07-04T08:22:53.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.374+0000 7f89dd7dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89d01b0410 con 0x5560b358a350 2024-07-04T08:22:53.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.374+0000 7f89dd7dd740 10 client.0 _ll_register_callbacks cb 0x7f89bc009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:53.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3569670 0x5560b393ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89d5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b393e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89cf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3569670 0x5560b393ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58084/0 (socket says 172.21.15.39:58084) 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89d5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b393e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44156/0 (socket says 172.21.15.39:44156) 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89cf400640 1 -- 172.21.15.39:0/3703421357 learned_addr learned my addr 172.21.15.39:0/3703421357 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89d5200640 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b358a350 0x5560b39adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89d5200640 1 -- 172.21.15.39:0/3703421357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b358a350 msgr2=0x5560b39adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89d5200640 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b358a350 0x5560b39adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89d5200640 1 -- 172.21.15.39:0/3703421357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3569670 msgr2=0x5560b393ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89d5200640 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3569670 0x5560b393ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89d5200640 1 -- 172.21.15.39:0/3703421357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560b39ae190 con 0x5560b3561490 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.375+0000 7f89cf400640 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3569670 0x5560b393ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:53.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.376+0000 7f89d5200640 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b393e940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005f10 tx=0x7f89c8011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac740e213ccb409 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:53.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.376+0000 7f89ba000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:22:53.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.376+0000 7f89ba000640 1 -- 172.21.15.39:0/3703421357 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8023040 con 0x5560b3561490 2024-07-04T08:22:53.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.376+0000 7f89dd7dd740 1 -- 172.21.15.39:0/3703421357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89bc00fda0 con 0x5560b3561490 2024-07-04T08:22:53.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.376+0000 7f89ba000640 1 -- 172.21.15.39:0/3703421357 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f89c800fb00 con 0x5560b3561490 2024-07-04T08:22:53.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.376+0000 7f89ba000640 1 -- 172.21.15.39:0/3703421357 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f89c800fd00 con 0x5560b3561490 2024-07-04T08:22:53.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.376+0000 7f89dd7dd740 10 client.27410 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:53.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.376+0000 7f89dd7dd740 1 -- 172.21.15.39:0/3703421357 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5560b39b55a0 con 0x5560b3561490 2024-07-04T08:22:53.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.377+0000 7f89ba000640 1 -- 172.21.15.39:0/3703421357 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89bc00fda0 con 0x5560b3561490 2024-07-04T08:22:53.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.377+0000 7f89ba000640 1 -- 172.21.15.39:0/3703421357 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f89c8029040 con 0x5560b3561490 2024-07-04T08:22:53.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.377+0000 7f89dd7dd740 10 client.27410 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.377+0000 7f89dd7dd740 10 client.27410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.377+0000 7f89dd7dd740 20 client.27410 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.377+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5560b351e080 0x5560b39b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.377+0000 7f89dd7dd740 4 client.27410 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:22:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.377+0000 7f89dd7dd740 1 -- 172.21.15.39:0/3703421357 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5560b38a28c0 con 0x5560b351e080 2024-07-04T08:22:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.378+0000 7f89cf400640 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5560b351e080 0x5560b39b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.378+0000 7f89cf400640 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5560b351e080 0x5560b39b7980 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4d3009b7e6c9aa server_cookie=9c70c3406c84de28 in_seq=0 out_seq=0 2024-07-04T08:22:53.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.378+0000 7f89ba000640 10 client.27410 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:53.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.379+0000 7f89ba000640 1 -- 172.21.15.39:0/3703421357 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5560b38a28c0 con 0x5560b351e080 2024-07-04T08:22:53.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.379+0000 7f89ba000640 10 client.27410 handle_command_reply: tid=0 2024-07-04T08:22:53.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.379+0000 7f89dd7dd740 1 client.27410 shutdown 2024-07-04T08:22:53.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 -- 172.21.15.39:0/3703421357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 msgr2=0x5560b393e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:53.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b393e940 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005f10 tx=0x7f89c8011ca0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 -- 172.21.15.39:0/3703421357 shutdown_connections 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5560b351e080 0x5560b39b7980 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5560b3569670 0x5560b393ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5560b358a350 0x5560b39adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 --2- 172.21.15.39:0/3703421357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5560b3561490 0x5560b393e940 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89ba000640 0 client.27410 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 -- 172.21.15.39:0/3703421357 >> 172.21.15.39:0/3703421357 conn(0x5560b357e740 msgr2=0x5560b392ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 -- 172.21.15.39:0/3703421357 shutdown_connections 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.380+0000 7f89dd7dd740 1 -- 172.21.15.39:0/3703421357 wait complete. 2024-07-04T08:22:53.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.381+0000 7f89dd7dd740 20 client.27410 trim_cache size 0 max 16384 2024-07-04T08:22:53.377 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:22:53.377 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:22:53.377 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "e94a7ae0-70eb-4cbe-9b25-4df120bc3de9", 2024-07-04T08:22:53.377 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:22:53.377 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:22:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.381+0000 7f89d5c00640 1 -- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89b40b29e0 msgr2=0x7f89b40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.381+0000 7f89d5c00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89b40b29e0 0x7f89b40b4ef0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x5560b392d480 tx=0x7f89c0106750 comp rx=0 tx=0).stop 2024-07-04T08:22:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.381+0000 7f89d5c00640 1 -- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 msgr2=0x7f89d00fde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.381+0000 7f89d5c00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 0x7f89d00fde00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f89c0005940 tx=0x7f89c00040c0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.382+0000 7f89d5c00640 1 -- 172.21.15.39:0/1242413052 shutdown_connections 2024-07-04T08:22:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.382+0000 7f89d5c00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89b40b29e0 0x7f89b40b4ef0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.382+0000 7f89d5c00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f89d01080b0 0x7f89d00fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.382+0000 7f89d5c00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89d0107760 0x7f89d00fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.382+0000 7f89d5c00640 1 --2- 172.21.15.39:0/1242413052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89d0101e20 0x7f89d00fde00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.382+0000 7f89d5c00640 1 -- 172.21.15.39:0/1242413052 >> 172.21.15.39:0/1242413052 conn(0x7f89d00fd8d0 msgr2=0x7f89d006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.382+0000 7f89d5c00640 1 -- 172.21.15.39:0/1242413052 shutdown_connections 2024-07-04T08:22:53.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:53.382+0000 7f89d5c00640 1 -- 172.21.15.39:0/1242413052 wait complete. 2024-07-04T08:22:53.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:53 smithi050 ceph-mon[36577]: pgmap v712: 129 pgs: 129 active+clean; 9.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 18 op/s 2024-07-04T08:22:53.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:53 smithi133 ceph-mon[37050]: pgmap v712: 129 pgs: 129 active+clean; 9.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 64 KiB/s wr, 18 op/s 2024-07-04T08:22:54.205 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:22:54.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:54 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:22:54.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:54 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:54.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:54 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:22:54.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:54 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:54.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:54 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:22:54.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:54 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:54.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:54 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:22:54.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:54 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:54.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:54 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:22:54.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:54 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:54.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:54 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:22:54.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:54 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:22:54.712 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:22:55.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:55 smithi133 ceph-mon[37050]: pgmap v713: 129 pgs: 129 active+clean; 9.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-07-04T08:22:55.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:55 smithi039 ceph-mon[32062]: pgmap v713: 129 pgs: 129 active+clean; 9.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-07-04T08:22:55.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:55 smithi050 ceph-mon[36577]: pgmap v713: 129 pgs: 129 active+clean; 9.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-07-04T08:22:56.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.000+0000 7f43c6600640 1 -- 172.21.15.39:0/1130101118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c0109510 msgr2=0x7f43c0109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:56.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.000+0000 7f43c6600640 1 --2- 172.21.15.39:0/1130101118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c0109510 0x7f43c0109920 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b402a410 comp rx=0 tx=0).stop 2024-07-04T08:22:56.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.000+0000 7f43c6600640 1 -- 172.21.15.39:0/1130101118 shutdown_connections 2024-07-04T08:22:56.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.000+0000 7f43c6600640 1 --2- 172.21.15.39:0/1130101118 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43c010f7a0 0x7f43c0115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:56.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.000+0000 7f43c6600640 1 --2- 172.21.15.39:0/1130101118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43c010ee50 0x7f43c010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:56.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.000+0000 7f43c6600640 1 --2- 172.21.15.39:0/1130101118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c0109510 0x7f43c0109920 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:56.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.000+0000 7f43c6600640 1 -- 172.21.15.39:0/1130101118 >> 172.21.15.39:0/1130101118 conn(0x7f43c0078b10 msgr2=0x7f43c0107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:56.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.000+0000 7f43c6600640 1 -- 172.21.15.39:0/1130101118 shutdown_connections 2024-07-04T08:22:56.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.000+0000 7f43c6600640 1 -- 172.21.15.39:0/1130101118 wait complete. 2024-07-04T08:22:56.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.002+0000 7f43c6600640 1 Processor -- start 2024-07-04T08:22:56.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.002+0000 7f43c6600640 1 -- start start 2024-07-04T08:22:56.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.002+0000 7f43c6600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43c0109510 0x7f43c019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:56.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.002+0000 7f43c6600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c010ee50 0x7f43c01a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:56.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.002+0000 7f43c6600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43c010f7a0 0x7f43c01a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:56.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.002+0000 7f43c6600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0119450 con 0x7f43c0109510 2024-07-04T08:22:56.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.002+0000 7f43c6600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01192d0 con 0x7f43c010ee50 2024-07-04T08:22:56.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.002+0000 7f43c6600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01195d0 con 0x7f43c010f7a0 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c010ee50 0x7f43c01a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43c5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43c010f7a0 0x7f43c01a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c010ee50 0x7f43c01a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44178/0 (socket says 172.21.15.39:44178) 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43bfe00640 1 -- 172.21.15.39:0/1378159022 learned_addr learned my addr 172.21.15.39:0/1378159022 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43c5200640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43c0109510 0x7f43c019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43bfe00640 1 -- 172.21.15.39:0/1378159022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43c010f7a0 msgr2=0x7f43c01a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43bfe00640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43c010f7a0 0x7f43c01a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43bfe00640 1 -- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43c0109510 msgr2=0x7f43c019fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43bfe00640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43c0109510 0x7f43c019fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:56.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.003+0000 7f43bfe00640 1 -- 172.21.15.39:0/1378159022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c01a5db0 con 0x7f43c010ee50 2024-07-04T08:22:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.004+0000 7f43bfe00640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c010ee50 0x7f43c01a03e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00e9f0 tx=0x7f43ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.004+0000 7f43bd600640 1 -- 172.21.15.39:0/1378159022 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac00ccc0 con 0x7f43c010ee50 2024-07-04T08:22:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.004+0000 7f43c6600640 1 -- 172.21.15.39:0/1378159022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c01a60a0 con 0x7f43c010ee50 2024-07-04T08:22:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.004+0000 7f43bd600640 1 -- 172.21.15.39:0/1378159022 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43ac0108e0 con 0x7f43c010ee50 2024-07-04T08:22:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.004+0000 7f43bd600640 1 -- 172.21.15.39:0/1378159022 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac018860 con 0x7f43c010ee50 2024-07-04T08:22:57.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.004+0000 7f43c6600640 1 -- 172.21.15.39:0/1378159022 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c01ac800 con 0x7f43c010ee50 2024-07-04T08:22:57.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.008+0000 7f43ce2a4740 1 Processor -- start 2024-07-04T08:22:57.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.008+0000 7f43ce2a4740 1 -- start start 2024-07-04T08:22:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.008+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x558587589910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.008+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x558587589e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.008+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 0x55858758a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.008+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5585874ffea0 con 0x5585871bc1b0 2024-07-04T08:22:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.008+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5585874ef440 con 0x5585871c4390 2024-07-04T08:22:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.008+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5585874f3480 con 0x5585871e5070 2024-07-04T08:22:57.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43c5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x558587589e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43c5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x558587589e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58116/0 (socket says 172.21.15.39:58116) 2024-07-04T08:22:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43c5200640 1 -- 172.21.15.39:0/2224269629 learned_addr learned my addr 172.21.15.39:0/2224269629 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43c5c00640 1 --2- 172.21.15.39:0/2224269629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x558587589910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43c5200640 1 -- 172.21.15.39:0/2224269629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 msgr2=0x558587589910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43c5200640 1 --2- 172.21.15.39:0/2224269629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x558587589910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43c5200640 1 -- 172.21.15.39:0/2224269629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 msgr2=0x55858758a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:22:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43c5200640 1 --2- 172.21.15.39:0/2224269629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 0x55858758a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43c5200640 1 -- 172.21.15.39:0/2224269629 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558587487610 con 0x5585871bc1b0 2024-07-04T08:22:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.009+0000 7f43bd600640 1 -- 172.21.15.39:0/1378159022 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f43ac010430 con 0x7f43c010ee50 2024-07-04T08:22:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.010+0000 7f43c5200640 1 --2- 172.21.15.39:0/2224269629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x558587589e50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b4032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f77c2aaa13c8ad5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.010+0000 7f43b2000640 1 -- 172.21.15.39:0/2224269629 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4040070 con 0x5585871bc1b0 2024-07-04T08:22:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.010+0000 7f43b2000640 1 -- 172.21.15.39:0/2224269629 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b4004030 con 0x5585871bc1b0 2024-07-04T08:22:57.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.010+0000 7f43b2000640 1 -- 172.21.15.39:0/2224269629 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4035040 con 0x5585871bc1b0 2024-07-04T08:22:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43ce2a4740 1 -- 172.21.15.39:0/2224269629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 msgr2=0x558587589e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/2224269629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x558587589e50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009900 tx=0x7f43b4032b90 comp rx=0 tx=0).stop 2024-07-04T08:22:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43bd600640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43a80b29e0 0x7f43a80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43ce2a4740 1 -- 172.21.15.39:0/2224269629 shutdown_connections 2024-07-04T08:22:57.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/2224269629 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x558587589e50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/2224269629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 0x55858758a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/2224269629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x558587589910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43ce2a4740 1 -- 172.21.15.39:0/2224269629 >> 172.21.15.39:0/2224269629 conn(0x5585871d9460 msgr2=0x558587587800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43bd600640 1 -- 172.21.15.39:0/1378159022 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f43ac0e2be0 con 0x7f43c010ee50 2024-07-04T08:22:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.011+0000 7f43ce2a4740 1 -- 172.21.15.39:0/2224269629 shutdown_connections 2024-07-04T08:22:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.012+0000 7f43c5c00640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43a80b29e0 0x7f43a80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.012+0000 7f43ce2a4740 1 -- 172.21.15.39:0/2224269629 wait complete. 2024-07-04T08:22:57.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.012+0000 7f43ce2a4740 1 -- start start 2024-07-04T08:22:57.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.012+0000 7f43c5c00640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43a80b29e0 0x7f43a80b4ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005fd0 tx=0x7f43b8009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:57.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.013+0000 7f43ce2a4740 1 Processor -- start 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.013+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x5585875954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.013+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 0x558587595a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.013+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.013+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558587353700 con 0x5585871bc1b0 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.013+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558587496b70 con 0x5585871c4390 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.013+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558587598090 con 0x5585871e5070 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.013+0000 7f43ce2a4740 10 client.0 _ll_register_callbacks cb 0x5585874e8530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x5585875954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43bfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 0x558587595a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56468/0 (socket says 172.21.15.39:56468) 2024-07-04T08:22:57.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5c00640 1 -- 172.21.15.39:0/815903703 learned_addr learned my addr 172.21.15.39:0/815903703 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:57.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5200640 1 -- 172.21.15.39:0/815903703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 msgr2=0x5585875a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:57.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5200640 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5200640 1 -- 172.21.15.39:0/815903703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 msgr2=0x558587595a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:57.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5200640 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 0x558587595a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5200640 1 -- 172.21.15.39:0/815903703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558587250650 con 0x5585871bc1b0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43c5c00640 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.014+0000 7f43bfe00640 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 0x558587595a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.015+0000 7f43c5200640 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x5585875954f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f43b4031440 tx=0x7f43b402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff44270629af735b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.015+0000 7f43a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.015+0000 7f43a6a00640 1 -- 172.21.15.39:0/815903703 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4040070 con 0x5585871bc1b0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.015+0000 7f43ce2a4740 1 -- 172.21.15.39:0/815903703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b4004030 con 0x5585871bc1b0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.015+0000 7f43a6a00640 1 -- 172.21.15.39:0/815903703 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f43b4005460 con 0x5585871bc1b0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.015+0000 7f43a6a00640 1 -- 172.21.15.39:0/815903703 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43b4056050 con 0x5585871bc1b0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.015+0000 7f43ce2a4740 10 client.17283 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.015+0000 7f43ce2a4740 1 -- 172.21.15.39:0/815903703 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585875a40d0 con 0x5585871bc1b0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.016+0000 7f43a6a00640 1 -- 172.21.15.39:0/815903703 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b4004030 con 0x5585871bc1b0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.016+0000 7f43a6a00640 1 -- 172.21.15.39:0/815903703 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f43b4004c10 con 0x5585871bc1b0 2024-07-04T08:22:57.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.016+0000 7f43ce2a4740 10 client.17283 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:57.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.016+0000 7f43ce2a4740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:57.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.016+0000 7f43ce2a4740 20 client.17283 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:57.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.016+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5585871a39f0 0x5585875a7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.016+0000 7f43ce2a4740 4 client.17283 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:22:57.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.016+0000 7f43ce2a4740 1 -- 172.21.15.39:0/815903703 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558587501df0 con 0x5585871a39f0 2024-07-04T08:22:57.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.017+0000 7f43bfe00640 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5585871a39f0 0x5585875a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.017+0000 7f43bfe00640 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5585871a39f0 0x5585875a7430 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00fce365b19eefb server_cookie=aa98438aba9621b6 in_seq=0 out_seq=0 2024-07-04T08:22:57.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.017+0000 7f43a6a00640 10 client.17283 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:57.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.018+0000 7f43a6a00640 1 -- 172.21.15.39:0/815903703 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558587501df0 con 0x5585871a39f0 2024-07-04T08:22:57.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.018+0000 7f43a6a00640 10 client.17283 handle_command_reply: tid=0 2024-07-04T08:22:57.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 client.17283 shutdown 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 -- 172.21.15.39:0/815903703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 msgr2=0x5585875954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x5585875954f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f43b4031440 tx=0x7f43b402a410 comp rx=0 tx=0).stop 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 -- 172.21.15.39:0/815903703 shutdown_connections 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5585871a39f0 0x5585875a7430 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871c4390 0x558587595a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43a6a00640 0 client.17283 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/815903703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871bc1b0 0x5585875954f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 -- 172.21.15.39:0/815903703 >> 172.21.15.39:0/815903703 conn(0x5585871d9460 msgr2=0x55858758caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 -- 172.21.15.39:0/815903703 shutdown_connections 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 1 -- 172.21.15.39:0/815903703 wait complete. 2024-07-04T08:22:57.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.019+0000 7f43ce2a4740 20 client.17283 trim_cache size 0 max 16384 2024-07-04T08:22:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.039+0000 7f43ce2a4740 1 Processor -- start 2024-07-04T08:22:57.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43ce2a4740 1 -- start start 2024-07-04T08:22:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871bc1b0 0x55858758f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01b6780 con 0x5585871c4390 2024-07-04T08:22:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01b82c0 con 0x5585871bc1b0 2024-07-04T08:22:57.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01b6e50 con 0x5585871e5070 2024-07-04T08:22:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43c5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871bc1b0 0x55858758f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.040+0000 7f43c5200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871bc1b0 0x55858758f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44200/0 (socket says 172.21.15.39:44200) 2024-07-04T08:22:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43c5200640 1 -- 172.21.15.39:0/1304401095 learned_addr learned my addr 172.21.15.39:0/1304401095 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58144/0 (socket says 172.21.15.39:58144) 2024-07-04T08:22:57.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43c5c00640 1 --2- 172.21.15.39:0/1304401095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43bfe00640 1 -- 172.21.15.39:0/1304401095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 msgr2=0x5585875a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43bfe00640 1 --2- 172.21.15.39:0/1304401095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43bfe00640 1 -- 172.21.15.39:0/1304401095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871bc1b0 msgr2=0x55858758f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43bfe00640 1 --2- 172.21.15.39:0/1304401095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871bc1b0 0x55858758f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43bfe00640 1 -- 172.21.15.39:0/1304401095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558587597a30 con 0x5585871c4390 2024-07-04T08:22:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43c5200640 1 --2- 172.21.15.39:0/1304401095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871bc1b0 0x55858758f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.041+0000 7f43bfe00640 1 --2- 172.21.15.39:0/1304401095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a1350 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0dff70 tx=0x7f43ac00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125e88bf2f4d98b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:57.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.042+0000 7f43b1600640 1 -- 172.21.15.39:0/1304401095 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac0a9bb0 con 0x5585871c4390 2024-07-04T08:22:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.042+0000 7f43b1600640 1 -- 172.21.15.39:0/1304401095 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43ac08d070 con 0x5585871c4390 2024-07-04T08:22:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.042+0000 7f43b1600640 1 -- 172.21.15.39:0/1304401095 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac0a5520 con 0x5585871c4390 2024-07-04T08:22:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 -- 172.21.15.39:0/1304401095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 msgr2=0x5585875a1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:57.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/1304401095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a1350 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0dff70 tx=0x7f43ac00cef0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 -- 172.21.15.39:0/1304401095 shutdown_connections 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/1304401095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a1350 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/1304401095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871e5070 0x5585875a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/1304401095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871bc1b0 0x55858758f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 -- 172.21.15.39:0/1304401095 >> 172.21.15.39:0/1304401095 conn(0x5585871d9460 msgr2=0x55858758ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 -- 172.21.15.39:0/1304401095 shutdown_connections 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 -- 172.21.15.39:0/1304401095 wait complete. 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.043+0000 7f43ce2a4740 1 -- start start 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.044+0000 7f43ce2a4740 1 Processor -- start 2024-07-04T08:22:57.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.044+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871bc1b0 0x5585875a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.044+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.044+0000 7f43ce2a4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871e5070 0x55858760d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.044+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01b7bf0 con 0x5585871c4390 2024-07-04T08:22:57.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.044+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01b7520 con 0x5585871e5070 2024-07-04T08:22:57.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.044+0000 7f43ce2a4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43c01b4760 con 0x5585871bc1b0 2024-07-04T08:22:57.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.044+0000 7f43ce2a4740 10 client.0 _ll_register_callbacks cb 0x7f43b8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:22:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43c5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871bc1b0 0x5585875a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43bfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58158/0 (socket says 172.21.15.39:58158) 2024-07-04T08:22:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43bfe00640 1 -- 172.21.15.39:0/4154488391 learned_addr learned my addr 172.21.15.39:0/4154488391 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:22:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43c5c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871e5070 0x55858760d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43c5200640 1 -- 172.21.15.39:0/4154488391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871e5070 msgr2=0x55858760d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:57.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43c5200640 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871e5070 0x55858760d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43c5200640 1 -- 172.21.15.39:0/4154488391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 msgr2=0x5585875a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43c5200640 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43c5200640 1 -- 172.21.15.39:0/4154488391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858760deb0 con 0x5585871bc1b0 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43c5c00640 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871e5070 0x55858760d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.045+0000 7f43bfe00640 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.046+0000 7f43c5200640 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871bc1b0 0x5585875a7ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f43b4001eb0 tx=0x7f43b40038d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57cc30015d980597 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.046+0000 7f43a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.046+0000 7f43a6a00640 1 -- 172.21.15.39:0/4154488391 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4040040 con 0x5585871bc1b0 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.046+0000 7f43a6a00640 1 -- 172.21.15.39:0/4154488391 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9424+0+0 (secure 0 0 0) 0x7f43b40318b0 con 0x5585871bc1b0 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.046+0000 7f43ce2a4740 1 -- 172.21.15.39:0/4154488391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43ac08d070 con 0x5585871bc1b0 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.046+0000 7f43a6a00640 1 -- 172.21.15.39:0/4154488391 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43b406a050 con 0x5585871bc1b0 2024-07-04T08:22:57.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.046+0000 7f43ce2a4740 10 client.27346 fetch_fsmap learned FSMap version 30 2024-07-04T08:22:57.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.047+0000 7f43ce2a4740 1 -- 172.21.15.39:0/4154488391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55858760f140 con 0x5585871bc1b0 2024-07-04T08:22:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.047+0000 7f43a6a00640 1 -- 172.21.15.39:0/4154488391 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43ac08d070 con 0x5585871bc1b0 2024-07-04T08:22:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.047+0000 7f43a6a00640 1 -- 172.21.15.39:0/4154488391 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f43b4003f40 con 0x5585871bc1b0 2024-07-04T08:22:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.047+0000 7f43ce2a4740 10 client.27346 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:22:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.047+0000 7f43ce2a4740 10 client.27346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:22:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.047+0000 7f43ce2a4740 20 client.27346 populate_metadata read hostname 'smithi039' 2024-07-04T08:22:57.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.047+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558587178da0 0x5585876118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.047+0000 7f43ce2a4740 4 client.27346 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.047+0000 7f43ce2a4740 1 -- 172.21.15.39:0/4154488391 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585874fd890 con 0x558587178da0 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.048+0000 7f43bfe00640 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558587178da0 0x5585876118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.048+0000 7f43bfe00640 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558587178da0 0x5585876118a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41604bef2cb289be server_cookie=64398081e56318fb in_seq=0 out_seq=0 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.048+0000 7f43a6a00640 10 client.27346 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43a6a00640 1 -- 172.21.15.39:0/4154488391 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5585874fd890 con 0x558587178da0 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43a6a00640 10 client.27346 handle_command_reply: tid=0 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 client.27346 shutdown 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 -- 172.21.15.39:0/4154488391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871bc1b0 msgr2=0x5585875a7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871bc1b0 0x5585875a7ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f43b4001eb0 tx=0x7f43b40038d0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 -- 172.21.15.39:0/4154488391 shutdown_connections 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558587178da0 0x5585876118a0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5585871c4390 0x5585875a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5585871e5070 0x55858760d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 --2- 172.21.15.39:0/4154488391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5585871bc1b0 0x5585875a7ac0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43a6a00640 0 client.27346 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:22:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 -- 172.21.15.39:0/4154488391 >> 172.21.15.39:0/4154488391 conn(0x5585871d9460 msgr2=0x5585875967a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 -- 172.21.15.39:0/4154488391 shutdown_connections 2024-07-04T08:22:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 1 -- 172.21.15.39:0/4154488391 wait complete. 2024-07-04T08:22:57.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.049+0000 7f43ce2a4740 20 client.27346 trim_cache size 0 max 16384 2024-07-04T08:22:57.046 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:22:57.046 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-04T08:22:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-07-04T08:22:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "e94a7ae0-70eb-4cbe-9b25-4df120bc3de9": { 2024-07-04T08:22:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-07-04T08:22:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "e94a7ae0-70eb-4cbe-9b25-4df120bc3de9", 2024-07-04T08:22:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-07-04T08:22:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:22:57.047 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:22:57.047 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.050+0000 7f43c6600640 1 -- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43a80b29e0 msgr2=0x7f43a80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.050+0000 7f43c6600640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43a80b29e0 0x7f43a80b4ef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005fd0 tx=0x7f43b8009500 comp rx=0 tx=0).stop 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.050+0000 7f43c6600640 1 -- 172.21.15.39:0/1378159022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c010ee50 msgr2=0x7f43c01a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.050+0000 7f43c6600640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c010ee50 0x7f43c01a03e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00e9f0 tx=0x7f43ac00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.051+0000 7f43c6600640 1 -- 172.21.15.39:0/1378159022 shutdown_connections 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.051+0000 7f43c6600640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43a80b29e0 0x7f43a80b4ef0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.051+0000 7f43c6600640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43c010f7a0 0x7f43c01a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.051+0000 7f43c6600640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43c010ee50 0x7f43c01a03e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.051+0000 7f43c6600640 1 --2- 172.21.15.39:0/1378159022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43c0109510 0x7f43c019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:22:57.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.051+0000 7f43c6600640 1 -- 172.21.15.39:0/1378159022 >> 172.21.15.39:0/1378159022 conn(0x7f43c0078b10 msgr2=0x7f43c00792e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:22:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.051+0000 7f43c6600640 1 -- 172.21.15.39:0/1378159022 shutdown_connections 2024-07-04T08:22:57.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:22:57.051+0000 7f43c6600640 1 -- 172.21.15.39:0/1378159022 wait complete. 2024-07-04T08:22:57.344 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:57 smithi039 ceph-mon[32062]: pgmap v714: 129 pgs: 129 active+clean; 9.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-07-04T08:22:57.439 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:57 smithi050 ceph-mon[36577]: pgmap v714: 129 pgs: 129 active+clean; 9.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-07-04T08:22:57.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:57 smithi133 ceph-mon[37050]: pgmap v714: 129 pgs: 129 active+clean; 9.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-07-04T08:22:57.995 INFO:tasks.cephfs.filesystem:scrub status for tag:e94a7ae0-70eb-4cbe-9b25-4df120bc3de9 - {'path': '/', 'tag': 'e94a7ae0-70eb-4cbe-9b25-4df120bc3de9', 'options': 'recursive,force'} 2024-07-04T08:22:58.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:58 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:22:58.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:58 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:22:58.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:58 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:22:59.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:22:59 smithi133 ceph-mon[37050]: pgmap v715: 129 pgs: 129 active+clean; 9.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-07-04T08:22:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:22:59 smithi039 ceph-mon[32062]: pgmap v715: 129 pgs: 129 active+clean; 9.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-07-04T08:22:59.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:22:59 smithi050 ceph-mon[36577]: pgmap v715: 129 pgs: 129 active+clean; 9.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-07-04T08:23:01.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:01 smithi133 ceph-mon[37050]: pgmap v716: 129 pgs: 129 active+clean; 9.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-07-04T08:23:01.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:01 smithi039 ceph-mon[32062]: pgmap v716: 129 pgs: 129 active+clean; 9.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-07-04T08:23:01.600 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:01 smithi050 ceph-mon[36577]: pgmap v716: 129 pgs: 129 active+clean; 9.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-07-04T08:23:03.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:03 smithi133 ceph-mon[37050]: pgmap v717: 129 pgs: 129 active+clean; 9.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-07-04T08:23:03.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:03 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:23:03.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:03 smithi039 ceph-mon[32062]: pgmap v717: 129 pgs: 129 active+clean; 9.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-07-04T08:23:03.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:03 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:23:03.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:03 smithi050 ceph-mon[36577]: pgmap v717: 129 pgs: 129 active+clean; 9.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-07-04T08:23:03.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:03 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:23:04.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:04 smithi133 ceph-mon[37050]: osdmap e95: 12 total, 12 up, 12 in 2024-07-04T08:23:04.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:23:04.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:23:04.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:04 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:23:04.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:04 smithi039 ceph-mon[32062]: osdmap e95: 12 total, 12 up, 12 in 2024-07-04T08:23:04.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:23:04.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:23:04.594 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:04 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:23:04.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:04 smithi050 ceph-mon[36577]: osdmap e95: 12 total, 12 up, 12 in 2024-07-04T08:23:04.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:23:04.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:23:04.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:04 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:23:05.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:05 smithi133 ceph-mon[37050]: pgmap v719: 129 pgs: 129 active+clean; 9.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-07-04T08:23:05.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:05 smithi039 ceph-mon[32062]: pgmap v719: 129 pgs: 129 active+clean; 9.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-07-04T08:23:05.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:05 smithi050 ceph-mon[36577]: pgmap v719: 129 pgs: 129 active+clean; 9.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-07-04T08:23:07.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:07 smithi133 ceph-mon[37050]: pgmap v720: 129 pgs: 129 active+clean; 9.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 102 KiB/s wr, 36 op/s 2024-07-04T08:23:07.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:07 smithi039 ceph-mon[32062]: pgmap v720: 129 pgs: 129 active+clean; 9.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 102 KiB/s wr, 36 op/s 2024-07-04T08:23:07.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:07 smithi050 ceph-mon[36577]: pgmap v720: 129 pgs: 129 active+clean; 9.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 102 KiB/s wr, 36 op/s 2024-07-04T08:23:09.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:09 smithi133 ceph-mon[37050]: pgmap v721: 129 pgs: 129 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 36 op/s 2024-07-04T08:23:09.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:09 smithi039 ceph-mon[32062]: pgmap v721: 129 pgs: 129 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 36 op/s 2024-07-04T08:23:09.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:09 smithi050 ceph-mon[36577]: pgmap v721: 129 pgs: 129 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 36 op/s 2024-07-04T08:23:11.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:11 smithi133 ceph-mon[37050]: pgmap v722: 129 pgs: 129 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 36 op/s 2024-07-04T08:23:11.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:11 smithi039 ceph-mon[32062]: pgmap v722: 129 pgs: 129 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 36 op/s 2024-07-04T08:23:11.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:11 smithi050 ceph-mon[36577]: pgmap v722: 129 pgs: 129 active+clean; 10 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 36 op/s 2024-07-04T08:23:12.559 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:12 smithi133 ceph-mon[37050]: pgmap v723: 129 pgs: 129 active+clean; 10 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-07-04T08:23:12.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:12 smithi039 ceph-mon[32062]: pgmap v723: 129 pgs: 129 active+clean; 10 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-07-04T08:23:12.638 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:12 smithi050 ceph-mon[36577]: pgmap v723: 129 pgs: 129 active+clean; 10 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-07-04T08:23:14.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:14 smithi039 ceph-mon[32062]: osdmap e96: 12 total, 12 up, 12 in 2024-07-04T08:23:14.844 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:14 smithi039 ceph-mon[32062]: pgmap v725: 129 pgs: 129 active+clean; 10 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-07-04T08:23:14.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:14 smithi050 ceph-mon[36577]: osdmap e96: 12 total, 12 up, 12 in 2024-07-04T08:23:14.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:14 smithi050 ceph-mon[36577]: pgmap v725: 129 pgs: 129 active+clean; 10 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-07-04T08:23:15.059 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:14 smithi133 ceph-mon[37050]: osdmap e96: 12 total, 12 up, 12 in 2024-07-04T08:23:15.059 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:14 smithi133 ceph-mon[37050]: pgmap v725: 129 pgs: 129 active+clean; 10 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 103 KiB/s wr, 35 op/s 2024-07-04T08:23:17.309 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:17 smithi133 ceph-mon[37050]: pgmap v726: 129 pgs: 129 active+clean; 10 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:23:17.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:17 smithi039 ceph-mon[32062]: pgmap v726: 129 pgs: 129 active+clean; 10 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:23:17.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:17 smithi050 ceph-mon[36577]: pgmap v726: 129 pgs: 129 active+clean; 10 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:23:19.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:19 smithi133 ceph-mon[37050]: pgmap v727: 129 pgs: 129 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-07-04T08:23:19.345 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:19 smithi039 ceph-mon[32062]: pgmap v727: 129 pgs: 129 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-07-04T08:23:19.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:19 smithi050 ceph-mon[36577]: pgmap v727: 129 pgs: 129 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-07-04T08:23:21.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:21 smithi133 ceph-mon[37050]: pgmap v728: 129 pgs: 129 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-07-04T08:23:21.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:21 smithi039 ceph-mon[32062]: pgmap v728: 129 pgs: 129 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-07-04T08:23:21.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:21 smithi050 ceph-mon[36577]: pgmap v728: 129 pgs: 129 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-07-04T08:23:21.947 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-07-04T08:23:21.947 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:23:22.435 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:23.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:23 smithi133 ceph-mon[37050]: pgmap v729: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:23:23.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:23 smithi039 ceph-mon[32062]: pgmap v729: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:23:23.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:23 smithi050 ceph-mon[36577]: pgmap v729: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:23:24.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.843+0000 7f0870000640 1 -- 172.21.15.39:0/1542481096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086810b590 msgr2=0x7f0868111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:24.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.843+0000 7f0870000640 1 --2- 172.21.15.39:0/1542481096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086810b590 0x7f0868111dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0860009be0 tx=0x7f086002a6c0 comp rx=0 tx=0).stop 2024-07-04T08:23:24.838 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:24 smithi039 ceph-mon[32062]: pgmap v730: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-07-04T08:23:24.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.843+0000 7f0870000640 1 -- 172.21.15.39:0/1542481096 shutdown_connections 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.843+0000 7f0870000640 1 --2- 172.21.15.39:0/1542481096 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f086810b590 0x7f0868111dc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.843+0000 7f0870000640 1 --2- 172.21.15.39:0/1542481096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086810ac40 0x7f086810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.843+0000 7f0870000640 1 --2- 172.21.15.39:0/1542481096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0868105300 0x7f0868105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.843+0000 7f0870000640 1 -- 172.21.15.39:0/1542481096 >> 172.21.15.39:0/1542481096 conn(0x7f0868100d10 msgr2=0x7f0868103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.843+0000 7f0870000640 1 -- 172.21.15.39:0/1542481096 shutdown_connections 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.844+0000 7f0870000640 1 -- 172.21.15.39:0/1542481096 wait complete. 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.845+0000 7f0870000640 1 Processor -- start 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.845+0000 7f0870000640 1 -- start start 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.845+0000 7f0870000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0868105300 0x7f086819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f0870000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f086810ac40 0x7f086819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f0870000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086810b590 0x7f08681a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f0870000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0868115240 con 0x7f086810ac40 2024-07-04T08:23:24.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f0870000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f08681150c0 con 0x7f0868105300 2024-07-04T08:23:24.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f0870000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08681153c0 con 0x7f086810b590 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f086d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0868105300 0x7f086819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f086cc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f086810ac40 0x7f086819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f086d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0868105300 0x7f086819bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35462/0 (socket says 172.21.15.39:35462) 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f086d600640 1 -- 172.21.15.39:0/24948048 learned_addr learned my addr 172.21.15.39:0/24948048 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.846+0000 7f086e000640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086810b590 0x7f08681a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f086cc00640 1 -- 172.21.15.39:0/24948048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086810b590 msgr2=0x7f08681a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f086cc00640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086810b590 0x7f08681a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f086cc00640 1 -- 172.21.15.39:0/24948048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0868105300 msgr2=0x7f086819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f086cc00640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0868105300 0x7f086819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f086cc00640 1 -- 172.21.15.39:0/24948048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08681a1b20 con 0x7f086810ac40 2024-07-04T08:23:24.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f086d600640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0868105300 0x7f086819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f086cc00640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f086810ac40 0x7f086819c150 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f086806bec0 tx=0x7f085400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f0866000640 1 -- 172.21.15.39:0/24948048 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085400cc40 con 0x7f086810ac40 2024-07-04T08:23:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08681a1e10 con 0x7f086810ac40 2024-07-04T08:23:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.847+0000 7f0866000640 1 -- 172.21.15.39:0/24948048 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f085400cde0 con 0x7f086810ac40 2024-07-04T08:23:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.848+0000 7f0866000640 1 -- 172.21.15.39:0/24948048 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854018760 con 0x7f086810ac40 2024-07-04T08:23:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.848+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08681a8570 con 0x7f086810ac40 2024-07-04T08:23:24.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.849+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0868105710 con 0x7f086810ac40 2024-07-04T08:23:24.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.852+0000 7f0866000640 1 -- 172.21.15.39:0/24948048 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0854010a90 con 0x7f086810ac40 2024-07-04T08:23:24.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.854+0000 7f0866000640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f08500b29e0 0x7f08500b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:24.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.855+0000 7f0866000640 1 -- 172.21.15.39:0/24948048 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f08540e2c20 con 0x7f086810ac40 2024-07-04T08:23:24.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.855+0000 7f086d600640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f08500b29e0 0x7f08500b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:24.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.856+0000 7f086d600640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f08500b29e0 0x7f08500b4ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f085c0096c0 tx=0x7f085c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:24.858+0000 7f0866000640 1 -- 172.21.15.39:0/24948048 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f08540a9bf0 con 0x7f086810ac40 2024-07-04T08:23:24.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:24 smithi050 ceph-mon[36577]: pgmap v730: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-07-04T08:23:25.062 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:24 smithi133 ceph-mon[37050]: pgmap v730: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-07-04T08:23:25.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.115+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f086810d3b0 con 0x7f086810ac40 2024-07-04T08:23:25.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.116+0000 7f0866000640 1 -- 172.21.15.39:0/24948048 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f08540b22c0 con 0x7f086810ac40 2024-07-04T08:23:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:23:25.110 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:23:25.110 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.119+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f08500b29e0 msgr2=0x7f08500b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.119+0000 7f0870000640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f08500b29e0 0x7f08500b4ef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f085c0096c0 tx=0x7f085c009260 comp rx=0 tx=0).stop 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.119+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f086810ac40 msgr2=0x7f086819c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.119+0000 7f0870000640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f086810ac40 0x7f086819c150 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f086806bec0 tx=0x7f085400ee50 comp rx=0 tx=0).stop 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.119+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 shutdown_connections 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.120+0000 7f0870000640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f08500b29e0 0x7f08500b4ef0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.120+0000 7f0870000640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f086810ac40 0x7f086819c150 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.120+0000 7f0870000640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f086810b590 0x7f08681a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.120+0000 7f0870000640 1 --2- 172.21.15.39:0/24948048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0868105300 0x7f086819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.120+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 >> 172.21.15.39:0/24948048 conn(0x7f0868100d10 msgr2=0x7f086810a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:25.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.120+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 shutdown_connections 2024-07-04T08:23:25.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:25.120+0000 7f0870000640 1 -- 172.21.15.39:0/24948048 wait complete. 2024-07-04T08:23:25.843 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:25 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/24948048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:23:25.886 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:25 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/24948048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:23:26.047 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-07-04T08:23:26.048 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-07-04T08:23:26.058 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:25 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/24948048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:23:26.547 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:26.844 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:26 smithi039 ceph-mon[32062]: pgmap v731: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-07-04T08:23:26.887 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:26 smithi050 ceph-mon[36577]: pgmap v731: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-07-04T08:23:27.058 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:26 smithi133 ceph-mon[37050]: pgmap v731: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-07-04T08:23:27.996 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:23:28.952 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:28.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.988+0000 7f34dd800640 1 -- 172.21.15.39:0/3678419403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34d81030d0 msgr2=0x7f34d81034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:28.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.988+0000 7f34dd800640 1 --2- 172.21.15.39:0/3678419403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34d81030d0 0x7f34d81034e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009900 tx=0x7f34c802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.988+0000 7f34dd800640 1 -- 172.21.15.39:0/3678419403 shutdown_connections 2024-07-04T08:23:28.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.988+0000 7f34dd800640 1 --2- 172.21.15.39:0/3678419403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34d8109360 0x7f34d810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.988+0000 7f34dd800640 1 --2- 172.21.15.39:0/3678419403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d8108a10 0x7f34d8108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.988+0000 7f34dd800640 1 --2- 172.21.15.39:0/3678419403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34d81030d0 0x7f34d81034e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.988+0000 7f34dd800640 1 -- 172.21.15.39:0/3678419403 >> 172.21.15.39:0/3678419403 conn(0x7f34d80feb20 msgr2=0x7f34d8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:28.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.988+0000 7f34dd800640 1 -- 172.21.15.39:0/3678419403 shutdown_connections 2024-07-04T08:23:28.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.989+0000 7f34dd800640 1 -- 172.21.15.39:0/3678419403 wait complete. 2024-07-04T08:23:28.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.990+0000 7f34dd800640 1 Processor -- start 2024-07-04T08:23:28.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.990+0000 7f34dd800640 1 -- start start 2024-07-04T08:23:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34dd800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d81030d0 0x7f34d819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34dd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34d8108a10 0x7f34d819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34dd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34d8109360 0x7f34d81a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34dd800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34d8113010 con 0x7f34d8108a10 2024-07-04T08:23:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34dd800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34d8112e90 con 0x7f34d81030d0 2024-07-04T08:23:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34dd800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34d8113190 con 0x7f34d8109360 2024-07-04T08:23:28.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34d6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d81030d0 0x7f34d819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:28.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34d6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34d8108a10 0x7f34d819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:28.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34d6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d81030d0 0x7f34d819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35486/0 (socket says 172.21.15.39:35486) 2024-07-04T08:23:28.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34d6a00640 1 -- 172.21.15.39:0/1114954531 learned_addr learned my addr 172.21.15.39:0/1114954531 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:28.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.991+0000 7f34d7400640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34d8109360 0x7f34d81a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:28.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.992+0000 7f34d6a00640 1 -- 172.21.15.39:0/1114954531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34d8109360 msgr2=0x7f34d81a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:28.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.992+0000 7f34d6a00640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34d8109360 0x7f34d81a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.992+0000 7f34d6a00640 1 -- 172.21.15.39:0/1114954531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34d8108a10 msgr2=0x7f34d819c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:28.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.992+0000 7f34d6a00640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34d8108a10 0x7f34d819c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.992+0000 7f34d6a00640 1 -- 172.21.15.39:0/1114954531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d81a1bf0 con 0x7f34d81030d0 2024-07-04T08:23:28.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.992+0000 7f34d6a00640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d81030d0 0x7f34d819bc80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f34c8005940 tx=0x7f34c80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:28.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.993+0000 7f34cf400640 1 -- 172.21.15.39:0/1114954531 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c803f030 con 0x7f34d81030d0 2024-07-04T08:23:28.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.993+0000 7f34cf400640 1 -- 172.21.15.39:0/1114954531 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c80044c0 con 0x7f34d81030d0 2024-07-04T08:23:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.993+0000 7f34dd800640 1 -- 172.21.15.39:0/1114954531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d81a1e80 con 0x7f34d81030d0 2024-07-04T08:23:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.993+0000 7f34cf400640 1 -- 172.21.15.39:0/1114954531 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8035070 con 0x7f34d81030d0 2024-07-04T08:23:28.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.993+0000 7f34dd800640 1 -- 172.21.15.39:0/1114954531 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d81a8640 con 0x7f34d81030d0 2024-07-04T08:23:28.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.998+0000 7f34e3dfb740 1 Processor -- start 2024-07-04T08:23:28.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.998+0000 7f34e3dfb740 1 -- start start 2024-07-04T08:23:28.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.998+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4da03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4da0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4da0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560dc4d15ce0 con 0x560dc49fb840 2024-07-04T08:23:28.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560dc4d05280 con 0x560dc49d2980 2024-07-04T08:23:28.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560dc4d092c0 con 0x560dc49dab60 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34cf400640 1 -- 172.21.15.39:0/1114954531 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f34c8042030 con 0x7f34d81030d0 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34d6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4da0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34d6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4da0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34686/0 (socket says 172.21.15.39:34686) 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34d6000640 1 -- 172.21.15.39:0/2366124907 learned_addr learned my addr 172.21.15.39:0/2366124907 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34d7400640 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4da03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:28.999+0000 7f34d6a00640 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4da0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.000+0000 7f34d6000640 1 -- 172.21.15.39:0/2366124907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 msgr2=0x560dc4da0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.000+0000 7f34d6000640 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4da0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.000+0000 7f34d6000640 1 -- 172.21.15.39:0/2366124907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 msgr2=0x560dc4da03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.000+0000 7f34d6000640 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4da03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.000+0000 7f34d6000640 1 -- 172.21.15.39:0/2366124907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dc4c9c6f0 con 0x560dc49dab60 2024-07-04T08:23:28.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.000+0000 7f34d7400640 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4da03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:28.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.000+0000 7f34d6000640 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4da0e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009c20 tx=0x7f34c4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa9ef3baa8e4c21 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:28.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.001+0000 7f34c3e00640 1 -- 172.21.15.39:0/2366124907 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4004230 con 0x560dc49dab60 2024-07-04T08:23:28.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.001+0000 7f34c3e00640 1 -- 172.21.15.39:0/2366124907 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c40043d0 con 0x560dc49dab60 2024-07-04T08:23:28.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.001+0000 7f34c3e00640 1 -- 172.21.15.39:0/2366124907 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4004d10 con 0x560dc49dab60 2024-07-04T08:23:28.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.001+0000 7f34cf400640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b29e0 0x7f34bc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.002+0000 7f34cf400640 1 -- 172.21.15.39:0/1114954531 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f34c8102ca0 con 0x7f34d81030d0 2024-07-04T08:23:28.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.002+0000 7f34d7400640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b29e0 0x7f34bc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:28.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.002+0000 7f34e3dfb740 1 -- 172.21.15.39:0/2366124907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 msgr2=0x560dc4da0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:28.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.002+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4da0e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009c20 tx=0x7f34c4011810 comp rx=0 tx=0).stop 2024-07-04T08:23:28.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.002+0000 7f34e3dfb740 1 -- 172.21.15.39:0/2366124907 shutdown_connections 2024-07-04T08:23:28.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.002+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4da0e40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.002+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4da0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.002+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/2366124907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4da03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:28.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.002+0000 7f34e3dfb740 1 -- 172.21.15.39:0/2366124907 >> 172.21.15.39:0/2366124907 conn(0x560dc49ef990 msgr2=0x560dc4d9e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:28.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.003+0000 7f34e3dfb740 1 -- 172.21.15.39:0/2366124907 shutdown_connections 2024-07-04T08:23:28.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.003+0000 7f34e3dfb740 1 -- 172.21.15.39:0/2366124907 wait complete. 2024-07-04T08:23:28.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.003+0000 7f34d7400640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b29e0 0x7f34bc0b4ef0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009fd0 tx=0x7f34d0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:28.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.003+0000 7f34e3dfb740 1 -- start start 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.004+0000 7f34e3dfb740 1 Processor -- start 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.004+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49d2980 0x560dc4dabed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.004+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49dab60 0x560dc4dac410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.004+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4db7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.004+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x560dc4b697a0 con 0x560dc49d2980 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.004+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560dc4cac6d0 con 0x560dc49dab60 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.004+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560dc4daeba0 con 0x560dc49fb840 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.004+0000 7f34e3dfb740 10 client.0 _ll_register_callbacks cb 0x560dc4a13090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49dab60 0x560dc4dac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4db7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:28.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4db7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:34688/0 (socket says 172.21.15.39:34688) 2024-07-04T08:23:28.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d7400640 1 -- 172.21.15.39:0/3093690864 learned_addr learned my addr 172.21.15.39:0/3093690864 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:28.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d6a00640 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49d2980 0x560dc4dabed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d7400640 1 -- 172.21.15.39:0/3093690864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49dab60 msgr2=0x560dc4dac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d7400640 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49dab60 0x560dc4dac410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d7400640 1 -- 172.21.15.39:0/3093690864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49d2980 msgr2=0x560dc4dabed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d7400640 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49d2980 0x560dc4dabed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.005+0000 7f34d7400640 1 -- 172.21.15.39:0/3093690864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dc4c9d1d0 con 0x560dc49fb840 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.006+0000 7f34d7400640 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4db7ef0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f34d000f6a0 tx=0x7f34d0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab93b59e2eb8943 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.006+0000 7f34c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.006+0000 7f34c0c00640 1 -- 172.21.15.39:0/3093690864 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0003970 con 0x560dc49fb840 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.006+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3093690864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c40043d0 con 0x560dc49fb840 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.006+0000 7f34c0c00640 1 -- 172.21.15.39:0/3093690864 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f34d0017030 con 0x560dc49fb840 2024-07-04T08:23:29.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.007+0000 7f34c0c00640 1 -- 172.21.15.39:0/3093690864 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34d0032050 con 0x560dc49fb840 2024-07-04T08:23:29.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.007+0000 7f34e3dfb740 10 client.27364 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:29.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.007+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3093690864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560dc4a75b90 con 0x560dc49fb840 2024-07-04T08:23:29.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.007+0000 7f34c0c00640 1 -- 172.21.15.39:0/3093690864 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c40043d0 con 0x560dc49fb840 2024-07-04T08:23:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.008+0000 7f34c0c00640 1 -- 172.21.15.39:0/3093690864 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f34d0027980 con 0x560dc49fb840 2024-07-04T08:23:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.008+0000 7f34e3dfb740 10 client.27364 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.008+0000 7f34e3dfb740 10 client.27364 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:23:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.008+0000 7f34e3dfb740 20 client.27364 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.008+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560dc49ba1c0 0x560dc4dbdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.008+0000 7f34e3dfb740 4 client.27364 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:23:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.008+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3093690864 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dc4d17c30 con 0x560dc49ba1c0 2024-07-04T08:23:29.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.008+0000 7f34d6a00640 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560dc49ba1c0 0x560dc4dbdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.009+0000 7f34d6a00640 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560dc49ba1c0 0x560dc4dbdf90 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc76090360be3404 server_cookie=812d91d028cbe62c in_seq=0 out_seq=0 2024-07-04T08:23:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.009+0000 7f34c0c00640 10 client.27364 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.010+0000 7f34c0c00640 1 -- 172.21.15.39:0/3093690864 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560dc4d17c30 con 0x560dc49ba1c0 2024-07-04T08:23:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.010+0000 7f34c0c00640 10 client.27364 handle_command_reply: tid=0 2024-07-04T08:23:29.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.010+0000 7f34e3dfb740 1 client.27364 shutdown 2024-07-04T08:23:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.010+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3093690864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 msgr2=0x560dc4db7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.010+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4db7ef0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f34d000f6a0 tx=0x7f34d0004ab0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.010+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3093690864 shutdown_connections 2024-07-04T08:23:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.010+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560dc49ba1c0 0x560dc4dbdf90 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.010+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4db7ef0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.011+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49dab60 0x560dc4dac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.011+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3093690864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49d2980 0x560dc4dabed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.011+0000 7f34c0c00640 0 client.27364 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.011+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3093690864 >> 172.21.15.39:0/3093690864 conn(0x560dc49ef990 msgr2=0x560dc4da35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.011+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3093690864 shutdown_connections 2024-07-04T08:23:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.011+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3093690864 wait complete. 2024-07-04T08:23:29.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.011+0000 7f34e3dfb740 20 client.27364 trim_cache size 0 max 16384 2024-07-04T08:23:29.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.032+0000 7f34e3dfb740 1 Processor -- start 2024-07-04T08:23:29.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.033+0000 7f34e3dfb740 1 -- start start 2024-07-04T08:23:29.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.033+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4d9d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:29.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.033+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4d9ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:29.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.033+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4dc3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:29.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.033+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34d81b25b0 con 0x560dc49fb840 2024-07-04T08:23:29.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.033+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34d81b40f0 con 0x560dc49d2980 2024-07-04T08:23:29.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.033+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34d81b2c80 con 0x560dc49dab60 2024-07-04T08:23:29.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4d9d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4d9ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4d9d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35510/0 (socket says 172.21.15.39:35510) 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6a00640 1 -- 172.21.15.39:0/3716157938 learned_addr learned my addr 172.21.15.39:0/3716157938 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6a00640 1 -- 172.21.15.39:0/3716157938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 msgr2=0x560dc4d9ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6a00640 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4d9ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6a00640 1 -- 172.21.15.39:0/3716157938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 msgr2=0x560dc4dc3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d7400640 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4dc3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6a00640 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4dc3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6a00640 1 -- 172.21.15.39:0/3716157938 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dc4dacf90 con 0x560dc49d2980 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d6000640 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4d9ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:29.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.034+0000 7f34d7400640 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4dc3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:23:29.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.035+0000 7f34d6a00640 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4d9d880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f34c80c9c40 tx=0x7f34c802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1585a7679d89373 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:29.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.035+0000 7f34c3400640 1 -- 172.21.15.39:0/3716157938 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8049040 con 0x560dc49d2980 2024-07-04T08:23:29.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.035+0000 7f34c3400640 1 -- 172.21.15.39:0/3716157938 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c80c9cb0 con 0x560dc49d2980 2024-07-04T08:23:29.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.035+0000 7f34c3400640 1 -- 172.21.15.39:0/3716157938 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c81002f0 con 0x560dc49d2980 2024-07-04T08:23:29.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.036+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3716157938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 msgr2=0x560dc4d9d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.036+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4d9d880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f34c80c9c40 tx=0x7f34c802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.036+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3716157938 shutdown_connections 2024-07-04T08:23:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.036+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49fb840 0x560dc4dc3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.036+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49dab60 0x560dc4d9ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.036+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/3716157938 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4d9d880 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.036+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3716157938 >> 172.21.15.39:0/3716157938 conn(0x560dc49ef990 msgr2=0x560dc4dad1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.036+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3716157938 shutdown_connections 2024-07-04T08:23:29.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.037+0000 7f34e3dfb740 1 -- 172.21.15.39:0/3716157938 wait complete. 2024-07-04T08:23:29.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.037+0000 7f34e3dfb740 1 -- start start 2024-07-04T08:23:29.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.037+0000 7f34e3dfb740 1 Processor -- start 2024-07-04T08:23:29.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4db04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:29.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49dab60 0x560dc4db0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:29.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34e3dfb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4e25720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34d81b3a20 con 0x560dc49dab60 2024-07-04T08:23:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34d81b3350 con 0x560dc49d2980 2024-07-04T08:23:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34e3dfb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34d81b0590 con 0x560dc49fb840 2024-07-04T08:23:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34e3dfb740 10 client.0 _ll_register_callbacks cb 0x7f34d0011050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34d6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49dab60 0x560dc4db0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34d6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49dab60 0x560dc4db0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56146/0 (socket says 172.21.15.39:56146) 2024-07-04T08:23:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34d6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4db04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34d6000640 1 -- 172.21.15.39:0/1712505822 learned_addr learned my addr 172.21.15.39:0/1712505822 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:29.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.038+0000 7f34d7400640 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4e25720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34d6a00640 1 -- 172.21.15.39:0/1712505822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 msgr2=0x560dc4e25720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34d6a00640 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4e25720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34d6a00640 1 -- 172.21.15.39:0/1712505822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49dab60 msgr2=0x560dc4db0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34d6a00640 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49dab60 0x560dc4db0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34d6a00640 1 -- 172.21.15.39:0/1712505822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dc4e25cb0 con 0x560dc49d2980 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34d6a00640 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4db04f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f34c80c9c40 tx=0x7f34c80c98f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76028cb96ac15b6a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34c0c00640 1 -- 172.21.15.39:0/1712505822 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8049040 con 0x560dc49d2980 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34c0c00640 1 -- 172.21.15.39:0/1712505822 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f34c80c62a0 con 0x560dc49d2980 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.039+0000 7f34e3dfb740 1 -- 172.21.15.39:0/1712505822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c80c9cb0 con 0x560dc49d2980 2024-07-04T08:23:29.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34c0c00640 1 -- 172.21.15.39:0/1712505822 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34c8100500 con 0x560dc49d2980 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34e3dfb740 10 client.27449 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34e3dfb740 1 -- 172.21.15.39:0/1712505822 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560dc4e26e80 con 0x560dc49d2980 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34c0c00640 1 -- 172.21.15.39:0/1712505822 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c80c9cb0 con 0x560dc49d2980 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34c0c00640 1 -- 172.21.15.39:0/1712505822 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f34c80c5620 con 0x560dc49d2980 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34e3dfb740 10 client.27449 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34e3dfb740 10 client.27449 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34e3dfb740 20 client.27449 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560dc498f570 0x560dc4e29620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34e3dfb740 4 client.27449 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.040+0000 7f34e3dfb740 1 -- 172.21.15.39:0/1712505822 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x560dc4d136d0 con 0x560dc498f570 2024-07-04T08:23:29.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.041+0000 7f34d6000640 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560dc498f570 0x560dc4e29620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:29.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.041+0000 7f34d6000640 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560dc498f570 0x560dc4e29620 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=458f87a18de62722 server_cookie=8d66c138f4966210 in_seq=0 out_seq=0 2024-07-04T08:23:29.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.041+0000 7f34c0c00640 10 client.27449 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:29.039 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:29 smithi039 ceph-mon[32062]: pgmap v732: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-07-04T08:23:29.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34c0c00640 1 -- 172.21.15.39:0/1712505822 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560dc4d136d0 con 0x560dc498f570 2024-07-04T08:23:29.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34c0c00640 10 client.27449 handle_command_reply: tid=0 2024-07-04T08:23:29.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 client.27449 shutdown 2024-07-04T08:23:29.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 -- 172.21.15.39:0/1712505822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 msgr2=0x560dc4db04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:29.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4db04f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f34c80c9c40 tx=0x7f34c80c98f0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 -- 172.21.15.39:0/1712505822 shutdown_connections 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560dc498f570 0x560dc4e29620 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560dc49dab60 0x560dc4db0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dc49fb840 0x560dc4e25720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 --2- 172.21.15.39:0/1712505822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560dc49d2980 0x560dc4db04f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34c0c00640 0 client.27449 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 -- 172.21.15.39:0/1712505822 >> 172.21.15.39:0/1712505822 conn(0x560dc49ef990 msgr2=0x560dc4daa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 -- 172.21.15.39:0/1712505822 shutdown_connections 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 1 -- 172.21.15.39:0/1712505822 wait complete. 2024-07-04T08:23:29.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.052+0000 7f34e3dfb740 20 client.27449 trim_cache size 0 max 16384 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 22, 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout: "3925f374": { 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 22, 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:23:29.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 140.1, 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:29.048 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:29.049 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:29.049 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:29.049 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:23:29.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 -- 172.21.15.39:0/1114954531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b29e0 msgr2=0x7f34bc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:29.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b29e0 0x7f34bc0b4ef0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009fd0 tx=0x7f34d0009500 comp rx=0 tx=0).stop 2024-07-04T08:23:29.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 -- 172.21.15.39:0/1114954531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d81030d0 msgr2=0x7f34d819bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:29.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d81030d0 0x7f34d819bc80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f34c8005940 tx=0x7f34c80040c0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 -- 172.21.15.39:0/1114954531 shutdown_connections 2024-07-04T08:23:29.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b29e0 0x7f34bc0b4ef0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34d8108a10 0x7f34d819c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34d8109360 0x7f34d81a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 --2- 172.21.15.39:0/1114954531 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34d81030d0 0x7f34d819bc80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:29.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 -- 172.21.15.39:0/1114954531 >> 172.21.15.39:0/1114954531 conn(0x7f34d80feb20 msgr2=0x7f34d8108140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:29.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.053+0000 7f34dd800640 1 -- 172.21.15.39:0/1114954531 shutdown_connections 2024-07-04T08:23:29.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:29.054+0000 7f34dd800640 1 -- 172.21.15.39:0/1114954531 wait complete. 2024-07-04T08:23:29.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:29 smithi133 ceph-mon[37050]: pgmap v732: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-07-04T08:23:29.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:29 smithi050 ceph-mon[36577]: pgmap v732: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-07-04T08:23:30.572 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3925f374', quiesce took 8.6 seconds. Will release after: 11.500000000000002 2024-07-04T08:23:31.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:31 smithi133 ceph-mon[37050]: pgmap v733: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-07-04T08:23:31.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:31 smithi039 ceph-mon[32062]: pgmap v733: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-07-04T08:23:31.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:31 smithi050 ceph-mon[36577]: pgmap v733: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 14 op/s 2024-07-04T08:23:31.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.928+0000 7fb485600640 1 -- 172.21.15.39:0/3740850725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb480108240 msgr2=0x7fb48010ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:31.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.928+0000 7fb485600640 1 --2- 172.21.15.39:0/3740850725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb480108240 0x7fb48010ea70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb478009be0 tx=0x7fb47802a6c0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.928+0000 7fb485600640 1 -- 172.21.15.39:0/3740850725 shutdown_connections 2024-07-04T08:23:31.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.928+0000 7fb485600640 1 --2- 172.21.15.39:0/3740850725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb480108240 0x7fb48010ea70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.928+0000 7fb485600640 1 --2- 172.21.15.39:0/3740850725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4801078f0 0x7fb480107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.928+0000 7fb485600640 1 --2- 172.21.15.39:0/3740850725 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb480101fb0 0x7fb4801023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.928+0000 7fb485600640 1 -- 172.21.15.39:0/3740850725 >> 172.21.15.39:0/3740850725 conn(0x7fb480078c80 msgr2=0x7fb480079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:31.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.928+0000 7fb485600640 1 -- 172.21.15.39:0/3740850725 shutdown_connections 2024-07-04T08:23:31.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.928+0000 7fb485600640 1 -- 172.21.15.39:0/3740850725 wait complete. 2024-07-04T08:23:31.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.930+0000 7fb485600640 1 Processor -- start 2024-07-04T08:23:31.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.930+0000 7fb485600640 1 -- start start 2024-07-04T08:23:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb485600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb480101fb0 0x7fb48019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb485600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4801078f0 0x7fb48019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb485600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb480108240 0x7fb4801a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb485600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb480111eb0 con 0x7fb4801078f0 2024-07-04T08:23:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb485600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb480111d30 con 0x7fb480108240 2024-07-04T08:23:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb485600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb480112030 con 0x7fb480101fb0 2024-07-04T08:23:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb477e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4801078f0 0x7fb48019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb47ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb480101fb0 0x7fb48019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb477e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4801078f0 0x7fb48019c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35464/0 (socket says 172.21.15.39:35464) 2024-07-04T08:23:31.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb477e00640 1 -- 172.21.15.39:0/601047983 learned_addr learned my addr 172.21.15.39:0/601047983 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.931+0000 7fb47f400640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb480108240 0x7fb4801a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.932+0000 7fb47ea00640 1 -- 172.21.15.39:0/601047983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb480108240 msgr2=0x7fb4801a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.932+0000 7fb47ea00640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb480108240 0x7fb4801a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.932+0000 7fb47ea00640 1 -- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4801078f0 msgr2=0x7fb48019c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.932+0000 7fb47ea00640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4801078f0 0x7fb48019c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.932+0000 7fb47ea00640 1 -- 172.21.15.39:0/601047983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4801a1b60 con 0x7fb480101fb0 2024-07-04T08:23:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.932+0000 7fb477e00640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4801078f0 0x7fb48019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:31.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.932+0000 7fb47ea00640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb480101fb0 0x7fb48019bbf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb47000e9f0 tx=0x7fb47000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:31.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.932+0000 7fb477400640 1 -- 172.21.15.39:0/601047983 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47000ccc0 con 0x7fb480101fb0 2024-07-04T08:23:31.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.932+0000 7fb477400640 1 -- 172.21.15.39:0/601047983 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4700035f0 con 0x7fb480101fb0 2024-07-04T08:23:31.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.933+0000 7fb485600640 1 -- 172.21.15.39:0/601047983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4801a1e50 con 0x7fb480101fb0 2024-07-04T08:23:31.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.933+0000 7fb477400640 1 -- 172.21.15.39:0/601047983 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47000ce60 con 0x7fb480101fb0 2024-07-04T08:23:31.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.933+0000 7fb485600640 1 -- 172.21.15.39:0/601047983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4801a85b0 con 0x7fb480101fb0 2024-07-04T08:23:31.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.937+0000 7fb48bd1c740 1 Processor -- start 2024-07-04T08:23:31.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.937+0000 7fb48bd1c740 1 -- start start 2024-07-04T08:23:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04c5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bdd043bea0 con 0x55bdd0121070 2024-07-04T08:23:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bdd042b440 con 0x55bdd00f81b0 2024-07-04T08:23:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bdd042f480 con 0x55bdd0100390 2024-07-04T08:23:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb47f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb47f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35480/0 (socket says 172.21.15.39:35480) 2024-07-04T08:23:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb47f400640 1 -- 172.21.15.39:0/4258782190 learned_addr learned my addr 172.21.15.39:0/4258782190 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb477400640 1 -- 172.21.15.39:0/601047983 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb4700037a0 con 0x7fb480101fb0 2024-07-04T08:23:31.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.938+0000 7fb477e00640 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.939+0000 7fb47f400640 1 -- 172.21.15.39:0/4258782190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 msgr2=0x55bdd04c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.939+0000 7fb47f400640 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.939+0000 7fb47ea00640 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.939+0000 7fb47f400640 1 -- 172.21.15.39:0/4258782190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 msgr2=0x55bdd04c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:31.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.939+0000 7fb47f400640 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.939+0000 7fb47f400640 1 -- 172.21.15.39:0/4258782190 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdd03c3610 con 0x55bdd0121070 2024-07-04T08:23:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.939+0000 7fb47ea00640 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:23:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.939+0000 7fb47f400640 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04c5770 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb478009be0 tx=0x7fb478030430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a54e8098f63390 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.940+0000 7fb46fe00640 1 -- 172.21.15.39:0/4258782190 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478041030 con 0x55bdd0121070 2024-07-04T08:23:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.940+0000 7fb46fe00640 1 -- 172.21.15.39:0/4258782190 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478003e20 con 0x55bdd0121070 2024-07-04T08:23:31.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.940+0000 7fb46fe00640 1 -- 172.21.15.39:0/4258782190 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478036040 con 0x55bdd0121070 2024-07-04T08:23:31.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb477400640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4640b29e0 0x7fb4640b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb48bd1c740 1 -- 172.21.15.39:0/4258782190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 msgr2=0x55bdd04c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:31.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04c5770 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb478009be0 tx=0x7fb478030430 comp rx=0 tx=0).stop 2024-07-04T08:23:31.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb477400640 1 -- 172.21.15.39:0/601047983 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7fb4700e2b70 con 0x7fb480101fb0 2024-07-04T08:23:31.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb47ea00640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4640b29e0 0x7fb4640b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb48bd1c740 1 -- 172.21.15.39:0/4258782190 shutdown_connections 2024-07-04T08:23:31.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/4258782190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04c5770 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb48bd1c740 1 -- 172.21.15.39:0/4258782190 >> 172.21.15.39:0/4258782190 conn(0x55bdd0115460 msgr2=0x55bdd04c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb48bd1c740 1 -- 172.21.15.39:0/4258782190 shutdown_connections 2024-07-04T08:23:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.941+0000 7fb48bd1c740 1 -- 172.21.15.39:0/4258782190 wait complete. 2024-07-04T08:23:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.942+0000 7fb47ea00640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4640b29e0 0x7fb4640b4ef0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55bdd04c43f0 tx=0x7fb4700a5a00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.942+0000 7fb48bd1c740 1 -- start start 2024-07-04T08:23:31.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.942+0000 7fb48bd1c740 1 Processor -- start 2024-07-04T08:23:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.943+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.943+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0100390 0x55bdd04d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.943+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0121070 0x55bdd04dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.943+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bdd028f700 con 0x55bdd0100390 2024-07-04T08:23:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.943+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bdd03d2b70 con 0x55bdd00f81b0 2024-07-04T08:23:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.943+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bdd04d3ef0 con 0x55bdd0121070 2024-07-04T08:23:31.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.943+0000 7fb48bd1c740 10 client.0 _ll_register_callbacks cb 0x55bdd0424530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb477e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0100390 0x55bdd04d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb477e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0100390 0x55bdd04d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35496/0 (socket says 172.21.15.39:35496) 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb477e00640 1 -- 172.21.15.39:0/1168621868 learned_addr learned my addr 172.21.15.39:0/1168621868 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb47f400640 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb47f400640 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0121070 0x55bdd04dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb477e00640 1 -- 172.21.15.39:0/1168621868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0121070 msgr2=0x55bdd04dd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb477e00640 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0121070 0x55bdd04dd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb477e00640 1 -- 172.21.15.39:0/1168621868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 msgr2=0x55bdd04d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb477e00640 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb477e00640 1 -- 172.21.15.39:0/1168621868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdd018c650 con 0x55bdd0100390 2024-07-04T08:23:31.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb47f400640 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0121070 0x55bdd04dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb477e00640 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0100390 0x55bdd04d18c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb468012d50 tx=0x7fb46800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f876c5d719960625 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.944+0000 7fb46cc00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.945+0000 7fb46cc00640 1 -- 172.21.15.39:0/1168621868 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46800fe20 con 0x55bdd0100390 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.945+0000 7fb46cc00640 1 -- 172.21.15.39:0/1168621868 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7fb468004030 con 0x55bdd0100390 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.945+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1168621868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb478003e20 con 0x55bdd0100390 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.945+0000 7fb46cc00640 1 -- 172.21.15.39:0/1168621868 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb468025050 con 0x55bdd0100390 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.945+0000 7fb48bd1c740 10 client.17322 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.945+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1168621868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bdd04dff20 con 0x55bdd0100390 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.946+0000 7fb46cc00640 1 -- 172.21.15.39:0/1168621868 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478003e20 con 0x55bdd0100390 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.946+0000 7fb46cc00640 1 -- 172.21.15.39:0/1168621868 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb468010430 con 0x55bdd0100390 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.946+0000 7fb48bd1c740 10 client.17322 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.946+0000 7fb48bd1c740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:23:31.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.946+0000 7fb48bd1c740 20 client.17322 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:31.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.946+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bdd00df9f0 0x55bdd04e3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.946+0000 7fb48bd1c740 4 client.17322 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:23:31.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.946+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1168621868 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdd043ddf0 con 0x55bdd00df9f0 2024-07-04T08:23:31.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.946+0000 7fb47f400640 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bdd00df9f0 0x55bdd04e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.947+0000 7fb47f400640 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bdd00df9f0 0x55bdd04e3450 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56d314a59c9607ff server_cookie=2305eb0fa6ff8834 in_seq=0 out_seq=0 2024-07-04T08:23:31.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.947+0000 7fb46cc00640 10 client.17322 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:31.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb46cc00640 1 -- 172.21.15.39:0/1168621868 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bdd043ddf0 con 0x55bdd00df9f0 2024-07-04T08:23:31.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb46cc00640 10 client.17322 handle_command_reply: tid=0 2024-07-04T08:23:31.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb48bd1c740 1 client.17322 shutdown 2024-07-04T08:23:31.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1168621868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0100390 msgr2=0x55bdd04d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:31.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0100390 0x55bdd04d18c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb468012d50 tx=0x7fb46800f790 comp rx=0 tx=0).stop 2024-07-04T08:23:31.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1168621868 shutdown_connections 2024-07-04T08:23:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bdd00df9f0 0x55bdd04e3450 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0100390 0x55bdd04d18c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0121070 0x55bdd04dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1168621868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb46cc00640 0 client.17322 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.948+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1168621868 >> 172.21.15.39:0/1168621868 conn(0x55bdd0115460 msgr2=0x55bdd04c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.949+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1168621868 shutdown_connections 2024-07-04T08:23:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.949+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1168621868 wait complete. 2024-07-04T08:23:31.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.949+0000 7fb48bd1c740 20 client.17322 trim_cache size 0 max 16384 2024-07-04T08:23:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.969+0000 7fb48bd1c740 1 Processor -- start 2024-07-04T08:23:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.969+0000 7fb48bd1c740 1 -- start start 2024-07-04T08:23:31.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.969+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd00f81b0 0x55bdd04c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.969+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd0100390 0x55bdd04d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.969+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.969+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b2590 con 0x55bdd0121070 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.969+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b40d0 con 0x55bdd0100390 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.969+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b2c60 con 0x55bdd00f81b0 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35510/0 (socket says 172.21.15.39:35510) 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 -- 172.21.15.39:0/1155879574 learned_addr learned my addr 172.21.15.39:0/1155879574 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb477e00640 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd0100390 0x55bdd04d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd00f81b0 0x55bdd04c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 -- 172.21.15.39:0/1155879574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd00f81b0 msgr2=0x55bdd04c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd00f81b0 0x55bdd04c2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 -- 172.21.15.39:0/1155879574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd0100390 msgr2=0x55bdd04d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd0100390 0x55bdd04d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 -- 172.21.15.39:0/1155879574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdd04c3ba0 con 0x55bdd0121070 2024-07-04T08:23:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb477e00640 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd0100390 0x55bdd04d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:31.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.970+0000 7fb47f400640 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04ca850 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb478030470 tx=0x7fb478030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15026efed180e9ea server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.971+0000 7fb46f400640 1 -- 172.21.15.39:0/1155879574 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478036040 con 0x55bdd0121070 2024-07-04T08:23:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.971+0000 7fb46f400640 1 -- 172.21.15.39:0/1155879574 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478003e20 con 0x55bdd0121070 2024-07-04T08:23:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.971+0000 7fb46f400640 1 -- 172.21.15.39:0/1155879574 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478041040 con 0x55bdd0121070 2024-07-04T08:23:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.972+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1155879574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 msgr2=0x55bdd04ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.972+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04ca850 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb478030470 tx=0x7fb478030d70 comp rx=0 tx=0).stop 2024-07-04T08:23:31.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.972+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1155879574 shutdown_connections 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.972+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04ca850 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.972+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd0100390 0x55bdd04d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.972+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/1155879574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd00f81b0 0x55bdd04c2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.972+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1155879574 >> 172.21.15.39:0/1155879574 conn(0x55bdd0115460 msgr2=0x55bdd04d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.972+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1155879574 shutdown_connections 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.972+0000 7fb48bd1c740 1 -- 172.21.15.39:0/1155879574 wait complete. 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.973+0000 7fb48bd1c740 1 -- start start 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.973+0000 7fb48bd1c740 1 Processor -- start 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.973+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.973+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.973+0000 7fb48bd1c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04d8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b3a00 con 0x55bdd0121070 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b3330 con 0x55bdd00f81b0 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb48bd1c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b0570 con 0x55bdd0100390 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb48bd1c740 10 client.0 _ll_register_callbacks cb 0x7fb468009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb47f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04dc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb477e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb47f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04d8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb477e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04dccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:45070/0 (socket says 172.21.15.39:45070) 2024-07-04T08:23:31.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb477e00640 1 -- 172.21.15.39:0/328352700 learned_addr learned my addr 172.21.15.39:0/328352700 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb477e00640 1 -- 172.21.15.39:0/328352700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 msgr2=0x55bdd04dc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb477e00640 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04dc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb477e00640 1 -- 172.21.15.39:0/328352700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 msgr2=0x55bdd04d8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb477e00640 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04d8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.974+0000 7fb477e00640 1 -- 172.21.15.39:0/328352700 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdd04d8680 con 0x55bdd0100390 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.975+0000 7fb477e00640 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04dccf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb4680109a0 tx=0x7fb468012850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17e9e5979a50a29 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.975+0000 7fb46cc00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.975+0000 7fb46cc00640 1 -- 172.21.15.39:0/328352700 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468022040 con 0x55bdd0100390 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.975+0000 7fb46cc00640 1 -- 172.21.15.39:0/328352700 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7fb468010430 con 0x55bdd0100390 2024-07-04T08:23:31.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.975+0000 7fb48bd1c740 1 -- 172.21.15.39:0/328352700 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb478003e20 con 0x55bdd0100390 2024-07-04T08:23:31.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.975+0000 7fb46cc00640 1 -- 172.21.15.39:0/328352700 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb468036050 con 0x55bdd0100390 2024-07-04T08:23:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.975+0000 7fb48bd1c740 10 client.27388 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.975+0000 7fb48bd1c740 1 -- 172.21.15.39:0/328352700 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bdd04d9940 con 0x55bdd0100390 2024-07-04T08:23:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.976+0000 7fb46cc00640 1 -- 172.21.15.39:0/328352700 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb478003e20 con 0x55bdd0100390 2024-07-04T08:23:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.976+0000 7fb46cc00640 1 -- 172.21.15.39:0/328352700 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb468031030 con 0x55bdd0100390 2024-07-04T08:23:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.976+0000 7fb48bd1c740 10 client.27388 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.976+0000 7fb48bd1c740 10 client.27388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:23:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.976+0000 7fb48bd1c740 20 client.27388 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.976+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bdd00b4da0 0x55bdd054ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:31.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.976+0000 7fb48bd1c740 4 client.27388 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.976+0000 7fb48bd1c740 1 -- 172.21.15.39:0/328352700 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bdd0439890 con 0x55bdd00b4da0 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.977+0000 7fb47f400640 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bdd00b4da0 0x55bdd054ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.977+0000 7fb47f400640 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bdd00b4da0 0x55bdd054ddb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d0024d5751167c server_cookie=bd3589049699f9b8 in_seq=0 out_seq=0 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.977+0000 7fb46cc00640 10 client.27388 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.977+0000 7fb46cc00640 1 -- 172.21.15.39:0/328352700 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bdd0439890 con 0x55bdd00b4da0 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.977+0000 7fb46cc00640 10 client.27388 handle_command_reply: tid=0 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.977+0000 7fb48bd1c740 1 client.27388 shutdown 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 -- 172.21.15.39:0/328352700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 msgr2=0x55bdd04dccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04dccf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb4680109a0 tx=0x7fb468012850 comp rx=0 tx=0).stop 2024-07-04T08:23:31.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 -- 172.21.15.39:0/328352700 shutdown_connections 2024-07-04T08:23:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55bdd00b4da0 0x55bdd054ddb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bdd0121070 0x55bdd04d8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bdd0100390 0x55bdd04dccf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 --2- 172.21.15.39:0/328352700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bdd00f81b0 0x55bdd04dc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb46cc00640 0 client.27388 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 -- 172.21.15.39:0/328352700 >> 172.21.15.39:0/328352700 conn(0x55bdd0115460 msgr2=0x55bdd04d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 -- 172.21.15.39:0/328352700 shutdown_connections 2024-07-04T08:23:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 1 -- 172.21.15.39:0/328352700 wait complete. 2024-07-04T08:23:31.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.978+0000 7fb48bd1c740 20 client.27388 trim_cache size 0 max 16384 2024-07-04T08:23:31.977 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:23:31.977 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:23:31.977 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:23:31.977 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:23:31.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 -- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4640b29e0 msgr2=0x7fb4640b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:31.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4640b29e0 0x7fb4640b4ef0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55bdd04c43f0 tx=0x7fb4700a5a00 comp rx=0 tx=0).stop 2024-07-04T08:23:31.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 -- 172.21.15.39:0/601047983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb480101fb0 msgr2=0x7fb48019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:31.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb480101fb0 0x7fb48019bbf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb47000e9f0 tx=0x7fb47000edf0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 -- 172.21.15.39:0/601047983 shutdown_connections 2024-07-04T08:23:31.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4640b29e0 0x7fb4640b4ef0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4801078f0 0x7fb48019c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb480108240 0x7fb4801a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 --2- 172.21.15.39:0/601047983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb480101fb0 0x7fb48019bbf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:31.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.979+0000 7fb485600640 1 -- 172.21.15.39:0/601047983 >> 172.21.15.39:0/601047983 conn(0x7fb480078c80 msgr2=0x7fb48006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:31.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.980+0000 7fb485600640 1 -- 172.21.15.39:0/601047983 shutdown_connections 2024-07-04T08:23:31.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:31.980+0000 7fb485600640 1 -- 172.21.15.39:0/601047983 wait complete. 2024-07-04T08:23:32.876 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:23:33.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:23:33.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:33 smithi039 ceph-mon[32062]: pgmap v734: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 16 op/s 2024-07-04T08:23:33.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:33 smithi133 ceph-mon[37050]: pgmap v734: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 16 op/s 2024-07-04T08:23:33.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:33 smithi050 ceph-mon[36577]: pgmap v734: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 58 KiB/s wr, 16 op/s 2024-07-04T08:23:33.503 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:35.235 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:35 smithi039 ceph-mon[32062]: pgmap v735: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-07-04T08:23:35.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:35 smithi133 ceph-mon[37050]: pgmap v735: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-07-04T08:23:35.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:35 smithi050 ceph-mon[36577]: pgmap v735: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-07-04T08:23:35.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.802+0000 7f836ec00640 1 -- 172.21.15.39:0/337831941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8368109510 msgr2=0x7f8368109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:35.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.802+0000 7f836ec00640 1 --2- 172.21.15.39:0/337831941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8368109510 0x7f8368109920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f835c009900 tx=0x7f835c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:23:35.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.802+0000 7f836ec00640 1 -- 172.21.15.39:0/337831941 shutdown_connections 2024-07-04T08:23:35.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.802+0000 7f836ec00640 1 --2- 172.21.15.39:0/337831941 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f836810ee50 0x7f836810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:35.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.802+0000 7f836ec00640 1 --2- 172.21.15.39:0/337831941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f836810f7a0 0x7f8368115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:35.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.802+0000 7f836ec00640 1 --2- 172.21.15.39:0/337831941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8368109510 0x7f8368109920 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:35.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.802+0000 7f836ec00640 1 -- 172.21.15.39:0/337831941 >> 172.21.15.39:0/337831941 conn(0x7f8368078b10 msgr2=0x7f8368107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:35.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.802+0000 7f836ec00640 1 -- 172.21.15.39:0/337831941 shutdown_connections 2024-07-04T08:23:35.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.802+0000 7f836ec00640 1 -- 172.21.15.39:0/337831941 wait complete. 2024-07-04T08:23:35.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.803+0000 7f836ec00640 1 Processor -- start 2024-07-04T08:23:35.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.804+0000 7f836ec00640 1 -- start start 2024-07-04T08:23:35.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.804+0000 7f836ec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8368109510 0x7f836819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.804+0000 7f836ec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f836810ee50 0x7f83681a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.804+0000 7f836ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f836810f7a0 0x7f83681a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.804+0000 7f836ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83681194a0 con 0x7f836810f7a0 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.804+0000 7f836ec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8368119320 con 0x7f836810ee50 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.804+0000 7f836ec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8368119620 con 0x7f8368109510 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.804+0000 7f836ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f836810ee50 0x7f83681a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8368109510 0x7f836819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f836810ee50 0x7f83681a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:55300/0 (socket says 172.21.15.39:55300) 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836ce00640 1 -- 172.21.15.39:0/3874509904 learned_addr learned my addr 172.21.15.39:0/3874509904 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:35.799 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836e200640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f836810f7a0 0x7f83681a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:35.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836ce00640 1 -- 172.21.15.39:0/3874509904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8368109510 msgr2=0x7f836819fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:35.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836ce00640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8368109510 0x7f836819fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:35.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836ce00640 1 -- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f836810f7a0 msgr2=0x7f83681a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:35.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836ce00640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f836810f7a0 0x7f83681a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:35.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836ce00640 1 -- 172.21.15.39:0/3874509904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83681a5e40 con 0x7f836810ee50 2024-07-04T08:23:35.800 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.805+0000 7f836ce00640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f836810ee50 0x7f83681a0470 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f835400e9f0 tx=0x7f835400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:35.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.806+0000 7f8366000640 1 -- 172.21.15.39:0/3874509904 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835400ccc0 con 0x7f836810ee50 2024-07-04T08:23:35.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.806+0000 7f8366000640 1 -- 172.21.15.39:0/3874509904 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83540108e0 con 0x7f836810ee50 2024-07-04T08:23:35.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.806+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83681a6130 con 0x7f836810ee50 2024-07-04T08:23:35.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.806+0000 7f8366000640 1 -- 172.21.15.39:0/3874509904 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354018860 con 0x7f836810ee50 2024-07-04T08:23:35.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.806+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681ac890 con 0x7f836810ee50 2024-07-04T08:23:35.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.807+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8368109920 con 0x7f836810ee50 2024-07-04T08:23:35.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.810+0000 7f8366000640 1 -- 172.21.15.39:0/3874509904 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8354010430 con 0x7f836810ee50 2024-07-04T08:23:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.812+0000 7f8366000640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83500b29e0 0x7f83500b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:35.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.812+0000 7f8366000640 1 -- 172.21.15.39:0/3874509904 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f83540e30e0 con 0x7f836810ee50 2024-07-04T08:23:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.813+0000 7f836d800640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83500b29e0 0x7f83500b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.813+0000 7f8366000640 1 -- 172.21.15.39:0/3874509904 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f83540aa0b0 con 0x7f836810ee50 2024-07-04T08:23:35.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:35.814+0000 7f836d800640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83500b29e0 0x7f83500b4ef0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f835c005ab0 tx=0x7f835c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:36.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.069+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83681a1000 con 0x7f836810ee50 2024-07-04T08:23:36.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.070+0000 7f8366000640 1 -- 172.21.15.39:0/3874509904 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f83540ae3b0 con 0x7f836810ee50 2024-07-04T08:23:36.063 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:23:36.064 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:23:36.064 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:23:36.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83500b29e0 msgr2=0x7f83500b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:36.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83500b29e0 0x7f83500b4ef0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f835c005ab0 tx=0x7f835c005a20 comp rx=0 tx=0).stop 2024-07-04T08:23:36.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f836810ee50 msgr2=0x7f83681a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:36.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f836810ee50 0x7f83681a0470 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f835400e9f0 tx=0x7f835400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:23:36.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 shutdown_connections 2024-07-04T08:23:36.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83500b29e0 0x7f83500b4ef0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:36.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f836810f7a0 0x7f83681a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:36.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f836810ee50 0x7f83681a0470 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:36.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 --2- 172.21.15.39:0/3874509904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8368109510 0x7f836819fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:36.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 >> 172.21.15.39:0/3874509904 conn(0x7f8368078b10 msgr2=0x7f8368107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:36.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 shutdown_connections 2024-07-04T08:23:36.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:36.073+0000 7f836ec00640 1 -- 172.21.15.39:0/3874509904 wait complete. 2024-07-04T08:23:36.958 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:23:37.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:37 smithi133 ceph-mon[37050]: pgmap v736: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-07-04T08:23:37.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:37 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3874509904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:23:37.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:37 smithi039 ceph-mon[32062]: pgmap v736: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-07-04T08:23:37.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:37 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3874509904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:23:37.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:37 smithi050 ceph-mon[36577]: pgmap v736: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-07-04T08:23:37.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:37 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3874509904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:23:37.438 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:39.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:39 smithi039 ceph-mon[32062]: pgmap v737: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-07-04T08:23:39.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:39 smithi050 ceph-mon[36577]: pgmap v737: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-07-04T08:23:39.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:39 smithi133 ceph-mon[37050]: pgmap v737: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-07-04T08:23:39.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.830+0000 7fb856c00640 1 -- 172.21.15.39:0/3513668078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8501089f0 msgr2=0x7fb850108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.830+0000 7fb856c00640 1 --2- 172.21.15.39:0/3513668078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8501089f0 0x7fb850108e00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb838009be0 tx=0x7fb83802a6d0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.830+0000 7fb856c00640 1 -- 172.21.15.39:0/3513668078 shutdown_connections 2024-07-04T08:23:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.830+0000 7fb856c00640 1 --2- 172.21.15.39:0/3513668078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb850109340 0x7fb85010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.830+0000 7fb856c00640 1 --2- 172.21.15.39:0/3513668078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8501089f0 0x7fb850108e00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.830+0000 7fb856c00640 1 --2- 172.21.15.39:0/3513668078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8501030b0 0x7fb8501034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.830+0000 7fb856c00640 1 -- 172.21.15.39:0/3513668078 >> 172.21.15.39:0/3513668078 conn(0x7fb85007ada0 msgr2=0x7fb85007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.830+0000 7fb856c00640 1 -- 172.21.15.39:0/3513668078 shutdown_connections 2024-07-04T08:23:39.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.830+0000 7fb856c00640 1 -- 172.21.15.39:0/3513668078 wait complete. 2024-07-04T08:23:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.831+0000 7fb856c00640 1 Processor -- start 2024-07-04T08:23:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.831+0000 7fb856c00640 1 -- start start 2024-07-04T08:23:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb856c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8501030b0 0x7fb85019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb856c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8501089f0 0x7fb85019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb856c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb850109340 0x7fb8501a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb856c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb850112fb0 con 0x7fb850109340 2024-07-04T08:23:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb856c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb850112e30 con 0x7fb8501030b0 2024-07-04T08:23:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb856c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb850113130 con 0x7fb8501089f0 2024-07-04T08:23:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb854c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb850109340 0x7fb8501a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb854c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb850109340 0x7fb8501a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35572/0 (socket says 172.21.15.39:35572) 2024-07-04T08:23:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb854c00640 1 -- 172.21.15.39:0/3500315042 learned_addr learned my addr 172.21.15.39:0/3500315042 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb84fe00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8501030b0 0x7fb85019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.832+0000 7fb847400640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8501089f0 0x7fb85019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.833+0000 7fb854c00640 1 -- 172.21.15.39:0/3500315042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8501089f0 msgr2=0x7fb85019c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.833+0000 7fb854c00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8501089f0 0x7fb85019c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.833+0000 7fb854c00640 1 -- 172.21.15.39:0/3500315042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8501030b0 msgr2=0x7fb85019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.833+0000 7fb854c00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8501030b0 0x7fb85019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.833+0000 7fb854c00640 1 -- 172.21.15.39:0/3500315042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8501a1c70 con 0x7fb850109340 2024-07-04T08:23:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.833+0000 7fb854c00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb850109340 0x7fb8501a1540 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb84800e940 tx=0x7fb84800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:39.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.834+0000 7fb84d600640 1 -- 172.21.15.39:0/3500315042 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84800cd80 con 0x7fb850109340 2024-07-04T08:23:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.834+0000 7fb856c00640 1 -- 172.21.15.39:0/3500315042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8501a1f60 con 0x7fb850109340 2024-07-04T08:23:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.834+0000 7fb84d600640 1 -- 172.21.15.39:0/3500315042 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8480035f0 con 0x7fb850109340 2024-07-04T08:23:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.834+0000 7fb856c00640 1 -- 172.21.15.39:0/3500315042 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8501a86c0 con 0x7fb850109340 2024-07-04T08:23:39.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.834+0000 7fb84d600640 1 -- 172.21.15.39:0/3500315042 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848010630 con 0x7fb850109340 2024-07-04T08:23:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.837+0000 7fb85d2d9740 1 Processor -- start 2024-07-04T08:23:39.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.838+0000 7fb84d600640 1 -- 172.21.15.39:0/3500315042 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb848003ac0 con 0x7fb850109340 2024-07-04T08:23:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.838+0000 7fb85d2d9740 1 -- start start 2024-07-04T08:23:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.838+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80bf070 0x55e2d84637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.838+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d8463d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.838+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8464250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.838+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e2d83d9f00 con 0x55e2d80961b0 2024-07-04T08:23:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.839+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e2d83c94a0 con 0x55e2d809e390 2024-07-04T08:23:39.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.839+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e2d83cd4e0 con 0x55e2d80bf070 2024-07-04T08:23:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.839+0000 7fb847400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8464250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.839+0000 7fb84fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d8463d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.839+0000 7fb847400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8464250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:55322/0 (socket says 172.21.15.39:55322) 2024-07-04T08:23:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.839+0000 7fb847400640 1 -- 172.21.15.39:0/2447404484 learned_addr learned my addr 172.21.15.39:0/2447404484 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.839+0000 7fb847400640 1 --2- 172.21.15.39:0/2447404484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80bf070 0x55e2d84637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb847400640 1 -- 172.21.15.39:0/2447404484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80bf070 msgr2=0x55e2d84637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb847400640 1 --2- 172.21.15.39:0/2447404484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80bf070 0x55e2d84637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb847400640 1 -- 172.21.15.39:0/2447404484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 msgr2=0x55e2d8463d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:39.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb847400640 1 --2- 172.21.15.39:0/2447404484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d8463d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb847400640 1 -- 172.21.15.39:0/2447404484 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2d8361670 con 0x55e2d809e390 2024-07-04T08:23:39.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb84fe00640 1 --2- 172.21.15.39:0/2447404484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d8463d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:39.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb847400640 1 --2- 172.21.15.39:0/2447404484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8464250 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb838000f80 tx=0x7fb838005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74cc3e35c6e15da9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:39.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb84d600640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb8340b2960 0x7fb8340b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb845600640 1 -- 172.21.15.39:0/2447404484 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838047070 con 0x55e2d809e390 2024-07-04T08:23:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb84d600640 1 -- 172.21.15.39:0/3500315042 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7fb8480e2bd0 con 0x7fb850109340 2024-07-04T08:23:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb845600640 1 -- 172.21.15.39:0/2447404484 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb838003db0 con 0x55e2d809e390 2024-07-04T08:23:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.840+0000 7fb845600640 1 -- 172.21.15.39:0/2447404484 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838036040 con 0x55e2d809e390 2024-07-04T08:23:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.841+0000 7fb84fe00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb8340b2960 0x7fb8340b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2447404484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 msgr2=0x55e2d8464250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:39.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2447404484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8464250 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb838000f80 tx=0x7fb838005fd0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2447404484 shutdown_connections 2024-07-04T08:23:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb84fe00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb8340b2960 0x7fb8340b4e70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55e2d8462450 tx=0x7fb840005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2447404484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d8463d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2447404484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8464250 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2447404484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80bf070 0x55e2d84637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2447404484 >> 172.21.15.39:0/2447404484 conn(0x55e2d80b3460 msgr2=0x55e2d84616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2447404484 shutdown_connections 2024-07-04T08:23:39.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.842+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2447404484 wait complete. 2024-07-04T08:23:39.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.843+0000 7fb85d2d9740 1 -- start start 2024-07-04T08:23:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.843+0000 7fb85d2d9740 1 Processor -- start 2024-07-04T08:23:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.843+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d846f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.843+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d809e390 0x55e2d846f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.843+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d847b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.843+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e2d822d700 con 0x55e2d809e390 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.843+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e2d8370bd0 con 0x55e2d80bf070 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.843+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e2d8471f50 con 0x55e2d80961b0 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb85d2d9740 10 client.0 _ll_register_callbacks cb 0x55e2d83c2590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb854c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d847b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb84fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d846f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb847400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d809e390 0x55e2d846f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb854c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d847b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:55326/0 (socket says 172.21.15.39:55326) 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb854c00640 1 -- 172.21.15.39:0/2878292113 learned_addr learned my addr 172.21.15.39:0/2878292113 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb84fe00640 1 -- 172.21.15.39:0/2878292113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 msgr2=0x55e2d847b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb84fe00640 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d847b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb84fe00640 1 -- 172.21.15.39:0/2878292113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d809e390 msgr2=0x55e2d846f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb84fe00640 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d809e390 0x55e2d846f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb84fe00640 1 -- 172.21.15.39:0/2878292113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2d812a650 con 0x55e2d80961b0 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb854c00640 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d847b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.844+0000 7fb847400640 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d809e390 0x55e2d846f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.845+0000 7fb84fe00640 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d846f400 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb840012cb0 tx=0x7fb840004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166c77ed7cb98bce server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.845+0000 7fb83e000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.845+0000 7fb83e000640 1 -- 172.21.15.39:0/2878292113 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84001f640 con 0x55e2d80961b0 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.845+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2878292113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb838003db0 con 0x55e2d80961b0 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.845+0000 7fb83e000640 1 -- 172.21.15.39:0/2878292113 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7fb84001f7e0 con 0x55e2d80961b0 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.845+0000 7fb83e000640 1 -- 172.21.15.39:0/2878292113 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb84001f9e0 con 0x55e2d80961b0 2024-07-04T08:23:39.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.845+0000 7fb85d2d9740 10 client.27406 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.845+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2878292113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e2d847e050 con 0x55e2d80961b0 2024-07-04T08:23:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.846+0000 7fb83e000640 1 -- 172.21.15.39:0/2878292113 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb838003db0 con 0x55e2d80961b0 2024-07-04T08:23:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.846+0000 7fb83e000640 1 -- 172.21.15.39:0/2878292113 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb840027070 con 0x55e2d80961b0 2024-07-04T08:23:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.846+0000 7fb85d2d9740 10 client.27406 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.846+0000 7fb85d2d9740 10 client.27406 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:23:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.846+0000 7fb85d2d9740 20 client.27406 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.846+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e2d807d9f0 0x55e2d8481550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.846+0000 7fb85d2d9740 4 client.27406 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:23:39.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.846+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2878292113 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2d83dbe50 con 0x55e2d807d9f0 2024-07-04T08:23:39.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.847+0000 7fb847400640 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e2d807d9f0 0x55e2d8481550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.847+0000 7fb847400640 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e2d807d9f0 0x55e2d8481550 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3aa70dcc708ce6e server_cookie=97b4da1d65e68c2f in_seq=0 out_seq=0 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.847+0000 7fb83e000640 10 client.27406 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.848+0000 7fb83e000640 1 -- 172.21.15.39:0/2878292113 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e2d83dbe50 con 0x55e2d807d9f0 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.848+0000 7fb83e000640 10 client.27406 handle_command_reply: tid=0 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.848+0000 7fb85d2d9740 1 client.27406 shutdown 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.848+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2878292113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 msgr2=0x55e2d846f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.848+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d846f400 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb840012cb0 tx=0x7fb840004730 comp rx=0 tx=0).stop 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2878292113 shutdown_connections 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e2d807d9f0 0x55e2d8481550 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d809e390 0x55e2d846f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d847b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/2878292113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d846f400 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb83e000640 0 client.27406 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:23:39.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2878292113 >> 172.21.15.39:0/2878292113 conn(0x55e2d80b3460 msgr2=0x55e2d8466960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:39.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2878292113 shutdown_connections 2024-07-04T08:23:39.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb85d2d9740 1 -- 172.21.15.39:0/2878292113 wait complete. 2024-07-04T08:23:39.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.849+0000 7fb85d2d9740 20 client.27406 trim_cache size 0 max 16384 2024-07-04T08:23:39.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.865+0000 7fb85d2d9740 1 Processor -- start 2024-07-04T08:23:39.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.865+0000 7fb85d2d9740 1 -- start start 2024-07-04T08:23:39.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.865+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d8466fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.865+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8460960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.865+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80bf070 0x55e2d8461020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.865+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb8501b26b0 con 0x55e2d80bf070 2024-07-04T08:23:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.865+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb8501b41f0 con 0x55e2d809e390 2024-07-04T08:23:39.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.865+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb8501b2d80 con 0x55e2d80961b0 2024-07-04T08:23:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb854c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80bf070 0x55e2d8461020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb847400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8460960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb84fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d8466fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb854c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80bf070 0x55e2d8461020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35586/0 (socket says 172.21.15.39:35586) 2024-07-04T08:23:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb854c00640 1 -- 172.21.15.39:0/317767497 learned_addr learned my addr 172.21.15.39:0/317767497 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb847400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8460960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:55340/0 (socket says 172.21.15.39:55340) 2024-07-04T08:23:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb84fe00640 1 --2- 172.21.15.39:0/317767497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d8466fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:45128/0 (socket says 172.21.15.39:45128) 2024-07-04T08:23:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb854c00640 1 -- 172.21.15.39:0/317767497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 msgr2=0x55e2d8466fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:39.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb854c00640 1 --2- 172.21.15.39:0/317767497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d8466fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb854c00640 1 -- 172.21.15.39:0/317767497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 msgr2=0x55e2d8460960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb854c00640 1 --2- 172.21.15.39:0/317767497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8460960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb854c00640 1 -- 172.21.15.39:0/317767497 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2d8465ac0 con 0x55e2d80bf070 2024-07-04T08:23:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.866+0000 7fb84fe00640 1 --2- 172.21.15.39:0/317767497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d8466fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.867+0000 7fb854c00640 1 --2- 172.21.15.39:0/317767497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80bf070 0x55e2d8461020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c6a0 tx=0x7fb8480a9390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f613e3dd53363c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.867+0000 7fb844c00640 1 -- 172.21.15.39:0/317767497 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8480c3070 con 0x55e2d80bf070 2024-07-04T08:23:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.867+0000 7fb844c00640 1 -- 172.21.15.39:0/317767497 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8480b22a0 con 0x55e2d80bf070 2024-07-04T08:23:39.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.867+0000 7fb844c00640 1 -- 172.21.15.39:0/317767497 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8480b24c0 con 0x55e2d80bf070 2024-07-04T08:23:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.868+0000 7fb85d2d9740 1 -- 172.21.15.39:0/317767497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80bf070 msgr2=0x55e2d8461020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.868+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/317767497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80bf070 0x55e2d8461020 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c6a0 tx=0x7fb8480a9390 comp rx=0 tx=0).stop 2024-07-04T08:23:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.868+0000 7fb85d2d9740 1 -- 172.21.15.39:0/317767497 shutdown_connections 2024-07-04T08:23:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.868+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/317767497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80bf070 0x55e2d8461020 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.868+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/317767497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d809e390 0x55e2d8460960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.868+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/317767497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d80961b0 0x55e2d8466fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.868+0000 7fb85d2d9740 1 -- 172.21.15.39:0/317767497 >> 172.21.15.39:0/317767497 conn(0x55e2d80b3460 msgr2=0x55e2d84668d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.868+0000 7fb85d2d9740 1 -- 172.21.15.39:0/317767497 shutdown_connections 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.868+0000 7fb85d2d9740 1 -- 172.21.15.39:0/317767497 wait complete. 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.869+0000 7fb85d2d9740 1 -- start start 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.869+0000 7fb85d2d9740 1 Processor -- start 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d847a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d809e390 0x55e2d847ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb85d2d9740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d84761d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb8501b3b20 con 0x55e2d80961b0 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb8501b3450 con 0x55e2d80bf070 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb85d2d9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb8501b0690 con 0x55e2d809e390 2024-07-04T08:23:39.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb85d2d9740 10 client.0 _ll_register_callbacks cb 0x7fb840009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb84fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d847a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb84fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d847a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:35600/0 (socket says 172.21.15.39:35600) 2024-07-04T08:23:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb84fe00640 1 -- 172.21.15.39:0/902291717 learned_addr learned my addr 172.21.15.39:0/902291717 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb854c00640 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d84761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.870+0000 7fb847400640 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d809e390 0x55e2d847ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb84fe00640 1 -- 172.21.15.39:0/902291717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d809e390 msgr2=0x55e2d847ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb84fe00640 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d809e390 0x55e2d847ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb84fe00640 1 -- 172.21.15.39:0/902291717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 msgr2=0x55e2d84761d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb84fe00640 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d84761d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb84fe00640 1 -- 172.21.15.39:0/902291717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2d8476710 con 0x55e2d80961b0 2024-07-04T08:23:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb854c00640 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d84761d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb847400640 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d809e390 0x55e2d847ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:39.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb84fe00640 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d847a840 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb840012cb0 tx=0x7fb84001d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28d900c6da47b65 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb83e000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:23:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.871+0000 7fb83e000640 1 -- 172.21.15.39:0/902291717 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84003b020 con 0x55e2d80961b0 2024-07-04T08:23:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.872+0000 7fb85d2d9740 1 -- 172.21.15.39:0/902291717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8480b22a0 con 0x55e2d80961b0 2024-07-04T08:23:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.872+0000 7fb83e000640 1 -- 172.21.15.39:0/902291717 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7fb840029440 con 0x55e2d80961b0 2024-07-04T08:23:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.872+0000 7fb83e000640 1 -- 172.21.15.39:0/902291717 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb840029660 con 0x55e2d80961b0 2024-07-04T08:23:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.872+0000 7fb85d2d9740 10 client.17352 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.872+0000 7fb85d2d9740 1 -- 172.21.15.39:0/902291717 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e2d8477990 con 0x55e2d80961b0 2024-07-04T08:23:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.873+0000 7fb83e000640 1 -- 172.21.15.39:0/902291717 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8480b22a0 con 0x55e2d80961b0 2024-07-04T08:23:39.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.873+0000 7fb83e000640 1 -- 172.21.15.39:0/902291717 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb840027040 con 0x55e2d80961b0 2024-07-04T08:23:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.873+0000 7fb85d2d9740 10 client.17352 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.873+0000 7fb85d2d9740 10 client.17352 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:23:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.873+0000 7fb85d2d9740 20 client.17352 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.873+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e2d8052da0 0x55e2d84ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.873+0000 7fb85d2d9740 4 client.17352 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:23:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.873+0000 7fb85d2d9740 1 -- 172.21.15.39:0/902291717 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2d83d78f0 con 0x55e2d8052da0 2024-07-04T08:23:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.874+0000 7fb847400640 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e2d8052da0 0x55e2d84ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.874+0000 7fb847400640 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e2d8052da0 0x55e2d84ebe00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac4bb3290a9282c6 server_cookie=1b1366fdc1501cd9 in_seq=0 out_seq=0 2024-07-04T08:23:39.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.874+0000 7fb83e000640 10 client.17352 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:23:39.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.874+0000 7fb83e000640 1 -- 172.21.15.39:0/902291717 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2d83d78f0 con 0x55e2d8052da0 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.874+0000 7fb83e000640 10 client.17352 handle_command_reply: tid=0 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.874+0000 7fb85d2d9740 1 client.17352 shutdown 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 -- 172.21.15.39:0/902291717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 msgr2=0x55e2d847a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d847a840 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb840012cb0 tx=0x7fb84001d630 comp rx=0 tx=0).stop 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 -- 172.21.15.39:0/902291717 shutdown_connections 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e2d8052da0 0x55e2d84ebe00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e2d80bf070 0x55e2d84761d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e2d809e390 0x55e2d847ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 --2- 172.21.15.39:0/902291717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e2d80961b0 0x55e2d847a840 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 -- 172.21.15.39:0/902291717 >> 172.21.15.39:0/902291717 conn(0x55e2d80b3460 msgr2=0x55e2d8460fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb83e000640 0 client.17352 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:23:39.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 -- 172.21.15.39:0/902291717 shutdown_connections 2024-07-04T08:23:39.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 1 -- 172.21.15.39:0/902291717 wait complete. 2024-07-04T08:23:39.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.875+0000 7fb85d2d9740 20 client.17352 trim_cache size 0 max 16384 2024-07-04T08:23:39.872 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 -- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb8340b2960 msgr2=0x7fb8340b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb8340b2960 0x7fb8340b4e70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55e2d8462450 tx=0x7fb840005f90 comp rx=0 tx=0).stop 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 -- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb850109340 msgr2=0x7fb8501a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb850109340 0x7fb8501a1540 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb84800e940 tx=0x7fb84800ee00 comp rx=0 tx=0).stop 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 -- 172.21.15.39:0/3500315042 shutdown_connections 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb8340b2960 0x7fb8340b4e70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb850109340 0x7fb8501a1540 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8501089f0 0x7fb85019c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 --2- 172.21.15.39:0/3500315042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb8501030b0 0x7fb85019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:39.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 -- 172.21.15.39:0/3500315042 >> 172.21.15.39:0/3500315042 conn(0x7fb85007ada0 msgr2=0x7fb850106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:39.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 -- 172.21.15.39:0/3500315042 shutdown_connections 2024-07-04T08:23:39.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:39.876+0000 7fb856c00640 1 -- 172.21.15.39:0/3500315042 wait complete. 2024-07-04T08:23:40.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:23:41.298 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:41.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:41 smithi039 ceph-mon[32062]: pgmap v738: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-07-04T08:23:41.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:41 smithi050 ceph-mon[36577]: pgmap v738: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-07-04T08:23:41.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:41 smithi133 ceph-mon[37050]: pgmap v738: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-07-04T08:23:42.074 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:23:43.001 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:43.323 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:43 smithi039 ceph-mon[32062]: pgmap v739: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-07-04T08:23:43.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:43 smithi050 ceph-mon[36577]: pgmap v739: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-07-04T08:23:43.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:43 smithi133 ceph-mon[37050]: pgmap v739: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-07-04T08:23:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.709+0000 7f4096a00640 1 -- 172.21.15.39:0/135960215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4090075320 msgr2=0x7f4090075730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.709+0000 7f4096a00640 1 --2- 172.21.15.39:0/135960215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4090075320 0x7f4090075730 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4078009be0 tx=0x7f407802a6c0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.709+0000 7f4096a00640 1 -- 172.21.15.39:0/135960215 shutdown_connections 2024-07-04T08:23:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.709+0000 7f4096a00640 1 --2- 172.21.15.39:0/135960215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4090113990 0x7f4090115e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.709+0000 7f4096a00640 1 --2- 172.21.15.39:0/135960215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4090075320 0x7f4090075730 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.709+0000 7f4096a00640 1 --2- 172.21.15.39:0/135960215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40900768d0 0x7f4090074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.709+0000 7f4096a00640 1 -- 172.21.15.39:0/135960215 >> 172.21.15.39:0/135960215 conn(0x7f40900feb30 msgr2=0x7f4090100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.709+0000 7f4096a00640 1 -- 172.21.15.39:0/135960215 shutdown_connections 2024-07-04T08:23:43.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.710+0000 7f4096a00640 1 -- 172.21.15.39:0/135960215 wait complete. 2024-07-04T08:23:43.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.711+0000 7f4096a00640 1 Processor -- start 2024-07-04T08:23:43.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.711+0000 7f4096a00640 1 -- start start 2024-07-04T08:23:43.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.711+0000 7f4096a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4090075320 0x7f409019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.711+0000 7f4096a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40900768d0 0x7f40901a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.711+0000 7f4096a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4090113990 0x7f40901a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.711+0000 7f4096a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40901195c0 con 0x7f4090113990 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.711+0000 7f4096a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4090119440 con 0x7f4090075320 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.711+0000 7f4096a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4090119740 con 0x7f40900768d0 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f408f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40900768d0 0x7f40901a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f4094a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4090113990 0x7f40901a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f408fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4090075320 0x7f409019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f408f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40900768d0 0x7f40901a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39198/0 (socket says 172.21.15.39:39198) 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f408f400640 1 -- 172.21.15.39:0/655359765 learned_addr learned my addr 172.21.15.39:0/655359765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f408fe00640 1 -- 172.21.15.39:0/655359765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40900768d0 msgr2=0x7f40901a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f408fe00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40900768d0 0x7f40901a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f408fe00640 1 -- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4090113990 msgr2=0x7f40901a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:43.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f408fe00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4090113990 0x7f40901a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f408fe00640 1 -- 172.21.15.39:0/655359765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40901a5f60 con 0x7f4090075320 2024-07-04T08:23:43.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.712+0000 7f4094a00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4090113990 0x7f40901a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:43.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.713+0000 7f408fe00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4090075320 0x7f409019fff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4080005fd0 tx=0x7f408000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:43.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.713+0000 7f408cc00640 1 -- 172.21.15.39:0/655359765 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408000cd00 con 0x7f4090075320 2024-07-04T08:23:43.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.713+0000 7f408cc00640 1 -- 172.21.15.39:0/655359765 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40800035f0 con 0x7f4090075320 2024-07-04T08:23:43.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.713+0000 7f408cc00640 1 -- 172.21.15.39:0/655359765 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080010670 con 0x7f4090075320 2024-07-04T08:23:43.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.713+0000 7f4096a00640 1 -- 172.21.15.39:0/655359765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40901a6250 con 0x7f4090075320 2024-07-04T08:23:43.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.713+0000 7f4096a00640 1 -- 172.21.15.39:0/655359765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40901ac9b0 con 0x7f4090075320 2024-07-04T08:23:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.717+0000 7f409cf33740 1 Processor -- start 2024-07-04T08:23:43.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.717+0000 7f409cf33740 1 -- start start 2024-07-04T08:23:43.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.717+0000 7f409cf33740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.717+0000 7f409cf33740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa48cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.717+0000 7f409cf33740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa49230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.717+0000 7f409cf33740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff9beee0 con 0x55b3ff683390 2024-07-04T08:23:43.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.717+0000 7f409cf33740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff9ae480 con 0x55b3ff6a4070 2024-07-04T08:23:43.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.717+0000 7f409cf33740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff9b24c0 con 0x55b3ff67b1b0 2024-07-04T08:23:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f4094a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f4094a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:36128/0 (socket says 172.21.15.39:36128) 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f4094a00640 1 -- 172.21.15.39:0/3968433528 learned_addr learned my addr 172.21.15.39:0/3968433528 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f408f400640 1 --2- 172.21.15.39:0/3968433528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa49230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f408cc00640 1 -- 172.21.15.39:0/655359765 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f40800037a0 con 0x7f4090075320 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f4094a00640 1 -- 172.21.15.39:0/3968433528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff67b1b0 msgr2=0x55b3ffa48cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f4094a00640 1 --2- 172.21.15.39:0/3968433528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa48cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f4094a00640 1 -- 172.21.15.39:0/3968433528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 msgr2=0x55b3ffa49230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f4094a00640 1 --2- 172.21.15.39:0/3968433528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa49230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f4094a00640 1 -- 172.21.15.39:0/3968433528 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ff946650 con 0x55b3ff6a4070 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.718+0000 7f408f400640 1 --2- 172.21.15.39:0/3968433528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa49230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.719+0000 7f4094a00640 1 --2- 172.21.15.39:0/3968433528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa487b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4088011540 tx=0x7f4088011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49ee3eea494489d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.719+0000 7f4085600640 1 -- 172.21.15.39:0/3968433528 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408800f5c0 con 0x55b3ff6a4070 2024-07-04T08:23:43.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.719+0000 7f4085600640 1 -- 172.21.15.39:0/3968433528 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408800fc10 con 0x55b3ff6a4070 2024-07-04T08:23:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.719+0000 7f4085600640 1 -- 172.21.15.39:0/3968433528 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088019c50 con 0x55b3ff6a4070 2024-07-04T08:23:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f409cf33740 1 -- 172.21.15.39:0/3968433528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 msgr2=0x55b3ffa487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:43.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f409cf33740 1 --2- 172.21.15.39:0/3968433528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa487b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4088011540 tx=0x7f4088011a00 comp rx=0 tx=0).stop 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f409cf33740 1 -- 172.21.15.39:0/3968433528 shutdown_connections 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f409cf33740 1 --2- 172.21.15.39:0/3968433528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa49230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f408cc00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40700b29e0 0x7f40700b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f409cf33740 1 --2- 172.21.15.39:0/3968433528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa48cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f409cf33740 1 --2- 172.21.15.39:0/3968433528 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa487b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f409cf33740 1 -- 172.21.15.39:0/3968433528 >> 172.21.15.39:0/3968433528 conn(0x55b3ff698460 msgr2=0x55b3ffa466a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f408cc00640 1 -- 172.21.15.39:0/655359765 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f40800e2ba0 con 0x7f4090075320 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f409cf33740 1 -- 172.21.15.39:0/3968433528 shutdown_connections 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f409cf33740 1 -- 172.21.15.39:0/3968433528 wait complete. 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.720+0000 7f4094a00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40700b29e0 0x7f40700b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.721+0000 7f409cf33740 1 -- start start 2024-07-04T08:23:43.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.721+0000 7f4094a00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40700b29e0 0x7f40700b4ef0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f40880198b0 tx=0x7f4088005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.721+0000 7f409cf33740 1 Processor -- start 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f409cf33740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f409cf33740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff683390 0x55b3ffa54920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f409cf33740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 0x55b3ffa602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f409cf33740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff812700 con 0x55b3ff67b1b0 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f409cf33740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff955bb0 con 0x55b3ff683390 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f409cf33740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b3ffa56f90 con 0x55b3ff6a4070 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f409cf33740 10 client.0 _ll_register_callbacks cb 0x55b3ff9a7570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f408fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f408fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39644/0 (socket says 172.21.15.39:39644) 2024-07-04T08:23:43.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.722+0000 7f408fe00640 1 -- 172.21.15.39:0/4226433754 learned_addr learned my addr 172.21.15.39:0/4226433754 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.723+0000 7f408f400640 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff683390 0x55b3ffa54920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.723+0000 7f4094a00640 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 0x55b3ffa602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.723+0000 7f408fe00640 1 -- 172.21.15.39:0/4226433754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 msgr2=0x55b3ffa602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.723+0000 7f408fe00640 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 0x55b3ffa602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.723+0000 7f408fe00640 1 -- 172.21.15.39:0/4226433754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff683390 msgr2=0x55b3ffa54920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.723+0000 7f408fe00640 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff683390 0x55b3ffa54920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.723+0000 7f408fe00640 1 -- 172.21.15.39:0/4226433754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ff70f650 con 0x55b3ff67b1b0 2024-07-04T08:23:43.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.723+0000 7f408fe00640 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa543a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f40800a9a50 tx=0x7f40800a9690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a7bb05453149d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.723+0000 7f407e000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.724+0000 7f407e000640 1 -- 172.21.15.39:0/4226433754 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800a9ac0 con 0x55b3ff67b1b0 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.724+0000 7f407e000640 1 -- 172.21.15.39:0/4226433754 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f40800a5d50 con 0x55b3ff67b1b0 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.724+0000 7f409cf33740 1 -- 172.21.15.39:0/4226433754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408800fc10 con 0x55b3ff67b1b0 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.724+0000 7f407e000640 1 -- 172.21.15.39:0/4226433754 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40800ade70 con 0x55b3ff67b1b0 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.724+0000 7f409cf33740 10 client.17364 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.724+0000 7f409cf33740 1 -- 172.21.15.39:0/4226433754 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3ffa63020 con 0x55b3ff67b1b0 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.724+0000 7f407e000640 1 -- 172.21.15.39:0/4226433754 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408800fc10 con 0x55b3ff67b1b0 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.725+0000 7f407e000640 1 -- 172.21.15.39:0/4226433754 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f40800a5430 con 0x55b3ff67b1b0 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.725+0000 7f409cf33740 10 client.17364 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.725+0000 7f409cf33740 10 client.17364 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:23:43.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.725+0000 7f409cf33740 20 client.17364 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.725+0000 7f409cf33740 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b3ff6629f0 0x55b3ffa66540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.725+0000 7f409cf33740 4 client.17364 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:23:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.725+0000 7f409cf33740 1 -- 172.21.15.39:0/4226433754 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3ff9c0e30 con 0x55b3ff6629f0 2024-07-04T08:23:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.725+0000 7f408f400640 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b3ff6629f0 0x55b3ffa66540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.726+0000 7f408f400640 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b3ff6629f0 0x55b3ffa66540 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=591c1ab8a3cdc67b server_cookie=713b45378f8f24ef in_seq=0 out_seq=0 2024-07-04T08:23:43.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.726+0000 7f407e000640 10 client.17364 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:23:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f407e000640 1 -- 172.21.15.39:0/4226433754 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b3ff9c0e30 con 0x55b3ff6629f0 2024-07-04T08:23:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f407e000640 10 client.17364 handle_command_reply: tid=0 2024-07-04T08:23:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 client.17364 shutdown 2024-07-04T08:23:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 -- 172.21.15.39:0/4226433754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 msgr2=0x55b3ffa543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa543a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f40800a9a50 tx=0x7f40800a9690 comp rx=0 tx=0).stop 2024-07-04T08:23:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 -- 172.21.15.39:0/4226433754 shutdown_connections 2024-07-04T08:23:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b3ff6629f0 0x55b3ffa66540 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 0x55b3ffa602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff683390 0x55b3ffa54920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 --2- 172.21.15.39:0/4226433754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa543a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f407e000640 0 client.17364 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:23:43.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 -- 172.21.15.39:0/4226433754 >> 172.21.15.39:0/4226433754 conn(0x55b3ff698460 msgr2=0x55b3ffa4b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:43.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 -- 172.21.15.39:0/4226433754 shutdown_connections 2024-07-04T08:23:43.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.727+0000 7f409cf33740 1 -- 172.21.15.39:0/4226433754 wait complete. 2024-07-04T08:23:43.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.728+0000 7f409cf33740 20 client.17364 trim_cache size 0 max 16384 2024-07-04T08:23:43.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.743+0000 7f409cf33740 1 Processor -- start 2024-07-04T08:23:43.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.743+0000 7f409cf33740 1 -- start start 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f409cf33740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa56080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f409cf33740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f409cf33740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 0x55b3ffa49ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f409cf33740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b69e0 con 0x55b3ff683390 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f409cf33740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b8520 con 0x55b3ff67b1b0 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f409cf33740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b70b0 con 0x55b3ff6a4070 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f408f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f408f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39652/0 (socket says 172.21.15.39:39652) 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f408f400640 1 -- 172.21.15.39:0/2920298452 learned_addr learned my addr 172.21.15.39:0/2920298452 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f408fe00640 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa56080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f4094a00640 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 0x55b3ffa49ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f408f400640 1 -- 172.21.15.39:0/2920298452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 msgr2=0x55b3ffa49ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:43.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f408f400640 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 0x55b3ffa49ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f408f400640 1 -- 172.21.15.39:0/2920298452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff67b1b0 msgr2=0x55b3ffa56080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:43.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.744+0000 7f408f400640 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa56080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.745+0000 7f408f400640 1 -- 172.21.15.39:0/2920298452 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ffa47380 con 0x55b3ff683390 2024-07-04T08:23:43.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.745+0000 7f4094a00640 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 0x55b3ffa49ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:23:43.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.745+0000 7f408fe00640 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa56080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:43.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.745+0000 7f408f400640 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa600a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4078009540 tx=0x7f4078009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749d0b8e1250c0e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:43.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.745+0000 7f4084c00640 1 -- 172.21.15.39:0/2920298452 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078036040 con 0x55b3ff683390 2024-07-04T08:23:43.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.745+0000 7f4084c00640 1 -- 172.21.15.39:0/2920298452 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4078030970 con 0x55b3ff683390 2024-07-04T08:23:43.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.745+0000 7f4084c00640 1 -- 172.21.15.39:0/2920298452 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40780043b0 con 0x55b3ff683390 2024-07-04T08:23:43.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.746+0000 7f409cf33740 1 -- 172.21.15.39:0/2920298452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 msgr2=0x55b3ffa600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:43.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.746+0000 7f409cf33740 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa600a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4078009540 tx=0x7f4078009570 comp rx=0 tx=0).stop 2024-07-04T08:23:43.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.746+0000 7f409cf33740 1 -- 172.21.15.39:0/2920298452 shutdown_connections 2024-07-04T08:23:43.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.746+0000 7f409cf33740 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff683390 0x55b3ffa600a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.746+0000 7f409cf33740 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff6a4070 0x55b3ffa49ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.746+0000 7f409cf33740 1 --2- 172.21.15.39:0/2920298452 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa56080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.746+0000 7f409cf33740 1 -- 172.21.15.39:0/2920298452 >> 172.21.15.39:0/2920298452 conn(0x55b3ff698460 msgr2=0x55b3ffa497e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:43.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.746+0000 7f409cf33740 1 -- 172.21.15.39:0/2920298452 shutdown_connections 2024-07-04T08:23:43.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.747+0000 7f409cf33740 1 -- 172.21.15.39:0/2920298452 wait complete. 2024-07-04T08:23:43.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.747+0000 7f409cf33740 1 -- start start 2024-07-04T08:23:43.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.747+0000 7f409cf33740 1 Processor -- start 2024-07-04T08:23:43.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f409cf33740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa5c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f409cf33740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff683390 0x55b3ffa5c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f409cf33740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f409cf33740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b7e50 con 0x55b3ff67b1b0 2024-07-04T08:23:43.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f409cf33740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b7780 con 0x55b3ff6a4070 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f409cf33740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40901b49c0 con 0x55b3ff683390 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f409cf33740 10 client.0 _ll_register_callbacks cb 0x7f40800046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f408fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa5c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f408fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa5c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39658/0 (socket says 172.21.15.39:39658) 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f408fe00640 1 -- 172.21.15.39:0/3490820124 learned_addr learned my addr 172.21.15.39:0/3490820124 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f408f400640 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff683390 0x55b3ffa5c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.748+0000 7f4094a00640 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.749+0000 7f408fe00640 1 -- 172.21.15.39:0/3490820124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff683390 msgr2=0x55b3ffa5c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.749+0000 7f408fe00640 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff683390 0x55b3ffa5c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.749+0000 7f408fe00640 1 -- 172.21.15.39:0/3490820124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 msgr2=0x55b3ffa571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.749+0000 7f408fe00640 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.749+0000 7f408fe00640 1 -- 172.21.15.39:0/3490820124 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ffa57710 con 0x55b3ff67b1b0 2024-07-04T08:23:43.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.749+0000 7f408f400640 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff683390 0x55b3ffa5c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:43.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.749+0000 7f408fe00640 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa5c220 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f40800a97c0 tx=0x7f4080003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d9c6183953378d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:43.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.749+0000 7f407e000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:23:43.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.750+0000 7f407e000640 1 -- 172.21.15.39:0/3490820124 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40800a9ac0 con 0x55b3ff67b1b0 2024-07-04T08:23:43.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.750+0000 7f407e000640 1 -- 172.21.15.39:0/3490820124 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f40800a5430 con 0x55b3ff67b1b0 2024-07-04T08:23:43.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.750+0000 7f409cf33740 1 -- 172.21.15.39:0/3490820124 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4078030970 con 0x55b3ff67b1b0 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.750+0000 7f407e000640 1 -- 172.21.15.39:0/3490820124 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f408008e050 con 0x55b3ff67b1b0 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.750+0000 7f409cf33740 10 client.17370 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.750+0000 7f409cf33740 1 -- 172.21.15.39:0/3490820124 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3ffacf2f0 con 0x55b3ff67b1b0 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.750+0000 7f407e000640 1 -- 172.21.15.39:0/3490820124 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4078030970 con 0x55b3ff67b1b0 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.750+0000 7f407e000640 1 -- 172.21.15.39:0/3490820124 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f40800b2c00 con 0x55b3ff67b1b0 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.751+0000 7f409cf33740 10 client.17370 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.751+0000 7f409cf33740 10 client.17370 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.751+0000 7f409cf33740 20 client.17370 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.751+0000 7f409cf33740 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b3ff637da0 0x55b3ffad1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.751+0000 7f409cf33740 4 client.17370 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:23:43.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.751+0000 7f409cf33740 1 -- 172.21.15.39:0/3490820124 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3ff9bc8d0 con 0x55b3ff637da0 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.751+0000 7f408f400640 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b3ff637da0 0x55b3ffad1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f408f400640 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b3ff637da0 0x55b3ffad1a50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4375af17b89eef16 server_cookie=5fc595b7cabbd806 in_seq=0 out_seq=0 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f407e000640 10 client.17370 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f407e000640 1 -- 172.21.15.39:0/3490820124 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3ff9bc8d0 con 0x55b3ff637da0 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f407e000640 10 client.17370 handle_command_reply: tid=0 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f409cf33740 1 client.17370 shutdown 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f409cf33740 1 -- 172.21.15.39:0/3490820124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 msgr2=0x55b3ffa5c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f409cf33740 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa5c220 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f40800a97c0 tx=0x7f4080003de0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f409cf33740 1 -- 172.21.15.39:0/3490820124 shutdown_connections 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f409cf33740 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b3ff637da0 0x55b3ffad1a50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f409cf33740 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b3ff6a4070 0x55b3ffa571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f409cf33740 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b3ff683390 0x55b3ffa5c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f409cf33740 1 --2- 172.21.15.39:0/3490820124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b3ff67b1b0 0x55b3ffa5c220 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f409cf33740 1 -- 172.21.15.39:0/3490820124 >> 172.21.15.39:0/3490820124 conn(0x55b3ff698460 msgr2=0x55b3ffa56010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:43.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.752+0000 7f407e000640 0 client.17370 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:23:43.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.753+0000 7f409cf33740 1 -- 172.21.15.39:0/3490820124 shutdown_connections 2024-07-04T08:23:43.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.753+0000 7f409cf33740 1 -- 172.21.15.39:0/3490820124 wait complete. 2024-07-04T08:23:43.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.753+0000 7f409cf33740 20 client.17370 trim_cache size 0 max 16384 2024-07-04T08:23:43.749 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:23:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.753+0000 7f4096a00640 1 -- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40700b29e0 msgr2=0x7f40700b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.753+0000 7f4096a00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40700b29e0 0x7f40700b4ef0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f40880198b0 tx=0x7f4088005850 comp rx=0 tx=0).stop 2024-07-04T08:23:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.753+0000 7f4096a00640 1 -- 172.21.15.39:0/655359765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4090075320 msgr2=0x7f409019fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.753+0000 7f4096a00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4090075320 0x7f409019fff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4080005fd0 tx=0x7f408000ee50 comp rx=0 tx=0).stop 2024-07-04T08:23:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.754+0000 7f4096a00640 1 -- 172.21.15.39:0/655359765 shutdown_connections 2024-07-04T08:23:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.754+0000 7f4096a00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40700b29e0 0x7f40700b4ef0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.754+0000 7f4096a00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4090113990 0x7f40901a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.754+0000 7f4096a00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40900768d0 0x7f40901a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.754+0000 7f4096a00640 1 --2- 172.21.15.39:0/655359765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4090075320 0x7f409019fff0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.754+0000 7f4096a00640 1 -- 172.21.15.39:0/655359765 >> 172.21.15.39:0/655359765 conn(0x7f40900feb30 msgr2=0x7f409010f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.754+0000 7f4096a00640 1 -- 172.21.15.39:0/655359765 shutdown_connections 2024-07-04T08:23:43.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:43.754+0000 7f4096a00640 1 -- 172.21.15.39:0/655359765 wait complete. 2024-07-04T08:23:45.097 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:23:45.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:45 smithi039 ceph-mon[32062]: pgmap v740: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-07-04T08:23:45.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:45 smithi050 ceph-mon[36577]: pgmap v740: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-07-04T08:23:45.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:45 smithi133 ceph-mon[37050]: pgmap v740: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-07-04T08:23:45.772 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:46.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.671+0000 7f620e200640 1 -- 172.21.15.39:0/2511463977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6208103f20 msgr2=0x7f6208104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:46.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.671+0000 7f620e200640 1 --2- 172.21.15.39:0/2511463977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6208103f20 0x7f6208104330 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009900 tx=0x7f61f802a410 comp rx=0 tx=0).stop 2024-07-04T08:23:46.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.671+0000 7f620e200640 1 -- 172.21.15.39:0/2511463977 shutdown_connections 2024-07-04T08:23:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.671+0000 7f620e200640 1 --2- 172.21.15.39:0/2511463977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f620810d380 0x7f620810f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.671+0000 7f620e200640 1 --2- 172.21.15.39:0/2511463977 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6208104870 0x7f620810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.671+0000 7f620e200640 1 --2- 172.21.15.39:0/2511463977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6208103f20 0x7f6208104330 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.671+0000 7f620e200640 1 -- 172.21.15.39:0/2511463977 >> 172.21.15.39:0/2511463977 conn(0x7f6208078c90 msgr2=0x7f62080790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.671+0000 7f620e200640 1 -- 172.21.15.39:0/2511463977 shutdown_connections 2024-07-04T08:23:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.671+0000 7f620e200640 1 -- 172.21.15.39:0/2511463977 wait complete. 2024-07-04T08:23:46.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.673+0000 7f620e200640 1 Processor -- start 2024-07-04T08:23:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.673+0000 7f620e200640 1 -- start start 2024-07-04T08:23:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.673+0000 7f620e200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6208103f20 0x7f6208199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.673+0000 7f620e200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6208104870 0x7f620819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.673+0000 7f620e200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f620810d380 0x7f620819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.673+0000 7f620e200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6208113010 con 0x7f6208104870 2024-07-04T08:23:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f620e200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6208112e90 con 0x7f6208103f20 2024-07-04T08:23:46.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f620e200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6208113190 con 0x7f620810d380 2024-07-04T08:23:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6207400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6208103f20 0x7f6208199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6206a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6208104870 0x7f620819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6207400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6208103f20 0x7f6208199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:36178/0 (socket says 172.21.15.39:36178) 2024-07-04T08:23:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6207400640 1 -- 172.21.15.39:0/2171307563 learned_addr learned my addr 172.21.15.39:0/2171307563 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:46.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6207e00640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f620810d380 0x7f620819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:46.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6207400640 1 -- 172.21.15.39:0/2171307563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f620810d380 msgr2=0x7f620819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:46.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6207400640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f620810d380 0x7f620819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:46.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6207400640 1 -- 172.21.15.39:0/2171307563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6208104870 msgr2=0x7f620819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:46.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6207400640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6208104870 0x7f620819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:46.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.674+0000 7f6207400640 1 -- 172.21.15.39:0/2171307563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620819f9d0 con 0x7f6208103f20 2024-07-04T08:23:46.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.675+0000 7f6207400640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6208103f20 0x7f6208199a60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f61f8032fd0 tx=0x7f61f8031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:46.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.675+0000 7f61ffe00640 1 -- 172.21.15.39:0/2171307563 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f803f030 con 0x7f6208103f20 2024-07-04T08:23:46.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.675+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620819fc60 con 0x7f6208103f20 2024-07-04T08:23:46.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.675+0000 7f61ffe00640 1 -- 172.21.15.39:0/2171307563 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61f8004490 con 0x7f6208103f20 2024-07-04T08:23:46.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.675+0000 7f61ffe00640 1 -- 172.21.15.39:0/2171307563 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8035070 con 0x7f6208103f20 2024-07-04T08:23:46.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.675+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62081a6420 con 0x7f6208103f20 2024-07-04T08:23:46.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.677+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620806b790 con 0x7f6208103f20 2024-07-04T08:23:46.672 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.679+0000 7f61ffe00640 1 -- 172.21.15.39:0/2171307563 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f61f8042030 con 0x7f6208103f20 2024-07-04T08:23:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.681+0000 7f61ffe00640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61ec0b2960 0x7f61ec0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:46.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.681+0000 7f61ffe00640 1 -- 172.21.15.39:0/2171307563 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f61f8102940 con 0x7f6208103f20 2024-07-04T08:23:46.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.681+0000 7f6206a00640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61ec0b2960 0x7f61ec0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:46.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.682+0000 7f6206a00640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61ec0b2960 0x7f61ec0b4e70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f620810f1a0 tx=0x7f61f4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:46.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.685+0000 7f61ffe00640 1 -- 172.21.15.39:0/2171307563 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f61f80c9910 con 0x7f6208103f20 2024-07-04T08:23:46.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.942+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6208104330 con 0x7f6208103f20 2024-07-04T08:23:46.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.943+0000 7f61ffe00640 1 -- 172.21.15.39:0/2171307563 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f61f80cdc10 con 0x7f6208103f20 2024-07-04T08:23:46.937 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:23:46.937 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:23:46.938 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:23:46.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61ec0b2960 msgr2=0x7f61ec0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:46.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61ec0b2960 0x7f61ec0b4e70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f620810f1a0 tx=0x7f61f4005f90 comp rx=0 tx=0).stop 2024-07-04T08:23:46.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6208103f20 msgr2=0x7f6208199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:46.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6208103f20 0x7f6208199a60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f61f8032fd0 tx=0x7f61f8031a20 comp rx=0 tx=0).stop 2024-07-04T08:23:46.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 shutdown_connections 2024-07-04T08:23:46.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f61ec0b2960 0x7f61ec0b4e70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:46.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6208104870 0x7f620819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f620810d380 0x7f620819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 --2- 172.21.15.39:0/2171307563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6208103f20 0x7f6208199a60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.946+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 >> 172.21.15.39:0/2171307563 conn(0x7f6208078c90 msgr2=0x7f62080feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.947+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 shutdown_connections 2024-07-04T08:23:46.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:46.947+0000 7f620e200640 1 -- 172.21.15.39:0/2171307563 wait complete. 2024-07-04T08:23:47.280 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:47 smithi039 ceph-mon[32062]: pgmap v741: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-07-04T08:23:47.280 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:47 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2171307563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:23:47.360 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:47 smithi133 ceph-mon[37050]: pgmap v741: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-07-04T08:23:47.360 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:47 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2171307563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:23:47.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:47 smithi050 ceph-mon[36577]: pgmap v741: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-07-04T08:23:47.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:47 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2171307563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:23:48.288 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id 3925f374 --release --await' 2024-07-04T08:23:48.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id 3925f374 --release --await 2024-07-04T08:23:48.942 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:48.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.960+0000 7f6ff5600640 1 -- 172.21.15.39:0/3926092587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0076880 msgr2=0x7f6ff0074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.960+0000 7f6ff5600640 1 --2- 172.21.15.39:0/3926092587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0076880 0x7f6ff0074d00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009900 tx=0x7f6fe002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.960+0000 7f6ff5600640 1 -- 172.21.15.39:0/3926092587 shutdown_connections 2024-07-04T08:23:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.960+0000 7f6ff5600640 1 --2- 172.21.15.39:0/3926092587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff0113980 0x7f6ff0115e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.960+0000 7f6ff5600640 1 --2- 172.21.15.39:0/3926092587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff0075240 0x7f6ff0075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.960+0000 7f6ff5600640 1 --2- 172.21.15.39:0/3926092587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0076880 0x7f6ff0074d00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.960+0000 7f6ff5600640 1 -- 172.21.15.39:0/3926092587 >> 172.21.15.39:0/3926092587 conn(0x7f6ff007ae60 msgr2=0x7f6ff007b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.961+0000 7f6ff5600640 1 -- 172.21.15.39:0/3926092587 shutdown_connections 2024-07-04T08:23:48.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.961+0000 7f6ff5600640 1 -- 172.21.15.39:0/3926092587 wait complete. 2024-07-04T08:23:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.962+0000 7f6ff5600640 1 Processor -- start 2024-07-04T08:23:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.962+0000 7f6ff5600640 1 -- start start 2024-07-04T08:23:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6ff5600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff0075240 0x7f6ff01a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6ff5600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff0076880 0x7f6ff01a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6ff5600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0113980 0x7f6ff01a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6ff5600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0119610 con 0x7f6ff0076880 2024-07-04T08:23:48.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6ff5600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0119490 con 0x7f6ff0075240 2024-07-04T08:23:48.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6ff5600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff0119790 con 0x7f6ff0113980 2024-07-04T08:23:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6feea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff0075240 0x7f6ff01a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6fee000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff0076880 0x7f6ff01a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6feea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff0075240 0x7f6ff01a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:36204/0 (socket says 172.21.15.39:36204) 2024-07-04T08:23:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6feea00640 1 -- 172.21.15.39:0/1838263140 learned_addr learned my addr 172.21.15.39:0/1838263140 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.963+0000 7f6fee000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff0076880 0x7f6ff01a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39680/0 (socket says 172.21.15.39:39680) 2024-07-04T08:23:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.964+0000 7f6fef400640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0113980 0x7f6ff01a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.964+0000 7f6feea00640 1 -- 172.21.15.39:0/1838263140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0113980 msgr2=0x7f6ff01a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:48.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.964+0000 7f6feea00640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0113980 0x7f6ff01a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.964+0000 7f6feea00640 1 -- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff0076880 msgr2=0x7f6ff01a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:48.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.964+0000 7f6feea00640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff0076880 0x7f6ff01a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.964+0000 7f6feea00640 1 -- 172.21.15.39:0/1838263140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff01a6030 con 0x7f6ff0075240 2024-07-04T08:23:48.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.964+0000 7f6fee000640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff0076880 0x7f6ff01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:48.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.964+0000 7f6feea00640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff0075240 0x7f6ff01a00c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0005940 tx=0x7f6fe00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:48.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.965+0000 7f6fe7400640 1 -- 172.21.15.39:0/1838263140 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe003f030 con 0x7f6ff0075240 2024-07-04T08:23:48.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.965+0000 7f6fe7400640 1 -- 172.21.15.39:0/1838263140 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe00044c0 con 0x7f6ff0075240 2024-07-04T08:23:48.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.965+0000 7f6ff5600640 1 -- 172.21.15.39:0/1838263140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff01a62c0 con 0x7f6ff0075240 2024-07-04T08:23:48.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.965+0000 7f6fe7400640 1 -- 172.21.15.39:0/1838263140 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0035070 con 0x7f6ff0075240 2024-07-04T08:23:48.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.965+0000 7f6ff5600640 1 -- 172.21.15.39:0/1838263140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff01aca80 con 0x7f6ff0075240 2024-07-04T08:23:48.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.969+0000 7f6ffbb5c740 1 Processor -- start 2024-07-04T08:23:48.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.969+0000 7f6ffbb5c740 1 -- start start 2024-07-04T08:23:48.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.969+0000 7f6fe7400640 1 -- 172.21.15.39:0/1838263140 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6fe0042030 con 0x7f6ff0075240 2024-07-04T08:23:48.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda25f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 0x55bda25f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda25f5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bda256aee0 con 0x55bda222f390 2024-07-04T08:23:48.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bda255a480 con 0x55bda2250070 2024-07-04T08:23:48.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bda255e4c0 con 0x55bda22271b0 2024-07-04T08:23:48.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6fef400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda25f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6fef400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda25f47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:36210/0 (socket says 172.21.15.39:36210) 2024-07-04T08:23:48.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6fef400640 1 -- 172.21.15.39:0/1100447825 learned_addr learned my addr 172.21.15.39:0/1100447825 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:48.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6feea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 0x55bda25f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.970+0000 7f6fee000640 1 --2- 172.21.15.39:0/1100447825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda25f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.971+0000 7f6fef400640 1 -- 172.21.15.39:0/1100447825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 msgr2=0x55bda25f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.971+0000 7f6fef400640 1 --2- 172.21.15.39:0/1100447825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 0x55bda25f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.971+0000 7f6fef400640 1 -- 172.21.15.39:0/1100447825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 msgr2=0x55bda25f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.971+0000 7f6fef400640 1 --2- 172.21.15.39:0/1100447825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda25f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.971+0000 7f6fef400640 1 -- 172.21.15.39:0/1100447825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda24f2650 con 0x55bda2250070 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.971+0000 7f6fef400640 1 --2- 172.21.15.39:0/1100447825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda25f47b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009f20 tx=0x7f6fe8010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52825c1c8efa508 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.971+0000 7f6fdbe00640 1 -- 172.21.15.39:0/1100447825 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8011700 con 0x55bda2250070 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.972+0000 7f6fdbe00640 1 -- 172.21.15.39:0/1100447825 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe8011d50 con 0x55bda2250070 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.972+0000 7f6fdbe00640 1 -- 172.21.15.39:0/1100447825 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe801ce10 con 0x55bda2250070 2024-07-04T08:23:48.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.972+0000 7f6fe7400640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b29e0 0x7f6fd40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/1100447825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 msgr2=0x55bda25f47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/1100447825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda25f47b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009f20 tx=0x7f6fe8010860 comp rx=0 tx=0).stop 2024-07-04T08:23:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6fe7400640 1 -- 172.21.15.39:0/1838263140 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f6fe0102d10 con 0x7f6ff0075240 2024-07-04T08:23:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6feea00640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b29e0 0x7f6fd40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/1100447825 shutdown_connections 2024-07-04T08:23:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/1100447825 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda25f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/1100447825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 0x55bda25f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/1100447825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda25f47b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/1100447825 >> 172.21.15.39:0/1100447825 conn(0x55bda2244460 msgr2=0x55bda25f26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/1100447825 shutdown_connections 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.973+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/1100447825 wait complete. 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.974+0000 7f6feea00640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b29e0 0x7f6fd40b4ef0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55bda25f3430 tx=0x7f6fe0106800 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.974+0000 7f6ffbb5c740 1 -- start start 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.974+0000 7f6ffbb5c740 1 Processor -- start 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 0x55bda2600330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda2600870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda260c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bda23be700 con 0x55bda222f390 2024-07-04T08:23:48.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bda2501bb0 con 0x55bda2250070 2024-07-04T08:23:48.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bda2602f90 con 0x55bda22271b0 2024-07-04T08:23:48.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6ffbb5c740 10 client.0 _ll_register_callbacks cb 0x55bda2553570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:48.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6fee000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda2600870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6fef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 0x55bda2600330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6fee000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda2600870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39694/0 (socket says 172.21.15.39:39694) 2024-07-04T08:23:48.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6fee000640 1 -- 172.21.15.39:0/3684021442 learned_addr learned my addr 172.21.15.39:0/3684021442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:48.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.975+0000 7f6fef400640 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda260c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.976+0000 7f6fee000640 1 -- 172.21.15.39:0/3684021442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 msgr2=0x55bda2600330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.976+0000 7f6fee000640 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 0x55bda2600330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.976+0000 7f6fee000640 1 -- 172.21.15.39:0/3684021442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 msgr2=0x55bda260c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.976+0000 7f6fee000640 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda260c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.976+0000 7f6fee000640 1 -- 172.21.15.39:0/3684021442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda22bb650 con 0x55bda222f390 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.976+0000 7f6fee000640 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda2600870 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0129e0 tx=0x7f6fdc012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375d7cf557f87b8e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.976+0000 7f6fd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.977+0000 7f6fd8c00640 1 -- 172.21.15.39:0/3684021442 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc004050 con 0x55bda222f390 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.977+0000 7f6fd8c00640 1 -- 172.21.15.39:0/3684021442 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f6fdc0041f0 con 0x55bda222f390 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.977+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/3684021442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe8011d50 con 0x55bda222f390 2024-07-04T08:23:48.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.977+0000 7f6fd8c00640 1 -- 172.21.15.39:0/3684021442 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fdc0043f0 con 0x55bda222f390 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.977+0000 7f6ffbb5c740 10 client.17388 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.977+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/3684021442 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bda260f030 con 0x55bda222f390 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.978+0000 7f6fd8c00640 1 -- 172.21.15.39:0/3684021442 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe8011d50 con 0x55bda222f390 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.978+0000 7f6fd8c00640 1 -- 172.21.15.39:0/3684021442 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6fdc003bb0 con 0x55bda222f390 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.978+0000 7f6ffbb5c740 10 client.17388 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.978+0000 7f6ffbb5c740 10 client.17388 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.978+0000 7f6ffbb5c740 20 client.17388 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.978+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55bda220e9f0 0x55bda26124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.978+0000 7f6ffbb5c740 4 client.17388 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.978+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/3684021442 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bda256ce30 con 0x55bda220e9f0 2024-07-04T08:23:48.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.979+0000 7f6fef400640 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55bda220e9f0 0x55bda26124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.979+0000 7f6fd8c00640 10 client.17388 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:23:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.979+0000 7f6fef400640 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55bda220e9f0 0x55bda26124f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc1a65bf61b535fe server_cookie=5a91d0394bb5009f in_seq=0 out_seq=0 2024-07-04T08:23:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.980+0000 7f6fd8c00640 1 -- 172.21.15.39:0/3684021442 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bda256ce30 con 0x55bda220e9f0 2024-07-04T08:23:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.980+0000 7f6fd8c00640 10 client.17388 handle_command_reply: tid=0 2024-07-04T08:23:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.980+0000 7f6ffbb5c740 1 client.17388 shutdown 2024-07-04T08:23:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/3684021442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 msgr2=0x55bda2600870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda2600870 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0129e0 tx=0x7f6fdc012ea0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/3684021442 shutdown_connections 2024-07-04T08:23:48.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55bda220e9f0 0x55bda26124f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda222f390 0x55bda2600870 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6fd8c00640 0 client.17388 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:23:48.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda260c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/3684021442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda22271b0 0x55bda2600330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/3684021442 >> 172.21.15.39:0/3684021442 conn(0x55bda2244460 msgr2=0x55bda25f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:48.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/3684021442 shutdown_connections 2024-07-04T08:23:48.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/3684021442 wait complete. 2024-07-04T08:23:48.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.981+0000 7f6ffbb5c740 20 client.17388 trim_cache size 0 max 16384 2024-07-04T08:23:48.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.998+0000 7f6ffbb5c740 1 Processor -- start 2024-07-04T08:23:48.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.998+0000 7f6ffbb5c740 1 -- start start 2024-07-04T08:23:48.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.998+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda25f1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.998+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda22271b0 0x55bda25ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.998+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda2250070 0x55bda25f9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:48.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.998+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b6a10 con 0x55bda2250070 2024-07-04T08:23:48.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.998+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b8550 con 0x55bda22271b0 2024-07-04T08:23:48.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.998+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b70e0 con 0x55bda220e9f0 2024-07-04T08:23:48.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda25f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda25f1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39292/0 (socket says 172.21.15.39:39292) 2024-07-04T08:23:48.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fef400640 1 -- 172.21.15.39:0/690933723 learned_addr learned my addr 172.21.15.39:0/690933723 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:48.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fee000640 1 --2- 172.21.15.39:0/690933723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda22271b0 0x55bda25ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fef400640 1 --2- 172.21.15.39:0/690933723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda2250070 0x55bda25f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:48.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fee000640 1 -- 172.21.15.39:0/690933723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 msgr2=0x55bda25f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:48.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fee000640 1 --2- 172.21.15.39:0/690933723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda25f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fee000640 1 -- 172.21.15.39:0/690933723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda2250070 msgr2=0x55bda25f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fee000640 1 --2- 172.21.15.39:0/690933723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda2250070 0x55bda25f9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:48.999+0000 7f6fee000640 1 -- 172.21.15.39:0/690933723 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bda25fff00 con 0x55bda22271b0 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.000+0000 7f6fee000640 1 --2- 172.21.15.39:0/690933723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda22271b0 0x55bda25ff0c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc003f80 tx=0x7f6fdc0123e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=353d089b9f0cc95b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.000+0000 7f6fdb400640 1 -- 172.21.15.39:0/690933723 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc010620 con 0x55bda22271b0 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.000+0000 7f6fdb400640 1 -- 172.21.15.39:0/690933723 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fdc0107c0 con 0x55bda22271b0 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.000+0000 7f6fdb400640 1 -- 172.21.15.39:0/690933723 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc00fd70 con 0x55bda22271b0 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.001+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/690933723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda22271b0 msgr2=0x55bda25ff0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.001+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/690933723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda22271b0 0x55bda25ff0c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc003f80 tx=0x7f6fdc0123e0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.001+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/690933723 shutdown_connections 2024-07-04T08:23:48.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.001+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/690933723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda2250070 0x55bda25f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.001+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/690933723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda22271b0 0x55bda25ff0c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.001+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/690933723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda25f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.001+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/690933723 >> 172.21.15.39:0/690933723 conn(0x55bda2244460 msgr2=0x55bda25fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.001+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/690933723 shutdown_connections 2024-07-04T08:23:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.001+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/690933723 wait complete. 2024-07-04T08:23:48.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.002+0000 7f6ffbb5c740 1 -- start start 2024-07-04T08:23:48.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.002+0000 7f6ffbb5c740 1 Processor -- start 2024-07-04T08:23:49.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda26089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:49.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda22271b0 0x55bda260bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:49.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6ffbb5c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda260c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:49.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b7e80 con 0x55bda22271b0 2024-07-04T08:23:49.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b77b0 con 0x55bda2250070 2024-07-04T08:23:49.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6ffbb5c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff01b49f0 con 0x55bda220e9f0 2024-07-04T08:23:49.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6ffbb5c740 10 client.0 _ll_register_callbacks cb 0x7f6fdc009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:49.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6fef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda26089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:49.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6fef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda26089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39308/0 (socket says 172.21.15.39:39308) 2024-07-04T08:23:49.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6fef400640 1 -- 172.21.15.39:0/152837048 learned_addr learned my addr 172.21.15.39:0/152837048 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:49.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6fee000640 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda22271b0 0x55bda260bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:49.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6fef400640 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda260c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:49.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6fef400640 1 -- 172.21.15.39:0/152837048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 msgr2=0x55bda260c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:49.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.003+0000 7f6fef400640 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda260c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.004+0000 7f6fef400640 1 -- 172.21.15.39:0/152837048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda22271b0 msgr2=0x55bda260bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:49.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.004+0000 7f6fef400640 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda22271b0 0x55bda260bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.004+0000 7f6fef400640 1 -- 172.21.15.39:0/152837048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bda260c9f0 con 0x55bda220e9f0 2024-07-04T08:23:49.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.004+0000 7f6fef400640 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda26089f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6fe801b000 tx=0x7f6fe8013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f9f759e1d85f8a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:49.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.004+0000 7f6fd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.004+0000 7f6fd8c00640 1 -- 172.21.15.39:0/152837048 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8004be0 con 0x55bda220e9f0 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.004+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/152837048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fdc0107c0 con 0x55bda220e9f0 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.004+0000 7f6fd8c00640 1 -- 172.21.15.39:0/152837048 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f6fe8004d80 con 0x55bda220e9f0 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6fd8c00640 1 -- 172.21.15.39:0/152837048 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6fe8045050 con 0x55bda220e9f0 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6ffbb5c740 10 client.27451 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/152837048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bda267ad00 con 0x55bda220e9f0 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6fd8c00640 1 -- 172.21.15.39:0/152837048 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fdc0107c0 con 0x55bda220e9f0 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6fd8c00640 1 -- 172.21.15.39:0/152837048 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6fe8005040 con 0x55bda220e9f0 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6ffbb5c740 10 client.27451 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:49.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6ffbb5c740 10 client.27451 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6ffbb5c740 20 client.27451 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55bda21e3da0 0x55bda267d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6ffbb5c740 4 client.27451 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.005+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/152837048 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bda25688d0 con 0x55bda21e3da0 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.006+0000 7f6fee000640 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55bda21e3da0 0x55bda267d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.006+0000 7f6fee000640 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55bda21e3da0 0x55bda267d310 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ded5bf384aa6a4e5 server_cookie=ac794f5c6eb00458 in_seq=0 out_seq=0 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.006+0000 7f6fd8c00640 10 client.27451 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.006+0000 7f6fd8c00640 1 -- 172.21.15.39:0/152837048 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bda25688d0 con 0x55bda21e3da0 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.006+0000 7f6fd8c00640 10 client.27451 handle_command_reply: tid=0 2024-07-04T08:23:49.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.006+0000 7f6ffbb5c740 1 client.27451 shutdown 2024-07-04T08:23:49.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/152837048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 msgr2=0x55bda26089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:49.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda26089f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6fe801b000 tx=0x7f6fe8013040 comp rx=0 tx=0).stop 2024-07-04T08:23:49.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/152837048 shutdown_connections 2024-07-04T08:23:49.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55bda21e3da0 0x55bda267d310 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bda22271b0 0x55bda260bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bda2250070 0x55bda260c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 --2- 172.21.15.39:0/152837048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bda220e9f0 0x55bda26089f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/152837048 >> 172.21.15.39:0/152837048 conn(0x55bda2244460 msgr2=0x55bda26015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:49.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6fd8c00640 0 client.27451 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:23:49.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/152837048 shutdown_connections 2024-07-04T08:23:49.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 1 -- 172.21.15.39:0/152837048 wait complete. 2024-07-04T08:23:49.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.007+0000 7f6ffbb5c740 20 client.27451 trim_cache size 0 max 16384 2024-07-04T08:23:49.007 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:23:49.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 -- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b29e0 msgr2=0x7f6fd40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:49.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b29e0 0x7f6fd40b4ef0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55bda25f3430 tx=0x7f6fe0106800 comp rx=0 tx=0).stop 2024-07-04T08:23:49.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 -- 172.21.15.39:0/1838263140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff0075240 msgr2=0x7f6ff01a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:49.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff0075240 0x7f6ff01a00c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0005940 tx=0x7f6fe00040c0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 -- 172.21.15.39:0/1838263140 shutdown_connections 2024-07-04T08:23:49.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6fd40b29e0 0x7f6fd40b4ef0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ff0076880 0x7f6ff01a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ff0113980 0x7f6ff01a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 --2- 172.21.15.39:0/1838263140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ff0075240 0x7f6ff01a00c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:49.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 -- 172.21.15.39:0/1838263140 >> 172.21.15.39:0/1838263140 conn(0x7f6ff007ae60 msgr2=0x7f6ff0112730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:49.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 -- 172.21.15.39:0/1838263140 shutdown_connections 2024-07-04T08:23:49.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:49.008+0000 7f6ff5600640 1 -- 172.21.15.39:0/1838263140 wait complete. 2024-07-04T08:23:49.233 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:49 smithi039 ceph-mon[32062]: pgmap v742: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-07-04T08:23:49.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:49 smithi050 ceph-mon[36577]: pgmap v742: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-07-04T08:23:49.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:49 smithi133 ceph-mon[37050]: pgmap v742: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 11 op/s 2024-07-04T08:23:50.552 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:23:51.129 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:51.271 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:51 smithi039 ceph-mon[32062]: pgmap v743: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-07-04T08:23:51.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:51 smithi050 ceph-mon[36577]: pgmap v743: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-07-04T08:23:51.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:51 smithi133 ceph-mon[37050]: pgmap v743: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.1 KiB/s wr, 10 op/s 2024-07-04T08:23:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.860+0000 7f0e73800640 1 -- 172.21.15.39:0/1753815963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e6c103f00 msgr2=0x7f0e6c104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.860+0000 7f0e73800640 1 --2- 172.21.15.39:0/1753815963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e6c103f00 0x7f0e6c104310 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009900 tx=0x7f0e6002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.860+0000 7f0e73800640 1 -- 172.21.15.39:0/1753815963 shutdown_connections 2024-07-04T08:23:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.860+0000 7f0e73800640 1 --2- 172.21.15.39:0/1753815963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c104850 0x7f0e6c10cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.860+0000 7f0e73800640 1 --2- 172.21.15.39:0/1753815963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c10d360 0x7f0e6c10f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.860+0000 7f0e73800640 1 --2- 172.21.15.39:0/1753815963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e6c103f00 0x7f0e6c104310 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.860+0000 7f0e73800640 1 -- 172.21.15.39:0/1753815963 >> 172.21.15.39:0/1753815963 conn(0x7f0e6c078c80 msgr2=0x7f0e6c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.861+0000 7f0e73800640 1 -- 172.21.15.39:0/1753815963 shutdown_connections 2024-07-04T08:23:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.861+0000 7f0e73800640 1 -- 172.21.15.39:0/1753815963 wait complete. 2024-07-04T08:23:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.862+0000 7f0e73800640 1 Processor -- start 2024-07-04T08:23:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.862+0000 7f0e73800640 1 -- start start 2024-07-04T08:23:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.863+0000 7f0e73800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c103f00 0x7f0e6c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.863+0000 7f0e73800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c104850 0x7f0e6c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.863+0000 7f0e73800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e6c10d360 0x7f0e6c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.863+0000 7f0e73800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c113020 con 0x7f0e6c103f00 2024-07-04T08:23:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.863+0000 7f0e73800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c112ea0 con 0x7f0e6c10d360 2024-07-04T08:23:51.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.863+0000 7f0e73800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1131a0 con 0x7f0e6c104850 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.863+0000 7f0e70e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c103f00 0x7f0e6c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.863+0000 7f0e70e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c103f00 0x7f0e6c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51858/0 (socket says 172.21.15.39:51858) 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.863+0000 7f0e70e00640 1 -- 172.21.15.39:0/2705347378 learned_addr learned my addr 172.21.15.39:0/2705347378 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.864+0000 7f0e6be00640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c104850 0x7f0e6c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.864+0000 7f0e71800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e6c10d360 0x7f0e6c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.864+0000 7f0e70e00640 1 -- 172.21.15.39:0/2705347378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c104850 msgr2=0x7f0e6c19c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.864+0000 7f0e70e00640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c104850 0x7f0e6c19c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.864+0000 7f0e70e00640 1 -- 172.21.15.39:0/2705347378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e6c10d360 msgr2=0x7f0e6c1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.864+0000 7f0e70e00640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e6c10d360 0x7f0e6c1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.864+0000 7f0e70e00640 1 -- 172.21.15.39:0/2705347378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6c1a1c80 con 0x7f0e6c103f00 2024-07-04T08:23:51.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.864+0000 7f0e71800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e6c10d360 0x7f0e6c1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:23:51.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.865+0000 7f0e70e00640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c103f00 0x7f0e6c19bd10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0e60005ab0 tx=0x7f0e60003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:51.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.865+0000 7f0e69600640 1 -- 172.21.15.39:0/2705347378 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6003f070 con 0x7f0e6c103f00 2024-07-04T08:23:51.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.865+0000 7f0e69600640 1 -- 172.21.15.39:0/2705347378 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e60031b50 con 0x7f0e6c103f00 2024-07-04T08:23:51.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.865+0000 7f0e73800640 1 -- 172.21.15.39:0/2705347378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6c1a1f10 con 0x7f0e6c103f00 2024-07-04T08:23:51.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.865+0000 7f0e69600640 1 -- 172.21.15.39:0/2705347378 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60035030 con 0x7f0e6c103f00 2024-07-04T08:23:51.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.865+0000 7f0e73800640 1 -- 172.21.15.39:0/2705347378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6c1a86d0 con 0x7f0e6c103f00 2024-07-04T08:23:51.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.869+0000 7f0e79e48740 1 Processor -- start 2024-07-04T08:23:51.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.870+0000 7f0e79e48740 1 -- start start 2024-07-04T08:23:51.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.870+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee30520 0x55607f1d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.870+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 0x55607f1d5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.870+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee0f840 0x55607f1d5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.870+0000 7f0e79e48740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55607f0efb20 con 0x55607ee0f840 2024-07-04T08:23:51.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.870+0000 7f0e79e48740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55607f14b2b0 con 0x55607ee07660 2024-07-04T08:23:51.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.870+0000 7f0e79e48740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55607f13a850 con 0x55607ee30520 2024-07-04T08:23:51.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e71800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee30520 0x55607f1d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e71800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee30520 0x55607f1d52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59626/0 (socket says 172.21.15.39:59626) 2024-07-04T08:23:51.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e71800640 1 -- 172.21.15.39:0/1712151460 learned_addr learned my addr 172.21.15.39:0/1712151460 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:51.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e71800640 1 -- 172.21.15.39:0/1712151460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 msgr2=0x55607f1d5800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:23:51.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e71800640 1 --2- 172.21.15.39:0/1712151460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 0x55607f1d5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e71800640 1 -- 172.21.15.39:0/1712151460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee0f840 msgr2=0x55607f1d5d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:23:51.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e6be00640 1 --2- 172.21.15.39:0/1712151460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee0f840 0x55607f1d5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e71800640 1 --2- 172.21.15.39:0/1712151460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee0f840 0x55607f1d5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e71800640 1 -- 172.21.15.39:0/1712151460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55607f0f4780 con 0x55607ee30520 2024-07-04T08:23:51.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.871+0000 7f0e6be00640 1 --2- 172.21.15.39:0/1712151460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee0f840 0x55607f1d5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:23:51.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.872+0000 7f0e69600640 1 -- 172.21.15.39:0/2705347378 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0e60042070 con 0x7f0e6c103f00 2024-07-04T08:23:51.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.872+0000 7f0e71800640 1 --2- 172.21.15.39:0/1712151460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee30520 0x55607f1d52c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009fd0 tx=0x7f0e64010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c78505f2bed899 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:51.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.872+0000 7f0e5e000640 1 -- 172.21.15.39:0/1712151460 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64011700 con 0x55607ee30520 2024-07-04T08:23:51.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.872+0000 7f0e5e000640 1 -- 172.21.15.39:0/1712151460 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e64011d50 con 0x55607ee30520 2024-07-04T08:23:51.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.872+0000 7f0e5e000640 1 -- 172.21.15.39:0/1712151460 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6400e3c0 con 0x55607ee30520 2024-07-04T08:23:51.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.873+0000 7f0e79e48740 1 -- 172.21.15.39:0/1712151460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee30520 msgr2=0x55607f1d52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:51.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.873+0000 7f0e79e48740 1 --2- 172.21.15.39:0/1712151460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee30520 0x55607f1d52c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009fd0 tx=0x7f0e64010860 comp rx=0 tx=0).stop 2024-07-04T08:23:51.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.874+0000 7f0e79e48740 1 -- 172.21.15.39:0/1712151460 shutdown_connections 2024-07-04T08:23:51.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.874+0000 7f0e79e48740 1 --2- 172.21.15.39:0/1712151460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee0f840 0x55607f1d5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.874+0000 7f0e79e48740 1 --2- 172.21.15.39:0/1712151460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 0x55607f1d5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.874+0000 7f0e79e48740 1 --2- 172.21.15.39:0/1712151460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee30520 0x55607f1d52c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.874+0000 7f0e79e48740 1 -- 172.21.15.39:0/1712151460 >> 172.21.15.39:0/1712151460 conn(0x55607ee24910 msgr2=0x55607f1d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:51.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.874+0000 7f0e79e48740 1 -- 172.21.15.39:0/1712151460 shutdown_connections 2024-07-04T08:23:51.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.874+0000 7f0e79e48740 1 -- 172.21.15.39:0/1712151460 wait complete. 2024-07-04T08:23:51.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.874+0000 7f0e69600640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b2870 0x7f0e540b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.874+0000 7f0e79e48740 1 -- start start 2024-07-04T08:23:51.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.875+0000 7f0e69600640 1 -- 172.21.15.39:0/2705347378 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f0e60103040 con 0x7f0e6c103f00 2024-07-04T08:23:51.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.875+0000 7f0e79e48740 1 Processor -- start 2024-07-04T08:23:51.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.875+0000 7f0e71800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b2870 0x7f0e540b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.876+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee07660 0x55607f1e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.876+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1e0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.876+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee30520 0x55607f1ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.876+0000 7f0e79e48740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55607f0e1d20 con 0x55607ee07660 2024-07-04T08:23:51.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.876+0000 7f0e79e48740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55607f13e890 con 0x55607ee30520 2024-07-04T08:23:51.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.876+0000 7f0e79e48740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55607ef9ebb0 con 0x55607ee0f840 2024-07-04T08:23:51.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.876+0000 7f0e79e48740 10 client.0 _ll_register_callbacks cb 0x55607f0e29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:51.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.876+0000 7f0e71800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee30520 0x55607f1ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.876+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1e0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1e0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59634/0 (socket says 172.21.15.39:59634) 2024-07-04T08:23:51.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e6be00640 1 -- 172.21.15.39:0/672550172 learned_addr learned my addr 172.21.15.39:0/672550172 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:51.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e70e00640 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee07660 0x55607f1e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e71800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b2870 0x7f0e540b4d80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0e64005bf0 tx=0x7f0e64005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:51.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e71800640 1 -- 172.21.15.39:0/672550172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 msgr2=0x55607f1e0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:51.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e71800640 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1e0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e71800640 1 -- 172.21.15.39:0/672550172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee07660 msgr2=0x55607f1e0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:51.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e71800640 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee07660 0x55607f1e0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e71800640 1 -- 172.21.15.39:0/672550172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55607f0d1660 con 0x55607ee30520 2024-07-04T08:23:51.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e6be00640 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1e0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:51.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.877+0000 7f0e70e00640 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee07660 0x55607f1e0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:51.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.878+0000 7f0e71800640 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee30520 0x55607f1ecc70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0e64005cc0 tx=0x7f0e64010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9afb0130be2cd5ae server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:51.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.878+0000 7f0e52a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:23:51.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.878+0000 7f0e52a00640 1 -- 172.21.15.39:0/672550172 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64005040 con 0x55607ee30520 2024-07-04T08:23:51.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.879+0000 7f0e52a00640 1 -- 172.21.15.39:0/672550172 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f0e64022040 con 0x55607ee30520 2024-07-04T08:23:51.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.879+0000 7f0e79e48740 1 -- 172.21.15.39:0/672550172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e64011d50 con 0x55607ee30520 2024-07-04T08:23:51.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.879+0000 7f0e52a00640 1 -- 172.21.15.39:0/672550172 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e6403c050 con 0x55607ee30520 2024-07-04T08:23:51.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.879+0000 7f0e79e48740 10 client.27539 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:51.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.879+0000 7f0e79e48740 1 -- 172.21.15.39:0/672550172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55607ee9bb00 con 0x55607ee30520 2024-07-04T08:23:51.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.880+0000 7f0e52a00640 1 -- 172.21.15.39:0/672550172 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e64011d50 con 0x55607ee30520 2024-07-04T08:23:51.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.880+0000 7f0e52a00640 1 -- 172.21.15.39:0/672550172 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0e64027440 con 0x55607ee30520 2024-07-04T08:23:51.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.880+0000 7f0e79e48740 10 client.27539 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:51.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.880+0000 7f0e79e48740 10 client.27539 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:23:51.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.880+0000 7f0e79e48740 20 client.27539 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:51.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.880+0000 7f0e79e48740 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607edeeea0 0x55607f1fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.880+0000 7f0e79e48740 4 client.27539 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:23:51.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.880+0000 7f0e79e48740 1 -- 172.21.15.39:0/672550172 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55607f14d200 con 0x55607edeeea0 2024-07-04T08:23:51.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.880+0000 7f0e70e00640 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607edeeea0 0x55607f1fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.881+0000 7f0e52a00640 10 client.27539 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:51.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.881+0000 7f0e70e00640 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607edeeea0 0x55607f1fb0e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f008aca452dddd67 server_cookie=5dabf41d91a1515b in_seq=0 out_seq=0 2024-07-04T08:23:51.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.882+0000 7f0e52a00640 1 -- 172.21.15.39:0/672550172 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55607f14d200 con 0x55607edeeea0 2024-07-04T08:23:51.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.882+0000 7f0e52a00640 10 client.27539 handle_command_reply: tid=0 2024-07-04T08:23:51.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.882+0000 7f0e79e48740 1 client.27539 shutdown 2024-07-04T08:23:51.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.882+0000 7f0e79e48740 1 -- 172.21.15.39:0/672550172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee30520 msgr2=0x55607f1ecc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:51.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.882+0000 7f0e79e48740 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee30520 0x55607f1ecc70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0e64005cc0 tx=0x7f0e64010860 comp rx=0 tx=0).stop 2024-07-04T08:23:51.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e79e48740 1 -- 172.21.15.39:0/672550172 shutdown_connections 2024-07-04T08:23:51.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e79e48740 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607edeeea0 0x55607f1fb0e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e79e48740 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee30520 0x55607f1ecc70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e52a00640 0 client.27539 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:51.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e79e48740 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1e0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e79e48740 1 --2- 172.21.15.39:0/672550172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee07660 0x55607f1e0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e79e48740 1 -- 172.21.15.39:0/672550172 >> 172.21.15.39:0/672550172 conn(0x55607ee24910 msgr2=0x55607f1d8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:51.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e79e48740 1 -- 172.21.15.39:0/672550172 shutdown_connections 2024-07-04T08:23:51.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e79e48740 1 -- 172.21.15.39:0/672550172 wait complete. 2024-07-04T08:23:51.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.883+0000 7f0e79e48740 20 client.27539 trim_cache size 0 max 16384 2024-07-04T08:23:51.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.903+0000 7f0e79e48740 1 Processor -- start 2024-07-04T08:23:51.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e79e48740 1 -- start start 2024-07-04T08:23:51.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 0x55607f1d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1d24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 0x55607f1e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e79e48740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1ae300 con 0x55607ee30520 2024-07-04T08:23:51.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e79e48740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1afe40 con 0x55607ee07660 2024-07-04T08:23:51.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e79e48740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1ae9d0 con 0x55607ee0f840 2024-07-04T08:23:51.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e70e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 0x55607f1d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e71800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 0x55607f1e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e70e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 0x55607f1d3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:49116/0 (socket says 172.21.15.39:49116) 2024-07-04T08:23:51.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.904+0000 7f0e70e00640 1 -- 172.21.15.39:0/1922372327 learned_addr learned my addr 172.21.15.39:0/1922372327 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.905+0000 7f0e6be00640 1 --2- 172.21.15.39:0/1922372327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1d24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.905+0000 7f0e70e00640 1 -- 172.21.15.39:0/1922372327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 msgr2=0x55607f1d24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.905+0000 7f0e70e00640 1 --2- 172.21.15.39:0/1922372327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1d24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.905+0000 7f0e70e00640 1 -- 172.21.15.39:0/1922372327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 msgr2=0x55607f1e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.905+0000 7f0e70e00640 1 --2- 172.21.15.39:0/1922372327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 0x55607f1e2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.905+0000 7f0e70e00640 1 -- 172.21.15.39:0/1922372327 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55607ef12960 con 0x55607ee07660 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.905+0000 7f0e70e00640 1 --2- 172.21.15.39:0/1922372327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 0x55607f1d3070 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0e600c9c60 tx=0x7f0e60000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a694371fffa6e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.905+0000 7f0e5d600640 1 -- 172.21.15.39:0/1922372327 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60049040 con 0x55607ee07660 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.906+0000 7f0e5d600640 1 -- 172.21.15.39:0/1922372327 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e600ce310 con 0x55607ee07660 2024-07-04T08:23:51.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.906+0000 7f0e5d600640 1 -- 172.21.15.39:0/1922372327 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60100650 con 0x55607ee07660 2024-07-04T08:23:51.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.906+0000 7f0e79e48740 1 -- 172.21.15.39:0/1922372327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 msgr2=0x55607f1d3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:51.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.907+0000 7f0e79e48740 1 --2- 172.21.15.39:0/1922372327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 0x55607f1d3070 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0e600c9c60 tx=0x7f0e60000bc0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.907+0000 7f0e79e48740 1 -- 172.21.15.39:0/1922372327 shutdown_connections 2024-07-04T08:23:51.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.907+0000 7f0e79e48740 1 --2- 172.21.15.39:0/1922372327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 0x55607f1e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.907+0000 7f0e79e48740 1 --2- 172.21.15.39:0/1922372327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee0f840 0x55607f1d24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.907+0000 7f0e79e48740 1 --2- 172.21.15.39:0/1922372327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee07660 0x55607f1d3070 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.907+0000 7f0e79e48740 1 -- 172.21.15.39:0/1922372327 >> 172.21.15.39:0/1922372327 conn(0x55607ee24910 msgr2=0x55607f1d8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:51.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.907+0000 7f0e79e48740 1 -- 172.21.15.39:0/1922372327 shutdown_connections 2024-07-04T08:23:51.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.907+0000 7f0e79e48740 1 -- 172.21.15.39:0/1922372327 wait complete. 2024-07-04T08:23:51.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.907+0000 7f0e79e48740 1 -- start start 2024-07-04T08:23:51.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.908+0000 7f0e79e48740 1 Processor -- start 2024-07-04T08:23:51.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.908+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee07660 0x55607f25b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.908+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee0f840 0x55607f25bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.908+0000 7f0e79e48740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 0x55607f1fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e79e48740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1af770 con 0x55607ee30520 2024-07-04T08:23:51.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e79e48740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1af0a0 con 0x55607ee0f840 2024-07-04T08:23:51.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e79e48740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c1ac360 con 0x55607ee07660 2024-07-04T08:23:51.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e79e48740 10 client.0 _ll_register_callbacks cb 0x7f0e64004e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:51.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee0f840 0x55607f25bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e70e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee07660 0x55607f25b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e70e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee07660 0x55607f25b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59656/0 (socket says 172.21.15.39:59656) 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e70e00640 1 -- 172.21.15.39:0/4269237959 learned_addr learned my addr 172.21.15.39:0/4269237959 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee0f840 0x55607f25bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:49124/0 (socket says 172.21.15.39:49124) 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e71800640 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 0x55607f1fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e70e00640 1 -- 172.21.15.39:0/4269237959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee0f840 msgr2=0x55607f25bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e70e00640 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee0f840 0x55607f25bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e70e00640 1 -- 172.21.15.39:0/4269237959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 msgr2=0x55607f1fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e70e00640 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 0x55607f1fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.909+0000 7f0e70e00640 1 -- 172.21.15.39:0/4269237959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55607f1078a0 con 0x55607ee07660 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.910+0000 7f0e70e00640 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee07660 0x55607f25b7e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0e600c71d0 tx=0x7f0e60003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5e9efdd2b9b611 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.910+0000 7f0e52a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.910+0000 7f0e52a00640 1 -- 172.21.15.39:0/4269237959 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600e4070 con 0x55607ee07660 2024-07-04T08:23:51.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.910+0000 7f0e52a00640 1 -- 172.21.15.39:0/4269237959 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f0e60100650 con 0x55607ee07660 2024-07-04T08:23:51.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.910+0000 7f0e79e48740 1 -- 172.21.15.39:0/4269237959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e600ce310 con 0x55607ee07660 2024-07-04T08:23:51.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.910+0000 7f0e52a00640 1 -- 172.21.15.39:0/4269237959 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e60049040 con 0x55607ee07660 2024-07-04T08:23:51.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.910+0000 7f0e79e48740 10 client.27469 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:51.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.910+0000 7f0e79e48740 1 -- 172.21.15.39:0/4269237959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55607f103450 con 0x55607ee07660 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.911+0000 7f0e52a00640 1 -- 172.21.15.39:0/4269237959 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e600ce310 con 0x55607ee07660 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.911+0000 7f0e52a00640 1 -- 172.21.15.39:0/4269237959 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0e600c91c0 con 0x55607ee07660 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.911+0000 7f0e79e48740 10 client.27469 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.911+0000 7f0e79e48740 10 client.27469 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.911+0000 7f0e79e48740 20 client.27469 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.911+0000 7f0e79e48740 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607edc4250 0x55607f265350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.911+0000 7f0e79e48740 4 client.27469 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3925f374", "release": true, "await": true}] 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.911+0000 7f0e79e48740 1 -- 172.21.15.39:0/4269237959 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3925f374", "release": true, "await": true}) -- 0x55607f148ca0 con 0x55607edc4250 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.912+0000 7f0e6be00640 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607edc4250 0x55607f265350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.912+0000 7f0e6be00640 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607edc4250 0x55607f265350 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc96ddb2b7117d61 server_cookie=386ed08c04c0a9a7 in_seq=0 out_seq=0 2024-07-04T08:23:51.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.912+0000 7f0e52a00640 10 client.27469 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:51.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.914+0000 7f0e52a00640 1 -- 172.21.15.39:0/4269237959 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55607f148ca0 con 0x55607edc4250 2024-07-04T08:23:51.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.914+0000 7f0e52a00640 10 client.27469 handle_command_reply: tid=0 2024-07-04T08:23:51.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.914+0000 7f0e79e48740 1 client.27469 shutdown 2024-07-04T08:23:51.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 -- 172.21.15.39:0/4269237959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee07660 msgr2=0x55607f25b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:51.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee07660 0x55607f25b7e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0e600c71d0 tx=0x7f0e60003f90 comp rx=0 tx=0).stop 2024-07-04T08:23:51.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 -- 172.21.15.39:0/4269237959 shutdown_connections 2024-07-04T08:23:51.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55607edc4250 0x55607f265350 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55607ee30520 0x55607f1fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55607ee0f840 0x55607f25bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 --2- 172.21.15.39:0/4269237959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55607ee07660 0x55607f25b7e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 -- 172.21.15.39:0/4269237959 >> 172.21.15.39:0/4269237959 conn(0x55607ee24910 msgr2=0x55607f1d6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:51.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e52a00640 0 client.27469 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:51.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 -- 172.21.15.39:0/4269237959 shutdown_connections 2024-07-04T08:23:51.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 1 -- 172.21.15.39:0/4269237959 wait complete. 2024-07-04T08:23:51.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.915+0000 7f0e79e48740 20 client.27469 trim_cache size 0 max 16384 2024-07-04T08:23:51.910 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:23:51.910 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:23:51.910 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:23:51.910 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 24, 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "3925f374": { 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 24, 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 140.1, 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:23:51.911 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:23:51.912 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:23:51.912 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:23:51.912 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:23:51.912 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:51.912 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:51.912 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:51.912 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:51.912 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:23:51.912 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:23:51.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.916+0000 7f0e73800640 1 -- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b2870 msgr2=0x7f0e540b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:51.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.916+0000 7f0e73800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b2870 0x7f0e540b4d80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0e64005bf0 tx=0x7f0e64005fd0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.916+0000 7f0e73800640 1 -- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c103f00 msgr2=0x7f0e6c19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:51.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.916+0000 7f0e73800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c103f00 0x7f0e6c19bd10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0e60005ab0 tx=0x7f0e60003b90 comp rx=0 tx=0).stop 2024-07-04T08:23:51.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.916+0000 7f0e73800640 1 -- 172.21.15.39:0/2705347378 shutdown_connections 2024-07-04T08:23:51.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.916+0000 7f0e73800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b2870 0x7f0e540b4d80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.916+0000 7f0e73800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e6c10d360 0x7f0e6c1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.916+0000 7f0e73800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e6c104850 0x7f0e6c19c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.916+0000 7f0e73800640 1 --2- 172.21.15.39:0/2705347378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e6c103f00 0x7f0e6c19bd10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:51.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.917+0000 7f0e73800640 1 -- 172.21.15.39:0/2705347378 >> 172.21.15.39:0/2705347378 conn(0x7f0e6c078c80 msgr2=0x7f0e6c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:51.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.917+0000 7f0e73800640 1 -- 172.21.15.39:0/2705347378 shutdown_connections 2024-07-04T08:23:51.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:51.917+0000 7f0e73800640 1 -- 172.21.15.39:0/2705347378 wait complete. 2024-07-04T08:23:53.358 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:53 smithi039 ceph-mon[32062]: pgmap v744: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-07-04T08:23:53.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:53 smithi050 ceph-mon[36577]: pgmap v744: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-07-04T08:23:53.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:53 smithi133 ceph-mon[37050]: pgmap v744: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-07-04T08:23:53.843 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3925f374', total seconds elapsed: 31.9 2024-07-04T08:23:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.565+0000 7fcc83e00640 1 -- 172.21.15.39:0/2132057698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc8410d170 msgr2=0x7fcc8410f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.565+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2132057698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc8410d170 0x7fcc8410f660 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009be0 tx=0x7fcc7c02a6c0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.565+0000 7fcc83e00640 1 -- 172.21.15.39:0/2132057698 shutdown_connections 2024-07-04T08:23:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.565+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2132057698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc8410d170 0x7fcc8410f660 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.565+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2132057698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc84104660 0x7fcc8410cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.565+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2132057698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc84103d10 0x7fcc84104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.565+0000 7fcc83e00640 1 -- 172.21.15.39:0/2132057698 >> 172.21.15.39:0/2132057698 conn(0x7fcc840fd810 msgr2=0x7fcc840ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:54.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.565+0000 7fcc83e00640 1 -- 172.21.15.39:0/2132057698 shutdown_connections 2024-07-04T08:23:54.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.565+0000 7fcc83e00640 1 -- 172.21.15.39:0/2132057698 wait complete. 2024-07-04T08:23:54.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.566+0000 7fcc83e00640 1 Processor -- start 2024-07-04T08:23:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.567+0000 7fcc83e00640 1 -- start start 2024-07-04T08:23:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.567+0000 7fcc83e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc84103d10 0x7fcc84199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.567+0000 7fcc83e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc84104660 0x7fcc84199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.567+0000 7fcc83e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc8410d170 0x7fcc8419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.567+0000 7fcc83e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84112df0 con 0x7fcc84103d10 2024-07-04T08:23:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.567+0000 7fcc83e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84112c70 con 0x7fcc8410d170 2024-07-04T08:23:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.567+0000 7fcc83e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84112f70 con 0x7fcc84104660 2024-07-04T08:23:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.567+0000 7fcc82a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc84103d10 0x7fcc84199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc82a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc84103d10 0x7fcc84199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51906/0 (socket says 172.21.15.39:51906) 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc82a00640 1 -- 172.21.15.39:0/2748000894 learned_addr learned my addr 172.21.15.39:0/2748000894 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc82000640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc84104660 0x7fcc84199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc83400640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc8410d170 0x7fcc8419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc82000640 1 -- 172.21.15.39:0/2748000894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc8410d170 msgr2=0x7fcc8419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc82000640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc8410d170 0x7fcc8419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc82000640 1 -- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc84103d10 msgr2=0x7fcc84199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc82000640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc84103d10 0x7fcc84199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc82000640 1 -- 172.21.15.39:0/2748000894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8419f6f0 con 0x7fcc84104660 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.568+0000 7fcc82a00640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc84103d10 0x7fcc84199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:54.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.569+0000 7fcc82000640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc84104660 0x7fcc84199d20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcc84069fe0 tx=0x7fcc7000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.569+0000 7fcc7b400640 1 -- 172.21.15.39:0/2748000894 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7000cc40 con 0x7fcc84104660 2024-07-04T08:23:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.569+0000 7fcc7b400640 1 -- 172.21.15.39:0/2748000894 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc7000cde0 con 0x7fcc84104660 2024-07-04T08:23:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.569+0000 7fcc83e00640 1 -- 172.21.15.39:0/2748000894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8419f9e0 con 0x7fcc84104660 2024-07-04T08:23:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.569+0000 7fcc7b400640 1 -- 172.21.15.39:0/2748000894 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc70004a00 con 0x7fcc84104660 2024-07-04T08:23:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.569+0000 7fcc83e00640 1 -- 172.21.15.39:0/2748000894 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc841a6140 con 0x7fcc84104660 2024-07-04T08:23:54.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.572+0000 7fcc8ff08740 1 Processor -- start 2024-07-04T08:23:54.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc8ff08740 1 -- start start 2024-07-04T08:23:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979edc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979edceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979edd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563979e52f00 con 0x563979b38070 2024-07-04T08:23:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563979e424a0 con 0x563979b17390 2024-07-04T08:23:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563979e464e0 con 0x563979b0f1b0 2024-07-04T08:23:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc83400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979edc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc83400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979edc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51910/0 (socket says 172.21.15.39:51910) 2024-07-04T08:23:54.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.573+0000 7fcc83400640 1 -- 172.21.15.39:0/1600341298 learned_addr learned my addr 172.21.15.39:0/1600341298 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:54.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.574+0000 7fcc82a00640 1 --2- 172.21.15.39:0/1600341298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979edceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.574+0000 7fcc7b400640 1 -- 172.21.15.39:0/2748000894 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcc70004ba0 con 0x7fcc84104660 2024-07-04T08:23:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.574+0000 7fcc83400640 1 -- 172.21.15.39:0/1600341298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 msgr2=0x563979edceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.574+0000 7fcc83400640 1 --2- 172.21.15.39:0/1600341298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979edceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.574+0000 7fcc83400640 1 -- 172.21.15.39:0/1600341298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 msgr2=0x563979edd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.574+0000 7fcc83400640 1 --2- 172.21.15.39:0/1600341298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979edd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.574+0000 7fcc83400640 1 -- 172.21.15.39:0/1600341298 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563979dda670 con 0x563979b38070 2024-07-04T08:23:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.575+0000 7fcc83400640 1 --2- 172.21.15.39:0/1600341298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979edc970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02acb0 tx=0x7fcc7c030420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da5d5406e100635 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.575+0000 7fcc6fe00640 1 -- 172.21.15.39:0/1600341298 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c041030 con 0x563979b38070 2024-07-04T08:23:54.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.575+0000 7fcc6fe00640 1 -- 172.21.15.39:0/1600341298 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc7c003e20 con 0x563979b38070 2024-07-04T08:23:54.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.575+0000 7fcc6fe00640 1 -- 172.21.15.39:0/1600341298 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c036040 con 0x563979b38070 2024-07-04T08:23:54.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.575+0000 7fcc7b400640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc680b2960 0x7fcc680b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc7b400640 1 -- 172.21.15.39:0/2748000894 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7fcc700e2900 con 0x7fcc84104660 2024-07-04T08:23:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc82a00640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc680b2960 0x7fcc680b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc8ff08740 1 -- 172.21.15.39:0/1600341298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 msgr2=0x563979edc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/1600341298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979edc970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02acb0 tx=0x7fcc7c030420 comp rx=0 tx=0).stop 2024-07-04T08:23:54.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc8ff08740 1 -- 172.21.15.39:0/1600341298 shutdown_connections 2024-07-04T08:23:54.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/1600341298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979edd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/1600341298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979edceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/1600341298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979edc970 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc8ff08740 1 -- 172.21.15.39:0/1600341298 >> 172.21.15.39:0/1600341298 conn(0x563979b2c460 msgr2=0x563979eda860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:54.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.576+0000 7fcc8ff08740 1 -- 172.21.15.39:0/1600341298 shutdown_connections 2024-07-04T08:23:54.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.577+0000 7fcc8ff08740 1 -- 172.21.15.39:0/1600341298 wait complete. 2024-07-04T08:23:54.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.577+0000 7fcc82a00640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc680b2960 0x7fcc680b4e70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x563979edb5f0 tx=0x7fcc74009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:54.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.577+0000 7fcc8ff08740 1 -- start start 2024-07-04T08:23:54.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.578+0000 7fcc8ff08740 1 Processor -- start 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.578+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979ee8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.578+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979ee8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.578+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979ef43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.578+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563979ca6700 con 0x563979b38070 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.578+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563979de9bd0 con 0x563979b17390 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.578+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563979eeb0f0 con 0x563979b0f1b0 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.578+0000 7fcc8ff08740 10 client.0 _ll_register_callbacks cb 0x563979e3b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979ee8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979ee8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979ee8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59678/0 (socket says 172.21.15.39:59678) 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc83400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979ef43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82a00640 1 -- 172.21.15.39:0/2266586595 learned_addr learned my addr 172.21.15.39:0/2266586595 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82000640 1 -- 172.21.15.39:0/2266586595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 msgr2=0x563979ee8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82000640 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979ee8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82000640 1 -- 172.21.15.39:0/2266586595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 msgr2=0x563979ef43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82000640 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979ef43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82000640 1 -- 172.21.15.39:0/2266586595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563979ba3650 con 0x563979b17390 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.579+0000 7fcc82a00640 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979ee8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.580+0000 7fcc82000640 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979ee8af0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcc700dfc90 tx=0x7fcc700a9670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c4de36949e37a27 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.580+0000 7fcc6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.580+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2266586595 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7001c030 con 0x563979b17390 2024-07-04T08:23:54.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.580+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2266586595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc7c003e20 con 0x563979b17390 2024-07-04T08:23:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.580+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2266586595 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7fcc700dff10 con 0x563979b17390 2024-07-04T08:23:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.580+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2266586595 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc700e0110 con 0x563979b17390 2024-07-04T08:23:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.580+0000 7fcc8ff08740 10 client.27554 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.580+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2266586595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563979ef7200 con 0x563979b17390 2024-07-04T08:23:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.580+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2266586595 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc7c003e20 con 0x563979b17390 2024-07-04T08:23:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.581+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2266586595 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcc700a8a80 con 0x563979b17390 2024-07-04T08:23:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.581+0000 7fcc8ff08740 10 client.27554 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.581+0000 7fcc8ff08740 10 client.27554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:23:54.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.581+0000 7fcc8ff08740 20 client.27554 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:54.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.581+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563979af69f0 0x563979efa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.581+0000 7fcc8ff08740 4 client.27554 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:23:54.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.581+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2266586595 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563979e54e50 con 0x563979af69f0 2024-07-04T08:23:54.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.581+0000 7fcc82a00640 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563979af69f0 0x563979efa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.582+0000 7fcc82a00640 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563979af69f0 0x563979efa670 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aba9f8f7998b5b51 server_cookie=550d22faceb3eb65 in_seq=0 out_seq=0 2024-07-04T08:23:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.582+0000 7fcc6cc00640 10 client.27554 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:23:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2266586595 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563979e54e50 con 0x563979af69f0 2024-07-04T08:23:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc6cc00640 10 client.27554 handle_command_reply: tid=0 2024-07-04T08:23:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 client.27554 shutdown 2024-07-04T08:23:54.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2266586595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 msgr2=0x563979ee8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979ee8af0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcc700dfc90 tx=0x7fcc700a9670 comp rx=0 tx=0).stop 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2266586595 shutdown_connections 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563979af69f0 0x563979efa670 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b38070 0x563979ef43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979ee8af0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2266586595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b0f1b0 0x563979ee8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc6cc00640 0 client.27554 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2266586595 >> 172.21.15.39:0/2266586595 conn(0x563979b2c460 msgr2=0x563979edfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2266586595 shutdown_connections 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2266586595 wait complete. 2024-07-04T08:23:54.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.583+0000 7fcc8ff08740 20 client.27554 trim_cache size 0 max 16384 2024-07-04T08:23:54.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.599+0000 7fcc8ff08740 1 Processor -- start 2024-07-04T08:23:54.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.599+0000 7fcc8ff08740 1 -- start start 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.599+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 0x563979ede470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.599+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979eea250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.599+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b38070 0x563979ef41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.599+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841b0010 con 0x563979b0f1b0 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.599+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841b1b50 con 0x563979b17390 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.599+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841b06e0 con 0x563979b38070 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc82000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979eea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc82000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979eea250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:49174/0 (socket says 172.21.15.39:49174) 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc82000640 1 -- 172.21.15.39:0/185241400 learned_addr learned my addr 172.21.15.39:0/185241400 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc82a00640 1 --2- 172.21.15.39:0/185241400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 0x563979ede470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc83400640 1 --2- 172.21.15.39:0/185241400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b38070 0x563979ef41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc82000640 1 -- 172.21.15.39:0/185241400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b38070 msgr2=0x563979ef41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:54.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc82000640 1 --2- 172.21.15.39:0/185241400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b38070 0x563979ef41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc82000640 1 -- 172.21.15.39:0/185241400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 msgr2=0x563979ede470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:54.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc82000640 1 --2- 172.21.15.39:0/185241400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 0x563979ede470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.600+0000 7fcc82000640 1 -- 172.21.15.39:0/185241400 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563979edf0c0 con 0x563979b17390 2024-07-04T08:23:54.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.601+0000 7fcc82000640 1 --2- 172.21.15.39:0/185241400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979eea250 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcc700a9520 tx=0x7fcc700057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307b1f038476768a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:54.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.601+0000 7fcc6f400640 1 -- 172.21.15.39:0/185241400 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc700b21f0 con 0x563979b17390 2024-07-04T08:23:54.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.601+0000 7fcc6f400640 1 -- 172.21.15.39:0/185241400 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc700a9f20 con 0x563979b17390 2024-07-04T08:23:54.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.601+0000 7fcc6f400640 1 -- 172.21.15.39:0/185241400 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc700dff10 con 0x563979b17390 2024-07-04T08:23:54.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 -- 172.21.15.39:0/185241400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 msgr2=0x563979eea250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:54.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/185241400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979eea250 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcc700a9520 tx=0x7fcc700057e0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 -- 172.21.15.39:0/185241400 shutdown_connections 2024-07-04T08:23:54.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/185241400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b38070 0x563979ef41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/185241400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b17390 0x563979eea250 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/185241400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 0x563979ede470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 -- 172.21.15.39:0/185241400 >> 172.21.15.39:0/185241400 conn(0x563979b2c460 msgr2=0x563979ed8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:54.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 -- 172.21.15.39:0/185241400 shutdown_connections 2024-07-04T08:23:54.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 -- 172.21.15.39:0/185241400 wait complete. 2024-07-04T08:23:54.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.602+0000 7fcc8ff08740 1 -- start start 2024-07-04T08:23:54.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.603+0000 7fcc8ff08740 1 Processor -- start 2024-07-04T08:23:54.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.603+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 0x563979ef08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.603+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b17390 0x563979eebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.603+0000 7fcc8ff08740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b38070 0x563979eec2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.603+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841b1480 con 0x563979b0f1b0 2024-07-04T08:23:54.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.603+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841b0db0 con 0x563979b38070 2024-07-04T08:23:54.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.603+0000 7fcc8ff08740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841adff0 con 0x563979b17390 2024-07-04T08:23:54.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc8ff08740 10 client.0 _ll_register_callbacks cb 0x7fcc700052f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b17390 0x563979eebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 0x563979ef08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b17390 0x563979eebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59706/0 (socket says 172.21.15.39:59706) 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82000640 1 -- 172.21.15.39:0/2522876268 learned_addr learned my addr 172.21.15.39:0/2522876268 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc83400640 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b38070 0x563979eec2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82000640 1 -- 172.21.15.39:0/2522876268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b38070 msgr2=0x563979eec2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82000640 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b38070 0x563979eec2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82000640 1 -- 172.21.15.39:0/2522876268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 msgr2=0x563979ef08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82000640 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 0x563979ef08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82000640 1 -- 172.21.15.39:0/2522876268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563979eec8f0 con 0x563979b17390 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc83400640 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b38070 0x563979eec2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.604+0000 7fcc82a00640 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 0x563979ef08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:54.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.605+0000 7fcc82000640 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b17390 0x563979eebda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcc700a57d0 tx=0x7fcc700053f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb5297ea699fb69f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.605+0000 7fcc6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:23:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.605+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2522876268 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc70087020 con 0x563979b17390 2024-07-04T08:23:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.605+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2522876268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc700a9f20 con 0x563979b17390 2024-07-04T08:23:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.605+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2522876268 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7fcc700b23d0 con 0x563979b17390 2024-07-04T08:23:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.605+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2522876268 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc700b21f0 con 0x563979b17390 2024-07-04T08:23:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.605+0000 7fcc8ff08740 10 client.27487 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.605+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2522876268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563979f62ba0 con 0x563979b17390 2024-07-04T08:23:54.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.606+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2522876268 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc700a9f20 con 0x563979b17390 2024-07-04T08:23:54.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.606+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2522876268 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcc700dff10 con 0x563979b17390 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.606+0000 7fcc8ff08740 10 client.27487 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.606+0000 7fcc8ff08740 10 client.27487 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.606+0000 7fcc8ff08740 20 client.27487 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.606+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563979acbda0 0x563979f65080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.606+0000 7fcc8ff08740 4 client.27487 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.606+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2522876268 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563979e508f0 con 0x563979acbda0 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.606+0000 7fcc82a00640 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563979acbda0 0x563979f65080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.607+0000 7fcc82a00640 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563979acbda0 0x563979f65080 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63b3887e30bf9cef server_cookie=8ec38a50544f8ac1 in_seq=0 out_seq=0 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.607+0000 7fcc6cc00640 10 client.27487 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.607+0000 7fcc6cc00640 1 -- 172.21.15.39:0/2522876268 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563979e508f0 con 0x563979acbda0 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.607+0000 7fcc6cc00640 10 client.27487 handle_command_reply: tid=0 2024-07-04T08:23:54.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.607+0000 7fcc8ff08740 1 client.27487 shutdown 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2522876268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b17390 msgr2=0x563979eebda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b17390 0x563979eebda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcc700a57d0 tx=0x7fcc700053f0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2522876268 shutdown_connections 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563979acbda0 0x563979f65080 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563979b38070 0x563979eec2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563979b17390 0x563979eebda0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 --2- 172.21.15.39:0/2522876268 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563979b0f1b0 0x563979ef08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc6cc00640 0 client.27487 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2522876268 >> 172.21.15.39:0/2522876268 conn(0x563979b2c460 msgr2=0x563979ed8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2522876268 shutdown_connections 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 1 -- 172.21.15.39:0/2522876268 wait complete. 2024-07-04T08:23:54.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.608+0000 7fcc8ff08740 20 client.27487 trim_cache size 0 max 16384 2024-07-04T08:23:54.604 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:23:54.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 -- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc680b2960 msgr2=0x7fcc680b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc680b2960 0x7fcc680b4e70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x563979edb5f0 tx=0x7fcc74009260 comp rx=0 tx=0).stop 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 -- 172.21.15.39:0/2748000894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc84104660 msgr2=0x7fcc84199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc84104660 0x7fcc84199d20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcc84069fe0 tx=0x7fcc7000ee50 comp rx=0 tx=0).stop 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 -- 172.21.15.39:0/2748000894 shutdown_connections 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc680b2960 0x7fcc680b4e70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc8410d170 0x7fcc8419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc84104660 0x7fcc84199d20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 --2- 172.21.15.39:0/2748000894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc84103d10 0x7fcc84199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 -- 172.21.15.39:0/2748000894 >> 172.21.15.39:0/2748000894 conn(0x7fcc840fd810 msgr2=0x7fcc84106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 -- 172.21.15.39:0/2748000894 shutdown_connections 2024-07-04T08:23:54.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:54.609+0000 7fcc83e00640 1 -- 172.21.15.39:0/2748000894 wait complete. 2024-07-04T08:23:55.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:55 smithi039 ceph-mon[32062]: pgmap v745: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-07-04T08:23:55.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:55 smithi050 ceph-mon[36577]: pgmap v745: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-07-04T08:23:55.458 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:23:55.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:55 smithi133 ceph-mon[37050]: pgmap v745: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-07-04T08:23:55.954 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:23:57.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:57 smithi039 ceph-mon[32062]: pgmap v746: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-07-04T08:23:57.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:57 smithi050 ceph-mon[36577]: pgmap v746: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-07-04T08:23:57.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:57 smithi133 ceph-mon[37050]: pgmap v746: 129 pgs: 129 active+clean; 11 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-07-04T08:23:58.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.359+0000 7f37aa800640 1 -- 172.21.15.39:0/1788280489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410ee50 msgr2=0x7f37a410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:58.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.359+0000 7f37aa800640 1 --2- 172.21.15.39:0/1788280489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410ee50 0x7f37a410f260 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3790009900 tx=0x7f379002a400 comp rx=0 tx=0).stop 2024-07-04T08:23:58.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.359+0000 7f37aa800640 1 -- 172.21.15.39:0/1788280489 shutdown_connections 2024-07-04T08:23:58.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.359+0000 7f37aa800640 1 --2- 172.21.15.39:0/1788280489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37a410f7a0 0x7f37a4115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.359+0000 7f37aa800640 1 --2- 172.21.15.39:0/1788280489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410ee50 0x7f37a410f260 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.359+0000 7f37aa800640 1 --2- 172.21.15.39:0/1788280489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37a4109510 0x7f37a4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.359+0000 7f37aa800640 1 -- 172.21.15.39:0/1788280489 >> 172.21.15.39:0/1788280489 conn(0x7f37a4078b10 msgr2=0x7f37a4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.359+0000 7f37aa800640 1 -- 172.21.15.39:0/1788280489 shutdown_connections 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.360+0000 7f37aa800640 1 -- 172.21.15.39:0/1788280489 wait complete. 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.361+0000 7f37aa800640 1 Processor -- start 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.361+0000 7f37aa800640 1 -- start start 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37aa800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37a4109510 0x7f37a419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37aa800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37a410ee50 0x7f37a41a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37aa800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410f7a0 0x7f37a41a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37aa800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4119410 con 0x7f37a4109510 2024-07-04T08:23:58.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37aa800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4119290 con 0x7f37a410f7a0 2024-07-04T08:23:58.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37aa800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4119590 con 0x7f37a410ee50 2024-07-04T08:23:58.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37a9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410f7a0 0x7f37a41a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37a9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37a4109510 0x7f37a419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37a9e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410f7a0 0x7f37a41a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:49186/0 (socket says 172.21.15.39:49186) 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37a8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37a410ee50 0x7f37a41a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.362+0000 7f37a9e00640 1 -- 172.21.15.39:0/3767283383 learned_addr learned my addr 172.21.15.39:0/3767283383 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.363+0000 7f37a9e00640 1 -- 172.21.15.39:0/3767283383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37a410ee50 msgr2=0x7f37a41a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.363+0000 7f37a9e00640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37a410ee50 0x7f37a41a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.363+0000 7f37a9e00640 1 -- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37a4109510 msgr2=0x7f37a419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.363+0000 7f37a9e00640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37a4109510 0x7f37a419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.363+0000 7f37a9e00640 1 -- 172.21.15.39:0/3767283383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a41a5d70 con 0x7f37a410f7a0 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.363+0000 7f37a9400640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37a4109510 0x7f37a419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.363+0000 7f37a9e00640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410f7a0 0x7f37a41a5640 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f379c00eb10 tx=0x7f379c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:58.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.363+0000 7f37a2000640 1 -- 172.21.15.39:0/3767283383 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c004b20 con 0x7f37a410f7a0 2024-07-04T08:23:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.364+0000 7f37a2000640 1 -- 172.21.15.39:0/3767283383 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f379c004cc0 con 0x7f37a410f7a0 2024-07-04T08:23:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.364+0000 7f37a2000640 1 -- 172.21.15.39:0/3767283383 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c010670 con 0x7f37a410f7a0 2024-07-04T08:23:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.364+0000 7f37aa800640 1 -- 172.21.15.39:0/3767283383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a41a6060 con 0x7f37a410f7a0 2024-07-04T08:23:58.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.364+0000 7f37aa800640 1 -- 172.21.15.39:0/3767283383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a41ac7c0 con 0x7f37a410f7a0 2024-07-04T08:23:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.367+0000 7f37b2335740 1 Processor -- start 2024-07-04T08:23:58.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.368+0000 7f37a2000640 1 -- 172.21.15.39:0/3767283383 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f379c005020 con 0x7f37a410f7a0 2024-07-04T08:23:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.368+0000 7f37b2335740 1 -- start start 2024-07-04T08:23:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.368+0000 7f37b2335740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.368+0000 7f37b2335740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531eddf1b0 0x56531f1aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.368+0000 7f37b2335740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531ede7390 0x56531f1ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.368+0000 7f37b2335740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56531f122f00 con 0x56531ede7390 2024-07-04T08:23:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.368+0000 7f37b2335740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56531f1124a0 con 0x56531eddf1b0 2024-07-04T08:23:58.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.368+0000 7f37b2335740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56531f1164e0 con 0x56531ee08070 2024-07-04T08:23:58.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59744/0 (socket says 172.21.15.39:59744) 2024-07-04T08:23:58.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531eddf1b0 0x56531f1aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a8a00640 1 -- 172.21.15.39:0/1190589216 learned_addr learned my addr 172.21.15.39:0/1190589216 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:58.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a8a00640 1 --2- 172.21.15.39:0/1190589216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531ede7390 0x56531f1ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a9400640 1 -- 172.21.15.39:0/1190589216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 msgr2=0x56531f1ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:58.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a9400640 1 --2- 172.21.15.39:0/1190589216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a9400640 1 -- 172.21.15.39:0/1190589216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531ede7390 msgr2=0x56531f1ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:58.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a9400640 1 --2- 172.21.15.39:0/1190589216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531ede7390 0x56531f1ad3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.369+0000 7f37a9400640 1 -- 172.21.15.39:0/1190589216 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531f0aa670 con 0x56531eddf1b0 2024-07-04T08:23:58.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.370+0000 7f37a9400640 1 --2- 172.21.15.39:0/1190589216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531eddf1b0 0x56531f1aceb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3798009fd0 tx=0x7f37980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb91f2d607771529 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:58.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.370+0000 7f3796a00640 1 -- 172.21.15.39:0/1190589216 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798004030 con 0x56531eddf1b0 2024-07-04T08:23:58.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.370+0000 7f3796a00640 1 -- 172.21.15.39:0/1190589216 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37980054d0 con 0x56531eddf1b0 2024-07-04T08:23:58.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.370+0000 7f3796a00640 1 -- 172.21.15.39:0/1190589216 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798004b80 con 0x56531eddf1b0 2024-07-04T08:23:58.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37a2000640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f378c0b29e0 0x7f378c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37a2000640 1 -- 172.21.15.39:0/3767283383 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f379c0e2ea0 con 0x7f37a410f7a0 2024-07-04T08:23:58.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37a8a00640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f378c0b29e0 0x7f378c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37b2335740 1 -- 172.21.15.39:0/1190589216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531eddf1b0 msgr2=0x56531f1aceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37b2335740 1 --2- 172.21.15.39:0/1190589216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531eddf1b0 0x56531f1aceb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3798009fd0 tx=0x7f37980118d0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37b2335740 1 -- 172.21.15.39:0/1190589216 shutdown_connections 2024-07-04T08:23:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37b2335740 1 --2- 172.21.15.39:0/1190589216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531ede7390 0x56531f1ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37b2335740 1 --2- 172.21.15.39:0/1190589216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531eddf1b0 0x56531f1aceb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37b2335740 1 --2- 172.21.15.39:0/1190589216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37b2335740 1 -- 172.21.15.39:0/1190589216 >> 172.21.15.39:0/1190589216 conn(0x56531edfc460 msgr2=0x56531f1aa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.371+0000 7f37b2335740 1 -- 172.21.15.39:0/1190589216 shutdown_connections 2024-07-04T08:23:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.372+0000 7f37b2335740 1 -- 172.21.15.39:0/1190589216 wait complete. 2024-07-04T08:23:58.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.372+0000 7f37a8a00640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f378c0b29e0 0x7f378c0b4ef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f37900096c0 tx=0x7f3790031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:58.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.372+0000 7f37b2335740 1 -- start start 2024-07-04T08:23:58.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.372+0000 7f37b2335740 1 Processor -- start 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.373+0000 7f37b2335740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.373+0000 7f37b2335740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1b8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.373+0000 7f37b2335740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.373+0000 7f37b2335740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56531ef76700 con 0x56531eddf1b0 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.373+0000 7f37b2335740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56531f0b9bd0 con 0x56531ede7390 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.373+0000 7f37b2335740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56531f1bb150 con 0x56531ee08070 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.373+0000 7f37b2335740 10 client.0 _ll_register_callbacks cb 0x56531f10b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.373+0000 7f37a9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.373+0000 7f37a9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51976/0 (socket says 172.21.15.39:51976) 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a9400640 1 -- 172.21.15.39:0/1912969945 learned_addr learned my addr 172.21.15.39:0/1912969945 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a9e00640 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a8a00640 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a9400640 1 -- 172.21.15.39:0/1912969945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 msgr2=0x56531f1c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:58.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a9400640 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a9400640 1 -- 172.21.15.39:0/1912969945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 msgr2=0x56531f1b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:58.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a9400640 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a9400640 1 -- 172.21.15.39:0/1912969945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531ee73650 con 0x56531eddf1b0 2024-07-04T08:23:58.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a9e00640 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:58.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.374+0000 7f37a8a00640 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:58.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.375+0000 7f37a9400640 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b8420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3798009230 tx=0x7f379800fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24890d013145ee10 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:58.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.375+0000 7f378b400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:23:58.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.375+0000 7f378b400640 1 -- 172.21.15.39:0/1912969945 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798028070 con 0x56531eddf1b0 2024-07-04T08:23:58.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.375+0000 7f378b400640 1 -- 172.21.15.39:0/1912969945 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f3798019660 con 0x56531eddf1b0 2024-07-04T08:23:58.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.375+0000 7f37b2335740 1 -- 172.21.15.39:0/1912969945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37980054d0 con 0x56531eddf1b0 2024-07-04T08:23:58.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.375+0000 7f378b400640 1 -- 172.21.15.39:0/1912969945 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3798034050 con 0x56531eddf1b0 2024-07-04T08:23:58.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.375+0000 7f37b2335740 10 client.17439 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:58.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.375+0000 7f37b2335740 1 -- 172.21.15.39:0/1912969945 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56531f1c70b0 con 0x56531eddf1b0 2024-07-04T08:23:58.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.376+0000 7f378b400640 1 -- 172.21.15.39:0/1912969945 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37980054d0 con 0x56531eddf1b0 2024-07-04T08:23:58.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.376+0000 7f378b400640 1 -- 172.21.15.39:0/1912969945 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3798024440 con 0x56531eddf1b0 2024-07-04T08:23:58.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.376+0000 7f37b2335740 10 client.17439 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.376+0000 7f37b2335740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.376+0000 7f37b2335740 20 client.17439 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.376+0000 7f37b2335740 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56531edc69f0 0x56531f1ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.376+0000 7f37b2335740 4 client.17439 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.376+0000 7f37b2335740 1 -- 172.21.15.39:0/1912969945 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56531f124e50 con 0x56531edc69f0 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.377+0000 7f37a8a00640 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56531edc69f0 0x56531f1ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.377+0000 7f37a8a00640 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56531edc69f0 0x56531f1ca470 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c4f0422fdf66ffd server_cookie=e97c9a8ac11d4d2b in_seq=0 out_seq=0 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.377+0000 7f378b400640 10 client.17439 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.378+0000 7f378b400640 1 -- 172.21.15.39:0/1912969945 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56531f124e50 con 0x56531edc69f0 2024-07-04T08:23:58.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.378+0000 7f378b400640 10 client.17439 handle_command_reply: tid=0 2024-07-04T08:23:58.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.378+0000 7f37b2335740 1 client.17439 shutdown 2024-07-04T08:23:58.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.378+0000 7f37b2335740 1 -- 172.21.15.39:0/1912969945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 msgr2=0x56531f1b8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:58.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.378+0000 7f37b2335740 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b8420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3798009230 tx=0x7f379800fca0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f37b2335740 1 -- 172.21.15.39:0/1912969945 shutdown_connections 2024-07-04T08:23:58.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f37b2335740 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56531edc69f0 0x56531f1ca470 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f37b2335740 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f37b2335740 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f37b2335740 1 --2- 172.21.15.39:0/1912969945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b8420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f37b2335740 1 -- 172.21.15.39:0/1912969945 >> 172.21.15.39:0/1912969945 conn(0x56531edfc460 msgr2=0x56531f1afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:58.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f378b400640 0 client.17439 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:58.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f37b2335740 1 -- 172.21.15.39:0/1912969945 shutdown_connections 2024-07-04T08:23:58.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f37b2335740 1 -- 172.21.15.39:0/1912969945 wait complete. 2024-07-04T08:23:58.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.379+0000 7f37b2335740 20 client.17439 trim_cache size 0 max 16384 2024-07-04T08:23:58.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37b2335740 1 Processor -- start 2024-07-04T08:23:58.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37b2335740 1 -- start start 2024-07-04T08:23:58.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37b2335740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37b2335740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ede7390 0x56531f1ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37b2335740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ee08070 0x56531f1c4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37b2335740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41b6700 con 0x56531eddf1b0 2024-07-04T08:23:58.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37b2335740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41b8240 con 0x56531ee08070 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37b2335740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41b6dd0 con 0x56531ede7390 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37a8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ede7390 0x56531f1ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37a8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ede7390 0x56531f1ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59754/0 (socket says 172.21.15.39:59754) 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37a8a00640 1 -- 172.21.15.39:0/2855039680 learned_addr learned my addr 172.21.15.39:0/2855039680 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.395+0000 7f37a9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.396+0000 7f37a9e00640 1 --2- 172.21.15.39:0/2855039680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ee08070 0x56531f1c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.396+0000 7f37a8a00640 1 -- 172.21.15.39:0/2855039680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ee08070 msgr2=0x56531f1c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.396+0000 7f37a8a00640 1 --2- 172.21.15.39:0/2855039680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ee08070 0x56531f1c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.396+0000 7f37a8a00640 1 -- 172.21.15.39:0/2855039680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 msgr2=0x56531f1b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.396+0000 7f37a8a00640 1 --2- 172.21.15.39:0/2855039680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.396+0000 7f37a8a00640 1 -- 172.21.15.39:0/2855039680 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56531f1aec60 con 0x56531ede7390 2024-07-04T08:23:58.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.396+0000 7f37a9400640 1 --2- 172.21.15.39:0/2855039680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.396+0000 7f37a8a00640 1 --2- 172.21.15.39:0/2855039680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ede7390 0x56531f1ae0b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3790055fd0 tx=0x7f3790009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd61df167379b8a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.396+0000 7f3796000640 1 -- 172.21.15.39:0/2855039680 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790035040 con 0x56531ede7390 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.397+0000 7f3796000640 1 -- 172.21.15.39:0/2855039680 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f379005b070 con 0x56531ede7390 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.397+0000 7f3796000640 1 -- 172.21.15.39:0/2855039680 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790004e50 con 0x56531ede7390 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.397+0000 7f37b2335740 1 -- 172.21.15.39:0/2855039680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ede7390 msgr2=0x56531f1ae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.397+0000 7f37b2335740 1 --2- 172.21.15.39:0/2855039680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ede7390 0x56531f1ae0b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3790055fd0 tx=0x7f3790009260 comp rx=0 tx=0).stop 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.398+0000 7f37b2335740 1 -- 172.21.15.39:0/2855039680 shutdown_connections 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.398+0000 7f37b2335740 1 --2- 172.21.15.39:0/2855039680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ee08070 0x56531f1c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.398+0000 7f37b2335740 1 --2- 172.21.15.39:0/2855039680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ede7390 0x56531f1ae0b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.398+0000 7f37b2335740 1 --2- 172.21.15.39:0/2855039680 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.398+0000 7f37b2335740 1 -- 172.21.15.39:0/2855039680 >> 172.21.15.39:0/2855039680 conn(0x56531edfc460 msgr2=0x56531f1ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:58.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.398+0000 7f37b2335740 1 -- 172.21.15.39:0/2855039680 shutdown_connections 2024-07-04T08:23:58.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.398+0000 7f37b2335740 1 -- 172.21.15.39:0/2855039680 wait complete. 2024-07-04T08:23:58.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.398+0000 7f37b2335740 1 -- start start 2024-07-04T08:23:58.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37b2335740 1 Processor -- start 2024-07-04T08:23:58.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37b2335740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37b2335740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1bb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37b2335740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37b2335740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41b7b70 con 0x56531eddf1b0 2024-07-04T08:23:58.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37b2335740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41b74a0 con 0x56531ede7390 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37b2335740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f37a41b46e0 con 0x56531ee08070 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37b2335740 10 client.0 _ll_register_callbacks cb 0x7f3798005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37a9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.399+0000 7f37a9400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51998/0 (socket says 172.21.15.39:51998) 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a9400640 1 -- 172.21.15.39:0/983963882 learned_addr learned my addr 172.21.15.39:0/983963882 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a8a00640 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a9e00640 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a9400640 1 -- 172.21.15.39:0/983963882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 msgr2=0x56531f1bbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a9400640 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1bbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a9400640 1 -- 172.21.15.39:0/983963882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 msgr2=0x56531f1bb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a9400640 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1bb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a9400640 1 -- 172.21.15.39:0/983963882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56531f230b10 con 0x56531eddf1b0 2024-07-04T08:23:58.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a8a00640 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1bb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:23:58.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f37a9400640 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1c0cd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3798011dd0 tx=0x7f379800f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9076ebd49d4ccdb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:23:58.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.400+0000 7f378b400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:23:58.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.401+0000 7f378b400640 1 -- 172.21.15.39:0/983963882 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798031020 con 0x56531eddf1b0 2024-07-04T08:23:58.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.401+0000 7f378b400640 1 -- 172.21.15.39:0/983963882 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9424+0+0 (secure 0 0 0) 0x7f3798024a80 con 0x56531eddf1b0 2024-07-04T08:23:58.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.401+0000 7f37b2335740 1 -- 172.21.15.39:0/983963882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f379005b070 con 0x56531eddf1b0 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.401+0000 7f378b400640 1 -- 172.21.15.39:0/983963882 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3798028040 con 0x56531eddf1b0 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.401+0000 7f37b2335740 10 client.17445 fetch_fsmap learned FSMap version 30 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.401+0000 7f37b2335740 1 -- 172.21.15.39:0/983963882 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56531f231c60 con 0x56531eddf1b0 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.401+0000 7f378b400640 1 -- 172.21.15.39:0/983963882 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f379005b070 con 0x56531eddf1b0 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.402+0000 7f378b400640 1 -- 172.21.15.39:0/983963882 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3798019660 con 0x56531eddf1b0 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.402+0000 7f37b2335740 10 client.17445 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.402+0000 7f37b2335740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.402+0000 7f37b2335740 20 client.17445 populate_metadata read hostname 'smithi039' 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.402+0000 7f37b2335740 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56531ed9bda0 0x56531f234410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.402+0000 7f37b2335740 4 client.17445 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:23:58.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.402+0000 7f37b2335740 1 -- 172.21.15.39:0/983963882 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56531f1208f0 con 0x56531ed9bda0 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.402+0000 7f37a8a00640 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56531ed9bda0 0x56531f234410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.403+0000 7f37a8a00640 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56531ed9bda0 0x56531f234410 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3daef7da62887 server_cookie=8fd4436931dc966d in_seq=0 out_seq=0 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.403+0000 7f378b400640 10 client.17445 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.403+0000 7f378b400640 1 -- 172.21.15.39:0/983963882 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56531f1208f0 con 0x56531ed9bda0 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.403+0000 7f378b400640 10 client.17445 handle_command_reply: tid=0 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.403+0000 7f37b2335740 1 client.17445 shutdown 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.403+0000 7f37b2335740 1 -- 172.21.15.39:0/983963882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 msgr2=0x56531f1c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.403+0000 7f37b2335740 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1c0cd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3798011dd0 tx=0x7f379800f980 comp rx=0 tx=0).stop 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f37b2335740 1 -- 172.21.15.39:0/983963882 shutdown_connections 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f37b2335740 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56531ed9bda0 0x56531f234410 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f37b2335740 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56531ee08070 0x56531f1bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f37b2335740 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56531ede7390 0x56531f1bb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f37b2335740 1 --2- 172.21.15.39:0/983963882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56531eddf1b0 0x56531f1c0cd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f37b2335740 1 -- 172.21.15.39:0/983963882 >> 172.21.15.39:0/983963882 conn(0x56531edfc460 msgr2=0x56531f1aa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:58.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f378b400640 0 client.17445 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:23:58.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f37b2335740 1 -- 172.21.15.39:0/983963882 shutdown_connections 2024-07-04T08:23:58.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f37b2335740 1 -- 172.21.15.39:0/983963882 wait complete. 2024-07-04T08:23:58.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.404+0000 7f37b2335740 20 client.17445 trim_cache size 0 max 16384 2024-07-04T08:23:58.399 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:23:58.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 -- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f378c0b29e0 msgr2=0x7f378c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:58.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f378c0b29e0 0x7f378c0b4ef0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f37900096c0 tx=0x7f3790031400 comp rx=0 tx=0).stop 2024-07-04T08:23:58.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 -- 172.21.15.39:0/3767283383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410f7a0 msgr2=0x7f37a41a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:23:58.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410f7a0 0x7f37a41a5640 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f379c00eb10 tx=0x7f379c00c2d0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 -- 172.21.15.39:0/3767283383 shutdown_connections 2024-07-04T08:23:58.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f378c0b29e0 0x7f378c0b4ef0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a410f7a0 0x7f37a41a5640 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37a410ee50 0x7f37a41a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 --2- 172.21.15.39:0/3767283383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37a4109510 0x7f37a419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:23:58.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 -- 172.21.15.39:0/3767283383 >> 172.21.15.39:0/3767283383 conn(0x7f37a4078b10 msgr2=0x7f37a410d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:23:58.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 -- 172.21.15.39:0/3767283383 shutdown_connections 2024-07-04T08:23:58.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:23:58.405+0000 7f37aa800640 1 -- 172.21.15.39:0/3767283383 wait complete. 2024-07-04T08:23:59.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:23:59 smithi050 ceph-mon[36577]: pgmap v747: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-07-04T08:23:59.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:23:59 smithi133 ceph-mon[37050]: pgmap v747: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-07-04T08:23:59.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:23:59 smithi039 ceph-mon[32062]: pgmap v747: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-07-04T08:24:00.257 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:24:00.257 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:24:00.766 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:01.171 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:01 smithi039 ceph-mon[32062]: pgmap v748: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-07-04T08:24:01.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:01 smithi133 ceph-mon[37050]: pgmap v748: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-07-04T08:24:01.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:01 smithi050 ceph-mon[36577]: pgmap v748: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-07-04T08:24:03.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.201+0000 7f5484a00640 1 -- 172.21.15.39:0/3713105664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010f7a0 msgr2=0x7f5480115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:03.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.201+0000 7f5484a00640 1 --2- 172.21.15.39:0/3713105664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010f7a0 0x7f5480115fd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5478009900 tx=0x7f547802a410 comp rx=0 tx=0).stop 2024-07-04T08:24:03.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.201+0000 7f5484a00640 1 -- 172.21.15.39:0/3713105664 shutdown_connections 2024-07-04T08:24:03.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.201+0000 7f5484a00640 1 --2- 172.21.15.39:0/3713105664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010f7a0 0x7f5480115fd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.201+0000 7f5484a00640 1 --2- 172.21.15.39:0/3713105664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548010ee50 0x7f548010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.201+0000 7f5484a00640 1 --2- 172.21.15.39:0/3713105664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5480109510 0x7f5480109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.201+0000 7f5484a00640 1 -- 172.21.15.39:0/3713105664 >> 172.21.15.39:0/3713105664 conn(0x7f5480078b10 msgr2=0x7f5480107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.201+0000 7f5484a00640 1 -- 172.21.15.39:0/3713105664 shutdown_connections 2024-07-04T08:24:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.201+0000 7f5484a00640 1 -- 172.21.15.39:0/3713105664 wait complete. 2024-07-04T08:24:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.203+0000 7f5484a00640 1 Processor -- start 2024-07-04T08:24:03.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.203+0000 7f5484a00640 1 -- start start 2024-07-04T08:24:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f5484a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5480109510 0x7f548019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f5484a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010ee50 0x7f54801a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f5484a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f548010f7a0 0x7f54801a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f5484a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f54801194c0 con 0x7f548010ee50 2024-07-04T08:24:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f5484a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5480119340 con 0x7f548010f7a0 2024-07-04T08:24:03.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f5484a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5480119640 con 0x7f5480109510 2024-07-04T08:24:03.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f547de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010ee50 0x7f54801a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f547e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5480109510 0x7f548019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f547e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5480109510 0x7f548019fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:55030/0 (socket says 172.21.15.39:55030) 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f547de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010ee50 0x7f54801a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55082/0 (socket says 172.21.15.39:55082) 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f547e800640 1 -- 172.21.15.39:0/1172698337 learned_addr learned my addr 172.21.15.39:0/1172698337 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.204+0000 7f547f200640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f548010f7a0 0x7f54801a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f547e800640 1 -- 172.21.15.39:0/1172698337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f548010f7a0 msgr2=0x7f54801a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f547e800640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f548010f7a0 0x7f54801a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f547e800640 1 -- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010ee50 msgr2=0x7f54801a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f547e800640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010ee50 0x7f54801a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f547e800640 1 -- 172.21.15.39:0/1172698337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54801a5de0 con 0x7f5480109510 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f547de00640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010ee50 0x7f54801a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:03.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f547e800640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5480109510 0x7f548019fe70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f547000ebc0 tx=0x7f547000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:03.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f5477400640 1 -- 172.21.15.39:0/1172698337 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470010630 con 0x7f5480109510 2024-07-04T08:24:03.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f5484a00640 1 -- 172.21.15.39:0/1172698337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54801a60d0 con 0x7f5480109510 2024-07-04T08:24:03.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f5477400640 1 -- 172.21.15.39:0/1172698337 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5470010c80 con 0x7f5480109510 2024-07-04T08:24:03.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.205+0000 7f5477400640 1 -- 172.21.15.39:0/1172698337 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470018b60 con 0x7f5480109510 2024-07-04T08:24:03.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.206+0000 7f5484a00640 1 -- 172.21.15.39:0/1172698337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54801ac830 con 0x7f5480109510 2024-07-04T08:24:03.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.211+0000 7f548b9d6740 1 Processor -- start 2024-07-04T08:24:03.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.211+0000 7f548b9d6740 1 -- start start 2024-07-04T08:24:03.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.211+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb57350 0x55fb2fefb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.211+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb2e490 0x55fb2fefbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.211+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 0x55fb2fefc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.211+0000 7f548b9d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fb2fe258e0 con 0x55fb2fb57350 2024-07-04T08:24:03.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.211+0000 7f548b9d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fb2fe71ec0 con 0x55fb2fb2e490 2024-07-04T08:24:03.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.211+0000 7f548b9d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fb2fe61460 con 0x55fb2fb36670 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.211+0000 7f5477400640 1 -- 172.21.15.39:0/1172698337 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5470018d00 con 0x7f5480109510 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb57350 0x55fb2fefb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb57350 0x55fb2fefb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55084/0 (socket says 172.21.15.39:55084) 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547f200640 1 -- 172.21.15.39:0/1173819470 learned_addr learned my addr 172.21.15.39:0/1173819470 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 0x55fb2fefc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547e800640 1 --2- 172.21.15.39:0/1173819470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb2e490 0x55fb2fefbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547de00640 1 -- 172.21.15.39:0/1173819470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb2e490 msgr2=0x55fb2fefbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547de00640 1 --2- 172.21.15.39:0/1173819470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb2e490 0x55fb2fefbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547de00640 1 -- 172.21.15.39:0/1173819470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb57350 msgr2=0x55fb2fefb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:03.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547de00640 1 --2- 172.21.15.39:0/1173819470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb57350 0x55fb2fefb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547de00640 1 -- 172.21.15.39:0/1173819470 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2fdf9280 con 0x55fb2fb36670 2024-07-04T08:24:03.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.212+0000 7f547e800640 1 --2- 172.21.15.39:0/1173819470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb2e490 0x55fb2fefbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:24:03.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.213+0000 7f547de00640 1 --2- 172.21.15.39:0/1173819470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 0x55fb2fefc270 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f548006bb00 tx=0x7f54680118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7d9c20af37d79e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:03.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.213+0000 7f546fe00640 1 -- 172.21.15.39:0/1173819470 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54680043d0 con 0x55fb2fb36670 2024-07-04T08:24:03.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.213+0000 7f546fe00640 1 -- 172.21.15.39:0/1173819470 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54680054d0 con 0x55fb2fb36670 2024-07-04T08:24:03.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.213+0000 7f546fe00640 1 -- 172.21.15.39:0/1173819470 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54680035f0 con 0x55fb2fb36670 2024-07-04T08:24:03.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.213+0000 7f5477400640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54640b2960 0x7f54640b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.213+0000 7f5477400640 1 -- 172.21.15.39:0/1172698337 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9512+0+0 (secure 0 0 0) 0x7f54700e2fe0 con 0x7f5480109510 2024-07-04T08:24:03.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.214+0000 7f547f200640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54640b2960 0x7f54640b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.214+0000 7f548b9d6740 1 -- 172.21.15.39:0/1173819470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 msgr2=0x55fb2fefc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:03.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.214+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1173819470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 0x55fb2fefc270 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f548006bb00 tx=0x7f54680118d0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.214+0000 7f548b9d6740 1 -- 172.21.15.39:0/1173819470 shutdown_connections 2024-07-04T08:24:03.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.214+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1173819470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 0x55fb2fefc270 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.214+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1173819470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb2e490 0x55fb2fefbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.214+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1173819470 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb57350 0x55fb2fefb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.214+0000 7f548b9d6740 1 -- 172.21.15.39:0/1173819470 >> 172.21.15.39:0/1173819470 conn(0x55fb2fb4b740 msgr2=0x55fb2fef96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:03.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.214+0000 7f548b9d6740 1 -- 172.21.15.39:0/1173819470 shutdown_connections 2024-07-04T08:24:03.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.215+0000 7f548b9d6740 1 -- 172.21.15.39:0/1173819470 wait complete. 2024-07-04T08:24:03.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.215+0000 7f547f200640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54640b2960 0x7f54640b4e70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f54780096c0 tx=0x7f5478005a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.215+0000 7f548b9d6740 1 -- start start 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.215+0000 7f548b9d6740 1 Processor -- start 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff07370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff13170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f548b9d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55fb2fe08760 con 0x55fb2fb36670 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f548b9d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55fb2fe654a0 con 0x55fb2fb57350 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f548b9d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fb2fcc5890 con 0x55fb2fb2e490 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f548b9d6740 10 client.0 _ll_register_callbacks cb 0x55fb2fe092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:03.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f547de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f547f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff13170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f547de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55096/0 (socket says 172.21.15.39:55096) 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.216+0000 7f547de00640 1 -- 172.21.15.39:0/2828112483 learned_addr learned my addr 172.21.15.39:0/2828112483 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.217+0000 7f547e800640 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff07370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.217+0000 7f547de00640 1 -- 172.21.15.39:0/2828112483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 msgr2=0x55fb2ff07370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.217+0000 7f547de00640 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff07370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.217+0000 7f547de00640 1 -- 172.21.15.39:0/2828112483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 msgr2=0x55fb2ff13170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.217+0000 7f547de00640 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff13170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.217+0000 7f547de00640 1 -- 172.21.15.39:0/2828112483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2fbc2930 con 0x55fb2fb36670 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.217+0000 7f547e800640 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff07370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.217+0000 7f547f200640 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff13170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.217+0000 7f547de00640 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff078b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5468004c70 tx=0x7f5468004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedaab3f90da1176 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:03.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.218+0000 7f546cc00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.218+0000 7f546cc00640 1 -- 172.21.15.39:0/2828112483 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468028030 con 0x55fb2fb36670 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.218+0000 7f548b9d6740 1 -- 172.21.15.39:0/2828112483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54680054d0 con 0x55fb2fb36670 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.218+0000 7f546cc00640 1 -- 172.21.15.39:0/2828112483 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9512+0+0 (secure 0 0 0) 0x7f5468004f20 con 0x55fb2fb36670 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.218+0000 7f546cc00640 1 -- 172.21.15.39:0/2828112483 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5468035050 con 0x55fb2fb36670 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.218+0000 7f548b9d6740 10 client.17457 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.218+0000 7f548b9d6740 1 -- 172.21.15.39:0/2828112483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb2ff15df0 con 0x55fb2fb36670 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.219+0000 7f546cc00640 1 -- 172.21.15.39:0/2828112483 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54680054d0 con 0x55fb2fb36670 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.219+0000 7f546cc00640 1 -- 172.21.15.39:0/2828112483 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f546802f4e0 con 0x55fb2fb36670 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.219+0000 7f548b9d6740 10 client.17457 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.219+0000 7f548b9d6740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.219+0000 7f548b9d6740 20 client.17457 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:03.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.219+0000 7f548b9d6740 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55fb2fb15cd0 0x55fb2ff19150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.219+0000 7f548b9d6740 4 client.17457 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:24:03.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.219+0000 7f548b9d6740 1 -- 172.21.15.39:0/2828112483 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb2fe73e10 con 0x55fb2fb15cd0 2024-07-04T08:24:03.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.220+0000 7f547e800640 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55fb2fb15cd0 0x55fb2ff19150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.220+0000 7f547e800640 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55fb2fb15cd0 0x55fb2ff19150 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece720e0c0a62788 server_cookie=4c61edde3108b11e in_seq=0 out_seq=0 2024-07-04T08:24:03.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.220+0000 7f546cc00640 10 client.17457 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:03.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.221+0000 7f546cc00640 1 -- 172.21.15.39:0/2828112483 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fb2fe73e10 con 0x55fb2fb15cd0 2024-07-04T08:24:03.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.221+0000 7f546cc00640 10 client.17457 handle_command_reply: tid=0 2024-07-04T08:24:03.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.221+0000 7f548b9d6740 1 client.17457 shutdown 2024-07-04T08:24:03.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 -- 172.21.15.39:0/2828112483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 msgr2=0x55fb2ff078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:03.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff078b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5468004c70 tx=0x7f5468004400 comp rx=0 tx=0).stop 2024-07-04T08:24:03.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 -- 172.21.15.39:0/2828112483 shutdown_connections 2024-07-04T08:24:03.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55fb2fb15cd0 0x55fb2ff19150 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff078b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff13170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 --2- 172.21.15.39:0/2828112483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff07370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 -- 172.21.15.39:0/2828112483 >> 172.21.15.39:0/2828112483 conn(0x55fb2fb4b740 msgr2=0x55fb2fefe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:03.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f546cc00640 0 client.17457 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:03.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 -- 172.21.15.39:0/2828112483 shutdown_connections 2024-07-04T08:24:03.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 1 -- 172.21.15.39:0/2828112483 wait complete. 2024-07-04T08:24:03.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.222+0000 7f548b9d6740 20 client.17457 trim_cache size 0 max 16384 2024-07-04T08:24:03.235 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.243+0000 7f548b9d6740 1 Processor -- start 2024-07-04T08:24:03.236 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:03 smithi039 ceph-mon[32062]: pgmap v749: 129 pgs: 129 active+clean; 12 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-07-04T08:24:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.243+0000 7f548b9d6740 1 -- start start 2024-07-04T08:24:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.243+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb2e490 0x55fb2ff01910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.243+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 0x55fb2ff08d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.243+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff12f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.243+0000 7f548b9d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f54801b69b0 con 0x55fb2fb2e490 2024-07-04T08:24:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.243+0000 7f548b9d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54801b84f0 con 0x55fb2fb57350 2024-07-04T08:24:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.243+0000 7f548b9d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f54801b7080 con 0x55fb2fb36670 2024-07-04T08:24:03.237 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb2e490 0x55fb2ff01910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb2e490 0x55fb2ff01910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55108/0 (socket says 172.21.15.39:55108) 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547e800640 1 -- 172.21.15.39:0/1839623923 learned_addr learned my addr 172.21.15.39:0/1839623923 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547de00640 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 0x55fb2ff08d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547e800640 1 -- 172.21.15.39:0/1839623923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 msgr2=0x55fb2ff08d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547f200640 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff12f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547e800640 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 0x55fb2ff08d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547e800640 1 -- 172.21.15.39:0/1839623923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 msgr2=0x55fb2ff12f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547e800640 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff12f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547e800640 1 -- 172.21.15.39:0/1839623923 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb2fefdae0 con 0x55fb2fb2e490 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.244+0000 7f547f200640 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff12f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:24:03.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.245+0000 7f547e800640 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb2e490 0x55fb2ff01910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f54700a6510 tx=0x7f547000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d8ce9db669af82 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.245+0000 7f546f400640 1 -- 172.21.15.39:0/1839623923 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54700cc040 con 0x55fb2fb2e490 2024-07-04T08:24:03.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.245+0000 7f546f400640 1 -- 172.21.15.39:0/1839623923 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54700a9a80 con 0x55fb2fb2e490 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.245+0000 7f546f400640 1 -- 172.21.15.39:0/1839623923 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54700e05b0 con 0x55fb2fb2e490 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.246+0000 7f548b9d6740 1 -- 172.21.15.39:0/1839623923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb2e490 msgr2=0x55fb2ff01910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.246+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb2e490 0x55fb2ff01910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f54700a6510 tx=0x7f547000cf90 comp rx=0 tx=0).stop 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.246+0000 7f548b9d6740 1 -- 172.21.15.39:0/1839623923 shutdown_connections 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.246+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff12f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.246+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb36670 0x55fb2ff08d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.246+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1839623923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb2e490 0x55fb2ff01910 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.246+0000 7f548b9d6740 1 -- 172.21.15.39:0/1839623923 >> 172.21.15.39:0/1839623923 conn(0x55fb2fb4b740 msgr2=0x55fb2fefd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.246+0000 7f548b9d6740 1 -- 172.21.15.39:0/1839623923 shutdown_connections 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.246+0000 7f548b9d6740 1 -- 172.21.15.39:0/1839623923 wait complete. 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.247+0000 7f548b9d6740 1 -- start start 2024-07-04T08:24:03.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.247+0000 7f548b9d6740 1 Processor -- start 2024-07-04T08:24:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff0a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff0ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f548b9d6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff796e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f548b9d6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f54801b7e20 con 0x55fb2fb36670 2024-07-04T08:24:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f548b9d6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f54801b7750 con 0x55fb2fb57350 2024-07-04T08:24:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f548b9d6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f54801b49b0 con 0x55fb2fb2e490 2024-07-04T08:24:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f548b9d6740 10 client.0 _ll_register_callbacks cb 0x7f5468004a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f547de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff0ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f547e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff0a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f547de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff0ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55110/0 (socket says 172.21.15.39:55110) 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.248+0000 7f547de00640 1 -- 172.21.15.39:0/1073855166 learned_addr learned my addr 172.21.15.39:0/1073855166 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.249+0000 7f547f200640 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.249+0000 7f547e800640 1 -- 172.21.15.39:0/1073855166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 msgr2=0x55fb2ff796e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.249+0000 7f547e800640 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff796e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.249+0000 7f547e800640 1 -- 172.21.15.39:0/1073855166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 msgr2=0x55fb2ff0ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.249+0000 7f547e800640 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff0ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.249+0000 7f547e800640 1 -- 172.21.15.39:0/1073855166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb2ff79d50 con 0x55fb2fb2e490 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.249+0000 7f547e800640 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff0a860 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f54700a5ff0 tx=0x7f5470010f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ae8f0e45a9be2fd server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.249+0000 7f546cc00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.250+0000 7f546cc00640 1 -- 172.21.15.39:0/1073855166 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54700cc070 con 0x55fb2fb2e490 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.250+0000 7f546cc00640 1 -- 172.21.15.39:0/1073855166 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9512+0+0 (secure 0 0 0) 0x7f54700107d0 con 0x55fb2fb2e490 2024-07-04T08:24:03.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.250+0000 7f548b9d6740 1 -- 172.21.15.39:0/1073855166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54700a9a80 con 0x55fb2fb2e490 2024-07-04T08:24:03.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.250+0000 7f546cc00640 1 -- 172.21.15.39:0/1073855166 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54700ae270 con 0x55fb2fb2e490 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.250+0000 7f548b9d6740 10 client.27523 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.250+0000 7f548b9d6740 1 -- 172.21.15.39:0/1073855166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb2ff81000 con 0x55fb2fb2e490 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.250+0000 7f546cc00640 1 -- 172.21.15.39:0/1073855166 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54700a9a80 con 0x55fb2fb2e490 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.250+0000 7f546cc00640 1 -- 172.21.15.39:0/1073855166 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f54700a8c40 con 0x55fb2fb2e490 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.251+0000 7f548b9d6740 10 client.27523 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.251+0000 7f548b9d6740 10 client.27523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.251+0000 7f548b9d6740 20 client.27523 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.251+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55fb2faeb080 0x55fb2ff836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.251+0000 7f548b9d6740 4 client.27523 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.251+0000 7f548b9d6740 1 -- 172.21.15.39:0/1073855166 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fb2fe6f8b0 con 0x55fb2faeb080 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.251+0000 7f547de00640 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55fb2faeb080 0x55fb2ff836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:03.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.252+0000 7f547de00640 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55fb2faeb080 0x55fb2ff836e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c8f77094b63bb33 server_cookie=fb579d167cb9bff4 in_seq=0 out_seq=0 2024-07-04T08:24:03.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.252+0000 7f546cc00640 10 client.27523 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:03.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.253+0000 7f546cc00640 1 -- 172.21.15.39:0/1073855166 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fb2fe6f8b0 con 0x55fb2faeb080 2024-07-04T08:24:03.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.253+0000 7f546cc00640 10 client.27523 handle_command_reply: tid=0 2024-07-04T08:24:03.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.253+0000 7f548b9d6740 1 client.27523 shutdown 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 -- 172.21.15.39:0/1073855166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 msgr2=0x55fb2ff0a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff0a860 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f54700a5ff0 tx=0x7f5470010f50 comp rx=0 tx=0).stop 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 -- 172.21.15.39:0/1073855166 shutdown_connections 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55fb2faeb080 0x55fb2ff836e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55fb2fb36670 0x55fb2ff0ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55fb2fb57350 0x55fb2ff796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 --2- 172.21.15.39:0/1073855166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fb2fb2e490 0x55fb2ff0a860 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 -- 172.21.15.39:0/1073855166 >> 172.21.15.39:0/1073855166 conn(0x55fb2fb4b740 msgr2=0x55fb2fefd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f546cc00640 0 client.27523 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 -- 172.21.15.39:0/1073855166 shutdown_connections 2024-07-04T08:24:03.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.254+0000 7f548b9d6740 1 -- 172.21.15.39:0/1073855166 wait complete. 2024-07-04T08:24:03.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.255+0000 7f548b9d6740 20 client.27523 trim_cache size 0 max 16384 2024-07-04T08:24:03.249 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:24:03.250 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:24:03.250 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "bf2a8232-8cdd-4022-b609-7adadc5fee62", 2024-07-04T08:24:03.250 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:24:03.250 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:24:03.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 -- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54640b2960 msgr2=0x7f54640b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:03.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54640b2960 0x7f54640b4e70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f54780096c0 tx=0x7f5478005a40 comp rx=0 tx=0).stop 2024-07-04T08:24:03.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 -- 172.21.15.39:0/1172698337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5480109510 msgr2=0x7f548019fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:03.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5480109510 0x7f548019fe70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f547000ebc0 tx=0x7f547000c6a0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 -- 172.21.15.39:0/1172698337 shutdown_connections 2024-07-04T08:24:03.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54640b2960 0x7f54640b4e70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f548010ee50 0x7f54801a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f548010f7a0 0x7f54801a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 --2- 172.21.15.39:0/1172698337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5480109510 0x7f548019fe70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:03.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.257+0000 7f5484a00640 1 -- 172.21.15.39:0/1172698337 >> 172.21.15.39:0/1172698337 conn(0x7f5480078b10 msgr2=0x7f5480079170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:03.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.258+0000 7f5484a00640 1 -- 172.21.15.39:0/1172698337 shutdown_connections 2024-07-04T08:24:03.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:03.258+0000 7f5484a00640 1 -- 172.21.15.39:0/1172698337 wait complete. 2024-07-04T08:24:03.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:03 smithi133 ceph-mon[37050]: pgmap v749: 129 pgs: 129 active+clean; 12 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-07-04T08:24:03.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:03 smithi050 ceph-mon[36577]: pgmap v749: 129 pgs: 129 active+clean; 12 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-07-04T08:24:04.142 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:24:04.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:04 smithi039 ceph-mon[32062]: osdmap e97: 12 total, 12 up, 12 in 2024-07-04T08:24:04.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:04 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:24:04.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:04 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:24:04.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:04 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:24:04.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:04 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:24:04.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:04 smithi133 ceph-mon[37050]: osdmap e97: 12 total, 12 up, 12 in 2024-07-04T08:24:04.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:04 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:24:04.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:04 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:24:04.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:04 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:24:04.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:04 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:24:04.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:04 smithi050 ceph-mon[36577]: osdmap e97: 12 total, 12 up, 12 in 2024-07-04T08:24:04.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:04 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:24:04.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:04 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:24:04.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:04 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:24:04.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:04 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:24:04.666 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:05.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:05 smithi133 ceph-mon[37050]: pgmap v751: 129 pgs: 129 active+clean; 12 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-07-04T08:24:05.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:05 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:24:05.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:05 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:24:05.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:05 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:24:05.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:05 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:24:05.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:05 smithi039 ceph-mon[32062]: pgmap v751: 129 pgs: 129 active+clean; 12 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-07-04T08:24:05.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:05 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:24:05.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:05 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:24:05.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:05 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:24:05.593 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:05 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:24:05.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:05 smithi050 ceph-mon[36577]: pgmap v751: 129 pgs: 129 active+clean; 12 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-07-04T08:24:05.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:05 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:24:05.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:05 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:24:05.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:05 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:24:05.637 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:05 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:24:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.972+0000 7fbeb2e00640 1 -- 172.21.15.39:0/1779216220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeac10b380 msgr2=0x7fbeac111bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.972+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/1779216220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeac10b380 0x7fbeac111bb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009be0 tx=0x7fbea402a6c0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.972+0000 7fbeb2e00640 1 -- 172.21.15.39:0/1779216220 shutdown_connections 2024-07-04T08:24:06.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.972+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/1779216220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeac10b380 0x7fbeac111bb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.972+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/1779216220 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeac10aa30 0x7fbeac10ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.972+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/1779216220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac1050f0 0x7fbeac105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.972+0000 7fbeb2e00640 1 -- 172.21.15.39:0/1779216220 >> 172.21.15.39:0/1779216220 conn(0x7fbeac100b60 msgr2=0x7fbeac102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.972+0000 7fbeb2e00640 1 -- 172.21.15.39:0/1779216220 shutdown_connections 2024-07-04T08:24:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.972+0000 7fbeb2e00640 1 -- 172.21.15.39:0/1779216220 wait complete. 2024-07-04T08:24:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.974+0000 7fbeb2e00640 1 Processor -- start 2024-07-04T08:24:06.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.974+0000 7fbeb2e00640 1 -- start start 2024-07-04T08:24:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.974+0000 7fbeb2e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeac1050f0 0x7fbeac19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.974+0000 7fbeb2e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeac10aa30 0x7fbeac19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.974+0000 7fbeb2e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac10b380 0x7fbeac1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.974+0000 7fbeb2e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac115030 con 0x7fbeac10b380 2024-07-04T08:24:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.974+0000 7fbeb2e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac114eb0 con 0x7fbeac1050f0 2024-07-04T08:24:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.974+0000 7fbeb2e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1151b0 con 0x7fbeac10aa30 2024-07-04T08:24:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeac1050f0 0x7fbeac19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeac10aa30 0x7fbeac19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeac1050f0 0x7fbeac19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44978/0 (socket says 172.21.15.39:44978) 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1a00640 1 -- 172.21.15.39:0/3557346342 learned_addr learned my addr 172.21.15.39:0/3557346342 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1000640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeac10aa30 0x7fbeac19bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:55082/0 (socket says 172.21.15.39:55082) 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb2400640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac10b380 0x7fbeac1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1a00640 1 -- 172.21.15.39:0/3557346342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeac10aa30 msgr2=0x7fbeac19bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeac10aa30 0x7fbeac19bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1a00640 1 -- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac10b380 msgr2=0x7fbeac1a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac10b380 0x7fbeac1a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb1a00640 1 -- 172.21.15.39:0/3557346342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac1a1950 con 0x7fbeac1050f0 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.975+0000 7fbeb2400640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac10b380 0x7fbeac1a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:24:06.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.976+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeac1050f0 0x7fbeac19b9e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbea000e9f0 tx=0x7fbea000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.976+0000 7fbeaa000640 1 -- 172.21.15.39:0/3557346342 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea000ccc0 con 0x7fbeac1050f0 2024-07-04T08:24:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.976+0000 7fbeb2e00640 1 -- 172.21.15.39:0/3557346342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac1a1c40 con 0x7fbeac1050f0 2024-07-04T08:24:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.976+0000 7fbeaa000640 1 -- 172.21.15.39:0/3557346342 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea00108e0 con 0x7fbeac1050f0 2024-07-04T08:24:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.976+0000 7fbeaa000640 1 -- 172.21.15.39:0/3557346342 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0018880 con 0x7fbeac1050f0 2024-07-04T08:24:06.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.976+0000 7fbeb2e00640 1 -- 172.21.15.39:0/3557346342 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac1a83a0 con 0x7fbeac1050f0 2024-07-04T08:24:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.980+0000 7fbebaae0740 1 Processor -- start 2024-07-04T08:24:06.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.980+0000 7fbebaae0740 1 -- start start 2024-07-04T08:24:06.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.980+0000 7fbeaa000640 1 -- 172.21.15.39:0/3557346342 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbea0010430 con 0x7fbeac1050f0 2024-07-04T08:24:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99d6070 0x562dc9d7a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99ad1b0 0x562dc9d7ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99b5390 0x562dc9d7b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbebaae0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562dc9cf0ea0 con 0x562dc99d6070 2024-07-04T08:24:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbebaae0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562dc9ce0440 con 0x562dc99ad1b0 2024-07-04T08:24:06.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbebaae0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562dc9ce4480 con 0x562dc99b5390 2024-07-04T08:24:06.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbeb2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99d6070 0x562dc9d7a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbeb2400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99d6070 0x562dc9d7a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55132/0 (socket says 172.21.15.39:55132) 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbeb2400640 1 -- 172.21.15.39:0/402183003 learned_addr learned my addr 172.21.15.39:0/402183003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/402183003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99ad1b0 0x562dc9d7ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.981+0000 7fbeb1000640 1 --2- 172.21.15.39:0/402183003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99b5390 0x562dc9d7b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.982+0000 7fbeb1a00640 1 -- 172.21.15.39:0/402183003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99b5390 msgr2=0x562dc9d7b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.982+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/402183003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99b5390 0x562dc9d7b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.982+0000 7fbeb1a00640 1 -- 172.21.15.39:0/402183003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99d6070 msgr2=0x562dc9d7a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.982+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/402183003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99d6070 0x562dc9d7a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.982+0000 7fbeb1a00640 1 -- 172.21.15.39:0/402183003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc9c78610 con 0x562dc99ad1b0 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.982+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/402183003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99ad1b0 0x562dc9d7ae50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbea00e00d0 tx=0x7fbea00e6f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e441799f7c2d82e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.983+0000 7fbeaa000640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe940b2960 0x7fbe940b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.983+0000 7fbe9ea00640 1 -- 172.21.15.39:0/402183003 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00e9bc0 con 0x562dc99ad1b0 2024-07-04T08:24:06.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.983+0000 7fbe9ea00640 1 -- 172.21.15.39:0/402183003 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea00a9770 con 0x562dc99ad1b0 2024-07-04T08:24:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.983+0000 7fbe9ea00640 1 -- 172.21.15.39:0/402183003 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea006e8b0 con 0x562dc99ad1b0 2024-07-04T08:24:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.983+0000 7fbeaa000640 1 -- 172.21.15.39:0/3557346342 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9512+0+0 (secure 0 0 0) 0x7fbea00e2dc0 con 0x7fbeac1050f0 2024-07-04T08:24:06.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.983+0000 7fbeb2400640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe940b2960 0x7fbe940b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbebaae0740 1 -- 172.21.15.39:0/402183003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99ad1b0 msgr2=0x562dc9d7ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbebaae0740 1 --2- 172.21.15.39:0/402183003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99ad1b0 0x562dc9d7ae50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbea00e00d0 tx=0x7fbea00e6f70 comp rx=0 tx=0).stop 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbebaae0740 1 -- 172.21.15.39:0/402183003 shutdown_connections 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbebaae0740 1 --2- 172.21.15.39:0/402183003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99b5390 0x562dc9d7b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbebaae0740 1 --2- 172.21.15.39:0/402183003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99ad1b0 0x562dc9d7ae50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbebaae0740 1 --2- 172.21.15.39:0/402183003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99d6070 0x562dc9d7a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbebaae0740 1 -- 172.21.15.39:0/402183003 >> 172.21.15.39:0/402183003 conn(0x562dc99ca460 msgr2=0x562dc9d78800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbeb2400640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe940b2960 0x7fbe940b4e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbea40099a0 tx=0x7fbea4005bf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbebaae0740 1 -- 172.21.15.39:0/402183003 shutdown_connections 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.984+0000 7fbebaae0740 1 -- 172.21.15.39:0/402183003 wait complete. 2024-07-04T08:24:06.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.985+0000 7fbebaae0740 1 -- start start 2024-07-04T08:24:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.985+0000 7fbebaae0740 1 Processor -- start 2024-07-04T08:24:06.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d86220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d867c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 0x562dc9d92470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbebaae0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562dc9b44700 con 0x562dc99b5390 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbebaae0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562dc9c87b70 con 0x562dc99d6070 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbebaae0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562dc9d890f0 con 0x562dc99ad1b0 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbebaae0740 10 client.0 _ll_register_callbacks cb 0x562dc9cd9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbeb1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d867c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbeb1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d86220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbeb2400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 0x562dc9d92470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbeb1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d867c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55144/0 (socket says 172.21.15.39:55144) 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.986+0000 7fbeb1000640 1 -- 172.21.15.39:0/809398823 learned_addr learned my addr 172.21.15.39:0/809398823 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.987+0000 7fbeb1000640 1 -- 172.21.15.39:0/809398823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 msgr2=0x562dc9d86220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:06.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.987+0000 7fbeb1000640 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d86220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.987+0000 7fbeb1000640 1 -- 172.21.15.39:0/809398823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 msgr2=0x562dc9d92470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.987+0000 7fbeb1000640 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 0x562dc9d92470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.987+0000 7fbeb1000640 1 -- 172.21.15.39:0/809398823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc9a41650 con 0x562dc99b5390 2024-07-04T08:24:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.987+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d86220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.987+0000 7fbeb1000640 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d867c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbe980126e0 tx=0x7fbe98012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78212e5eb453c7ee server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:06.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.987+0000 7fbe93400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:24:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.988+0000 7fbe93400640 1 -- 172.21.15.39:0/809398823 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98012190 con 0x562dc99b5390 2024-07-04T08:24:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.988+0000 7fbe93400640 1 -- 172.21.15.39:0/809398823 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9512+0+0 (secure 0 0 0) 0x7fbe9800f920 con 0x562dc99b5390 2024-07-04T08:24:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.988+0000 7fbebaae0740 1 -- 172.21.15.39:0/809398823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea00a9770 con 0x562dc99b5390 2024-07-04T08:24:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.988+0000 7fbe93400640 1 -- 172.21.15.39:0/809398823 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe9800fb20 con 0x562dc99b5390 2024-07-04T08:24:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.988+0000 7fbebaae0740 10 client.17475 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.988+0000 7fbebaae0740 1 -- 172.21.15.39:0/809398823 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562dc9d95070 con 0x562dc99b5390 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.988+0000 7fbe93400640 1 -- 172.21.15.39:0/809398823 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea00a9770 con 0x562dc99b5390 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.989+0000 7fbe93400640 1 -- 172.21.15.39:0/809398823 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbe98010c90 con 0x562dc99b5390 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.989+0000 7fbebaae0740 10 client.17475 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.989+0000 7fbebaae0740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.989+0000 7fbebaae0740 20 client.17475 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.989+0000 7fbebaae0740 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562dc99949f0 0x562dc9d98ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.989+0000 7fbebaae0740 4 client.17475 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.989+0000 7fbebaae0740 1 -- 172.21.15.39:0/809398823 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dc9cf2df0 con 0x562dc99949f0 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.989+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562dc99949f0 0x562dc9d98ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.990+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562dc99949f0 0x562dc9d98ad0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9616b73103ed86 server_cookie=f7c15d35909b223c in_seq=0 out_seq=0 2024-07-04T08:24:06.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.990+0000 7fbe93400640 10 client.17475 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbe93400640 1 -- 172.21.15.39:0/809398823 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562dc9cf2df0 con 0x562dc99949f0 2024-07-04T08:24:06.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbe93400640 10 client.17475 handle_command_reply: tid=0 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbebaae0740 1 client.17475 shutdown 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbebaae0740 1 -- 172.21.15.39:0/809398823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 msgr2=0x562dc9d867c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbebaae0740 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d867c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbe980126e0 tx=0x7fbe98012ba0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbebaae0740 1 -- 172.21.15.39:0/809398823 shutdown_connections 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbebaae0740 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562dc99949f0 0x562dc9d98ad0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbebaae0740 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d867c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbebaae0740 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 0x562dc9d92470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbebaae0740 1 --2- 172.21.15.39:0/809398823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d86220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbebaae0740 1 -- 172.21.15.39:0/809398823 >> 172.21.15.39:0/809398823 conn(0x562dc99ca460 msgr2=0x562dc9d7daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.991+0000 7fbe93400640 0 client.17475 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.992+0000 7fbebaae0740 1 -- 172.21.15.39:0/809398823 shutdown_connections 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.992+0000 7fbebaae0740 1 -- 172.21.15.39:0/809398823 wait complete. 2024-07-04T08:24:06.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:06.992+0000 7fbebaae0740 20 client.17475 trim_cache size 0 max 16384 2024-07-04T08:24:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.012+0000 7fbebaae0740 1 Processor -- start 2024-07-04T08:24:07.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.012+0000 7fbebaae0740 1 -- start start 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d80a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 0x562dc9d923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbebaae0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c010010 con 0x562dc99b5390 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbebaae0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c011b50 con 0x562dc99d6070 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbebaae0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c0106e0 con 0x562dc99ad1b0 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbeb1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbeb1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d87e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:55156/0 (socket says 172.21.15.39:55156) 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbeb1000640 1 -- 172.21.15.39:0/2368923229 learned_addr learned my addr 172.21.15.39:0/2368923229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.013+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d80a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:07.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.014+0000 7fbeb2400640 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 0x562dc9d923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:07.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.014+0000 7fbeb1000640 1 -- 172.21.15.39:0/2368923229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 msgr2=0x562dc9d80a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:07.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.014+0000 7fbeb1000640 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d80a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.014+0000 7fbeb1000640 1 -- 172.21.15.39:0/2368923229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 msgr2=0x562dc9d923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:07.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.014+0000 7fbeb1000640 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 0x562dc9d923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.007 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.014+0000 7fbeb1000640 1 -- 172.21.15.39:0/2368923229 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc9d7cc00 con 0x562dc99b5390 2024-07-04T08:24:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.014+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d80a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.014+0000 7fbeb1000640 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d87e50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbe9801c570 tx=0x7fbe9800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff340fab693a92b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.015+0000 7fbe9e000640 1 -- 172.21.15.39:0/2368923229 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98012190 con 0x562dc99b5390 2024-07-04T08:24:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.015+0000 7fbe9e000640 1 -- 172.21.15.39:0/2368923229 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9800fcc0 con 0x562dc99b5390 2024-07-04T08:24:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.015+0000 7fbe9e000640 1 -- 172.21.15.39:0/2368923229 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe980053d0 con 0x562dc99b5390 2024-07-04T08:24:07.008 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.016+0000 7fbebaae0740 1 -- 172.21.15.39:0/2368923229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 msgr2=0x562dc9d87e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.016+0000 7fbebaae0740 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d87e50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbe9801c570 tx=0x7fbe9800fab0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.016+0000 7fbebaae0740 1 -- 172.21.15.39:0/2368923229 shutdown_connections 2024-07-04T08:24:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.016+0000 7fbebaae0740 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99b5390 0x562dc9d87e50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.016+0000 7fbebaae0740 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99d6070 0x562dc9d923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.016+0000 7fbebaae0740 1 --2- 172.21.15.39:0/2368923229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99ad1b0 0x562dc9d80a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.016+0000 7fbebaae0740 1 -- 172.21.15.39:0/2368923229 >> 172.21.15.39:0/2368923229 conn(0x562dc99ca460 msgr2=0x562dc9d87740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.016+0000 7fbebaae0740 1 -- 172.21.15.39:0/2368923229 shutdown_connections 2024-07-04T08:24:07.009 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.016+0000 7fbebaae0740 1 -- 172.21.15.39:0/2368923229 wait complete. 2024-07-04T08:24:07.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.017+0000 7fbebaae0740 1 -- start start 2024-07-04T08:24:07.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.017+0000 7fbebaae0740 1 Processor -- start 2024-07-04T08:24:07.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99ad1b0 0x562dc9d99200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:07.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99b5390 0x562dc9d997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:07.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbebaae0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99d6070 0x562dc9df9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbebaae0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c011480 con 0x562dc99ad1b0 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbebaae0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c010db0 con 0x562dc99b5390 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbebaae0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c00e010 con 0x562dc99d6070 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbebaae0740 10 client.0 _ll_register_callbacks cb 0x7fbe98009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbeb1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99ad1b0 0x562dc9d99200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbeb2400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99d6070 0x562dc9df9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbeb1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99b5390 0x562dc9d997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbeb1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99b5390 0x562dc9d997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45018/0 (socket says 172.21.15.39:45018) 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbeb2400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99d6070 0x562dc9df9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:55138/0 (socket says 172.21.15.39:55138) 2024-07-04T08:24:07.011 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.018+0000 7fbeb1000640 1 -- 172.21.15.39:0/3037843742 learned_addr learned my addr 172.21.15.39:0/3037843742 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:07.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.019+0000 7fbeb1000640 1 -- 172.21.15.39:0/3037843742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99d6070 msgr2=0x562dc9df9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:07.012 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.019+0000 7fbeb1000640 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99d6070 0x562dc9df9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.019+0000 7fbeb1000640 1 -- 172.21.15.39:0/3037843742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99ad1b0 msgr2=0x562dc9d99200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.019+0000 7fbeb1000640 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99ad1b0 0x562dc9d99200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.019+0000 7fbeb1000640 1 -- 172.21.15.39:0/3037843742 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc9df96c0 con 0x562dc99b5390 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.019+0000 7fbeb2400640 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99d6070 0x562dc9df9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.019+0000 7fbeb1000640 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99b5390 0x562dc9d997a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006010 tx=0x7fbe98005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe194570620cbdf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.019+0000 7fbe93400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.020+0000 7fbe93400640 1 -- 172.21.15.39:0/3037843742 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9800fe70 con 0x562dc99b5390 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.020+0000 7fbebaae0740 1 -- 172.21.15.39:0/3037843742 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe9800fcc0 con 0x562dc99b5390 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.020+0000 7fbe93400640 1 -- 172.21.15.39:0/3037843742 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9512+0+0 (secure 0 0 0) 0x7fbe98018040 con 0x562dc99b5390 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.020+0000 7fbe93400640 1 -- 172.21.15.39:0/3037843742 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe9803d050 con 0x562dc99b5390 2024-07-04T08:24:07.013 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.020+0000 7fbebaae0740 10 client.27611 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:07.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.020+0000 7fbebaae0740 1 -- 172.21.15.39:0/3037843742 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562dc9e00a00 con 0x562dc99b5390 2024-07-04T08:24:07.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.020+0000 7fbe93400640 1 -- 172.21.15.39:0/3037843742 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9800fcc0 con 0x562dc99b5390 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.021+0000 7fbe93400640 1 -- 172.21.15.39:0/3037843742 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbe98003990 con 0x562dc99b5390 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.021+0000 7fbebaae0740 10 client.27611 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.021+0000 7fbebaae0740 10 client.27611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.021+0000 7fbebaae0740 20 client.27611 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.021+0000 7fbebaae0740 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562dc9969da0 0x562dc9e030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.021+0000 7fbebaae0740 4 client.27611 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.021+0000 7fbebaae0740 1 -- 172.21.15.39:0/3037843742 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562dc9cee890 con 0x562dc9969da0 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.021+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562dc9969da0 0x562dc9e030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.022+0000 7fbeb1a00640 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562dc9969da0 0x562dc9e030d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e64ba551c77a3c6b server_cookie=2ded35f2a92d6201 in_seq=0 out_seq=0 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.022+0000 7fbe93400640 10 client.27611 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.022+0000 7fbe93400640 1 -- 172.21.15.39:0/3037843742 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562dc9cee890 con 0x562dc9969da0 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.022+0000 7fbe93400640 10 client.27611 handle_command_reply: tid=0 2024-07-04T08:24:07.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.022+0000 7fbebaae0740 1 client.27611 shutdown 2024-07-04T08:24:07.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.022+0000 7fbebaae0740 1 -- 172.21.15.39:0/3037843742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99b5390 msgr2=0x562dc9d997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:07.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.022+0000 7fbebaae0740 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99b5390 0x562dc9d997a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbe98006010 tx=0x7fbe98005970 comp rx=0 tx=0).stop 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbebaae0740 1 -- 172.21.15.39:0/3037843742 shutdown_connections 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbebaae0740 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562dc9969da0 0x562dc9e030d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbebaae0740 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562dc99d6070 0x562dc9df9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbebaae0740 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562dc99b5390 0x562dc9d997a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbebaae0740 1 --2- 172.21.15.39:0/3037843742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562dc99ad1b0 0x562dc9d99200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbe93400640 0 client.27611 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbebaae0740 1 -- 172.21.15.39:0/3037843742 >> 172.21.15.39:0/3037843742 conn(0x562dc99ca460 msgr2=0x562dc9d76dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbebaae0740 1 -- 172.21.15.39:0/3037843742 shutdown_connections 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbebaae0740 1 -- 172.21.15.39:0/3037843742 wait complete. 2024-07-04T08:24:07.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.023+0000 7fbebaae0740 20 client.27611 trim_cache size 0 max 16384 2024-07-04T08:24:07.018 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:24:07.018 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-04T08:24:07.018 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-07-04T08:24:07.018 INFO:teuthology.orchestra.run.smithi039.stdout: "bf2a8232-8cdd-4022-b609-7adadc5fee62": { 2024-07-04T08:24:07.018 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-07-04T08:24:07.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "bf2a8232-8cdd-4022-b609-7adadc5fee62", 2024-07-04T08:24:07.018 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-07-04T08:24:07.019 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:24:07.019 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:24:07.019 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:24:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 -- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe940b2960 msgr2=0x7fbe940b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe940b2960 0x7fbe940b4e70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbea40099a0 tx=0x7fbea4005bf0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 -- 172.21.15.39:0/3557346342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeac1050f0 msgr2=0x7fbeac19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeac1050f0 0x7fbeac19b9e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbea000e9f0 tx=0x7fbea000edf0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 -- 172.21.15.39:0/3557346342 shutdown_connections 2024-07-04T08:24:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbe940b2960 0x7fbe940b4e70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeac10b380 0x7fbeac1a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbeac10aa30 0x7fbeac19bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 --2- 172.21.15.39:0/3557346342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbeac1050f0 0x7fbeac19b9e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 -- 172.21.15.39:0/3557346342 >> 172.21.15.39:0/3557346342 conn(0x7fbeac100b60 msgr2=0x7fbeac102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.024+0000 7fbeb2e00640 1 -- 172.21.15.39:0/3557346342 shutdown_connections 2024-07-04T08:24:07.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:07.025+0000 7fbeb2e00640 1 -- 172.21.15.39:0/3557346342 wait complete. 2024-07-04T08:24:07.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:07 smithi039 ceph-mon[32062]: pgmap v752: 129 pgs: 129 active+clean; 12 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-07-04T08:24:07.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:07 smithi133 ceph-mon[37050]: pgmap v752: 129 pgs: 129 active+clean; 12 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-07-04T08:24:07.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:07 smithi050 ceph-mon[36577]: pgmap v752: 129 pgs: 129 active+clean; 12 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-07-04T08:24:07.949 INFO:tasks.cephfs.filesystem:scrub status for tag:bf2a8232-8cdd-4022-b609-7adadc5fee62 - {'path': '/', 'tag': 'bf2a8232-8cdd-4022-b609-7adadc5fee62', 'options': 'recursive,force'} 2024-07-04T08:24:08.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:08 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:24:08.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:08 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:24:08.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:08 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:24:09.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:09 smithi133 ceph-mon[37050]: pgmap v753: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-07-04T08:24:09.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:09 smithi039 ceph-mon[32062]: pgmap v753: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-07-04T08:24:09.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:09 smithi050 ceph-mon[36577]: pgmap v753: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-07-04T08:24:11.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:11 smithi133 ceph-mon[37050]: pgmap v754: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-07-04T08:24:11.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:11 smithi039 ceph-mon[32062]: pgmap v754: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-07-04T08:24:11.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:11 smithi050 ceph-mon[36577]: pgmap v754: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 104 KiB/s wr, 36 op/s 2024-07-04T08:24:13.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:13 smithi133 ceph-mon[37050]: pgmap v755: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 108 KiB/s wr, 37 op/s 2024-07-04T08:24:13.596 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:13 smithi039 ceph-mon[32062]: pgmap v755: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 108 KiB/s wr, 37 op/s 2024-07-04T08:24:13.636 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:13 smithi050 ceph-mon[36577]: pgmap v755: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 108 KiB/s wr, 37 op/s 2024-07-04T08:24:14.111 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.306 INFO:tasks.workunit.client.0.smithi039.stdout:251/600: open fd = 3 2024-07-04T08:24:14.306 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.306 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:252/600: open fd = 3 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:253/600: open fd = 3 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:254/600: open fd = 3 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.307 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:255/600: open fd = 3 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:256/600: open fd = 3 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:257/600: open fd = 3 2024-07-04T08:24:14.308 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:258/600: open fd = 3 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:259/600: open fd = 3 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.309 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:260/600: open fd = 3 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:261/600: open fd = 3 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:262/600: open fd = 3 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.310 INFO:tasks.workunit.client.0.smithi039.stdout:263/600: open fd = 3 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:264/600: open fd = 3 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:265/600: open fd = 3 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.311 INFO:tasks.workunit.client.0.smithi039.stdout:266/600: open fd = 3 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:267/600: open fd = 3 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:268/600: open fd = 3 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:269/600: open fd = 3 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.312 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:270/600: open fd = 3 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:271/600: open fd = 3 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:272/600: open fd = 3 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.313 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.314 INFO:tasks.workunit.client.0.smithi039.stdout:273/600: open fd = 3 2024-07-04T08:24:14.314 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.314 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.314 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.314 INFO:tasks.workunit.client.0.smithi039.stdout:274/600: open fd = 3 2024-07-04T08:24:14.314 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.314 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.314 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.314 INFO:tasks.workunit.client.0.smithi039.stdout:275/600: open fd = 3 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:276/600: open fd = 3 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:277/600: open fd = 3 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:278/600: open fd = 3 2024-07-04T08:24:14.315 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.316 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.316 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.316 INFO:tasks.workunit.client.0.smithi039.stdout:279/600: open fd = 3 2024-07-04T08:24:14.316 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.316 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.316 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.316 INFO:tasks.workunit.client.0.smithi039.stdout:280/600: open fd = 3 2024-07-04T08:24:14.316 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.317 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.317 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.317 INFO:tasks.workunit.client.0.smithi039.stdout:281/600: open fd = 3 2024-07-04T08:24:14.317 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.318 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.318 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.318 INFO:tasks.workunit.client.0.smithi039.stdout:282/600: open fd = 3 2024-07-04T08:24:14.319 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.319 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.319 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.319 INFO:tasks.workunit.client.0.smithi039.stdout:283/600: open fd = 3 2024-07-04T08:24:14.320 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.320 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.320 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.320 INFO:tasks.workunit.client.0.smithi039.stdout:284/600: open fd = 3 2024-07-04T08:24:14.321 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.321 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.321 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.321 INFO:tasks.workunit.client.0.smithi039.stdout:285/600: open fd = 3 2024-07-04T08:24:14.321 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.322 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.322 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.322 INFO:tasks.workunit.client.0.smithi039.stdout:286/600: open fd = 3 2024-07-04T08:24:14.322 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.322 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.322 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.323 INFO:tasks.workunit.client.0.smithi039.stdout:287/600: open fd = 3 2024-07-04T08:24:14.323 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.323 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.323 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.323 INFO:tasks.workunit.client.0.smithi039.stdout:288/600: open fd = 3 2024-07-04T08:24:14.323 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.324 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.324 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.324 INFO:tasks.workunit.client.0.smithi039.stdout:289/600: open fd = 3 2024-07-04T08:24:14.324 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.324 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.324 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.325 INFO:tasks.workunit.client.0.smithi039.stdout:290/600: open fd = 3 2024-07-04T08:24:14.325 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.325 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.325 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.325 INFO:tasks.workunit.client.0.smithi039.stdout:291/600: open fd = 3 2024-07-04T08:24:14.325 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.326 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.326 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.326 INFO:tasks.workunit.client.0.smithi039.stdout:292/600: open fd = 3 2024-07-04T08:24:14.326 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.326 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.326 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.327 INFO:tasks.workunit.client.0.smithi039.stdout:293/600: open fd = 3 2024-07-04T08:24:14.327 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.327 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.327 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.327 INFO:tasks.workunit.client.0.smithi039.stdout:294/600: open fd = 3 2024-07-04T08:24:14.327 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.328 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.329 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.329 INFO:tasks.workunit.client.0.smithi039.stdout:295/600: open fd = 3 2024-07-04T08:24:14.329 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.329 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.329 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.329 INFO:tasks.workunit.client.0.smithi039.stdout:296/600: open fd = 3 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:297/600: open fd = 3 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:298/600: open fd = 3 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.330 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:299/600: open fd = 3 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:300/600: open fd = 3 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:301/600: open fd = 3 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.331 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.332 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.332 INFO:tasks.workunit.client.0.smithi039.stdout:302/600: open fd = 3 2024-07-04T08:24:14.332 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.332 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.332 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.332 INFO:tasks.workunit.client.0.smithi039.stdout:303/600: open fd = 3 2024-07-04T08:24:14.332 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.332 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.332 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:304/600: open fd = 3 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:305/600: open fd = 3 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:306/600: open fd = 3 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.333 INFO:tasks.workunit.client.0.smithi039.stdout:307/600: open fd = 3 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:308/600: open fd = 3 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:309/600: open fd = 3 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:310/600: open fd = 3 2024-07-04T08:24:14.334 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.335 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.335 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.335 INFO:tasks.workunit.client.0.smithi039.stdout:311/600: open fd = 3 2024-07-04T08:24:14.335 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.335 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.335 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:24:14.335 INFO:tasks.workunit.client.0.smithi039.stdout:312/600: open fd = 3 2024-07-04T08:24:14.335 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.335 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:24:14.886 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:14 smithi050 ceph-mon[36577]: osdmap e98: 12 total, 12 up, 12 in 2024-07-04T08:24:14.886 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:14 smithi050 ceph-mon[36577]: pgmap v757: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 36 op/s 2024-07-04T08:24:15.058 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:14 smithi133 ceph-mon[37050]: osdmap e98: 12 total, 12 up, 12 in 2024-07-04T08:24:15.058 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:14 smithi133 ceph-mon[37050]: pgmap v757: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 36 op/s 2024-07-04T08:24:15.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:14 smithi039 ceph-mon[32062]: osdmap e98: 12 total, 12 up, 12 in 2024-07-04T08:24:15.093 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:14 smithi039 ceph-mon[32062]: pgmap v757: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 36 op/s 2024-07-04T08:24:17.092 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:17 smithi039 ceph-mon[32062]: pgmap v758: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-07-04T08:24:17.307 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:17 smithi133 ceph-mon[37050]: pgmap v758: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-07-04T08:24:17.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:17 smithi050 ceph-mon[36577]: pgmap v758: 129 pgs: 129 active+clean; 12 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-07-04T08:24:19.308 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:19 smithi133 ceph-mon[37050]: pgmap v759: 129 pgs: 129 active+clean; 12 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:24:19.343 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:19 smithi039 ceph-mon[32062]: pgmap v759: 129 pgs: 129 active+clean; 12 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:24:19.391 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:19 smithi050 ceph-mon[36577]: pgmap v759: 129 pgs: 129 active+clean; 12 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:24:21.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:21 smithi039 ceph-mon[32062]: pgmap v760: 129 pgs: 129 active+clean; 12 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:24:21.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:21 smithi050 ceph-mon[36577]: pgmap v760: 129 pgs: 129 active+clean; 12 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:24:21.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:21 smithi133 ceph-mon[37050]: pgmap v760: 129 pgs: 129 active+clean; 12 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:24:23.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:23 smithi039 ceph-mon[32062]: pgmap v761: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:24:23.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:23 smithi050 ceph-mon[36577]: pgmap v761: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:24:23.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:23 smithi133 ceph-mon[37050]: pgmap v761: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:24:25.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:25 smithi039 ceph-mon[32062]: pgmap v762: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:24:25.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:25 smithi050 ceph-mon[36577]: pgmap v762: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:24:25.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:25 smithi133 ceph-mon[37050]: pgmap v762: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:24:27.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:27 smithi039 ceph-mon[32062]: pgmap v763: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-07-04T08:24:27.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:27 smithi050 ceph-mon[36577]: pgmap v763: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-07-04T08:24:27.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:27 smithi133 ceph-mon[37050]: pgmap v763: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-07-04T08:24:29.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:29 smithi039 ceph-mon[32062]: pgmap v764: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 17 op/s 2024-07-04T08:24:29.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:29 smithi050 ceph-mon[36577]: pgmap v764: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 17 op/s 2024-07-04T08:24:29.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:29 smithi133 ceph-mon[37050]: pgmap v764: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 17 op/s 2024-07-04T08:24:31.172 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.1 2024-07-04T08:24:31.173 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:24:31.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:31 smithi050 ceph-mon[36577]: pgmap v765: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-07-04T08:24:31.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:31 smithi133 ceph-mon[37050]: pgmap v765: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-07-04T08:24:31.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:31 smithi039 ceph-mon[32062]: pgmap v765: 129 pgs: 129 active+clean; 13 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 14 op/s 2024-07-04T08:24:31.672 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:33.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:33 smithi050 ceph-mon[36577]: pgmap v766: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 18 op/s 2024-07-04T08:24:33.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:33 smithi039 ceph-mon[32062]: pgmap v766: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 18 op/s 2024-07-04T08:24:33.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:33 smithi133 ceph-mon[37050]: pgmap v766: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 18 op/s 2024-07-04T08:24:34.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.160+0000 7fd59fe00640 1 -- 172.21.15.39:0/1800223915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0101150 msgr2=0x7fd5a0101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.160+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1800223915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0101150 0x7fd5a0101560 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd588009c10 tx=0x7fd58802a6f0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.160+0000 7fd59fe00640 1 -- 172.21.15.39:0/1800223915 shutdown_connections 2024-07-04T08:24:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.160+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1800223915 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5a0101aa0 0x7fd5a010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.160+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1800223915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0101150 0x7fd5a0101560 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.160+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1800223915 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a0107a60 0x7fd5a0107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.160+0000 7fd59fe00640 1 -- 172.21.15.39:0/1800223915 >> 172.21.15.39:0/1800223915 conn(0x7fd5a0076ff0 msgr2=0x7fd5a0077430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.160+0000 7fd59fe00640 1 -- 172.21.15.39:0/1800223915 shutdown_connections 2024-07-04T08:24:34.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.160+0000 7fd59fe00640 1 -- 172.21.15.39:0/1800223915 wait complete. 2024-07-04T08:24:34.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.162+0000 7fd59fe00640 1 Processor -- start 2024-07-04T08:24:34.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.162+0000 7fd59fe00640 1 -- start start 2024-07-04T08:24:34.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5a0101150 0x7fd5a019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0101aa0 0x7fd5a019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a0107a60 0x7fd5a01a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0112de0 con 0x7fd5a0107a60 2024-07-04T08:24:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0112c60 con 0x7fd5a0101150 2024-07-04T08:24:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0112f60 con 0x7fd5a0101aa0 2024-07-04T08:24:34.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5a0101150 0x7fd5a019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0101aa0 0x7fd5a019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a0107a60 0x7fd5a01a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0101aa0 0x7fd5a019bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37108/0 (socket says 172.21.15.39:37108) 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59e000640 1 -- 172.21.15.39:0/1108812831 learned_addr learned my addr 172.21.15.39:0/1108812831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.163+0000 7fd59ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5a0101150 0x7fd5a019ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47050/0 (socket says 172.21.15.39:47050) 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd59ea00640 1 -- 172.21.15.39:0/1108812831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0101aa0 msgr2=0x7fd5a019bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd59ea00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0101aa0 0x7fd5a019bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd59ea00640 1 -- 172.21.15.39:0/1108812831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a0107a60 msgr2=0x7fd5a01a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd59ea00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a0107a60 0x7fd5a01a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd59ea00640 1 -- 172.21.15.39:0/1108812831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a01a19c0 con 0x7fd5a0101150 2024-07-04T08:24:34.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd59ea00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5a0101150 0x7fd5a019ba50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e790 tx=0x7fd59000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:34.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd5a4a00640 1 -- 172.21.15.39:0/1108812831 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59000eea0 con 0x7fd5a0101150 2024-07-04T08:24:34.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd5a4a00640 1 -- 172.21.15.39:0/1108812831 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd59000ccc0 con 0x7fd5a0101150 2024-07-04T08:24:34.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a01a1cb0 con 0x7fd5a0101150 2024-07-04T08:24:34.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.164+0000 7fd5a4a00640 1 -- 172.21.15.39:0/1108812831 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590018680 con 0x7fd5a0101150 2024-07-04T08:24:34.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.165+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a01a8410 con 0x7fd5a0101150 2024-07-04T08:24:34.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.166+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a0101560 con 0x7fd5a0101150 2024-07-04T08:24:34.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.168+0000 7fd5a4a00640 1 -- 172.21.15.39:0/1108812831 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd59004ab30 con 0x7fd5a0101150 2024-07-04T08:24:34.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.170+0000 7fd5a4a00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5800b2960 0x7fd5800b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:34.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.170+0000 7fd5a4a00640 1 -- 172.21.15.39:0/1108812831 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7fd5900e2b10 con 0x7fd5a0101150 2024-07-04T08:24:34.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.170+0000 7fd59e000640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5800b2960 0x7fd5800b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:34.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.171+0000 7fd59e000640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5800b2960 0x7fd5800b4e70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd588006010 tx=0x7fd588009280 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:34.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.171+0000 7fd5a4a00640 1 -- 172.21.15.39:0/1108812831 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fd59004ae20 con 0x7fd5a0101150 2024-07-04T08:24:34.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.437+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5a01a86b0 con 0x7fd5a0101150 2024-07-04T08:24:34.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.438+0000 7fd5a4a00640 1 -- 172.21.15.39:0/1108812831 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fd5900b21b0 con 0x7fd5a0101150 2024-07-04T08:24:34.432 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:24:34.432 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:24:34.432 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:24:34.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5800b2960 msgr2=0x7fd5800b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:34.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5800b2960 0x7fd5800b4e70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd588006010 tx=0x7fd588009280 comp rx=0 tx=0).stop 2024-07-04T08:24:34.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5a0101150 msgr2=0x7fd5a019ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:34.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5a0101150 0x7fd5a019ba50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e790 tx=0x7fd59000ec50 comp rx=0 tx=0).stop 2024-07-04T08:24:34.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 shutdown_connections 2024-07-04T08:24:34.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5800b2960 0x7fd5800b4e70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5a0107a60 0x7fd5a01a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5a0101aa0 0x7fd5a019bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 --2- 172.21.15.39:0/1108812831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5a0101150 0x7fd5a019ba50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:34.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.441+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 >> 172.21.15.39:0/1108812831 conn(0x7fd5a0076ff0 msgr2=0x7fd5a01057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:34.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.442+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 shutdown_connections 2024-07-04T08:24:34.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:34.442+0000 7fd59fe00640 1 -- 172.21.15.39:0/1108812831 wait complete. 2024-07-04T08:24:35.359 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:35 smithi039 ceph-mon[32062]: pgmap v767: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-07-04T08:24:35.359 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:35 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1108812831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:24:35.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:35 smithi050 ceph-mon[36577]: pgmap v767: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-07-04T08:24:35.386 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:35 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1108812831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:24:35.420 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 142.1 --await-for 45.0' 2024-07-04T08:24:35.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 142.1 --await-for 45.0 2024-07-04T08:24:35.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:35 smithi133 ceph-mon[37050]: pgmap v767: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 13 op/s 2024-07-04T08:24:35.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:35 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1108812831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:24:35.896 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:37.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:37 smithi039 ceph-mon[32062]: pgmap v768: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-07-04T08:24:37.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:37 smithi050 ceph-mon[36577]: pgmap v768: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-07-04T08:24:37.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:37 smithi133 ceph-mon[37050]: pgmap v768: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-07-04T08:24:37.950 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:24:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.248+0000 7f8451e00640 1 -- 172.21.15.39:0/2723903368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c10d1e0 msgr2=0x7f844c10f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.248+0000 7f8451e00640 1 --2- 172.21.15.39:0/2723903368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c10d1e0 0x7f844c10f680 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8444009be0 tx=0x7f844402a6d0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.248+0000 7f8451e00640 1 -- 172.21.15.39:0/2723903368 shutdown_connections 2024-07-04T08:24:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.248+0000 7f8451e00640 1 --2- 172.21.15.39:0/2723903368 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c10d1e0 0x7f844c10f680 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.248+0000 7f8451e00640 1 --2- 172.21.15.39:0/2723903368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f844c104660 0x7f844c10cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.248+0000 7f8451e00640 1 --2- 172.21.15.39:0/2723903368 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f844c103c30 0x7f844c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.248+0000 7f8451e00640 1 -- 172.21.15.39:0/2723903368 >> 172.21.15.39:0/2723903368 conn(0x7f844c0fd880 msgr2=0x7f844c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:38.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.249+0000 7f8451e00640 1 -- 172.21.15.39:0/2723903368 shutdown_connections 2024-07-04T08:24:38.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.249+0000 7f8451e00640 1 -- 172.21.15.39:0/2723903368 wait complete. 2024-07-04T08:24:38.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.250+0000 7f8451e00640 1 Processor -- start 2024-07-04T08:24:38.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.251+0000 7f8451e00640 1 -- start start 2024-07-04T08:24:38.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.251+0000 7f8451e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f844c103c30 0x7f844c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.251+0000 7f8451e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f844c104660 0x7f844c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.251+0000 7f8451e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c10d1e0 0x7f844c1a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.251+0000 7f8451e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f844c112df0 con 0x7f844c104660 2024-07-04T08:24:38.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.251+0000 7f8451e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f844c112c70 con 0x7f844c10d1e0 2024-07-04T08:24:38.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.251+0000 7f8451e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f844c112f70 con 0x7f844c103c30 2024-07-04T08:24:38.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.251+0000 7f8450a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f844c103c30 0x7f844c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f8450a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f844c103c30 0x7f844c19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37114/0 (socket says 172.21.15.39:37114) 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f8450a00640 1 -- 172.21.15.39:0/965772657 learned_addr learned my addr 172.21.15.39:0/965772657 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f8451400640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c10d1e0 0x7f844c1a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f844be00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f844c104660 0x7f844c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f8450a00640 1 -- 172.21.15.39:0/965772657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c10d1e0 msgr2=0x7f844c1a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f8450a00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c10d1e0 0x7f844c1a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f8450a00640 1 -- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f844c104660 msgr2=0x7f844c19bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f8450a00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f844c104660 0x7f844c19bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f8450a00640 1 -- 172.21.15.39:0/965772657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c1a1910 con 0x7f844c103c30 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.252+0000 7f8450a00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f844c103c30 0x7f844c19b9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f844000e9c0 tx=0x7f844000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.253+0000 7f8449600640 1 -- 172.21.15.39:0/965772657 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844000ccc0 con 0x7f844c103c30 2024-07-04T08:24:38.246 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.253+0000 7f8449600640 1 -- 172.21.15.39:0/965772657 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84400054d0 con 0x7f844c103c30 2024-07-04T08:24:38.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.253+0000 7f8449600640 1 -- 172.21.15.39:0/965772657 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440010430 con 0x7f844c103c30 2024-07-04T08:24:38.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.253+0000 7f8451e00640 1 -- 172.21.15.39:0/965772657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c1a1c00 con 0x7f844c103c30 2024-07-04T08:24:38.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.253+0000 7f8451e00640 1 -- 172.21.15.39:0/965772657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c1a8360 con 0x7f844c103c30 2024-07-04T08:24:38.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.257+0000 7f84599b8740 1 Processor -- start 2024-07-04T08:24:38.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.257+0000 7f84599b8740 1 -- start start 2024-07-04T08:24:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.257+0000 7f84599b8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efec0840 0x55b1f0265560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.257+0000 7f84599b8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 0x55b1f0265aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.257+0000 7f84599b8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 0x55b1f0265fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.257+0000 7f84599b8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b1f01dacc0 con 0x55b1efe9fb60 2024-07-04T08:24:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.257+0000 7f84599b8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b1f01ca260 con 0x55b1efe97980 2024-07-04T08:24:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.257+0000 7f84599b8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b1f01ce2a0 con 0x55b1efec0840 2024-07-04T08:24:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f8451400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efec0840 0x55b1f0265560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f8451400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efec0840 0x55b1f0265560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37124/0 (socket says 172.21.15.39:37124) 2024-07-04T08:24:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f8451400640 1 -- 172.21.15.39:0/1758972775 learned_addr learned my addr 172.21.15.39:0/1758972775 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f844be00640 1 --2- 172.21.15.39:0/1758972775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 0x55b1f0265fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f8449600640 1 -- 172.21.15.39:0/965772657 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8440004f80 con 0x7f844c103c30 2024-07-04T08:24:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f8451400640 1 -- 172.21.15.39:0/1758972775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 msgr2=0x55b1f0265aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:24:38.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f8451400640 1 --2- 172.21.15.39:0/1758972775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 0x55b1f0265aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f8451400640 1 -- 172.21.15.39:0/1758972775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 msgr2=0x55b1f0265fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:38.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f8451400640 1 --2- 172.21.15.39:0/1758972775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 0x55b1f0265fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.258+0000 7f8451400640 1 -- 172.21.15.39:0/1758972775 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f01616d0 con 0x55b1efec0840 2024-07-04T08:24:38.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.259+0000 7f8451400640 1 --2- 172.21.15.39:0/1758972775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efec0840 0x55b1f0265560 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f844402acc0 tx=0x7f8444030420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6a0151415120be3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:38.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.259+0000 7f843e000640 1 -- 172.21.15.39:0/1758972775 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444041030 con 0x55b1efec0840 2024-07-04T08:24:38.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.259+0000 7f843e000640 1 -- 172.21.15.39:0/1758972775 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8444003e20 con 0x55b1efec0840 2024-07-04T08:24:38.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.259+0000 7f843e000640 1 -- 172.21.15.39:0/1758972775 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444036040 con 0x55b1efec0840 2024-07-04T08:24:38.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.260+0000 7f8449600640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f84340b29d0 0x7f84340b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.260+0000 7f8449600640 1 -- 172.21.15.39:0/965772657 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f84400e2d80 con 0x7f844c103c30 2024-07-04T08:24:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.260+0000 7f84599b8740 1 -- 172.21.15.39:0/1758972775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efec0840 msgr2=0x55b1f0265560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.260+0000 7f84599b8740 1 --2- 172.21.15.39:0/1758972775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efec0840 0x55b1f0265560 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f844402acc0 tx=0x7f8444030420 comp rx=0 tx=0).stop 2024-07-04T08:24:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.261+0000 7f8450a00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f84340b29d0 0x7f84340b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.261+0000 7f84599b8740 1 -- 172.21.15.39:0/1758972775 shutdown_connections 2024-07-04T08:24:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.261+0000 7f84599b8740 1 --2- 172.21.15.39:0/1758972775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 0x55b1f0265fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.254 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.261+0000 7f84599b8740 1 --2- 172.21.15.39:0/1758972775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 0x55b1f0265aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.261+0000 7f84599b8740 1 --2- 172.21.15.39:0/1758972775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efec0840 0x55b1f0265560 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.261+0000 7f84599b8740 1 -- 172.21.15.39:0/1758972775 >> 172.21.15.39:0/1758972775 conn(0x55b1efeb4990 msgr2=0x55b1f0263450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.261+0000 7f84599b8740 1 -- 172.21.15.39:0/1758972775 shutdown_connections 2024-07-04T08:24:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.261+0000 7f84599b8740 1 -- 172.21.15.39:0/1758972775 wait complete. 2024-07-04T08:24:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.261+0000 7f8450a00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f84340b29d0 0x7f84340b4ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55b1f02641e0 tx=0x7f84400e3000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.262+0000 7f84599b8740 1 -- start start 2024-07-04T08:24:38.255 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.262+0000 7f84599b8740 1 Processor -- start 2024-07-04T08:24:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.263+0000 7f84599b8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 0x55b1f02799a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.263+0000 7f84599b8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe9fb60 0x55b1f0279f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.263+0000 7f84599b8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efec0840 0x55b1f0285740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.263+0000 7f84599b8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b1f002e7a0 con 0x55b1efec0840 2024-07-04T08:24:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.263+0000 7f84599b8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b1f01716b0 con 0x55b1efe97980 2024-07-04T08:24:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.263+0000 7f84599b8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b1f027c410 con 0x55b1efe9fb60 2024-07-04T08:24:38.256 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.263+0000 7f84599b8740 10 client.0 _ll_register_callbacks cb 0x55b1efed8090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:38.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f8451400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 0x55b1f02799a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f8451400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efec0840 0x55b1f0285740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f844be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe9fb60 0x55b1f0279f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.257 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f8451400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 0x55b1f02799a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47090/0 (socket says 172.21.15.39:47090) 2024-07-04T08:24:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f844be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe9fb60 0x55b1f0279f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37136/0 (socket says 172.21.15.39:37136) 2024-07-04T08:24:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f8451400640 1 -- 172.21.15.39:0/624051135 learned_addr learned my addr 172.21.15.39:0/624051135 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f8451400640 1 -- 172.21.15.39:0/624051135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe9fb60 msgr2=0x55b1f0279f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f8451400640 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe9fb60 0x55b1f0279f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f8451400640 1 -- 172.21.15.39:0/624051135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efec0840 msgr2=0x55b1f0285740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f8451400640 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efec0840 0x55b1f0285740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.264+0000 7f8451400640 1 -- 172.21.15.39:0/624051135 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f01621b0 con 0x55b1efe97980 2024-07-04T08:24:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.265+0000 7f8451400640 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 0x55b1f02799a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8444003af0 tx=0x7f844402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46bd0ce0b03de98d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:38.258 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.265+0000 7f8432a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:24:38.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.265+0000 7f8432a00640 1 -- 172.21.15.39:0/624051135 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444036070 con 0x55b1efe97980 2024-07-04T08:24:38.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.265+0000 7f84599b8740 1 -- 172.21.15.39:0/624051135 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8444003e20 con 0x55b1efe97980 2024-07-04T08:24:38.259 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.265+0000 7f8432a00640 1 -- 172.21.15.39:0/624051135 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f8444003b50 con 0x55b1efe97980 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f8432a00640 1 -- 172.21.15.39:0/624051135 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8444041040 con 0x55b1efe97980 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f84599b8740 10 client.27626 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f84599b8740 1 -- 172.21.15.39:0/624051135 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1eff3ab90 con 0x55b1efe97980 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f8432a00640 1 -- 172.21.15.39:0/624051135 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8444003e20 con 0x55b1efe97980 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f8432a00640 1 -- 172.21.15.39:0/624051135 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f844403d460 con 0x55b1efe97980 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f84599b8740 10 client.27626 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f84599b8740 10 client.27626 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f84599b8740 20 client.27626 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f84599b8740 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b1efe7f1c0 0x55b1f028b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f84599b8740 4 client.27626 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.266+0000 7f84599b8740 1 -- 172.21.15.39:0/624051135 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1f01dcc10 con 0x55b1efe7f1c0 2024-07-04T08:24:38.260 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.267+0000 7f844be00640 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b1efe7f1c0 0x55b1f028b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.267+0000 7f844be00640 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b1efe7f1c0 0x55b1f028b670 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15747c8acf9c0ad server_cookie=5216a50350fdd850 in_seq=0 out_seq=0 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.267+0000 7f8432a00640 10 client.27626 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.268+0000 7f8432a00640 1 -- 172.21.15.39:0/624051135 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b1f01dcc10 con 0x55b1efe7f1c0 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.268+0000 7f8432a00640 10 client.27626 handle_command_reply: tid=0 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.268+0000 7f84599b8740 1 client.27626 shutdown 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 -- 172.21.15.39:0/624051135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 msgr2=0x55b1f02799a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 0x55b1f02799a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8444003af0 tx=0x7f844402a6d0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 -- 172.21.15.39:0/624051135 shutdown_connections 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b1efe7f1c0 0x55b1f028b670 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efec0840 0x55b1f0285740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.262 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe9fb60 0x55b1f0279f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 --2- 172.21.15.39:0/624051135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe97980 0x55b1f02799a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f8432a00640 0 client.27626 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:38.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 -- 172.21.15.39:0/624051135 >> 172.21.15.39:0/624051135 conn(0x55b1efeb4990 msgr2=0x55b1f0261ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:38.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 -- 172.21.15.39:0/624051135 shutdown_connections 2024-07-04T08:24:38.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 1 -- 172.21.15.39:0/624051135 wait complete. 2024-07-04T08:24:38.263 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.269+0000 7f84599b8740 20 client.27626 trim_cache size 0 max 16384 2024-07-04T08:24:38.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.294+0000 7f84599b8740 1 Processor -- start 2024-07-04T08:24:38.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.295+0000 7f84599b8740 1 -- start start 2024-07-04T08:24:38.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.295+0000 7f84599b8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f02856b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.295+0000 7f84599b8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe9fb60 0x55b1f0285bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.295+0000 7f84599b8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efec0840 0x55b1f0265220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.295+0000 7f84599b8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1adfa0 con 0x55b1efec0840 2024-07-04T08:24:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.295+0000 7f84599b8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1afae0 con 0x55b1efe9fb60 2024-07-04T08:24:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.295+0000 7f84599b8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1ae670 con 0x55b1efe97980 2024-07-04T08:24:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.296+0000 7f844be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe9fb60 0x55b1f0285bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.296+0000 7f844be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe9fb60 0x55b1f0285bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47096/0 (socket says 172.21.15.39:47096) 2024-07-04T08:24:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.296+0000 7f844be00640 1 -- 172.21.15.39:0/273596456 learned_addr learned my addr 172.21.15.39:0/273596456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:38.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.296+0000 7f8451400640 1 --2- 172.21.15.39:0/273596456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f02856b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.296+0000 7f844be00640 1 -- 172.21.15.39:0/273596456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 msgr2=0x55b1f02856b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:38.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.296+0000 7f844be00640 1 --2- 172.21.15.39:0/273596456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f02856b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.296+0000 7f844be00640 1 -- 172.21.15.39:0/273596456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efec0840 msgr2=0x55b1f0265220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.296+0000 7f844be00640 1 --2- 172.21.15.39:0/273596456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efec0840 0x55b1f0265220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.296+0000 7f844be00640 1 -- 172.21.15.39:0/273596456 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f0278700 con 0x55b1efe9fb60 2024-07-04T08:24:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.297+0000 7f844be00640 1 --2- 172.21.15.39:0/273596456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe9fb60 0x55b1f0285bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8438000c00 tx=0x7f843800df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5810321f3e55fc89 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.298+0000 7f843d600640 1 -- 172.21.15.39:0/273596456 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438022020 con 0x55b1efe9fb60 2024-07-04T08:24:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.298+0000 7f843d600640 1 -- 172.21.15.39:0/273596456 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8438003e40 con 0x55b1efe9fb60 2024-07-04T08:24:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.298+0000 7f843d600640 1 -- 172.21.15.39:0/273596456 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84380119d0 con 0x55b1efe9fb60 2024-07-04T08:24:38.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.298+0000 7f84599b8740 1 -- 172.21.15.39:0/273596456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe9fb60 msgr2=0x55b1f0285bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.299+0000 7f84599b8740 1 --2- 172.21.15.39:0/273596456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe9fb60 0x55b1f0285bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8438000c00 tx=0x7f843800df80 comp rx=0 tx=0).stop 2024-07-04T08:24:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.299+0000 7f84599b8740 1 -- 172.21.15.39:0/273596456 shutdown_connections 2024-07-04T08:24:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.299+0000 7f84599b8740 1 --2- 172.21.15.39:0/273596456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efec0840 0x55b1f0265220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.299+0000 7f84599b8740 1 --2- 172.21.15.39:0/273596456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efe9fb60 0x55b1f0285bf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.299+0000 7f84599b8740 1 --2- 172.21.15.39:0/273596456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f02856b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.299+0000 7f84599b8740 1 -- 172.21.15.39:0/273596456 >> 172.21.15.39:0/273596456 conn(0x55b1efeb4990 msgr2=0x55b1f0262d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.299+0000 7f84599b8740 1 -- 172.21.15.39:0/273596456 shutdown_connections 2024-07-04T08:24:38.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.299+0000 7f84599b8740 1 -- 172.21.15.39:0/273596456 wait complete. 2024-07-04T08:24:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.299+0000 7f84599b8740 1 -- start start 2024-07-04T08:24:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.300+0000 7f84599b8740 1 Processor -- start 2024-07-04T08:24:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.300+0000 7f84599b8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f028ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.300+0000 7f84599b8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 0x55b1f028b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.300+0000 7f84599b8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efec0840 0x55b1f02f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.300+0000 7f84599b8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1af410 con 0x55b1efe9fb60 2024-07-04T08:24:38.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.300+0000 7f84599b8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1aed40 con 0x55b1efec0840 2024-07-04T08:24:38.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.300+0000 7f84599b8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1abfc0 con 0x55b1efe97980 2024-07-04T08:24:38.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.300+0000 7f84599b8740 10 client.0 _ll_register_callbacks cb 0x7f843800d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.301+0000 7f8451400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f028ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.301+0000 7f844be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 0x55b1f028b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.301+0000 7f8451400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f028ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37146/0 (socket says 172.21.15.39:37146) 2024-07-04T08:24:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.301+0000 7f8451400640 1 -- 172.21.15.39:0/3814121201 learned_addr learned my addr 172.21.15.39:0/3814121201 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.301+0000 7f8451400640 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efec0840 0x55b1f02f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.302+0000 7f844be00640 1 -- 172.21.15.39:0/3814121201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 msgr2=0x55b1f028ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.302+0000 7f844be00640 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f028ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.302+0000 7f844be00640 1 -- 172.21.15.39:0/3814121201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efec0840 msgr2=0x55b1f02f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:38.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.302+0000 7f844be00640 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efec0840 0x55b1f02f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.302+0000 7f844be00640 1 -- 172.21.15.39:0/3814121201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f02f3220 con 0x55b1efe9fb60 2024-07-04T08:24:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.302+0000 7f8451400640 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f028ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.302+0000 7f844be00640 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 0x55b1f028b3c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f843800e010 tx=0x7f8438011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4694e04315cc6b5d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:38.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.302+0000 7f8432a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:24:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.303+0000 7f8432a00640 1 -- 172.21.15.39:0/3814121201 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438022030 con 0x55b1efe9fb60 2024-07-04T08:24:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.303+0000 7f8432a00640 1 -- 172.21.15.39:0/3814121201 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f8438005450 con 0x55b1efe9fb60 2024-07-04T08:24:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.303+0000 7f84599b8740 1 -- 172.21.15.39:0/3814121201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8438003e40 con 0x55b1efe9fb60 2024-07-04T08:24:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.303+0000 7f8432a00640 1 -- 172.21.15.39:0/3814121201 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8438011750 con 0x55b1efe9fb60 2024-07-04T08:24:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.303+0000 7f84599b8740 10 client.17502 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.303+0000 7f84599b8740 1 -- 172.21.15.39:0/3814121201 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1f02f4460 con 0x55b1efe9fb60 2024-07-04T08:24:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.304+0000 7f8432a00640 1 -- 172.21.15.39:0/3814121201 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8438003e40 con 0x55b1efe9fb60 2024-07-04T08:24:38.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.304+0000 7f8432a00640 1 -- 172.21.15.39:0/3814121201 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8438004aa0 con 0x55b1efe9fb60 2024-07-04T08:24:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.304+0000 7f84599b8740 10 client.17502 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.304+0000 7f84599b8740 10 client.17502 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:24:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.304+0000 7f84599b8740 20 client.17502 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.304+0000 7f84599b8740 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b1efe54570 0x55b1f02f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.304+0000 7f84599b8740 4 client.17502 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.1, "await_for": 45.0}] 2024-07-04T08:24:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.304+0000 7f84599b8740 1 -- 172.21.15.39:0/3814121201 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.1, "await_for": 45.0}) -- 0x55b1f01d86b0 con 0x55b1efe54570 2024-07-04T08:24:38.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.305+0000 7f8451400640 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b1efe54570 0x55b1f02f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:38.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.305+0000 7f8451400640 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b1efe54570 0x55b1f02f6ba0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533e80530bbcecfa server_cookie=5740254e8ba35449 in_seq=0 out_seq=0 2024-07-04T08:24:38.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.305+0000 7f8432a00640 10 client.17502 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:38.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.316+0000 7f8432a00640 1 -- 172.21.15.39:0/3814121201 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b1f01d86b0 con 0x55b1efe54570 2024-07-04T08:24:38.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.316+0000 7f8432a00640 10 client.17502 handle_command_reply: tid=0 2024-07-04T08:24:38.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.316+0000 7f84599b8740 1 client.17502 shutdown 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.316+0000 7f84599b8740 1 -- 172.21.15.39:0/3814121201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 msgr2=0x55b1f028b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.316+0000 7f84599b8740 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 0x55b1f028b3c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f843800e010 tx=0x7f8438011440 comp rx=0 tx=0).stop 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.316+0000 7f84599b8740 1 -- 172.21.15.39:0/3814121201 shutdown_connections 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.316+0000 7f84599b8740 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b1efe54570 0x55b1f02f6ba0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.317+0000 7f84599b8740 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b1efe9fb60 0x55b1f028b3c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.317+0000 7f8432a00640 0 client.17502 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.317+0000 7f84599b8740 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b1efec0840 0x55b1f02f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.317+0000 7f84599b8740 1 --2- 172.21.15.39:0/3814121201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b1efe97980 0x55b1f028ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.317+0000 7f84599b8740 1 -- 172.21.15.39:0/3814121201 >> 172.21.15.39:0/3814121201 conn(0x55b1efeb4990 msgr2=0x55b1f0262d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.317+0000 7f84599b8740 1 -- 172.21.15.39:0/3814121201 shutdown_connections 2024-07-04T08:24:38.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.317+0000 7f84599b8740 1 -- 172.21.15.39:0/3814121201 wait complete. 2024-07-04T08:24:38.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.317+0000 7f84599b8740 20 client.17502 trim_cache size 0 max 16384 2024-07-04T08:24:38.311 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:24:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:24:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:24:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 26, 2024-07-04T08:24:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:24:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "34031fc7": { 2024-07-04T08:24:38.311 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 26, 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 142.1, 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:24:38.312 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:24:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:24:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:24:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:24:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:24:38.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:24:38.313 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:24:38.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.318+0000 7f8451e00640 1 -- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f84340b29d0 msgr2=0x7f84340b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.318+0000 7f8451e00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f84340b29d0 0x7f84340b4ee0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x55b1f02641e0 tx=0x7f84400e3000 comp rx=0 tx=0).stop 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.318+0000 7f8451e00640 1 -- 172.21.15.39:0/965772657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f844c103c30 msgr2=0x7f844c19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.318+0000 7f8451e00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f844c103c30 0x7f844c19b9a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f844000e9c0 tx=0x7f844000ee80 comp rx=0 tx=0).stop 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.319+0000 7f8451e00640 1 -- 172.21.15.39:0/965772657 shutdown_connections 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.319+0000 7f8451e00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f84340b29d0 0x7f84340b4ee0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.319+0000 7f8451e00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f844c104660 0x7f844c19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.319+0000 7f8451e00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f844c10d1e0 0x7f844c1a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.319+0000 7f8451e00640 1 --2- 172.21.15.39:0/965772657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f844c103c30 0x7f844c19b9a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.319+0000 7f8451e00640 1 -- 172.21.15.39:0/965772657 >> 172.21.15.39:0/965772657 conn(0x7f844c0fd880 msgr2=0x7f844c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.319+0000 7f8451e00640 1 -- 172.21.15.39:0/965772657 shutdown_connections 2024-07-04T08:24:38.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:38.319+0000 7f8451e00640 1 -- 172.21.15.39:0/965772657 wait complete. 2024-07-04T08:24:39.081 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:39.127 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '34031fc7', quiesce took 8.0 seconds. Will release after: 14.100000000000001 2024-07-04T08:24:39.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:39 smithi039 ceph-mon[32062]: pgmap v769: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-07-04T08:24:39.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:39 smithi050 ceph-mon[36577]: pgmap v769: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-07-04T08:24:39.558 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:39 smithi133 ceph-mon[37050]: pgmap v769: 129 pgs: 129 active+clean; 13 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 KiB/s wr, 17 op/s 2024-07-04T08:24:41.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:41 smithi050 ceph-mon[36577]: pgmap v770: 129 pgs: 129 active+clean; 14 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-07-04T08:24:41.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.425+0000 7f34e1a00640 1 -- 172.21.15.39:0/1331184834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc075650 msgr2=0x7f34dc10f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:41.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.425+0000 7f34e1a00640 1 --2- 172.21.15.39:0/1331184834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc075650 0x7f34dc10f0e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009ca0 tx=0x7f34c402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.426+0000 7f34e1a00640 1 -- 172.21.15.39:0/1331184834 shutdown_connections 2024-07-04T08:24:41.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.426+0000 7f34e1a00640 1 --2- 172.21.15.39:0/1331184834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34dc10f620 0x7f34dc111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.426+0000 7f34e1a00640 1 --2- 172.21.15.39:0/1331184834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc075650 0x7f34dc10f0e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.426+0000 7f34e1a00640 1 --2- 172.21.15.39:0/1331184834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34dc074d00 0x7f34dc075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.426+0000 7f34e1a00640 1 -- 172.21.15.39:0/1331184834 >> 172.21.15.39:0/1331184834 conn(0x7f34dc0feb30 msgr2=0x7f34dc100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.426+0000 7f34e1a00640 1 -- 172.21.15.39:0/1331184834 shutdown_connections 2024-07-04T08:24:41.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.426+0000 7f34e1a00640 1 -- 172.21.15.39:0/1331184834 wait complete. 2024-07-04T08:24:41.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:41 smithi039 ceph-mon[32062]: pgmap v770: 129 pgs: 129 active+clean; 14 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-07-04T08:24:41.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.427+0000 7f34e1a00640 1 Processor -- start 2024-07-04T08:24:41.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.427+0000 7f34e1a00640 1 -- start start 2024-07-04T08:24:41.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.428+0000 7f34e1a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34dc074d00 0x7f34dc19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.428+0000 7f34e1a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc075650 0x7f34dc19c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.428+0000 7f34e1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34dc10f620 0x7f34dc1a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.428+0000 7f34e1a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1151f0 con 0x7f34dc074d00 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.428+0000 7f34e1a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc115070 con 0x7f34dc075650 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.428+0000 7f34e1a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc115370 con 0x7f34dc10f620 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.428+0000 7f34daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34dc074d00 0x7f34dc19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.428+0000 7f34daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34dc074d00 0x7f34dc19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56154/0 (socket says 172.21.15.39:56154) 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.428+0000 7f34daa00640 1 -- 172.21.15.39:0/3998786126 learned_addr learned my addr 172.21.15.39:0/3998786126 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.429+0000 7f34da000640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc075650 0x7f34dc19c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.429+0000 7f34db400640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34dc10f620 0x7f34dc1a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.429+0000 7f34da000640 1 -- 172.21.15.39:0/3998786126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34dc10f620 msgr2=0x7f34dc1a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:41.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.429+0000 7f34da000640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34dc10f620 0x7f34dc1a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.429+0000 7f34da000640 1 -- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34dc074d00 msgr2=0x7f34dc19bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.429+0000 7f34da000640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34dc074d00 0x7f34dc19bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.429+0000 7f34da000640 1 -- 172.21.15.39:0/3998786126 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc1a1b50 con 0x7f34dc075650 2024-07-04T08:24:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.429+0000 7f34daa00640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34dc074d00 0x7f34dc19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:41.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.430+0000 7f34da000640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc075650 0x7f34dc19c180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f34c402aeb0 tx=0x7f34c4031750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:41.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.430+0000 7f34d3400640 1 -- 172.21.15.39:0/3998786126 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4003b70 con 0x7f34dc075650 2024-07-04T08:24:41.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.430+0000 7f34d3400640 1 -- 172.21.15.39:0/3998786126 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34c4003d10 con 0x7f34dc075650 2024-07-04T08:24:41.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.430+0000 7f34d3400640 1 -- 172.21.15.39:0/3998786126 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c402fa30 con 0x7f34dc075650 2024-07-04T08:24:41.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.430+0000 7f34e1a00640 1 -- 172.21.15.39:0/3998786126 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc1a1de0 con 0x7f34dc075650 2024-07-04T08:24:41.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.430+0000 7f34e1a00640 1 -- 172.21.15.39:0/3998786126 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc1a85a0 con 0x7f34dc075650 2024-07-04T08:24:41.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.433+0000 7f34e805b740 1 Processor -- start 2024-07-04T08:24:41.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34e805b740 1 -- start start 2024-07-04T08:24:41.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34d3400640 1 -- 172.21.15.39:0/3998786126 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f34c4004580 con 0x7f34dc075650 2024-07-04T08:24:41.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34e805b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c418127a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34e805b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c414451b0 0x558c41812ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34e805b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4144d390 0x558c41813220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34e805b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558c41788ed0 con 0x558c4146e070 2024-07-04T08:24:41.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34e805b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c41778470 con 0x558c4144d390 2024-07-04T08:24:41.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34e805b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558c4177c4b0 con 0x558c414451b0 2024-07-04T08:24:41.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c418127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c418127a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56168/0 (socket says 172.21.15.39:56168) 2024-07-04T08:24:41.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34daa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c414451b0 0x558c41812ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.434+0000 7f34db400640 1 -- 172.21.15.39:0/2611539391 learned_addr learned my addr 172.21.15.39:0/2611539391 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:41.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.435+0000 7f34da000640 1 --2- 172.21.15.39:0/2611539391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4144d390 0x558c41813220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.435+0000 7f34daa00640 1 -- 172.21.15.39:0/2611539391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4144d390 msgr2=0x558c41813220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:41.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.435+0000 7f34daa00640 1 --2- 172.21.15.39:0/2611539391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4144d390 0x558c41813220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.435+0000 7f34daa00640 1 -- 172.21.15.39:0/2611539391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 msgr2=0x558c418127a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:41.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.435+0000 7f34daa00640 1 --2- 172.21.15.39:0/2611539391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c418127a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.435+0000 7f34daa00640 1 -- 172.21.15.39:0/2611539391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c41710640 con 0x558c414451b0 2024-07-04T08:24:41.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.435+0000 7f34daa00640 1 --2- 172.21.15.39:0/2611539391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c414451b0 0x558c41812ce0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0114f0 tx=0x7f34cc0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25efa12a96e8500 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.436+0000 7f34cbe00640 1 -- 172.21.15.39:0/2611539391 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc01e070 con 0x558c414451b0 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.436+0000 7f34cbe00640 1 -- 172.21.15.39:0/2611539391 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34cc0035f0 con 0x558c414451b0 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.436+0000 7f34cbe00640 1 -- 172.21.15.39:0/2611539391 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc00f7c0 con 0x558c414451b0 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.436+0000 7f34d3400640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b2960 0x7f34bc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34d3400640 1 -- 172.21.15.39:0/3998786126 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f34c4103b30 con 0x7f34dc075650 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34e805b740 1 -- 172.21.15.39:0/2611539391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c414451b0 msgr2=0x558c41812ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34e805b740 1 --2- 172.21.15.39:0/2611539391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c414451b0 0x558c41812ce0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0114f0 tx=0x7f34cc0119b0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34db400640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b2960 0x7f34bc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34e805b740 1 -- 172.21.15.39:0/2611539391 shutdown_connections 2024-07-04T08:24:41.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34e805b740 1 --2- 172.21.15.39:0/2611539391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4144d390 0x558c41813220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34e805b740 1 --2- 172.21.15.39:0/2611539391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c414451b0 0x558c41812ce0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34e805b740 1 --2- 172.21.15.39:0/2611539391 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c418127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34e805b740 1 -- 172.21.15.39:0/2611539391 >> 172.21.15.39:0/2611539391 conn(0x558c41462460 msgr2=0x558c41810690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:41.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34e805b740 1 -- 172.21.15.39:0/2611539391 shutdown_connections 2024-07-04T08:24:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.437+0000 7f34e805b740 1 -- 172.21.15.39:0/2611539391 wait complete. 2024-07-04T08:24:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.438+0000 7f34db400640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b2960 0x7f34bc0b4e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009ea0 tx=0x7f34d4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.438+0000 7f34e805b740 1 -- start start 2024-07-04T08:24:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.438+0000 7f34e805b740 1 Processor -- start 2024-07-04T08:24:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34e805b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c414451b0 0x558c41827210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34e805b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c418277f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34e805b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4146e070 0x558c4182c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34e805b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558c415dc700 con 0x558c414451b0 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34e805b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558c4171fba0 con 0x558c4146e070 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34e805b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558c41820f80 con 0x558c4144d390 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34e805b740 10 client.0 _ll_register_callbacks cb 0x558c41771560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c414451b0 0x558c41827210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34db400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4146e070 0x558c4182c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c414451b0 0x558c41827210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56182/0 (socket says 172.21.15.39:56182) 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34da000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c418277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34db400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4146e070 0x558c4182c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:42058/0 (socket says 172.21.15.39:42058) 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.439+0000 7f34daa00640 1 -- 172.21.15.39:0/4028011923 learned_addr learned my addr 172.21.15.39:0/4028011923 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.440+0000 7f34db400640 1 -- 172.21.15.39:0/4028011923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 msgr2=0x558c418277f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.440+0000 7f34db400640 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c418277f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.440+0000 7f34db400640 1 -- 172.21.15.39:0/4028011923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c414451b0 msgr2=0x558c41827210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.440+0000 7f34db400640 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c414451b0 0x558c41827210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.440+0000 7f34db400640 1 -- 172.21.15.39:0/4028011923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c414d9650 con 0x558c4146e070 2024-07-04T08:24:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.440+0000 7f34db400640 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4146e070 0x558c4182c810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f34d401e9d0 tx=0x7f34d4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7493e49f6f9e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.440+0000 7f34c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:24:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.440+0000 7f34c8c00640 1 -- 172.21.15.39:0/4028011923 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4005020 con 0x558c4146e070 2024-07-04T08:24:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.440+0000 7f34e805b740 1 -- 172.21.15.39:0/4028011923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34cc0035f0 con 0x558c4146e070 2024-07-04T08:24:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.441+0000 7f34c8c00640 1 -- 172.21.15.39:0/4028011923 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f34d4018030 con 0x558c4146e070 2024-07-04T08:24:41.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.441+0000 7f34c8c00640 1 -- 172.21.15.39:0/4028011923 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34d4034050 con 0x558c4146e070 2024-07-04T08:24:41.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.441+0000 7f34e805b740 10 client.27644 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:41.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.441+0000 7f34e805b740 1 -- 172.21.15.39:0/4028011923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c41835810 con 0x558c4146e070 2024-07-04T08:24:41.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.441+0000 7f34c8c00640 1 -- 172.21.15.39:0/4028011923 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34cc0035f0 con 0x558c4146e070 2024-07-04T08:24:41.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.442+0000 7f34c8c00640 1 -- 172.21.15.39:0/4028011923 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f34d4029420 con 0x558c4146e070 2024-07-04T08:24:41.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.442+0000 7f34e805b740 10 client.27644 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:41.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.442+0000 7f34e805b740 10 client.27644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:24:41.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.442+0000 7f34e805b740 20 client.27644 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.442+0000 7f34e805b740 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558c4142c9f0 0x558c41838db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.442+0000 7f34e805b740 4 client.27644 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.442+0000 7f34e805b740 1 -- 172.21.15.39:0/4028011923 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c4178ae20 con 0x558c4142c9f0 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.442+0000 7f34daa00640 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558c4142c9f0 0x558c41838db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.443+0000 7f34daa00640 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558c4142c9f0 0x558c41838db0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b2687dcf0cc489 server_cookie=ecea86b97ebfa346 in_seq=0 out_seq=0 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.443+0000 7f34c8c00640 10 client.27644 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34c8c00640 1 -- 172.21.15.39:0/4028011923 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558c4178ae20 con 0x558c4142c9f0 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34c8c00640 10 client.27644 handle_command_reply: tid=0 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 client.27644 shutdown 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 -- 172.21.15.39:0/4028011923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4146e070 msgr2=0x558c4182c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4146e070 0x558c4182c810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f34d401e9d0 tx=0x7f34d4004730 comp rx=0 tx=0).stop 2024-07-04T08:24:41.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 -- 172.21.15.39:0/4028011923 shutdown_connections 2024-07-04T08:24:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558c4142c9f0 0x558c41838db0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c4146e070 0x558c4182c810 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c418277f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 --2- 172.21.15.39:0/4028011923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c414451b0 0x558c41827210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 -- 172.21.15.39:0/4028011923 >> 172.21.15.39:0/4028011923 conn(0x558c41462460 msgr2=0x558c41815980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34c8c00640 0 client.27644 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 -- 172.21.15.39:0/4028011923 shutdown_connections 2024-07-04T08:24:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.444+0000 7f34e805b740 1 -- 172.21.15.39:0/4028011923 wait complete. 2024-07-04T08:24:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.445+0000 7f34e805b740 20 client.27644 trim_cache size 0 max 16384 2024-07-04T08:24:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.465+0000 7f34e805b740 1 Processor -- start 2024-07-04T08:24:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.465+0000 7f34e805b740 1 -- start start 2024-07-04T08:24:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.465+0000 7f34e805b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c418202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.465+0000 7f34e805b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c4182afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.465+0000 7f34e805b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c4182bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.458 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.465+0000 7f34e805b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1ae120 con 0x558c4146e070 2024-07-04T08:24:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.465+0000 7f34e805b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1afc60 con 0x558c414451b0 2024-07-04T08:24:41.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.465+0000 7f34e805b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1ae7f0 con 0x558c4144d390 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c418202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c418202a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:42072/0 (socket says 172.21.15.39:42072) 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34daa00640 1 -- 172.21.15.39:0/4199862736 learned_addr learned my addr 172.21.15.39:0/4199862736 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34db400640 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c4182bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34da000640 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c4182afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34daa00640 1 -- 172.21.15.39:0/4199862736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 msgr2=0x558c4182afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34daa00640 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c4182afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34daa00640 1 -- 172.21.15.39:0/4199862736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 msgr2=0x558c4182bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34daa00640 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c4182bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34daa00640 1 -- 172.21.15.39:0/4199862736 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c41827b60 con 0x558c414451b0 2024-07-04T08:24:41.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.466+0000 7f34da000640 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c4182afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:24:41.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.467+0000 7f34daa00640 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c418202a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f34cc003fd0 tx=0x7f34cc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2553d7a5b196afed server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:41.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.467+0000 7f34cb400640 1 -- 172.21.15.39:0/4199862736 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc01e040 con 0x558c414451b0 2024-07-04T08:24:41.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.467+0000 7f34cb400640 1 -- 172.21.15.39:0/4199862736 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34cc01f040 con 0x558c414451b0 2024-07-04T08:24:41.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.467+0000 7f34cb400640 1 -- 172.21.15.39:0/4199862736 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc005410 con 0x558c414451b0 2024-07-04T08:24:41.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 -- 172.21.15.39:0/4199862736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 msgr2=0x558c418202a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:41.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c418202a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f34cc003fd0 tx=0x7f34cc005fd0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 -- 172.21.15.39:0/4199862736 shutdown_connections 2024-07-04T08:24:41.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c4182bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c4182afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 --2- 172.21.15.39:0/4199862736 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c418202a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 -- 172.21.15.39:0/4199862736 >> 172.21.15.39:0/4199862736 conn(0x558c41462460 msgr2=0x558c4182b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 -- 172.21.15.39:0/4199862736 shutdown_connections 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 -- 172.21.15.39:0/4199862736 wait complete. 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.468+0000 7f34e805b740 1 -- start start 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.469+0000 7f34e805b740 1 Processor -- start 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.469+0000 7f34e805b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c41833240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.469+0000 7f34e805b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c4182a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.469+0000 7f34e805b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c4182a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.469+0000 7f34e805b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1af590 con 0x558c4146e070 2024-07-04T08:24:41.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.469+0000 7f34e805b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1aeec0 con 0x558c414451b0 2024-07-04T08:24:41.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.469+0000 7f34e805b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1ac160 con 0x558c4144d390 2024-07-04T08:24:41.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.469+0000 7f34e805b740 10 client.0 _ll_register_callbacks cb 0x7f34d4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:41.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c4182a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c41833240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c4182a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56204/0 (socket says 172.21.15.39:56204) 2024-07-04T08:24:41.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34db400640 1 -- 172.21.15.39:0/511027311 learned_addr learned my addr 172.21.15.39:0/511027311 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:41.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34da000640 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c4182a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34daa00640 1 -- 172.21.15.39:0/511027311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 msgr2=0x558c4182a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:41.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34daa00640 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c4182a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34daa00640 1 -- 172.21.15.39:0/511027311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 msgr2=0x558c4182a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34daa00640 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c4182a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34daa00640 1 -- 172.21.15.39:0/511027311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c4182aec0 con 0x558c414451b0 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34daa00640 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c41833240 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34cc003fd0 tx=0x7f34cc005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0c717b99adbdde server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.470+0000 7f34c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.471+0000 7f34c8c00640 1 -- 172.21.15.39:0/511027311 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc01e070 con 0x558c414451b0 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.471+0000 7f34c8c00640 1 -- 172.21.15.39:0/511027311 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f34cc0053a0 con 0x558c414451b0 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.471+0000 7f34e805b740 1 -- 172.21.15.39:0/511027311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34cc01f040 con 0x558c414451b0 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.471+0000 7f34c8c00640 1 -- 172.21.15.39:0/511027311 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34cc03f050 con 0x558c414451b0 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.471+0000 7f34e805b740 10 client.27650 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.471+0000 7f34e805b740 1 -- 172.21.15.39:0/511027311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558c4182f930 con 0x558c414451b0 2024-07-04T08:24:41.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.472+0000 7f34c8c00640 1 -- 172.21.15.39:0/511027311 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34cc01f040 con 0x558c414451b0 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.472+0000 7f34c8c00640 1 -- 172.21.15.39:0/511027311 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f34cc004750 con 0x558c414451b0 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.472+0000 7f34e805b740 10 client.27650 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.472+0000 7f34e805b740 10 client.27650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.472+0000 7f34e805b740 20 client.27650 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.472+0000 7f34e805b740 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558c41401da0 0x558c418a3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.472+0000 7f34e805b740 4 client.27650 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.472+0000 7f34e805b740 1 -- 172.21.15.39:0/511027311 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c417868c0 con 0x558c41401da0 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.472+0000 7f34da000640 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558c41401da0 0x558c418a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34da000640 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558c41401da0 0x558c418a3820 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df81db940ee9ec2e server_cookie=8ae1682c7d793c5d in_seq=0 out_seq=0 2024-07-04T08:24:41.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34c8c00640 10 client.27650 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:41.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34c8c00640 1 -- 172.21.15.39:0/511027311 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f34c4109050 con 0x558c41401da0 2024-07-04T08:24:41.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34c8c00640 10 client.27650 handle_command_reply: tid=0 2024-07-04T08:24:41.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34e805b740 1 client.27650 shutdown 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34e805b740 1 -- 172.21.15.39:0/511027311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 msgr2=0x558c41833240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34e805b740 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c41833240 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34cc003fd0 tx=0x7f34cc005c50 comp rx=0 tx=0).stop 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34e805b740 1 -- 172.21.15.39:0/511027311 shutdown_connections 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34e805b740 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558c41401da0 0x558c418a3820 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34e805b740 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558c4146e070 0x558c4182a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.473+0000 7f34e805b740 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c4144d390 0x558c4182a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.474+0000 7f34e805b740 1 --2- 172.21.15.39:0/511027311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558c414451b0 0x558c41833240 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.474+0000 7f34e805b740 1 -- 172.21.15.39:0/511027311 >> 172.21.15.39:0/511027311 conn(0x558c41462460 msgr2=0x558c4182be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.474+0000 7f34c8c00640 0 client.27650 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.474+0000 7f34e805b740 1 -- 172.21.15.39:0/511027311 shutdown_connections 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.474+0000 7f34e805b740 1 -- 172.21.15.39:0/511027311 wait complete. 2024-07-04T08:24:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.474+0000 7f34e805b740 20 client.27650 trim_cache size 0 max 16384 2024-07-04T08:24:41.470 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:24:41.470 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:24:41.470 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:24:41.470 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 -- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b2960 msgr2=0x7f34bc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b2960 0x7f34bc0b4e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009ea0 tx=0x7f34d4005f90 comp rx=0 tx=0).stop 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 -- 172.21.15.39:0/3998786126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc075650 msgr2=0x7f34dc19c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc075650 0x7f34dc19c180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f34c402aeb0 tx=0x7f34c4031750 comp rx=0 tx=0).stop 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 -- 172.21.15.39:0/3998786126 shutdown_connections 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34bc0b2960 0x7f34bc0b4e70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34dc10f620 0x7f34dc1a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34dc075650 0x7f34dc19c180 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 --2- 172.21.15.39:0/3998786126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34dc074d00 0x7f34dc19bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 -- 172.21.15.39:0/3998786126 >> 172.21.15.39:0/3998786126 conn(0x7f34dc0feb30 msgr2=0x7f34dc1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 -- 172.21.15.39:0/3998786126 shutdown_connections 2024-07-04T08:24:41.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:41.475+0000 7f34e1a00640 1 -- 172.21.15.39:0/3998786126 wait complete. 2024-07-04T08:24:41.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:41 smithi133 ceph-mon[37050]: pgmap v770: 129 pgs: 129 active+clean; 14 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 16 op/s 2024-07-04T08:24:42.464 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:24:42.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:24:42.964 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:43.373 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:43 smithi039 ceph-mon[32062]: pgmap v771: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-07-04T08:24:43.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:43 smithi050 ceph-mon[36577]: pgmap v771: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-07-04T08:24:43.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:43 smithi133 ceph-mon[37050]: pgmap v771: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 17 op/s 2024-07-04T08:24:45.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:45 smithi039 ceph-mon[32062]: pgmap v772: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-07-04T08:24:45.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.393+0000 7f27e9800640 1 -- 172.21.15.39:0/3913646394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4108a40 msgr2=0x7f27e4108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.393+0000 7f27e9800640 1 --2- 172.21.15.39:0/3913646394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4108a40 0x7f27e4108e50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009900 tx=0x7f27cc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.393+0000 7f27e9800640 1 -- 172.21.15.39:0/3913646394 shutdown_connections 2024-07-04T08:24:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.393+0000 7f27e9800640 1 --2- 172.21.15.39:0/3913646394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4108a40 0x7f27e4108e50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.393+0000 7f27e9800640 1 --2- 172.21.15.39:0/3913646394 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27e4109390 0x7f27e410fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.393+0000 7f27e9800640 1 --2- 172.21.15.39:0/3913646394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e4103100 0x7f27e4103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.393+0000 7f27e9800640 1 -- 172.21.15.39:0/3913646394 >> 172.21.15.39:0/3913646394 conn(0x7f27e40feb50 msgr2=0x7f27e4100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.393+0000 7f27e9800640 1 -- 172.21.15.39:0/3913646394 shutdown_connections 2024-07-04T08:24:45.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.394+0000 7f27e9800640 1 -- 172.21.15.39:0/3913646394 wait complete. 2024-07-04T08:24:45.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.395+0000 7f27e9800640 1 Processor -- start 2024-07-04T08:24:45.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.395+0000 7f27e9800640 1 -- start start 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4103100 0x7f27e4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e4108a40 0x7f27e4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e9800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27e4109390 0x7f27e419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e9800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4113000 con 0x7f27e4103100 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e9800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4112e80 con 0x7f27e4109390 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e9800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4113180 con 0x7f27e4108a40 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4103100 0x7f27e4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4103100 0x7f27e4199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56210/0 (socket says 172.21.15.39:56210) 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e2a00640 1 -- 172.21.15.39:0/968423606 learned_addr learned my addr 172.21.15.39:0/968423606 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:45.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27dbe00640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e4108a40 0x7f27e4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:45.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.396+0000 7f27e3400640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27e4109390 0x7f27e419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:45.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.397+0000 7f27e2a00640 1 -- 172.21.15.39:0/968423606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e4108a40 msgr2=0x7f27e4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.397+0000 7f27e2a00640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e4108a40 0x7f27e4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.397+0000 7f27e2a00640 1 -- 172.21.15.39:0/968423606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27e4109390 msgr2=0x7f27e419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.397+0000 7f27e2a00640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27e4109390 0x7f27e419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.397+0000 7f27e2a00640 1 -- 172.21.15.39:0/968423606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e419f9c0 con 0x7f27e4103100 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.397+0000 7f27e3400640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27e4109390 0x7f27e419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.397+0000 7f27dbe00640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e4108a40 0x7f27e4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.397+0000 7f27e2a00640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4103100 0x7f27e4199a50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f27d400e9f0 tx=0x7f27d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.398+0000 7f27db400640 1 -- 172.21.15.39:0/968423606 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d400cd80 con 0x7f27e4103100 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.398+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e419fcb0 con 0x7f27e4103100 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.398+0000 7f27db400640 1 -- 172.21.15.39:0/968423606 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d40108e0 con 0x7f27e4103100 2024-07-04T08:24:45.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.398+0000 7f27db400640 1 -- 172.21.15.39:0/968423606 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4018980 con 0x7f27e4103100 2024-07-04T08:24:45.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.398+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e41a6410 con 0x7f27e4103100 2024-07-04T08:24:45.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.399+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e4103510 con 0x7f27e4103100 2024-07-04T08:24:45.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.401+0000 7f27db400640 1 -- 172.21.15.39:0/968423606 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f27d4010430 con 0x7f27e4103100 2024-07-04T08:24:45.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.403+0000 7f27db400640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f27c80b2960 0x7f27c80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:45.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.403+0000 7f27db400640 1 -- 172.21.15.39:0/968423606 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f27d40e2db0 con 0x7f27e4103100 2024-07-04T08:24:45.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.403+0000 7f27dbe00640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f27c80b2960 0x7f27c80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:45.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.404+0000 7f27dbe00640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f27c80b2960 0x7f27c80b4e70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009230 tx=0x7f27cc031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:45.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.407+0000 7f27db400640 1 -- 172.21.15.39:0/968423606 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f27d40e7070 con 0x7f27e4103100 2024-07-04T08:24:45.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:45 smithi133 ceph-mon[37050]: pgmap v772: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-07-04T08:24:45.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:45 smithi050 ceph-mon[36577]: pgmap v772: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-07-04T08:24:45.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.669+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27e419ab80 con 0x7f27e4103100 2024-07-04T08:24:45.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.672+0000 7f27db400640 1 -- 172.21.15.39:0/968423606 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f27d40aa3d0 con 0x7f27e4103100 2024-07-04T08:24:45.664 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:24:45.664 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:24:45.665 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:24:45.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.675+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f27c80b2960 msgr2=0x7f27c80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:45.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.675+0000 7f27e9800640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f27c80b2960 0x7f27c80b4e70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009230 tx=0x7f27cc031400 comp rx=0 tx=0).stop 2024-07-04T08:24:45.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.675+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4103100 msgr2=0x7f27e4199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:45.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.675+0000 7f27e9800640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4103100 0x7f27e4199a50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f27d400e9f0 tx=0x7f27d400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.676+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 shutdown_connections 2024-07-04T08:24:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.676+0000 7f27e9800640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f27c80b2960 0x7f27c80b4e70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.676+0000 7f27e9800640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27e4109390 0x7f27e419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.676+0000 7f27e9800640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27e4108a40 0x7f27e4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.676+0000 7f27e9800640 1 --2- 172.21.15.39:0/968423606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27e4103100 0x7f27e4199a50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.676+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 >> 172.21.15.39:0/968423606 conn(0x7f27e40feb50 msgr2=0x7f27e4100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.676+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 shutdown_connections 2024-07-04T08:24:45.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:45.676+0000 7f27e9800640 1 -- 172.21.15.39:0/968423606 wait complete. 2024-07-04T08:24:46.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:46 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/968423606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:24:46.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:46 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/968423606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:24:46.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:24:46.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:46 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/968423606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:24:47.017 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:47.443 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:47 smithi039 ceph-mon[32062]: pgmap v773: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-07-04T08:24:47.458 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:47 smithi133 ceph-mon[37050]: pgmap v773: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-07-04T08:24:47.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:47 smithi050 ceph-mon[36577]: pgmap v773: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-07-04T08:24:49.269 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:49 smithi039 ceph-mon[32062]: pgmap v774: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-07-04T08:24:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.494+0000 7fb465800640 1 -- 172.21.15.39:0/4186436539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46010f990 msgr2=0x7fb4601161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:49.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.494+0000 7fb465800640 1 --2- 172.21.15.39:0/4186436539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46010f990 0x7fb4601161c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb458009ca0 tx=0x7fb45802a8b0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.494+0000 7fb465800640 1 -- 172.21.15.39:0/4186436539 shutdown_connections 2024-07-04T08:24:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.494+0000 7fb465800640 1 --2- 172.21.15.39:0/4186436539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46010f990 0x7fb4601161c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.494+0000 7fb465800640 1 --2- 172.21.15.39:0/4186436539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46010f040 0x7fb46010f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.494+0000 7fb465800640 1 --2- 172.21.15.39:0/4186436539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb460109700 0x7fb460109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.494+0000 7fb465800640 1 -- 172.21.15.39:0/4186436539 >> 172.21.15.39:0/4186436539 conn(0x7fb460077630 msgr2=0x7fb460107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.495+0000 7fb465800640 1 -- 172.21.15.39:0/4186436539 shutdown_connections 2024-07-04T08:24:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.495+0000 7fb465800640 1 -- 172.21.15.39:0/4186436539 wait complete. 2024-07-04T08:24:49.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.496+0000 7fb465800640 1 Processor -- start 2024-07-04T08:24:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.496+0000 7fb465800640 1 -- start start 2024-07-04T08:24:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb465800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460109700 0x7fb4601a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb465800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46010f040 0x7fb4601a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb465800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46010f990 0x7fb4601a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb465800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb460119640 con 0x7fb46010f990 2024-07-04T08:24:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb465800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601194c0 con 0x7fb46010f040 2024-07-04T08:24:49.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb465800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601197c0 con 0x7fb460109700 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb45ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460109700 0x7fb4601a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb457e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46010f040 0x7fb4601a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb45ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460109700 0x7fb4601a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36918/0 (socket says 172.21.15.39:36918) 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb45f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46010f990 0x7fb4601a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb45ea00640 1 -- 172.21.15.39:0/3363120364 learned_addr learned my addr 172.21.15.39:0/3363120364 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.497+0000 7fb457e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46010f040 0x7fb4601a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:42102/0 (socket says 172.21.15.39:42102) 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb45ea00640 1 -- 172.21.15.39:0/3363120364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46010f040 msgr2=0x7fb4601a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb45ea00640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46010f040 0x7fb4601a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb45ea00640 1 -- 172.21.15.39:0/3363120364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46010f990 msgr2=0x7fb4601a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb45ea00640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46010f990 0x7fb4601a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb45ea00640 1 -- 172.21.15.39:0/3363120364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4601a5fa0 con 0x7fb460109700 2024-07-04T08:24:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb45ea00640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460109700 0x7fb4601a0030 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb45000e9f0 tx=0x7fb45000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb457400640 1 -- 172.21.15.39:0/3363120364 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45000cd80 con 0x7fb460109700 2024-07-04T08:24:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb457400640 1 -- 172.21.15.39:0/3363120364 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4500108e0 con 0x7fb460109700 2024-07-04T08:24:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb457400640 1 -- 172.21.15.39:0/3363120364 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450018980 con 0x7fb460109700 2024-07-04T08:24:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.498+0000 7fb465800640 1 -- 172.21.15.39:0/3363120364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4601a6290 con 0x7fb460109700 2024-07-04T08:24:49.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.499+0000 7fb465800640 1 -- 172.21.15.39:0/3363120364 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4601ac9f0 con 0x7fb460109700 2024-07-04T08:24:49.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.502+0000 7fb46bebb740 1 Processor -- start 2024-07-04T08:24:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.502+0000 7fb46bebb740 1 -- start start 2024-07-04T08:24:49.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 0x55c073b577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b57cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b58230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb46bebb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c073acdee0 con 0x55c073792390 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb46bebb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c073abd480 con 0x55c0737b3070 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb46bebb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c073ac14c0 con 0x55c07378a1b0 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb457400640 1 -- 172.21.15.39:0/3363120364 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb450010430 con 0x7fb460109700 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb457e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b58230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb45f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 0x55c073b577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb457e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b58230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56240/0 (socket says 172.21.15.39:56240) 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb457e00640 1 -- 172.21.15.39:0/3260679646 learned_addr learned my addr 172.21.15.39:0/3260679646 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:49.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.503+0000 7fb45ea00640 1 --2- 172.21.15.39:0/3260679646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b57cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb45f400640 1 -- 172.21.15.39:0/3260679646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 msgr2=0x55c073b57cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:49.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb45f400640 1 --2- 172.21.15.39:0/3260679646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b57cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb45f400640 1 -- 172.21.15.39:0/3260679646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 msgr2=0x55c073b58230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:49.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb45f400640 1 --2- 172.21.15.39:0/3260679646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b58230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb45f400640 1 -- 172.21.15.39:0/3260679646 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c073a55650 con 0x55c0737b3070 2024-07-04T08:24:49.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb45ea00640 1 --2- 172.21.15.39:0/3260679646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b57cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:24:49.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb45f400640 1 --2- 172.21.15.39:0/3260679646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 0x55c073b577b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb458005e30 tx=0x7fb458009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f35dd9d15fa09b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:49.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb44fe00640 1 -- 172.21.15.39:0/3260679646 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4580305a0 con 0x55c0737b3070 2024-07-04T08:24:49.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb44fe00640 1 -- 172.21.15.39:0/3260679646 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb458036030 con 0x55c0737b3070 2024-07-04T08:24:49.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.504+0000 7fb44fe00640 1 -- 172.21.15.39:0/3260679646 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4580042a0 con 0x55c0737b3070 2024-07-04T08:24:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.505+0000 7fb46bebb740 1 -- 172.21.15.39:0/3260679646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 msgr2=0x55c073b577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.505+0000 7fb46bebb740 1 --2- 172.21.15.39:0/3260679646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 0x55c073b577b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb458005e30 tx=0x7fb458009800 comp rx=0 tx=0).stop 2024-07-04T08:24:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.505+0000 7fb457400640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4440b29e0 0x7fb4440b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.506+0000 7fb46bebb740 1 -- 172.21.15.39:0/3260679646 shutdown_connections 2024-07-04T08:24:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.506+0000 7fb46bebb740 1 --2- 172.21.15.39:0/3260679646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b58230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.506+0000 7fb457400640 1 -- 172.21.15.39:0/3363120364 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7fb4500e2c90 con 0x7fb460109700 2024-07-04T08:24:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.506+0000 7fb46bebb740 1 --2- 172.21.15.39:0/3260679646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b57cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.506+0000 7fb46bebb740 1 --2- 172.21.15.39:0/3260679646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 0x55c073b577b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.506+0000 7fb46bebb740 1 -- 172.21.15.39:0/3260679646 >> 172.21.15.39:0/3260679646 conn(0x55c0737a7460 msgr2=0x55c073b556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.506+0000 7fb45f400640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4440b29e0 0x7fb4440b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.506+0000 7fb46bebb740 1 -- 172.21.15.39:0/3260679646 shutdown_connections 2024-07-04T08:24:49.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.506+0000 7fb46bebb740 1 -- 172.21.15.39:0/3260679646 wait complete. 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.507+0000 7fb45f400640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4440b29e0 0x7fb4440b4ef0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb4580094e0 tx=0x7fb458003f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.507+0000 7fb46bebb740 1 -- start start 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.507+0000 7fb46bebb740 1 Processor -- start 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.507+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b63350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.507+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b63890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 0x55c073b6f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb46bebb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c073921700 con 0x55c073792390 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb46bebb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c073a64bb0 con 0x55c0737b3070 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb46bebb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c073b65f30 con 0x55c07378a1b0 2024-07-04T08:24:49.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb46bebb740 10 client.0 _ll_register_callbacks cb 0x55c073ab6570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb45ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b63350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb45ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b63350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36932/0 (socket says 172.21.15.39:36932) 2024-07-04T08:24:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb45ea00640 1 -- 172.21.15.39:0/1046271610 learned_addr learned my addr 172.21.15.39:0/1046271610 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb45f400640 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 0x55c073b6f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb457e00640 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b63890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb45ea00640 1 -- 172.21.15.39:0/1046271610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 msgr2=0x55c073b6f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb45ea00640 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 0x55c073b6f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb45ea00640 1 -- 172.21.15.39:0/1046271610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 msgr2=0x55c073b63890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:49.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb45ea00640 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b63890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.508+0000 7fb45ea00640 1 -- 172.21.15.39:0/1046271610 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c07381e650 con 0x55c07378a1b0 2024-07-04T08:24:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.509+0000 7fb45ea00640 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b63350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb450000c00 tx=0x7fb4500a93d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9810307abd7f5c14 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.509+0000 7fb44cc00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:24:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.509+0000 7fb44cc00640 1 -- 172.21.15.39:0/1046271610 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4500cae60 con 0x55c07378a1b0 2024-07-04T08:24:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.509+0000 7fb44cc00640 1 -- 172.21.15.39:0/1046271610 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7fb4500e02a0 con 0x55c07378a1b0 2024-07-04T08:24:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.509+0000 7fb46bebb740 1 -- 172.21.15.39:0/1046271610 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb458036030 con 0x55c07378a1b0 2024-07-04T08:24:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.509+0000 7fb44cc00640 1 -- 172.21.15.39:0/1046271610 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb4500e04a0 con 0x55c07378a1b0 2024-07-04T08:24:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.509+0000 7fb46bebb740 10 client.27601 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:49.503 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.509+0000 7fb46bebb740 1 -- 172.21.15.39:0/1046271610 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c073b71ef0 con 0x55c07378a1b0 2024-07-04T08:24:49.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.510+0000 7fb44cc00640 1 -- 172.21.15.39:0/1046271610 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb458036030 con 0x55c07378a1b0 2024-07-04T08:24:49.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.510+0000 7fb44cc00640 1 -- 172.21.15.39:0/1046271610 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb450018b20 con 0x55c07378a1b0 2024-07-04T08:24:49.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.510+0000 7fb46bebb740 10 client.27601 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:49.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.510+0000 7fb46bebb740 10 client.27601 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:24:49.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.510+0000 7fb46bebb740 20 client.27601 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:49.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.510+0000 7fb46bebb740 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55c0737719f0 0x55c073b75320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.510+0000 7fb46bebb740 4 client.27601 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:24:49.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.510+0000 7fb46bebb740 1 -- 172.21.15.39:0/1046271610 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c073acfe30 con 0x55c0737719f0 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.510+0000 7fb457e00640 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55c0737719f0 0x55c073b75320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.511+0000 7fb457e00640 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55c0737719f0 0x55c073b75320 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb8cf79ef979523e server_cookie=e108333fbe9c07d1 in_seq=0 out_seq=0 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.511+0000 7fb44cc00640 10 client.27601 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.512+0000 7fb44cc00640 1 -- 172.21.15.39:0/1046271610 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c073acfe30 con 0x55c0737719f0 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.512+0000 7fb44cc00640 10 client.27601 handle_command_reply: tid=0 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.512+0000 7fb46bebb740 1 client.27601 shutdown 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 -- 172.21.15.39:0/1046271610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 msgr2=0x55c073b63350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b63350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb450000c00 tx=0x7fb4500a93d0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 -- 172.21.15.39:0/1046271610 shutdown_connections 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55c0737719f0 0x55c073b75320 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b63890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0737b3070 0x55c073b6f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 --2- 172.21.15.39:0/1046271610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c07378a1b0 0x55c073b63350 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb44cc00640 0 client.27601 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:24:49.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 -- 172.21.15.39:0/1046271610 >> 172.21.15.39:0/1046271610 conn(0x55c0737a7460 msgr2=0x55c073b5a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:49.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 -- 172.21.15.39:0/1046271610 shutdown_connections 2024-07-04T08:24:49.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 1 -- 172.21.15.39:0/1046271610 wait complete. 2024-07-04T08:24:49.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.513+0000 7fb46bebb740 20 client.27601 trim_cache size 0 max 16384 2024-07-04T08:24:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.529+0000 7fb46bebb740 1 Processor -- start 2024-07-04T08:24:49.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.529+0000 7fb46bebb740 1 -- start start 2024-07-04T08:24:49.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.529+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.529+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c073792390 0x55c073b64d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.529+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0737b3070 0x55c073b5ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.529+0000 7fb46bebb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b69c0 con 0x55c0737b3070 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.529+0000 7fb46bebb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b8500 con 0x55c07378a1b0 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.529+0000 7fb46bebb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b7090 con 0x55c073792390 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.529+0000 7fb457e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c073792390 0x55c073b64d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb45f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0737b3070 0x55c073b5ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb457e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c073792390 0x55c073b64d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36948/0 (socket says 172.21.15.39:36948) 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb457e00640 1 -- 172.21.15.39:0/3529359998 learned_addr learned my addr 172.21.15.39:0/3529359998 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb45ea00640 1 --2- 172.21.15.39:0/3529359998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb457e00640 1 -- 172.21.15.39:0/3529359998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 msgr2=0x55c073b591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb457e00640 1 --2- 172.21.15.39:0/3529359998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb457e00640 1 -- 172.21.15.39:0/3529359998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0737b3070 msgr2=0x55c073b5ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb457e00640 1 --2- 172.21.15.39:0/3529359998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0737b3070 0x55c073b5ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb457e00640 1 -- 172.21.15.39:0/3529359998 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c073b55f90 con 0x55c073792390 2024-07-04T08:24:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.530+0000 7fb457e00640 1 --2- 172.21.15.39:0/3529359998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c073792390 0x55c073b64d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55c073b75a80 tx=0x7fb448005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6cdcd062774606 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.531+0000 7fb44f400640 1 -- 172.21.15.39:0/3529359998 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448004110 con 0x55c073792390 2024-07-04T08:24:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.531+0000 7fb44f400640 1 -- 172.21.15.39:0/3529359998 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4480042b0 con 0x55c073792390 2024-07-04T08:24:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.531+0000 7fb44f400640 1 -- 172.21.15.39:0/3529359998 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44801e7a0 con 0x55c073792390 2024-07-04T08:24:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 -- 172.21.15.39:0/3529359998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c073792390 msgr2=0x55c073b64d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 --2- 172.21.15.39:0/3529359998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c073792390 0x55c073b64d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55c073b75a80 tx=0x7fb448005f90 comp rx=0 tx=0).stop 2024-07-04T08:24:49.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 -- 172.21.15.39:0/3529359998 shutdown_connections 2024-07-04T08:24:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 --2- 172.21.15.39:0/3529359998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c0737b3070 0x55c073b5ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 --2- 172.21.15.39:0/3529359998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c073792390 0x55c073b64d80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 --2- 172.21.15.39:0/3529359998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 -- 172.21.15.39:0/3529359998 >> 172.21.15.39:0/3529359998 conn(0x55c0737a7460 msgr2=0x55c073b64670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 -- 172.21.15.39:0/3529359998 shutdown_connections 2024-07-04T08:24:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 -- 172.21.15.39:0/3529359998 wait complete. 2024-07-04T08:24:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.532+0000 7fb46bebb740 1 -- start start 2024-07-04T08:24:49.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.533+0000 7fb46bebb740 1 Processor -- start 2024-07-04T08:24:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.533+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b6b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.533+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b6b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.533+0000 7fb46bebb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0737b3070 0x55c073b66140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.533+0000 7fb46bebb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b7e30 con 0x55c073792390 2024-07-04T08:24:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.533+0000 7fb46bebb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b7760 con 0x55c07378a1b0 2024-07-04T08:24:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.533+0000 7fb46bebb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b49a0 con 0x55c0737b3070 2024-07-04T08:24:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.533+0000 7fb46bebb740 10 client.0 _ll_register_callbacks cb 0x7fb45000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b6b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb457e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b6b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b6b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:42150/0 (socket says 172.21.15.39:42150) 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45ea00640 1 -- 172.21.15.39:0/2403878778 learned_addr learned my addr 172.21.15.39:0/2403878778 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45f400640 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0737b3070 0x55c073b66140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45ea00640 1 -- 172.21.15.39:0/2403878778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0737b3070 msgr2=0x55c073b66140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45ea00640 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0737b3070 0x55c073b66140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45ea00640 1 -- 172.21.15.39:0/2403878778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 msgr2=0x55c073b6b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45ea00640 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b6b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45ea00640 1 -- 172.21.15.39:0/2403878778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c073b66680 con 0x55c07378a1b0 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb45ea00640 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b6b190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c6a0 tx=0x7fb450009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d95498fd1d78bc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.534+0000 7fb44cc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.535+0000 7fb44cc00640 1 -- 172.21.15.39:0/2403878778 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4500c8030 con 0x55c07378a1b0 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.535+0000 7fb44cc00640 1 -- 172.21.15.39:0/2403878778 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7fb4500a6250 con 0x55c07378a1b0 2024-07-04T08:24:49.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.535+0000 7fb46bebb740 1 -- 172.21.15.39:0/2403878778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4480042b0 con 0x55c07378a1b0 2024-07-04T08:24:49.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.535+0000 7fb44cc00640 1 -- 172.21.15.39:0/2403878778 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb4500a9a30 con 0x55c07378a1b0 2024-07-04T08:24:49.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.535+0000 7fb46bebb740 10 client.27674 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:49.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.535+0000 7fb46bebb740 1 -- 172.21.15.39:0/2403878778 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c073bde1d0 con 0x55c07378a1b0 2024-07-04T08:24:49.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.535+0000 7fb44cc00640 1 -- 172.21.15.39:0/2403878778 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4480042b0 con 0x55c07378a1b0 2024-07-04T08:24:49.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.536+0000 7fb44cc00640 1 -- 172.21.15.39:0/2403878778 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb4500e02a0 con 0x55c07378a1b0 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.536+0000 7fb46bebb740 10 client.27674 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.536+0000 7fb46bebb740 10 client.27674 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.536+0000 7fb46bebb740 20 client.27674 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.536+0000 7fb46bebb740 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55c073746da0 0x55c073be07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.536+0000 7fb46bebb740 4 client.27674 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.536+0000 7fb46bebb740 1 -- 172.21.15.39:0/2403878778 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c073acb8d0 con 0x55c073746da0 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.536+0000 7fb457e00640 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55c073746da0 0x55c073be07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb457e00640 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55c073746da0 0x55c073be07d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=502e9a5f067c2c7 server_cookie=7ee91f05fa44d428 in_seq=0 out_seq=0 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb44cc00640 10 client.27674 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:24:49.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb44cc00640 1 -- 172.21.15.39:0/2403878778 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c073acb8d0 con 0x55c073746da0 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb44cc00640 10 client.27674 handle_command_reply: tid=0 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb46bebb740 1 client.27674 shutdown 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb46bebb740 1 -- 172.21.15.39:0/2403878778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 msgr2=0x55c073b6b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb46bebb740 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b6b190 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c6a0 tx=0x7fb450009770 comp rx=0 tx=0).stop 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb46bebb740 1 -- 172.21.15.39:0/2403878778 shutdown_connections 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb46bebb740 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55c073746da0 0x55c073be07d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb46bebb740 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c073792390 0x55c073b6b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb46bebb740 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0737b3070 0x55c073b66140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb46bebb740 1 --2- 172.21.15.39:0/2403878778 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c07378a1b0 0x55c073b6b190 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb46bebb740 1 -- 172.21.15.39:0/2403878778 >> 172.21.15.39:0/2403878778 conn(0x55c0737a7460 msgr2=0x55c073b64fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.537+0000 7fb44cc00640 0 client.27674 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:24:49.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.538+0000 7fb46bebb740 1 -- 172.21.15.39:0/2403878778 shutdown_connections 2024-07-04T08:24:49.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.538+0000 7fb46bebb740 1 -- 172.21.15.39:0/2403878778 wait complete. 2024-07-04T08:24:49.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.538+0000 7fb46bebb740 20 client.27674 trim_cache size 0 max 16384 2024-07-04T08:24:49.533 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:24:49.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 -- 172.21.15.39:0/3363120364 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4440b29e0 msgr2=0x7fb4440b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4440b29e0 0x7fb4440b4ef0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb4580094e0 tx=0x7fb458003f90 comp rx=0 tx=0).stop 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 -- 172.21.15.39:0/3363120364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460109700 msgr2=0x7fb4601a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460109700 0x7fb4601a0030 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb45000e9f0 tx=0x7fb45000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 -- 172.21.15.39:0/3363120364 shutdown_connections 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb4440b29e0 0x7fb4440b4ef0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb46010f990 0x7fb4601a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb46010f040 0x7fb4601a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 --2- 172.21.15.39:0/3363120364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460109700 0x7fb4601a0030 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 -- 172.21.15.39:0/3363120364 >> 172.21.15.39:0/3363120364 conn(0x7fb460077630 msgr2=0x7fb460077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 -- 172.21.15.39:0/3363120364 shutdown_connections 2024-07-04T08:24:49.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:49.539+0000 7fb465800640 1 -- 172.21.15.39:0/3363120364 wait complete. 2024-07-04T08:24:49.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:49 smithi133 ceph-mon[37050]: pgmap v774: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-07-04T08:24:49.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:49 smithi050 ceph-mon[36577]: pgmap v774: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-07-04T08:24:50.469 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:24:50.950 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:51.421 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:51 smithi039 ceph-mon[32062]: pgmap v775: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-07-04T08:24:51.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:51 smithi133 ceph-mon[37050]: pgmap v775: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-07-04T08:24:51.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:51 smithi050 ceph-mon[36577]: pgmap v775: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-07-04T08:24:53.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.187+0000 7f3ade600640 1 -- 172.21.15.39:0/3357026011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 msgr2=0x7f3ad8101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:53.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.187+0000 7f3ade600640 1 --2- 172.21.15.39:0/3357026011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 0x7f3ad8101d40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009ca0 tx=0x7f3ac802a950 comp rx=0 tx=0).stop 2024-07-04T08:24:53.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.188+0000 7f3ade600640 1 -- 172.21.15.39:0/3357026011 shutdown_connections 2024-07-04T08:24:53.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.188+0000 7f3ade600640 1 --2- 172.21.15.39:0/3357026011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ad8102280 0x7f3ad810efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.188+0000 7f3ade600640 1 --2- 172.21.15.39:0/3357026011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad810f4e0 0x7f3ad8111900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.188+0000 7f3ade600640 1 --2- 172.21.15.39:0/3357026011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 0x7f3ad8101d40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.188+0000 7f3ade600640 1 -- 172.21.15.39:0/3357026011 >> 172.21.15.39:0/3357026011 conn(0x7f3ad80fd7f0 msgr2=0x7f3ad80ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:53.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.188+0000 7f3ade600640 1 -- 172.21.15.39:0/3357026011 shutdown_connections 2024-07-04T08:24:53.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.188+0000 7f3ade600640 1 -- 172.21.15.39:0/3357026011 wait complete. 2024-07-04T08:24:53.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.189+0000 7f3ade600640 1 Processor -- start 2024-07-04T08:24:53.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.190+0000 7f3ade600640 1 -- start start 2024-07-04T08:24:53.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.190+0000 7f3ade600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 0x7f3ad819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.190+0000 7f3ade600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad8102280 0x7f3ad819bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.190+0000 7f3ade600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ad810f4e0 0x7f3ad81a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.190+0000 7f3ade600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8114fb0 con 0x7f3ad810f4e0 2024-07-04T08:24:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.190+0000 7f3ade600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8114e30 con 0x7f3ad8101930 2024-07-04T08:24:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.190+0000 7f3ade600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8115130 con 0x7f3ad8102280 2024-07-04T08:24:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3add200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 0x7f3ad819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3add200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 0x7f3ad819b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50690/0 (socket says 172.21.15.39:50690) 2024-07-04T08:24:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3add200640 1 -- 172.21.15.39:0/841698051 learned_addr learned my addr 172.21.15.39:0/841698051 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3addc00640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ad810f4e0 0x7f3ad81a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad8102280 0x7f3ad819bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3add200640 1 -- 172.21.15.39:0/841698051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad8102280 msgr2=0x7f3ad819bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3add200640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad8102280 0x7f3ad819bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3add200640 1 -- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ad810f4e0 msgr2=0x7f3ad81a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3add200640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ad810f4e0 0x7f3ad81a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.191+0000 7f3add200640 1 -- 172.21.15.39:0/841698051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad81a1930 con 0x7f3ad8101930 2024-07-04T08:24:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.192+0000 7f3add200640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 0x7f3ad819b9c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8005cc0 tx=0x7f3ac802f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.192+0000 7f3ad5600640 1 -- 172.21.15.39:0/841698051 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8034040 con 0x7f3ad8101930 2024-07-04T08:24:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.192+0000 7f3ad5600640 1 -- 172.21.15.39:0/841698051 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac80308f0 con 0x7f3ad8101930 2024-07-04T08:24:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.192+0000 7f3ade600640 1 -- 172.21.15.39:0/841698051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad81a1bc0 con 0x7f3ad8101930 2024-07-04T08:24:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.192+0000 7f3ad5600640 1 -- 172.21.15.39:0/841698051 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8004640 con 0x7f3ad8101930 2024-07-04T08:24:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.192+0000 7f3ade600640 1 -- 172.21.15.39:0/841698051 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad81a8380 con 0x7f3ad8101930 2024-07-04T08:24:53.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.196+0000 7f3ae61f2740 1 Processor -- start 2024-07-04T08:24:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.196+0000 7f3ae61f2740 1 -- start start 2024-07-04T08:24:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f05586950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f05586e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 0x557f055873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557f054fcee0 con 0x557f051b91b0 2024-07-04T08:24:53.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557f054ec480 con 0x557f051c1390 2024-07-04T08:24:53.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557f054f04c0 con 0x557f051e2070 2024-07-04T08:24:53.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3ad5600640 1 -- 172.21.15.39:0/841698051 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3ac8037020 con 0x7f3ad8101930 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3addc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f05586950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3addc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f05586950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:52318/0 (socket says 172.21.15.39:52318) 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3addc00640 1 -- 172.21.15.39:0/4192951385 learned_addr learned my addr 172.21.15.39:0/4192951385 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3add200640 1 --2- 172.21.15.39:0/4192951385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f05586e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.197+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/4192951385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 0x557f055873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.198+0000 7f3addc00640 1 -- 172.21.15.39:0/4192951385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 msgr2=0x557f055873d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.198+0000 7f3addc00640 1 --2- 172.21.15.39:0/4192951385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 0x557f055873d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.198+0000 7f3addc00640 1 -- 172.21.15.39:0/4192951385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 msgr2=0x557f05586e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.198+0000 7f3addc00640 1 --2- 172.21.15.39:0/4192951385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f05586e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.198+0000 7f3addc00640 1 -- 172.21.15.39:0/4192951385 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f05484650 con 0x557f051e2070 2024-07-04T08:24:53.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.198+0000 7f3addc00640 1 --2- 172.21.15.39:0/4192951385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f05586950 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009ca0 tx=0x7f3ad0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=190d52c20f338d6d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.199+0000 7f3ad5600640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3abc0b29e0 0x7f3abc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.199+0000 7f3ad5600640 1 -- 172.21.15.39:0/841698051 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f3ac8103610 con 0x7f3ad8101930 2024-07-04T08:24:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.199+0000 7f3add200640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3abc0b29e0 0x7f3abc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.200+0000 7f3add200640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3abc0b29e0 0x7f3abc0b4ef0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x557f055855d0 tx=0x7f3ac8109530 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.203+0000 7f3ace000640 1 -- 172.21.15.39:0/4192951385 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad001e070 con 0x557f051e2070 2024-07-04T08:24:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.203+0000 7f3ace000640 1 -- 172.21.15.39:0/4192951385 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad0009e50 con 0x557f051e2070 2024-07-04T08:24:53.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.203+0000 7f3ace000640 1 -- 172.21.15.39:0/4192951385 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000ea50 con 0x557f051e2070 2024-07-04T08:24:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.204+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4192951385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 msgr2=0x557f05586950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.204+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/4192951385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f05586950 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009ca0 tx=0x7f3ad0011810 comp rx=0 tx=0).stop 2024-07-04T08:24:53.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.204+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4192951385 shutdown_connections 2024-07-04T08:24:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.204+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/4192951385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f05586e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.204+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/4192951385 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 0x557f055873d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.204+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/4192951385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f05586950 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.204+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4192951385 >> 172.21.15.39:0/4192951385 conn(0x557f051d6460 msgr2=0x557f05584840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.204+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4192951385 shutdown_connections 2024-07-04T08:24:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.204+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4192951385 wait complete. 2024-07-04T08:24:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.205+0000 7f3ae61f2740 1 -- start start 2024-07-04T08:24:53.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.205+0000 7f3ae61f2740 1 Processor -- start 2024-07-04T08:24:53.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.205+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051b91b0 0x557f05592570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051c1390 0x557f05592ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f0559e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557f05350700 con 0x557f051c1390 2024-07-04T08:24:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557f05493bb0 con 0x557f051b91b0 2024-07-04T08:24:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557f05595130 con 0x557f051e2070 2024-07-04T08:24:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3ae61f2740 10 client.0 _ll_register_callbacks cb 0x557f054e5570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3addc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051b91b0 0x557f05592570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3addc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051b91b0 0x557f05592570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50704/0 (socket says 172.21.15.39:50704) 2024-07-04T08:24:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3addc00640 1 -- 172.21.15.39:0/260598556 learned_addr learned my addr 172.21.15.39:0/260598556 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:53.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3ad7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051c1390 0x557f05592ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.206+0000 7f3addc00640 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f0559e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3addc00640 1 -- 172.21.15.39:0/260598556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 msgr2=0x557f0559e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:53.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3addc00640 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f0559e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3addc00640 1 -- 172.21.15.39:0/260598556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051c1390 msgr2=0x557f05592ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:53.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3addc00640 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051c1390 0x557f05592ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3addc00640 1 -- 172.21.15.39:0/260598556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0524d650 con 0x557f051b91b0 2024-07-04T08:24:53.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3addc00640 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051b91b0 0x557f05592570 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000e4f0 tx=0x7f3ad000e5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d340a40124a57c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:53.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3ac2a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3ac2a00640 1 -- 172.21.15.39:0/260598556 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad002e060 con 0x557f051b91b0 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3ae61f2740 1 -- 172.21.15.39:0/260598556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ad0009e50 con 0x557f051b91b0 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.207+0000 7f3ac2a00640 1 -- 172.21.15.39:0/260598556 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f3ad0004260 con 0x557f051b91b0 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.208+0000 7f3ac2a00640 1 -- 172.21.15.39:0/260598556 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ad0035050 con 0x557f051b91b0 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.208+0000 7f3ae61f2740 10 client.27686 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.208+0000 7f3ae61f2740 1 -- 172.21.15.39:0/260598556 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f055a1180 con 0x557f051b91b0 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.208+0000 7f3ac2a00640 1 -- 172.21.15.39:0/260598556 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad0009e50 con 0x557f051b91b0 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.208+0000 7f3ac2a00640 1 -- 172.21.15.39:0/260598556 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3ad0019430 con 0x557f051b91b0 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.209+0000 7f3ae61f2740 10 client.27686 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.209+0000 7f3ae61f2740 10 client.27686 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.209+0000 7f3ae61f2740 20 client.27686 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.209+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557f051a09f0 0x557f055a4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.209+0000 7f3ae61f2740 4 client.27686 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:24:53.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.209+0000 7f3ae61f2740 1 -- 172.21.15.39:0/260598556 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f054fee30 con 0x557f051a09f0 2024-07-04T08:24:53.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.209+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557f051a09f0 0x557f055a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.210+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557f051a09f0 0x557f055a4540 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c017f59149998471 server_cookie=3f6880c44e1a8aa4 in_seq=0 out_seq=0 2024-07-04T08:24:53.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.210+0000 7f3ac2a00640 10 client.27686 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:24:53.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.210+0000 7f3ac2a00640 1 -- 172.21.15.39:0/260598556 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f054fee30 con 0x557f051a09f0 2024-07-04T08:24:53.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.210+0000 7f3ac2a00640 10 client.27686 handle_command_reply: tid=0 2024-07-04T08:24:53.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 client.27686 shutdown 2024-07-04T08:24:53.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 -- 172.21.15.39:0/260598556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051b91b0 msgr2=0x557f05592570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:53.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051b91b0 0x557f05592570 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000e4f0 tx=0x7f3ad000e5f0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 -- 172.21.15.39:0/260598556 shutdown_connections 2024-07-04T08:24:53.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557f051a09f0 0x557f055a4540 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051c1390 0x557f05592ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f0559e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/260598556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051b91b0 0x557f05592570 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ac2a00640 0 client.27686 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:24:53.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 -- 172.21.15.39:0/260598556 >> 172.21.15.39:0/260598556 conn(0x557f051d6460 msgr2=0x557f05589ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:53.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 -- 172.21.15.39:0/260598556 shutdown_connections 2024-07-04T08:24:53.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 1 -- 172.21.15.39:0/260598556 wait complete. 2024-07-04T08:24:53.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.211+0000 7f3ae61f2740 20 client.27686 trim_cache size 0 max 16384 2024-07-04T08:24:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.227+0000 7f3ae61f2740 1 Processor -- start 2024-07-04T08:24:53.219 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:53 smithi039 ceph-mon[32062]: pgmap v776: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-07-04T08:24:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.227+0000 7f3ae61f2740 1 -- start start 2024-07-04T08:24:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0558c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051c1390 0x557f0559e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051e2070 0x557f05583e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81b2320 con 0x557f051b91b0 2024-07-04T08:24:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81b3e60 con 0x557f051e2070 2024-07-04T08:24:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81b29f0 con 0x557f051c1390 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3addc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0558c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3addc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0558c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42522/0 (socket says 172.21.15.39:42522) 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3addc00640 1 -- 172.21.15.39:0/4206523916 learned_addr learned my addr 172.21.15.39:0/4206523916 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/4206523916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051c1390 0x557f0559e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.228+0000 7f3addc00640 1 --2- 172.21.15.39:0/4206523916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051e2070 0x557f05583e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.229+0000 7f3ad7e00640 1 -- 172.21.15.39:0/4206523916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051e2070 msgr2=0x557f05583e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.229+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/4206523916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051e2070 0x557f05583e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.229+0000 7f3ad7e00640 1 -- 172.21.15.39:0/4206523916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 msgr2=0x557f0558c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.229+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/4206523916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0558c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.229+0000 7f3ad7e00640 1 -- 172.21.15.39:0/4206523916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f05585600 con 0x557f051c1390 2024-07-04T08:24:53.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.229+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/4206523916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051c1390 0x557f0559e240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x557f05493bb0 tx=0x7f3ac4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd6d2057391b415a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:53.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.229+0000 7f3acd600640 1 -- 172.21.15.39:0/4206523916 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4027620 con 0x557f051c1390 2024-07-04T08:24:53.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.229+0000 7f3acd600640 1 -- 172.21.15.39:0/4206523916 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac401e8a0 con 0x557f051c1390 2024-07-04T08:24:53.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.229+0000 7f3acd600640 1 -- 172.21.15.39:0/4206523916 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40269f0 con 0x557f051c1390 2024-07-04T08:24:53.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4206523916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051c1390 msgr2=0x557f0559e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/4206523916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051c1390 0x557f0559e240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x557f05493bb0 tx=0x7f3ac4005f90 comp rx=0 tx=0).stop 2024-07-04T08:24:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4206523916 shutdown_connections 2024-07-04T08:24:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/4206523916 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051e2070 0x557f05583e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/4206523916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051c1390 0x557f0559e240 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/4206523916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0558c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4206523916 >> 172.21.15.39:0/4206523916 conn(0x557f051d6460 msgr2=0x557f0558ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4206523916 shutdown_connections 2024-07-04T08:24:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 -- 172.21.15.39:0/4206523916 wait complete. 2024-07-04T08:24:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.231+0000 7f3ae61f2740 1 -- start start 2024-07-04T08:24:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.232+0000 7f3ae61f2740 1 Processor -- start 2024-07-04T08:24:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.232+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0559a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.232+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 0x557f05595e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.232+0000 7f3ae61f2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f055963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.232+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81b3790 con 0x557f051b91b0 2024-07-04T08:24:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.232+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81b30c0 con 0x557f051c1390 2024-07-04T08:24:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.232+0000 7f3ae61f2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81b0300 con 0x557f051e2070 2024-07-04T08:24:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.232+0000 7f3ae61f2740 10 client.0 _ll_register_callbacks cb 0x7f3ad001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3addc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0559a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3addc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0559a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42534/0 (socket says 172.21.15.39:42534) 2024-07-04T08:24:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3addc00640 1 -- 172.21.15.39:0/3378854806 learned_addr learned my addr 172.21.15.39:0/3378854806 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 0x557f05595e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3addc00640 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f055963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3ad7e00640 1 -- 172.21.15.39:0/3378854806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 msgr2=0x557f055963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f055963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3ad7e00640 1 -- 172.21.15.39:0/3378854806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 msgr2=0x557f0559a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0559a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.233+0000 7f3ad7e00640 1 -- 172.21.15.39:0/3378854806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f05596a90 con 0x557f051c1390 2024-07-04T08:24:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.234+0000 7f3ad7e00640 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 0x557f05595e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4010fd0 tx=0x7f3ac4026440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9f27f2bb72ec096 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:53.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.234+0000 7f3ac2a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:24:53.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.234+0000 7f3ac2a00640 1 -- 172.21.15.39:0/3378854806 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac401e670 con 0x557f051c1390 2024-07-04T08:24:53.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.234+0000 7f3ae61f2740 1 -- 172.21.15.39:0/3378854806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac401e8a0 con 0x557f051c1390 2024-07-04T08:24:53.227 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.234+0000 7f3ac2a00640 1 -- 172.21.15.39:0/3378854806 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f3ac401ea50 con 0x557f051c1390 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.234+0000 7f3ac2a00640 1 -- 172.21.15.39:0/3378854806 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ac4039050 con 0x557f051c1390 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.234+0000 7f3ae61f2740 10 client.27692 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.234+0000 7f3ae61f2740 1 -- 172.21.15.39:0/3378854806 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f0560cac0 con 0x557f051c1390 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.235+0000 7f3ac2a00640 1 -- 172.21.15.39:0/3378854806 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ac401e8a0 con 0x557f051c1390 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.235+0000 7f3ac2a00640 1 -- 172.21.15.39:0/3378854806 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3ac40053b0 con 0x557f051c1390 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.235+0000 7f3ae61f2740 10 client.27692 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.235+0000 7f3ae61f2740 10 client.27692 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.235+0000 7f3ae61f2740 20 client.27692 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.235+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557f05175da0 0x557f0560f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:53.228 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.235+0000 7f3ae61f2740 4 client.27692 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:24:53.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.235+0000 7f3ae61f2740 1 -- 172.21.15.39:0/3378854806 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f054fa8d0 con 0x557f05175da0 2024-07-04T08:24:53.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.236+0000 7f3addc00640 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557f05175da0 0x557f0560f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:53.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.236+0000 7f3ac2a00640 10 client.27692 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:24:53.229 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.236+0000 7f3addc00640 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557f05175da0 0x557f0560f200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a807add900310546 server_cookie=8331b701dd06fd0c in_seq=0 out_seq=0 2024-07-04T08:24:53.230 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:24:53.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ac2a00640 1 -- 172.21.15.39:0/3378854806 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f054fa8d0 con 0x557f05175da0 2024-07-04T08:24:53.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ac2a00640 10 client.27692 handle_command_reply: tid=0 2024-07-04T08:24:53.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 client.27692 shutdown 2024-07-04T08:24:53.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 -- 172.21.15.39:0/3378854806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 msgr2=0x557f05595e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:53.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 0x557f05595e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4010fd0 tx=0x7f3ac4026440 comp rx=0 tx=0).stop 2024-07-04T08:24:53.231 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 -- 172.21.15.39:0/3378854806 shutdown_connections 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557f05175da0 0x557f0560f200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557f051e2070 0x557f055963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557f051c1390 0x557f05595e60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 --2- 172.21.15.39:0/3378854806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557f051b91b0 0x557f0559a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ac2a00640 0 client.27692 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 -- 172.21.15.39:0/3378854806 >> 172.21.15.39:0/3378854806 conn(0x557f051d6460 msgr2=0x557f05593820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 -- 172.21.15.39:0/3378854806 shutdown_connections 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.237+0000 7f3ae61f2740 1 -- 172.21.15.39:0/3378854806 wait complete. 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.238+0000 7f3ae61f2740 20 client.27692 trim_cache size 0 max 16384 2024-07-04T08:24:53.232 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:24:53.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.238+0000 7f3ade600640 1 -- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3abc0b29e0 msgr2=0x7f3abc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:53.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.238+0000 7f3ade600640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3abc0b29e0 0x7f3abc0b4ef0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x557f055855d0 tx=0x7f3ac8109530 comp rx=0 tx=0).stop 2024-07-04T08:24:53.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.238+0000 7f3ade600640 1 -- 172.21.15.39:0/841698051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 msgr2=0x7f3ad819b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:53.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.238+0000 7f3ade600640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 0x7f3ad819b9c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8005cc0 tx=0x7f3ac802f3a0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.233 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.239+0000 7f3ade600640 1 -- 172.21.15.39:0/841698051 shutdown_connections 2024-07-04T08:24:53.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.239+0000 7f3ade600640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3abc0b29e0 0x7f3abc0b4ef0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.239+0000 7f3ade600640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3ad810f4e0 0x7f3ad81a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.239+0000 7f3ade600640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad8102280 0x7f3ad819bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.239+0000 7f3ade600640 1 --2- 172.21.15.39:0/841698051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3ad8101930 0x7f3ad819b9c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:53.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.239+0000 7f3ade600640 1 -- 172.21.15.39:0/841698051 >> 172.21.15.39:0/841698051 conn(0x7f3ad80fd7f0 msgr2=0x7f3ad81005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:53.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.239+0000 7f3ade600640 1 -- 172.21.15.39:0/841698051 shutdown_connections 2024-07-04T08:24:53.234 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:53.239+0000 7f3ade600640 1 -- 172.21.15.39:0/841698051 wait complete. 2024-07-04T08:24:53.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:53 smithi133 ceph-mon[37050]: pgmap v776: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-07-04T08:24:53.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:53 smithi050 ceph-mon[36577]: pgmap v776: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-07-04T08:24:54.110 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:24:54.157 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:55.038 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:55.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:55 smithi133 ceph-mon[37050]: pgmap v777: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-07-04T08:24:55.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:55 smithi039 ceph-mon[32062]: pgmap v777: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-07-04T08:24:55.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:55 smithi050 ceph-mon[36577]: pgmap v777: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-07-04T08:24:56.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.593+0000 7f4b6a400640 1 -- 172.21.15.39:0/1290443592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 msgr2=0x7f4b641034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:56.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.593+0000 7f4b6a400640 1 --2- 172.21.15.39:0/1290443592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 0x7f4b641034e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009ca0 tx=0x7f4b5402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.593+0000 7f4b6a400640 1 -- 172.21.15.39:0/1290443592 shutdown_connections 2024-07-04T08:24:56.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.593+0000 7f4b6a400640 1 --2- 172.21.15.39:0/1290443592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b64109360 0x7f4b6410fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.593+0000 7f4b6a400640 1 --2- 172.21.15.39:0/1290443592 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b64108a10 0x7f4b64108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.593+0000 7f4b6a400640 1 --2- 172.21.15.39:0/1290443592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 0x7f4b641034e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.593+0000 7f4b6a400640 1 -- 172.21.15.39:0/1290443592 >> 172.21.15.39:0/1290443592 conn(0x7f4b640feae0 msgr2=0x7f4b64100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.593+0000 7f4b6a400640 1 -- 172.21.15.39:0/1290443592 shutdown_connections 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.593+0000 7f4b6a400640 1 -- 172.21.15.39:0/1290443592 wait complete. 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.595+0000 7f4b6a400640 1 Processor -- start 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.595+0000 7f4b6a400640 1 -- start start 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.595+0000 7f4b6a400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 0x7f4b64074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.595+0000 7f4b6a400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b64108a10 0x7f4b640751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.595+0000 7f4b6a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b64109360 0x7f4b640756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:56.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.595+0000 7f4b6a400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64113030 con 0x7f4b641030d0 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.595+0000 7f4b6a400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64112eb0 con 0x7f4b64109360 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.595+0000 7f4b6a400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4b641131b0 con 0x7f4b64108a10 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b63400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 0x7f4b64074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b5aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b64108a10 0x7f4b640751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b5aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b64108a10 0x7f4b640751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:52350/0 (socket says 172.21.15.39:52350) 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b63400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 0x7f4b64074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42556/0 (socket says 172.21.15.39:42556) 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b5aa00640 1 -- 172.21.15.39:0/2176064935 learned_addr learned my addr 172.21.15.39:0/2176064935 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b63e00640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b64109360 0x7f4b640756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b63400640 1 -- 172.21.15.39:0/2176064935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b64108a10 msgr2=0x7f4b640751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b63400640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b64108a10 0x7f4b640751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b63400640 1 -- 172.21.15.39:0/2176064935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b64109360 msgr2=0x7f4b640756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b63400640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b64109360 0x7f4b640756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b63400640 1 -- 172.21.15.39:0/2176064935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b641a7d00 con 0x7f4b641030d0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.596+0000 7f4b63e00640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b64109360 0x7f4b640756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.597+0000 7f4b63400640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 0x7f4b64074c60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b5402ade0 tx=0x7f4b54031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.597+0000 7f4b60c00640 1 -- 172.21.15.39:0/2176064935 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54003b70 con 0x7f4b641030d0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.597+0000 7f4b60c00640 1 -- 172.21.15.39:0/2176064935 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b54031910 con 0x7f4b641030d0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.597+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b641a7f90 con 0x7f4b641030d0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.597+0000 7f4b60c00640 1 -- 172.21.15.39:0/2176064935 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5402f950 con 0x7f4b641030d0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.597+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b641a8440 con 0x7f4b641030d0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.598+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b641034e0 con 0x7f4b641030d0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.601+0000 7f4b60c00640 1 -- 172.21.15.39:0/2176064935 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4b5402faf0 con 0x7f4b641030d0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.602+0000 7f4b60c00640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4b480b2960 0x7f4b480b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.603+0000 7f4b60c00640 1 -- 172.21.15.39:0/2176064935 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f4b54103bc0 con 0x7f4b641030d0 2024-07-04T08:24:56.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.603+0000 7f4b5aa00640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4b480b2960 0x7f4b480b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:56.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.604+0000 7f4b5aa00640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4b480b2960 0x7f4b480b4e70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4b64101340 tx=0x7f4b4c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:56.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.605+0000 7f4b60c00640 1 -- 172.21.15.39:0/2176064935 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f4b540cb2f0 con 0x7f4b641030d0 2024-07-04T08:24:56.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.864+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b64075f60 con 0x7f4b641030d0 2024-07-04T08:24:56.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.866+0000 7f4b60c00640 1 -- 172.21.15.39:0/2176064935 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f4b64075f60 con 0x7f4b641030d0 2024-07-04T08:24:56.858 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:24:56.858 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:24:56.858 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:24:56.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4b480b2960 msgr2=0x7f4b480b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:56.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4b480b2960 0x7f4b480b4e70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4b64101340 tx=0x7f4b4c005f90 comp rx=0 tx=0).stop 2024-07-04T08:24:56.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 msgr2=0x7f4b64074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:56.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 0x7f4b64074c60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4b5402ade0 tx=0x7f4b54031660 comp rx=0 tx=0).stop 2024-07-04T08:24:56.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 shutdown_connections 2024-07-04T08:24:56.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4b480b2960 0x7f4b480b4e70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b64109360 0x7f4b640756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b64108a10 0x7f4b640751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 --2- 172.21.15.39:0/2176064935 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4b641030d0 0x7f4b64074c60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:56.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 >> 172.21.15.39:0/2176064935 conn(0x7f4b640feae0 msgr2=0x7f4b64101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:56.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.869+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 shutdown_connections 2024-07-04T08:24:56.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:56.870+0000 7f4b6a400640 1 -- 172.21.15.39:0/2176064935 wait complete. 2024-07-04T08:24:57.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:57 smithi133 ceph-mon[37050]: pgmap v778: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-07-04T08:24:57.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:57 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2176064935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:24:57.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:57 smithi039 ceph-mon[32062]: pgmap v778: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-07-04T08:24:57.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:57 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2176064935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:24:57.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:57 smithi050 ceph-mon[36577]: pgmap v778: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-07-04T08:24:57.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:57 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2176064935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:24:57.841 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id 34031fc7 --release --await' 2024-07-04T08:24:57.841 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id 34031fc7 --release --await 2024-07-04T08:24:58.328 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:24:59.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.103+0000 7fb1f8a00640 1 -- 172.21.15.39:0/1996545354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f410d360 msgr2=0x7fb1f410f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.103+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/1996545354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f410d360 0x7fb1f410f850 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009ca0 tx=0x7fb1ec02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.103+0000 7fb1f8a00640 1 -- 172.21.15.39:0/1996545354 shutdown_connections 2024-07-04T08:24:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.103+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/1996545354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f410d360 0x7fb1f410f850 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.103+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/1996545354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f4104850 0x7fb1f410cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.103+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/1996545354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1f4103f00 0x7fb1f4104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.103+0000 7fb1f8a00640 1 -- 172.21.15.39:0/1996545354 >> 172.21.15.39:0/1996545354 conn(0x7fb1f4078c80 msgr2=0x7fb1f4079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:59.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.103+0000 7fb1f8a00640 1 -- 172.21.15.39:0/1996545354 shutdown_connections 2024-07-04T08:24:59.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.103+0000 7fb1f8a00640 1 -- 172.21.15.39:0/1996545354 wait complete. 2024-07-04T08:24:59.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.104+0000 7fb1f8a00640 1 Processor -- start 2024-07-04T08:24:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.105+0000 7fb1f8a00640 1 -- start start 2024-07-04T08:24:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.105+0000 7fb1f8a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1f4103f00 0x7fb1f4101c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.105+0000 7fb1f8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f4104850 0x7fb1f4102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.105+0000 7fb1f8a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f410d360 0x7fb1f41026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.105+0000 7fb1f8a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4112fc0 con 0x7fb1f410d360 2024-07-04T08:24:59.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.105+0000 7fb1f8a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4112e40 con 0x7fb1f4103f00 2024-07-04T08:24:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.105+0000 7fb1f8a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4113140 con 0x7fb1f4104850 2024-07-04T08:24:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.105+0000 7fb1f1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1f4103f00 0x7fb1f4101c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.105+0000 7fb1f1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1f4103f00 0x7fb1f4101c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50744/0 (socket says 172.21.15.39:50744) 2024-07-04T08:24:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.106+0000 7fb1f1e00640 1 -- 172.21.15.39:0/2045552785 learned_addr learned my addr 172.21.15.39:0/2045552785 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:59.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.106+0000 7fb1f2800640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f410d360 0x7fb1f41026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.106+0000 7fb1e9400640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f4104850 0x7fb1f4102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.106+0000 7fb1e9400640 1 -- 172.21.15.39:0/2045552785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1f4103f00 msgr2=0x7fb1f4101c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.106+0000 7fb1e9400640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1f4103f00 0x7fb1f4101c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.106+0000 7fb1e9400640 1 -- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f410d360 msgr2=0x7fb1f41026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:59.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.106+0000 7fb1e9400640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f410d360 0x7fb1f41026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.106+0000 7fb1e9400640 1 -- 172.21.15.39:0/2045552785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f41a7c90 con 0x7fb1f4104850 2024-07-04T08:24:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.106+0000 7fb1e9400640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f4104850 0x7fb1f4102180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb1f406a280 tx=0x7fb1dc00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.107+0000 7fb1eb400640 1 -- 172.21.15.39:0/2045552785 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00cc40 con 0x7fb1f4104850 2024-07-04T08:24:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.107+0000 7fb1f8a00640 1 -- 172.21.15.39:0/2045552785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f41a7f80 con 0x7fb1f4104850 2024-07-04T08:24:59.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.107+0000 7fb1eb400640 1 -- 172.21.15.39:0/2045552785 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1dc00cde0 con 0x7fb1f4104850 2024-07-04T08:24:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.107+0000 7fb1eb400640 1 -- 172.21.15.39:0/2045552785 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc018760 con 0x7fb1f4104850 2024-07-04T08:24:59.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.107+0000 7fb1f8a00640 1 -- 172.21.15.39:0/2045552785 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f41a83e0 con 0x7fb1f4104850 2024-07-04T08:24:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.110+0000 7fb1fefae740 1 Processor -- start 2024-07-04T08:24:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.111+0000 7fb1fefae740 1 -- start start 2024-07-04T08:24:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.111+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d30e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.111+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d1b1b0 0x5621d30e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.111+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d23390 0x5621d30e9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.111+0000 7fb1fefae740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5621d305eee0 con 0x5621d2d44070 2024-07-04T08:24:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.111+0000 7fb1fefae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5621d304e480 con 0x5621d2d23390 2024-07-04T08:24:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.111+0000 7fb1fefae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5621d30524c0 con 0x5621d2d1b1b0 2024-07-04T08:24:59.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.111+0000 7fb1eb400640 1 -- 172.21.15.39:0/2045552785 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb1dc018900 con 0x7fb1f4104850 2024-07-04T08:24:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.111+0000 7fb1f2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d30e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1f2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d30e87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42588/0 (socket says 172.21.15.39:42588) 2024-07-04T08:24:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1f2800640 1 -- 172.21.15.39:0/130704602 learned_addr learned my addr 172.21.15.39:0/130704602 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/130704602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d1b1b0 0x5621d30e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1e9400640 1 --2- 172.21.15.39:0/130704602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d23390 0x5621d30e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1f1e00640 1 -- 172.21.15.39:0/130704602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d23390 msgr2=0x5621d30e9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:59.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/130704602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d23390 0x5621d30e9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1f1e00640 1 -- 172.21.15.39:0/130704602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 msgr2=0x5621d30e87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/130704602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d30e87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1f1e00640 1 -- 172.21.15.39:0/130704602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d2fe6650 con 0x5621d2d1b1b0 2024-07-04T08:24:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.112+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/130704602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d1b1b0 0x5621d30e8d10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40058c0 tx=0x7fb1e4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb5a99b4050193a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.113+0000 7fb1e3400640 1 -- 172.21.15.39:0/130704602 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e401e070 con 0x5621d2d1b1b0 2024-07-04T08:24:59.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.113+0000 7fb1e3400640 1 -- 172.21.15.39:0/130704602 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e4009e50 con 0x5621d2d1b1b0 2024-07-04T08:24:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.113+0000 7fb1e3400640 1 -- 172.21.15.39:0/130704602 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40186e0 con 0x5621d2d1b1b0 2024-07-04T08:24:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.113+0000 7fb1eb400640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1d80b2960 0x7fb1d80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.113+0000 7fb1eb400640 1 -- 172.21.15.39:0/2045552785 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7fb1dc0e2ca0 con 0x7fb1f4104850 2024-07-04T08:24:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1f2800640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1d80b2960 0x7fb1d80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1fefae740 1 -- 172.21.15.39:0/130704602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d1b1b0 msgr2=0x5621d30e8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:59.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1fefae740 1 --2- 172.21.15.39:0/130704602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d1b1b0 0x5621d30e8d10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40058c0 tx=0x7fb1e4011810 comp rx=0 tx=0).stop 2024-07-04T08:24:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1fefae740 1 -- 172.21.15.39:0/130704602 shutdown_connections 2024-07-04T08:24:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1fefae740 1 --2- 172.21.15.39:0/130704602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d23390 0x5621d30e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1fefae740 1 --2- 172.21.15.39:0/130704602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d1b1b0 0x5621d30e8d10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1fefae740 1 --2- 172.21.15.39:0/130704602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d30e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1fefae740 1 -- 172.21.15.39:0/130704602 >> 172.21.15.39:0/130704602 conn(0x5621d2d38460 msgr2=0x5621d30e66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1fefae740 1 -- 172.21.15.39:0/130704602 shutdown_connections 2024-07-04T08:24:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.114+0000 7fb1f2800640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1d80b2960 0x7fb1d80b4e70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec02f500 tx=0x7fb1ec02f310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:59.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.115+0000 7fb1fefae740 1 -- 172.21.15.39:0/130704602 wait complete. 2024-07-04T08:24:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.115+0000 7fb1fefae740 1 -- start start 2024-07-04T08:24:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.116+0000 7fb1fefae740 1 Processor -- start 2024-07-04T08:24:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.116+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d1b1b0 0x5621d30f4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.116+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.116+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d31002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.116+0000 7fb1fefae740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5621d2eb2700 con 0x5621d2d44070 2024-07-04T08:24:59.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.116+0000 7fb1fefae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5621d2ff5bb0 con 0x5621d2d1b1b0 2024-07-04T08:24:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.116+0000 7fb1fefae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5621d30f6fb0 con 0x5621d2d23390 2024-07-04T08:24:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.116+0000 7fb1fefae740 10 client.0 _ll_register_callbacks cb 0x5621d3047570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1f2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d31002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1f1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d1b1b0 0x5621d30f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1f1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d1b1b0 0x5621d30f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50758/0 (socket says 172.21.15.39:50758) 2024-07-04T08:24:59.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1e9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1f2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d31002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42600/0 (socket says 172.21.15.39:42600) 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1f1e00640 1 -- 172.21.15.39:0/2948391229 learned_addr learned my addr 172.21.15.39:0/2948391229 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1e9400640 1 -- 172.21.15.39:0/2948391229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d1b1b0 msgr2=0x5621d30f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1e9400640 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d1b1b0 0x5621d30f4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1e9400640 1 -- 172.21.15.39:0/2948391229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 msgr2=0x5621d31002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1e9400640 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d31002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.117+0000 7fb1e9400640 1 -- 172.21.15.39:0/2948391229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d2daf650 con 0x5621d2d23390 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.118+0000 7fb1e9400640 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30f48a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0e0030 tx=0x7fb1dc010e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ccbac716c1f42ae server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.118+0000 7fb1d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.118+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2948391229 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0a93e0 con 0x5621d2d23390 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.118+0000 7fb1fefae740 1 -- 172.21.15.39:0/2948391229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e4009e50 con 0x5621d2d23390 2024-07-04T08:24:59.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.118+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2948391229 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7fb1dc0a6260 con 0x5621d2d23390 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.118+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2948391229 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1dc0e04c0 con 0x5621d2d23390 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.118+0000 7fb1fefae740 10 client.27643 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.118+0000 7fb1fefae740 1 -- 172.21.15.39:0/2948391229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621d3102fe0 con 0x5621d2d23390 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.119+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2948391229 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e4009e50 con 0x5621d2d23390 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.119+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2948391229 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb1dc0e8060 con 0x5621d2d23390 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.119+0000 7fb1fefae740 10 client.27643 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.119+0000 7fb1fefae740 10 client.27643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.119+0000 7fb1fefae740 20 client.27643 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.119+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5621d2d029f0 0x5621d31064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.119+0000 7fb1fefae740 4 client.27643 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:24:59.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.119+0000 7fb1fefae740 1 -- 172.21.15.39:0/2948391229 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621d3060e30 con 0x5621d2d029f0 2024-07-04T08:24:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.119+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5621d2d029f0 0x5621d31064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.120+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5621d2d029f0 0x5621d31064a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc9110233d34a9f8 server_cookie=647718a97c0d8ccb in_seq=0 out_seq=0 2024-07-04T08:24:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.120+0000 7fb1d7e00640 10 client.27643 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:24:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.121+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2948391229 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5621d3060e30 con 0x5621d2d029f0 2024-07-04T08:24:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.121+0000 7fb1d7e00640 10 client.27643 handle_command_reply: tid=0 2024-07-04T08:24:59.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.121+0000 7fb1fefae740 1 client.27643 shutdown 2024-07-04T08:24:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 -- 172.21.15.39:0/2948391229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 msgr2=0x5621d30f48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30f48a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0e0030 tx=0x7fb1dc010e30 comp rx=0 tx=0).stop 2024-07-04T08:24:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 -- 172.21.15.39:0/2948391229 shutdown_connections 2024-07-04T08:24:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5621d2d029f0 0x5621d31064a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d44070 0x5621d31002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1d7e00640 0 client.27643 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:24:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30f48a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2948391229 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d1b1b0 0x5621d30f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 -- 172.21.15.39:0/2948391229 >> 172.21.15.39:0/2948391229 conn(0x5621d2d38460 msgr2=0x5621d30eb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 -- 172.21.15.39:0/2948391229 shutdown_connections 2024-07-04T08:24:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 1 -- 172.21.15.39:0/2948391229 wait complete. 2024-07-04T08:24:59.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.122+0000 7fb1fefae740 20 client.27643 trim_cache size 0 max 16384 2024-07-04T08:24:59.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.138+0000 7fb1fefae740 1 Processor -- start 2024-07-04T08:24:59.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.138+0000 7fb1fefae740 1 -- start start 2024-07-04T08:24:59.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.138+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.138+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.138+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d30eda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.138+0000 7fb1fefae740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41b25f0 con 0x5621d2d1b1b0 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1fefae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41b4130 con 0x5621d2d44070 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1fefae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41b2cc0 con 0x5621d2d23390 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1f1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1f1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30e59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42612/0 (socket says 172.21.15.39:42612) 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1f1e00640 1 -- 172.21.15.39:0/4272422526 learned_addr learned my addr 172.21.15.39:0/4272422526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1f2800640 1 --2- 172.21.15.39:0/4272422526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d30eda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1e9400640 1 --2- 172.21.15.39:0/4272422526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1f1e00640 1 -- 172.21.15.39:0/4272422526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 msgr2=0x5621d30f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/4272422526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1f1e00640 1 -- 172.21.15.39:0/4272422526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 msgr2=0x5621d30eda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/4272422526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d30eda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.139+0000 7fb1f1e00640 1 -- 172.21.15.39:0/4272422526 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621d30f5420 con 0x5621d2d1b1b0 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.140+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/4272422526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30e59d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4019e70 tx=0x7fb1e4011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a1dfc42a4ea362 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.140+0000 7fb1e2a00640 1 -- 172.21.15.39:0/4272422526 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4029070 con 0x5621d2d1b1b0 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.140+0000 7fb1e2a00640 1 -- 172.21.15.39:0/4272422526 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e4009e50 con 0x5621d2d1b1b0 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.140+0000 7fb1e2a00640 1 -- 172.21.15.39:0/4272422526 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e401e040 con 0x5621d2d1b1b0 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.141+0000 7fb1fefae740 1 -- 172.21.15.39:0/4272422526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 msgr2=0x5621d30e59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.141+0000 7fb1fefae740 1 --2- 172.21.15.39:0/4272422526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30e59d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4019e70 tx=0x7fb1e4011dc0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.141+0000 7fb1fefae740 1 -- 172.21.15.39:0/4272422526 shutdown_connections 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.141+0000 7fb1fefae740 1 --2- 172.21.15.39:0/4272422526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d30eda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.141+0000 7fb1fefae740 1 --2- 172.21.15.39:0/4272422526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.141+0000 7fb1fefae740 1 --2- 172.21.15.39:0/4272422526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30e59d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.141+0000 7fb1fefae740 1 -- 172.21.15.39:0/4272422526 >> 172.21.15.39:0/4272422526 conn(0x5621d2d38460 msgr2=0x5621d30f5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.141+0000 7fb1fefae740 1 -- 172.21.15.39:0/4272422526 shutdown_connections 2024-07-04T08:24:59.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.141+0000 7fb1fefae740 1 -- 172.21.15.39:0/4272422526 wait complete. 2024-07-04T08:24:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.142+0000 7fb1fefae740 1 -- start start 2024-07-04T08:24:59.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.142+0000 7fb1fefae740 1 Processor -- start 2024-07-04T08:24:59.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30fc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1fefae740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d3100130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1fefae740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41b3a60 con 0x5621d2d1b1b0 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1fefae740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41b3390 con 0x5621d2d44070 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1fefae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41b05d0 con 0x5621d2d23390 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1fefae740 10 client.0 _ll_register_callbacks cb 0x7fb1dc0e6440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1f1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30fc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1f2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d3100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1f1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30fc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42618/0 (socket says 172.21.15.39:42618) 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1f2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d3100130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50766/0 (socket says 172.21.15.39:50766) 2024-07-04T08:24:59.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1f1e00640 1 -- 172.21.15.39:0/2768377955 learned_addr learned my addr 172.21.15.39:0/2768377955 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.143+0000 7fb1e9400640 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1f2800640 1 -- 172.21.15.39:0/2768377955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 msgr2=0x5621d30ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1f2800640 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1f2800640 1 -- 172.21.15.39:0/2768377955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 msgr2=0x5621d30fc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1f2800640 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30fc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1f2800640 1 -- 172.21.15.39:0/2768377955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621d31008c0 con 0x5621d2d44070 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1f2800640 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d3100130 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec02ade0 tx=0x7fb1ec0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d931cc0d447f66 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2768377955 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec058060 con 0x5621d2d44070 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2768377955 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7fb1ec004040 con 0x5621d2d44070 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.144+0000 7fb1fefae740 1 -- 172.21.15.39:0/2768377955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e4009e50 con 0x5621d2d44070 2024-07-04T08:24:59.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.145+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2768377955 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1ec05f050 con 0x5621d2d44070 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.145+0000 7fb1fefae740 10 client.27716 fetch_fsmap learned FSMap version 30 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.145+0000 7fb1fefae740 1 -- 172.21.15.39:0/2768377955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621d316ec50 con 0x5621d2d44070 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.145+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2768377955 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e4009e50 con 0x5621d2d44070 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.146+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2768377955 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb1ec02fae0 con 0x5621d2d44070 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.146+0000 7fb1fefae740 10 client.27716 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.146+0000 7fb1fefae740 10 client.27716 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.146+0000 7fb1fefae740 20 client.27716 populate_metadata read hostname 'smithi039' 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.146+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5621d2cd7da0 0x5621d3171410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.146+0000 7fb1fefae740 4 client.27716 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.146+0000 7fb1fefae740 1 -- 172.21.15.39:0/2768377955 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621d305c8d0 con 0x5621d2cd7da0 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.146+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5621d2cd7da0 0x5621d3171410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.147+0000 7fb1f1e00640 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5621d2cd7da0 0x5621d3171410 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68522638b14fa915 server_cookie=c58fc60d2deb41e5 in_seq=0 out_seq=0 2024-07-04T08:24:59.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.147+0000 7fb1d7e00640 10 client.27716 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:24:59.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.147+0000 7fb1d7e00640 1 -- 172.21.15.39:0/2768377955 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621d305c8d0 con 0x5621d2cd7da0 2024-07-04T08:24:59.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.147+0000 7fb1d7e00640 10 client.27716 handle_command_reply: tid=0 2024-07-04T08:24:59.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.147+0000 7fb1fefae740 1 client.27716 shutdown 2024-07-04T08:24:59.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.147+0000 7fb1fefae740 1 -- 172.21.15.39:0/2768377955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 msgr2=0x5621d3100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:59.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.147+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d3100130 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec02ade0 tx=0x7fb1ec0095c0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1fefae740 1 -- 172.21.15.39:0/2768377955 shutdown_connections 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5621d2cd7da0 0x5621d3171410 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5621d2d44070 0x5621d3100130 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621d2d23390 0x5621d30ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1fefae740 1 --2- 172.21.15.39:0/2768377955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5621d2d1b1b0 0x5621d30fc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1d7e00640 0 client.27716 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1fefae740 1 -- 172.21.15.39:0/2768377955 >> 172.21.15.39:0/2768377955 conn(0x5621d2d38460 msgr2=0x5621d30f55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1fefae740 1 -- 172.21.15.39:0/2768377955 shutdown_connections 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1fefae740 1 -- 172.21.15.39:0/2768377955 wait complete. 2024-07-04T08:24:59.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.148+0000 7fb1fefae740 20 client.27716 trim_cache size 0 max 16384 2024-07-04T08:24:59.144 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:24:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 -- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1d80b2960 msgr2=0x7fb1d80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1d80b2960 0x7fb1d80b4e70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec02f500 tx=0x7fb1ec02f310 comp rx=0 tx=0).stop 2024-07-04T08:24:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 -- 172.21.15.39:0/2045552785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f4104850 msgr2=0x7fb1f4102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:24:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f4104850 0x7fb1f4102180 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb1f406a280 tx=0x7fb1dc00ee50 comp rx=0 tx=0).stop 2024-07-04T08:24:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 -- 172.21.15.39:0/2045552785 shutdown_connections 2024-07-04T08:24:59.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb1d80b2960 0x7fb1d80b4e70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1f410d360 0x7fb1f41026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f4104850 0x7fb1f4102180 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 --2- 172.21.15.39:0/2045552785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb1f4103f00 0x7fb1f4101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:24:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 -- 172.21.15.39:0/2045552785 >> 172.21.15.39:0/2045552785 conn(0x7fb1f4078c80 msgr2=0x7fb1f40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:24:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 -- 172.21.15.39:0/2045552785 shutdown_connections 2024-07-04T08:24:59.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:24:59.149+0000 7fb1f8a00640 1 -- 172.21.15.39:0/2045552785 wait complete. 2024-07-04T08:24:59.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:24:59 smithi039 ceph-mon[32062]: pgmap v779: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-07-04T08:24:59.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:24:59 smithi133 ceph-mon[37050]: pgmap v779: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-07-04T08:24:59.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:24:59 smithi050 ceph-mon[36577]: pgmap v779: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-07-04T08:25:00.445 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:25:00.970 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:25:01.315 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:01 smithi039 ceph-mon[32062]: pgmap v780: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-07-04T08:25:01.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:01 smithi133 ceph-mon[37050]: pgmap v780: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-07-04T08:25:01.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:01 smithi050 ceph-mon[36577]: pgmap v780: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-07-04T08:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.709+0000 7f0dc5800640 1 -- 172.21.15.39:0/800372126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc01051a0 msgr2=0x7f0dc01055b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.709+0000 7f0dc5800640 1 --2- 172.21.15.39:0/800372126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc01051a0 0x7f0dc01055b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009900 tx=0x7f0db002a400 comp rx=0 tx=0).stop 2024-07-04T08:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.709+0000 7f0dc5800640 1 -- 172.21.15.39:0/800372126 shutdown_connections 2024-07-04T08:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.709+0000 7f0dc5800640 1 --2- 172.21.15.39:0/800372126 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dc010b3a0 0x7f0dc0111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.709+0000 7f0dc5800640 1 --2- 172.21.15.39:0/800372126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc010aa50 0x7f0dc010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.709+0000 7f0dc5800640 1 --2- 172.21.15.39:0/800372126 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc01051a0 0x7f0dc01055b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.709+0000 7f0dc5800640 1 -- 172.21.15.39:0/800372126 >> 172.21.15.39:0/800372126 conn(0x7f0dc0100c30 msgr2=0x7f0dc0103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.709+0000 7f0dc5800640 1 -- 172.21.15.39:0/800372126 shutdown_connections 2024-07-04T08:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.709+0000 7f0dc5800640 1 -- 172.21.15.39:0/800372126 wait complete. 2024-07-04T08:25:01.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.710+0000 7f0dc5800640 1 Processor -- start 2024-07-04T08:25:01.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.711+0000 7f0dc5800640 1 -- start start 2024-07-04T08:25:01.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.711+0000 7f0dc5800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dc01051a0 0x7f0dc019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.711+0000 7f0dc5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc010aa50 0x7f0dc019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.711+0000 7f0dc5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc010b3a0 0x7f0dc01a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.711+0000 7f0dc5800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0115050 con 0x7f0dc010b3a0 2024-07-04T08:25:01.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.711+0000 7f0dc5800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc0114ed0 con 0x7f0dc01051a0 2024-07-04T08:25:01.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.711+0000 7f0dc5800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01151d0 con 0x7f0dc010aa50 2024-07-04T08:25:01.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc010aa50 0x7f0dc019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dc4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc010b3a0 0x7f0dc01a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc010aa50 0x7f0dc019c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51250/0 (socket says 172.21.15.39:51250) 2024-07-04T08:25:01.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbf400640 1 -- 172.21.15.39:0/1900227381 learned_addr learned my addr 172.21.15.39:0/1900227381 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:01.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dc01051a0 0x7f0dc019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbf400640 1 -- 172.21.15.39:0/1900227381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dc01051a0 msgr2=0x7f0dc019baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:01.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbf400640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dc01051a0 0x7f0dc019baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbf400640 1 -- 172.21.15.39:0/1900227381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc010b3a0 msgr2=0x7f0dc01a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:01.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbf400640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc010b3a0 0x7f0dc01a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbf400640 1 -- 172.21.15.39:0/1900227381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc01a1a10 con 0x7f0dc010aa50 2024-07-04T08:25:01.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.712+0000 7f0dbf400640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc010aa50 0x7f0dc019c040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e970 tx=0x7f0da800ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:01.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.713+0000 7f0dbcc00640 1 -- 172.21.15.39:0/1900227381 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da800ccc0 con 0x7f0dc010aa50 2024-07-04T08:25:01.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.713+0000 7f0dbcc00640 1 -- 172.21.15.39:0/1900227381 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da80035f0 con 0x7f0dc010aa50 2024-07-04T08:25:01.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.713+0000 7f0dbcc00640 1 -- 172.21.15.39:0/1900227381 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da800ce60 con 0x7f0dc010aa50 2024-07-04T08:25:01.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.713+0000 7f0dc5800640 1 -- 172.21.15.39:0/1900227381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc01a1d00 con 0x7f0dc010aa50 2024-07-04T08:25:01.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.713+0000 7f0dc5800640 1 -- 172.21.15.39:0/1900227381 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc01a8460 con 0x7f0dc010aa50 2024-07-04T08:25:01.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.717+0000 7f0dcd4cb740 1 Processor -- start 2024-07-04T08:25:01.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.717+0000 7f0dcd4cb740 1 -- start start 2024-07-04T08:25:01.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.717+0000 7f0dbcc00640 1 -- 172.21.15.39:0/1900227381 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0da8003aa0 con 0x7f0dc010aa50 2024-07-04T08:25:01.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.717+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.717+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.717+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 0x55ddfa0ddf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.717+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9ff7b20 con 0x55ddf9d0f660 2024-07-04T08:25:01.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.717+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ddfa0532b0 con 0x55ddf9d17840 2024-07-04T08:25:01.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.717+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ddfa042850 con 0x55ddf9d38520 2024-07-04T08:25:01.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dc4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dc4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51258/0 (socket says 172.21.15.39:51258) 2024-07-04T08:25:01.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dc4e00640 1 -- 172.21.15.39:0/3075558284 learned_addr learned my addr 172.21.15.39:0/3075558284 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:01.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dbf400640 1 --2- 172.21.15.39:0/3075558284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 0x55ddfa0ddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/3075558284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dc4e00640 1 -- 172.21.15.39:0/3075558284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 msgr2=0x55ddfa0ddf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:01.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dc4e00640 1 --2- 172.21.15.39:0/3075558284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 0x55ddfa0ddf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dc4e00640 1 -- 172.21.15.39:0/3075558284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 msgr2=0x55ddfa0dd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:01.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dc4e00640 1 --2- 172.21.15.39:0/3075558284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0dd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.718+0000 7f0dc4e00640 1 -- 172.21.15.39:0/3075558284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf9ffc780 con 0x55ddf9d38520 2024-07-04T08:25:01.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.719+0000 7f0dc4e00640 1 --2- 172.21.15.39:0/3075558284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dd490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0db8011440 tx=0x7f0db8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d603eb9be05961 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:01.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.719+0000 7f0db5600640 1 -- 172.21.15.39:0/3075558284 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8003c30 con 0x55ddf9d38520 2024-07-04T08:25:01.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.719+0000 7f0db5600640 1 -- 172.21.15.39:0/3075558284 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db8003dd0 con 0x55ddf9d38520 2024-07-04T08:25:01.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.719+0000 7f0db5600640 1 -- 172.21.15.39:0/3075558284 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db800fae0 con 0x55ddf9d38520 2024-07-04T08:25:01.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.719+0000 7f0dbcc00640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0da40b29e0 0x7f0da40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.720+0000 7f0dbcc00640 1 -- 172.21.15.39:0/1900227381 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f0da80e2c00 con 0x7f0dc010aa50 2024-07-04T08:25:01.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.720+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0da40b29e0 0x7f0da40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.720+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/3075558284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 msgr2=0x55ddfa0dd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:01.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.720+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/3075558284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dd490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0db8011440 tx=0x7f0db8011900 comp rx=0 tx=0).stop 2024-07-04T08:25:01.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.720+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/3075558284 shutdown_connections 2024-07-04T08:25:01.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.720+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/3075558284 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0dd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.720+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/3075558284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 0x55ddfa0ddf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.720+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/3075558284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dd490 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.720+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/3075558284 >> 172.21.15.39:0/3075558284 conn(0x55ddf9d2c910 msgr2=0x55ddfa0db380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.721+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/3075558284 shutdown_connections 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.721+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/3075558284 wait complete. 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.721+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0da40b29e0 0x7f0da40b4ef0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0db0031440 tx=0x7f0db0009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.721+0000 7f0dcd4cb740 1 -- start start 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.722+0000 7f0dcd4cb740 1 Processor -- start 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.722+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d0f660 0x55ddfa0e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.722+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d17840 0x55ddfa0e9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.722+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.722+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9fe9d20 con 0x55ddf9d17840 2024-07-04T08:25:01.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.722+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ddfa046890 con 0x55ddf9d0f660 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.722+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ddf9ea6bb0 con 0x55ddf9d38520 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.722+0000 7f0dcd4cb740 10 client.0 _ll_register_callbacks cb 0x55ddf9fea9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dbf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d17840 0x55ddfa0e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dbfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d0f660 0x55ddfa0e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dbfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d0f660 0x55ddfa0e8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52038/0 (socket says 172.21.15.39:52038) 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dbfe00640 1 -- 172.21.15.39:0/2242315765 learned_addr learned my addr 172.21.15.39:0/2242315765 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dc4e00640 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dbfe00640 1 -- 172.21.15.39:0/2242315765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 msgr2=0x55ddfa0f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dbfe00640 1 -- 172.21.15.39:0/2242315765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d17840 msgr2=0x55ddfa0e9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d17840 0x55ddfa0e9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.723+0000 7f0dbfe00640 1 -- 172.21.15.39:0/2242315765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf9fd9660 con 0x55ddf9d0f660 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.724+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d0f660 0x55ddfa0e8b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009900 tx=0x7f0db0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f82677a16a65d5b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.724+0000 7f0dae000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.724+0000 7f0dae000640 1 -- 172.21.15.39:0/2242315765 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db004d070 con 0x55ddf9d0f660 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.724+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2242315765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db8003dd0 con 0x55ddf9d0f660 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.724+0000 7f0dae000640 1 -- 172.21.15.39:0/2242315765 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f0db0005230 con 0x55ddf9d0f660 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dae000640 1 -- 172.21.15.39:0/2242315765 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0db0059050 con 0x55ddf9d0f660 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dcd4cb740 10 client.27728 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2242315765 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ddf9da3b00 con 0x55ddf9d0f660 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dae000640 1 -- 172.21.15.39:0/2242315765 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db8003dd0 con 0x55ddf9d0f660 2024-07-04T08:25:01.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dae000640 1 -- 172.21.15.39:0/2242315765 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0db0003680 con 0x55ddf9d0f660 2024-07-04T08:25:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dcd4cb740 10 client.27728 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dcd4cb740 10 client.27728 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:25:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dcd4cb740 20 client.27728 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ddf9cf6ea0 0x55ddfa0faaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dcd4cb740 4 client.27728 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:25:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.725+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2242315765 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddfa055200 con 0x55ddf9cf6ea0 2024-07-04T08:25:01.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.726+0000 7f0dbf400640 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ddf9cf6ea0 0x55ddfa0faaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.726+0000 7f0dbf400640 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ddf9cf6ea0 0x55ddfa0faaf0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880922f47f898f42 server_cookie=a31ee919f83a6407 in_seq=0 out_seq=0 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.726+0000 7f0dae000640 10 client.27728 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.727+0000 7f0dae000640 1 -- 172.21.15.39:0/2242315765 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ddfa055200 con 0x55ddf9cf6ea0 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.727+0000 7f0dae000640 10 client.27728 handle_command_reply: tid=0 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.727+0000 7f0dcd4cb740 1 client.27728 shutdown 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2242315765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d0f660 msgr2=0x55ddfa0e8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d0f660 0x55ddfa0e8b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009900 tx=0x7f0db0009260 comp rx=0 tx=0).stop 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2242315765 shutdown_connections 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ddf9cf6ea0 0x55ddfa0faaf0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d17840 0x55ddfa0e9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2242315765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d0f660 0x55ddfa0e8b20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2242315765 >> 172.21.15.39:0/2242315765 conn(0x55ddf9d2c910 msgr2=0x55ddfa0e0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dae000640 0 client.27728 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2242315765 shutdown_connections 2024-07-04T08:25:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2242315765 wait complete. 2024-07-04T08:25:01.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.728+0000 7f0dcd4cb740 20 client.27728 trim_cache size 0 max 16384 2024-07-04T08:25:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.748+0000 7f0dcd4cb740 1 Processor -- start 2024-07-04T08:25:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.748+0000 7f0dcd4cb740 1 -- start start 2024-07-04T08:25:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 0x55ddfa0e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b2430 con 0x55ddf9d0f660 2024-07-04T08:25:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b3f70 con 0x55ddf9d17840 2024-07-04T08:25:01.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b2b00 con 0x55ddf9d38520 2024-07-04T08:25:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dbfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dbfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0ea6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47526/0 (socket says 172.21.15.39:47526) 2024-07-04T08:25:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dbfe00640 1 -- 172.21.15.39:0/1592890794 learned_addr learned my addr 172.21.15.39:0/1592890794 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dbf400640 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 0x55ddfa0e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.749+0000 7f0dc4e00640 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.750+0000 7f0dbf400640 1 -- 172.21.15.39:0/1592890794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 msgr2=0x55ddfa0dee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.750+0000 7f0dbf400640 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.750+0000 7f0dbf400640 1 -- 172.21.15.39:0/1592890794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 msgr2=0x55ddfa0ea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:01.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.750+0000 7f0dbf400640 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0ea6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.750+0000 7f0dbf400640 1 -- 172.21.15.39:0/1592890794 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf9e1a960 con 0x55ddf9d17840 2024-07-04T08:25:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.750+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:25:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.750+0000 7f0dbf400640 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 0x55ddfa0e73a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0da80dff90 tx=0x7f0da8091040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf7cf880cd4ca9e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.751+0000 7f0db4c00640 1 -- 172.21.15.39:0/1592890794 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da80a9290 con 0x55ddf9d17840 2024-07-04T08:25:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.751+0000 7f0db4c00640 1 -- 172.21.15.39:0/1592890794 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da80aae50 con 0x55ddf9d17840 2024-07-04T08:25:01.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.751+0000 7f0db4c00640 1 -- 172.21.15.39:0/1592890794 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da80a52b0 con 0x55ddf9d17840 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.751+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/1592890794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 msgr2=0x55ddfa0e73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.751+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 0x55ddfa0e73a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0da80dff90 tx=0x7f0da8091040 comp rx=0 tx=0).stop 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.751+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/1592890794 shutdown_connections 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.751+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d38520 0x55ddfa0dee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.752+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d17840 0x55ddfa0e73a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.752+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/1592890794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.752+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/1592890794 >> 172.21.15.39:0/1592890794 conn(0x55ddf9d2c910 msgr2=0x55ddfa0d9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.752+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/1592890794 shutdown_connections 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.752+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/1592890794 wait complete. 2024-07-04T08:25:01.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.752+0000 7f0dcd4cb740 1 -- start start 2024-07-04T08:25:01.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.752+0000 7f0dcd4cb740 1 Processor -- start 2024-07-04T08:25:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d17840 0x55ddfa0eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dcd4cb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d38520 0x55ddfa0ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b38a0 con 0x55ddf9d0f660 2024-07-04T08:25:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b31d0 con 0x55ddf9d38520 2024-07-04T08:25:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dcd4cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc01b0410 con 0x55ddf9d17840 2024-07-04T08:25:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dcd4cb740 10 client.0 _ll_register_callbacks cb 0x7f0db0032c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dbfe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dc4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d38520 0x55ddfa0ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dc4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d38520 0x55ddfa0ef2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52046/0 (socket says 172.21.15.39:52046) 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.753+0000 7f0dc4e00640 1 -- 172.21.15.39:0/2385127753 learned_addr learned my addr 172.21.15.39:0/2385127753 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.754+0000 7f0dbf400640 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d17840 0x55ddfa0eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.754+0000 7f0dc4e00640 1 -- 172.21.15.39:0/2385127753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d17840 msgr2=0x55ddfa0eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.754+0000 7f0dc4e00640 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d17840 0x55ddfa0eebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.754+0000 7f0dc4e00640 1 -- 172.21.15.39:0/2385127753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 msgr2=0x55ddfa0f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.754+0000 7f0dc4e00640 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.754+0000 7f0dc4e00640 1 -- 172.21.15.39:0/2385127753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddfa00f8a0 con 0x55ddf9d38520 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.754+0000 7f0dc4e00640 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d38520 0x55ddfa0ef2c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0db8014040 tx=0x7f0db800f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=348a974c9b7c9ace server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.754+0000 7f0dae000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.755+0000 7f0dae000640 1 -- 172.21.15.39:0/2385127753 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8009820 con 0x55ddf9d38520 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.755+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2385127753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da80aae50 con 0x55ddf9d38520 2024-07-04T08:25:01.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.755+0000 7f0dae000640 1 -- 172.21.15.39:0/2385127753 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9424+0+0 (secure 0 0 0) 0x7f0db800fca0 con 0x55ddf9d38520 2024-07-04T08:25:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.755+0000 7f0dae000640 1 -- 172.21.15.39:0/2385127753 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0db800fea0 con 0x55ddf9d38520 2024-07-04T08:25:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.755+0000 7f0dcd4cb740 10 client.27734 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.755+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2385127753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ddfa00b450 con 0x55ddf9d38520 2024-07-04T08:25:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.755+0000 7f0dae000640 1 -- 172.21.15.39:0/2385127753 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0da80aae50 con 0x55ddf9d38520 2024-07-04T08:25:01.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.756+0000 7f0dae000640 1 -- 172.21.15.39:0/2385127753 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0db802e470 con 0x55ddf9d38520 2024-07-04T08:25:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.756+0000 7f0dcd4cb740 10 client.27734 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.756+0000 7f0dcd4cb740 10 client.27734 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:25:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.756+0000 7f0dcd4cb740 20 client.27734 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.756+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ddf9ccc250 0x55ddfa16df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.756+0000 7f0dcd4cb740 4 client.27734 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "34031fc7", "release": true, "await": true}] 2024-07-04T08:25:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.756+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2385127753 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "34031fc7", "release": true, "await": true}) -- 0x55ddfa050ca0 con 0x55ddf9ccc250 2024-07-04T08:25:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.756+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ddf9ccc250 0x55ddfa16df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.757+0000 7f0dbfe00640 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ddf9ccc250 0x55ddfa16df30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4356e70f6f5cc301 server_cookie=a98703b797f0b4a0 in_seq=0 out_seq=0 2024-07-04T08:25:01.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.757+0000 7f0dae000640 10 client.27734 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dae000640 1 -- 172.21.15.39:0/2385127753 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ddfa050ca0 con 0x55ddf9ccc250 2024-07-04T08:25:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dae000640 10 client.27734 handle_command_reply: tid=0 2024-07-04T08:25:01.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dcd4cb740 1 client.27734 shutdown 2024-07-04T08:25:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2385127753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d38520 msgr2=0x55ddfa0ef2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d38520 0x55ddfa0ef2c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0db8014040 tx=0x7f0db800f920 comp rx=0 tx=0).stop 2024-07-04T08:25:01.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2385127753 shutdown_connections 2024-07-04T08:25:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ddf9ccc250 0x55ddfa16df30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ddf9d38520 0x55ddfa0ef2c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ddf9d17840 0x55ddfa0eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dcd4cb740 1 --2- 172.21.15.39:0/2385127753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ddf9d0f660 0x55ddfa0f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dae000640 0 client.27734 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.760+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2385127753 >> 172.21.15.39:0/2385127753 conn(0x55ddf9d2c910 msgr2=0x55ddfa0dec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.761+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2385127753 shutdown_connections 2024-07-04T08:25:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.761+0000 7f0dcd4cb740 1 -- 172.21.15.39:0/2385127753 wait complete. 2024-07-04T08:25:01.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.761+0000 7f0dcd4cb740 20 client.27734 trim_cache size 0 max 16384 2024-07-04T08:25:01.754 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:25:01.754 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:25:01.755 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:25:01.755 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 28, 2024-07-04T08:25:01.755 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:25:01.755 INFO:teuthology.orchestra.run.smithi039.stdout: "34031fc7": { 2024-07-04T08:25:01.755 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 28, 2024-07-04T08:25:01.755 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:25:01.755 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:25:01.755 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:25:01.756 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:25:01.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:25:01.756 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:25:01.756 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 142.1, 2024-07-04T08:25:01.756 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:25:01.757 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:25:01.757 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:25:01.757 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:25:01.757 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:25:01.757 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:25:01.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:01.757 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:01.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:01.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:01.758 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:01.758 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:25:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 -- 172.21.15.39:0/1900227381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0da40b29e0 msgr2=0x7f0da40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0da40b29e0 0x7f0da40b4ef0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0db0031440 tx=0x7f0db0009560 comp rx=0 tx=0).stop 2024-07-04T08:25:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 -- 172.21.15.39:0/1900227381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc010aa50 msgr2=0x7f0dc019c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc010aa50 0x7f0dc019c040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e970 tx=0x7f0da800ed70 comp rx=0 tx=0).stop 2024-07-04T08:25:01.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 -- 172.21.15.39:0/1900227381 shutdown_connections 2024-07-04T08:25:01.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0da40b29e0 0x7f0da40b4ef0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0dc010b3a0 0x7f0dc01a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0dc010aa50 0x7f0dc019c040 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 --2- 172.21.15.39:0/1900227381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0dc01051a0 0x7f0dc019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 -- 172.21.15.39:0/1900227381 >> 172.21.15.39:0/1900227381 conn(0x7f0dc0100c30 msgr2=0x7f0dc0103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 -- 172.21.15.39:0/1900227381 shutdown_connections 2024-07-04T08:25:01.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:01.762+0000 7f0dc5800640 1 -- 172.21.15.39:0/1900227381 wait complete. 2024-07-04T08:25:02.994 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '34031fc7', total seconds elapsed: 31.8 2024-07-04T08:25:03.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:03 smithi133 ceph-mon[37050]: pgmap v781: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-07-04T08:25:03.568 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:03 smithi039 ceph-mon[32062]: pgmap v781: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-07-04T08:25:03.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:03 smithi050 ceph-mon[36577]: pgmap v781: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-07-04T08:25:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.515+0000 7f8f73a00640 1 -- 172.21.15.39:0/1967304216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6c0755f0 msgr2=0x7f8f6c10f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:04.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.515+0000 7f8f73a00640 1 --2- 172.21.15.39:0/1967304216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6c0755f0 0x7f8f6c10f130 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009900 tx=0x7f8f5802a410 comp rx=0 tx=0).stop 2024-07-04T08:25:04.507 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:04 smithi039 ceph-mon[32062]: osdmap e99: 12 total, 12 up, 12 in 2024-07-04T08:25:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.515+0000 7f8f73a00640 1 -- 172.21.15.39:0/1967304216 shutdown_connections 2024-07-04T08:25:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.515+0000 7f8f73a00640 1 --2- 172.21.15.39:0/1967304216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c10f670 0x7f8f6c111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.515+0000 7f8f73a00640 1 --2- 172.21.15.39:0/1967304216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6c0755f0 0x7f8f6c10f130 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.515+0000 7f8f73a00640 1 --2- 172.21.15.39:0/1967304216 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f6c074ca0 0x7f8f6c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.515+0000 7f8f73a00640 1 -- 172.21.15.39:0/1967304216 >> 172.21.15.39:0/1967304216 conn(0x7f8f6c07ada0 msgr2=0x7f8f6c07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.515+0000 7f8f73a00640 1 -- 172.21.15.39:0/1967304216 shutdown_connections 2024-07-04T08:25:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.515+0000 7f8f73a00640 1 -- 172.21.15.39:0/1967304216 wait complete. 2024-07-04T08:25:04.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.517+0000 7f8f73a00640 1 Processor -- start 2024-07-04T08:25:04.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.517+0000 7f8f73a00640 1 -- start start 2024-07-04T08:25:04.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f73a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c074ca0 0x7f8f6c19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f73a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6c0755f0 0x7f8f6c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f73a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f6c10f670 0x7f8f6c1a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f73a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c115190 con 0x7f8f6c074ca0 2024-07-04T08:25:04.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f73a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c115010 con 0x7f8f6c10f670 2024-07-04T08:25:04.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f73a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c115310 con 0x7f8f6c0755f0 2024-07-04T08:25:04.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f71000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c074ca0 0x7f8f6c19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f6be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6c0755f0 0x7f8f6c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f71000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c074ca0 0x7f8f6c19bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47560/0 (socket says 172.21.15.39:47560) 2024-07-04T08:25:04.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f71000640 1 -- 172.21.15.39:0/972155788 learned_addr learned my addr 172.21.15.39:0/972155788 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:04.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.518+0000 7f8f71a00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f6c10f670 0x7f8f6c1a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.519+0000 7f8f6be00640 1 -- 172.21.15.39:0/972155788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f6c10f670 msgr2=0x7f8f6c1a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:04.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.519+0000 7f8f6be00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f6c10f670 0x7f8f6c1a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.519+0000 7f8f6be00640 1 -- 172.21.15.39:0/972155788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c074ca0 msgr2=0x7f8f6c19bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:04.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.519+0000 7f8f6be00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c074ca0 0x7f8f6c19bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.519+0000 7f8f6be00640 1 -- 172.21.15.39:0/972155788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c1a1bb0 con 0x7f8f6c0755f0 2024-07-04T08:25:04.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.520+0000 7f8f6be00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6c0755f0 0x7f8f6c19c1e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8f5802a930 tx=0x7f8f58031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:04.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.520+0000 7f8f69600640 1 -- 172.21.15.39:0/972155788 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5803f070 con 0x7f8f6c0755f0 2024-07-04T08:25:04.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.520+0000 7f8f69600640 1 -- 172.21.15.39:0/972155788 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f58004490 con 0x7f8f6c0755f0 2024-07-04T08:25:04.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.520+0000 7f8f69600640 1 -- 172.21.15.39:0/972155788 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58035070 con 0x7f8f6c0755f0 2024-07-04T08:25:04.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.520+0000 7f8f73a00640 1 -- 172.21.15.39:0/972155788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c1a1e40 con 0x7f8f6c0755f0 2024-07-04T08:25:04.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.520+0000 7f8f73a00640 1 -- 172.21.15.39:0/972155788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c1a8600 con 0x7f8f6c0755f0 2024-07-04T08:25:04.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.524+0000 7f8f7a043740 1 Processor -- start 2024-07-04T08:25:04.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.524+0000 7f8f7a043740 1 -- start start 2024-07-04T08:25:04.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.524+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 0x5607be8267d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.524+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be826d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.524+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be827250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.524+0000 7f8f7a043740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5607be79cf00 con 0x5607be461390 2024-07-04T08:25:04.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.524+0000 7f8f7a043740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5607be78c4a0 con 0x5607be4591b0 2024-07-04T08:25:04.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.524+0000 7f8f7a043740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5607be7904e0 con 0x5607be482070 2024-07-04T08:25:04.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f69600640 1 -- 172.21.15.39:0/972155788 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8f58043400 con 0x7f8f6c0755f0 2024-07-04T08:25:04.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f71a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 0x5607be8267d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f71a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 0x5607be8267d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51306/0 (socket says 172.21.15.39:51306) 2024-07-04T08:25:04.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f71a00640 1 -- 172.21.15.39:0/3689460347 learned_addr learned my addr 172.21.15.39:0/3689460347 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:04.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f71000640 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be826d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f71000640 1 -- 172.21.15.39:0/3689460347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 msgr2=0x5607be8267d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f71000640 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 0x5607be8267d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f71000640 1 -- 172.21.15.39:0/3689460347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 msgr2=0x5607be827250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f6be00640 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be827250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f71000640 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be827250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f71000640 1 -- 172.21.15.39:0/3689460347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607be724670 con 0x5607be4591b0 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.525+0000 7f8f6be00640 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be827250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.526+0000 7f8f71000640 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be826d10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009c80 tx=0x7f8f60011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2111135fcbda1d52 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.526+0000 7f8f5e000640 1 -- 172.21.15.39:0/3689460347 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60004230 con 0x5607be4591b0 2024-07-04T08:25:04.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.526+0000 7f8f5e000640 1 -- 172.21.15.39:0/3689460347 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f600043d0 con 0x5607be4591b0 2024-07-04T08:25:04.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.526+0000 7f8f5e000640 1 -- 172.21.15.39:0/3689460347 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60004cd0 con 0x5607be4591b0 2024-07-04T08:25:04.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.527+0000 7f8f69600640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8f540b29e0 0x7f8f540b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.527+0000 7f8f69600640 1 -- 172.21.15.39:0/972155788 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9512+0+0 (secure 0 0 0) 0x7f8f58102e30 con 0x7f8f6c0755f0 2024-07-04T08:25:04.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.527+0000 7f8f71a00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8f540b29e0 0x7f8f540b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.527+0000 7f8f7a043740 1 -- 172.21.15.39:0/3689460347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 msgr2=0x5607be826d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:04.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.528+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be826d10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009c80 tx=0x7f8f60011810 comp rx=0 tx=0).stop 2024-07-04T08:25:04.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.528+0000 7f8f7a043740 1 -- 172.21.15.39:0/3689460347 shutdown_connections 2024-07-04T08:25:04.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.528+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be827250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.528+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be826d10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.528+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3689460347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 0x5607be8267d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.528+0000 7f8f7a043740 1 -- 172.21.15.39:0/3689460347 >> 172.21.15.39:0/3689460347 conn(0x5607be476460 msgr2=0x5607be8246c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:04.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.528+0000 7f8f7a043740 1 -- 172.21.15.39:0/3689460347 shutdown_connections 2024-07-04T08:25:04.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.528+0000 7f8f71a00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8f540b29e0 0x7f8f540b4ef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8f64011630 tx=0x7f8f64009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:04.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.528+0000 7f8f7a043740 1 -- 172.21.15.39:0/3689460347 wait complete. 2024-07-04T08:25:04.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.529+0000 7f8f7a043740 1 -- start start 2024-07-04T08:25:04.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.529+0000 7f8f7a043740 1 Processor -- start 2024-07-04T08:25:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.529+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be832330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.529+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be832870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.529+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 0x5607be83e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.529+0000 7f8f7a043740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5607be5f0700 con 0x5607be4591b0 2024-07-04T08:25:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f7a043740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5607be733bd0 con 0x5607be482070 2024-07-04T08:25:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f7a043740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5607be834fb0 con 0x5607be461390 2024-07-04T08:25:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f7a043740 10 client.0 _ll_register_callbacks cb 0x5607be785590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f71000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be832330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f6be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be832870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f71000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be832330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47574/0 (socket says 172.21.15.39:47574) 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f71000640 1 -- 172.21.15.39:0/281425458 learned_addr learned my addr 172.21.15.39:0/281425458 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f71a00640 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 0x5607be83e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f6be00640 1 -- 172.21.15.39:0/281425458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 msgr2=0x5607be83e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f6be00640 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 0x5607be83e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f6be00640 1 -- 172.21.15.39:0/281425458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 msgr2=0x5607be832330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f6be00640 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be832330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.530+0000 7f8f6be00640 1 -- 172.21.15.39:0/281425458 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607be4ed650 con 0x5607be461390 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.531+0000 7f8f6be00640 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be832870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8f58100140 tx=0x7f8f5802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c92cb70688989a39 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.531+0000 7f8f52a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.531+0000 7f8f52a00640 1 -- 172.21.15.39:0/281425458 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58048030 con 0x5607be461390 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.531+0000 7f8f7a043740 1 -- 172.21.15.39:0/281425458 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f600043d0 con 0x5607be461390 2024-07-04T08:25:04.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.531+0000 7f8f52a00640 1 -- 172.21.15.39:0/281425458 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9512+0+0 (secure 0 0 0) 0x7f8f581003c0 con 0x5607be461390 2024-07-04T08:25:04.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.531+0000 7f8f52a00640 1 -- 172.21.15.39:0/281425458 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8f581005c0 con 0x5607be461390 2024-07-04T08:25:04.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.531+0000 7f8f7a043740 10 client.27679 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:04.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.531+0000 7f8f7a043740 1 -- 172.21.15.39:0/281425458 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607be840f10 con 0x5607be461390 2024-07-04T08:25:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.532+0000 7f8f52a00640 1 -- 172.21.15.39:0/281425458 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f580e1070 con 0x5607be461390 2024-07-04T08:25:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.532+0000 7f8f52a00640 1 -- 172.21.15.39:0/281425458 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8f5804b060 con 0x5607be461390 2024-07-04T08:25:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.532+0000 7f8f7a043740 10 client.27679 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.532+0000 7f8f7a043740 10 client.27679 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:25:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.532+0000 7f8f7a043740 20 client.27679 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.532+0000 7f8f7a043740 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5607be4409f0 0x5607be844400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.532+0000 7f8f7a043740 4 client.27679 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:25:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.532+0000 7f8f7a043740 1 -- 172.21.15.39:0/281425458 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607be79ee50 con 0x5607be4409f0 2024-07-04T08:25:04.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.533+0000 7f8f71000640 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5607be4409f0 0x5607be844400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.533+0000 7f8f52a00640 10 client.27679 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:25:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.533+0000 7f8f71000640 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5607be4409f0 0x5607be844400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f64c263e29a213b server_cookie=f29c3545106d7ff0 in_seq=0 out_seq=0 2024-07-04T08:25:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.534+0000 7f8f52a00640 1 -- 172.21.15.39:0/281425458 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5607be79ee50 con 0x5607be4409f0 2024-07-04T08:25:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.534+0000 7f8f52a00640 10 client.27679 handle_command_reply: tid=0 2024-07-04T08:25:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.534+0000 7f8f7a043740 1 client.27679 shutdown 2024-07-04T08:25:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 -- 172.21.15.39:0/281425458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 msgr2=0x5607be832870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:04.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be832870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8f58100140 tx=0x7f8f5802a410 comp rx=0 tx=0).stop 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 -- 172.21.15.39:0/281425458 shutdown_connections 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5607be4409f0 0x5607be844400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 0x5607be83e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be832870 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 --2- 172.21.15.39:0/281425458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be832330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f52a00640 0 client.27679 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 -- 172.21.15.39:0/281425458 >> 172.21.15.39:0/281425458 conn(0x5607be476460 msgr2=0x5607be829960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 -- 172.21.15.39:0/281425458 shutdown_connections 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 1 -- 172.21.15.39:0/281425458 wait complete. 2024-07-04T08:25:04.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.535+0000 7f8f7a043740 20 client.27679 trim_cache size 0 max 16384 2024-07-04T08:25:04.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.563+0000 7f8f7a043740 1 Processor -- start 2024-07-04T08:25:04.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.564+0000 7f8f7a043740 1 -- start start 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.564+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be823960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.564+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be8246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.564+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 0x5607be82b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.564+0000 7f8f7a043740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1b2650 con 0x5607be4591b0 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.564+0000 7f8f7a043740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1b4190 con 0x5607be482070 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.564+0000 7f8f7a043740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1b2d20 con 0x5607be461390 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.564+0000 7f8f71000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be823960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.564+0000 7f8f6be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be8246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f6be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be8246f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51332/0 (socket says 172.21.15.39:51332) 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f6be00640 1 -- 172.21.15.39:0/2288201590 learned_addr learned my addr 172.21.15.39:0/2288201590 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f71000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be823960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47582/0 (socket says 172.21.15.39:47582) 2024-07-04T08:25:04.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:04 smithi133 ceph-mon[37050]: osdmap e99: 12 total, 12 up, 12 in 2024-07-04T08:25:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f71a00640 1 --2- 172.21.15.39:0/2288201590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 0x5607be82b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f6be00640 1 -- 172.21.15.39:0/2288201590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 msgr2=0x5607be82b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f6be00640 1 --2- 172.21.15.39:0/2288201590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 0x5607be82b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f6be00640 1 -- 172.21.15.39:0/2288201590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 msgr2=0x5607be823960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f6be00640 1 --2- 172.21.15.39:0/2288201590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be823960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f6be00640 1 -- 172.21.15.39:0/2288201590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607be832c00 con 0x5607be461390 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.565+0000 7f8f6be00640 1 --2- 172.21.15.39:0/2288201590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be8246f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8f580c9a80 tx=0x7f8f58003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b72059f47d64e735 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.566+0000 7f8f5d600640 1 -- 172.21.15.39:0/2288201590 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f580c5c40 con 0x5607be461390 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.566+0000 7f8f5d600640 1 -- 172.21.15.39:0/2288201590 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f600043d0 con 0x5607be461390 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.566+0000 7f8f5d600640 1 -- 172.21.15.39:0/2288201590 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f581003c0 con 0x5607be461390 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 -- 172.21.15.39:0/2288201590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 msgr2=0x5607be8246f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 --2- 172.21.15.39:0/2288201590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be8246f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8f580c9a80 tx=0x7f8f58003f50 comp rx=0 tx=0).stop 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 -- 172.21.15.39:0/2288201590 shutdown_connections 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 --2- 172.21.15.39:0/2288201590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be482070 0x5607be82b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 --2- 172.21.15.39:0/2288201590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be461390 0x5607be8246f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 --2- 172.21.15.39:0/2288201590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be4591b0 0x5607be823960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 -- 172.21.15.39:0/2288201590 >> 172.21.15.39:0/2288201590 conn(0x5607be476460 msgr2=0x5607be823fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 -- 172.21.15.39:0/2288201590 shutdown_connections 2024-07-04T08:25:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 -- 172.21.15.39:0/2288201590 wait complete. 2024-07-04T08:25:04.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.567+0000 7f8f7a043740 1 -- start start 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.568+0000 7f8f7a043740 1 Processor -- start 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.568+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be83df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.568+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be83e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.568+0000 7f8f7a043740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 0x5607be83e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.568+0000 7f8f7a043740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1b3ac0 con 0x5607be461390 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.568+0000 7f8f7a043740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1b33f0 con 0x5607be4591b0 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.568+0000 7f8f7a043740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1b0650 con 0x5607be482070 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.568+0000 7f8f7a043740 10 client.0 _ll_register_callbacks cb 0x7f8f58005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be83e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f71000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be83df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f6be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be83e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47590/0 (socket says 172.21.15.39:47590) 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f71000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be83df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52086/0 (socket says 172.21.15.39:52086) 2024-07-04T08:25:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f6be00640 1 -- 172.21.15.39:0/3768671755 learned_addr learned my addr 172.21.15.39:0/3768671755 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f71a00640 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 0x5607be83e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f71000640 1 -- 172.21.15.39:0/3768671755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 msgr2=0x5607be83e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f71000640 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 0x5607be83e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f71000640 1 -- 172.21.15.39:0/3768671755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 msgr2=0x5607be83e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f71000640 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be83e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f71000640 1 -- 172.21.15.39:0/3768671755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607be8365a0 con 0x5607be4591b0 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.569+0000 7f8f6be00640 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be83e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.570+0000 7f8f71000640 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be83df10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8f60004950 tx=0x7f8f600098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5fe8eb4de9fadc9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.570+0000 7f8f52a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.570+0000 7f8f52a00640 1 -- 172.21.15.39:0/3768671755 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6001e040 con 0x5607be4591b0 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.570+0000 7f8f7a043740 1 -- 172.21.15.39:0/3768671755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f600043d0 con 0x5607be4591b0 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.570+0000 7f8f52a00640 1 -- 172.21.15.39:0/3768671755 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9512+0+0 (secure 0 0 0) 0x7f8f60026910 con 0x5607be4591b0 2024-07-04T08:25:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.570+0000 7f8f52a00640 1 -- 172.21.15.39:0/3768671755 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8f6003b050 con 0x5607be4591b0 2024-07-04T08:25:04.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.570+0000 7f8f7a043740 10 client.27752 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:04.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.570+0000 7f8f7a043740 1 -- 172.21.15.39:0/3768671755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5607be8ace40 con 0x5607be4591b0 2024-07-04T08:25:04.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.571+0000 7f8f52a00640 1 -- 172.21.15.39:0/3768671755 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f600043d0 con 0x5607be4591b0 2024-07-04T08:25:04.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.571+0000 7f8f52a00640 1 -- 172.21.15.39:0/3768671755 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8f60024040 con 0x5607be4591b0 2024-07-04T08:25:04.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.571+0000 7f8f7a043740 10 client.27752 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:04.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.571+0000 7f8f7a043740 10 client.27752 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:25:04.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.571+0000 7f8f7a043740 20 client.27752 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:04.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.571+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5607be415da0 0x5607be8af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.571+0000 7f8f7a043740 4 client.27752 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.571+0000 7f8f7a043740 1 -- 172.21.15.39:0/3768671755 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607be79a8f0 con 0x5607be415da0 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.572+0000 7f8f6be00640 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5607be415da0 0x5607be8af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.572+0000 7f8f6be00640 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5607be415da0 0x5607be8af3e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47deebad590af753 server_cookie=99ca050c54fd4468 in_seq=0 out_seq=0 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.572+0000 7f8f52a00640 10 client.27752 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f52a00640 1 -- 172.21.15.39:0/3768671755 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607be79a8f0 con 0x5607be415da0 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f52a00640 10 client.27752 handle_command_reply: tid=0 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 client.27752 shutdown 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 -- 172.21.15.39:0/3768671755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 msgr2=0x5607be83df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be83df10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8f60004950 tx=0x7f8f600098d0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 -- 172.21.15.39:0/3768671755 shutdown_connections 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5607be415da0 0x5607be8af3e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5607be461390 0x5607be83e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5607be482070 0x5607be83e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 --2- 172.21.15.39:0/3768671755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5607be4591b0 0x5607be83df10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 -- 172.21.15.39:0/3768671755 >> 172.21.15.39:0/3768671755 conn(0x5607be476460 msgr2=0x5607be8335e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f52a00640 0 client.27752 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.573+0000 7f8f7a043740 1 -- 172.21.15.39:0/3768671755 shutdown_connections 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.574+0000 7f8f7a043740 1 -- 172.21.15.39:0/3768671755 wait complete. 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.574+0000 7f8f7a043740 20 client.27752 trim_cache size 0 max 16384 2024-07-04T08:25:04.567 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:25:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.574+0000 7f8f73a00640 1 -- 172.21.15.39:0/972155788 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8f540b29e0 msgr2=0x7f8f540b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.574+0000 7f8f73a00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8f540b29e0 0x7f8f540b4ef0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8f64011630 tx=0x7f8f64009500 comp rx=0 tx=0).stop 2024-07-04T08:25:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 -- 172.21.15.39:0/972155788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6c0755f0 msgr2=0x7f8f6c19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6c0755f0 0x7f8f6c19c1e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8f5802a930 tx=0x7f8f58031a20 comp rx=0 tx=0).stop 2024-07-04T08:25:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 -- 172.21.15.39:0/972155788 shutdown_connections 2024-07-04T08:25:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8f540b29e0 0x7f8f540b4ef0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8f6c10f670 0x7f8f6c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f6c0755f0 0x7f8f6c19c1e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 --2- 172.21.15.39:0/972155788 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f6c074ca0 0x7f8f6c19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:04.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 -- 172.21.15.39:0/972155788 >> 172.21.15.39:0/972155788 conn(0x7f8f6c07ada0 msgr2=0x7f8f6c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:04.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 -- 172.21.15.39:0/972155788 shutdown_connections 2024-07-04T08:25:04.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:04.575+0000 7f8f73a00640 1 -- 172.21.15.39:0/972155788 wait complete. 2024-07-04T08:25:04.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:04 smithi050 ceph-mon[36577]: osdmap e99: 12 total, 12 up, 12 in 2024-07-04T08:25:05.511 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:05 smithi039 ceph-mon[32062]: pgmap v783: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-07-04T08:25:05.514 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:25:05.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:05 smithi133 ceph-mon[37050]: pgmap v783: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-07-04T08:25:05.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:05 smithi050 ceph-mon[36577]: pgmap v783: 129 pgs: 129 active+clean; 14 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-07-04T08:25:05.981 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:25:06.430 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:06 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:25:06.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:06 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:25:06.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:06 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:25:07.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:07 smithi133 ceph-mon[37050]: pgmap v784: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 32 op/s 2024-07-04T08:25:07.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:07 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:25:07.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:07 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:25:07.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:07 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:25:07.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:07 smithi039 ceph-mon[32062]: pgmap v784: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 32 op/s 2024-07-04T08:25:07.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:07 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:25:07.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:07 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:25:07.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:07 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:25:07.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:07 smithi050 ceph-mon[36577]: pgmap v784: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 32 op/s 2024-07-04T08:25:07.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:07 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:25:07.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:07 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:25:07.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:07 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:25:08.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.186+0000 7fb071c00640 1 -- 172.21.15.39:0/3203805027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb06c108840 msgr2=0x7fb06c108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:08.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.186+0000 7fb071c00640 1 --2- 172.21.15.39:0/3203805027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb06c108840 0x7fb06c108c50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb054009900 tx=0x7fb05402a410 comp rx=0 tx=0).stop 2024-07-04T08:25:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.186+0000 7fb071c00640 1 -- 172.21.15.39:0/3203805027 shutdown_connections 2024-07-04T08:25:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.186+0000 7fb071c00640 1 --2- 172.21.15.39:0/3203805027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb06c108840 0x7fb06c108c50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.186+0000 7fb071c00640 1 --2- 172.21.15.39:0/3203805027 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb06c109190 0x7fb06c10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.186+0000 7fb071c00640 1 --2- 172.21.15.39:0/3203805027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb06c102f00 0x7fb06c103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.186+0000 7fb071c00640 1 -- 172.21.15.39:0/3203805027 >> 172.21.15.39:0/3203805027 conn(0x7fb06c0fe950 msgr2=0x7fb06c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.186+0000 7fb071c00640 1 -- 172.21.15.39:0/3203805027 shutdown_connections 2024-07-04T08:25:08.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.186+0000 7fb071c00640 1 -- 172.21.15.39:0/3203805027 wait complete. 2024-07-04T08:25:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.188+0000 7fb071c00640 1 Processor -- start 2024-07-04T08:25:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.188+0000 7fb071c00640 1 -- start start 2024-07-04T08:25:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb071c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb06c102f00 0x7fb06c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb071c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb06c108840 0x7fb06c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb071c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb06c109190 0x7fb06c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb071c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c112ea0 con 0x7fb06c109190 2024-07-04T08:25:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb071c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c112d20 con 0x7fb06c102f00 2024-07-04T08:25:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb071c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c113020 con 0x7fb06c108840 2024-07-04T08:25:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb071200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb06c109190 0x7fb06c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb06be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb06c102f00 0x7fb06c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb06be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb06c102f00 0x7fb06c19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:52102/0 (socket says 172.21.15.39:52102) 2024-07-04T08:25:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb06be00640 1 -- 172.21.15.39:0/1811850237 learned_addr learned my addr 172.21.15.39:0/1811850237 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:08.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.189+0000 7fb063400640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb06c108840 0x7fb06c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb06be00640 1 -- 172.21.15.39:0/1811850237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb06c108840 msgr2=0x7fb06c19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb06be00640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb06c108840 0x7fb06c19bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb06be00640 1 -- 172.21.15.39:0/1811850237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb06c109190 msgr2=0x7fb06c1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb06be00640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb06c109190 0x7fb06c1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb06be00640 1 -- 172.21.15.39:0/1811850237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c1a19c0 con 0x7fb06c102f00 2024-07-04T08:25:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb06be00640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb06c102f00 0x7fb06c19ba50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00eb90 tx=0x7fb05c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb069600640 1 -- 172.21.15.39:0/1811850237 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c004d20 con 0x7fb06c102f00 2024-07-04T08:25:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb069600640 1 -- 172.21.15.39:0/1811850237 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c00ee90 con 0x7fb06c102f00 2024-07-04T08:25:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb069600640 1 -- 172.21.15.39:0/1811850237 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c010670 con 0x7fb06c102f00 2024-07-04T08:25:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.190+0000 7fb071c00640 1 -- 172.21.15.39:0/1811850237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c1a1cb0 con 0x7fb06c102f00 2024-07-04T08:25:08.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.191+0000 7fb071c00640 1 -- 172.21.15.39:0/1811850237 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1a8410 con 0x7fb06c102f00 2024-07-04T08:25:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.194+0000 7fb0798ef740 1 Processor -- start 2024-07-04T08:25:08.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.194+0000 7fb0798ef740 1 -- start start 2024-07-04T08:25:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.194+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389584070 0x556389928950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.194+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 0x556389928e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.194+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x5563899293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.194+0000 7fb0798ef740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55638989eee0 con 0x556389563390 2024-07-04T08:25:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.194+0000 7fb0798ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55638988e480 con 0x55638955b1b0 2024-07-04T08:25:08.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.194+0000 7fb0798ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5563898924c0 con 0x556389584070 2024-07-04T08:25:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb063400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x5563899293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb069600640 1 -- 172.21.15.39:0/1811850237 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb05c005020 con 0x7fb06c102f00 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb071200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389584070 0x556389928950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb071200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389584070 0x556389928950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51358/0 (socket says 172.21.15.39:51358) 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb071200640 1 -- 172.21.15.39:0/397489901 learned_addr learned my addr 172.21.15.39:0/397489901 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb06be00640 1 --2- 172.21.15.39:0/397489901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 0x556389928e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb063400640 1 -- 172.21.15.39:0/397489901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389584070 msgr2=0x556389928950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb063400640 1 --2- 172.21.15.39:0/397489901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389584070 0x556389928950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb063400640 1 -- 172.21.15.39:0/397489901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 msgr2=0x556389928e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb063400640 1 --2- 172.21.15.39:0/397489901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 0x556389928e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.195+0000 7fb063400640 1 -- 172.21.15.39:0/397489901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556389826650 con 0x556389563390 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.196+0000 7fb071200640 1 --2- 172.21.15.39:0/397489901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389584070 0x556389928950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.196+0000 7fb063400640 1 --2- 172.21.15.39:0/397489901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x5563899293d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb054009230 tx=0x7fb054004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3a47c5b1957c01 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.196+0000 7fb061600640 1 -- 172.21.15.39:0/397489901 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054035040 con 0x556389563390 2024-07-04T08:25:08.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.196+0000 7fb061600640 1 -- 172.21.15.39:0/397489901 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0540054f0 con 0x556389563390 2024-07-04T08:25:08.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.196+0000 7fb061600640 1 -- 172.21.15.39:0/397489901 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540035f0 con 0x556389563390 2024-07-04T08:25:08.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.197+0000 7fb069600640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb0500b2960 0x7fb0500b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.197+0000 7fb0798ef740 1 -- 172.21.15.39:0/397489901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 msgr2=0x5563899293d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.197+0000 7fb0798ef740 1 --2- 172.21.15.39:0/397489901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x5563899293d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb054009230 tx=0x7fb054004040 comp rx=0 tx=0).stop 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.197+0000 7fb069600640 1 -- 172.21.15.39:0/1811850237 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9512+0+0 (secure 0 0 0) 0x7fb05c0e3090 con 0x7fb06c102f00 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.197+0000 7fb071200640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb0500b2960 0x7fb0500b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.198+0000 7fb0798ef740 1 -- 172.21.15.39:0/397489901 shutdown_connections 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.198+0000 7fb0798ef740 1 --2- 172.21.15.39:0/397489901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x5563899293d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.198+0000 7fb0798ef740 1 --2- 172.21.15.39:0/397489901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 0x556389928e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.198+0000 7fb0798ef740 1 --2- 172.21.15.39:0/397489901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389584070 0x556389928950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.198+0000 7fb0798ef740 1 -- 172.21.15.39:0/397489901 >> 172.21.15.39:0/397489901 conn(0x556389578460 msgr2=0x556389926840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.198+0000 7fb0798ef740 1 -- 172.21.15.39:0/397489901 shutdown_connections 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.198+0000 7fb0798ef740 1 -- 172.21.15.39:0/397489901 wait complete. 2024-07-04T08:25:08.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.198+0000 7fb071200640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb0500b2960 0x7fb0500b4e70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb064009ea0 tx=0x7fb064005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.198+0000 7fb0798ef740 1 -- start start 2024-07-04T08:25:08.191 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.199+0000 7fb0798ef740 1 Processor -- start 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.199+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 0x556389934570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.199+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389563390 0x556389934b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.199+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389584070 0x5563899403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.199+0000 7fb0798ef740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5563896f2700 con 0x556389584070 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.199+0000 7fb0798ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556389835bb0 con 0x55638955b1b0 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.199+0000 7fb0798ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5563899370d0 con 0x556389563390 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb0798ef740 10 client.0 _ll_register_callbacks cb 0x556389887570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb063400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389563390 0x556389934b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb06be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 0x556389934570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb063400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389563390 0x556389934b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51364/0 (socket says 172.21.15.39:51364) 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb063400640 1 -- 172.21.15.39:0/2439331726 learned_addr learned my addr 172.21.15.39:0/2439331726 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:08.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb071200640 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389584070 0x5563899403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb063400640 1 -- 172.21.15.39:0/2439331726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 msgr2=0x556389934570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb063400640 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 0x556389934570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb063400640 1 -- 172.21.15.39:0/2439331726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389584070 msgr2=0x5563899403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb063400640 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389584070 0x5563899403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.200+0000 7fb063400640 1 -- 172.21.15.39:0/2439331726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563895ef650 con 0x556389563390 2024-07-04T08:25:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.201+0000 7fb063400640 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389563390 0x556389934b10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb05402aa00 tx=0x7fb0540045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=415fbc2b62f7af15 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:08.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.201+0000 7fb05a000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:25:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.201+0000 7fb05a000640 1 -- 172.21.15.39:0/2439331726 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054035070 con 0x556389563390 2024-07-04T08:25:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.201+0000 7fb0798ef740 1 -- 172.21.15.39:0/2439331726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0540054f0 con 0x556389563390 2024-07-04T08:25:08.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.201+0000 7fb05a000640 1 -- 172.21.15.39:0/2439331726 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9512+0+0 (secure 0 0 0) 0x7fb054004e70 con 0x556389563390 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.201+0000 7fb05a000640 1 -- 172.21.15.39:0/2439331726 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb054058050 con 0x556389563390 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.201+0000 7fb0798ef740 10 client.27697 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.201+0000 7fb0798ef740 1 -- 172.21.15.39:0/2439331726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556389943160 con 0x556389563390 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.202+0000 7fb05a000640 1 -- 172.21.15.39:0/2439331726 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0540054f0 con 0x556389563390 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.202+0000 7fb05a000640 1 -- 172.21.15.39:0/2439331726 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb054003c30 con 0x556389563390 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.202+0000 7fb0798ef740 10 client.27697 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.202+0000 7fb0798ef740 10 client.27697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.202+0000 7fb0798ef740 20 client.27697 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.202+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563895429f0 0x5563899464c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.202+0000 7fb0798ef740 4 client.27697 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.202+0000 7fb0798ef740 1 -- 172.21.15.39:0/2439331726 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563898a0e30 con 0x5563895429f0 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.203+0000 7fb06be00640 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563895429f0 0x5563899464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.203+0000 7fb06be00640 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563895429f0 0x5563899464c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f02e3cb716f919 server_cookie=a5152aac0cf7103e in_seq=0 out_seq=0 2024-07-04T08:25:08.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.203+0000 7fb05a000640 10 client.27697 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:08.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb05a000640 1 -- 172.21.15.39:0/2439331726 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5563898a0e30 con 0x5563895429f0 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb05a000640 10 client.27697 handle_command_reply: tid=0 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb0798ef740 1 client.27697 shutdown 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb0798ef740 1 -- 172.21.15.39:0/2439331726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389563390 msgr2=0x556389934b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389563390 0x556389934b10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb05402aa00 tx=0x7fb0540045a0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb0798ef740 1 -- 172.21.15.39:0/2439331726 shutdown_connections 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5563895429f0 0x5563899464c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389584070 0x5563899403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556389563390 0x556389934b10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2439331726 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55638955b1b0 0x556389934570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb05a000640 0 client.27697 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.204+0000 7fb0798ef740 1 -- 172.21.15.39:0/2439331726 >> 172.21.15.39:0/2439331726 conn(0x556389578460 msgr2=0x55638992bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:08.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.205+0000 7fb0798ef740 1 -- 172.21.15.39:0/2439331726 shutdown_connections 2024-07-04T08:25:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.205+0000 7fb0798ef740 1 -- 172.21.15.39:0/2439331726 wait complete. 2024-07-04T08:25:08.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.205+0000 7fb0798ef740 20 client.27697 trim_cache size 0 max 16384 2024-07-04T08:25:08.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.220+0000 7fb0798ef740 1 Processor -- start 2024-07-04T08:25:08.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.220+0000 7fb0798ef740 1 -- start start 2024-07-04T08:25:08.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x55638992e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389940330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 0x556389940870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb0798ef740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b2420 con 0x556389563390 2024-07-04T08:25:08.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb0798ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b3f60 con 0x556389584070 2024-07-04T08:25:08.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb0798ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b2af0 con 0x55638955b1b0 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb06be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x55638992e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb063400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389940330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb06be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x55638992e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51380/0 (socket says 172.21.15.39:51380) 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb071200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 0x556389940870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb063400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389940330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47644/0 (socket says 172.21.15.39:47644) 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.221+0000 7fb06be00640 1 -- 172.21.15.39:0/2643905811 learned_addr learned my addr 172.21.15.39:0/2643905811 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb06be00640 1 -- 172.21.15.39:0/2643905811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 msgr2=0x556389940870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb06be00640 1 --2- 172.21.15.39:0/2643905811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 0x556389940870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb06be00640 1 -- 172.21.15.39:0/2643905811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 msgr2=0x556389940330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb06be00640 1 --2- 172.21.15.39:0/2643905811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389940330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb06be00640 1 -- 172.21.15.39:0/2643905811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556389936ab0 con 0x55638955b1b0 2024-07-04T08:25:08.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb063400640 1 --2- 172.21.15.39:0/2643905811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389940330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:25:08.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb071200640 1 --2- 172.21.15.39:0/2643905811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 0x556389940870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:25:08.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb06be00640 1 --2- 172.21.15.39:0/2643905811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x55638992e170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0e0400 tx=0x7fb05c005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e3da66e4a34ef8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:08.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb060c00640 1 -- 172.21.15.39:0/2643905811 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0a9a40 con 0x55638955b1b0 2024-07-04T08:25:08.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.222+0000 7fb060c00640 1 -- 172.21.15.39:0/2643905811 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c0aa630 con 0x55638955b1b0 2024-07-04T08:25:08.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.223+0000 7fb060c00640 1 -- 172.21.15.39:0/2643905811 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0e50e0 con 0x55638955b1b0 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.223+0000 7fb0798ef740 1 -- 172.21.15.39:0/2643905811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 msgr2=0x55638992e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.223+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2643905811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x55638992e170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0e0400 tx=0x7fb05c005870 comp rx=0 tx=0).stop 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.224+0000 7fb0798ef740 1 -- 172.21.15.39:0/2643905811 shutdown_connections 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.224+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2643905811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389940330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.224+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2643905811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 0x556389940870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.224+0000 7fb0798ef740 1 --2- 172.21.15.39:0/2643905811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x55638992e170 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.224+0000 7fb0798ef740 1 -- 172.21.15.39:0/2643905811 >> 172.21.15.39:0/2643905811 conn(0x556389578460 msgr2=0x55638992da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.224+0000 7fb0798ef740 1 -- 172.21.15.39:0/2643905811 shutdown_connections 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.224+0000 7fb0798ef740 1 -- 172.21.15.39:0/2643905811 wait complete. 2024-07-04T08:25:08.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.224+0000 7fb0798ef740 1 -- start start 2024-07-04T08:25:08.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.225+0000 7fb0798ef740 1 Processor -- start 2024-07-04T08:25:08.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.225+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x556389949e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.225+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389947360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.225+0000 7fb0798ef740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 0x556389947a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.225+0000 7fb0798ef740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b3890 con 0x556389563390 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.225+0000 7fb0798ef740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b31c0 con 0x556389584070 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.225+0000 7fb0798ef740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b0420 con 0x55638955b1b0 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.225+0000 7fb0798ef740 10 client.0 _ll_register_callbacks cb 0x7fb064009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb06be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x556389949e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb06be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x556389949e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51390/0 (socket says 172.21.15.39:51390) 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb06be00640 1 -- 172.21.15.39:0/1278828907 learned_addr learned my addr 172.21.15.39:0/1278828907 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb063400640 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389947360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb071200640 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 0x556389947a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb06be00640 1 -- 172.21.15.39:0/1278828907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 msgr2=0x556389947a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb06be00640 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 0x556389947a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb06be00640 1 -- 172.21.15.39:0/1278828907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 msgr2=0x556389947360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb06be00640 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389947360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb06be00640 1 -- 172.21.15.39:0/1278828907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563899ada10 con 0x55638955b1b0 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb063400640 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389947360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb06be00640 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x556389949e10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0a5fb0 tx=0x7fb05c0a9b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66b5cd2db88d4940 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.226+0000 7fb05a000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.227+0000 7fb05a000640 1 -- 172.21.15.39:0/1278828907 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0e0830 con 0x55638955b1b0 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.227+0000 7fb05a000640 1 -- 172.21.15.39:0/1278828907 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9512+0+0 (secure 0 0 0) 0x7fb05c0e50d0 con 0x55638955b1b0 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.227+0000 7fb0798ef740 1 -- 172.21.15.39:0/1278828907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c0aa630 con 0x55638955b1b0 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.227+0000 7fb05a000640 1 -- 172.21.15.39:0/1278828907 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb05c0ae2e0 con 0x55638955b1b0 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.227+0000 7fb0798ef740 10 client.27703 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.227+0000 7fb0798ef740 1 -- 172.21.15.39:0/1278828907 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5563899aeba0 con 0x55638955b1b0 2024-07-04T08:25:08.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.227+0000 7fb05a000640 1 -- 172.21.15.39:0/1278828907 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c0aa630 con 0x55638955b1b0 2024-07-04T08:25:08.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.227+0000 7fb05a000640 1 -- 172.21.15.39:0/1278828907 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb05c0e0af0 con 0x55638955b1b0 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.228+0000 7fb0798ef740 10 client.27703 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.228+0000 7fb0798ef740 10 client.27703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.228+0000 7fb0798ef740 20 client.27703 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.228+0000 7fb0798ef740 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556389517da0 0x5563899b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.228+0000 7fb0798ef740 4 client.27703 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.228+0000 7fb0798ef740 1 -- 172.21.15.39:0/1278828907 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55638989c8d0 con 0x556389517da0 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.228+0000 7fb063400640 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556389517da0 0x5563899b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb05a000640 10 client.27703 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb063400640 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556389517da0 0x5563899b1300 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=193c9674cc81b85c server_cookie=f9e6daccdeb5f8bb in_seq=0 out_seq=0 2024-07-04T08:25:08.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb05a000640 1 -- 172.21.15.39:0/1278828907 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55638989c8d0 con 0x556389517da0 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb05a000640 10 client.27703 handle_command_reply: tid=0 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb0798ef740 1 client.27703 shutdown 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb0798ef740 1 -- 172.21.15.39:0/1278828907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 msgr2=0x556389949e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb0798ef740 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x556389949e10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0a5fb0 tx=0x7fb05c0a9b50 comp rx=0 tx=0).stop 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb0798ef740 1 -- 172.21.15.39:0/1278828907 shutdown_connections 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb0798ef740 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556389517da0 0x5563899b1300 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb0798ef740 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556389563390 0x556389947360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb0798ef740 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556389584070 0x556389947a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb0798ef740 1 --2- 172.21.15.39:0/1278828907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55638955b1b0 0x556389949e10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb05a000640 0 client.27703 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.229+0000 7fb0798ef740 1 -- 172.21.15.39:0/1278828907 >> 172.21.15.39:0/1278828907 conn(0x556389578460 msgr2=0x556389929980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.230+0000 7fb0798ef740 1 -- 172.21.15.39:0/1278828907 shutdown_connections 2024-07-04T08:25:08.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.230+0000 7fb0798ef740 1 -- 172.21.15.39:0/1278828907 wait complete. 2024-07-04T08:25:08.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.230+0000 7fb0798ef740 20 client.27703 trim_cache size 0 max 16384 2024-07-04T08:25:08.224 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:25:08.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 -- 172.21.15.39:0/1811850237 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb0500b2960 msgr2=0x7fb0500b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb0500b2960 0x7fb0500b4e70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb064009ea0 tx=0x7fb064005f90 comp rx=0 tx=0).stop 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 -- 172.21.15.39:0/1811850237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb06c102f00 msgr2=0x7fb06c19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb06c102f00 0x7fb06c19ba50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00eb90 tx=0x7fb05c00c6a0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 -- 172.21.15.39:0/1811850237 shutdown_connections 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb0500b2960 0x7fb0500b4e70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb06c109190 0x7fb06c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb06c108840 0x7fb06c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 --2- 172.21.15.39:0/1811850237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb06c102f00 0x7fb06c19ba50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 -- 172.21.15.39:0/1811850237 >> 172.21.15.39:0/1811850237 conn(0x7fb06c0fe950 msgr2=0x7fb06c10dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 -- 172.21.15.39:0/1811850237 shutdown_connections 2024-07-04T08:25:08.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:08.231+0000 7fb071c00640 1 -- 172.21.15.39:0/1811850237 wait complete. 2024-07-04T08:25:08.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:08 smithi133 ceph-mon[37050]: pgmap v785: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 32 op/s 2024-07-04T08:25:08.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:08 smithi039 ceph-mon[32062]: pgmap v785: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 32 op/s 2024-07-04T08:25:08.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:08 smithi050 ceph-mon[36577]: pgmap v785: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 32 op/s 2024-07-04T08:25:10.122 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:25:10.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:25:10.631 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:25:11.342 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:11 smithi039 ceph-mon[32062]: pgmap v786: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 34 op/s 2024-07-04T08:25:11.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:11 smithi050 ceph-mon[36577]: pgmap v786: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 34 op/s 2024-07-04T08:25:11.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:11 smithi133 ceph-mon[37050]: pgmap v786: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 34 op/s 2024-07-04T08:25:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.033+0000 7f9252400640 1 -- 172.21.15.39:0/3882784703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f924c10b590 msgr2=0x7f924c111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:13.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.033+0000 7f9252400640 1 --2- 172.21.15.39:0/3882784703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f924c10b590 0x7f924c111dc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9244009ca0 tx=0x7f924402a8b0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.034+0000 7f9252400640 1 -- 172.21.15.39:0/3882784703 shutdown_connections 2024-07-04T08:25:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.034+0000 7f9252400640 1 --2- 172.21.15.39:0/3882784703 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f924c10ac40 0x7f924c10b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.034+0000 7f9252400640 1 --2- 172.21.15.39:0/3882784703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f924c10b590 0x7f924c111dc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.034+0000 7f9252400640 1 --2- 172.21.15.39:0/3882784703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f924c105300 0x7f924c105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.034+0000 7f9252400640 1 -- 172.21.15.39:0/3882784703 >> 172.21.15.39:0/3882784703 conn(0x7f924c100d10 msgr2=0x7f924c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.034+0000 7f9252400640 1 -- 172.21.15.39:0/3882784703 shutdown_connections 2024-07-04T08:25:13.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.034+0000 7f9252400640 1 -- 172.21.15.39:0/3882784703 wait complete. 2024-07-04T08:25:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.035+0000 7f9252400640 1 Processor -- start 2024-07-04T08:25:13.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.035+0000 7f9252400640 1 -- start start 2024-07-04T08:25:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f9252400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f924c105300 0x7f924c19bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f9252400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f924c10ac40 0x7f924c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f9252400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f924c10b590 0x7f924c1a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f9252400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f924c115240 con 0x7f924c10ac40 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f9252400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1150c0 con 0x7f924c105300 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f9252400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1153c0 con 0x7f924c10b590 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f924aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f924c10ac40 0x7f924c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f924aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f924c10ac40 0x7f924c19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52014/0 (socket says 172.21.15.39:52014) 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f924aa00640 1 -- 172.21.15.39:0/623485601 learned_addr learned my addr 172.21.15.39:0/623485601 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.036+0000 7f924b400640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f924c105300 0x7f924c19bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.037+0000 7f924be00640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f924c10b590 0x7f924c1a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.037+0000 7f924aa00640 1 -- 172.21.15.39:0/623485601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f924c10b590 msgr2=0x7f924c1a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:13.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.037+0000 7f924aa00640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f924c10b590 0x7f924c1a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.037+0000 7f924aa00640 1 -- 172.21.15.39:0/623485601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f924c105300 msgr2=0x7f924c19bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.037+0000 7f924aa00640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f924c105300 0x7f924c19bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.037+0000 7f924aa00640 1 -- 172.21.15.39:0/623485601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924c1a1b80 con 0x7f924c10ac40 2024-07-04T08:25:13.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.037+0000 7f924aa00640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f924c10ac40 0x7f924c19c1b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f923400e970 tx=0x7f923400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.038+0000 7f9243e00640 1 -- 172.21.15.39:0/623485601 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923400cd80 con 0x7f924c10ac40 2024-07-04T08:25:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.038+0000 7f9243e00640 1 -- 172.21.15.39:0/623485601 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92340035f0 con 0x7f924c10ac40 2024-07-04T08:25:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.038+0000 7f9243e00640 1 -- 172.21.15.39:0/623485601 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234010690 con 0x7f924c10ac40 2024-07-04T08:25:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.038+0000 7f9252400640 1 -- 172.21.15.39:0/623485601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924c1a1e70 con 0x7f924c10ac40 2024-07-04T08:25:13.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.038+0000 7f9252400640 1 -- 172.21.15.39:0/623485601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924c1a85d0 con 0x7f924c10ac40 2024-07-04T08:25:13.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.041+0000 7f925894a740 1 Processor -- start 2024-07-04T08:25:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f925894a740 1 -- start start 2024-07-04T08:25:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f925894a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a113640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f925894a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a113b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f9243e00640 1 -- 172.21.15.39:0/623485601 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9234003aa0 con 0x7f924c10ac40 2024-07-04T08:25:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f925894a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f925894a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56002a03d8e0 con 0x560029d6f350 2024-07-04T08:25:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f925894a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56002a089ec0 con 0x560029d46490 2024-07-04T08:25:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f925894a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56002a079460 con 0x560029d4e670 2024-07-04T08:25:13.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f924be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a113640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f924b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f924be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a113640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52018/0 (socket says 172.21.15.39:52018) 2024-07-04T08:25:13.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f924be00640 1 -- 172.21.15.39:0/2565379616 learned_addr learned my addr 172.21.15.39:0/2565379616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:13.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.042+0000 7f924aa00640 1 --2- 172.21.15.39:0/2565379616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.043+0000 7f924b400640 1 -- 172.21.15.39:0/2565379616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 msgr2=0x56002a1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:13.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.043+0000 7f924b400640 1 --2- 172.21.15.39:0/2565379616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.043+0000 7f924b400640 1 -- 172.21.15.39:0/2565379616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 msgr2=0x56002a113640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:13.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.043+0000 7f924b400640 1 --2- 172.21.15.39:0/2565379616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a113640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.043+0000 7f924b400640 1 -- 172.21.15.39:0/2565379616 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56002a011280 con 0x560029d46490 2024-07-04T08:25:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.043+0000 7f924b400640 1 --2- 172.21.15.39:0/2565379616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a113b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f923c009c20 tx=0x7f923c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=702f5e5d2afc9933 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.044+0000 7f9240c00640 1 -- 172.21.15.39:0/2565379616 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c01e070 con 0x560029d46490 2024-07-04T08:25:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.044+0000 7f9240c00640 1 -- 172.21.15.39:0/2565379616 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f923c009dd0 con 0x560029d46490 2024-07-04T08:25:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.044+0000 7f9240c00640 1 -- 172.21.15.39:0/2565379616 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c018730 con 0x560029d46490 2024-07-04T08:25:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.044+0000 7f9243e00640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92300b2960 0x7f92300b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.044+0000 7f9243e00640 1 -- 172.21.15.39:0/623485601 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9512+0+0 (secure 0 0 0) 0x7f92340e2c10 con 0x7f924c10ac40 2024-07-04T08:25:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.044+0000 7f924be00640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92300b2960 0x7f92300b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f925894a740 1 -- 172.21.15.39:0/2565379616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 msgr2=0x56002a113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f925894a740 1 --2- 172.21.15.39:0/2565379616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a113b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f923c009c20 tx=0x7f923c011810 comp rx=0 tx=0).stop 2024-07-04T08:25:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f925894a740 1 -- 172.21.15.39:0/2565379616 shutdown_connections 2024-07-04T08:25:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f925894a740 1 --2- 172.21.15.39:0/2565379616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f925894a740 1 --2- 172.21.15.39:0/2565379616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a113b80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f925894a740 1 --2- 172.21.15.39:0/2565379616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a113640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f925894a740 1 -- 172.21.15.39:0/2565379616 >> 172.21.15.39:0/2565379616 conn(0x560029d63740 msgr2=0x56002a111530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f925894a740 1 -- 172.21.15.39:0/2565379616 shutdown_connections 2024-07-04T08:25:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f924be00640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92300b2960 0x7f92300b4e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9244035000 tx=0x7f92440093a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:13.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.045+0000 7f925894a740 1 -- 172.21.15.39:0/2565379616 wait complete. 2024-07-04T08:25:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.046+0000 7f925894a740 1 -- start start 2024-07-04T08:25:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.046+0000 7f925894a740 1 Processor -- start 2024-07-04T08:25:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f925894a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a11f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f925894a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a11f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f925894a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a12afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f925894a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56002a020760 con 0x560029d6f350 2024-07-04T08:25:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f925894a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56002a07d4a0 con 0x560029d46490 2024-07-04T08:25:13.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f925894a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560029edd890 con 0x560029d4e670 2024-07-04T08:25:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f925894a740 10 client.0 _ll_register_callbacks cb 0x56002a0212b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f924be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a12afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f924aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a11f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f924aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a11f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53104/0 (socket says 172.21.15.39:53104) 2024-07-04T08:25:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f924aa00640 1 -- 172.21.15.39:0/1550002388 learned_addr learned my addr 172.21.15.39:0/1550002388 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.047+0000 7f924b400640 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a11f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f924aa00640 1 -- 172.21.15.39:0/1550002388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 msgr2=0x56002a11f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:13.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f924aa00640 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a11f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f924aa00640 1 -- 172.21.15.39:0/1550002388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 msgr2=0x56002a12afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f924aa00640 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a12afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f924aa00640 1 -- 172.21.15.39:0/1550002388 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560029dda930 con 0x560029d4e670 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f924be00640 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a12afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f924aa00640 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a11f6e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f92340a9a40 tx=0x7f92340a5a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deec0ed0bbd4b091 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f9239600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f9239600640 1 -- 172.21.15.39:0/1550002388 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340e03b0 con 0x560029d4e670 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f9239600640 1 -- 172.21.15.39:0/1550002388 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9512+0+0 (secure 0 0 0) 0x7f92340a60a0 con 0x560029d4e670 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.048+0000 7f925894a740 1 -- 172.21.15.39:0/1550002388 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f923c009dd0 con 0x560029d4e670 2024-07-04T08:25:13.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.049+0000 7f9239600640 1 -- 172.21.15.39:0/1550002388 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92340a95c0 con 0x560029d4e670 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.049+0000 7f925894a740 10 client.27715 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.049+0000 7f925894a740 1 -- 172.21.15.39:0/1550002388 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56002a12dc90 con 0x560029d4e670 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.049+0000 7f9239600640 1 -- 172.21.15.39:0/1550002388 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f923c009dd0 con 0x560029d4e670 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.049+0000 7f9239600640 1 -- 172.21.15.39:0/1550002388 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f92340eb070 con 0x560029d4e670 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.050+0000 7f925894a740 10 client.27715 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.050+0000 7f925894a740 10 client.27715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.050+0000 7f925894a740 20 client.27715 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.050+0000 7f925894a740 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560029d2dcd0 0x56002a131160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.050+0000 7f925894a740 4 client.27715 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.050+0000 7f925894a740 1 -- 172.21.15.39:0/1550002388 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56002a08be10 con 0x560029d2dcd0 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.050+0000 7f924b400640 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560029d2dcd0 0x56002a131160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.051+0000 7f924b400640 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560029d2dcd0 0x56002a131160 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e70f29f18d27d9 server_cookie=902281035200f84 in_seq=0 out_seq=0 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.051+0000 7f9239600640 10 client.27715 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f9239600640 1 -- 172.21.15.39:0/1550002388 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56002a08be10 con 0x560029d2dcd0 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f9239600640 10 client.27715 handle_command_reply: tid=0 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 client.27715 shutdown 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 -- 172.21.15.39:0/1550002388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 msgr2=0x56002a11f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a11f6e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f92340a9a40 tx=0x7f92340a5a30 comp rx=0 tx=0).stop 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 -- 172.21.15.39:0/1550002388 shutdown_connections 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560029d2dcd0 0x56002a131160 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a12afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a11f6e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 --2- 172.21.15.39:0/1550002388 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a11f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f9239600640 0 client.27715 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 -- 172.21.15.39:0/1550002388 >> 172.21.15.39:0/1550002388 conn(0x560029d63740 msgr2=0x56002a116820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 -- 172.21.15.39:0/1550002388 shutdown_connections 2024-07-04T08:25:13.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.052+0000 7f925894a740 1 -- 172.21.15.39:0/1550002388 wait complete. 2024-07-04T08:25:13.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.053+0000 7f925894a740 20 client.27715 trim_cache size 0 max 16384 2024-07-04T08:25:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.073+0000 7f925894a740 1 Processor -- start 2024-07-04T08:25:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.073+0000 7f925894a740 1 -- start start 2024-07-04T08:25:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f925894a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d46490 0x56002a118720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f925894a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a118c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f925894a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d6f350 0x56002a12adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f925894a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1b2690 con 0x560029d46490 2024-07-04T08:25:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f925894a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1b41d0 con 0x560029d6f350 2024-07-04T08:25:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f925894a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1b2d60 con 0x560029d4e670 2024-07-04T08:25:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f924b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d46490 0x56002a118720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f924b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d46490 0x56002a118720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52048/0 (socket says 172.21.15.39:52048) 2024-07-04T08:25:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f924b400640 1 -- 172.21.15.39:0/593994547 learned_addr learned my addr 172.21.15.39:0/593994547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f924aa00640 1 --2- 172.21.15.39:0/593994547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a118c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.074+0000 7f924be00640 1 --2- 172.21.15.39:0/593994547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d6f350 0x56002a12adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.075+0000 7f924aa00640 1 -- 172.21.15.39:0/593994547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d6f350 msgr2=0x56002a12adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.075+0000 7f924aa00640 1 --2- 172.21.15.39:0/593994547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d6f350 0x56002a12adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.075+0000 7f924aa00640 1 -- 172.21.15.39:0/593994547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d46490 msgr2=0x56002a118720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.075+0000 7f924aa00640 1 --2- 172.21.15.39:0/593994547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d46490 0x56002a118720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.075+0000 7f924aa00640 1 -- 172.21.15.39:0/593994547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56002a11fa70 con 0x560029d4e670 2024-07-04T08:25:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.075+0000 7f924aa00640 1 --2- 172.21.15.39:0/593994547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a118c60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f92340098c0 tx=0x7f9234000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb7fbc011023a8ce server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.075+0000 7f923be00640 1 -- 172.21.15.39:0/593994547 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340e03b0 con 0x560029d4e670 2024-07-04T08:25:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.075+0000 7f923be00640 1 -- 172.21.15.39:0/593994547 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92340aa1b0 con 0x560029d4e670 2024-07-04T08:25:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.075+0000 7f923be00640 1 -- 172.21.15.39:0/593994547 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340a5240 con 0x560029d4e670 2024-07-04T08:25:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.076+0000 7f925894a740 1 -- 172.21.15.39:0/593994547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 msgr2=0x56002a118c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.076+0000 7f925894a740 1 --2- 172.21.15.39:0/593994547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a118c60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f92340098c0 tx=0x7f9234000bc0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.076+0000 7f925894a740 1 -- 172.21.15.39:0/593994547 shutdown_connections 2024-07-04T08:25:13.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.076+0000 7f925894a740 1 --2- 172.21.15.39:0/593994547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d6f350 0x56002a12adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.076+0000 7f925894a740 1 --2- 172.21.15.39:0/593994547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a118c60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.076+0000 7f925894a740 1 --2- 172.21.15.39:0/593994547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d46490 0x56002a118720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.076+0000 7f925894a740 1 -- 172.21.15.39:0/593994547 >> 172.21.15.39:0/593994547 conn(0x560029d63740 msgr2=0x56002a116db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.077+0000 7f925894a740 1 -- 172.21.15.39:0/593994547 shutdown_connections 2024-07-04T08:25:13.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.077+0000 7f925894a740 1 -- 172.21.15.39:0/593994547 wait complete. 2024-07-04T08:25:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.077+0000 7f925894a740 1 -- start start 2024-07-04T08:25:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.078+0000 7f925894a740 1 Processor -- start 2024-07-04T08:25:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.078+0000 7f925894a740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a126f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.078+0000 7f925894a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a127470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.078+0000 7f925894a740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a121ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.078+0000 7f925894a740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1b3b00 con 0x560029d6f350 2024-07-04T08:25:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.078+0000 7f925894a740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1b3430 con 0x560029d46490 2024-07-04T08:25:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.078+0000 7f925894a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f924c1b0690 con 0x560029d4e670 2024-07-04T08:25:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.078+0000 7f925894a740 10 client.0 _ll_register_callbacks cb 0x7f92340e63b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.078+0000 7f924be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a121ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a127470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a121ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52060/0 (socket says 172.21.15.39:52060) 2024-07-04T08:25:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924be00640 1 -- 172.21.15.39:0/526474139 learned_addr learned my addr 172.21.15.39:0/526474139 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a127470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53126/0 (socket says 172.21.15.39:53126) 2024-07-04T08:25:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924b400640 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a126f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924be00640 1 -- 172.21.15.39:0/526474139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 msgr2=0x56002a127470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924be00640 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a127470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924be00640 1 -- 172.21.15.39:0/526474139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 msgr2=0x56002a126f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924be00640 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a126f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.079+0000 7f924be00640 1 -- 172.21.15.39:0/526474139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56002a11ce30 con 0x560029d6f350 2024-07-04T08:25:13.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.080+0000 7f924be00640 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a121ee0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9244009a60 tx=0x7f92440095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32e3de9b5100cc0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.080+0000 7f9239600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.080+0000 7f9239600640 1 -- 172.21.15.39:0/526474139 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9244058060 con 0x560029d6f350 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.080+0000 7f925894a740 1 -- 172.21.15.39:0/526474139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92340aa1b0 con 0x560029d6f350 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.080+0000 7f9239600640 1 -- 172.21.15.39:0/526474139 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9512+0+0 (secure 0 0 0) 0x7f9244004250 con 0x560029d6f350 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.080+0000 7f9239600640 1 -- 172.21.15.39:0/526474139 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f924405f050 con 0x560029d6f350 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.080+0000 7f925894a740 10 client.17655 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.080+0000 7f925894a740 1 -- 172.21.15.39:0/526474139 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56002a199c70 con 0x560029d6f350 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.081+0000 7f9239600640 1 -- 172.21.15.39:0/526474139 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92340aa1b0 con 0x560029d6f350 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.081+0000 7f9239600640 1 -- 172.21.15.39:0/526474139 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9244049a60 con 0x560029d6f350 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.081+0000 7f925894a740 10 client.17655 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.081+0000 7f925894a740 10 client.17655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:25:13.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.081+0000 7f925894a740 20 client.17655 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:13.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.081+0000 7f925894a740 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560029d03080 0x56002a19c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:13.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.081+0000 7f925894a740 4 client.17655 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:25:13.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.081+0000 7f925894a740 1 -- 172.21.15.39:0/526474139 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56002a0878b0 con 0x560029d03080 2024-07-04T08:25:13.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.082+0000 7f924b400640 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560029d03080 0x56002a19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.082+0000 7f924b400640 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560029d03080 0x56002a19c440 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a49cbbc54966cbfc server_cookie=efb148e0018d78c0 in_seq=0 out_seq=0 2024-07-04T08:25:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.082+0000 7f9239600640 10 client.17655 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.083+0000 7f9239600640 1 -- 172.21.15.39:0/526474139 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56002a0878b0 con 0x560029d03080 2024-07-04T08:25:13.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.083+0000 7f9239600640 10 client.17655 handle_command_reply: tid=0 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.083+0000 7f925894a740 1 client.17655 shutdown 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 -- 172.21.15.39:0/526474139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 msgr2=0x56002a121ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a121ee0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9244009a60 tx=0x7f92440095c0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 -- 172.21.15.39:0/526474139 shutdown_connections 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x560029d03080 0x56002a19c440 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x560029d6f350 0x56002a121ee0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560029d4e670 0x56002a127470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 --2- 172.21.15.39:0/526474139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560029d46490 0x56002a126f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 -- 172.21.15.39:0/526474139 >> 172.21.15.39:0/526474139 conn(0x560029d63740 msgr2=0x56002a116db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f9239600640 0 client.17655 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:13.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 -- 172.21.15.39:0/526474139 shutdown_connections 2024-07-04T08:25:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.084+0000 7f925894a740 1 -- 172.21.15.39:0/526474139 wait complete. 2024-07-04T08:25:13.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.085+0000 7f925894a740 20 client.17655 trim_cache size 0 max 16384 2024-07-04T08:25:13.080 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:25:13.080 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:25:13.080 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "212b7312-ba59-47b6-9356-e0ad07696229", 2024-07-04T08:25:13.080 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:25:13.080 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:25:13.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 -- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92300b2960 msgr2=0x7f92300b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:13.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92300b2960 0x7f92300b4e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9244035000 tx=0x7f92440093a0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 -- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f924c10ac40 msgr2=0x7f924c19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f924c10ac40 0x7f924c19c1b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f923400e970 tx=0x7f923400ee30 comp rx=0 tx=0).stop 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 -- 172.21.15.39:0/623485601 shutdown_connections 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92300b2960 0x7f92300b4e70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f924c10ac40 0x7f924c19c1b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f924c10b590 0x7f924c1a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 --2- 172.21.15.39:0/623485601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f924c105300 0x7f924c19bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.086+0000 7f9252400640 1 -- 172.21.15.39:0/623485601 >> 172.21.15.39:0/623485601 conn(0x7f924c100d10 msgr2=0x7f924c108140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.087+0000 7f9252400640 1 -- 172.21.15.39:0/623485601 shutdown_connections 2024-07-04T08:25:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:13.087+0000 7f9252400640 1 -- 172.21.15.39:0/623485601 wait complete. 2024-07-04T08:25:13.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:13 smithi039 ceph-mon[32062]: pgmap v787: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-07-04T08:25:13.387 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:13 smithi050 ceph-mon[36577]: pgmap v787: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-07-04T08:25:13.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:13 smithi133 ceph-mon[37050]: pgmap v787: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-07-04T08:25:13.973 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:25:14.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:14 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:25:14.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:14 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:25:14.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:14 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:25:14.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:14 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:25:14.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:14 smithi039 ceph-mon[32062]: osdmap e100: 12 total, 12 up, 12 in 2024-07-04T08:25:14.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:14 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:25:14.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:14 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:25:14.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:14 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:25:14.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:14 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:25:14.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:14 smithi050 ceph-mon[36577]: osdmap e100: 12 total, 12 up, 12 in 2024-07-04T08:25:14.451 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:25:14.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:14 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:25:14.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:14 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:25:14.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:14 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:25:14.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:14 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:25:14.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:14 smithi133 ceph-mon[37050]: osdmap e100: 12 total, 12 up, 12 in 2024-07-04T08:25:15.248 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:15 smithi039 ceph-mon[32062]: pgmap v789: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-07-04T08:25:15.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:15 smithi050 ceph-mon[36577]: pgmap v789: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-07-04T08:25:15.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:15 smithi133 ceph-mon[37050]: pgmap v789: 129 pgs: 129 active+clean; 14 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 102 KiB/s wr, 35 op/s 2024-07-04T08:25:16.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.751+0000 7f34a7800640 1 -- 172.21.15.39:0/179224454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a01078f0 msgr2=0x7f34a0107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:16.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.751+0000 7f34a7800640 1 --2- 172.21.15.39:0/179224454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a01078f0 0x7f34a0107d00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3488009860 tx=0x7f348802a340 comp rx=0 tx=0).stop 2024-07-04T08:25:16.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.751+0000 7f34a7800640 1 -- 172.21.15.39:0/179224454 shutdown_connections 2024-07-04T08:25:16.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.751+0000 7f34a7800640 1 --2- 172.21.15.39:0/179224454 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a0108240 0x7f34a010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.751+0000 7f34a7800640 1 --2- 172.21.15.39:0/179224454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a01078f0 0x7f34a0107d00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.751+0000 7f34a7800640 1 --2- 172.21.15.39:0/179224454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34a0101fb0 0x7f34a01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.751+0000 7f34a7800640 1 -- 172.21.15.39:0/179224454 >> 172.21.15.39:0/179224454 conn(0x7f34a0078c80 msgr2=0x7f34a0079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:16.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.751+0000 7f34a7800640 1 -- 172.21.15.39:0/179224454 shutdown_connections 2024-07-04T08:25:16.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.751+0000 7f34a7800640 1 -- 172.21.15.39:0/179224454 wait complete. 2024-07-04T08:25:16.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.752+0000 7f34a7800640 1 Processor -- start 2024-07-04T08:25:16.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.753+0000 7f34a7800640 1 -- start start 2024-07-04T08:25:16.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.753+0000 7f34a7800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a0101fb0 0x7f34a00febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.753+0000 7f34a7800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a01078f0 0x7f34a00ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.753+0000 7f34a7800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34a0108240 0x7f34a00ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.753+0000 7f34a7800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0111eb0 con 0x7f34a01078f0 2024-07-04T08:25:16.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.753+0000 7f34a7800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0111d30 con 0x7f34a0101fb0 2024-07-04T08:25:16.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.753+0000 7f34a7800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34a0112030 con 0x7f34a0108240 2024-07-04T08:25:16.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f349fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a01078f0 0x7f34a00ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f34a5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34a0108240 0x7f34a00ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f349fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a01078f0 0x7f34a00ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52080/0 (socket says 172.21.15.39:52080) 2024-07-04T08:25:16.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f34a4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a0101fb0 0x7f34a00febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f349fe00640 1 -- 172.21.15.39:0/3766354942 learned_addr learned my addr 172.21.15.39:0/3766354942 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:16.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f34a4e00640 1 -- 172.21.15.39:0/3766354942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34a0108240 msgr2=0x7f34a00ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:16.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f34a4e00640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34a0108240 0x7f34a00ff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f34a4e00640 1 -- 172.21.15.39:0/3766354942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a01078f0 msgr2=0x7f34a00ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:16.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f34a4e00640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a01078f0 0x7f34a00ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.754+0000 7f34a4e00640 1 -- 172.21.15.39:0/3766354942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a01a7c90 con 0x7f34a0101fb0 2024-07-04T08:25:16.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.755+0000 7f34a4e00640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a0101fb0 0x7f34a00febc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3490009fd0 tx=0x7f349000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:16.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.755+0000 7f349d600640 1 -- 172.21.15.39:0/3766354942 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490004b20 con 0x7f34a0101fb0 2024-07-04T08:25:16.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.755+0000 7f349d600640 1 -- 172.21.15.39:0/3766354942 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3490004cc0 con 0x7f34a0101fb0 2024-07-04T08:25:16.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.755+0000 7f349d600640 1 -- 172.21.15.39:0/3766354942 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3490010720 con 0x7f34a0101fb0 2024-07-04T08:25:16.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.755+0000 7f34a7800640 1 -- 172.21.15.39:0/3766354942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a01a7f80 con 0x7f34a0101fb0 2024-07-04T08:25:16.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.755+0000 7f34a7800640 1 -- 172.21.15.39:0/3766354942 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a01a83e0 con 0x7f34a0101fb0 2024-07-04T08:25:16.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.759+0000 7f34add34740 1 Processor -- start 2024-07-04T08:25:16.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.759+0000 7f34add34740 1 -- start start 2024-07-04T08:25:16.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.759+0000 7f34add34740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.759+0000 7f34add34740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bce1b0 0x55d0c6f9bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.759+0000 7f34add34740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bd6390 0x55d0c6f9c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.759+0000 7f34add34740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d0c6f11ea0 con 0x55d0c6bce1b0 2024-07-04T08:25:16.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.759+0000 7f34add34740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d0c6f01440 con 0x55d0c6bf7070 2024-07-04T08:25:16.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.759+0000 7f34add34740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d0c6f05480 con 0x55d0c6bd6390 2024-07-04T08:25:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.760+0000 7f34a5800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.760+0000 7f34a5800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:39612/0 (socket says 172.21.15.39:39612) 2024-07-04T08:25:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.760+0000 7f34a5800640 1 -- 172.21.15.39:0/443284547 learned_addr learned my addr 172.21.15.39:0/443284547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.760+0000 7f349fe00640 1 --2- 172.21.15.39:0/443284547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bd6390 0x55d0c6f9c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.760+0000 7f34a5800640 1 -- 172.21.15.39:0/443284547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bd6390 msgr2=0x55d0c6f9c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.760+0000 7f34a5800640 1 --2- 172.21.15.39:0/443284547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bd6390 0x55d0c6f9c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.760+0000 7f34a5800640 1 -- 172.21.15.39:0/443284547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bce1b0 msgr2=0x55d0c6f9bcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:25:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.760+0000 7f34a5800640 1 --2- 172.21.15.39:0/443284547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bce1b0 0x55d0c6f9bcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.760+0000 7f34a5800640 1 -- 172.21.15.39:0/443284547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c6e99610 con 0x55d0c6bf7070 2024-07-04T08:25:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.761+0000 7f34a5800640 1 --2- 172.21.15.39:0/443284547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9b770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3498009c40 tx=0x7f3498011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90294de6ced01931 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.761+0000 7f349d600640 1 -- 172.21.15.39:0/3766354942 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3490005020 con 0x7f34a0101fb0 2024-07-04T08:25:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.761+0000 7f3496000640 1 -- 172.21.15.39:0/443284547 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498004230 con 0x55d0c6bf7070 2024-07-04T08:25:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.761+0000 7f3496000640 1 -- 172.21.15.39:0/443284547 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34980043d0 con 0x55d0c6bf7070 2024-07-04T08:25:16.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.761+0000 7f3496000640 1 -- 172.21.15.39:0/443284547 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498019800 con 0x55d0c6bf7070 2024-07-04T08:25:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.762+0000 7f34add34740 1 -- 172.21.15.39:0/443284547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bf7070 msgr2=0x55d0c6f9b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.762+0000 7f34add34740 1 --2- 172.21.15.39:0/443284547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9b770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3498009c40 tx=0x7f3498011810 comp rx=0 tx=0).stop 2024-07-04T08:25:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.762+0000 7f34add34740 1 -- 172.21.15.39:0/443284547 shutdown_connections 2024-07-04T08:25:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.762+0000 7f34add34740 1 --2- 172.21.15.39:0/443284547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bce1b0 0x55d0c6f9bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.762+0000 7f34add34740 1 --2- 172.21.15.39:0/443284547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bd6390 0x55d0c6f9c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.762+0000 7f34add34740 1 --2- 172.21.15.39:0/443284547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9b770 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.762+0000 7f34add34740 1 -- 172.21.15.39:0/443284547 >> 172.21.15.39:0/443284547 conn(0x55d0c6beb460 msgr2=0x55d0c6f99660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:16.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.762+0000 7f34add34740 1 -- 172.21.15.39:0/443284547 shutdown_connections 2024-07-04T08:25:16.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.762+0000 7f34add34740 1 -- 172.21.15.39:0/443284547 wait complete. 2024-07-04T08:25:16.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.763+0000 7f349d600640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34840b28f0 0x7f34840b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.763+0000 7f349d600640 1 -- 172.21.15.39:0/3766354942 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f34900e30e0 con 0x7f34a0101fb0 2024-07-04T08:25:16.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.763+0000 7f349fe00640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34840b28f0 0x7f34840b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.763+0000 7f34add34740 1 -- start start 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.764+0000 7f34add34740 1 Processor -- start 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.764+0000 7f349fe00640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34840b28f0 0x7f34840b4e00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3488005eb0 tx=0x7f348802c040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.764+0000 7f34add34740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fa73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.764+0000 7f34add34740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6fa7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.764+0000 7f34add34740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bf7070 0x55d0c6fb32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.764+0000 7f34add34740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d0c6d65700 con 0x55d0c6bce1b0 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.764+0000 7f34add34740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d0c6ea8b70 con 0x55d0c6bd6390 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.764+0000 7f34add34740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d0c6fa9f50 con 0x55d0c6bf7070 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.764+0000 7f34add34740 10 client.0 _ll_register_callbacks cb 0x55d0c6efa530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:16.756 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f34a5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bf7070 0x55d0c6fb32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f34a5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bf7070 0x55d0c6fb32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53158/0 (socket says 172.21.15.39:53158) 2024-07-04T08:25:16.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f34a5800640 1 -- 172.21.15.39:0/3645975723 learned_addr learned my addr 172.21.15.39:0/3645975723 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:16.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f34a4e00640 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fa73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.757 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f349fe00640 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6fa7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f34a5800640 1 -- 172.21.15.39:0/3645975723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 msgr2=0x55d0c6fa7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f34a5800640 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6fa7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f34a5800640 1 -- 172.21.15.39:0/3645975723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bce1b0 msgr2=0x55d0c6fa73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f34a5800640 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fa73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.765+0000 7f34a5800640 1 -- 172.21.15.39:0/3645975723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c6c62650 con 0x55d0c6bf7070 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.766+0000 7f34a5800640 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bf7070 0x55d0c6fb32a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f349801b040 tx=0x7f3498003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585ac2a8fbda2114 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.766+0000 7f348ea00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.766+0000 7f348ea00640 1 -- 172.21.15.39:0/3645975723 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498028070 con 0x55d0c6bf7070 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.766+0000 7f34add34740 1 -- 172.21.15.39:0/3645975723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34980043d0 con 0x55d0c6bf7070 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.766+0000 7f348ea00640 1 -- 172.21.15.39:0/3645975723 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f3498005040 con 0x55d0c6bf7070 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.766+0000 7f348ea00640 1 -- 172.21.15.39:0/3645975723 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3498034050 con 0x55d0c6bf7070 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.766+0000 7f34add34740 10 client.27733 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:16.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.766+0000 7f34add34740 1 -- 172.21.15.39:0/3645975723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0c6fb6010 con 0x55d0c6bf7070 2024-07-04T08:25:16.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.767+0000 7f348ea00640 1 -- 172.21.15.39:0/3645975723 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34980043d0 con 0x55d0c6bf7070 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.767+0000 7f348ea00640 1 -- 172.21.15.39:0/3645975723 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3498004990 con 0x55d0c6bf7070 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.767+0000 7f34add34740 10 client.27733 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.767+0000 7f34add34740 10 client.27733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.767+0000 7f34add34740 20 client.27733 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.767+0000 7f34add34740 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d0c6bb59f0 0x55d0c6fb9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.767+0000 7f34add34740 4 client.27733 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.767+0000 7f34add34740 1 -- 172.21.15.39:0/3645975723 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0c6f13df0 con 0x55d0c6bb59f0 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.768+0000 7f34a4e00640 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d0c6bb59f0 0x55d0c6fb9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.768+0000 7f34a4e00640 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d0c6bb59f0 0x55d0c6fb9430 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc93a82a91c072a6 server_cookie=9cc3db4209d2936c in_seq=0 out_seq=0 2024-07-04T08:25:16.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.768+0000 7f348ea00640 10 client.27733 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:16.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.769+0000 7f348ea00640 1 -- 172.21.15.39:0/3645975723 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d0c6f13df0 con 0x55d0c6bb59f0 2024-07-04T08:25:16.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.769+0000 7f348ea00640 10 client.27733 handle_command_reply: tid=0 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.769+0000 7f34add34740 1 client.27733 shutdown 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.769+0000 7f34add34740 1 -- 172.21.15.39:0/3645975723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bf7070 msgr2=0x55d0c6fb32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.769+0000 7f34add34740 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bf7070 0x55d0c6fb32a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f349801b040 tx=0x7f3498003820 comp rx=0 tx=0).stop 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f34add34740 1 -- 172.21.15.39:0/3645975723 shutdown_connections 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f34add34740 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d0c6bb59f0 0x55d0c6fb9430 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f34add34740 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bf7070 0x55d0c6fb32a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f34add34740 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6fa7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f34add34740 1 --2- 172.21.15.39:0/3645975723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fa73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f348ea00640 0 client.27733 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f34add34740 1 -- 172.21.15.39:0/3645975723 >> 172.21.15.39:0/3645975723 conn(0x55d0c6beb460 msgr2=0x55d0c6f9e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f34add34740 1 -- 172.21.15.39:0/3645975723 shutdown_connections 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f34add34740 1 -- 172.21.15.39:0/3645975723 wait complete. 2024-07-04T08:25:16.762 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.770+0000 7f34add34740 20 client.27733 trim_cache size 0 max 16384 2024-07-04T08:25:16.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.790+0000 7f34add34740 1 Processor -- start 2024-07-04T08:25:16.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.790+0000 7f34add34740 1 -- start start 2024-07-04T08:25:16.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f34add34740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fa90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f34add34740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6f9ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f34add34740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f34add34740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01b2510 con 0x55d0c6bf7070 2024-07-04T08:25:16.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f34add34740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01b4050 con 0x55d0c6bd6390 2024-07-04T08:25:16.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f34add34740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01b2be0 con 0x55d0c6bce1b0 2024-07-04T08:25:16.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f349fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6f9ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f34a5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f34a4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fa90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f349fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6f9ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:39642/0 (socket says 172.21.15.39:39642) 2024-07-04T08:25:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.791+0000 7f349fe00640 1 -- 172.21.15.39:0/2966691914 learned_addr learned my addr 172.21.15.39:0/2966691914 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.792+0000 7f34a4e00640 1 -- 172.21.15.39:0/2966691914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 msgr2=0x55d0c6f9ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.792+0000 7f34a4e00640 1 --2- 172.21.15.39:0/2966691914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6f9ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.792+0000 7f34a4e00640 1 -- 172.21.15.39:0/2966691914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 msgr2=0x55d0c6f9d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.792+0000 7f34a4e00640 1 --2- 172.21.15.39:0/2966691914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.792+0000 7f34a4e00640 1 -- 172.21.15.39:0/2966691914 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c6fa56e0 con 0x55d0c6bce1b0 2024-07-04T08:25:16.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.792+0000 7f34a4e00640 1 --2- 172.21.15.39:0/2966691914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fa90c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34900e0470 tx=0x7f349000c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db1449d8b5c78d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.793+0000 7f3495600640 1 -- 172.21.15.39:0/2966691914 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900c3070 con 0x55d0c6bce1b0 2024-07-04T08:25:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.793+0000 7f3495600640 1 -- 172.21.15.39:0/2966691914 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34900ae3b0 con 0x55d0c6bce1b0 2024-07-04T08:25:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.793+0000 7f3495600640 1 -- 172.21.15.39:0/2966691914 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900b2c30 con 0x55d0c6bce1b0 2024-07-04T08:25:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.794+0000 7f34add34740 1 -- 172.21.15.39:0/2966691914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 msgr2=0x55d0c6fa90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:16.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.794+0000 7f34add34740 1 --2- 172.21.15.39:0/2966691914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fa90c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34900e0470 tx=0x7f349000c890 comp rx=0 tx=0).stop 2024-07-04T08:25:16.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.794+0000 7f34add34740 1 -- 172.21.15.39:0/2966691914 shutdown_connections 2024-07-04T08:25:16.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.794+0000 7f34add34740 1 --2- 172.21.15.39:0/2966691914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 0x55d0c6f9d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.794+0000 7f34add34740 1 --2- 172.21.15.39:0/2966691914 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6f9ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.794+0000 7f34add34740 1 --2- 172.21.15.39:0/2966691914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fa90c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.794+0000 7f34add34740 1 -- 172.21.15.39:0/2966691914 >> 172.21.15.39:0/2966691914 conn(0x55d0c6beb460 msgr2=0x55d0c6f9c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:16.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.794+0000 7f34add34740 1 -- 172.21.15.39:0/2966691914 shutdown_connections 2024-07-04T08:25:16.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.794+0000 7f34add34740 1 -- 172.21.15.39:0/2966691914 wait complete. 2024-07-04T08:25:16.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.795+0000 7f34add34740 1 -- start start 2024-07-04T08:25:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.795+0000 7f34add34740 1 Processor -- start 2024-07-04T08:25:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.796+0000 7f34add34740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.796+0000 7f34add34740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6fab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.796+0000 7f34add34740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 0x55d0c7020a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.796+0000 7f34add34740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01b3980 con 0x55d0c6bf7070 2024-07-04T08:25:16.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.796+0000 7f34add34740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01b32b0 con 0x55d0c6bd6390 2024-07-04T08:25:16.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.796+0000 7f34add34740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34a01b04f0 con 0x55d0c6bce1b0 2024-07-04T08:25:16.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.796+0000 7f34add34740 10 client.0 _ll_register_callbacks cb 0x7f3498005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f34a4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f34a5800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 0x55d0c7020a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f34a4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53178/0 (socket says 172.21.15.39:53178) 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f34a4e00640 1 -- 172.21.15.39:0/2667459354 learned_addr learned my addr 172.21.15.39:0/2667459354 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f349fe00640 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6fab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f34a4e00640 1 -- 172.21.15.39:0/2667459354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 msgr2=0x55d0c6fab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f34a4e00640 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6fab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f34a4e00640 1 -- 172.21.15.39:0/2667459354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 msgr2=0x55d0c7020a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f34a4e00640 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 0x55d0c7020a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.797+0000 7f34a4e00640 1 -- 172.21.15.39:0/2667459354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c6fac160 con 0x55d0c6bce1b0 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.798+0000 7f34a4e00640 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fab380 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3490005fd0 tx=0x7f34900b2f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15323ecf56aaade0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:16.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.798+0000 7f348ea00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:25:16.790 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.798+0000 7f348ea00640 1 -- 172.21.15.39:0/2667459354 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34900c8020 con 0x55d0c6bce1b0 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.798+0000 7f34add34740 1 -- 172.21.15.39:0/2667459354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34900ae3b0 con 0x55d0c6bce1b0 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.798+0000 7f348ea00640 1 -- 172.21.15.39:0/2667459354 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f34900b29d0 con 0x55d0c6bce1b0 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.798+0000 7f348ea00640 1 -- 172.21.15.39:0/2667459354 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f34900c3070 con 0x55d0c6bce1b0 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.798+0000 7f34add34740 10 client.27739 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.799+0000 7f34add34740 1 -- 172.21.15.39:0/2667459354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0c7021fd0 con 0x55d0c6bce1b0 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.799+0000 7f348ea00640 1 -- 172.21.15.39:0/2667459354 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34900ae3b0 con 0x55d0c6bce1b0 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.799+0000 7f348ea00640 1 -- 172.21.15.39:0/2667459354 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3490005310 con 0x55d0c6bce1b0 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.799+0000 7f34add34740 10 client.27739 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.799+0000 7f34add34740 10 client.27739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.799+0000 7f34add34740 20 client.27739 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:16.791 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.799+0000 7f34add34740 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d0c6b8ada0 0x55d0c7024720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.799+0000 7f34add34740 4 client.27739 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:25:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.799+0000 7f34add34740 1 -- 172.21.15.39:0/2667459354 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d0c6f0f890 con 0x55d0c6b8ada0 2024-07-04T08:25:16.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.800+0000 7f349fe00640 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d0c6b8ada0 0x55d0c7024720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.800+0000 7f349fe00640 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d0c6b8ada0 0x55d0c7024720 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad82ad0a06719ab server_cookie=fe79e80b70feccfd in_seq=0 out_seq=0 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.800+0000 7f348ea00640 10 client.27739 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f348ea00640 1 -- 172.21.15.39:0/2667459354 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d0c6f0f890 con 0x55d0c6b8ada0 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f348ea00640 10 client.27739 handle_command_reply: tid=0 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 client.27739 shutdown 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 -- 172.21.15.39:0/2667459354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 msgr2=0x55d0c6fab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fab380 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3490005fd0 tx=0x7f34900b2f10 comp rx=0 tx=0).stop 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 -- 172.21.15.39:0/2667459354 shutdown_connections 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d0c6b8ada0 0x55d0c7024720 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d0c6bf7070 0x55d0c7020a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d0c6bd6390 0x55d0c6fab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 --2- 172.21.15.39:0/2667459354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c6bce1b0 0x55d0c6fab380 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 -- 172.21.15.39:0/2667459354 >> 172.21.15.39:0/2667459354 conn(0x55d0c6beb460 msgr2=0x55d0c6f9c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f348ea00640 0 client.27739 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.801+0000 7f34add34740 1 -- 172.21.15.39:0/2667459354 shutdown_connections 2024-07-04T08:25:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.802+0000 7f34add34740 1 -- 172.21.15.39:0/2667459354 wait complete. 2024-07-04T08:25:16.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.802+0000 7f34add34740 20 client.27739 trim_cache size 0 max 16384 2024-07-04T08:25:16.794 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:25:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-04T08:25:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-07-04T08:25:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "212b7312-ba59-47b6-9356-e0ad07696229": { 2024-07-04T08:25:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-07-04T08:25:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "212b7312-ba59-47b6-9356-e0ad07696229", 2024-07-04T08:25:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-07-04T08:25:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:16.795 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:16.795 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 -- 172.21.15.39:0/3766354942 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34840b28f0 msgr2=0x7f34840b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34840b28f0 0x7f34840b4e00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3488005eb0 tx=0x7f348802c040 comp rx=0 tx=0).stop 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 -- 172.21.15.39:0/3766354942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a0101fb0 msgr2=0x7f34a00febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a0101fb0 0x7f34a00febc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3490009fd0 tx=0x7f349000ef90 comp rx=0 tx=0).stop 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 -- 172.21.15.39:0/3766354942 shutdown_connections 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f34840b28f0 0x7f34840b4e00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34a01078f0 0x7f34a00ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34a0108240 0x7f34a00ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 --2- 172.21.15.39:0/3766354942 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f34a0101fb0 0x7f34a00febc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:16.796 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 -- 172.21.15.39:0/3766354942 >> 172.21.15.39:0/3766354942 conn(0x7f34a0078c80 msgr2=0x7f34a006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:16.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 -- 172.21.15.39:0/3766354942 shutdown_connections 2024-07-04T08:25:16.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:16.803+0000 7f34a7800640 1 -- 172.21.15.39:0/3766354942 wait complete. 2024-07-04T08:25:17.359 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:17 smithi133 ceph-mon[37050]: pgmap v790: 129 pgs: 129 active+clean; 14 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-07-04T08:25:17.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:17 smithi050 ceph-mon[36577]: pgmap v790: 129 pgs: 129 active+clean; 14 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-07-04T08:25:17.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:17 smithi039 ceph-mon[32062]: pgmap v790: 129 pgs: 129 active+clean; 14 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-07-04T08:25:17.720 INFO:tasks.cephfs.filesystem:scrub status for tag:212b7312-ba59-47b6-9356-e0ad07696229 - {'path': '/', 'tag': '212b7312-ba59-47b6-9356-e0ad07696229', 'options': 'recursive,force'} 2024-07-04T08:25:18.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:18 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:25:18.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:18 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:25:18.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:18 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:25:19.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:19 smithi050 ceph-mon[36577]: pgmap v791: 129 pgs: 129 active+clean; 14 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:25:19.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:19 smithi133 ceph-mon[37050]: pgmap v791: 129 pgs: 129 active+clean; 14 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:25:19.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:19 smithi039 ceph-mon[32062]: pgmap v791: 129 pgs: 129 active+clean; 14 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:25:21.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:21 smithi050 ceph-mon[36577]: pgmap v792: 129 pgs: 129 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:25:21.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:21 smithi133 ceph-mon[37050]: pgmap v792: 129 pgs: 129 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:25:21.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:21 smithi039 ceph-mon[32062]: pgmap v792: 129 pgs: 129 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:25:23.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:23 smithi050 ceph-mon[36577]: pgmap v793: 129 pgs: 129 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:25:23.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:23 smithi133 ceph-mon[37050]: pgmap v793: 129 pgs: 129 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:25:23.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:23 smithi039 ceph-mon[32062]: pgmap v793: 129 pgs: 129 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 15 op/s 2024-07-04T08:25:25.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:25 smithi050 ceph-mon[36577]: pgmap v794: 129 pgs: 129 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-07-04T08:25:25.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:25 smithi133 ceph-mon[37050]: pgmap v794: 129 pgs: 129 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-07-04T08:25:25.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:25 smithi039 ceph-mon[32062]: pgmap v794: 129 pgs: 129 active+clean; 15 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-07-04T08:25:27.385 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:27 smithi050 ceph-mon[36577]: pgmap v795: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 67 KiB/s wr, 16 op/s 2024-07-04T08:25:27.553 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:27 smithi039 ceph-mon[32062]: pgmap v795: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 67 KiB/s wr, 16 op/s 2024-07-04T08:25:27.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:27 smithi133 ceph-mon[37050]: pgmap v795: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 67 KiB/s wr, 16 op/s 2024-07-04T08:25:29.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:29 smithi133 ceph-mon[37050]: pgmap v796: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-07-04T08:25:29.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:29 smithi039 ceph-mon[32062]: pgmap v796: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-07-04T08:25:29.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:29 smithi050 ceph-mon[36577]: pgmap v796: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-07-04T08:25:31.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:31 smithi133 ceph-mon[37050]: pgmap v797: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 71 KiB/s wr, 16 op/s 2024-07-04T08:25:31.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:31 smithi039 ceph-mon[32062]: pgmap v797: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 71 KiB/s wr, 16 op/s 2024-07-04T08:25:31.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:31 smithi050 ceph-mon[36577]: pgmap v797: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 71 KiB/s wr, 16 op/s 2024-07-04T08:25:33.557 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:33 smithi133 ceph-mon[37050]: pgmap v798: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-07-04T08:25:33.592 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:33 smithi039 ceph-mon[32062]: pgmap v798: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-07-04T08:25:33.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:33 smithi050 ceph-mon[36577]: pgmap v798: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 49 KiB/s wr, 16 op/s 2024-07-04T08:25:35.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:35 smithi133 ceph-mon[37050]: pgmap v799: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-07-04T08:25:35.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:35 smithi039 ceph-mon[32062]: pgmap v799: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-07-04T08:25:35.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:35 smithi050 ceph-mon[36577]: pgmap v799: 129 pgs: 129 active+clean; 15 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 49 KiB/s wr, 13 op/s 2024-07-04T08:25:37.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:37 smithi133 ceph-mon[37050]: pgmap v800: 129 pgs: 129 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 17 op/s 2024-07-04T08:25:37.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:37 smithi039 ceph-mon[32062]: pgmap v800: 129 pgs: 129 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 17 op/s 2024-07-04T08:25:37.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:37 smithi050 ceph-mon[36577]: pgmap v800: 129 pgs: 129 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 72 KiB/s wr, 17 op/s 2024-07-04T08:25:39.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:39 smithi133 ceph-mon[37050]: pgmap v801: 129 pgs: 129 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:25:39.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:39 smithi039 ceph-mon[32062]: pgmap v801: 129 pgs: 129 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:25:39.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:39 smithi050 ceph-mon[36577]: pgmap v801: 129 pgs: 129 active+clean; 15 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:25:40.236 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:313/600: open fd = 3 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:314/600: open fd = 3 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:315/600: open fd = 3 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.237 INFO:tasks.workunit.client.0.smithi039.stdout:316/600: open fd = 3 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:317/600: open fd = 3 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:318/600: open fd = 3 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.238 INFO:tasks.workunit.client.0.smithi039.stdout:319/600: open fd = 3 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:320/600: open fd = 3 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:321/600: open fd = 3 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:322/600: open fd = 3 2024-07-04T08:25:40.239 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:323/600: open fd = 3 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:324/600: open fd = 3 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:325/600: open fd = 3 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.240 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:326/600: open fd = 3 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:327/600: open fd = 3 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:328/600: open fd = 3 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.241 INFO:tasks.workunit.client.0.smithi039.stdout:329/600: open fd = 3 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:330/600: open fd = 3 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:331/600: open fd = 3 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:332/600: open fd = 3 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.242 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:333/600: open fd = 3 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:334/600: open fd = 3 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:335/600: open fd = 3 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.243 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:336/600: open fd = 3 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:337/600: open fd = 3 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:338/600: open fd = 3 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.244 INFO:tasks.workunit.client.0.smithi039.stdout:339/600: open fd = 3 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:340/600: open fd = 3 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:341/600: open fd = 3 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:342/600: open fd = 3 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.245 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:343/600: open fd = 3 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:344/600: open fd = 3 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:345/600: open fd = 3 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.246 INFO:tasks.workunit.client.0.smithi039.stdout:346/600: open fd = 3 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:347/600: open fd = 3 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:348/600: open fd = 3 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:349/600: open fd = 3 2024-07-04T08:25:40.247 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:350/600: open fd = 3 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:351/600: open fd = 3 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:352/600: open fd = 3 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.248 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:353/600: open fd = 3 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:354/600: open fd = 3 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:355/600: open fd = 3 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.249 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:356/600: open fd = 3 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:357/600: open fd = 3 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:358/600: open fd = 3 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:359/600: open fd = 3 2024-07-04T08:25:40.250 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:360/600: open fd = 3 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:361/600: open fd = 3 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:362/600: open fd = 3 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.251 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:363/600: open fd = 3 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:364/600: open fd = 3 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:365/600: open fd = 3 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.252 INFO:tasks.workunit.client.0.smithi039.stdout:366/600: open fd = 3 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:367/600: open fd = 3 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:368/600: open fd = 3 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:369/600: open fd = 3 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.253 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:370/600: open fd = 3 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:371/600: open fd = 3 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:372/600: open fd = 3 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.254 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.255 INFO:tasks.workunit.client.0.smithi039.stdout:373/600: open fd = 3 2024-07-04T08:25:40.255 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.255 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.255 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:40.255 INFO:tasks.workunit.client.0.smithi039.stdout:374/600: open fd = 3 2024-07-04T08:25:40.255 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.255 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:25:40.255 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:25:41.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:41 smithi133 ceph-mon[37050]: pgmap v802: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 17 op/s 2024-07-04T08:25:41.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:41 smithi039 ceph-mon[32062]: pgmap v802: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 17 op/s 2024-07-04T08:25:41.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:41 smithi050 ceph-mon[36577]: pgmap v802: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 17 op/s 2024-07-04T08:25:43.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:43 smithi133 ceph-mon[37050]: pgmap v803: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-07-04T08:25:43.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:43 smithi039 ceph-mon[32062]: pgmap v803: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-07-04T08:25:43.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:43 smithi050 ceph-mon[36577]: pgmap v803: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 16 op/s 2024-07-04T08:25:44.038 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-04T08:25:44.038 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-07-04T08:25:44.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:25:44.541 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:25:45.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:45 smithi039 ceph-mon[32062]: pgmap v804: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-07-04T08:25:45.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:45 smithi133 ceph-mon[37050]: pgmap v804: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-07-04T08:25:45.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:45 smithi050 ceph-mon[36577]: pgmap v804: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 13 op/s 2024-07-04T08:25:46.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.855+0000 7fc106200640 1 -- 172.21.15.39:0/1267237996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10010ac20 msgr2=0x7fc10010b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:46.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.855+0000 7fc106200640 1 --2- 172.21.15.39:0/1267237996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10010ac20 0x7fc10010b030 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009900 tx=0x7fc0ec02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:25:46.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.855+0000 7fc106200640 1 -- 172.21.15.39:0/1267237996 shutdown_connections 2024-07-04T08:25:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.855+0000 7fc106200640 1 --2- 172.21.15.39:0/1267237996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc10010b570 0x7fc100111da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.855+0000 7fc106200640 1 --2- 172.21.15.39:0/1267237996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10010ac20 0x7fc10010b030 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.855+0000 7fc106200640 1 --2- 172.21.15.39:0/1267237996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1001052e0 0x7fc1001056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.855+0000 7fc106200640 1 -- 172.21.15.39:0/1267237996 >> 172.21.15.39:0/1267237996 conn(0x7fc100100db0 msgr2=0x7fc100103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.855+0000 7fc106200640 1 -- 172.21.15.39:0/1267237996 shutdown_connections 2024-07-04T08:25:46.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.856+0000 7fc106200640 1 -- 172.21.15.39:0/1267237996 wait complete. 2024-07-04T08:25:46.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.857+0000 7fc106200640 1 Processor -- start 2024-07-04T08:25:46.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.857+0000 7fc106200640 1 -- start start 2024-07-04T08:25:46.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc106200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1001052e0 0x7fc10019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:46.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc106200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc10010ac20 0x7fc10019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:46.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc106200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10010b570 0x7fc1001a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:46.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc106200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc1001151e0 con 0x7fc1001052e0 2024-07-04T08:25:46.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc106200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc100115060 con 0x7fc10010b570 2024-07-04T08:25:46.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc106200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc100115360 con 0x7fc10010ac20 2024-07-04T08:25:46.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc0fea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc10010ac20 0x7fc10019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:46.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc0fea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc10010ac20 0x7fc10019c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:60088/0 (socket says 172.21.15.39:60088) 2024-07-04T08:25:46.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc0fea00640 1 -- 172.21.15.39:0/3242191163 learned_addr learned my addr 172.21.15.39:0/3242191163 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:46.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.858+0000 7fc0ff400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1001052e0 0x7fc10019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:46.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.859+0000 7fc0ffe00640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10010b570 0x7fc1001a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:46.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.859+0000 7fc0fea00640 1 -- 172.21.15.39:0/3242191163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10010b570 msgr2=0x7fc1001a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.859+0000 7fc0fea00640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10010b570 0x7fc1001a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.859+0000 7fc0fea00640 1 -- 172.21.15.39:0/3242191163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1001052e0 msgr2=0x7fc10019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.859+0000 7fc0fea00640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1001052e0 0x7fc10019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.859+0000 7fc0fea00640 1 -- 172.21.15.39:0/3242191163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1001a1c00 con 0x7fc10010ac20 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.859+0000 7fc0fea00640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc10010ac20 0x7fc10019c230 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec02a910 tx=0x7fc0ec031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.860+0000 7fc0f7e00640 1 -- 172.21.15.39:0/3242191163 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec03f030 con 0x7fc10010ac20 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.860+0000 7fc0f7e00640 1 -- 172.21.15.39:0/3242191163 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0ec004490 con 0x7fc10010ac20 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.860+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1001a1e90 con 0x7fc10010ac20 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.860+0000 7fc0f7e00640 1 -- 172.21.15.39:0/3242191163 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec035070 con 0x7fc10010ac20 2024-07-04T08:25:46.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.860+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1001a8650 con 0x7fc10010ac20 2024-07-04T08:25:46.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.861+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1001056f0 con 0x7fc10010ac20 2024-07-04T08:25:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.865+0000 7fc0f7e00640 1 -- 172.21.15.39:0/3242191163 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc0ec042030 con 0x7fc10010ac20 2024-07-04T08:25:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.866+0000 7fc0f7e00640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e40b2960 0x7fc0e40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.866+0000 7fc0f7e00640 1 -- 172.21.15.39:0/3242191163 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7fc0ec103180 con 0x7fc10010ac20 2024-07-04T08:25:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.867+0000 7fc0ff400640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e40b2960 0x7fc0e40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.867+0000 7fc0f7e00640 1 -- 172.21.15.39:0/3242191163 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fc0ec0ca150 con 0x7fc10010ac20 2024-07-04T08:25:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:46.867+0000 7fc0ff400640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e40b2960 0x7fc0e40b4e70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0005fd0 tx=0x7fc0f0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:47.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.114+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc10019cdc0 con 0x7fc10010ac20 2024-07-04T08:25:47.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.115+0000 7fc0f7e00640 1 -- 172.21.15.39:0/3242191163 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fc10019cdc0 con 0x7fc10010ac20 2024-07-04T08:25:47.107 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:25:47.107 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:25:47.107 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:25:47.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e40b2960 msgr2=0x7fc0e40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:47.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e40b2960 0x7fc0e40b4e70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc0f0005fd0 tx=0x7fc0f0009500 comp rx=0 tx=0).stop 2024-07-04T08:25:47.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc10010ac20 msgr2=0x7fc10019c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc10010ac20 0x7fc10019c230 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec02a910 tx=0x7fc0ec031a20 comp rx=0 tx=0).stop 2024-07-04T08:25:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 shutdown_connections 2024-07-04T08:25:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc0e40b2960 0x7fc0e40b4e70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc10010b570 0x7fc1001a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc10010ac20 0x7fc10019c230 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 --2- 172.21.15.39:0/3242191163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1001052e0 0x7fc10019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 >> 172.21.15.39:0/3242191163 conn(0x7fc100100db0 msgr2=0x7fc1001031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 shutdown_connections 2024-07-04T08:25:47.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:47.119+0000 7fc106200640 1 -- 172.21.15.39:0/3242191163 wait complete. 2024-07-04T08:25:47.511 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:47 smithi039 ceph-mon[32062]: pgmap v805: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 16 op/s 2024-07-04T08:25:47.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:47 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3242191163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:25:47.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:47 smithi133 ceph-mon[37050]: pgmap v805: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 16 op/s 2024-07-04T08:25:47.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:47 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3242191163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:25:47.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:47 smithi050 ceph-mon[36577]: pgmap v805: 129 pgs: 129 active+clean; 16 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 71 KiB/s wr, 16 op/s 2024-07-04T08:25:47.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:47 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3242191163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:25:47.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:25:47.919 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-07-04T08:25:47.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-07-04T08:25:48.231 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:25:49.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:49 smithi133 ceph-mon[37050]: pgmap v806: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-07-04T08:25:49.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:49 smithi039 ceph-mon[32062]: pgmap v806: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-07-04T08:25:49.616 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:25:49.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:49 smithi050 ceph-mon[36577]: pgmap v806: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 15 op/s 2024-07-04T08:25:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.077+0000 7f5324e00640 1 -- 172.21.15.39:0/2914840441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53200777e0 msgr2=0x7f5320113620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:51.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.077+0000 7f5324e00640 1 --2- 172.21.15.39:0/2914840441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53200777e0 0x7f5320113620 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5308009900 tx=0x7f530802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.077+0000 7f5324e00640 1 -- 172.21.15.39:0/2914840441 shutdown_connections 2024-07-04T08:25:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.077+0000 7f5324e00640 1 --2- 172.21.15.39:0/2914840441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5320113b60 0x7f5320115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.077+0000 7f5324e00640 1 --2- 172.21.15.39:0/2914840441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53200777e0 0x7f5320113620 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.077+0000 7f5324e00640 1 --2- 172.21.15.39:0/2914840441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5320076e90 0x7f53200772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.077+0000 7f5324e00640 1 -- 172.21.15.39:0/2914840441 >> 172.21.15.39:0/2914840441 conn(0x7f5320100c30 msgr2=0x7f5320103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.077+0000 7f5324e00640 1 -- 172.21.15.39:0/2914840441 shutdown_connections 2024-07-04T08:25:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.077+0000 7f5324e00640 1 -- 172.21.15.39:0/2914840441 wait complete. 2024-07-04T08:25:51.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.079+0000 7f5324e00640 1 Processor -- start 2024-07-04T08:25:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.079+0000 7f5324e00640 1 -- start start 2024-07-04T08:25:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f5324e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5320076e90 0x7f532019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f5324e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53200777e0 0x7f53201a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f5324e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5320113b60 0x7f53201a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f5324e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5320119420 con 0x7f5320076e90 2024-07-04T08:25:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f5324e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53201192a0 con 0x7f53200777e0 2024-07-04T08:25:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f5324e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f53201195a0 con 0x7f5320113b60 2024-07-04T08:25:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5320076e90 0x7f532019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5320113b60 0x7f53201a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5320076e90 0x7f532019fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41428/0 (socket says 172.21.15.39:41428) 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531f400640 1 -- 172.21.15.39:0/2699943570 learned_addr learned my addr 172.21.15.39:0/2699943570 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f5316a00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53200777e0 0x7f53201a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531fe00640 1 -- 172.21.15.39:0/2699943570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53200777e0 msgr2=0x7f53201a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531fe00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53200777e0 0x7f53201a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531fe00640 1 -- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5320076e90 msgr2=0x7f532019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531fe00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5320076e90 0x7f532019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531fe00640 1 -- 172.21.15.39:0/2699943570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53201a5dc0 con 0x7f5320113b60 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.080+0000 7f531f400640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5320076e90 0x7f532019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.081+0000 7f531fe00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5320113b60 0x7f53201a5690 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5318005fd0 tx=0x7f531800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.081+0000 7f531cc00640 1 -- 172.21.15.39:0/2699943570 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318003b70 con 0x7f5320113b60 2024-07-04T08:25:51.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.081+0000 7f5324e00640 1 -- 172.21.15.39:0/2699943570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53201a60b0 con 0x7f5320113b60 2024-07-04T08:25:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.081+0000 7f531cc00640 1 -- 172.21.15.39:0/2699943570 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5318003d10 con 0x7f5320113b60 2024-07-04T08:25:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.081+0000 7f531cc00640 1 -- 172.21.15.39:0/2699943570 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180107a0 con 0x7f5320113b60 2024-07-04T08:25:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.081+0000 7f5324e00640 1 -- 172.21.15.39:0/2699943570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53201ac810 con 0x7f5320113b60 2024-07-04T08:25:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.084+0000 7f532c992740 1 Processor -- start 2024-07-04T08:25:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.085+0000 7f531cc00640 1 -- 172.21.15.39:0/2699943570 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5318003ec0 con 0x7f5320113b60 2024-07-04T08:25:51.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.085+0000 7f532c992740 1 -- start start 2024-07-04T08:25:51.388 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:51 smithi039 ceph-mon[32062]: pgmap v807: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-07-04T08:25:51.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f532c992740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439c8070 0x561643d6c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f532c992740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 0x561643d6ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f532c992740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 0x561643d6d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f532c992740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561643ce2ed0 con 0x5616439c8070 2024-07-04T08:25:51.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f532c992740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561643cd2470 con 0x5616439a7390 2024-07-04T08:25:51.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f532c992740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561643cd64b0 con 0x56164399f1b0 2024-07-04T08:25:51.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f5316a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439c8070 0x561643d6c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f5316a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439c8070 0x561643d6c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41444/0 (socket says 172.21.15.39:41444) 2024-07-04T08:25:51.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f5316a00640 1 -- 172.21.15.39:0/2412862213 learned_addr learned my addr 172.21.15.39:0/2412862213 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:51.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.086+0000 7f531f400640 1 --2- 172.21.15.39:0/2412862213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 0x561643d6ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.087+0000 7f5316a00640 1 --2- 172.21.15.39:0/2412862213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 0x561643d6d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.087+0000 7f5316a00640 1 -- 172.21.15.39:0/2412862213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 msgr2=0x561643d6ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:51.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.087+0000 7f5316a00640 1 --2- 172.21.15.39:0/2412862213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 0x561643d6ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.087+0000 7f5316a00640 1 -- 172.21.15.39:0/2412862213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 msgr2=0x561643d6d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:51.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.087+0000 7f5316a00640 1 --2- 172.21.15.39:0/2412862213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 0x561643d6d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.087+0000 7f5316a00640 1 -- 172.21.15.39:0/2412862213 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561643c6a640 con 0x5616439c8070 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.087+0000 7f5316a00640 1 --2- 172.21.15.39:0/2412862213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439c8070 0x561643d6c940 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5308009900 tx=0x7f5308005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a703b6a33134fb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.088+0000 7f5314c00640 1 -- 172.21.15.39:0/2412862213 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308044020 con 0x5616439c8070 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.088+0000 7f5314c00640 1 -- 172.21.15.39:0/2412862213 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53080054f0 con 0x5616439c8070 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.088+0000 7f5314c00640 1 -- 172.21.15.39:0/2412862213 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308035040 con 0x5616439c8070 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.088+0000 7f531cc00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f53040b2960 0x7f53040b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.088+0000 7f531cc00640 1 -- 172.21.15.39:0/2699943570 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f53180e2f10 con 0x7f5320113b60 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.088+0000 7f531f400640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f53040b2960 0x7f53040b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f532c992740 1 -- 172.21.15.39:0/2412862213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439c8070 msgr2=0x561643d6c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f532c992740 1 --2- 172.21.15.39:0/2412862213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439c8070 0x561643d6c940 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5308009900 tx=0x7f5308005fd0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f532c992740 1 -- 172.21.15.39:0/2412862213 shutdown_connections 2024-07-04T08:25:51.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f532c992740 1 --2- 172.21.15.39:0/2412862213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 0x561643d6d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f532c992740 1 --2- 172.21.15.39:0/2412862213 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 0x561643d6ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f532c992740 1 --2- 172.21.15.39:0/2412862213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439c8070 0x561643d6c940 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f532c992740 1 -- 172.21.15.39:0/2412862213 >> 172.21.15.39:0/2412862213 conn(0x5616439bc460 msgr2=0x561643d6a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f532c992740 1 -- 172.21.15.39:0/2412862213 shutdown_connections 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f531f400640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f53040b2960 0x7f53040b4e70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x561643d6b5c0 tx=0x7f5310005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.089+0000 7f532c992740 1 -- 172.21.15.39:0/2412862213 wait complete. 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.090+0000 7f532c992740 1 -- start start 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.090+0000 7f532c992740 1 Processor -- start 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f532c992740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56164399f1b0 0x561643d785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f532c992740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d78b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f532c992740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 0x561643d843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f532c992740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561643b36700 con 0x5616439a7390 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f532c992740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561643c79ba0 con 0x56164399f1b0 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f532c992740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561643d7b0c0 con 0x5616439c8070 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f532c992740 10 client.0 _ll_register_callbacks cb 0x561643ccb560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f531f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56164399f1b0 0x561643d785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f5316a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d78b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f531f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56164399f1b0 0x561643d785b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43110/0 (socket says 172.21.15.39:43110) 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f531f400640 1 -- 172.21.15.39:0/1286922376 learned_addr learned my addr 172.21.15.39:0/1286922376 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.091+0000 7f531fe00640 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 0x561643d843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f531f400640 1 -- 172.21.15.39:0/1286922376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 msgr2=0x561643d843b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f531f400640 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 0x561643d843b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f531f400640 1 -- 172.21.15.39:0/1286922376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 msgr2=0x561643d78b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:51.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f531f400640 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d78b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f531f400640 1 -- 172.21.15.39:0/1286922376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561643a33650 con 0x56164399f1b0 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f531f400640 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56164399f1b0 0x561643d785b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f531000f8c0 tx=0x7f5310004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e2a4de2fe30042 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f530d600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f530d600640 1 -- 172.21.15.39:0/1286922376 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531000fe40 con 0x56164399f1b0 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f532c992740 1 -- 172.21.15.39:0/1286922376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53080054f0 con 0x56164399f1b0 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.092+0000 7f530d600640 1 -- 172.21.15.39:0/1286922376 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f5310011bd0 con 0x56164399f1b0 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f530d600640 1 -- 172.21.15.39:0/1286922376 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5310011dd0 con 0x56164399f1b0 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f532c992740 10 client.27824 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f532c992740 1 -- 172.21.15.39:0/1286922376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561643d87230 con 0x56164399f1b0 2024-07-04T08:25:51.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f530d600640 1 -- 172.21.15.39:0/1286922376 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53080054f0 con 0x56164399f1b0 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f530d600640 1 -- 172.21.15.39:0/1286922376 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5310027070 con 0x56164399f1b0 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f532c992740 10 client.27824 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f532c992740 10 client.27824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f532c992740 20 client.27824 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f532c992740 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5616439869f0 0x561643d8a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f532c992740 4 client.27824 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.093+0000 7f532c992740 1 -- 172.21.15.39:0/1286922376 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561643ce4e20 con 0x5616439869f0 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.094+0000 7f5316a00640 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5616439869f0 0x561643d8a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.094+0000 7f5316a00640 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5616439869f0 0x561643d8a730 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14983fd8dcaed681 server_cookie=a1b0ba5f2c772f81 in_seq=0 out_seq=0 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.094+0000 7f530d600640 10 client.27824 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.095+0000 7f530d600640 1 -- 172.21.15.39:0/1286922376 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561643ce4e20 con 0x5616439869f0 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.095+0000 7f530d600640 10 client.27824 handle_command_reply: tid=0 2024-07-04T08:25:51.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.095+0000 7f532c992740 1 client.27824 shutdown 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 -- 172.21.15.39:0/1286922376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56164399f1b0 msgr2=0x561643d785b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56164399f1b0 0x561643d785b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f531000f8c0 tx=0x7f5310004730 comp rx=0 tx=0).stop 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 -- 172.21.15.39:0/1286922376 shutdown_connections 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5616439869f0 0x561643d8a730 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d78b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 0x561643d843b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 --2- 172.21.15.39:0/1286922376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56164399f1b0 0x561643d785b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f530d600640 0 client.27824 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 -- 172.21.15.39:0/1286922376 >> 172.21.15.39:0/1286922376 conn(0x5616439bc460 msgr2=0x561643d6fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 -- 172.21.15.39:0/1286922376 shutdown_connections 2024-07-04T08:25:51.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 1 -- 172.21.15.39:0/1286922376 wait complete. 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.096+0000 7f532c992740 20 client.27824 trim_cache size 0 max 16384 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.116+0000 7f532c992740 1 Processor -- start 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.117+0000 7f532c992740 1 -- start start 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.117+0000 7f532c992740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56164399f1b0 0x561643d72160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.117+0000 7f532c992740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 0x561643d8ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.117+0000 7f532c992740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 0x561643d90470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.117+0000 7f532c992740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f53201b6640 con 0x56164399f1b0 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.117+0000 7f532c992740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53201b8180 con 0x5616439a7390 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.117+0000 7f532c992740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f53201b6d10 con 0x5616439c8070 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.117+0000 7f531f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56164399f1b0 0x561643d72160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f531f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56164399f1b0 0x561643d72160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41460/0 (socket says 172.21.15.39:41460) 2024-07-04T08:25:51.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f5316a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 0x561643d8ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f531f400640 1 -- 172.21.15.39:0/479743264 learned_addr learned my addr 172.21.15.39:0/479743264 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f531fe00640 1 --2- 172.21.15.39:0/479743264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 0x561643d90470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f531f400640 1 -- 172.21.15.39:0/479743264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 msgr2=0x561643d90470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f531f400640 1 --2- 172.21.15.39:0/479743264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 0x561643d90470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f531f400640 1 -- 172.21.15.39:0/479743264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 msgr2=0x561643d8ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f531f400640 1 --2- 172.21.15.39:0/479743264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 0x561643d8ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f531f400640 1 -- 172.21.15.39:0/479743264 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561643d77310 con 0x56164399f1b0 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f5316a00640 1 --2- 172.21.15.39:0/479743264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 0x561643d8ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.118+0000 7f531f400640 1 --2- 172.21.15.39:0/479743264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56164399f1b0 0x561643d72160 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5310003eb0 tx=0x7f53100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2b7723768a4a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.119+0000 7f530fe00640 1 -- 172.21.15.39:0/479743264 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310004000 con 0x56164399f1b0 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.119+0000 7f530fe00640 1 -- 172.21.15.39:0/479743264 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5310005020 con 0x56164399f1b0 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.119+0000 7f530fe00640 1 -- 172.21.15.39:0/479743264 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310004ba0 con 0x56164399f1b0 2024-07-04T08:25:51.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 -- 172.21.15.39:0/479743264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56164399f1b0 msgr2=0x561643d72160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 --2- 172.21.15.39:0/479743264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56164399f1b0 0x561643d72160 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5310003eb0 tx=0x7f53100046c0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 -- 172.21.15.39:0/479743264 shutdown_connections 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 --2- 172.21.15.39:0/479743264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616439c8070 0x561643d90470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 --2- 172.21.15.39:0/479743264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439a7390 0x561643d8ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 --2- 172.21.15.39:0/479743264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56164399f1b0 0x561643d72160 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 -- 172.21.15.39:0/479743264 >> 172.21.15.39:0/479743264 conn(0x5616439bc460 msgr2=0x561643d71a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 -- 172.21.15.39:0/479743264 shutdown_connections 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 -- 172.21.15.39:0/479743264 wait complete. 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.120+0000 7f532c992740 1 -- start start 2024-07-04T08:25:51.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.121+0000 7f532c992740 1 Processor -- start 2024-07-04T08:25:51.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.121+0000 7f532c992740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 0x561643d7c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.121+0000 7f532c992740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d7c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.121+0000 7f532c992740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439c8070 0x561643d7cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.121+0000 7f532c992740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f53201b7ab0 con 0x5616439a7390 2024-07-04T08:25:51.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.121+0000 7f532c992740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f53201b73e0 con 0x5616439c8070 2024-07-04T08:25:51.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.121+0000 7f532c992740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f53201b4620 con 0x56164399f1b0 2024-07-04T08:25:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.121+0000 7f532c992740 10 client.0 _ll_register_callbacks cb 0x7f531000f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.121+0000 7f5316a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d7c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f5316a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d7c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41464/0 (socket says 172.21.15.39:41464) 2024-07-04T08:25:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f5316a00640 1 -- 172.21.15.39:0/68687175 learned_addr learned my addr 172.21.15.39:0/68687175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f531f400640 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 0x561643d7c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f531fe00640 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439c8070 0x561643d7cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f5316a00640 1 -- 172.21.15.39:0/68687175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 msgr2=0x561643d7c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f5316a00640 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 0x561643d7c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f5316a00640 1 -- 172.21.15.39:0/68687175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439c8070 msgr2=0x561643d7cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f5316a00640 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439c8070 0x561643d7cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f5316a00640 1 -- 172.21.15.39:0/68687175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561643d6f1d0 con 0x5616439a7390 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f5316a00640 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d7c6e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f53080049d0 tx=0x7f5308004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12038ff607b2d33d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.122+0000 7f530d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.123+0000 7f530d600640 1 -- 172.21.15.39:0/68687175 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308035050 con 0x5616439a7390 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.123+0000 7f530d600640 1 -- 172.21.15.39:0/68687175 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f5308031b40 con 0x5616439a7390 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.123+0000 7f532c992740 1 -- 172.21.15.39:0/68687175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5310005020 con 0x5616439a7390 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.123+0000 7f530d600640 1 -- 172.21.15.39:0/68687175 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5308044040 con 0x5616439a7390 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.123+0000 7f532c992740 10 client.17694 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.123+0000 7f532c992740 1 -- 172.21.15.39:0/68687175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561643df2f90 con 0x5616439a7390 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.123+0000 7f530d600640 1 -- 172.21.15.39:0/68687175 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5310005020 con 0x5616439a7390 2024-07-04T08:25:51.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.124+0000 7f530d600640 1 -- 172.21.15.39:0/68687175 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5308004b10 con 0x5616439a7390 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.124+0000 7f532c992740 10 client.17694 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.124+0000 7f532c992740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.124+0000 7f532c992740 20 client.17694 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.124+0000 7f532c992740 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56164395bda0 0x561643df52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.124+0000 7f532c992740 4 client.17694 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.124+0000 7f532c992740 1 -- 172.21.15.39:0/68687175 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561643ce08c0 con 0x56164395bda0 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.124+0000 7f531f400640 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56164395bda0 0x561643df52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.125+0000 7f531f400640 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56164395bda0 0x561643df52d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4fbb4185e06e37 server_cookie=62fa12b513ddb517 in_seq=0 out_seq=0 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.125+0000 7f530d600640 10 client.17694 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.125+0000 7f530d600640 1 -- 172.21.15.39:0/68687175 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561643ce08c0 con 0x56164395bda0 2024-07-04T08:25:51.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.125+0000 7f530d600640 10 client.17694 handle_command_reply: tid=0 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.125+0000 7f532c992740 1 client.17694 shutdown 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.125+0000 7f532c992740 1 -- 172.21.15.39:0/68687175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 msgr2=0x561643d7c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.125+0000 7f532c992740 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d7c6e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f53080049d0 tx=0x7f5308004aa0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.125+0000 7f532c992740 1 -- 172.21.15.39:0/68687175 shutdown_connections 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.125+0000 7f532c992740 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56164395bda0 0x561643df52d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.126+0000 7f532c992740 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5616439a7390 0x561643d7c6e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.126+0000 7f532c992740 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5616439c8070 0x561643d7cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.126+0000 7f530d600640 0 client.17694 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.126+0000 7f532c992740 1 --2- 172.21.15.39:0/68687175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56164399f1b0 0x561643d7c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.126+0000 7f532c992740 1 -- 172.21.15.39:0/68687175 >> 172.21.15.39:0/68687175 conn(0x5616439bc460 msgr2=0x561643d6d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.126+0000 7f532c992740 1 -- 172.21.15.39:0/68687175 shutdown_connections 2024-07-04T08:25:51.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.126+0000 7f532c992740 1 -- 172.21.15.39:0/68687175 wait complete. 2024-07-04T08:25:51.407 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:25:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:25:51.407 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:25:51.407 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:25:51.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.126+0000 7f532c992740 20 client.17694 trim_cache size 0 max 16384 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 -- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f53040b2960 msgr2=0x7f53040b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f53040b2960 0x7f53040b4e70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x561643d6b5c0 tx=0x7f5310005f90 comp rx=0 tx=0).stop 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 -- 172.21.15.39:0/2699943570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5320113b60 msgr2=0x7f53201a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5320113b60 0x7f53201a5690 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5318005fd0 tx=0x7f531800c6a0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 -- 172.21.15.39:0/2699943570 shutdown_connections 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f53040b2960 0x7f53040b4e70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5320113b60 0x7f53201a5690 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f53200777e0 0x7f53201a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 --2- 172.21.15.39:0/2699943570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5320076e90 0x7f532019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 -- 172.21.15.39:0/2699943570 >> 172.21.15.39:0/2699943570 conn(0x7f5320100c30 msgr2=0x7f5320103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:51.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 -- 172.21.15.39:0/2699943570 shutdown_connections 2024-07-04T08:25:51.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:51.127+0000 7f5324e00640 1 -- 172.21.15.39:0/2699943570 wait complete. 2024-07-04T08:25:51.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:51 smithi133 ceph-mon[37050]: pgmap v807: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-07-04T08:25:51.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:51 smithi050 ceph-mon[36577]: pgmap v807: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-07-04T08:25:52.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.385+0000 7fc9b0000640 1 -- 172.21.15.39:0/3489114545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8104840 msgr2=0x7fc9a810ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:52.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.385+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3489114545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8104840 0x7fc9a810ce80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc994009900 tx=0x7fc99402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.385+0000 7fc9b0000640 1 -- 172.21.15.39:0/3489114545 shutdown_connections 2024-07-04T08:25:52.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.385+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3489114545 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810d3c0 0x7fc9a810f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.385+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3489114545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8104840 0x7fc9a810ce80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.385+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3489114545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9a8103e10 0x7fc9a8104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.385+0000 7fc9b0000640 1 -- 172.21.15.39:0/3489114545 >> 172.21.15.39:0/3489114545 conn(0x7fc9a8078cb0 msgr2=0x7fc9a80790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:52.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.385+0000 7fc9b0000640 1 -- 172.21.15.39:0/3489114545 shutdown_connections 2024-07-04T08:25:52.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.385+0000 7fc9b0000640 1 -- 172.21.15.39:0/3489114545 wait complete. 2024-07-04T08:25:52.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.386+0000 7fc9b0000640 1 Processor -- start 2024-07-04T08:25:52.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.386+0000 7fc9b0000640 1 -- start start 2024-07-04T08:25:52.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9b0000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9a8103e10 0x7fc9a819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9b0000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8104840 0x7fc9a819c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9b0000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810d3c0 0x7fc9a81a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9b0000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8112f90 con 0x7fc9a810d3c0 2024-07-04T08:25:52.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9b0000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8112e10 con 0x7fc9a8104840 2024-07-04T08:25:52.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9b0000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a8113110 con 0x7fc9a8103e10 2024-07-04T08:25:52.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810d3c0 0x7fc9a81a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810d3c0 0x7fc9a81a1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41482/0 (socket says 172.21.15.39:41482) 2024-07-04T08:25:52.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9ae000640 1 -- 172.21.15.39:0/3041845804 learned_addr learned my addr 172.21.15.39:0/3041845804 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:52.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9ad600640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9a8103e10 0x7fc9a819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.379 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.387+0000 7fc9acc00640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8104840 0x7fc9a819c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.388+0000 7fc9ae000640 1 -- 172.21.15.39:0/3041845804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9a8103e10 msgr2=0x7fc9a819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.388+0000 7fc9ae000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9a8103e10 0x7fc9a819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.388+0000 7fc9ae000640 1 -- 172.21.15.39:0/3041845804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8104840 msgr2=0x7fc9a819c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.388+0000 7fc9ae000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8104840 0x7fc9a819c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.388+0000 7fc9ae000640 1 -- 172.21.15.39:0/3041845804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a81a1b50 con 0x7fc9a810d3c0 2024-07-04T08:25:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.388+0000 7fc9ae000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810d3c0 0x7fc9a81a1420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc99800e970 tx=0x7fc99800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.388+0000 7fc9a6000640 1 -- 172.21.15.39:0/3041845804 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99800ccc0 con 0x7fc9a810d3c0 2024-07-04T08:25:52.380 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.389+0000 7fc9b0000640 1 -- 172.21.15.39:0/3041845804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a81a1e40 con 0x7fc9a810d3c0 2024-07-04T08:25:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.389+0000 7fc9a6000640 1 -- 172.21.15.39:0/3041845804 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc99800ce60 con 0x7fc9a810d3c0 2024-07-04T08:25:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.389+0000 7fc9a6000640 1 -- 172.21.15.39:0/3041845804 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc998010430 con 0x7fc9a810d3c0 2024-07-04T08:25:52.381 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.389+0000 7fc9b0000640 1 -- 172.21.15.39:0/3041845804 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a81a85a0 con 0x7fc9a810d3c0 2024-07-04T08:25:52.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.392+0000 7fc9b6584740 1 Processor -- start 2024-07-04T08:25:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.393+0000 7fc9a6000640 1 -- 172.21.15.39:0/3041845804 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc998004ff0 con 0x7fc9a810d3c0 2024-07-04T08:25:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9b6584740 1 -- start start 2024-07-04T08:25:52.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558861614980 0x5588619e2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886161cb60 0x5588619e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9b6584740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558861957ce0 con 0x55886163d840 2024-07-04T08:25:52.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9b6584740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558861947280 con 0x55886161cb60 2024-07-04T08:25:52.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9b6584740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55886194b2c0 con 0x558861614980 2024-07-04T08:25:52.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9acc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9acc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619e23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41494/0 (socket says 172.21.15.39:41494) 2024-07-04T08:25:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9acc00640 1 -- 172.21.15.39:0/2987267713 learned_addr learned my addr 172.21.15.39:0/2987267713 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.394+0000 7fc9ad600640 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558861614980 0x5588619e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.395+0000 7fc9acc00640 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886161cb60 0x5588619e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.395+0000 7fc9ad600640 1 -- 172.21.15.39:0/2987267713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886161cb60 msgr2=0x5588619e2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.395+0000 7fc9ad600640 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886161cb60 0x5588619e2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.395+0000 7fc9ad600640 1 -- 172.21.15.39:0/2987267713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 msgr2=0x5588619e23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.395+0000 7fc9ad600640 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619e23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.395+0000 7fc9ad600640 1 -- 172.21.15.39:0/2987267713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588618de6f0 con 0x558861614980 2024-07-04T08:25:52.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.395+0000 7fc9acc00640 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886161cb60 0x5588619e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:25:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.395+0000 7fc9ad600640 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558861614980 0x5588619e2900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005fd0 tx=0x7fc9a0010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5374080affa95061 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.396+0000 7fc99ea00640 1 -- 172.21.15.39:0/2987267713 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0016070 con 0x558861614980 2024-07-04T08:25:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.396+0000 7fc99ea00640 1 -- 172.21.15.39:0/2987267713 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9a0012bc0 con 0x558861614980 2024-07-04T08:25:52.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.396+0000 7fc99ea00640 1 -- 172.21.15.39:0/2987267713 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0011bd0 con 0x558861614980 2024-07-04T08:25:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.396+0000 7fc9b6584740 1 -- 172.21.15.39:0/2987267713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558861614980 msgr2=0x5588619e2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.396+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558861614980 0x5588619e2900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005fd0 tx=0x7fc9a0010800 comp rx=0 tx=0).stop 2024-07-04T08:25:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9b6584740 1 -- 172.21.15.39:0/2987267713 shutdown_connections 2024-07-04T08:25:52.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886161cb60 0x5588619e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558861614980 0x5588619e2900 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2987267713 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9b6584740 1 -- 172.21.15.39:0/2987267713 >> 172.21.15.39:0/2987267713 conn(0x558861631990 msgr2=0x5588619e02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9b6584740 1 -- 172.21.15.39:0/2987267713 shutdown_connections 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9b6584740 1 -- 172.21.15.39:0/2987267713 wait complete. 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9a6000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc98c0b2870 0x7fc98c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9a6000640 1 -- 172.21.15.39:0/3041845804 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7fc9980e2dc0 con 0x7fc9a810d3c0 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.397+0000 7fc9ad600640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc98c0b2870 0x7fc98c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.398+0000 7fc9b6584740 1 -- start start 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.398+0000 7fc9b6584740 1 Processor -- start 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.398+0000 7fc9ad600640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc98c0b2870 0x7fc98c0b4d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009d80 tx=0x7fc9a0005bd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:52.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619edf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886161cb60 0x5588619ee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886163d840 0x5588619f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9b6584740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5588617ab7a0 con 0x55886161cb60 2024-07-04T08:25:52.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9b6584740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5588618ee6d0 con 0x558861614980 2024-07-04T08:25:52.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9b6584740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5588619f0ba0 con 0x55886163d840 2024-07-04T08:25:52.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9b6584740 10 client.0 _ll_register_callbacks cb 0x558861655090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:52.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9acc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886161cb60 0x5588619ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9ad600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9acc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886161cb60 0x5588619ee490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41496/0 (socket says 172.21.15.39:41496) 2024-07-04T08:25:52.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9ad600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619edf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43154/0 (socket says 172.21.15.39:43154) 2024-07-04T08:25:52.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9acc00640 1 -- 172.21.15.39:0/3119276930 learned_addr learned my addr 172.21.15.39:0/3119276930 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:52.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.399+0000 7fc9ae000640 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886163d840 0x5588619f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.400+0000 7fc9acc00640 1 -- 172.21.15.39:0/3119276930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886163d840 msgr2=0x5588619f9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.400+0000 7fc9acc00640 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886163d840 0x5588619f9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.400+0000 7fc9acc00640 1 -- 172.21.15.39:0/3119276930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 msgr2=0x5588619edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.400+0000 7fc9acc00640 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619edf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.400+0000 7fc9acc00640 1 -- 172.21.15.39:0/3119276930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588618df1d0 con 0x55886161cb60 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.400+0000 7fc9acc00640 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886161cb60 0x5588619ee490 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc994006010 tx=0x7fc9940314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec64057f83f687f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.400+0000 7fc993400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.401+0000 7fc993400640 1 -- 172.21.15.39:0/3119276930 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994035040 con 0x55886161cb60 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.401+0000 7fc993400640 1 -- 172.21.15.39:0/3119276930 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7fc994031810 con 0x55886161cb60 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.401+0000 7fc9b6584740 1 -- 172.21.15.39:0/3119276930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a0012bc0 con 0x55886161cb60 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.401+0000 7fc993400640 1 -- 172.21.15.39:0/3119276930 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc994004040 con 0x55886161cb60 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.401+0000 7fc9b6584740 10 client.17706 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:52.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.401+0000 7fc9b6584740 1 -- 172.21.15.39:0/3119276930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5588616b7b90 con 0x55886161cb60 2024-07-04T08:25:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.401+0000 7fc993400640 1 -- 172.21.15.39:0/3119276930 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9a0012bc0 con 0x55886161cb60 2024-07-04T08:25:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.401+0000 7fc993400640 1 -- 172.21.15.39:0/3119276930 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc9940047e0 con 0x55886161cb60 2024-07-04T08:25:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.402+0000 7fc9b6584740 10 client.17706 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.402+0000 7fc9b6584740 10 client.17706 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:25:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.402+0000 7fc9b6584740 20 client.17706 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.402+0000 7fc9b6584740 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5588615fc1c0 0x5588619ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.402+0000 7fc9b6584740 4 client.17706 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:25:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.402+0000 7fc9b6584740 1 -- 172.21.15.39:0/3119276930 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558861959c30 con 0x5588615fc1c0 2024-07-04T08:25:52.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.402+0000 7fc9ad600640 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5588615fc1c0 0x5588619ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.403+0000 7fc9ad600640 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5588615fc1c0 0x5588619ffe50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bb0bb6810fd142d server_cookie=26833cc34a08042b in_seq=0 out_seq=0 2024-07-04T08:25:52.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.403+0000 7fc993400640 10 client.17706 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:52.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.404+0000 7fc993400640 1 -- 172.21.15.39:0/3119276930 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558861959c30 con 0x5588615fc1c0 2024-07-04T08:25:52.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.404+0000 7fc993400640 10 client.17706 handle_command_reply: tid=0 2024-07-04T08:25:52.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.404+0000 7fc9b6584740 1 client.17706 shutdown 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 -- 172.21.15.39:0/3119276930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886161cb60 msgr2=0x5588619ee490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886161cb60 0x5588619ee490 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc994006010 tx=0x7fc9940314a0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 -- 172.21.15.39:0/3119276930 shutdown_connections 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5588615fc1c0 0x5588619ffe50 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886161cb60 0x5588619ee490 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886163d840 0x5588619f9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 --2- 172.21.15.39:0/3119276930 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 -- 172.21.15.39:0/3119276930 >> 172.21.15.39:0/3119276930 conn(0x558861631990 msgr2=0x5588619e55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc993400640 0 client.17706 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 -- 172.21.15.39:0/3119276930 shutdown_connections 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 1 -- 172.21.15.39:0/3119276930 wait complete. 2024-07-04T08:25:52.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.405+0000 7fc9b6584740 20 client.17706 trim_cache size 0 max 16384 2024-07-04T08:25:52.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.427+0000 7fc9b6584740 1 Processor -- start 2024-07-04T08:25:52.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.427+0000 7fc9b6584740 1 -- start start 2024-07-04T08:25:52.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.427+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619efc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.427+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 0x5588619e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.427+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619df880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.427+0000 7fc9b6584740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81b2660 con 0x55886163d840 2024-07-04T08:25:52.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.427+0000 7fc9b6584740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81b41a0 con 0x558861614980 2024-07-04T08:25:52.419 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.427+0000 7fc9b6584740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81b2d30 con 0x55886161cb60 2024-07-04T08:25:52.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619df880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41500/0 (socket says 172.21.15.39:41500) 2024-07-04T08:25:52.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9ad600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9ae000640 1 -- 172.21.15.39:0/803382810 learned_addr learned my addr 172.21.15.39:0/803382810 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:52.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9acc00640 1 --2- 172.21.15.39:0/803382810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 0x5588619e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9ad600640 1 -- 172.21.15.39:0/803382810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 msgr2=0x5588619e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:52.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9ad600640 1 --2- 172.21.15.39:0/803382810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 0x5588619e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9ad600640 1 -- 172.21.15.39:0/803382810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 msgr2=0x5588619df880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:52.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9ad600640 1 --2- 172.21.15.39:0/803382810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619df880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.428+0000 7fc9ad600640 1 -- 172.21.15.39:0/803382810 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588619e7bb0 con 0x558861614980 2024-07-04T08:25:52.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.429+0000 7fc9ad600640 1 --2- 172.21.15.39:0/803382810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619efc30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0011870 tx=0x7fc9a0009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb0d3f85d97ca2a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:52.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.429+0000 7fc99e000640 1 -- 172.21.15.39:0/803382810 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0016040 con 0x558861614980 2024-07-04T08:25:52.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.429+0000 7fc99e000640 1 -- 172.21.15.39:0/803382810 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9a0012bc0 con 0x558861614980 2024-07-04T08:25:52.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.429+0000 7fc99e000640 1 -- 172.21.15.39:0/803382810 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0034c00 con 0x558861614980 2024-07-04T08:25:52.423 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.430+0000 7fc9b6584740 1 -- 172.21.15.39:0/803382810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 msgr2=0x5588619efc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:52.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.430+0000 7fc9b6584740 1 --2- 172.21.15.39:0/803382810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619efc30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0011870 tx=0x7fc9a0009670 comp rx=0 tx=0).stop 2024-07-04T08:25:52.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.430+0000 7fc9b6584740 1 -- 172.21.15.39:0/803382810 shutdown_connections 2024-07-04T08:25:52.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.430+0000 7fc9b6584740 1 --2- 172.21.15.39:0/803382810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55886163d840 0x5588619df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.430+0000 7fc9b6584740 1 --2- 172.21.15.39:0/803382810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 0x5588619e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.430+0000 7fc9b6584740 1 --2- 172.21.15.39:0/803382810 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558861614980 0x5588619efc30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.430+0000 7fc9b6584740 1 -- 172.21.15.39:0/803382810 >> 172.21.15.39:0/803382810 conn(0x558861631990 msgr2=0x5588619ec5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:52.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.430+0000 7fc9b6584740 1 -- 172.21.15.39:0/803382810 shutdown_connections 2024-07-04T08:25:52.424 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.430+0000 7fc9b6584740 1 -- 172.21.15.39:0/803382810 wait complete. 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.431+0000 7fc9b6584740 1 -- start start 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.431+0000 7fc9b6584740 1 Processor -- start 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558861614980 0x5588619f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 0x5588619f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9b6584740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886163d840 0x5588619f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9b6584740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81b3ad0 con 0x558861614980 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9b6584740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81b3400 con 0x55886163d840 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9b6584740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a81b0640 con 0x55886161cb60 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9b6584740 10 client.0 _ll_register_callbacks cb 0x7fc994009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886163d840 0x5588619f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9ad600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558861614980 0x5588619f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.425 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9ae000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886163d840 0x5588619f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43172/0 (socket says 172.21.15.39:43172) 2024-07-04T08:25:52.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9ae000640 1 -- 172.21.15.39:0/2663729796 learned_addr learned my addr 172.21.15.39:0/2663729796 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:52.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9acc00640 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 0x5588619f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9ae000640 1 -- 172.21.15.39:0/2663729796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 msgr2=0x5588619f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:52.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9ae000640 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 0x5588619f22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9ae000640 1 -- 172.21.15.39:0/2663729796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558861614980 msgr2=0x5588619f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:25:52.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9ae000640 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558861614980 0x5588619f1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.432+0000 7fc9ae000640 1 -- 172.21.15.39:0/2663729796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558861a67660 con 0x55886163d840 2024-07-04T08:25:52.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.433+0000 7fc9ae000640 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886163d840 0x5588619f27e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc99800e940 tx=0x7fc9980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41607ff3c75fe08b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:52.426 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.433+0000 7fc993400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:25:52.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.433+0000 7fc993400640 1 -- 172.21.15.39:0/2663729796 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9980c43a0 con 0x55886163d840 2024-07-04T08:25:52.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.433+0000 7fc9b6584740 1 -- 172.21.15.39:0/2663729796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a0012bc0 con 0x55886163d840 2024-07-04T08:25:52.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.433+0000 7fc993400640 1 -- 172.21.15.39:0/2663729796 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7fc9980b28c0 con 0x55886163d840 2024-07-04T08:25:52.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.433+0000 7fc993400640 1 -- 172.21.15.39:0/2663729796 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc9980a9ce0 con 0x55886163d840 2024-07-04T08:25:52.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.433+0000 7fc9b6584740 10 client.27845 fetch_fsmap learned FSMap version 30 2024-07-04T08:25:52.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.433+0000 7fc9b6584740 1 -- 172.21.15.39:0/2663729796 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558861a68910 con 0x55886163d840 2024-07-04T08:25:52.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.434+0000 7fc993400640 1 -- 172.21.15.39:0/2663729796 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9a0012bc0 con 0x55886163d840 2024-07-04T08:25:52.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.434+0000 7fc993400640 1 -- 172.21.15.39:0/2663729796 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc9980a89b0 con 0x55886163d840 2024-07-04T08:25:52.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.434+0000 7fc9b6584740 10 client.27845 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:25:52.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.434+0000 7fc9b6584740 10 client.27845 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:25:52.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.434+0000 7fc9b6584740 20 client.27845 populate_metadata read hostname 'smithi039' 2024-07-04T08:25:52.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.434+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5588615d1570 0x558861a6b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:52.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.434+0000 7fc9b6584740 4 client.27845 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-07-04T08:25:52.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.434+0000 7fc9b6584740 1 -- 172.21.15.39:0/2663729796 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5588619556d0 con 0x5588615d1570 2024-07-04T08:25:52.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.434+0000 7fc9ad600640 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5588615d1570 0x558861a6b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:52.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.435+0000 7fc9ad600640 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5588615d1570 0x558861a6b0f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d285e4795ec8590 server_cookie=5f47e356d41ea47b in_seq=0 out_seq=0 2024-07-04T08:25:52.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.435+0000 7fc993400640 10 client.27845 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:52.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.445+0000 7fc993400640 1 -- 172.21.15.39:0/2663729796 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5588619556d0 con 0x5588615d1570 2024-07-04T08:25:52.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.445+0000 7fc993400640 10 client.27845 handle_command_reply: tid=0 2024-07-04T08:25:52.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.445+0000 7fc9b6584740 1 client.27845 shutdown 2024-07-04T08:25:52.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.445+0000 7fc9b6584740 1 -- 172.21.15.39:0/2663729796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886163d840 msgr2=0x5588619f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:52.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.445+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886163d840 0x5588619f27e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc99800e940 tx=0x7fc9980046c0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.445+0000 7fc9b6584740 1 -- 172.21.15.39:0/2663729796 shutdown_connections 2024-07-04T08:25:52.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.445+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5588615d1570 0x558861a6b0f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.446+0000 7fc993400640 0 client.27845 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:25:52.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.445+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55886163d840 0x5588619f27e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.446+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55886161cb60 0x5588619f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.446+0000 7fc9b6584740 1 --2- 172.21.15.39:0/2663729796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558861614980 0x5588619f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.446+0000 7fc9b6584740 1 -- 172.21.15.39:0/2663729796 >> 172.21.15.39:0/2663729796 conn(0x558861631990 msgr2=0x5588619e3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:52.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.446+0000 7fc9b6584740 1 -- 172.21.15.39:0/2663729796 shutdown_connections 2024-07-04T08:25:52.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.446+0000 7fc9b6584740 1 -- 172.21.15.39:0/2663729796 wait complete. 2024-07-04T08:25:52.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.446+0000 7fc9b6584740 20 client.27845 trim_cache size 0 max 16384 2024-07-04T08:25:52.438 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:25:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:25:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:25:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 30, 2024-07-04T08:25:52.438 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "1289c4b8": { 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 30, 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 150.0, 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:25:52.439 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:25:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:25:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:25:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:25:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:52.440 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:25:52.440 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:25:52.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 -- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc98c0b2870 msgr2=0x7fc98c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:52.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc98c0b2870 0x7fc98c0b4d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009d80 tx=0x7fc9a0005bd0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 -- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810d3c0 msgr2=0x7fc9a81a1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:52.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810d3c0 0x7fc9a81a1420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc99800e970 tx=0x7fc99800ed70 comp rx=0 tx=0).stop 2024-07-04T08:25:52.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 -- 172.21.15.39:0/3041845804 shutdown_connections 2024-07-04T08:25:52.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc98c0b2870 0x7fc98c0b4d80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9a810d3c0 0x7fc9a81a1420 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc9a8104840 0x7fc9a819c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 --2- 172.21.15.39:0/3041845804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9a8103e10 0x7fc9a819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:52.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 -- 172.21.15.39:0/3041845804 >> 172.21.15.39:0/3041845804 conn(0x7fc9a8078cb0 msgr2=0x7fc9a80ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:52.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 -- 172.21.15.39:0/3041845804 shutdown_connections 2024-07-04T08:25:52.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:52.447+0000 7fc9b0000640 1 -- 172.21.15.39:0/3041845804 wait complete. 2024-07-04T08:25:52.970 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:25:52.971 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:25:53.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:53 smithi133 ceph-mon[37050]: pgmap v808: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 16 op/s 2024-07-04T08:25:53.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:53 smithi039 ceph-mon[32062]: pgmap v808: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 16 op/s 2024-07-04T08:25:53.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:53 smithi050 ceph-mon[36577]: pgmap v808: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 52 KiB/s wr, 16 op/s 2024-07-04T08:25:53.781 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1289c4b8', quiesce took 9.7 seconds. Will release after: 20.3 2024-07-04T08:25:54.071 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:25:55.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:55 smithi039 ceph-mon[32062]: pgmap v809: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-07-04T08:25:55.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:55 smithi133 ceph-mon[37050]: pgmap v809: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-07-04T08:25:55.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:55 smithi050 ceph-mon[36577]: pgmap v809: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 52 KiB/s wr, 13 op/s 2024-07-04T08:25:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.087+0000 7fbd54a00640 1 -- 172.21.15.39:0/2200988381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd501052d0 msgr2=0x7fbd501056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.087+0000 7fbd54a00640 1 --2- 172.21.15.39:0/2200988381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd501052d0 0x7fbd501056e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009900 tx=0x7fbd4002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.087+0000 7fbd54a00640 1 -- 172.21.15.39:0/2200988381 shutdown_connections 2024-07-04T08:25:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.087+0000 7fbd54a00640 1 --2- 172.21.15.39:0/2200988381 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd5010b560 0x7fbd50111d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.087+0000 7fbd54a00640 1 --2- 172.21.15.39:0/2200988381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd5010ac10 0x7fbd5010b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.087+0000 7fbd54a00640 1 --2- 172.21.15.39:0/2200988381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd501052d0 0x7fbd501056e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.087+0000 7fbd54a00640 1 -- 172.21.15.39:0/2200988381 >> 172.21.15.39:0/2200988381 conn(0x7fbd50100d60 msgr2=0x7fbd501031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.088+0000 7fbd54a00640 1 -- 172.21.15.39:0/2200988381 shutdown_connections 2024-07-04T08:25:56.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.088+0000 7fbd54a00640 1 -- 172.21.15.39:0/2200988381 wait complete. 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.089+0000 7fbd54a00640 1 Processor -- start 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.089+0000 7fbd54a00640 1 -- start start 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd54a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd501052d0 0x7fbd5019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd54a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd5010ac10 0x7fbd5019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd54a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd5010b560 0x7fbd501a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd54a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbd50115260 con 0x7fbd5010ac10 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd54a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd501150e0 con 0x7fbd501052d0 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd54a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbd501153e0 con 0x7fbd5010b560 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd4d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd5010ac10 0x7fbd5019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:56.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd4d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd5010ac10 0x7fbd5019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41530/0 (socket says 172.21.15.39:41530) 2024-07-04T08:25:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd4d400640 1 -- 172.21.15.39:0/3186419694 learned_addr learned my addr 172.21.15.39:0/3186419694 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:25:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.090+0000 7fbd4de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd501052d0 0x7fbd5019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.091+0000 7fbd4e800640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd5010b560 0x7fbd501a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:56.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.091+0000 7fbd4de00640 1 -- 172.21.15.39:0/3186419694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd5010b560 msgr2=0x7fbd501a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.091+0000 7fbd4de00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd5010b560 0x7fbd501a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.091+0000 7fbd4de00640 1 -- 172.21.15.39:0/3186419694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd5010ac10 msgr2=0x7fbd5019c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.091+0000 7fbd4de00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd5010ac10 0x7fbd5019c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.091+0000 7fbd4de00640 1 -- 172.21.15.39:0/3186419694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd501a1ba0 con 0x7fbd501052d0 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.091+0000 7fbd4de00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd501052d0 0x7fbd5019bc30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbd4002a9e0 tx=0x7fbd40003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.092+0000 7fbd46a00640 1 -- 172.21.15.39:0/3186419694 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4003f070 con 0x7fbd501052d0 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.092+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd501a1e30 con 0x7fbd501052d0 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.092+0000 7fbd46a00640 1 -- 172.21.15.39:0/3186419694 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd40031b50 con 0x7fbd501052d0 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.092+0000 7fbd46a00640 1 -- 172.21.15.39:0/3186419694 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40035030 con 0x7fbd501052d0 2024-07-04T08:25:56.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.092+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd501a85f0 con 0x7fbd501052d0 2024-07-04T08:25:56.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.093+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd501056e0 con 0x7fbd501052d0 2024-07-04T08:25:56.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.096+0000 7fbd46a00640 1 -- 172.21.15.39:0/3186419694 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbd40042070 con 0x7fbd501052d0 2024-07-04T08:25:56.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.097+0000 7fbd46a00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd300b2960 0x7fbd300b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:25:56.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.098+0000 7fbd46a00640 1 -- 172.21.15.39:0/3186419694 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7fbd40103120 con 0x7fbd501052d0 2024-07-04T08:25:56.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.098+0000 7fbd4d400640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd300b2960 0x7fbd300b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:25:56.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.098+0000 7fbd4d400640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd300b2960 0x7fbd300b4e70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbd5010a750 tx=0x7fbd38009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:25:56.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.099+0000 7fbd46a00640 1 -- 172.21.15.39:0/3186419694 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fbd40031d00 con 0x7fbd501052d0 2024-07-04T08:25:56.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.375+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd5019cd60 con 0x7fbd501052d0 2024-07-04T08:25:56.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.377+0000 7fbd46a00640 1 -- 172.21.15.39:0/3186419694 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fbd400ca0f0 con 0x7fbd501052d0 2024-07-04T08:25:56.368 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:25:56.368 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:25:56.369 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd300b2960 msgr2=0x7fbd300b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd300b2960 0x7fbd300b4e70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbd5010a750 tx=0x7fbd38009500 comp rx=0 tx=0).stop 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd501052d0 msgr2=0x7fbd5019bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd501052d0 0x7fbd5019bc30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbd4002a9e0 tx=0x7fbd40003b90 comp rx=0 tx=0).stop 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 shutdown_connections 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd300b2960 0x7fbd300b4e70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd5010ac10 0x7fbd5019c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd5010b560 0x7fbd501a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 --2- 172.21.15.39:0/3186419694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd501052d0 0x7fbd5019bc30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 >> 172.21.15.39:0/3186419694 conn(0x7fbd50100d60 msgr2=0x7fbd50103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:25:56.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 shutdown_connections 2024-07-04T08:25:56.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:25:56.380+0000 7fbd54a00640 1 -- 172.21.15.39:0/3186419694 wait complete. 2024-07-04T08:25:57.269 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:25:57.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:57 smithi133 ceph-mon[37050]: pgmap v810: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-07-04T08:25:57.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:57 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3186419694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:25:57.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:57 smithi039 ceph-mon[32062]: pgmap v810: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-07-04T08:25:57.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:57 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3186419694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:25:57.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:57 smithi050 ceph-mon[36577]: pgmap v810: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 66 KiB/s wr, 16 op/s 2024-07-04T08:25:57.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:57 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3186419694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:25:57.759 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:25:59.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:25:59 smithi039 ceph-mon[32062]: pgmap v811: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-07-04T08:25:59.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:25:59 smithi133 ceph-mon[37050]: pgmap v811: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-07-04T08:25:59.635 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:25:59 smithi050 ceph-mon[36577]: pgmap v811: 129 pgs: 129 active+clean; 16 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-07-04T08:26:00.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.141+0000 7f17aee00640 1 -- 172.21.15.39:0/1711639659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a8108820 msgr2=0x7f17a8108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:00.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.141+0000 7f17aee00640 1 --2- 172.21.15.39:0/1711639659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a8108820 0x7f17a8108c30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1794009860 tx=0x7f179402a340 comp rx=0 tx=0).stop 2024-07-04T08:26:00.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.141+0000 7f17aee00640 1 -- 172.21.15.39:0/1711639659 shutdown_connections 2024-07-04T08:26:00.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.141+0000 7f17aee00640 1 --2- 172.21.15.39:0/1711639659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a8109170 0x7f17a810f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.141+0000 7f17aee00640 1 --2- 172.21.15.39:0/1711639659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a8108820 0x7f17a8108c30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.141+0000 7f17aee00640 1 --2- 172.21.15.39:0/1711639659 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a8102ee0 0x7f17a81032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.141+0000 7f17aee00640 1 -- 172.21.15.39:0/1711639659 >> 172.21.15.39:0/1711639659 conn(0x7f17a80fe930 msgr2=0x7f17a8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:00.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.141+0000 7f17aee00640 1 -- 172.21.15.39:0/1711639659 shutdown_connections 2024-07-04T08:26:00.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.141+0000 7f17aee00640 1 -- 172.21.15.39:0/1711639659 wait complete. 2024-07-04T08:26:00.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.142+0000 7f17aee00640 1 Processor -- start 2024-07-04T08:26:00.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.143+0000 7f17aee00640 1 -- start start 2024-07-04T08:26:00.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.143+0000 7f17aee00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a8102ee0 0x7f17a819ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.143+0000 7f17aee00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a8108820 0x7f17a819c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.143+0000 7f17aee00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a8109170 0x7f17a81a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.143+0000 7f17aee00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8112e30 con 0x7f17a8108820 2024-07-04T08:26:00.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.143+0000 7f17aee00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8112cb0 con 0x7f17a8102ee0 2024-07-04T08:26:00.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.143+0000 7f17aee00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8112fb0 con 0x7f17a8109170 2024-07-04T08:26:00.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ad000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a8108820 0x7f17a819c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ada00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a8102ee0 0x7f17a819ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ad000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a8108820 0x7f17a819c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41558/0 (socket says 172.21.15.39:41558) 2024-07-04T08:26:00.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ad000640 1 -- 172.21.15.39:0/1735390756 learned_addr learned my addr 172.21.15.39:0/1735390756 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:00.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ae400640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a8109170 0x7f17a81a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ada00640 1 -- 172.21.15.39:0/1735390756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a8109170 msgr2=0x7f17a81a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:00.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ada00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a8109170 0x7f17a81a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ada00640 1 -- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a8108820 msgr2=0x7f17a819c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:00.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ada00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a8108820 0x7f17a819c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ada00640 1 -- 172.21.15.39:0/1735390756 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a81a19f0 con 0x7f17a8102ee0 2024-07-04T08:26:00.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.144+0000 7f17ad000640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a8108820 0x7f17a819c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:00.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.145+0000 7f17ada00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a8102ee0 0x7f17a819ba80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f179c005fd0 tx=0x7f179c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:00.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.145+0000 7f17a6000640 1 -- 172.21.15.39:0/1735390756 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c009c30 con 0x7f17a8102ee0 2024-07-04T08:26:00.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.145+0000 7f17a6000640 1 -- 172.21.15.39:0/1735390756 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f179c009dd0 con 0x7f17a8102ee0 2024-07-04T08:26:00.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.145+0000 7f17aee00640 1 -- 172.21.15.39:0/1735390756 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a81a1ce0 con 0x7f17a8102ee0 2024-07-04T08:26:00.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.145+0000 7f17a6000640 1 -- 172.21.15.39:0/1735390756 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c010460 con 0x7f17a8102ee0 2024-07-04T08:26:00.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.145+0000 7f17aee00640 1 -- 172.21.15.39:0/1735390756 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a81a8440 con 0x7f17a8102ee0 2024-07-04T08:26:00.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.148+0000 7f17b69e1740 1 Processor -- start 2024-07-04T08:26:00.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17b69e1740 1 -- start start 2024-07-04T08:26:00.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91e0070 0x5562e9584990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 0x5562e9584ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 0x5562e9585410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17b69e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5562e94faf00 con 0x5562e91e0070 2024-07-04T08:26:00.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17b69e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5562e94ea4a0 con 0x5562e91bf390 2024-07-04T08:26:00.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17b69e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5562e94ee4e0 con 0x5562e91b71b0 2024-07-04T08:26:00.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17ae400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91e0070 0x5562e9584990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17ae400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91e0070 0x5562e9584990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41564/0 (socket says 172.21.15.39:41564) 2024-07-04T08:26:00.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17ae400640 1 -- 172.21.15.39:0/3293068702 learned_addr learned my addr 172.21.15.39:0/3293068702 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:00.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17ad000640 1 --2- 172.21.15.39:0/3293068702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 0x5562e9585410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.149+0000 7f17a6000640 1 -- 172.21.15.39:0/1735390756 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f179c003e80 con 0x7f17a8102ee0 2024-07-04T08:26:00.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.150+0000 7f17ae400640 1 -- 172.21.15.39:0/3293068702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 msgr2=0x5562e9584ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:00.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.150+0000 7f17ae400640 1 --2- 172.21.15.39:0/3293068702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 0x5562e9584ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.150+0000 7f17ae400640 1 -- 172.21.15.39:0/3293068702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 msgr2=0x5562e9585410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:00.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.150+0000 7f17ae400640 1 --2- 172.21.15.39:0/3293068702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 0x5562e9585410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.150+0000 7f17ae400640 1 -- 172.21.15.39:0/3293068702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e9482670 con 0x5562e91e0070 2024-07-04T08:26:00.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.150+0000 7f17ae400640 1 --2- 172.21.15.39:0/3293068702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91e0070 0x5562e9584990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f17a00099f0 tx=0x7f17a00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7598c8532e26d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:00.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.151+0000 7f179aa00640 1 -- 172.21.15.39:0/3293068702 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a001d070 con 0x5562e91e0070 2024-07-04T08:26:00.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.151+0000 7f179aa00640 1 -- 172.21.15.39:0/3293068702 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a0003e40 con 0x5562e91e0070 2024-07-04T08:26:00.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.151+0000 7f179aa00640 1 -- 172.21.15.39:0/3293068702 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a000d9d0 con 0x5562e91e0070 2024-07-04T08:26:00.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.151+0000 7f17a6000640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f17900b2960 0x7f17900b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17b69e1740 1 -- 172.21.15.39:0/3293068702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91e0070 msgr2=0x5562e9584990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17b69e1740 1 --2- 172.21.15.39:0/3293068702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91e0070 0x5562e9584990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f17a00099f0 tx=0x7f17a00095c0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17a6000640 1 -- 172.21.15.39:0/1735390756 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f179c0e2900 con 0x7f17a8102ee0 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17b69e1740 1 -- 172.21.15.39:0/3293068702 shutdown_connections 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17ada00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f17900b2960 0x7f17900b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17b69e1740 1 --2- 172.21.15.39:0/3293068702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 0x5562e9585410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17b69e1740 1 --2- 172.21.15.39:0/3293068702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 0x5562e9584ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17b69e1740 1 --2- 172.21.15.39:0/3293068702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91e0070 0x5562e9584990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17b69e1740 1 -- 172.21.15.39:0/3293068702 >> 172.21.15.39:0/3293068702 conn(0x5562e91d4460 msgr2=0x5562e9582880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:00.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17b69e1740 1 -- 172.21.15.39:0/3293068702 shutdown_connections 2024-07-04T08:26:00.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.152+0000 7f17b69e1740 1 -- 172.21.15.39:0/3293068702 wait complete. 2024-07-04T08:26:00.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.153+0000 7f17ada00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f17900b2960 0x7f17900b4e70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x5562e9583610 tx=0x7f179c00bef0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:00.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.153+0000 7f17b69e1740 1 -- start start 2024-07-04T08:26:00.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.153+0000 7f17b69e1740 1 Processor -- start 2024-07-04T08:26:00.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91bf390 0x5562e9590b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 0x5562e959c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17b69e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5562e934e700 con 0x5562e91b71b0 2024-07-04T08:26:00.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17b69e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5562e9491bd0 con 0x5562e91e0070 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17b69e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5562e9593170 con 0x5562e91bf390 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17b69e1740 10 client.0 _ll_register_callbacks cb 0x5562e94e3590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17ae400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17ae400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95905b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41580/0 (socket says 172.21.15.39:41580) 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17ae400640 1 -- 172.21.15.39:0/2389243764 learned_addr learned my addr 172.21.15.39:0/2389243764 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17ad000640 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91bf390 0x5562e9590b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.154+0000 7f17ae400640 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 0x5562e959c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f17ae400640 1 -- 172.21.15.39:0/2389243764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91bf390 msgr2=0x5562e9590b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f17ae400640 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91bf390 0x5562e9590b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f17ae400640 1 -- 172.21.15.39:0/2389243764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 msgr2=0x5562e959c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f17ae400640 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 0x5562e959c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f17ae400640 1 -- 172.21.15.39:0/2389243764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e924b650 con 0x5562e91b71b0 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f17ae400640 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95905b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f17a001b040 tx=0x7f17a0004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549a1783ce94b589 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f178f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f178f400640 1 -- 172.21.15.39:0/2389243764 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a001d030 con 0x5562e91b71b0 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f178f400640 1 -- 172.21.15.39:0/2389243764 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f17a0004590 con 0x5562e91b71b0 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.155+0000 7f17b69e1740 1 -- 172.21.15.39:0/2389243764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562e94fce50 con 0x5562e91b71b0 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f178f400640 1 -- 172.21.15.39:0/2389243764 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17a0004a40 con 0x5562e91b71b0 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f17b69e1740 10 client.17730 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f17b69e1740 1 -- 172.21.15.39:0/2389243764 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562e959f200 con 0x5562e91b71b0 2024-07-04T08:26:00.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f178f400640 1 -- 172.21.15.39:0/2389243764 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5562e94fce50 con 0x5562e91b71b0 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f178f400640 1 -- 172.21.15.39:0/2389243764 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f17a000b040 con 0x5562e91b71b0 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f17b69e1740 10 client.17730 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f17b69e1740 10 client.17730 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f17b69e1740 20 client.17730 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562e919e9f0 0x5562e95a2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f17b69e1740 4 client.17730 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.156+0000 7f17b69e1740 1 -- 172.21.15.39:0/2389243764 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562e94f88f0 con 0x5562e919e9f0 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.157+0000 7f17ad000640 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562e919e9f0 0x5562e95a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.158+0000 7f17ad000640 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562e919e9f0 0x5562e95a2610 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=470571fad0e7715b server_cookie=736d7822e22061c3 in_seq=0 out_seq=0 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.158+0000 7f178f400640 10 client.17730 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:26:00.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f178f400640 1 -- 172.21.15.39:0/2389243764 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5562e94f88f0 con 0x5562e919e9f0 2024-07-04T08:26:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f178f400640 10 client.17730 handle_command_reply: tid=0 2024-07-04T08:26:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 client.17730 shutdown 2024-07-04T08:26:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 -- 172.21.15.39:0/2389243764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 msgr2=0x5562e95905b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95905b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f17a001b040 tx=0x7f17a0004940 comp rx=0 tx=0).stop 2024-07-04T08:26:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 -- 172.21.15.39:0/2389243764 shutdown_connections 2024-07-04T08:26:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562e919e9f0 0x5562e95a2610 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 0x5562e959c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91bf390 0x5562e9590b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2389243764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95905b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 -- 172.21.15.39:0/2389243764 >> 172.21.15.39:0/2389243764 conn(0x5562e91d4460 msgr2=0x5562e9587b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:00.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f178f400640 0 client.17730 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:26:00.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 -- 172.21.15.39:0/2389243764 shutdown_connections 2024-07-04T08:26:00.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 1 -- 172.21.15.39:0/2389243764 wait complete. 2024-07-04T08:26:00.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.159+0000 7f17b69e1740 20 client.17730 trim_cache size 0 max 16384 2024-07-04T08:26:00.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.175+0000 7f17b69e1740 1 Processor -- start 2024-07-04T08:26:00.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.175+0000 7f17b69e1740 1 -- start start 2024-07-04T08:26:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.175+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.175+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 0x5562e959c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.175+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91e0070 0x5562e959c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.175+0000 7f17b69e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81b2490 con 0x5562e91b71b0 2024-07-04T08:26:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.175+0000 7f17b69e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81b3fd0 con 0x5562e91bf390 2024-07-04T08:26:00.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.175+0000 7f17b69e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81b2b60 con 0x5562e91e0070 2024-07-04T08:26:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.176+0000 7f17ae400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.176+0000 7f17ae400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95860d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:41586/0 (socket says 172.21.15.39:41586) 2024-07-04T08:26:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.176+0000 7f17ae400640 1 -- 172.21.15.39:0/2840821156 learned_addr learned my addr 172.21.15.39:0/2840821156 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.176+0000 7f17ad000640 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 0x5562e959c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.176+0000 7f17ae400640 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91e0070 0x5562e959c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.176+0000 7f17ae400640 1 -- 172.21.15.39:0/2840821156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91e0070 msgr2=0x5562e959c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.176+0000 7f17ae400640 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91e0070 0x5562e959c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.176+0000 7f17ae400640 1 -- 172.21.15.39:0/2840821156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 msgr2=0x5562e959c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.176+0000 7f17ae400640 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 0x5562e959c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.177+0000 7f17ae400640 1 -- 172.21.15.39:0/2840821156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e95877f0 con 0x5562e91b71b0 2024-07-04T08:26:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.177+0000 7f17ad000640 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 0x5562e959c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.177+0000 7f17ae400640 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95860d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005120 tx=0x7f17a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805005a18a059458 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.177+0000 7f179a000640 1 -- 172.21.15.39:0/2840821156 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a001d040 con 0x5562e91b71b0 2024-07-04T08:26:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.177+0000 7f179a000640 1 -- 172.21.15.39:0/2840821156 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a0004790 con 0x5562e91b71b0 2024-07-04T08:26:00.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.177+0000 7f179a000640 1 -- 172.21.15.39:0/2840821156 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0004e40 con 0x5562e91b71b0 2024-07-04T08:26:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.178+0000 7f17b69e1740 1 -- 172.21.15.39:0/2840821156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 msgr2=0x5562e95860d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.178+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95860d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005120 tx=0x7f17a0005fd0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.178+0000 7f17b69e1740 1 -- 172.21.15.39:0/2840821156 shutdown_connections 2024-07-04T08:26:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.178+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91e0070 0x5562e959c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.178+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91bf390 0x5562e959c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.178+0000 7f17b69e1740 1 --2- 172.21.15.39:0/2840821156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91b71b0 0x5562e95860d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.178+0000 7f17b69e1740 1 -- 172.21.15.39:0/2840821156 >> 172.21.15.39:0/2840821156 conn(0x5562e91d4460 msgr2=0x5562e95859c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.178+0000 7f17b69e1740 1 -- 172.21.15.39:0/2840821156 shutdown_connections 2024-07-04T08:26:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.178+0000 7f17b69e1740 1 -- 172.21.15.39:0/2840821156 wait complete. 2024-07-04T08:26:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.179+0000 7f17b69e1740 1 -- start start 2024-07-04T08:26:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.179+0000 7f17b69e1740 1 Processor -- start 2024-07-04T08:26:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 0x5562e95a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91bf390 0x5562e95a3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17b69e1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 0x5562e9611360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17b69e1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81b3900 con 0x5562e91bf390 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17b69e1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81b3230 con 0x5562e91e0070 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17b69e1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81b0470 con 0x5562e91b71b0 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17b69e1740 10 client.0 _ll_register_callbacks cb 0x7f17a001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17ae400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 0x5562e95a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17ad000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91bf390 0x5562e95a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17ae400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 0x5562e95a2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:54382/0 (socket says 172.21.15.39:54382) 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17ae400640 1 -- 172.21.15.39:0/1264029274 learned_addr learned my addr 172.21.15.39:0/1264029274 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.180+0000 7f17ae400640 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 0x5562e9611360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.181+0000 7f17ad000640 1 -- 172.21.15.39:0/1264029274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 msgr2=0x5562e95a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.181+0000 7f17ad000640 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 0x5562e95a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.181+0000 7f17ad000640 1 -- 172.21.15.39:0/1264029274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 msgr2=0x5562e9611360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.181+0000 7f17ad000640 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 0x5562e9611360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.181+0000 7f17ad000640 1 -- 172.21.15.39:0/1264029274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e96119d0 con 0x5562e91bf390 2024-07-04T08:26:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.181+0000 7f17ae400640 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 0x5562e95a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.181+0000 7f17ad000640 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91bf390 0x5562e95a3380 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1794009830 tx=0x7f1794004110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2593924160aa420 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.181+0000 7f178f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:26:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.181+0000 7f178f400640 1 -- 172.21.15.39:0/1264029274 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794037040 con 0x5562e91bf390 2024-07-04T08:26:00.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.182+0000 7f178f400640 1 -- 172.21.15.39:0/1264029274 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f17940042c0 con 0x5562e91bf390 2024-07-04T08:26:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.182+0000 7f17b69e1740 1 -- 172.21.15.39:0/1264029274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a0004790 con 0x5562e91bf390 2024-07-04T08:26:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.182+0000 7f178f400640 1 -- 172.21.15.39:0/1264029274 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17940044c0 con 0x5562e91bf390 2024-07-04T08:26:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.182+0000 7f17b69e1740 10 client.17736 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.182+0000 7f17b69e1740 1 -- 172.21.15.39:0/1264029274 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562e9612cb0 con 0x5562e91bf390 2024-07-04T08:26:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.182+0000 7f178f400640 1 -- 172.21.15.39:0/1264029274 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a0004790 con 0x5562e91bf390 2024-07-04T08:26:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.183+0000 7f178f400640 1 -- 172.21.15.39:0/1264029274 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f17940035f0 con 0x5562e91bf390 2024-07-04T08:26:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.183+0000 7f17b69e1740 10 client.17736 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.183+0000 7f17b69e1740 10 client.17736 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:26:00.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.183+0000 7f17b69e1740 20 client.17736 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.183+0000 7f17b69e1740 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562e9173da0 0x5562e96154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.183+0000 7f17b69e1740 4 client.17736 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.183+0000 7f17b69e1740 1 -- 172.21.15.39:0/1264029274 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562e94f6030 con 0x5562e9173da0 2024-07-04T08:26:00.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.183+0000 7f17ae400640 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562e9173da0 0x5562e96154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:00.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17ae400640 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562e9173da0 0x5562e96154c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e6c3503ea688ec68 server_cookie=6e8a4643b5ee93a4 in_seq=0 out_seq=0 2024-07-04T08:26:00.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f178f400640 10 client.17736 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f178f400640 1 -- 172.21.15.39:0/1264029274 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562e94f6030 con 0x5562e9173da0 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f178f400640 10 client.17736 handle_command_reply: tid=0 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17b69e1740 1 client.17736 shutdown 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17b69e1740 1 -- 172.21.15.39:0/1264029274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91bf390 msgr2=0x5562e95a3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17b69e1740 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91bf390 0x5562e95a3380 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1794009830 tx=0x7f1794004110 comp rx=0 tx=0).stop 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17b69e1740 1 -- 172.21.15.39:0/1264029274 shutdown_connections 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17b69e1740 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5562e9173da0 0x5562e96154c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17b69e1740 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5562e91bf390 0x5562e95a3380 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17b69e1740 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5562e91e0070 0x5562e9611360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17b69e1740 1 --2- 172.21.15.39:0/1264029274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5562e91b71b0 0x5562e95a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.184+0000 7f17b69e1740 1 -- 172.21.15.39:0/1264029274 >> 172.21.15.39:0/1264029274 conn(0x5562e91d4460 msgr2=0x5562e9591860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:00.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.185+0000 7f178f400640 0 client.17736 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:26:00.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.185+0000 7f17b69e1740 1 -- 172.21.15.39:0/1264029274 shutdown_connections 2024-07-04T08:26:00.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.185+0000 7f17b69e1740 1 -- 172.21.15.39:0/1264029274 wait complete. 2024-07-04T08:26:00.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.185+0000 7f17b69e1740 20 client.17736 trim_cache size 0 max 16384 2024-07-04T08:26:00.182 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:00.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 -- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f17900b2960 msgr2=0x7f17900b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:00.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f17900b2960 0x7f17900b4e70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x5562e9583610 tx=0x7f179c00bef0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 -- 172.21.15.39:0/1735390756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a8102ee0 msgr2=0x7f17a819ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a8102ee0 0x7f17a819ba80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f179c005fd0 tx=0x7f179c00dde0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 -- 172.21.15.39:0/1735390756 shutdown_connections 2024-07-04T08:26:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f17900b2960 0x7f17900b4e70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a8108820 0x7f17a819c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a8109170 0x7f17a81a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 --2- 172.21.15.39:0/1735390756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17a8102ee0 0x7f17a819ba80 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 -- 172.21.15.39:0/1735390756 >> 172.21.15.39:0/1735390756 conn(0x7f17a80fe930 msgr2=0x7f17a8107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 -- 172.21.15.39:0/1735390756 shutdown_connections 2024-07-04T08:26:00.184 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:00.186+0000 7f17aee00640 1 -- 172.21.15.39:0/1735390756 wait complete. 2024-07-04T08:26:01.057 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:26:01.544 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:01.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:01 smithi133 ceph-mon[37050]: pgmap v812: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-07-04T08:26:01.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:01 smithi039 ceph-mon[32062]: pgmap v812: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-07-04T08:26:01.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:01 smithi050 ceph-mon[36577]: pgmap v812: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-07-04T08:26:03.529 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:03 smithi039 ceph-mon[32062]: pgmap v813: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-07-04T08:26:03.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:03 smithi133 ceph-mon[37050]: pgmap v813: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-07-04T08:26:03.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:03 smithi050 ceph-mon[36577]: pgmap v813: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-07-04T08:26:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.949+0000 7f81cca00640 1 -- 172.21.15.39:0/1223938533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c8074d00 msgr2=0x7f81c8075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.949+0000 7f81cca00640 1 --2- 172.21.15.39:0/1223938533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c8074d00 0x7f81c8075110 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009900 tx=0x7f81b802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.949+0000 7f81cca00640 1 -- 172.21.15.39:0/1223938533 shutdown_connections 2024-07-04T08:26:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.949+0000 7f81cca00640 1 --2- 172.21.15.39:0/1223938533 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81c810f460 0x7f81c81118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.949+0000 7f81cca00640 1 --2- 172.21.15.39:0/1223938533 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c8075650 0x7f81c810ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.949+0000 7f81cca00640 1 --2- 172.21.15.39:0/1223938533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c8074d00 0x7f81c8075110 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.949+0000 7f81cca00640 1 -- 172.21.15.39:0/1223938533 >> 172.21.15.39:0/1223938533 conn(0x7f81c80fe990 msgr2=0x7f81c8100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:03.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.949+0000 7f81cca00640 1 -- 172.21.15.39:0/1223938533 shutdown_connections 2024-07-04T08:26:03.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.949+0000 7f81cca00640 1 -- 172.21.15.39:0/1223938533 wait complete. 2024-07-04T08:26:03.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.950+0000 7f81cca00640 1 Processor -- start 2024-07-04T08:26:03.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.951+0000 7f81cca00640 1 -- start start 2024-07-04T08:26:03.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.951+0000 7f81cca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c8074d00 0x7f81c819baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.951+0000 7f81cca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81c8075650 0x7f81c819c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.951+0000 7f81cca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c810f460 0x7f81c81a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.951+0000 7f81cca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8115010 con 0x7f81c8075650 2024-07-04T08:26:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.951+0000 7f81cca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8114e90 con 0x7f81c8074d00 2024-07-04T08:26:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.951+0000 7f81cca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8115190 con 0x7f81c810f460 2024-07-04T08:26:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81c8075650 0x7f81c819c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c8074d00 0x7f81c819baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c8074d00 0x7f81c819baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37648/0 (socket says 172.21.15.39:37648) 2024-07-04T08:26:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7400640 1 -- 172.21.15.39:0/2759825848 learned_addr learned my addr 172.21.15.39:0/2759825848 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:03.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7e00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c810f460 0x7f81c81a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7400640 1 -- 172.21.15.39:0/2759825848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c810f460 msgr2=0x7f81c81a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:03.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7400640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c810f460 0x7f81c81a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7400640 1 -- 172.21.15.39:0/2759825848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81c8075650 msgr2=0x7f81c819c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7400640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81c8075650 0x7f81c819c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7400640 1 -- 172.21.15.39:0/2759825848 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c81a1a10 con 0x7f81c8074d00 2024-07-04T08:26:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.952+0000 7f81c7400640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c8074d00 0x7f81c819baa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f81b8005940 tx=0x7f81b8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.953+0000 7f81bfe00640 1 -- 172.21.15.39:0/2759825848 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b803f030 con 0x7f81c8074d00 2024-07-04T08:26:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.953+0000 7f81cca00640 1 -- 172.21.15.39:0/2759825848 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c81a1ca0 con 0x7f81c8074d00 2024-07-04T08:26:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.953+0000 7f81bfe00640 1 -- 172.21.15.39:0/2759825848 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b8003aa0 con 0x7f81c8074d00 2024-07-04T08:26:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.953+0000 7f81bfe00640 1 -- 172.21.15.39:0/2759825848 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b8035070 con 0x7f81c8074d00 2024-07-04T08:26:03.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.953+0000 7f81cca00640 1 -- 172.21.15.39:0/2759825848 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c81a8460 con 0x7f81c8074d00 2024-07-04T08:26:03.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.956+0000 7f81d4563740 1 Processor -- start 2024-07-04T08:26:03.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.957+0000 7f81d4563740 1 -- start start 2024-07-04T08:26:03.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.957+0000 7f81bfe00640 1 -- 172.21.15.39:0/2759825848 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f81b8042030 con 0x7f81c8074d00 2024-07-04T08:26:03.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.957+0000 7f81d4563740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148045070 0x55b1483e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.957+0000 7f81d4563740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b14801c1b0 0x55b1483e9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.957+0000 7f81d4563740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.957+0000 7f81d4563740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b14835fee0 con 0x55b148045070 2024-07-04T08:26:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.957+0000 7f81d4563740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b14834f480 con 0x55b148024390 2024-07-04T08:26:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.957+0000 7f81d4563740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b1483534c0 con 0x55b14801c1b0 2024-07-04T08:26:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148045070 0x55b1483e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483ea3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37660/0 (socket says 172.21.15.39:37660) 2024-07-04T08:26:03.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c6a00640 1 -- 172.21.15.39:0/1000261998 learned_addr learned my addr 172.21.15.39:0/1000261998 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148045070 0x55b1483e9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49832/0 (socket says 172.21.15.39:49832) 2024-07-04T08:26:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c7400640 1 --2- 172.21.15.39:0/1000261998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b14801c1b0 0x55b1483e9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c6a00640 1 -- 172.21.15.39:0/1000261998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b14801c1b0 msgr2=0x55b1483e9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c6a00640 1 --2- 172.21.15.39:0/1000261998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b14801c1b0 0x55b1483e9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c6a00640 1 -- 172.21.15.39:0/1000261998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148045070 msgr2=0x55b1483e9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c6a00640 1 --2- 172.21.15.39:0/1000261998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148045070 0x55b1483e9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c6a00640 1 -- 172.21.15.39:0/1000261998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1482e7650 con 0x55b148024390 2024-07-04T08:26:03.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.958+0000 7f81c7e00640 1 --2- 172.21.15.39:0/1000261998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148045070 0x55b1483e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.959+0000 7f81c6a00640 1 --2- 172.21.15.39:0/1000261998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483ea3d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009c60 tx=0x7f81b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72bc79c51cccbfe1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.959+0000 7f81bcc00640 1 -- 172.21.15.39:0/1000261998 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0004220 con 0x55b148024390 2024-07-04T08:26:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.959+0000 7f81bcc00640 1 -- 172.21.15.39:0/1000261998 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b00043c0 con 0x55b148024390 2024-07-04T08:26:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.959+0000 7f81bcc00640 1 -- 172.21.15.39:0/1000261998 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0019810 con 0x55b148024390 2024-07-04T08:26:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81bfe00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81ac0b2960 0x7f81ac0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81bfe00640 1 -- 172.21.15.39:0/2759825848 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f81b8102d50 con 0x7f81c8074d00 2024-07-04T08:26:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81d4563740 1 -- 172.21.15.39:0/1000261998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 msgr2=0x55b1483ea3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81d4563740 1 --2- 172.21.15.39:0/1000261998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483ea3d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009c60 tx=0x7f81b0011810 comp rx=0 tx=0).stop 2024-07-04T08:26:03.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81c7e00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81ac0b2960 0x7f81ac0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81d4563740 1 -- 172.21.15.39:0/1000261998 shutdown_connections 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81d4563740 1 --2- 172.21.15.39:0/1000261998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483ea3d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81d4563740 1 --2- 172.21.15.39:0/1000261998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b14801c1b0 0x55b1483e9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81d4563740 1 --2- 172.21.15.39:0/1000261998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148045070 0x55b1483e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81d4563740 1 -- 172.21.15.39:0/1000261998 >> 172.21.15.39:0/1000261998 conn(0x55b148039460 msgr2=0x55b1483e7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81d4563740 1 -- 172.21.15.39:0/1000261998 shutdown_connections 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.960+0000 7f81d4563740 1 -- 172.21.15.39:0/1000261998 wait complete. 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.961+0000 7f81c7e00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81ac0b2960 0x7f81ac0b4e70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009cf0 tx=0x7f81c0009600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.961+0000 7f81d4563740 1 -- start start 2024-07-04T08:26:03.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.961+0000 7f81d4563740 1 Processor -- start 2024-07-04T08:26:03.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81d4563740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b14801c1b0 0x55b1483f5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81d4563740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148024390 0x55b1483f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81d4563740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b148401420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81d4563740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b1481b3700 con 0x55b148024390 2024-07-04T08:26:03.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81d4563740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b1482f6bb0 con 0x55b14801c1b0 2024-07-04T08:26:03.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81d4563740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b1483f8130 con 0x55b148045070 2024-07-04T08:26:03.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81d4563740 10 client.0 _ll_register_callbacks cb 0x55b148348570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:03.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81c6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148024390 0x55b1483f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81c7400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b14801c1b0 0x55b1483f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81c6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148024390 0x55b1483f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49840/0 (socket says 172.21.15.39:49840) 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81c6a00640 1 -- 172.21.15.39:0/1941530120 learned_addr learned my addr 172.21.15.39:0/1941530120 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.962+0000 7f81c7e00640 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b148401420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.963+0000 7f81c7e00640 1 -- 172.21.15.39:0/1941530120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b14801c1b0 msgr2=0x55b1483f5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.963+0000 7f81c7e00640 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b14801c1b0 0x55b1483f5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.963+0000 7f81c7e00640 1 -- 172.21.15.39:0/1941530120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148024390 msgr2=0x55b1483f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.963+0000 7f81c7e00640 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148024390 0x55b1483f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.963+0000 7f81c7e00640 1 -- 172.21.15.39:0/1941530120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1480b0650 con 0x55b148045070 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.963+0000 7f81c6a00640 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148024390 0x55b1483f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.963+0000 7f81c7e00640 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b148401420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f81c0005fd0 tx=0x7f81c0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff91861ad7b46d17 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.963+0000 7f81b5600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:03.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.964+0000 7f81b5600640 1 -- 172.21.15.39:0/1941530120 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0020620 con 0x55b148045070 2024-07-04T08:26:03.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.964+0000 7f81d4563740 1 -- 172.21.15.39:0/1941530120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81b00043c0 con 0x55b148045070 2024-07-04T08:26:03.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.964+0000 7f81b5600640 1 -- 172.21.15.39:0/1941530120 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f81c00207c0 con 0x55b148045070 2024-07-04T08:26:03.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.964+0000 7f81b5600640 1 -- 172.21.15.39:0/1941530120 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81c0033050 con 0x55b148045070 2024-07-04T08:26:03.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.964+0000 7f81d4563740 10 client.27814 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:03.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.964+0000 7f81d4563740 1 -- 172.21.15.39:0/1941530120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b1484040f0 con 0x55b148045070 2024-07-04T08:26:03.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.964+0000 7f81b5600640 1 -- 172.21.15.39:0/1941530120 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81b00043c0 con 0x55b148045070 2024-07-04T08:26:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.965+0000 7f81b5600640 1 -- 172.21.15.39:0/1941530120 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f81c000f040 con 0x55b148045070 2024-07-04T08:26:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.965+0000 7f81d4563740 10 client.27814 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.965+0000 7f81d4563740 10 client.27814 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:26:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.965+0000 7f81d4563740 20 client.27814 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.965+0000 7f81d4563740 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b1480039f0 0x55b1484076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.965+0000 7f81d4563740 4 client.27814 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:03.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.965+0000 7f81d4563740 1 -- 172.21.15.39:0/1941530120 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b148361e30 con 0x55b1480039f0 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.965+0000 7f81c7400640 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b1480039f0 0x55b1484076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.966+0000 7f81c7400640 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b1480039f0 0x55b1484076d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f393ddd5135a2cd server_cookie=b3d00ab8efa28a1c in_seq=0 out_seq=0 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.966+0000 7f81b5600640 10 client.27814 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81b5600640 1 -- 172.21.15.39:0/1941530120 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b148361e30 con 0x55b1480039f0 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81b5600640 10 client.27814 handle_command_reply: tid=0 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81d4563740 1 client.27814 shutdown 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81d4563740 1 -- 172.21.15.39:0/1941530120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 msgr2=0x55b148401420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81d4563740 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b148401420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f81c0005fd0 tx=0x7f81c0004a40 comp rx=0 tx=0).stop 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81d4563740 1 -- 172.21.15.39:0/1941530120 shutdown_connections 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81d4563740 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b1480039f0 0x55b1484076d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81d4563740 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b148024390 0x55b1483f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81d4563740 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b148401420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81d4563740 1 --2- 172.21.15.39:0/1941530120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b14801c1b0 0x55b1483f5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81b5600640 0 client.27814 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:26:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.967+0000 7f81d4563740 1 -- 172.21.15.39:0/1941530120 >> 172.21.15.39:0/1941530120 conn(0x55b148039460 msgr2=0x55b1483ecb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.968+0000 7f81d4563740 1 -- 172.21.15.39:0/1941530120 shutdown_connections 2024-07-04T08:26:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.968+0000 7f81d4563740 1 -- 172.21.15.39:0/1941530120 wait complete. 2024-07-04T08:26:03.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.968+0000 7f81d4563740 20 client.27814 trim_cache size 0 max 16384 2024-07-04T08:26:03.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.984+0000 7f81d4563740 1 Processor -- start 2024-07-04T08:26:03.974 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.984+0000 7f81d4563740 1 -- start start 2024-07-04T08:26:03.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.984+0000 7f81d4563740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 0x55b1483ed520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.984+0000 7f81d4563740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.984+0000 7f81d4563740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b1483eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.984+0000 7f81d4563740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b23f0 con 0x55b14801c1b0 2024-07-04T08:26:03.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.984+0000 7f81d4563740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b3f30 con 0x55b148024390 2024-07-04T08:26:03.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.984+0000 7f81d4563740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b2ac0 con 0x55b148045070 2024-07-04T08:26:03.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 0x55b1483ed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b1483eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b1483eb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50452/0 (socket says 172.21.15.39:50452) 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 0x55b1483ed520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49852/0 (socket says 172.21.15.39:49852) 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7e00640 1 -- 172.21.15.39:0/3273791127 learned_addr learned my addr 172.21.15.39:0/3273791127 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c6a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483f7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37674/0 (socket says 172.21.15.39:37674) 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7e00640 1 -- 172.21.15.39:0/3273791127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 msgr2=0x55b1483f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7e00640 1 --2- 172.21.15.39:0/3273791127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7e00640 1 -- 172.21.15.39:0/3273791127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 msgr2=0x55b1483ed520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7e00640 1 --2- 172.21.15.39:0/3273791127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 0x55b1483ed520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.985+0000 7f81c7e00640 1 -- 172.21.15.39:0/3273791127 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1483f5d10 con 0x55b148045070 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.986+0000 7f81c7e00640 1 --2- 172.21.15.39:0/3273791127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b1483eb040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f81c001da20 tx=0x7f81c0005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80a0fb5c8f2b4db9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.986+0000 7f81b7e00640 1 -- 172.21.15.39:0/3273791127 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c001de10 con 0x55b148045070 2024-07-04T08:26:03.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.986+0000 7f81b7e00640 1 -- 172.21.15.39:0/3273791127 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c001fe80 con 0x55b148045070 2024-07-04T08:26:03.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.986+0000 7f81b7e00640 1 -- 172.21.15.39:0/3273791127 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0005450 con 0x55b148045070 2024-07-04T08:26:03.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.987+0000 7f81d4563740 1 -- 172.21.15.39:0/3273791127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 msgr2=0x55b1483eb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:03.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.987+0000 7f81d4563740 1 --2- 172.21.15.39:0/3273791127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b1483eb040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f81c001da20 tx=0x7f81c0005cf0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.987+0000 7f81d4563740 1 -- 172.21.15.39:0/3273791127 shutdown_connections 2024-07-04T08:26:03.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.987+0000 7f81d4563740 1 --2- 172.21.15.39:0/3273791127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148045070 0x55b1483eb040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.987+0000 7f81d4563740 1 --2- 172.21.15.39:0/3273791127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148024390 0x55b1483f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.987+0000 7f81d4563740 1 --2- 172.21.15.39:0/3273791127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 0x55b1483ed520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.987+0000 7f81d4563740 1 -- 172.21.15.39:0/3273791127 >> 172.21.15.39:0/3273791127 conn(0x55b148039460 msgr2=0x55b1483ea930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.987+0000 7f81d4563740 1 -- 172.21.15.39:0/3273791127 shutdown_connections 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.987+0000 7f81d4563740 1 -- 172.21.15.39:0/3273791127 wait complete. 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.988+0000 7f81d4563740 1 -- start start 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.988+0000 7f81d4563740 1 Processor -- start 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81d4563740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 0x55b1483fd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81d4563740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148024390 0x55b1483f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81d4563740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148045070 0x55b1483f9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81d4563740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b3860 con 0x55b14801c1b0 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81d4563740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b3190 con 0x55b148045070 2024-07-04T08:26:03.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81d4563740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81b03d0 con 0x55b148024390 2024-07-04T08:26:03.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81d4563740 10 client.0 _ll_register_callbacks cb 0x7f81c0009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:03.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148045070 0x55b1483f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148045070 0x55b1483f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37686/0 (socket says 172.21.15.39:37686) 2024-07-04T08:26:03.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c7e00640 1 -- 172.21.15.39:0/3731470664 learned_addr learned my addr 172.21.15.39:0/3731470664 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:03.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c6a00640 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148024390 0x55b1483f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c7400640 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 0x55b1483fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c7e00640 1 -- 172.21.15.39:0/3731470664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148024390 msgr2=0x55b1483f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:03.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c7e00640 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148024390 0x55b1483f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c7e00640 1 -- 172.21.15.39:0/3731470664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 msgr2=0x55b1483fd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c7e00640 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 0x55b1483fd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.989+0000 7f81c7e00640 1 -- 172.21.15.39:0/3731470664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1483f9640 con 0x55b148045070 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.990+0000 7f81c7e00640 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148045070 0x55b1483f9100 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009e90 tx=0x7f81c001de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=143e3d633367ffae server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.990+0000 7f81b5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.990+0000 7f81b5600640 1 -- 172.21.15.39:0/3731470664 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c001d6e0 con 0x55b148045070 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.990+0000 7f81d4563740 1 -- 172.21.15.39:0/3731470664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b14835d8d0 con 0x55b148045070 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.990+0000 7f81b5600640 1 -- 172.21.15.39:0/3731470664 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f81c0004a00 con 0x55b148045070 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81b5600640 1 -- 172.21.15.39:0/3731470664 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81c004d050 con 0x55b148045070 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81d4563740 10 client.27887 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81d4563740 1 -- 172.21.15.39:0/3731470664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b148478270 con 0x55b148045070 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81b5600640 1 -- 172.21.15.39:0/3731470664 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b14835d8d0 con 0x55b148045070 2024-07-04T08:26:03.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81b5600640 1 -- 172.21.15.39:0/3731470664 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f81c0048030 con 0x55b148045070 2024-07-04T08:26:03.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81d4563740 10 client.27887 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:03.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81d4563740 10 client.27887 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:26:03.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81d4563740 20 client.27887 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:03.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81d4563740 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b147fd8da0 0x55b14847a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:03.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81d4563740 4 client.27887 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:03.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.991+0000 7f81d4563740 1 -- 172.21.15.39:0/3731470664 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b14835b010 con 0x55b147fd8da0 2024-07-04T08:26:03.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.992+0000 7f81c7400640 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b147fd8da0 0x55b14847a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:03.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.992+0000 7f81c7400640 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b147fd8da0 0x55b14847a940 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd7f488f28efdbac server_cookie=78924835b53e03f in_seq=0 out_seq=0 2024-07-04T08:26:03.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.992+0000 7f81b5600640 10 client.27887 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:26:03.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.992+0000 7f81b5600640 1 -- 172.21.15.39:0/3731470664 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b14835b010 con 0x55b147fd8da0 2024-07-04T08:26:03.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.992+0000 7f81b5600640 10 client.27887 handle_command_reply: tid=0 2024-07-04T08:26:03.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.992+0000 7f81d4563740 1 client.27887 shutdown 2024-07-04T08:26:03.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 -- 172.21.15.39:0/3731470664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148045070 msgr2=0x55b1483f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:03.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148045070 0x55b1483f9100 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009e90 tx=0x7f81c001de40 comp rx=0 tx=0).stop 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 -- 172.21.15.39:0/3731470664 shutdown_connections 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55b147fd8da0 0x55b14847a940 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b148045070 0x55b1483f9100 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b148024390 0x55b1483f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 --2- 172.21.15.39:0/3731470664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b14801c1b0 0x55b1483fd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81b5600640 0 client.27887 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 -- 172.21.15.39:0/3731470664 >> 172.21.15.39:0/3731470664 conn(0x55b148039460 msgr2=0x55b1483f6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 -- 172.21.15.39:0/3731470664 shutdown_connections 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 1 -- 172.21.15.39:0/3731470664 wait complete. 2024-07-04T08:26:03.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.993+0000 7f81d4563740 20 client.27887 trim_cache size 0 max 16384 2024-07-04T08:26:03.987 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:03.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 -- 172.21.15.39:0/2759825848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81ac0b2960 msgr2=0x7f81ac0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:03.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81ac0b2960 0x7f81ac0b4e70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009cf0 tx=0x7f81c0009600 comp rx=0 tx=0).stop 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 -- 172.21.15.39:0/2759825848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c8074d00 msgr2=0x7f81c819baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c8074d00 0x7f81c819baa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f81b8005940 tx=0x7f81b8004270 comp rx=0 tx=0).stop 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 -- 172.21.15.39:0/2759825848 shutdown_connections 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81ac0b2960 0x7f81ac0b4e70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81c8075650 0x7f81c819c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81c810f460 0x7f81c81a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 --2- 172.21.15.39:0/2759825848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c8074d00 0x7f81c819baa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 -- 172.21.15.39:0/2759825848 >> 172.21.15.39:0/2759825848 conn(0x7f81c80fe990 msgr2=0x7f81c8109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 -- 172.21.15.39:0/2759825848 shutdown_connections 2024-07-04T08:26:03.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:03.994+0000 7f81cca00640 1 -- 172.21.15.39:0/2759825848 wait complete. 2024-07-04T08:26:04.910 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:26:05.400 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:05.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:05 smithi039 ceph-mon[32062]: pgmap v814: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-07-04T08:26:05.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:05 smithi133 ceph-mon[37050]: pgmap v814: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-07-04T08:26:05.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:05 smithi050 ceph-mon[36577]: pgmap v814: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-07-04T08:26:07.473 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:07 smithi039 ceph-mon[32062]: pgmap v815: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 65 KiB/s wr, 29 op/s 2024-07-04T08:26:07.473 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:07 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:26:07.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:07 smithi133 ceph-mon[37050]: pgmap v815: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 65 KiB/s wr, 29 op/s 2024-07-04T08:26:07.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:07 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:26:07.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:07 smithi050 ceph-mon[36577]: pgmap v815: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 65 KiB/s wr, 29 op/s 2024-07-04T08:26:07.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:07 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:26:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.705+0000 7f8e76400640 1 -- 172.21.15.39:0/3792181316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e70104660 msgr2=0x7f8e7010cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.705+0000 7f8e76400640 1 --2- 172.21.15.39:0/3792181316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e70104660 0x7f8e7010cbb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009900 tx=0x7f8e5802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.705+0000 7f8e76400640 1 -- 172.21.15.39:0/3792181316 shutdown_connections 2024-07-04T08:26:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.705+0000 7f8e76400640 1 --2- 172.21.15.39:0/3792181316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e70104660 0x7f8e7010cbb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.705+0000 7f8e76400640 1 --2- 172.21.15.39:0/3792181316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e7010d170 0x7f8e7010f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.705+0000 7f8e76400640 1 --2- 172.21.15.39:0/3792181316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70103d10 0x7f8e70104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.705+0000 7f8e76400640 1 -- 172.21.15.39:0/3792181316 >> 172.21.15.39:0/3792181316 conn(0x7f8e700fd810 msgr2=0x7f8e700ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.706+0000 7f8e76400640 1 -- 172.21.15.39:0/3792181316 shutdown_connections 2024-07-04T08:26:07.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.706+0000 7f8e76400640 1 -- 172.21.15.39:0/3792181316 wait complete. 2024-07-04T08:26:07.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.707+0000 7f8e76400640 1 Processor -- start 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.707+0000 7f8e76400640 1 -- start start 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.707+0000 7f8e76400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e70103d10 0x7f8e7019ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.707+0000 7f8e76400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70104660 0x7f8e7019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.707+0000 7f8e76400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7010d170 0x7f8e701a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.707+0000 7f8e76400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70112d90 con 0x7f8e7010d170 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.707+0000 7f8e76400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70112c10 con 0x7f8e70103d10 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.707+0000 7f8e76400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e70112f10 con 0x7f8e70104660 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e75a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7010d170 0x7f8e701a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e75000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e70103d10 0x7f8e7019ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e75a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7010d170 0x7f8e701a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49886/0 (socket says 172.21.15.39:49886) 2024-07-04T08:26:07.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e6fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70104660 0x7f8e7019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e75a00640 1 -- 172.21.15.39:0/3932792358 learned_addr learned my addr 172.21.15.39:0/3932792358 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e75000640 1 -- 172.21.15.39:0/3932792358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70104660 msgr2=0x7f8e7019c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e75000640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70104660 0x7f8e7019c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e75000640 1 -- 172.21.15.39:0/3932792358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7010d170 msgr2=0x7f8e701a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e75000640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7010d170 0x7f8e701a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.708+0000 7f8e75000640 1 -- 172.21.15.39:0/3932792358 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e701a19f0 con 0x7f8e70103d10 2024-07-04T08:26:07.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.709+0000 7f8e75000640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e70103d10 0x7f8e7019ba80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000eb00 tx=0x7f8e6000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:07.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.709+0000 7f8e6d600640 1 -- 172.21.15.39:0/3932792358 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60010430 con 0x7f8e70103d10 2024-07-04T08:26:07.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.709+0000 7f8e76400640 1 -- 172.21.15.39:0/3932792358 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e701a1ce0 con 0x7f8e70103d10 2024-07-04T08:26:07.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.709+0000 7f8e6d600640 1 -- 172.21.15.39:0/3932792358 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e60010a80 con 0x7f8e70103d10 2024-07-04T08:26:07.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.709+0000 7f8e6d600640 1 -- 172.21.15.39:0/3932792358 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60017a40 con 0x7f8e70103d10 2024-07-04T08:26:07.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.709+0000 7f8e76400640 1 -- 172.21.15.39:0/3932792358 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e701a8440 con 0x7f8e70103d10 2024-07-04T08:26:07.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.713+0000 7f8e7e0e2740 1 Processor -- start 2024-07-04T08:26:07.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.713+0000 7f8e7e0e2740 1 -- start start 2024-07-04T08:26:07.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.713+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.713+0000 7f8e6d600640 1 -- 172.21.15.39:0/3932792358 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8e60017ce0 con 0x7f8e70103d10 2024-07-04T08:26:07.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.713+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 0x5622975d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.713+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975d53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.713+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56229754af00 con 0x56229720f390 2024-07-04T08:26:07.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.713+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56229753a4a0 con 0x562297230070 2024-07-04T08:26:07.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.713+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56229753e4e0 con 0x5622972071b0 2024-07-04T08:26:07.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975d53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e75000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 0x5622975d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975d53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49890/0 (socket says 172.21.15.39:49890) 2024-07-04T08:26:07.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e6fe00640 1 -- 172.21.15.39:0/2576196560 learned_addr learned my addr 172.21.15.39:0/2576196560 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:07.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e75a00640 1 --2- 172.21.15.39:0/2576196560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e6fe00640 1 -- 172.21.15.39:0/2576196560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 msgr2=0x5622975d4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:07.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/2576196560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 0x5622975d4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e6fe00640 1 -- 172.21.15.39:0/2576196560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 msgr2=0x5622975d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:07.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/2576196560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.714+0000 7f8e6fe00640 1 -- 172.21.15.39:0/2576196560 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622974d2670 con 0x56229720f390 2024-07-04T08:26:07.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.715+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/2576196560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975d53f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009900 tx=0x7f8e58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9fe7ab8fc90b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:07.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.715+0000 7f8e66000640 1 -- 172.21.15.39:0/2576196560 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58040070 con 0x56229720f390 2024-07-04T08:26:07.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.715+0000 7f8e66000640 1 -- 172.21.15.39:0/2576196560 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e58004030 con 0x56229720f390 2024-07-04T08:26:07.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.715+0000 7f8e66000640 1 -- 172.21.15.39:0/2576196560 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58035040 con 0x56229720f390 2024-07-04T08:26:07.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e6d600640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8e540b2960 0x7f8e540b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2576196560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 msgr2=0x5622975d53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:07.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2576196560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975d53f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009900 tx=0x7f8e58000bc0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e6d600640 1 -- 172.21.15.39:0/3932792358 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f8e600e3430 con 0x7f8e70103d10 2024-07-04T08:26:07.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e75a00640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8e540b2960 0x7f8e540b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2576196560 shutdown_connections 2024-07-04T08:26:07.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2576196560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975d53f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2576196560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 0x5622975d4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2576196560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.716+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2576196560 >> 172.21.15.39:0/2576196560 conn(0x562297224460 msgr2=0x5622975d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:07.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.717+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2576196560 shutdown_connections 2024-07-04T08:26:07.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.717+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2576196560 wait complete. 2024-07-04T08:26:07.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.717+0000 7f8e75a00640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8e540b2960 0x7f8e540b4e70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8e68005bc0 tx=0x7f8e68005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:07.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.717+0000 7f8e7e0e2740 1 -- start start 2024-07-04T08:26:07.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.718+0000 7f8e7e0e2740 1 Processor -- start 2024-07-04T08:26:07.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.718+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 0x5622975e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.718+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.718+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975ec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.718+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56229739e700 con 0x56229720f390 2024-07-04T08:26:07.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.718+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5622974e1bd0 con 0x562297230070 2024-07-04T08:26:07.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.718+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5622975e30f0 con 0x5622972071b0 2024-07-04T08:26:07.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.718+0000 7f8e7e0e2740 10 client.0 _ll_register_callbacks cb 0x562297533590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:07.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975e0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49892/0 (socket says 172.21.15.39:49892) 2024-07-04T08:26:07.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e6fe00640 1 -- 172.21.15.39:0/2529868324 learned_addr learned my addr 172.21.15.39:0/2529868324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:07.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e75000640 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 0x5622975e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e75a00640 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975ec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e75000640 1 -- 172.21.15.39:0/2529868324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 msgr2=0x5622975ec3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e75000640 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975ec3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e75000640 1 -- 172.21.15.39:0/2529868324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 msgr2=0x5622975e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e75000640 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e75000640 1 -- 172.21.15.39:0/2529868324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56229729b650 con 0x5622972071b0 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.719+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.720+0000 7f8e75000640 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 0x5622975e04e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8e600e07c0 tx=0x7f8e600a6210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d401eea045b6fc4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.720+0000 7f8e5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.720+0000 7f8e5ea00640 1 -- 172.21.15.39:0/2529868324 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e600a9ac0 con 0x5622972071b0 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.720+0000 7f8e5ea00640 1 -- 172.21.15.39:0/2529868324 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f8e600e3150 con 0x5622972071b0 2024-07-04T08:26:07.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.720+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2529868324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e58004030 con 0x5622972071b0 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.720+0000 7f8e5ea00640 1 -- 172.21.15.39:0/2529868324 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8e600a9e60 con 0x5622972071b0 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.720+0000 7f8e7e0e2740 10 client.27832 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.720+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2529868324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5622975ef0b0 con 0x5622972071b0 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.720+0000 7f8e5ea00640 1 -- 172.21.15.39:0/2529868324 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e58004030 con 0x5622972071b0 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.721+0000 7f8e5ea00640 1 -- 172.21.15.39:0/2529868324 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8e600e0a40 con 0x5622972071b0 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.721+0000 7f8e7e0e2740 10 client.27832 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.721+0000 7f8e7e0e2740 10 client.27832 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.721+0000 7f8e7e0e2740 20 client.27832 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.721+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5622971ee9f0 0x5622975f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.721+0000 7f8e7e0e2740 4 client.27832 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.721+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2529868324 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56229754ce50 con 0x5622971ee9f0 2024-07-04T08:26:07.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.721+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5622971ee9f0 0x5622975f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.722+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5622971ee9f0 0x5622975f25b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a5715572d1a5ba7 server_cookie=191052b299099a75 in_seq=0 out_seq=0 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.722+0000 7f8e5ea00640 10 client.27832 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e5ea00640 1 -- 172.21.15.39:0/2529868324 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56229754ce50 con 0x5622971ee9f0 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e5ea00640 10 client.27832 handle_command_reply: tid=0 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 client.27832 shutdown 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2529868324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 msgr2=0x5622975e04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 0x5622975e04e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8e600e07c0 tx=0x7f8e600a6210 comp rx=0 tx=0).stop 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2529868324 shutdown_connections 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5622971ee9f0 0x5622975f25b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56229720f390 0x5622975e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975ec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/2529868324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5622972071b0 0x5622975e04e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e5ea00640 0 client.27832 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:26:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2529868324 >> 172.21.15.39:0/2529868324 conn(0x562297224460 msgr2=0x5622975d7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2529868324 shutdown_connections 2024-07-04T08:26:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/2529868324 wait complete. 2024-07-04T08:26:07.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.723+0000 7f8e7e0e2740 20 client.27832 trim_cache size 0 max 16384 2024-07-04T08:26:07.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.739+0000 7f8e7e0e2740 1 Processor -- start 2024-07-04T08:26:07.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.739+0000 7f8e7e0e2740 1 -- start start 2024-07-04T08:26:07.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.739+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56229720f390 0x5622975d6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701b2440 con 0x5622972071b0 2024-07-04T08:26:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701b3f80 con 0x562297230070 2024-07-04T08:26:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701b2b10 con 0x56229720f390 2024-07-04T08:26:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e75000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e6fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56229720f390 0x5622975d6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e6fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56229720f390 0x5622975d6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50518/0 (socket says 172.21.15.39:50518) 2024-07-04T08:26:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e6fe00640 1 -- 172.21.15.39:0/1145460870 learned_addr learned my addr 172.21.15.39:0/1145460870 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:07.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e75a00640 1 --2- 172.21.15.39:0/1145460870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e6fe00640 1 -- 172.21.15.39:0/1145460870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 msgr2=0x5622975e1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:07.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/1145460870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975e1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e6fe00640 1 -- 172.21.15.39:0/1145460870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 msgr2=0x5622975d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:07.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/1145460870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.740+0000 7f8e6fe00640 1 -- 172.21.15.39:0/1145460870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622975d77d0 con 0x56229720f390 2024-07-04T08:26:07.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.741+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/1145460870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56229720f390 0x5622975d6360 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009900 tx=0x7f8e5802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875511782689cb0c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:07.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.741+0000 7f8e65600640 1 -- 172.21.15.39:0/1145460870 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58040040 con 0x56229720f390 2024-07-04T08:26:07.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.741+0000 7f8e65600640 1 -- 172.21.15.39:0/1145460870 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e58004030 con 0x56229720f390 2024-07-04T08:26:07.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.741+0000 7f8e65600640 1 -- 172.21.15.39:0/1145460870 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58035040 con 0x56229720f390 2024-07-04T08:26:07.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.742+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/1145460870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56229720f390 msgr2=0x5622975d6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:07.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.742+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/1145460870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56229720f390 0x5622975d6360 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009900 tx=0x7f8e5802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.742+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/1145460870 shutdown_connections 2024-07-04T08:26:07.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.742+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/1145460870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562297230070 0x5622975e1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.742+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/1145460870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56229720f390 0x5622975d6360 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.742+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/1145460870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.742+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/1145460870 >> 172.21.15.39:0/1145460870 conn(0x562297224460 msgr2=0x5622975e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:07.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.742+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/1145460870 shutdown_connections 2024-07-04T08:26:07.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.742+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/1145460870 wait complete. 2024-07-04T08:26:07.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.743+0000 7f8e7e0e2740 1 -- start start 2024-07-04T08:26:07.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.743+0000 7f8e7e0e2740 1 Processor -- start 2024-07-04T08:26:07.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975e8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56229720f390 0x5622975e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e7e0e2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562297230070 0x5622975e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701b38b0 con 0x5622972071b0 2024-07-04T08:26:07.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701b31e0 con 0x56229720f390 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e7e0e2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e701b0420 con 0x562297230070 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e7e0e2740 10 client.0 _ll_register_callbacks cb 0x7f8e60005a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e75000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e75000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975e8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49900/0 (socket says 172.21.15.39:49900) 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e75000640 1 -- 172.21.15.39:0/3063373681 learned_addr learned my addr 172.21.15.39:0/3063373681 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56229720f390 0x5622975e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.744+0000 7f8e75a00640 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562297230070 0x5622975e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e75000640 1 -- 172.21.15.39:0/3063373681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562297230070 msgr2=0x5622975e3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e75000640 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562297230070 0x5622975e3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e75000640 1 -- 172.21.15.39:0/3063373681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56229720f390 msgr2=0x5622975e8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e75000640 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56229720f390 0x5622975e8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e75000640 1 -- 172.21.15.39:0/3063373681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622975e44c0 con 0x5622972071b0 2024-07-04T08:26:07.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e75000640 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975e8680 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8e600a63d0 tx=0x7f8e600a9dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221af9f5a9b64cc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e5ea00640 1 -- 172.21.15.39:0/3063373681 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e600a9ac0 con 0x5622972071b0 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e5ea00640 1 -- 172.21.15.39:0/3063373681 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9424+0+0 (secure 0 0 0) 0x7f8e600105d0 con 0x5622972071b0 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.745+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/3063373681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e58004030 con 0x5622972071b0 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.746+0000 7f8e5ea00640 1 -- 172.21.15.39:0/3063373681 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8e600aa350 con 0x5622972071b0 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.746+0000 7f8e7e0e2740 10 client.17772 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.746+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/3063373681 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56229765a900 con 0x5622972071b0 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.746+0000 7f8e5ea00640 1 -- 172.21.15.39:0/3063373681 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e58004030 con 0x5622972071b0 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.746+0000 7f8e5ea00640 1 -- 172.21.15.39:0/3063373681 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8e600a9020 con 0x5622972071b0 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.746+0000 7f8e7e0e2740 10 client.17772 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:07.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.746+0000 7f8e7e0e2740 10 client.17772 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:26:07.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.746+0000 7f8e7e0e2740 20 client.17772 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:07.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.747+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5622971c3da0 0x56229765cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:07.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.747+0000 7f8e7e0e2740 4 client.17772 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:07.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.747+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/3063373681 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622975488f0 con 0x5622971c3da0 2024-07-04T08:26:07.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.747+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5622971c3da0 0x56229765cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:07.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e6fe00640 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5622971c3da0 0x56229765cf10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=805d808673ce07f2 server_cookie=874b92ebc80950e5 in_seq=0 out_seq=0 2024-07-04T08:26:07.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e5ea00640 10 client.17772 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:26:07.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e5ea00640 1 -- 172.21.15.39:0/3063373681 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622975488f0 con 0x5622971c3da0 2024-07-04T08:26:07.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e5ea00640 10 client.17772 handle_command_reply: tid=0 2024-07-04T08:26:07.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 client.17772 shutdown 2024-07-04T08:26:07.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/3063373681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 msgr2=0x5622975e8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:07.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975e8680 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8e600a63d0 tx=0x7f8e600a9dd0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/3063373681 shutdown_connections 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5622971c3da0 0x56229765cf10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562297230070 0x5622975e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56229720f390 0x5622975e8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 --2- 172.21.15.39:0/3063373681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5622972071b0 0x5622975e8680 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e5ea00640 0 client.17772 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/3063373681 >> 172.21.15.39:0/3063373681 conn(0x562297224460 msgr2=0x5622975d60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.748+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/3063373681 shutdown_connections 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.749+0000 7f8e7e0e2740 1 -- 172.21.15.39:0/3063373681 wait complete. 2024-07-04T08:26:07.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.749+0000 7f8e7e0e2740 20 client.17772 trim_cache size 0 max 16384 2024-07-04T08:26:07.742 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:07.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.749+0000 7f8e76400640 1 -- 172.21.15.39:0/3932792358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8e540b2960 msgr2=0x7f8e540b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:07.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.749+0000 7f8e76400640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8e540b2960 0x7f8e540b4e70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8e68005bc0 tx=0x7f8e68005f90 comp rx=0 tx=0).stop 2024-07-04T08:26:07.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.749+0000 7f8e76400640 1 -- 172.21.15.39:0/3932792358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e70103d10 msgr2=0x7f8e7019ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.749+0000 7f8e76400640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e70103d10 0x7f8e7019ba80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000eb00 tx=0x7f8e6000c2d0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.750+0000 7f8e76400640 1 -- 172.21.15.39:0/3932792358 shutdown_connections 2024-07-04T08:26:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.750+0000 7f8e76400640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8e540b2960 0x7f8e540b4e70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.750+0000 7f8e76400640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e7010d170 0x7f8e701a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.750+0000 7f8e76400640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e70104660 0x7f8e7019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.750+0000 7f8e76400640 1 --2- 172.21.15.39:0/3932792358 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8e70103d10 0x7f8e7019ba80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.750+0000 7f8e76400640 1 -- 172.21.15.39:0/3932792358 >> 172.21.15.39:0/3932792358 conn(0x7f8e700fd810 msgr2=0x7f8e70101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.750+0000 7f8e76400640 1 -- 172.21.15.39:0/3932792358 shutdown_connections 2024-07-04T08:26:07.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:07.750+0000 7f8e76400640 1 -- 172.21.15.39:0/3932792358 wait complete. 2024-07-04T08:26:08.470 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:26:08.470 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:26:08.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:26:08.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:08 smithi039 ceph-mon[32062]: pgmap v816: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 51 KiB/s wr, 28 op/s 2024-07-04T08:26:08.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:26:08.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:26:08.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:26:08.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:26:08.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:08 smithi133 ceph-mon[37050]: pgmap v816: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 51 KiB/s wr, 28 op/s 2024-07-04T08:26:08.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:26:08.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:26:08.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:26:08.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:08 smithi050 ceph-mon[36577]: pgmap v816: 129 pgs: 129 active+clean; 17 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 51 KiB/s wr, 28 op/s 2024-07-04T08:26:08.996 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:09.403 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:09 smithi039 ceph-mon[32062]: osdmap e101: 12 total, 12 up, 12 in 2024-07-04T08:26:09.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:09 smithi133 ceph-mon[37050]: osdmap e101: 12 total, 12 up, 12 in 2024-07-04T08:26:09.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:09 smithi050 ceph-mon[36577]: osdmap e101: 12 total, 12 up, 12 in 2024-07-04T08:26:10.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:10 smithi133 ceph-mon[37050]: pgmap v818: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-07-04T08:26:10.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:10 smithi039 ceph-mon[32062]: pgmap v818: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-07-04T08:26:10.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:10 smithi050 ceph-mon[36577]: pgmap v818: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-07-04T08:26:11.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.297+0000 7fc4bba00640 1 -- 172.21.15.39:0/101201518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4b4104850 msgr2=0x7fc4b410cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.297+0000 7fc4bba00640 1 --2- 172.21.15.39:0/101201518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4b4104850 0x7fc4b410cda0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009900 tx=0x7fc4a002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.297+0000 7fc4bba00640 1 -- 172.21.15.39:0/101201518 shutdown_connections 2024-07-04T08:26:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.297+0000 7fc4bba00640 1 --2- 172.21.15.39:0/101201518 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4b4104850 0x7fc4b410cda0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.297+0000 7fc4bba00640 1 --2- 172.21.15.39:0/101201518 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4b410d360 0x7fc4b410f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.297+0000 7fc4bba00640 1 --2- 172.21.15.39:0/101201518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b4103f00 0x7fc4b4104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.297+0000 7fc4bba00640 1 -- 172.21.15.39:0/101201518 >> 172.21.15.39:0/101201518 conn(0x7fc4b4078c80 msgr2=0x7fc4b4079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.298+0000 7fc4bba00640 1 -- 172.21.15.39:0/101201518 shutdown_connections 2024-07-04T08:26:11.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.298+0000 7fc4bba00640 1 -- 172.21.15.39:0/101201518 wait complete. 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.299+0000 7fc4bba00640 1 Processor -- start 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.299+0000 7fc4bba00640 1 -- start start 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.299+0000 7fc4bba00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4b4103f00 0x7fc4b419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.299+0000 7fc4bba00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4b4104850 0x7fc4b419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.299+0000 7fc4bba00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b410d360 0x7fc4b41a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.299+0000 7fc4bba00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4112fa0 con 0x7fc4b4104850 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.299+0000 7fc4bba00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4112e20 con 0x7fc4b4103f00 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.299+0000 7fc4bba00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4113120 con 0x7fc4b410d360 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b9000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4b4103f00 0x7fc4b419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b9000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4b4103f00 0x7fc4b419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41566/0 (socket says 172.21.15.39:41566) 2024-07-04T08:26:11.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b9000640 1 -- 172.21.15.39:0/2274106236 learned_addr learned my addr 172.21.15.39:0/2274106236 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4b4104850 0x7fc4b419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b9a00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b410d360 0x7fc4b41a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b9000640 1 -- 172.21.15.39:0/2274106236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b410d360 msgr2=0x7fc4b41a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b9000640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b410d360 0x7fc4b41a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b9000640 1 -- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4b4104850 msgr2=0x7fc4b419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b9000640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4b4104850 0x7fc4b419c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b9000640 1 -- 172.21.15.39:0/2274106236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b41a1b80 con 0x7fc4b4103f00 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.300+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4b4104850 0x7fc4b419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.301+0000 7fc4b9000640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4b4103f00 0x7fc4b419bc10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800e9c0 tx=0x7fc4a800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.301+0000 7fc4b1600640 1 -- 172.21.15.39:0/2274106236 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800ccc0 con 0x7fc4b4103f00 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.301+0000 7fc4b1600640 1 -- 172.21.15.39:0/2274106236 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a80035f0 con 0x7fc4b4103f00 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.301+0000 7fc4bba00640 1 -- 172.21.15.39:0/2274106236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b41a1e70 con 0x7fc4b4103f00 2024-07-04T08:26:11.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.301+0000 7fc4b1600640 1 -- 172.21.15.39:0/2274106236 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800ce60 con 0x7fc4b4103f00 2024-07-04T08:26:11.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.301+0000 7fc4bba00640 1 -- 172.21.15.39:0/2274106236 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b41a85d0 con 0x7fc4b4103f00 2024-07-04T08:26:11.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.305+0000 7fc4c209b740 1 Processor -- start 2024-07-04T08:26:11.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.306+0000 7fc4c209b740 1 -- start start 2024-07-04T08:26:11.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.306+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd1962e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.306+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1962ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.306+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1962f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.306+0000 7fc4c209b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cd195a4ee0 con 0x55cd192611b0 2024-07-04T08:26:11.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.306+0000 7fc4c209b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cd19594480 con 0x55cd1928a070 2024-07-04T08:26:11.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.306+0000 7fc4c209b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cd195984c0 con 0x55cd19269390 2024-07-04T08:26:11.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1962f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b9a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd1962e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1962f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58578/0 (socket says 172.21.15.39:58578) 2024-07-04T08:26:11.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3160885670 learned_addr learned my addr 172.21.15.39:0/3160885670 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:11.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b1600640 1 -- 172.21.15.39:0/2274106236 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc4a8004000 con 0x7fc4b4103f00 2024-07-04T08:26:11.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3160885670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 msgr2=0x55cd1962e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:11.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/3160885670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd1962e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3160885670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 msgr2=0x55cd1962ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:11.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/3160885670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1962ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3160885670 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd1952c650 con 0x55cd19269390 2024-07-04T08:26:11.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.307+0000 7fc4b9a00640 1 --2- 172.21.15.39:0/3160885670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd1962e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:11.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.308+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/3160885670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1962f230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0031440 tx=0x7fc4a0032c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4787d9c531f31a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:11.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.308+0000 7fc4a6000640 1 -- 172.21.15.39:0/3160885670 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a003f070 con 0x55cd19269390 2024-07-04T08:26:11.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.308+0000 7fc4a6000640 1 -- 172.21.15.39:0/3160885670 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a00316a0 con 0x55cd19269390 2024-07-04T08:26:11.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.308+0000 7fc4a6000640 1 -- 172.21.15.39:0/3160885670 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0035070 con 0x55cd19269390 2024-07-04T08:26:11.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4b1600640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc49c0b2960 0x7fc49c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4b1600640 1 -- 172.21.15.39:0/2274106236 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9512+0+0 (secure 0 0 0) 0x7fc4a80e2ff0 con 0x7fc4b4103f00 2024-07-04T08:26:11.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4c209b740 1 -- 172.21.15.39:0/3160885670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 msgr2=0x55cd1962f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:11.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3160885670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1962f230 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0031440 tx=0x7fc4a0032c50 comp rx=0 tx=0).stop 2024-07-04T08:26:11.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4b9a00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc49c0b2960 0x7fc49c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4c209b740 1 -- 172.21.15.39:0/3160885670 shutdown_connections 2024-07-04T08:26:11.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3160885670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1962ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3160885670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1962f230 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3160885670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd1962e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.309+0000 7fc4c209b740 1 -- 172.21.15.39:0/3160885670 >> 172.21.15.39:0/3160885670 conn(0x55cd1927e460 msgr2=0x55cd1962c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:11.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.310+0000 7fc4c209b740 1 -- 172.21.15.39:0/3160885670 shutdown_connections 2024-07-04T08:26:11.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.310+0000 7fc4c209b740 1 -- 172.21.15.39:0/3160885670 wait complete. 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.310+0000 7fc4b9a00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc49c0b2960 0x7fc49c0b4e70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009d50 tx=0x7fc4ac009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.310+0000 7fc4c209b740 1 -- start start 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.311+0000 7fc4c209b740 1 Processor -- start 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.311+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1963a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.311+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd19269390 0x55cd1963a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.311+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 0x55cd19646220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.311+0000 7fc4c209b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55cd193f8700 con 0x55cd192611b0 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.311+0000 7fc4c209b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55cd1953bbb0 con 0x55cd19269390 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.311+0000 7fc4c209b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cd1963cf30 con 0x55cd1928a070 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.311+0000 7fc4c209b740 10 client.0 _ll_register_callbacks cb 0x55cd1958d570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:11.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1963a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd19269390 0x55cd1963a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1963a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54632/0 (socket says 172.21.15.39:54632) 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b9000640 1 -- 172.21.15.39:0/3941056642 learned_addr learned my addr 172.21.15.39:0/3941056642 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b9a00640 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 0x55cd19646220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3941056642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 msgr2=0x55cd19646220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 0x55cd19646220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3941056642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 msgr2=0x55cd1963a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1963a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3941056642 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd192f5650 con 0x55cd19269390 2024-07-04T08:26:11.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd19269390 0x55cd1963a900 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc4a002a430 tx=0x7fc4a00314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6a42c894e430b60 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:11.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.312+0000 7fc49aa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:11.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.313+0000 7fc49aa00640 1 -- 172.21.15.39:0/3941056642 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a003f070 con 0x55cd19269390 2024-07-04T08:26:11.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.313+0000 7fc4c209b740 1 -- 172.21.15.39:0/3941056642 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a00316a0 con 0x55cd19269390 2024-07-04T08:26:11.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.313+0000 7fc49aa00640 1 -- 172.21.15.39:0/3941056642 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9512+0+0 (secure 0 0 0) 0x7fc4a0004440 con 0x55cd19269390 2024-07-04T08:26:11.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.313+0000 7fc49aa00640 1 -- 172.21.15.39:0/3941056642 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4a0054050 con 0x55cd19269390 2024-07-04T08:26:11.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.313+0000 7fc4c209b740 10 client.27917 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:11.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.313+0000 7fc4c209b740 1 -- 172.21.15.39:0/3941056642 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd19648f40 con 0x55cd19269390 2024-07-04T08:26:11.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.314+0000 7fc49aa00640 1 -- 172.21.15.39:0/3941056642 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a00316a0 con 0x55cd19269390 2024-07-04T08:26:11.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.314+0000 7fc49aa00640 1 -- 172.21.15.39:0/3941056642 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc4a00047e0 con 0x55cd19269390 2024-07-04T08:26:11.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.314+0000 7fc4c209b740 10 client.27917 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:11.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.314+0000 7fc4c209b740 10 client.27917 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:26:11.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.314+0000 7fc4c209b740 20 client.27917 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:11.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.314+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55cd192489f0 0x55cd1964c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.314+0000 7fc4c209b740 4 client.27917 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:11.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.314+0000 7fc4c209b740 1 -- 172.21.15.39:0/3941056642 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd195a6e30 con 0x55cd192489f0 2024-07-04T08:26:11.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.315+0000 7fc4b9000640 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55cd192489f0 0x55cd1964c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.315+0000 7fc4b9000640 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55cd192489f0 0x55cd1964c2a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c563566ae3f2d53e server_cookie=aa493cd733505702 in_seq=0 out_seq=0 2024-07-04T08:26:11.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.315+0000 7fc49aa00640 10 client.27917 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:26:11.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.316+0000 7fc49aa00640 1 -- 172.21.15.39:0/3941056642 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cd195a6e30 con 0x55cd192489f0 2024-07-04T08:26:11.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.316+0000 7fc49aa00640 10 client.27917 handle_command_reply: tid=0 2024-07-04T08:26:11.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.316+0000 7fc4c209b740 1 client.27917 shutdown 2024-07-04T08:26:11.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.316+0000 7fc4c209b740 1 -- 172.21.15.39:0/3941056642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd19269390 msgr2=0x55cd1963a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:11.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.316+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd19269390 0x55cd1963a900 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc4a002a430 tx=0x7fc4a00314a0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.316+0000 7fc4c209b740 1 -- 172.21.15.39:0/3941056642 shutdown_connections 2024-07-04T08:26:11.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.316+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55cd192489f0 0x55cd1964c2a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.317+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 0x55cd19646220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.317+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd19269390 0x55cd1963a900 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.317+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3941056642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1963a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.317+0000 7fc4c209b740 1 -- 172.21.15.39:0/3941056642 >> 172.21.15.39:0/3941056642 conn(0x55cd1927e460 msgr2=0x55cd19631990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:11.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.317+0000 7fc49aa00640 0 client.27917 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:26:11.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.317+0000 7fc4c209b740 1 -- 172.21.15.39:0/3941056642 shutdown_connections 2024-07-04T08:26:11.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.317+0000 7fc4c209b740 1 -- 172.21.15.39:0/3941056642 wait complete. 2024-07-04T08:26:11.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.317+0000 7fc4c209b740 20 client.27917 trim_cache size 0 max 16384 2024-07-04T08:26:11.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.332+0000 7fc4c209b740 1 Processor -- start 2024-07-04T08:26:11.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.332+0000 7fc4c209b740 1 -- start start 2024-07-04T08:26:11.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd192611b0 0x55cd196301a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd19269390 0x55cd1963bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 0x55cd19633890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4c209b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b25c0 con 0x55cd19269390 2024-07-04T08:26:11.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4c209b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b4100 con 0x55cd192611b0 2024-07-04T08:26:11.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4c209b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b2c90 con 0x55cd1928a070 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd19269390 0x55cd1963bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4b9000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd192611b0 0x55cd196301a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4b9000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd192611b0 0x55cd196301a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41600/0 (socket says 172.21.15.39:41600) 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4b9000640 1 -- 172.21.15.39:0/1551434848 learned_addr learned my addr 172.21.15.39:0/1551434848 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4b9a00640 1 --2- 172.21.15.39:0/1551434848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 0x55cd19633890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.333+0000 7fc4b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd19269390 0x55cd1963bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54638/0 (socket says 172.21.15.39:54638) 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.334+0000 7fc4b9000640 1 -- 172.21.15.39:0/1551434848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 msgr2=0x55cd19633890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.334+0000 7fc4b9000640 1 --2- 172.21.15.39:0/1551434848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 0x55cd19633890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.334+0000 7fc4b9000640 1 -- 172.21.15.39:0/1551434848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd19269390 msgr2=0x55cd1963bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.334+0000 7fc4b9000640 1 --2- 172.21.15.39:0/1551434848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd19269390 0x55cd1963bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.334+0000 7fc4b9000640 1 -- 172.21.15.39:0/1551434848 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd196265c0 con 0x55cd192611b0 2024-07-04T08:26:11.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.334+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/1551434848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd19269390 0x55cd1963bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:11.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.334+0000 7fc4b9000640 1 --2- 172.21.15.39:0/1551434848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd192611b0 0x55cd196301a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009810 tx=0x7fc4a80e02d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=710bc70b3443e661 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:11.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.334+0000 7fc4a5600640 1 -- 172.21.15.39:0/1551434848 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80a9e90 con 0x55cd192611b0 2024-07-04T08:26:11.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.335+0000 7fc4a5600640 1 -- 172.21.15.39:0/1551434848 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a80ae240 con 0x55cd192611b0 2024-07-04T08:26:11.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.335+0000 7fc4a5600640 1 -- 172.21.15.39:0/1551434848 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80042f0 con 0x55cd192611b0 2024-07-04T08:26:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.336+0000 7fc4c209b740 1 -- 172.21.15.39:0/1551434848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd192611b0 msgr2=0x55cd196301a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.336+0000 7fc4c209b740 1 --2- 172.21.15.39:0/1551434848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd192611b0 0x55cd196301a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009810 tx=0x7fc4a80e02d0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.336+0000 7fc4c209b740 1 -- 172.21.15.39:0/1551434848 shutdown_connections 2024-07-04T08:26:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.336+0000 7fc4c209b740 1 --2- 172.21.15.39:0/1551434848 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd19269390 0x55cd1963bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.336+0000 7fc4c209b740 1 --2- 172.21.15.39:0/1551434848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd1928a070 0x55cd19633890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.336+0000 7fc4c209b740 1 --2- 172.21.15.39:0/1551434848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd192611b0 0x55cd196301a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.336+0000 7fc4c209b740 1 -- 172.21.15.39:0/1551434848 >> 172.21.15.39:0/1551434848 conn(0x55cd1927e460 msgr2=0x55cd1963b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.336+0000 7fc4c209b740 1 -- 172.21.15.39:0/1551434848 shutdown_connections 2024-07-04T08:26:11.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.336+0000 7fc4c209b740 1 -- 172.21.15.39:0/1551434848 wait complete. 2024-07-04T08:26:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.337+0000 7fc4c209b740 1 -- start start 2024-07-04T08:26:11.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.337+0000 7fc4c209b740 1 Processor -- start 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1964cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1964d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4c209b740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd196b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4c209b740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b3a30 con 0x55cd192611b0 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4c209b740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b3360 con 0x55cd1928a070 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4c209b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41b05c0 con 0x55cd19269390 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4c209b740 10 client.0 _ll_register_callbacks cb 0x7fc4ac009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4b9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1964cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4b3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1964d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4b3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1964d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58590/0 (socket says 172.21.15.39:58590) 2024-07-04T08:26:11.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4b9000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1964cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54650/0 (socket says 172.21.15.39:54650) 2024-07-04T08:26:11.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.338+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3980495320 learned_addr learned my addr 172.21.15.39:0/3980495320 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:11.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.339+0000 7fc4b9a00640 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd196b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.339+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3980495320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 msgr2=0x55cd196b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:11.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.339+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd196b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.339+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3980495320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 msgr2=0x55cd1964cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.339+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1964cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.339+0000 7fc4b3e00640 1 -- 172.21.15.39:0/3980495320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd196b4620 con 0x55cd19269390 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.339+0000 7fc4b9a00640 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd196b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.339+0000 7fc4b3e00640 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1964d1e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0033000 tx=0x7fc4a0031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e6dcf1b1cb961ba server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.339+0000 7fc49aa00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.340+0000 7fc49aa00640 1 -- 172.21.15.39:0/3980495320 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a005c070 con 0x55cd19269390 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.340+0000 7fc4c209b740 1 -- 172.21.15.39:0/3980495320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a80ae240 con 0x55cd19269390 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.340+0000 7fc49aa00640 1 -- 172.21.15.39:0/3980495320 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9512+0+0 (secure 0 0 0) 0x7fc4a00046c0 con 0x55cd19269390 2024-07-04T08:26:11.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.340+0000 7fc49aa00640 1 -- 172.21.15.39:0/3980495320 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4a0067050 con 0x55cd19269390 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.340+0000 7fc4c209b740 10 client.27856 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.340+0000 7fc4c209b740 1 -- 172.21.15.39:0/3980495320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cd196bcac0 con 0x55cd19269390 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.340+0000 7fc49aa00640 1 -- 172.21.15.39:0/3980495320 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a80ae240 con 0x55cd19269390 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.341+0000 7fc49aa00640 1 -- 172.21.15.39:0/3980495320 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc4a0003c10 con 0x55cd19269390 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.341+0000 7fc4c209b740 10 client.27856 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.341+0000 7fc4c209b740 10 client.27856 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.341+0000 7fc4c209b740 20 client.27856 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.341+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55cd1921dda0 0x55cd196bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.341+0000 7fc4c209b740 4 client.27856 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.341+0000 7fc4c209b740 1 -- 172.21.15.39:0/3980495320 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd195a28d0 con 0x55cd1921dda0 2024-07-04T08:26:11.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.341+0000 7fc4b9000640 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55cd1921dda0 0x55cd196bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.342+0000 7fc4b9000640 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55cd1921dda0 0x55cd196bf1f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b880aa822c74dbab server_cookie=2e3001eeeade8ec3 in_seq=0 out_seq=0 2024-07-04T08:26:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.342+0000 7fc49aa00640 10 client.27856 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:26:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.342+0000 7fc49aa00640 1 -- 172.21.15.39:0/3980495320 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd195a28d0 con 0x55cd1921dda0 2024-07-04T08:26:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.342+0000 7fc49aa00640 10 client.27856 handle_command_reply: tid=0 2024-07-04T08:26:11.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.342+0000 7fc4c209b740 1 client.27856 shutdown 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.342+0000 7fc4c209b740 1 -- 172.21.15.39:0/3980495320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 msgr2=0x55cd1964d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.342+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1964d1e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0033000 tx=0x7fc4a0031400 comp rx=0 tx=0).stop 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc4c209b740 1 -- 172.21.15.39:0/3980495320 shutdown_connections 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55cd1921dda0 0x55cd196bf1f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55cd1928a070 0x55cd196b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cd19269390 0x55cd1964d1e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc49aa00640 0 client.27856 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc4c209b740 1 --2- 172.21.15.39:0/3980495320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55cd192611b0 0x55cd1964cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc4c209b740 1 -- 172.21.15.39:0/3980495320 >> 172.21.15.39:0/3980495320 conn(0x55cd1927e460 msgr2=0x55cd1963c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:11.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc4c209b740 1 -- 172.21.15.39:0/3980495320 shutdown_connections 2024-07-04T08:26:11.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc4c209b740 1 -- 172.21.15.39:0/3980495320 wait complete. 2024-07-04T08:26:11.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.343+0000 7fc4c209b740 20 client.27856 trim_cache size 0 max 16384 2024-07-04T08:26:11.337 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 -- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc49c0b2960 msgr2=0x7fc49c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc49c0b2960 0x7fc49c0b4e70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009d50 tx=0x7fc4ac009500 comp rx=0 tx=0).stop 2024-07-04T08:26:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 -- 172.21.15.39:0/2274106236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4b4103f00 msgr2=0x7fc4b419bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4b4103f00 0x7fc4b419bc10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800e9c0 tx=0x7fc4a800edc0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 -- 172.21.15.39:0/2274106236 shutdown_connections 2024-07-04T08:26:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc49c0b2960 0x7fc49c0b4e70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4b4104850 0x7fc4b419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4b410d360 0x7fc4b41a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 --2- 172.21.15.39:0/2274106236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc4b4103f00 0x7fc4b419bc10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 -- 172.21.15.39:0/2274106236 >> 172.21.15.39:0/2274106236 conn(0x7fc4b4078c80 msgr2=0x7fc4b41002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 -- 172.21.15.39:0/2274106236 shutdown_connections 2024-07-04T08:26:11.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:11.344+0000 7fc4bba00640 1 -- 172.21.15.39:0/2274106236 wait complete. 2024-07-04T08:26:12.071 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:26:12.562 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:13.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:13 smithi039 ceph-mon[32062]: pgmap v819: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-07-04T08:26:13.384 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:13 smithi050 ceph-mon[36577]: pgmap v819: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-07-04T08:26:13.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:13 smithi133 ceph-mon[37050]: pgmap v819: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-07-04T08:26:14.372 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:26:14.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.926+0000 7f2616400640 1 -- 172.21.15.39:0/494792808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2610104850 msgr2=0x7f261010cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.926+0000 7f2616400640 1 --2- 172.21.15.39:0/494792808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2610104850 0x7f261010cda0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009900 tx=0x7f25fc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.926+0000 7f2616400640 1 -- 172.21.15.39:0/494792808 shutdown_connections 2024-07-04T08:26:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.926+0000 7f2616400640 1 --2- 172.21.15.39:0/494792808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2610104850 0x7f261010cda0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.927+0000 7f2616400640 1 --2- 172.21.15.39:0/494792808 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f261010d360 0x7f261010f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.927+0000 7f2616400640 1 --2- 172.21.15.39:0/494792808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2610103f00 0x7f2610104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.927+0000 7f2616400640 1 -- 172.21.15.39:0/494792808 >> 172.21.15.39:0/494792808 conn(0x7f2610078c80 msgr2=0x7f2610079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.927+0000 7f2616400640 1 -- 172.21.15.39:0/494792808 shutdown_connections 2024-07-04T08:26:14.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.927+0000 7f2616400640 1 -- 172.21.15.39:0/494792808 wait complete. 2024-07-04T08:26:14.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.928+0000 7f2616400640 1 Processor -- start 2024-07-04T08:26:14.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.928+0000 7f2616400640 1 -- start start 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f2616400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2610103f00 0x7f2610108b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f2616400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2610104850 0x7f2610104e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f2616400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f261010d360 0x7f2610105580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f2616400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2610112f80 con 0x7f2610104850 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f2616400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2610112e00 con 0x7f261010d360 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f2616400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2610113100 con 0x7f2610103f00 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f260ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2610104850 0x7f2610104e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f260ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2610104850 0x7f2610104e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54666/0 (socket says 172.21.15.39:54666) 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f260ea00640 1 -- 172.21.15.39:0/2809289396 learned_addr learned my addr 172.21.15.39:0/2809289396 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f260fe00640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f261010d360 0x7f2610105580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.929+0000 7f260f400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2610103f00 0x7f2610108b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.930+0000 7f260f400640 1 -- 172.21.15.39:0/2809289396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f261010d360 msgr2=0x7f2610105580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:14.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.930+0000 7f260f400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f261010d360 0x7f2610105580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.930+0000 7f260f400640 1 -- 172.21.15.39:0/2809289396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2610104850 msgr2=0x7f2610104e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:14.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.930+0000 7f260f400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2610104850 0x7f2610104e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.930+0000 7f260f400640 1 -- 172.21.15.39:0/2809289396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26101ac1e0 con 0x7f2610103f00 2024-07-04T08:26:14.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.930+0000 7f260f400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2610103f00 0x7f2610108b80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f260000e9c0 tx=0x7f260000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:14.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.930+0000 7f2607e00640 1 -- 172.21.15.39:0/2809289396 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000ccc0 con 0x7f2610103f00 2024-07-04T08:26:14.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.930+0000 7f2607e00640 1 -- 172.21.15.39:0/2809289396 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26000035f0 con 0x7f2610103f00 2024-07-04T08:26:14.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.931+0000 7f2607e00640 1 -- 172.21.15.39:0/2809289396 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000ce60 con 0x7f2610103f00 2024-07-04T08:26:14.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.931+0000 7f2616400640 1 -- 172.21.15.39:0/2809289396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26101ac410 con 0x7f2610103f00 2024-07-04T08:26:14.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.931+0000 7f2616400640 1 -- 172.21.15.39:0/2809289396 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101ac900 con 0x7f2610103f00 2024-07-04T08:26:14.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.935+0000 7f261ca9e740 1 Processor -- start 2024-07-04T08:26:14.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.935+0000 7f261ca9e740 1 -- start start 2024-07-04T08:26:14.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.936+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6968b070 0x55ae69a2f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.936+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 0x55ae69a2fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.936+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6966a390 0x55ae69a30250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.936+0000 7f261ca9e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae699a5f00 con 0x55ae6966a390 2024-07-04T08:26:14.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.936+0000 7f261ca9e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ae699954a0 con 0x55ae696621b0 2024-07-04T08:26:14.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.936+0000 7f261ca9e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ae699994e0 con 0x55ae6968b070 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.936+0000 7f260ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6966a390 0x55ae69a30250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.936+0000 7f2607e00640 1 -- 172.21.15.39:0/2809289396 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2600003a80 con 0x7f2610103f00 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.937+0000 7f260ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6966a390 0x55ae69a30250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54672/0 (socket says 172.21.15.39:54672) 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.937+0000 7f260ea00640 1 -- 172.21.15.39:0/3268331852 learned_addr learned my addr 172.21.15.39:0/3268331852 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.936+0000 7f260fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6968b070 0x55ae69a2f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.937+0000 7f260f400640 1 --2- 172.21.15.39:0/3268331852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 0x55ae69a2fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.937+0000 7f260ea00640 1 -- 172.21.15.39:0/3268331852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6968b070 msgr2=0x55ae69a2f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.937+0000 7f260ea00640 1 --2- 172.21.15.39:0/3268331852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6968b070 0x55ae69a2f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.937+0000 7f260ea00640 1 -- 172.21.15.39:0/3268331852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 msgr2=0x55ae69a2fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.937+0000 7f260ea00640 1 --2- 172.21.15.39:0/3268331852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 0x55ae69a2fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.937+0000 7f260ea00640 1 -- 172.21.15.39:0/3268331852 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6992d670 con 0x55ae6966a390 2024-07-04T08:26:14.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.938+0000 7f260ea00640 1 --2- 172.21.15.39:0/3268331852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6966a390 0x55ae69a30250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009230 tx=0x7f25fc031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c6a0901b94d208 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:14.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.938+0000 7f2604c00640 1 -- 172.21.15.39:0/3268331852 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc040070 con 0x55ae6966a390 2024-07-04T08:26:14.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.938+0000 7f2604c00640 1 -- 172.21.15.39:0/3268331852 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25fc004030 con 0x55ae6966a390 2024-07-04T08:26:14.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.938+0000 7f2604c00640 1 -- 172.21.15.39:0/3268331852 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc035040 con 0x55ae6966a390 2024-07-04T08:26:14.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.939+0000 7f2607e00640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f25f40b2960 0x7f25f40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.939+0000 7f2607e00640 1 -- 172.21.15.39:0/2809289396 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9512+0+0 (secure 0 0 0) 0x7f26000e2e50 con 0x7f2610103f00 2024-07-04T08:26:14.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.939+0000 7f260fe00640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f25f40b2960 0x7f25f40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.939+0000 7f261ca9e740 1 -- 172.21.15.39:0/3268331852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6966a390 msgr2=0x55ae69a30250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.939+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3268331852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6966a390 0x55ae69a30250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009230 tx=0x7f25fc031b10 comp rx=0 tx=0).stop 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.940+0000 7f261ca9e740 1 -- 172.21.15.39:0/3268331852 shutdown_connections 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.940+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3268331852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6966a390 0x55ae69a30250 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.940+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3268331852 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 0x55ae69a2fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.940+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3268331852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6968b070 0x55ae69a2f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.940+0000 7f261ca9e740 1 -- 172.21.15.39:0/3268331852 >> 172.21.15.39:0/3268331852 conn(0x55ae6967f460 msgr2=0x55ae69a2d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.940+0000 7f261ca9e740 1 -- 172.21.15.39:0/3268331852 shutdown_connections 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.940+0000 7f260fe00640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f25f40b2960 0x7f25f40b4e70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2608005fd0 tx=0x7f2608009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.940+0000 7f261ca9e740 1 -- 172.21.15.39:0/3268331852 wait complete. 2024-07-04T08:26:14.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.941+0000 7f261ca9e740 1 -- start start 2024-07-04T08:26:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.941+0000 7f261ca9e740 1 Processor -- start 2024-07-04T08:26:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae696621b0 0x55ae69a3b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6966a390 0x55ae69a3b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 0x55ae69a47240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f261ca9e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ae697f9700 con 0x55ae6968b070 2024-07-04T08:26:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f261ca9e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ae6993cbd0 con 0x55ae6966a390 2024-07-04T08:26:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f261ca9e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ae69a3df50 con 0x55ae696621b0 2024-07-04T08:26:14.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f261ca9e740 10 client.0 _ll_register_callbacks cb 0x55ae6998e590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:14.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae696621b0 0x55ae69a3b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6966a390 0x55ae69a3b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 0x55ae69a47240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae696621b0 0x55ae69a3b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58628/0 (socket says 172.21.15.39:58628) 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260f400640 1 -- 172.21.15.39:0/2251268033 learned_addr learned my addr 172.21.15.39:0/2251268033 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260f400640 1 -- 172.21.15.39:0/2251268033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6966a390 msgr2=0x55ae69a3b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260f400640 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6966a390 0x55ae69a3b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260f400640 1 -- 172.21.15.39:0/2251268033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 msgr2=0x55ae69a47240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260f400640 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 0x55ae69a47240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.942+0000 7f260f400640 1 -- 172.21.15.39:0/2251268033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae696f6650 con 0x55ae696621b0 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.943+0000 7f260f400640 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae696621b0 0x55ae69a3b2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f26000a5db0 tx=0x7f26000a94a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2060c410e3fbaa server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.943+0000 7f25f9600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.943+0000 7f25f9600640 1 -- 172.21.15.39:0/2251268033 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26000e0630 con 0x55ae696621b0 2024-07-04T08:26:14.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.943+0000 7f261ca9e740 1 -- 172.21.15.39:0/2251268033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc004030 con 0x55ae696621b0 2024-07-04T08:26:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.943+0000 7f25f9600640 1 -- 172.21.15.39:0/2251268033 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9512+0+0 (secure 0 0 0) 0x7f26000a8f90 con 0x55ae696621b0 2024-07-04T08:26:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.943+0000 7f25f9600640 1 -- 172.21.15.39:0/2251268033 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f26000a9d30 con 0x55ae696621b0 2024-07-04T08:26:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.943+0000 7f261ca9e740 10 client.27868 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.943+0000 7f261ca9e740 1 -- 172.21.15.39:0/2251268033 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ae69a49ec0 con 0x55ae696621b0 2024-07-04T08:26:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.944+0000 7f25f9600640 1 -- 172.21.15.39:0/2251268033 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25fc004030 con 0x55ae696621b0 2024-07-04T08:26:14.936 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.944+0000 7f25f9600640 1 -- 172.21.15.39:0/2251268033 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f26000c4280 con 0x55ae696621b0 2024-07-04T08:26:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.944+0000 7f261ca9e740 10 client.27868 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.944+0000 7f261ca9e740 10 client.27868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:14.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.944+0000 7f261ca9e740 20 client.27868 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.944+0000 7f261ca9e740 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ae696499f0 0x55ae69a4d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.944+0000 7f261ca9e740 4 client.27868 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.944+0000 7f261ca9e740 1 -- 172.21.15.39:0/2251268033 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae699a7e50 con 0x55ae696499f0 2024-07-04T08:26:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.945+0000 7f260ea00640 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ae696499f0 0x55ae69a4d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.945+0000 7f25f9600640 10 client.27868 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.945+0000 7f260ea00640 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ae696499f0 0x55ae69a4d440 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a5243245445e01 server_cookie=bd4d265fb2f3071e in_seq=0 out_seq=0 2024-07-04T08:26:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.946+0000 7f25f9600640 1 -- 172.21.15.39:0/2251268033 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ae699a7e50 con 0x55ae696499f0 2024-07-04T08:26:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.946+0000 7f25f9600640 10 client.27868 handle_command_reply: tid=0 2024-07-04T08:26:14.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.946+0000 7f261ca9e740 1 client.27868 shutdown 2024-07-04T08:26:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 -- 172.21.15.39:0/2251268033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae696621b0 msgr2=0x55ae69a3b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae696621b0 0x55ae69a3b2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f26000a5db0 tx=0x7f26000a94a0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 -- 172.21.15.39:0/2251268033 shutdown_connections 2024-07-04T08:26:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ae696499f0 0x55ae69a4d440 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 0x55ae69a47240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6966a390 0x55ae69a3b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 --2- 172.21.15.39:0/2251268033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae696621b0 0x55ae69a3b2f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f25f9600640 0 client.27868 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 -- 172.21.15.39:0/2251268033 >> 172.21.15.39:0/2251268033 conn(0x55ae6967f460 msgr2=0x55ae69a32960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 -- 172.21.15.39:0/2251268033 shutdown_connections 2024-07-04T08:26:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 1 -- 172.21.15.39:0/2251268033 wait complete. 2024-07-04T08:26:14.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.947+0000 7f261ca9e740 20 client.27868 trim_cache size 0 max 16384 2024-07-04T08:26:14.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.963+0000 7f261ca9e740 1 Processor -- start 2024-07-04T08:26:14.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.963+0000 7f261ca9e740 1 -- start start 2024-07-04T08:26:14.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.963+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 0x55ae69a32fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.963+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a34a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.963+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 0x55ae69a47030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.963+0000 7f261ca9e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b6aa0 con 0x55ae6968b070 2024-07-04T08:26:14.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.963+0000 7f261ca9e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b85e0 con 0x55ae696621b0 2024-07-04T08:26:14.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.963+0000 7f261ca9e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b7170 con 0x55ae6966a390 2024-07-04T08:26:14.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 0x55ae69a47030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a34a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a34a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58634/0 (socket says 172.21.15.39:58634) 2024-07-04T08:26:14.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 0x55ae69a47030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54678/0 (socket says 172.21.15.39:54678) 2024-07-04T08:26:14.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260ea00640 1 -- 172.21.15.39:0/189545212 learned_addr learned my addr 172.21.15.39:0/189545212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:14.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260f400640 1 --2- 172.21.15.39:0/189545212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 0x55ae69a32fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260ea00640 1 -- 172.21.15.39:0/189545212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 msgr2=0x55ae69a32fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:14.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260ea00640 1 --2- 172.21.15.39:0/189545212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 0x55ae69a32fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260ea00640 1 -- 172.21.15.39:0/189545212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 msgr2=0x55ae69a47030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:14.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260ea00640 1 --2- 172.21.15.39:0/189545212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 0x55ae69a47030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.964+0000 7f260ea00640 1 -- 172.21.15.39:0/189545212 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae69a395f0 con 0x55ae6966a390 2024-07-04T08:26:14.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.965+0000 7f260ea00640 1 --2- 172.21.15.39:0/189545212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a34a50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0095d0 tx=0x7f25fc005130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b5f7c09c626925 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:14.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.965+0000 7f25fbe00640 1 -- 172.21.15.39:0/189545212 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc040040 con 0x55ae6966a390 2024-07-04T08:26:14.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.965+0000 7f25fbe00640 1 -- 172.21.15.39:0/189545212 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25fc0314a0 con 0x55ae6966a390 2024-07-04T08:26:14.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.965+0000 7f25fbe00640 1 -- 172.21.15.39:0/189545212 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc035040 con 0x55ae6966a390 2024-07-04T08:26:14.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.966+0000 7f261ca9e740 1 -- 172.21.15.39:0/189545212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 msgr2=0x55ae69a34a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:14.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.966+0000 7f261ca9e740 1 --2- 172.21.15.39:0/189545212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a34a50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0095d0 tx=0x7f25fc005130 comp rx=0 tx=0).stop 2024-07-04T08:26:14.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.966+0000 7f261ca9e740 1 -- 172.21.15.39:0/189545212 shutdown_connections 2024-07-04T08:26:14.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.966+0000 7f261ca9e740 1 --2- 172.21.15.39:0/189545212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae6968b070 0x55ae69a47030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.966+0000 7f261ca9e740 1 --2- 172.21.15.39:0/189545212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a34a50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.966+0000 7f261ca9e740 1 --2- 172.21.15.39:0/189545212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae696621b0 0x55ae69a32fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.966+0000 7f261ca9e740 1 -- 172.21.15.39:0/189545212 >> 172.21.15.39:0/189545212 conn(0x55ae6967f460 msgr2=0x55ae69a328d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:14.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.966+0000 7f261ca9e740 1 -- 172.21.15.39:0/189545212 shutdown_connections 2024-07-04T08:26:14.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.966+0000 7f261ca9e740 1 -- 172.21.15.39:0/189545212 wait complete. 2024-07-04T08:26:14.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.967+0000 7f261ca9e740 1 -- start start 2024-07-04T08:26:14.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.967+0000 7f261ca9e740 1 Processor -- start 2024-07-04T08:26:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae696621b0 0x55ae69a3f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a3f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f261ca9e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6968b070 0x55ae69ab4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f261ca9e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b7f10 con 0x55ae696621b0 2024-07-04T08:26:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f261ca9e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b7840 con 0x55ae6968b070 2024-07-04T08:26:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f261ca9e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b4aa0 con 0x55ae6966a390 2024-07-04T08:26:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f261ca9e740 10 client.0 _ll_register_callbacks cb 0x7f26000046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f260f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae696621b0 0x55ae69a3f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f260f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae696621b0 0x55ae69a3f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:54690/0 (socket says 172.21.15.39:54690) 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f260f400640 1 -- 172.21.15.39:0/3066032543 learned_addr learned my addr 172.21.15.39:0/3066032543 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f260ea00640 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a3f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.968+0000 7f260fe00640 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6968b070 0x55ae69ab4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f260f400640 1 -- 172.21.15.39:0/3066032543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 msgr2=0x55ae69a3f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f260f400640 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a3f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f260f400640 1 -- 172.21.15.39:0/3066032543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6968b070 msgr2=0x55ae69ab4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f260f400640 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6968b070 0x55ae69ab4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f260f400640 1 -- 172.21.15.39:0/3066032543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae69a40030 con 0x55ae696621b0 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f260ea00640 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a3f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:14.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f260fe00640 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6968b070 0x55ae69ab4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:14.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f260f400640 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae696621b0 0x55ae69a3f2e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f260000c6a0 tx=0x7f26000a58b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad13e43cb58dadb6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:14.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f25f9600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:26:14.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f25f9600640 1 -- 172.21.15.39:0/3066032543 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26000e0630 con 0x55ae696621b0 2024-07-04T08:26:14.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f25f9600640 1 -- 172.21.15.39:0/3066032543 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9512+0+0 (secure 0 0 0) 0x7f26000c4290 con 0x55ae696621b0 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.969+0000 7f261ca9e740 1 -- 172.21.15.39:0/3066032543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc0314a0 con 0x55ae696621b0 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f25f9600640 1 -- 172.21.15.39:0/3066032543 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f26000a5bf0 con 0x55ae696621b0 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f261ca9e740 10 client.17805 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f261ca9e740 1 -- 172.21.15.39:0/3066032543 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ae69ab60d0 con 0x55ae696621b0 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f25f9600640 1 -- 172.21.15.39:0/3066032543 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25fc0314a0 con 0x55ae696621b0 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f25f9600640 1 -- 172.21.15.39:0/3066032543 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f26000a5040 con 0x55ae696621b0 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f261ca9e740 10 client.17805 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f261ca9e740 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f261ca9e740 20 client.17805 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ae6961eda0 0x55ae69ab8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f261ca9e740 4 client.17805 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:14.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.970+0000 7f261ca9e740 1 -- 172.21.15.39:0/3066032543 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae699a38f0 con 0x55ae6961eda0 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.971+0000 7f260ea00640 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ae6961eda0 0x55ae69ab8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.971+0000 7f260ea00640 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ae6961eda0 0x55ae69ab8700 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee1783f75b579267 server_cookie=8cb63b78ed1e77db in_seq=0 out_seq=0 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.971+0000 7f25f9600640 10 client.17805 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f25f9600640 1 -- 172.21.15.39:0/3066032543 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae699a38f0 con 0x55ae6961eda0 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f25f9600640 10 client.17805 handle_command_reply: tid=0 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 client.17805 shutdown 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 -- 172.21.15.39:0/3066032543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae696621b0 msgr2=0x55ae69a3f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae696621b0 0x55ae69a3f2e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f260000c6a0 tx=0x7f26000a58b0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 -- 172.21.15.39:0/3066032543 shutdown_connections 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ae6961eda0 0x55ae69ab8700 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ae6968b070 0x55ae69ab4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ae6966a390 0x55ae69a3f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 --2- 172.21.15.39:0/3066032543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ae696621b0 0x55ae69a3f2e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 -- 172.21.15.39:0/3066032543 >> 172.21.15.39:0/3066032543 conn(0x55ae6967f460 msgr2=0x55ae69a30800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f25f9600640 0 client.17805 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 -- 172.21.15.39:0/3066032543 shutdown_connections 2024-07-04T08:26:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.972+0000 7f261ca9e740 1 -- 172.21.15.39:0/3066032543 wait complete. 2024-07-04T08:26:14.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f261ca9e740 20 client.17805 trim_cache size 0 max 16384 2024-07-04T08:26:14.967 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 -- 172.21.15.39:0/2809289396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f25f40b2960 msgr2=0x7f25f40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f25f40b2960 0x7f25f40b4e70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2608005fd0 tx=0x7f2608009500 comp rx=0 tx=0).stop 2024-07-04T08:26:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 -- 172.21.15.39:0/2809289396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2610103f00 msgr2=0x7f2610108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2610103f00 0x7f2610108b80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f260000e9c0 tx=0x7f260000ee80 comp rx=0 tx=0).stop 2024-07-04T08:26:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 -- 172.21.15.39:0/2809289396 shutdown_connections 2024-07-04T08:26:14.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f25f40b2960 0x7f25f40b4e70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2610104850 0x7f2610104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f261010d360 0x7f2610105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 --2- 172.21.15.39:0/2809289396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2610103f00 0x7f2610108b80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.973+0000 7f2616400640 1 -- 172.21.15.39:0/2809289396 >> 172.21.15.39:0/2809289396 conn(0x7f2610078c80 msgr2=0x7f26100ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.974+0000 7f2616400640 1 -- 172.21.15.39:0/2809289396 shutdown_connections 2024-07-04T08:26:14.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:14.974+0000 7f2616400640 1 -- 172.21.15.39:0/2809289396 wait complete. 2024-07-04T08:26:15.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:15 smithi039 ceph-mon[32062]: pgmap v820: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-07-04T08:26:15.384 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:15 smithi050 ceph-mon[36577]: pgmap v820: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-07-04T08:26:15.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:15 smithi133 ceph-mon[37050]: pgmap v820: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-07-04T08:26:17.297 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:26:17.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:26:17.384 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:17 smithi050 ceph-mon[36577]: pgmap v821: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-07-04T08:26:17.509 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:17 smithi039 ceph-mon[32062]: pgmap v821: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-07-04T08:26:17.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:17 smithi133 ceph-mon[37050]: pgmap v821: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-07-04T08:26:18.040 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:18.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.071+0000 7fcc9d600640 1 -- 172.21.15.39:0/811046967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc980768d0 msgr2=0x7fcc98074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:18.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.071+0000 7fcc9d600640 1 --2- 172.21.15.39:0/811046967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc980768d0 0x7fcc98074d00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009900 tx=0x7fcc8802a410 comp rx=0 tx=0).stop 2024-07-04T08:26:18.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.071+0000 7fcc9d600640 1 -- 172.21.15.39:0/811046967 shutdown_connections 2024-07-04T08:26:18.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.071+0000 7fcc9d600640 1 --2- 172.21.15.39:0/811046967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc98075320 0x7fcc98075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:18.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.071+0000 7fcc9d600640 1 --2- 172.21.15.39:0/811046967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc98113990 0x7fcc98115e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:18.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.071+0000 7fcc9d600640 1 --2- 172.21.15.39:0/811046967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc980768d0 0x7fcc98074d00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:18.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.071+0000 7fcc9d600640 1 -- 172.21.15.39:0/811046967 >> 172.21.15.39:0/811046967 conn(0x7fcc980feb30 msgr2=0x7fcc98100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:18.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.071+0000 7fcc9d600640 1 -- 172.21.15.39:0/811046967 shutdown_connections 2024-07-04T08:26:18.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.071+0000 7fcc9d600640 1 -- 172.21.15.39:0/811046967 wait complete. 2024-07-04T08:26:18.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.073+0000 7fcc9d600640 1 Processor -- start 2024-07-04T08:26:18.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.073+0000 7fcc9d600640 1 -- start start 2024-07-04T08:26:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc9d600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc98075320 0x7fcc981a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc9d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc980768d0 0x7fcc981a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc9d600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc98113990 0x7fcc981a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc9d600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981195e0 con 0x7fcc98113990 2024-07-04T08:26:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc9d600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc98119460 con 0x7fcc98075320 2024-07-04T08:26:18.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc9d600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc98119760 con 0x7fcc980768d0 2024-07-04T08:26:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc96000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc980768d0 0x7fcc981a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc98075320 0x7fcc981a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc96000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc980768d0 0x7fcc981a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58672/0 (socket says 172.21.15.39:58672) 2024-07-04T08:26:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc96000640 1 -- 172.21.15.39:0/2227612436 learned_addr learned my addr 172.21.15.39:0/2227612436 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc98075320 0x7fcc981a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41674/0 (socket says 172.21.15.39:41674) 2024-07-04T08:26:18.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.074+0000 7fcc97400640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc98113990 0x7fcc981a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.075+0000 7fcc96000640 1 -- 172.21.15.39:0/2227612436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc98075320 msgr2=0x7fcc981a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.075+0000 7fcc96000640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc98075320 0x7fcc981a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.075+0000 7fcc96000640 1 -- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc98113990 msgr2=0x7fcc981a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.075+0000 7fcc96000640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc98113990 0x7fcc981a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:18.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.075+0000 7fcc96000640 1 -- 172.21.15.39:0/2227612436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc981a5fa0 con 0x7fcc980768d0 2024-07-04T08:26:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.075+0000 7fcc96000640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc980768d0 0x7fcc981a05d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400e9c0 tx=0x7fcc8400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.075+0000 7fcc8f400640 1 -- 172.21.15.39:0/2227612436 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400cd80 con 0x7fcc980768d0 2024-07-04T08:26:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.075+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc981a6290 con 0x7fcc980768d0 2024-07-04T08:26:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.075+0000 7fcc8f400640 1 -- 172.21.15.39:0/2227612436 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc840054d0 con 0x7fcc980768d0 2024-07-04T08:26:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.076+0000 7fcc8f400640 1 -- 172.21.15.39:0/2227612436 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84010430 con 0x7fcc980768d0 2024-07-04T08:26:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.076+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc981ac9f0 con 0x7fcc980768d0 2024-07-04T08:26:18.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.077+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc98075730 con 0x7fcc980768d0 2024-07-04T08:26:18.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.080+0000 7fcc8f400640 1 -- 172.21.15.39:0/2227612436 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcc84004fb0 con 0x7fcc980768d0 2024-07-04T08:26:18.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.083+0000 7fcc8f400640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc7c0b2960 0x7fcc7c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:18.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.083+0000 7fcc8f400640 1 -- 172.21.15.39:0/2227612436 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9512+0+0 (secure 0 0 0) 0x7fcc840e3070 con 0x7fcc980768d0 2024-07-04T08:26:18.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.083+0000 7fcc96a00640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc7c0b2960 0x7fcc7c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:18.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.083+0000 7fcc8f400640 1 -- 172.21.15.39:0/2227612436 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fcc840aa610 con 0x7fcc980768d0 2024-07-04T08:26:18.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.084+0000 7fcc96a00640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc7c0b2960 0x7fcc7c0b4e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcc88031440 tx=0x7fcc88009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:18.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.342+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc981a1160 con 0x7fcc980768d0 2024-07-04T08:26:18.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.343+0000 7fcc8f400640 1 -- 172.21.15.39:0/2227612436 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fcc840a9f10 con 0x7fcc980768d0 2024-07-04T08:26:18.339 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:26:18.339 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:26:18.339 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:26:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.346+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc7c0b2960 msgr2=0x7fcc7c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.346+0000 7fcc9d600640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc7c0b2960 0x7fcc7c0b4e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcc88031440 tx=0x7fcc88009560 comp rx=0 tx=0).stop 2024-07-04T08:26:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.346+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc980768d0 msgr2=0x7fcc981a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.346+0000 7fcc9d600640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc980768d0 0x7fcc981a05d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400e9c0 tx=0x7fcc8400ee80 comp rx=0 tx=0).stop 2024-07-04T08:26:18.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.347+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 shutdown_connections 2024-07-04T08:26:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.347+0000 7fcc9d600640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc7c0b2960 0x7fcc7c0b4e70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.347+0000 7fcc9d600640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc98113990 0x7fcc981a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.347+0000 7fcc9d600640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc980768d0 0x7fcc981a05d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.347+0000 7fcc9d600640 1 --2- 172.21.15.39:0/2227612436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc98075320 0x7fcc981a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.347+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 >> 172.21.15.39:0/2227612436 conn(0x7fcc980feb30 msgr2=0x7fcc9810f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.347+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 shutdown_connections 2024-07-04T08:26:18.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:18.347+0000 7fcc9d600640 1 -- 172.21.15.39:0/2227612436 wait complete. 2024-07-04T08:26:19.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:19 smithi133 ceph-mon[37050]: pgmap v822: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-07-04T08:26:19.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:19 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2227612436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:19.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:19 smithi133 ceph-mon[37050]: osdmap e102: 12 total, 12 up, 12 in 2024-07-04T08:26:19.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:19 smithi039 ceph-mon[32062]: pgmap v822: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-07-04T08:26:19.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:19 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2227612436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:19.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:19 smithi039 ceph-mon[32062]: osdmap e102: 12 total, 12 up, 12 in 2024-07-04T08:26:19.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:19 smithi050 ceph-mon[36577]: pgmap v822: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-07-04T08:26:19.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:19 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2227612436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:19.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:19 smithi050 ceph-mon[36577]: osdmap e102: 12 total, 12 up, 12 in 2024-07-04T08:26:19.803 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id 1289c4b8 --release --await' 2024-07-04T08:26:19.803 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id 1289c4b8 --release --await 2024-07-04T08:26:20.298 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.060+0000 7f13c2000640 1 -- 172.21.15.39:0/1151078435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 msgr2=0x7f13bc113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.060+0000 7f13c2000640 1 --2- 172.21.15.39:0/1151078435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 0x7f13bc113780 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f13a8009900 tx=0x7f13a802a410 comp rx=0 tx=0).stop 2024-07-04T08:26:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.060+0000 7f13c2000640 1 -- 172.21.15.39:0/1151078435 shutdown_connections 2024-07-04T08:26:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.060+0000 7f13c2000640 1 --2- 172.21.15.39:0/1151078435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 0x7f13bc113780 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.060+0000 7f13c2000640 1 --2- 172.21.15.39:0/1151078435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13bc113cc0 0x7f13bc1160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.060+0000 7f13c2000640 1 --2- 172.21.15.39:0/1151078435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc076e90 0x7f13bc0772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.060+0000 7f13c2000640 1 -- 172.21.15.39:0/1151078435 >> 172.21.15.39:0/1151078435 conn(0x7f13bc100d10 msgr2=0x7f13bc103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:21.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.060+0000 7f13c2000640 1 -- 172.21.15.39:0/1151078435 shutdown_connections 2024-07-04T08:26:21.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.060+0000 7f13c2000640 1 -- 172.21.15.39:0/1151078435 wait complete. 2024-07-04T08:26:21.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.062+0000 7f13c2000640 1 Processor -- start 2024-07-04T08:26:21.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.062+0000 7f13c2000640 1 -- start start 2024-07-04T08:26:21.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.062+0000 7f13c2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc076e90 0x7f13bc1a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.062+0000 7f13c2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 0x7f13bc1a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.062+0000 7f13c2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13bc113cc0 0x7f13bc1a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.062+0000 7f13c2000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc119660 con 0x7f13bc0777e0 2024-07-04T08:26:21.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.062+0000 7f13c2000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1194e0 con 0x7f13bc076e90 2024-07-04T08:26:21.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.062+0000 7f13c2000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1197e0 con 0x7f13bc113cc0 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13baa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 0x7f13bc1a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13baa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 0x7f13bc1a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40454/0 (socket says 172.21.15.39:40454) 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13baa00640 1 -- 172.21.15.39:0/2568040428 learned_addr learned my addr 172.21.15.39:0/2568040428 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13bb400640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc076e90 0x7f13bc1a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13bbe00640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13bc113cc0 0x7f13bc1a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13baa00640 1 -- 172.21.15.39:0/2568040428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13bc113cc0 msgr2=0x7f13bc1a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13baa00640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13bc113cc0 0x7f13bc1a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13baa00640 1 -- 172.21.15.39:0/2568040428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc076e90 msgr2=0x7f13bc1a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13baa00640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc076e90 0x7f13bc1a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13baa00640 1 -- 172.21.15.39:0/2568040428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc1a5f80 con 0x7f13bc0777e0 2024-07-04T08:26:21.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.063+0000 7f13bb400640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc076e90 0x7f13bc1a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:21.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.064+0000 7f13baa00640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 0x7f13bc1a05b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f13a8005ab0 tx=0x7f13a8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:21.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.064+0000 7f13b3e00640 1 -- 172.21.15.39:0/2568040428 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a803f070 con 0x7f13bc0777e0 2024-07-04T08:26:21.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.064+0000 7f13b3e00640 1 -- 172.21.15.39:0/2568040428 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a8031b50 con 0x7f13bc0777e0 2024-07-04T08:26:21.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.064+0000 7f13c2000640 1 -- 172.21.15.39:0/2568040428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc1a6210 con 0x7f13bc0777e0 2024-07-04T08:26:21.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.064+0000 7f13b3e00640 1 -- 172.21.15.39:0/2568040428 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8035030 con 0x7f13bc0777e0 2024-07-04T08:26:21.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.064+0000 7f13c2000640 1 -- 172.21.15.39:0/2568040428 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc1ac9d0 con 0x7f13bc0777e0 2024-07-04T08:26:21.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.068+0000 7f13c8535740 1 Processor -- start 2024-07-04T08:26:21.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.068+0000 7f13c8535740 1 -- start start 2024-07-04T08:26:21.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.068+0000 7f13b3e00640 1 -- 172.21.15.39:0/2568040428 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f13a8042070 con 0x7f13bc0777e0 2024-07-04T08:26:21.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.068+0000 7f13c8535740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac8e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.068+0000 7f13c8535740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac8eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.068+0000 7f13c8535740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac8f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.068+0000 7f13c8535740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562f2abb88e0 con 0x562f2a8ea350 2024-07-04T08:26:21.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.068+0000 7f13c8535740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562f2ac04ec0 con 0x562f2a8c1490 2024-07-04T08:26:21.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.068+0000 7f13c8535740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562f2abf4460 con 0x562f2a8c9670 2024-07-04T08:26:21.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac8eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac8eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43810/0 (socket says 172.21.15.39:43810) 2024-07-04T08:26:21.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bb400640 1 -- 172.21.15.39:0/1906137085 learned_addr learned my addr 172.21.15.39:0/1906137085 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13baa00640 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac8f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bbe00640 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac8e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bb400640 1 -- 172.21.15.39:0/1906137085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 msgr2=0x562f2ac8f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bb400640 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac8f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bb400640 1 -- 172.21.15.39:0/1906137085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 msgr2=0x562f2ac8e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bb400640 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac8e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bb400640 1 -- 172.21.15.39:0/1906137085 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f2ab8c280 con 0x562f2a8c1490 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.069+0000 7f13bbe00640 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac8e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.070+0000 7f13bb400640 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac8eb80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f13ac011540 tx=0x7f13ac011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7338579ba31fa3e1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:21.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.070+0000 7f13b0c00640 1 -- 172.21.15.39:0/1906137085 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00f640 con 0x562f2a8c1490 2024-07-04T08:26:21.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.070+0000 7f13b0c00640 1 -- 172.21.15.39:0/1906137085 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac00fc90 con 0x562f2a8c1490 2024-07-04T08:26:21.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.070+0000 7f13b0c00640 1 -- 172.21.15.39:0/1906137085 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac019c80 con 0x562f2a8c1490 2024-07-04T08:26:21.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.070+0000 7f13b3e00640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f13a00b29e0 0x7f13a00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.071+0000 7f13b3e00640 1 -- 172.21.15.39:0/2568040428 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f13a8102b80 con 0x7f13bc0777e0 2024-07-04T08:26:21.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.071+0000 7f13bbe00640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f13a00b29e0 0x7f13a00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.071+0000 7f13c8535740 1 -- 172.21.15.39:0/1906137085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 msgr2=0x562f2ac8eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.071+0000 7f13c8535740 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac8eb80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f13ac011540 tx=0x7f13ac011a00 comp rx=0 tx=0).stop 2024-07-04T08:26:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.072+0000 7f13c8535740 1 -- 172.21.15.39:0/1906137085 shutdown_connections 2024-07-04T08:26:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.072+0000 7f13c8535740 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac8f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.072+0000 7f13c8535740 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac8eb80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.072+0000 7f13c8535740 1 --2- 172.21.15.39:0/1906137085 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac8e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.072+0000 7f13c8535740 1 -- 172.21.15.39:0/1906137085 >> 172.21.15.39:0/1906137085 conn(0x562f2a8de740 msgr2=0x562f2ac8c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.072+0000 7f13bbe00640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f13a00b29e0 0x7f13a00b4ef0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f13b4005fd0 tx=0x7f13b4009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:21.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.072+0000 7f13c8535740 1 -- 172.21.15.39:0/1906137085 shutdown_connections 2024-07-04T08:26:21.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.072+0000 7f13c8535740 1 -- 172.21.15.39:0/1906137085 wait complete. 2024-07-04T08:26:21.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.072+0000 7f13c8535740 1 -- start start 2024-07-04T08:26:21.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.073+0000 7f13c8535740 1 Processor -- start 2024-07-04T08:26:21.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.073+0000 7f13c8535740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac9a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.073+0000 7f13c8535740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c9670 0x562f2ac9a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.073+0000 7f13c8535740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8ea350 0x562f2aca5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.073+0000 7f13c8535740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562f2ab9b760 con 0x562f2a8c1490 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.073+0000 7f13c8535740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562f2abf84a0 con 0x562f2a8c9670 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.073+0000 7f13c8535740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562f2aa58890 con 0x562f2a8ea350 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.073+0000 7f13c8535740 10 client.0 _ll_register_callbacks cb 0x562f2ab9c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c9670 0x562f2ac9a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac9a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13baa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c9670 0x562f2ac9a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43824/0 (socket says 172.21.15.39:43824) 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13baa00640 1 -- 172.21.15.39:0/2131400008 learned_addr learned my addr 172.21.15.39:0/2131400008 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:21.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13bbe00640 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8ea350 0x562f2aca5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13baa00640 1 -- 172.21.15.39:0/2131400008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8ea350 msgr2=0x562f2aca5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13baa00640 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8ea350 0x562f2aca5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13baa00640 1 -- 172.21.15.39:0/2131400008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 msgr2=0x562f2ac9a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13baa00640 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac9a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13baa00640 1 -- 172.21.15.39:0/2131400008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f2a955930 con 0x562f2a8c9670 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.074+0000 7f13bbe00640 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8ea350 0x562f2aca5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.075+0000 7f13baa00640 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c9670 0x562f2ac9a6e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f13a80c9ae0 tx=0x7f13a8003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b56364634b1cb683 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.075+0000 7f13a5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.075+0000 7f13a5600640 1 -- 172.21.15.39:0/2131400008 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8049070 con 0x562f2a8c9670 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.075+0000 7f13a5600640 1 -- 172.21.15.39:0/2131400008 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f13a80c6140 con 0x562f2a8c9670 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.075+0000 7f13c8535740 1 -- 172.21.15.39:0/2131400008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ac00fc90 con 0x562f2a8c9670 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.075+0000 7f13a5600640 1 -- 172.21.15.39:0/2131400008 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f13a80c94a0 con 0x562f2a8c9670 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.075+0000 7f13c8535740 10 client.27959 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:21.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.075+0000 7f13c8535740 1 -- 172.21.15.39:0/2131400008 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f2aca8c60 con 0x562f2a8c9670 2024-07-04T08:26:21.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.075+0000 7f13a5600640 1 -- 172.21.15.39:0/2131400008 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac00fc90 con 0x562f2a8c9670 2024-07-04T08:26:21.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.076+0000 7f13a5600640 1 -- 172.21.15.39:0/2131400008 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f13a804c060 con 0x562f2a8c9670 2024-07-04T08:26:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.076+0000 7f13c8535740 10 client.27959 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.076+0000 7f13c8535740 10 client.27959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.076+0000 7f13c8535740 20 client.27959 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.076+0000 7f13c8535740 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f2a8a8cd0 0x562f2acac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.076+0000 7f13c8535740 4 client.27959 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.076+0000 7f13c8535740 1 -- 172.21.15.39:0/2131400008 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f2ac06e10 con 0x562f2a8a8cd0 2024-07-04T08:26:21.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.076+0000 7f13bb400640 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f2a8a8cd0 0x562f2acac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.077+0000 7f13bb400640 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f2a8a8cd0 0x562f2acac250 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8f7e2d990c73af server_cookie=56c229b06f76be1c in_seq=0 out_seq=0 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.077+0000 7f13a5600640 10 client.27959 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13a5600640 1 -- 172.21.15.39:0/2131400008 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562f2ac06e10 con 0x562f2a8a8cd0 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13a5600640 10 client.27959 handle_command_reply: tid=0 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 client.27959 shutdown 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 -- 172.21.15.39:0/2131400008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c9670 msgr2=0x562f2ac9a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c9670 0x562f2ac9a6e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f13a80c9ae0 tx=0x7f13a8003f90 comp rx=0 tx=0).stop 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 -- 172.21.15.39:0/2131400008 shutdown_connections 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f2a8a8cd0 0x562f2acac250 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8ea350 0x562f2aca5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c9670 0x562f2ac9a6e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 --2- 172.21.15.39:0/2131400008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac9a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13a5600640 0 client.27959 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 -- 172.21.15.39:0/2131400008 >> 172.21.15.39:0/2131400008 conn(0x562f2a8de740 msgr2=0x562f2ac91820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 -- 172.21.15.39:0/2131400008 shutdown_connections 2024-07-04T08:26:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.078+0000 7f13c8535740 1 -- 172.21.15.39:0/2131400008 wait complete. 2024-07-04T08:26:21.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.079+0000 7f13c8535740 20 client.27959 trim_cache size 0 max 16384 2024-07-04T08:26:21.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13c8535740 1 Processor -- start 2024-07-04T08:26:21.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13c8535740 1 -- start start 2024-07-04T08:26:21.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13c8535740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac8c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13c8535740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac9bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13c8535740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8ea350 0x562f2aca5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13c8535740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b68e0 con 0x562f2a8c1490 2024-07-04T08:26:21.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13c8535740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b8420 con 0x562f2a8ea350 2024-07-04T08:26:21.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13c8535740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b6fb0 con 0x562f2a8c9670 2024-07-04T08:26:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac8c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.099+0000 7f13bb400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac8c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40474/0 (socket says 172.21.15.39:40474) 2024-07-04T08:26:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13bb400640 1 -- 172.21.15.39:0/1083449386 learned_addr learned my addr 172.21.15.39:0/1083449386 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13baa00640 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac9bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13bbe00640 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8ea350 0x562f2aca5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13bb400640 1 -- 172.21.15.39:0/1083449386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 msgr2=0x562f2ac9bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:21.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13bb400640 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac9bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13bb400640 1 -- 172.21.15.39:0/1083449386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8ea350 msgr2=0x562f2aca5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13bb400640 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8ea350 0x562f2aca5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13bb400640 1 -- 172.21.15.39:0/1083449386 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f2ac9aa50 con 0x562f2a8c1490 2024-07-04T08:26:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13bbe00640 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8ea350 0x562f2aca5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.100+0000 7f13bb400640 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac8c4c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f13ac011f00 tx=0x7f13ac0198a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9f0184b7bee71a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.101+0000 7f13a7e00640 1 -- 172.21.15.39:0/1083449386 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac019650 con 0x562f2a8c1490 2024-07-04T08:26:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.101+0000 7f13a7e00640 1 -- 172.21.15.39:0/1083449386 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac019dc0 con 0x562f2a8c1490 2024-07-04T08:26:21.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.101+0000 7f13a7e00640 1 -- 172.21.15.39:0/1083449386 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00f640 con 0x562f2a8c1490 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.101+0000 7f13c8535740 1 -- 172.21.15.39:0/1083449386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 msgr2=0x562f2ac8c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.101+0000 7f13c8535740 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac8c4c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f13ac011f00 tx=0x7f13ac0198a0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.102+0000 7f13c8535740 1 -- 172.21.15.39:0/1083449386 shutdown_connections 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.102+0000 7f13c8535740 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8ea350 0x562f2aca5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.102+0000 7f13c8535740 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac9bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.102+0000 7f13c8535740 1 --2- 172.21.15.39:0/1083449386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8c1490 0x562f2ac8c4c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.102+0000 7f13c8535740 1 -- 172.21.15.39:0/1083449386 >> 172.21.15.39:0/1083449386 conn(0x562f2a8de740 msgr2=0x562f2ac9b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.102+0000 7f13c8535740 1 -- 172.21.15.39:0/1083449386 shutdown_connections 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.102+0000 7f13c8535740 1 -- 172.21.15.39:0/1083449386 wait complete. 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.102+0000 7f13c8535740 1 -- start start 2024-07-04T08:26:21.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.103+0000 7f13c8535740 1 Processor -- start 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.103+0000 7f13c8535740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac9de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.103+0000 7f13c8535740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac9e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.103+0000 7f13c8535740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.103+0000 7f13c8535740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b7d50 con 0x562f2a8ea350 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.103+0000 7f13c8535740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b7680 con 0x562f2a8c1490 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.103+0000 7f13c8535740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1b48c0 con 0x562f2a8c9670 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.103+0000 7f13c8535740 10 client.0 _ll_register_callbacks cb 0x7f13a8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13baa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac9e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac9e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40488/0 (socket says 172.21.15.39:40488) 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bb400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac9de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bbe00640 1 -- 172.21.15.39:0/3939532344 learned_addr learned my addr 172.21.15.39:0/3939532344 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bb400640 1 -- 172.21.15.39:0/3939532344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 msgr2=0x562f2ac9e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bb400640 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac9e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bb400640 1 -- 172.21.15.39:0/3939532344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 msgr2=0x562f2ac9e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bb400640 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac9e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bb400640 1 -- 172.21.15.39:0/3939532344 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f2ad13aa0 con 0x562f2a8c1490 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13baa00640 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac9e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13bb400640 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac9de80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006010 tx=0x7f13ac009d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756800f9289173a6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.104+0000 7f13a5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.105+0000 7f13a5600640 1 -- 172.21.15.39:0/3939532344 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac019910 con 0x562f2a8c1490 2024-07-04T08:26:21.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.105+0000 7f13c8535740 1 -- 172.21.15.39:0/3939532344 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ac019dc0 con 0x562f2a8c1490 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.105+0000 7f13a5600640 1 -- 172.21.15.39:0/3939532344 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f13ac019ab0 con 0x562f2a8c1490 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.105+0000 7f13a5600640 1 -- 172.21.15.39:0/3939532344 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f13ac040050 con 0x562f2a8c1490 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.105+0000 7f13c8535740 10 client.27965 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.105+0000 7f13c8535740 1 -- 172.21.15.39:0/3939532344 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f2ad14d40 con 0x562f2a8c1490 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.106+0000 7f13a5600640 1 -- 172.21.15.39:0/3939532344 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac019dc0 con 0x562f2a8c1490 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.106+0000 7f13a5600640 1 -- 172.21.15.39:0/3939532344 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f13ac02b020 con 0x562f2a8c1490 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.106+0000 7f13c8535740 10 client.27965 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.106+0000 7f13c8535740 10 client.27965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.106+0000 7f13c8535740 20 client.27965 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.106+0000 7f13c8535740 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f2a87e080 0x562f2ad171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.106+0000 7f13c8535740 4 client.27965 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.106+0000 7f13c8535740 1 -- 172.21.15.39:0/3939532344 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562f2ac028b0 con 0x562f2a87e080 2024-07-04T08:26:21.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.106+0000 7f13baa00640 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f2a87e080 0x562f2ad171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:21.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.107+0000 7f13baa00640 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f2a87e080 0x562f2ad171d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed43ed68d82320dd server_cookie=edcb1efaad94e9f6 in_seq=0 out_seq=0 2024-07-04T08:26:21.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.107+0000 7f13a5600640 10 client.27965 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13a5600640 1 -- 172.21.15.39:0/3939532344 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562f2ac028b0 con 0x562f2a87e080 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13a5600640 10 client.27965 handle_command_reply: tid=0 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13c8535740 1 client.27965 shutdown 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13c8535740 1 -- 172.21.15.39:0/3939532344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 msgr2=0x562f2ac9de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13c8535740 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac9de80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006010 tx=0x7f13ac009d50 comp rx=0 tx=0).stop 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13c8535740 1 -- 172.21.15.39:0/3939532344 shutdown_connections 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13c8535740 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562f2a87e080 0x562f2ad171d0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13c8535740 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562f2a8ea350 0x562f2ac9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13c8535740 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562f2a8c9670 0x562f2ac9e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13c8535740 1 --2- 172.21.15.39:0/3939532344 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562f2a8c1490 0x562f2ac9de80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13c8535740 1 -- 172.21.15.39:0/3939532344 >> 172.21.15.39:0/3939532344 conn(0x562f2a8de740 msgr2=0x562f2ac90030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:21.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.108+0000 7f13a5600640 0 client.27965 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:21.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.109+0000 7f13c8535740 1 -- 172.21.15.39:0/3939532344 shutdown_connections 2024-07-04T08:26:21.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.109+0000 7f13c8535740 1 -- 172.21.15.39:0/3939532344 wait complete. 2024-07-04T08:26:21.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.109+0000 7f13c8535740 20 client.27965 trim_cache size 0 max 16384 2024-07-04T08:26:21.102 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:26:21.102 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:26:21.102 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "aa8f8e3c-1fee-4856-a922-2f34fa9862f8", 2024-07-04T08:26:21.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:26:21.102 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:26:21.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 -- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f13a00b29e0 msgr2=0x7f13a00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:21.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f13a00b29e0 0x7f13a00b4ef0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f13b4005fd0 tx=0x7f13b4009500 comp rx=0 tx=0).stop 2024-07-04T08:26:21.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 -- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 msgr2=0x7f13bc1a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:21.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 0x7f13bc1a05b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f13a8005ab0 tx=0x7f13a8003b90 comp rx=0 tx=0).stop 2024-07-04T08:26:21.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 -- 172.21.15.39:0/2568040428 shutdown_connections 2024-07-04T08:26:21.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f13a00b29e0 0x7f13a00b4ef0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13bc0777e0 0x7f13bc1a05b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13bc113cc0 0x7f13bc1a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 --2- 172.21.15.39:0/2568040428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13bc076e90 0x7f13bc1a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:21.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 -- 172.21.15.39:0/2568040428 >> 172.21.15.39:0/2568040428 conn(0x7f13bc100d10 msgr2=0x7f13bc103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:21.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 -- 172.21.15.39:0/2568040428 shutdown_connections 2024-07-04T08:26:21.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:21.110+0000 7f13c2000640 1 -- 172.21.15.39:0/2568040428 wait complete. 2024-07-04T08:26:21.232 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:21 smithi039 ceph-mon[32062]: pgmap v824: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-04T08:26:21.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:21 smithi133 ceph-mon[37050]: pgmap v824: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-04T08:26:21.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:21 smithi050 ceph-mon[36577]: pgmap v824: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-04T08:26:22.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:22 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:26:22.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:22 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:26:22.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:22 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:26:22.539 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:22 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:26:22.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:22 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:26:22.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:22 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:26:22.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:22 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:26:22.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:22 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:26:22.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:22 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:26:22.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:22 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:26:22.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:22 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:26:22.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:22 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:26:22.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:26:23.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:23 smithi039 ceph-mon[32062]: pgmap v825: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:26:23.441 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:23 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:26:23.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:23 smithi133 ceph-mon[37050]: pgmap v825: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:26:23.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:23 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:26:23.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.567+0000 7f1a2b800640 1 -- 172.21.15.39:0/381944974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a24107910 msgr2=0x7f1a24107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:23.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.567+0000 7f1a2b800640 1 --2- 172.21.15.39:0/381944974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a24107910 0x7f1a24107d20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009900 tx=0x7f1a0c02a410 comp rx=0 tx=0).stop 2024-07-04T08:26:23.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.567+0000 7f1a2b800640 1 -- 172.21.15.39:0/381944974 shutdown_connections 2024-07-04T08:26:23.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.567+0000 7f1a2b800640 1 --2- 172.21.15.39:0/381944974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a24107910 0x7f1a24107d20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.567+0000 7f1a2b800640 1 --2- 172.21.15.39:0/381944974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a24108260 0x7f1a2410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.567+0000 7f1a2b800640 1 --2- 172.21.15.39:0/381944974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a24101fd0 0x7f1a241023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.567+0000 7f1a2b800640 1 -- 172.21.15.39:0/381944974 >> 172.21.15.39:0/381944974 conn(0x7f1a24078c90 msgr2=0x7f1a240790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:23.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.567+0000 7f1a2b800640 1 -- 172.21.15.39:0/381944974 shutdown_connections 2024-07-04T08:26:23.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.568+0000 7f1a2b800640 1 -- 172.21.15.39:0/381944974 wait complete. 2024-07-04T08:26:23.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.569+0000 7f1a2b800640 1 Processor -- start 2024-07-04T08:26:23.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.569+0000 7f1a2b800640 1 -- start start 2024-07-04T08:26:23.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.569+0000 7f1a2b800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a24101fd0 0x7f1a240febe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.569+0000 7f1a2b800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a24107910 0x7f1a240ff120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.569+0000 7f1a2b800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a24108260 0x7f1a240ff660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.569+0000 7f1a2b800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24111ed0 con 0x7f1a24108260 2024-07-04T08:26:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.569+0000 7f1a2b800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24111d50 con 0x7f1a24101fd0 2024-07-04T08:26:23.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.569+0000 7f1a2b800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24112050 con 0x7f1a24107910 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a28e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a24101fd0 0x7f1a240febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a29800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a24108260 0x7f1a240ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a28e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a24101fd0 0x7f1a240febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43866/0 (socket says 172.21.15.39:43866) 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a28e00640 1 -- 172.21.15.39:0/2973058540 learned_addr learned my addr 172.21.15.39:0/2973058540 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a1be00640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a24107910 0x7f1a240ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a29800640 1 -- 172.21.15.39:0/2973058540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a24107910 msgr2=0x7f1a240ff120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a29800640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a24107910 0x7f1a240ff120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a29800640 1 -- 172.21.15.39:0/2973058540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a24101fd0 msgr2=0x7f1a240febe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a29800640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a24101fd0 0x7f1a240febe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a29800640 1 -- 172.21.15.39:0/2973058540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a241a7c90 con 0x7f1a24108260 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.570+0000 7f1a1be00640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a24107910 0x7f1a240ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:23.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.571+0000 7f1a28e00640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a24101fd0 0x7f1a240febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:23.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.571+0000 7f1a29800640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a24108260 0x7f1a240ff660 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009fd0 tx=0x7f1a1c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:23.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.571+0000 7f1a22000640 1 -- 172.21.15.39:0/2973058540 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00ce20 con 0x7f1a24108260 2024-07-04T08:26:23.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.571+0000 7f1a2b800640 1 -- 172.21.15.39:0/2973058540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a241a7f80 con 0x7f1a24108260 2024-07-04T08:26:23.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.571+0000 7f1a2b800640 1 -- 172.21.15.39:0/2973058540 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a241a8430 con 0x7f1a24108260 2024-07-04T08:26:23.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.571+0000 7f1a22000640 1 -- 172.21.15.39:0/2973058540 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1c005020 con 0x7f1a24108260 2024-07-04T08:26:23.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.571+0000 7f1a22000640 1 -- 172.21.15.39:0/2973058540 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c010690 con 0x7f1a24108260 2024-07-04T08:26:23.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.575+0000 7f1a31f09740 1 Processor -- start 2024-07-04T08:26:23.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.576+0000 7f1a22000640 1 -- 172.21.15.39:0/2973058540 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1a1c010830 con 0x7f1a24108260 2024-07-04T08:26:23.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.576+0000 7f1a31f09740 1 -- start start 2024-07-04T08:26:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe352520 0x5582fe6f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe6f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe6f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a31f09740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5582fe611b00 con 0x5582fe331840 2024-07-04T08:26:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a31f09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5582fe66d290 con 0x5582fe352520 2024-07-04T08:26:23.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a31f09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5582fe65c830 con 0x5582fe329660 2024-07-04T08:26:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a28e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe6f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a1be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe6f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a28e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe6f77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59606/0 (socket says 172.21.15.39:59606) 2024-07-04T08:26:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a28e00640 1 -- 172.21.15.39:0/1808747803 learned_addr learned my addr 172.21.15.39:0/1808747803 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.577+0000 7f1a1be00640 1 --2- 172.21.15.39:0/1808747803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe352520 0x5582fe6f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.578+0000 7f1a28e00640 1 -- 172.21.15.39:0/1808747803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe352520 msgr2=0x5582fe6f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:23.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.578+0000 7f1a28e00640 1 --2- 172.21.15.39:0/1808747803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe352520 0x5582fe6f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.578+0000 7f1a28e00640 1 -- 172.21.15.39:0/1808747803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 msgr2=0x5582fe6f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:23.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.578+0000 7f1a28e00640 1 --2- 172.21.15.39:0/1808747803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe6f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.578+0000 7f1a28e00640 1 -- 172.21.15.39:0/1808747803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582fe616760 con 0x5582fe329660 2024-07-04T08:26:23.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.578+0000 7f1a22000640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a080b29e0 0x7f1a080b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.578+0000 7f1a28e00640 1 --2- 172.21.15.39:0/1808747803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe6f77e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1a14010610 tx=0x7f1a14010ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d281723f180b11 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:23.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.578+0000 7f1a22000640 1 -- 172.21.15.39:0/2973058540 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f1a1c0e2f70 con 0x7f1a24108260 2024-07-04T08:26:23.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.579+0000 7f1a1be00640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a080b29e0 0x7f1a080b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.579+0000 7f1a1a000640 1 -- 172.21.15.39:0/1808747803 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14005220 con 0x5582fe329660 2024-07-04T08:26:23.571 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.579+0000 7f1a1a000640 1 -- 172.21.15.39:0/1808747803 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a14004c50 con 0x5582fe329660 2024-07-04T08:26:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.579+0000 7f1a1a000640 1 -- 172.21.15.39:0/1808747803 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400eac0 con 0x5582fe329660 2024-07-04T08:26:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a31f09740 1 -- 172.21.15.39:0/1808747803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 msgr2=0x5582fe6f77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1808747803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe6f77e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1a14010610 tx=0x7f1a14010ad0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a1be00640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a080b29e0 0x7f1a080b4ef0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c000c00 tx=0x7f1a0c005780 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a31f09740 1 -- 172.21.15.39:0/1808747803 shutdown_connections 2024-07-04T08:26:23.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1808747803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe6f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1808747803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe6f77e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1808747803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe352520 0x5582fe6f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a31f09740 1 -- 172.21.15.39:0/1808747803 >> 172.21.15.39:0/1808747803 conn(0x5582fe346910 msgr2=0x5582fe6f5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a31f09740 1 -- 172.21.15.39:0/1808747803 shutdown_connections 2024-07-04T08:26:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.580+0000 7f1a31f09740 1 -- 172.21.15.39:0/1808747803 wait complete. 2024-07-04T08:26:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.581+0000 7f1a31f09740 1 -- start start 2024-07-04T08:26:23.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.581+0000 7f1a31f09740 1 Processor -- start 2024-07-04T08:26:23.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe702900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe331840 0x5582fe702e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe352520 0x5582fe70ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a31f09740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5582fe603d00 con 0x5582fe352520 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a31f09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5582fe660870 con 0x5582fe331840 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a31f09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5582fe4c0bb0 con 0x5582fe329660 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a31f09740 10 client.0 _ll_register_callbacks cb 0x5582fe6049c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a28e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe702900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a28e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe702900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59608/0 (socket says 172.21.15.39:59608) 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a28e00640 1 -- 172.21.15.39:0/1548119158 learned_addr learned my addr 172.21.15.39:0/1548119158 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a1be00640 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe331840 0x5582fe702e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a29800640 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe352520 0x5582fe70ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.582+0000 7f1a28e00640 1 -- 172.21.15.39:0/1548119158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe331840 msgr2=0x5582fe702e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:23.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.583+0000 7f1a28e00640 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe331840 0x5582fe702e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.583+0000 7f1a28e00640 1 -- 172.21.15.39:0/1548119158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe352520 msgr2=0x5582fe70ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:23.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.583+0000 7f1a28e00640 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe352520 0x5582fe70ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.583+0000 7f1a28e00640 1 -- 172.21.15.39:0/1548119158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582fe5f3640 con 0x5582fe329660 2024-07-04T08:26:23.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.583+0000 7f1a28e00640 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe702900 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400e660 tx=0x7f1a14004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5d37d4b4842a206 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:23.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.583+0000 7f1a12a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:23.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.583+0000 7f1a12a00640 1 -- 172.21.15.39:0/1548119158 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1402a060 con 0x5582fe329660 2024-07-04T08:26:23.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.583+0000 7f1a12a00640 1 -- 172.21.15.39:0/1548119158 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f1a1400e840 con 0x5582fe329660 2024-07-04T08:26:23.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.583+0000 7f1a31f09740 1 -- 172.21.15.39:0/1548119158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a14004c50 con 0x5582fe329660 2024-07-04T08:26:23.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.584+0000 7f1a12a00640 1 -- 172.21.15.39:0/1548119158 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a14031050 con 0x5582fe329660 2024-07-04T08:26:23.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.584+0000 7f1a31f09740 10 client.27910 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:23.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.584+0000 7f1a31f09740 1 -- 172.21.15.39:0/1548119158 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582fe3bdb00 con 0x5582fe329660 2024-07-04T08:26:23.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.584+0000 7f1a12a00640 1 -- 172.21.15.39:0/1548119158 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a14004c50 con 0x5582fe329660 2024-07-04T08:26:23.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.585+0000 7f1a12a00640 1 -- 172.21.15.39:0/1548119158 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1a1401a480 con 0x5582fe329660 2024-07-04T08:26:23.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.585+0000 7f1a31f09740 10 client.27910 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:23.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.585+0000 7f1a31f09740 10 client.27910 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:26:23.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.585+0000 7f1a31f09740 20 client.27910 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:23.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.585+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5582fe310ea0 0x5582fe714820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.585+0000 7f1a31f09740 4 client.27910 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:23.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.585+0000 7f1a31f09740 1 -- 172.21.15.39:0/1548119158 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582fe66f1e0 con 0x5582fe310ea0 2024-07-04T08:26:23.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.585+0000 7f1a1be00640 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5582fe310ea0 0x5582fe714820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.586+0000 7f1a1be00640 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5582fe310ea0 0x5582fe714820 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5830bf1e702079 server_cookie=e6aed7bca6f390bf in_seq=0 out_seq=0 2024-07-04T08:26:23.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.586+0000 7f1a12a00640 10 client.27910 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:23.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a12a00640 1 -- 172.21.15.39:0/1548119158 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5582fe66f1e0 con 0x5582fe310ea0 2024-07-04T08:26:23.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a12a00640 10 client.27910 handle_command_reply: tid=0 2024-07-04T08:26:23.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a31f09740 1 client.27910 shutdown 2024-07-04T08:26:23.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a31f09740 1 -- 172.21.15.39:0/1548119158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 msgr2=0x5582fe702900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:23.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe702900 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400e660 tx=0x7f1a14004750 comp rx=0 tx=0).stop 2024-07-04T08:26:23.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a31f09740 1 -- 172.21.15.39:0/1548119158 shutdown_connections 2024-07-04T08:26:23.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5582fe310ea0 0x5582fe714820 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe352520 0x5582fe70ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe331840 0x5582fe702e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1548119158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe329660 0x5582fe702900 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a31f09740 1 -- 172.21.15.39:0/1548119158 >> 172.21.15.39:0/1548119158 conn(0x5582fe346910 msgr2=0x5582fe6fa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:23.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.587+0000 7f1a12a00640 0 client.27910 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:23.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.588+0000 7f1a31f09740 1 -- 172.21.15.39:0/1548119158 shutdown_connections 2024-07-04T08:26:23.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.588+0000 7f1a31f09740 1 -- 172.21.15.39:0/1548119158 wait complete. 2024-07-04T08:26:23.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.588+0000 7f1a31f09740 20 client.27910 trim_cache size 0 max 16384 2024-07-04T08:26:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.608+0000 7f1a31f09740 1 Processor -- start 2024-07-04T08:26:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.608+0000 7f1a31f09740 1 -- start start 2024-07-04T08:26:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.608+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe6fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.608+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe6fc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.608+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 0x5582fe6fc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.608+0000 7f1a31f09740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241b25b0 con 0x5582fe331840 2024-07-04T08:26:23.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.608+0000 7f1a31f09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241b40f0 con 0x5582fe329660 2024-07-04T08:26:23.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.608+0000 7f1a31f09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241b2c80 con 0x5582fe352520 2024-07-04T08:26:23.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a28e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe6fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a1be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe6fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a28e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe6fe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43904/0 (socket says 172.21.15.39:43904) 2024-07-04T08:26:23.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a28e00640 1 -- 172.21.15.39:0/1429543160 learned_addr learned my addr 172.21.15.39:0/1429543160 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:23.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a29800640 1 --2- 172.21.15.39:0/1429543160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 0x5582fe6fc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a28e00640 1 -- 172.21.15.39:0/1429543160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 msgr2=0x5582fe6fc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a28e00640 1 --2- 172.21.15.39:0/1429543160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 0x5582fe6fc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a28e00640 1 -- 172.21.15.39:0/1429543160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 msgr2=0x5582fe6fc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a28e00640 1 --2- 172.21.15.39:0/1429543160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe6fc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.609+0000 7f1a28e00640 1 -- 172.21.15.39:0/1429543160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582fe434960 con 0x5582fe329660 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.610+0000 7f1a28e00640 1 --2- 172.21.15.39:0/1429543160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe6fe0e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1a14010b10 tx=0x7f1a1400bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fcb13201b7e91c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.610+0000 7f1a19600640 1 -- 172.21.15.39:0/1429543160 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1402a070 con 0x5582fe329660 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.610+0000 7f1a19600640 1 -- 172.21.15.39:0/1429543160 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a14004520 con 0x5582fe329660 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.610+0000 7f1a19600640 1 -- 172.21.15.39:0/1429543160 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1401a480 con 0x5582fe329660 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.611+0000 7f1a31f09740 1 -- 172.21.15.39:0/1429543160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 msgr2=0x5582fe6fe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.611+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1429543160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe6fe0e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1a14010b10 tx=0x7f1a1400bc70 comp rx=0 tx=0).stop 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.611+0000 7f1a31f09740 1 -- 172.21.15.39:0/1429543160 shutdown_connections 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.611+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1429543160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe6fc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.611+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1429543160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 0x5582fe6fc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.611+0000 7f1a31f09740 1 --2- 172.21.15.39:0/1429543160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe6fe0e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.611+0000 7f1a31f09740 1 -- 172.21.15.39:0/1429543160 >> 172.21.15.39:0/1429543160 conn(0x5582fe346910 msgr2=0x5582fe7047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.611+0000 7f1a31f09740 1 -- 172.21.15.39:0/1429543160 shutdown_connections 2024-07-04T08:26:23.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.611+0000 7f1a31f09740 1 -- 172.21.15.39:0/1429543160 wait complete. 2024-07-04T08:26:23.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.612+0000 7f1a31f09740 1 -- start start 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.612+0000 7f1a31f09740 1 Processor -- start 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe7159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe715ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a31f09740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 0x5582fe77bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a31f09740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241b3a20 con 0x5582fe331840 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a31f09740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241b3350 con 0x5582fe329660 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a31f09740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241b0590 con 0x5582fe352520 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a31f09740 10 client.0 _ll_register_callbacks cb 0x7f1a14010460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a1be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe715ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a28e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe7159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a1be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe715ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40550/0 (socket says 172.21.15.39:40550) 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a1be00640 1 -- 172.21.15.39:0/3729513324 learned_addr learned my addr 172.21.15.39:0/3729513324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:23.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.613+0000 7f1a29800640 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 0x5582fe77bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.614+0000 7f1a28e00640 1 -- 172.21.15.39:0/3729513324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 msgr2=0x5582fe77bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:23.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.614+0000 7f1a28e00640 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 0x5582fe77bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.614+0000 7f1a28e00640 1 -- 172.21.15.39:0/3729513324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 msgr2=0x5582fe715ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:23.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.614+0000 7f1a28e00640 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe715ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.614+0000 7f1a28e00640 1 -- 172.21.15.39:0/3729513324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582fe629880 con 0x5582fe329660 2024-07-04T08:26:23.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.614+0000 7f1a28e00640 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe7159a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400ef90 tx=0x7f1a1400bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52bae1618b5b6624 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:23.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.614+0000 7f1a12a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:23.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.615+0000 7f1a12a00640 1 -- 172.21.15.39:0/3729513324 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1402a040 con 0x5582fe329660 2024-07-04T08:26:23.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.615+0000 7f1a31f09740 1 -- 172.21.15.39:0/3729513324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a14004520 con 0x5582fe329660 2024-07-04T08:26:23.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.615+0000 7f1a12a00640 1 -- 172.21.15.39:0/3729513324 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f1a1401aae0 con 0x5582fe329660 2024-07-04T08:26:23.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.615+0000 7f1a12a00640 1 -- 172.21.15.39:0/3729513324 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a14048050 con 0x5582fe329660 2024-07-04T08:26:23.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.615+0000 7f1a31f09740 10 client.27983 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:23.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.615+0000 7f1a31f09740 1 -- 172.21.15.39:0/3729513324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582fe625430 con 0x5582fe329660 2024-07-04T08:26:23.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.616+0000 7f1a12a00640 1 -- 172.21.15.39:0/3729513324 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a14004520 con 0x5582fe329660 2024-07-04T08:26:23.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.616+0000 7f1a12a00640 1 -- 172.21.15.39:0/3729513324 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1a14005100 con 0x5582fe329660 2024-07-04T08:26:23.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.616+0000 7f1a31f09740 10 client.27983 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:23.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.616+0000 7f1a31f09740 10 client.27983 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:26:23.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.616+0000 7f1a31f09740 20 client.27983 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:23.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.616+0000 7f1a31f09740 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5582fe2e6250 0x5582fe77f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:23.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.616+0000 7f1a31f09740 4 client.27983 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1289c4b8", "release": true, "await": true}] 2024-07-04T08:26:23.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.616+0000 7f1a31f09740 1 -- 172.21.15.39:0/3729513324 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1289c4b8", "release": true, "await": true}) -- 0x5582fe66ac80 con 0x5582fe2e6250 2024-07-04T08:26:23.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.616+0000 7f1a1be00640 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5582fe2e6250 0x5582fe77f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:23.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.617+0000 7f1a12a00640 10 client.27983 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:23.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.617+0000 7f1a1be00640 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5582fe2e6250 0x5582fe77f510 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d5529aecf08216 server_cookie=ad2762ba34ea7691 in_seq=0 out_seq=0 2024-07-04T08:26:23.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.619+0000 7f1a12a00640 1 -- 172.21.15.39:0/3729513324 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5582fe66ac80 con 0x5582fe2e6250 2024-07-04T08:26:23.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.619+0000 7f1a12a00640 10 client.27983 handle_command_reply: tid=0 2024-07-04T08:26:23.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.619+0000 7f1a31f09740 1 client.27983 shutdown 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.619+0000 7f1a31f09740 1 -- 172.21.15.39:0/3729513324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 msgr2=0x5582fe7159a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.619+0000 7f1a31f09740 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe7159a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400ef90 tx=0x7f1a1400bc00 comp rx=0 tx=0).stop 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.619+0000 7f1a31f09740 1 -- 172.21.15.39:0/3729513324 shutdown_connections 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.619+0000 7f1a31f09740 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5582fe2e6250 0x5582fe77f510 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.620+0000 7f1a31f09740 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582fe331840 0x5582fe715ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.620+0000 7f1a12a00640 0 client.27983 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.620+0000 7f1a31f09740 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582fe352520 0x5582fe77bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.620+0000 7f1a31f09740 1 --2- 172.21.15.39:0/3729513324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582fe329660 0x5582fe7159a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.620+0000 7f1a31f09740 1 -- 172.21.15.39:0/3729513324 >> 172.21.15.39:0/3729513324 conn(0x5582fe346910 msgr2=0x5582fe70db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.620+0000 7f1a31f09740 1 -- 172.21.15.39:0/3729513324 shutdown_connections 2024-07-04T08:26:23.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.620+0000 7f1a31f09740 1 -- 172.21.15.39:0/3729513324 wait complete. 2024-07-04T08:26:23.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.620+0000 7f1a31f09740 20 client.27983 trim_cache size 0 max 16384 2024-07-04T08:26:23.613 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:26:23.613 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:26:23.613 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:26:23.613 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 32, 2024-07-04T08:26:23.613 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:26:23.613 INFO:teuthology.orchestra.run.smithi039.stdout: "1289c4b8": { 2024-07-04T08:26:23.613 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 32, 2024-07-04T08:26:23.613 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 150.0, 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:23.614 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:23.615 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:23.615 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:23.615 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:23.615 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:26:23.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 -- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a080b29e0 msgr2=0x7f1a080b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:23.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a080b29e0 0x7f1a080b4ef0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c000c00 tx=0x7f1a0c005780 comp rx=0 tx=0).stop 2024-07-04T08:26:23.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 -- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a24108260 msgr2=0x7f1a240ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:23.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a24108260 0x7f1a240ff660 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009fd0 tx=0x7f1a1c00ef10 comp rx=0 tx=0).stop 2024-07-04T08:26:23.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 -- 172.21.15.39:0/2973058540 shutdown_connections 2024-07-04T08:26:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1a080b29e0 0x7f1a080b4ef0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1a24108260 0x7f1a240ff660 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a24107910 0x7f1a240ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 --2- 172.21.15.39:0/2973058540 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a24101fd0 0x7f1a240febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.621+0000 7f1a2b800640 1 -- 172.21.15.39:0/2973058540 >> 172.21.15.39:0/2973058540 conn(0x7f1a24078c90 msgr2=0x7f1a2406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.622+0000 7f1a2b800640 1 -- 172.21.15.39:0/2973058540 shutdown_connections 2024-07-04T08:26:23.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:23.622+0000 7f1a2b800640 1 -- 172.21.15.39:0/2973058540 wait complete. 2024-07-04T08:26:23.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:23 smithi050 ceph-mon[36577]: pgmap v825: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:26:23.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:23 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:26:24.665 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1289c4b8', total seconds elapsed: 40.6 2024-07-04T08:26:24.665 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-07-04T08:26:24.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:26:25.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:25 smithi039 ceph-mon[32062]: pgmap v826: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:26:25.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:25 smithi133 ceph-mon[37050]: pgmap v826: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:26:25.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:25 smithi050 ceph-mon[36577]: pgmap v826: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:26:25.782 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:26.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.824+0000 7ff4eb800640 1 -- 172.21.15.39:0/2679704867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4e410b570 msgr2=0x7ff4e4111da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:26.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.824+0000 7ff4eb800640 1 --2- 172.21.15.39:0/2679704867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4e410b570 0x7ff4e4111da0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009be0 tx=0x7ff4dc02a6c0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.824+0000 7ff4eb800640 1 -- 172.21.15.39:0/2679704867 shutdown_connections 2024-07-04T08:26:26.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.824+0000 7ff4eb800640 1 --2- 172.21.15.39:0/2679704867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4e410b570 0x7ff4e4111da0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.824+0000 7ff4eb800640 1 --2- 172.21.15.39:0/2679704867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e410ac20 0x7ff4e410b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.824+0000 7ff4eb800640 1 --2- 172.21.15.39:0/2679704867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e41052e0 0x7ff4e41056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.824+0000 7ff4eb800640 1 -- 172.21.15.39:0/2679704867 >> 172.21.15.39:0/2679704867 conn(0x7ff4e4100d10 msgr2=0x7ff4e4103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:26.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.825+0000 7ff4eb800640 1 -- 172.21.15.39:0/2679704867 shutdown_connections 2024-07-04T08:26:26.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.825+0000 7ff4eb800640 1 -- 172.21.15.39:0/2679704867 wait complete. 2024-07-04T08:26:26.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.826+0000 7ff4eb800640 1 Processor -- start 2024-07-04T08:26:26.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.826+0000 7ff4eb800640 1 -- start start 2024-07-04T08:26:26.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4eb800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e41052e0 0x7ff4e419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4eb800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e410ac20 0x7ff4e419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4eb800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4e410b570 0x7ff4e41a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4eb800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4115220 con 0x7ff4e41052e0 2024-07-04T08:26:26.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4eb800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41150a0 con 0x7ff4e410b570 2024-07-04T08:26:26.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4eb800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41153a0 con 0x7ff4e410ac20 2024-07-04T08:26:26.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4e8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e41052e0 0x7ff4e419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4e8e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e41052e0 0x7ff4e419bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40562/0 (socket says 172.21.15.39:40562) 2024-07-04T08:26:26.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4e8e00640 1 -- 172.21.15.39:0/226506101 learned_addr learned my addr 172.21.15.39:0/226506101 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:26.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.827+0000 7ff4e3e00640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e410ac20 0x7ff4e419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.828+0000 7ff4e9800640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4e410b570 0x7ff4e41a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.828+0000 7ff4e8e00640 1 -- 172.21.15.39:0/226506101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e410ac20 msgr2=0x7ff4e419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:26.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.828+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e410ac20 0x7ff4e419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.828+0000 7ff4e8e00640 1 -- 172.21.15.39:0/226506101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4e410b570 msgr2=0x7ff4e41a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:26.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.828+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4e410b570 0x7ff4e41a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.828+0000 7ff4e8e00640 1 -- 172.21.15.39:0/226506101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e41a1ae0 con 0x7ff4e41052e0 2024-07-04T08:26:26.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.828+0000 7ff4e3e00640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e410ac20 0x7ff4e419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:26.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.828+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e41052e0 0x7ff4e419bb70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800e9c0 tx=0x7ff4d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:26.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.829+0000 7ff4e1600640 1 -- 172.21.15.39:0/226506101 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d800cde0 con 0x7ff4e41052e0 2024-07-04T08:26:26.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.829+0000 7ff4e1600640 1 -- 172.21.15.39:0/226506101 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d80035f0 con 0x7ff4e41052e0 2024-07-04T08:26:26.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.829+0000 7ff4eb800640 1 -- 172.21.15.39:0/226506101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e41a1dd0 con 0x7ff4e41052e0 2024-07-04T08:26:26.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.829+0000 7ff4e1600640 1 -- 172.21.15.39:0/226506101 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8010690 con 0x7ff4e41052e0 2024-07-04T08:26:26.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.829+0000 7ff4eb800640 1 -- 172.21.15.39:0/226506101 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41a8530 con 0x7ff4e41052e0 2024-07-04T08:26:26.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.832+0000 7ff4f1eee740 1 Processor -- start 2024-07-04T08:26:26.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.832+0000 7ff4f1eee740 1 -- start start 2024-07-04T08:26:26.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a05770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1986381b0 0x55d198a05cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d19897bea0 con 0x55d1986381b0 2024-07-04T08:26:26.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d19896b440 con 0x55d198640390 2024-07-04T08:26:26.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d19896f480 con 0x55d198661070 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4e1600640 1 -- 172.21.15.39:0/226506101 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff4d8004000 con 0x7ff4e41052e0 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4e9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a05770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43940/0 (socket says 172.21.15.39:43940) 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.833+0000 7ff4e3e00640 1 -- 172.21.15.39:0/836415319 learned_addr learned my addr 172.21.15.39:0/836415319 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.834+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/836415319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1986381b0 0x55d198a05cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.834+0000 7ff4e3e00640 1 -- 172.21.15.39:0/836415319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 msgr2=0x55d198a05770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.834+0000 7ff4e3e00640 1 --2- 172.21.15.39:0/836415319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a05770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.834+0000 7ff4e3e00640 1 -- 172.21.15.39:0/836415319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1986381b0 msgr2=0x55d198a05cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.834+0000 7ff4e3e00640 1 --2- 172.21.15.39:0/836415319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1986381b0 0x55d198a05cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.834+0000 7ff4e3e00640 1 -- 172.21.15.39:0/836415319 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d198903610 con 0x55d198640390 2024-07-04T08:26:26.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.834+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/836415319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1986381b0 0x55d198a05cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:26:26.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.834+0000 7ff4e3e00640 1 --2- 172.21.15.39:0/836415319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a061f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0011470 tx=0x7ff4d0011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8344cb6a266439 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:26.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.835+0000 7ff4d6000640 1 -- 172.21.15.39:0/836415319 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d001e070 con 0x55d198640390 2024-07-04T08:26:26.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.835+0000 7ff4d6000640 1 -- 172.21.15.39:0/836415319 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d00035f0 con 0x55d198640390 2024-07-04T08:26:26.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.835+0000 7ff4d6000640 1 -- 172.21.15.39:0/836415319 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000f860 con 0x55d198640390 2024-07-04T08:26:26.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4e1600640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4cc0b2960 0x7ff4cc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4f1eee740 1 -- 172.21.15.39:0/836415319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 msgr2=0x55d198a061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:26.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/836415319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a061f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0011470 tx=0x7ff4d0011930 comp rx=0 tx=0).stop 2024-07-04T08:26:26.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4e1600640 1 -- 172.21.15.39:0/226506101 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7ff4d80e2c90 con 0x7ff4e41052e0 2024-07-04T08:26:26.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4f1eee740 1 -- 172.21.15.39:0/836415319 shutdown_connections 2024-07-04T08:26:26.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/836415319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d1986381b0 0x55d198a05cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/836415319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a061f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/836415319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a05770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4e9800640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4cc0b2960 0x7ff4cc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4f1eee740 1 -- 172.21.15.39:0/836415319 >> 172.21.15.39:0/836415319 conn(0x55d198655460 msgr2=0x55d198a03660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:26.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4f1eee740 1 -- 172.21.15.39:0/836415319 shutdown_connections 2024-07-04T08:26:26.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.836+0000 7ff4f1eee740 1 -- 172.21.15.39:0/836415319 wait complete. 2024-07-04T08:26:26.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.837+0000 7ff4e9800640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4cc0b2960 0x7ff4cc0b4e70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009370 tx=0x7ff4dc004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:26.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.837+0000 7ff4f1eee740 1 -- start start 2024-07-04T08:26:26.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.837+0000 7ff4f1eee740 1 Processor -- start 2024-07-04T08:26:26.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.838+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.838+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a11820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.838+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198661070 0x55d198a1d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.838+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d1987cf700 con 0x55d198661070 2024-07-04T08:26:26.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.838+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d198912b70 con 0x55d198640390 2024-07-04T08:26:26.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.838+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d198a13f50 con 0x55d1986381b0 2024-07-04T08:26:26.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.838+0000 7ff4f1eee740 10 client.0 _ll_register_callbacks cb 0x55d198964530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.838+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a11820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.838+0000 7ff4e9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198661070 0x55d198a1d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a11820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:43946/0 (socket says 172.21.15.39:43946) 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4e3e00640 1 -- 172.21.15.39:0/2245845442 learned_addr learned my addr 172.21.15.39:0/2245845442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4e3e00640 1 -- 172.21.15.39:0/2245845442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 msgr2=0x55d198a112e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4e3e00640 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a112e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4e3e00640 1 -- 172.21.15.39:0/2245845442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198661070 msgr2=0x55d198a1d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4e3e00640 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198661070 0x55d198a1d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4e3e00640 1 -- 172.21.15.39:0/2245845442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1986cc650 con 0x55d198640390 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4e3e00640 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a11820 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009560 tx=0x7ff4d00042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d80a5b3bbfb0fe3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.839+0000 7ff4caa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.840+0000 7ff4caa00640 1 -- 172.21.15.39:0/2245845442 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d001e040 con 0x55d198640390 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.840+0000 7ff4f1eee740 1 -- 172.21.15.39:0/2245845442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d00035f0 con 0x55d198640390 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.840+0000 7ff4caa00640 1 -- 172.21.15.39:0/2245845442 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7ff4d0005040 con 0x55d198640390 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.840+0000 7ff4caa00640 1 -- 172.21.15.39:0/2245845442 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4d0032050 con 0x55d198640390 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.840+0000 7ff4f1eee740 10 client.27995 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.840+0000 7ff4f1eee740 1 -- 172.21.15.39:0/2245845442 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d198a1fe50 con 0x55d198640390 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.840+0000 7ff4caa00640 1 -- 172.21.15.39:0/2245845442 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d00035f0 con 0x55d198640390 2024-07-04T08:26:26.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.841+0000 7ff4caa00640 1 -- 172.21.15.39:0/2245845442 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff4d0021030 con 0x55d198640390 2024-07-04T08:26:26.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.841+0000 7ff4f1eee740 10 client.27995 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:26.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.841+0000 7ff4f1eee740 10 client.27995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:26.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.841+0000 7ff4f1eee740 20 client.27995 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:26.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.841+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d19861f9f0 0x55d198a23210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.841+0000 7ff4f1eee740 4 client.27995 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:26.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.841+0000 7ff4f1eee740 1 -- 172.21.15.39:0/2245845442 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d19897ddf0 con 0x55d19861f9f0 2024-07-04T08:26:26.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.841+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d19861f9f0 0x55d198a23210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.842+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d19861f9f0 0x55d198a23210 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37868072609c74da server_cookie=d5625d3884264733 in_seq=0 out_seq=0 2024-07-04T08:26:26.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.842+0000 7ff4caa00640 10 client.27995 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:26.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4caa00640 1 -- 172.21.15.39:0/2245845442 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d19897ddf0 con 0x55d19861f9f0 2024-07-04T08:26:26.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4caa00640 10 client.27995 handle_command_reply: tid=0 2024-07-04T08:26:26.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 client.27995 shutdown 2024-07-04T08:26:26.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 -- 172.21.15.39:0/2245845442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 msgr2=0x55d198a11820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:26.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a11820 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009560 tx=0x7ff4d00042b0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 -- 172.21.15.39:0/2245845442 shutdown_connections 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d19861f9f0 0x55d198a23210 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198661070 0x55d198a1d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198640390 0x55d198a11820 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/2245845442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4caa00640 0 client.27995 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 -- 172.21.15.39:0/2245845442 >> 172.21.15.39:0/2245845442 conn(0x55d198655460 msgr2=0x55d198a08900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 -- 172.21.15.39:0/2245845442 shutdown_connections 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.843+0000 7ff4f1eee740 1 -- 172.21.15.39:0/2245845442 wait complete. 2024-07-04T08:26:26.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.844+0000 7ff4f1eee740 20 client.27995 trim_cache size 0 max 16384 2024-07-04T08:26:26.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.865+0000 7ff4f1eee740 1 Processor -- start 2024-07-04T08:26:26.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.865+0000 7ff4f1eee740 1 -- start start 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.865+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a02900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.865+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 0x55d198a12d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.865+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198661070 0x55d198a0a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.865+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b2570 con 0x55d198640390 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.865+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b40b0 con 0x55d198661070 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.865+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b2c40 con 0x55d1986381b0 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a02900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 0x55d198a12d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a02900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59668/0 (socket says 172.21.15.39:59668) 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e8e00640 1 -- 172.21.15.39:0/3201894406 learned_addr learned my addr 172.21.15.39:0/3201894406 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:26.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e9800640 1 --2- 172.21.15.39:0/3201894406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198661070 0x55d198a0a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e8e00640 1 -- 172.21.15.39:0/3201894406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198661070 msgr2=0x55d198a0a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:26.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/3201894406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198661070 0x55d198a0a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e8e00640 1 -- 172.21.15.39:0/3201894406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 msgr2=0x55d198a12d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:26.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/3201894406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 0x55d198a12d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e8e00640 1 -- 172.21.15.39:0/3201894406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d198a0bd60 con 0x55d1986381b0 2024-07-04T08:26:26.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.866+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/3201894406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a02900 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff4d80a5c70 tx=0x7ff4d80e5120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebfabb57045b7057 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:26.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.867+0000 7ff4d5600640 1 -- 172.21.15.39:0/3201894406 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d80b23a0 con 0x55d1986381b0 2024-07-04T08:26:26.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.867+0000 7ff4d5600640 1 -- 172.21.15.39:0/3201894406 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d80aa2f0 con 0x55d1986381b0 2024-07-04T08:26:26.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.867+0000 7ff4d5600640 1 -- 172.21.15.39:0/3201894406 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d80037a0 con 0x55d1986381b0 2024-07-04T08:26:26.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 -- 172.21.15.39:0/3201894406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 msgr2=0x55d198a02900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:26.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/3201894406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a02900 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff4d80a5c70 tx=0x7ff4d80e5120 comp rx=0 tx=0).stop 2024-07-04T08:26:26.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 -- 172.21.15.39:0/3201894406 shutdown_connections 2024-07-04T08:26:26.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/3201894406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 0x55d198a12d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/3201894406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d198661070 0x55d198a0a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/3201894406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1986381b0 0x55d198a02900 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 -- 172.21.15.39:0/3201894406 >> 172.21.15.39:0/3201894406 conn(0x55d198655460 msgr2=0x55d198a12600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:26.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 -- 172.21.15.39:0/3201894406 shutdown_connections 2024-07-04T08:26:26.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 -- 172.21.15.39:0/3201894406 wait complete. 2024-07-04T08:26:26.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.868+0000 7ff4f1eee740 1 -- start start 2024-07-04T08:26:26.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.869+0000 7ff4f1eee740 1 Processor -- start 2024-07-04T08:26:26.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.869+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d1986381b0 0x55d198a19800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.869+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 0x55d198a1ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.869+0000 7ff4f1eee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a1cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.869+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b39e0 con 0x55d198640390 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.869+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b3310 con 0x55d1986381b0 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.869+0000 7ff4f1eee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e41b0550 con 0x55d198661070 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.869+0000 7ff4f1eee740 10 client.0 _ll_register_callbacks cb 0x7ff4d001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a1cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 0x55d198a1ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d1986381b0 0x55d198a19800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a1cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:59680/0 (socket says 172.21.15.39:59680) 2024-07-04T08:26:26.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e9800640 1 -- 172.21.15.39:0/408922502 learned_addr learned my addr 172.21.15.39:0/408922502 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:26.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e9800640 1 -- 172.21.15.39:0/408922502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d1986381b0 msgr2=0x55d198a19800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:26.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e9800640 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d1986381b0 0x55d198a19800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e9800640 1 -- 172.21.15.39:0/408922502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 msgr2=0x55d198a1ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:26.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e9800640 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 0x55d198a1ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.870+0000 7ff4e9800640 1 -- 172.21.15.39:0/408922502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d198a1d620 con 0x55d198661070 2024-07-04T08:26:26.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.871+0000 7ff4e9800640 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a1cf80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc02acb0 tx=0x7ff4dc009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50b1a6ee3bfec84 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:26.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.871+0000 7ff4caa00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:26.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.871+0000 7ff4caa00640 1 -- 172.21.15.39:0/408922502 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc036040 con 0x55d198661070 2024-07-04T08:26:26.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.871+0000 7ff4f1eee740 1 -- 172.21.15.39:0/408922502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d80aa2f0 con 0x55d198661070 2024-07-04T08:26:26.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.871+0000 7ff4caa00640 1 -- 172.21.15.39:0/408922502 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7ff4dc004240 con 0x55d198661070 2024-07-04T08:26:26.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.871+0000 7ff4caa00640 1 -- 172.21.15.39:0/408922502 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff4dc05e050 con 0x55d198661070 2024-07-04T08:26:26.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.871+0000 7ff4f1eee740 10 client.27934 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.871+0000 7ff4f1eee740 1 -- 172.21.15.39:0/408922502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d198a16050 con 0x55d198661070 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.872+0000 7ff4caa00640 1 -- 172.21.15.39:0/408922502 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d80aa2f0 con 0x55d198661070 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.872+0000 7ff4caa00640 1 -- 172.21.15.39:0/408922502 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff4dc053020 con 0x55d198661070 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.872+0000 7ff4f1eee740 10 client.27934 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.872+0000 7ff4f1eee740 10 client.27934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.872+0000 7ff4f1eee740 20 client.27934 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.872+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d1985f4da0 0x55d198a8ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.872+0000 7ff4f1eee740 4 client.27934 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.872+0000 7ff4f1eee740 1 -- 172.21.15.39:0/408922502 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d198979890 con 0x55d1985f4da0 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.872+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d1985f4da0 0x55d198a8ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4e8e00640 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d1985f4da0 0x55d198a8ddf0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404d5d272d092ad4 server_cookie=4f38d3d57ad4d413 in_seq=0 out_seq=0 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4caa00640 10 client.27934 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:26.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4caa00640 1 -- 172.21.15.39:0/408922502 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d198979890 con 0x55d1985f4da0 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4caa00640 10 client.27934 handle_command_reply: tid=0 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4f1eee740 1 client.27934 shutdown 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4f1eee740 1 -- 172.21.15.39:0/408922502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 msgr2=0x55d198a1cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a1cf80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc02acb0 tx=0x7ff4dc009500 comp rx=0 tx=0).stop 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4f1eee740 1 -- 172.21.15.39:0/408922502 shutdown_connections 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d1985f4da0 0x55d198a8ddf0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d198640390 0x55d198a1ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d198661070 0x55d198a1cf80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4f1eee740 1 --2- 172.21.15.39:0/408922502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d1986381b0 0x55d198a19800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4caa00640 0 client.27934 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.873+0000 7ff4f1eee740 1 -- 172.21.15.39:0/408922502 >> 172.21.15.39:0/408922502 conn(0x55d198655460 msgr2=0x55d198a12590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.874+0000 7ff4f1eee740 1 -- 172.21.15.39:0/408922502 shutdown_connections 2024-07-04T08:26:26.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.874+0000 7ff4f1eee740 1 -- 172.21.15.39:0/408922502 wait complete. 2024-07-04T08:26:26.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.874+0000 7ff4f1eee740 20 client.27934 trim_cache size 0 max 16384 2024-07-04T08:26:26.867 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:26:26.867 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:26:26.867 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:26:26.867 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 -- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4cc0b2960 msgr2=0x7ff4cc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4cc0b2960 0x7ff4cc0b4e70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009370 tx=0x7ff4dc004a40 comp rx=0 tx=0).stop 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 -- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e41052e0 msgr2=0x7ff4e419bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e41052e0 0x7ff4e419bb70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800e9c0 tx=0x7ff4d800ee80 comp rx=0 tx=0).stop 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 -- 172.21.15.39:0/226506101 shutdown_connections 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4cc0b2960 0x7ff4cc0b4e70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4e410b570 0x7ff4e41a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e410ac20 0x7ff4e419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 --2- 172.21.15.39:0/226506101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4e41052e0 0x7ff4e419bb70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:26.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 -- 172.21.15.39:0/226506101 >> 172.21.15.39:0/226506101 conn(0x7ff4e4100d10 msgr2=0x7ff4e410a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 -- 172.21.15.39:0/226506101 shutdown_connections 2024-07-04T08:26:26.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:26.875+0000 7ff4eb800640 1 -- 172.21.15.39:0/226506101 wait complete. 2024-07-04T08:26:27.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:27 smithi133 ceph-mon[37050]: pgmap v827: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-07-04T08:26:27.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:27 smithi039 ceph-mon[32062]: pgmap v827: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-07-04T08:26:27.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:27 smithi050 ceph-mon[36577]: pgmap v827: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-07-04T08:26:28.142 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:26:28.142 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:26:28.744 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:29.420 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:29 smithi039 ceph-mon[32062]: pgmap v828: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-07-04T08:26:29.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.475+0000 7f03c3e00640 1 -- 172.21.15.39:0/2087185065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c410f7a0 msgr2=0x7f03c4115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:29.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.475+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2087185065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c410f7a0 0x7f03c4115fd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009ca0 tx=0x7f03bc02a8b0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.475+0000 7f03c3e00640 1 -- 172.21.15.39:0/2087185065 shutdown_connections 2024-07-04T08:26:29.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.475+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2087185065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c410f7a0 0x7f03c4115fd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.475+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2087185065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c410ee50 0x7f03c410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.475+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2087185065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03c4109510 0x7f03c4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.475+0000 7f03c3e00640 1 -- 172.21.15.39:0/2087185065 >> 172.21.15.39:0/2087185065 conn(0x7f03c4078b10 msgr2=0x7f03c4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:29.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.475+0000 7f03c3e00640 1 -- 172.21.15.39:0/2087185065 shutdown_connections 2024-07-04T08:26:29.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.475+0000 7f03c3e00640 1 -- 172.21.15.39:0/2087185065 wait complete. 2024-07-04T08:26:29.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.477+0000 7f03c3e00640 1 Processor -- start 2024-07-04T08:26:29.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.477+0000 7f03c3e00640 1 -- start start 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.477+0000 7f03c3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c4109510 0x7f03c419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.477+0000 7f03c3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c410ee50 0x7f03c41a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.477+0000 7f03c3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03c410f7a0 0x7f03c41a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.477+0000 7f03c3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f03c4119450 con 0x7f03c4109510 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.477+0000 7f03c3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41192d0 con 0x7f03c410f7a0 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.477+0000 7f03c3e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f03c41195d0 con 0x7f03c410ee50 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c4109510 0x7f03c419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c4109510 0x7f03c419fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:40616/0 (socket says 172.21.15.39:40616) 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c2a00640 1 -- 172.21.15.39:0/2546714907 learned_addr learned my addr 172.21.15.39:0/2546714907 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c3400640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03c410f7a0 0x7f03c41a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:29.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c2000640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c410ee50 0x7f03c41a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:29.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c2a00640 1 -- 172.21.15.39:0/2546714907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c410ee50 msgr2=0x7f03c41a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:29.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c2a00640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c410ee50 0x7f03c41a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c2a00640 1 -- 172.21.15.39:0/2546714907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03c410f7a0 msgr2=0x7f03c41a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:29.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c2a00640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03c410f7a0 0x7f03c41a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c2a00640 1 -- 172.21.15.39:0/2546714907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c41a5df0 con 0x7f03c4109510 2024-07-04T08:26:29.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.478+0000 7f03c3400640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03c410f7a0 0x7f03c41a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:29.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.479+0000 7f03c2a00640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c4109510 0x7f03c419fe80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e9f0 tx=0x7f03b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:29.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.479+0000 7f03bb400640 1 -- 172.21.15.39:0/2546714907 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b400cd80 con 0x7f03c4109510 2024-07-04T08:26:29.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.479+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c41a60e0 con 0x7f03c4109510 2024-07-04T08:26:29.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.479+0000 7f03bb400640 1 -- 172.21.15.39:0/2546714907 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b40108e0 con 0x7f03c4109510 2024-07-04T08:26:29.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.479+0000 7f03bb400640 1 -- 172.21.15.39:0/2546714907 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b4018980 con 0x7f03c4109510 2024-07-04T08:26:29.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.479+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c41ac840 con 0x7f03c4109510 2024-07-04T08:26:29.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.481+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a40052c0 con 0x7f03c4109510 2024-07-04T08:26:29.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.483+0000 7f03bb400640 1 -- 172.21.15.39:0/2546714907 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f03b4010430 con 0x7f03c4109510 2024-07-04T08:26:29.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.487+0000 7f03bb400640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f03a80b29e0 0x7f03a80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:29.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.488+0000 7f03c2000640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f03a80b29e0 0x7f03a80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:29.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.488+0000 7f03bb400640 1 -- 172.21.15.39:0/2546714907 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f03b40e2aa0 con 0x7f03c4109510 2024-07-04T08:26:29.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.489+0000 7f03c2000640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f03a80b29e0 0x7f03a80b4ef0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f03c410f580 tx=0x7f03b0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:29.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.490+0000 7f03bb400640 1 -- 172.21.15.39:0/2546714907 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f03b40a9a70 con 0x7f03c4109510 2024-07-04T08:26:29.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:29 smithi133 ceph-mon[37050]: pgmap v828: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-07-04T08:26:29.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:29 smithi050 ceph-mon[36577]: pgmap v828: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-07-04T08:26:29.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.752+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03a4005880 con 0x7f03c4109510 2024-07-04T08:26:29.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.754+0000 7f03bb400640 1 -- 172.21.15.39:0/2546714907 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f03b40add70 con 0x7f03c4109510 2024-07-04T08:26:29.744 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:26:29.745 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:26:29.745 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:26:29.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f03a80b29e0 msgr2=0x7f03a80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:29.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f03a80b29e0 0x7f03a80b4ef0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f03c410f580 tx=0x7f03b0005f90 comp rx=0 tx=0).stop 2024-07-04T08:26:29.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c4109510 msgr2=0x7f03c419fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:29.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c4109510 0x7f03c419fe80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f03b400e9f0 tx=0x7f03b400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 shutdown_connections 2024-07-04T08:26:29.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f03a80b29e0 0x7f03a80b4ef0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f03c410f7a0 0x7f03c41a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c410ee50 0x7f03c41a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 --2- 172.21.15.39:0/2546714907 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03c4109510 0x7f03c419fe80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:29.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 >> 172.21.15.39:0/2546714907 conn(0x7f03c4078b10 msgr2=0x7f03c41079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:29.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 shutdown_connections 2024-07-04T08:26:29.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:29.757+0000 7f03c3e00640 1 -- 172.21.15.39:0/2546714907 wait complete. 2024-07-04T08:26:30.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:30 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2546714907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:30.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:30 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2546714907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:30.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:30 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2546714907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:30.730 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-07-04T08:26:30.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-07-04T08:26:31.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:31 smithi133 ceph-mon[37050]: pgmap v829: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-07-04T08:26:31.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:31 smithi039 ceph-mon[32062]: pgmap v829: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-07-04T08:26:31.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:31 smithi050 ceph-mon[36577]: pgmap v829: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-07-04T08:26:32.052 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:32.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.170+0000 7fa8c4400640 1 -- 172.21.15.39:0/594305287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc075320 msgr2=0x7fa8bc075730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:32.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.170+0000 7fa8c4400640 1 --2- 172.21.15.39:0/594305287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc075320 0x7fa8bc075730 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009900 tx=0x7fa8a802a410 comp rx=0 tx=0).stop 2024-07-04T08:26:32.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.170+0000 7fa8c4400640 1 -- 172.21.15.39:0/594305287 shutdown_connections 2024-07-04T08:26:32.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.170+0000 7fa8c4400640 1 --2- 172.21.15.39:0/594305287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc075320 0x7fa8bc075730 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:32.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.170+0000 7fa8c4400640 1 --2- 172.21.15.39:0/594305287 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8bc113990 0x7fa8bc115e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:32.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.170+0000 7fa8c4400640 1 --2- 172.21.15.39:0/594305287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8bc0768d0 0x7fa8bc074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:32.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.170+0000 7fa8c4400640 1 -- 172.21.15.39:0/594305287 >> 172.21.15.39:0/594305287 conn(0x7fa8bc0feb30 msgr2=0x7fa8bc100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:32.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.171+0000 7fa8c4400640 1 -- 172.21.15.39:0/594305287 shutdown_connections 2024-07-04T08:26:32.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.171+0000 7fa8c4400640 1 -- 172.21.15.39:0/594305287 wait complete. 2024-07-04T08:26:32.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.172+0000 7fa8c4400640 1 Processor -- start 2024-07-04T08:26:32.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.172+0000 7fa8c4400640 1 -- start start 2024-07-04T08:26:32.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c4400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8bc075320 0x7fa8bc1a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c4400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc0768d0 0x7fa8bc1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c4400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8bc113990 0x7fa8bc1a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c4400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc1195a0 con 0x7fa8bc0768d0 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c4400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc119420 con 0x7fa8bc075320 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c4400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc119720 con 0x7fa8bc113990 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc0768d0 0x7fa8bc1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c1a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8bc075320 0x7fa8bc1a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc0768d0 0x7fa8bc1a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52216/0 (socket says 172.21.15.39:52216) 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c1000640 1 -- 172.21.15.39:0/3688847526 learned_addr learned my addr 172.21.15.39:0/3688847526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:32.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.173+0000 7fa8c2400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8bc113990 0x7fa8bc1a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:32.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.174+0000 7fa8c2400640 1 -- 172.21.15.39:0/3688847526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8bc075320 msgr2=0x7fa8bc1a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:32.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.174+0000 7fa8c2400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8bc075320 0x7fa8bc1a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.174+0000 7fa8c2400640 1 -- 172.21.15.39:0/3688847526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc0768d0 msgr2=0x7fa8bc1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.174+0000 7fa8c2400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc0768d0 0x7fa8bc1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.174+0000 7fa8c2400640 1 -- 172.21.15.39:0/3688847526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc1a5fa0 con 0x7fa8bc113990 2024-07-04T08:26:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.174+0000 7fa8c1a00640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8bc075320 0x7fa8bc1a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.174+0000 7fa8c2400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8bc113990 0x7fa8bc1a5870 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009fd0 tx=0x7fa8b400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.175+0000 7fa8ba000640 1 -- 172.21.15.39:0/3688847526 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b400ce20 con 0x7fa8bc113990 2024-07-04T08:26:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.175+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc1a6290 con 0x7fa8bc113990 2024-07-04T08:26:32.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.175+0000 7fa8ba000640 1 -- 172.21.15.39:0/3688847526 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b4005020 con 0x7fa8bc113990 2024-07-04T08:26:32.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.175+0000 7fa8ba000640 1 -- 172.21.15.39:0/3688847526 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4010690 con 0x7fa8bc113990 2024-07-04T08:26:32.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.175+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc1ac9f0 con 0x7fa8bc113990 2024-07-04T08:26:32.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.176+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc075730 con 0x7fa8bc113990 2024-07-04T08:26:32.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.179+0000 7fa8ba000640 1 -- 172.21.15.39:0/3688847526 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa8b40054d0 con 0x7fa8bc113990 2024-07-04T08:26:32.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.180+0000 7fa8ba000640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8a40b2960 0x7fa8a40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:32.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.180+0000 7fa8ba000640 1 -- 172.21.15.39:0/3688847526 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fa8b40e2ff0 con 0x7fa8bc113990 2024-07-04T08:26:32.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.180+0000 7fa8c1a00640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8a40b2960 0x7fa8a40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:32.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.181+0000 7fa8c1a00640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8a40b2960 0x7fa8a40b4e70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009fc0 tx=0x7fa8b0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:32.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.182+0000 7fa8ba000640 1 -- 172.21.15.39:0/3688847526 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fa8b40aa630 con 0x7fa8bc113990 2024-07-04T08:26:32.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.443+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8bc1a1160 con 0x7fa8bc113990 2024-07-04T08:26:32.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.444+0000 7fa8ba000640 1 -- 172.21.15.39:0/3688847526 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fa8b40a9f30 con 0x7fa8bc113990 2024-07-04T08:26:32.435 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:26:32.435 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:26:32.435 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:26:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8a40b2960 msgr2=0x7fa8a40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8a40b2960 0x7fa8a40b4e70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009fc0 tx=0x7fa8b0005f90 comp rx=0 tx=0).stop 2024-07-04T08:26:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8bc113990 msgr2=0x7fa8bc1a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8bc113990 0x7fa8bc1a5870 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009fd0 tx=0x7fa8b400ef10 comp rx=0 tx=0).stop 2024-07-04T08:26:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 shutdown_connections 2024-07-04T08:26:32.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa8a40b2960 0x7fa8a40b4e70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:32.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8bc0768d0 0x7fa8bc1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:32.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8bc113990 0x7fa8bc1a5870 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:32.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 --2- 172.21.15.39:0/3688847526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa8bc075320 0x7fa8bc1a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:32.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.447+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 >> 172.21.15.39:0/3688847526 conn(0x7fa8bc0feb30 msgr2=0x7fa8bc10f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:32.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.448+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 shutdown_connections 2024-07-04T08:26:32.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:32.448+0000 7fa8c4400640 1 -- 172.21.15.39:0/3688847526 wait complete. 2024-07-04T08:26:33.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:33 smithi039 ceph-mon[32062]: pgmap v830: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-07-04T08:26:33.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:33 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3688847526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:33.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:33 smithi133 ceph-mon[37050]: pgmap v830: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-07-04T08:26:33.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:33 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3688847526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:33.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:33 smithi050 ceph-mon[36577]: pgmap v830: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-07-04T08:26:33.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:33 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3688847526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:34.223 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:26:34.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.911+0000 7fdda3200640 1 -- 172.21.15.39:0/2766272169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda410f500 msgr2=0x7fdda4111920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:34.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.911+0000 7fdda3200640 1 --2- 172.21.15.39:0/2766272169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda410f500 0x7fdda4111920 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009ca0 tx=0x7fdd9c02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.911+0000 7fdda3200640 1 -- 172.21.15.39:0/2766272169 shutdown_connections 2024-07-04T08:26:34.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.911+0000 7fdda3200640 1 --2- 172.21.15.39:0/2766272169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda410f500 0x7fdda4111920 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.911+0000 7fdda3200640 1 --2- 172.21.15.39:0/2766272169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda40755d0 0x7fdda410efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.911+0000 7fdda3200640 1 --2- 172.21.15.39:0/2766272169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdda4074c80 0x7fdda4075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.911+0000 7fdda3200640 1 -- 172.21.15.39:0/2766272169 >> 172.21.15.39:0/2766272169 conn(0x7fdda40fe970 msgr2=0x7fdda4100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:34.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.912+0000 7fdda3200640 1 -- 172.21.15.39:0/2766272169 shutdown_connections 2024-07-04T08:26:34.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.912+0000 7fdda3200640 1 -- 172.21.15.39:0/2766272169 wait complete. 2024-07-04T08:26:34.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.913+0000 7fdda3200640 1 Processor -- start 2024-07-04T08:26:34.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.913+0000 7fdda3200640 1 -- start start 2024-07-04T08:26:34.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.913+0000 7fdda3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdda4074c80 0x7fdda419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.913+0000 7fdda3200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda40755d0 0x7fdda419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.913+0000 7fdda3200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda410f500 0x7fdda41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda3200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4115050 con 0x7fdda40755d0 2024-07-04T08:26:34.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda3200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4114ed0 con 0x7fdda410f500 2024-07-04T08:26:34.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41151d0 con 0x7fdda4074c80 2024-07-04T08:26:34.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdda4074c80 0x7fdda419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdda4074c80 0x7fdda419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51588/0 (socket says 172.21.15.39:51588) 2024-07-04T08:26:34.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda1e00640 1 -- 172.21.15.39:0/543638521 learned_addr learned my addr 172.21.15.39:0/543638521 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:34.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda1400640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda40755d0 0x7fdda419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda2800640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda410f500 0x7fdda41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda2800640 1 -- 172.21.15.39:0/543638521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdda4074c80 msgr2=0x7fdda419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:34.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda2800640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdda4074c80 0x7fdda419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda2800640 1 -- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda40755d0 msgr2=0x7fdda419bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:34.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda2800640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda40755d0 0x7fdda419bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda2800640 1 -- 172.21.15.39:0/543638521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda41a1990 con 0x7fdda410f500 2024-07-04T08:26:34.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.914+0000 7fdda1e00640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdda4074c80 0x7fdda419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:34.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.915+0000 7fdda2800640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda410f500 0x7fdda41a1260 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c02ade0 tx=0x7fdd9c031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:34.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.915+0000 7fdd9aa00640 1 -- 172.21.15.39:0/543638521 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c003b70 con 0x7fdda410f500 2024-07-04T08:26:34.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.915+0000 7fdd9aa00640 1 -- 172.21.15.39:0/543638521 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd9c031910 con 0x7fdda410f500 2024-07-04T08:26:34.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.915+0000 7fdd9aa00640 1 -- 172.21.15.39:0/543638521 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c02f950 con 0x7fdda410f500 2024-07-04T08:26:34.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.915+0000 7fdda3200640 1 -- 172.21.15.39:0/543638521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda41a1c20 con 0x7fdda410f500 2024-07-04T08:26:34.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.916+0000 7fdda3200640 1 -- 172.21.15.39:0/543638521 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda41a83e0 con 0x7fdda410f500 2024-07-04T08:26:34.911 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:34.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.919+0000 7fddaf3ee740 1 Processor -- start 2024-07-04T08:26:34.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.919+0000 7fdd9aa00640 1 -- 172.21.15.39:0/543638521 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdd9c02faf0 con 0x7fdda410f500 2024-07-04T08:26:34.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.920+0000 7fddaf3ee740 1 -- start start 2024-07-04T08:26:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.920+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x557081728580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.920+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081728ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.920+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x557081729000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.920+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55708169dce0 con 0x557081383840 2024-07-04T08:26:34.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.920+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55708168d280 con 0x557081362b60 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.920+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5570816912c0 con 0x55708135a980 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.920+0000 7fdda1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x557081728580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x557081728580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52258/0 (socket says 172.21.15.39:52258) 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1400640 1 -- 172.21.15.39:0/1216760471 learned_addr learned my addr 172.21.15.39:0/1216760471 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1e00640 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081728ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1400640 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x557081729000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1e00640 1 -- 172.21.15.39:0/1216760471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 msgr2=0x557081729000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1e00640 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x557081729000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1e00640 1 -- 172.21.15.39:0/1216760471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 msgr2=0x557081728580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1e00640 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x557081728580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.920 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1e00640 1 -- 172.21.15.39:0/1216760471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570816246f0 con 0x55708135a980 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.921+0000 7fdda1400640 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x557081728580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.922+0000 7fdda1e00640 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081728ac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdd94011510 tx=0x7fdd940119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e46b2ff4e1dd3e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.922+0000 7fdd8f400640 1 -- 172.21.15.39:0/1216760471 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94005020 con 0x55708135a980 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.922+0000 7fdd9aa00640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd880b29d0 0x7fdd880b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.922+0000 7fdd8f400640 1 -- 172.21.15.39:0/1216760471 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd940047a0 con 0x55708135a980 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.922+0000 7fdd8f400640 1 -- 172.21.15.39:0/1216760471 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400f8a0 con 0x55708135a980 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.922+0000 7fdd9aa00640 1 -- 172.21.15.39:0/543638521 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fdd9c103ca0 con 0x7fdda410f500 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.923+0000 7fdda1400640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd880b29d0 0x7fdd880b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.923+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1216760471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 msgr2=0x557081728ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.923+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081728ac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdd94011510 tx=0x7fdd940119d0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.923+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1216760471 shutdown_connections 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.923+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x557081729000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.923+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081728ac0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.924+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1216760471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x557081728580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.924+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1216760471 >> 172.21.15.39:0/1216760471 conn(0x557081377990 msgr2=0x557081726470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.924+0000 7fdda1400640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd880b29d0 0x7fdd880b4ee0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdd90010700 tx=0x7fdd90009910 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.924+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1216760471 shutdown_connections 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.924+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1216760471 wait complete. 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.925+0000 7fddaf3ee740 1 -- start start 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.925+0000 7fddaf3ee740 1 Processor -- start 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081734180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 0x5570817346c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081383840 0x557081740080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5570814f17a0 con 0x557081362b60 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5570816346d0 con 0x557081383840 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557081736d60 con 0x55708135a980 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fddaf3ee740 10 client.0 _ll_register_callbacks cb 0x55708139b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fdda1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081734180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fdda1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 0x5570817346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fdda1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081734180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51596/0 (socket says 172.21.15.39:51596) 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fdda1e00640 1 -- 172.21.15.39:0/457252398 learned_addr learned my addr 172.21.15.39:0/457252398 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.926+0000 7fdda2800640 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081383840 0x557081740080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fdda1e00640 1 -- 172.21.15.39:0/457252398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081383840 msgr2=0x557081740080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fdda1e00640 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081383840 0x557081740080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fdda1e00640 1 -- 172.21.15.39:0/457252398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 msgr2=0x5570817346c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fdda1e00640 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 0x5570817346c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fdda1e00640 1 -- 172.21.15.39:0/457252398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570816251d0 con 0x55708135a980 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fdda1e00640 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081734180 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdd940042a0 tx=0x7fdd940042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de31cef3b753bb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fdd83e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fdd83e00640 1 -- 172.21.15.39:0/457252398 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94027070 con 0x55708135a980 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fdd83e00640 1 -- 172.21.15.39:0/457252398 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fdd94004ec0 con 0x55708135a980 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.927+0000 7fddaf3ee740 1 -- 172.21.15.39:0/457252398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd940047a0 con 0x55708135a980 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.928+0000 7fdd83e00640 1 -- 172.21.15.39:0/457252398 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd94033050 con 0x55708135a980 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.928+0000 7fddaf3ee740 10 client.27958 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.928+0000 7fddaf3ee740 1 -- 172.21.15.39:0/457252398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5570813fdb90 con 0x55708135a980 2024-07-04T08:26:34.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.928+0000 7fdd83e00640 1 -- 172.21.15.39:0/457252398 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd940047a0 con 0x55708135a980 2024-07-04T08:26:34.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.929+0000 7fdd83e00640 1 -- 172.21.15.39:0/457252398 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdd9400fac0 con 0x55708135a980 2024-07-04T08:26:34.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.929+0000 7fddaf3ee740 10 client.27958 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:34.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.929+0000 7fddaf3ee740 10 client.27958 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:26:34.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.929+0000 7fddaf3ee740 20 client.27958 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:34.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.929+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5570813421c0 0x557081745ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.929+0000 7fddaf3ee740 4 client.27958 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.929+0000 7fddaf3ee740 1 -- 172.21.15.39:0/457252398 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55708169fc30 con 0x5570813421c0 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.929+0000 7fdda1400640 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5570813421c0 0x557081745ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.930+0000 7fdda1400640 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5570813421c0 0x557081745ff0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fd567a9e7d5fc5 server_cookie=506928c48e43af61 in_seq=0 out_seq=0 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.930+0000 7fdd83e00640 10 client.27958 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fdd83e00640 1 -- 172.21.15.39:0/457252398 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55708169fc30 con 0x5570813421c0 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fdd83e00640 10 client.27958 handle_command_reply: tid=0 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 client.27958 shutdown 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 -- 172.21.15.39:0/457252398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 msgr2=0x557081734180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081734180 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdd940042a0 tx=0x7fdd940042d0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 -- 172.21.15.39:0/457252398 shutdown_connections 2024-07-04T08:26:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5570813421c0 0x557081745ff0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 0x5570817346c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081383840 0x557081740080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/457252398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x557081734180 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fdd83e00640 0 client.27958 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 -- 172.21.15.39:0/457252398 >> 172.21.15.39:0/457252398 conn(0x557081377990 msgr2=0x55708172b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.931+0000 7fddaf3ee740 1 -- 172.21.15.39:0/457252398 shutdown_connections 2024-07-04T08:26:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.932+0000 7fddaf3ee740 1 -- 172.21.15.39:0/457252398 wait complete. 2024-07-04T08:26:34.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.932+0000 7fddaf3ee740 20 client.27958 trim_cache size 0 max 16384 2024-07-04T08:26:34.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.952+0000 7fddaf3ee740 1 Processor -- start 2024-07-04T08:26:34.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.952+0000 7fddaf3ee740 1 -- start start 2024-07-04T08:26:34.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.952+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x5570817254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.952+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x55708173fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.952+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x5570817403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.952+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b2320 con 0x557081383840 2024-07-04T08:26:34.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.952+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b3e60 con 0x557081362b60 2024-07-04T08:26:34.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.952+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b29f0 con 0x55708135a980 2024-07-04T08:26:34.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x55708173fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x55708173fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33286/0 (socket says 172.21.15.39:33286) 2024-07-04T08:26:34.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda1400640 1 -- 172.21.15.39:0/2757901539 learned_addr learned my addr 172.21.15.39:0/2757901539 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:34.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda1e00640 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x5570817254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda2800640 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x5570817403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda1400640 1 -- 172.21.15.39:0/2757901539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 msgr2=0x5570817254b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:34.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda1400640 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x5570817254b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda1400640 1 -- 172.21.15.39:0/2757901539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 msgr2=0x5570817403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:34.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda1400640 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x5570817403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda1400640 1 -- 172.21.15.39:0/2757901539 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55708172b3e0 con 0x557081362b60 2024-07-04T08:26:34.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.953+0000 7fdda2800640 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x5570817403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:34.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.954+0000 7fdda1400640 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x55708173fe70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x557081736930 tx=0x7fdd90015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2d4e4c1229ba76a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:34.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.954+0000 7fdd8ea00640 1 -- 172.21.15.39:0/2757901539 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9002c5b0 con 0x557081362b60 2024-07-04T08:26:34.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.954+0000 7fdd8ea00640 1 -- 172.21.15.39:0/2757901539 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd9002cc00 con 0x557081362b60 2024-07-04T08:26:34.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.954+0000 7fdd8ea00640 1 -- 172.21.15.39:0/2757901539 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9000fe20 con 0x557081362b60 2024-07-04T08:26:34.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.955+0000 7fddaf3ee740 1 -- 172.21.15.39:0/2757901539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 msgr2=0x55708173fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:34.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.955+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x55708173fe70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x557081736930 tx=0x7fdd90015000 comp rx=0 tx=0).stop 2024-07-04T08:26:34.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.955+0000 7fddaf3ee740 1 -- 172.21.15.39:0/2757901539 shutdown_connections 2024-07-04T08:26:34.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.955+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081383840 0x5570817403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.955+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557081362b60 0x55708173fe70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.955+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/2757901539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55708135a980 0x5570817254b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.955+0000 7fddaf3ee740 1 -- 172.21.15.39:0/2757901539 >> 172.21.15.39:0/2757901539 conn(0x557081377990 msgr2=0x5570817263b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:34.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.956+0000 7fddaf3ee740 1 -- 172.21.15.39:0/2757901539 shutdown_connections 2024-07-04T08:26:34.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.956+0000 7fddaf3ee740 1 -- 172.21.15.39:0/2757901539 wait complete. 2024-07-04T08:26:34.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.956+0000 7fddaf3ee740 1 -- start start 2024-07-04T08:26:34.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.957+0000 7fddaf3ee740 1 Processor -- start 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.957+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708135a980 0x55708173c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.957+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 0x557081737d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.957+0000 7fddaf3ee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557081383840 0x557081738440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.957+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b3790 con 0x557081362b60 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.957+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b30c0 con 0x55708135a980 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.957+0000 7fddaf3ee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b0300 con 0x557081383840 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.957+0000 7fddaf3ee740 10 client.0 _ll_register_callbacks cb 0x7fdd90009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708135a980 0x55708173c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557081383840 0x557081738440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda1e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708135a980 0x55708173c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33290/0 (socket says 172.21.15.39:33290) 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda1e00640 1 -- 172.21.15.39:0/1436743324 learned_addr learned my addr 172.21.15.39:0/1436743324 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda1400640 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 0x557081737d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda1e00640 1 -- 172.21.15.39:0/1436743324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557081383840 msgr2=0x557081738440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda1e00640 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557081383840 0x557081738440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda1e00640 1 -- 172.21.15.39:0/1436743324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 msgr2=0x557081737d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda1e00640 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 0x557081737d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.958+0000 7fdda1e00640 1 -- 172.21.15.39:0/1436743324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570817ad690 con 0x55708135a980 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.959+0000 7fdda1e00640 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708135a980 0x55708173c880 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdd94004270 tx=0x7fdd940043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989fb12dd5d62965 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.959+0000 7fdd83e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.959+0000 7fdd83e00640 1 -- 172.21.15.39:0/1436743324 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94027040 con 0x55708135a980 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.959+0000 7fdd83e00640 1 -- 172.21.15.39:0/1436743324 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fdd94003ce0 con 0x55708135a980 2024-07-04T08:26:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.959+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1436743324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd9002cc00 con 0x55708135a980 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.960+0000 7fdd83e00640 1 -- 172.21.15.39:0/1436743324 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd94009470 con 0x55708135a980 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.960+0000 7fddaf3ee740 10 client.28031 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.960+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1436743324 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5570817b6fa0 con 0x55708135a980 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.960+0000 7fdd83e00640 1 -- 172.21.15.39:0/1436743324 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd9002cc00 con 0x55708135a980 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.961+0000 7fdd83e00640 1 -- 172.21.15.39:0/1436743324 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdd940426e0 con 0x55708135a980 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.961+0000 7fddaf3ee740 10 client.28031 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.961+0000 7fddaf3ee740 10 client.28031 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.961+0000 7fddaf3ee740 20 client.28031 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.961+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557081317570 0x5570817b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:34.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.961+0000 7fddaf3ee740 4 client.28031 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-07-04T08:26:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.961+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1436743324 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55708169b6d0 con 0x557081317570 2024-07-04T08:26:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.962+0000 7fdda1400640 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557081317570 0x5570817b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.962+0000 7fdd83e00640 10 client.28031 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:34.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.962+0000 7fdda1400640 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557081317570 0x5570817b96e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca08e732f4f0960 server_cookie=6330afc2960503b6 in_seq=0 out_seq=0 2024-07-04T08:26:34.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fdd83e00640 1 -- 172.21.15.39:0/1436743324 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55708169b6d0 con 0x557081317570 2024-07-04T08:26:34.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fdd83e00640 10 client.28031 handle_command_reply: tid=0 2024-07-04T08:26:34.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 client.28031 shutdown 2024-07-04T08:26:34.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1436743324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708135a980 msgr2=0x55708173c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:34.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708135a980 0x55708173c880 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdd94004270 tx=0x7fdd940043b0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1436743324 shutdown_connections 2024-07-04T08:26:34.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x557081317570 0x5570817b96e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557081362b60 0x557081737d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557081383840 0x557081738440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 --2- 172.21.15.39:0/1436743324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708135a980 0x55708173c880 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fdd83e00640 0 client.28031 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:34.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1436743324 >> 172.21.15.39:0/1436743324 conn(0x557081377990 msgr2=0x557081735430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:34.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.973+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1436743324 shutdown_connections 2024-07-04T08:26:34.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.974+0000 7fddaf3ee740 1 -- 172.21.15.39:0/1436743324 wait complete. 2024-07-04T08:26:34.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.974+0000 7fddaf3ee740 20 client.28031 trim_cache size 0 max 16384 2024-07-04T08:26:34.965 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:26:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:26:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:26:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 34, 2024-07-04T08:26:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:26:34.965 INFO:teuthology.orchestra.run.smithi039.stdout: "b7445762": { 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 34, 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 150.0, 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:26:34.966 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:26:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:26:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:26:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:34.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:26:34.967 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.974+0000 7fdda3200640 1 -- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd880b29d0 msgr2=0x7fdd880b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.974+0000 7fdda3200640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd880b29d0 0x7fdd880b4ee0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdd90010700 tx=0x7fdd90009910 comp rx=0 tx=0).stop 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.974+0000 7fdda3200640 1 -- 172.21.15.39:0/543638521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda410f500 msgr2=0x7fdda41a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.974+0000 7fdda3200640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda410f500 0x7fdda41a1260 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c02ade0 tx=0x7fdd9c031660 comp rx=0 tx=0).stop 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.975+0000 7fdda3200640 1 -- 172.21.15.39:0/543638521 shutdown_connections 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.975+0000 7fdda3200640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdd880b29d0 0x7fdd880b4ee0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.975+0000 7fdda3200640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdda40755d0 0x7fdda419bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.975+0000 7fdda3200640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdda410f500 0x7fdda41a1260 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.975+0000 7fdda3200640 1 --2- 172.21.15.39:0/543638521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdda4074c80 0x7fdda419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:34.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.975+0000 7fdda3200640 1 -- 172.21.15.39:0/543638521 >> 172.21.15.39:0/543638521 conn(0x7fdda40fe970 msgr2=0x7fdda4108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:34.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.975+0000 7fdda3200640 1 -- 172.21.15.39:0/543638521 shutdown_connections 2024-07-04T08:26:34.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:34.975+0000 7fdda3200640 1 -- 172.21.15.39:0/543638521 wait complete. 2024-07-04T08:26:35.318 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:35 smithi039 ceph-mon[32062]: pgmap v831: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-07-04T08:26:35.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:35 smithi133 ceph-mon[37050]: pgmap v831: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-07-04T08:26:35.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:35 smithi050 ceph-mon[36577]: pgmap v831: 129 pgs: 129 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-07-04T08:26:36.594 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b7445762', quiesce took 11.9 seconds. Will release after: 18.1 2024-07-04T08:26:37.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:37 smithi039 ceph-mon[32062]: pgmap v832: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-07-04T08:26:37.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:37 smithi133 ceph-mon[37050]: pgmap v832: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-07-04T08:26:37.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:37 smithi050 ceph-mon[36577]: pgmap v832: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-07-04T08:26:37.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.957+0000 7f0c42600640 1 -- 172.21.15.39:0/2161041496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c10aa50 msgr2=0x7f0c3c10ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:37.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.957+0000 7f0c42600640 1 --2- 172.21.15.39:0/2161041496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c10aa50 0x7f0c3c10ae60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009900 tx=0x7f0c2802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.957+0000 7f0c42600640 1 -- 172.21.15.39:0/2161041496 shutdown_connections 2024-07-04T08:26:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.957+0000 7f0c42600640 1 --2- 172.21.15.39:0/2161041496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c3c10b3a0 0x7f0c3c111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.957+0000 7f0c42600640 1 --2- 172.21.15.39:0/2161041496 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c10aa50 0x7f0c3c10ae60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.957+0000 7f0c42600640 1 --2- 172.21.15.39:0/2161041496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c3c105110 0x7f0c3c105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.957+0000 7f0c42600640 1 -- 172.21.15.39:0/2161041496 >> 172.21.15.39:0/2161041496 conn(0x7f0c3c100bc0 msgr2=0x7f0c3c103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.957+0000 7f0c42600640 1 -- 172.21.15.39:0/2161041496 shutdown_connections 2024-07-04T08:26:37.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.957+0000 7f0c42600640 1 -- 172.21.15.39:0/2161041496 wait complete. 2024-07-04T08:26:37.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.958+0000 7f0c42600640 1 Processor -- start 2024-07-04T08:26:37.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.958+0000 7f0c42600640 1 -- start start 2024-07-04T08:26:37.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c42600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c105110 0x7f0c3c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c42600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c3c10aa50 0x7f0c3c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c42600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c3c10b3a0 0x7f0c3c1a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c42600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c115010 con 0x7f0c3c10aa50 2024-07-04T08:26:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c42600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c114e90 con 0x7f0c3c105110 2024-07-04T08:26:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c42600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c115190 con 0x7f0c3c10b3a0 2024-07-04T08:26:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c41200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c105110 0x7f0c3c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c3c10aa50 0x7f0c3c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c41200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c105110 0x7f0c3c19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33298/0 (socket says 172.21.15.39:33298) 2024-07-04T08:26:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c41200640 1 -- 172.21.15.39:0/2900618979 learned_addr learned my addr 172.21.15.39:0/2900618979 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:37.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.959+0000 7f0c41c00640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c3c10b3a0 0x7f0c3c1a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c41200640 1 -- 172.21.15.39:0/2900618979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c3c10b3a0 msgr2=0x7f0c3c1a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:37.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c41200640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c3c10b3a0 0x7f0c3c1a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c41200640 1 -- 172.21.15.39:0/2900618979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c3c10aa50 msgr2=0x7f0c3c19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:37.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c41200640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c3c10aa50 0x7f0c3c19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c41200640 1 -- 172.21.15.39:0/2900618979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3c1a19b0 con 0x7f0c3c105110 2024-07-04T08:26:37.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c41200640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c105110 0x7f0c3c19ba40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000e9f0 tx=0x7f0c3000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:37.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c39600640 1 -- 172.21.15.39:0/2900618979 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000ccc0 con 0x7f0c3c105110 2024-07-04T08:26:37.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c39600640 1 -- 172.21.15.39:0/2900618979 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c300108e0 con 0x7f0c3c105110 2024-07-04T08:26:37.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c42600640 1 -- 172.21.15.39:0/2900618979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3c1a1ca0 con 0x7f0c3c105110 2024-07-04T08:26:37.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c39600640 1 -- 172.21.15.39:0/2900618979 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30018880 con 0x7f0c3c105110 2024-07-04T08:26:37.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.960+0000 7f0c42600640 1 -- 172.21.15.39:0/2900618979 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3c1a8400 con 0x7f0c3c105110 2024-07-04T08:26:37.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.964+0000 7f0c4a300740 1 Processor -- start 2024-07-04T08:26:37.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.964+0000 7f0c4a300740 1 -- start start 2024-07-04T08:26:37.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.964+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d7b6070 0x55e91db5a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.964+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 0x55e91db5ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.964+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d795390 0x55e91db5b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.964+0000 7f0c4a300740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e91dad0ee0 con 0x55e91d795390 2024-07-04T08:26:37.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.964+0000 7f0c4a300740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e91dac0480 con 0x55e91d78d1b0 2024-07-04T08:26:37.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.964+0000 7f0c4a300740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e91dac44c0 con 0x55e91d7b6070 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c39600640 1 -- 172.21.15.39:0/2900618979 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0c30010430 con 0x7f0c3c105110 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d795390 0x55e91db5b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c41c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d7b6070 0x55e91db5a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d795390 0x55e91db5b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52332/0 (socket says 172.21.15.39:52332) 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c3be00640 1 -- 172.21.15.39:0/695169727 learned_addr learned my addr 172.21.15.39:0/695169727 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c41c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d7b6070 0x55e91db5a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51672/0 (socket says 172.21.15.39:51672) 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c41200640 1 --2- 172.21.15.39:0/695169727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 0x55e91db5ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c41c00640 1 -- 172.21.15.39:0/695169727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 msgr2=0x55e91db5ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c41c00640 1 --2- 172.21.15.39:0/695169727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 0x55e91db5ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c41c00640 1 -- 172.21.15.39:0/695169727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d795390 msgr2=0x55e91db5b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c41c00640 1 --2- 172.21.15.39:0/695169727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d795390 0x55e91db5b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c41c00640 1 -- 172.21.15.39:0/695169727 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91da58650 con 0x55e91d7b6070 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.965+0000 7f0c3be00640 1 --2- 172.21.15.39:0/695169727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d795390 0x55e91db5b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.966+0000 7f0c41c00640 1 --2- 172.21.15.39:0/695169727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d7b6070 0x55e91db5a950 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009f10 tx=0x7f0c34011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1d421c1f49acbad server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.966+0000 7f0c2e000640 1 -- 172.21.15.39:0/695169727 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34004d20 con 0x55e91d7b6070 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.966+0000 7f0c2e000640 1 -- 172.21.15.39:0/695169727 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c34004030 con 0x55e91d7b6070 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.966+0000 7f0c2e000640 1 -- 172.21.15.39:0/695169727 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34005280 con 0x55e91d7b6070 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.967+0000 7f0c39600640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c240b2960 0x7f0c240b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.967+0000 7f0c39600640 1 -- 172.21.15.39:0/2900618979 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f0c300e2dc0 con 0x7f0c3c105110 2024-07-04T08:26:37.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.967+0000 7f0c41200640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c240b2960 0x7f0c240b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.967+0000 7f0c4a300740 1 -- 172.21.15.39:0/695169727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d7b6070 msgr2=0x55e91db5a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.967+0000 7f0c4a300740 1 --2- 172.21.15.39:0/695169727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d7b6070 0x55e91db5a950 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009f10 tx=0x7f0c34011810 comp rx=0 tx=0).stop 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.968+0000 7f0c4a300740 1 -- 172.21.15.39:0/695169727 shutdown_connections 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.968+0000 7f0c4a300740 1 --2- 172.21.15.39:0/695169727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d795390 0x55e91db5b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.968+0000 7f0c4a300740 1 --2- 172.21.15.39:0/695169727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 0x55e91db5ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.968+0000 7f0c4a300740 1 --2- 172.21.15.39:0/695169727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d7b6070 0x55e91db5a950 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.968+0000 7f0c4a300740 1 -- 172.21.15.39:0/695169727 >> 172.21.15.39:0/695169727 conn(0x55e91d7aa460 msgr2=0x55e91db58840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.968+0000 7f0c4a300740 1 -- 172.21.15.39:0/695169727 shutdown_connections 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.968+0000 7f0c41200640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c240b2960 0x7f0c240b4e70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55e91db595d0 tx=0x7f0c3000ce60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.968+0000 7f0c4a300740 1 -- 172.21.15.39:0/695169727 wait complete. 2024-07-04T08:26:37.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.968+0000 7f0c4a300740 1 -- start start 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.969+0000 7f0c4a300740 1 Processor -- start 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.969+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d78d1b0 0x55e91db66470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.969+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d795390 0x55e91db669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.969+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 0x55e91db72450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.969+0000 7f0c4a300740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e91d924700 con 0x55e91d7b6070 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.969+0000 7f0c4a300740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e91da67bb0 con 0x55e91d795390 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.969+0000 7f0c4a300740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e91db69130 con 0x55e91d78d1b0 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c4a300740 10 client.0 _ll_register_callbacks cb 0x55e91dab9570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d795390 0x55e91db669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c41c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d78d1b0 0x55e91db66470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d795390 0x55e91db669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33308/0 (socket says 172.21.15.39:33308) 2024-07-04T08:26:37.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c3be00640 1 -- 172.21.15.39:0/35296815 learned_addr learned my addr 172.21.15.39:0/35296815 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c41c00640 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 0x55e91db72450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c3be00640 1 -- 172.21.15.39:0/35296815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d78d1b0 msgr2=0x55e91db66470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c3be00640 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d78d1b0 0x55e91db66470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c3be00640 1 -- 172.21.15.39:0/35296815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 msgr2=0x55e91db72450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c3be00640 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 0x55e91db72450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c3be00640 1 -- 172.21.15.39:0/35296815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e91d821650 con 0x55e91d795390 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.970+0000 7f0c41c00640 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d78d1b0 0x55e91db66470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.971+0000 7f0c3be00640 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d795390 0x55e91db669b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c28032e20 tx=0x7f0c28032e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319bb51904aeda69 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.971+0000 7f0c22a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.971+0000 7f0c22a00640 1 -- 172.21.15.39:0/35296815 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28035040 con 0x55e91d795390 2024-07-04T08:26:37.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.971+0000 7f0c4a300740 1 -- 172.21.15.39:0/35296815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c34004030 con 0x55e91d795390 2024-07-04T08:26:37.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.971+0000 7f0c22a00640 1 -- 172.21.15.39:0/35296815 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f0c28031760 con 0x55e91d795390 2024-07-04T08:26:37.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.971+0000 7f0c22a00640 1 -- 172.21.15.39:0/35296815 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c28004040 con 0x55e91d795390 2024-07-04T08:26:37.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.971+0000 7f0c4a300740 10 client.28043 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:37.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.971+0000 7f0c4a300740 1 -- 172.21.15.39:0/35296815 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e91db75120 con 0x55e91d795390 2024-07-04T08:26:37.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.972+0000 7f0c22a00640 1 -- 172.21.15.39:0/35296815 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c34004030 con 0x55e91d795390 2024-07-04T08:26:37.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.972+0000 7f0c22a00640 1 -- 172.21.15.39:0/35296815 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0c28004750 con 0x55e91d795390 2024-07-04T08:26:37.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.972+0000 7f0c4a300740 10 client.28043 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.972+0000 7f0c4a300740 10 client.28043 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.972+0000 7f0c4a300740 20 client.28043 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.972+0000 7f0c4a300740 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e91d7749f0 0x55e91db78590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.972+0000 7f0c4a300740 4 client.28043 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.972+0000 7f0c4a300740 1 -- 172.21.15.39:0/35296815 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e91dad2e30 con 0x55e91d7749f0 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.972+0000 7f0c41c00640 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e91d7749f0 0x55e91db78590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.973+0000 7f0c41c00640 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e91d7749f0 0x55e91db78590 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee69ef8f20393306 server_cookie=69fcecfd9063eebd in_seq=0 out_seq=0 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.973+0000 7f0c22a00640 10 client.28043 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c22a00640 1 -- 172.21.15.39:0/35296815 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e91dad2e30 con 0x55e91d7749f0 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c22a00640 10 client.28043 handle_command_reply: tid=0 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c4a300740 1 client.28043 shutdown 2024-07-04T08:26:37.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c4a300740 1 -- 172.21.15.39:0/35296815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d795390 msgr2=0x55e91db669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c4a300740 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d795390 0x55e91db669b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0c28032e20 tx=0x7f0c28032e50 comp rx=0 tx=0).stop 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c4a300740 1 -- 172.21.15.39:0/35296815 shutdown_connections 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c4a300740 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e91d7749f0 0x55e91db78590 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c4a300740 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 0x55e91db72450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c4a300740 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d795390 0x55e91db669b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c4a300740 1 --2- 172.21.15.39:0/35296815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d78d1b0 0x55e91db66470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c22a00640 0 client.28043 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.974+0000 7f0c4a300740 1 -- 172.21.15.39:0/35296815 >> 172.21.15.39:0/35296815 conn(0x55e91d7aa460 msgr2=0x55e91db5dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.975+0000 7f0c4a300740 1 -- 172.21.15.39:0/35296815 shutdown_connections 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.975+0000 7f0c4a300740 1 -- 172.21.15.39:0/35296815 wait complete. 2024-07-04T08:26:37.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.975+0000 7f0c4a300740 20 client.28043 trim_cache size 0 max 16384 2024-07-04T08:26:37.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.990+0000 7f0c4a300740 1 Processor -- start 2024-07-04T08:26:37.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.990+0000 7f0c4a300740 1 -- start start 2024-07-04T08:26:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.991+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 0x55e91db60a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.991+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db67ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.991+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 0x55e91db72240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.991+0000 7f0c4a300740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1b23e0 con 0x55e91d7b6070 2024-07-04T08:26:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.991+0000 7f0c4a300740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1b3f20 con 0x55e91d78d1b0 2024-07-04T08:26:37.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.991+0000 7f0c4a300740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1b2ab0 con 0x55e91d795390 2024-07-04T08:26:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.991+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db67ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.991+0000 7f0c41c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 0x55e91db72240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.992+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db67ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51698/0 (socket says 172.21.15.39:51698) 2024-07-04T08:26:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.992+0000 7f0c3be00640 1 -- 172.21.15.39:0/1009501103 learned_addr learned my addr 172.21.15.39:0/1009501103 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:37.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.992+0000 7f0c41c00640 1 --2- 172.21.15.39:0/1009501103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 0x55e91db60a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.992+0000 7f0c3be00640 1 -- 172.21.15.39:0/1009501103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 msgr2=0x55e91db60a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.992+0000 7f0c3be00640 1 --2- 172.21.15.39:0/1009501103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 0x55e91db60a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.992+0000 7f0c3be00640 1 -- 172.21.15.39:0/1009501103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 msgr2=0x55e91db72240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.992+0000 7f0c3be00640 1 --2- 172.21.15.39:0/1009501103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 0x55e91db72240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.992+0000 7f0c3be00640 1 -- 172.21.15.39:0/1009501103 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91db64770 con 0x55e91d795390 2024-07-04T08:26:37.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.992+0000 7f0c41c00640 1 --2- 172.21.15.39:0/1009501103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 0x55e91db60a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:37.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.993+0000 7f0c3be00640 1 --2- 172.21.15.39:0/1009501103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db67ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c2802a430 tx=0x7f0c28005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb5657fd55fffbb9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:37.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.993+0000 7f0c2d600640 1 -- 172.21.15.39:0/1009501103 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28035030 con 0x55e91d795390 2024-07-04T08:26:37.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.993+0000 7f0c2d600640 1 -- 172.21.15.39:0/1009501103 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c280048c0 con 0x55e91d795390 2024-07-04T08:26:37.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.993+0000 7f0c2d600640 1 -- 172.21.15.39:0/1009501103 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28004ff0 con 0x55e91d795390 2024-07-04T08:26:37.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.994+0000 7f0c4a300740 1 -- 172.21.15.39:0/1009501103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 msgr2=0x55e91db67ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:37.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.994+0000 7f0c4a300740 1 --2- 172.21.15.39:0/1009501103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db67ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c2802a430 tx=0x7f0c28005870 comp rx=0 tx=0).stop 2024-07-04T08:26:37.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.994+0000 7f0c4a300740 1 -- 172.21.15.39:0/1009501103 shutdown_connections 2024-07-04T08:26:37.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.994+0000 7f0c4a300740 1 --2- 172.21.15.39:0/1009501103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d7b6070 0x55e91db72240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.994+0000 7f0c4a300740 1 --2- 172.21.15.39:0/1009501103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db67ea0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.994+0000 7f0c4a300740 1 --2- 172.21.15.39:0/1009501103 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d78d1b0 0x55e91db60a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.994+0000 7f0c4a300740 1 -- 172.21.15.39:0/1009501103 >> 172.21.15.39:0/1009501103 conn(0x55e91d7aa460 msgr2=0x55e91db67790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:37.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.994+0000 7f0c4a300740 1 -- 172.21.15.39:0/1009501103 shutdown_connections 2024-07-04T08:26:37.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.995+0000 7f0c4a300740 1 -- 172.21.15.39:0/1009501103 wait complete. 2024-07-04T08:26:37.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.995+0000 7f0c4a300740 1 -- start start 2024-07-04T08:26:37.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.995+0000 7f0c4a300740 1 Processor -- start 2024-07-04T08:26:37.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d78d1b0 0x55e91db6a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db6aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c4a300740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d7b6070 0x55e91dbdfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c4a300740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1b3850 con 0x55e91d78d1b0 2024-07-04T08:26:37.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c4a300740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1b3180 con 0x55e91d7b6070 2024-07-04T08:26:37.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c4a300740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3c1b03c0 con 0x55e91d795390 2024-07-04T08:26:37.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c4a300740 10 client.0 _ll_register_callbacks cb 0x7f0c28009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:37.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c41c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d78d1b0 0x55e91db6a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db6aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c3be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db6aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:51702/0 (socket says 172.21.15.39:51702) 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c3be00640 1 -- 172.21.15.39:0/4126906208 learned_addr learned my addr 172.21.15.39:0/4126906208 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.996+0000 7f0c41c00640 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d7b6070 0x55e91dbdfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.997+0000 7f0c41c00640 1 -- 172.21.15.39:0/4126906208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 msgr2=0x55e91db6aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.997+0000 7f0c41c00640 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db6aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.997+0000 7f0c41c00640 1 -- 172.21.15.39:0/4126906208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d7b6070 msgr2=0x55e91dbdfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.997+0000 7f0c41c00640 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d7b6070 0x55e91dbdfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.997+0000 7f0c41c00640 1 -- 172.21.15.39:0/4126906208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e91db6b360 con 0x55e91d78d1b0 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.997+0000 7f0c41c00640 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d78d1b0 0x55e91db6a580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0c34004890 tx=0x7f0c34009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1760d5895094ea server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:37.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.997+0000 7f0c22a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:26:37.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.998+0000 7f0c22a00640 1 -- 172.21.15.39:0/4126906208 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3401e040 con 0x55e91d78d1b0 2024-07-04T08:26:37.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.998+0000 7f0c22a00640 1 -- 172.21.15.39:0/4126906208 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f0c34003c30 con 0x55e91d78d1b0 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.998+0000 7f0c4a300740 1 -- 172.21.15.39:0/4126906208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c280048c0 con 0x55e91d78d1b0 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.998+0000 7f0c22a00640 1 -- 172.21.15.39:0/4126906208 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c34003e30 con 0x55e91d78d1b0 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.998+0000 7f0c4a300740 10 client.17910 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.998+0000 7f0c4a300740 1 -- 172.21.15.39:0/4126906208 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e91dbe1280 con 0x55e91d78d1b0 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.998+0000 7f0c22a00640 1 -- 172.21.15.39:0/4126906208 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c280048c0 con 0x55e91d78d1b0 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.999+0000 7f0c22a00640 1 -- 172.21.15.39:0/4126906208 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0c34024070 con 0x55e91d78d1b0 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.999+0000 7f0c4a300740 10 client.17910 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.999+0000 7f0c4a300740 10 client.17910 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.999+0000 7f0c4a300740 20 client.17910 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.999+0000 7f0c4a300740 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e91d749da0 0x55e91dbe3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.999+0000 7f0c4a300740 4 client.17910 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.999+0000 7f0c4a300740 1 -- 172.21.15.39:0/4126906208 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e91dace8d0 con 0x55e91d749da0 2024-07-04T08:26:37.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:37.999+0000 7f0c3be00640 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e91d749da0 0x55e91dbe3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.000+0000 7f0c3be00640 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e91d749da0 0x55e91dbe3870 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d81a93b21152017 server_cookie=1b6ad7a79941852 in_seq=0 out_seq=0 2024-07-04T08:26:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.000+0000 7f0c22a00640 10 client.17910 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:26:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.000+0000 7f0c22a00640 1 -- 172.21.15.39:0/4126906208 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e91dace8d0 con 0x55e91d749da0 2024-07-04T08:26:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.000+0000 7f0c22a00640 10 client.17910 handle_command_reply: tid=0 2024-07-04T08:26:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.000+0000 7f0c4a300740 1 client.17910 shutdown 2024-07-04T08:26:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.000+0000 7f0c4a300740 1 -- 172.21.15.39:0/4126906208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d78d1b0 msgr2=0x55e91db6a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.000+0000 7f0c4a300740 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d78d1b0 0x55e91db6a580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0c34004890 tx=0x7f0c34009900 comp rx=0 tx=0).stop 2024-07-04T08:26:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c4a300740 1 -- 172.21.15.39:0/4126906208 shutdown_connections 2024-07-04T08:26:37.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c4a300740 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55e91d749da0 0x55e91dbe3870 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c4a300740 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e91d7b6070 0x55e91dbdfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c4a300740 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e91d795390 0x55e91db6aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c4a300740 1 --2- 172.21.15.39:0/4126906208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e91d78d1b0 0x55e91db6a580 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c4a300740 1 -- 172.21.15.39:0/4126906208 >> 172.21.15.39:0/4126906208 conn(0x55e91d7aa460 msgr2=0x55e91db56e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c22a00640 0 client.17910 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:26:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c4a300740 1 -- 172.21.15.39:0/4126906208 shutdown_connections 2024-07-04T08:26:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c4a300740 1 -- 172.21.15.39:0/4126906208 wait complete. 2024-07-04T08:26:37.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.001+0000 7f0c4a300740 20 client.17910 trim_cache size 0 max 16384 2024-07-04T08:26:37.993 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:37.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 -- 172.21.15.39:0/2900618979 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c240b2960 msgr2=0x7f0c240b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:37.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c240b2960 0x7f0c240b4e70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55e91db595d0 tx=0x7f0c3000ce60 comp rx=0 tx=0).stop 2024-07-04T08:26:37.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 -- 172.21.15.39:0/2900618979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c105110 msgr2=0x7f0c3c19ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:37.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c105110 0x7f0c3c19ba40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000e9f0 tx=0x7f0c3000edf0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 -- 172.21.15.39:0/2900618979 shutdown_connections 2024-07-04T08:26:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0c240b2960 0x7f0c240b4e70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c3c10aa50 0x7f0c3c19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c3c10b3a0 0x7f0c3c1a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 --2- 172.21.15.39:0/2900618979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c3c105110 0x7f0c3c19ba40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 -- 172.21.15.39:0/2900618979 >> 172.21.15.39:0/2900618979 conn(0x7f0c3c100bc0 msgr2=0x7f0c3c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 -- 172.21.15.39:0/2900618979 shutdown_connections 2024-07-04T08:26:37.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:38.002+0000 7f0c42600640 1 -- 172.21.15.39:0/2900618979 wait complete. 2024-07-04T08:26:38.829 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:26:39.318 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:39.508 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:39 smithi039 ceph-mon[32062]: pgmap v833: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-07-04T08:26:39.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:39 smithi133 ceph-mon[37050]: pgmap v833: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-07-04T08:26:39.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:39 smithi050 ceph-mon[36577]: pgmap v833: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 14 op/s 2024-07-04T08:26:41.471 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:41 smithi039 ceph-mon[32062]: pgmap v834: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-07-04T08:26:41.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:41 smithi133 ceph-mon[37050]: pgmap v834: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-07-04T08:26:41.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:41 smithi050 ceph-mon[36577]: pgmap v834: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 49 KiB/s wr, 14 op/s 2024-07-04T08:26:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.699+0000 7f4d25c00640 1 -- 172.21.15.39:0/4258439041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2010f7a0 msgr2=0x7f4d20115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.699+0000 7f4d25c00640 1 --2- 172.21.15.39:0/4258439041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2010f7a0 0x7f4d20115fd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009ca0 tx=0x7f4d1802a8b0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.699+0000 7f4d25c00640 1 -- 172.21.15.39:0/4258439041 shutdown_connections 2024-07-04T08:26:41.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.699+0000 7f4d25c00640 1 --2- 172.21.15.39:0/4258439041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2010f7a0 0x7f4d20115fd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.699+0000 7f4d25c00640 1 --2- 172.21.15.39:0/4258439041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d2010ee50 0x7f4d2010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.699+0000 7f4d25c00640 1 --2- 172.21.15.39:0/4258439041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d20109510 0x7f4d20109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.699+0000 7f4d25c00640 1 -- 172.21.15.39:0/4258439041 >> 172.21.15.39:0/4258439041 conn(0x7f4d20078b10 msgr2=0x7f4d20107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:41.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.699+0000 7f4d25c00640 1 -- 172.21.15.39:0/4258439041 shutdown_connections 2024-07-04T08:26:41.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.699+0000 7f4d25c00640 1 -- 172.21.15.39:0/4258439041 wait complete. 2024-07-04T08:26:41.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.700+0000 7f4d25c00640 1 Processor -- start 2024-07-04T08:26:41.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d25c00640 1 -- start start 2024-07-04T08:26:41.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d25c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d20109510 0x7f4d2019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d25c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2010ee50 0x7f4d201a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d25c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d2010f7a0 0x7f4d201a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d25c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20119450 con 0x7f4d2010ee50 2024-07-04T08:26:41.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d25c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201192d0 con 0x7f4d20109510 2024-07-04T08:26:41.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d25c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201195d0 con 0x7f4d2010f7a0 2024-07-04T08:26:41.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2010ee50 0x7f4d201a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2010ee50 0x7f4d201a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38924/0 (socket says 172.21.15.39:38924) 2024-07-04T08:26:41.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d25200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d2010f7a0 0x7f4d201a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d17e00640 1 -- 172.21.15.39:0/3436600962 learned_addr learned my addr 172.21.15.39:0/3436600962 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:41.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.701+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d20109510 0x7f4d2019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.702+0000 7f4d25200640 1 -- 172.21.15.39:0/3436600962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d20109510 msgr2=0x7f4d2019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:41.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.702+0000 7f4d25200640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d20109510 0x7f4d2019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.702+0000 7f4d25200640 1 -- 172.21.15.39:0/3436600962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2010ee50 msgr2=0x7f4d201a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:41.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.702+0000 7f4d25200640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2010ee50 0x7f4d201a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.702+0000 7f4d25200640 1 -- 172.21.15.39:0/3436600962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d201a5d50 con 0x7f4d2010f7a0 2024-07-04T08:26:41.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.702+0000 7f4d25200640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d2010f7a0 0x7f4d201a5620 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4d1802add0 tx=0x7f4d18030d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:41.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.703+0000 7f4d1d600640 1 -- 172.21.15.39:0/3436600962 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18003e40 con 0x7f4d2010f7a0 2024-07-04T08:26:41.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.703+0000 7f4d25c00640 1 -- 172.21.15.39:0/3436600962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d201a5fe0 con 0x7f4d2010f7a0 2024-07-04T08:26:41.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.703+0000 7f4d1d600640 1 -- 172.21.15.39:0/3436600962 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d18040070 con 0x7f4d2010f7a0 2024-07-04T08:26:41.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.703+0000 7f4d1d600640 1 -- 172.21.15.39:0/3436600962 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d180459a0 con 0x7f4d2010f7a0 2024-07-04T08:26:41.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.703+0000 7f4d25c00640 1 -- 172.21.15.39:0/3436600962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d201ac7a0 con 0x7f4d2010f7a0 2024-07-04T08:26:41.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.707+0000 7f4d2d7be740 1 Processor -- start 2024-07-04T08:26:41.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.708+0000 7f4d1d600640 1 -- 172.21.15.39:0/3436600962 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4d18004680 con 0x7f4d2010f7a0 2024-07-04T08:26:41.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.708+0000 7f4d2d7be740 1 -- start start 2024-07-04T08:26:41.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d8a070 0x557b0a12e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a12ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d69390 0x557b0a12f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557b0a0a4ee0 con 0x557b09d8a070 2024-07-04T08:26:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557b0a094480 con 0x557b09d69390 2024-07-04T08:26:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557b0a0984c0 con 0x557b09d611b0 2024-07-04T08:26:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a12ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d8a070 0x557b0a12e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a12ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:40278/0 (socket says 172.21.15.39:40278) 2024-07-04T08:26:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.709+0000 7f4d1fe00640 1 -- 172.21.15.39:0/447500436 learned_addr learned my addr 172.21.15.39:0/447500436 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d17e00640 1 --2- 172.21.15.39:0/447500436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d69390 0x557b0a12f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d1fe00640 1 -- 172.21.15.39:0/447500436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d69390 msgr2=0x557b0a12f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/447500436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d69390 0x557b0a12f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d1fe00640 1 -- 172.21.15.39:0/447500436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d8a070 msgr2=0x557b0a12e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/447500436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d8a070 0x557b0a12e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d1fe00640 1 -- 172.21.15.39:0/447500436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0a02c650 con 0x557b09d611b0 2024-07-04T08:26:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d17e00640 1 --2- 172.21.15.39:0/447500436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d69390 0x557b0a12f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:26:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d1d600640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4d040b2960 0x7f4d040b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/447500436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a12ee90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009cf0 tx=0x7f4d10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7875d2bd1d242c2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:41.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d1d600640 1 -- 172.21.15.39:0/3436600962 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f4d18103800 con 0x7f4d2010f7a0 2024-07-04T08:26:41.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d17e00640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4d040b2960 0x7f4d040b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.710+0000 7f4d15600640 1 -- 172.21.15.39:0/447500436 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1001e070 con 0x557b09d611b0 2024-07-04T08:26:41.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.711+0000 7f4d15600640 1 -- 172.21.15.39:0/447500436 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d100198e0 con 0x557b09d611b0 2024-07-04T08:26:41.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.711+0000 7f4d15600640 1 -- 172.21.15.39:0/447500436 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000e880 con 0x557b09d611b0 2024-07-04T08:26:41.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d2d7be740 1 -- 172.21.15.39:0/447500436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 msgr2=0x557b0a12ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:41.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/447500436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a12ee90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009cf0 tx=0x7f4d10011810 comp rx=0 tx=0).stop 2024-07-04T08:26:41.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d17e00640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4d040b2960 0x7f4d040b4e70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009790 tx=0x7f4d08009800 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:41.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d2d7be740 1 -- 172.21.15.39:0/447500436 shutdown_connections 2024-07-04T08:26:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/447500436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d69390 0x557b0a12f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/447500436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a12ee90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/447500436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d8a070 0x557b0a12e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d2d7be740 1 -- 172.21.15.39:0/447500436 >> 172.21.15.39:0/447500436 conn(0x557b09d7e460 msgr2=0x557b0a12c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d2d7be740 1 -- 172.21.15.39:0/447500436 shutdown_connections 2024-07-04T08:26:41.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.712+0000 7f4d2d7be740 1 -- 172.21.15.39:0/447500436 wait complete. 2024-07-04T08:26:41.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.713+0000 7f4d2d7be740 1 -- start start 2024-07-04T08:26:41.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.713+0000 7f4d2d7be740 1 Processor -- start 2024-07-04T08:26:41.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.713+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a13a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.713+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a13aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.713+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.713+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x557b09ef8700 con 0x557b09d69390 2024-07-04T08:26:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.713+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557b0a03bbb0 con 0x557b09d8a070 2024-07-04T08:26:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.713+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557b0a13d130 con 0x557b09d611b0 2024-07-04T08:26:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.713+0000 7f4d2d7be740 10 client.0 _ll_register_callbacks cb 0x557b0a08d570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a13a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a13a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:40292/0 (socket says 172.21.15.39:40292) 2024-07-04T08:26:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d1fe00640 1 -- 172.21.15.39:0/3420009266 learned_addr learned my addr 172.21.15.39:0/3420009266 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d17e00640 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a13aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d25200640 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d1fe00640 1 -- 172.21.15.39:0/3420009266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 msgr2=0x557b0a146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d1fe00640 1 -- 172.21.15.39:0/3420009266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 msgr2=0x557b0a13aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a13aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d1fe00640 1 -- 172.21.15.39:0/3420009266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b09df5650 con 0x557b09d611b0 2024-07-04T08:26:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.714+0000 7f4d25200640 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.715+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a13a4d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000c00 tx=0x7f4d10009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aebed444a9cb29bc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.715+0000 7f4d0e000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.715+0000 7f4d0e000640 1 -- 172.21.15.39:0/3420009266 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10027070 con 0x557b09d611b0 2024-07-04T08:26:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.715+0000 7f4d0e000640 1 -- 172.21.15.39:0/3420009266 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f4d10005040 con 0x557b09d611b0 2024-07-04T08:26:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.715+0000 7f4d2d7be740 1 -- 172.21.15.39:0/3420009266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d100198e0 con 0x557b09d611b0 2024-07-04T08:26:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.715+0000 7f4d0e000640 1 -- 172.21.15.39:0/3420009266 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d10033050 con 0x557b09d611b0 2024-07-04T08:26:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.715+0000 7f4d2d7be740 10 client.27994 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.715+0000 7f4d2d7be740 1 -- 172.21.15.39:0/3420009266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b0a149070 con 0x557b09d611b0 2024-07-04T08:26:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.716+0000 7f4d0e000640 1 -- 172.21.15.39:0/3420009266 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d100198e0 con 0x557b09d611b0 2024-07-04T08:26:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.716+0000 7f4d0e000640 1 -- 172.21.15.39:0/3420009266 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4d10004ba0 con 0x557b09d611b0 2024-07-04T08:26:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.716+0000 7f4d2d7be740 10 client.27994 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.716+0000 7f4d2d7be740 10 client.27994 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:26:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.716+0000 7f4d2d7be740 20 client.27994 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.716+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557b09d489f0 0x557b0a14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.716+0000 7f4d2d7be740 4 client.27994 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.716+0000 7f4d2d7be740 1 -- 172.21.15.39:0/3420009266 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b0a0a6e30 con 0x557b09d489f0 2024-07-04T08:26:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.717+0000 7f4d17e00640 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557b09d489f0 0x557b0a14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.717+0000 7f4d17e00640 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557b09d489f0 0x557b0a14c3d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c34263616a1b04f server_cookie=f77a7911b55f0531 in_seq=0 out_seq=0 2024-07-04T08:26:41.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.717+0000 7f4d0e000640 10 client.27994 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:26:41.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d0e000640 1 -- 172.21.15.39:0/3420009266 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557b0a0a6e30 con 0x557b09d489f0 2024-07-04T08:26:41.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d0e000640 10 client.27994 handle_command_reply: tid=0 2024-07-04T08:26:41.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d2d7be740 1 client.27994 shutdown 2024-07-04T08:26:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d2d7be740 1 -- 172.21.15.39:0/3420009266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 msgr2=0x557b0a13a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a13a4d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000c00 tx=0x7f4d10009910 comp rx=0 tx=0).stop 2024-07-04T08:26:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d2d7be740 1 -- 172.21.15.39:0/3420009266 shutdown_connections 2024-07-04T08:26:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557b09d489f0 0x557b0a14c3d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a13aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/3420009266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a13a4d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.718+0000 7f4d0e000640 0 client.27994 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:26:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.719+0000 7f4d2d7be740 1 -- 172.21.15.39:0/3420009266 >> 172.21.15.39:0/3420009266 conn(0x557b09d7e460 msgr2=0x557b0a131b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.719+0000 7f4d2d7be740 1 -- 172.21.15.39:0/3420009266 shutdown_connections 2024-07-04T08:26:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.719+0000 7f4d2d7be740 1 -- 172.21.15.39:0/3420009266 wait complete. 2024-07-04T08:26:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.719+0000 7f4d2d7be740 20 client.27994 trim_cache size 0 max 16384 2024-07-04T08:26:41.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.735+0000 7f4d2d7be740 1 Processor -- start 2024-07-04T08:26:41.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.735+0000 7f4d2d7be740 1 -- start start 2024-07-04T08:26:41.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.735+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d611b0 0x557b0a130090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.735+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.735+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d8a070 0x557b0a134110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.735+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b67a0 con 0x557b09d69390 2024-07-04T08:26:41.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.735+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b82e0 con 0x557b09d611b0 2024-07-04T08:26:41.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.735+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b6e70 con 0x557b09d8a070 2024-07-04T08:26:41.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a133bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38960/0 (socket says 172.21.15.39:38960) 2024-07-04T08:26:41.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d17e00640 1 -- 172.21.15.39:0/2221079100 learned_addr learned my addr 172.21.15.39:0/2221079100 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:41.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d25200640 1 --2- 172.21.15.39:0/2221079100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d8a070 0x557b0a134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/2221079100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d611b0 0x557b0a130090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d17e00640 1 -- 172.21.15.39:0/2221079100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d8a070 msgr2=0x557b0a134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:41.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d17e00640 1 --2- 172.21.15.39:0/2221079100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d8a070 0x557b0a134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d17e00640 1 -- 172.21.15.39:0/2221079100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d611b0 msgr2=0x557b0a130090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:41.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d17e00640 1 --2- 172.21.15.39:0/2221079100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d611b0 0x557b0a130090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.736+0000 7f4d17e00640 1 -- 172.21.15.39:0/2221079100 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0a1387d0 con 0x557b09d69390 2024-07-04T08:26:41.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.737+0000 7f4d17e00640 1 --2- 172.21.15.39:0/2221079100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a133bd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x557b0a14ca90 tx=0x7f4d08018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c4c00f39282d6f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:41.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.737+0000 7f4d14c00640 1 -- 172.21.15.39:0/2221079100 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08003c40 con 0x557b09d69390 2024-07-04T08:26:41.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.737+0000 7f4d14c00640 1 -- 172.21.15.39:0/2221079100 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d08003de0 con 0x557b09d69390 2024-07-04T08:26:41.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.737+0000 7f4d14c00640 1 -- 172.21.15.39:0/2221079100 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080207f0 con 0x557b09d69390 2024-07-04T08:26:41.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.738+0000 7f4d2d7be740 1 -- 172.21.15.39:0/2221079100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 msgr2=0x557b0a133bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:41.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.738+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/2221079100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a133bd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x557b0a14ca90 tx=0x7f4d08018000 comp rx=0 tx=0).stop 2024-07-04T08:26:41.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.738+0000 7f4d2d7be740 1 -- 172.21.15.39:0/2221079100 shutdown_connections 2024-07-04T08:26:41.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.738+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/2221079100 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a133bd0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.738+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/2221079100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d8a070 0x557b0a134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.738+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/2221079100 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d611b0 0x557b0a130090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.738+0000 7f4d2d7be740 1 -- 172.21.15.39:0/2221079100 >> 172.21.15.39:0/2221079100 conn(0x557b09d7e460 msgr2=0x557b0a12f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:41.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.738+0000 7f4d2d7be740 1 -- 172.21.15.39:0/2221079100 shutdown_connections 2024-07-04T08:26:41.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.738+0000 7f4d2d7be740 1 -- 172.21.15.39:0/2221079100 wait complete. 2024-07-04T08:26:41.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.739+0000 7f4d2d7be740 1 -- start start 2024-07-04T08:26:41.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.739+0000 7f4d2d7be740 1 Processor -- start 2024-07-04T08:26:41.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.739+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a142a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.739+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a145be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.739+0000 7f4d2d7be740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.739+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b7c10 con 0x557b09d69390 2024-07-04T08:26:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.739+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b7540 con 0x557b09d8a070 2024-07-04T08:26:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.739+0000 7f4d2d7be740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201b4780 con 0x557b09d611b0 2024-07-04T08:26:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.739+0000 7f4d2d7be740 10 client.0 _ll_register_callbacks cb 0x7f4d1001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a145be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38962/0 (socket says 172.21.15.39:38962) 2024-07-04T08:26:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d17e00640 1 -- 172.21.15.39:0/1523292753 learned_addr learned my addr 172.21.15.39:0/1523292753 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d25200640 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d25200640 1 -- 172.21.15.39:0/1523292753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 msgr2=0x557b0a142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d25200640 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d25200640 1 -- 172.21.15.39:0/1523292753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 msgr2=0x557b0a145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d25200640 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d25200640 1 -- 172.21.15.39:0/1523292753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b0a146810 con 0x557b09d8a070 2024-07-04T08:26:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.740+0000 7f4d17e00640 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.741+0000 7f4d25200640 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146120 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4d18100ce0 tx=0x7f4d18005c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2defc1baf8fb48 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.741+0000 7f4d0e000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.741+0000 7f4d0e000640 1 -- 172.21.15.39:0/1523292753 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d180ca410 con 0x557b09d8a070 2024-07-04T08:26:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.741+0000 7f4d0e000640 1 -- 172.21.15.39:0/1523292753 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f4d18103460 con 0x557b09d8a070 2024-07-04T08:26:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.741+0000 7f4d2d7be740 1 -- 172.21.15.39:0/1523292753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d08003de0 con 0x557b09d8a070 2024-07-04T08:26:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.741+0000 7f4d0e000640 1 -- 172.21.15.39:0/1523292753 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d18103660 con 0x557b09d8a070 2024-07-04T08:26:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.741+0000 7f4d2d7be740 10 client.28064 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.741+0000 7f4d2d7be740 1 -- 172.21.15.39:0/1523292753 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557b0a1b4ac0 con 0x557b09d8a070 2024-07-04T08:26:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.741+0000 7f4d0e000640 1 -- 172.21.15.39:0/1523292753 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d08003de0 con 0x557b09d8a070 2024-07-04T08:26:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.742+0000 7f4d0e000640 1 -- 172.21.15.39:0/1523292753 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4d1804d030 con 0x557b09d8a070 2024-07-04T08:26:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.742+0000 7f4d2d7be740 10 client.28064 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.742+0000 7f4d2d7be740 10 client.28064 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:26:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.742+0000 7f4d2d7be740 20 client.28064 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.742+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557b09d1dda0 0x557b0a1b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.742+0000 7f4d2d7be740 4 client.28064 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:41.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.742+0000 7f4d2d7be740 1 -- 172.21.15.39:0/1523292753 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b0a0a28d0 con 0x557b09d1dda0 2024-07-04T08:26:41.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.742+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557b09d1dda0 0x557b0a1b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:41.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d0e000640 10 client.28064 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:26:41.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d1fe00640 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557b09d1dda0 0x557b0a1b7180 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=577093e4e23a4f7c server_cookie=2af7e906c7e85ec0 in_seq=0 out_seq=0 2024-07-04T08:26:41.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d0e000640 1 -- 172.21.15.39:0/1523292753 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b0a0a28d0 con 0x557b09d1dda0 2024-07-04T08:26:41.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d0e000640 10 client.28064 handle_command_reply: tid=0 2024-07-04T08:26:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d2d7be740 1 client.28064 shutdown 2024-07-04T08:26:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d2d7be740 1 -- 172.21.15.39:0/1523292753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 msgr2=0x557b0a146120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146120 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4d18100ce0 tx=0x7f4d18005c30 comp rx=0 tx=0).stop 2024-07-04T08:26:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d2d7be740 1 -- 172.21.15.39:0/1523292753 shutdown_connections 2024-07-04T08:26:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x557b09d1dda0 0x557b0a1b7180 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557b09d69390 0x557b0a145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557b09d8a070 0x557b0a146120 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d2d7be740 1 --2- 172.21.15.39:0/1523292753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557b09d611b0 0x557b0a142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d2d7be740 1 -- 172.21.15.39:0/1523292753 >> 172.21.15.39:0/1523292753 conn(0x557b09d7e460 msgr2=0x557b0a12c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.743+0000 7f4d0e000640 0 client.28064 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:26:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.744+0000 7f4d2d7be740 1 -- 172.21.15.39:0/1523292753 shutdown_connections 2024-07-04T08:26:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.744+0000 7f4d2d7be740 1 -- 172.21.15.39:0/1523292753 wait complete. 2024-07-04T08:26:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.744+0000 7f4d2d7be740 20 client.28064 trim_cache size 0 max 16384 2024-07-04T08:26:41.749 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.744+0000 7f4d25c00640 1 -- 172.21.15.39:0/3436600962 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4d040b2960 msgr2=0x7f4d040b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.744+0000 7f4d25c00640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4d040b2960 0x7f4d040b4e70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009790 tx=0x7f4d08009800 comp rx=0 tx=0).stop 2024-07-04T08:26:41.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.744+0000 7f4d25c00640 1 -- 172.21.15.39:0/3436600962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d2010f7a0 msgr2=0x7f4d201a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:41.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.745+0000 7f4d25c00640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d2010f7a0 0x7f4d201a5620 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4d1802add0 tx=0x7f4d18030d80 comp rx=0 tx=0).stop 2024-07-04T08:26:41.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.745+0000 7f4d25c00640 1 -- 172.21.15.39:0/3436600962 shutdown_connections 2024-07-04T08:26:41.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.745+0000 7f4d25c00640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4d040b2960 0x7f4d040b4e70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.745+0000 7f4d25c00640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d2010ee50 0x7f4d201a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.745+0000 7f4d25c00640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d2010f7a0 0x7f4d201a5620 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.745+0000 7f4d25c00640 1 --2- 172.21.15.39:0/3436600962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4d20109510 0x7f4d2019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.745+0000 7f4d25c00640 1 -- 172.21.15.39:0/3436600962 >> 172.21.15.39:0/3436600962 conn(0x7f4d20078b10 msgr2=0x7f4d2010c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.746+0000 7f4d25c00640 1 -- 172.21.15.39:0/3436600962 shutdown_connections 2024-07-04T08:26:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:41.746+0000 7f4d25c00640 1 -- 172.21.15.39:0/3436600962 wait complete. 2024-07-04T08:26:42.661 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:26:43.162 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:43.521 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:43 smithi039 ceph-mon[32062]: pgmap v835: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-07-04T08:26:43.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:43 smithi133 ceph-mon[37050]: pgmap v835: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-07-04T08:26:43.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:43 smithi050 ceph-mon[36577]: pgmap v835: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-07-04T08:26:45.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:45 smithi133 ceph-mon[37050]: pgmap v836: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-07-04T08:26:45.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:45 smithi039 ceph-mon[32062]: pgmap v836: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-07-04T08:26:45.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.618+0000 7fca3b200640 1 -- 172.21.15.39:0/2487164900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c108840 msgr2=0x7fca3c108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:45.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.618+0000 7fca3b200640 1 --2- 172.21.15.39:0/2487164900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c108840 0x7fca3c108c50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fca28009860 tx=0x7fca2802a340 comp rx=0 tx=0).stop 2024-07-04T08:26:45.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.618+0000 7fca3b200640 1 -- 172.21.15.39:0/2487164900 shutdown_connections 2024-07-04T08:26:45.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.618+0000 7fca3b200640 1 --2- 172.21.15.39:0/2487164900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca3c109190 0x7fca3c10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.618+0000 7fca3b200640 1 --2- 172.21.15.39:0/2487164900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c108840 0x7fca3c108c50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.618+0000 7fca3b200640 1 --2- 172.21.15.39:0/2487164900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca3c102f00 0x7fca3c103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.618+0000 7fca3b200640 1 -- 172.21.15.39:0/2487164900 >> 172.21.15.39:0/2487164900 conn(0x7fca3c0fe990 msgr2=0x7fca3c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:45.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.618+0000 7fca3b200640 1 -- 172.21.15.39:0/2487164900 shutdown_connections 2024-07-04T08:26:45.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.618+0000 7fca3b200640 1 -- 172.21.15.39:0/2487164900 wait complete. 2024-07-04T08:26:45.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.620+0000 7fca3b200640 1 Processor -- start 2024-07-04T08:26:45.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.620+0000 7fca3b200640 1 -- start start 2024-07-04T08:26:45.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca3b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca3c102f00 0x7fca3c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca3b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca3c108840 0x7fca3c0751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca3b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c109190 0x7fca3c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca3b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c112e50 con 0x7fca3c102f00 2024-07-04T08:26:45.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca3b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c112cd0 con 0x7fca3c108840 2024-07-04T08:26:45.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca3b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c112fd0 con 0x7fca3c109190 2024-07-04T08:26:45.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca39e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca3c102f00 0x7fca3c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca39e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca3c102f00 0x7fca3c074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38986/0 (socket says 172.21.15.39:38986) 2024-07-04T08:26:45.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca39e00640 1 -- 172.21.15.39:0/130189859 learned_addr learned my addr 172.21.15.39:0/130189859 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:45.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca39400640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca3c108840 0x7fca3c0751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.621+0000 7fca3a800640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c109190 0x7fca3c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.622+0000 7fca39400640 1 -- 172.21.15.39:0/130189859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c109190 msgr2=0x7fca3c0756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.622+0000 7fca39400640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c109190 0x7fca3c0756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.622+0000 7fca39400640 1 -- 172.21.15.39:0/130189859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca3c102f00 msgr2=0x7fca3c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.622+0000 7fca39400640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca3c102f00 0x7fca3c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.622+0000 7fca39400640 1 -- 172.21.15.39:0/130189859 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c1a7b40 con 0x7fca3c108840 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.622+0000 7fca39400640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca3c108840 0x7fca3c0751a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fca28009780 tx=0x7fca28032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.622+0000 7fca32a00640 1 -- 172.21.15.39:0/130189859 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28037040 con 0x7fca3c108840 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.622+0000 7fca32a00640 1 -- 172.21.15.39:0/130189859 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca28030c10 con 0x7fca3c108840 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.622+0000 7fca32a00640 1 -- 172.21.15.39:0/130189859 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca280035f0 con 0x7fca3c108840 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.623+0000 7fca3b200640 1 -- 172.21.15.39:0/130189859 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c1a7dd0 con 0x7fca3c108840 2024-07-04T08:26:45.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.623+0000 7fca3b200640 1 -- 172.21.15.39:0/130189859 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3c1a8170 con 0x7fca3c108840 2024-07-04T08:26:45.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.626+0000 7fca47705740 1 Processor -- start 2024-07-04T08:26:45.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.626+0000 7fca47705740 1 -- start start 2024-07-04T08:26:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca47705740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744efc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca47705740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744efce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca32a00640 1 -- 172.21.15.39:0/130189859 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fca28043020 con 0x7fca3c108840 2024-07-04T08:26:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca47705740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744efd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca47705740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564744e72ee0 con 0x564744b58070 2024-07-04T08:26:45.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca47705740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564744e62480 con 0x564744b37390 2024-07-04T08:26:45.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca47705740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564744e664c0 con 0x564744b2f1b0 2024-07-04T08:26:45.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca3a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744efc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca39400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744efd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca3a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744efc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38990/0 (socket says 172.21.15.39:38990) 2024-07-04T08:26:45.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.627+0000 7fca3a800640 1 -- 172.21.15.39:0/1914448546 learned_addr learned my addr 172.21.15.39:0/1914448546 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.628+0000 7fca39e00640 1 --2- 172.21.15.39:0/1914448546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744efce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.628+0000 7fca39400640 1 -- 172.21.15.39:0/1914448546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 msgr2=0x564744efce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.628+0000 7fca39400640 1 --2- 172.21.15.39:0/1914448546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744efce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.628+0000 7fca39400640 1 -- 172.21.15.39:0/1914448546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 msgr2=0x564744efc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.628+0000 7fca39400640 1 --2- 172.21.15.39:0/1914448546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744efc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.628+0000 7fca39400640 1 -- 172.21.15.39:0/1914448546 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564744dfa650 con 0x564744b37390 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.628+0000 7fca3a800640 1 --2- 172.21.15.39:0/1914448546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744efc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.628+0000 7fca39400640 1 --2- 172.21.15.39:0/1914448546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744efd3d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fca280fede0 tx=0x7fca28107b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c83c7e66d637d80 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.629+0000 7fca27400640 1 -- 172.21.15.39:0/1914448546 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28110070 con 0x564744b37390 2024-07-04T08:26:45.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.629+0000 7fca27400640 1 -- 172.21.15.39:0/1914448546 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2810c420 con 0x564744b37390 2024-07-04T08:26:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.629+0000 7fca27400640 1 -- 172.21.15.39:0/1914448546 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28103330 con 0x564744b37390 2024-07-04T08:26:45.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.629+0000 7fca32a00640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fca200b2960 0x7fca200b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca47705740 1 -- 172.21.15.39:0/1914448546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 msgr2=0x564744efd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca47705740 1 --2- 172.21.15.39:0/1914448546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744efd3d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fca280fede0 tx=0x7fca28107b20 comp rx=0 tx=0).stop 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca32a00640 1 -- 172.21.15.39:0/130189859 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fca28101940 con 0x7fca3c108840 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca47705740 1 -- 172.21.15.39:0/1914448546 shutdown_connections 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca47705740 1 --2- 172.21.15.39:0/1914448546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744efd3d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca47705740 1 --2- 172.21.15.39:0/1914448546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744efce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca47705740 1 --2- 172.21.15.39:0/1914448546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744efc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca47705740 1 -- 172.21.15.39:0/1914448546 >> 172.21.15.39:0/1914448546 conn(0x564744b4c460 msgr2=0x564744efa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca3a800640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fca200b2960 0x7fca200b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.630+0000 7fca47705740 1 -- 172.21.15.39:0/1914448546 shutdown_connections 2024-07-04T08:26:45.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.631+0000 7fca47705740 1 -- 172.21.15.39:0/1914448546 wait complete. 2024-07-04T08:26:45.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.631+0000 7fca3a800640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fca200b2960 0x7fca200b4e70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fca34005fd0 tx=0x7fca34010040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.631+0000 7fca47705740 1 -- start start 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.631+0000 7fca47705740 1 Processor -- start 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.632+0000 7fca47705740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f08440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.632+0000 7fca47705740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b37390 0x564744f08980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.632+0000 7fca47705740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b58070 0x564744f14150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.632+0000 7fca47705740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564744cc6700 con 0x564744b37390 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.632+0000 7fca47705740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564744e09bb0 con 0x564744b58070 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.632+0000 7fca47705740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564744f0aef0 con 0x564744b2f1b0 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.632+0000 7fca47705740 10 client.0 _ll_register_callbacks cb 0x564744e5b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.632+0000 7fca39400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b37390 0x564744f08980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.632+0000 7fca3a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b58070 0x564744f14150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca39e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f08440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca39400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b37390 0x564744f08980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39002/0 (socket says 172.21.15.39:39002) 2024-07-04T08:26:45.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca39400640 1 -- 172.21.15.39:0/1916652687 learned_addr learned my addr 172.21.15.39:0/1916652687 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:45.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca3a800640 1 -- 172.21.15.39:0/1916652687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 msgr2=0x564744f08440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:45.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca3a800640 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f08440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca3a800640 1 -- 172.21.15.39:0/1916652687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b37390 msgr2=0x564744f08980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:45.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca3a800640 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b37390 0x564744f08980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca3a800640 1 -- 172.21.15.39:0/1916652687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564744bc3650 con 0x564744b58070 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca39e00640 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f08440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca39400640 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b37390 0x564744f08980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca3a800640 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b58070 0x564744f14150 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca340045b0 tx=0x7fca34004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa1df08e7ef73f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.633+0000 7fca1be00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.634+0000 7fca1be00640 1 -- 172.21.15.39:0/1916652687 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34018460 con 0x564744b58070 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.634+0000 7fca47705740 1 -- 172.21.15.39:0/1916652687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2810c420 con 0x564744b58070 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.634+0000 7fca1be00640 1 -- 172.21.15.39:0/1916652687 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fca34018600 con 0x564744b58070 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.634+0000 7fca1be00640 1 -- 172.21.15.39:0/1916652687 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca34032050 con 0x564744b58070 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.634+0000 7fca47705740 10 client.28076 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.634+0000 7fca47705740 1 -- 172.21.15.39:0/1916652687 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564744f16e50 con 0x564744b58070 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.635+0000 7fca1be00640 1 -- 172.21.15.39:0/1916652687 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca2810c420 con 0x564744b58070 2024-07-04T08:26:45.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.635+0000 7fca1be00640 1 -- 172.21.15.39:0/1916652687 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fca3400b040 con 0x564744b58070 2024-07-04T08:26:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.635+0000 7fca47705740 10 client.28076 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.635+0000 7fca47705740 10 client.28076 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:26:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.635+0000 7fca47705740 20 client.28076 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.635+0000 7fca47705740 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x564744b169f0 0x564744f1a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.635+0000 7fca47705740 4 client.28076 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:45.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.635+0000 7fca47705740 1 -- 172.21.15.39:0/1916652687 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564744e74e30 con 0x564744b169f0 2024-07-04T08:26:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.636+0000 7fca39e00640 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x564744b169f0 0x564744f1a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.636+0000 7fca39e00640 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x564744b169f0 0x564744f1a330 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=239e1a9eb3537526 server_cookie=44dbf387753408e6 in_seq=0 out_seq=0 2024-07-04T08:26:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.636+0000 7fca1be00640 10 client.28076 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:26:45.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.637+0000 7fca1be00640 1 -- 172.21.15.39:0/1916652687 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564744e74e30 con 0x564744b169f0 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.637+0000 7fca1be00640 10 client.28076 handle_command_reply: tid=0 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 client.28076 shutdown 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 -- 172.21.15.39:0/1916652687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b58070 msgr2=0x564744f14150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b58070 0x564744f14150 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca340045b0 tx=0x7fca34004ab0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 -- 172.21.15.39:0/1916652687 shutdown_connections 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x564744b169f0 0x564744f1a330 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b37390 0x564744f08980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b58070 0x564744f14150 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 --2- 172.21.15.39:0/1916652687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f08440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca1be00640 0 client.28076 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 -- 172.21.15.39:0/1916652687 >> 172.21.15.39:0/1916652687 conn(0x564744b4c460 msgr2=0x564744effb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 -- 172.21.15.39:0/1916652687 shutdown_connections 2024-07-04T08:26:45.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 1 -- 172.21.15.39:0/1916652687 wait complete. 2024-07-04T08:26:45.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.638+0000 7fca47705740 20 client.28076 trim_cache size 0 max 16384 2024-07-04T08:26:45.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:45 smithi050 ceph-mon[36577]: pgmap v836: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-07-04T08:26:45.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca47705740 1 Processor -- start 2024-07-04T08:26:45.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca47705740 1 -- start start 2024-07-04T08:26:45.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca47705740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b2f1b0 0x564744f00410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca47705740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f09e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca47705740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b58070 0x564744f13f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca47705740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b2410 con 0x564744b2f1b0 2024-07-04T08:26:45.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca47705740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b3f50 con 0x564744b37390 2024-07-04T08:26:45.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca47705740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b2ae0 con 0x564744b58070 2024-07-04T08:26:45.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca39400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f09e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.654+0000 7fca39e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b2f1b0 0x564744f00410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca3a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b58070 0x564744f13f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca39400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f09e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54720/0 (socket says 172.21.15.39:54720) 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca3a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b58070 0x564744f13f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:40348/0 (socket says 172.21.15.39:40348) 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca39400640 1 -- 172.21.15.39:0/1257138068 learned_addr learned my addr 172.21.15.39:0/1257138068 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca3a800640 1 -- 172.21.15.39:0/1257138068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 msgr2=0x564744f09e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca3a800640 1 --2- 172.21.15.39:0/1257138068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f09e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca3a800640 1 -- 172.21.15.39:0/1257138068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b2f1b0 msgr2=0x564744f00410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca3a800640 1 --2- 172.21.15.39:0/1257138068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b2f1b0 0x564744f00410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca3a800640 1 -- 172.21.15.39:0/1257138068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564744efb130 con 0x564744b58070 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.655+0000 7fca3a800640 1 --2- 172.21.15.39:0/1257138068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b58070 0x564744f13f40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fca3401a8b0 tx=0x7fca3400df00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d51a71a3a1ceeed7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.656+0000 7fca26a00640 1 -- 172.21.15.39:0/1257138068 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3401a650 con 0x564744b58070 2024-07-04T08:26:45.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.656+0000 7fca26a00640 1 -- 172.21.15.39:0/1257138068 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca34029070 con 0x564744b58070 2024-07-04T08:26:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.656+0000 7fca26a00640 1 -- 172.21.15.39:0/1257138068 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3402eb60 con 0x564744b58070 2024-07-04T08:26:45.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 -- 172.21.15.39:0/1257138068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b58070 msgr2=0x564744f13f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 --2- 172.21.15.39:0/1257138068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b58070 0x564744f13f40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fca3401a8b0 tx=0x7fca3400df00 comp rx=0 tx=0).stop 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 -- 172.21.15.39:0/1257138068 shutdown_connections 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 --2- 172.21.15.39:0/1257138068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b58070 0x564744f13f40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 --2- 172.21.15.39:0/1257138068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 --2- 172.21.15.39:0/1257138068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b2f1b0 0x564744f00410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 -- 172.21.15.39:0/1257138068 >> 172.21.15.39:0/1257138068 conn(0x564744b4c460 msgr2=0x564744f09760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 -- 172.21.15.39:0/1257138068 shutdown_connections 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 -- 172.21.15.39:0/1257138068 wait complete. 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.657+0000 7fca47705740 1 -- start start 2024-07-04T08:26:45.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.658+0000 7fca47705740 1 Processor -- start 2024-07-04T08:26:45.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.658+0000 7fca47705740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.658+0000 7fca47705740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.658+0000 7fca47705740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744f0be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.658+0000 7fca47705740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b3880 con 0x564744b58070 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.658+0000 7fca47705740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b31b0 con 0x564744b37390 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.658+0000 7fca47705740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c1b03f0 con 0x564744b2f1b0 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.658+0000 7fca47705740 10 client.0 _ll_register_callbacks cb 0x7fca34009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca3a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744f0be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f0b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54722/0 (socket says 172.21.15.39:54722) 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39400640 1 -- 172.21.15.39:0/818512662 learned_addr learned my addr 172.21.15.39:0/818512662 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:45.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39400640 1 -- 172.21.15.39:0/818512662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 msgr2=0x564744f10450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39400640 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f10450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39400640 1 -- 172.21.15.39:0/818512662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 msgr2=0x564744f0be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39400640 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744f0be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39400640 1 -- 172.21.15.39:0/818512662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564744f0c380 con 0x564744b37390 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39e00640 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca39400640 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f0b900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fca280057a0 tx=0x7fca280303b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2c511f9c5332d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.659+0000 7fca1be00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.660+0000 7fca1be00640 1 -- 172.21.15.39:0/818512662 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28110040 con 0x564744b37390 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.660+0000 7fca1be00640 1 -- 172.21.15.39:0/818512662 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fca280c8de0 con 0x564744b37390 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.660+0000 7fca47705740 1 -- 172.21.15.39:0/818512662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca34029070 con 0x564744b37390 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.660+0000 7fca1be00640 1 -- 172.21.15.39:0/818512662 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fca280c4830 con 0x564744b37390 2024-07-04T08:26:45.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.660+0000 7fca47705740 10 client.28082 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:45.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.660+0000 7fca47705740 1 -- 172.21.15.39:0/818512662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564744f826c0 con 0x564744b37390 2024-07-04T08:26:45.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.660+0000 7fca1be00640 1 -- 172.21.15.39:0/818512662 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca34029070 con 0x564744b37390 2024-07-04T08:26:45.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.661+0000 7fca1be00640 1 -- 172.21.15.39:0/818512662 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fca280e2030 con 0x564744b37390 2024-07-04T08:26:45.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.661+0000 7fca47705740 10 client.28082 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:45.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.661+0000 7fca47705740 10 client.28082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:26:45.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.661+0000 7fca47705740 20 client.28082 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:45.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.661+0000 7fca47705740 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x564744aebda0 0x564744f84c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:45.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.661+0000 7fca47705740 4 client.28082 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.661+0000 7fca47705740 1 -- 172.21.15.39:0/818512662 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564744e708d0 con 0x564744aebda0 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.661+0000 7fca39e00640 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x564744aebda0 0x564744f84c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca39e00640 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x564744aebda0 0x564744f84c30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bdca56a566fab4b server_cookie=816417a31f098e62 in_seq=0 out_seq=0 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca1be00640 10 client.28082 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca1be00640 1 -- 172.21.15.39:0/818512662 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564744e708d0 con 0x564744aebda0 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca1be00640 10 client.28082 handle_command_reply: tid=0 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca47705740 1 client.28082 shutdown 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca47705740 1 -- 172.21.15.39:0/818512662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 msgr2=0x564744f0b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca47705740 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f0b900 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fca280057a0 tx=0x7fca280303b0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca47705740 1 -- 172.21.15.39:0/818512662 shutdown_connections 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca47705740 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x564744aebda0 0x564744f84c30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca47705740 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564744b58070 0x564744f0be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.663+0000 7fca1be00640 0 client.28082 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:26:45.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.662+0000 7fca47705740 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564744b37390 0x564744f0b900 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.663+0000 7fca47705740 1 --2- 172.21.15.39:0/818512662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564744b2f1b0 0x564744f10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.663+0000 7fca47705740 1 -- 172.21.15.39:0/818512662 >> 172.21.15.39:0/818512662 conn(0x564744b4c460 msgr2=0x564744f001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:45.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.663+0000 7fca47705740 1 -- 172.21.15.39:0/818512662 shutdown_connections 2024-07-04T08:26:45.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.663+0000 7fca47705740 1 -- 172.21.15.39:0/818512662 wait complete. 2024-07-04T08:26:45.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.663+0000 7fca47705740 20 client.28082 trim_cache size 0 max 16384 2024-07-04T08:26:45.655 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:45.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 -- 172.21.15.39:0/130189859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fca200b2960 msgr2=0x7fca200b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:45.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fca200b2960 0x7fca200b4e70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fca34005fd0 tx=0x7fca34010040 comp rx=0 tx=0).stop 2024-07-04T08:26:45.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 -- 172.21.15.39:0/130189859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca3c108840 msgr2=0x7fca3c0751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:45.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca3c108840 0x7fca3c0751a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fca28009780 tx=0x7fca28032d50 comp rx=0 tx=0).stop 2024-07-04T08:26:45.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 -- 172.21.15.39:0/130189859 shutdown_connections 2024-07-04T08:26:45.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fca200b2960 0x7fca200b4e70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.656 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c109190 0x7fca3c0756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca3c108840 0x7fca3c0751a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 --2- 172.21.15.39:0/130189859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca3c102f00 0x7fca3c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:45.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 -- 172.21.15.39:0/130189859 >> 172.21.15.39:0/130189859 conn(0x7fca3c0fe990 msgr2=0x7fca3c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:45.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 -- 172.21.15.39:0/130189859 shutdown_connections 2024-07-04T08:26:45.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:45.664+0000 7fca3b200640 1 -- 172.21.15.39:0/130189859 wait complete. 2024-07-04T08:26:46.588 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:26:47.086 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:47.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:47 smithi133 ceph-mon[37050]: pgmap v837: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-07-04T08:26:47.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:47 smithi039 ceph-mon[32062]: pgmap v837: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-07-04T08:26:47.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:47 smithi050 ceph-mon[36577]: pgmap v837: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 13 op/s 2024-07-04T08:26:48.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:48 smithi039 ceph-mon[32062]: pgmap v838: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-07-04T08:26:48.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:48 smithi050 ceph-mon[36577]: pgmap v838: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-07-04T08:26:48.805 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:48 smithi133 ceph-mon[37050]: pgmap v838: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-07-04T08:26:49.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.330+0000 7f0188400640 1 -- 172.21.15.39:0/1397283280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 msgr2=0x7f0180109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.330+0000 7f0188400640 1 --2- 172.21.15.39:0/1397283280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 0x7f0180109b10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0174009900 tx=0x7f017402a410 comp rx=0 tx=0).stop 2024-07-04T08:26:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.330+0000 7f0188400640 1 -- 172.21.15.39:0/1397283280 shutdown_connections 2024-07-04T08:26:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.330+0000 7f0188400640 1 --2- 172.21.15.39:0/1397283280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018010f990 0x7f01801161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.330+0000 7f0188400640 1 --2- 172.21.15.39:0/1397283280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f018010f040 0x7f018010f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.330+0000 7f0188400640 1 --2- 172.21.15.39:0/1397283280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 0x7f0180109b10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.330+0000 7f0188400640 1 -- 172.21.15.39:0/1397283280 >> 172.21.15.39:0/1397283280 conn(0x7f0180077630 msgr2=0x7f0180107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.330+0000 7f0188400640 1 -- 172.21.15.39:0/1397283280 shutdown_connections 2024-07-04T08:26:49.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.330+0000 7f0188400640 1 -- 172.21.15.39:0/1397283280 wait complete. 2024-07-04T08:26:49.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.331+0000 7f0188400640 1 Processor -- start 2024-07-04T08:26:49.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.332+0000 7f0188400640 1 -- start start 2024-07-04T08:26:49.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.332+0000 7f0188400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 0x7f01801a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.332+0000 7f0188400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018010f040 0x7f01801a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.332+0000 7f0188400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f018010f990 0x7f01801a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.332+0000 7f0188400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0180119660 con 0x7f0180109700 2024-07-04T08:26:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.332+0000 7f0188400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f01801194e0 con 0x7f018010f040 2024-07-04T08:26:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.332+0000 7f0188400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f01801197e0 con 0x7f018010f990 2024-07-04T08:26:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 0x7f01801a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 0x7f01801a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39040/0 (socket says 172.21.15.39:39040) 2024-07-04T08:26:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185a00640 1 -- 172.21.15.39:0/390165424 learned_addr learned my addr 172.21.15.39:0/390165424 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185000640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018010f040 0x7f01801a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0186400640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f018010f990 0x7f01801a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185000640 1 -- 172.21.15.39:0/390165424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f018010f990 msgr2=0x7f01801a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:49.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185000640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f018010f990 0x7f01801a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185000640 1 -- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 msgr2=0x7f01801a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:49.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185000640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 0x7f01801a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185000640 1 -- 172.21.15.39:0/390165424 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01801a6020 con 0x7f018010f040 2024-07-04T08:26:49.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.333+0000 7f0185a00640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 0x7f01801a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:49.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.334+0000 7f0185000640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018010f040 0x7f01801a0650 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f018006bf70 tx=0x7f016c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:49.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.334+0000 7f017e000640 1 -- 172.21.15.39:0/390165424 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c00cc40 con 0x7f018010f040 2024-07-04T08:26:49.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.334+0000 7f0188400640 1 -- 172.21.15.39:0/390165424 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01801a6310 con 0x7f018010f040 2024-07-04T08:26:49.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.334+0000 7f017e000640 1 -- 172.21.15.39:0/390165424 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f016c00cde0 con 0x7f018010f040 2024-07-04T08:26:49.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.334+0000 7f017e000640 1 -- 172.21.15.39:0/390165424 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c004a00 con 0x7f018010f040 2024-07-04T08:26:49.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.334+0000 7f0188400640 1 -- 172.21.15.39:0/390165424 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01801aca70 con 0x7f018010f040 2024-07-04T08:26:49.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.338+0000 7f018e9bb740 1 Processor -- start 2024-07-04T08:26:49.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.338+0000 7f018e9bb740 1 -- start start 2024-07-04T08:26:49.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.338+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca6c070 0x55e92ce107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.338+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca431b0 0x55e92ce10cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.338+0000 7f017e000640 1 -- 172.21.15.39:0/390165424 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f016c004ba0 con 0x7f018010f040 2024-07-04T08:26:49.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.338+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca4b390 0x55e92ce11230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.338+0000 7f018e9bb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e92cd86ee0 con 0x55e92ca6c070 2024-07-04T08:26:49.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.338+0000 7f018e9bb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e92cd76480 con 0x55e92ca4b390 2024-07-04T08:26:49.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.338+0000 7f018e9bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e92cd7a4c0 con 0x55e92ca431b0 2024-07-04T08:26:49.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0186400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca6c070 0x55e92ce107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0185000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca4b390 0x55e92ce11230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0185a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca431b0 0x55e92ce10cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0185a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca431b0 0x55e92ce10cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:40392/0 (socket says 172.21.15.39:40392) 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0185a00640 1 -- 172.21.15.39:0/3503904442 learned_addr learned my addr 172.21.15.39:0/3503904442 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0185a00640 1 -- 172.21.15.39:0/3503904442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca4b390 msgr2=0x55e92ce11230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0185a00640 1 --2- 172.21.15.39:0/3503904442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca4b390 0x55e92ce11230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0185a00640 1 -- 172.21.15.39:0/3503904442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca6c070 msgr2=0x55e92ce107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0185a00640 1 --2- 172.21.15.39:0/3503904442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca6c070 0x55e92ce107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0185a00640 1 -- 172.21.15.39:0/3503904442 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92cd0e650 con 0x55e92ca431b0 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.339+0000 7f0186400640 1 --2- 172.21.15.39:0/3503904442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca6c070 0x55e92ce107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:49.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.340+0000 7f0185a00640 1 --2- 172.21.15.39:0/3503904442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca431b0 0x55e92ce10cf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0174000c00 tx=0x7f0174005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9188bcb90a1b994d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.340+0000 7f0172a00640 1 -- 172.21.15.39:0/3503904442 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174035040 con 0x55e92ca431b0 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.340+0000 7f0172a00640 1 -- 172.21.15.39:0/3503904442 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0174031820 con 0x55e92ca431b0 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.340+0000 7f0172a00640 1 -- 172.21.15.39:0/3503904442 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01740049e0 con 0x55e92ca431b0 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.341+0000 7f017e000640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01680b2960 0x7f01680b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.341+0000 7f017e000640 1 -- 172.21.15.39:0/390165424 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f016c0e2900 con 0x7f018010f040 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.341+0000 7f0186400640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01680b2960 0x7f01680b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.341+0000 7f018e9bb740 1 -- 172.21.15.39:0/3503904442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca431b0 msgr2=0x55e92ce10cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.341+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3503904442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca431b0 0x55e92ce10cf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0174000c00 tx=0x7f0174005ed0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.342+0000 7f018e9bb740 1 -- 172.21.15.39:0/3503904442 shutdown_connections 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.342+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3503904442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca4b390 0x55e92ce11230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.342+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3503904442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca431b0 0x55e92ce10cf0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.342+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3503904442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca6c070 0x55e92ce107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.342+0000 7f018e9bb740 1 -- 172.21.15.39:0/3503904442 >> 172.21.15.39:0/3503904442 conn(0x55e92ca60460 msgr2=0x55e92ce0e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:49.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.342+0000 7f018e9bb740 1 -- 172.21.15.39:0/3503904442 shutdown_connections 2024-07-04T08:26:49.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.342+0000 7f0186400640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01680b2960 0x7f01680b4e70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0178009fd0 tx=0x7f0178009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:49.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.342+0000 7f018e9bb740 1 -- 172.21.15.39:0/3503904442 wait complete. 2024-07-04T08:26:49.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.342+0000 7f018e9bb740 1 -- start start 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.343+0000 7f018e9bb740 1 Processor -- start 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.343+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce1c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.343+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce1c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.343+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce28280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.343+0000 7f018e9bb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55e92cbda700 con 0x55e92ca4b390 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.343+0000 7f018e9bb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e92cd1dbb0 con 0x55e92ca431b0 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.343+0000 7f018e9bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e92ce1ef30 con 0x55e92ca6c070 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.343+0000 7f018e9bb740 10 client.0 _ll_register_callbacks cb 0x55e92cd6f570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce1c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce1c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54750/0 (socket says 172.21.15.39:54750) 2024-07-04T08:26:49.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185a00640 1 -- 172.21.15.39:0/3374130927 learned_addr learned my addr 172.21.15.39:0/3374130927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185000640 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce1c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0186400640 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce28280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185a00640 1 -- 172.21.15.39:0/3374130927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 msgr2=0x55e92ce28280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185a00640 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce28280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185a00640 1 -- 172.21.15.39:0/3374130927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 msgr2=0x55e92ce1c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185a00640 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce1c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185a00640 1 -- 172.21.15.39:0/3374130927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92cad7650 con 0x55e92ca431b0 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.344+0000 7f0185000640 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce1c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.345+0000 7f0185a00640 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce1c370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f01740042a0 tx=0x7f01740054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd4acb1d3db5e7b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.345+0000 7f0167400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.345+0000 7f0167400640 1 -- 172.21.15.39:0/3374130927 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017404f060 con 0x55e92ca431b0 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.345+0000 7f018e9bb740 1 -- 172.21.15.39:0/3374130927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0174031820 con 0x55e92ca431b0 2024-07-04T08:26:49.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.345+0000 7f0167400640 1 -- 172.21.15.39:0/3374130927 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f0174004480 con 0x55e92ca431b0 2024-07-04T08:26:49.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.345+0000 7f0167400640 1 -- 172.21.15.39:0/3374130927 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0174056050 con 0x55e92ca431b0 2024-07-04T08:26:49.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.345+0000 7f018e9bb740 10 client.28094 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:49.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.345+0000 7f018e9bb740 1 -- 172.21.15.39:0/3374130927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e92ce2af50 con 0x55e92ca431b0 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.346+0000 7f0167400640 1 -- 172.21.15.39:0/3374130927 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0174031820 con 0x55e92ca431b0 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.346+0000 7f0167400640 1 -- 172.21.15.39:0/3374130927 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f01740466f0 con 0x55e92ca431b0 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.346+0000 7f018e9bb740 10 client.28094 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.346+0000 7f018e9bb740 10 client.28094 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.346+0000 7f018e9bb740 20 client.28094 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.346+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55e92ca2a9f0 0x55e92ce2e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.346+0000 7f018e9bb740 4 client.28094 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.346+0000 7f018e9bb740 1 -- 172.21.15.39:0/3374130927 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e92cd88e30 con 0x55e92ca2a9f0 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.346+0000 7f0185000640 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55e92ca2a9f0 0x55e92ce2e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.347+0000 7f0185000640 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55e92ca2a9f0 0x55e92ce2e2b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9346bbecc49efc1f server_cookie=2d752ce499d8997a in_seq=0 out_seq=0 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.347+0000 7f0167400640 10 client.28094 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:26:49.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f0167400640 1 -- 172.21.15.39:0/3374130927 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e92cd88e30 con 0x55e92ca2a9f0 2024-07-04T08:26:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f0167400640 10 client.28094 handle_command_reply: tid=0 2024-07-04T08:26:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 client.28094 shutdown 2024-07-04T08:26:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 -- 172.21.15.39:0/3374130927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 msgr2=0x55e92ce1c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce1c370 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f01740042a0 tx=0x7f01740054e0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 -- 172.21.15.39:0/3374130927 shutdown_connections 2024-07-04T08:26:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55e92ca2a9f0 0x55e92ce2e2b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce1c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce28280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3374130927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce1c370 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 -- 172.21.15.39:0/3374130927 >> 172.21.15.39:0/3374130927 conn(0x55e92ca60460 msgr2=0x55e92ce13990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f0167400640 0 client.28094 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:26:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.348+0000 7f018e9bb740 1 -- 172.21.15.39:0/3374130927 shutdown_connections 2024-07-04T08:26:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.349+0000 7f018e9bb740 1 -- 172.21.15.39:0/3374130927 wait complete. 2024-07-04T08:26:49.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.349+0000 7f018e9bb740 20 client.28094 trim_cache size 0 max 16384 2024-07-04T08:26:49.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.364+0000 7f018e9bb740 1 Processor -- start 2024-07-04T08:26:49.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.364+0000 7f018e9bb740 1 -- start start 2024-07-04T08:26:49.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce0d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce0e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f018e9bb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f01801b6930 con 0x55e92ca4b390 2024-07-04T08:26:49.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f018e9bb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f01801b8470 con 0x55e92ca431b0 2024-07-04T08:26:49.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f018e9bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f01801b7000 con 0x55e92ca6c070 2024-07-04T08:26:49.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f0185a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce0d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f0185000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce0e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f0185a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce0d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54760/0 (socket says 172.21.15.39:54760) 2024-07-04T08:26:49.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f0185a00640 1 -- 172.21.15.39:0/4113249286 learned_addr learned my addr 172.21.15.39:0/4113249286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:49.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f0186400640 1 --2- 172.21.15.39:0/4113249286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f0185a00640 1 -- 172.21.15.39:0/4113249286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 msgr2=0x55e92ce281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f0185a00640 1 --2- 172.21.15.39:0/4113249286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f0185a00640 1 -- 172.21.15.39:0/4113249286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 msgr2=0x55e92ce0e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.365+0000 7f0185a00640 1 --2- 172.21.15.39:0/4113249286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce0e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.366+0000 7f0185a00640 1 -- 172.21.15.39:0/4113249286 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92ce1d430 con 0x55e92ca431b0 2024-07-04T08:26:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.366+0000 7f0185a00640 1 --2- 172.21.15.39:0/4113249286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce0d940 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f01740092a0 tx=0x7f0174005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cad34318bc71bef server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.366+0000 7f0172000640 1 -- 172.21.15.39:0/4113249286 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017404f070 con 0x55e92ca431b0 2024-07-04T08:26:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.366+0000 7f0172000640 1 -- 172.21.15.39:0/4113249286 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0174009520 con 0x55e92ca431b0 2024-07-04T08:26:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.366+0000 7f0172000640 1 -- 172.21.15.39:0/4113249286 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174035040 con 0x55e92ca431b0 2024-07-04T08:26:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.367+0000 7f018e9bb740 1 -- 172.21.15.39:0/4113249286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 msgr2=0x55e92ce0d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.367+0000 7f018e9bb740 1 --2- 172.21.15.39:0/4113249286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce0d940 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f01740092a0 tx=0x7f0174005fd0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.368+0000 7f018e9bb740 1 -- 172.21.15.39:0/4113249286 shutdown_connections 2024-07-04T08:26:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.368+0000 7f018e9bb740 1 --2- 172.21.15.39:0/4113249286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce0e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.368+0000 7f018e9bb740 1 --2- 172.21.15.39:0/4113249286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.368+0000 7f018e9bb740 1 --2- 172.21.15.39:0/4113249286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce0d940 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.368+0000 7f018e9bb740 1 -- 172.21.15.39:0/4113249286 >> 172.21.15.39:0/4113249286 conn(0x55e92ca60460 msgr2=0x55e92ce0dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.368+0000 7f018e9bb740 1 -- 172.21.15.39:0/4113249286 shutdown_connections 2024-07-04T08:26:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.368+0000 7f018e9bb740 1 -- 172.21.15.39:0/4113249286 wait complete. 2024-07-04T08:26:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.368+0000 7f018e9bb740 1 -- start start 2024-07-04T08:26:49.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.369+0000 7f018e9bb740 1 Processor -- start 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce31820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce2cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f018e9bb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce2d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f018e9bb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f01801b7da0 con 0x55e92ca4b390 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f018e9bb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f01801b76d0 con 0x55e92ca431b0 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f018e9bb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f01801b4910 con 0x55e92ca6c070 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f018e9bb740 10 client.0 _ll_register_callbacks cb 0x7f0174040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0185a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce31820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0185a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce31820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54772/0 (socket says 172.21.15.39:54772) 2024-07-04T08:26:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0185a00640 1 -- 172.21.15.39:0/3613013558 learned_addr learned my addr 172.21.15.39:0/3613013558 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0185000640 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce2cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0186400640 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce2d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0185a00640 1 -- 172.21.15.39:0/3613013558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 msgr2=0x55e92ce2d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0185a00640 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce2d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0185a00640 1 -- 172.21.15.39:0/3613013558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 msgr2=0x55e92ce2cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0185a00640 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce2cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.370+0000 7f0185a00640 1 -- 172.21.15.39:0/3613013558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92ce2d8a0 con 0x55e92ca431b0 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.371+0000 7f0186400640 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce2d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.371+0000 7f0185a00640 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce31820 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0174031440 tx=0x7f0174004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b885c31d4985497e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:49.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.371+0000 7f0167400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.371+0000 7f0167400640 1 -- 172.21.15.39:0/3613013558 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0174035030 con 0x55e92ca431b0 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.371+0000 7f018e9bb740 1 -- 172.21.15.39:0/3613013558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0174009520 con 0x55e92ca431b0 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.371+0000 7f0167400640 1 -- 172.21.15.39:0/3613013558 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f01740468a0 con 0x55e92ca431b0 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.372+0000 7f0167400640 1 -- 172.21.15.39:0/3613013558 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f017406f050 con 0x55e92ca431b0 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.372+0000 7f018e9bb740 10 client.28100 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.372+0000 7f018e9bb740 1 -- 172.21.15.39:0/3613013558 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e92ce96590 con 0x55e92ca431b0 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.372+0000 7f0167400640 1 -- 172.21.15.39:0/3613013558 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0174009520 con 0x55e92ca431b0 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.373+0000 7f0167400640 1 -- 172.21.15.39:0/3613013558 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f017406a030 con 0x55e92ca431b0 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.373+0000 7f018e9bb740 10 client.28100 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:49.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.373+0000 7f018e9bb740 10 client.28100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.373+0000 7f018e9bb740 20 client.28100 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.373+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55e92c9ffda0 0x55e92ce98cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.373+0000 7f018e9bb740 4 client.28100 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.373+0000 7f018e9bb740 1 -- 172.21.15.39:0/3613013558 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e92cd848d0 con 0x55e92c9ffda0 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.373+0000 7f0185000640 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55e92c9ffda0 0x55e92ce98cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f0185000640 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55e92c9ffda0 0x55e92ce98cb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc592c5ad37cd5d1 server_cookie=3ebe42ea7f3e1af1 in_seq=0 out_seq=0 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f0167400640 10 client.28100 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f0167400640 1 -- 172.21.15.39:0/3613013558 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e92cd848d0 con 0x55e92c9ffda0 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f0167400640 10 client.28100 handle_command_reply: tid=0 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f018e9bb740 1 client.28100 shutdown 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f018e9bb740 1 -- 172.21.15.39:0/3613013558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 msgr2=0x55e92ce31820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:49.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce31820 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0174031440 tx=0x7f0174004530 comp rx=0 tx=0).stop 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f018e9bb740 1 -- 172.21.15.39:0/3613013558 shutdown_connections 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55e92c9ffda0 0x55e92ce98cb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55e92ca4b390 0x55e92ce2cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f0167400640 0 client.28100 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e92ca6c070 0x55e92ce2d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f018e9bb740 1 --2- 172.21.15.39:0/3613013558 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e92ca431b0 0x55e92ce31820 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.374+0000 7f018e9bb740 1 -- 172.21.15.39:0/3613013558 >> 172.21.15.39:0/3613013558 conn(0x55e92ca60460 msgr2=0x55e92ce1d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.375+0000 7f018e9bb740 1 -- 172.21.15.39:0/3613013558 shutdown_connections 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.375+0000 7f018e9bb740 1 -- 172.21.15.39:0/3613013558 wait complete. 2024-07-04T08:26:49.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.375+0000 7f018e9bb740 20 client.28100 trim_cache size 0 max 16384 2024-07-04T08:26:49.368 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:49.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 -- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01680b2960 msgr2=0x7f01680b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01680b2960 0x7f01680b4e70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0178009fd0 tx=0x7f0178009500 comp rx=0 tx=0).stop 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 -- 172.21.15.39:0/390165424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018010f040 msgr2=0x7f01801a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018010f040 0x7f01801a0650 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f018006bf70 tx=0x7f016c00ee50 comp rx=0 tx=0).stop 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 -- 172.21.15.39:0/390165424 shutdown_connections 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f01680b2960 0x7f01680b4e70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f018010f990 0x7f01801a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f018010f040 0x7f01801a0650 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 --2- 172.21.15.39:0/390165424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0180109700 0x7f01801a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 -- 172.21.15.39:0/390165424 >> 172.21.15.39:0/390165424 conn(0x7f0180077630 msgr2=0x7f0180107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 -- 172.21.15.39:0/390165424 shutdown_connections 2024-07-04T08:26:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:49.376+0000 7f0188400640 1 -- 172.21.15.39:0/390165424 wait complete. 2024-07-04T08:26:50.217 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:26:50.696 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:51.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:51 smithi133 ceph-mon[37050]: pgmap v839: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-07-04T08:26:51.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:51 smithi039 ceph-mon[32062]: pgmap v839: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-07-04T08:26:51.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:51 smithi050 ceph-mon[36577]: pgmap v839: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-07-04T08:26:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.061+0000 7f2418400640 1 -- 172.21.15.39:0/1131151357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2410105300 msgr2=0x7f2410105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.061+0000 7f2418400640 1 --2- 172.21.15.39:0/1131151357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2410105300 0x7f2410105710 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2404009900 tx=0x7f240402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.062+0000 7f2418400640 1 -- 172.21.15.39:0/1131151357 shutdown_connections 2024-07-04T08:26:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.062+0000 7f2418400640 1 --2- 172.21.15.39:0/1131151357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f241010b590 0x7f2410111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.062+0000 7f2418400640 1 --2- 172.21.15.39:0/1131151357 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241010ac40 0x7f241010b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.062+0000 7f2418400640 1 --2- 172.21.15.39:0/1131151357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2410105300 0x7f2410105710 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.062+0000 7f2418400640 1 -- 172.21.15.39:0/1131151357 >> 172.21.15.39:0/1131151357 conn(0x7f2410100d10 msgr2=0x7f2410103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.062+0000 7f2418400640 1 -- 172.21.15.39:0/1131151357 shutdown_connections 2024-07-04T08:26:53.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.062+0000 7f2418400640 1 -- 172.21.15.39:0/1131151357 wait complete. 2024-07-04T08:26:53.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.064+0000 7f2418400640 1 Processor -- start 2024-07-04T08:26:53.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.064+0000 7f2418400640 1 -- start start 2024-07-04T08:26:53.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.064+0000 7f2418400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2410105300 0x7f241019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.064+0000 7f2418400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241010ac40 0x7f241019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.064+0000 7f2418400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241010b590 0x7f24101a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.064+0000 7f2418400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2410115240 con 0x7f241010ac40 2024-07-04T08:26:53.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.064+0000 7f2418400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24101150c0 con 0x7f241010b590 2024-07-04T08:26:53.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.064+0000 7f2418400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24101153c0 con 0x7f2410105300 2024-07-04T08:26:53.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241010ac40 0x7f241019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2410105300 0x7f241019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241010ac40 0x7f241019c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59274/0 (socket says 172.21.15.39:59274) 2024-07-04T08:26:53.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2410105300 0x7f241019bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41210/0 (socket says 172.21.15.39:41210) 2024-07-04T08:26:53.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415000640 1 -- 172.21.15.39:0/4249568515 learned_addr learned my addr 172.21.15.39:0/4249568515 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2416400640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241010b590 0x7f24101a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415000640 1 -- 172.21.15.39:0/4249568515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2410105300 msgr2=0x7f241019bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415000640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2410105300 0x7f241019bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415000640 1 -- 172.21.15.39:0/4249568515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241010b590 msgr2=0x7f24101a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415000640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241010b590 0x7f24101a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415000640 1 -- 172.21.15.39:0/4249568515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24101a1be0 con 0x7f241010ac40 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.065+0000 7f2415a00640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2410105300 0x7f241019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.066+0000 7f2415000640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241010ac40 0x7f241019c210 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f240000d950 tx=0x7f240000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.066+0000 7f240e000640 1 -- 172.21.15.39:0/4249568515 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400017070 con 0x7f241010ac40 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.066+0000 7f240e000640 1 -- 172.21.15.39:0/4249568515 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2400009c30 con 0x7f241010ac40 2024-07-04T08:26:53.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.066+0000 7f240e000640 1 -- 172.21.15.39:0/4249568515 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000047a0 con 0x7f241010ac40 2024-07-04T08:26:53.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.066+0000 7f2418400640 1 -- 172.21.15.39:0/4249568515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24101a1ed0 con 0x7f241010ac40 2024-07-04T08:26:53.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.066+0000 7f2418400640 1 -- 172.21.15.39:0/4249568515 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101a8630 con 0x7f241010ac40 2024-07-04T08:26:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.070+0000 7f241e9f7740 1 Processor -- start 2024-07-04T08:26:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.070+0000 7f241e9f7740 1 -- start start 2024-07-04T08:26:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc4e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc4ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 0x56311dc4f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f240e000640 1 -- 172.21.15.39:0/4249568515 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2400004020 con 0x7f241010ac40 2024-07-04T08:26:53.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f241e9f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56311dbc4ee0 con 0x56311d889390 2024-07-04T08:26:53.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f241e9f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56311dbb4480 con 0x56311d8811b0 2024-07-04T08:26:53.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f241e9f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56311dbb84c0 con 0x56311d8aa070 2024-07-04T08:26:53.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f2416400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc4e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f2415a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc4ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f2415000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 0x56311dc4f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f2416400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc4e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41222/0 (socket says 172.21.15.39:41222) 2024-07-04T08:26:53.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.071+0000 7f2416400640 1 -- 172.21.15.39:0/79956429 learned_addr learned my addr 172.21.15.39:0/79956429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:53.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.072+0000 7f2415a00640 1 -- 172.21.15.39:0/79956429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 msgr2=0x56311dc4e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:53.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.072+0000 7f2415a00640 1 --2- 172.21.15.39:0/79956429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc4e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.072+0000 7f2415a00640 1 -- 172.21.15.39:0/79956429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 msgr2=0x56311dc4f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:53.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.072+0000 7f2415a00640 1 --2- 172.21.15.39:0/79956429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 0x56311dc4f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.072+0000 7f2415a00640 1 -- 172.21.15.39:0/79956429 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311db4c650 con 0x56311d8811b0 2024-07-04T08:26:53.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.072+0000 7f2415a00640 1 --2- 172.21.15.39:0/79956429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc4ecf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2404009230 tx=0x7f2404031550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1bd2d078ebd945b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.073+0000 7f23fea00640 1 -- 172.21.15.39:0/79956429 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404040070 con 0x56311d8811b0 2024-07-04T08:26:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.073+0000 7f23fea00640 1 -- 172.21.15.39:0/79956429 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24040318c0 con 0x56311d8811b0 2024-07-04T08:26:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.073+0000 7f23fea00640 1 -- 172.21.15.39:0/79956429 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404035040 con 0x56311d8811b0 2024-07-04T08:26:53.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.073+0000 7f240e000640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23f80b29e0 0x7f23f80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.073+0000 7f240e000640 1 -- 172.21.15.39:0/4249568515 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f24000e25e0 con 0x7f241010ac40 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.073+0000 7f2416400640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23f80b29e0 0x7f23f80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f241e9f7740 1 -- 172.21.15.39:0/79956429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 msgr2=0x56311dc4ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f241e9f7740 1 --2- 172.21.15.39:0/79956429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc4ecf0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2404009230 tx=0x7f2404031550 comp rx=0 tx=0).stop 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f241e9f7740 1 -- 172.21.15.39:0/79956429 shutdown_connections 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f241e9f7740 1 --2- 172.21.15.39:0/79956429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 0x56311dc4f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f241e9f7740 1 --2- 172.21.15.39:0/79956429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc4ecf0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f241e9f7740 1 --2- 172.21.15.39:0/79956429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc4e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f241e9f7740 1 -- 172.21.15.39:0/79956429 >> 172.21.15.39:0/79956429 conn(0x56311d89e460 msgr2=0x56311dc4c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f241e9f7740 1 -- 172.21.15.39:0/79956429 shutdown_connections 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f241e9f7740 1 -- 172.21.15.39:0/79956429 wait complete. 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.074+0000 7f2416400640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23f80b29e0 0x7f23f80b4ef0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2408005f40 tx=0x7f2408009600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:53.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.075+0000 7f241e9f7740 1 -- start start 2024-07-04T08:26:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.075+0000 7f241e9f7740 1 Processor -- start 2024-07-04T08:26:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 0x56311dc5a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc5a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc66280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f241e9f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56311da18700 con 0x56311d8811b0 2024-07-04T08:26:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f241e9f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56311db5bbb0 con 0x56311d889390 2024-07-04T08:26:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f241e9f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56311dc5cf30 con 0x56311d8aa070 2024-07-04T08:26:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f241e9f7740 10 client.0 _ll_register_callbacks cb 0x56311dbad570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:53.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f2415a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 0x56311dc5a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f2416400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc66280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f2415000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc5a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f2416400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc66280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41226/0 (socket says 172.21.15.39:41226) 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f2416400640 1 -- 172.21.15.39:0/1064424753 learned_addr learned my addr 172.21.15.39:0/1064424753 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f2416400640 1 -- 172.21.15.39:0/1064424753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 msgr2=0x56311dc5a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f2416400640 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc5a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f2416400640 1 -- 172.21.15.39:0/1064424753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 msgr2=0x56311dc5a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.076+0000 7f2416400640 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 0x56311dc5a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.077+0000 7f2416400640 1 -- 172.21.15.39:0/1064424753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311d915650 con 0x56311d8aa070 2024-07-04T08:26:53.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.077+0000 7f2416400640 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc66280 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2408009fd0 tx=0x7f2408004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d621c0b52b7b645 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.077+0000 7f23f3400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.077+0000 7f23f3400640 1 -- 172.21.15.39:0/1064424753 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408025070 con 0x56311d8aa070 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.077+0000 7f23f3400640 1 -- 172.21.15.39:0/1064424753 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f2408011cd0 con 0x56311d8aa070 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.077+0000 7f241e9f7740 1 -- 172.21.15.39:0/1064424753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24040318c0 con 0x56311d8aa070 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.078+0000 7f23f3400640 1 -- 172.21.15.39:0/1064424753 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2408031050 con 0x56311d8aa070 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.078+0000 7f241e9f7740 10 client.28045 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.078+0000 7f241e9f7740 1 -- 172.21.15.39:0/1064424753 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56311dc68fe0 con 0x56311d8aa070 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.078+0000 7f23f3400640 1 -- 172.21.15.39:0/1064424753 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24040318c0 con 0x56311d8aa070 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.079+0000 7f23f3400640 1 -- 172.21.15.39:0/1064424753 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f240800f040 con 0x56311d8aa070 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.079+0000 7f241e9f7740 10 client.28045 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:53.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.079+0000 7f241e9f7740 10 client.28045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.079+0000 7f241e9f7740 20 client.28045 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.079+0000 7f241e9f7740 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56311d8689f0 0x56311dc6c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.079+0000 7f241e9f7740 4 client.28045 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.079+0000 7f241e9f7740 1 -- 172.21.15.39:0/1064424753 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56311dbc6e30 con 0x56311d8689f0 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.079+0000 7f2415a00640 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56311d8689f0 0x56311dc6c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.080+0000 7f2415a00640 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56311d8689f0 0x56311dc6c5f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4283038844d8f68 server_cookie=99ec793e00682f5a in_seq=0 out_seq=0 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.080+0000 7f23f3400640 10 client.28045 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f23f3400640 1 -- 172.21.15.39:0/1064424753 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56311dbc6e30 con 0x56311d8689f0 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f23f3400640 10 client.28045 handle_command_reply: tid=0 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f241e9f7740 1 client.28045 shutdown 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f241e9f7740 1 -- 172.21.15.39:0/1064424753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 msgr2=0x56311dc66280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:53.075 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f241e9f7740 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc66280 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2408009fd0 tx=0x7f2408004a40 comp rx=0 tx=0).stop 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f241e9f7740 1 -- 172.21.15.39:0/1064424753 shutdown_connections 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f241e9f7740 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56311d8689f0 0x56311dc6c5f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f241e9f7740 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc66280 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f241e9f7740 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc5a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f241e9f7740 1 --2- 172.21.15.39:0/1064424753 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 0x56311dc5a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f23f3400640 0 client.28045 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.081+0000 7f241e9f7740 1 -- 172.21.15.39:0/1064424753 >> 172.21.15.39:0/1064424753 conn(0x56311d89e460 msgr2=0x56311dc51940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.082+0000 7f241e9f7740 1 -- 172.21.15.39:0/1064424753 shutdown_connections 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.082+0000 7f241e9f7740 1 -- 172.21.15.39:0/1064424753 wait complete. 2024-07-04T08:26:53.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.082+0000 7f241e9f7740 20 client.28045 trim_cache size 0 max 16384 2024-07-04T08:26:53.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.097+0000 7f241e9f7740 1 Processor -- start 2024-07-04T08:26:53.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.097+0000 7f241e9f7740 1 -- start start 2024-07-04T08:26:53.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 0x56311dc4b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc4c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f241e9f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b2620 con 0x56311d8811b0 2024-07-04T08:26:53.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f241e9f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b4160 con 0x56311d889390 2024-07-04T08:26:53.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f241e9f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b2cf0 con 0x56311d8aa070 2024-07-04T08:26:53.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f2415000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc4c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f2415000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc4c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60056/0 (socket says 172.21.15.39:60056) 2024-07-04T08:26:53.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f2415a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 0x56311dc4b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f2415000640 1 -- 172.21.15.39:0/912904737 learned_addr learned my addr 172.21.15.39:0/912904737 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.098+0000 7f2416400640 1 --2- 172.21.15.39:0/912904737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc53840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.099+0000 7f2415000640 1 -- 172.21.15.39:0/912904737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 msgr2=0x56311dc53840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:53.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.099+0000 7f2415000640 1 --2- 172.21.15.39:0/912904737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc53840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.099+0000 7f2415000640 1 -- 172.21.15.39:0/912904737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 msgr2=0x56311dc4b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.099+0000 7f2415000640 1 --2- 172.21.15.39:0/912904737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 0x56311dc4b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.099+0000 7f2415000640 1 -- 172.21.15.39:0/912904737 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311dc51040 con 0x56311d889390 2024-07-04T08:26:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.099+0000 7f2415000640 1 --2- 172.21.15.39:0/912904737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc4c6f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f24000df9d0 tx=0x7f2400091040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c9b609b0794772 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.100+0000 7f23fe000640 1 -- 172.21.15.39:0/912904737 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000910b0 con 0x56311d889390 2024-07-04T08:26:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.100+0000 7f23fe000640 1 -- 172.21.15.39:0/912904737 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24000a9010 con 0x56311d889390 2024-07-04T08:26:53.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.100+0000 7f23fe000640 1 -- 172.21.15.39:0/912904737 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000b2360 con 0x56311d889390 2024-07-04T08:26:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.100+0000 7f241e9f7740 1 -- 172.21.15.39:0/912904737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 msgr2=0x56311dc4c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.101+0000 7f241e9f7740 1 --2- 172.21.15.39:0/912904737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc4c6f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f24000df9d0 tx=0x7f2400091040 comp rx=0 tx=0).stop 2024-07-04T08:26:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.101+0000 7f241e9f7740 1 -- 172.21.15.39:0/912904737 shutdown_connections 2024-07-04T08:26:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.101+0000 7f241e9f7740 1 --2- 172.21.15.39:0/912904737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc53840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.101+0000 7f241e9f7740 1 --2- 172.21.15.39:0/912904737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d889390 0x56311dc4c6f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.101+0000 7f241e9f7740 1 --2- 172.21.15.39:0/912904737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d8811b0 0x56311dc4b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.101+0000 7f241e9f7740 1 -- 172.21.15.39:0/912904737 >> 172.21.15.39:0/912904737 conn(0x56311d89e460 msgr2=0x56311dc4bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.101+0000 7f241e9f7740 1 -- 172.21.15.39:0/912904737 shutdown_connections 2024-07-04T08:26:53.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.101+0000 7f241e9f7740 1 -- 172.21.15.39:0/912904737 wait complete. 2024-07-04T08:26:53.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.101+0000 7f241e9f7740 1 -- start start 2024-07-04T08:26:53.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.102+0000 7f241e9f7740 1 Processor -- start 2024-07-04T08:26:53.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.102+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc62850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.102+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 0x56311dc65a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.102+0000 7f241e9f7740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc65fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.102+0000 7f241e9f7740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b3a90 con 0x56311d889390 2024-07-04T08:26:53.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.102+0000 7f241e9f7740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b33c0 con 0x56311d8811b0 2024-07-04T08:26:53.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.102+0000 7f241e9f7740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b0600 con 0x56311d8aa070 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.102+0000 7f241e9f7740 10 client.0 _ll_register_callbacks cb 0x7f2408009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc62850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc62850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60066/0 (socket says 172.21.15.39:60066) 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415a00640 1 -- 172.21.15.39:0/383651719 learned_addr learned my addr 172.21.15.39:0/383651719 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415000640 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 0x56311dc65a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2416400640 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc65fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415a00640 1 -- 172.21.15.39:0/383651719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 msgr2=0x56311dc65fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415a00640 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc65fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415a00640 1 -- 172.21.15.39:0/383651719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 msgr2=0x56311dc65a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415a00640 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 0x56311dc65a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415a00640 1 -- 172.21.15.39:0/383651719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311dc66700 con 0x56311d8811b0 2024-07-04T08:26:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f2415a00640 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc62850 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2404005690 tx=0x7f240402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d27ecfa3bd0ca99 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.103+0000 7f23f3400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:26:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.104+0000 7f23f3400640 1 -- 172.21.15.39:0/383651719 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240405b070 con 0x56311d8811b0 2024-07-04T08:26:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.104+0000 7f241e9f7740 1 -- 172.21.15.39:0/383651719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24000a9010 con 0x56311d8811b0 2024-07-04T08:26:53.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.104+0000 7f23f3400640 1 -- 172.21.15.39:0/383651719 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f24040042e0 con 0x56311d8811b0 2024-07-04T08:26:53.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.104+0000 7f23f3400640 1 -- 172.21.15.39:0/383651719 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2404035040 con 0x56311d8811b0 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.104+0000 7f241e9f7740 10 client.28118 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.104+0000 7f241e9f7740 1 -- 172.21.15.39:0/383651719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56311dcd4d80 con 0x56311d8811b0 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.104+0000 7f23f3400640 1 -- 172.21.15.39:0/383651719 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24000a9010 con 0x56311d8811b0 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.105+0000 7f23f3400640 1 -- 172.21.15.39:0/383651719 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2404003710 con 0x56311d8811b0 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.105+0000 7f241e9f7740 10 client.28118 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.105+0000 7f241e9f7740 10 client.28118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.105+0000 7f241e9f7740 20 client.28118 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.105+0000 7f241e9f7740 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56311d83dda0 0x56311dcd7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.105+0000 7f241e9f7740 4 client.28118 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.105+0000 7f241e9f7740 1 -- 172.21.15.39:0/383651719 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56311dbc28d0 con 0x56311d83dda0 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.105+0000 7f2415000640 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56311d83dda0 0x56311dcd7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:53.097 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f2415000640 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56311d83dda0 0x56311dcd7060 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473ea1c9a4e87f5 server_cookie=d0b85672a9893ce in_seq=0 out_seq=0 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f23f3400640 10 client.28118 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f23f3400640 1 -- 172.21.15.39:0/383651719 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56311dbc28d0 con 0x56311d83dda0 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f23f3400640 10 client.28118 handle_command_reply: tid=0 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f241e9f7740 1 client.28118 shutdown 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f241e9f7740 1 -- 172.21.15.39:0/383651719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 msgr2=0x56311dc62850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f241e9f7740 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc62850 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2404005690 tx=0x7f240402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f241e9f7740 1 -- 172.21.15.39:0/383651719 shutdown_connections 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f241e9f7740 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56311d83dda0 0x56311dcd7060 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f241e9f7740 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56311d889390 0x56311dc65a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f241e9f7740 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56311d8aa070 0x56311dc65fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f23f3400640 0 client.28118 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:53.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f241e9f7740 1 --2- 172.21.15.39:0/383651719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56311d8811b0 0x56311dc62850 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.106+0000 7f241e9f7740 1 -- 172.21.15.39:0/383651719 >> 172.21.15.39:0/383651719 conn(0x56311d89e460 msgr2=0x56311dc4f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.107+0000 7f241e9f7740 1 -- 172.21.15.39:0/383651719 shutdown_connections 2024-07-04T08:26:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.107+0000 7f241e9f7740 1 -- 172.21.15.39:0/383651719 wait complete. 2024-07-04T08:26:53.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.107+0000 7f241e9f7740 20 client.28118 trim_cache size 0 max 16384 2024-07-04T08:26:53.099 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:26:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.107+0000 7f2418400640 1 -- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23f80b29e0 msgr2=0x7f23f80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.107+0000 7f2418400640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23f80b29e0 0x7f23f80b4ef0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2408005f40 tx=0x7f2408009600 comp rx=0 tx=0).stop 2024-07-04T08:26:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.107+0000 7f2418400640 1 -- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241010ac40 msgr2=0x7f241019c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.107+0000 7f2418400640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241010ac40 0x7f241019c210 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f240000d950 tx=0x7f240000de10 comp rx=0 tx=0).stop 2024-07-04T08:26:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.108+0000 7f2418400640 1 -- 172.21.15.39:0/4249568515 shutdown_connections 2024-07-04T08:26:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.108+0000 7f2418400640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f23f80b29e0 0x7f23f80b4ef0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.108+0000 7f2418400640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f241010ac40 0x7f241019c210 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.108+0000 7f2418400640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f241010b590 0x7f24101a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.108+0000 7f2418400640 1 --2- 172.21.15.39:0/4249568515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2410105300 0x7f241019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:53.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.108+0000 7f2418400640 1 -- 172.21.15.39:0/4249568515 >> 172.21.15.39:0/4249568515 conn(0x7f2410100d10 msgr2=0x7f2410104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:53.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.108+0000 7f2418400640 1 -- 172.21.15.39:0/4249568515 shutdown_connections 2024-07-04T08:26:53.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:53.108+0000 7f2418400640 1 -- 172.21.15.39:0/4249568515 wait complete. 2024-07-04T08:26:53.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:53 smithi039 ceph-mon[32062]: pgmap v840: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-07-04T08:26:53.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:53 smithi133 ceph-mon[37050]: pgmap v840: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-07-04T08:26:53.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:53 smithi050 ceph-mon[36577]: pgmap v840: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-07-04T08:26:54.695 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:26:54.962 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:26:54.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:26:55.173 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:55.322 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:55 smithi039 ceph-mon[32062]: pgmap v841: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-07-04T08:26:55.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:55 smithi133 ceph-mon[37050]: pgmap v841: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-07-04T08:26:55.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:55 smithi050 ceph-mon[36577]: pgmap v841: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-07-04T08:26:56.579 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:26:57.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:57 smithi039 ceph-mon[32062]: pgmap v842: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-07-04T08:26:57.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:57 smithi133 ceph-mon[37050]: pgmap v842: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-07-04T08:26:57.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:57 smithi050 ceph-mon[36577]: pgmap v842: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.0 KiB/s wr, 12 op/s 2024-07-04T08:26:58.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.041+0000 7f0ac7e00640 1 -- 172.21.15.39:0/582082791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac010ac40 msgr2=0x7f0ac010b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:58.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.041+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/582082791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac010ac40 0x7f0ac010b050 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009900 tx=0x7f0aac02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.041+0000 7f0ac7e00640 1 -- 172.21.15.39:0/582082791 shutdown_connections 2024-07-04T08:26:58.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.041+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/582082791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac010ac40 0x7f0ac010b050 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.041+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/582082791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac010b590 0x7f0ac0111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.041+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/582082791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ac0105300 0x7f0ac0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.041+0000 7f0ac7e00640 1 -- 172.21.15.39:0/582082791 >> 172.21.15.39:0/582082791 conn(0x7f0ac0100d10 msgr2=0x7f0ac0103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:58.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.041+0000 7f0ac7e00640 1 -- 172.21.15.39:0/582082791 shutdown_connections 2024-07-04T08:26:58.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.042+0000 7f0ac7e00640 1 -- 172.21.15.39:0/582082791 wait complete. 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.043+0000 7f0ac7e00640 1 Processor -- start 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.043+0000 7f0ac7e00640 1 -- start start 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ac0105300 0x7f0ac019bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac010ac40 0x7f0ac019c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac010b590 0x7f0ac01a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0115270 con 0x7f0ac010ac40 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac7e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01150f0 con 0x7f0ac0105300 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01153f0 con 0x7f0ac010b590 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ac0105300 0x7f0ac019bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac5400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ac0105300 0x7f0ac019bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60084/0 (socket says 172.21.15.39:60084) 2024-07-04T08:26:58.034 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac5400640 1 -- 172.21.15.39:0/128231787 learned_addr learned my addr 172.21.15.39:0/128231787 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:58.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac4a00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac010ac40 0x7f0ac019c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac5e00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac010b590 0x7f0ac01a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac5400640 1 -- 172.21.15.39:0/128231787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac010b590 msgr2=0x7f0ac01a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac5400640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac010b590 0x7f0ac01a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac5400640 1 -- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac010ac40 msgr2=0x7f0ac019c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac5400640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac010ac40 0x7f0ac019c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.044+0000 7f0ac5400640 1 -- 172.21.15.39:0/128231787 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac01a1bd0 con 0x7f0ac0105300 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.045+0000 7f0ac5e00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac010b590 0x7f0ac01a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.045+0000 7f0ac5400640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ac0105300 0x7f0ac019bc60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400e9f0 tx=0x7f0ab400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.045+0000 7f0abe000640 1 -- 172.21.15.39:0/128231787 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400cd80 con 0x7f0ac0105300 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.045+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac01a1ec0 con 0x7f0ac0105300 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.045+0000 7f0abe000640 1 -- 172.21.15.39:0/128231787 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab40108e0 con 0x7f0ac0105300 2024-07-04T08:26:58.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.045+0000 7f0abe000640 1 -- 172.21.15.39:0/128231787 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4018980 con 0x7f0ac0105300 2024-07-04T08:26:58.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.045+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac01a8620 con 0x7f0ac0105300 2024-07-04T08:26:58.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.047+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac0105710 con 0x7f0ac0105300 2024-07-04T08:26:58.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.049+0000 7f0abe000640 1 -- 172.21.15.39:0/128231787 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0ab4010430 con 0x7f0ac0105300 2024-07-04T08:26:58.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.051+0000 7f0abe000640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0aa80b2960 0x7f0aa80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:58.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.051+0000 7f0abe000640 1 -- 172.21.15.39:0/128231787 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f0ab40e3230 con 0x7f0ac0105300 2024-07-04T08:26:58.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.051+0000 7f0ac4a00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0aa80b2960 0x7f0aa80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:58.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.052+0000 7f0ac4a00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0aa80b2960 0x7f0aa80b4e70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0aac02a9e0 tx=0x7f0aac005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:58.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.052+0000 7f0abe000640 1 -- 172.21.15.39:0/128231787 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f0ab40aa850 con 0x7f0ac0105300 2024-07-04T08:26:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.311+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ac019cd90 con 0x7f0ac0105300 2024-07-04T08:26:58.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.312+0000 7f0abe000640 1 -- 172.21.15.39:0/128231787 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f0ab40aa150 con 0x7f0ac0105300 2024-07-04T08:26:58.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:26:58.303 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:26:58.303 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:26:58.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.315+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0aa80b2960 msgr2=0x7f0aa80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:58.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.315+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0aa80b2960 0x7f0aa80b4e70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0aac02a9e0 tx=0x7f0aac005a20 comp rx=0 tx=0).stop 2024-07-04T08:26:58.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.315+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ac0105300 msgr2=0x7f0ac019bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:58.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.315+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ac0105300 0x7f0ac019bc60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400e9f0 tx=0x7f0ab400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.316+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 shutdown_connections 2024-07-04T08:26:58.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.316+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0aa80b2960 0x7f0aa80b4e70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.316+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ac010ac40 0x7f0ac019c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.316+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ac010b590 0x7f0ac01a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.316+0000 7f0ac7e00640 1 --2- 172.21.15.39:0/128231787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ac0105300 0x7f0ac019bc60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:58.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.316+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 >> 172.21.15.39:0/128231787 conn(0x7f0ac0100d10 msgr2=0x7f0ac01041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:58.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.316+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 shutdown_connections 2024-07-04T08:26:58.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:58.316+0000 7f0ac7e00640 1 -- 172.21.15.39:0/128231787 wait complete. 2024-07-04T08:26:59.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:59 smithi039 ceph-mon[32062]: pgmap v843: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-07-04T08:26:59.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:26:59 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/128231787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:59.436 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:59 smithi050 ceph-mon[36577]: pgmap v843: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-07-04T08:26:59.436 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:26:59 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/128231787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.377+0000 7f2675800640 1 -- 172.21.15.39:0/3896305320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26701013d0 msgr2=0x7f26701017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.377+0000 7f2675800640 1 --2- 172.21.15.39:0/3896305320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26701013d0 0x7f26701017e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2660009900 tx=0x7f266002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.377+0000 7f2675800640 1 -- 172.21.15.39:0/3896305320 shutdown_connections 2024-07-04T08:26:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.377+0000 7f2675800640 1 --2- 172.21.15.39:0/3896305320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2670105f70 0x7f267010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.377+0000 7f2675800640 1 --2- 172.21.15.39:0/3896305320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2670105620 0x7f2670105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.377+0000 7f2675800640 1 --2- 172.21.15.39:0/3896305320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26701013d0 0x7f26701017e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.377+0000 7f2675800640 1 -- 172.21.15.39:0/3896305320 >> 172.21.15.39:0/3896305320 conn(0x7f2670077000 msgr2=0x7f2670077440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.377+0000 7f2675800640 1 -- 172.21.15.39:0/3896305320 shutdown_connections 2024-07-04T08:26:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.377+0000 7f2675800640 1 -- 172.21.15.39:0/3896305320 wait complete. 2024-07-04T08:26:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.378+0000 7f2675800640 1 Processor -- start 2024-07-04T08:26:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.378+0000 7f2675800640 1 -- start start 2024-07-04T08:26:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f2675800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26701013d0 0x7f267019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f2675800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2670105620 0x7f267019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f2675800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2670105f70 0x7f26701a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f2675800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2670112eb0 con 0x7f2670105620 2024-07-04T08:26:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f2675800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2670112d30 con 0x7f2670105f70 2024-07-04T08:26:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f2675800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2670113030 con 0x7f26701013d0 2024-07-04T08:26:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f266f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2670105620 0x7f267019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f266fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26701013d0 0x7f267019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f266fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26701013d0 0x7f267019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41270/0 (socket says 172.21.15.39:41270) 2024-07-04T08:26:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f266fe00640 1 -- 172.21.15.39:0/3578568367 learned_addr learned my addr 172.21.15.39:0/3578568367 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.379+0000 7f2674e00640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2670105f70 0x7f26701a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266fe00640 1 -- 172.21.15.39:0/3578568367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2670105f70 msgr2=0x7f26701a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266fe00640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2670105f70 0x7f26701a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266fe00640 1 -- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2670105620 msgr2=0x7f267019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:59.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266fe00640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2670105620 0x7f267019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266fe00640 1 -- 172.21.15.39:0/3578568367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26701a1990 con 0x7f26701013d0 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266f400640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2670105620 0x7f267019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266fe00640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26701013d0 0x7f267019ba20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f266002a910 tx=0x7f2660003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266cc00640 1 -- 172.21.15.39:0/3578568367 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266003f070 con 0x7f26701013d0 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f2675800640 1 -- 172.21.15.39:0/3578568367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26701a1c20 con 0x7f26701013d0 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266cc00640 1 -- 172.21.15.39:0/3578568367 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266003bb10 con 0x7f26701013d0 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.380+0000 7f266cc00640 1 -- 172.21.15.39:0/3578568367 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660035030 con 0x7f26701013d0 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.381+0000 7f2675800640 1 -- 172.21.15.39:0/3578568367 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26701a83e0 con 0x7f26701013d0 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.385+0000 7f267d511740 1 Processor -- start 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.385+0000 7f267d511740 1 -- start start 2024-07-04T08:26:59.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.386+0000 7f267d511740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc9787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.386+0000 7f267d511740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc978d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.386+0000 7f267d511740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc979270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.386+0000 7f267d511740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5612dc8a28e0 con 0x5612dc5b3670 2024-07-04T08:26:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.386+0000 7f267d511740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5612dc8eeec0 con 0x5612dc5ab490 2024-07-04T08:26:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.386+0000 7f267d511740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5612dc8de460 con 0x5612dc5d4350 2024-07-04T08:26:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.386+0000 7f266cc00640 1 -- 172.21.15.39:0/3578568367 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2660042070 con 0x7f26701013d0 2024-07-04T08:26:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.386+0000 7f266f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc979270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f266f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc979270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59358/0 (socket says 172.21.15.39:59358) 2024-07-04T08:26:59.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f2674e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc9787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f266f400640 1 -- 172.21.15.39:0/2261379962 learned_addr learned my addr 172.21.15.39:0/2261379962 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f266fe00640 1 --2- 172.21.15.39:0/2261379962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc978d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f2674e00640 1 -- 172.21.15.39:0/2261379962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 msgr2=0x5612dc978d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f2674e00640 1 --2- 172.21.15.39:0/2261379962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc978d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f2674e00640 1 -- 172.21.15.39:0/2261379962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 msgr2=0x5612dc979270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f2674e00640 1 --2- 172.21.15.39:0/2261379962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc979270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f2674e00640 1 -- 172.21.15.39:0/2261379962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612dc876280 con 0x5612dc5d4350 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f266f400640 1 --2- 172.21.15.39:0/2261379962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc979270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.387+0000 7f2674e00640 1 --2- 172.21.15.39:0/2261379962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc9787f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2668009f00 tx=0x7f2668011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78b382fabfec119c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.388+0000 7f2665600640 1 -- 172.21.15.39:0/2261379962 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668004d20 con 0x5612dc5d4350 2024-07-04T08:26:59.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.388+0000 7f2665600640 1 -- 172.21.15.39:0/2261379962 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2668004030 con 0x5612dc5d4350 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.388+0000 7f2665600640 1 -- 172.21.15.39:0/2261379962 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668005280 con 0x5612dc5d4350 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 -- 172.21.15.39:0/2261379962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 msgr2=0x5612dc9787f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 --2- 172.21.15.39:0/2261379962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc9787f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2668009f00 tx=0x7f2668011810 comp rx=0 tx=0).stop 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 -- 172.21.15.39:0/2261379962 shutdown_connections 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 --2- 172.21.15.39:0/2261379962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc979270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 --2- 172.21.15.39:0/2261379962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc978d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 --2- 172.21.15.39:0/2261379962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc9787f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 -- 172.21.15.39:0/2261379962 >> 172.21.15.39:0/2261379962 conn(0x5612dc5c8740 msgr2=0x5612dc9766e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 -- 172.21.15.39:0/2261379962 shutdown_connections 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 -- 172.21.15.39:0/2261379962 wait complete. 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.389+0000 7f267d511740 1 -- start start 2024-07-04T08:26:59.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.390+0000 7f266cc00640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26540b2870 0x7f26540b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.390+0000 7f267d511740 1 Processor -- start 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.390+0000 7f2674e00640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26540b2870 0x7f26540b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.390+0000 7f266cc00640 1 -- 172.21.15.39:0/3578568367 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f2660102f90 con 0x7f26701013d0 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.390+0000 7f267d511740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5ab490 0x5612dc984370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.390+0000 7f267d511740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5b3670 0x5612dc9848b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.390+0000 7f267d511740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5d4350 0x5612dc990140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.390+0000 7f267d511740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5612dc885760 con 0x5612dc5ab490 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.390+0000 7f267d511740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5612dc8e24a0 con 0x5612dc5d4350 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f267d511740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5612dc742890 con 0x5612dc5b3670 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f267d511740 10 client.0 _ll_register_callbacks cb 0x5612dc8862b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f2674e00640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26540b2870 0x7f26540b4d80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2668009270 tx=0x7f2668009ca0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:59.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f266fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5ab490 0x5612dc984370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f2674e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5d4350 0x5612dc990140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f266f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5b3670 0x5612dc9848b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f266fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5ab490 0x5612dc984370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:59360/0 (socket says 172.21.15.39:59360) 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f266fe00640 1 -- 172.21.15.39:0/459405781 learned_addr learned my addr 172.21.15.39:0/459405781 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f266fe00640 1 -- 172.21.15.39:0/459405781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5b3670 msgr2=0x5612dc9848b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f266fe00640 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5b3670 0x5612dc9848b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f266fe00640 1 -- 172.21.15.39:0/459405781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5d4350 msgr2=0x5612dc990140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f266fe00640 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5d4350 0x5612dc990140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f266fe00640 1 -- 172.21.15.39:0/459405781 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612dc63f930 con 0x5612dc5ab490 2024-07-04T08:26:59.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.391+0000 7f2674e00640 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5d4350 0x5612dc990140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.392+0000 7f266fe00640 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5ab490 0x5612dc984370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2660000c00 tx=0x7f2660009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a680dcb1923157ca server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.392+0000 7f265a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.392+0000 7f265a000640 1 -- 172.21.15.39:0/459405781 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660049030 con 0x5612dc5ab490 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.392+0000 7f265a000640 1 -- 172.21.15.39:0/459405781 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f26601005a0 con 0x5612dc5ab490 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.392+0000 7f267d511740 1 -- 172.21.15.39:0/459405781 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2668004030 con 0x5612dc5ab490 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.392+0000 7f265a000640 1 -- 172.21.15.39:0/459405781 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f26600c98b0 con 0x5612dc5ab490 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.392+0000 7f267d511740 10 client.18000 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.393+0000 7f267d511740 1 -- 172.21.15.39:0/459405781 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612dc992df0 con 0x5612dc5ab490 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.393+0000 7f265a000640 1 -- 172.21.15.39:0/459405781 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2668004030 con 0x5612dc5ab490 2024-07-04T08:26:59.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.393+0000 7f265a000640 1 -- 172.21.15.39:0/459405781 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f26600ca1c0 con 0x5612dc5ab490 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.393+0000 7f267d511740 10 client.18000 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.393+0000 7f267d511740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.393+0000 7f267d511740 20 client.18000 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.393+0000 7f267d511740 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5612dc592cd0 0x5612dc996250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.393+0000 7f267d511740 4 client.18000 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.393+0000 7f267d511740 1 -- 172.21.15.39:0/459405781 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612dc8f0e10 con 0x5612dc592cd0 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.394+0000 7f266f400640 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5612dc592cd0 0x5612dc996250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.394+0000 7f265a000640 10 client.18000 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.394+0000 7f266f400640 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5612dc592cd0 0x5612dc996250 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720126cf0b32e205 server_cookie=833d7adb65a5e677 in_seq=0 out_seq=0 2024-07-04T08:26:59.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.395+0000 7f265a000640 1 -- 172.21.15.39:0/459405781 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5612dc8f0e10 con 0x5612dc592cd0 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.395+0000 7f265a000640 10 client.18000 handle_command_reply: tid=0 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.395+0000 7f267d511740 1 client.18000 shutdown 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 -- 172.21.15.39:0/459405781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5ab490 msgr2=0x5612dc984370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5ab490 0x5612dc984370 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2660000c00 tx=0x7f2660009620 comp rx=0 tx=0).stop 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 -- 172.21.15.39:0/459405781 shutdown_connections 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5612dc592cd0 0x5612dc996250 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5d4350 0x5612dc990140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5b3670 0x5612dc9848b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 --2- 172.21.15.39:0/459405781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5ab490 0x5612dc984370 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 -- 172.21.15.39:0/459405781 >> 172.21.15.39:0/459405781 conn(0x5612dc5c8740 msgr2=0x5612dc97b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f265a000640 0 client.18000 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:59.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 -- 172.21.15.39:0/459405781 shutdown_connections 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 1 -- 172.21.15.39:0/459405781 wait complete. 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.396+0000 7f267d511740 20 client.18000 trim_cache size 0 max 16384 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.416+0000 7f267d511740 1 Processor -- start 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.416+0000 7f267d511740 1 -- start start 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f267d511740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc97e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f267d511740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc98ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f267d511740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc99ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f267d511740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26701b2370 con 0x5612dc5b3670 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f267d511740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26701b3eb0 con 0x5612dc5ab490 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f267d511740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f26701b2a40 con 0x5612dc5d4350 2024-07-04T08:26:59.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f266fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc97e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f266fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc97e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:60124/0 (socket says 172.21.15.39:60124) 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f266fe00640 1 -- 172.21.15.39:0/2990884808 learned_addr learned my addr 172.21.15.39:0/2990884808 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f266f400640 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc98ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f2674e00640 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc99ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f266fe00640 1 -- 172.21.15.39:0/2990884808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 msgr2=0x5612dc99ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f266fe00640 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc99ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f266fe00640 1 -- 172.21.15.39:0/2990884808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 msgr2=0x5612dc98ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f266fe00640 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc98ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f266fe00640 1 -- 172.21.15.39:0/2990884808 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612dc9708e0 con 0x5612dc5ab490 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.417+0000 7f2674e00640 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc99ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.418+0000 7f266fe00640 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc97e910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f26600c71d0 tx=0x7f2660103230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540161677d3f7b2f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.418+0000 7f2664c00640 1 -- 172.21.15.39:0/2990884808 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660049070 con 0x5612dc5ab490 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.418+0000 7f2664c00640 1 -- 172.21.15.39:0/2990884808 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26600c9a70 con 0x5612dc5ab490 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.418+0000 7f2664c00640 1 -- 172.21.15.39:0/2990884808 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26600c96d0 con 0x5612dc5ab490 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.419+0000 7f267d511740 1 -- 172.21.15.39:0/2990884808 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 msgr2=0x5612dc97e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.419+0000 7f267d511740 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc97e910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f26600c71d0 tx=0x7f2660103230 comp rx=0 tx=0).stop 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.419+0000 7f267d511740 1 -- 172.21.15.39:0/2990884808 shutdown_connections 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.419+0000 7f267d511740 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5b3670 0x5612dc98ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.419+0000 7f267d511740 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5d4350 0x5612dc99ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.419+0000 7f267d511740 1 --2- 172.21.15.39:0/2990884808 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5ab490 0x5612dc97e910 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.419+0000 7f267d511740 1 -- 172.21.15.39:0/2990884808 >> 172.21.15.39:0/2990884808 conn(0x5612dc5c8740 msgr2=0x5612dc986050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.419+0000 7f267d511740 1 -- 172.21.15.39:0/2990884808 shutdown_connections 2024-07-04T08:26:59.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.419+0000 7f267d511740 1 -- 172.21.15.39:0/2990884808 wait complete. 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.420+0000 7f267d511740 1 -- start start 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.420+0000 7f267d511740 1 Processor -- start 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.420+0000 7f267d511740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5ab490 0x5612dc988640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.420+0000 7f267d511740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5b3670 0x5612dc988b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.420+0000 7f267d511740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5d4350 0x5612dc9fd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f267d511740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f26701b37e0 con 0x5612dc5d4350 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f267d511740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26701b3110 con 0x5612dc5b3670 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f267d511740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f26701b0350 con 0x5612dc5ab490 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f267d511740 10 client.0 _ll_register_callbacks cb 0x7f26600045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f266fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5ab490 0x5612dc988640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f266fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5ab490 0x5612dc988640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41284/0 (socket says 172.21.15.39:41284) 2024-07-04T08:26:59.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f266fe00640 1 -- 172.21.15.39:0/1948594316 learned_addr learned my addr 172.21.15.39:0/1948594316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f2674e00640 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5d4350 0x5612dc9fd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f266f400640 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5b3670 0x5612dc988b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f266fe00640 1 -- 172.21.15.39:0/1948594316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5b3670 msgr2=0x5612dc988b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f266fe00640 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5b3670 0x5612dc988b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f266fe00640 1 -- 172.21.15.39:0/1948594316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5d4350 msgr2=0x5612dc9fd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f266fe00640 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5d4350 0x5612dc9fd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.421+0000 7f266fe00640 1 -- 172.21.15.39:0/1948594316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612dc9fdf20 con 0x5612dc5ab490 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.422+0000 7f266fe00640 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5ab490 0x5612dc988640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2660106ae0 tx=0x7f26600c6400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33321aa5f01f5dcc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.422+0000 7f265a000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.422+0000 7f265a000640 1 -- 172.21.15.39:0/1948594316 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660049020 con 0x5612dc5ab490 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.422+0000 7f267d511740 1 -- 172.21.15.39:0/1948594316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26600c9a70 con 0x5612dc5ab490 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.422+0000 7f265a000640 1 -- 172.21.15.39:0/1948594316 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f26601005a0 con 0x5612dc5ab490 2024-07-04T08:26:59.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.422+0000 7f265a000640 1 -- 172.21.15.39:0/1948594316 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f26601007a0 con 0x5612dc5ab490 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.422+0000 7f267d511740 10 client.28072 fetch_fsmap learned FSMap version 30 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.422+0000 7f267d511740 1 -- 172.21.15.39:0/1948594316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612dc9fdf20 con 0x5612dc5ab490 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.423+0000 7f265a000640 1 -- 172.21.15.39:0/1948594316 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26600c9a70 con 0x5612dc5ab490 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.423+0000 7f265a000640 1 -- 172.21.15.39:0/1948594316 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f266003b780 con 0x5612dc5ab490 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.423+0000 7f267d511740 10 client.28072 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.423+0000 7f267d511740 10 client.28072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.423+0000 7f267d511740 20 client.28072 populate_metadata read hostname 'smithi039' 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.423+0000 7f267d511740 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5612dc568080 0x5612dca01470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.423+0000 7f267d511740 4 client.28072 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.423+0000 7f267d511740 1 -- 172.21.15.39:0/1948594316 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5612dc8ec8b0 con 0x5612dc568080 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.423+0000 7f266f400640 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5612dc568080 0x5612dca01470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.424+0000 7f266f400640 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5612dc568080 0x5612dca01470 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53bb7daf25843e12 server_cookie=f5e293d9852ad7b9 in_seq=0 out_seq=0 2024-07-04T08:26:59.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.424+0000 7f265a000640 10 client.28072 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.425+0000 7f265a000640 1 -- 172.21.15.39:0/1948594316 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5612dc8ec8b0 con 0x5612dc568080 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.425+0000 7f265a000640 10 client.28072 handle_command_reply: tid=0 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.425+0000 7f267d511740 1 client.28072 shutdown 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.425+0000 7f267d511740 1 -- 172.21.15.39:0/1948594316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5ab490 msgr2=0x5612dc988640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.425+0000 7f267d511740 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5ab490 0x5612dc988640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2660106ae0 tx=0x7f26600c6400 comp rx=0 tx=0).stop 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f267d511740 1 -- 172.21.15.39:0/1948594316 shutdown_connections 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f267d511740 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5612dc568080 0x5612dca01470 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f267d511740 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5612dc5d4350 0x5612dc9fd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f265a000640 0 client.28072 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f267d511740 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5612dc5b3670 0x5612dc988b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f267d511740 1 --2- 172.21.15.39:0/1948594316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5612dc5ab490 0x5612dc988640 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f267d511740 1 -- 172.21.15.39:0/1948594316 >> 172.21.15.39:0/1948594316 conn(0x5612dc5c8740 msgr2=0x5612dc985fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:59.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f267d511740 1 -- 172.21.15.39:0/1948594316 shutdown_connections 2024-07-04T08:26:59.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f267d511740 1 -- 172.21.15.39:0/1948594316 wait complete. 2024-07-04T08:26:59.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.426+0000 7f267d511740 20 client.28072 trim_cache size 0 max 16384 2024-07-04T08:26:59.479 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:26:59.479 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:26:59.479 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "7e77bf78-3310-49fe-83e0-7a191c81cb1a", 2024-07-04T08:26:59.479 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:26:59.479 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:26:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 -- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26540b2870 msgr2=0x7f26540b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26540b2870 0x7f26540b4d80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2668009270 tx=0x7f2668009ca0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 -- 172.21.15.39:0/3578568367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26701013d0 msgr2=0x7f267019ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:26:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26701013d0 0x7f267019ba20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f266002a910 tx=0x7f2660003b90 comp rx=0 tx=0).stop 2024-07-04T08:26:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 -- 172.21.15.39:0/3578568367 shutdown_connections 2024-07-04T08:26:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f26540b2870 0x7f26540b4d80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2670105620 0x7f267019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2670105f70 0x7f26701a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 --2- 172.21.15.39:0/3578568367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26701013d0 0x7f267019ba20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:26:59.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 -- 172.21.15.39:0/3578568367 >> 172.21.15.39:0/3578568367 conn(0x7f2670077000 msgr2=0x7f2670107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:26:59.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 -- 172.21.15.39:0/3578568367 shutdown_connections 2024-07-04T08:26:59.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:26:59.427+0000 7f2675800640 1 -- 172.21.15.39:0/3578568367 wait complete. 2024-07-04T08:26:59.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:59 smithi133 ceph-mon[37050]: pgmap v843: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-07-04T08:26:59.556 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:26:59 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/128231787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:26:59.871 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id b7445762 --release --await' 2024-07-04T08:26:59.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id b7445762 --release --await 2024-07-04T08:27:00.438 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:00 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:27:00.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:00 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:00.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:00 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:27:00.439 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:00 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:00.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:27:00.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:00 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:27:00.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:00 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:00.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:00 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:27:00.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:00 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:00.629 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:00.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:00 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:27:00.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:00 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:00.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:00 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:27:00.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:00 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:01.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:01 smithi039 ceph-mon[32062]: pgmap v844: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:27:01.443 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:01.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:01 smithi133 ceph-mon[37050]: pgmap v844: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:27:01.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:01 smithi050 ceph-mon[36577]: pgmap v844: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:27:03.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:03 smithi039 ceph-mon[32062]: pgmap v845: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-07-04T08:27:03.341 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:03 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:27:03.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:03 smithi133 ceph-mon[37050]: pgmap v845: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-07-04T08:27:03.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:03 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:27:03.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:03 smithi050 ceph-mon[36577]: pgmap v845: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-07-04T08:27:03.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:03 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:27:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.428+0000 7fb2df200640 1 -- 172.21.15.39:0/10519428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d81137d0 msgr2=0x7fb2d8115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.428+0000 7fb2df200640 1 --2- 172.21.15.39:0/10519428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d81137d0 0x7fb2d8115ca0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009900 tx=0x7fb2d402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.428+0000 7fb2df200640 1 -- 172.21.15.39:0/10519428 shutdown_connections 2024-07-04T08:27:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.428+0000 7fb2df200640 1 --2- 172.21.15.39:0/10519428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d8075320 0x7fb2d8075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.428+0000 7fb2df200640 1 --2- 172.21.15.39:0/10519428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d81137d0 0x7fb2d8115ca0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.428+0000 7fb2df200640 1 --2- 172.21.15.39:0/10519428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d80768d0 0x7fb2d8074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.428+0000 7fb2df200640 1 -- 172.21.15.39:0/10519428 >> 172.21.15.39:0/10519428 conn(0x7fb2d80fe990 msgr2=0x7fb2d8100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.428+0000 7fb2df200640 1 -- 172.21.15.39:0/10519428 shutdown_connections 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.428+0000 7fb2df200640 1 -- 172.21.15.39:0/10519428 wait complete. 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.430+0000 7fb2df200640 1 Processor -- start 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.430+0000 7fb2df200640 1 -- start start 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.430+0000 7fb2df200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d8075320 0x7fb2d819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.430+0000 7fb2df200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d80768d0 0x7fb2d81a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.430+0000 7fb2df200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d81137d0 0x7fb2d81a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.430+0000 7fb2df200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8119410 con 0x7fb2d81137d0 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.430+0000 7fb2df200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8119290 con 0x7fb2d8075320 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.430+0000 7fb2df200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8119590 con 0x7fb2d80768d0 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2de800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d81137d0 0x7fb2d81a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2dde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d8075320 0x7fb2d819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2dde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d8075320 0x7fb2d819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53400/0 (socket says 172.21.15.39:53400) 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2dde00640 1 -- 172.21.15.39:0/4209880282 learned_addr learned my addr 172.21.15.39:0/4209880282 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2dd400640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d80768d0 0x7fb2d81a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2dde00640 1 -- 172.21.15.39:0/4209880282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d80768d0 msgr2=0x7fb2d81a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2dde00640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d80768d0 0x7fb2d81a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2dde00640 1 -- 172.21.15.39:0/4209880282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d81137d0 msgr2=0x7fb2d81a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2dde00640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d81137d0 0x7fb2d81a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.431+0000 7fb2dde00640 1 -- 172.21.15.39:0/4209880282 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d81a5d50 con 0x7fb2d8075320 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.432+0000 7fb2dde00640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d8075320 0x7fb2d819fde0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000eb90 tx=0x7fb2d000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.432+0000 7fb2cea00640 1 -- 172.21.15.39:0/4209880282 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0004d20 con 0x7fb2d8075320 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.432+0000 7fb2cea00640 1 -- 172.21.15.39:0/4209880282 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d000ee90 con 0x7fb2d8075320 2024-07-04T08:27:03.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.432+0000 7fb2df200640 1 -- 172.21.15.39:0/4209880282 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d81a6040 con 0x7fb2d8075320 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.432+0000 7fb2cea00640 1 -- 172.21.15.39:0/4209880282 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0010670 con 0x7fb2d8075320 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.432+0000 7fb2df200640 1 -- 172.21.15.39:0/4209880282 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81ac7a0 con 0x7fb2d8075320 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.436+0000 7fb2e6d9d740 1 Processor -- start 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.436+0000 7fb2e6d9d740 1 -- start start 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.436+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.436+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 0x55f5cc6b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.436+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2eb840 0x55f5cc6b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.436+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f5cc5cbb00 con 0x55f5cc2eb840 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.436+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f5cc627290 con 0x55f5cc2e3660 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.436+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f5cc616830 con 0x55f5cc30c520 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.436+0000 7fb2cea00640 1 -- 172.21.15.39:0/4209880282 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb2d0005020 con 0x7fb2d8075320 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2dd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2eb840 0x55f5cc6b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2de800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2dde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 0x55f5cc6b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2de800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57530/0 (socket says 172.21.15.39:57530) 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2de800640 1 -- 172.21.15.39:0/2449467260 learned_addr learned my addr 172.21.15.39:0/2449467260 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2de800640 1 -- 172.21.15.39:0/2449467260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 msgr2=0x55f5cc6b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2de800640 1 --2- 172.21.15.39:0/2449467260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 0x55f5cc6b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2de800640 1 -- 172.21.15.39:0/2449467260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2eb840 msgr2=0x55f5cc6b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2de800640 1 --2- 172.21.15.39:0/2449467260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2eb840 0x55f5cc6b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2de800640 1 -- 172.21.15.39:0/2449467260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5cc5d0760 con 0x55f5cc30c520 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.437+0000 7fb2dde00640 1 --2- 172.21.15.39:0/2449467260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 0x55f5cc6b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.438+0000 7fb2de800640 1 --2- 172.21.15.39:0/2449467260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6b1470 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009900 tx=0x7fb2d4032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=883d31df3237dec0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.438+0000 7fb2c3400640 1 -- 172.21.15.39:0/2449467260 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4040070 con 0x55f5cc30c520 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.438+0000 7fb2c3400640 1 -- 172.21.15.39:0/2449467260 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d4004030 con 0x55f5cc30c520 2024-07-04T08:27:03.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.438+0000 7fb2c3400640 1 -- 172.21.15.39:0/2449467260 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4035040 con 0x55f5cc30c520 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.438+0000 7fb2cea00640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c40b29e0 0x7fb2c40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.438+0000 7fb2dde00640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c40b29e0 0x7fb2c40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.438+0000 7fb2cea00640 1 -- 172.21.15.39:0/4209880282 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fb2d00e2f50 con 0x7fb2d8075320 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/2449467260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 msgr2=0x55f5cc6b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/2449467260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6b1470 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009900 tx=0x7fb2d4032b20 comp rx=0 tx=0).stop 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2dde00640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c40b29e0 0x7fb2c40b4ef0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55f5cc6b00f0 tx=0x7fb2d00e6a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/2449467260 shutdown_connections 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/2449467260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2eb840 0x55f5cc6b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/2449467260 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 0x55f5cc6b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/2449467260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6b1470 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/2449467260 >> 172.21.15.39:0/2449467260 conn(0x55f5cc300910 msgr2=0x55f5cc6af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:03.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/2449467260 shutdown_connections 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.439+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/2449467260 wait complete. 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.440+0000 7fb2e6d9d740 1 -- start start 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.440+0000 7fb2e6d9d740 1 Processor -- start 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2e3660 0x55f5cc6bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f5cc5bdd00 con 0x55f5cc30c520 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f5cc61a870 con 0x55f5cc2eb840 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f5cc47abb0 con 0x55f5cc2e3660 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2e6d9d740 10 client.0 _ll_register_callbacks cb 0x55f5cc5be9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2dd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2de800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2dd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53426/0 (socket says 172.21.15.39:53426) 2024-07-04T08:27:03.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2dd400640 1 -- 172.21.15.39:0/3623394585 learned_addr learned my addr 172.21.15.39:0/3623394585 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.441+0000 7fb2de800640 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2e3660 0x55f5cc6bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2dd400640 1 -- 172.21.15.39:0/3623394585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2e3660 msgr2=0x55f5cc6bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2dd400640 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2e3660 0x55f5cc6bcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2dd400640 1 -- 172.21.15.39:0/3623394585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 msgr2=0x55f5cc6c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2dd400640 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2dd400640 1 -- 172.21.15.39:0/3623394585 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5cc5ad640 con 0x55f5cc2eb840 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2de800640 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2dd400640 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bd0b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8012700 tx=0x7fb2c8012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a212a79b652b773d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2b7e00640 1 -- 172.21.15.39:0/3623394585 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800f8c0 con 0x55f5cc2eb840 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/3623394585 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d4004030 con 0x55f5cc2eb840 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.442+0000 7fb2b7e00640 1 -- 172.21.15.39:0/3623394585 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fb2c800fa60 con 0x55f5cc2eb840 2024-07-04T08:27:03.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2b7e00640 1 -- 172.21.15.39:0/3623394585 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2c8025050 con 0x55f5cc2eb840 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2e6d9d740 10 client.28154 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/3623394585 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5cc377b00 con 0x55f5cc2eb840 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2b7e00640 1 -- 172.21.15.39:0/3623394585 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d4004030 con 0x55f5cc2eb840 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2b7e00640 1 -- 172.21.15.39:0/3623394585 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb2c80050b0 con 0x55f5cc2eb840 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2e6d9d740 10 client.28154 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2e6d9d740 10 client.28154 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2e6d9d740 20 client.28154 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f5cc2caea0 0x55f5cc6cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2e6d9d740 4 client.28154 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.443+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/3623394585 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5cc6291e0 con 0x55f5cc2caea0 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.444+0000 7fb2de800640 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f5cc2caea0 0x55f5cc6cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.444+0000 7fb2de800640 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f5cc2caea0 0x55f5cc6cec60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6d88927a8fe36a server_cookie=8e1ec251d2567f07 in_seq=0 out_seq=0 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.444+0000 7fb2b7e00640 10 client.28154 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:03.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.445+0000 7fb2b7e00640 1 -- 172.21.15.39:0/3623394585 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f5cc6291e0 con 0x55f5cc2caea0 2024-07-04T08:27:03.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.445+0000 7fb2b7e00640 10 client.28154 handle_command_reply: tid=0 2024-07-04T08:27:03.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.445+0000 7fb2e6d9d740 1 client.28154 shutdown 2024-07-04T08:27:03.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/3623394585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 msgr2=0x55f5cc6bd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:03.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bd0b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8012700 tx=0x7fb2c8012bc0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/3623394585 shutdown_connections 2024-07-04T08:27:03.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f5cc2caea0 0x55f5cc6cec60 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bd0b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2b7e00640 0 client.28154 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/3623394585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2e3660 0x55f5cc6bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/3623394585 >> 172.21.15.39:0/3623394585 conn(0x55f5cc300910 msgr2=0x55f5cc6b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/3623394585 shutdown_connections 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/3623394585 wait complete. 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.446+0000 7fb2e6d9d740 20 client.28154 trim_cache size 0 max 16384 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.466+0000 7fb2e6d9d740 1 Processor -- start 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.466+0000 7fb2e6d9d740 1 -- start start 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.466+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2e3660 0x55f5cc6bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81b6750 con 0x55f5cc2e3660 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81b8290 con 0x55f5cc2eb840 2024-07-04T08:27:03.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81b6e20 con 0x55f5cc30c520 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2de800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2e3660 0x55f5cc6bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2dd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2de800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2e3660 0x55f5cc6bbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58706/0 (socket says 172.21.15.39:58706) 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2de800640 1 -- 172.21.15.39:0/1608419199 learned_addr learned my addr 172.21.15.39:0/1608419199 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.467+0000 7fb2de800640 1 --2- 172.21.15.39:0/1608419199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.468+0000 7fb2dd400640 1 -- 172.21.15.39:0/1608419199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 msgr2=0x55f5cc6c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.468+0000 7fb2dd400640 1 --2- 172.21.15.39:0/1608419199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.468+0000 7fb2dd400640 1 -- 172.21.15.39:0/1608419199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2e3660 msgr2=0x55f5cc6bbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.468+0000 7fb2dd400640 1 --2- 172.21.15.39:0/1608419199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2e3660 0x55f5cc6bbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.468+0000 7fb2dd400640 1 -- 172.21.15.39:0/1608419199 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5cc3ee960 con 0x55f5cc2eb840 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.468+0000 7fb2de800640 1 --2- 172.21.15.39:0/1608419199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.468+0000 7fb2dd400640 1 --2- 172.21.15.39:0/1608419199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bc040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009e00 tx=0x7fb2c80096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36871cc666c8a01c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.469+0000 7fb2c2a00640 1 -- 172.21.15.39:0/1608419199 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8022040 con 0x55f5cc2eb840 2024-07-04T08:27:03.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.469+0000 7fb2c2a00640 1 -- 172.21.15.39:0/1608419199 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c800fd00 con 0x55f5cc2eb840 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.469+0000 7fb2c2a00640 1 -- 172.21.15.39:0/1608419199 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8010430 con 0x55f5cc2eb840 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1608419199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 msgr2=0x55f5cc6bc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1608419199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bc040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009e00 tx=0x7fb2c80096c0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1608419199 shutdown_connections 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1608419199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc30c520 0x55f5cc6c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1608419199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2eb840 0x55f5cc6bc040 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1608419199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc2e3660 0x55f5cc6bbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1608419199 >> 172.21.15.39:0/1608419199 conn(0x55f5cc300910 msgr2=0x55f5cc6bb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1608419199 shutdown_connections 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1608419199 wait complete. 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.470+0000 7fb2e6d9d740 1 -- start start 2024-07-04T08:27:03.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.471+0000 7fb2e6d9d740 1 Processor -- start 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.471+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 0x55f5cc6c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.471+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2eb840 0x55f5cc6c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.471+0000 7fb2e6d9d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.471+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81b7bc0 con 0x55f5cc30c520 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.471+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81b74f0 con 0x55f5cc2e3660 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.471+0000 7fb2e6d9d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d81b4730 con 0x55f5cc2eb840 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.471+0000 7fb2e6d9d740 10 client.0 _ll_register_callbacks cb 0x7fb2c8009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.471+0000 7fb2de800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2de800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58716/0 (socket says 172.21.15.39:58716) 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2de800640 1 -- 172.21.15.39:0/1232358461 learned_addr learned my addr 172.21.15.39:0/1232358461 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2dd400640 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2eb840 0x55f5cc6c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2de800640 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 0x55f5cc6c0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2de800640 1 -- 172.21.15.39:0/1232358461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2eb840 msgr2=0x55f5cc6c12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:03.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2de800640 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2eb840 0x55f5cc6c12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2de800640 1 -- 172.21.15.39:0/1232358461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 msgr2=0x55f5cc6c0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2de800640 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 0x55f5cc6c0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2de800640 1 -- 172.21.15.39:0/1232358461 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5cc5e3880 con 0x55f5cc30c520 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2dd400640 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2eb840 0x55f5cc6c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2de800640 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c1820 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4000c00 tx=0x7fb2d4005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300c39dea5264a66 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.472+0000 7fb2b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.473+0000 7fb2b7e00640 1 -- 172.21.15.39:0/1232358461 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4035040 con 0x55f5cc30c520 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.473+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1232358461 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c800fd00 con 0x55f5cc30c520 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.473+0000 7fb2b7e00640 1 -- 172.21.15.39:0/1232358461 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7fb2d40316c0 con 0x55f5cc30c520 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.473+0000 7fb2b7e00640 1 -- 172.21.15.39:0/1232358461 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2d4068050 con 0x55f5cc30c520 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.473+0000 7fb2e6d9d740 10 client.18024 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.473+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1232358461 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f5cc5df430 con 0x55f5cc30c520 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.473+0000 7fb2b7e00640 1 -- 172.21.15.39:0/1232358461 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c800fd00 con 0x55f5cc30c520 2024-07-04T08:27:03.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.474+0000 7fb2b7e00640 1 -- 172.21.15.39:0/1232358461 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb2d4052040 con 0x55f5cc30c520 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.474+0000 7fb2e6d9d740 10 client.18024 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.474+0000 7fb2e6d9d740 10 client.18024 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.474+0000 7fb2e6d9d740 20 client.18024 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.474+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f5cc2a0250 0x55f5cc7397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.474+0000 7fb2e6d9d740 4 client.18024 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b7445762", "release": true, "await": true}] 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.474+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1232358461 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b7445762", "release": true, "await": true}) -- 0x55f5cc624c80 con 0x55f5cc2a0250 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.474+0000 7fb2dd400640 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f5cc2a0250 0x55f5cc7397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.475+0000 7fb2dd400640 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f5cc2a0250 0x55f5cc7397b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=625c41b2244ac097 server_cookie=8925ae19ecfdf9bd in_seq=0 out_seq=0 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.475+0000 7fb2b7e00640 10 client.18024 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.478+0000 7fb2b7e00640 1 -- 172.21.15.39:0/1232358461 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f5cc624c80 con 0x55f5cc2a0250 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.478+0000 7fb2b7e00640 10 client.18024 handle_command_reply: tid=0 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 client.18024 shutdown 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1232358461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 msgr2=0x55f5cc6c1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:03.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c1820 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4000c00 tx=0x7fb2d4005150 comp rx=0 tx=0).stop 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1232358461 shutdown_connections 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f5cc2a0250 0x55f5cc7397b0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f5cc30c520 0x55f5cc6c1820 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f5cc2eb840 0x55f5cc6c12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 --2- 172.21.15.39:0/1232358461 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f5cc2e3660 0x55f5cc6c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1232358461 >> 172.21.15.39:0/1232358461 conn(0x55f5cc300910 msgr2=0x55f5cc6bbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2b7e00640 0 client.18024 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:27:03.750 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 36, 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "b7445762": { 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 36, 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 150.0, 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:27:03.751 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1232358461 shutdown_connections 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 1 -- 172.21.15.39:0/1232358461 wait complete. 2024-07-04T08:27:03.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.479+0000 7fb2e6d9d740 20 client.18024 trim_cache size 0 max 16384 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 -- 172.21.15.39:0/4209880282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c40b29e0 msgr2=0x7fb2c40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c40b29e0 0x7fb2c40b4ef0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55f5cc6b00f0 tx=0x7fb2d00e6a40 comp rx=0 tx=0).stop 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 -- 172.21.15.39:0/4209880282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d8075320 msgr2=0x7fb2d819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d8075320 0x7fb2d819fde0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000eb90 tx=0x7fb2d000c6a0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 -- 172.21.15.39:0/4209880282 shutdown_connections 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c40b29e0 0x7fb2c40b4ef0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d81137d0 0x7fb2d81a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2d80768d0 0x7fb2d81a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 --2- 172.21.15.39:0/4209880282 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2d8075320 0x7fb2d819fde0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 -- 172.21.15.39:0/4209880282 >> 172.21.15.39:0/4209880282 conn(0x7fb2d80fe990 msgr2=0x7fb2d810f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 -- 172.21.15.39:0/4209880282 shutdown_connections 2024-07-04T08:27:03.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:03.480+0000 7fb2df200640 1 -- 172.21.15.39:0/4209880282 wait complete. 2024-07-04T08:27:04.584 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b7445762', total seconds elapsed: 39.9 2024-07-04T08:27:05.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.170+0000 7f5dc2800640 1 -- 172.21.15.39:0/801093601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dbc1137b0 msgr2=0x7f5dbc115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:05.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.170+0000 7f5dc2800640 1 --2- 172.21.15.39:0/801093601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dbc1137b0 0x7f5dbc115c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009be0 tx=0x7f5db402a6c0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.170+0000 7f5dc2800640 1 -- 172.21.15.39:0/801093601 shutdown_connections 2024-07-04T08:27:05.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.170+0000 7f5dc2800640 1 --2- 172.21.15.39:0/801093601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dbc1137b0 0x7f5dbc115c80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.170+0000 7f5dc2800640 1 --2- 172.21.15.39:0/801093601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dbc075320 0x7f5dbc075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.170+0000 7f5dc2800640 1 --2- 172.21.15.39:0/801093601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dbc0768d0 0x7f5dbc074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.170+0000 7f5dc2800640 1 -- 172.21.15.39:0/801093601 >> 172.21.15.39:0/801093601 conn(0x7f5dbc0fe930 msgr2=0x7f5dbc100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:05.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.170+0000 7f5dc2800640 1 -- 172.21.15.39:0/801093601 shutdown_connections 2024-07-04T08:27:05.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.170+0000 7f5dc2800640 1 -- 172.21.15.39:0/801093601 wait complete. 2024-07-04T08:27:05.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.172+0000 7f5dc2800640 1 Processor -- start 2024-07-04T08:27:05.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.172+0000 7f5dc2800640 1 -- start start 2024-07-04T08:27:05.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc2800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dbc075320 0x7f5dbc19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dbc0768d0 0x7f5dbc1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc2800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dbc1137b0 0x7f5dbc1a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc2800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc119400 con 0x7f5dbc075320 2024-07-04T08:27:05.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc2800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc119280 con 0x7f5dbc1137b0 2024-07-04T08:27:05.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc2800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc119580 con 0x7f5dbc0768d0 2024-07-04T08:27:05.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dbc075320 0x7f5dbc19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dbc075320 0x7f5dbc19fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58736/0 (socket says 172.21.15.39:58736) 2024-07-04T08:27:05.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc1400640 1 -- 172.21.15.39:0/3680696319 learned_addr learned my addr 172.21.15.39:0/3680696319 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:05.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dbc0768d0 0x7f5dbc1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.173+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dbc1137b0 0x7f5dbc1a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.174+0000 7f5dc1400640 1 -- 172.21.15.39:0/3680696319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dbc0768d0 msgr2=0x7f5dbc1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:05.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.174+0000 7f5dc1400640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dbc0768d0 0x7f5dbc1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.174+0000 7f5dc1400640 1 -- 172.21.15.39:0/3680696319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dbc1137b0 msgr2=0x7f5dbc1a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:05.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.174+0000 7f5dc1400640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dbc1137b0 0x7f5dbc1a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.174+0000 7f5dc1400640 1 -- 172.21.15.39:0/3680696319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc1a5da0 con 0x7f5dbc075320 2024-07-04T08:27:05.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.174+0000 7f5dc1400640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dbc075320 0x7f5dbc19fe30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e9c0 tx=0x7f5db000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:05.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.175+0000 7f5dba000640 1 -- 172.21.15.39:0/3680696319 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000ccc0 con 0x7f5dbc075320 2024-07-04T08:27:05.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.175+0000 7f5dc2800640 1 -- 172.21.15.39:0/3680696319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc1a6090 con 0x7f5dbc075320 2024-07-04T08:27:05.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.175+0000 7f5dba000640 1 -- 172.21.15.39:0/3680696319 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5db00054d0 con 0x7f5dbc075320 2024-07-04T08:27:05.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.175+0000 7f5dba000640 1 -- 172.21.15.39:0/3680696319 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0010430 con 0x7f5dbc075320 2024-07-04T08:27:05.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.175+0000 7f5dc2800640 1 -- 172.21.15.39:0/3680696319 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc1ac7f0 con 0x7f5dbc075320 2024-07-04T08:27:05.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.178+0000 7f5dca382740 1 Processor -- start 2024-07-04T08:27:05.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.178+0000 7f5dca382740 1 -- start start 2024-07-04T08:27:05.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dca382740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266c07070 0x55d266fab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dca382740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 0x55d266fabe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dca382740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266be6390 0x55d266fac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dca382740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d266f21ea0 con 0x55d266c07070 2024-07-04T08:27:05.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dca382740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d266f11440 con 0x55d266be6390 2024-07-04T08:27:05.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dca382740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d266f15480 con 0x55d266bde1b0 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dba000640 1 -- 172.21.15.39:0/3680696319 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5db0004f80 con 0x7f5dbc075320 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dc1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266c07070 0x55d266fab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dc0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266be6390 0x55d266fac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dc0a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266be6390 0x55d266fac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53472/0 (socket says 172.21.15.39:53472) 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dc1e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266c07070 0x55d266fab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58740/0 (socket says 172.21.15.39:58740) 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.179+0000 7f5dc0a00640 1 -- 172.21.15.39:0/3174709232 learned_addr learned my addr 172.21.15.39:0/3174709232 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.180+0000 7f5dc1400640 1 --2- 172.21.15.39:0/3174709232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 0x55d266fabe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.180+0000 7f5dc0a00640 1 -- 172.21.15.39:0/3174709232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 msgr2=0x55d266fabe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.180+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/3174709232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 0x55d266fabe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.180+0000 7f5dc0a00640 1 -- 172.21.15.39:0/3174709232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266c07070 msgr2=0x55d266fab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:05.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.180+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/3174709232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266c07070 0x55d266fab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.180+0000 7f5dc0a00640 1 -- 172.21.15.39:0/3174709232 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d266ea9610 con 0x55d266be6390 2024-07-04T08:27:05.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.180+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/3174709232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266be6390 0x55d266fac390 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009c60 tx=0x7f5dac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25aac45245d35fa3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:05.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.180+0000 7f5daaa00640 1 -- 172.21.15.39:0/3174709232 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac004230 con 0x55d266be6390 2024-07-04T08:27:05.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.181+0000 7f5daaa00640 1 -- 172.21.15.39:0/3174709232 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dac0043d0 con 0x55d266be6390 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.181+0000 7f5daaa00640 1 -- 172.21.15.39:0/3174709232 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0196f0 con 0x55d266be6390 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.181+0000 7f5dba000640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5da40b29e0 0x7f5da40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.181+0000 7f5dba000640 1 -- 172.21.15.39:0/3680696319 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f5db00e2cb0 con 0x7f5dbc075320 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.181+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5da40b29e0 0x7f5da40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.181+0000 7f5dca382740 1 -- 172.21.15.39:0/3174709232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266be6390 msgr2=0x55d266fac390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.181+0000 7f5dca382740 1 --2- 172.21.15.39:0/3174709232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266be6390 0x55d266fac390 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009c60 tx=0x7f5dac011810 comp rx=0 tx=0).stop 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.182+0000 7f5dca382740 1 -- 172.21.15.39:0/3174709232 shutdown_connections 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.182+0000 7f5dca382740 1 --2- 172.21.15.39:0/3174709232 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266be6390 0x55d266fac390 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.182+0000 7f5dca382740 1 --2- 172.21.15.39:0/3174709232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 0x55d266fabe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.182+0000 7f5dca382740 1 --2- 172.21.15.39:0/3174709232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266c07070 0x55d266fab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.182+0000 7f5dca382740 1 -- 172.21.15.39:0/3174709232 >> 172.21.15.39:0/3174709232 conn(0x55d266bfb460 msgr2=0x55d266fa9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:05.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.182+0000 7f5dca382740 1 -- 172.21.15.39:0/3174709232 shutdown_connections 2024-07-04T08:27:05.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.182+0000 7f5dca382740 1 -- 172.21.15.39:0/3174709232 wait complete. 2024-07-04T08:27:05.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.183+0000 7f5dca382740 1 -- start start 2024-07-04T08:27:05.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.183+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5da40b29e0 0x7f5da40b4ef0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009370 tx=0x7f5db4004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:05.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.183+0000 7f5dca382740 1 Processor -- start 2024-07-04T08:27:05.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.183+0000 7f5dca382740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fb74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.183+0000 7f5dca382740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fb7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dca382740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266c07070 0x55d266fc3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dca382740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d266d75700 con 0x55d266bde1b0 2024-07-04T08:27:05.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dca382740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d266eb8b70 con 0x55d266c07070 2024-07-04T08:27:05.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dca382740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d266fba0f0 con 0x55d266be6390 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dca382740 10 client.0 _ll_register_callbacks cb 0x55d266f0a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fb74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fb74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58748/0 (socket says 172.21.15.39:58748) 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1400640 1 -- 172.21.15.39:0/3425824005 learned_addr learned my addr 172.21.15.39:0/3425824005 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266c07070 0x55d266fc3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fb7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1400640 1 -- 172.21.15.39:0/3425824005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 msgr2=0x55d266fb7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1400640 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fb7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1400640 1 -- 172.21.15.39:0/3425824005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266c07070 msgr2=0x55d266fc3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1400640 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266c07070 0x55d266fc3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1400640 1 -- 172.21.15.39:0/3425824005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d266c72650 con 0x55d266bde1b0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.184+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266c07070 0x55d266fc3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.185+0000 7f5dc1400640 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fb74c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5db00a98d0 tx=0x7f5db00a9a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d078313043ed6fd9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.185+0000 7f5d9f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.185+0000 7f5d9f400640 1 -- 172.21.15.39:0/3425824005 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db00e04d0 con 0x55d266bde1b0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.185+0000 7f5d9f400640 1 -- 172.21.15.39:0/3425824005 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f5db00a8e30 con 0x55d266bde1b0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.185+0000 7f5dca382740 1 -- 172.21.15.39:0/3425824005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dac0043d0 con 0x55d266bde1b0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.185+0000 7f5d9f400640 1 -- 172.21.15.39:0/3425824005 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5db00b2370 con 0x55d266bde1b0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5dca382740 10 client.18036 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5dca382740 1 -- 172.21.15.39:0/3425824005 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d266fc6070 con 0x55d266bde1b0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5d9f400640 1 -- 172.21.15.39:0/3425824005 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dac0043d0 con 0x55d266bde1b0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5d9f400640 1 -- 172.21.15.39:0/3425824005 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5db00a5040 con 0x55d266bde1b0 2024-07-04T08:27:05.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5dca382740 10 client.18036 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5dca382740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:27:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5dca382740 20 client.18036 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5dca382740 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d266bc59f0 0x55d266fc94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5dca382740 4 client.18036 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:27:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.186+0000 7f5dca382740 1 -- 172.21.15.39:0/3425824005 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d266f23df0 con 0x55d266bc59f0 2024-07-04T08:27:05.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.187+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d266bc59f0 0x55d266fc94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.187+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d266bc59f0 0x55d266fc94f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a70360cc8185591e server_cookie=80a3f8a617a5217b in_seq=0 out_seq=0 2024-07-04T08:27:05.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.187+0000 7f5d9f400640 10 client.18036 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:05.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.188+0000 7f5d9f400640 1 -- 172.21.15.39:0/3425824005 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d266f23df0 con 0x55d266bc59f0 2024-07-04T08:27:05.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.188+0000 7f5d9f400640 10 client.18036 handle_command_reply: tid=0 2024-07-04T08:27:05.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.188+0000 7f5dca382740 1 client.18036 shutdown 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 -- 172.21.15.39:0/3425824005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 msgr2=0x55d266fb74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fb74c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5db00a98d0 tx=0x7f5db00a9a60 comp rx=0 tx=0).stop 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 -- 172.21.15.39:0/3425824005 shutdown_connections 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d266bc59f0 0x55d266fc94f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266c07070 0x55d266fc3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fb7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 --2- 172.21.15.39:0/3425824005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fb74c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5d9f400640 0 client.18036 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 -- 172.21.15.39:0/3425824005 >> 172.21.15.39:0/3425824005 conn(0x55d266bfb460 msgr2=0x55d266faeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 -- 172.21.15.39:0/3425824005 shutdown_connections 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 1 -- 172.21.15.39:0/3425824005 wait complete. 2024-07-04T08:27:05.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.189+0000 7f5dca382740 20 client.18036 trim_cache size 0 max 16384 2024-07-04T08:27:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.209+0000 7f5dca382740 1 Processor -- start 2024-07-04T08:27:05.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.209+0000 7f5dca382740 1 -- start start 2024-07-04T08:27:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dca382740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fad300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dca382740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 0x55d266fb8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dca382740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266be6390 0x55d266fc3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dca382740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b6820 con 0x55d266be6390 2024-07-04T08:27:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dca382740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b8360 con 0x55d266bfb460 2024-07-04T08:27:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dca382740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b6ef0 con 0x55d266bde1b0 2024-07-04T08:27:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dc1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dc1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fad300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53486/0 (socket says 172.21.15.39:53486) 2024-07-04T08:27:05.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dc1400640 1 -- 172.21.15.39:0/126879422 learned_addr learned my addr 172.21.15.39:0/126879422 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/126879422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 0x55d266fb8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.210+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/126879422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266be6390 0x55d266fc3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.211+0000 7f5dc1400640 1 -- 172.21.15.39:0/126879422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 msgr2=0x55d266fb8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.211+0000 7f5dc1400640 1 --2- 172.21.15.39:0/126879422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 0x55d266fb8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.211+0000 7f5dc1400640 1 -- 172.21.15.39:0/126879422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266be6390 msgr2=0x55d266fc3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.211+0000 7f5dc1400640 1 --2- 172.21.15.39:0/126879422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266be6390 0x55d266fc3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.211+0000 7f5dc1400640 1 -- 172.21.15.39:0/126879422 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d266fae770 con 0x55d266bfb460 2024-07-04T08:27:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.211+0000 7f5dc1400640 1 --2- 172.21.15.39:0/126879422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fad300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5db00b2b00 tx=0x7f5db00a9ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b47581884ced87 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.211+0000 7f5daa000640 1 -- 172.21.15.39:0/126879422 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db00e04d0 con 0x55d266bfb460 2024-07-04T08:27:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.211+0000 7f5daa000640 1 -- 172.21.15.39:0/126879422 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5db00e6230 con 0x55d266bfb460 2024-07-04T08:27:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.211+0000 7f5daa000640 1 -- 172.21.15.39:0/126879422 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db00a9340 con 0x55d266bfb460 2024-07-04T08:27:05.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.212+0000 7f5dca382740 1 -- 172.21.15.39:0/126879422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 msgr2=0x55d266fad300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.212+0000 7f5dca382740 1 --2- 172.21.15.39:0/126879422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fad300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5db00b2b00 tx=0x7f5db00a9ad0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.204 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.212+0000 7f5dca382740 1 -- 172.21.15.39:0/126879422 shutdown_connections 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.212+0000 7f5dca382740 1 --2- 172.21.15.39:0/126879422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266be6390 0x55d266fc3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.212+0000 7f5dca382740 1 --2- 172.21.15.39:0/126879422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266bde1b0 0x55d266fb8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.212+0000 7f5dca382740 1 --2- 172.21.15.39:0/126879422 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fad300 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.212+0000 7f5dca382740 1 -- 172.21.15.39:0/126879422 >> 172.21.15.39:0/126879422 conn(0x55d266fb7b80 msgr2=0x55d266fb87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.212+0000 7f5dca382740 1 -- 172.21.15.39:0/126879422 shutdown_connections 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.213+0000 7f5dca382740 1 -- 172.21.15.39:0/126879422 wait complete. 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.213+0000 7f5dca382740 1 -- start start 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.213+0000 7f5dca382740 1 Processor -- start 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dca382740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fbf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dca382740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fbf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dca382740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.205 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dca382740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b7c90 con 0x55d266bde1b0 2024-07-04T08:27:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dca382740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b75c0 con 0x55d266bfb460 2024-07-04T08:27:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dca382740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1b4800 con 0x55d266be6390 2024-07-04T08:27:05.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dca382740 10 client.0 _ll_register_callbacks cb 0x7f5db0004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dc0a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fbf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dc1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fbf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dc0a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fbf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57634/0 (socket says 172.21.15.39:57634) 2024-07-04T08:27:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dc0a00640 1 -- 172.21.15.39:0/1636818441 learned_addr learned my addr 172.21.15.39:0/1636818441 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:05.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.214+0000 7f5dc1e00640 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5dc0a00640 1 -- 172.21.15.39:0/1636818441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 msgr2=0x55d266fba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5dc0a00640 1 -- 172.21.15.39:0/1636818441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 msgr2=0x55d266fbf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fbf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5dc0a00640 1 -- 172.21.15.39:0/1636818441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d266fcf760 con 0x55d266be6390 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5dc0a00640 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fbf8f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009800 tx=0x7f5dac009270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f50e885f477c23 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5d9f400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5d9f400640 1 -- 172.21.15.39:0/1636818441 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac032020 con 0x55d266be6390 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5d9f400640 1 -- 172.21.15.39:0/1636818441 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9424+0+0 (secure 0 0 0) 0x7f5dac005400 con 0x55d266be6390 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.215+0000 7f5dca382740 1 -- 172.21.15.39:0/1636818441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d266f1f890 con 0x55d266be6390 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.216+0000 7f5d9f400640 1 -- 172.21.15.39:0/1636818441 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5dac03d050 con 0x55d266be6390 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.216+0000 7f5dca382740 10 client.28108 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:05.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.216+0000 7f5dca382740 1 -- 172.21.15.39:0/1636818441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d267032070 con 0x55d266be6390 2024-07-04T08:27:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.216+0000 7f5d9f400640 1 -- 172.21.15.39:0/1636818441 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d266f1f890 con 0x55d266be6390 2024-07-04T08:27:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.216+0000 7f5d9f400640 1 -- 172.21.15.39:0/1636818441 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5dac018070 con 0x55d266be6390 2024-07-04T08:27:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.217+0000 7f5dca382740 10 client.28108 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.217+0000 7f5dca382740 10 client.28108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:27:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.217+0000 7f5dca382740 20 client.28108 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.217+0000 7f5dca382740 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d266c07070 0x55d2670346c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.217+0000 7f5dca382740 4 client.28108 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:27:05.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.217+0000 7f5dca382740 1 -- 172.21.15.39:0/1636818441 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d266f1cfd0 con 0x55d266c07070 2024-07-04T08:27:05.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.217+0000 7f5dc1400640 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d266c07070 0x55d2670346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.217+0000 7f5dc1400640 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d266c07070 0x55d2670346c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a330cf1129d1ce server_cookie=286dbc5057160be4 in_seq=0 out_seq=0 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.217+0000 7f5d9f400640 10 client.28108 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5d9f400640 1 -- 172.21.15.39:0/1636818441 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d266f1cfd0 con 0x55d266c07070 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5d9f400640 10 client.28108 handle_command_reply: tid=0 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 client.28108 shutdown 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 -- 172.21.15.39:0/1636818441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 msgr2=0x55d266fbf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fbf8f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009800 tx=0x7f5dac009270 comp rx=0 tx=0).stop 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 -- 172.21.15.39:0/1636818441 shutdown_connections 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55d266c07070 0x55d2670346c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d266bfb460 0x55d266fba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d266be6390 0x55d266fbf8f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 --2- 172.21.15.39:0/1636818441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d266bde1b0 0x55d266fbf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5d9f400640 0 client.28108 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 -- 172.21.15.39:0/1636818441 >> 172.21.15.39:0/1636818441 conn(0x55d266fb7b80 msgr2=0x55d266faf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 -- 172.21.15.39:0/1636818441 shutdown_connections 2024-07-04T08:27:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 1 -- 172.21.15.39:0/1636818441 wait complete. 2024-07-04T08:27:05.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.218+0000 7f5dca382740 20 client.28108 trim_cache size 0 max 16384 2024-07-04T08:27:05.212 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:27:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:27:05.212 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:27:05.213 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:27:05.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 -- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5da40b29e0 msgr2=0x7f5da40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:05.213 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5da40b29e0 0x7f5da40b4ef0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009370 tx=0x7f5db4004a40 comp rx=0 tx=0).stop 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 -- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dbc075320 msgr2=0x7f5dbc19fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dbc075320 0x7f5dbc19fe30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e9c0 tx=0x7f5db000ee80 comp rx=0 tx=0).stop 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 -- 172.21.15.39:0/3680696319 shutdown_connections 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5da40b29e0 0x7f5da40b4ef0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5dbc1137b0 0x7f5dbc1a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dbc0768d0 0x7f5dbc1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 --2- 172.21.15.39:0/3680696319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dbc075320 0x7f5dbc19fe30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.219+0000 7f5dc2800640 1 -- 172.21.15.39:0/3680696319 >> 172.21.15.39:0/3680696319 conn(0x7f5dbc0fe930 msgr2=0x7f5dbc10f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.220+0000 7f5dc2800640 1 -- 172.21.15.39:0/3680696319 shutdown_connections 2024-07-04T08:27:05.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:05.220+0000 7f5dc2800640 1 -- 172.21.15.39:0/3680696319 wait complete. 2024-07-04T08:27:05.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:05 smithi133 ceph-mon[37050]: pgmap v846: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-07-04T08:27:05.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:05 smithi039 ceph-mon[32062]: pgmap v846: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-07-04T08:27:05.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:05 smithi050 ceph-mon[36577]: pgmap v846: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-07-04T08:27:06.203 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:27:06.203 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:27:06.701 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:07.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:07 smithi133 ceph-mon[37050]: pgmap v847: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 28 op/s 2024-07-04T08:27:07.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:07 smithi039 ceph-mon[32062]: pgmap v847: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 28 op/s 2024-07-04T08:27:07.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:07 smithi050 ceph-mon[36577]: pgmap v847: 129 pgs: 129 active+clean; 17 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 28 op/s 2024-07-04T08:27:08.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:08 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:27:08.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:08 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:27:08.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:08 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:27:08.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.947+0000 7f09c5c00640 1 -- 172.21.15.39:0/1846275223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c01030b0 msgr2=0x7f09c01034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:08.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.947+0000 7f09c5c00640 1 --2- 172.21.15.39:0/1846275223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c01030b0 0x7f09c01034c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009900 tx=0x7f09b002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:27:08.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.947+0000 7f09c5c00640 1 -- 172.21.15.39:0/1846275223 shutdown_connections 2024-07-04T08:27:08.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.947+0000 7f09c5c00640 1 --2- 172.21.15.39:0/1846275223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c0109340 0x7f09c010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:08.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.947+0000 7f09c5c00640 1 --2- 172.21.15.39:0/1846275223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09c01089f0 0x7f09c0108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:08.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.947+0000 7f09c5c00640 1 --2- 172.21.15.39:0/1846275223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c01030b0 0x7f09c01034c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:08.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.947+0000 7f09c5c00640 1 -- 172.21.15.39:0/1846275223 >> 172.21.15.39:0/1846275223 conn(0x7f09c007ada0 msgr2=0x7f09c007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:08.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.947+0000 7f09c5c00640 1 -- 172.21.15.39:0/1846275223 shutdown_connections 2024-07-04T08:27:08.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.947+0000 7f09c5c00640 1 -- 172.21.15.39:0/1846275223 wait complete. 2024-07-04T08:27:08.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.948+0000 7f09c5c00640 1 Processor -- start 2024-07-04T08:27:08.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09c5c00640 1 -- start start 2024-07-04T08:27:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09c5c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c01030b0 0x7f09c0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09c5c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09c01089f0 0x7f09c0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09c5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0109340 0x7f09c019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09c5c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0113060 con 0x7f09c01030b0 2024-07-04T08:27:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09c5c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0112ee0 con 0x7f09c01089f0 2024-07-04T08:27:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09c5c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09c01131e0 con 0x7f09c0109340 2024-07-04T08:27:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09bea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c01030b0 0x7f09c0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09bea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c01030b0 0x7f09c0199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58782/0 (socket says 172.21.15.39:58782) 2024-07-04T08:27:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.949+0000 7f09bea00640 1 -- 172.21.15.39:0/862304867 learned_addr learned my addr 172.21.15.39:0/862304867 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.950+0000 7f09bf400640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0109340 0x7f09c019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.950+0000 7f09be000640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09c01089f0 0x7f09c0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.950+0000 7f09bea00640 1 -- 172.21.15.39:0/862304867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0109340 msgr2=0x7f09c019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.950+0000 7f09bea00640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0109340 0x7f09c019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.950+0000 7f09bea00640 1 -- 172.21.15.39:0/862304867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09c01089f0 msgr2=0x7f09c0199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.950+0000 7f09bea00640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09c01089f0 0x7f09c0199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.950+0000 7f09bea00640 1 -- 172.21.15.39:0/862304867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c019f9a0 con 0x7f09c01030b0 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.950+0000 7f09bea00640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c01030b0 0x7f09c0199a30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f09b0005ab0 tx=0x7f09b0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.951+0000 7f09b7400640 1 -- 172.21.15.39:0/862304867 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b003f070 con 0x7f09c01030b0 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.951+0000 7f09b7400640 1 -- 172.21.15.39:0/862304867 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b0031d60 con 0x7f09c01030b0 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.951+0000 7f09b7400640 1 -- 172.21.15.39:0/862304867 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0035030 con 0x7f09c01030b0 2024-07-04T08:27:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.951+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c019fc30 con 0x7f09c01030b0 2024-07-04T08:27:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.951+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c01a63f0 con 0x7f09c01030b0 2024-07-04T08:27:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.952+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c01034c0 con 0x7f09c01030b0 2024-07-04T08:27:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.955+0000 7f09b7400640 1 -- 172.21.15.39:0/862304867 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f09b0042070 con 0x7f09c01030b0 2024-07-04T08:27:08.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.957+0000 7f09b7400640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09a00b29e0 0x7f09a00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:08.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.957+0000 7f09b7400640 1 -- 172.21.15.39:0/862304867 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9512+0+0 (secure 0 0 0) 0x7f09b0103780 con 0x7f09c01030b0 2024-07-04T08:27:08.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.957+0000 7f09be000640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09a00b29e0 0x7f09a00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:08.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.958+0000 7f09be000640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09a00b29e0 0x7f09a00b4ef0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09c0105c00 tx=0x7f09a8009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:08.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:08.960+0000 7f09b7400640 1 -- 172.21.15.39:0/862304867 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f09b00ca6d0 con 0x7f09c01030b0 2024-07-04T08:27:09.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.215+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09c002de10 con 0x7f09c01030b0 2024-07-04T08:27:09.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.217+0000 7f09b7400640 1 -- 172.21.15.39:0/862304867 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f09b00ce9d0 con 0x7f09c01030b0 2024-07-04T08:27:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:27:09.208 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:27:09.208 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:27:09.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09a00b29e0 msgr2=0x7f09a00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:09.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09a00b29e0 0x7f09a00b4ef0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09c0105c00 tx=0x7f09a8009500 comp rx=0 tx=0).stop 2024-07-04T08:27:09.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c01030b0 msgr2=0x7f09c0199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:09.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c01030b0 0x7f09c0199a30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f09b0005ab0 tx=0x7f09b0003c80 comp rx=0 tx=0).stop 2024-07-04T08:27:09.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 shutdown_connections 2024-07-04T08:27:09.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f09a00b29e0 0x7f09a00b4ef0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:09.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0109340 0x7f09c019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:09.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f09c01089f0 0x7f09c0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:09.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 --2- 172.21.15.39:0/862304867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c01030b0 0x7f09c0199a30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:09.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 >> 172.21.15.39:0/862304867 conn(0x7f09c007ada0 msgr2=0x7f09c0106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:09.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 shutdown_connections 2024-07-04T08:27:09.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:09.220+0000 7f09c5c00640 1 -- 172.21.15.39:0/862304867 wait complete. 2024-07-04T08:27:09.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:09 smithi133 ceph-mon[37050]: pgmap v848: 129 pgs: 129 active+clean; 18 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 KiB/s wr, 29 op/s 2024-07-04T08:27:09.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:09 smithi133 ceph-mon[37050]: osdmap e103: 12 total, 12 up, 12 in 2024-07-04T08:27:09.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:27:09.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:27:09.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:27:09.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:09 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/862304867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:27:09.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:09 smithi039 ceph-mon[32062]: pgmap v848: 129 pgs: 129 active+clean; 18 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 KiB/s wr, 29 op/s 2024-07-04T08:27:09.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:09 smithi039 ceph-mon[32062]: osdmap e103: 12 total, 12 up, 12 in 2024-07-04T08:27:09.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:27:09.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:27:09.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:27:09.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:09 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/862304867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:27:09.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:09 smithi050 ceph-mon[36577]: pgmap v848: 129 pgs: 129 active+clean; 18 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 KiB/s wr, 29 op/s 2024-07-04T08:27:09.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:09 smithi050 ceph-mon[36577]: osdmap e103: 12 total, 12 up, 12 in 2024-07-04T08:27:09.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:27:09.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:27:09.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:27:09.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:09 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/862304867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:27:10.105 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:27:10.605 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:11.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:11 smithi133 ceph-mon[37050]: pgmap v850: 129 pgs: 129 active+clean; 18 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-07-04T08:27:11.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:11 smithi039 ceph-mon[32062]: pgmap v850: 129 pgs: 129 active+clean; 18 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-07-04T08:27:11.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:11 smithi050 ceph-mon[36577]: pgmap v850: 129 pgs: 129 active+clean; 18 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-07-04T08:27:13.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.056+0000 7fe762000640 1 -- 172.21.15.39:0/108654474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 msgr2=0x7fe75c104350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.056+0000 7fe762000640 1 --2- 172.21.15.39:0/108654474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 0x7fe75c104350 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009860 tx=0x7fe74c02a340 comp rx=0 tx=0).stop 2024-07-04T08:27:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.056+0000 7fe762000640 1 -- 172.21.15.39:0/108654474 shutdown_connections 2024-07-04T08:27:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.056+0000 7fe762000640 1 --2- 172.21.15.39:0/108654474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c10d3a0 0x7fe75c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.056+0000 7fe762000640 1 --2- 172.21.15.39:0/108654474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c104890 0x7fe75c10cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.056+0000 7fe762000640 1 --2- 172.21.15.39:0/108654474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 0x7fe75c104350 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.056+0000 7fe762000640 1 -- 172.21.15.39:0/108654474 >> 172.21.15.39:0/108654474 conn(0x7fe75c078cb0 msgr2=0x7fe75c0790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.057+0000 7fe762000640 1 -- 172.21.15.39:0/108654474 shutdown_connections 2024-07-04T08:27:13.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.057+0000 7fe762000640 1 -- 172.21.15.39:0/108654474 wait complete. 2024-07-04T08:27:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.058+0000 7fe762000640 1 Processor -- start 2024-07-04T08:27:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.058+0000 7fe762000640 1 -- start start 2024-07-04T08:27:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe762000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 0x7fe75c19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe762000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c104890 0x7fe75c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe762000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c10d3a0 0x7fe75c1a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.049 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe762000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c113010 con 0x7fe75c104890 2024-07-04T08:27:13.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe762000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c112e90 con 0x7fe75c103f40 2024-07-04T08:27:13.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe762000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c113190 con 0x7fe75c10d3a0 2024-07-04T08:27:13.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c104890 0x7fe75c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe75b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 0x7fe75c19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe75b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 0x7fe75c19bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33302/0 (socket says 172.21.15.39:33302) 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe75b400640 1 -- 172.21.15.39:0/73559822 learned_addr learned my addr 172.21.15.39:0/73559822 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.059+0000 7fe75be00640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c10d3a0 0x7fe75c1a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.060+0000 7fe75aa00640 1 -- 172.21.15.39:0/73559822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c10d3a0 msgr2=0x7fe75c1a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.060+0000 7fe75aa00640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c10d3a0 0x7fe75c1a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.060+0000 7fe75aa00640 1 -- 172.21.15.39:0/73559822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 msgr2=0x7fe75c19bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.060+0000 7fe75aa00640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 0x7fe75c19bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.060+0000 7fe75aa00640 1 -- 172.21.15.39:0/73559822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c1a1bb0 con 0x7fe75c104890 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.060+0000 7fe75b400640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 0x7fe75c19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:13.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.060+0000 7fe75be00640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c10d3a0 0x7fe75c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:13.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.060+0000 7fe75aa00640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c104890 0x7fe75c19c1e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe75c06acf0 tx=0x7fe74800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:13.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.060+0000 7fe753e00640 1 -- 172.21.15.39:0/73559822 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74800cc40 con 0x7fe75c104890 2024-07-04T08:27:13.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.061+0000 7fe753e00640 1 -- 172.21.15.39:0/73559822 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74800cde0 con 0x7fe75c104890 2024-07-04T08:27:13.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.061+0000 7fe753e00640 1 -- 172.21.15.39:0/73559822 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748018780 con 0x7fe75c104890 2024-07-04T08:27:13.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.061+0000 7fe762000640 1 -- 172.21.15.39:0/73559822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c1a1ea0 con 0x7fe75c104890 2024-07-04T08:27:13.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.061+0000 7fe762000640 1 -- 172.21.15.39:0/73559822 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c1a8600 con 0x7fe75c104890 2024-07-04T08:27:13.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.064+0000 7fe7686a2740 1 Processor -- start 2024-07-04T08:27:13.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.064+0000 7fe7686a2740 1 -- start start 2024-07-04T08:27:13.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe753e00640 1 -- 172.21.15.39:0/73559822 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe748010a90 con 0x7fe75c104890 2024-07-04T08:27:13.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f96c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f96ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 0x55a12f96d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe7686a2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a12f8e2ee0 con 0x55a12f5a7390 2024-07-04T08:27:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe7686a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a12f8d2480 con 0x55a12f5c8070 2024-07-04T08:27:13.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe7686a2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a12f8d64c0 con 0x55a12f59f1b0 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 0x55a12f96d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe75be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f96c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe75be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f96c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33310/0 (socket says 172.21.15.39:33310) 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 0x55a12f96d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38884/0 (socket says 172.21.15.39:38884) 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe75be00640 1 -- 172.21.15.39:0/766584588 learned_addr learned my addr 172.21.15.39:0/766584588 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.065+0000 7fe75b400640 1 --2- 172.21.15.39:0/766584588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f96ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.066+0000 7fe75be00640 1 -- 172.21.15.39:0/766584588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 msgr2=0x55a12f96ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.066+0000 7fe75be00640 1 --2- 172.21.15.39:0/766584588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f96ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.066+0000 7fe75be00640 1 -- 172.21.15.39:0/766584588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 msgr2=0x55a12f96d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.066+0000 7fe75be00640 1 --2- 172.21.15.39:0/766584588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 0x55a12f96d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.066+0000 7fe75be00640 1 -- 172.21.15.39:0/766584588 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a12f86a650 con 0x55a12f5c8070 2024-07-04T08:27:13.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.066+0000 7fe75be00640 1 --2- 172.21.15.39:0/766584588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f96c7b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe754009fd0 tx=0x7fe75400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2819f528dff58549 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.067+0000 7fe750c00640 1 -- 172.21.15.39:0/766584588 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75401d070 con 0x55a12f5c8070 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.067+0000 7fe750c00640 1 -- 172.21.15.39:0/766584588 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe754003e20 con 0x55a12f5c8070 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.067+0000 7fe750c00640 1 -- 172.21.15.39:0/766584588 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540047f0 con 0x55a12f5c8070 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.067+0000 7fe753e00640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7400b2960 0x7fe7400b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.067+0000 7fe753e00640 1 -- 172.21.15.39:0/73559822 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9512+0+0 (secure 0 0 0) 0x7fe7480e2d30 con 0x7fe75c104890 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.067+0000 7fe75b400640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7400b2960 0x7fe7400b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe7686a2740 1 -- 172.21.15.39:0/766584588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 msgr2=0x55a12f96c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe7686a2740 1 --2- 172.21.15.39:0/766584588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f96c7b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe754009fd0 tx=0x7fe75400ec80 comp rx=0 tx=0).stop 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe7686a2740 1 -- 172.21.15.39:0/766584588 shutdown_connections 2024-07-04T08:27:13.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe7686a2740 1 --2- 172.21.15.39:0/766584588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 0x55a12f96d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe7686a2740 1 --2- 172.21.15.39:0/766584588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f96ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe7686a2740 1 --2- 172.21.15.39:0/766584588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f96c7b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe7686a2740 1 -- 172.21.15.39:0/766584588 >> 172.21.15.39:0/766584588 conn(0x55a12f5bc460 msgr2=0x55a12f96a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe7686a2740 1 -- 172.21.15.39:0/766584588 shutdown_connections 2024-07-04T08:27:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe7686a2740 1 -- 172.21.15.39:0/766584588 wait complete. 2024-07-04T08:27:13.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.068+0000 7fe75b400640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7400b2960 0x7fe7400b4e70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0057c0 tx=0x7fe74c02c040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.069+0000 7fe7686a2740 1 -- start start 2024-07-04T08:27:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.069+0000 7fe7686a2740 1 Processor -- start 2024-07-04T08:27:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f59f1b0 0x55a12f978340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f5a7390 0x55a12f978880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f9842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe7686a2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a12f736700 con 0x55a12f5c8070 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe7686a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a12f879bb0 con 0x55a12f59f1b0 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe7686a2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a12f97af90 con 0x55a12f5a7390 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe7686a2740 10 client.0 _ll_register_callbacks cb 0x55a12f8cb570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe75be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f9842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f5a7390 0x55a12f978880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe75be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f9842b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38892/0 (socket says 172.21.15.39:38892) 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe75b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f59f1b0 0x55a12f978340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.070+0000 7fe75be00640 1 -- 172.21.15.39:0/2826624384 learned_addr learned my addr 172.21.15.39:0/2826624384 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe75be00640 1 -- 172.21.15.39:0/2826624384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f5a7390 msgr2=0x55a12f978880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe75be00640 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f5a7390 0x55a12f978880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe75be00640 1 -- 172.21.15.39:0/2826624384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f59f1b0 msgr2=0x55a12f978340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe75be00640 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f59f1b0 0x55a12f978340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe75be00640 1 -- 172.21.15.39:0/2826624384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a12f633650 con 0x55a12f5c8070 2024-07-04T08:27:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe75be00640 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f9842b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe754005c40 tx=0x7fe754003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f206ea22ffd36183 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe745600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:27:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe745600640 1 -- 172.21.15.39:0/2826624384 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754026030 con 0x55a12f5c8070 2024-07-04T08:27:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe745600640 1 -- 172.21.15.39:0/2826624384 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9512+0+0 (secure 0 0 0) 0x7fe754004530 con 0x55a12f5c8070 2024-07-04T08:27:13.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.071+0000 7fe7686a2740 1 -- 172.21.15.39:0/2826624384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe754003e20 con 0x55a12f5c8070 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.072+0000 7fe745600640 1 -- 172.21.15.39:0/2826624384 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe754030050 con 0x55a12f5c8070 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.072+0000 7fe7686a2740 10 client.18060 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.072+0000 7fe7686a2740 1 -- 172.21.15.39:0/2826624384 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a12f986fa0 con 0x55a12f5c8070 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.072+0000 7fe745600640 1 -- 172.21.15.39:0/2826624384 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe754003e20 con 0x55a12f5c8070 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.072+0000 7fe745600640 1 -- 172.21.15.39:0/2826624384 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe75400b040 con 0x55a12f5c8070 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.073+0000 7fe7686a2740 10 client.18060 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.073+0000 7fe7686a2740 10 client.18060 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.073+0000 7fe7686a2740 20 client.18060 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.073+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a12f5869f0 0x55a12f98a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.073+0000 7fe7686a2740 4 client.18060 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.073+0000 7fe7686a2740 1 -- 172.21.15.39:0/2826624384 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a12f8e4e30 con 0x55a12f5869f0 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.073+0000 7fe75b400640 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a12f5869f0 0x55a12f98a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.074+0000 7fe75b400640 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a12f5869f0 0x55a12f98a3b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=389136df290ef86e server_cookie=963bbbfdfe79b711 in_seq=0 out_seq=0 2024-07-04T08:27:13.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.074+0000 7fe745600640 10 client.18060 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:27:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe745600640 1 -- 172.21.15.39:0/2826624384 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a12f8e4e30 con 0x55a12f5869f0 2024-07-04T08:27:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe745600640 10 client.18060 handle_command_reply: tid=0 2024-07-04T08:27:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe7686a2740 1 client.18060 shutdown 2024-07-04T08:27:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe7686a2740 1 -- 172.21.15.39:0/2826624384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 msgr2=0x55a12f9842b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f9842b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe754005c40 tx=0x7fe754003c20 comp rx=0 tx=0).stop 2024-07-04T08:27:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe7686a2740 1 -- 172.21.15.39:0/2826624384 shutdown_connections 2024-07-04T08:27:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a12f5869f0 0x55a12f98a3b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f9842b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f5a7390 0x55a12f978880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2826624384 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f59f1b0 0x55a12f978340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe7686a2740 1 -- 172.21.15.39:0/2826624384 >> 172.21.15.39:0/2826624384 conn(0x55a12f5bc460 msgr2=0x55a12f96f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.075+0000 7fe745600640 0 client.18060 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:27:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.076+0000 7fe7686a2740 1 -- 172.21.15.39:0/2826624384 shutdown_connections 2024-07-04T08:27:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.076+0000 7fe7686a2740 1 -- 172.21.15.39:0/2826624384 wait complete. 2024-07-04T08:27:13.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.076+0000 7fe7686a2740 20 client.18060 trim_cache size 0 max 16384 2024-07-04T08:27:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.091+0000 7fe7686a2740 1 Processor -- start 2024-07-04T08:27:13.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.091+0000 7fe7686a2740 1 -- start start 2024-07-04T08:27:13.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f978a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 0x55a12f9840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f9845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe7686a2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b2710 con 0x55a12f5a7390 2024-07-04T08:27:13.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe7686a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b4250 con 0x55a12f5c8070 2024-07-04T08:27:13.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe7686a2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b2de0 con 0x55a12f59f1b0 2024-07-04T08:27:13.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe75b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f978a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe75b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f978a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:54332/0 (socket says 172.21.15.39:54332) 2024-07-04T08:27:13.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe75b400640 1 -- 172.21.15.39:0/2378166497 learned_addr learned my addr 172.21.15.39:0/2378166497 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:13.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe75aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 0x55a12f9840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.092+0000 7fe75be00640 1 --2- 172.21.15.39:0/2378166497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f9845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.093+0000 7fe75b400640 1 -- 172.21.15.39:0/2378166497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 msgr2=0x55a12f9845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:13.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.093+0000 7fe75b400640 1 --2- 172.21.15.39:0/2378166497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f9845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.093+0000 7fe75b400640 1 -- 172.21.15.39:0/2378166497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 msgr2=0x55a12f9840a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:13.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.093+0000 7fe75b400640 1 --2- 172.21.15.39:0/2378166497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 0x55a12f9840a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.093+0000 7fe75b400640 1 -- 172.21.15.39:0/2378166497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a12f96f640 con 0x55a12f59f1b0 2024-07-04T08:27:13.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.093+0000 7fe75b400640 1 --2- 172.21.15.39:0/2378166497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f978a50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0303f0 tx=0x7fe74c03e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dd3d7e1c6b5d00c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:13.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.093+0000 7fe747e00640 1 -- 172.21.15.39:0/2378166497 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c037040 con 0x55a12f59f1b0 2024-07-04T08:27:13.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.093+0000 7fe747e00640 1 -- 172.21.15.39:0/2378166497 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74c050070 con 0x55a12f59f1b0 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.093+0000 7fe747e00640 1 -- 172.21.15.39:0/2378166497 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c003c20 con 0x55a12f59f1b0 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.094+0000 7fe7686a2740 1 -- 172.21.15.39:0/2378166497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 msgr2=0x55a12f978a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.094+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2378166497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f978a50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0303f0 tx=0x7fe74c03e000 comp rx=0 tx=0).stop 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.094+0000 7fe7686a2740 1 -- 172.21.15.39:0/2378166497 shutdown_connections 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.094+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2378166497 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5a7390 0x55a12f9840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.094+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2378166497 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5c8070 0x55a12f9845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.094+0000 7fe7686a2740 1 --2- 172.21.15.39:0/2378166497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f978a50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.094+0000 7fe7686a2740 1 -- 172.21.15.39:0/2378166497 >> 172.21.15.39:0/2378166497 conn(0x55a12f5bc460 msgr2=0x55a12f978340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.094+0000 7fe7686a2740 1 -- 172.21.15.39:0/2378166497 shutdown_connections 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.094+0000 7fe7686a2740 1 -- 172.21.15.39:0/2378166497 wait complete. 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.095+0000 7fe7686a2740 1 -- start start 2024-07-04T08:27:13.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.095+0000 7fe7686a2740 1 Processor -- start 2024-07-04T08:27:13.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f9807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5a7390 0x55a12f97dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe7686a2740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f97e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe7686a2740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b3b80 con 0x55a12f5c8070 2024-07-04T08:27:13.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe7686a2740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b34b0 con 0x55a12f5a7390 2024-07-04T08:27:13.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe7686a2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c1b0710 con 0x55a12f59f1b0 2024-07-04T08:27:13.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe7686a2740 10 client.0 _ll_register_callbacks cb 0x7fe75401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe75be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f97e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe75be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f97e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38910/0 (socket says 172.21.15.39:38910) 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe75be00640 1 -- 172.21.15.39:0/3528455864 learned_addr learned my addr 172.21.15.39:0/3528455864 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe75b400640 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f9807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.096+0000 7fe75aa00640 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5a7390 0x55a12f97dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.097+0000 7fe75b400640 1 -- 172.21.15.39:0/3528455864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5a7390 msgr2=0x55a12f97dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.097+0000 7fe75b400640 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5a7390 0x55a12f97dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.097+0000 7fe75b400640 1 -- 172.21.15.39:0/3528455864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 msgr2=0x55a12f97e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.097+0000 7fe75b400640 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f97e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.097+0000 7fe75b400640 1 -- 172.21.15.39:0/3528455864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a12f97e8a0 con 0x55a12f59f1b0 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.097+0000 7fe75be00640 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f97e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.097+0000 7fe75b400640 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f9807f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe74c02a930 tx=0x7fe74c030340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64339e2a60dce52a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:13.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.097+0000 7fe745600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:27:13.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.097+0000 7fe745600640 1 -- 172.21.15.39:0/3528455864 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c037070 con 0x55a12f59f1b0 2024-07-04T08:27:13.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe7686a2740 1 -- 172.21.15.39:0/3528455864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe74c050070 con 0x55a12f59f1b0 2024-07-04T08:27:13.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe745600640 1 -- 172.21.15.39:0/3528455864 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9512+0+0 (secure 0 0 0) 0x7fe74c003960 con 0x55a12f59f1b0 2024-07-04T08:27:13.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe745600640 1 -- 172.21.15.39:0/3528455864 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe74c06b050 con 0x55a12f59f1b0 2024-07-04T08:27:13.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe7686a2740 10 client.28132 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:13.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe7686a2740 1 -- 172.21.15.39:0/3528455864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a12f9f2800 con 0x55a12f59f1b0 2024-07-04T08:27:13.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe745600640 1 -- 172.21.15.39:0/3528455864 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74c050070 con 0x55a12f59f1b0 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe745600640 1 -- 172.21.15.39:0/3528455864 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe74c066030 con 0x55a12f59f1b0 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe7686a2740 10 client.28132 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe7686a2740 10 client.28132 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe7686a2740 20 client.28132 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe7686a2740 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a12f55bda0 0x55a12f9f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe7686a2740 4 client.28132 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.098+0000 7fe7686a2740 1 -- 172.21.15.39:0/3528455864 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a12f8e08d0 con 0x55a12f55bda0 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.099+0000 7fe75aa00640 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a12f55bda0 0x55a12f9f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.099+0000 7fe75aa00640 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a12f55bda0 0x55a12f9f4fa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98f2c32e700f9f5f server_cookie=218a408c70fef22b in_seq=0 out_seq=0 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.099+0000 7fe745600640 10 client.28132 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:27:13.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe745600640 1 -- 172.21.15.39:0/3528455864 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a12f8e08d0 con 0x55a12f55bda0 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe745600640 10 client.28132 handle_command_reply: tid=0 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 client.28132 shutdown 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 -- 172.21.15.39:0/3528455864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 msgr2=0x55a12f9807f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f9807f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe74c02a930 tx=0x7fe74c030340 comp rx=0 tx=0).stop 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 -- 172.21.15.39:0/3528455864 shutdown_connections 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55a12f55bda0 0x55a12f9f4fa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a12f5c8070 0x55a12f97e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a12f5a7390 0x55a12f97dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 --2- 172.21.15.39:0/3528455864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a12f59f1b0 0x55a12f9807f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe745600640 0 client.28132 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 -- 172.21.15.39:0/3528455864 >> 172.21.15.39:0/3528455864 conn(0x55a12f5bc460 msgr2=0x55a12f9795d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 -- 172.21.15.39:0/3528455864 shutdown_connections 2024-07-04T08:27:13.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 1 -- 172.21.15.39:0/3528455864 wait complete. 2024-07-04T08:27:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.100+0000 7fe7686a2740 20 client.28132 trim_cache size 0 max 16384 2024-07-04T08:27:13.094 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:27:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 -- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7400b2960 msgr2=0x7fe7400b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:13.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7400b2960 0x7fe7400b4e70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0057c0 tx=0x7fe74c02c040 comp rx=0 tx=0).stop 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 -- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c104890 msgr2=0x7fe75c19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c104890 0x7fe75c19c1e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe75c06acf0 tx=0x7fe74800ee50 comp rx=0 tx=0).stop 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 -- 172.21.15.39:0/73559822 shutdown_connections 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7400b2960 0x7fe7400b4e70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75c104890 0x7fe75c19c1e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe75c10d3a0 0x7fe75c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 --2- 172.21.15.39:0/73559822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe75c103f40 0x7fe75c19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 -- 172.21.15.39:0/73559822 >> 172.21.15.39:0/73559822 conn(0x7fe75c078cb0 msgr2=0x7fe75c10c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 -- 172.21.15.39:0/73559822 shutdown_connections 2024-07-04T08:27:13.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:13.101+0000 7fe762000640 1 -- 172.21.15.39:0/73559822 wait complete. 2024-07-04T08:27:13.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:13 smithi039 ceph-mon[32062]: pgmap v851: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 96 KiB/s wr, 33 op/s 2024-07-04T08:27:13.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:13 smithi133 ceph-mon[37050]: pgmap v851: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 96 KiB/s wr, 33 op/s 2024-07-04T08:27:13.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:13 smithi050 ceph-mon[36577]: pgmap v851: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 96 KiB/s wr, 33 op/s 2024-07-04T08:27:14.000 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:27:14.497 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:15.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:15 smithi133 ceph-mon[37050]: pgmap v852: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 96 KiB/s wr, 33 op/s 2024-07-04T08:27:15.568 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:15 smithi039 ceph-mon[32062]: pgmap v852: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 96 KiB/s wr, 33 op/s 2024-07-04T08:27:15.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:15 smithi050 ceph-mon[36577]: pgmap v852: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 96 KiB/s wr, 33 op/s 2024-07-04T08:27:17.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.110+0000 7fcc30e00640 1 -- 172.21.15.39:0/2983049014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc2c0768d0 msgr2=0x7fcc2c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:17.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.110+0000 7fcc30e00640 1 --2- 172.21.15.39:0/2983049014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc2c0768d0 0x7fcc2c074d00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009900 tx=0x7fcc1c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.111+0000 7fcc30e00640 1 -- 172.21.15.39:0/2983049014 shutdown_connections 2024-07-04T08:27:17.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.111+0000 7fcc30e00640 1 --2- 172.21.15.39:0/2983049014 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2c1137d0 0x7fcc2c115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.111+0000 7fcc30e00640 1 --2- 172.21.15.39:0/2983049014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc2c075320 0x7fcc2c075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.111+0000 7fcc30e00640 1 --2- 172.21.15.39:0/2983049014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc2c0768d0 0x7fcc2c074d00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.111+0000 7fcc30e00640 1 -- 172.21.15.39:0/2983049014 >> 172.21.15.39:0/2983049014 conn(0x7fcc2c0fe990 msgr2=0x7fcc2c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:17.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.111+0000 7fcc30e00640 1 -- 172.21.15.39:0/2983049014 shutdown_connections 2024-07-04T08:27:17.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.111+0000 7fcc30e00640 1 -- 172.21.15.39:0/2983049014 wait complete. 2024-07-04T08:27:17.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.112+0000 7fcc30e00640 1 Processor -- start 2024-07-04T08:27:17.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.112+0000 7fcc30e00640 1 -- start start 2024-07-04T08:27:17.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc30e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2c075320 0x7fcc2c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc30e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc2c0768d0 0x7fcc2c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc30e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc2c1137d0 0x7fcc2c1a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc30e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c119470 con 0x7fcc2c1137d0 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc30e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1192f0 con 0x7fcc2c0768d0 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc30e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1195f0 con 0x7fcc2c075320 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2c075320 0x7fcc2c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc2c1137d0 0x7fcc2c1a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2c075320 0x7fcc2c19fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:54362/0 (socket says 172.21.15.39:54362) 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc2b400640 1 -- 172.21.15.39:0/124143743 learned_addr learned my addr 172.21.15.39:0/124143743 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:17.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc2c0768d0 0x7fcc2c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.113+0000 7fcc2b400640 1 -- 172.21.15.39:0/124143743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc2c0768d0 msgr2=0x7fcc2c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:17.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.114+0000 7fcc2b400640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc2c0768d0 0x7fcc2c1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.114+0000 7fcc2b400640 1 -- 172.21.15.39:0/124143743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc2c1137d0 msgr2=0x7fcc2c1a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:17.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.114+0000 7fcc2b400640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc2c1137d0 0x7fcc2c1a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.114+0000 7fcc2b400640 1 -- 172.21.15.39:0/124143743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c1a5db0 con 0x7fcc2c075320 2024-07-04T08:27:17.105 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.114+0000 7fcc2b400640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2c075320 0x7fcc2c19fe40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c005ab0 tx=0x7fcc1c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:17.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.116+0000 7fcc23e00640 1 -- 172.21.15.39:0/124143743 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c03f070 con 0x7fcc2c075320 2024-07-04T08:27:17.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.116+0000 7fcc23e00640 1 -- 172.21.15.39:0/124143743 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c031b60 con 0x7fcc2c075320 2024-07-04T08:27:17.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.116+0000 7fcc23e00640 1 -- 172.21.15.39:0/124143743 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c035030 con 0x7fcc2c075320 2024-07-04T08:27:17.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.116+0000 7fcc30e00640 1 -- 172.21.15.39:0/124143743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c1a6040 con 0x7fcc2c075320 2024-07-04T08:27:17.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.116+0000 7fcc30e00640 1 -- 172.21.15.39:0/124143743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c1ac800 con 0x7fcc2c075320 2024-07-04T08:27:17.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.118+0000 7fcc38968740 1 Processor -- start 2024-07-04T08:27:17.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.119+0000 7fcc38968740 1 -- start start 2024-07-04T08:27:17.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.119+0000 7fcc38968740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e689070 0x55793ea2d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.119+0000 7fcc38968740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e6601b0 0x55793ea2deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.119+0000 7fcc38968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 0x55793ea2e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.119+0000 7fcc38968740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55793e9a3f00 con 0x55793e689070 2024-07-04T08:27:17.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.119+0000 7fcc38968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55793e9934a0 con 0x55793e668390 2024-07-04T08:27:17.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.119+0000 7fcc38968740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55793e9974e0 con 0x55793e6601b0 2024-07-04T08:27:17.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.119+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e689070 0x55793ea2d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.110 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.119+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e689070 0x55793ea2d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38926/0 (socket says 172.21.15.39:38926) 2024-07-04T08:27:17.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.120+0000 7fcc2be00640 1 -- 172.21.15.39:0/2434425397 learned_addr learned my addr 172.21.15.39:0/2434425397 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:17.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.120+0000 7fcc23e00640 1 -- 172.21.15.39:0/124143743 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcc1c042070 con 0x7fcc2c075320 2024-07-04T08:27:17.111 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.120+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/2434425397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 0x55793ea2e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.120+0000 7fcc2be00640 1 -- 172.21.15.39:0/2434425397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e6601b0 msgr2=0x55793ea2deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.120+0000 7fcc2be00640 1 --2- 172.21.15.39:0/2434425397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e6601b0 0x55793ea2deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.120+0000 7fcc2be00640 1 -- 172.21.15.39:0/2434425397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 msgr2=0x55793ea2e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.120+0000 7fcc2be00640 1 --2- 172.21.15.39:0/2434425397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 0x55793ea2e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.120+0000 7fcc2be00640 1 -- 172.21.15.39:0/2434425397 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793e92b670 con 0x55793e689070 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.121+0000 7fcc2be00640 1 --2- 172.21.15.39:0/2434425397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e689070 0x55793ea2d970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009fd0 tx=0x7fcc240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299689040da2df1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.121+0000 7fcc20c00640 1 -- 172.21.15.39:0/2434425397 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24004d20 con 0x55793e689070 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.121+0000 7fcc20c00640 1 -- 172.21.15.39:0/2434425397 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc24004050 con 0x55793e689070 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.121+0000 7fcc20c00640 1 -- 172.21.15.39:0/2434425397 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24005240 con 0x55793e689070 2024-07-04T08:27:17.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc23e00640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc100b29e0 0x7fcc100b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc38968740 1 -- 172.21.15.39:0/2434425397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e689070 msgr2=0x55793ea2d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc38968740 1 --2- 172.21.15.39:0/2434425397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e689070 0x55793ea2d970 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009fd0 tx=0x7fcc240118d0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc38968740 1 -- 172.21.15.39:0/2434425397 shutdown_connections 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc38968740 1 --2- 172.21.15.39:0/2434425397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 0x55793ea2e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc38968740 1 --2- 172.21.15.39:0/2434425397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e6601b0 0x55793ea2deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc38968740 1 --2- 172.21.15.39:0/2434425397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e689070 0x55793ea2d970 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc38968740 1 -- 172.21.15.39:0/2434425397 >> 172.21.15.39:0/2434425397 conn(0x55793e67d460 msgr2=0x55793ea2b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc23e00640 1 -- 172.21.15.39:0/124143743 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9512+0+0 (secure 0 0 0) 0x7fcc1c103030 con 0x7fcc2c075320 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc2b400640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc100b29e0 0x7fcc100b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc38968740 1 -- 172.21.15.39:0/2434425397 shutdown_connections 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.122+0000 7fcc38968740 1 -- 172.21.15.39:0/2434425397 wait complete. 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.123+0000 7fcc2b400640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc100b29e0 0x7fcc100b4ef0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55793ea2c5f0 tx=0x7fcc1c009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.123+0000 7fcc38968740 1 -- start start 2024-07-04T08:27:17.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.123+0000 7fcc38968740 1 Processor -- start 2024-07-04T08:27:17.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc38968740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea39550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc38968740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea39a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc38968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 0x55793ea45470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc38968740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55793e7f7700 con 0x55793e6601b0 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc38968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55793e93abd0 con 0x55793e689070 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc38968740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55793ea3c150 con 0x55793e668390 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc38968740 10 client.0 _ll_register_callbacks cb 0x55793e98c590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea39550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea39550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38942/0 (socket says 172.21.15.39:38942) 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc2be00640 1 -- 172.21.15.39:0/2562250167 learned_addr learned my addr 172.21.15.39:0/2562250167 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.124+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea39a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc2be00640 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 0x55793ea45470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc2be00640 1 -- 172.21.15.39:0/2562250167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 msgr2=0x55793ea39a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc2be00640 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea39a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc2be00640 1 -- 172.21.15.39:0/2562250167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 msgr2=0x55793ea45470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:17.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc2be00640 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 0x55793ea45470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc2be00640 1 -- 172.21.15.39:0/2562250167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793e6f4650 con 0x55793e6601b0 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea39a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc2be00640 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea39550 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcc24004910 tx=0x7fcc24003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd97060bb5e9be2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc19600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc19600640 1 -- 172.21.15.39:0/2562250167 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24004a40 con 0x55793e6601b0 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc19600640 1 -- 172.21.15.39:0/2562250167 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9512+0+0 (secure 0 0 0) 0x7fcc24004be0 con 0x55793e6601b0 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc38968740 1 -- 172.21.15.39:0/2562250167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc24004050 con 0x55793e6601b0 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc19600640 1 -- 172.21.15.39:0/2562250167 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc24004de0 con 0x55793e6601b0 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc38968740 10 client.18078 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:17.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc38968740 1 -- 172.21.15.39:0/2562250167 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55793ea48160 con 0x55793e6601b0 2024-07-04T08:27:17.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.126+0000 7fcc19600640 1 -- 172.21.15.39:0/2562250167 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc24004050 con 0x55793e6601b0 2024-07-04T08:27:17.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.127+0000 7fcc19600640 1 -- 172.21.15.39:0/2562250167 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcc2400f4e0 con 0x55793e6601b0 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.127+0000 7fcc38968740 10 client.18078 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.127+0000 7fcc38968740 10 client.18078 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.127+0000 7fcc38968740 20 client.18078 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.127+0000 7fcc38968740 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55793e6479f0 0x55793ea4b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.127+0000 7fcc38968740 4 client.18078 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.127+0000 7fcc38968740 1 -- 172.21.15.39:0/2562250167 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55793e9a5e50 con 0x55793e6479f0 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.127+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55793e6479f0 0x55793ea4b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.128+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55793e6479f0 0x55793ea4b570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53e22f22fd81e5c2 server_cookie=c189a890630fc0a2 in_seq=0 out_seq=0 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.128+0000 7fcc19600640 10 client.18078 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc19600640 1 -- 172.21.15.39:0/2562250167 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55793e9a5e50 con 0x55793e6479f0 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc19600640 10 client.18078 handle_command_reply: tid=0 2024-07-04T08:27:17.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 client.18078 shutdown 2024-07-04T08:27:17.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 -- 172.21.15.39:0/2562250167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 msgr2=0x55793ea39550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:17.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea39550 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcc24004910 tx=0x7fcc24003f50 comp rx=0 tx=0).stop 2024-07-04T08:27:17.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 -- 172.21.15.39:0/2562250167 shutdown_connections 2024-07-04T08:27:17.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55793e6479f0 0x55793ea4b570 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 0x55793ea45470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea39a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 --2- 172.21.15.39:0/2562250167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea39550 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc19600640 0 client.18078 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:27:17.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 -- 172.21.15.39:0/2562250167 >> 172.21.15.39:0/2562250167 conn(0x55793e67d460 msgr2=0x55793ea30b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:17.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 -- 172.21.15.39:0/2562250167 shutdown_connections 2024-07-04T08:27:17.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 1 -- 172.21.15.39:0/2562250167 wait complete. 2024-07-04T08:27:17.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.130+0000 7fcc38968740 20 client.18078 trim_cache size 0 max 16384 2024-07-04T08:27:17.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.145+0000 7fcc38968740 1 Processor -- start 2024-07-04T08:27:17.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.145+0000 7fcc38968740 1 -- start start 2024-07-04T08:27:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.146+0000 7fcc38968740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea2f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.146+0000 7fcc38968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 0x55793ea3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.146+0000 7fcc38968740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e689070 0x55793ea45260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.146+0000 7fcc38968740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b6850 con 0x55793e6601b0 2024-07-04T08:27:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.146+0000 7fcc38968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b8390 con 0x55793e668390 2024-07-04T08:27:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.146+0000 7fcc38968740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b6f20 con 0x55793e689070 2024-07-04T08:27:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.146+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea2f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.146+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea2f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38958/0 (socket says 172.21.15.39:38958) 2024-07-04T08:27:17.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.146+0000 7fcc2be00640 1 -- 172.21.15.39:0/3106723286 learned_addr learned my addr 172.21.15.39:0/3106723286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:17.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.147+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/3106723286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 0x55793ea3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.147+0000 7fcc2be00640 1 --2- 172.21.15.39:0/3106723286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e689070 0x55793ea45260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.147+0000 7fcc2be00640 1 -- 172.21.15.39:0/3106723286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e689070 msgr2=0x55793ea45260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:17.138 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.147+0000 7fcc2be00640 1 --2- 172.21.15.39:0/3106723286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e689070 0x55793ea45260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.147+0000 7fcc2be00640 1 -- 172.21.15.39:0/3106723286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 msgr2=0x55793ea3af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:17.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.147+0000 7fcc2be00640 1 --2- 172.21.15.39:0/3106723286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 0x55793ea3af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.147+0000 7fcc2be00640 1 -- 172.21.15.39:0/3106723286 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793ea300a0 con 0x55793e6601b0 2024-07-04T08:27:17.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.147+0000 7fcc2be00640 1 --2- 172.21.15.39:0/3106723286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea2f360 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005d70 tx=0x7fcc240021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c935ba2aa8b5fc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:17.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.148+0000 7fcc1be00640 1 -- 172.21.15.39:0/3106723286 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2400f4e0 con 0x55793e6601b0 2024-07-04T08:27:17.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.148+0000 7fcc1be00640 1 -- 172.21.15.39:0/3106723286 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc2402d070 con 0x55793e6601b0 2024-07-04T08:27:17.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.148+0000 7fcc1be00640 1 -- 172.21.15.39:0/3106723286 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24004ba0 con 0x55793e6601b0 2024-07-04T08:27:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.149+0000 7fcc38968740 1 -- 172.21.15.39:0/3106723286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 msgr2=0x55793ea2f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.149+0000 7fcc38968740 1 --2- 172.21.15.39:0/3106723286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea2f360 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005d70 tx=0x7fcc240021d0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.149+0000 7fcc38968740 1 -- 172.21.15.39:0/3106723286 shutdown_connections 2024-07-04T08:27:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.149+0000 7fcc38968740 1 --2- 172.21.15.39:0/3106723286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e689070 0x55793ea45260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.149+0000 7fcc38968740 1 --2- 172.21.15.39:0/3106723286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e668390 0x55793ea3af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.149+0000 7fcc38968740 1 --2- 172.21.15.39:0/3106723286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea2f360 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.149+0000 7fcc38968740 1 -- 172.21.15.39:0/3106723286 >> 172.21.15.39:0/3106723286 conn(0x55793e67d460 msgr2=0x55793ea3a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.149+0000 7fcc38968740 1 -- 172.21.15.39:0/3106723286 shutdown_connections 2024-07-04T08:27:17.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.149+0000 7fcc38968740 1 -- 172.21.15.39:0/3106723286 wait complete. 2024-07-04T08:27:17.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.150+0000 7fcc38968740 1 -- start start 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.150+0000 7fcc38968740 1 Processor -- start 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc38968740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea41790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc38968740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea3cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc38968740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 0x55793ea3d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc38968740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b7cc0 con 0x55793e6601b0 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc38968740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b75f0 con 0x55793e689070 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc38968740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b4850 con 0x55793e668390 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc38968740 10 client.0 _ll_register_callbacks cb 0x7fcc14009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea41790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea41790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38964/0 (socket says 172.21.15.39:38964) 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc2be00640 1 -- 172.21.15.39:0/3052816571 learned_addr learned my addr 172.21.15.39:0/3052816571 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:17.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea3cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.151+0000 7fcc2be00640 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 0x55793ea3d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc2aa00640 1 -- 172.21.15.39:0/3052816571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 msgr2=0x55793ea3d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 0x55793ea3d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc2aa00640 1 -- 172.21.15.39:0/3052816571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 msgr2=0x55793ea41790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea41790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc2aa00640 1 -- 172.21.15.39:0/3052816571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793ea3d6c0 con 0x55793e668390 2024-07-04T08:27:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc2aa00640 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea3cc40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55793ea4bb10 tx=0x7fcc14012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff27afbd106a9d24 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc19600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:27:17.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc19600640 1 -- 172.21.15.39:0/3052816571 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14004230 con 0x55793e668390 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc19600640 1 -- 172.21.15.39:0/3052816571 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9512+0+0 (secure 0 0 0) 0x7fcc140043d0 con 0x55793e668390 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.152+0000 7fcc38968740 1 -- 172.21.15.39:0/3052816571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2402d070 con 0x55793e668390 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.153+0000 7fcc19600640 1 -- 172.21.15.39:0/3052816571 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc1401f960 con 0x55793e668390 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.153+0000 7fcc38968740 10 client.28147 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.153+0000 7fcc38968740 1 -- 172.21.15.39:0/3052816571 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55793eab3820 con 0x55793e668390 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.153+0000 7fcc19600640 1 -- 172.21.15.39:0/3052816571 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc2402d070 con 0x55793e668390 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.153+0000 7fcc19600640 1 -- 172.21.15.39:0/3052816571 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcc14010430 con 0x55793e668390 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.153+0000 7fcc38968740 10 client.28147 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.153+0000 7fcc38968740 10 client.28147 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.154+0000 7fcc38968740 20 client.28147 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.154+0000 7fcc38968740 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55793e61cda0 0x55793eab5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.154+0000 7fcc38968740 4 client.28147 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:27:17.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.154+0000 7fcc38968740 1 -- 172.21.15.39:0/3052816571 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55793e9a18f0 con 0x55793e61cda0 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.154+0000 7fcc2be00640 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55793e61cda0 0x55793eab5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc2be00640 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55793e61cda0 0x55793eab5fe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8dffc16bebea8a6b server_cookie=2f0b6c6271c46f87 in_seq=0 out_seq=0 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc19600640 10 client.28147 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc19600640 1 -- 172.21.15.39:0/3052816571 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55793e9a18f0 con 0x55793e61cda0 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc19600640 10 client.28147 handle_command_reply: tid=0 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc38968740 1 client.28147 shutdown 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc38968740 1 -- 172.21.15.39:0/3052816571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 msgr2=0x55793ea3cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc38968740 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea3cc40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x55793ea4bb10 tx=0x7fcc14012eb0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc38968740 1 -- 172.21.15.39:0/3052816571 shutdown_connections 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc38968740 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55793e61cda0 0x55793eab5fe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc38968740 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55793e689070 0x55793ea3d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc38968740 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55793e668390 0x55793ea3cc40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc19600640 0 client.28147 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:27:17.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc38968740 1 --2- 172.21.15.39:0/3052816571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55793e6601b0 0x55793ea41790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.155+0000 7fcc38968740 1 -- 172.21.15.39:0/3052816571 >> 172.21.15.39:0/3052816571 conn(0x55793e67d460 msgr2=0x55793ea3b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:17.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.156+0000 7fcc38968740 1 -- 172.21.15.39:0/3052816571 shutdown_connections 2024-07-04T08:27:17.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.156+0000 7fcc38968740 1 -- 172.21.15.39:0/3052816571 wait complete. 2024-07-04T08:27:17.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.156+0000 7fcc38968740 20 client.28147 trim_cache size 0 max 16384 2024-07-04T08:27:17.148 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:27:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.156+0000 7fcc30e00640 1 -- 172.21.15.39:0/124143743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc100b29e0 msgr2=0x7fcc100b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.156+0000 7fcc30e00640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc100b29e0 0x7fcc100b4ef0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55793ea2c5f0 tx=0x7fcc1c009260 comp rx=0 tx=0).stop 2024-07-04T08:27:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.156+0000 7fcc30e00640 1 -- 172.21.15.39:0/124143743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2c075320 msgr2=0x7fcc2c19fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.156+0000 7fcc30e00640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2c075320 0x7fcc2c19fe40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c005ab0 tx=0x7fcc1c003c10 comp rx=0 tx=0).stop 2024-07-04T08:27:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.157+0000 7fcc30e00640 1 -- 172.21.15.39:0/124143743 shutdown_connections 2024-07-04T08:27:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.157+0000 7fcc30e00640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcc100b29e0 0x7fcc100b4ef0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.157+0000 7fcc30e00640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc2c1137d0 0x7fcc2c1a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.157+0000 7fcc30e00640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcc2c0768d0 0x7fcc2c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.157+0000 7fcc30e00640 1 --2- 172.21.15.39:0/124143743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2c075320 0x7fcc2c19fe40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.157+0000 7fcc30e00640 1 -- 172.21.15.39:0/124143743 >> 172.21.15.39:0/124143743 conn(0x7fcc2c0fe990 msgr2=0x7fcc2c10f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.157+0000 7fcc30e00640 1 -- 172.21.15.39:0/124143743 shutdown_connections 2024-07-04T08:27:17.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:17.157+0000 7fcc30e00640 1 -- 172.21.15.39:0/124143743 wait complete. 2024-07-04T08:27:17.527 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:17 smithi039 ceph-mon[32062]: pgmap v853: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-07-04T08:27:17.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:17 smithi133 ceph-mon[37050]: pgmap v853: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-07-04T08:27:17.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:17 smithi050 ceph-mon[36577]: pgmap v853: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-07-04T08:27:18.010 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:27:18.498 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:19.500 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:19 smithi039 ceph-mon[32062]: pgmap v854: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:19.500 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:19 smithi039 ceph-mon[32062]: osdmap e104: 12 total, 12 up, 12 in 2024-07-04T08:27:19.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:19 smithi133 ceph-mon[37050]: pgmap v854: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:19.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:19 smithi133 ceph-mon[37050]: osdmap e104: 12 total, 12 up, 12 in 2024-07-04T08:27:19.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:19 smithi050 ceph-mon[36577]: pgmap v854: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:19.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:19 smithi050 ceph-mon[36577]: osdmap e104: 12 total, 12 up, 12 in 2024-07-04T08:27:20.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.705+0000 7ff7dde00640 1 -- 172.21.15.39:0/1764213218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8104660 msgr2=0x7ff7d810cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:20.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.705+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1764213218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8104660 0x7ff7d810cbb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009900 tx=0x7ff7c402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.705+0000 7ff7dde00640 1 -- 172.21.15.39:0/1764213218 shutdown_connections 2024-07-04T08:27:20.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.705+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1764213218 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8104660 0x7ff7d810cbb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.705+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1764213218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d810d170 0x7ff7d810f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.705+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1764213218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d8103d10 0x7ff7d8104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.705+0000 7ff7dde00640 1 -- 172.21.15.39:0/1764213218 >> 172.21.15.39:0/1764213218 conn(0x7ff7d80fd810 msgr2=0x7ff7d80ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:20.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.705+0000 7ff7dde00640 1 -- 172.21.15.39:0/1764213218 shutdown_connections 2024-07-04T08:27:20.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.705+0000 7ff7dde00640 1 -- 172.21.15.39:0/1764213218 wait complete. 2024-07-04T08:27:20.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.707+0000 7ff7dde00640 1 Processor -- start 2024-07-04T08:27:20.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.707+0000 7ff7dde00640 1 -- start start 2024-07-04T08:27:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dde00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d8103d10 0x7ff7d8101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8104660 0x7ff7d8101f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d810d170 0x7ff7d81024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dde00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8112e20 con 0x7ff7d8104660 2024-07-04T08:27:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dde00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8112ca0 con 0x7ff7d810d170 2024-07-04T08:27:20.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dde00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8112fa0 con 0x7ff7d8103d10 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d8103d10 0x7ff7d8101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d8103d10 0x7ff7d8101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:54420/0 (socket says 172.21.15.39:54420) 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dca00640 1 -- 172.21.15.39:0/1017895365 learned_addr learned my addr 172.21.15.39:0/1017895365 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8104660 0x7ff7d8101f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.708+0000 7ff7dd400640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d810d170 0x7ff7d81024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7dca00640 1 -- 172.21.15.39:0/1017895365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d810d170 msgr2=0x7ff7d81024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7dca00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d810d170 0x7ff7d81024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7dca00640 1 -- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8104660 msgr2=0x7ff7d8101f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7dca00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8104660 0x7ff7d8101f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7dca00640 1 -- 172.21.15.39:0/1017895365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d81a7ad0 con 0x7ff7d8103d10 2024-07-04T08:27:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7dca00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d8103d10 0x7ff7d8101a50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800ea60 tx=0x7ff7c800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:20.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7d6000640 1 -- 172.21.15.39:0/1017895365 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c800cd20 con 0x7ff7d8103d10 2024-07-04T08:27:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7d6000640 1 -- 172.21.15.39:0/1017895365 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7c80035f0 con 0x7ff7d8103d10 2024-07-04T08:27:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7dde00640 1 -- 172.21.15.39:0/1017895365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d81a7dc0 con 0x7ff7d8103d10 2024-07-04T08:27:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7d6000640 1 -- 172.21.15.39:0/1017895365 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8010690 con 0x7ff7d8103d10 2024-07-04T08:27:20.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.709+0000 7ff7dde00640 1 -- 172.21.15.39:0/1017895365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d81a8270 con 0x7ff7d8103d10 2024-07-04T08:27:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.713+0000 7ff7e59cf740 1 Processor -- start 2024-07-04T08:27:20.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.713+0000 7ff7e59cf740 1 -- start start 2024-07-04T08:27:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 0x55b738f25eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b738e9bf00 con 0x55b738b81070 2024-07-04T08:27:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b738e8b4a0 con 0x55b738b581b0 2024-07-04T08:27:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b738e8f4e0 con 0x55b738b60390 2024-07-04T08:27:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7d6000640 1 -- 172.21.15.39:0/1017895365 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff7c80037a0 con 0x7ff7d8103d10 2024-07-04T08:27:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f25970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38982/0 (socket says 172.21.15.39:38982) 2024-07-04T08:27:20.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7dd400640 1 -- 172.21.15.39:0/1617756401 learned_addr learned my addr 172.21.15.39:0/1617756401 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:20.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/1617756401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.714+0000 7ff7dca00640 1 --2- 172.21.15.39:0/1617756401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 0x55b738f25eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.715+0000 7ff7dd400640 1 -- 172.21.15.39:0/1617756401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 msgr2=0x55b738f263f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.715+0000 7ff7dd400640 1 --2- 172.21.15.39:0/1617756401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f263f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.715+0000 7ff7dd400640 1 -- 172.21.15.39:0/1617756401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 msgr2=0x55b738f25eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.715+0000 7ff7dd400640 1 --2- 172.21.15.39:0/1617756401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 0x55b738f25eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.715+0000 7ff7dd400640 1 -- 172.21.15.39:0/1617756401 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b738e23670 con 0x55b738b81070 2024-07-04T08:27:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.715+0000 7ff7dd400640 1 --2- 172.21.15.39:0/1617756401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f25970 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0011440 tx=0x7ff7d0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac43b0186ed2cd1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.715+0000 7ff7ce000640 1 -- 172.21.15.39:0/1617756401 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0003c30 con 0x55b738b81070 2024-07-04T08:27:20.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.716+0000 7ff7ce000640 1 -- 172.21.15.39:0/1617756401 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d0003dd0 con 0x55b738b81070 2024-07-04T08:27:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.716+0000 7ff7ce000640 1 -- 172.21.15.39:0/1617756401 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000fa80 con 0x55b738b81070 2024-07-04T08:27:20.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.716+0000 7ff7d6000640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff7bc0b29e0 0x7ff7bc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.716+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1617756401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 msgr2=0x55b738f25970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:20.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.716+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1617756401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f25970 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0011440 tx=0x7ff7d0011900 comp rx=0 tx=0).stop 2024-07-04T08:27:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.716+0000 7ff7d6000640 1 -- 172.21.15.39:0/1017895365 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7ff7c80e2c80 con 0x7ff7d8103d10 2024-07-04T08:27:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.716+0000 7ff7dca00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff7bc0b29e0 0x7ff7bc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.717+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1617756401 shutdown_connections 2024-07-04T08:27:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.717+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1617756401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f263f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.717+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1617756401 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 0x55b738f25eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.717+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1617756401 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f25970 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.717+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1617756401 >> 172.21.15.39:0/1617756401 conn(0x55b738b75460 msgr2=0x55b738f23860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:20.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.717+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1617756401 shutdown_connections 2024-07-04T08:27:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.717+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1617756401 wait complete. 2024-07-04T08:27:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.717+0000 7ff7dca00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff7bc0b29e0 0x7ff7bc0b4ef0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55b738f245f0 tx=0x7ff7c80e6730 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.718+0000 7ff7e59cf740 1 -- start start 2024-07-04T08:27:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.718+0000 7ff7e59cf740 1 Processor -- start 2024-07-04T08:27:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.718+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 0x55b738f31620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.718+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f31be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.718+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f3d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.718+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b738cef700 con 0x55b738b81070 2024-07-04T08:27:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.718+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b738e32bd0 con 0x55b738b581b0 2024-07-04T08:27:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.718+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b738f34150 con 0x55b738b60390 2024-07-04T08:27:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7e59cf740 10 client.0 _ll_register_callbacks cb 0x55b738e84590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7cfe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f31be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f3d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7cfe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f31be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:54448/0 (socket says 172.21.15.39:54448) 2024-07-04T08:27:20.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7cfe00640 1 -- 172.21.15.39:0/117315219 learned_addr learned my addr 172.21.15.39:0/117315219 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:20.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7dd400640 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 0x55b738f31620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7cfe00640 1 -- 172.21.15.39:0/117315219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 msgr2=0x55b738f31620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 0x55b738f31620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7cfe00640 1 -- 172.21.15.39:0/117315219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 msgr2=0x55b738f3d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f3d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7cfe00640 1 -- 172.21.15.39:0/117315219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b738bec650 con 0x55b738b60390 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.719+0000 7ff7dd400640 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 0x55b738f31620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.720+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f31be0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4000c00 tx=0x7ff7c4005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6adae2d6655b0b5b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.720+0000 7ff7c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.720+0000 7ff7c2a00640 1 -- 172.21.15.39:0/117315219 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4035040 con 0x55b738b60390 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.720+0000 7ff7e59cf740 1 -- 172.21.15.39:0/117315219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d0003dd0 con 0x55b738b60390 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.720+0000 7ff7c2a00640 1 -- 172.21.15.39:0/117315219 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7ff7c40048c0 con 0x55b738b60390 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.721+0000 7ff7c2a00640 1 -- 172.21.15.39:0/117315219 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7c4004ac0 con 0x55b738b60390 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.721+0000 7ff7e59cf740 10 client.28159 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.721+0000 7ff7e59cf740 1 -- 172.21.15.39:0/117315219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b738f401f0 con 0x55b738b60390 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.721+0000 7ff7c2a00640 1 -- 172.21.15.39:0/117315219 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d0003dd0 con 0x55b738b60390 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.721+0000 7ff7c2a00640 1 -- 172.21.15.39:0/117315219 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff7c4003b30 con 0x55b738b60390 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.721+0000 7ff7e59cf740 10 client.28159 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:20.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.721+0000 7ff7e59cf740 10 client.28159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.721+0000 7ff7e59cf740 20 client.28159 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.722+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b738b3f9f0 0x55b738f43740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.722+0000 7ff7e59cf740 4 client.28159 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.722+0000 7ff7e59cf740 1 -- 172.21.15.39:0/117315219 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b738e9de50 con 0x55b738b3f9f0 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.722+0000 7ff7dd400640 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b738b3f9f0 0x55b738f43740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.722+0000 7ff7dd400640 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b738b3f9f0 0x55b738f43740 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d010b92cb39ab246 server_cookie=4789f66d46fc527c in_seq=0 out_seq=0 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.722+0000 7ff7c2a00640 10 client.28159 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.723+0000 7ff7c2a00640 1 -- 172.21.15.39:0/117315219 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b738e9de50 con 0x55b738b3f9f0 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.723+0000 7ff7c2a00640 10 client.28159 handle_command_reply: tid=0 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.723+0000 7ff7e59cf740 1 client.28159 shutdown 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 -- 172.21.15.39:0/117315219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 msgr2=0x55b738f31be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f31be0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4000c00 tx=0x7ff7c4005270 comp rx=0 tx=0).stop 2024-07-04T08:27:20.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 -- 172.21.15.39:0/117315219 shutdown_connections 2024-07-04T08:27:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b738b3f9f0 0x55b738f43740 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b81070 0x55b738f3d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f31be0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/117315219 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b581b0 0x55b738f31620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7c2a00640 0 client.28159 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:27:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 -- 172.21.15.39:0/117315219 >> 172.21.15.39:0/117315219 conn(0x55b738b75460 msgr2=0x55b738f28b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 -- 172.21.15.39:0/117315219 shutdown_connections 2024-07-04T08:27:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 1 -- 172.21.15.39:0/117315219 wait complete. 2024-07-04T08:27:20.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.724+0000 7ff7e59cf740 20 client.28159 trim_cache size 0 max 16384 2024-07-04T08:27:20.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.740+0000 7ff7e59cf740 1 Processor -- start 2024-07-04T08:27:20.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.740+0000 7ff7e59cf740 1 -- start start 2024-07-04T08:27:20.729 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.740+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f31c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.740+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f3d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.740+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b81070 0x55b738f2aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.740+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81b2420 con 0x55b738b581b0 2024-07-04T08:27:20.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.740+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81b3f60 con 0x55b738b81070 2024-07-04T08:27:20.730 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.740+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81b2af0 con 0x55b738b60390 2024-07-04T08:27:20.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f31c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f31c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38988/0 (socket says 172.21.15.39:38988) 2024-07-04T08:27:20.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 -- 172.21.15.39:0/2255776928 learned_addr learned my addr 172.21.15.39:0/2255776928 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:20.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f3d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.731 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b81070 0x55b738f2aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 -- 172.21.15.39:0/2255776928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 msgr2=0x55b738f3d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f3d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 -- 172.21.15.39:0/2255776928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b81070 msgr2=0x55b738f2aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b81070 0x55b738f2aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 -- 172.21.15.39:0/2255776928 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b738f24620 con 0x55b738b581b0 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f3d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.741+0000 7ff7dd400640 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f31c90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000f650 tx=0x7ff7d000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97ca293e53b863c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.742+0000 7ff7cd600640 1 -- 172.21.15.39:0/2255776928 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000fbc0 con 0x55b738b581b0 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.742+0000 7ff7cd600640 1 -- 172.21.15.39:0/2255776928 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d000fd60 con 0x55b738b581b0 2024-07-04T08:27:20.732 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.742+0000 7ff7cd600640 1 -- 172.21.15.39:0/2255776928 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0031440 con 0x55b738b581b0 2024-07-04T08:27:20.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.743+0000 7ff7e59cf740 1 -- 172.21.15.39:0/2255776928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 msgr2=0x55b738f31c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:20.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.743+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f31c90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000f650 tx=0x7ff7d000f5e0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.743+0000 7ff7e59cf740 1 -- 172.21.15.39:0/2255776928 shutdown_connections 2024-07-04T08:27:20.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.743+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b81070 0x55b738f2aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.743+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b60390 0x55b738f3d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.743+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/2255776928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f31c90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.743+0000 7ff7e59cf740 1 -- 172.21.15.39:0/2255776928 >> 172.21.15.39:0/2255776928 conn(0x55b738b75460 msgr2=0x55b738f2fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:20.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.743+0000 7ff7e59cf740 1 -- 172.21.15.39:0/2255776928 shutdown_connections 2024-07-04T08:27:20.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.743+0000 7ff7e59cf740 1 -- 172.21.15.39:0/2255776928 wait complete. 2024-07-04T08:27:20.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.744+0000 7ff7e59cf740 1 -- start start 2024-07-04T08:27:20.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.744+0000 7ff7e59cf740 1 Processor -- start 2024-07-04T08:27:20.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.745+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f34cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.745+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b60390 0x55b738f35290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.745+0000 7ff7e59cf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b81070 0x55b738fa3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.745+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81b3890 con 0x55b738b581b0 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.745+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81b31c0 con 0x55b738b60390 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.745+0000 7ff7e59cf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81b0400 con 0x55b738b81070 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.745+0000 7ff7e59cf740 10 client.0 _ll_register_callbacks cb 0x7ff7c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f34cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7cfe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b60390 0x55b738f35290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f34cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38998/0 (socket says 172.21.15.39:38998) 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7dd400640 1 -- 172.21.15.39:0/1987515548 learned_addr learned my addr 172.21.15.39:0/1987515548 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:20.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7dd400640 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b81070 0x55b738fa3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7cfe00640 1 -- 172.21.15.39:0/1987515548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b81070 msgr2=0x55b738fa3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b81070 0x55b738fa3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7cfe00640 1 -- 172.21.15.39:0/1987515548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 msgr2=0x55b738f34cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f34cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7cfe00640 1 -- 172.21.15.39:0/1987515548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b738fa4650 con 0x55b738b60390 2024-07-04T08:27:20.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.746+0000 7ff7dd400640 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f34cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.747+0000 7ff7cfe00640 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b60390 0x55b738f35290 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4003f80 tx=0x7ff7c4004e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a99e0da4e8ded6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.747+0000 7ff7c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.747+0000 7ff7c2a00640 1 -- 172.21.15.39:0/1987515548 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4058060 con 0x55b738b60390 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.747+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1987515548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d000fd60 con 0x55b738b60390 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.747+0000 7ff7c2a00640 1 -- 172.21.15.39:0/1987515548 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7ff7c4031c90 con 0x55b738b60390 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.747+0000 7ff7c2a00640 1 -- 172.21.15.39:0/1987515548 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7c4035040 con 0x55b738b60390 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.747+0000 7ff7e59cf740 10 client.28238 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.747+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1987515548 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b738fab990 con 0x55b738b60390 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.748+0000 7ff7c2a00640 1 -- 172.21.15.39:0/1987515548 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d000fd60 con 0x55b738b60390 2024-07-04T08:27:20.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.748+0000 7ff7c2a00640 1 -- 172.21.15.39:0/1987515548 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff7c4003aa0 con 0x55b738b60390 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.748+0000 7ff7e59cf740 10 client.28238 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.748+0000 7ff7e59cf740 10 client.28238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.748+0000 7ff7e59cf740 20 client.28238 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.748+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b738b14da0 0x55b738fae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.748+0000 7ff7e59cf740 4 client.28238 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.748+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1987515548 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b738e998f0 con 0x55b738b14da0 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.749+0000 7ff7dd400640 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b738b14da0 0x55b738fae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.749+0000 7ff7dd400640 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b738b14da0 0x55b738fae120 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1c09a1a308b4128 server_cookie=54eb98c11b46487e in_seq=0 out_seq=0 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.749+0000 7ff7c2a00640 10 client.28238 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.749+0000 7ff7c2a00640 1 -- 172.21.15.39:0/1987515548 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b738e998f0 con 0x55b738b14da0 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.749+0000 7ff7c2a00640 10 client.28238 handle_command_reply: tid=0 2024-07-04T08:27:20.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 client.28238 shutdown 2024-07-04T08:27:20.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1987515548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b60390 msgr2=0x55b738f35290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:20.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b60390 0x55b738f35290 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4003f80 tx=0x7ff7c4004e90 comp rx=0 tx=0).stop 2024-07-04T08:27:20.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1987515548 shutdown_connections 2024-07-04T08:27:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55b738b14da0 0x55b738fae120 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b738b81070 0x55b738fa3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b738b60390 0x55b738f35290 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 --2- 172.21.15.39:0/1987515548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b738b581b0 0x55b738f34cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7c2a00640 0 client.28238 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:27:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1987515548 >> 172.21.15.39:0/1987515548 conn(0x55b738b75460 msgr2=0x55b738f2aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1987515548 shutdown_connections 2024-07-04T08:27:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 1 -- 172.21.15.39:0/1987515548 wait complete. 2024-07-04T08:27:20.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.750+0000 7ff7e59cf740 20 client.28238 trim_cache size 0 max 16384 2024-07-04T08:27:20.743 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:27:20.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 -- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff7bc0b29e0 msgr2=0x7ff7bc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:20.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff7bc0b29e0 0x7ff7bc0b4ef0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55b738f245f0 tx=0x7ff7c80e6730 comp rx=0 tx=0).stop 2024-07-04T08:27:20.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 -- 172.21.15.39:0/1017895365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d8103d10 msgr2=0x7ff7d8101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:20.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d8103d10 0x7ff7d8101a50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800ea60 tx=0x7ff7c800ee60 comp rx=0 tx=0).stop 2024-07-04T08:27:20.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 -- 172.21.15.39:0/1017895365 shutdown_connections 2024-07-04T08:27:20.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff7bc0b29e0 0x7ff7bc0b4ef0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7d8104660 0x7ff7d8101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7d810d170 0x7ff7d81024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 --2- 172.21.15.39:0/1017895365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7d8103d10 0x7ff7d8101a50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:20.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.751+0000 7ff7dde00640 1 -- 172.21.15.39:0/1017895365 >> 172.21.15.39:0/1017895365 conn(0x7ff7d80fd810 msgr2=0x7ff7d8106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:20.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.752+0000 7ff7dde00640 1 -- 172.21.15.39:0/1017895365 shutdown_connections 2024-07-04T08:27:20.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:20.752+0000 7ff7dde00640 1 -- 172.21.15.39:0/1017895365 wait complete. 2024-07-04T08:27:21.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:21 smithi133 ceph-mon[37050]: pgmap v856: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:21.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:21 smithi039 ceph-mon[32062]: pgmap v856: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:21.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:21 smithi050 ceph-mon[36577]: pgmap v856: 129 pgs: 129 active+clean; 18 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:21.692 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:27:22.188 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:23.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:23 smithi133 ceph-mon[37050]: pgmap v857: 129 pgs: 129 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:23.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:23 smithi039 ceph-mon[32062]: pgmap v857: 129 pgs: 129 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:23.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:23 smithi050 ceph-mon[36577]: pgmap v857: 129 pgs: 129 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:24.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.606+0000 7fa121e00640 1 -- 172.21.15.39:0/18917820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c075650 msgr2=0x7fa11c10f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:24.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.606+0000 7fa121e00640 1 --2- 172.21.15.39:0/18917820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c075650 0x7fa11c10f0f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa104009ca0 tx=0x7fa10402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.606+0000 7fa121e00640 1 -- 172.21.15.39:0/18917820 shutdown_connections 2024-07-04T08:27:24.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.606+0000 7fa121e00640 1 --2- 172.21.15.39:0/18917820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c075650 0x7fa11c10f0f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.606+0000 7fa121e00640 1 --2- 172.21.15.39:0/18917820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa11c10f630 0x7fa11c111a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.606+0000 7fa121e00640 1 --2- 172.21.15.39:0/18917820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11c074d00 0x7fa11c075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.606+0000 7fa121e00640 1 -- 172.21.15.39:0/18917820 >> 172.21.15.39:0/18917820 conn(0x7fa11c0feae0 msgr2=0x7fa11c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:24.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.607+0000 7fa121e00640 1 -- 172.21.15.39:0/18917820 shutdown_connections 2024-07-04T08:27:24.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.607+0000 7fa121e00640 1 -- 172.21.15.39:0/18917820 wait complete. 2024-07-04T08:27:24.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.608+0000 7fa121e00640 1 Processor -- start 2024-07-04T08:27:24.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.608+0000 7fa121e00640 1 -- start start 2024-07-04T08:27:24.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa121e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c074d00 0x7fa11c19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa121e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11c075650 0x7fa11c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa121e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa11c10f630 0x7fa11c1a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa121e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1151d0 con 0x7fa11c074d00 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa121e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c115050 con 0x7fa11c10f630 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa121e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c115350 con 0x7fa11c075650 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa11a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11c075650 0x7fa11c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa11a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11c075650 0x7fa11c19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57464/0 (socket says 172.21.15.39:57464) 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa11a000640 1 -- 172.21.15.39:0/2024423632 learned_addr learned my addr 172.21.15.39:0/2024423632 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.609+0000 7fa11aa00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c074d00 0x7fa11c19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.610+0000 7fa11b400640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa11c10f630 0x7fa11c1a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.610+0000 7fa11a000640 1 -- 172.21.15.39:0/2024423632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa11c10f630 msgr2=0x7fa11c1a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.610+0000 7fa11a000640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa11c10f630 0x7fa11c1a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.610+0000 7fa11a000640 1 -- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c074d00 msgr2=0x7fa11c19bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:24.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.610+0000 7fa11a000640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c074d00 0x7fa11c19bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.610+0000 7fa11a000640 1 -- 172.21.15.39:0/2024423632 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c1a1bb0 con 0x7fa11c075650 2024-07-04T08:27:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.610+0000 7fa11b400640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa11c10f630 0x7fa11c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:27:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.610+0000 7fa11a000640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11c075650 0x7fa11c19c1e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa104033040 tx=0x7fa1040316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.611+0000 7fa120a00640 1 -- 172.21.15.39:0/2024423632 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104003b70 con 0x7fa11c075650 2024-07-04T08:27:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.611+0000 7fa120a00640 1 -- 172.21.15.39:0/2024423632 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa104003d10 con 0x7fa11c075650 2024-07-04T08:27:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.611+0000 7fa121e00640 1 -- 172.21.15.39:0/2024423632 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c1a1e40 con 0x7fa11c075650 2024-07-04T08:27:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.611+0000 7fa120a00640 1 -- 172.21.15.39:0/2024423632 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10402fa80 con 0x7fa11c075650 2024-07-04T08:27:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.611+0000 7fa121e00640 1 -- 172.21.15.39:0/2024423632 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11c1a8600 con 0x7fa11c075650 2024-07-04T08:27:24.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.614+0000 7fa128368740 1 Processor -- start 2024-07-04T08:27:24.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.614+0000 7fa128368740 1 -- start start 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa128368740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x5565763767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa128368740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x556576376d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa128368740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x556576377270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa128368740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5565762ecf00 con 0x556575fa91b0 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa128368740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5565762dc4a0 con 0x556575fd2070 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa128368740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5565762e04e0 con 0x556575fb1390 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x556576376d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa11b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x5565763767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x556576376d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42266/0 (socket says 172.21.15.39:42266) 2024-07-04T08:27:24.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa11b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x5565763767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53408/0 (socket says 172.21.15.39:53408) 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa11aa00640 1 -- 172.21.15.39:0/1472863844 learned_addr learned my addr 172.21.15.39:0/1472863844 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.615+0000 7fa120a00640 1 -- 172.21.15.39:0/2024423632 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa10402fc20 con 0x7fa11c075650 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa11a000640 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x556576377270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa11b400640 1 -- 172.21.15.39:0/1472863844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 msgr2=0x556576377270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa11b400640 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x556576377270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa11b400640 1 -- 172.21.15.39:0/1472863844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 msgr2=0x556576376d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa11b400640 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x556576376d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa11b400640 1 -- 172.21.15.39:0/1472863844 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556576274670 con 0x556575fd2070 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa11aa00640 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x556576376d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa11a000640 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x556576377270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa11b400640 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x5565763767f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa114009bb0 tx=0x7fa1140117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f80a3080e9cec56 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:24.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.616+0000 7fa110c00640 1 -- 172.21.15.39:0/1472863844 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114005020 con 0x556575fd2070 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.617+0000 7fa110c00640 1 -- 172.21.15.39:0/1472863844 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1140047a0 con 0x556575fd2070 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.617+0000 7fa110c00640 1 -- 172.21.15.39:0/1472863844 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114012650 con 0x556575fd2070 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.617+0000 7fa120a00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2960 0x7fa0fc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.617+0000 7fa120a00640 1 -- 172.21.15.39:0/2024423632 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7fa104103f00 con 0x7fa11c075650 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.617+0000 7fa11aa00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2960 0x7fa0fc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.617+0000 7fa128368740 1 -- 172.21.15.39:0/1472863844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 msgr2=0x5565763767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.617+0000 7fa128368740 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x5565763767f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa114009bb0 tx=0x7fa1140117b0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.618+0000 7fa128368740 1 -- 172.21.15.39:0/1472863844 shutdown_connections 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.618+0000 7fa128368740 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x556576376d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.618+0000 7fa128368740 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x556576377270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.610 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.618+0000 7fa128368740 1 --2- 172.21.15.39:0/1472863844 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x5565763767f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.618+0000 7fa128368740 1 -- 172.21.15.39:0/1472863844 >> 172.21.15.39:0/1472863844 conn(0x556575fc6460 msgr2=0x5565763746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:24.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.618+0000 7fa128368740 1 -- 172.21.15.39:0/1472863844 shutdown_connections 2024-07-04T08:27:24.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.618+0000 7fa128368740 1 -- 172.21.15.39:0/1472863844 wait complete. 2024-07-04T08:27:24.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.618+0000 7fa11aa00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2960 0x7fa0fc0b4e70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x556576375470 tx=0x7fa10c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:24.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.619+0000 7fa128368740 1 -- start start 2024-07-04T08:27:24.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.619+0000 7fa128368740 1 Processor -- start 2024-07-04T08:27:24.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.619+0000 7fa128368740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fa91b0 0x556576382480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa128368740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 0x5565763829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa128368740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fd2070 0x55657638e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa128368740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556576140700 con 0x556575fb1390 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa128368740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556576283bd0 con 0x556575fa91b0 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa128368740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556576384fd0 con 0x556575fd2070 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa128368740 10 client.0 _ll_register_callbacks cb 0x5565762d5590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fa91b0 0x556576382480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 0x5565763829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fa91b0 0x556576382480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:53416/0 (socket says 172.21.15.39:53416) 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11aa00640 1 -- 172.21.15.39:0/3997332999 learned_addr learned my addr 172.21.15.39:0/3997332999 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:24.614 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11aa00640 1 -- 172.21.15.39:0/3997332999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fd2070 msgr2=0x55657638e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11b400640 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fd2070 0x55657638e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11aa00640 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fd2070 0x55657638e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11aa00640 1 -- 172.21.15.39:0/3997332999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 msgr2=0x5565763829c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11aa00640 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 0x5565763829c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11aa00640 1 -- 172.21.15.39:0/3997332999 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55657603d650 con 0x556575fa91b0 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11a000640 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 0x5565763829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.620+0000 7fa11b400640 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fd2070 0x55657638e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.621+0000 7fa11aa00640 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fa91b0 0x556576382480 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009710 tx=0x7fa10c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=536949757d07b89f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.621+0000 7fa109600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:27:24.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.621+0000 7fa109600640 1 -- 172.21.15.39:0/3997332999 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c005020 con 0x556575fa91b0 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.621+0000 7fa109600640 1 -- 172.21.15.39:0/3997332999 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7fa10c018030 con 0x556575fa91b0 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.621+0000 7fa128368740 1 -- 172.21.15.39:0/3997332999 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1140047a0 con 0x556575fa91b0 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa109600640 1 -- 172.21.15.39:0/3997332999 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa10c034050 con 0x556575fa91b0 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa128368740 10 client.28250 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa128368740 1 -- 172.21.15.39:0/3997332999 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556576390fb0 con 0x556575fa91b0 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa109600640 1 -- 172.21.15.39:0/3997332999 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1140047a0 con 0x556575fa91b0 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa109600640 1 -- 172.21.15.39:0/3997332999 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa10c029420 con 0x556575fa91b0 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa128368740 10 client.28250 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa128368740 10 client.28250 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa128368740 20 client.28250 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:24.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa128368740 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556575f909f0 0x556576394550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa128368740 4 client.28250 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:27:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.622+0000 7fa128368740 1 -- 172.21.15.39:0/3997332999 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565762eee50 con 0x556575f909f0 2024-07-04T08:27:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.623+0000 7fa11a000640 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556575f909f0 0x556576394550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.623+0000 7fa11a000640 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556575f909f0 0x556576394550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d356bd4079f695d server_cookie=99fe2e89954c39bf in_seq=0 out_seq=0 2024-07-04T08:27:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.623+0000 7fa109600640 10 client.28250 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:27:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.624+0000 7fa109600640 1 -- 172.21.15.39:0/3997332999 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5565762eee50 con 0x556575f909f0 2024-07-04T08:27:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.624+0000 7fa109600640 10 client.28250 handle_command_reply: tid=0 2024-07-04T08:27:24.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.624+0000 7fa128368740 1 client.28250 shutdown 2024-07-04T08:27:24.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.624+0000 7fa128368740 1 -- 172.21.15.39:0/3997332999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fa91b0 msgr2=0x556576382480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:24.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.624+0000 7fa128368740 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fa91b0 0x556576382480 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009710 tx=0x7fa10c004730 comp rx=0 tx=0).stop 2024-07-04T08:27:24.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa128368740 1 -- 172.21.15.39:0/3997332999 shutdown_connections 2024-07-04T08:27:24.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa128368740 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556575f909f0 0x556576394550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa128368740 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 0x5565763829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa128368740 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fd2070 0x55657638e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa109600640 0 client.28250 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:27:24.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa128368740 1 --2- 172.21.15.39:0/3997332999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fa91b0 0x556576382480 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa128368740 1 -- 172.21.15.39:0/3997332999 >> 172.21.15.39:0/3997332999 conn(0x556575fc6460 msgr2=0x5565763799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:24.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa128368740 1 -- 172.21.15.39:0/3997332999 shutdown_connections 2024-07-04T08:27:24.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa128368740 1 -- 172.21.15.39:0/3997332999 wait complete. 2024-07-04T08:27:24.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.625+0000 7fa128368740 20 client.28250 trim_cache size 0 max 16384 2024-07-04T08:27:24.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.640+0000 7fa128368740 1 Processor -- start 2024-07-04T08:27:24.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa128368740 1 -- start start 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa128368740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fa91b0 0x55657637a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa128368740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 0x55657638e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa128368740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x55657637ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa128368740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b2620 con 0x556575fb1390 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa128368740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b4160 con 0x556575fd2070 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa128368740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b2cf0 con 0x556575fa91b0 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fa91b0 0x55657637a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa11b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x55657637ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fa91b0 0x55657637a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57490/0 (socket says 172.21.15.39:57490) 2024-07-04T08:27:24.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.641+0000 7fa11aa00640 1 -- 172.21.15.39:0/1160818593 learned_addr learned my addr 172.21.15.39:0/1160818593 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:24.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.642+0000 7fa11a000640 1 --2- 172.21.15.39:0/1160818593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 0x55657638e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.642+0000 7fa11aa00640 1 -- 172.21.15.39:0/1160818593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 msgr2=0x55657637ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:24.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.642+0000 7fa11aa00640 1 --2- 172.21.15.39:0/1160818593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x55657637ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.642+0000 7fa11aa00640 1 -- 172.21.15.39:0/1160818593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 msgr2=0x55657638e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.642+0000 7fa11aa00640 1 --2- 172.21.15.39:0/1160818593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 0x55657638e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.642+0000 7fa11aa00640 1 -- 172.21.15.39:0/1160818593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556576374c10 con 0x556575fa91b0 2024-07-04T08:27:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.642+0000 7fa11b400640 1 --2- 172.21.15.39:0/1160818593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x55657637ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.642+0000 7fa11aa00640 1 --2- 172.21.15.39:0/1160818593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fa91b0 0x55657637a000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa10c004700 tx=0x7fa10c009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe1a7ebd02a093a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.643+0000 7fa10be00640 1 -- 172.21.15.39:0/1160818593 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c029b70 con 0x556575fa91b0 2024-07-04T08:27:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.643+0000 7fa10be00640 1 -- 172.21.15.39:0/1160818593 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c0054d0 con 0x556575fa91b0 2024-07-04T08:27:24.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.643+0000 7fa10be00640 1 -- 172.21.15.39:0/1160818593 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0049e0 con 0x556575fa91b0 2024-07-04T08:27:24.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.644+0000 7fa128368740 1 -- 172.21.15.39:0/1160818593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fa91b0 msgr2=0x55657637a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:24.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.644+0000 7fa128368740 1 --2- 172.21.15.39:0/1160818593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fa91b0 0x55657637a000 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa10c004700 tx=0x7fa10c009f60 comp rx=0 tx=0).stop 2024-07-04T08:27:24.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.644+0000 7fa128368740 1 -- 172.21.15.39:0/1160818593 shutdown_connections 2024-07-04T08:27:24.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.644+0000 7fa128368740 1 --2- 172.21.15.39:0/1160818593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fb1390 0x55657638e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.644+0000 7fa128368740 1 --2- 172.21.15.39:0/1160818593 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x55657637ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.644+0000 7fa128368740 1 --2- 172.21.15.39:0/1160818593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fa91b0 0x55657637a000 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.644+0000 7fa128368740 1 -- 172.21.15.39:0/1160818593 >> 172.21.15.39:0/1160818593 conn(0x556575fc6460 msgr2=0x5565763798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.644+0000 7fa128368740 1 -- 172.21.15.39:0/1160818593 shutdown_connections 2024-07-04T08:27:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.644+0000 7fa128368740 1 -- 172.21.15.39:0/1160818593 wait complete. 2024-07-04T08:27:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.645+0000 7fa128368740 1 -- start start 2024-07-04T08:27:24.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.645+0000 7fa128368740 1 Processor -- start 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa128368740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x55657638a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa128368740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x55657638aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa128368740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x556576385b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa128368740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b3a90 con 0x556575fa91b0 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa128368740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b33c0 con 0x556575fd2070 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa128368740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b0600 con 0x556575fb1390 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa128368740 10 client.0 _ll_register_callbacks cb 0x7fa10c009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x55657638a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x55657638aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x55657638a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42290/0 (socket says 172.21.15.39:42290) 2024-07-04T08:27:24.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11aa00640 1 -- 172.21.15.39:0/1573435108 learned_addr learned my addr 172.21.15.39:0/1573435108 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11b400640 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x556576385b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11a000640 1 -- 172.21.15.39:0/1573435108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 msgr2=0x556576385b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11a000640 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x556576385b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11a000640 1 -- 172.21.15.39:0/1573435108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 msgr2=0x55657638a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11a000640 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x55657638a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.646+0000 7fa11a000640 1 -- 172.21.15.39:0/1573435108 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556576386270 con 0x556575fb1390 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.647+0000 7fa11aa00640 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x55657638a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.647+0000 7fa11a000640 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x55657638aab0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa1041013a0 tx=0x7fa1040095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7085bfe2f7dc5c48 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.647+0000 7fa109600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.647+0000 7fa109600640 1 -- 172.21.15.39:0/1573435108 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1040e5070 con 0x556575fb1390 2024-07-04T08:27:24.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.647+0000 7fa128368740 1 -- 172.21.15.39:0/1573435108 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa10c0054d0 con 0x556575fb1390 2024-07-04T08:27:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.647+0000 7fa109600640 1 -- 172.21.15.39:0/1573435108 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7fa104101510 con 0x556575fb1390 2024-07-04T08:27:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.647+0000 7fa109600640 1 -- 172.21.15.39:0/1573435108 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa104101710 con 0x556575fb1390 2024-07-04T08:27:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.647+0000 7fa128368740 10 client.28177 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:24.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.647+0000 7fa128368740 1 -- 172.21.15.39:0/1573435108 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565763fc8c0 con 0x556575fb1390 2024-07-04T08:27:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.648+0000 7fa109600640 1 -- 172.21.15.39:0/1573435108 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c0054d0 con 0x556575fb1390 2024-07-04T08:27:24.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.648+0000 7fa109600640 1 -- 172.21.15.39:0/1573435108 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa104105040 con 0x556575fb1390 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.648+0000 7fa128368740 10 client.28177 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.648+0000 7fa128368740 10 client.28177 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.648+0000 7fa128368740 20 client.28177 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.648+0000 7fa128368740 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556575f65da0 0x5565763ff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.648+0000 7fa128368740 4 client.28177 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.648+0000 7fa128368740 1 -- 172.21.15.39:0/1573435108 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565762ea8f0 con 0x556575f65da0 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.649+0000 7fa11aa00640 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556575f65da0 0x5565763ff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.649+0000 7fa11aa00640 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556575f65da0 0x5565763ff000 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af62a5da2d65401b server_cookie=aae2cdc374dd417d in_seq=0 out_seq=0 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.649+0000 7fa109600640 10 client.28177 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:27:24.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.649+0000 7fa109600640 1 -- 172.21.15.39:0/1573435108 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565762ea8f0 con 0x556575f65da0 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.649+0000 7fa109600640 10 client.28177 handle_command_reply: tid=0 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.649+0000 7fa128368740 1 client.28177 shutdown 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 -- 172.21.15.39:0/1573435108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 msgr2=0x55657638aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x55657638aab0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa1041013a0 tx=0x7fa1040095c0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 -- 172.21.15.39:0/1573435108 shutdown_connections 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x556575f65da0 0x5565763ff000 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556575fd2070 0x556576385b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556575fb1390 0x55657638aab0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 --2- 172.21.15.39:0/1573435108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556575fa91b0 0x55657638a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 -- 172.21.15.39:0/1573435108 >> 172.21.15.39:0/1573435108 conn(0x556575fc6460 msgr2=0x556576377820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:24.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa109600640 0 client.28177 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:27:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 -- 172.21.15.39:0/1573435108 shutdown_connections 2024-07-04T08:27:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 1 -- 172.21.15.39:0/1573435108 wait complete. 2024-07-04T08:27:24.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.650+0000 7fa128368740 20 client.28177 trim_cache size 0 max 16384 2024-07-04T08:27:24.643 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:27:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 -- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2960 msgr2=0x7fa0fc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2960 0x7fa0fc0b4e70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x556576375470 tx=0x7fa10c005f90 comp rx=0 tx=0).stop 2024-07-04T08:27:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 -- 172.21.15.39:0/2024423632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11c075650 msgr2=0x7fa11c19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11c075650 0x7fa11c19c1e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa104033040 tx=0x7fa1040316f0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 -- 172.21.15.39:0/2024423632 shutdown_connections 2024-07-04T08:27:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa0fc0b2960 0x7fa0fc0b4e70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa11c10f630 0x7fa11c1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11c075650 0x7fa11c19c1e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 --2- 172.21.15.39:0/2024423632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c074d00 0x7fa11c19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 -- 172.21.15.39:0/2024423632 >> 172.21.15.39:0/2024423632 conn(0x7fa11c0feae0 msgr2=0x7fa11c111940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 -- 172.21.15.39:0/2024423632 shutdown_connections 2024-07-04T08:27:24.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:24.651+0000 7fa121e00640 1 -- 172.21.15.39:0/2024423632 wait complete. 2024-07-04T08:27:25.019 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:24 smithi039 ceph-mon[32062]: pgmap v858: 129 pgs: 129 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:25.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:24 smithi133 ceph-mon[37050]: pgmap v858: 129 pgs: 129 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:25.133 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:24 smithi050 ceph-mon[36577]: pgmap v858: 129 pgs: 129 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:25.552 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:27:26.039 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:27.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:27 smithi133 ceph-mon[37050]: pgmap v859: 129 pgs: 129 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:27.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:27 smithi039 ceph-mon[32062]: pgmap v859: 129 pgs: 129 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:27.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:27 smithi050 ceph-mon[36577]: pgmap v859: 129 pgs: 129 active+clean; 18 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:27:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.557+0000 7fd077200640 1 -- 172.21.15.39:0/2927980610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810f7a0 msgr2=0x7fd078115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.557+0000 7fd077200640 1 --2- 172.21.15.39:0/2927980610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810f7a0 0x7fd078115fd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd070009ca0 tx=0x7fd07002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.557+0000 7fd077200640 1 -- 172.21.15.39:0/2927980610 shutdown_connections 2024-07-04T08:27:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.557+0000 7fd077200640 1 --2- 172.21.15.39:0/2927980610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810f7a0 0x7fd078115fd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.557+0000 7fd077200640 1 --2- 172.21.15.39:0/2927980610 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd07810ee50 0x7fd07810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.557+0000 7fd077200640 1 --2- 172.21.15.39:0/2927980610 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd078109510 0x7fd078109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.557+0000 7fd077200640 1 -- 172.21.15.39:0/2927980610 >> 172.21.15.39:0/2927980610 conn(0x7fd078078b10 msgr2=0x7fd078107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.557+0000 7fd077200640 1 -- 172.21.15.39:0/2927980610 shutdown_connections 2024-07-04T08:27:28.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.557+0000 7fd077200640 1 -- 172.21.15.39:0/2927980610 wait complete. 2024-07-04T08:27:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.558+0000 7fd077200640 1 Processor -- start 2024-07-04T08:27:28.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.558+0000 7fd077200640 1 -- start start 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.559+0000 7fd077200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd078109510 0x7fd07819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.559+0000 7fd077200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810ee50 0x7fd0781a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.559+0000 7fd077200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd07810f7a0 0x7fd0781a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.559+0000 7fd077200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781194c0 con 0x7fd078109510 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.559+0000 7fd077200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd078119340 con 0x7fd07810f7a0 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.559+0000 7fd077200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd078119640 con 0x7fd07810ee50 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.559+0000 7fd06fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810ee50 0x7fd0781a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd06fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810ee50 0x7fd0781a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57504/0 (socket says 172.21.15.39:57504) 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd06fe00640 1 -- 172.21.15.39:0/1478863081 learned_addr learned my addr 172.21.15.39:0/1478863081 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:28.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd076800640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd07810f7a0 0x7fd0781a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd075e00640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd078109510 0x7fd07819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd06fe00640 1 -- 172.21.15.39:0/1478863081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd07810f7a0 msgr2=0x7fd0781a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd06fe00640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd07810f7a0 0x7fd0781a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd06fe00640 1 -- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd078109510 msgr2=0x7fd07819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd06fe00640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd078109510 0x7fd07819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd06fe00640 1 -- 172.21.15.39:0/1478863081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0781a5de0 con 0x7fd07810ee50 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.560+0000 7fd076800640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd07810f7a0 0x7fd0781a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.561+0000 7fd06fe00640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810ee50 0x7fd0781a0410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd06000e9c0 tx=0x7fd06000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.561+0000 7fd06ea00640 1 -- 172.21.15.39:0/1478863081 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06000ccc0 con 0x7fd07810ee50 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.561+0000 7fd077200640 1 -- 172.21.15.39:0/1478863081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0781a60d0 con 0x7fd07810ee50 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.561+0000 7fd06ea00640 1 -- 172.21.15.39:0/1478863081 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0600054d0 con 0x7fd07810ee50 2024-07-04T08:27:28.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.561+0000 7fd06ea00640 1 -- 172.21.15.39:0/1478863081 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060010430 con 0x7fd07810ee50 2024-07-04T08:27:28.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.561+0000 7fd077200640 1 -- 172.21.15.39:0/1478863081 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0781ac830 con 0x7fd07810ee50 2024-07-04T08:27:28.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.564+0000 7fd082fee740 1 Processor -- start 2024-07-04T08:27:28.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.565+0000 7fd082fee740 1 -- start start 2024-07-04T08:27:28.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.565+0000 7fd082fee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb0510070 0x55dfb08b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.565+0000 7fd082fee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04e71b0 0x55dfb08b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.565+0000 7fd082fee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ef390 0x55dfb08b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.565+0000 7fd082fee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55dfb082aee0 con 0x55dfb0510070 2024-07-04T08:27:28.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.565+0000 7fd082fee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dfb081a480 con 0x55dfb04ef390 2024-07-04T08:27:28.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.565+0000 7fd082fee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55dfb081e4c0 con 0x55dfb04e71b0 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd06ea00640 1 -- 172.21.15.39:0/1478863081 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd060005020 con 0x7fd07810ee50 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd076800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb0510070 0x55dfb08b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd075e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04e71b0 0x55dfb08b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd076800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb0510070 0x55dfb08b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42304/0 (socket says 172.21.15.39:42304) 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd076800640 1 -- 172.21.15.39:0/4107590704 learned_addr learned my addr 172.21.15.39:0/4107590704 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd06fe00640 1 --2- 172.21.15.39:0/4107590704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ef390 0x55dfb08b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd076800640 1 -- 172.21.15.39:0/4107590704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04e71b0 msgr2=0x55dfb08b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd076800640 1 --2- 172.21.15.39:0/4107590704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04e71b0 0x55dfb08b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd076800640 1 -- 172.21.15.39:0/4107590704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ef390 msgr2=0x55dfb08b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd076800640 1 --2- 172.21.15.39:0/4107590704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ef390 0x55dfb08b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd076800640 1 -- 172.21.15.39:0/4107590704 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb07b2650 con 0x55dfb0510070 2024-07-04T08:27:28.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.566+0000 7fd06fe00640 1 --2- 172.21.15.39:0/4107590704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ef390 0x55dfb08b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:27:28.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.567+0000 7fd076800640 1 --2- 172.21.15.39:0/4107590704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb0510070 0x55dfb08b4950 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd070009ca0 tx=0x7fd070009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280e2d0f55285fb8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.567+0000 7fd067400640 1 -- 172.21.15.39:0/4107590704 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07002f3d0 con 0x55dfb0510070 2024-07-04T08:27:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.567+0000 7fd067400640 1 -- 172.21.15.39:0/4107590704 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd070036040 con 0x55dfb0510070 2024-07-04T08:27:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.567+0000 7fd067400640 1 -- 172.21.15.39:0/4107590704 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0700044b0 con 0x55dfb0510070 2024-07-04T08:27:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd06ea00640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd05c0b2960 0x7fd05c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd082fee740 1 -- 172.21.15.39:0/4107590704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb0510070 msgr2=0x55dfb08b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd082fee740 1 --2- 172.21.15.39:0/4107590704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb0510070 0x55dfb08b4950 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd070009ca0 tx=0x7fd070009490 comp rx=0 tx=0).stop 2024-07-04T08:27:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd06ea00640 1 -- 172.21.15.39:0/1478863081 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7fd0600e2bf0 con 0x7fd07810ee50 2024-07-04T08:27:28.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd075e00640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd05c0b2960 0x7fd05c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd082fee740 1 -- 172.21.15.39:0/4107590704 shutdown_connections 2024-07-04T08:27:28.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd082fee740 1 --2- 172.21.15.39:0/4107590704 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ef390 0x55dfb08b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd082fee740 1 --2- 172.21.15.39:0/4107590704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04e71b0 0x55dfb08b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd082fee740 1 --2- 172.21.15.39:0/4107590704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb0510070 0x55dfb08b4950 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd082fee740 1 -- 172.21.15.39:0/4107590704 >> 172.21.15.39:0/4107590704 conn(0x55dfb0504460 msgr2=0x55dfb08b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:28.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.568+0000 7fd082fee740 1 -- 172.21.15.39:0/4107590704 shutdown_connections 2024-07-04T08:27:28.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.569+0000 7fd082fee740 1 -- 172.21.15.39:0/4107590704 wait complete. 2024-07-04T08:27:28.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.569+0000 7fd075e00640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd05c0b2960 0x7fd05c0b4e70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55dfb08b35d0 tx=0x7fd068005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:28.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.569+0000 7fd082fee740 1 -- start start 2024-07-04T08:27:28.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.569+0000 7fd082fee740 1 Processor -- start 2024-07-04T08:27:28.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.570+0000 7fd082fee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.570+0000 7fd082fee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb08c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.570+0000 7fd082fee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb0510070 0x55dfb08cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.570+0000 7fd082fee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55dfb067e700 con 0x55dfb04ef390 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.570+0000 7fd082fee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dfb07c1bb0 con 0x55dfb04e71b0 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.570+0000 7fd082fee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55dfb08c2f50 con 0x55dfb0510070 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.570+0000 7fd082fee740 10 client.0 _ll_register_callbacks cb 0x55dfb0813570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.570+0000 7fd06fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb08c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.570+0000 7fd06fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb08c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42310/0 (socket says 172.21.15.39:42310) 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd06fe00640 1 -- 172.21.15.39:0/684665192 learned_addr learned my addr 172.21.15.39:0/684665192 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd075e00640 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd076800640 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb0510070 0x55dfb08cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd06fe00640 1 -- 172.21.15.39:0/684665192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb0510070 msgr2=0x55dfb08cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd06fe00640 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb0510070 0x55dfb08cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd06fe00640 1 -- 172.21.15.39:0/684665192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 msgr2=0x55dfb08c0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:28.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd06fe00640 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd06fe00640 1 -- 172.21.15.39:0/684665192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb057b650 con 0x55dfb04ef390 2024-07-04T08:27:28.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd075e00640 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:28.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd06fe00640 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb08c0ab0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd060004650 tx=0x7fd06000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5305f0d309f4f1c0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.571+0000 7fd05be00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.572+0000 7fd05be00640 1 -- 172.21.15.39:0/684665192 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0600c3060 con 0x55dfb04ef390 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.572+0000 7fd05be00640 1 -- 172.21.15.39:0/684665192 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7fd0600a6140 con 0x55dfb04ef390 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.572+0000 7fd082fee740 1 -- 172.21.15.39:0/684665192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd070036040 con 0x55dfb04ef390 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.572+0000 7fd05be00640 1 -- 172.21.15.39:0/684665192 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd0600e0450 con 0x55dfb04ef390 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.572+0000 7fd082fee740 10 client.18132 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.572+0000 7fd082fee740 1 -- 172.21.15.39:0/684665192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfb08cef10 con 0x55dfb04ef390 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.572+0000 7fd05be00640 1 -- 172.21.15.39:0/684665192 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd070036040 con 0x55dfb04ef390 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.573+0000 7fd05be00640 1 -- 172.21.15.39:0/684665192 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd0600b2520 con 0x55dfb04ef390 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.573+0000 7fd082fee740 10 client.18132 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.573+0000 7fd082fee740 10 client.18132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:27:28.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.573+0000 7fd082fee740 20 client.18132 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.573+0000 7fd082fee740 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55dfb04ce9f0 0x55dfb08d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.573+0000 7fd082fee740 4 client.18132 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:27:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.573+0000 7fd082fee740 1 -- 172.21.15.39:0/684665192 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfb082ce30 con 0x55dfb04ce9f0 2024-07-04T08:27:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.573+0000 7fd075e00640 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55dfb04ce9f0 0x55dfb08d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.574+0000 7fd075e00640 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55dfb04ce9f0 0x55dfb08d2330 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594b79486ee28aa6 server_cookie=dbfe5d5a414fbdcf in_seq=0 out_seq=0 2024-07-04T08:27:28.565 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.574+0000 7fd05be00640 10 client.18132 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd05be00640 1 -- 172.21.15.39:0/684665192 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dfb082ce30 con 0x55dfb04ce9f0 2024-07-04T08:27:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd05be00640 10 client.18132 handle_command_reply: tid=0 2024-07-04T08:27:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd082fee740 1 client.18132 shutdown 2024-07-04T08:27:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd082fee740 1 -- 172.21.15.39:0/684665192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 msgr2=0x55dfb08c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd082fee740 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb08c0ab0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd060004650 tx=0x7fd06000cf20 comp rx=0 tx=0).stop 2024-07-04T08:27:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd082fee740 1 -- 172.21.15.39:0/684665192 shutdown_connections 2024-07-04T08:27:28.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd082fee740 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55dfb04ce9f0 0x55dfb08d2330 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd082fee740 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb08c0ab0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd082fee740 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb0510070 0x55dfb08cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.575+0000 7fd082fee740 1 --2- 172.21.15.39:0/684665192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.576+0000 7fd05be00640 0 client.18132 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.576+0000 7fd082fee740 1 -- 172.21.15.39:0/684665192 >> 172.21.15.39:0/684665192 conn(0x55dfb0504460 msgr2=0x55dfb08b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.576+0000 7fd082fee740 1 -- 172.21.15.39:0/684665192 shutdown_connections 2024-07-04T08:27:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.576+0000 7fd082fee740 1 -- 172.21.15.39:0/684665192 wait complete. 2024-07-04T08:27:28.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.576+0000 7fd082fee740 20 client.18132 trim_cache size 0 max 16384 2024-07-04T08:27:28.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.592+0000 7fd082fee740 1 Processor -- start 2024-07-04T08:27:28.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.592+0000 7fd082fee740 1 -- start start 2024-07-04T08:27:28.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.592+0000 7fd082fee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ce9f0 0x55dfb08ba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.592+0000 7fd082fee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04e71b0 0x55dfb08cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.592+0000 7fd082fee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ef390 0x55dfb08d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.592+0000 7fd082fee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781b6790 con 0x55dfb04e71b0 2024-07-04T08:27:28.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.592+0000 7fd082fee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781b82d0 con 0x55dfb04ce9f0 2024-07-04T08:27:28.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.592+0000 7fd082fee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781b6e60 con 0x55dfb04ef390 2024-07-04T08:27:28.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd076800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ef390 0x55dfb08d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd075e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ce9f0 0x55dfb08ba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd076800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ef390 0x55dfb08d7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:57540/0 (socket says 172.21.15.39:57540) 2024-07-04T08:27:28.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd076800640 1 -- 172.21.15.39:0/946677511 learned_addr learned my addr 172.21.15.39:0/946677511 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:28.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd06fe00640 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04e71b0 0x55dfb08cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd076800640 1 -- 172.21.15.39:0/946677511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ce9f0 msgr2=0x55dfb08ba170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd076800640 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ce9f0 0x55dfb08ba170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd076800640 1 -- 172.21.15.39:0/946677511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04e71b0 msgr2=0x55dfb08cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd076800640 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04e71b0 0x55dfb08cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd076800640 1 -- 172.21.15.39:0/946677511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb08b71e0 con 0x55dfb04ef390 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd075e00640 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ce9f0 0x55dfb08ba170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.593+0000 7fd06fe00640 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04e71b0 0x55dfb08cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.594+0000 7fd076800640 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ef390 0x55dfb08d7a80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd07002aeb0 tx=0x7fd070009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17615b2a13cb31c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.594+0000 7fd066a00640 1 -- 172.21.15.39:0/946677511 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070055070 con 0x55dfb04ef390 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.594+0000 7fd066a00640 1 -- 172.21.15.39:0/946677511 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd070036040 con 0x55dfb04ef390 2024-07-04T08:27:28.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.594+0000 7fd066a00640 1 -- 172.21.15.39:0/946677511 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0700042a0 con 0x55dfb04ef390 2024-07-04T08:27:28.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.595+0000 7fd082fee740 1 -- 172.21.15.39:0/946677511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ef390 msgr2=0x55dfb08d7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:28.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.595+0000 7fd082fee740 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ef390 0x55dfb08d7a80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd07002aeb0 tx=0x7fd070009a20 comp rx=0 tx=0).stop 2024-07-04T08:27:28.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.595+0000 7fd082fee740 1 -- 172.21.15.39:0/946677511 shutdown_connections 2024-07-04T08:27:28.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.595+0000 7fd082fee740 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04e71b0 0x55dfb08cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.595+0000 7fd082fee740 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ef390 0x55dfb08d7a80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.595+0000 7fd082fee740 1 --2- 172.21.15.39:0/946677511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04ce9f0 0x55dfb08ba170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.595+0000 7fd082fee740 1 -- 172.21.15.39:0/946677511 >> 172.21.15.39:0/946677511 conn(0x55dfb0504460 msgr2=0x55dfb08b9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:28.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.595+0000 7fd082fee740 1 -- 172.21.15.39:0/946677511 shutdown_connections 2024-07-04T08:27:28.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.595+0000 7fd082fee740 1 -- 172.21.15.39:0/946677511 wait complete. 2024-07-04T08:27:28.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.596+0000 7fd082fee740 1 -- start start 2024-07-04T08:27:28.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.596+0000 7fd082fee740 1 Processor -- start 2024-07-04T08:27:28.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd082fee740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ce9f0 0x55dfb08c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd082fee740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd082fee740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb0939a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd082fee740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781b7c00 con 0x55dfb04ef390 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd082fee740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781b7530 con 0x55dfb04e71b0 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd082fee740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781b4770 con 0x55dfb04ce9f0 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd082fee740 10 client.0 _ll_register_callbacks cb 0x7fd0600096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd076800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb0939a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd06fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd076800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb0939a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42324/0 (socket says 172.21.15.39:42324) 2024-07-04T08:27:28.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd075e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ce9f0 0x55dfb08c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.597+0000 7fd076800640 1 -- 172.21.15.39:0/2486558197 learned_addr learned my addr 172.21.15.39:0/2486558197 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd075e00640 1 -- 172.21.15.39:0/2486558197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 msgr2=0x55dfb08c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd075e00640 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd075e00640 1 -- 172.21.15.39:0/2486558197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 msgr2=0x55dfb0939a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd075e00640 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb0939a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd075e00640 1 -- 172.21.15.39:0/2486558197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb0939f90 con 0x55dfb04ce9f0 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd06fe00640 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd076800640 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb0939a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd075e00640 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ce9f0 0x55dfb08c47a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55dfb08d29c0 tx=0x7fd0680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5f9e8105c0102a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd05be00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd05be00640 1 -- 172.21.15.39:0/2486558197 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068030030 con 0x55dfb04ce9f0 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd05be00640 1 -- 172.21.15.39:0/2486558197 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7fd068018030 con 0x55dfb04ce9f0 2024-07-04T08:27:28.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.598+0000 7fd082fee740 1 -- 172.21.15.39:0/2486558197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd070036040 con 0x55dfb04ce9f0 2024-07-04T08:27:28.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.599+0000 7fd05be00640 1 -- 172.21.15.39:0/2486558197 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd06802b5c0 con 0x55dfb04ce9f0 2024-07-04T08:27:28.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.599+0000 7fd082fee740 10 client.28195 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:28.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.599+0000 7fd082fee740 1 -- 172.21.15.39:0/2486558197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfb093b240 con 0x55dfb04ce9f0 2024-07-04T08:27:28.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.599+0000 7fd05be00640 1 -- 172.21.15.39:0/2486558197 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd070036040 con 0x55dfb04ce9f0 2024-07-04T08:27:28.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.600+0000 7fd05be00640 1 -- 172.21.15.39:0/2486558197 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd06801f8c0 con 0x55dfb04ce9f0 2024-07-04T08:27:28.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.600+0000 7fd082fee740 10 client.28195 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.600+0000 7fd082fee740 10 client.28195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.600+0000 7fd082fee740 20 client.28195 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.600+0000 7fd082fee740 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55dfb0510070 0x55dfb093da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.600+0000 7fd082fee740 4 client.28195 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.600+0000 7fd082fee740 1 -- 172.21.15.39:0/2486558197 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfb08288d0 con 0x55dfb0510070 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.600+0000 7fd06fe00640 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55dfb0510070 0x55dfb093da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd06fe00640 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55dfb0510070 0x55dfb093da00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3259dc7c09cde6ca server_cookie=96d844b3c5ae5e4e in_seq=0 out_seq=0 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd05be00640 10 client.28195 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd05be00640 1 -- 172.21.15.39:0/2486558197 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfb08288d0 con 0x55dfb0510070 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd05be00640 10 client.28195 handle_command_reply: tid=0 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 client.28195 shutdown 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 -- 172.21.15.39:0/2486558197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ce9f0 msgr2=0x55dfb08c47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:28.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ce9f0 0x55dfb08c47a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x55dfb08d29c0 tx=0x7fd0680046c0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 -- 172.21.15.39:0/2486558197 shutdown_connections 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55dfb0510070 0x55dfb093da00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55dfb04ef390 0x55dfb0939a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dfb04e71b0 0x55dfb08c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 --2- 172.21.15.39:0/2486558197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dfb04ce9f0 0x55dfb08c47a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd05be00640 0 client.28195 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 -- 172.21.15.39:0/2486558197 >> 172.21.15.39:0/2486558197 conn(0x55dfb0504460 msgr2=0x55dfb08b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 -- 172.21.15.39:0/2486558197 shutdown_connections 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.601+0000 7fd082fee740 1 -- 172.21.15.39:0/2486558197 wait complete. 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd082fee740 20 client.28195 trim_cache size 0 max 16384 2024-07-04T08:27:28.593 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:27:28.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 -- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd05c0b2960 msgr2=0x7fd05c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:28.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd05c0b2960 0x7fd05c0b4e70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55dfb08b35d0 tx=0x7fd068005f90 comp rx=0 tx=0).stop 2024-07-04T08:27:28.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 -- 172.21.15.39:0/1478863081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810ee50 msgr2=0x7fd0781a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:28.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810ee50 0x7fd0781a0410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd06000e9c0 tx=0x7fd06000ee80 comp rx=0 tx=0).stop 2024-07-04T08:27:28.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 -- 172.21.15.39:0/1478863081 shutdown_connections 2024-07-04T08:27:28.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd05c0b2960 0x7fd05c0b4e70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd07810f7a0 0x7fd0781a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd07810ee50 0x7fd0781a0410 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 --2- 172.21.15.39:0/1478863081 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd078109510 0x7fd07819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:28.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.602+0000 7fd077200640 1 -- 172.21.15.39:0/1478863081 >> 172.21.15.39:0/1478863081 conn(0x7fd078078b10 msgr2=0x7fd078079170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:28.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.603+0000 7fd077200640 1 -- 172.21.15.39:0/1478863081 shutdown_connections 2024-07-04T08:27:28.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:28.603+0000 7fd077200640 1 -- 172.21.15.39:0/1478863081 wait complete. 2024-07-04T08:27:29.340 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:29 smithi039 ceph-mon[32062]: pgmap v860: 129 pgs: 129 active+clean; 19 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 15 op/s 2024-07-04T08:27:29.435 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:29 smithi050 ceph-mon[36577]: pgmap v860: 129 pgs: 129 active+clean; 19 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 15 op/s 2024-07-04T08:27:29.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:29 smithi133 ceph-mon[37050]: pgmap v860: 129 pgs: 129 active+clean; 19 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 67 KiB/s wr, 15 op/s 2024-07-04T08:27:30.499 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:27:30.500 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:27:30.982 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:31.483 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:31 smithi039 ceph-mon[32062]: pgmap v861: 129 pgs: 129 active+clean; 19 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-07-04T08:27:31.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:31 smithi133 ceph-mon[37050]: pgmap v861: 129 pgs: 129 active+clean; 19 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-07-04T08:27:31.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:31 smithi050 ceph-mon[36577]: pgmap v861: 129 pgs: 129 active+clean; 19 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 15 op/s 2024-07-04T08:27:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.329+0000 7fdf57e00640 1 -- 172.21.15.39:0/2019108150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf5810aa40 msgr2=0x7fdf5810ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.329+0000 7fdf57e00640 1 --2- 172.21.15.39:0/2019108150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf5810aa40 0x7fdf5810ae50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009900 tx=0x7fdf4002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.329+0000 7fdf57e00640 1 -- 172.21.15.39:0/2019108150 shutdown_connections 2024-07-04T08:27:33.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.329+0000 7fdf57e00640 1 --2- 172.21.15.39:0/2019108150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf5810aa40 0x7fdf5810ae50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.329+0000 7fdf57e00640 1 --2- 172.21.15.39:0/2019108150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf5810b390 0x7fdf58111bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.329+0000 7fdf57e00640 1 --2- 172.21.15.39:0/2019108150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf58105100 0x7fdf58105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.329+0000 7fdf57e00640 1 -- 172.21.15.39:0/2019108150 >> 172.21.15.39:0/2019108150 conn(0x7fdf58100bd0 msgr2=0x7fdf58103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:33.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.329+0000 7fdf57e00640 1 -- 172.21.15.39:0/2019108150 shutdown_connections 2024-07-04T08:27:33.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.329+0000 7fdf57e00640 1 -- 172.21.15.39:0/2019108150 wait complete. 2024-07-04T08:27:33.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.331+0000 7fdf57e00640 1 Processor -- start 2024-07-04T08:27:33.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.331+0000 7fdf57e00640 1 -- start start 2024-07-04T08:27:33.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf58105100 0x7fdf5819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf5810aa40 0x7fdf5819c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf5810b390 0x7fdf581a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf57e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58115020 con 0x7fdf5810aa40 2024-07-04T08:27:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf57e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58114ea0 con 0x7fdf5810b390 2024-07-04T08:27:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf57e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581151a0 con 0x7fdf58105100 2024-07-04T08:27:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf56a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf58105100 0x7fdf5819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf4fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf5810aa40 0x7fdf5819c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf56a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf58105100 0x7fdf5819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37698/0 (socket says 172.21.15.39:37698) 2024-07-04T08:27:33.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.332+0000 7fdf56a00640 1 -- 172.21.15.39:0/3637269389 learned_addr learned my addr 172.21.15.39:0/3637269389 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:33.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.333+0000 7fdf57400640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf5810b390 0x7fdf581a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.333+0000 7fdf56a00640 1 -- 172.21.15.39:0/3637269389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf5810b390 msgr2=0x7fdf581a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.333+0000 7fdf56a00640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf5810b390 0x7fdf581a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.333+0000 7fdf56a00640 1 -- 172.21.15.39:0/3637269389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf5810aa40 msgr2=0x7fdf5819c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.333+0000 7fdf56a00640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf5810aa40 0x7fdf5819c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.333+0000 7fdf56a00640 1 -- 172.21.15.39:0/3637269389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf581a1a00 con 0x7fdf58105100 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.333+0000 7fdf57400640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf5810b390 0x7fdf581a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.333+0000 7fdf56a00640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf58105100 0x7fdf5819ba90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800ea60 tx=0x7fdf4800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.333+0000 7fdf4f400640 1 -- 172.21.15.39:0/3637269389 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800cde0 con 0x7fdf58105100 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.334+0000 7fdf57e00640 1 -- 172.21.15.39:0/3637269389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf581a1cf0 con 0x7fdf58105100 2024-07-04T08:27:33.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.334+0000 7fdf4f400640 1 -- 172.21.15.39:0/3637269389 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf480035f0 con 0x7fdf58105100 2024-07-04T08:27:33.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.334+0000 7fdf4f400640 1 -- 172.21.15.39:0/3637269389 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48010690 con 0x7fdf58105100 2024-07-04T08:27:33.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.334+0000 7fdf57e00640 1 -- 172.21.15.39:0/3637269389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf581a8450 con 0x7fdf58105100 2024-07-04T08:27:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.337+0000 7fdf63dad740 1 Processor -- start 2024-07-04T08:27:33.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.337+0000 7fdf63dad740 1 -- start start 2024-07-04T08:27:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.337+0000 7fdf4f400640 1 -- 172.21.15.39:0/3637269389 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fdf480037a0 con 0x7fdf58105100 2024-07-04T08:27:33.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e355217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e35521d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 0x562e35522270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf63dad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562e3544b8e0 con 0x562e3517d350 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf63dad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562e35497ec0 con 0x562e3515c670 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf63dad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562e35487460 con 0x562e35154490 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf56a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e35521d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf57400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e355217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf57400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e355217f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52072/0 (socket says 172.21.15.39:52072) 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf57400640 1 -- 172.21.15.39:0/3746896293 learned_addr learned my addr 172.21.15.39:0/3746896293 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:33.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.338+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/3746896293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 0x562e35522270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.339+0000 7fdf56a00640 1 -- 172.21.15.39:0/3746896293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 msgr2=0x562e35522270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.339+0000 7fdf56a00640 1 --2- 172.21.15.39:0/3746896293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 0x562e35522270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.339+0000 7fdf56a00640 1 -- 172.21.15.39:0/3746896293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 msgr2=0x562e355217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.339+0000 7fdf56a00640 1 --2- 172.21.15.39:0/3746896293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e355217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.339+0000 7fdf56a00640 1 -- 172.21.15.39:0/3746896293 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e3541f280 con 0x562e35154490 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.339+0000 7fdf56a00640 1 --2- 172.21.15.39:0/3746896293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e35521d30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdf480e00b0 tx=0x7fdf480eb7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d6c91444fe1107 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.340+0000 7fdf47e00640 1 -- 172.21.15.39:0/3746896293 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf480f4640 con 0x562e35154490 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.340+0000 7fdf47e00640 1 -- 172.21.15.39:0/3746896293 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf480e88a0 con 0x562e35154490 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.340+0000 7fdf47e00640 1 -- 172.21.15.39:0/3746896293 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf480f38d0 con 0x562e35154490 2024-07-04T08:27:33.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.340+0000 7fdf4f400640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf3c0b2960 0x7fdf3c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.340+0000 7fdf4f400640 1 -- 172.21.15.39:0/3637269389 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7fdf480e2d20 con 0x7fdf58105100 2024-07-04T08:27:33.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.340+0000 7fdf57400640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf3c0b2960 0x7fdf3c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf63dad740 1 -- 172.21.15.39:0/3746896293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 msgr2=0x562e35521d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:33.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3746896293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e35521d30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdf480e00b0 tx=0x7fdf480eb7e0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf63dad740 1 -- 172.21.15.39:0/3746896293 shutdown_connections 2024-07-04T08:27:33.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3746896293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 0x562e35522270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3746896293 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e35521d30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3746896293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e355217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf63dad740 1 -- 172.21.15.39:0/3746896293 >> 172.21.15.39:0/3746896293 conn(0x562e35171740 msgr2=0x562e3551f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf63dad740 1 -- 172.21.15.39:0/3746896293 shutdown_connections 2024-07-04T08:27:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf57400640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf3c0b2960 0x7fdf3c0b4e70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdf50005c40 tx=0x7fdf50009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.341+0000 7fdf63dad740 1 -- 172.21.15.39:0/3746896293 wait complete. 2024-07-04T08:27:33.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.342+0000 7fdf63dad740 1 -- start start 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.342+0000 7fdf63dad740 1 Processor -- start 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 0x562e3552d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e35539170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf63dad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562e3542e760 con 0x562e3517d350 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf63dad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562e3548b4a0 con 0x562e3515c670 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf63dad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562e352eb890 con 0x562e35154490 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf63dad740 10 client.0 _ll_register_callbacks cb 0x562e3542f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf56a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf56a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37716/0 (socket says 172.21.15.39:37716) 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf56a00640 1 -- 172.21.15.39:0/3465942987 learned_addr learned my addr 172.21.15.39:0/3465942987 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:33.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 0x562e3552d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.343+0000 7fdf57400640 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e35539170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf56a00640 1 -- 172.21.15.39:0/3465942987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 msgr2=0x562e3552d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf56a00640 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 0x562e3552d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf56a00640 1 -- 172.21.15.39:0/3465942987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 msgr2=0x562e35539170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf56a00640 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e35539170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf56a00640 1 -- 172.21.15.39:0/3465942987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e351e8930 con 0x562e35154490 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf57400640 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e35539170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf56a00640 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552d070 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdf480a9940 tx=0x7fdf480f3450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17b4cd1c5f7227b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf44c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf44c00640 1 -- 172.21.15.39:0/3465942987 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf480e5070 con 0x562e35154490 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf44c00640 1 -- 172.21.15.39:0/3465942987 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7fdf480e8a50 con 0x562e35154490 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.344+0000 7fdf63dad740 1 -- 172.21.15.39:0/3465942987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf480e88a0 con 0x562e35154490 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf44c00640 1 -- 172.21.15.39:0/3465942987 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf480a5b00 con 0x562e35154490 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf63dad740 10 client.28207 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf63dad740 1 -- 172.21.15.39:0/3465942987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e3553bd10 con 0x562e35154490 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf44c00640 1 -- 172.21.15.39:0/3465942987 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf480e88a0 con 0x562e35154490 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf44c00640 1 -- 172.21.15.39:0/3465942987 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdf480f2070 con 0x562e35154490 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf63dad740 10 client.28207 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:33.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf63dad740 10 client.28207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:27:33.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf63dad740 20 client.28207 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:33.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562e3513bcd0 0x562e3553f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf63dad740 4 client.28207 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:27:33.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.345+0000 7fdf63dad740 1 -- 172.21.15.39:0/3465942987 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e35499e10 con 0x562e3513bcd0 2024-07-04T08:27:33.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.346+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562e3513bcd0 0x562e3553f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.346+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562e3513bcd0 0x562e3553f070 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f8a8fe625b20524 server_cookie=686f0d3a83e075c7 in_seq=0 out_seq=0 2024-07-04T08:27:33.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.346+0000 7fdf44c00640 10 client.28207 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:33.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.347+0000 7fdf44c00640 1 -- 172.21.15.39:0/3465942987 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562e35499e10 con 0x562e3513bcd0 2024-07-04T08:27:33.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.347+0000 7fdf44c00640 10 client.28207 handle_command_reply: tid=0 2024-07-04T08:27:33.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 client.28207 shutdown 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 -- 172.21.15.39:0/3465942987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 msgr2=0x562e3552d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552d070 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdf480a9940 tx=0x7fdf480f3450 comp rx=0 tx=0).stop 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 -- 172.21.15.39:0/3465942987 shutdown_connections 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562e3513bcd0 0x562e3553f070 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3517d350 0x562e35539170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3515c670 0x562e3552d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf44c00640 0 client.28207 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 --2- 172.21.15.39:0/3465942987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552d070 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 -- 172.21.15.39:0/3465942987 >> 172.21.15.39:0/3465942987 conn(0x562e35171740 msgr2=0x562e355249d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 -- 172.21.15.39:0/3465942987 shutdown_connections 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 1 -- 172.21.15.39:0/3465942987 wait complete. 2024-07-04T08:27:33.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.348+0000 7fdf63dad740 20 client.28207 trim_cache size 0 max 16384 2024-07-04T08:27:33.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.368+0000 7fdf63dad740 1 Processor -- start 2024-07-04T08:27:33.358 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:33 smithi039 ceph-mon[32062]: pgmap v862: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 17 op/s 2024-07-04T08:27:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.369+0000 7fdf63dad740 1 -- start start 2024-07-04T08:27:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.369+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.369+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e355390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.369+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 0x562e35539620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.369+0000 7fdf63dad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b23e0 con 0x562e3515c670 2024-07-04T08:27:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.369+0000 7fdf63dad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b3f20 con 0x562e3517d350 2024-07-04T08:27:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.369+0000 7fdf63dad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b2ab0 con 0x562e35154490 2024-07-04T08:27:33.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf4fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e355390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf56a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf57400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 0x562e35539620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf56a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37722/0 (socket says 172.21.15.39:37722) 2024-07-04T08:27:33.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf4fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e355390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52088/0 (socket says 172.21.15.39:52088) 2024-07-04T08:27:33.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf56a00640 1 -- 172.21.15.39:0/2621139900 learned_addr learned my addr 172.21.15.39:0/2621139900 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf4fe00640 1 -- 172.21.15.39:0/2621139900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 msgr2=0x562e3552eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/2621139900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf4fe00640 1 -- 172.21.15.39:0/2621139900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 msgr2=0x562e35539620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:33.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/2621139900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 0x562e35539620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf4fe00640 1 -- 172.21.15.39:0/2621139900 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e3552da70 con 0x562e3515c670 2024-07-04T08:27:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf56a00640 1 --2- 172.21.15.39:0/2621139900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.370+0000 7fdf57400640 1 --2- 172.21.15.39:0/2621139900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 0x562e35539620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:27:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.371+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/2621139900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e355390e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf4002a910 tx=0x7fdf40031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3d2479135726a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.371+0000 7fdf47400640 1 -- 172.21.15.39:0/2621139900 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4004b070 con 0x562e3515c670 2024-07-04T08:27:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.371+0000 7fdf47400640 1 -- 172.21.15.39:0/2621139900 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf40004380 con 0x562e3515c670 2024-07-04T08:27:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.371+0000 7fdf47400640 1 -- 172.21.15.39:0/2621139900 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40035040 con 0x562e3515c670 2024-07-04T08:27:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.372+0000 7fdf63dad740 1 -- 172.21.15.39:0/2621139900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 msgr2=0x562e355390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:33.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.372+0000 7fdf63dad740 1 --2- 172.21.15.39:0/2621139900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e355390e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf4002a910 tx=0x7fdf40031400 comp rx=0 tx=0).stop 2024-07-04T08:27:33.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.372+0000 7fdf63dad740 1 -- 172.21.15.39:0/2621139900 shutdown_connections 2024-07-04T08:27:33.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.372+0000 7fdf63dad740 1 --2- 172.21.15.39:0/2621139900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e355390e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.372+0000 7fdf63dad740 1 --2- 172.21.15.39:0/2621139900 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 0x562e35539620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.372+0000 7fdf63dad740 1 --2- 172.21.15.39:0/2621139900 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e3552eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.372+0000 7fdf63dad740 1 -- 172.21.15.39:0/2621139900 >> 172.21.15.39:0/2621139900 conn(0x562e35171740 msgr2=0x562e3552e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:33.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.372+0000 7fdf63dad740 1 -- 172.21.15.39:0/2621139900 shutdown_connections 2024-07-04T08:27:33.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.372+0000 7fdf63dad740 1 -- 172.21.15.39:0/2621139900 wait complete. 2024-07-04T08:27:33.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.373+0000 7fdf63dad740 1 -- start start 2024-07-04T08:27:33.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.373+0000 7fdf63dad740 1 Processor -- start 2024-07-04T08:27:33.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.374+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e35540650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.374+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e35540b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.374+0000 7fdf63dad740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 0x562e355a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.374+0000 7fdf63dad740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b3850 con 0x562e3515c670 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.374+0000 7fdf63dad740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b3180 con 0x562e3517d350 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.374+0000 7fdf63dad740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581b03c0 con 0x562e35154490 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.374+0000 7fdf63dad740 10 client.0 _ll_register_callbacks cb 0x7fdf480e6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf4fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e35540b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf4fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e35540b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52094/0 (socket says 172.21.15.39:52094) 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf4fe00640 1 -- 172.21.15.39:0/791390512 learned_addr learned my addr 172.21.15.39:0/791390512 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf57400640 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 0x562e355a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf56a00640 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e35540650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf4fe00640 1 -- 172.21.15.39:0/791390512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 msgr2=0x562e35540650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e35540650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf4fe00640 1 -- 172.21.15.39:0/791390512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 msgr2=0x562e355a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 0x562e355a65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.375+0000 7fdf4fe00640 1 -- 172.21.15.39:0/791390512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e355a6b00 con 0x562e3515c670 2024-07-04T08:27:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.376+0000 7fdf4fe00640 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e35540b90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009230 tx=0x7fdf40005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6021beaf11f1c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.376+0000 7fdf44c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:27:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.376+0000 7fdf44c00640 1 -- 172.21.15.39:0/791390512 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40053020 con 0x562e3515c670 2024-07-04T08:27:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.376+0000 7fdf44c00640 1 -- 172.21.15.39:0/791390512 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7fdf40004950 con 0x562e3515c670 2024-07-04T08:27:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.376+0000 7fdf63dad740 1 -- 172.21.15.39:0/791390512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf40004380 con 0x562e3515c670 2024-07-04T08:27:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.376+0000 7fdf44c00640 1 -- 172.21.15.39:0/791390512 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf40035050 con 0x562e3515c670 2024-07-04T08:27:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.376+0000 7fdf63dad740 10 client.18156 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:33.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.376+0000 7fdf63dad740 1 -- 172.21.15.39:0/791390512 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562e355a7d50 con 0x562e3515c670 2024-07-04T08:27:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.377+0000 7fdf44c00640 1 -- 172.21.15.39:0/791390512 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf40004380 con 0x562e3515c670 2024-07-04T08:27:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.377+0000 7fdf44c00640 1 -- 172.21.15.39:0/791390512 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdf4004e070 con 0x562e3515c670 2024-07-04T08:27:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.377+0000 7fdf63dad740 10 client.18156 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.377+0000 7fdf63dad740 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:27:33.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.377+0000 7fdf63dad740 20 client.18156 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:33.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.377+0000 7fdf63dad740 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562e35111080 0x562e355aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:33.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.377+0000 7fdf63dad740 4 client.18156 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:27:33.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.377+0000 7fdf63dad740 1 -- 172.21.15.39:0/791390512 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562e354958b0 con 0x562e35111080 2024-07-04T08:27:33.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.378+0000 7fdf56a00640 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562e35111080 0x562e355aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:33.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.378+0000 7fdf56a00640 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562e35111080 0x562e355aa500 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64e337416f1f34f server_cookie=c88b5a68478eb842 in_seq=0 out_seq=0 2024-07-04T08:27:33.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.378+0000 7fdf44c00640 10 client.18156 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:33.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.380+0000 7fdf44c00640 1 -- 172.21.15.39:0/791390512 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562e354958b0 con 0x562e35111080 2024-07-04T08:27:33.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.380+0000 7fdf44c00640 10 client.18156 handle_command_reply: tid=0 2024-07-04T08:27:33.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.380+0000 7fdf63dad740 1 client.18156 shutdown 2024-07-04T08:27:33.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.380+0000 7fdf63dad740 1 -- 172.21.15.39:0/791390512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 msgr2=0x562e35540b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:33.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.380+0000 7fdf63dad740 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e35540b90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009230 tx=0x7fdf40005db0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.380+0000 7fdf63dad740 1 -- 172.21.15.39:0/791390512 shutdown_connections 2024-07-04T08:27:33.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.380+0000 7fdf63dad740 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562e35111080 0x562e355aa500 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.381+0000 7fdf63dad740 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562e3515c670 0x562e35540b90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.381+0000 7fdf63dad740 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562e3517d350 0x562e355a65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.381+0000 7fdf63dad740 1 --2- 172.21.15.39:0/791390512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e35154490 0x562e35540650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.381+0000 7fdf63dad740 1 -- 172.21.15.39:0/791390512 >> 172.21.15.39:0/791390512 conn(0x562e35171740 msgr2=0x562e3551dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:33.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.381+0000 7fdf44c00640 0 client.18156 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:33.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.381+0000 7fdf63dad740 1 -- 172.21.15.39:0/791390512 shutdown_connections 2024-07-04T08:27:33.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.381+0000 7fdf63dad740 1 -- 172.21.15.39:0/791390512 wait complete. 2024-07-04T08:27:33.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.381+0000 7fdf63dad740 20 client.18156 trim_cache size 0 max 16384 2024-07-04T08:27:33.371 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:27:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:27:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "012b5943-86d4-412a-b67a-3fd6d4f7f09b", 2024-07-04T08:27:33.372 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:27:33.372 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:27:33.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 -- 172.21.15.39:0/3637269389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf3c0b2960 msgr2=0x7fdf3c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf3c0b2960 0x7fdf3c0b4e70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdf50005c40 tx=0x7fdf50009500 comp rx=0 tx=0).stop 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 -- 172.21.15.39:0/3637269389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf58105100 msgr2=0x7fdf5819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf58105100 0x7fdf5819ba90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800ea60 tx=0x7fdf4800ef20 comp rx=0 tx=0).stop 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 -- 172.21.15.39:0/3637269389 shutdown_connections 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fdf3c0b2960 0x7fdf3c0b4e70 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdf5810aa40 0x7fdf5819c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf5810b390 0x7fdf581a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 --2- 172.21.15.39:0/3637269389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf58105100 0x7fdf5819ba90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.382+0000 7fdf57e00640 1 -- 172.21.15.39:0/3637269389 >> 172.21.15.39:0/3637269389 conn(0x7fdf58100bd0 msgr2=0x7fdf58102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.383+0000 7fdf57e00640 1 -- 172.21.15.39:0/3637269389 shutdown_connections 2024-07-04T08:27:33.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:33.383+0000 7fdf57e00640 1 -- 172.21.15.39:0/3637269389 wait complete. 2024-07-04T08:27:33.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:33 smithi133 ceph-mon[37050]: pgmap v862: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 17 op/s 2024-07-04T08:27:33.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:33 smithi050 ceph-mon[36577]: pgmap v862: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 82 KiB/s wr, 17 op/s 2024-07-04T08:27:34.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:27:34.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:34 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:27:34.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:34 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:34.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:34 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:27:34.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:34 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:34.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:34 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:27:34.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:34 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:34.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:34 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:27:34.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:34 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:34.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:34 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:27:34.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:34 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:34.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:34 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:27:34.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:34 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:27:34.723 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:27:35.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:35 smithi133 ceph-mon[37050]: pgmap v863: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 13 op/s 2024-07-04T08:27:35.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:35 smithi039 ceph-mon[32062]: pgmap v863: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 13 op/s 2024-07-04T08:27:35.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:35 smithi050 ceph-mon[36577]: pgmap v863: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 13 op/s 2024-07-04T08:27:37.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.048+0000 7f084b200640 1 -- 172.21.15.39:0/2931708351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084410b380 msgr2=0x7f0844111bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:37.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.048+0000 7f084b200640 1 --2- 172.21.15.39:0/2931708351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084410b380 0x7f0844111bb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0840009ca0 tx=0x7f084002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.048+0000 7f084b200640 1 -- 172.21.15.39:0/2931708351 shutdown_connections 2024-07-04T08:27:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.048+0000 7f084b200640 1 --2- 172.21.15.39:0/2931708351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084410aa30 0x7f084410ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.048+0000 7f084b200640 1 --2- 172.21.15.39:0/2931708351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084410b380 0x7f0844111bb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.048+0000 7f084b200640 1 --2- 172.21.15.39:0/2931708351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08441050f0 0x7f0844105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.048+0000 7f084b200640 1 -- 172.21.15.39:0/2931708351 >> 172.21.15.39:0/2931708351 conn(0x7f0844100b60 msgr2=0x7f0844102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.048+0000 7f084b200640 1 -- 172.21.15.39:0/2931708351 shutdown_connections 2024-07-04T08:27:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.048+0000 7f084b200640 1 -- 172.21.15.39:0/2931708351 wait complete. 2024-07-04T08:27:37.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.049+0000 7f084b200640 1 Processor -- start 2024-07-04T08:27:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f084b200640 1 -- start start 2024-07-04T08:27:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f084b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08441050f0 0x7f084419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f084b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084410aa30 0x7f084419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f084b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084410b380 0x7f08441a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f084b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0844115030 con 0x7f084410aa30 2024-07-04T08:27:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f084b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0844114eb0 con 0x7f08441050f0 2024-07-04T08:27:37.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f084b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08441151b0 con 0x7f084410b380 2024-07-04T08:27:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f0849400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084410aa30 0x7f084419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f084a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084410b380 0x7f08441a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f0849400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084410aa30 0x7f084419bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52110/0 (socket says 172.21.15.39:52110) 2024-07-04T08:27:37.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.050+0000 7f0849400640 1 -- 172.21.15.39:0/3924668146 learned_addr learned my addr 172.21.15.39:0/3924668146 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:37.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.051+0000 7f0849e00640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08441050f0 0x7f084419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.051+0000 7f0849400640 1 -- 172.21.15.39:0/3924668146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084410b380 msgr2=0x7f08441a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:37.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.051+0000 7f0849400640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084410b380 0x7f08441a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.051+0000 7f0849400640 1 -- 172.21.15.39:0/3924668146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08441050f0 msgr2=0x7f084419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:37.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.051+0000 7f0849400640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08441050f0 0x7f084419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.051+0000 7f0849400640 1 -- 172.21.15.39:0/3924668146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08441a1950 con 0x7f084410aa30 2024-07-04T08:27:37.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.051+0000 7f0849400640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084410aa30 0x7f084419bf80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f083400e9f0 tx=0x7f083400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:37.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.052+0000 7f083ea00640 1 -- 172.21.15.39:0/3924668146 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083400ccc0 con 0x7f084410aa30 2024-07-04T08:27:37.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.052+0000 7f083ea00640 1 -- 172.21.15.39:0/3924668146 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08340108e0 con 0x7f084410aa30 2024-07-04T08:27:37.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.052+0000 7f084b200640 1 -- 172.21.15.39:0/3924668146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08441a1c40 con 0x7f084410aa30 2024-07-04T08:27:37.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.052+0000 7f083ea00640 1 -- 172.21.15.39:0/3924668146 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834018860 con 0x7f084410aa30 2024-07-04T08:27:37.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.052+0000 7f084b200640 1 -- 172.21.15.39:0/3924668146 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08441a83a0 con 0x7f084410aa30 2024-07-04T08:27:37.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.056+0000 7f0852ed1740 1 Processor -- start 2024-07-04T08:27:37.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.057+0000 7f0852ed1740 1 -- start start 2024-07-04T08:27:37.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.057+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 0x558e45290910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.057+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e45290e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.057+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ecb390 0x558e45291390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.057+0000 7f0852ed1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558e45206ea0 con 0x558e44eec070 2024-07-04T08:27:37.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.057+0000 7f0852ed1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558e451f6440 con 0x558e44ecb390 2024-07-04T08:27:37.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.057+0000 7f0852ed1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558e451fa480 con 0x558e44ec31b0 2024-07-04T08:27:37.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f083ea00640 1 -- 172.21.15.39:0/3924668146 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0834010430 con 0x7f084410aa30 2024-07-04T08:27:37.047 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f084a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 0x558e45290910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f0849e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e45290e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f0849e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e45290e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37744/0 (socket says 172.21.15.39:37744) 2024-07-04T08:27:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f0849e00640 1 -- 172.21.15.39:0/1669480789 learned_addr learned my addr 172.21.15.39:0/1669480789 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f0849e00640 1 -- 172.21.15.39:0/1669480789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ecb390 msgr2=0x558e45291390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f0849e00640 1 --2- 172.21.15.39:0/1669480789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ecb390 0x558e45291390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f0849e00640 1 -- 172.21.15.39:0/1669480789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 msgr2=0x558e45290910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f0849e00640 1 --2- 172.21.15.39:0/1669480789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 0x558e45290910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.058+0000 7f0849e00640 1 -- 172.21.15.39:0/1669480789 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e4518e610 con 0x558e44ec31b0 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.059+0000 7f0849e00640 1 --2- 172.21.15.39:0/1669480789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e45290e50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0838009cf0 tx=0x7f0838011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=907204ca996b61be server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.059+0000 7f0833400640 1 -- 172.21.15.39:0/1669480789 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838004230 con 0x558e44ec31b0 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.059+0000 7f0833400640 1 -- 172.21.15.39:0/1669480789 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08380043d0 con 0x558e44ec31b0 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.059+0000 7f0833400640 1 -- 172.21.15.39:0/1669480789 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838019880 con 0x558e44ec31b0 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f083ea00640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f082c0b2960 0x7f082c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f083ea00640 1 -- 172.21.15.39:0/3924668146 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7f08340e2c40 con 0x7f084410aa30 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f0852ed1740 1 -- 172.21.15.39:0/1669480789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 msgr2=0x558e45290e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f0852ed1740 1 --2- 172.21.15.39:0/1669480789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e45290e50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0838009cf0 tx=0x7f0838011810 comp rx=0 tx=0).stop 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f084a800640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f082c0b2960 0x7f082c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f0852ed1740 1 -- 172.21.15.39:0/1669480789 shutdown_connections 2024-07-04T08:27:37.050 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f0852ed1740 1 --2- 172.21.15.39:0/1669480789 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ecb390 0x558e45291390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f0852ed1740 1 --2- 172.21.15.39:0/1669480789 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e45290e50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f0852ed1740 1 --2- 172.21.15.39:0/1669480789 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 0x558e45290910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.060+0000 7f0852ed1740 1 -- 172.21.15.39:0/1669480789 >> 172.21.15.39:0/1669480789 conn(0x558e44ee0460 msgr2=0x558e4528e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:37.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.061+0000 7f0852ed1740 1 -- 172.21.15.39:0/1669480789 shutdown_connections 2024-07-04T08:27:37.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.061+0000 7f0852ed1740 1 -- 172.21.15.39:0/1669480789 wait complete. 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.061+0000 7f084a800640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f082c0b2960 0x7f082c0b4e70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f084002f500 tx=0x7f084002f310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.061+0000 7f0852ed1740 1 -- start start 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.062+0000 7f0852ed1740 1 Processor -- start 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.062+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ec31b0 0x558e4529c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.062+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ecb390 0x558e4529cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.062+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44eec070 0x558e452a8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.062+0000 7f0852ed1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558e4505a700 con 0x558e44ecb390 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.062+0000 7f0852ed1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558e4519db70 con 0x558e44ec31b0 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.062+0000 7f0852ed1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558e4529f0f0 con 0x558e44eec070 2024-07-04T08:27:37.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.062+0000 7f0852ed1740 10 client.0 _ll_register_callbacks cb 0x558e451ef530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:37.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ecb390 0x558e4529cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ecb390 0x558e4529cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52116/0 (socket says 172.21.15.39:52116) 2024-07-04T08:27:37.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849400640 1 -- 172.21.15.39:0/2060623609 learned_addr learned my addr 172.21.15.39:0/2060623609 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:37.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f084a800640 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44eec070 0x558e452a8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849e00640 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ec31b0 0x558e4529c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849400640 1 -- 172.21.15.39:0/2060623609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44eec070 msgr2=0x558e452a8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:37.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849400640 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44eec070 0x558e452a8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849400640 1 -- 172.21.15.39:0/2060623609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ec31b0 msgr2=0x558e4529c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849400640 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ec31b0 0x558e4529c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849400640 1 -- 172.21.15.39:0/2060623609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e44f57650 con 0x558e44ecb390 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f084a800640 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44eec070 0x558e452a8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0849400640 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ecb390 0x558e4529cb40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f08340a9be0 tx=0x7f08340a96d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d9affb37fca151 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.063+0000 7f0827e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.064+0000 7f0827e00640 1 -- 172.21.15.39:0/2060623609 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834091040 con 0x558e44ecb390 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.064+0000 7f0827e00640 1 -- 172.21.15.39:0/2060623609 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7f08340a8eb0 con 0x558e44ecb390 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.064+0000 7f0852ed1740 1 -- 172.21.15.39:0/2060623609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08380043d0 con 0x558e44ecb390 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.064+0000 7f0827e00640 1 -- 172.21.15.39:0/2060623609 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f08340e04a0 con 0x558e44ecb390 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.064+0000 7f0852ed1740 10 client.18168 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.064+0000 7f0852ed1740 1 -- 172.21.15.39:0/2060623609 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e452ab210 con 0x558e44ecb390 2024-07-04T08:27:37.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.064+0000 7f0827e00640 1 -- 172.21.15.39:0/2060623609 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08380043d0 con 0x558e44ecb390 2024-07-04T08:27:37.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.065+0000 7f0827e00640 1 -- 172.21.15.39:0/2060623609 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f08340a8920 con 0x558e44ecb390 2024-07-04T08:27:37.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.065+0000 7f0852ed1740 10 client.18168 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:37.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.065+0000 7f0852ed1740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:27:37.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.065+0000 7f0852ed1740 20 client.18168 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:37.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.065+0000 7f0852ed1740 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558e44eaa9f0 0x558e452ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.065+0000 7f0852ed1740 4 client.18168 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:27:37.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.065+0000 7f0852ed1740 1 -- 172.21.15.39:0/2060623609 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e45208df0 con 0x558e44eaa9f0 2024-07-04T08:27:37.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.065+0000 7f0849e00640 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558e44eaa9f0 0x558e452ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.066+0000 7f0849e00640 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558e44eaa9f0 0x558e452ae780 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359fda37cce0767e server_cookie=ba542c3ceefaccf3 in_seq=0 out_seq=0 2024-07-04T08:27:37.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.066+0000 7f0827e00640 10 client.18168 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:37.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0827e00640 1 -- 172.21.15.39:0/2060623609 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558e45208df0 con 0x558e44eaa9f0 2024-07-04T08:27:37.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0827e00640 10 client.18168 handle_command_reply: tid=0 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0852ed1740 1 client.18168 shutdown 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0852ed1740 1 -- 172.21.15.39:0/2060623609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ecb390 msgr2=0x558e4529cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0852ed1740 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ecb390 0x558e4529cb40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f08340a9be0 tx=0x7f08340a96d0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0852ed1740 1 -- 172.21.15.39:0/2060623609 shutdown_connections 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0852ed1740 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558e44eaa9f0 0x558e452ae780 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0852ed1740 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ecb390 0x558e4529cb40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0852ed1740 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44eec070 0x558e452a8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0852ed1740 1 --2- 172.21.15.39:0/2060623609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ec31b0 0x558e4529c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0852ed1740 1 -- 172.21.15.39:0/2060623609 >> 172.21.15.39:0/2060623609 conn(0x558e44ee0460 msgr2=0x558e45293af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.067+0000 7f0827e00640 0 client.18168 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.068+0000 7f0852ed1740 1 -- 172.21.15.39:0/2060623609 shutdown_connections 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.068+0000 7f0852ed1740 1 -- 172.21.15.39:0/2060623609 wait complete. 2024-07-04T08:27:37.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.068+0000 7f0852ed1740 20 client.18168 trim_cache size 0 max 16384 2024-07-04T08:27:37.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.088+0000 7f0852ed1740 1 Processor -- start 2024-07-04T08:27:37.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.088+0000 7f0852ed1740 1 -- start start 2024-07-04T08:27:37.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e4529e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ecb390 0x558e4529b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 0x558e4529b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f0852ed1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f08441b23f0 con 0x558e44eec070 2024-07-04T08:27:37.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f0852ed1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f08441b3f30 con 0x558e44ecb390 2024-07-04T08:27:37.078 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f0852ed1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08441b2ac0 con 0x558e44ec31b0 2024-07-04T08:27:37.079 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f084a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 0x558e4529b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f084a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 0x558e4529b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:52122/0 (socket says 172.21.15.39:52122) 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f0849e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e4529e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f084a800640 1 -- 172.21.15.39:0/3676277084 learned_addr learned my addr 172.21.15.39:0/3676277084 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.089+0000 7f0849400640 1 --2- 172.21.15.39:0/3676277084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ecb390 0x558e4529b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.090+0000 7f0849e00640 1 -- 172.21.15.39:0/3676277084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ecb390 msgr2=0x558e4529b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.090+0000 7f0849e00640 1 --2- 172.21.15.39:0/3676277084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ecb390 0x558e4529b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.090+0000 7f0849e00640 1 -- 172.21.15.39:0/3676277084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 msgr2=0x558e4529b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.090+0000 7f0849e00640 1 --2- 172.21.15.39:0/3676277084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 0x558e4529b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.090+0000 7f0849e00640 1 -- 172.21.15.39:0/3676277084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e452aeaf0 con 0x558e44ec31b0 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.090+0000 7f0849e00640 1 --2- 172.21.15.39:0/3676277084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e4529e280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f083801b040 tx=0x7f0838011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12aae057d632363d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:37.080 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.090+0000 7f0832a00640 1 -- 172.21.15.39:0/3676277084 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838030070 con 0x558e44ec31b0 2024-07-04T08:27:37.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.091+0000 7f0832a00640 1 -- 172.21.15.39:0/3676277084 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0838005040 con 0x558e44ec31b0 2024-07-04T08:27:37.081 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.091+0000 7f0832a00640 1 -- 172.21.15.39:0/3676277084 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083802b600 con 0x558e44ec31b0 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.091+0000 7f0852ed1740 1 -- 172.21.15.39:0/3676277084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 msgr2=0x558e4529e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.092+0000 7f0852ed1740 1 --2- 172.21.15.39:0/3676277084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e4529e280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f083801b040 tx=0x7f0838011810 comp rx=0 tx=0).stop 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.092+0000 7f0852ed1740 1 -- 172.21.15.39:0/3676277084 shutdown_connections 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.092+0000 7f0852ed1740 1 --2- 172.21.15.39:0/3676277084 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44eec070 0x558e4529b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.092+0000 7f0852ed1740 1 --2- 172.21.15.39:0/3676277084 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44ecb390 0x558e4529b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.092+0000 7f0852ed1740 1 --2- 172.21.15.39:0/3676277084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ec31b0 0x558e4529e280 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.092+0000 7f0852ed1740 1 -- 172.21.15.39:0/3676277084 >> 172.21.15.39:0/3676277084 conn(0x558e44ee0460 msgr2=0x558e4529ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.092+0000 7f0852ed1740 1 -- 172.21.15.39:0/3676277084 shutdown_connections 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.092+0000 7f0852ed1740 1 -- 172.21.15.39:0/3676277084 wait complete. 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.092+0000 7f0852ed1740 1 -- start start 2024-07-04T08:27:37.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.093+0000 7f0852ed1740 1 Processor -- start 2024-07-04T08:27:37.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.093+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ec31b0 0x558e452a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.093+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ecb390 0x558e452a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.093+0000 7f0852ed1740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44eec070 0x558e4529fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.093+0000 7f0852ed1740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f08441b3860 con 0x558e44ec31b0 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.093+0000 7f0852ed1740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f08441b3190 con 0x558e44eec070 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.093+0000 7f0852ed1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08441b03d0 con 0x558e44ecb390 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.093+0000 7f0852ed1740 10 client.0 _ll_register_callbacks cb 0x7f08340a5b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f0849400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ecb390 0x558e452a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f0849e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ec31b0 0x558e452a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f0849400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ecb390 0x558e452a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37772/0 (socket says 172.21.15.39:37772) 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f0849400640 1 -- 172.21.15.39:0/477554608 learned_addr learned my addr 172.21.15.39:0/477554608 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f084a800640 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44eec070 0x558e4529fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f0849400640 1 -- 172.21.15.39:0/477554608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44eec070 msgr2=0x558e4529fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f0849400640 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44eec070 0x558e4529fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f0849400640 1 -- 172.21.15.39:0/477554608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ec31b0 msgr2=0x558e452a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f0849400640 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ec31b0 0x558e452a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.094+0000 7f0849400640 1 -- 172.21.15.39:0/477554608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e452a0460 con 0x558e44ecb390 2024-07-04T08:27:37.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.095+0000 7f0849400640 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ecb390 0x558e452a4c50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f083401f040 tx=0x7f08340c42e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cbf24ee9f27593a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:27:37.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.095+0000 7f0827e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:27:37.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.095+0000 7f0827e00640 1 -- 172.21.15.39:0/477554608 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834091040 con 0x558e44ecb390 2024-07-04T08:27:37.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.095+0000 7f0827e00640 1 -- 172.21.15.39:0/477554608 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9424+0+0 (secure 0 0 0) 0x7f08340a8eb0 con 0x558e44ecb390 2024-07-04T08:27:37.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.095+0000 7f0852ed1740 1 -- 172.21.15.39:0/477554608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0838005040 con 0x558e44ecb390 2024-07-04T08:27:37.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.095+0000 7f0827e00640 1 -- 172.21.15.39:0/477554608 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f08340a93c0 con 0x558e44ecb390 2024-07-04T08:27:37.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.095+0000 7f0852ed1740 10 client.28231 fetch_fsmap learned FSMap version 30 2024-07-04T08:27:37.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.095+0000 7f0852ed1740 1 -- 172.21.15.39:0/477554608 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e45316aa0 con 0x558e44ecb390 2024-07-04T08:27:37.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.095+0000 7f0827e00640 1 -- 172.21.15.39:0/477554608 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0838005040 con 0x558e44ecb390 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.096+0000 7f0827e00640 1 -- 172.21.15.39:0/477554608 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f08340cc0d0 con 0x558e44ecb390 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.096+0000 7f0852ed1740 10 client.28231 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.096+0000 7f0852ed1740 10 client.28231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.096+0000 7f0852ed1740 20 client.28231 populate_metadata read hostname 'smithi039' 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.096+0000 7f0852ed1740 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558e44e7fda0 0x558e45319260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.096+0000 7f0852ed1740 4 client.28231 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.096+0000 7f0852ed1740 1 -- 172.21.15.39:0/477554608 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e45204890 con 0x558e44e7fda0 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.096+0000 7f0849e00640 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558e44e7fda0 0x558e45319260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0849e00640 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558e44e7fda0 0x558e45319260 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb4031dc9a35ae5 server_cookie=8e56b03a813f3df5 in_seq=0 out_seq=0 2024-07-04T08:27:37.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0827e00640 10 client.28231 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0827e00640 1 -- 172.21.15.39:0/477554608 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558e45204890 con 0x558e44e7fda0 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0827e00640 10 client.28231 handle_command_reply: tid=0 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0852ed1740 1 client.28231 shutdown 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0852ed1740 1 -- 172.21.15.39:0/477554608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ecb390 msgr2=0x558e452a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0852ed1740 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ecb390 0x558e452a4c50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f083401f040 tx=0x7f08340c42e0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0852ed1740 1 -- 172.21.15.39:0/477554608 shutdown_connections 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0852ed1740 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558e44e7fda0 0x558e45319260 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0852ed1740 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558e44eec070 0x558e4529fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0852ed1740 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e44ecb390 0x558e452a4c50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0852ed1740 1 --2- 172.21.15.39:0/477554608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e44ec31b0 0x558e452a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0827e00640 0 client.28231 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.097+0000 7f0852ed1740 1 -- 172.21.15.39:0/477554608 >> 172.21.15.39:0/477554608 conn(0x558e44ee0460 msgr2=0x558e4529b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.098+0000 7f0852ed1740 1 -- 172.21.15.39:0/477554608 shutdown_connections 2024-07-04T08:27:37.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.098+0000 7f0852ed1740 1 -- 172.21.15.39:0/477554608 wait complete. 2024-07-04T08:27:37.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.098+0000 7f0852ed1740 20 client.28231 trim_cache size 0 max 16384 2024-07-04T08:27:37.090 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:27:37.090 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-04T08:27:37.090 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-07-04T08:27:37.090 INFO:teuthology.orchestra.run.smithi039.stdout: "012b5943-86d4-412a-b67a-3fd6d4f7f09b": { 2024-07-04T08:27:37.090 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-07-04T08:27:37.090 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "012b5943-86d4-412a-b67a-3fd6d4f7f09b", 2024-07-04T08:27:37.090 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-07-04T08:27:37.090 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:27:37.091 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:27:37.091 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:27:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.098+0000 7f084b200640 1 -- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f082c0b2960 msgr2=0x7f082c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.098+0000 7f084b200640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f082c0b2960 0x7f082c0b4e70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f084002f500 tx=0x7f084002f310 comp rx=0 tx=0).stop 2024-07-04T08:27:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.098+0000 7f084b200640 1 -- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084410aa30 msgr2=0x7f084419bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:27:37.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.098+0000 7f084b200640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084410aa30 0x7f084419bf80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f083400e9f0 tx=0x7f083400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.099+0000 7f084b200640 1 -- 172.21.15.39:0/3924668146 shutdown_connections 2024-07-04T08:27:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.099+0000 7f084b200640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f082c0b2960 0x7f082c0b4e70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.099+0000 7f084b200640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f084410aa30 0x7f084419bf80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.099+0000 7f084b200640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084410b380 0x7f08441a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.099+0000 7f084b200640 1 --2- 172.21.15.39:0/3924668146 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f08441050f0 0x7f084419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:27:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.099+0000 7f084b200640 1 -- 172.21.15.39:0/3924668146 >> 172.21.15.39:0/3924668146 conn(0x7f0844100b60 msgr2=0x7f0844102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:27:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.099+0000 7f084b200640 1 -- 172.21.15.39:0/3924668146 shutdown_connections 2024-07-04T08:27:37.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:27:37.099+0000 7f084b200640 1 -- 172.21.15.39:0/3924668146 wait complete. 2024-07-04T08:27:37.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:37 smithi039 ceph-mon[32062]: pgmap v864: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:27:37.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:37 smithi133 ceph-mon[37050]: pgmap v864: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:27:37.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:37 smithi050 ceph-mon[36577]: pgmap v864: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 15 op/s 2024-07-04T08:27:38.006 INFO:tasks.cephfs.filesystem:scrub status for tag:012b5943-86d4-412a-b67a-3fd6d4f7f09b - {'path': '/', 'tag': '012b5943-86d4-412a-b67a-3fd6d4f7f09b', 'options': 'recursive,force'} 2024-07-04T08:27:38.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:38 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:27:38.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:38 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:27:38.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:38 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:27:39.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:39 smithi133 ceph-mon[37050]: pgmap v865: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 17 op/s 2024-07-04T08:27:39.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:39 smithi039 ceph-mon[32062]: pgmap v865: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 17 op/s 2024-07-04T08:27:39.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:39 smithi050 ceph-mon[36577]: pgmap v865: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 87 KiB/s wr, 17 op/s 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:375/600: open fd = 3 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:376/600: open fd = 3 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:377/600: open fd = 3 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.920 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:378/600: open fd = 3 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:379/600: open fd = 3 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:380/600: open fd = 3 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.921 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:381/600: open fd = 3 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:382/600: open fd = 3 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:383/600: open fd = 3 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.922 INFO:tasks.workunit.client.0.smithi039.stdout:384/600: open fd = 3 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:385/600: open fd = 3 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:386/600: open fd = 3 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:387/600: open fd = 3 2024-07-04T08:27:40.923 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:388/600: open fd = 3 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:389/600: open fd = 3 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:390/600: open fd = 3 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.924 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:391/600: open fd = 3 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:392/600: open fd = 3 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:393/600: open fd = 3 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.925 INFO:tasks.workunit.client.0.smithi039.stdout:394/600: open fd = 3 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:395/600: open fd = 3 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:396/600: open fd = 3 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:397/600: open fd = 3 2024-07-04T08:27:40.926 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:398/600: open fd = 3 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:399/600: open fd = 3 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:400/600: open fd = 3 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.927 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:401/600: open fd = 3 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:402/600: open fd = 3 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:403/600: open fd = 3 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.928 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:404/600: open fd = 3 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:405/600: open fd = 3 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:406/600: open fd = 3 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:407/600: open fd = 3 2024-07-04T08:27:40.929 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:408/600: open fd = 3 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:409/600: open fd = 3 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:410/600: open fd = 3 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.930 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:411/600: open fd = 3 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:412/600: open fd = 3 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:413/600: open fd = 3 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.931 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:414/600: open fd = 3 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:415/600: open fd = 3 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:416/600: open fd = 3 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:417/600: open fd = 3 2024-07-04T08:27:40.932 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:418/600: open fd = 3 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:419/600: open fd = 3 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:420/600: open fd = 3 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.933 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:421/600: open fd = 3 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:422/600: open fd = 3 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:423/600: open fd = 3 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.934 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:424/600: open fd = 3 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:425/600: open fd = 3 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:426/600: open fd = 3 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.935 INFO:tasks.workunit.client.0.smithi039.stdout:427/600: open fd = 3 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:428/600: open fd = 3 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:429/600: open fd = 3 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:430/600: open fd = 3 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.936 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:431/600: open fd = 3 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:432/600: open fd = 3 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:433/600: open fd = 3 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.937 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:434/600: open fd = 3 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:435/600: open fd = 3 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:436/600: open fd = 3 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:27:40.938 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:27:41.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:41 smithi133 ceph-mon[37050]: pgmap v866: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-07-04T08:27:41.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:41 smithi039 ceph-mon[32062]: pgmap v866: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-07-04T08:27:41.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:41 smithi050 ceph-mon[36577]: pgmap v866: 129 pgs: 129 active+clean; 19 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-07-04T08:27:43.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:43 smithi133 ceph-mon[37050]: pgmap v867: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 18 op/s 2024-07-04T08:27:43.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:43 smithi039 ceph-mon[32062]: pgmap v867: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 18 op/s 2024-07-04T08:27:43.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:43 smithi050 ceph-mon[36577]: pgmap v867: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 83 KiB/s wr, 18 op/s 2024-07-04T08:27:45.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:45 smithi133 ceph-mon[37050]: pgmap v868: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 13 op/s 2024-07-04T08:27:45.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:45 smithi039 ceph-mon[32062]: pgmap v868: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 13 op/s 2024-07-04T08:27:45.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:45 smithi050 ceph-mon[36577]: pgmap v868: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 13 op/s 2024-07-04T08:27:47.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:47 smithi133 ceph-mon[37050]: pgmap v869: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-07-04T08:27:47.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:47 smithi039 ceph-mon[32062]: pgmap v869: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-07-04T08:27:47.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:47 smithi050 ceph-mon[36577]: pgmap v869: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-07-04T08:27:49.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:49 smithi133 ceph-mon[37050]: pgmap v870: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 17 op/s 2024-07-04T08:27:49.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:49 smithi039 ceph-mon[32062]: pgmap v870: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 17 op/s 2024-07-04T08:27:49.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:49 smithi050 ceph-mon[36577]: pgmap v870: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 86 KiB/s wr, 17 op/s 2024-07-04T08:27:51.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:51 smithi133 ceph-mon[37050]: pgmap v871: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-07-04T08:27:51.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:51 smithi039 ceph-mon[32062]: pgmap v871: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-07-04T08:27:51.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:51 smithi050 ceph-mon[36577]: pgmap v871: 129 pgs: 129 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-07-04T08:27:53.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:53 smithi133 ceph-mon[37050]: pgmap v872: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-07-04T08:27:53.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:53 smithi039 ceph-mon[32062]: pgmap v872: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-07-04T08:27:53.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:53 smithi050 ceph-mon[36577]: pgmap v872: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 76 KiB/s wr, 17 op/s 2024-07-04T08:27:55.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:55 smithi133 ceph-mon[37050]: pgmap v873: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 50 KiB/s wr, 13 op/s 2024-07-04T08:27:55.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:55 smithi039 ceph-mon[32062]: pgmap v873: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 50 KiB/s wr, 13 op/s 2024-07-04T08:27:55.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:55 smithi050 ceph-mon[36577]: pgmap v873: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 50 KiB/s wr, 13 op/s 2024-07-04T08:27:57.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:57 smithi133 ceph-mon[37050]: pgmap v874: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:27:57.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:57 smithi039 ceph-mon[32062]: pgmap v874: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:27:57.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:57 smithi050 ceph-mon[36577]: pgmap v874: 129 pgs: 129 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 50 KiB/s wr, 15 op/s 2024-07-04T08:27:59.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:27:59 smithi133 ceph-mon[37050]: pgmap v875: 129 pgs: 129 active+clean; 21 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 79 KiB/s wr, 17 op/s 2024-07-04T08:27:59.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:27:59 smithi039 ceph-mon[32062]: pgmap v875: 129 pgs: 129 active+clean; 21 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 79 KiB/s wr, 17 op/s 2024-07-04T08:27:59.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:27:59 smithi050 ceph-mon[36577]: pgmap v875: 129 pgs: 129 active+clean; 21 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 79 KiB/s wr, 17 op/s 2024-07-04T08:28:00.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:00 smithi050 ceph-mon[36577]: pgmap v876: 129 pgs: 129 active+clean; 21 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-07-04T08:28:00.804 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:00 smithi133 ceph-mon[37050]: pgmap v876: 129 pgs: 129 active+clean; 21 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-07-04T08:28:00.839 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:00 smithi039 ceph-mon[32062]: pgmap v876: 129 pgs: 129 active+clean; 21 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-07-04T08:28:03.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:03 smithi133 ceph-mon[37050]: pgmap v877: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 80 KiB/s wr, 18 op/s 2024-07-04T08:28:03.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:03 smithi039 ceph-mon[32062]: pgmap v877: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 80 KiB/s wr, 18 op/s 2024-07-04T08:28:03.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:03 smithi050 ceph-mon[36577]: pgmap v877: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 80 KiB/s wr, 18 op/s 2024-07-04T08:28:04.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:04 smithi133 ceph-mon[37050]: osdmap e105: 12 total, 12 up, 12 in 2024-07-04T08:28:04.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:04 smithi039 ceph-mon[32062]: osdmap e105: 12 total, 12 up, 12 in 2024-07-04T08:28:04.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:04 smithi050 ceph-mon[36577]: osdmap e105: 12 total, 12 up, 12 in 2024-07-04T08:28:05.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:05 smithi133 ceph-mon[37050]: pgmap v879: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-07-04T08:28:05.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:05 smithi039 ceph-mon[32062]: pgmap v879: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-07-04T08:28:05.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:05 smithi050 ceph-mon[36577]: pgmap v879: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-07-04T08:28:07.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:07 smithi133 ceph-mon[37050]: pgmap v880: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 36 op/s 2024-07-04T08:28:07.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:07 smithi039 ceph-mon[32062]: pgmap v880: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 36 op/s 2024-07-04T08:28:07.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:07 smithi050 ceph-mon[36577]: pgmap v880: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 123 KiB/s wr, 36 op/s 2024-07-04T08:28:08.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:28:08.497 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:09.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:09 smithi039 ceph-mon[32062]: pgmap v881: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 118 KiB/s wr, 35 op/s 2024-07-04T08:28:09.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:09 smithi039 ceph-mon[32062]: osdmap e106: 12 total, 12 up, 12 in 2024-07-04T08:28:09.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:28:09.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:09 smithi133 ceph-mon[37050]: pgmap v881: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 118 KiB/s wr, 35 op/s 2024-07-04T08:28:09.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:09 smithi133 ceph-mon[37050]: osdmap e106: 12 total, 12 up, 12 in 2024-07-04T08:28:09.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:28:09.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:09 smithi050 ceph-mon[36577]: pgmap v881: 129 pgs: 129 active+clean; 21 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 118 KiB/s wr, 35 op/s 2024-07-04T08:28:09.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:09 smithi050 ceph-mon[36577]: osdmap e106: 12 total, 12 up, 12 in 2024-07-04T08:28:09.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:28:10.495 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.495 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.495 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.495 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.495 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:28:10.496 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:28:10.496 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:28:10.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:28:10.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:28:10.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:28:10.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:28:10.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:28:10.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:28:10.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:28:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.847+0000 7f116fe00640 1 -- 172.21.15.39:0/3066319480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11701045f0 msgr2=0x7f117010cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:10.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.847+0000 7f116fe00640 1 --2- 172.21.15.39:0/3066319480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11701045f0 0x7f117010cc30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1158009ca0 tx=0x7f115802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.847+0000 7f116fe00640 1 -- 172.21.15.39:0/3066319480 shutdown_connections 2024-07-04T08:28:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.847+0000 7f116fe00640 1 --2- 172.21.15.39:0/3066319480 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11701045f0 0x7f117010cc30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.847+0000 7f116fe00640 1 --2- 172.21.15.39:0/3066319480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f117010d170 0x7f117010f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.847+0000 7f116fe00640 1 --2- 172.21.15.39:0/3066319480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1170103bc0 0x7f1170103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.847+0000 7f116fe00640 1 -- 172.21.15.39:0/3066319480 >> 172.21.15.39:0/3066319480 conn(0x7f11700fd810 msgr2=0x7f11700ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.847+0000 7f116fe00640 1 -- 172.21.15.39:0/3066319480 shutdown_connections 2024-07-04T08:28:10.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.847+0000 7f116fe00640 1 -- 172.21.15.39:0/3066319480 wait complete. 2024-07-04T08:28:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.849+0000 7f116fe00640 1 Processor -- start 2024-07-04T08:28:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.849+0000 7f116fe00640 1 -- start start 2024-07-04T08:28:10.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.849+0000 7f116fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1170103bc0 0x7f117019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.849+0000 7f116fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11701045f0 0x7f117019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.849+0000 7f116fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f117010d170 0x7f11701a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.849+0000 7f116fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112d90 con 0x7f1170103bc0 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.849+0000 7f116fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112c10 con 0x7f117010d170 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.849+0000 7f116fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1170112f10 con 0x7f11701045f0 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1170103bc0 0x7f117019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11701045f0 0x7f117019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11701045f0 0x7f117019bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:47720/0 (socket says 172.21.15.39:47720) 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1170103bc0 0x7f117019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45978/0 (socket says 172.21.15.39:45978) 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116e000640 1 -- 172.21.15.39:0/272936798 learned_addr learned my addr 172.21.15.39:0/272936798 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:10.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116f400640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f117010d170 0x7f11701a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116f400640 1 -- 172.21.15.39:0/272936798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11701045f0 msgr2=0x7f117019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:10.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116f400640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11701045f0 0x7f117019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116f400640 1 -- 172.21.15.39:0/272936798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1170103bc0 msgr2=0x7f117019ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116f400640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1170103bc0 0x7f117019ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.850+0000 7f116f400640 1 -- 172.21.15.39:0/272936798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11701a1990 con 0x7f117010d170 2024-07-04T08:28:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.851+0000 7f116f400640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f117010d170 0x7f11701a1260 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f116800e970 tx=0x7f116800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.851+0000 7f1167400640 1 -- 172.21.15.39:0/272936798 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800ccc0 con 0x7f117010d170 2024-07-04T08:28:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.851+0000 7f1167400640 1 -- 172.21.15.39:0/272936798 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11680035f0 con 0x7f117010d170 2024-07-04T08:28:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.851+0000 7f116fe00640 1 -- 172.21.15.39:0/272936798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11701a1c80 con 0x7f117010d170 2024-07-04T08:28:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.851+0000 7f1167400640 1 -- 172.21.15.39:0/272936798 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116800ce60 con 0x7f117010d170 2024-07-04T08:28:10.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.851+0000 7f116fe00640 1 -- 172.21.15.39:0/272936798 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701a83e0 con 0x7f117010d170 2024-07-04T08:28:10.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.854+0000 7f117bdb3740 1 Processor -- start 2024-07-04T08:28:10.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.855+0000 7f1167400640 1 -- 172.21.15.39:0/272936798 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1168004020 con 0x7f117010d170 2024-07-04T08:28:10.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.855+0000 7f117bdb3740 1 -- start start 2024-07-04T08:28:10.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x559915815910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5599154481b0 0x559915815e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559915450390 0x559915816390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f117bdb3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55991578bea0 con 0x559915450390 2024-07-04T08:28:10.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f117bdb3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55991577b440 con 0x5599154481b0 2024-07-04T08:28:10.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f117bdb3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55991577f480 con 0x559915471070 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f116ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5599154481b0 0x559915815e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f116e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x559915815910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f116ea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5599154481b0 0x559915815e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:48524/0 (socket says 172.21.15.39:48524) 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f116ea00640 1 -- 172.21.15.39:0/3921709949 learned_addr learned my addr 172.21.15.39:0/3921709949 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.856+0000 7f116e000640 1 --2- 172.21.15.39:0/3921709949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559915450390 0x559915816390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.857+0000 7f116ea00640 1 -- 172.21.15.39:0/3921709949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 msgr2=0x559915815910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.857+0000 7f116ea00640 1 --2- 172.21.15.39:0/3921709949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x559915815910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.857+0000 7f116ea00640 1 -- 172.21.15.39:0/3921709949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559915450390 msgr2=0x559915816390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.857+0000 7f116ea00640 1 --2- 172.21.15.39:0/3921709949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559915450390 0x559915816390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.857+0000 7f116ea00640 1 -- 172.21.15.39:0/3921709949 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559915713610 con 0x5599154481b0 2024-07-04T08:28:10.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.857+0000 7f1167400640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f11500b2960 0x7f11500b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.857+0000 7f116ea00640 1 --2- 172.21.15.39:0/3921709949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5599154481b0 0x559915815e50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1160009cf0 tx=0x7f1160011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a2f8143e0e28cc3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.857+0000 7f1167400640 1 -- 172.21.15.39:0/272936798 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f11680e2ba0 con 0x7f117010d170 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.857+0000 7f116e000640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f11500b2960 0x7f11500b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.858+0000 7f115fe00640 1 -- 172.21.15.39:0/3921709949 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116001e070 con 0x5599154481b0 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.858+0000 7f115fe00640 1 -- 172.21.15.39:0/3921709949 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1160019ae0 con 0x5599154481b0 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.858+0000 7f115fe00640 1 -- 172.21.15.39:0/3921709949 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116000e9e0 con 0x5599154481b0 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f117bdb3740 1 -- 172.21.15.39:0/3921709949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5599154481b0 msgr2=0x559915815e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f116e000640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f11500b2960 0x7f11500b4e70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1158005cc0 tx=0x7f115803f040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f117bdb3740 1 --2- 172.21.15.39:0/3921709949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5599154481b0 0x559915815e50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1160009cf0 tx=0x7f1160011810 comp rx=0 tx=0).stop 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f117bdb3740 1 -- 172.21.15.39:0/3921709949 shutdown_connections 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f117bdb3740 1 --2- 172.21.15.39:0/3921709949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559915450390 0x559915816390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f117bdb3740 1 --2- 172.21.15.39:0/3921709949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5599154481b0 0x559915815e50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f117bdb3740 1 --2- 172.21.15.39:0/3921709949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x559915815910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f117bdb3740 1 -- 172.21.15.39:0/3921709949 >> 172.21.15.39:0/3921709949 conn(0x559915465460 msgr2=0x559915813800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:10.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f117bdb3740 1 -- 172.21.15.39:0/3921709949 shutdown_connections 2024-07-04T08:28:10.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.859+0000 7f117bdb3740 1 -- 172.21.15.39:0/3921709949 wait complete. 2024-07-04T08:28:10.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.860+0000 7f117bdb3740 1 -- start start 2024-07-04T08:28:10.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.860+0000 7f117bdb3740 1 Processor -- start 2024-07-04T08:28:10.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x5599158214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 0x5599158219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x55991582d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f117bdb3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5599155df700 con 0x5599154481b0 2024-07-04T08:28:10.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f117bdb3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559915722b70 con 0x559915450390 2024-07-04T08:28:10.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f117bdb3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5599158240f0 con 0x559915471070 2024-07-04T08:28:10.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f117bdb3740 10 client.0 _ll_register_callbacks cb 0x559915774530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:10.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f116ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x5599158214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f116ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x5599158214b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45992/0 (socket says 172.21.15.39:45992) 2024-07-04T08:28:10.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f116ea00640 1 -- 172.21.15.39:0/2272900312 learned_addr learned my addr 172.21.15.39:0/2272900312 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:10.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f116e000640 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 0x5599158219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.861+0000 7f116f400640 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x55991582d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f116ea00640 1 -- 172.21.15.39:0/2272900312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 msgr2=0x55991582d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f116ea00640 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x55991582d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f116ea00640 1 -- 172.21.15.39:0/2272900312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 msgr2=0x5599158219f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f116ea00640 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 0x5599158219f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f116ea00640 1 -- 172.21.15.39:0/2272900312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599154dc650 con 0x5599154481b0 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f116e000640 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 0x5599158219f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f116ea00640 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x5599158214b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f116000e600 tx=0x7f1160009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c95d3560e74c9e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f115cc00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f115cc00640 1 -- 172.21.15.39:0/2272900312 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116001e040 con 0x5599154481b0 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f117bdb3740 1 -- 172.21.15.39:0/2272900312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1160019ae0 con 0x5599154481b0 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.862+0000 7f115cc00640 1 -- 172.21.15.39:0/2272900312 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f1160019840 con 0x5599154481b0 2024-07-04T08:28:10.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.863+0000 7f115cc00640 1 -- 172.21.15.39:0/2272900312 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1160004030 con 0x5599154481b0 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.863+0000 7f117bdb3740 10 client.18186 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.863+0000 7f117bdb3740 1 -- 172.21.15.39:0/2272900312 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559915830160 con 0x5599154481b0 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.863+0000 7f115cc00640 1 -- 172.21.15.39:0/2272900312 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1160019ae0 con 0x5599154481b0 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.863+0000 7f115cc00640 1 -- 172.21.15.39:0/2272900312 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f116000e760 con 0x5599154481b0 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.863+0000 7f117bdb3740 10 client.18186 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.863+0000 7f117bdb3740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.864+0000 7f117bdb3740 20 client.18186 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.864+0000 7f117bdb3740 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55991542f9f0 0x5599158334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.864+0000 7f117bdb3740 4 client.18186 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.864+0000 7f117bdb3740 1 -- 172.21.15.39:0/2272900312 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55991578ddf0 con 0x55991542f9f0 2024-07-04T08:28:10.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.864+0000 7f116e000640 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55991542f9f0 0x5599158334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.865+0000 7f116e000640 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55991542f9f0 0x5599158334c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8f6a1ceec3f08b server_cookie=cdc450dc702398cb in_seq=0 out_seq=0 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.865+0000 7f115cc00640 10 client.18186 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.865+0000 7f115cc00640 1 -- 172.21.15.39:0/2272900312 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55991578ddf0 con 0x55991542f9f0 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.865+0000 7f115cc00640 10 client.18186 handle_command_reply: tid=0 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.865+0000 7f117bdb3740 1 client.18186 shutdown 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 -- 172.21.15.39:0/2272900312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 msgr2=0x5599158214b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x5599158214b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f116000e600 tx=0x7f1160009a70 comp rx=0 tx=0).stop 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 -- 172.21.15.39:0/2272900312 shutdown_connections 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55991542f9f0 0x5599158334c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x55991582d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 0x5599158219f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 --2- 172.21.15.39:0/2272900312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x5599158214b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 -- 172.21.15.39:0/2272900312 >> 172.21.15.39:0/2272900312 conn(0x559915465460 msgr2=0x559915818aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:10.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f115cc00640 0 client.18186 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:10.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 -- 172.21.15.39:0/2272900312 shutdown_connections 2024-07-04T08:28:10.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 1 -- 172.21.15.39:0/2272900312 wait complete. 2024-07-04T08:28:10.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.866+0000 7f117bdb3740 20 client.18186 trim_cache size 0 max 16384 2024-07-04T08:28:10.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.886+0000 7f117bdb3740 1 Processor -- start 2024-07-04T08:28:10.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.886+0000 7f117bdb3740 1 -- start start 2024-07-04T08:28:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x55991581ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 0x5599158193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x559915822ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f117bdb3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f11701b2430 con 0x5599154481b0 2024-07-04T08:28:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f117bdb3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f11701b3f70 con 0x559915450390 2024-07-04T08:28:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f117bdb3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f11701b2b00 con 0x559915471070 2024-07-04T08:28:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f116ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x55991581ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f116e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 0x5599158193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f116f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x559915822ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f116ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x55991581ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46004/0 (socket says 172.21.15.39:46004) 2024-07-04T08:28:10.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.887+0000 7f116ea00640 1 -- 172.21.15.39:0/1739086208 learned_addr learned my addr 172.21.15.39:0/1739086208 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f116f400640 1 -- 172.21.15.39:0/1739086208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 msgr2=0x5599158193d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f116f400640 1 --2- 172.21.15.39:0/1739086208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 0x5599158193d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f116f400640 1 -- 172.21.15.39:0/1739086208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 msgr2=0x55991581ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f116f400640 1 --2- 172.21.15.39:0/1739086208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x55991581ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f116f400640 1 -- 172.21.15.39:0/1739086208 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599158140f0 con 0x559915471070 2024-07-04T08:28:10.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f116ea00640 1 --2- 172.21.15.39:0/1739086208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x55991581ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f116f400640 1 --2- 172.21.15.39:0/1739086208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x559915822ee0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f11680dff70 tx=0x7f11680a5af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92549dcd9938278b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f115f400640 1 -- 172.21.15.39:0/1739086208 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680c3070 con 0x559915471070 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f115f400640 1 -- 172.21.15.39:0/1739086208 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11680aa200 con 0x559915471070 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.888+0000 7f115f400640 1 -- 172.21.15.39:0/1739086208 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680e01f0 con 0x559915471070 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.889+0000 7f117bdb3740 1 -- 172.21.15.39:0/1739086208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 msgr2=0x559915822ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.889+0000 7f117bdb3740 1 --2- 172.21.15.39:0/1739086208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x559915822ee0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f11680dff70 tx=0x7f11680a5af0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.889+0000 7f117bdb3740 1 -- 172.21.15.39:0/1739086208 shutdown_connections 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.889+0000 7f117bdb3740 1 --2- 172.21.15.39:0/1739086208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915471070 0x559915822ee0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.889+0000 7f117bdb3740 1 --2- 172.21.15.39:0/1739086208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915450390 0x5599158193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.889+0000 7f117bdb3740 1 --2- 172.21.15.39:0/1739086208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x55991581ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.889+0000 7f117bdb3740 1 -- 172.21.15.39:0/1739086208 >> 172.21.15.39:0/1739086208 conn(0x559915465460 msgr2=0x5599158227d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.889+0000 7f117bdb3740 1 -- 172.21.15.39:0/1739086208 shutdown_connections 2024-07-04T08:28:10.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.889+0000 7f117bdb3740 1 -- 172.21.15.39:0/1739086208 wait complete. 2024-07-04T08:28:10.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.890+0000 7f117bdb3740 1 -- start start 2024-07-04T08:28:10.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.890+0000 7f117bdb3740 1 Processor -- start 2024-07-04T08:28:10.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x559915824360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915450390 0x5599158248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f117bdb3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915471070 0x559915824de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f117bdb3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f11701b38a0 con 0x5599154481b0 2024-07-04T08:28:10.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f117bdb3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f11701b31d0 con 0x559915471070 2024-07-04T08:28:10.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f117bdb3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f11701b0410 con 0x559915450390 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f117bdb3740 10 client.0 _ll_register_callbacks cb 0x7f116001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f116ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x559915824360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f116e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915450390 0x5599158248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f116ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x559915824360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:46018/0 (socket says 172.21.15.39:46018) 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f116ea00640 1 -- 172.21.15.39:0/990337172 learned_addr learned my addr 172.21.15.39:0/990337172 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.891+0000 7f116f400640 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915471070 0x559915824de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f116ea00640 1 -- 172.21.15.39:0/990337172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915450390 msgr2=0x5599158248a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f116ea00640 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915450390 0x5599158248a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f116ea00640 1 -- 172.21.15.39:0/990337172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915471070 msgr2=0x559915824de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f116ea00640 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915471070 0x559915824de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f116ea00640 1 -- 172.21.15.39:0/990337172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559915899bd0 con 0x5599154481b0 2024-07-04T08:28:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f116ea00640 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x559915824360 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f116000eb80 tx=0x7f1160009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e1a7a47a06139e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f115cc00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:28:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f115cc00640 1 -- 172.21.15.39:0/990337172 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116003b020 con 0x5599154481b0 2024-07-04T08:28:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f115cc00640 1 -- 172.21.15.39:0/990337172 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f11600041d0 con 0x5599154481b0 2024-07-04T08:28:10.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.892+0000 7f117bdb3740 1 -- 172.21.15.39:0/990337172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11680aa200 con 0x5599154481b0 2024-07-04T08:28:10.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f115cc00640 1 -- 172.21.15.39:0/990337172 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1160046050 con 0x5599154481b0 2024-07-04T08:28:10.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f117bdb3740 10 client.18192 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:10.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f117bdb3740 1 -- 172.21.15.39:0/990337172 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55991589ae40 con 0x5599154481b0 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f115cc00640 1 -- 172.21.15.39:0/990337172 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11680aa200 con 0x5599154481b0 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f115cc00640 1 -- 172.21.15.39:0/990337172 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1160019b80 con 0x5599154481b0 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f117bdb3740 10 client.18192 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f117bdb3740 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f117bdb3740 20 client.18192 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f117bdb3740 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559915404da0 0x55991589d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f117bdb3740 4 client.18192 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.893+0000 7f117bdb3740 1 -- 172.21.15.39:0/990337172 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559915789890 con 0x559915404da0 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.894+0000 7f116e000640 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559915404da0 0x55991589d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.894+0000 7f116e000640 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559915404da0 0x55991589d680 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e5f1bb8b2d8802 server_cookie=a24bfc1d4bfc6b0c in_seq=0 out_seq=0 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.894+0000 7f115cc00640 10 client.18192 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f115cc00640 1 -- 172.21.15.39:0/990337172 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559915789890 con 0x559915404da0 2024-07-04T08:28:10.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f115cc00640 10 client.18192 handle_command_reply: tid=0 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 client.18192 shutdown 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 -- 172.21.15.39:0/990337172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 msgr2=0x559915824360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x559915824360 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f116000eb80 tx=0x7f1160009f80 comp rx=0 tx=0).stop 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 -- 172.21.15.39:0/990337172 shutdown_connections 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x559915404da0 0x55991589d680 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559915471070 0x559915824de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559915450390 0x5599158248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 --2- 172.21.15.39:0/990337172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5599154481b0 0x559915824360 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f115cc00640 0 client.18192 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 -- 172.21.15.39:0/990337172 >> 172.21.15.39:0/990337172 conn(0x559915465460 msgr2=0x559915819020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 -- 172.21.15.39:0/990337172 shutdown_connections 2024-07-04T08:28:10.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 1 -- 172.21.15.39:0/990337172 wait complete. 2024-07-04T08:28:10.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.895+0000 7f117bdb3740 20 client.18192 trim_cache size 0 max 16384 2024-07-04T08:28:10.887 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:28:10.887 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:28:10.887 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:28:10.887 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 -- 172.21.15.39:0/272936798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f11500b2960 msgr2=0x7f11500b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f11500b2960 0x7f11500b4e70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1158005cc0 tx=0x7f115803f040 comp rx=0 tx=0).stop 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 -- 172.21.15.39:0/272936798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f117010d170 msgr2=0x7f11701a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f117010d170 0x7f11701a1260 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f116800e970 tx=0x7f116800ee30 comp rx=0 tx=0).stop 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 -- 172.21.15.39:0/272936798 shutdown_connections 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f11500b2960 0x7f11500b4e70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f117010d170 0x7f11701a1260 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11701045f0 0x7f117019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 --2- 172.21.15.39:0/272936798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1170103bc0 0x7f117019ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.896+0000 7f116fe00640 1 -- 172.21.15.39:0/272936798 >> 172.21.15.39:0/272936798 conn(0x7f11700fd810 msgr2=0x7f1170101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:10.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.897+0000 7f116fe00640 1 -- 172.21.15.39:0/272936798 shutdown_connections 2024-07-04T08:28:10.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:10.897+0000 7f116fe00640 1 -- 172.21.15.39:0/272936798 wait complete. 2024-07-04T08:28:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:11 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi039 to 176.4M 2024-07-04T08:28:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:11 smithi133 ceph-mon[37050]: Unable to set osd_memory_target on smithi039 to 185068441: error parsing value: Value '185068441' is below minimum 939524096 2024-07-04T08:28:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:11 smithi133 ceph-mon[37050]: pgmap v883: 129 pgs: 129 active+clean; 22 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 141 KiB/s wr, 39 op/s 2024-07-04T08:28:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:11 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi039 to 176.4M 2024-07-04T08:28:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:11 smithi039 ceph-mon[32062]: Unable to set osd_memory_target on smithi039 to 185068441: error parsing value: Value '185068441' is below minimum 939524096 2024-07-04T08:28:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:11 smithi039 ceph-mon[32062]: pgmap v883: 129 pgs: 129 active+clean; 22 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 141 KiB/s wr, 39 op/s 2024-07-04T08:28:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:11 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi039 to 176.4M 2024-07-04T08:28:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:11 smithi050 ceph-mon[36577]: Unable to set osd_memory_target on smithi039 to 185068441: error parsing value: Value '185068441' is below minimum 939524096 2024-07-04T08:28:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:11 smithi050 ceph-mon[36577]: pgmap v883: 129 pgs: 129 active+clean; 22 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 141 KiB/s wr, 39 op/s 2024-07-04T08:28:11.753 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:28:11.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:28:12.248 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:13.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:13 smithi133 ceph-mon[37050]: pgmap v884: 129 pgs: 129 active+clean; 22 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 127 KiB/s wr, 39 op/s 2024-07-04T08:28:13.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:13 smithi039 ceph-mon[32062]: pgmap v884: 129 pgs: 129 active+clean; 22 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 127 KiB/s wr, 39 op/s 2024-07-04T08:28:13.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:13 smithi050 ceph-mon[36577]: pgmap v884: 129 pgs: 129 active+clean; 22 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 127 KiB/s wr, 39 op/s 2024-07-04T08:28:14.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.578+0000 7f6838c00640 1 -- 172.21.15.39:0/2419532744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6834103ba0 msgr2=0x7f6834103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.578+0000 7f6838c00640 1 --2- 172.21.15.39:0/2419532744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6834103ba0 0x7f6834103fb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6824009be0 tx=0x7f682402a6c0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.578+0000 7f6838c00640 1 -- 172.21.15.39:0/2419532744 shutdown_connections 2024-07-04T08:28:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.578+0000 7f6838c00640 1 --2- 172.21.15.39:0/2419532744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f683410d150 0x7f683410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.578+0000 7f6838c00640 1 --2- 172.21.15.39:0/2419532744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68341045d0 0x7f683410cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.578+0000 7f6838c00640 1 --2- 172.21.15.39:0/2419532744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6834103ba0 0x7f6834103fb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.578+0000 7f6838c00640 1 -- 172.21.15.39:0/2419532744 >> 172.21.15.39:0/2419532744 conn(0x7f68340fd810 msgr2=0x7f68340ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.578+0000 7f6838c00640 1 -- 172.21.15.39:0/2419532744 shutdown_connections 2024-07-04T08:28:14.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.578+0000 7f6838c00640 1 -- 172.21.15.39:0/2419532744 wait complete. 2024-07-04T08:28:14.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.580+0000 7f6838c00640 1 Processor -- start 2024-07-04T08:28:14.569 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.580+0000 7f6838c00640 1 -- start start 2024-07-04T08:28:14.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6838c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834103ba0 0x7f6834101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:14.570 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6838c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68341045d0 0x7f6834101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6838c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f683410d150 0x7f6834102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6838c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6834112df0 con 0x7f683410d150 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6838c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6834112c70 con 0x7f68341045d0 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6838c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6834112f70 con 0x7f6834103ba0 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6832a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68341045d0 0x7f6834101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6833400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834103ba0 0x7f6834101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6832a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68341045d0 0x7f6834101ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:36490/0 (socket says 172.21.15.39:36490) 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6832a00640 1 -- 172.21.15.39:0/2320017221 learned_addr learned my addr 172.21.15.39:0/2320017221 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.581+0000 7f6833e00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f683410d150 0x7f6834102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.582+0000 7f6832a00640 1 -- 172.21.15.39:0/2320017221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834103ba0 msgr2=0x7f6834101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.582+0000 7f6832a00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834103ba0 0x7f6834101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.582+0000 7f6832a00640 1 -- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f683410d150 msgr2=0x7f6834102530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:14.571 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.582+0000 7f6832a00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f683410d150 0x7f6834102530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.582+0000 7f6832a00640 1 -- 172.21.15.39:0/2320017221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68341a7b20 con 0x7f68341045d0 2024-07-04T08:28:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.582+0000 7f6832a00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68341045d0 0x7f6834101ff0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f682000e7e0 tx=0x7f682000ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.582+0000 7f682be00640 1 -- 172.21.15.39:0/2320017221 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682000cab0 con 0x7f68341045d0 2024-07-04T08:28:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.582+0000 7f682be00640 1 -- 172.21.15.39:0/2320017221 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68200054d0 con 0x7f68341045d0 2024-07-04T08:28:14.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.582+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68341a7e10 con 0x7f68341045d0 2024-07-04T08:28:14.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.583+0000 7f682be00640 1 -- 172.21.15.39:0/2320017221 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820004dd0 con 0x7f68341045d0 2024-07-04T08:28:14.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.583+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68341a8260 con 0x7f68341045d0 2024-07-04T08:28:14.573 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.584+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6834103fb0 con 0x7f68341045d0 2024-07-04T08:28:14.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.588+0000 7f682be00640 1 -- 172.21.15.39:0/2320017221 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f682000cc50 con 0x7f68341045d0 2024-07-04T08:28:14.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.589+0000 7f682be00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68180b29e0 0x7f68180b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:14.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.589+0000 7f682be00640 1 -- 172.21.15.39:0/2320017221 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f68200e2bb0 con 0x7f68341045d0 2024-07-04T08:28:14.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.590+0000 7f6833400640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68180b29e0 0x7f68180b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:14.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.590+0000 7f6833400640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68180b29e0 0x7f68180b4ef0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6834103500 tx=0x7f6824004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:14.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.594+0000 7f682be00640 1 -- 172.21.15.39:0/2320017221 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f68200aa1d0 con 0x7f68341045d0 2024-07-04T08:28:14.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.852+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68341a8520 con 0x7f68341045d0 2024-07-04T08:28:14.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.853+0000 7f682be00640 1 -- 172.21.15.39:0/2320017221 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f68200a9ad0 con 0x7f68341045d0 2024-07-04T08:28:14.843 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:28:14.843 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:28:14.843 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:28:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68180b29e0 msgr2=0x7f68180b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68180b29e0 0x7f68180b4ef0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6834103500 tx=0x7f6824004a40 comp rx=0 tx=0).stop 2024-07-04T08:28:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68341045d0 msgr2=0x7f6834101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:14.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68341045d0 0x7f6834101ff0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f682000e7e0 tx=0x7f682000ebe0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 shutdown_connections 2024-07-04T08:28:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68180b29e0 0x7f68180b4ef0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f683410d150 0x7f6834102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68341045d0 0x7f6834101ff0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 --2- 172.21.15.39:0/2320017221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6834103ba0 0x7f6834101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:14.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.857+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 >> 172.21.15.39:0/2320017221 conn(0x7f68340fd810 msgr2=0x7f6834106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:14.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.858+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 shutdown_connections 2024-07-04T08:28:14.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:14.858+0000 7f6838c00640 1 -- 172.21.15.39:0/2320017221 wait complete. 2024-07-04T08:28:15.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:15 smithi133 ceph-mon[37050]: pgmap v885: 129 pgs: 129 active+clean; 22 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-07-04T08:28:15.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:15 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2320017221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:28:15.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:15 smithi039 ceph-mon[32062]: pgmap v885: 129 pgs: 129 active+clean; 22 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-07-04T08:28:15.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:15 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2320017221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:28:15.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:15 smithi050 ceph-mon[36577]: pgmap v885: 129 pgs: 129 active+clean; 22 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 113 KiB/s wr, 35 op/s 2024-07-04T08:28:15.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:15 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2320017221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:28:15.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:28:16.184 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:17.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:17 smithi133 ceph-mon[37050]: pgmap v886: 129 pgs: 129 active+clean; 22 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 100 KiB/s wr, 18 op/s 2024-07-04T08:28:17.558 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:17 smithi039 ceph-mon[32062]: pgmap v886: 129 pgs: 129 active+clean; 22 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 100 KiB/s wr, 18 op/s 2024-07-04T08:28:17.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:17 smithi050 ceph-mon[36577]: pgmap v886: 129 pgs: 129 active+clean; 22 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 100 KiB/s wr, 18 op/s 2024-07-04T08:28:18.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.499+0000 7f1da4e00640 1 -- 172.21.15.39:0/3522286324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da0074c80 msgr2=0x7f1da0075090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:18.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.499+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3522286324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da0074c80 0x7f1da0075090 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009860 tx=0x7f1d9002a340 comp rx=0 tx=0).stop 2024-07-04T08:28:18.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.499+0000 7f1da4e00640 1 -- 172.21.15.39:0/3522286324 shutdown_connections 2024-07-04T08:28:18.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.499+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3522286324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da010f690 0x7f1da0111ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.499+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3522286324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da00755d0 0x7f1da010f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.499+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3522286324 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da0074c80 0x7f1da0075090 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.499+0000 7f1da4e00640 1 -- 172.21.15.39:0/3522286324 >> 172.21.15.39:0/3522286324 conn(0x7f1da007ada0 msgr2=0x7f1da007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.499+0000 7f1da4e00640 1 -- 172.21.15.39:0/3522286324 shutdown_connections 2024-07-04T08:28:18.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.499+0000 7f1da4e00640 1 -- 172.21.15.39:0/3522286324 wait complete. 2024-07-04T08:28:18.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.500+0000 7f1da4e00640 1 Processor -- start 2024-07-04T08:28:18.492 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.500+0000 7f1da4e00640 1 -- start start 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.501+0000 7f1da4e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da0074c80 0x7f1da019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.501+0000 7f1da4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da00755d0 0x7f1da019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.501+0000 7f1da4e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da010f690 0x7f1da01a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.501+0000 7f1da4e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0115200 con 0x7f1da010f690 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.501+0000 7f1da4e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0115080 con 0x7f1da0074c80 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.501+0000 7f1da4e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0115380 con 0x7f1da00755d0 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.501+0000 7f1d9e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da010f690 0x7f1da01a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.501+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da00755d0 0x7f1da019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.502+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da00755d0 0x7f1da019c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37042/0 (socket says 172.21.15.39:37042) 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.502+0000 7f1d97e00640 1 -- 172.21.15.39:0/3097451457 learned_addr learned my addr 172.21.15.39:0/3097451457 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:18.493 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.502+0000 7f1d9de00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da0074c80 0x7f1da019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.502+0000 7f1d9e800640 1 -- 172.21.15.39:0/3097451457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da00755d0 msgr2=0x7f1da019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.502+0000 7f1d9e800640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da00755d0 0x7f1da019c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.502+0000 7f1d9e800640 1 -- 172.21.15.39:0/3097451457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da0074c80 msgr2=0x7f1da019bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.502+0000 7f1d9e800640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da0074c80 0x7f1da019bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.502+0000 7f1d9e800640 1 -- 172.21.15.39:0/3097451457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da01a1be0 con 0x7f1da010f690 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.502+0000 7f1d97e00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da00755d0 0x7f1da019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.503+0000 7f1d9e800640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da010f690 0x7f1da01a14b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800ef00 tx=0x7f1d9800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.503+0000 7f1d96a00640 1 -- 172.21.15.39:0/3097451457 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98004b20 con 0x7f1da010f690 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.503+0000 7f1da4e00640 1 -- 172.21.15.39:0/3097451457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da01a1ed0 con 0x7f1da010f690 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.503+0000 7f1d96a00640 1 -- 172.21.15.39:0/3097451457 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d98003e20 con 0x7f1da010f690 2024-07-04T08:28:18.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.503+0000 7f1d96a00640 1 -- 172.21.15.39:0/3097451457 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800b570 con 0x7f1da010f690 2024-07-04T08:28:18.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.503+0000 7f1da4e00640 1 -- 172.21.15.39:0/3097451457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a8630 con 0x7f1da010f690 2024-07-04T08:28:18.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.506+0000 7f1dab528740 1 Processor -- start 2024-07-04T08:28:18.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.507+0000 7f1d96a00640 1 -- 172.21.15.39:0/3097451457 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1d9801d020 con 0x7f1da010f690 2024-07-04T08:28:18.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.507+0000 7f1dab528740 1 -- start start 2024-07-04T08:28:18.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1dab528740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1dab528740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a42d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1dab528740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a43250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1dab528740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5559f99b8f00 con 0x5559f969e070 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1dab528740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5559f99a84a0 con 0x5559f967d390 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1dab528740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5559f99ac4e0 con 0x5559f96751b0 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a42d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a42d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37046/0 (socket says 172.21.15.39:37046) 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1d9de00640 1 -- 172.21.15.39:0/1045540339 learned_addr learned my addr 172.21.15.39:0/1045540339 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:18.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1d9de00640 1 -- 172.21.15.39:0/1045540339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 msgr2=0x5559f9a43250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.508+0000 7f1d97e00640 1 --2- 172.21.15.39:0/1045540339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a43250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d9de00640 1 --2- 172.21.15.39:0/1045540339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a43250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d9de00640 1 -- 172.21.15.39:0/1045540339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 msgr2=0x5559f9a427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d9de00640 1 --2- 172.21.15.39:0/1045540339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d9de00640 1 -- 172.21.15.39:0/1045540339 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f9940670 con 0x5559f96751b0 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d97e00640 1 --2- 172.21.15.39:0/1045540339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a43250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d9de00640 1 --2- 172.21.15.39:0/1045540339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a42d10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009860 tx=0x7f1d90005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69618fcf6306ecb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d8f400640 1 -- 172.21.15.39:0/1045540339 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90037040 con 0x5559f96751b0 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d8f400640 1 -- 172.21.15.39:0/1045540339 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d90030450 con 0x5559f96751b0 2024-07-04T08:28:18.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d96a00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d840b2960 0x7f1d840b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.509+0000 7f1d8f400640 1 -- 172.21.15.39:0/1045540339 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90003950 con 0x5559f96751b0 2024-07-04T08:28:18.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.510+0000 7f1d96a00640 1 -- 172.21.15.39:0/3097451457 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f1d980e27c0 con 0x7f1da010f690 2024-07-04T08:28:18.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.510+0000 7f1d97e00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d840b2960 0x7f1d840b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.510+0000 7f1dab528740 1 -- 172.21.15.39:0/1045540339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 msgr2=0x5559f9a42d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.510+0000 7f1d97e00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d840b2960 0x7f1d840b4e70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1d880095e0 tx=0x7f1d88009bf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.510+0000 7f1dab528740 1 --2- 172.21.15.39:0/1045540339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a42d10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009860 tx=0x7f1d90005d30 comp rx=0 tx=0).stop 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.511+0000 7f1dab528740 1 -- 172.21.15.39:0/1045540339 shutdown_connections 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.511+0000 7f1dab528740 1 --2- 172.21.15.39:0/1045540339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a43250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.511+0000 7f1dab528740 1 --2- 172.21.15.39:0/1045540339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a42d10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.511+0000 7f1dab528740 1 --2- 172.21.15.39:0/1045540339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.511+0000 7f1dab528740 1 -- 172.21.15.39:0/1045540339 >> 172.21.15.39:0/1045540339 conn(0x5559f9692460 msgr2=0x5559f9a406c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.511+0000 7f1dab528740 1 -- 172.21.15.39:0/1045540339 shutdown_connections 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.511+0000 7f1dab528740 1 -- 172.21.15.39:0/1045540339 wait complete. 2024-07-04T08:28:18.505 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.512+0000 7f1dab528740 1 -- start start 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.512+0000 7f1dab528740 1 Processor -- start 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.512+0000 7f1dab528740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f96751b0 0x5559f9a4e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.512+0000 7f1dab528740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f967d390 0x5559f9a4e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.512+0000 7f1dab528740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a5a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.512+0000 7f1dab528740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5559f980c700 con 0x5559f969e070 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1dab528740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5559f994fbd0 con 0x5559f96751b0 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1dab528740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5559f9a50f50 con 0x5559f967d390 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1dab528740 10 client.0 _ll_register_callbacks cb 0x5559f99a1590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f967d390 0x5559f9a4e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d97e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f967d390 0x5559f9a4e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37058/0 (socket says 172.21.15.39:37058) 2024-07-04T08:28:18.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f96751b0 0x5559f9a4e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d97e00640 1 -- 172.21.15.39:0/444075186 learned_addr learned my addr 172.21.15.39:0/444075186 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d9e800640 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a5a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d97e00640 1 -- 172.21.15.39:0/444075186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f96751b0 msgr2=0x5559f9a4e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d97e00640 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f96751b0 0x5559f9a4e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d97e00640 1 -- 172.21.15.39:0/444075186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 msgr2=0x5559f9a5a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d97e00640 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a5a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d97e00640 1 -- 172.21.15.39:0/444075186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f9709650 con 0x5559f967d390 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.513+0000 7f1d9e800640 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a5a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.514+0000 7f1d97e00640 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f967d390 0x5559f9a4e8e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800fe90 tx=0x7f1d88004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=299e76011e314d47 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.514+0000 7f1d83e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.514+0000 7f1d83e00640 1 -- 172.21.15.39:0/444075186 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88012e30 con 0x5559f967d390 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.514+0000 7f1dab528740 1 -- 172.21.15.39:0/444075186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d90030450 con 0x5559f967d390 2024-07-04T08:28:18.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.514+0000 7f1d83e00640 1 -- 172.21.15.39:0/444075186 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f1d88019030 con 0x5559f967d390 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.514+0000 7f1d83e00640 1 -- 172.21.15.39:0/444075186 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d8801f920 con 0x5559f967d390 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.514+0000 7f1dab528740 10 client.28267 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.515+0000 7f1dab528740 1 -- 172.21.15.39:0/444075186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559f9a5cf40 con 0x5559f967d390 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.515+0000 7f1d83e00640 1 -- 172.21.15.39:0/444075186 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d90030450 con 0x5559f967d390 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.515+0000 7f1d83e00640 1 -- 172.21.15.39:0/444075186 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d880048c0 con 0x5559f967d390 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.515+0000 7f1dab528740 10 client.28267 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.515+0000 7f1dab528740 10 client.28267 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.515+0000 7f1dab528740 20 client.28267 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.515+0000 7f1dab528740 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5559f965c9f0 0x5559f9a604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.515+0000 7f1dab528740 4 client.28267 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.515+0000 7f1dab528740 1 -- 172.21.15.39:0/444075186 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559f99bae50 con 0x5559f965c9f0 2024-07-04T08:28:18.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.516+0000 7f1d9de00640 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5559f965c9f0 0x5559f9a604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.516+0000 7f1d9de00640 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5559f965c9f0 0x5559f9a604e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=451bd993ba64ac9f server_cookie=efb244da82c77eaa in_seq=0 out_seq=0 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.516+0000 7f1d83e00640 10 client.28267 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.517+0000 7f1d83e00640 1 -- 172.21.15.39:0/444075186 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5559f99bae50 con 0x5559f965c9f0 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.517+0000 7f1d83e00640 10 client.28267 handle_command_reply: tid=0 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.517+0000 7f1dab528740 1 client.28267 shutdown 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.517+0000 7f1dab528740 1 -- 172.21.15.39:0/444075186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f967d390 msgr2=0x5559f9a4e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.517+0000 7f1dab528740 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f967d390 0x5559f9a4e8e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800fe90 tx=0x7f1d88004730 comp rx=0 tx=0).stop 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1dab528740 1 -- 172.21.15.39:0/444075186 shutdown_connections 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1dab528740 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5559f965c9f0 0x5559f9a604e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1dab528740 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a5a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1dab528740 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f967d390 0x5559f9a4e8e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1dab528740 1 --2- 172.21.15.39:0/444075186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f96751b0 0x5559f9a4e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1d83e00640 0 client.28267 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:28:18.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1dab528740 1 -- 172.21.15.39:0/444075186 >> 172.21.15.39:0/444075186 conn(0x5559f9692460 msgr2=0x5559f9a459b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:18.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1dab528740 1 -- 172.21.15.39:0/444075186 shutdown_connections 2024-07-04T08:28:18.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1dab528740 1 -- 172.21.15.39:0/444075186 wait complete. 2024-07-04T08:28:18.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.518+0000 7f1dab528740 20 client.28267 trim_cache size 0 max 16384 2024-07-04T08:28:18.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.533+0000 7f1dab528740 1 Processor -- start 2024-07-04T08:28:18.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.534+0000 7f1dab528740 1 -- start start 2024-07-04T08:28:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.534+0000 7f1dab528740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a3f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.534+0000 7f1dab528740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a45fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.534+0000 7f1dab528740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.534+0000 7f1dab528740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01b2620 con 0x5559f969e070 2024-07-04T08:28:18.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.534+0000 7f1dab528740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01b4160 con 0x5559f967d390 2024-07-04T08:28:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.534+0000 7f1dab528740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01b2cf0 con 0x5559f96751b0 2024-07-04T08:28:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.534+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a3f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.534+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a3f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:37060/0 (socket says 172.21.15.39:37060) 2024-07-04T08:28:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d9de00640 1 -- 172.21.15.39:0/887939064 learned_addr learned my addr 172.21.15.39:0/887939064 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d9e800640 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d97e00640 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a45fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d9de00640 1 -- 172.21.15.39:0/887939064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 msgr2=0x5559f9a45fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d9de00640 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a45fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d9de00640 1 -- 172.21.15.39:0/887939064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 msgr2=0x5559f9a478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d9de00640 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d9de00640 1 -- 172.21.15.39:0/887939064 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f9a4c6a0 con 0x5559f96751b0 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d9e800640 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.535+0000 7f1d9de00640 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a3f960 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5559f9a41450 tx=0x7f1d900058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37ae16d0ae0047d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.536+0000 7f1d8ea00640 1 -- 172.21.15.39:0/887939064 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90037040 con 0x5559f96751b0 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.536+0000 7f1d8ea00640 1 -- 172.21.15.39:0/887939064 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d90005500 con 0x5559f96751b0 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.536+0000 7f1d8ea00640 1 -- 172.21.15.39:0/887939064 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90004b30 con 0x5559f96751b0 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.536+0000 7f1dab528740 1 -- 172.21.15.39:0/887939064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 msgr2=0x5559f9a3f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:18.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.536+0000 7f1dab528740 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a3f960 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5559f9a41450 tx=0x7f1d900058d0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.536+0000 7f1dab528740 1 -- 172.21.15.39:0/887939064 shutdown_connections 2024-07-04T08:28:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.537+0000 7f1dab528740 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f969e070 0x5559f9a478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.537+0000 7f1dab528740 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a45fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.537+0000 7f1dab528740 1 --2- 172.21.15.39:0/887939064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f96751b0 0x5559f9a3f960 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.537+0000 7f1dab528740 1 -- 172.21.15.39:0/887939064 >> 172.21.15.39:0/887939064 conn(0x5559f9692460 msgr2=0x5559f9a458d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.537+0000 7f1dab528740 1 -- 172.21.15.39:0/887939064 shutdown_connections 2024-07-04T08:28:18.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.537+0000 7f1dab528740 1 -- 172.21.15.39:0/887939064 wait complete. 2024-07-04T08:28:18.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.537+0000 7f1dab528740 1 -- start start 2024-07-04T08:28:18.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.538+0000 7f1dab528740 1 Processor -- start 2024-07-04T08:28:18.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.538+0000 7f1dab528740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f96751b0 0x5559f9a564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.538+0000 7f1dab528740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a56a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.538+0000 7f1dab528740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f969e070 0x5559f9a598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.538+0000 7f1dab528740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01b3a90 con 0x5559f96751b0 2024-07-04T08:28:18.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.538+0000 7f1dab528740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01b33c0 con 0x5559f967d390 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.538+0000 7f1dab528740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1da01b0600 con 0x5559f969e070 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.538+0000 7f1dab528740 10 client.0 _ll_register_callbacks cb 0x7f1d8800fd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.538+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f96751b0 0x5559f9a564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d9de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f96751b0 0x5559f9a564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49166/0 (socket says 172.21.15.39:49166) 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d9de00640 1 -- 172.21.15.39:0/441227018 learned_addr learned my addr 172.21.15.39:0/441227018 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d9e800640 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f969e070 0x5559f9a598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d97e00640 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a56a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d9de00640 1 -- 172.21.15.39:0/441227018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f969e070 msgr2=0x5559f9a598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d9de00640 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f969e070 0x5559f9a598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d9de00640 1 -- 172.21.15.39:0/441227018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 msgr2=0x5559f9a56a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d9de00640 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a56a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d9de00640 1 -- 172.21.15.39:0/441227018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f9a5a020 con 0x5559f96751b0 2024-07-04T08:28:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d9de00640 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f96751b0 0x5559f9a564a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1d90005730 tx=0x7f1d90005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92e9f77631810c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:18.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.539+0000 7f1d83e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:28:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.540+0000 7f1d83e00640 1 -- 172.21.15.39:0/441227018 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90037050 con 0x5559f96751b0 2024-07-04T08:28:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.540+0000 7f1d83e00640 1 -- 172.21.15.39:0/441227018 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f1d90004040 con 0x5559f96751b0 2024-07-04T08:28:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.540+0000 7f1dab528740 1 -- 172.21.15.39:0/441227018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d90005500 con 0x5559f96751b0 2024-07-04T08:28:18.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.540+0000 7f1d83e00640 1 -- 172.21.15.39:0/441227018 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d90064050 con 0x5559f96751b0 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.540+0000 7f1dab528740 10 client.18216 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.540+0000 7f1dab528740 1 -- 172.21.15.39:0/441227018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559f9a55d20 con 0x5559f96751b0 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.541+0000 7f1d83e00640 1 -- 172.21.15.39:0/441227018 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d90005500 con 0x5559f96751b0 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.541+0000 7f1d83e00640 1 -- 172.21.15.39:0/441227018 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d90004860 con 0x5559f96751b0 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.541+0000 7f1dab528740 10 client.18216 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.541+0000 7f1dab528740 10 client.18216 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.541+0000 7f1dab528740 20 client.18216 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.541+0000 7f1dab528740 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5559f9631da0 0x5559f9acac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.541+0000 7f1dab528740 4 client.18216 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.541+0000 7f1dab528740 1 -- 172.21.15.39:0/441227018 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559f99b68f0 con 0x5559f9631da0 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.542+0000 7f1d97e00640 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5559f9631da0 0x5559f9acac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.542+0000 7f1d97e00640 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5559f9631da0 0x5559f9acac80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e249432fe692747a server_cookie=5656e837edf52759 in_seq=0 out_seq=0 2024-07-04T08:28:18.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.542+0000 7f1d83e00640 10 client.18216 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:28:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.542+0000 7f1d83e00640 1 -- 172.21.15.39:0/441227018 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559f99b68f0 con 0x5559f9631da0 2024-07-04T08:28:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.542+0000 7f1d83e00640 10 client.18216 handle_command_reply: tid=0 2024-07-04T08:28:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.542+0000 7f1dab528740 1 client.18216 shutdown 2024-07-04T08:28:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 -- 172.21.15.39:0/441227018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f96751b0 msgr2=0x5559f9a564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:18.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f96751b0 0x5559f9a564a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1d90005730 tx=0x7f1d90005760 comp rx=0 tx=0).stop 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 -- 172.21.15.39:0/441227018 shutdown_connections 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5559f9631da0 0x5559f9acac80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5559f969e070 0x5559f9a598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559f967d390 0x5559f9a56a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 --2- 172.21.15.39:0/441227018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5559f96751b0 0x5559f9a564a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 -- 172.21.15.39:0/441227018 >> 172.21.15.39:0/441227018 conn(0x5559f9692460 msgr2=0x5559f9a3ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1d83e00640 0 client.18216 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 -- 172.21.15.39:0/441227018 shutdown_connections 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 1 -- 172.21.15.39:0/441227018 wait complete. 2024-07-04T08:28:18.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.543+0000 7f1dab528740 20 client.18216 trim_cache size 0 max 16384 2024-07-04T08:28:18.535 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:28:18.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 -- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d840b2960 msgr2=0x7f1d840b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:18.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d840b2960 0x7f1d840b4e70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1d880095e0 tx=0x7f1d88009bf0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 -- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da010f690 msgr2=0x7f1da01a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:18.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da010f690 0x7f1da01a14b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800ef00 tx=0x7f1d9800c740 comp rx=0 tx=0).stop 2024-07-04T08:28:18.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 -- 172.21.15.39:0/3097451457 shutdown_connections 2024-07-04T08:28:18.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1d840b2960 0x7f1d840b4e70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1da010f690 0x7f1da01a14b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1da00755d0 0x7f1da019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 --2- 172.21.15.39:0/3097451457 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1da0074c80 0x7f1da019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:18.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.544+0000 7f1da4e00640 1 -- 172.21.15.39:0/3097451457 >> 172.21.15.39:0/3097451457 conn(0x7f1da007ada0 msgr2=0x7f1da01092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:18.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.545+0000 7f1da4e00640 1 -- 172.21.15.39:0/3097451457 shutdown_connections 2024-07-04T08:28:18.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:18.545+0000 7f1da4e00640 1 -- 172.21.15.39:0/3097451457 wait complete. 2024-07-04T08:28:19.287 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:28:19.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:19 smithi133 ceph-mon[37050]: pgmap v887: 129 pgs: 129 active+clean; 22 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 69 KiB/s wr, 16 op/s 2024-07-04T08:28:19.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:19 smithi039 ceph-mon[32062]: pgmap v887: 129 pgs: 129 active+clean; 22 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 69 KiB/s wr, 16 op/s 2024-07-04T08:28:19.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:19 smithi050 ceph-mon[36577]: pgmap v887: 129 pgs: 129 active+clean; 22 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 69 KiB/s wr, 16 op/s 2024-07-04T08:28:19.788 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:21.488 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:21 smithi039 ceph-mon[32062]: pgmap v888: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 90 KiB/s wr, 18 op/s 2024-07-04T08:28:21.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:21 smithi133 ceph-mon[37050]: pgmap v888: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 90 KiB/s wr, 18 op/s 2024-07-04T08:28:21.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:21 smithi050 ceph-mon[36577]: pgmap v888: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 90 KiB/s wr, 18 op/s 2024-07-04T08:28:22.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.092+0000 7f9e93e00640 1 -- 172.21.15.39:0/200579828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e9410b390 msgr2=0x7f9e94111bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:22.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.092+0000 7f9e93e00640 1 --2- 172.21.15.39:0/200579828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e9410b390 0x7f9e94111bc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009ca0 tx=0x7f9e8c02a8b0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.092+0000 7f9e93e00640 1 -- 172.21.15.39:0/200579828 shutdown_connections 2024-07-04T08:28:22.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.092+0000 7f9e93e00640 1 --2- 172.21.15.39:0/200579828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e9410aa40 0x7f9e9410ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.092+0000 7f9e93e00640 1 --2- 172.21.15.39:0/200579828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e9410b390 0x7f9e94111bc0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.092+0000 7f9e93e00640 1 --2- 172.21.15.39:0/200579828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e94105190 0x7f9e941055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.092+0000 7f9e93e00640 1 -- 172.21.15.39:0/200579828 >> 172.21.15.39:0/200579828 conn(0x7f9e94100be0 msgr2=0x7f9e94103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:22.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.092+0000 7f9e93e00640 1 -- 172.21.15.39:0/200579828 shutdown_connections 2024-07-04T08:28:22.082 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.093+0000 7f9e93e00640 1 -- 172.21.15.39:0/200579828 wait complete. 2024-07-04T08:28:22.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.094+0000 7f9e93e00640 1 Processor -- start 2024-07-04T08:28:22.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.094+0000 7f9e93e00640 1 -- start start 2024-07-04T08:28:22.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.094+0000 7f9e93e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e94105190 0x7f9e9419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.094+0000 7f9e93e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e9410aa40 0x7f9e9419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.094+0000 7f9e93e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9410b390 0x7f9e941a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.094+0000 7f9e93e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94115040 con 0x7f9e94105190 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.094+0000 7f9e93e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94114ec0 con 0x7f9e9410aa40 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.094+0000 7f9e93e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941151c0 con 0x7f9e9410b390 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e92a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e94105190 0x7f9e9419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e8be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e9410aa40 0x7f9e9419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e92a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e94105190 0x7f9e9419b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47364/0 (socket says 172.21.15.39:47364) 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e92a00640 1 -- 172.21.15.39:0/2159748198 learned_addr learned my addr 172.21.15.39:0/2159748198 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:22.084 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e93400640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9410b390 0x7f9e941a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e92a00640 1 -- 172.21.15.39:0/2159748198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9410b390 msgr2=0x7f9e941a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:22.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e92a00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9410b390 0x7f9e941a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e92a00640 1 -- 172.21.15.39:0/2159748198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e9410aa40 msgr2=0x7f9e9419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e92a00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e9410aa40 0x7f9e9419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e92a00640 1 -- 172.21.15.39:0/2159748198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e941a18e0 con 0x7f9e94105190 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.095+0000 7f9e8be00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e9410aa40 0x7f9e9419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.096+0000 7f9e92a00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e94105190 0x7f9e9419b970 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e9f0 tx=0x7f9e8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.096+0000 7f9e98a00640 1 -- 172.21.15.39:0/2159748198 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400cd80 con 0x7f9e94105190 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.096+0000 7f9e93e00640 1 -- 172.21.15.39:0/2159748198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e941a1bd0 con 0x7f9e94105190 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.096+0000 7f9e98a00640 1 -- 172.21.15.39:0/2159748198 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e840108e0 con 0x7f9e94105190 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.096+0000 7f9e98a00640 1 -- 172.21.15.39:0/2159748198 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84018980 con 0x7f9e94105190 2024-07-04T08:28:22.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.096+0000 7f9e93e00640 1 -- 172.21.15.39:0/2159748198 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e941a8330 con 0x7f9e94105190 2024-07-04T08:28:22.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.100+0000 7f9ea04cb740 1 Processor -- start 2024-07-04T08:28:22.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.100+0000 7f9ea04cb740 1 -- start start 2024-07-04T08:28:22.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.100+0000 7f9e98a00640 1 -- 172.21.15.39:0/2159748198 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9e84010430 con 0x7f9e94105190 2024-07-04T08:28:22.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.100+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c7437950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.100+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7437e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.100+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c74383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.100+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a0c73adee0 con 0x55a0c7072390 2024-07-04T08:28:22.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.100+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a0c739d480 con 0x55a0c706a1b0 2024-07-04T08:28:22.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.100+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a0c73a14c0 con 0x55a0c7093070 2024-07-04T08:28:22.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.101+0000 7f9e92a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7437e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.101+0000 7f9e93400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c7437950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.101+0000 7f9e92a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7437e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34308/0 (socket says 172.21.15.39:34308) 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.101+0000 7f9e92a00640 1 -- 172.21.15.39:0/1626482323 learned_addr learned my addr 172.21.15.39:0/1626482323 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.101+0000 7f9e93400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c7437950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44618/0 (socket says 172.21.15.39:44618) 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.101+0000 7f9e8be00640 1 --2- 172.21.15.39:0/1626482323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c74383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.102+0000 7f9e92a00640 1 -- 172.21.15.39:0/1626482323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 msgr2=0x55a0c7437950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.102+0000 7f9e92a00640 1 --2- 172.21.15.39:0/1626482323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c7437950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.102+0000 7f9e92a00640 1 -- 172.21.15.39:0/1626482323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 msgr2=0x55a0c74383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.102+0000 7f9e92a00640 1 --2- 172.21.15.39:0/1626482323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c74383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.094 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.102+0000 7f9e92a00640 1 -- 172.21.15.39:0/1626482323 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c7335650 con 0x55a0c706a1b0 2024-07-04T08:28:22.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.102+0000 7f9e93400640 1 --2- 172.21.15.39:0/1626482323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c7437950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:22.095 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.102+0000 7f9e92a00640 1 --2- 172.21.15.39:0/1626482323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7437e90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9e840dff60 tx=0x7f9e840eb750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d7f304e3c7c9283 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:22.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.102+0000 7f9e88c00640 1 -- 172.21.15.39:0/1626482323 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e840ebe60 con 0x55a0c706a1b0 2024-07-04T08:28:22.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.102+0000 7f9e88c00640 1 -- 172.21.15.39:0/1626482323 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e840f0070 con 0x55a0c706a1b0 2024-07-04T08:28:22.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.103+0000 7f9e88c00640 1 -- 172.21.15.39:0/1626482323 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e840ecb40 con 0x55a0c706a1b0 2024-07-04T08:28:22.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.103+0000 7f9e98a00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e780b29e0 0x7f9e780b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.103+0000 7f9e98a00640 1 -- 172.21.15.39:0/2159748198 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f9e840e2c90 con 0x7f9e94105190 2024-07-04T08:28:22.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.103+0000 7f9e93400640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e780b29e0 0x7f9e780b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9ea04cb740 1 -- 172.21.15.39:0/1626482323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 msgr2=0x55a0c7437e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:22.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/1626482323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7437e90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9e840dff60 tx=0x7f9e840eb750 comp rx=0 tx=0).stop 2024-07-04T08:28:22.098 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9ea04cb740 1 -- 172.21.15.39:0/1626482323 shutdown_connections 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/1626482323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c74383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/1626482323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7437e90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/1626482323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c7437950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9ea04cb740 1 -- 172.21.15.39:0/1626482323 >> 172.21.15.39:0/1626482323 conn(0x55a0c7087460 msgr2=0x55a0c7435840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9ea04cb740 1 -- 172.21.15.39:0/1626482323 shutdown_connections 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9ea04cb740 1 -- 172.21.15.39:0/1626482323 wait complete. 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.104+0000 7f9e93400640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e780b29e0 0x7f9e780b4ef0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c006010 tx=0x7f9e8c009860 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.105+0000 7f9ea04cb740 1 -- start start 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.105+0000 7f9ea04cb740 1 Processor -- start 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7443560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7072390 0x55a0c7443ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.099 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 0x55a0c744f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a0c7201700 con 0x55a0c7093070 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a0c7344bb0 con 0x55a0c706a1b0 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a0c7446130 con 0x55a0c7072390 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9ea04cb740 10 client.0 _ll_register_callbacks cb 0x55a0c7396570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e92a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7443560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e92a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7443560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34310/0 (socket says 172.21.15.39:34310) 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e92a00640 1 -- 172.21.15.39:0/4159492203 learned_addr learned my addr 172.21.15.39:0/4159492203 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e93400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 0x55a0c744f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e8be00640 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7072390 0x55a0c7443ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e92a00640 1 -- 172.21.15.39:0/4159492203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7072390 msgr2=0x55a0c7443ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e92a00640 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7072390 0x55a0c7443ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e92a00640 1 -- 172.21.15.39:0/4159492203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 msgr2=0x55a0c744f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:22.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e92a00640 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 0x55a0c744f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.106+0000 7f9e92a00640 1 -- 172.21.15.39:0/4159492203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c70fe650 con 0x55a0c706a1b0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.107+0000 7f9e92a00640 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7443560 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9e840a5ce0 tx=0x7f9e84000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2254f29b64b6259 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.107+0000 7f9e81600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.107+0000 7f9e81600640 1 -- 172.21.15.39:0/4159492203 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e840cae60 con 0x55a0c706a1b0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.107+0000 7f9e81600640 1 -- 172.21.15.39:0/4159492203 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f9e840e02a0 con 0x55a0c706a1b0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.107+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4159492203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e840f0070 con 0x55a0c706a1b0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.107+0000 7f9e81600640 1 -- 172.21.15.39:0/4159492203 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e840a5b20 con 0x55a0c706a1b0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.107+0000 7f9ea04cb740 10 client.28355 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.107+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4159492203 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0c74521f0 con 0x55a0c706a1b0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.108+0000 7f9e81600640 1 -- 172.21.15.39:0/4159492203 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e840f0070 con 0x55a0c706a1b0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.108+0000 7f9e81600640 1 -- 172.21.15.39:0/4159492203 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9e840a9ec0 con 0x55a0c706a1b0 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.108+0000 7f9ea04cb740 10 client.28355 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.108+0000 7f9ea04cb740 10 client.28355 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:28:22.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.108+0000 7f9ea04cb740 20 client.28355 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:22.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.108+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55a0c70519f0 0x55a0c7455550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.108+0000 7f9ea04cb740 4 client.28355 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:28:22.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.108+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4159492203 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0c73afe30 con 0x55a0c70519f0 2024-07-04T08:28:22.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.108+0000 7f9e8be00640 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55a0c70519f0 0x55a0c7455550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.109+0000 7f9e8be00640 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55a0c70519f0 0x55a0c7455550 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=24e8bf5ec53555f9 server_cookie=1464026d1226d113 in_seq=0 out_seq=0 2024-07-04T08:28:22.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.109+0000 7f9e81600640 10 client.28355 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:28:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9e81600640 1 -- 172.21.15.39:0/4159492203 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a0c73afe30 con 0x55a0c70519f0 2024-07-04T08:28:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9e81600640 10 client.28355 handle_command_reply: tid=0 2024-07-04T08:28:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 client.28355 shutdown 2024-07-04T08:28:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4159492203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 msgr2=0x55a0c7443560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7443560 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9e840a5ce0 tx=0x7f9e84000bc0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4159492203 shutdown_connections 2024-07-04T08:28:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55a0c70519f0 0x55a0c7455550 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 0x55a0c744f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7072390 0x55a0c7443ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4159492203 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7443560 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4159492203 >> 172.21.15.39:0/4159492203 conn(0x55a0c7087460 msgr2=0x55a0c7438980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9e81600640 0 client.28355 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:28:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4159492203 shutdown_connections 2024-07-04T08:28:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.110+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4159492203 wait complete. 2024-07-04T08:28:22.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.111+0000 7f9ea04cb740 20 client.28355 trim_cache size 0 max 16384 2024-07-04T08:28:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.126+0000 7f9ea04cb740 1 Processor -- start 2024-07-04T08:28:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.126+0000 7f9ea04cb740 1 -- start start 2024-07-04T08:28:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.126+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c706a1b0 0x55a0c744f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c7072390 0x55a0c743ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 0x55a0c743cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b2460 con 0x55a0c7093070 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b3fa0 con 0x55a0c7072390 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b2b30 con 0x55a0c706a1b0 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e93400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 0x55a0c743cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e93400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 0x55a0c743cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47402/0 (socket says 172.21.15.39:47402) 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e93400640 1 -- 172.21.15.39:0/4136711291 learned_addr learned my addr 172.21.15.39:0/4136711291 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e8be00640 1 --2- 172.21.15.39:0/4136711291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c7072390 0x55a0c743ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e92a00640 1 --2- 172.21.15.39:0/4136711291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c706a1b0 0x55a0c744f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e8be00640 1 -- 172.21.15.39:0/4136711291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c706a1b0 msgr2=0x55a0c744f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e8be00640 1 --2- 172.21.15.39:0/4136711291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c706a1b0 0x55a0c744f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e8be00640 1 -- 172.21.15.39:0/4136711291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 msgr2=0x55a0c743cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e8be00640 1 --2- 172.21.15.39:0/4136711291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 0x55a0c743cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.127+0000 7f9e8be00640 1 -- 172.21.15.39:0/4136711291 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c7445aa0 con 0x55a0c7072390 2024-07-04T08:28:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.128+0000 7f9e8be00640 1 --2- 172.21.15.39:0/4136711291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c7072390 0x55a0c743ca60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55a0c7446160 tx=0x7f9e7c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1eb97077ca03d8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.128+0000 7f9e83e00640 1 -- 172.21.15.39:0/4136711291 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c012e40 con 0x55a0c7072390 2024-07-04T08:28:22.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.128+0000 7f9e83e00640 1 -- 172.21.15.39:0/4136711291 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7c0108e0 con 0x55a0c7072390 2024-07-04T08:28:22.118 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.128+0000 7f9e83e00640 1 -- 172.21.15.39:0/4136711291 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c003a10 con 0x55a0c7072390 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.129+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4136711291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c7072390 msgr2=0x55a0c743ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.129+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4136711291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c7072390 0x55a0c743ca60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55a0c7446160 tx=0x7f9e7c005f90 comp rx=0 tx=0).stop 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.129+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4136711291 shutdown_connections 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.129+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4136711291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7093070 0x55a0c743cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.129+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4136711291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c7072390 0x55a0c743ca60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.129+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/4136711291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c706a1b0 0x55a0c744f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.129+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4136711291 >> 172.21.15.39:0/4136711291 conn(0x55a0c7087460 msgr2=0x55a0c743e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.129+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4136711291 shutdown_connections 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.129+0000 7f9ea04cb740 1 -- 172.21.15.39:0/4136711291 wait complete. 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.130+0000 7f9ea04cb740 1 -- start start 2024-07-04T08:28:22.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.130+0000 7f9ea04cb740 1 Processor -- start 2024-07-04T08:28:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7455eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c7456490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9ea04cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c74b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b38d0 con 0x55a0c7072390 2024-07-04T08:28:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b3200 con 0x55a0c706a1b0 2024-07-04T08:28:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9ea04cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941b0440 con 0x55a0c7093070 2024-07-04T08:28:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9ea04cb740 10 client.0 _ll_register_callbacks cb 0x7f9e840e4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:22.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9e8be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c7456490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9e8be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c7456490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47416/0 (socket says 172.21.15.39:47416) 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9e8be00640 1 -- 172.21.15.39:0/3810069145 learned_addr learned my addr 172.21.15.39:0/3810069145 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9e92a00640 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7455eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.131+0000 7f9e93400640 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c74b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e8be00640 1 -- 172.21.15.39:0/3810069145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 msgr2=0x55a0c74b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e8be00640 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c74b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e8be00640 1 -- 172.21.15.39:0/3810069145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 msgr2=0x55a0c7455eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e8be00640 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7455eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e8be00640 1 -- 172.21.15.39:0/3810069145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c74b50c0 con 0x55a0c7072390 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e92a00640 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7455eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e8be00640 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c7456490 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c003630 tx=0x7f9e7c0093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28fc23cadf5df56 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e81600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:28:22.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e81600640 1 -- 172.21.15.39:0/3810069145 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c033020 con 0x55a0c7072390 2024-07-04T08:28:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9ea04cb740 1 -- 172.21.15.39:0/3810069145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e7c0108e0 con 0x55a0c7072390 2024-07-04T08:28:22.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.132+0000 7f9e81600640 1 -- 172.21.15.39:0/3810069145 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f9e7c010d60 con 0x55a0c7072390 2024-07-04T08:28:22.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9e81600640 1 -- 172.21.15.39:0/3810069145 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e7c03e050 con 0x55a0c7072390 2024-07-04T08:28:22.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9ea04cb740 10 client.18234 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:22.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9ea04cb740 1 -- 172.21.15.39:0/3810069145 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a0c74bd560 con 0x55a0c7072390 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9e81600640 1 -- 172.21.15.39:0/3810069145 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7c0108e0 con 0x55a0c7072390 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9e81600640 1 -- 172.21.15.39:0/3810069145 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9e7c02f4a0 con 0x55a0c7072390 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9ea04cb740 10 client.18234 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9ea04cb740 10 client.18234 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9ea04cb740 20 client.18234 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55a0c7026da0 0x55a0c74bfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9ea04cb740 4 client.18234 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.133+0000 7f9ea04cb740 1 -- 172.21.15.39:0/3810069145 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0c73ab8d0 con 0x55a0c7026da0 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.134+0000 7f9e92a00640 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55a0c7026da0 0x55a0c74bfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.134+0000 7f9e92a00640 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55a0c7026da0 0x55a0c74bfc90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab9f92c8443f830b server_cookie=70580ee68915fd9c in_seq=0 out_seq=0 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.134+0000 7f9e81600640 10 client.18234 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:28:22.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9e81600640 1 -- 172.21.15.39:0/3810069145 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0c73ab8d0 con 0x55a0c7026da0 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9e81600640 10 client.18234 handle_command_reply: tid=0 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 client.18234 shutdown 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 -- 172.21.15.39:0/3810069145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 msgr2=0x55a0c7456490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c7456490 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c003630 tx=0x7f9e7c0093a0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 -- 172.21.15.39:0/3810069145 shutdown_connections 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55a0c7026da0 0x55a0c74bfc90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a0c7072390 0x55a0c7456490 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0c7093070 0x55a0c74b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 --2- 172.21.15.39:0/3810069145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a0c706a1b0 0x55a0c7455eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 -- 172.21.15.39:0/3810069145 >> 172.21.15.39:0/3810069145 conn(0x55a0c7087460 msgr2=0x55a0c7438960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9e81600640 0 client.18234 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 -- 172.21.15.39:0/3810069145 shutdown_connections 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 1 -- 172.21.15.39:0/3810069145 wait complete. 2024-07-04T08:28:22.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.135+0000 7f9ea04cb740 20 client.18234 trim_cache size 0 max 16384 2024-07-04T08:28:22.127 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 -- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e780b29e0 msgr2=0x7f9e780b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e780b29e0 0x7f9e780b4ef0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c006010 tx=0x7f9e8c009860 comp rx=0 tx=0).stop 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 -- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e94105190 msgr2=0x7f9e9419b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e94105190 0x7f9e9419b970 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e9f0 tx=0x7f9e8400eeb0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 -- 172.21.15.39:0/2159748198 shutdown_connections 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9e780b29e0 0x7f9e780b4ef0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e9410b390 0x7f9e941a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e9410aa40 0x7f9e9419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 --2- 172.21.15.39:0/2159748198 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e94105190 0x7f9e9419b970 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.136+0000 7f9e93e00640 1 -- 172.21.15.39:0/2159748198 >> 172.21.15.39:0/2159748198 conn(0x7f9e94100be0 msgr2=0x7f9e94103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:22.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.137+0000 7f9e93e00640 1 -- 172.21.15.39:0/2159748198 shutdown_connections 2024-07-04T08:28:22.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:22.137+0000 7f9e93e00640 1 -- 172.21.15.39:0/2159748198 wait complete. 2024-07-04T08:28:23.004 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:28:23.490 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:23.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:23 smithi133 ceph-mon[37050]: pgmap v889: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-07-04T08:28:23.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:23 smithi039 ceph-mon[32062]: pgmap v889: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-07-04T08:28:23.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:23 smithi050 ceph-mon[36577]: pgmap v889: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 16 op/s 2024-07-04T08:28:25.548 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:25 smithi039 ceph-mon[32062]: pgmap v890: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-07-04T08:28:25.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:25 smithi133 ceph-mon[37050]: pgmap v890: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-07-04T08:28:25.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:25 smithi050 ceph-mon[36577]: pgmap v890: 129 pgs: 129 active+clean; 22 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-07-04T08:28:25.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.900+0000 7fce9be00640 1 -- 172.21.15.39:0/1687232856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9c10b3a0 msgr2=0x7fce9c111bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:25.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.900+0000 7fce9be00640 1 --2- 172.21.15.39:0/1687232856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9c10b3a0 0x7fce9c111bd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fce94009ca0 tx=0x7fce9402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.900+0000 7fce9be00640 1 -- 172.21.15.39:0/1687232856 shutdown_connections 2024-07-04T08:28:25.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.900+0000 7fce9be00640 1 --2- 172.21.15.39:0/1687232856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9c10b3a0 0x7fce9c111bd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.900+0000 7fce9be00640 1 --2- 172.21.15.39:0/1687232856 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce9c10aa50 0x7fce9c10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.900+0000 7fce9be00640 1 --2- 172.21.15.39:0/1687232856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce9c105110 0x7fce9c105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.900+0000 7fce9be00640 1 -- 172.21.15.39:0/1687232856 >> 172.21.15.39:0/1687232856 conn(0x7fce9c100b80 msgr2=0x7fce9c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:25.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.900+0000 7fce9be00640 1 -- 172.21.15.39:0/1687232856 shutdown_connections 2024-07-04T08:28:25.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.900+0000 7fce9be00640 1 -- 172.21.15.39:0/1687232856 wait complete. 2024-07-04T08:28:25.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.901+0000 7fce9be00640 1 Processor -- start 2024-07-04T08:28:25.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.902+0000 7fce9be00640 1 -- start start 2024-07-04T08:28:25.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.902+0000 7fce9be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce9c105110 0x7fce9c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.902+0000 7fce9be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9c10aa50 0x7fce9c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.902+0000 7fce9be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce9c10b3a0 0x7fce9c1a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.902+0000 7fce9be00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c115050 con 0x7fce9c10b3a0 2024-07-04T08:28:25.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.902+0000 7fce9be00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c114ed0 con 0x7fce9c105110 2024-07-04T08:28:25.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.902+0000 7fce9be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c1151d0 con 0x7fce9c10aa50 2024-07-04T08:28:25.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce9c10b3a0 0x7fce9c1a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce9c105110 0x7fce9c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce9c105110 0x7fce9c19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34332/0 (socket says 172.21.15.39:34332) 2024-07-04T08:28:25.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9aa00640 1 -- 172.21.15.39:0/4290456494 learned_addr learned my addr 172.21.15.39:0/4290456494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:25.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce9c10b3a0 0x7fce9c1a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47424/0 (socket says 172.21.15.39:47424) 2024-07-04T08:28:25.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9a000640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9c10aa50 0x7fce9c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9aa00640 1 -- 172.21.15.39:0/4290456494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9c10aa50 msgr2=0x7fce9c19bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:25.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9aa00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9c10aa50 0x7fce9c19bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9aa00640 1 -- 172.21.15.39:0/4290456494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce9c10b3a0 msgr2=0x7fce9c1a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:25.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9aa00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce9c10b3a0 0x7fce9c1a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9aa00640 1 -- 172.21.15.39:0/4290456494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c1a1930 con 0x7fce9c105110 2024-07-04T08:28:25.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.903+0000 7fce9aa00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce9c105110 0x7fce9c19b9c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00e9f0 tx=0x7fce8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:25.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.904+0000 7fce93400640 1 -- 172.21.15.39:0/4290456494 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c00cd80 con 0x7fce9c105110 2024-07-04T08:28:25.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.904+0000 7fce9be00640 1 -- 172.21.15.39:0/4290456494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c1a1c20 con 0x7fce9c105110 2024-07-04T08:28:25.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.904+0000 7fce93400640 1 -- 172.21.15.39:0/4290456494 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce8c0108e0 con 0x7fce9c105110 2024-07-04T08:28:25.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.904+0000 7fce93400640 1 -- 172.21.15.39:0/4290456494 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c018980 con 0x7fce9c105110 2024-07-04T08:28:25.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.904+0000 7fce9be00640 1 -- 172.21.15.39:0/4290456494 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c1a8380 con 0x7fce9c105110 2024-07-04T08:28:25.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.908+0000 7fcea804e740 1 Processor -- start 2024-07-04T08:28:25.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.909+0000 7fcea804e740 1 -- start start 2024-07-04T08:28:25.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.909+0000 7fcea804e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c5f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.909+0000 7fcea804e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c5feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.909+0000 7fcea804e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.909+0000 7fcea804e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5582f7bd5f00 con 0x5582f78bb070 2024-07-04T08:28:25.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.909+0000 7fcea804e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5582f7bc54a0 con 0x5582f78921b0 2024-07-04T08:28:25.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.909+0000 7fcea804e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5582f7bc94e0 con 0x5582f789a390 2024-07-04T08:28:25.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.909+0000 7fce9b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c5f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c5f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47432/0 (socket says 172.21.15.39:47432) 2024-07-04T08:28:25.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9b400640 1 -- 172.21.15.39:0/1663901748 learned_addr learned my addr 172.21.15.39:0/1663901748 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:25.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9a000640 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce93400640 1 -- 172.21.15.39:0/4290456494 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fce8c010430 con 0x7fce9c105110 2024-07-04T08:28:25.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9aa00640 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c5feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9b400640 1 -- 172.21.15.39:0/1663901748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 msgr2=0x5582f7c603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:25.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9b400640 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9b400640 1 -- 172.21.15.39:0/1663901748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 msgr2=0x5582f7c5feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:25.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9b400640 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c5feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9b400640 1 -- 172.21.15.39:0/1663901748 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f7b5d670 con 0x5582f78bb070 2024-07-04T08:28:25.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.910+0000 7fce9aa00640 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c5feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:28:25.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.911+0000 7fce9b400640 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c5f970 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fce9402f410 tx=0x7fce9402f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104dcff687e99321 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:25.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.911+0000 7fce8be00640 1 -- 172.21.15.39:0/1663901748 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94031cf0 con 0x5582f78bb070 2024-07-04T08:28:25.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.911+0000 7fce8be00640 1 -- 172.21.15.39:0/1663901748 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce94036030 con 0x5582f78bb070 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.911+0000 7fce8be00640 1 -- 172.21.15.39:0/1663901748 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce940042a0 con 0x5582f78bb070 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fce93400640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce800b29e0 0x7fce800b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fce93400640 1 -- 172.21.15.39:0/4290456494 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7fce8c0e2d00 con 0x7fce9c105110 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fcea804e740 1 -- 172.21.15.39:0/1663901748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 msgr2=0x5582f7c5f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fcea804e740 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c5f970 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fce9402f410 tx=0x7fce9402f440 comp rx=0 tx=0).stop 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fce9aa00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce800b29e0 0x7fce800b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fcea804e740 1 -- 172.21.15.39:0/1663901748 shutdown_connections 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fcea804e740 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fcea804e740 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c5feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fcea804e740 1 --2- 172.21.15.39:0/1663901748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c5f970 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fcea804e740 1 -- 172.21.15.39:0/1663901748 >> 172.21.15.39:0/1663901748 conn(0x5582f78af460 msgr2=0x5582f7c5d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:25.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.912+0000 7fcea804e740 1 -- 172.21.15.39:0/1663901748 shutdown_connections 2024-07-04T08:28:25.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.913+0000 7fcea804e740 1 -- 172.21.15.39:0/1663901748 wait complete. 2024-07-04T08:28:25.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.913+0000 7fce9aa00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce800b29e0 0x7fce800b4ef0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5582f7c5e5f0 tx=0x7fce8c00cf20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:25.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.913+0000 7fcea804e740 1 -- start start 2024-07-04T08:28:25.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.914+0000 7fcea804e740 1 Processor -- start 2024-07-04T08:28:25.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.914+0000 7fcea804e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78921b0 0x5582f7c6b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.914+0000 7fcea804e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c6ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.914+0000 7fcea804e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 0x5582f7c77290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.914+0000 7fcea804e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5582f7a29700 con 0x5582f78921b0 2024-07-04T08:28:25.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.914+0000 7fcea804e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5582f7b6cbd0 con 0x5582f78bb070 2024-07-04T08:28:25.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.914+0000 7fcea804e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5582f7c6df70 con 0x5582f789a390 2024-07-04T08:28:25.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.914+0000 7fcea804e740 10 client.0 _ll_register_callbacks cb 0x5582f7bbe590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c6ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c6ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44684/0 (socket says 172.21.15.39:44684) 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9a000640 1 -- 172.21.15.39:0/3956450349 learned_addr learned my addr 172.21.15.39:0/3956450349 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9b400640 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78921b0 0x5582f7c6b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9b400640 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 0x5582f7c77290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9a000640 1 -- 172.21.15.39:0/3956450349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 msgr2=0x5582f7c77290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9a000640 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 0x5582f7c77290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9a000640 1 -- 172.21.15.39:0/3956450349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78921b0 msgr2=0x5582f7c6b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9a000640 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78921b0 0x5582f7c6b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9a000640 1 -- 172.21.15.39:0/3956450349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f7926650 con 0x5582f789a390 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9b400640 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 0x5582f7c77290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:28:25.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce9a000640 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c6ba80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fce840126e0 tx=0x7fce84012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd9a5cd14c21df54 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.915+0000 7fce88c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:28:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.916+0000 7fce88c00640 1 -- 172.21.15.39:0/3956450349 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84012190 con 0x5582f789a390 2024-07-04T08:28:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.916+0000 7fcea804e740 1 -- 172.21.15.39:0/3956450349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce94036030 con 0x5582f789a390 2024-07-04T08:28:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.916+0000 7fce88c00640 1 -- 172.21.15.39:0/3956450349 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7fce8400f920 con 0x5582f789a390 2024-07-04T08:28:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.916+0000 7fce88c00640 1 -- 172.21.15.39:0/3956450349 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce8400fb20 con 0x5582f789a390 2024-07-04T08:28:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.916+0000 7fcea804e740 10 client.28303 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:25.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.916+0000 7fcea804e740 1 -- 172.21.15.39:0/3956450349 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582f7c79f80 con 0x5582f789a390 2024-07-04T08:28:25.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.917+0000 7fce88c00640 1 -- 172.21.15.39:0/3956450349 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce94036030 con 0x5582f789a390 2024-07-04T08:28:25.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.917+0000 7fce88c00640 1 -- 172.21.15.39:0/3956450349 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fce84010cb0 con 0x5582f789a390 2024-07-04T08:28:25.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.917+0000 7fcea804e740 10 client.28303 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:25.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.917+0000 7fcea804e740 10 client.28303 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:28:25.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.917+0000 7fcea804e740 20 client.28303 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:25.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.917+0000 7fcea804e740 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5582f78799f0 0x5582f7c7d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.917+0000 7fcea804e740 4 client.28303 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:28:25.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.917+0000 7fcea804e740 1 -- 172.21.15.39:0/3956450349 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582f7bd7e50 con 0x5582f78799f0 2024-07-04T08:28:25.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.917+0000 7fce9b400640 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5582f78799f0 0x5582f7c7d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.918+0000 7fce9b400640 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5582f78799f0 0x5582f7c7d550 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f91fd9a74fa1241 server_cookie=19e70f5db9a7e765 in_seq=0 out_seq=0 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.918+0000 7fce88c00640 10 client.28303 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fce88c00640 1 -- 172.21.15.39:0/3956450349 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5582f7bd7e50 con 0x5582f78799f0 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fce88c00640 10 client.28303 handle_command_reply: tid=0 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fcea804e740 1 client.28303 shutdown 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fcea804e740 1 -- 172.21.15.39:0/3956450349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 msgr2=0x5582f7c6ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fcea804e740 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c6ba80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fce840126e0 tx=0x7fce84012ba0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fcea804e740 1 -- 172.21.15.39:0/3956450349 shutdown_connections 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fcea804e740 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5582f78799f0 0x5582f7c7d550 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fcea804e740 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 0x5582f7c77290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fcea804e740 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c6ba80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fcea804e740 1 --2- 172.21.15.39:0/3956450349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78921b0 0x5582f7c6b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fce88c00640 0 client.28303 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:28:25.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.919+0000 7fcea804e740 1 -- 172.21.15.39:0/3956450349 >> 172.21.15.39:0/3956450349 conn(0x5582f78af460 msgr2=0x5582f7c62b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:25.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.920+0000 7fcea804e740 1 -- 172.21.15.39:0/3956450349 shutdown_connections 2024-07-04T08:28:25.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.920+0000 7fcea804e740 1 -- 172.21.15.39:0/3956450349 wait complete. 2024-07-04T08:28:25.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.920+0000 7fcea804e740 20 client.28303 trim_cache size 0 max 16384 2024-07-04T08:28:25.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.935+0000 7fcea804e740 1 Processor -- start 2024-07-04T08:28:25.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.935+0000 7fcea804e740 1 -- start start 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fcea804e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f78921b0 0x5582f7c610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fcea804e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f789a390 0x5582f7c77080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fcea804e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 0x5582f7c775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fcea804e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c1b2300 con 0x5582f789a390 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fcea804e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c1b3e40 con 0x5582f78bb070 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fcea804e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c1b29d0 con 0x5582f78921b0 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fce9a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f789a390 0x5582f7c77080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fce9b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f78921b0 0x5582f7c610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fce9a000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f789a390 0x5582f7c77080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47462/0 (socket says 172.21.15.39:47462) 2024-07-04T08:28:25.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fce9a000640 1 -- 172.21.15.39:0/2714577868 learned_addr learned my addr 172.21.15.39:0/2714577868 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:25.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.936+0000 7fce9b400640 1 --2- 172.21.15.39:0/2714577868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 0x5582f7c775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.937+0000 7fce9a000640 1 -- 172.21.15.39:0/2714577868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f78921b0 msgr2=0x5582f7c610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:25.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.937+0000 7fce9a000640 1 --2- 172.21.15.39:0/2714577868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f78921b0 0x5582f7c610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.937+0000 7fce9a000640 1 -- 172.21.15.39:0/2714577868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 msgr2=0x5582f7c775c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:25.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.937+0000 7fce9a000640 1 --2- 172.21.15.39:0/2714577868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 0x5582f7c775c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.937+0000 7fce9a000640 1 -- 172.21.15.39:0/2714577868 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f7c5e1d0 con 0x5582f789a390 2024-07-04T08:28:25.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.937+0000 7fce9a000640 1 --2- 172.21.15.39:0/2714577868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f789a390 0x5582f7c77080 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fce8400f9a0 tx=0x7fce8400f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c38f6f98ed9d1b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:25.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.937+0000 7fce8b400640 1 -- 172.21.15.39:0/2714577868 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84012210 con 0x5582f789a390 2024-07-04T08:28:25.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.937+0000 7fce8b400640 1 -- 172.21.15.39:0/2714577868 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce8400fcc0 con 0x5582f789a390 2024-07-04T08:28:25.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.937+0000 7fce8b400640 1 -- 172.21.15.39:0/2714577868 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84005650 con 0x5582f789a390 2024-07-04T08:28:25.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.938+0000 7fcea804e740 1 -- 172.21.15.39:0/2714577868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f789a390 msgr2=0x5582f7c77080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:25.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.938+0000 7fcea804e740 1 --2- 172.21.15.39:0/2714577868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f789a390 0x5582f7c77080 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fce8400f9a0 tx=0x7fce8400f9d0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.938+0000 7fcea804e740 1 -- 172.21.15.39:0/2714577868 shutdown_connections 2024-07-04T08:28:25.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.938+0000 7fcea804e740 1 --2- 172.21.15.39:0/2714577868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f789a390 0x5582f7c77080 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.938+0000 7fcea804e740 1 --2- 172.21.15.39:0/2714577868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78bb070 0x5582f7c775c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.938+0000 7fcea804e740 1 --2- 172.21.15.39:0/2714577868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f78921b0 0x5582f7c610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.938+0000 7fcea804e740 1 -- 172.21.15.39:0/2714577868 >> 172.21.15.39:0/2714577868 conn(0x5582f78af460 msgr2=0x5582f7c609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:25.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.938+0000 7fcea804e740 1 -- 172.21.15.39:0/2714577868 shutdown_connections 2024-07-04T08:28:25.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.939+0000 7fcea804e740 1 -- 172.21.15.39:0/2714577868 wait complete. 2024-07-04T08:28:25.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.939+0000 7fcea804e740 1 -- start start 2024-07-04T08:28:25.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.939+0000 7fcea804e740 1 Processor -- start 2024-07-04T08:28:25.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fcea804e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c6fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fcea804e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fcea804e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c72f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fcea804e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c1b3770 con 0x5582f78bb070 2024-07-04T08:28:25.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fcea804e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c1b30a0 con 0x5582f78921b0 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fcea804e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c1b02e0 con 0x5582f789a390 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fcea804e740 10 client.0 _ll_register_callbacks cb 0x7fce84009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fce9b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c72f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fce9a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fce9b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c6fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fce9a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44694/0 (socket says 172.21.15.39:44694) 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fce9a000640 1 -- 172.21.15.39:0/2710098251 learned_addr learned my addr 172.21.15.39:0/2710098251 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.940+0000 7fce9b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c6fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34372/0 (socket says 172.21.15.39:34372) 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fce9a000640 1 -- 172.21.15.39:0/2710098251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 msgr2=0x5582f7c6fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fce9a000640 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c6fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fce9a000640 1 -- 172.21.15.39:0/2710098251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 msgr2=0x5582f7c72f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:25.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fce9a000640 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c72f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fce9a000640 1 -- 172.21.15.39:0/2710098251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f7c735a0 con 0x5582f789a390 2024-07-04T08:28:25.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fce9a000640 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c705d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fce84006010 tx=0x7fce84012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf2a63937338146a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:25.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fce88c00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:28:25.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fce88c00640 1 -- 172.21.15.39:0/2710098251 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84005a70 con 0x5582f789a390 2024-07-04T08:28:25.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fcea804e740 1 -- 172.21.15.39:0/2710098251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce8400fcc0 con 0x5582f789a390 2024-07-04T08:28:25.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.941+0000 7fce88c00640 1 -- 172.21.15.39:0/2710098251 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7fce8401cd20 con 0x5582f789a390 2024-07-04T08:28:25.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fce88c00640 1 -- 172.21.15.39:0/2710098251 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce84005390 con 0x5582f789a390 2024-07-04T08:28:25.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fcea804e740 10 client.28309 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:25.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fcea804e740 1 -- 172.21.15.39:0/2710098251 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582f7ce5740 con 0x5582f789a390 2024-07-04T08:28:25.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fce88c00640 1 -- 172.21.15.39:0/2710098251 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce8400fcc0 con 0x5582f789a390 2024-07-04T08:28:25.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fce88c00640 1 -- 172.21.15.39:0/2710098251 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fce8402a440 con 0x5582f789a390 2024-07-04T08:28:25.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fcea804e740 10 client.28309 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:25.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fcea804e740 10 client.28309 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:28:25.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fcea804e740 20 client.28309 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:25.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fcea804e740 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5582f784eda0 0x5582f7ce7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:25.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.942+0000 7fcea804e740 4 client.28309 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.943+0000 7fcea804e740 1 -- 172.21.15.39:0/2710098251 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582f7bd38f0 con 0x5582f784eda0 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.943+0000 7fce9b400640 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5582f784eda0 0x5582f7ce7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.943+0000 7fce9b400640 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5582f784eda0 0x5582f7ce7d40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b23c24124c0161d server_cookie=334f7bf28167a387 in_seq=0 out_seq=0 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.943+0000 7fce88c00640 10 client.28309 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fce88c00640 1 -- 172.21.15.39:0/2710098251 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582f7bd38f0 con 0x5582f784eda0 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fce88c00640 10 client.28309 handle_command_reply: tid=0 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 client.28309 shutdown 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 -- 172.21.15.39:0/2710098251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 msgr2=0x5582f7c705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c705d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fce84006010 tx=0x7fce84012ba0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 -- 172.21.15.39:0/2710098251 shutdown_connections 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5582f784eda0 0x5582f7ce7d40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5582f78bb070 0x5582f7c72f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fce88c00640 0 client.28309 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:28:25.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5582f789a390 0x5582f7c705d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 --2- 172.21.15.39:0/2710098251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5582f78921b0 0x5582f7c6fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 -- 172.21.15.39:0/2710098251 >> 172.21.15.39:0/2710098251 conn(0x5582f78af460 msgr2=0x5582f7c62a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:25.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 -- 172.21.15.39:0/2710098251 shutdown_connections 2024-07-04T08:28:25.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 1 -- 172.21.15.39:0/2710098251 wait complete. 2024-07-04T08:28:25.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.944+0000 7fcea804e740 20 client.28309 trim_cache size 0 max 16384 2024-07-04T08:28:25.936 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:28:25.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 -- 172.21.15.39:0/4290456494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce800b29e0 msgr2=0x7fce800b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:25.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce800b29e0 0x7fce800b4ef0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5582f7c5e5f0 tx=0x7fce8c00cf20 comp rx=0 tx=0).stop 2024-07-04T08:28:25.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 -- 172.21.15.39:0/4290456494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce9c105110 msgr2=0x7fce9c19b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:25.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce9c105110 0x7fce9c19b9c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00e9f0 tx=0x7fce8c00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 -- 172.21.15.39:0/4290456494 shutdown_connections 2024-07-04T08:28:25.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fce800b29e0 0x7fce800b4ef0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce9c10b3a0 0x7fce9c1a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9c10aa50 0x7fce9c19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 --2- 172.21.15.39:0/4290456494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce9c105110 0x7fce9c19b9c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:25.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 -- 172.21.15.39:0/4290456494 >> 172.21.15.39:0/4290456494 conn(0x7fce9c100b80 msgr2=0x7fce9c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:25.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 -- 172.21.15.39:0/4290456494 shutdown_connections 2024-07-04T08:28:25.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:25.945+0000 7fce9be00640 1 -- 172.21.15.39:0/4290456494 wait complete. 2024-07-04T08:28:26.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:28:27.315 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:27.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:27 smithi133 ceph-mon[37050]: pgmap v891: 129 pgs: 129 active+clean; 23 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 18 op/s 2024-07-04T08:28:27.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:27 smithi039 ceph-mon[32062]: pgmap v891: 129 pgs: 129 active+clean; 23 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 18 op/s 2024-07-04T08:28:27.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:27 smithi050 ceph-mon[36577]: pgmap v891: 129 pgs: 129 active+clean; 23 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 89 KiB/s wr, 18 op/s 2024-07-04T08:28:29.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:29 smithi133 ceph-mon[37050]: pgmap v892: 129 pgs: 129 active+clean; 23 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-07-04T08:28:29.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:29 smithi039 ceph-mon[32062]: pgmap v892: 129 pgs: 129 active+clean; 23 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-07-04T08:28:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.609+0000 7f3c17200640 1 -- 172.21.15.39:0/4110582323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c1810aa50 msgr2=0x7f3c1810ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:29.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.609+0000 7f3c17200640 1 --2- 172.21.15.39:0/4110582323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c1810aa50 0x7f3c1810ae60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009900 tx=0x7f3c0002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.610+0000 7f3c17200640 1 -- 172.21.15.39:0/4110582323 shutdown_connections 2024-07-04T08:28:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.610+0000 7f3c17200640 1 --2- 172.21.15.39:0/4110582323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c1810aa50 0x7f3c1810ae60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.610+0000 7f3c17200640 1 --2- 172.21.15.39:0/4110582323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1810b3a0 0x7f3c18111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.610+0000 7f3c17200640 1 --2- 172.21.15.39:0/4110582323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c18105110 0x7f3c18105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.610+0000 7f3c17200640 1 -- 172.21.15.39:0/4110582323 >> 172.21.15.39:0/4110582323 conn(0x7f3c18100b60 msgr2=0x7f3c18102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.610+0000 7f3c17200640 1 -- 172.21.15.39:0/4110582323 shutdown_connections 2024-07-04T08:28:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.610+0000 7f3c17200640 1 -- 172.21.15.39:0/4110582323 wait complete. 2024-07-04T08:28:29.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.611+0000 7f3c17200640 1 Processor -- start 2024-07-04T08:28:29.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.611+0000 7f3c17200640 1 -- start start 2024-07-04T08:28:29.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c17200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c18105110 0x7f3c1819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c17200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c1810aa50 0x7f3c1819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c17200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1810b3a0 0x7f3c181a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c17200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18115010 con 0x7f3c1810aa50 2024-07-04T08:28:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c17200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18114e90 con 0x7f3c18105110 2024-07-04T08:28:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c17200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18115190 con 0x7f3c1810b3a0 2024-07-04T08:28:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c15e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c18105110 0x7f3c1819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c15e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c18105110 0x7f3c1819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34400/0 (socket says 172.21.15.39:34400) 2024-07-04T08:28:29.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c15e00640 1 -- 172.21.15.39:0/2722933249 learned_addr learned my addr 172.21.15.39:0/2722933249 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:29.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.612+0000 7f3c15400640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c1810aa50 0x7f3c1819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.613+0000 7f3c16800640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1810b3a0 0x7f3c181a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.613+0000 7f3c15e00640 1 -- 172.21.15.39:0/2722933249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1810b3a0 msgr2=0x7f3c181a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:29.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.613+0000 7f3c15e00640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1810b3a0 0x7f3c181a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.613+0000 7f3c15e00640 1 -- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c1810aa50 msgr2=0x7f3c1819bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:29.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.613+0000 7f3c15e00640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c1810aa50 0x7f3c1819bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.613+0000 7f3c15e00640 1 -- 172.21.15.39:0/2722933249 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c181a1990 con 0x7f3c18105110 2024-07-04T08:28:29.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.613+0000 7f3c15e00640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c18105110 0x7f3c1819ba20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800ebc0 tx=0x7f3c0800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:29.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.614+0000 7f3c0ea00640 1 -- 172.21.15.39:0/2722933249 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08010630 con 0x7f3c18105110 2024-07-04T08:28:29.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.614+0000 7f3c0ea00640 1 -- 172.21.15.39:0/2722933249 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c08010c80 con 0x7f3c18105110 2024-07-04T08:28:29.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.614+0000 7f3c17200640 1 -- 172.21.15.39:0/2722933249 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c181a1c80 con 0x7f3c18105110 2024-07-04T08:28:29.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.614+0000 7f3c0ea00640 1 -- 172.21.15.39:0/2722933249 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08018b60 con 0x7f3c18105110 2024-07-04T08:28:29.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.614+0000 7f3c17200640 1 -- 172.21.15.39:0/2722933249 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c181a83e0 con 0x7f3c18105110 2024-07-04T08:28:29.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.617+0000 7f3c23781740 1 Processor -- start 2024-07-04T08:28:29.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c23781740 1 -- start start 2024-07-04T08:28:29.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c0ea00640 1 -- 172.21.15.39:0/2722933249 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f3c08018d00 con 0x7f3c18105110 2024-07-04T08:28:29.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c23781740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe641970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c23781740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe2741b0 0x5567fe641eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c23781740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe27c390 0x5567fe6423f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c23781740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5567fe5b7f00 con 0x5567fe27c390 2024-07-04T08:28:29.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c23781740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5567fe5a74a0 con 0x5567fe29d070 2024-07-04T08:28:29.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c23781740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5567fe5ab4e0 con 0x5567fe2741b0 2024-07-04T08:28:29.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c16800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe641970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c16800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe641970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34408/0 (socket says 172.21.15.39:34408) 2024-07-04T08:28:29.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.618+0000 7f3c16800640 1 -- 172.21.15.39:0/2515135928 learned_addr learned my addr 172.21.15.39:0/2515135928 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:29.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.619+0000 7f3c15e00640 1 --2- 172.21.15.39:0/2515135928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe2741b0 0x5567fe641eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.619+0000 7f3c15400640 1 --2- 172.21.15.39:0/2515135928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe27c390 0x5567fe6423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.619+0000 7f3c16800640 1 -- 172.21.15.39:0/2515135928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe2741b0 msgr2=0x5567fe641eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:29.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.619+0000 7f3c16800640 1 --2- 172.21.15.39:0/2515135928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe2741b0 0x5567fe641eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.619+0000 7f3c16800640 1 -- 172.21.15.39:0/2515135928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe27c390 msgr2=0x5567fe6423f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:29.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.619+0000 7f3c16800640 1 --2- 172.21.15.39:0/2515135928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe27c390 0x5567fe6423f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.619+0000 7f3c16800640 1 -- 172.21.15.39:0/2515135928 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567fe53f670 con 0x5567fe29d070 2024-07-04T08:28:29.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.619+0000 7f3c16800640 1 --2- 172.21.15.39:0/2515135928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe641970 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009ca0 tx=0x7f3c10011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7780dd4f7cac7f4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:29.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.619+0000 7f3c07400640 1 -- 172.21.15.39:0/2515135928 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1001e070 con 0x5567fe29d070 2024-07-04T08:28:29.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.620+0000 7f3c07400640 1 -- 172.21.15.39:0/2515135928 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c10009e50 con 0x5567fe29d070 2024-07-04T08:28:29.609 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.620+0000 7f3c07400640 1 -- 172.21.15.39:0/2515135928 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000ea50 con 0x5567fe29d070 2024-07-04T08:28:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.620+0000 7f3c0ea00640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3bfc0b29e0 0x7f3bfc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.620+0000 7f3c0ea00640 1 -- 172.21.15.39:0/2722933249 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f3c080e2f30 con 0x7f3c18105110 2024-07-04T08:28:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.620+0000 7f3c15e00640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3bfc0b29e0 0x7f3bfc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.620+0000 7f3c23781740 1 -- 172.21.15.39:0/2515135928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 msgr2=0x5567fe641970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:29.611 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.621+0000 7f3c23781740 1 --2- 172.21.15.39:0/2515135928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe641970 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009ca0 tx=0x7f3c10011810 comp rx=0 tx=0).stop 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.621+0000 7f3c23781740 1 -- 172.21.15.39:0/2515135928 shutdown_connections 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.621+0000 7f3c23781740 1 --2- 172.21.15.39:0/2515135928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe27c390 0x5567fe6423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.621+0000 7f3c23781740 1 --2- 172.21.15.39:0/2515135928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe2741b0 0x5567fe641eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.621+0000 7f3c23781740 1 --2- 172.21.15.39:0/2515135928 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe641970 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.621+0000 7f3c23781740 1 -- 172.21.15.39:0/2515135928 >> 172.21.15.39:0/2515135928 conn(0x5567fe291460 msgr2=0x5567fe63f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.621+0000 7f3c23781740 1 -- 172.21.15.39:0/2515135928 shutdown_connections 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.621+0000 7f3c23781740 1 -- 172.21.15.39:0/2515135928 wait complete. 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.621+0000 7f3c15e00640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3bfc0b29e0 0x7f3bfc0b4ef0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5567fe6405f0 tx=0x7f3c0800cf90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.622+0000 7f3c23781740 1 -- start start 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.622+0000 7f3c23781740 1 Processor -- start 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c23781740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe64d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c23781740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe27c390 0x5567fe64da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.612 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c23781740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe29d070 0x5567fe659470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c23781740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5567fe40b700 con 0x5567fe2741b0 2024-07-04T08:28:29.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c23781740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5567fe54ebd0 con 0x5567fe27c390 2024-07-04T08:28:29.613 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c23781740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5567fe650150 con 0x5567fe29d070 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c23781740 10 client.0 _ll_register_callbacks cb 0x5567fe5a0590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c15400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe27c390 0x5567fe64da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c16800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe64d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c15400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe27c390 0x5567fe64da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:34412/0 (socket says 172.21.15.39:34412) 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c15400640 1 -- 172.21.15.39:0/745782188 learned_addr learned my addr 172.21.15.39:0/745782188 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c16800640 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe29d070 0x5567fe659470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c15400640 1 -- 172.21.15.39:0/745782188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe29d070 msgr2=0x5567fe659470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.623+0000 7f3c15400640 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe29d070 0x5567fe659470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.624+0000 7f3c15400640 1 -- 172.21.15.39:0/745782188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 msgr2=0x5567fe64d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.624+0000 7f3c15400640 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe64d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.615 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.624+0000 7f3c15400640 1 -- 172.21.15.39:0/745782188 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567fe308650 con 0x5567fe27c390 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.624+0000 7f3c16800640 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe64d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.624+0000 7f3c15400640 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe27c390 0x5567fe64da40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3c1806ae70 tx=0x7f3c00032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e587b6b8cbe5f30 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.624+0000 7f3bfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.624+0000 7f3bfbe00640 1 -- 172.21.15.39:0/745782188 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00035040 con 0x5567fe27c390 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.624+0000 7f3c23781740 1 -- 172.21.15.39:0/745782188 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c10009e50 con 0x5567fe27c390 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.624+0000 7f3bfbe00640 1 -- 172.21.15.39:0/745782188 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f3c000316a0 con 0x5567fe27c390 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.625+0000 7f3bfbe00640 1 -- 172.21.15.39:0/745782188 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c0004e050 con 0x5567fe27c390 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.625+0000 7f3c23781740 10 client.28385 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.625+0000 7f3c23781740 1 -- 172.21.15.39:0/745782188 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567fe65c190 con 0x5567fe27c390 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.625+0000 7f3bfbe00640 1 -- 172.21.15.39:0/745782188 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c10009e50 con 0x5567fe27c390 2024-07-04T08:28:29.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.625+0000 7f3bfbe00640 1 -- 172.21.15.39:0/745782188 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3c000054b0 con 0x5567fe27c390 2024-07-04T08:28:29.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.626+0000 7f3c23781740 10 client.28385 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:29.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.626+0000 7f3c23781740 10 client.28385 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:28:29.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.626+0000 7f3c23781740 20 client.28385 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:29.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.626+0000 7f3c23781740 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5567fe25b9f0 0x5567fe65f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.626+0000 7f3c23781740 4 client.28385 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:28:29.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.626+0000 7f3c23781740 1 -- 172.21.15.39:0/745782188 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567fe5b9e50 con 0x5567fe25b9f0 2024-07-04T08:28:29.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.626+0000 7f3c16800640 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5567fe25b9f0 0x5567fe65f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.627+0000 7f3c16800640 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5567fe25b9f0 0x5567fe65f680 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8d7b1c0ee52834b server_cookie=673e4973b7990808 in_seq=0 out_seq=0 2024-07-04T08:28:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.627+0000 7f3bfbe00640 10 client.28385 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:28:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3bfbe00640 1 -- 172.21.15.39:0/745782188 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5567fe5b9e50 con 0x5567fe25b9f0 2024-07-04T08:28:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3bfbe00640 10 client.28385 handle_command_reply: tid=0 2024-07-04T08:28:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3c23781740 1 client.28385 shutdown 2024-07-04T08:28:29.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3c23781740 1 -- 172.21.15.39:0/745782188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe27c390 msgr2=0x5567fe64da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3c23781740 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe27c390 0x5567fe64da40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3c1806ae70 tx=0x7f3c00032da0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3c23781740 1 -- 172.21.15.39:0/745782188 shutdown_connections 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3c23781740 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5567fe25b9f0 0x5567fe65f680 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3c23781740 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe29d070 0x5567fe659470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3c23781740 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe27c390 0x5567fe64da40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3bfbe00640 0 client.28385 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3c23781740 1 --2- 172.21.15.39:0/745782188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe64d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.628+0000 7f3c23781740 1 -- 172.21.15.39:0/745782188 >> 172.21.15.39:0/745782188 conn(0x5567fe291460 msgr2=0x5567fe644b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.629+0000 7f3c23781740 1 -- 172.21.15.39:0/745782188 shutdown_connections 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.629+0000 7f3c23781740 1 -- 172.21.15.39:0/745782188 wait complete. 2024-07-04T08:28:29.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.629+0000 7f3c23781740 20 client.28385 trim_cache size 0 max 16384 2024-07-04T08:28:29.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:29 smithi050 ceph-mon[36577]: pgmap v892: 129 pgs: 129 active+clean; 23 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-07-04T08:28:29.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.645+0000 7f3c23781740 1 Processor -- start 2024-07-04T08:28:29.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.645+0000 7f3c23781740 1 -- start start 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c23781740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe63e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c23781740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 0x5567fe64ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c23781740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe659260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c23781740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181b2470 con 0x5567fe2741b0 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c23781740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181b3fb0 con 0x5567fe29d070 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c23781740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181b2b40 con 0x5567fe27c390 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c16800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe63e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c16800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe63e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47494/0 (socket says 172.21.15.39:47494) 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c16800640 1 -- 172.21.15.39:0/4069489302 learned_addr learned my addr 172.21.15.39:0/4069489302 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:29.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c16800640 1 --2- 172.21.15.39:0/4069489302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe659260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.646+0000 7f3c15400640 1 --2- 172.21.15.39:0/4069489302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 0x5567fe64ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.647+0000 7f3c16800640 1 -- 172.21.15.39:0/4069489302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 msgr2=0x5567fe64ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:29.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.647+0000 7f3c16800640 1 --2- 172.21.15.39:0/4069489302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 0x5567fe64ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.647+0000 7f3c16800640 1 -- 172.21.15.39:0/4069489302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 msgr2=0x5567fe659260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:29.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.647+0000 7f3c16800640 1 --2- 172.21.15.39:0/4069489302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe659260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.647+0000 7f3c16800640 1 -- 172.21.15.39:0/4069489302 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567fe6447d0 con 0x5567fe2741b0 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.647+0000 7f3c16800640 1 --2- 172.21.15.39:0/4069489302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe63e7f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000e520 tx=0x7f3c1000e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba92418ec609b9f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.647+0000 7f3c06a00640 1 -- 172.21.15.39:0/4069489302 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1001e040 con 0x5567fe2741b0 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.648+0000 7f3c06a00640 1 -- 172.21.15.39:0/4069489302 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c1000ee30 con 0x5567fe2741b0 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.648+0000 7f3c06a00640 1 -- 172.21.15.39:0/4069489302 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10003e60 con 0x5567fe2741b0 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.648+0000 7f3c23781740 1 -- 172.21.15.39:0/4069489302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 msgr2=0x5567fe63e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.648+0000 7f3c23781740 1 --2- 172.21.15.39:0/4069489302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe63e7f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000e520 tx=0x7f3c1000e690 comp rx=0 tx=0).stop 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.649+0000 7f3c23781740 1 -- 172.21.15.39:0/4069489302 shutdown_connections 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.649+0000 7f3c23781740 1 --2- 172.21.15.39:0/4069489302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe659260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.649+0000 7f3c23781740 1 --2- 172.21.15.39:0/4069489302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 0x5567fe64ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.649+0000 7f3c23781740 1 --2- 172.21.15.39:0/4069489302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe63e7f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.649+0000 7f3c23781740 1 -- 172.21.15.39:0/4069489302 >> 172.21.15.39:0/4069489302 conn(0x5567fe291460 msgr2=0x5567fe64e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:29.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.649+0000 7f3c23781740 1 -- 172.21.15.39:0/4069489302 shutdown_connections 2024-07-04T08:28:29.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.649+0000 7f3c23781740 1 -- 172.21.15.39:0/4069489302 wait complete. 2024-07-04T08:28:29.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.649+0000 7f3c23781740 1 -- start start 2024-07-04T08:28:29.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.649+0000 7f3c23781740 1 Processor -- start 2024-07-04T08:28:29.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c23781740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe655900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c23781740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 0x5567fe650e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c23781740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe651340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c23781740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181b38e0 con 0x5567fe2741b0 2024-07-04T08:28:29.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c23781740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181b3210 con 0x5567fe29d070 2024-07-04T08:28:29.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c23781740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181b0450 con 0x5567fe27c390 2024-07-04T08:28:29.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c23781740 10 client.0 _ll_register_callbacks cb 0x7f3c10005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:29.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c16800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe655900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.644 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c16800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe655900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47506/0 (socket says 172.21.15.39:47506) 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.650+0000 7f3c16800640 1 -- 172.21.15.39:0/4021959944 learned_addr learned my addr 172.21.15.39:0/4021959944 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3c15400640 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 0x5567fe650e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3c16800640 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe651340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3c16800640 1 -- 172.21.15.39:0/4021959944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 msgr2=0x5567fe650e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3c16800640 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 0x5567fe650e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3c16800640 1 -- 172.21.15.39:0/4021959944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 msgr2=0x5567fe651340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3c16800640 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe651340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3c16800640 1 -- 172.21.15.39:0/4021959944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567fe651930 con 0x5567fe2741b0 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3c15400640 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 0x5567fe650e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3c16800640 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe655900 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3c10014040 tx=0x7f3c10005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12ef4843eb53ebd server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.651+0000 7f3bfbe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:28:29.645 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.652+0000 7f3bfbe00640 1 -- 172.21.15.39:0/4021959944 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1001e040 con 0x5567fe2741b0 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.652+0000 7f3bfbe00640 1 -- 172.21.15.39:0/4021959944 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f3c1000e980 con 0x5567fe2741b0 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.652+0000 7f3c23781740 1 -- 172.21.15.39:0/4021959944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c1000ee30 con 0x5567fe2741b0 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.652+0000 7f3bfbe00640 1 -- 172.21.15.39:0/4021959944 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c1000eb80 con 0x5567fe2741b0 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.652+0000 7f3c23781740 10 client.18270 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.652+0000 7f3c23781740 1 -- 172.21.15.39:0/4021959944 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5567fe6c7bb0 con 0x5567fe2741b0 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.652+0000 7f3bfbe00640 1 -- 172.21.15.39:0/4021959944 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c1000ee30 con 0x5567fe2741b0 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.653+0000 7f3bfbe00640 1 -- 172.21.15.39:0/4021959944 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3c10024040 con 0x5567fe2741b0 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.653+0000 7f3c23781740 10 client.18270 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.653+0000 7f3c23781740 10 client.18270 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.653+0000 7f3c23781740 20 client.18270 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.653+0000 7f3c23781740 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5567fe230da0 0x5567fe6ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.653+0000 7f3c23781740 4 client.18270 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:28:29.646 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.653+0000 7f3c23781740 1 -- 172.21.15.39:0/4021959944 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567fe5b58f0 con 0x5567fe230da0 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.653+0000 7f3c15400640 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5567fe230da0 0x5567fe6ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c15400640 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5567fe230da0 0x5567fe6ca140 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a40f37b935f5340 server_cookie=ae846a91b4d202c2 in_seq=0 out_seq=0 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3bfbe00640 10 client.18270 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3bfbe00640 1 -- 172.21.15.39:0/4021959944 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567fe5b58f0 con 0x5567fe230da0 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3bfbe00640 10 client.18270 handle_command_reply: tid=0 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c23781740 1 client.18270 shutdown 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c23781740 1 -- 172.21.15.39:0/4021959944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 msgr2=0x5567fe655900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c23781740 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe655900 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3c10014040 tx=0x7f3c10005670 comp rx=0 tx=0).stop 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c23781740 1 -- 172.21.15.39:0/4021959944 shutdown_connections 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c23781740 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5567fe230da0 0x5567fe6ca140 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c23781740 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5567fe29d070 0x5567fe651340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c23781740 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5567fe27c390 0x5567fe650e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c23781740 1 --2- 172.21.15.39:0/4021959944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5567fe2741b0 0x5567fe655900 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.647 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3c23781740 1 -- 172.21.15.39:0/4021959944 >> 172.21.15.39:0/4021959944 conn(0x5567fe291460 msgr2=0x5567fe63e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:29.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.654+0000 7f3bfbe00640 0 client.18270 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:28:29.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.655+0000 7f3c23781740 1 -- 172.21.15.39:0/4021959944 shutdown_connections 2024-07-04T08:28:29.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.655+0000 7f3c23781740 1 -- 172.21.15.39:0/4021959944 wait complete. 2024-07-04T08:28:29.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.655+0000 7f3c23781740 20 client.18270 trim_cache size 0 max 16384 2024-07-04T08:28:29.648 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:28:29.648 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.655+0000 7f3c17200640 1 -- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3bfc0b29e0 msgr2=0x7f3bfc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.655+0000 7f3c17200640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3bfc0b29e0 0x7f3bfc0b4ef0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5567fe6405f0 tx=0x7f3c0800cf90 comp rx=0 tx=0).stop 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.655+0000 7f3c17200640 1 -- 172.21.15.39:0/2722933249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c18105110 msgr2=0x7f3c1819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.656+0000 7f3c17200640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c18105110 0x7f3c1819ba20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800ebc0 tx=0x7f3c0800c6a0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.656+0000 7f3c17200640 1 -- 172.21.15.39:0/2722933249 shutdown_connections 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.656+0000 7f3c17200640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f3bfc0b29e0 0x7f3bfc0b4ef0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.656+0000 7f3c17200640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c1810aa50 0x7f3c1819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.656+0000 7f3c17200640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1810b3a0 0x7f3c181a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.656+0000 7f3c17200640 1 --2- 172.21.15.39:0/2722933249 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3c18105110 0x7f3c1819ba20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.656+0000 7f3c17200640 1 -- 172.21.15.39:0/2722933249 >> 172.21.15.39:0/2722933249 conn(0x7f3c18100b60 msgr2=0x7f3c18102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.656+0000 7f3c17200640 1 -- 172.21.15.39:0/2722933249 shutdown_connections 2024-07-04T08:28:29.649 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:29.656+0000 7f3c17200640 1 -- 172.21.15.39:0/2722933249 wait complete. 2024-07-04T08:28:30.565 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:28:31.076 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:31.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:31 smithi133 ceph-mon[37050]: pgmap v893: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 84 KiB/s wr, 16 op/s 2024-07-04T08:28:31.562 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:31 smithi039 ceph-mon[32062]: pgmap v893: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 84 KiB/s wr, 16 op/s 2024-07-04T08:28:31.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:31 smithi050 ceph-mon[36577]: pgmap v893: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 84 KiB/s wr, 16 op/s 2024-07-04T08:28:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.412+0000 7f281ec00640 1 -- 172.21.15.39:0/1891069285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818113aa0 msgr2=0x7f2818115ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:33.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.412+0000 7f281ec00640 1 --2- 172.21.15.39:0/1891069285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818113aa0 0x7f2818115ec0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2810009860 tx=0x7f281002a340 comp rx=0 tx=0).stop 2024-07-04T08:28:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.412+0000 7f281ec00640 1 -- 172.21.15.39:0/1891069285 shutdown_connections 2024-07-04T08:28:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.412+0000 7f281ec00640 1 --2- 172.21.15.39:0/1891069285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818113aa0 0x7f2818115ec0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.412+0000 7f281ec00640 1 --2- 172.21.15.39:0/1891069285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28180777e0 0x7f2818113560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.412+0000 7f281ec00640 1 --2- 172.21.15.39:0/1891069285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818076e90 0x7f28180772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.412+0000 7f281ec00640 1 -- 172.21.15.39:0/1891069285 >> 172.21.15.39:0/1891069285 conn(0x7f2818100b30 msgr2=0x7f2818102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.412+0000 7f281ec00640 1 -- 172.21.15.39:0/1891069285 shutdown_connections 2024-07-04T08:28:33.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.412+0000 7f281ec00640 1 -- 172.21.15.39:0/1891069285 wait complete. 2024-07-04T08:28:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.414+0000 7f281ec00640 1 Processor -- start 2024-07-04T08:28:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.414+0000 7f281ec00640 1 -- start start 2024-07-04T08:28:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281ec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818076e90 0x7f281819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281ec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28180777e0 0x7f28181a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281ec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818113aa0 0x7f28181a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281ec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2818119420 con 0x7f2818076e90 2024-07-04T08:28:33.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281ec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f28181192a0 con 0x7f28180777e0 2024-07-04T08:28:33.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281ec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f28181195a0 con 0x7f2818113aa0 2024-07-04T08:28:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818076e90 0x7f281819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818076e90 0x7f281819fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38850/0 (socket says 172.21.15.39:38850) 2024-07-04T08:28:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281d800640 1 -- 172.21.15.39:0/1733194550 learned_addr learned my addr 172.21.15.39:0/1733194550 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281e200640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818113aa0 0x7f28181a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.415+0000 7f281ce00640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28180777e0 0x7f28181a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.416+0000 7f281d800640 1 -- 172.21.15.39:0/1733194550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818113aa0 msgr2=0x7f28181a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:33.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.416+0000 7f281d800640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818113aa0 0x7f28181a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.416+0000 7f281d800640 1 -- 172.21.15.39:0/1733194550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28180777e0 msgr2=0x7f28181a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.416+0000 7f281d800640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28180777e0 0x7f28181a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.416+0000 7f281d800640 1 -- 172.21.15.39:0/1733194550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28181a5d80 con 0x7f2818076e90 2024-07-04T08:28:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.416+0000 7f281d800640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818076e90 0x7f281819fe10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f280c00eb90 tx=0x7f280c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.416+0000 7f2816000640 1 -- 172.21.15.39:0/1733194550 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c004d20 con 0x7f2818076e90 2024-07-04T08:28:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.416+0000 7f2816000640 1 -- 172.21.15.39:0/1733194550 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f280c00ee90 con 0x7f2818076e90 2024-07-04T08:28:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.417+0000 7f2816000640 1 -- 172.21.15.39:0/1733194550 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c010670 con 0x7f2818076e90 2024-07-04T08:28:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.417+0000 7f281ec00640 1 -- 172.21.15.39:0/1733194550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28181a6070 con 0x7f2818076e90 2024-07-04T08:28:33.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.417+0000 7f281ec00640 1 -- 172.21.15.39:0/1733194550 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181ac7d0 con 0x7f2818076e90 2024-07-04T08:28:33.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.420+0000 7f2826775740 1 Processor -- start 2024-07-04T08:28:33.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.420+0000 7f2826775740 1 -- start start 2024-07-04T08:28:33.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f2816000640 1 -- 172.21.15.39:0/1733194550 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f280c005020 con 0x7f2818076e90 2024-07-04T08:28:33.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f2826775740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762cc070 0x55ad76670970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f2826775740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76670eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f2826775740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad766713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f2826775740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ad765e6f00 con 0x55ad762a31b0 2024-07-04T08:28:33.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f2826775740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad765d64a0 con 0x55ad762ab390 2024-07-04T08:28:33.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f2826775740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad765da4e0 con 0x55ad762cc070 2024-07-04T08:28:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f281ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad766713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f281e200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762cc070 0x55ad76670970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f281d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76670eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f281ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad766713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:39268/0 (socket says 172.21.15.39:39268) 2024-07-04T08:28:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.421+0000 7f281ce00640 1 -- 172.21.15.39:0/3753297784 learned_addr learned my addr 172.21.15.39:0/3753297784 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f281ce00640 1 -- 172.21.15.39:0/3753297784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762cc070 msgr2=0x55ad76670970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:33.411 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f281ce00640 1 --2- 172.21.15.39:0/3753297784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762cc070 0x55ad76670970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f281ce00640 1 -- 172.21.15.39:0/3753297784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 msgr2=0x55ad76670eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f281ce00640 1 --2- 172.21.15.39:0/3753297784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76670eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f281ce00640 1 -- 172.21.15.39:0/3753297784 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad7656e670 con 0x55ad762ab390 2024-07-04T08:28:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f281d800640 1 --2- 172.21.15.39:0/3753297784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76670eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:28:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f281ce00640 1 --2- 172.21.15.39:0/3753297784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad766713f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2804009c30 tx=0x7f2804011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731399e34b6ba48c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f280aa00640 1 -- 172.21.15.39:0/3753297784 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804004020 con 0x55ad762ab390 2024-07-04T08:28:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f280aa00640 1 -- 172.21.15.39:0/3753297784 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28040041c0 con 0x55ad762ab390 2024-07-04T08:28:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.422+0000 7f280aa00640 1 -- 172.21.15.39:0/3753297784 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804019690 con 0x55ad762ab390 2024-07-04T08:28:33.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.423+0000 7f2816000640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28000b29e0 0x7f28000b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.423+0000 7f2816000640 1 -- 172.21.15.39:0/1733194550 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f280c0e2f60 con 0x7f2818076e90 2024-07-04T08:28:33.413 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.423+0000 7f281e200640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28000b29e0 0x7f28000b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.423+0000 7f2826775740 1 -- 172.21.15.39:0/3753297784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 msgr2=0x55ad766713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.423+0000 7f2826775740 1 --2- 172.21.15.39:0/3753297784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad766713f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2804009c30 tx=0x7f2804011810 comp rx=0 tx=0).stop 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.424+0000 7f2826775740 1 -- 172.21.15.39:0/3753297784 shutdown_connections 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.424+0000 7f2826775740 1 --2- 172.21.15.39:0/3753297784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76670eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.424+0000 7f2826775740 1 --2- 172.21.15.39:0/3753297784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad766713f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.424+0000 7f2826775740 1 --2- 172.21.15.39:0/3753297784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762cc070 0x55ad76670970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.424+0000 7f2826775740 1 -- 172.21.15.39:0/3753297784 >> 172.21.15.39:0/3753297784 conn(0x55ad762c0460 msgr2=0x55ad7666e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.424+0000 7f2826775740 1 -- 172.21.15.39:0/3753297784 shutdown_connections 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.424+0000 7f2826775740 1 -- 172.21.15.39:0/3753297784 wait complete. 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.424+0000 7f281e200640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28000b29e0 0x7f28000b4ef0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f28100303f0 tx=0x7f281002c040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.425+0000 7f2826775740 1 -- start start 2024-07-04T08:28:33.414 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.425+0000 7f2826775740 1 Processor -- start 2024-07-04T08:28:33.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.425+0000 7f2826775740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762a31b0 0x55ad7667c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f2826775740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 0x55ad7667cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f2826775740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 0x55ad76688470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f2826775740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ad7643a700 con 0x55ad762cc070 2024-07-04T08:28:33.416 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f2826775740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ad7657dbd0 con 0x55ad762a31b0 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f2826775740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad7667f150 con 0x55ad762ab390 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f2826775740 10 client.0 _ll_register_callbacks cb 0x55ad765cf590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762a31b0 0x55ad7667c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281e200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 0x55ad76688470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762a31b0 0x55ad7667c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:39276/0 (socket says 172.21.15.39:39276) 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281d800640 1 -- 172.21.15.39:0/2231545597 learned_addr learned my addr 172.21.15.39:0/2231545597 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281ce00640 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 0x55ad7667cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281d800640 1 -- 172.21.15.39:0/2231545597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 msgr2=0x55ad7667cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281d800640 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 0x55ad7667cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281d800640 1 -- 172.21.15.39:0/2231545597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 msgr2=0x55ad76688470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281d800640 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 0x55ad76688470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.417 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.426+0000 7f281d800640 1 -- 172.21.15.39:0/2231545597 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad76337650 con 0x55ad762a31b0 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.427+0000 7f281ce00640 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 0x55ad7667cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.427+0000 7f281d800640 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762a31b0 0x55ad7667c5c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f280c0049d0 tx=0x7f280c0e08b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f46b9d03b89d6b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.427+0000 7f27ff400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.427+0000 7f27ff400640 1 -- 172.21.15.39:0/2231545597 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0a9990 con 0x55ad762a31b0 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.427+0000 7f2826775740 1 -- 172.21.15.39:0/2231545597 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28040041c0 con 0x55ad762a31b0 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.427+0000 7f27ff400640 1 -- 172.21.15.39:0/2231545597 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f280c0aa2f0 con 0x55ad762a31b0 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.427+0000 7f27ff400640 1 -- 172.21.15.39:0/2231545597 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f280c0aa580 con 0x55ad762a31b0 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.427+0000 7f2826775740 10 client.28403 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.427+0000 7f2826775740 1 -- 172.21.15.39:0/2231545597 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad7668b1f0 con 0x55ad762a31b0 2024-07-04T08:28:33.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.428+0000 7f27ff400640 1 -- 172.21.15.39:0/2231545597 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28040041c0 con 0x55ad762a31b0 2024-07-04T08:28:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.428+0000 7f27ff400640 1 -- 172.21.15.39:0/2231545597 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f280c0a90e0 con 0x55ad762a31b0 2024-07-04T08:28:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.428+0000 7f2826775740 10 client.28403 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.428+0000 7f2826775740 10 client.28403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:28:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.428+0000 7f2826775740 20 client.28403 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.428+0000 7f2826775740 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ad7628a9f0 0x55ad7668e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.428+0000 7f2826775740 4 client.28403 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:28:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.428+0000 7f2826775740 1 -- 172.21.15.39:0/2231545597 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad765e8e50 con 0x55ad7628a9f0 2024-07-04T08:28:33.420 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.429+0000 7f281ce00640 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ad7628a9f0 0x55ad7668e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.429+0000 7f281ce00640 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ad7628a9f0 0x55ad7668e6e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9161d0ca1420190f server_cookie=f607c4d717fcb9a6 in_seq=0 out_seq=0 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.429+0000 7f27ff400640 10 client.28403 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.430+0000 7f27ff400640 1 -- 172.21.15.39:0/2231545597 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ad765e8e50 con 0x55ad7628a9f0 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.430+0000 7f27ff400640 10 client.28403 handle_command_reply: tid=0 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.430+0000 7f2826775740 1 client.28403 shutdown 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.430+0000 7f2826775740 1 -- 172.21.15.39:0/2231545597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762a31b0 msgr2=0x55ad7667c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762a31b0 0x55ad7667c5c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f280c0049d0 tx=0x7f280c0e08b0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 1 -- 172.21.15.39:0/2231545597 shutdown_connections 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ad7628a9f0 0x55ad7668e6e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 0x55ad76688470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 0x55ad7667cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 1 --2- 172.21.15.39:0/2231545597 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762a31b0 0x55ad7667c5c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 1 -- 172.21.15.39:0/2231545597 >> 172.21.15.39:0/2231545597 conn(0x55ad762c0460 msgr2=0x55ad76673b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f27ff400640 0 client.28403 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 1 -- 172.21.15.39:0/2231545597 shutdown_connections 2024-07-04T08:28:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 1 -- 172.21.15.39:0/2231545597 wait complete. 2024-07-04T08:28:33.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.431+0000 7f2826775740 20 client.28403 trim_cache size 0 max 16384 2024-07-04T08:28:33.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.446+0000 7f2826775740 1 Processor -- start 2024-07-04T08:28:33.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:33 smithi039 ceph-mon[32062]: pgmap v894: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-07-04T08:28:33.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.446+0000 7f2826775740 1 -- start start 2024-07-04T08:28:33.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f2826775740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762a31b0 0x55ad76672360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f2826775740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad7667dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f2826775740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 0x55ad7666de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f2826775740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b6740 con 0x55ad762cc070 2024-07-04T08:28:33.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f2826775740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b8280 con 0x55ad762ab390 2024-07-04T08:28:33.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f2826775740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b6e10 con 0x55ad762a31b0 2024-07-04T08:28:33.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f281d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762a31b0 0x55ad76672360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f281ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad7667dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f281d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762a31b0 0x55ad76672360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:52182/0 (socket says 172.21.15.39:52182) 2024-07-04T08:28:33.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f281ce00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad7667dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:39290/0 (socket says 172.21.15.39:39290) 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f281ce00640 1 -- 172.21.15.39:0/1175268006 learned_addr learned my addr 172.21.15.39:0/1175268006 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.447+0000 7f281e200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 0x55ad7666de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.448+0000 7f281ce00640 1 -- 172.21.15.39:0/1175268006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762a31b0 msgr2=0x55ad76672360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.448+0000 7f281ce00640 1 --2- 172.21.15.39:0/1175268006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762a31b0 0x55ad76672360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.448+0000 7f281ce00640 1 -- 172.21.15.39:0/1175268006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 msgr2=0x55ad7666de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.448+0000 7f281ce00640 1 --2- 172.21.15.39:0/1175268006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 0x55ad7666de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.448+0000 7f281ce00640 1 -- 172.21.15.39:0/1175268006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad76673200 con 0x55ad762ab390 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.448+0000 7f281d800640 1 --2- 172.21.15.39:0/1175268006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762a31b0 0x55ad76672360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.448+0000 7f281ce00640 1 --2- 172.21.15.39:0/1175268006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad7667dff0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2804011d10 tx=0x7f2804009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e9c205b5dc93cb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.449+0000 7f280a000640 1 -- 172.21.15.39:0/1175268006 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28040043f0 con 0x55ad762ab390 2024-07-04T08:28:33.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.449+0000 7f280a000640 1 -- 172.21.15.39:0/1175268006 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28040040c0 con 0x55ad762ab390 2024-07-04T08:28:33.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.449+0000 7f280a000640 1 -- 172.21.15.39:0/1175268006 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804003770 con 0x55ad762ab390 2024-07-04T08:28:33.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.450+0000 7f2826775740 1 -- 172.21.15.39:0/1175268006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 msgr2=0x55ad7667dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.450+0000 7f2826775740 1 --2- 172.21.15.39:0/1175268006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad7667dff0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2804011d10 tx=0x7f2804009640 comp rx=0 tx=0).stop 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.450+0000 7f2826775740 1 -- 172.21.15.39:0/1175268006 shutdown_connections 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.450+0000 7f2826775740 1 --2- 172.21.15.39:0/1175268006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762cc070 0x55ad7666de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.450+0000 7f2826775740 1 --2- 172.21.15.39:0/1175268006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762ab390 0x55ad7667dff0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.450+0000 7f2826775740 1 --2- 172.21.15.39:0/1175268006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762a31b0 0x55ad76672360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.450+0000 7f2826775740 1 -- 172.21.15.39:0/1175268006 >> 172.21.15.39:0/1175268006 conn(0x55ad762c0460 msgr2=0x55ad7667d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.450+0000 7f2826775740 1 -- 172.21.15.39:0/1175268006 shutdown_connections 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.450+0000 7f2826775740 1 -- 172.21.15.39:0/1175268006 wait complete. 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.451+0000 7f2826775740 1 -- start start 2024-07-04T08:28:33.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.451+0000 7f2826775740 1 Processor -- start 2024-07-04T08:28:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.451+0000 7f2826775740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76684c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.451+0000 7f2826775740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 0x55ad76687e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.451+0000 7f2826775740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762cc070 0x55ad76688540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.451+0000 7f2826775740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b7bb0 con 0x55ad762a31b0 2024-07-04T08:28:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.451+0000 7f2826775740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b74e0 con 0x55ad762cc070 2024-07-04T08:28:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.451+0000 7f2826775740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f28181b4720 con 0x55ad762ab390 2024-07-04T08:28:33.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.451+0000 7f2826775740 10 client.0 _ll_register_callbacks cb 0x7f280c0a5d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76684c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76684c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38888/0 (socket says 172.21.15.39:38888) 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281d800640 1 -- 172.21.15.39:0/1108692246 learned_addr learned my addr 172.21.15.39:0/1108692246 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281ce00640 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 0x55ad76687e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281e200640 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762cc070 0x55ad76688540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281d800640 1 -- 172.21.15.39:0/1108692246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 msgr2=0x55ad76687e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281d800640 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 0x55ad76687e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281d800640 1 -- 172.21.15.39:0/1108692246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762cc070 msgr2=0x55ad76688540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281d800640 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762cc070 0x55ad76688540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281d800640 1 -- 172.21.15.39:0/1108692246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad76680670 con 0x55ad762a31b0 2024-07-04T08:28:33.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.452+0000 7f281e200640 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762cc070 0x55ad76688540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:28:33.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.453+0000 7f281d800640 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76684c20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f280c00cba0 tx=0x7f280c0a9d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40eae660f7f615e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:33.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.453+0000 7f27ff400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:28:33.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.453+0000 7f27ff400640 1 -- 172.21.15.39:0/1108692246 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0ae230 con 0x55ad762a31b0 2024-07-04T08:28:33.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.453+0000 7f27ff400640 1 -- 172.21.15.39:0/1108692246 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f280c0e6110 con 0x55ad762a31b0 2024-07-04T08:28:33.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.453+0000 7f2826775740 1 -- 172.21.15.39:0/1108692246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28040040c0 con 0x55ad762a31b0 2024-07-04T08:28:33.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.453+0000 7f27ff400640 1 -- 172.21.15.39:0/1108692246 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f280c0e6310 con 0x55ad762a31b0 2024-07-04T08:28:33.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.453+0000 7f2826775740 10 client.18288 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:33.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.453+0000 7f2826775740 1 -- 172.21.15.39:0/1108692246 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad766f6f60 con 0x55ad762a31b0 2024-07-04T08:28:33.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.454+0000 7f27ff400640 1 -- 172.21.15.39:0/1108692246 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28040040c0 con 0x55ad762a31b0 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.454+0000 7f27ff400640 1 -- 172.21.15.39:0/1108692246 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f280c0a8b50 con 0x55ad762a31b0 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.454+0000 7f2826775740 10 client.18288 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.454+0000 7f2826775740 10 client.18288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.454+0000 7f2826775740 20 client.18288 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.454+0000 7f2826775740 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ad7625fda0 0x55ad766f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.454+0000 7f2826775740 4 client.18288 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.454+0000 7f2826775740 1 -- 172.21.15.39:0/1108692246 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad765e48f0 con 0x55ad7625fda0 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.454+0000 7f281ce00640 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ad7625fda0 0x55ad766f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.455+0000 7f281ce00640 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ad7625fda0 0x55ad766f95c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b2d4b6dc8b7bdc server_cookie=686063e2fefbbbed in_seq=0 out_seq=0 2024-07-04T08:28:33.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.455+0000 7f27ff400640 10 client.18288 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.455+0000 7f27ff400640 1 -- 172.21.15.39:0/1108692246 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad765e48f0 con 0x55ad7625fda0 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.455+0000 7f27ff400640 10 client.18288 handle_command_reply: tid=0 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.455+0000 7f2826775740 1 client.18288 shutdown 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 -- 172.21.15.39:0/1108692246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 msgr2=0x55ad76684c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76684c20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f280c00cba0 tx=0x7f280c0a9d20 comp rx=0 tx=0).stop 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 -- 172.21.15.39:0/1108692246 shutdown_connections 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55ad7625fda0 0x55ad766f95c0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ad762cc070 0x55ad76688540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad762ab390 0x55ad76687e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 --2- 172.21.15.39:0/1108692246 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ad762a31b0 0x55ad76684c20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 -- 172.21.15.39:0/1108692246 >> 172.21.15.39:0/1108692246 conn(0x55ad762c0460 msgr2=0x55ad76672030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f27ff400640 0 client.18288 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 -- 172.21.15.39:0/1108692246 shutdown_connections 2024-07-04T08:28:33.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 1 -- 172.21.15.39:0/1108692246 wait complete. 2024-07-04T08:28:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.456+0000 7f2826775740 20 client.18288 trim_cache size 0 max 16384 2024-07-04T08:28:33.448 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:28:33.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 -- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28000b29e0 msgr2=0x7f28000b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28000b29e0 0x7f28000b4ef0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f28100303f0 tx=0x7f281002c040 comp rx=0 tx=0).stop 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 -- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818076e90 msgr2=0x7f281819fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818076e90 0x7f281819fe10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f280c00eb90 tx=0x7f280c00c6a0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 -- 172.21.15.39:0/1733194550 shutdown_connections 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28000b29e0 0x7f28000b4ef0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2818113aa0 0x7f28181a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28180777e0 0x7f28181a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 --2- 172.21.15.39:0/1733194550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2818076e90 0x7f281819fe10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.457+0000 7f281ec00640 1 -- 172.21.15.39:0/1733194550 >> 172.21.15.39:0/1733194550 conn(0x7f2818100b30 msgr2=0x7f2818115b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:33.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.458+0000 7f281ec00640 1 -- 172.21.15.39:0/1733194550 shutdown_connections 2024-07-04T08:28:33.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:33.458+0000 7f281ec00640 1 -- 172.21.15.39:0/1733194550 wait complete. 2024-07-04T08:28:33.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:33 smithi133 ceph-mon[37050]: pgmap v894: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-07-04T08:28:33.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:33 smithi050 ceph-mon[36577]: pgmap v894: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 KiB/s wr, 15 op/s 2024-07-04T08:28:34.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:34 smithi039 ceph-mon[32062]: pgmap v895: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 57 KiB/s wr, 13 op/s 2024-07-04T08:28:34.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:34 smithi050 ceph-mon[36577]: pgmap v895: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 57 KiB/s wr, 13 op/s 2024-07-04T08:28:34.804 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:34 smithi133 ceph-mon[37050]: pgmap v895: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 57 KiB/s wr, 13 op/s 2024-07-04T08:28:35.263 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:28:35.264 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:28:35.766 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:37.524 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:37 smithi039 ceph-mon[32062]: pgmap v896: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 89 KiB/s wr, 17 op/s 2024-07-04T08:28:37.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:37 smithi133 ceph-mon[37050]: pgmap v896: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 89 KiB/s wr, 17 op/s 2024-07-04T08:28:37.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:37 smithi050 ceph-mon[36577]: pgmap v896: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 89 KiB/s wr, 17 op/s 2024-07-04T08:28:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.123+0000 7fa137000640 1 -- 172.21.15.39:0/2647452922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa130105300 msgr2=0x7fa130105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.123+0000 7fa137000640 1 --2- 172.21.15.39:0/2647452922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa130105300 0x7fa130105710 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa120009900 tx=0x7fa12002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.112 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.123+0000 7fa137000640 1 -- 172.21.15.39:0/2647452922 shutdown_connections 2024-07-04T08:28:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.123+0000 7fa137000640 1 --2- 172.21.15.39:0/2647452922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa13010b590 0x7fa130111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.123+0000 7fa137000640 1 --2- 172.21.15.39:0/2647452922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13010ac40 0x7fa13010b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.123+0000 7fa137000640 1 --2- 172.21.15.39:0/2647452922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa130105300 0x7fa130105710 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.123+0000 7fa137000640 1 -- 172.21.15.39:0/2647452922 >> 172.21.15.39:0/2647452922 conn(0x7fa130100d10 msgr2=0x7fa130103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.123+0000 7fa137000640 1 -- 172.21.15.39:0/2647452922 shutdown_connections 2024-07-04T08:28:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.123+0000 7fa137000640 1 -- 172.21.15.39:0/2647452922 wait complete. 2024-07-04T08:28:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.125+0000 7fa137000640 1 Processor -- start 2024-07-04T08:28:38.113 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.125+0000 7fa137000640 1 -- start start 2024-07-04T08:28:38.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.125+0000 7fa137000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa130105300 0x7fa13019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.114 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.125+0000 7fa137000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa13010ac40 0x7fa13019c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa137000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13010b590 0x7fa1301a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa137000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301152b0 con 0x7fa13010ac40 2024-07-04T08:28:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa137000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa130115130 con 0x7fa130105300 2024-07-04T08:28:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa137000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa130115430 con 0x7fa13010b590 2024-07-04T08:28:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa130105300 0x7fa13019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa130105300 0x7fa13019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:39308/0 (socket says 172.21.15.39:39308) 2024-07-04T08:28:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa12fe00640 1 -- 172.21.15.39:0/1228952310 learned_addr learned my addr 172.21.15.39:0/1228952310 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa12f400640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa13010ac40 0x7fa13019c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.115 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa135000640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13010b590 0x7fa1301a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa12fe00640 1 -- 172.21.15.39:0/1228952310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13010b590 msgr2=0x7fa1301a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa12fe00640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13010b590 0x7fa1301a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.116 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa12fe00640 1 -- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa13010ac40 msgr2=0x7fa13019c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa12fe00640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa13010ac40 0x7fa13019c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.126+0000 7fa12fe00640 1 -- 172.21.15.39:0/1228952310 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1301a1c10 con 0x7fa130105300 2024-07-04T08:28:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.127+0000 7fa12fe00640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa130105300 0x7fa13019bca0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa120005ab0 tx=0x7fa120003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.127+0000 7fa12cc00640 1 -- 172.21.15.39:0/1228952310 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12003f070 con 0x7fa130105300 2024-07-04T08:28:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.127+0000 7fa137000640 1 -- 172.21.15.39:0/1228952310 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1301a1ea0 con 0x7fa130105300 2024-07-04T08:28:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.127+0000 7fa137000640 1 -- 172.21.15.39:0/1228952310 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1301a8660 con 0x7fa130105300 2024-07-04T08:28:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.128+0000 7fa12cc00640 1 -- 172.21.15.39:0/1228952310 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa12003b910 con 0x7fa130105300 2024-07-04T08:28:38.117 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.128+0000 7fa12cc00640 1 -- 172.21.15.39:0/1228952310 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120049060 con 0x7fa130105300 2024-07-04T08:28:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.131+0000 7fa13d6ed740 1 Processor -- start 2024-07-04T08:28:38.119 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.131+0000 7fa13d6ed740 1 -- start start 2024-07-04T08:28:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.131+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b64640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.131+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563935797490 0x563935b64b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.120 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.131+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.131+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563935a8e8e0 con 0x56393579f670 2024-07-04T08:28:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.131+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563935adaec0 con 0x563935797490 2024-07-04T08:28:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.131+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563935aca460 con 0x5639357c0350 2024-07-04T08:28:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa12cc00640 1 -- 172.21.15.39:0/1228952310 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fa120042070 con 0x7fa130105300 2024-07-04T08:28:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa12f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa135000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b64640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa12f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b650c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38912/0 (socket says 172.21.15.39:38912) 2024-07-04T08:28:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa12f400640 1 -- 172.21.15.39:0/1908258419 learned_addr learned my addr 172.21.15.39:0/1908258419 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:38.121 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa12fe00640 1 --2- 172.21.15.39:0/1908258419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563935797490 0x563935b64b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa12f400640 1 -- 172.21.15.39:0/1908258419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 msgr2=0x563935b64640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:38.122 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa12f400640 1 --2- 172.21.15.39:0/1908258419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b64640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa12f400640 1 -- 172.21.15.39:0/1908258419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563935797490 msgr2=0x563935b64b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.132+0000 7fa12f400640 1 --2- 172.21.15.39:0/1908258419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563935797490 0x563935b64b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.133+0000 7fa12f400640 1 -- 172.21.15.39:0/1908258419 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563935a62280 con 0x56393579f670 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.133+0000 7fa135000640 1 --2- 172.21.15.39:0/1908258419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b64640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.133+0000 7fa12f400640 1 --2- 172.21.15.39:0/1908258419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b650c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009c20 tx=0x7fa11c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7504eb65595509 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.133+0000 7fa125600640 1 -- 172.21.15.39:0/1908258419 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c004230 con 0x56393579f670 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.133+0000 7fa125600640 1 -- 172.21.15.39:0/1908258419 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11c0043d0 con 0x56393579f670 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.133+0000 7fa125600640 1 -- 172.21.15.39:0/1908258419 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c004ce0 con 0x56393579f670 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.134+0000 7fa12cc00640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa1140b2960 0x7fa1140b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.134+0000 7fa12cc00640 1 -- 172.21.15.39:0/1228952310 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7fa120102c80 con 0x7fa130105300 2024-07-04T08:28:38.123 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.134+0000 7fa135000640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa1140b2960 0x7fa1140b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.134+0000 7fa13d6ed740 1 -- 172.21.15.39:0/1908258419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 msgr2=0x563935b650c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:38.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.134+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/1908258419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b650c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009c20 tx=0x7fa11c011810 comp rx=0 tx=0).stop 2024-07-04T08:28:38.124 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.135+0000 7fa13d6ed740 1 -- 172.21.15.39:0/1908258419 shutdown_connections 2024-07-04T08:28:38.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.135+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/1908258419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b650c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.135+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/1908258419 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563935797490 0x563935b64b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.135+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/1908258419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b64640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.135+0000 7fa13d6ed740 1 -- 172.21.15.39:0/1908258419 >> 172.21.15.39:0/1908258419 conn(0x5639357b4740 msgr2=0x563935b62530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:38.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.135+0000 7fa13d6ed740 1 -- 172.21.15.39:0/1908258419 shutdown_connections 2024-07-04T08:28:38.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.135+0000 7fa13d6ed740 1 -- 172.21.15.39:0/1908258419 wait complete. 2024-07-04T08:28:38.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.135+0000 7fa135000640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa1140b2960 0x7fa1140b4e70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa128009fd0 tx=0x7fa128009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:38.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.136+0000 7fa13d6ed740 1 -- start start 2024-07-04T08:28:38.125 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.136+0000 7fa13d6ed740 1 Processor -- start 2024-07-04T08:28:38.126 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.136+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b6fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.136+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.136+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b7c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.136+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563935a71760 con 0x563935797490 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.136+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563935ace4a0 con 0x56393579f670 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.136+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56393592e890 con 0x5639357c0350 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa13d6ed740 10 client.0 _ll_register_callbacks cb 0x563935a722b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa135000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b7c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b6fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:39314/0 (socket says 172.21.15.39:39314) 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12f400640 1 -- 172.21.15.39:0/2642784573 learned_addr learned my addr 172.21.15.39:0/2642784573 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa135000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b7c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:52238/0 (socket says 172.21.15.39:52238) 2024-07-04T08:28:38.127 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12fe00640 1 -- 172.21.15.39:0/2642784573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 msgr2=0x563935b7c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12fe00640 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b7c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12fe00640 1 -- 172.21.15.39:0/2642784573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 msgr2=0x563935b704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12fe00640 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12fe00640 1 -- 172.21.15.39:0/2642784573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56393582b930 con 0x563935797490 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.137+0000 7fa12f400640 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.138+0000 7fa12fe00640 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b6fee0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa120100010 tx=0x7fa1201005b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144d09e14a539629 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.138+0000 7fa11a000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.138+0000 7fa11a000640 1 -- 172.21.15.39:0/2642784573 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1200b1050 con 0x563935797490 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.138+0000 7fa11a000640 1 -- 172.21.15.39:0/2642784573 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7fa1200d2780 con 0x563935797490 2024-07-04T08:28:38.128 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.138+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2642784573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11c0043d0 con 0x563935797490 2024-07-04T08:28:38.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.138+0000 7fa11a000640 1 -- 172.21.15.39:0/2642784573 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1200c9a20 con 0x563935797490 2024-07-04T08:28:38.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa13d6ed740 10 client.18300 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:38.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2642784573 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563935b7ebd0 con 0x563935797490 2024-07-04T08:28:38.129 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa11a000640 1 -- 172.21.15.39:0/2642784573 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11c0043d0 con 0x563935797490 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa11a000640 1 -- 172.21.15.39:0/2642784573 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa12010a040 con 0x563935797490 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa13d6ed740 10 client.18300 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa13d6ed740 10 client.18300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa13d6ed740 20 client.18300 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56393577ecd0 0x563935b82050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa13d6ed740 4 client.18300 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.139+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2642784573 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563935adce10 con 0x56393577ecd0 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.140+0000 7fa12f400640 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56393577ecd0 0x563935b82050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.140+0000 7fa12f400640 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56393577ecd0 0x563935b82050 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a5b2d70c6b1599 server_cookie=4bf71375646256b in_seq=0 out_seq=0 2024-07-04T08:28:38.130 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.140+0000 7fa11a000640 10 client.18300 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:38.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.141+0000 7fa11a000640 1 -- 172.21.15.39:0/2642784573 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563935adce10 con 0x56393577ecd0 2024-07-04T08:28:38.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.141+0000 7fa11a000640 10 client.18300 handle_command_reply: tid=0 2024-07-04T08:28:38.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.141+0000 7fa13d6ed740 1 client.18300 shutdown 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2642784573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 msgr2=0x563935b6fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b6fee0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa120100010 tx=0x7fa1201005b0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2642784573 shutdown_connections 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56393577ecd0 0x563935b82050 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b7c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2642784573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b6fee0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2642784573 >> 172.21.15.39:0/2642784573 conn(0x5639357b4740 msgr2=0x563935b677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa11a000640 0 client.18300 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2642784573 shutdown_connections 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2642784573 wait complete. 2024-07-04T08:28:38.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.142+0000 7fa13d6ed740 20 client.18300 trim_cache size 0 max 16384 2024-07-04T08:28:38.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.163+0000 7fa13d6ed740 1 Processor -- start 2024-07-04T08:28:38.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.163+0000 7fa13d6ed740 1 -- start start 2024-07-04T08:28:38.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.163+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563935797490 0x563935b66030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.163+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.163+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639357c0350 0x563935b7be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.163+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa11000fec0 con 0x56393579f670 2024-07-04T08:28:38.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.163+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa110011a00 con 0x5639357c0350 2024-07-04T08:28:38.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.163+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa110010590 con 0x563935797490 2024-07-04T08:28:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563935797490 0x563935b66030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563935797490 0x563935b66030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:52254/0 (socket says 172.21.15.39:52254) 2024-07-04T08:28:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12fe00640 1 -- 172.21.15.39:0/2566788003 learned_addr learned my addr 172.21.15.39:0/2566788003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa135000640 1 --2- 172.21.15.39:0/2566788003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639357c0350 0x563935b7be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12fe00640 1 -- 172.21.15.39:0/2566788003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639357c0350 msgr2=0x563935b7be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:38.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12fe00640 1 --2- 172.21.15.39:0/2566788003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639357c0350 0x563935b7be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12fe00640 1 -- 172.21.15.39:0/2566788003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 msgr2=0x563935b71b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:38.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12fe00640 1 --2- 172.21.15.39:0/2566788003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b71b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12fe00640 1 -- 172.21.15.39:0/2566788003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563935b711a0 con 0x563935797490 2024-07-04T08:28:38.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.164+0000 7fa12f400640 1 --2- 172.21.15.39:0/2566788003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b71b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:38.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.165+0000 7fa12fe00640 1 --2- 172.21.15.39:0/2566788003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563935797490 0x563935b66030 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa120000c00 tx=0x7fa120031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aabad643f5a977f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.165+0000 7fa124c00640 1 -- 172.21.15.39:0/2566788003 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1200b1040 con 0x563935797490 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.165+0000 7fa124c00640 1 -- 172.21.15.39:0/2566788003 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1200eae50 con 0x563935797490 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.165+0000 7fa124c00640 1 -- 172.21.15.39:0/2566788003 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1200c53e0 con 0x563935797490 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.166+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2566788003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563935797490 msgr2=0x563935b66030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.166+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2566788003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563935797490 0x563935b66030 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa120000c00 tx=0x7fa120031400 comp rx=0 tx=0).stop 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.166+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2566788003 shutdown_connections 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.166+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2566788003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56393579f670 0x563935b71b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.166+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2566788003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639357c0350 0x563935b7be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.166+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/2566788003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563935797490 0x563935b66030 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.166+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2566788003 >> 172.21.15.39:0/2566788003 conn(0x5639357b4740 msgr2=0x563935b71400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.166+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2566788003 shutdown_connections 2024-07-04T08:28:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.166+0000 7fa13d6ed740 1 -- 172.21.15.39:0/2566788003 wait complete. 2024-07-04T08:28:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.167+0000 7fa13d6ed740 1 -- start start 2024-07-04T08:28:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.167+0000 7fa13d6ed740 1 Processor -- start 2024-07-04T08:28:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b73f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b74470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa13d6ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b749b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fa110011330 con 0x563935797490 2024-07-04T08:28:38.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa110010c60 con 0x56393579f670 2024-07-04T08:28:38.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa13d6ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa11000dea0 con 0x5639357c0350 2024-07-04T08:28:38.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa13d6ed740 10 client.0 _ll_register_callbacks cb 0x7fa1200c6110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b73f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa12f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b74470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b73f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38924/0 (socket says 172.21.15.39:38924) 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa12fe00640 1 -- 172.21.15.39:0/3605799174 learned_addr learned my addr 172.21.15.39:0/3605799174 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.168+0000 7fa135000640 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.169+0000 7fa12f400640 1 -- 172.21.15.39:0/3605799174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 msgr2=0x563935b749b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.169+0000 7fa12f400640 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b749b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.169+0000 7fa12f400640 1 -- 172.21.15.39:0/3605799174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 msgr2=0x563935b73f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.169+0000 7fa12f400640 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b73f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.169+0000 7fa12f400640 1 -- 172.21.15.39:0/3605799174 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563935be9890 con 0x56393579f670 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.169+0000 7fa12fe00640 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b73f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:38.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.169+0000 7fa12f400640 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b74470 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x563935b827b0 tx=0x7fa11c0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=851195dff6c946ca server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:38.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.169+0000 7fa11a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:28:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.169+0000 7fa11a000640 1 -- 172.21.15.39:0/3605799174 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c01e040 con 0x56393579f670 2024-07-04T08:28:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.170+0000 7fa13d6ed740 1 -- 172.21.15.39:0/3605799174 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1200eae50 con 0x56393579f670 2024-07-04T08:28:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.170+0000 7fa11a000640 1 -- 172.21.15.39:0/3605799174 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7fa11c019700 con 0x56393579f670 2024-07-04T08:28:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.170+0000 7fa11a000640 1 -- 172.21.15.39:0/3605799174 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa11c03c050 con 0x56393579f670 2024-07-04T08:28:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.170+0000 7fa13d6ed740 10 client.28427 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.170+0000 7fa13d6ed740 1 -- 172.21.15.39:0/3605799174 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563935beab30 con 0x56393579f670 2024-07-04T08:28:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.171+0000 7fa11a000640 1 -- 172.21.15.39:0/3605799174 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1200eae50 con 0x56393579f670 2024-07-04T08:28:38.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.171+0000 7fa11a000640 1 -- 172.21.15.39:0/3605799174 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa11c024040 con 0x56393579f670 2024-07-04T08:28:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.171+0000 7fa13d6ed740 10 client.28427 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.171+0000 7fa13d6ed740 10 client.28427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:28:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.171+0000 7fa13d6ed740 20 client.28427 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.171+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563935754080 0x563935bed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.171+0000 7fa13d6ed740 4 client.28427 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:28:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.171+0000 7fa13d6ed740 1 -- 172.21.15.39:0/3605799174 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563935ad88b0 con 0x563935754080 2024-07-04T08:28:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.171+0000 7fa12fe00640 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563935754080 0x563935bed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:38.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.172+0000 7fa12fe00640 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563935754080 0x563935bed180 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782bbb291d6e274 server_cookie=af4d36e72f6cd4c7 in_seq=0 out_seq=0 2024-07-04T08:28:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.172+0000 7fa11a000640 10 client.28427 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.173+0000 7fa11a000640 1 -- 172.21.15.39:0/3605799174 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563935ad88b0 con 0x563935754080 2024-07-04T08:28:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.173+0000 7fa11a000640 10 client.28427 handle_command_reply: tid=0 2024-07-04T08:28:38.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.173+0000 7fa13d6ed740 1 client.28427 shutdown 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 -- 172.21.15.39:0/3605799174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 msgr2=0x563935b74470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b74470 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x563935b827b0 tx=0x7fa11c0047e0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 -- 172.21.15.39:0/3605799174 shutdown_connections 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563935754080 0x563935bed180 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639357c0350 0x563935b749b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56393579f670 0x563935b74470 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 --2- 172.21.15.39:0/3605799174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563935797490 0x563935b73f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa11a000640 0 client.28427 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 -- 172.21.15.39:0/3605799174 >> 172.21.15.39:0/3605799174 conn(0x5639357b4740 msgr2=0x563935b65de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:38.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 -- 172.21.15.39:0/3605799174 shutdown_connections 2024-07-04T08:28:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.174+0000 7fa13d6ed740 1 -- 172.21.15.39:0/3605799174 wait complete. 2024-07-04T08:28:38.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.175+0000 7fa13d6ed740 20 client.28427 trim_cache size 0 max 16384 2024-07-04T08:28:38.164 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:28:38.164 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:28:38.164 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "34eb089e-a1f5-4f50-8dbc-155432b282eb", 2024-07-04T08:28:38.164 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:28:38.164 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:28:38.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.175+0000 7fa137000640 1 -- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa1140b2960 msgr2=0x7fa1140b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:38.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.175+0000 7fa137000640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa1140b2960 0x7fa1140b4e70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa128009fd0 tx=0x7fa128009500 comp rx=0 tx=0).stop 2024-07-04T08:28:38.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.175+0000 7fa137000640 1 -- 172.21.15.39:0/1228952310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa130105300 msgr2=0x7fa13019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:38.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.175+0000 7fa137000640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa130105300 0x7fa13019bca0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa120005ab0 tx=0x7fa120003c10 comp rx=0 tx=0).stop 2024-07-04T08:28:38.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.176+0000 7fa137000640 1 -- 172.21.15.39:0/1228952310 shutdown_connections 2024-07-04T08:28:38.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.176+0000 7fa137000640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fa1140b2960 0x7fa1140b4e70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.176+0000 7fa137000640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa13010ac40 0x7fa13019c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.176+0000 7fa137000640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa13010b590 0x7fa1301a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.176+0000 7fa137000640 1 --2- 172.21.15.39:0/1228952310 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa130105300 0x7fa13019bca0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:38.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.176+0000 7fa137000640 1 -- 172.21.15.39:0/1228952310 >> 172.21.15.39:0/1228952310 conn(0x7fa130100d10 msgr2=0x7fa1301040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:38.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.176+0000 7fa137000640 1 -- 172.21.15.39:0/1228952310 shutdown_connections 2024-07-04T08:28:38.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:38.176+0000 7fa137000640 1 -- 172.21.15.39:0/1228952310 wait complete. 2024-07-04T08:28:39.073 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:28:39.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:39 smithi039 ceph-mon[32062]: pgmap v897: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:28:39.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:39 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:28:39.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:39 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:28:39.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:39 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:28:39.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:39 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:28:39.553 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:39.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:39 smithi133 ceph-mon[37050]: pgmap v897: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:28:39.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:39 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:28:39.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:39 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:28:39.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:39 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:28:39.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:39 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:28:39.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:39 smithi050 ceph-mon[36577]: pgmap v897: 129 pgs: 129 active+clean; 23 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:28:39.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:39 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:28:39.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:39 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:28:39.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:39 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:28:39.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:39 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:28:41.465 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:41 smithi039 ceph-mon[32062]: pgmap v898: 129 pgs: 129 active+clean; 24 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 90 KiB/s wr, 17 op/s 2024-07-04T08:28:41.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:41 smithi133 ceph-mon[37050]: pgmap v898: 129 pgs: 129 active+clean; 24 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 90 KiB/s wr, 17 op/s 2024-07-04T08:28:41.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:41 smithi050 ceph-mon[36577]: pgmap v898: 129 pgs: 129 active+clean; 24 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 90 KiB/s wr, 17 op/s 2024-07-04T08:28:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.872+0000 7f456b200640 1 -- 172.21.15.39:0/1110596439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456c10f460 msgr2=0x7f456c1118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.872+0000 7f456b200640 1 --2- 172.21.15.39:0/1110596439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456c10f460 0x7f456c1118a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4564009ca0 tx=0x7f456402a8c0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.873+0000 7f456b200640 1 -- 172.21.15.39:0/1110596439 shutdown_connections 2024-07-04T08:28:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.873+0000 7f456b200640 1 --2- 172.21.15.39:0/1110596439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456c10f460 0x7f456c1118a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.873+0000 7f456b200640 1 --2- 172.21.15.39:0/1110596439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f456c075650 0x7f456c10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.873+0000 7f456b200640 1 --2- 172.21.15.39:0/1110596439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456c074d00 0x7f456c075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.873+0000 7f456b200640 1 -- 172.21.15.39:0/1110596439 >> 172.21.15.39:0/1110596439 conn(0x7f456c0fe990 msgr2=0x7f456c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.873+0000 7f456b200640 1 -- 172.21.15.39:0/1110596439 shutdown_connections 2024-07-04T08:28:41.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.873+0000 7f456b200640 1 -- 172.21.15.39:0/1110596439 wait complete. 2024-07-04T08:28:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.874+0000 7f456b200640 1 Processor -- start 2024-07-04T08:28:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.874+0000 7f456b200640 1 -- start start 2024-07-04T08:28:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.875+0000 7f456b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456c074d00 0x7f456c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.875+0000 7f456b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456c075650 0x7f456c19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.875+0000 7f456b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f456c10f460 0x7f456c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.875+0000 7f456b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f456c115010 con 0x7f456c074d00 2024-07-04T08:28:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.875+0000 7f456b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f456c114e90 con 0x7f456c075650 2024-07-04T08:28:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.875+0000 7f456b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f456c115190 con 0x7f456c10f460 2024-07-04T08:28:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.875+0000 7f4569e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456c074d00 0x7f456c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.876+0000 7f456a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f456c10f460 0x7f456c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.876+0000 7f4569e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456c074d00 0x7f456c19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42626/0 (socket says 172.21.15.39:42626) 2024-07-04T08:28:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.876+0000 7f4569e00640 1 -- 172.21.15.39:0/3978075159 learned_addr learned my addr 172.21.15.39:0/3978075159 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:41.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.876+0000 7f4561400640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456c075650 0x7f456c19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.876+0000 7f456a800640 1 -- 172.21.15.39:0/3978075159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456c075650 msgr2=0x7f456c19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:41.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.876+0000 7f456a800640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456c075650 0x7f456c19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.876+0000 7f456a800640 1 -- 172.21.15.39:0/3978075159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456c074d00 msgr2=0x7f456c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.876+0000 7f456a800640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456c074d00 0x7f456c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.876+0000 7f456a800640 1 -- 172.21.15.39:0/3978075159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c1a19d0 con 0x7f456c10f460 2024-07-04T08:28:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.877+0000 7f456a800640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f456c10f460 0x7f456c1a12a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f456402ade0 tx=0x7f4564031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.877+0000 7f4563400640 1 -- 172.21.15.39:0/3978075159 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564003b70 con 0x7f456c10f460 2024-07-04T08:28:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.877+0000 7f4563400640 1 -- 172.21.15.39:0/3978075159 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4564031910 con 0x7f456c10f460 2024-07-04T08:28:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.877+0000 7f456b200640 1 -- 172.21.15.39:0/3978075159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c1a1c60 con 0x7f456c10f460 2024-07-04T08:28:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.877+0000 7f4563400640 1 -- 172.21.15.39:0/3978075159 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456402f950 con 0x7f456c10f460 2024-07-04T08:28:41.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.877+0000 7f456b200640 1 -- 172.21.15.39:0/3978075159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c1a8420 con 0x7f456c10f460 2024-07-04T08:28:41.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.880+0000 7f4577149740 1 Processor -- start 2024-07-04T08:28:41.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.881+0000 7f4563400640 1 -- 172.21.15.39:0/3978075159 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f456402faf0 con 0x7f456c10f460 2024-07-04T08:28:41.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.881+0000 7f4577149740 1 -- start start 2024-07-04T08:28:41.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4577149740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f3231070 0x5584f35d5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4577149740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f32081b0 0x5584f35d5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4577149740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35d63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4577149740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5584f354bed0 con 0x5584f32081b0 2024-07-04T08:28:41.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4577149740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5584f353b470 con 0x5584f3231070 2024-07-04T08:28:41.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4577149740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584f353f4b0 con 0x5584f3210390 2024-07-04T08:28:41.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4569e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f32081b0 0x5584f35d5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4561400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f3231070 0x5584f35d5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4569e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f32081b0 0x5584f35d5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42642/0 (socket says 172.21.15.39:42642) 2024-07-04T08:28:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4569e00640 1 -- 172.21.15.39:0/512819056 learned_addr learned my addr 172.21.15.39:0/512819056 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.882+0000 7f4561400640 1 --2- 172.21.15.39:0/512819056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35d63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f4561400640 1 -- 172.21.15.39:0/512819056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 msgr2=0x5584f35d63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f4561400640 1 --2- 172.21.15.39:0/512819056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35d63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f4561400640 1 -- 172.21.15.39:0/512819056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f32081b0 msgr2=0x5584f35d5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f4561400640 1 --2- 172.21.15.39:0/512819056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f32081b0 0x5584f35d5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f4561400640 1 -- 172.21.15.39:0/512819056 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f34d3640 con 0x5584f3231070 2024-07-04T08:28:41.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f4569e00640 1 --2- 172.21.15.39:0/512819056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f32081b0 0x5584f35d5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:41.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f4561400640 1 --2- 172.21.15.39:0/512819056 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f3231070 0x5584f35d5940 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f456c111810 tx=0x7f45540096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9e49c949839027a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:41.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f455b400640 1 -- 172.21.15.39:0/512819056 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554004020 con 0x5584f3231070 2024-07-04T08:28:41.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f455b400640 1 -- 172.21.15.39:0/512819056 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45540041c0 con 0x5584f3231070 2024-07-04T08:28:41.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.883+0000 7f455b400640 1 -- 172.21.15.39:0/512819056 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455400f770 con 0x5584f3231070 2024-07-04T08:28:41.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.884+0000 7f4563400640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f45500b2960 0x7f45500b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.884+0000 7f4563400640 1 -- 172.21.15.39:0/3978075159 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f4564103b60 con 0x7f456c10f460 2024-07-04T08:28:41.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.884+0000 7f4569e00640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f45500b2960 0x7f45500b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.884+0000 7f4577149740 1 -- 172.21.15.39:0/512819056 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f3231070 msgr2=0x5584f35d5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.884+0000 7f4577149740 1 --2- 172.21.15.39:0/512819056 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f3231070 0x5584f35d5940 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f456c111810 tx=0x7f45540096a0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.885+0000 7f4577149740 1 -- 172.21.15.39:0/512819056 shutdown_connections 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.885+0000 7f4577149740 1 --2- 172.21.15.39:0/512819056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f32081b0 0x5584f35d5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.885+0000 7f4577149740 1 --2- 172.21.15.39:0/512819056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35d63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.885+0000 7f4577149740 1 --2- 172.21.15.39:0/512819056 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f3231070 0x5584f35d5940 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.885+0000 7f4577149740 1 -- 172.21.15.39:0/512819056 >> 172.21.15.39:0/512819056 conn(0x5584f3225460 msgr2=0x5584f35d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.885+0000 7f4577149740 1 -- 172.21.15.39:0/512819056 shutdown_connections 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.885+0000 7f4569e00640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f45500b2960 0x7f45500b4e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5584f35d45c0 tx=0x7f455c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.885+0000 7f4577149740 1 -- 172.21.15.39:0/512819056 wait complete. 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.886+0000 7f4577149740 1 -- start start 2024-07-04T08:28:41.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.886+0000 7f4577149740 1 Processor -- start 2024-07-04T08:28:41.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.886+0000 7f4577149740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.886+0000 7f4577149740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35e1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.886+0000 7f4577149740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4577149740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5584f339f700 con 0x5584f3231070 2024-07-04T08:28:41.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4577149740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5584f34e2ba0 con 0x5584f32081b0 2024-07-04T08:28:41.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4577149740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584f35e4120 con 0x5584f3210390 2024-07-04T08:28:41.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4577149740 10 client.0 _ll_register_callbacks cb 0x5584f3534560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4569e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4569e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38730/0 (socket says 172.21.15.39:38730) 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4569e00640 1 -- 172.21.15.39:0/2606537483 learned_addr learned my addr 172.21.15.39:0/2606537483 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4561400640 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35e1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f456a800640 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4569e00640 1 -- 172.21.15.39:0/2606537483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 msgr2=0x5584f35e1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4569e00640 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35e1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4569e00640 1 -- 172.21.15.39:0/2606537483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 msgr2=0x5584f35ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4569e00640 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.887+0000 7f4569e00640 1 -- 172.21.15.39:0/2606537483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f329c650 con 0x5584f32081b0 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.888+0000 7f4569e00640 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e15c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f455c00fb10 tx=0x7f455c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2dfe9fba97e72b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:41.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.888+0000 7f454fe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:28:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.888+0000 7f454fe00640 1 -- 172.21.15.39:0/2606537483 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c02d060 con 0x5584f32081b0 2024-07-04T08:28:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.888+0000 7f454fe00640 1 -- 172.21.15.39:0/2606537483 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f455c018030 con 0x5584f32081b0 2024-07-04T08:28:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.888+0000 7f4577149740 1 -- 172.21.15.39:0/2606537483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45540041c0 con 0x5584f32081b0 2024-07-04T08:28:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.889+0000 7f454fe00640 1 -- 172.21.15.39:0/2606537483 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f455c034050 con 0x5584f32081b0 2024-07-04T08:28:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.889+0000 7f4577149740 10 client.28439 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:41.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.889+0000 7f4577149740 1 -- 172.21.15.39:0/2606537483 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5584f35f0240 con 0x5584f32081b0 2024-07-04T08:28:41.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.889+0000 7f454fe00640 1 -- 172.21.15.39:0/2606537483 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45540041c0 con 0x5584f32081b0 2024-07-04T08:28:41.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.890+0000 7f454fe00640 1 -- 172.21.15.39:0/2606537483 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f455c028990 con 0x5584f32081b0 2024-07-04T08:28:41.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.890+0000 7f4577149740 10 client.28439 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:41.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.890+0000 7f4577149740 10 client.28439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:28:41.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.890+0000 7f4577149740 20 client.28439 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.890+0000 7f4577149740 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5584f31ef9f0 0x5584f35f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.890+0000 7f4577149740 4 client.28439 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:28:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.890+0000 7f4577149740 1 -- 172.21.15.39:0/2606537483 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584f354de20 con 0x5584f31ef9f0 2024-07-04T08:28:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.890+0000 7f4561400640 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5584f31ef9f0 0x5584f35f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.891+0000 7f4561400640 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5584f31ef9f0 0x5584f35f3840 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec8050d4068c639 server_cookie=3d7fc64a296a825 in_seq=0 out_seq=0 2024-07-04T08:28:41.881 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.891+0000 7f454fe00640 10 client.28439 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.892+0000 7f454fe00640 1 -- 172.21.15.39:0/2606537483 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5584f354de20 con 0x5584f31ef9f0 2024-07-04T08:28:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.892+0000 7f454fe00640 10 client.28439 handle_command_reply: tid=0 2024-07-04T08:28:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.892+0000 7f4577149740 1 client.28439 shutdown 2024-07-04T08:28:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.892+0000 7f4577149740 1 -- 172.21.15.39:0/2606537483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 msgr2=0x5584f35e15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.892+0000 7f4577149740 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e15c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f455c00fb10 tx=0x7f455c004730 comp rx=0 tx=0).stop 2024-07-04T08:28:41.882 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.892+0000 7f4577149740 1 -- 172.21.15.39:0/2606537483 shutdown_connections 2024-07-04T08:28:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.892+0000 7f4577149740 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5584f31ef9f0 0x5584f35f3840 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.892+0000 7f4577149740 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.892+0000 7f4577149740 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35e1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.893+0000 7f4577149740 1 --2- 172.21.15.39:0/2606537483 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e15c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.893+0000 7f4577149740 1 -- 172.21.15.39:0/2606537483 >> 172.21.15.39:0/2606537483 conn(0x5584f3225460 msgr2=0x5584f35d8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.893+0000 7f454fe00640 0 client.28439 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.893+0000 7f4577149740 1 -- 172.21.15.39:0/2606537483 shutdown_connections 2024-07-04T08:28:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.893+0000 7f4577149740 1 -- 172.21.15.39:0/2606537483 wait complete. 2024-07-04T08:28:41.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.893+0000 7f4577149740 20 client.28439 trim_cache size 0 max 16384 2024-07-04T08:28:41.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.913+0000 7f4577149740 1 Processor -- start 2024-07-04T08:28:41.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.913+0000 7f4577149740 1 -- start start 2024-07-04T08:28:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.913+0000 7f4577149740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35db160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.913+0000 7f4577149740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3210390 0x5584f35ed230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.913+0000 7f4577149740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3231070 0x5584f35f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.913+0000 7f4577149740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1b2470 con 0x5584f3210390 2024-07-04T08:28:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.913+0000 7f4577149740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1b3fb0 con 0x5584f32081b0 2024-07-04T08:28:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.913+0000 7f4577149740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1b2b40 con 0x5584f3231070 2024-07-04T08:28:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f4561400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3210390 0x5584f35ed230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f4569e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35db160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f4569e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35db160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38744/0 (socket says 172.21.15.39:38744) 2024-07-04T08:28:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f4569e00640 1 -- 172.21.15.39:0/2982489674 learned_addr learned my addr 172.21.15.39:0/2982489674 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:41.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f456a800640 1 --2- 172.21.15.39:0/2982489674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3231070 0x5584f35f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f4569e00640 1 -- 172.21.15.39:0/2982489674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3231070 msgr2=0x5584f35f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f4569e00640 1 --2- 172.21.15.39:0/2982489674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3231070 0x5584f35f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f4569e00640 1 -- 172.21.15.39:0/2982489674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3210390 msgr2=0x5584f35ed230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f4569e00640 1 --2- 172.21.15.39:0/2982489674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3210390 0x5584f35ed230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.914+0000 7f4569e00640 1 -- 172.21.15.39:0/2982489674 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f35e0320 con 0x5584f32081b0 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.915+0000 7f4569e00640 1 --2- 172.21.15.39:0/2982489674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35db160 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f455c012f10 tx=0x7f455c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc2ec55f9490519 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.915+0000 7f455aa00640 1 -- 172.21.15.39:0/2982489674 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c01c4e0 con 0x5584f32081b0 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.915+0000 7f455aa00640 1 -- 172.21.15.39:0/2982489674 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f455c0044e0 con 0x5584f32081b0 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.915+0000 7f455aa00640 1 -- 172.21.15.39:0/2982489674 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c01d900 con 0x5584f32081b0 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.916+0000 7f4577149740 1 -- 172.21.15.39:0/2982489674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 msgr2=0x5584f35db160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.916+0000 7f4577149740 1 --2- 172.21.15.39:0/2982489674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35db160 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f455c012f10 tx=0x7f455c0046c0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.916+0000 7f4577149740 1 -- 172.21.15.39:0/2982489674 shutdown_connections 2024-07-04T08:28:41.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.916+0000 7f4577149740 1 --2- 172.21.15.39:0/2982489674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3210390 0x5584f35ed230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.916+0000 7f4577149740 1 --2- 172.21.15.39:0/2982489674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3231070 0x5584f35f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.916+0000 7f4577149740 1 --2- 172.21.15.39:0/2982489674 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35db160 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.916+0000 7f4577149740 1 -- 172.21.15.39:0/2982489674 >> 172.21.15.39:0/2982489674 conn(0x5584f3225460 msgr2=0x5584f35daa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.916+0000 7f4577149740 1 -- 172.21.15.39:0/2982489674 shutdown_connections 2024-07-04T08:28:41.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.916+0000 7f4577149740 1 -- 172.21.15.39:0/2982489674 wait complete. 2024-07-04T08:28:41.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.917+0000 7f4577149740 1 -- start start 2024-07-04T08:28:41.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.917+0000 7f4577149740 1 Processor -- start 2024-07-04T08:28:41.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.917+0000 7f4577149740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.917+0000 7f4577149740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35e5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.917+0000 7f4577149740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.917+0000 7f4577149740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1b38e0 con 0x5584f3231070 2024-07-04T08:28:41.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.917+0000 7f4577149740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1b3210 con 0x5584f32081b0 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.917+0000 7f4577149740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1b0450 con 0x5584f3210390 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.917+0000 7f4577149740 10 client.0 _ll_register_callbacks cb 0x7f455c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4569e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f456a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4569e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38756/0 (socket says 172.21.15.39:38756) 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f456a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35e5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:42678/0 (socket says 172.21.15.39:42678) 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4569e00640 1 -- 172.21.15.39:0/787829231 learned_addr learned my addr 172.21.15.39:0/787829231 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4561400640 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4569e00640 1 -- 172.21.15.39:0/787829231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 msgr2=0x5584f35e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4569e00640 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4569e00640 1 -- 172.21.15.39:0/787829231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 msgr2=0x5584f35e5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4569e00640 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35e5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4569e00640 1 -- 172.21.15.39:0/787829231 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f365b190 con 0x5584f32081b0 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f456a800640 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35e5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.918+0000 7f4561400640 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.919+0000 7f4569e00640 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e5440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f455c028de0 tx=0x7f455c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1599d104f46c5241 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.919+0000 7f454fe00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.919+0000 7f454fe00640 1 -- 172.21.15.39:0/787829231 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c03c020 con 0x5584f32081b0 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.919+0000 7f4577149740 1 -- 172.21.15.39:0/787829231 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f455c0044e0 con 0x5584f32081b0 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.919+0000 7f454fe00640 1 -- 172.21.15.39:0/787829231 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7f455c028680 con 0x5584f32081b0 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f454fe00640 1 -- 172.21.15.39:0/787829231 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f455c047050 con 0x5584f32081b0 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f4577149740 10 client.28445 fetch_fsmap learned FSMap version 30 2024-07-04T08:28:41.910 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f4577149740 1 -- 172.21.15.39:0/787829231 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5584f365c420 con 0x5584f32081b0 2024-07-04T08:28:41.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f454fe00640 1 -- 172.21.15.39:0/787829231 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f455c0044e0 con 0x5584f32081b0 2024-07-04T08:28:41.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f454fe00640 1 -- 172.21.15.39:0/787829231 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f455c037030 con 0x5584f32081b0 2024-07-04T08:28:41.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f4577149740 10 client.28445 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:28:41.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f4577149740 10 client.28445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:28:41.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f4577149740 20 client.28445 populate_metadata read hostname 'smithi039' 2024-07-04T08:28:41.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f4577149740 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5584f31c4da0 0x5584f365eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:41.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f4577149740 4 client.28445 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.920+0000 7f4577149740 1 -- 172.21.15.39:0/787829231 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584f35498c0 con 0x5584f31c4da0 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.921+0000 7f4561400640 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5584f31c4da0 0x5584f365eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.921+0000 7f4561400640 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5584f31c4da0 0x5584f365eb10 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6021abb105d0f6 server_cookie=aec34edef5d3a9ea in_seq=0 out_seq=0 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.921+0000 7f454fe00640 10 client.28445 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.921+0000 7f454fe00640 1 -- 172.21.15.39:0/787829231 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5584f35498c0 con 0x5584f31c4da0 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.921+0000 7f454fe00640 10 client.28445 handle_command_reply: tid=0 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.921+0000 7f4577149740 1 client.28445 shutdown 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 -- 172.21.15.39:0/787829231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 msgr2=0x5584f35e5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e5440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f455c028de0 tx=0x7f455c005740 comp rx=0 tx=0).stop 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 -- 172.21.15.39:0/787829231 shutdown_connections 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5584f31c4da0 0x5584f365eb10 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5584f3231070 0x5584f35e5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584f3210390 0x5584f35e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 --2- 172.21.15.39:0/787829231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5584f32081b0 0x5584f35e5440 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f454fe00640 0 client.28445 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 -- 172.21.15.39:0/787829231 >> 172.21.15.39:0/787829231 conn(0x5584f3225460 msgr2=0x5584f35d6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 -- 172.21.15.39:0/787829231 shutdown_connections 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 1 -- 172.21.15.39:0/787829231 wait complete. 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.922+0000 7f4577149740 20 client.28445 trim_cache size 0 max 16384 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": { 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "34eb089e-a1f5-4f50-8dbc-155432b282eb": { 2024-07-04T08:28:41.914 INFO:teuthology.orchestra.run.smithi039.stdout: "path": "/", 2024-07-04T08:28:41.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tag": "34eb089e-a1f5-4f50-8dbc-155432b282eb", 2024-07-04T08:28:41.915 INFO:teuthology.orchestra.run.smithi039.stdout: "options": "recursive,force" 2024-07-04T08:28:41.915 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:28:41.915 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:28:41.915 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:28:41.915 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 -- 172.21.15.39:0/3978075159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f45500b2960 msgr2=0x7f45500b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f45500b2960 0x7f45500b4e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5584f35d45c0 tx=0x7f455c005f90 comp rx=0 tx=0).stop 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 -- 172.21.15.39:0/3978075159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f456c10f460 msgr2=0x7f456c1a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f456c10f460 0x7f456c1a12a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f456402ade0 tx=0x7f4564031660 comp rx=0 tx=0).stop 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 -- 172.21.15.39:0/3978075159 shutdown_connections 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f45500b2960 0x7f45500b4e70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f456c10f460 0x7f456c1a12a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f456c075650 0x7f456c19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 --2- 172.21.15.39:0/3978075159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456c074d00 0x7f456c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 -- 172.21.15.39:0/3978075159 >> 172.21.15.39:0/3978075159 conn(0x7f456c0fe990 msgr2=0x7f456c109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 -- 172.21.15.39:0/3978075159 shutdown_connections 2024-07-04T08:28:41.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:41.923+0000 7f456b200640 1 -- 172.21.15.39:0/3978075159 wait complete. 2024-07-04T08:28:42.834 INFO:tasks.cephfs.filesystem:scrub status for tag:34eb089e-a1f5-4f50-8dbc-155432b282eb - {'path': '/', 'tag': '34eb089e-a1f5-4f50-8dbc-155432b282eb', 'options': 'recursive,force'} 2024-07-04T08:28:43.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:43 smithi133 ceph-mon[37050]: pgmap v899: 129 pgs: 129 active+clean; 24 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-07-04T08:28:43.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:43 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:28:43.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:43 smithi039 ceph-mon[32062]: pgmap v899: 129 pgs: 129 active+clean; 24 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-07-04T08:28:43.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:43 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:28:43.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:43 smithi050 ceph-mon[36577]: pgmap v899: 129 pgs: 129 active+clean; 24 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 62 KiB/s wr, 16 op/s 2024-07-04T08:28:43.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:43 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:28:43.806 INFO:tasks.workunit.client.0.smithi039.stdout:437/600: open fd = 3 2024-07-04T08:28:43.806 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:438/600: open fd = 3 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:439/600: open fd = 3 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.807 INFO:tasks.workunit.client.0.smithi039.stdout:440/600: open fd = 3 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:441/600: open fd = 3 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:442/600: open fd = 3 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.808 INFO:tasks.workunit.client.0.smithi039.stdout:443/600: open fd = 3 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:444/600: open fd = 3 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:445/600: open fd = 3 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.809 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:446/600: open fd = 3 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:447/600: open fd = 3 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:448/600: open fd = 3 2024-07-04T08:28:43.810 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:449/600: open fd = 3 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:450/600: open fd = 3 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:451/600: open fd = 3 2024-07-04T08:28:43.811 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:452/600: open fd = 3 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:453/600: open fd = 3 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.812 INFO:tasks.workunit.client.0.smithi039.stdout:454/600: open fd = 3 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:455/600: open fd = 3 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:456/600: open fd = 3 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.813 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:457/600: open fd = 3 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:458/600: open fd = 3 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:459/600: open fd = 3 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.814 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:460/600: open fd = 3 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:461/600: open fd = 3 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:462/600: open fd = 3 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.815 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:463/600: open fd = 3 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:464/600: open fd = 3 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:465/600: open fd = 3 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.816 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:466/600: open fd = 3 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:467/600: open fd = 3 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:468/600: open fd = 3 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.817 INFO:tasks.workunit.client.0.smithi039.stdout:469/600: open fd = 3 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:470/600: open fd = 3 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:471/600: open fd = 3 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.818 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:472/600: open fd = 3 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:473/600: open fd = 3 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:474/600: open fd = 3 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.819 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:475/600: open fd = 3 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:476/600: open fd = 3 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:477/600: open fd = 3 2024-07-04T08:28:43.820 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:478/600: open fd = 3 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:479/600: open fd = 3 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:480/600: open fd = 3 2024-07-04T08:28:43.821 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:481/600: open fd = 3 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:482/600: open fd = 3 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.822 INFO:tasks.workunit.client.0.smithi039.stdout:483/600: open fd = 3 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:484/600: open fd = 3 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:485/600: open fd = 3 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.823 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:486/600: open fd = 3 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:487/600: open fd = 3 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:488/600: open fd = 3 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.824 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:489/600: open fd = 3 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:490/600: open fd = 3 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:491/600: open fd = 3 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.825 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:492/600: open fd = 3 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:493/600: open fd = 3 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.826 INFO:tasks.workunit.client.0.smithi039.stdout:494/600: open fd = 3 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:495/600: open fd = 3 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:496/600: open fd = 3 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.827 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.828 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.828 INFO:tasks.workunit.client.0.smithi039.stdout:497/600: open fd = 3 2024-07-04T08:28:43.828 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.828 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.828 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:43.828 INFO:tasks.workunit.client.0.smithi039.stdout:498/600: open fd = 3 2024-07-04T08:28:43.828 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.828 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:28:43.828 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:28:45.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:45 smithi133 ceph-mon[37050]: pgmap v900: 129 pgs: 129 active+clean; 24 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 13 op/s 2024-07-04T08:28:45.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:45 smithi039 ceph-mon[32062]: pgmap v900: 129 pgs: 129 active+clean; 24 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 13 op/s 2024-07-04T08:28:45.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:45 smithi050 ceph-mon[36577]: pgmap v900: 129 pgs: 129 active+clean; 24 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 13 op/s 2024-07-04T08:28:47.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:47 smithi133 ceph-mon[37050]: pgmap v901: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 94 KiB/s wr, 18 op/s 2024-07-04T08:28:47.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:47 smithi039 ceph-mon[32062]: pgmap v901: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 94 KiB/s wr, 18 op/s 2024-07-04T08:28:47.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:47 smithi050 ceph-mon[36577]: pgmap v901: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 94 KiB/s wr, 18 op/s 2024-07-04T08:28:49.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:49 smithi133 ceph-mon[37050]: pgmap v902: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-07-04T08:28:49.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:49 smithi039 ceph-mon[32062]: pgmap v902: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-07-04T08:28:49.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:49 smithi050 ceph-mon[36577]: pgmap v902: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-07-04T08:28:51.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:51 smithi133 ceph-mon[37050]: pgmap v903: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 89 KiB/s wr, 17 op/s 2024-07-04T08:28:51.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:51 smithi039 ceph-mon[32062]: pgmap v903: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 89 KiB/s wr, 17 op/s 2024-07-04T08:28:51.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:51 smithi050 ceph-mon[36577]: pgmap v903: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 89 KiB/s wr, 17 op/s 2024-07-04T08:28:53.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:53 smithi133 ceph-mon[37050]: pgmap v904: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:28:53.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:53 smithi039 ceph-mon[32062]: pgmap v904: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:28:53.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:53 smithi050 ceph-mon[36577]: pgmap v904: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:28:55.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:55 smithi133 ceph-mon[37050]: pgmap v905: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 13 op/s 2024-07-04T08:28:55.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:55 smithi039 ceph-mon[32062]: pgmap v905: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 13 op/s 2024-07-04T08:28:55.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:55 smithi050 ceph-mon[36577]: pgmap v905: 129 pgs: 129 active+clean; 24 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 KiB/s wr, 13 op/s 2024-07-04T08:28:56.014 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.2 2024-07-04T08:28:56.014 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:28:56.511 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:28:57.536 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:57 smithi039 ceph-mon[32062]: pgmap v906: 129 pgs: 129 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 17 op/s 2024-07-04T08:28:57.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:57 smithi133 ceph-mon[37050]: pgmap v906: 129 pgs: 129 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 17 op/s 2024-07-04T08:28:57.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:57 smithi050 ceph-mon[36577]: pgmap v906: 129 pgs: 129 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 93 KiB/s wr, 17 op/s 2024-07-04T08:28:58.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.812+0000 7ff615000640 1 -- 172.21.15.39:0/2283146812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 msgr2=0x7ff610103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.812+0000 7ff615000640 1 --2- 172.21.15.39:0/2283146812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 0x7ff610103310 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff600009900 tx=0x7ff60002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:28:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.812+0000 7ff615000640 1 -- 172.21.15.39:0/2283146812 shutdown_connections 2024-07-04T08:28:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.812+0000 7ff615000640 1 --2- 172.21.15.39:0/2283146812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff610109190 0x7ff61010f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.812+0000 7ff615000640 1 --2- 172.21.15.39:0/2283146812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff610108840 0x7ff610108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.812+0000 7ff615000640 1 --2- 172.21.15.39:0/2283146812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 0x7ff610103310 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:58.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.812+0000 7ff615000640 1 -- 172.21.15.39:0/2283146812 >> 172.21.15.39:0/2283146812 conn(0x7ff6100fe950 msgr2=0x7ff610100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.812+0000 7ff615000640 1 -- 172.21.15.39:0/2283146812 shutdown_connections 2024-07-04T08:28:58.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.812+0000 7ff615000640 1 -- 172.21.15.39:0/2283146812 wait complete. 2024-07-04T08:28:58.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.813+0000 7ff615000640 1 Processor -- start 2024-07-04T08:28:58.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.814+0000 7ff615000640 1 -- start start 2024-07-04T08:28:58.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.814+0000 7ff615000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 0x7ff61019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:58.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.814+0000 7ff615000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff610108840 0x7ff61019c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:58.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.814+0000 7ff615000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff610109190 0x7ff6101a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.814+0000 7ff615000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff610112e50 con 0x7ff610102f00 2024-07-04T08:28:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.814+0000 7ff615000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff610112cd0 con 0x7ff610108840 2024-07-04T08:28:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.814+0000 7ff615000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff610112fd0 con 0x7ff610109190 2024-07-04T08:28:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.814+0000 7ff60f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 0x7ff61019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 0x7ff61019bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:38988/0 (socket says 172.21.15.39:38988) 2024-07-04T08:28:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60f400640 1 -- 172.21.15.39:0/2074879430 learned_addr learned my addr 172.21.15.39:0/2074879430 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:28:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60ea00640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff610108840 0x7ff61019c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:58.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60fe00640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff610109190 0x7ff6101a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60f400640 1 -- 172.21.15.39:0/2074879430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff610109190 msgr2=0x7ff6101a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60f400640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff610109190 0x7ff6101a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60f400640 1 -- 172.21.15.39:0/2074879430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff610108840 msgr2=0x7ff61019c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60f400640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff610108840 0x7ff61019c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60f400640 1 -- 172.21.15.39:0/2074879430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6101a1a50 con 0x7ff610102f00 2024-07-04T08:28:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60ea00640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff610108840 0x7ff61019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:28:58.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.815+0000 7ff60f400640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 0x7ff61019bae0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff600005940 tx=0x7ff6000040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.816+0000 7ff607e00640 1 -- 172.21.15.39:0/2074879430 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60003f030 con 0x7ff610102f00 2024-07-04T08:28:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.816+0000 7ff607e00640 1 -- 172.21.15.39:0/2074879430 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6000044c0 con 0x7ff610102f00 2024-07-04T08:28:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.816+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6101a1ce0 con 0x7ff610102f00 2024-07-04T08:28:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.816+0000 7ff607e00640 1 -- 172.21.15.39:0/2074879430 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600035070 con 0x7ff610102f00 2024-07-04T08:28:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.816+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6101a84a0 con 0x7ff610102f00 2024-07-04T08:28:58.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.817+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff610103310 con 0x7ff610102f00 2024-07-04T08:28:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.820+0000 7ff607e00640 1 -- 172.21.15.39:0/2074879430 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff600042030 con 0x7ff610102f00 2024-07-04T08:28:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.823+0000 7ff607e00640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5f40b2960 0x7ff5f40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:28:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.823+0000 7ff607e00640 1 -- 172.21.15.39:0/2074879430 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7ff600102f60 con 0x7ff610102f00 2024-07-04T08:28:58.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.823+0000 7ff60ea00640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5f40b2960 0x7ff5f40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:28:58.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.823+0000 7ff60ea00640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5f40b2960 0x7ff5f40b4e70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80095f0 tx=0x7ff5f8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:28:58.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:58.824+0000 7ff607e00640 1 -- 172.21.15.39:0/2074879430 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7ff6000c9f30 con 0x7ff610102f00 2024-07-04T08:28:59.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.081+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6101a86a0 con 0x7ff610102f00 2024-07-04T08:28:59.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.084+0000 7ff607e00640 1 -- 172.21.15.39:0/2074879430 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7ff6000ce230 con 0x7ff610102f00 2024-07-04T08:28:59.073 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:28:59.074 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:28:59.074 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:28:59.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5f40b2960 msgr2=0x7ff5f40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:59.076 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5f40b2960 0x7ff5f40b4e70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80095f0 tx=0x7ff5f8005f90 comp rx=0 tx=0).stop 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 msgr2=0x7ff61019bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 0x7ff61019bae0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff600005940 tx=0x7ff6000040c0 comp rx=0 tx=0).stop 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 shutdown_connections 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff5f40b2960 0x7ff5f40b4e70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff610109190 0x7ff6101a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff610108840 0x7ff61019c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 --2- 172.21.15.39:0/2074879430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff610102f00 0x7ff61019bae0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 >> 172.21.15.39:0/2074879430 conn(0x7ff6100fe950 msgr2=0x7ff610107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 shutdown_connections 2024-07-04T08:28:59.077 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:28:59.087+0000 7ff615000640 1 -- 172.21.15.39:0/2074879430 wait complete. 2024-07-04T08:28:59.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:59 smithi039 ceph-mon[32062]: pgmap v907: 129 pgs: 129 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-07-04T08:28:59.339 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:28:59 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2074879430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:28:59.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:59 smithi133 ceph-mon[37050]: pgmap v907: 129 pgs: 129 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-07-04T08:28:59.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:28:59 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2074879430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:28:59.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:59 smithi050 ceph-mon[36577]: pgmap v907: 129 pgs: 129 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 60 KiB/s wr, 15 op/s 2024-07-04T08:28:59.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:28:59 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2074879430' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:28:59.941 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 140.2 --await-for 45.0' 2024-07-04T08:28:59.941 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 140.2 --await-for 45.0 2024-07-04T08:29:00.444 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:01.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:01 smithi133 ceph-mon[37050]: pgmap v908: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 17 op/s 2024-07-04T08:29:01.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:01 smithi039 ceph-mon[32062]: pgmap v908: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 17 op/s 2024-07-04T08:29:01.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:01 smithi050 ceph-mon[36577]: pgmap v908: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 89 KiB/s wr, 17 op/s 2024-07-04T08:29:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.706+0000 7fb55f200640 1 -- 172.21.15.39:0/2654690448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb56010b3d0 msgr2=0x7fb560111c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.706+0000 7fb55f200640 1 --2- 172.21.15.39:0/2654690448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb56010b3d0 0x7fb560111c00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb558009900 tx=0x7fb55802a400 comp rx=0 tx=0).stop 2024-07-04T08:29:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.706+0000 7fb55f200640 1 -- 172.21.15.39:0/2654690448 shutdown_connections 2024-07-04T08:29:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.706+0000 7fb55f200640 1 --2- 172.21.15.39:0/2654690448 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb56010b3d0 0x7fb560111c00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.706+0000 7fb55f200640 1 --2- 172.21.15.39:0/2654690448 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb56010aa80 0x7fb56010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.706+0000 7fb55f200640 1 --2- 172.21.15.39:0/2654690448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb560105140 0x7fb560105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.706+0000 7fb55f200640 1 -- 172.21.15.39:0/2654690448 >> 172.21.15.39:0/2654690448 conn(0x7fb560100bd0 msgr2=0x7fb560103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.706+0000 7fb55f200640 1 -- 172.21.15.39:0/2654690448 shutdown_connections 2024-07-04T08:29:02.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.706+0000 7fb55f200640 1 -- 172.21.15.39:0/2654690448 wait complete. 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.707+0000 7fb55f200640 1 Processor -- start 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.708+0000 7fb55f200640 1 -- start start 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.708+0000 7fb55f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb560105140 0x7fb56019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.708+0000 7fb55f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb56010aa80 0x7fb56019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.708+0000 7fb55f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb56010b3d0 0x7fb5601a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.708+0000 7fb55f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5601150d0 con 0x7fb56010b3d0 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.708+0000 7fb55f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb560114f50 con 0x7fb56010aa80 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.708+0000 7fb55f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb560115250 con 0x7fb560105140 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb560105140 0x7fb56019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb560105140 0x7fb56019b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53294/0 (socket says 172.21.15.39:53294) 2024-07-04T08:29:02.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55de00640 1 -- 172.21.15.39:0/214532316 learned_addr learned my addr 172.21.15.39:0/214532316 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:02.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb56010b3d0 0x7fb5601a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55d400640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb56010aa80 0x7fb56019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55de00640 1 -- 172.21.15.39:0/214532316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb56010aa80 msgr2=0x7fb56019bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55de00640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb56010aa80 0x7fb56019bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55de00640 1 -- 172.21.15.39:0/214532316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb56010b3d0 msgr2=0x7fb5601a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55de00640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb56010b3d0 0x7fb5601a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.709+0000 7fb55de00640 1 -- 172.21.15.39:0/214532316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5601a1950 con 0x7fb560105140 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.710+0000 7fb55de00640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb560105140 0x7fb56019b9e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb55000e9f0 tx=0x7fb55000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.710+0000 7fb556a00640 1 -- 172.21.15.39:0/214532316 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55000cd80 con 0x7fb560105140 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.710+0000 7fb556a00640 1 -- 172.21.15.39:0/214532316 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5500108e0 con 0x7fb560105140 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.710+0000 7fb55f200640 1 -- 172.21.15.39:0/214532316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5601a1c40 con 0x7fb560105140 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.710+0000 7fb556a00640 1 -- 172.21.15.39:0/214532316 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550018980 con 0x7fb560105140 2024-07-04T08:29:02.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.710+0000 7fb55f200640 1 -- 172.21.15.39:0/214532316 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5601a83a0 con 0x7fb560105140 2024-07-04T08:29:02.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.714+0000 7fb56b5d4740 1 Processor -- start 2024-07-04T08:29:02.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.714+0000 7fb56b5d4740 1 -- start start 2024-07-04T08:29:02.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.714+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccbb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.714+0000 7fb556a00640 1 -- 172.21.15.39:0/214532316 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb550010430 con 0x7fb560105140 2024-07-04T08:29:02.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.714+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8ed980 0x55a1bccbbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.714+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 0x55a1bccbbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a1bcc30cc0 con 0x55a1bc8ed980 2024-07-04T08:29:02.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a1bcc20260 con 0x55a1bc916840 2024-07-04T08:29:02.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a1bcc242a0 con 0x55a1bc8f5b60 2024-07-04T08:29:02.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccbb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccbb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33586/0 (socket says 172.21.15.39:33586) 2024-07-04T08:29:02.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55e800640 1 -- 172.21.15.39:0/3070085152 learned_addr learned my addr 172.21.15.39:0/3070085152 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:02.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55de00640 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8ed980 0x55a1bccbbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55d400640 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 0x55a1bccbbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55e800640 1 -- 172.21.15.39:0/3070085152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 msgr2=0x55a1bccbbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:02.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55e800640 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 0x55a1bccbbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55e800640 1 -- 172.21.15.39:0/3070085152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8ed980 msgr2=0x55a1bccbbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:02.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55e800640 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8ed980 0x55a1bccbbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55e800640 1 -- 172.21.15.39:0/3070085152 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1bcbb76d0 con 0x55a1bc916840 2024-07-04T08:29:02.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.715+0000 7fb55de00640 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8ed980 0x55a1bccbbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:29:02.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.716+0000 7fb55e800640 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccbb560 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb558009900 tx=0x7fb558005f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79cce45f0e4cf7a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:02.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.716+0000 7fb54f400640 1 -- 172.21.15.39:0/3070085152 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558040070 con 0x55a1bc916840 2024-07-04T08:29:02.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.716+0000 7fb54f400640 1 -- 172.21.15.39:0/3070085152 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5580044c0 con 0x55a1bc916840 2024-07-04T08:29:02.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.716+0000 7fb54f400640 1 -- 172.21.15.39:0/3070085152 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558035040 con 0x55a1bc916840 2024-07-04T08:29:02.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.717+0000 7fb556a00640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5400b2950 0x7fb5400b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.717+0000 7fb556a00640 1 -- 172.21.15.39:0/214532316 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7fb5500e2d00 con 0x7fb560105140 2024-07-04T08:29:02.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.717+0000 7fb55de00640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5400b2950 0x7fb5400b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.717+0000 7fb56b5d4740 1 -- 172.21.15.39:0/3070085152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 msgr2=0x55a1bccbb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:02.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.717+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccbb560 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb558009900 tx=0x7fb558005f30 comp rx=0 tx=0).stop 2024-07-04T08:29:02.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.717+0000 7fb56b5d4740 1 -- 172.21.15.39:0/3070085152 shutdown_connections 2024-07-04T08:29:02.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.717+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8ed980 0x55a1bccbbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.717+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 0x55a1bccbbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.717+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/3070085152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccbb560 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.718+0000 7fb56b5d4740 1 -- 172.21.15.39:0/3070085152 >> 172.21.15.39:0/3070085152 conn(0x55a1bc90a990 msgr2=0x55a1bccb9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:02.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.718+0000 7fb56b5d4740 1 -- 172.21.15.39:0/3070085152 shutdown_connections 2024-07-04T08:29:02.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.718+0000 7fb56b5d4740 1 -- 172.21.15.39:0/3070085152 wait complete. 2024-07-04T08:29:02.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.718+0000 7fb55de00640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5400b2950 0x7fb5400b4e60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55a1bccba1e0 tx=0x7fb550000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:02.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.719+0000 7fb56b5d4740 1 -- start start 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.719+0000 7fb56b5d4740 1 Processor -- start 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.719+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccc7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.719+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8f5b60 0x55a1bccc7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.719+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccd2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.719+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a1bca847a0 con 0x55a1bc8f5b60 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.719+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a1bcbc76b0 con 0x55a1bc916840 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.719+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a1bccc9ce0 con 0x55a1bc8ed980 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb56b5d4740 10 client.0 _ll_register_callbacks cb 0x55a1bc92e090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccc7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccc7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53310/0 (socket says 172.21.15.39:53310) 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55e800640 1 -- 172.21.15.39:0/842500768 learned_addr learned my addr 172.21.15.39:0/842500768 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55e800640 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccd2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55d400640 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8f5b60 0x55a1bccc7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55e800640 1 -- 172.21.15.39:0/842500768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 msgr2=0x55a1bccd2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:02.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55e800640 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccd2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55e800640 1 -- 172.21.15.39:0/842500768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8f5b60 msgr2=0x55a1bccc7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:02.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55e800640 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8f5b60 0x55a1bccc7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.720+0000 7fb55e800640 1 -- 172.21.15.39:0/842500768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1bcbb81b0 con 0x55a1bc8ed980 2024-07-04T08:29:02.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.721+0000 7fb55e800640 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccc7150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb5580047a0 tx=0x7fb558004c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52b35a9daf382574 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:02.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.721+0000 7fb547e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:29:02.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.721+0000 7fb547e00640 1 -- 172.21.15.39:0/842500768 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558040070 con 0x55a1bc8ed980 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.721+0000 7fb56b5d4740 1 -- 172.21.15.39:0/842500768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1bcc32c10 con 0x55a1bc8ed980 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.721+0000 7fb547e00640 1 -- 172.21.15.39:0/842500768 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7fb558004fa0 con 0x55a1bc8ed980 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.721+0000 7fb547e00640 1 -- 172.21.15.39:0/842500768 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb558058050 con 0x55a1bc8ed980 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.721+0000 7fb56b5d4740 10 client.28399 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.722+0000 7fb56b5d4740 1 -- 172.21.15.39:0/842500768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1bc990b90 con 0x55a1bc8ed980 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.722+0000 7fb547e00640 1 -- 172.21.15.39:0/842500768 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a1bcc32c10 con 0x55a1bc8ed980 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.722+0000 7fb547e00640 1 -- 172.21.15.39:0/842500768 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb558003b90 con 0x55a1bc8ed980 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.722+0000 7fb56b5d4740 10 client.28399 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.722+0000 7fb56b5d4740 10 client.28399 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.722+0000 7fb56b5d4740 20 client.28399 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.722+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1bc8d51c0 0x55a1bccd8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.722+0000 7fb56b5d4740 4 client.28399 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:02.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.722+0000 7fb56b5d4740 1 -- 172.21.15.39:0/842500768 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1bcc2e6b0 con 0x55a1bc8d51c0 2024-07-04T08:29:02.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.723+0000 7fb55d400640 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1bc8d51c0 0x55a1bccd8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.723+0000 7fb55d400640 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1bc8d51c0 0x55a1bccd8e50 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=132565522ba894fe server_cookie=5f20f77d5e2e2af4 in_seq=0 out_seq=0 2024-07-04T08:29:02.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.723+0000 7fb547e00640 10 client.28399 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.724+0000 7fb547e00640 1 -- 172.21.15.39:0/842500768 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a1bcc2e6b0 con 0x55a1bc8d51c0 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.724+0000 7fb547e00640 10 client.28399 handle_command_reply: tid=0 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.724+0000 7fb56b5d4740 1 client.28399 shutdown 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 -- 172.21.15.39:0/842500768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 msgr2=0x55a1bccc7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccc7150 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb5580047a0 tx=0x7fb558004c20 comp rx=0 tx=0).stop 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 -- 172.21.15.39:0/842500768 shutdown_connections 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1bc8d51c0 0x55a1bccd8e50 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8f5b60 0x55a1bccc7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc916840 0x55a1bccd2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb547e00640 0 client.28399 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:02.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/842500768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccc7150 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 -- 172.21.15.39:0/842500768 >> 172.21.15.39:0/842500768 conn(0x55a1bc90a990 msgr2=0x55a1bccbe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:02.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 -- 172.21.15.39:0/842500768 shutdown_connections 2024-07-04T08:29:02.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 1 -- 172.21.15.39:0/842500768 wait complete. 2024-07-04T08:29:02.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.725+0000 7fb56b5d4740 20 client.28399 trim_cache size 0 max 16384 2024-07-04T08:29:02.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.745+0000 7fb56b5d4740 1 Processor -- start 2024-07-04T08:29:02.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.745+0000 7fb56b5d4740 1 -- start start 2024-07-04T08:29:02.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccd2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8ed980 0x55a1bccd3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 0x55a1bccdbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5601b23b0 con 0x55a1bc8d51c0 2024-07-04T08:29:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5601b3ef0 con 0x55a1bc8ed980 2024-07-04T08:29:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb5601b2a80 con 0x55a1bc8f5b60 2024-07-04T08:29:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb55e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccd2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb55e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccd2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51018/0 (socket says 172.21.15.39:51018) 2024-07-04T08:29:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb55e800640 1 -- 172.21.15.39:0/2072248030 learned_addr learned my addr 172.21.15.39:0/2072248030 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb55d400640 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8ed980 0x55a1bccd3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.746+0000 7fb55e800640 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 0x55a1bccdbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb55e800640 1 -- 172.21.15.39:0/2072248030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 msgr2=0x55a1bccdbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb55e800640 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 0x55a1bccdbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb55e800640 1 -- 172.21.15.39:0/2072248030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8ed980 msgr2=0x55a1bccd3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:02.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb55e800640 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8ed980 0x55a1bccd3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb55e800640 1 -- 172.21.15.39:0/2072248030 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1bccbe3c0 con 0x55a1bc8d51c0 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb55d400640 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8ed980 0x55a1bccd3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb55e800640 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccd2f40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb558004bf0 tx=0x7fb558005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c938c4c9b71f5da3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb54ea00640 1 -- 172.21.15.39:0/2072248030 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558049020 con 0x55a1bc8d51c0 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb54ea00640 1 -- 172.21.15.39:0/2072248030 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb558009520 con 0x55a1bc8d51c0 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.747+0000 7fb54ea00640 1 -- 172.21.15.39:0/2072248030 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558040040 con 0x55a1bc8d51c0 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.748+0000 7fb56b5d4740 1 -- 172.21.15.39:0/2072248030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 msgr2=0x55a1bccd2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.748+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccd2f40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb558004bf0 tx=0x7fb558005f30 comp rx=0 tx=0).stop 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.748+0000 7fb56b5d4740 1 -- 172.21.15.39:0/2072248030 shutdown_connections 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.748+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8f5b60 0x55a1bccdbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.748+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8ed980 0x55a1bccd3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.748+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/2072248030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccd2f40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.748+0000 7fb56b5d4740 1 -- 172.21.15.39:0/2072248030 >> 172.21.15.39:0/2072248030 conn(0x55a1bc90a990 msgr2=0x55a1bccb7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.749+0000 7fb56b5d4740 1 -- 172.21.15.39:0/2072248030 shutdown_connections 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.749+0000 7fb56b5d4740 1 -- 172.21.15.39:0/2072248030 wait complete. 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.749+0000 7fb56b5d4740 1 -- start start 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.749+0000 7fb56b5d4740 1 Processor -- start 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccda5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccdabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb56b5d4740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8f5b60 0x55a1bcd403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb5601b3820 con 0x55a1bc8d51c0 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb5601b3150 con 0x55a1bc8f5b60 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb56b5d4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb5601b0390 con 0x55a1bc8ed980 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb56b5d4740 10 client.0 _ll_register_callbacks cb 0x7fb548009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:02.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb55e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccda5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb55e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccda5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51032/0 (socket says 172.21.15.39:51032) 2024-07-04T08:29:02.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb55e800640 1 -- 172.21.15.39:0/234842186 learned_addr learned my addr 172.21.15.39:0/234842186 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:02.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb55d400640 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccdabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.750+0000 7fb55e800640 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8f5b60 0x55a1bcd403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb55e800640 1 -- 172.21.15.39:0/234842186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 msgr2=0x55a1bccdabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:02.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb55e800640 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccdabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb55e800640 1 -- 172.21.15.39:0/234842186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8f5b60 msgr2=0x55a1bcd403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:02.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb55e800640 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8f5b60 0x55a1bcd403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb55e800640 1 -- 172.21.15.39:0/234842186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1bcd40a10 con 0x55a1bc8d51c0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb55e800640 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccda5e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb558005a80 tx=0x7fb558005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77050a917bdf44e3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb547e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb547e00640 1 -- 172.21.15.39:0/234842186 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558049070 con 0x55a1bc8d51c0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb547e00640 1 -- 172.21.15.39:0/234842186 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9424+0+0 (secure 0 0 0) 0x7fb558003c90 con 0x55a1bc8d51c0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.751+0000 7fb56b5d4740 1 -- 172.21.15.39:0/234842186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb558009520 con 0x55a1bc8d51c0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb547e00640 1 -- 172.21.15.39:0/234842186 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb558040040 con 0x55a1bc8d51c0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb56b5d4740 10 client.18348 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb56b5d4740 1 -- 172.21.15.39:0/234842186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1bcd41c70 con 0x55a1bc8d51c0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb547e00640 1 -- 172.21.15.39:0/234842186 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb558009520 con 0x55a1bc8d51c0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb547e00640 1 -- 172.21.15.39:0/234842186 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb558004f20 con 0x55a1bc8d51c0 2024-07-04T08:29:02.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb56b5d4740 10 client.18348 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:02.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb56b5d4740 10 client.18348 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:29:02.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb56b5d4740 20 client.18348 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:02.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1bc916840 0x55a1bcd44030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:02.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb56b5d4740 4 client.18348 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.2, "await_for": 45.0}] 2024-07-04T08:29:02.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.752+0000 7fb56b5d4740 1 -- 172.21.15.39:0/234842186 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.2, "await_for": 45.0}) -- 0x55a1bcc2bdf0 con 0x55a1bc916840 2024-07-04T08:29:02.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.753+0000 7fb55d400640 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1bc916840 0x55a1bcd44030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:02.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.753+0000 7fb55d400640 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1bc916840 0x55a1bcd44030 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94bd6ff1ef453bd server_cookie=cd413044b08e6454 in_seq=0 out_seq=0 2024-07-04T08:29:02.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.753+0000 7fb547e00640 10 client.18348 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:02.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.770+0000 7fb547e00640 1 -- 172.21.15.39:0/234842186 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a1bcc2bdf0 con 0x55a1bc916840 2024-07-04T08:29:02.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.770+0000 7fb547e00640 10 client.18348 handle_command_reply: tid=0 2024-07-04T08:29:02.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 client.18348 shutdown 2024-07-04T08:29:02.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 -- 172.21.15.39:0/234842186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 msgr2=0x55a1bccda5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:02.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccda5e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb558005a80 tx=0x7fb558005e30 comp rx=0 tx=0).stop 2024-07-04T08:29:02.759 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 -- 172.21.15.39:0/234842186 shutdown_connections 2024-07-04T08:29:02.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a1bc916840 0x55a1bcd44030 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a1bc8f5b60 0x55a1bcd403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a1bc8ed980 0x55a1bccdabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 --2- 172.21.15.39:0/234842186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a1bc8d51c0 0x55a1bccda5e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 -- 172.21.15.39:0/234842186 >> 172.21.15.39:0/234842186 conn(0x55a1bc90a990 msgr2=0x55a1bccc57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:02.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb547e00640 0 client.18348 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:02.760 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 -- 172.21.15.39:0/234842186 shutdown_connections 2024-07-04T08:29:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.771+0000 7fb56b5d4740 1 -- 172.21.15.39:0/234842186 wait complete. 2024-07-04T08:29:02.761 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.772+0000 7fb56b5d4740 20 client.18348 trim_cache size 0 max 16384 2024-07-04T08:29:02.761 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:29:02.761 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:29:02.761 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:29:02.761 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 38, 2024-07-04T08:29:02.761 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:29:02.761 INFO:teuthology.orchestra.run.smithi039.stdout: "b0a3c823": { 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 38, 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 140.2, 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:29:02.762 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:29:02.763 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:29:02.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:02.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:02.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:02.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:02.763 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:02.763 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.772+0000 7fb55f200640 1 -- 172.21.15.39:0/214532316 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5400b2950 msgr2=0x7fb5400b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.772+0000 7fb55f200640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5400b2950 0x7fb5400b4e60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55a1bccba1e0 tx=0x7fb550000bc0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.772+0000 7fb55f200640 1 -- 172.21.15.39:0/214532316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb560105140 msgr2=0x7fb56019b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.772+0000 7fb55f200640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb560105140 0x7fb56019b9e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb55000e9f0 tx=0x7fb55000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.773+0000 7fb55f200640 1 -- 172.21.15.39:0/214532316 shutdown_connections 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.773+0000 7fb55f200640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb5400b2950 0x7fb5400b4e60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.773+0000 7fb55f200640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb56010b3d0 0x7fb5601a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.773+0000 7fb55f200640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb56010aa80 0x7fb56019bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.773+0000 7fb55f200640 1 --2- 172.21.15.39:0/214532316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb560105140 0x7fb56019b9e0 secure :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb55000e9f0 tx=0x7fb55000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.773+0000 7fb55f200640 1 -- 172.21.15.39:0/214532316 >> 172.21.15.39:0/214532316 conn(0x7fb560100bd0 msgr2=0x7fb560109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.773+0000 7fb55f200640 1 -- 172.21.15.39:0/214532316 shutdown_connections 2024-07-04T08:29:02.764 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:02.773+0000 7fb55f200640 1 -- 172.21.15.39:0/214532316 wait complete. 2024-07-04T08:29:03.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:03 smithi133 ceph-mon[37050]: pgmap v909: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:29:03.584 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:03 smithi039 ceph-mon[32062]: pgmap v909: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:29:03.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:03 smithi050 ceph-mon[36577]: pgmap v909: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 15 op/s 2024-07-04T08:29:03.669 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b0a3c823', quiesce took 7.7 seconds. Will release after: 12.5 2024-07-04T08:29:04.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:04 smithi133 ceph-mon[37050]: osdmap e107: 12 total, 12 up, 12 in 2024-07-04T08:29:04.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:04 smithi039 ceph-mon[32062]: osdmap e107: 12 total, 12 up, 12 in 2024-07-04T08:29:04.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:04 smithi050 ceph-mon[36577]: osdmap e107: 12 total, 12 up, 12 in 2024-07-04T08:29:05.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:05 smithi133 ceph-mon[37050]: pgmap v911: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 15 op/s 2024-07-04T08:29:05.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:05 smithi039 ceph-mon[32062]: pgmap v911: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 15 op/s 2024-07-04T08:29:05.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:05 smithi050 ceph-mon[36577]: pgmap v911: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 15 op/s 2024-07-04T08:29:06.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:06 smithi039 ceph-mon[32062]: pgmap v912: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 121 KiB/s wr, 35 op/s 2024-07-04T08:29:06.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:06 smithi050 ceph-mon[36577]: pgmap v912: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 121 KiB/s wr, 35 op/s 2024-07-04T08:29:06.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:06 smithi133 ceph-mon[37050]: pgmap v912: 129 pgs: 129 active+clean; 25 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 121 KiB/s wr, 35 op/s 2024-07-04T08:29:09.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:09 smithi133 ceph-mon[37050]: pgmap v913: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 121 KiB/s wr, 35 op/s 2024-07-04T08:29:09.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:09 smithi133 ceph-mon[37050]: osdmap e108: 12 total, 12 up, 12 in 2024-07-04T08:29:09.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:09 smithi039 ceph-mon[32062]: pgmap v913: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 121 KiB/s wr, 35 op/s 2024-07-04T08:29:09.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:09 smithi039 ceph-mon[32062]: osdmap e108: 12 total, 12 up, 12 in 2024-07-04T08:29:09.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:09 smithi050 ceph-mon[36577]: pgmap v913: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 121 KiB/s wr, 35 op/s 2024-07-04T08:29:09.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:09 smithi050 ceph-mon[36577]: osdmap e108: 12 total, 12 up, 12 in 2024-07-04T08:29:10.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:10 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:29:10.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:10 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:29:10.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:10 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:29:11.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: pgmap v915: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 109 KiB/s wr, 38 op/s 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:29:11.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:29:11.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:29:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: pgmap v915: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 109 KiB/s wr, 38 op/s 2024-07-04T08:29:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:29:11.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:29:11.590 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:29:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: pgmap v915: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 109 KiB/s wr, 38 op/s 2024-07-04T08:29:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-04T08:29:11.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:29:11.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:29:11.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:29:12.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:12 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-07-04T08:29:12.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:12 smithi133 ceph-mon[37050]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-07-04T08:29:12.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:12 smithi133 ceph-mon[37050]: Unable to set osd_memory_target on smithi133 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-04T08:29:12.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:12 smithi133 ceph-mon[37050]: Unable to set osd_memory_target on smithi050 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-07-04T08:29:12.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:12 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-07-04T08:29:12.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:12 smithi039 ceph-mon[32062]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-07-04T08:29:12.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:12 smithi039 ceph-mon[32062]: Unable to set osd_memory_target on smithi133 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-04T08:29:12.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:12 smithi039 ceph-mon[32062]: Unable to set osd_memory_target on smithi050 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-07-04T08:29:12.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:12 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-07-04T08:29:12.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:12 smithi050 ceph-mon[36577]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-07-04T08:29:12.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:12 smithi050 ceph-mon[36577]: Unable to set osd_memory_target on smithi133 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-04T08:29:12.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:12 smithi050 ceph-mon[36577]: Unable to set osd_memory_target on smithi050 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-07-04T08:29:12.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:29:13.319 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:13.526 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:13 smithi039 ceph-mon[32062]: pgmap v916: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 98 KiB/s wr, 37 op/s 2024-07-04T08:29:13.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:13 smithi133 ceph-mon[37050]: pgmap v916: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 98 KiB/s wr, 37 op/s 2024-07-04T08:29:13.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:13 smithi050 ceph-mon[36577]: pgmap v916: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 98 KiB/s wr, 37 op/s 2024-07-04T08:29:15.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:15 smithi039 ceph-mon[32062]: pgmap v917: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-07-04T08:29:15.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:15 smithi133 ceph-mon[37050]: pgmap v917: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-07-04T08:29:15.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:15 smithi050 ceph-mon[36577]: pgmap v917: 129 pgs: 129 active+clean; 25 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-07-04T08:29:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.896+0000 7f8844e00640 1 -- 172.21.15.39:0/1348339550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840109170 msgr2=0x7f884010f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:15.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.896+0000 7f8844e00640 1 --2- 172.21.15.39:0/1348339550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840109170 0x7f884010f9a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8838009900 tx=0x7f883802a410 comp rx=0 tx=0).stop 2024-07-04T08:29:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.896+0000 7f8844e00640 1 -- 172.21.15.39:0/1348339550 shutdown_connections 2024-07-04T08:29:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.896+0000 7f8844e00640 1 --2- 172.21.15.39:0/1348339550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840109170 0x7f884010f9a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.896+0000 7f8844e00640 1 --2- 172.21.15.39:0/1348339550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8840108820 0x7f8840108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.896+0000 7f8844e00640 1 --2- 172.21.15.39:0/1348339550 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8840102ee0 0x7f88401032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.896+0000 7f8844e00640 1 -- 172.21.15.39:0/1348339550 >> 172.21.15.39:0/1348339550 conn(0x7f88400fe930 msgr2=0x7f8840100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.897+0000 7f8844e00640 1 -- 172.21.15.39:0/1348339550 shutdown_connections 2024-07-04T08:29:15.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.897+0000 7f8844e00640 1 -- 172.21.15.39:0/1348339550 wait complete. 2024-07-04T08:29:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.898+0000 7f8844e00640 1 Processor -- start 2024-07-04T08:29:15.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.898+0000 7f8844e00640 1 -- start start 2024-07-04T08:29:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f8844e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840102ee0 0x7f884019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f8844e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8840108820 0x7f884019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f8844e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8840109170 0x7f88401a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f8844e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8840112e30 con 0x7f8840102ee0 2024-07-04T08:29:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f8844e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8840112cb0 con 0x7f8840109170 2024-07-04T08:29:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f8844e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8840112fb0 con 0x7f8840108820 2024-07-04T08:29:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f883f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840102ee0 0x7f884019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f883f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840102ee0 0x7f884019b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51448/0 (socket says 172.21.15.39:51448) 2024-07-04T08:29:15.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f883f400640 1 -- 172.21.15.39:0/1229271574 learned_addr learned my addr 172.21.15.39:0/1229271574 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:15.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f883fe00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8840109170 0x7f88401a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.899+0000 7f8836a00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8840108820 0x7f884019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f883f400640 1 -- 172.21.15.39:0/1229271574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8840108820 msgr2=0x7f884019bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:15.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f883f400640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8840108820 0x7f884019bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f883f400640 1 -- 172.21.15.39:0/1229271574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8840109170 msgr2=0x7f88401a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:15.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f883f400640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8840109170 0x7f88401a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f883f400640 1 -- 172.21.15.39:0/1229271574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88401a1950 con 0x7f8840102ee0 2024-07-04T08:29:15.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f8836a00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8840108820 0x7f884019bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f883f400640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840102ee0 0x7f884019b9e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9f0 tx=0x7f883000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f883cc00640 1 -- 172.21.15.39:0/1229271574 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883000cd80 con 0x7f8840102ee0 2024-07-04T08:29:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f883cc00640 1 -- 172.21.15.39:0/1229271574 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88300108e0 con 0x7f8840102ee0 2024-07-04T08:29:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f883cc00640 1 -- 172.21.15.39:0/1229271574 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830018980 con 0x7f8840102ee0 2024-07-04T08:29:15.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.900+0000 7f8844e00640 1 -- 172.21.15.39:0/1229271574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88401a1c40 con 0x7f8840102ee0 2024-07-04T08:29:15.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.901+0000 7f8844e00640 1 -- 172.21.15.39:0/1229271574 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88401a83a0 con 0x7f8840102ee0 2024-07-04T08:29:15.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.905+0000 7f884ca3d740 1 Processor -- start 2024-07-04T08:29:15.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.905+0000 7f884ca3d740 1 -- start start 2024-07-04T08:29:15.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.905+0000 7f883cc00640 1 -- 172.21.15.39:0/1229271574 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8830010430 con 0x7f8840102ee0 2024-07-04T08:29:15.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.905+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559587416070 0x5595877ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.905+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 0x5595877bae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.905+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873f5390 0x5595877bb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.905+0000 7f884ca3d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559587730ea0 con 0x559587416070 2024-07-04T08:29:15.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.905+0000 7f884ca3d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559587720440 con 0x5595873f5390 2024-07-04T08:29:15.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.905+0000 7f884ca3d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559587724480 con 0x5595873ed1b0 2024-07-04T08:29:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f8836a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873f5390 0x5595877bb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f883fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559587416070 0x5595877ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f8836a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873f5390 0x5595877bb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37676/0 (socket says 172.21.15.39:37676) 2024-07-04T08:29:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f8836a00640 1 -- 172.21.15.39:0/4236693429 learned_addr learned my addr 172.21.15.39:0/4236693429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f883fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559587416070 0x5595877ba910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51454/0 (socket says 172.21.15.39:51454) 2024-07-04T08:29:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f883f400640 1 --2- 172.21.15.39:0/4236693429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 0x5595877bae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f8836a00640 1 -- 172.21.15.39:0/4236693429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 msgr2=0x5595877bae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f8836a00640 1 --2- 172.21.15.39:0/4236693429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 0x5595877bae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f8836a00640 1 -- 172.21.15.39:0/4236693429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559587416070 msgr2=0x5595877ba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f8836a00640 1 --2- 172.21.15.39:0/4236693429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559587416070 0x5595877ba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.906+0000 7f8836a00640 1 -- 172.21.15.39:0/4236693429 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595876b8610 con 0x5595873f5390 2024-07-04T08:29:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.907+0000 7f8836a00640 1 --2- 172.21.15.39:0/4236693429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873f5390 0x5595877bb390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f88280117e0 tx=0x7f8828011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4489750da6a30b4d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.907+0000 7f8834c00640 1 -- 172.21.15.39:0/4236693429 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828003c30 con 0x5595873f5390 2024-07-04T08:29:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.907+0000 7f8834c00640 1 -- 172.21.15.39:0/4236693429 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882800fe50 con 0x5595873f5390 2024-07-04T08:29:15.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.907+0000 7f8834c00640 1 -- 172.21.15.39:0/4236693429 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280193e0 con 0x5595873f5390 2024-07-04T08:29:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f883cc00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88240b2960 0x7f88240b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f883cc00640 1 -- 172.21.15.39:0/1229271574 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f88300e2d00 con 0x7f8840102ee0 2024-07-04T08:29:15.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f883fe00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88240b2960 0x7f88240b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f884ca3d740 1 -- 172.21.15.39:0/4236693429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873f5390 msgr2=0x5595877bb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4236693429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873f5390 0x5595877bb390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f88280117e0 tx=0x7f8828011ca0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f884ca3d740 1 -- 172.21.15.39:0/4236693429 shutdown_connections 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4236693429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873f5390 0x5595877bb390 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4236693429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 0x5595877bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4236693429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559587416070 0x5595877ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f884ca3d740 1 -- 172.21.15.39:0/4236693429 >> 172.21.15.39:0/4236693429 conn(0x55958740a460 msgr2=0x5595877b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.908+0000 7f884ca3d740 1 -- 172.21.15.39:0/4236693429 shutdown_connections 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.909+0000 7f884ca3d740 1 -- 172.21.15.39:0/4236693429 wait complete. 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.909+0000 7f883fe00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88240b2960 0x7f88240b4e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8838006010 tx=0x7f88380058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:15.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.909+0000 7f884ca3d740 1 -- start start 2024-07-04T08:29:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.910+0000 7f884ca3d740 1 Processor -- start 2024-07-04T08:29:15.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.910+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.910+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.910+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 0x5595877d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.910+0000 7f884ca3d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559587584700 con 0x5595873f5390 2024-07-04T08:29:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.910+0000 7f884ca3d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5595876c7b70 con 0x5595873ed1b0 2024-07-04T08:29:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.910+0000 7f884ca3d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5595877c90f0 con 0x559587416070 2024-07-04T08:29:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.910+0000 7f884ca3d740 10 client.0 _ll_register_callbacks cb 0x559587719530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:15.902 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f883fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 0x5595877d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f8836a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f883fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 0x5595877d2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:54824/0 (socket says 172.21.15.39:54824) 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f883fe00640 1 -- 172.21.15.39:0/4164210755 learned_addr learned my addr 172.21.15.39:0/4164210755 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f8836a00640 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51456/0 (socket says 172.21.15.39:51456) 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f883f400640 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f8836a00640 1 -- 172.21.15.39:0/4164210755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 msgr2=0x5595877d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f8836a00640 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 0x5595877d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f8836a00640 1 -- 172.21.15.39:0/4164210755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 msgr2=0x5595877c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f8836a00640 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.911+0000 7f8836a00640 1 -- 172.21.15.39:0/4164210755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559587481650 con 0x5595873f5390 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.912+0000 7f8836a00640 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877c6a10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8828000f80 tx=0x7f882800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5aa10e289d09719 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.912+0000 7f882d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:29:15.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.912+0000 7f882d600640 1 -- 172.21.15.39:0/4164210755 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828028020 con 0x5595873f5390 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.912+0000 7f882d600640 1 -- 172.21.15.39:0/4164210755 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f8828004310 con 0x5595873f5390 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.912+0000 7f884ca3d740 1 -- 172.21.15.39:0/4164210755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882800fe50 con 0x5595873f5390 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.912+0000 7f882d600640 1 -- 172.21.15.39:0/4164210755 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8828033050 con 0x5595873f5390 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.912+0000 7f884ca3d740 10 client.18360 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.912+0000 7f884ca3d740 1 -- 172.21.15.39:0/4164210755 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595877d5060 con 0x5595873f5390 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.913+0000 7f882d600640 1 -- 172.21.15.39:0/4164210755 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f882800fe50 con 0x5595873f5390 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.913+0000 7f882d600640 1 -- 172.21.15.39:0/4164210755 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8828018070 con 0x5595873f5390 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.913+0000 7f884ca3d740 10 client.18360 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.913+0000 7f884ca3d740 10 client.18360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.913+0000 7f884ca3d740 20 client.18360 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.913+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5595873d49f0 0x5595877d8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.913+0000 7f884ca3d740 4 client.18360 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:15.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.913+0000 7f884ca3d740 1 -- 172.21.15.39:0/4164210755 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559587732df0 con 0x5595873d49f0 2024-07-04T08:29:15.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.914+0000 7f883f400640 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5595873d49f0 0x5595877d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.914+0000 7f883f400640 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5595873d49f0 0x5595877d8470 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03ee555a18cd3c8 server_cookie=78553ce0e6cc7fb7 in_seq=0 out_seq=0 2024-07-04T08:29:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.914+0000 7f882d600640 10 client.18360 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:15.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.915+0000 7f882d600640 1 -- 172.21.15.39:0/4164210755 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559587732df0 con 0x5595873d49f0 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.915+0000 7f882d600640 10 client.18360 handle_command_reply: tid=0 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.915+0000 7f884ca3d740 1 client.18360 shutdown 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.915+0000 7f884ca3d740 1 -- 172.21.15.39:0/4164210755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 msgr2=0x5595877c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.915+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877c6a10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8828000f80 tx=0x7f882800fc30 comp rx=0 tx=0).stop 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f884ca3d740 1 -- 172.21.15.39:0/4164210755 shutdown_connections 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5595873d49f0 0x5595877d8470 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877c6a10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 0x5595877d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f884ca3d740 1 --2- 172.21.15.39:0/4164210755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f882d600640 0 client.18360 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f884ca3d740 1 -- 172.21.15.39:0/4164210755 >> 172.21.15.39:0/4164210755 conn(0x55958740a460 msgr2=0x5595877bdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f884ca3d740 1 -- 172.21.15.39:0/4164210755 shutdown_connections 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f884ca3d740 1 -- 172.21.15.39:0/4164210755 wait complete. 2024-07-04T08:29:15.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.916+0000 7f884ca3d740 20 client.18360 trim_cache size 0 max 16384 2024-07-04T08:29:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.936+0000 7f884ca3d740 1 Processor -- start 2024-07-04T08:29:15.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.936+0000 7f884ca3d740 1 -- start start 2024-07-04T08:29:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 0x5595877be120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f884ca3d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b23f0 con 0x5595873f5390 2024-07-04T08:29:15.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f884ca3d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b3f30 con 0x5595873ed1b0 2024-07-04T08:29:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f884ca3d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b2ac0 con 0x559587416070 2024-07-04T08:29:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f8836a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f883f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f883f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:37682/0 (socket says 172.21.15.39:37682) 2024-07-04T08:29:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f883f400640 1 -- 172.21.15.39:0/2888562495 learned_addr learned my addr 172.21.15.39:0/2888562495 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f8836a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51470/0 (socket says 172.21.15.39:51470) 2024-07-04T08:29:15.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f883fe00640 1 --2- 172.21.15.39:0/2888562495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 0x5595877be120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f883f400640 1 -- 172.21.15.39:0/2888562495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 msgr2=0x5595877be120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f883f400640 1 --2- 172.21.15.39:0/2888562495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 0x5595877be120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f883f400640 1 -- 172.21.15.39:0/2888562495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 msgr2=0x5595877bb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f883f400640 1 --2- 172.21.15.39:0/2888562495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.937+0000 7f883f400640 1 -- 172.21.15.39:0/2888562495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595877c7590 con 0x5595873ed1b0 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.938+0000 7f8836a00640 1 --2- 172.21.15.39:0/2888562495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.938+0000 7f883f400640 1 --2- 172.21.15.39:0/2888562495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c81b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f88300a5d50 tx=0x7f88300e67b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33f6e8aa5e6f3963 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.938+0000 7f882fe00640 1 -- 172.21.15.39:0/2888562495 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830010720 con 0x5595873ed1b0 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.938+0000 7f882fe00640 1 -- 172.21.15.39:0/2888562495 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88300b23c0 con 0x5595873ed1b0 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.938+0000 7f882fe00640 1 -- 172.21.15.39:0/2888562495 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88300b2a80 con 0x5595873ed1b0 2024-07-04T08:29:15.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.939+0000 7f884ca3d740 1 -- 172.21.15.39:0/2888562495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 msgr2=0x5595877c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.939+0000 7f884ca3d740 1 --2- 172.21.15.39:0/2888562495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c81b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f88300a5d50 tx=0x7f88300e67b0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.939+0000 7f884ca3d740 1 -- 172.21.15.39:0/2888562495 shutdown_connections 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.939+0000 7f884ca3d740 1 --2- 172.21.15.39:0/2888562495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.939+0000 7f884ca3d740 1 --2- 172.21.15.39:0/2888562495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559587416070 0x5595877be120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.939+0000 7f884ca3d740 1 --2- 172.21.15.39:0/2888562495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595873ed1b0 0x5595877c81b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.939+0000 7f884ca3d740 1 -- 172.21.15.39:0/2888562495 >> 172.21.15.39:0/2888562495 conn(0x55958740a460 msgr2=0x5595877bda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.939+0000 7f884ca3d740 1 -- 172.21.15.39:0/2888562495 shutdown_connections 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.939+0000 7f884ca3d740 1 -- 172.21.15.39:0/2888562495 wait complete. 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.940+0000 7f884ca3d740 1 -- start start 2024-07-04T08:29:15.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.940+0000 7f884ca3d740 1 Processor -- start 2024-07-04T08:29:15.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.940+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 0x5595877ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.940+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.940+0000 7f884ca3d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559587416070 0x5595877bfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.940+0000 7f884ca3d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b3860 con 0x5595873f5390 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.940+0000 7f884ca3d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b3190 con 0x559587416070 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.940+0000 7f884ca3d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b03d0 con 0x5595873ed1b0 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f884ca3d740 10 client.0 _ll_register_callbacks cb 0x7f88280093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f8836a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f883fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559587416070 0x5595877bfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f8836a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51482/0 (socket says 172.21.15.39:51482) 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f8836a00640 1 -- 172.21.15.39:0/293109070 learned_addr learned my addr 172.21.15.39:0/293109070 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f883f400640 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 0x5595877ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f8836a00640 1 -- 172.21.15.39:0/293109070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 msgr2=0x5595877ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:15.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f8836a00640 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 0x5595877ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f8836a00640 1 -- 172.21.15.39:0/293109070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559587416070 msgr2=0x5595877bfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:15.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f8836a00640 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559587416070 0x5595877bfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f8836a00640 1 -- 172.21.15.39:0/293109070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595877c0270 con 0x5595873f5390 2024-07-04T08:29:15.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f883fe00640 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559587416070 0x5595877bfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:15.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.941+0000 7f883f400640 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 0x5595877ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:15.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.942+0000 7f8836a00640 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bf460 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8828005c40 tx=0x7f88280116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b6039f1e7b28c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:15.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.942+0000 7f882d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:29:15.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.942+0000 7f882d600640 1 -- 172.21.15.39:0/293109070 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828028040 con 0x5595873f5390 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.942+0000 7f882d600640 1 -- 172.21.15.39:0/293109070 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f8828004110 con 0x5595873f5390 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.942+0000 7f884ca3d740 1 -- 172.21.15.39:0/293109070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55958772e890 con 0x5595873f5390 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.942+0000 7f882d600640 1 -- 172.21.15.39:0/293109070 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f882801f040 con 0x5595873f5390 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.942+0000 7f884ca3d740 10 client.18366 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.942+0000 7f884ca3d740 1 -- 172.21.15.39:0/293109070 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559587840be0 con 0x5595873f5390 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.943+0000 7f882d600640 1 -- 172.21.15.39:0/293109070 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55958772e890 con 0x5595873f5390 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.943+0000 7f882d600640 1 -- 172.21.15.39:0/293109070 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8828018070 con 0x5595873f5390 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.943+0000 7f884ca3d740 10 client.18366 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.943+0000 7f884ca3d740 10 client.18366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.943+0000 7f884ca3d740 20 client.18366 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.943+0000 7f884ca3d740 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5595873a9da0 0x559587843340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:15.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.943+0000 7f884ca3d740 4 client.18366 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:29:15.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.943+0000 7f884ca3d740 1 -- 172.21.15.39:0/293109070 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55958772bfd0 con 0x5595873a9da0 2024-07-04T08:29:15.935 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.943+0000 7f883f400640 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5595873a9da0 0x559587843340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.944+0000 7f882d600640 10 client.18366 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.944+0000 7f883f400640 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5595873a9da0 0x559587843340 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6d9c434ea71b6b server_cookie=36b690dcbce7b24e in_seq=0 out_seq=0 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.944+0000 7f882d600640 1 -- 172.21.15.39:0/293109070 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55958772bfd0 con 0x5595873a9da0 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.944+0000 7f882d600640 10 client.18366 handle_command_reply: tid=0 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.944+0000 7f884ca3d740 1 client.18366 shutdown 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 -- 172.21.15.39:0/293109070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 msgr2=0x5595877bf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bf460 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8828005c40 tx=0x7f88280116f0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 -- 172.21.15.39:0/293109070 shutdown_connections 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5595873a9da0 0x559587843340 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595873f5390 0x5595877bf460 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559587416070 0x5595877bfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 --2- 172.21.15.39:0/293109070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595873ed1b0 0x5595877ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 -- 172.21.15.39:0/293109070 >> 172.21.15.39:0/293109070 conn(0x55958740a460 msgr2=0x5595877c7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f882d600640 0 client.18366 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 -- 172.21.15.39:0/293109070 shutdown_connections 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 1 -- 172.21.15.39:0/293109070 wait complete. 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.945+0000 7f884ca3d740 20 client.18366 trim_cache size 0 max 16384 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:29:15.937 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:29:15.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 -- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88240b2960 msgr2=0x7f88240b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:15.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88240b2960 0x7f88240b4e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8838006010 tx=0x7f88380058d0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 -- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840102ee0 msgr2=0x7f884019b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:15.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840102ee0 0x7f884019b9e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9f0 tx=0x7f883000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 -- 172.21.15.39:0/1229271574 shutdown_connections 2024-07-04T08:29:15.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f88240b2960 0x7f88240b4e70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8840109170 0x7f88401a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8840108820 0x7f884019bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 --2- 172.21.15.39:0/1229271574 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8840102ee0 0x7f884019b9e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:15.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 -- 172.21.15.39:0/1229271574 >> 172.21.15.39:0/1229271574 conn(0x7f88400fe930 msgr2=0x7f884010dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:15.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 -- 172.21.15.39:0/1229271574 shutdown_connections 2024-07-04T08:29:15.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:15.946+0000 7f8844e00640 1 -- 172.21.15.39:0/1229271574 wait complete. 2024-07-04T08:29:16.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:29:16.720 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:29:16.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:29:16.759 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:17.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:17 smithi133 ceph-mon[37050]: pgmap v918: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-07-04T08:29:17.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:17 smithi039 ceph-mon[32062]: pgmap v918: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-07-04T08:29:17.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:17 smithi050 ceph-mon[36577]: pgmap v918: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-07-04T08:29:17.654 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:19.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.241+0000 7f56f2000640 1 -- 172.21.15.39:0/192276215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec108820 msgr2=0x7f56ec108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:19.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.241+0000 7f56f2000640 1 --2- 172.21.15.39:0/192276215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec108820 0x7f56ec108c30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009be0 tx=0x7f56dc02a6c0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.242+0000 7f56f2000640 1 -- 172.21.15.39:0/192276215 shutdown_connections 2024-07-04T08:29:19.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.242+0000 7f56f2000640 1 --2- 172.21.15.39:0/192276215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56ec109170 0x7f56ec10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.242+0000 7f56f2000640 1 --2- 172.21.15.39:0/192276215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec108820 0x7f56ec108c30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.242+0000 7f56f2000640 1 --2- 172.21.15.39:0/192276215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56ec102ee0 0x7f56ec1032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.242+0000 7f56f2000640 1 -- 172.21.15.39:0/192276215 >> 172.21.15.39:0/192276215 conn(0x7f56ec0fe930 msgr2=0x7f56ec100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.242+0000 7f56f2000640 1 -- 172.21.15.39:0/192276215 shutdown_connections 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.242+0000 7f56f2000640 1 -- 172.21.15.39:0/192276215 wait complete. 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.244+0000 7f56f2000640 1 Processor -- start 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.244+0000 7f56f2000640 1 -- start start 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.245+0000 7f56f2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56ec102ee0 0x7f56ec199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.245+0000 7f56f2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56ec108820 0x7f56ec199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.245+0000 7f56f2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec109170 0x7f56ec19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.245+0000 7f56f2000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec112e30 con 0x7f56ec108820 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.245+0000 7f56f2000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec112cb0 con 0x7f56ec109170 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.245+0000 7f56f2000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec112fb0 con 0x7f56ec102ee0 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.245+0000 7f56f0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56ec102ee0 0x7f56ec199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:19.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.245+0000 7f56f0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56ec102ee0 0x7f56ec199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:54846/0 (socket says 172.21.15.39:54846) 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.245+0000 7f56f0c00640 1 -- 172.21.15.39:0/1733109440 learned_addr learned my addr 172.21.15.39:0/1733109440 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.246+0000 7f56ebe00640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56ec108820 0x7f56ec199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.246+0000 7f56f1600640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec109170 0x7f56ec19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.246+0000 7f56f0c00640 1 -- 172.21.15.39:0/1733109440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec109170 msgr2=0x7f56ec19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.246+0000 7f56f0c00640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec109170 0x7f56ec19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.246+0000 7f56f0c00640 1 -- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56ec108820 msgr2=0x7f56ec199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.246+0000 7f56f0c00640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56ec108820 0x7f56ec199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.246+0000 7f56f0c00640 1 -- 172.21.15.39:0/1733109440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec19f790 con 0x7f56ec102ee0 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.246+0000 7f56f0c00640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56ec102ee0 0x7f56ec199820 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f56d400e9c0 tx=0x7f56d400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.247+0000 7f56e9600640 1 -- 172.21.15.39:0/1733109440 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d400ccc0 con 0x7f56ec102ee0 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.247+0000 7f56e9600640 1 -- 172.21.15.39:0/1733109440 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56d40054d0 con 0x7f56ec102ee0 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.247+0000 7f56e9600640 1 -- 172.21.15.39:0/1733109440 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4010430 con 0x7f56ec102ee0 2024-07-04T08:29:19.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.247+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec19fa80 con 0x7f56ec102ee0 2024-07-04T08:29:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.247+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec1a61e0 con 0x7f56ec102ee0 2024-07-04T08:29:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.248+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec104160 con 0x7f56ec102ee0 2024-07-04T08:29:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.251+0000 7f56e9600640 1 -- 172.21.15.39:0/1733109440 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f56d4004f80 con 0x7f56ec102ee0 2024-07-04T08:29:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.253+0000 7f56e9600640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56d00b2960 0x7f56d00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.253+0000 7f56e9600640 1 -- 172.21.15.39:0/1733109440 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f56d40e34b0 con 0x7f56ec102ee0 2024-07-04T08:29:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.253+0000 7f56ebe00640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56d00b2960 0x7f56d00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.254+0000 7f56ebe00640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56d00b2960 0x7f56d00b4e70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f56dc02acb0 tx=0x7f56dc004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:19.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.254+0000 7f56e9600640 1 -- 172.21.15.39:0/1733109440 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f56d40aaad0 con 0x7f56ec102ee0 2024-07-04T08:29:19.501 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.512+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56ec10afc0 con 0x7f56ec102ee0 2024-07-04T08:29:19.501 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:19 smithi039 ceph-mon[32062]: pgmap v919: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 KiB/s wr, 11 op/s 2024-07-04T08:29:19.502 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.514+0000 7f56e9600640 1 -- 172.21.15.39:0/1733109440 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f56d40aa3d0 con 0x7f56ec102ee0 2024-07-04T08:29:19.502 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:29:19.503 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:29:19.503 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:29:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.517+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56d00b2960 msgr2=0x7f56d00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.517+0000 7f56f2000640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56d00b2960 0x7f56d00b4e70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f56dc02acb0 tx=0x7f56dc004a40 comp rx=0 tx=0).stop 2024-07-04T08:29:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.517+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56ec102ee0 msgr2=0x7f56ec199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.517+0000 7f56f2000640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56ec102ee0 0x7f56ec199820 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f56d400e9c0 tx=0x7f56d400ee80 comp rx=0 tx=0).stop 2024-07-04T08:29:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.518+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 shutdown_connections 2024-07-04T08:29:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.518+0000 7f56f2000640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f56d00b2960 0x7f56d00b4e70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.518+0000 7f56f2000640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56ec108820 0x7f56ec199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.506 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.518+0000 7f56f2000640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f56ec109170 0x7f56ec19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.518+0000 7f56f2000640 1 --2- 172.21.15.39:0/1733109440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56ec102ee0 0x7f56ec199820 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.518+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 >> 172.21.15.39:0/1733109440 conn(0x7f56ec0fe930 msgr2=0x7f56ec100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.518+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 shutdown_connections 2024-07-04T08:29:19.507 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:19.518+0000 7f56f2000640 1 -- 172.21.15.39:0/1733109440 wait complete. 2024-07-04T08:29:19.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:19 smithi133 ceph-mon[37050]: pgmap v919: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 KiB/s wr, 11 op/s 2024-07-04T08:29:19.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:19 smithi050 ceph-mon[36577]: pgmap v919: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.3 KiB/s wr, 11 op/s 2024-07-04T08:29:20.478 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:20 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1733109440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:29:20.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:20 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1733109440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:29:20.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:20 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1733109440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:29:20.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.970+0000 7f7882000640 1 -- 172.21.15.39:0/475601334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c109370 msgr2=0x7f787c10fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:20.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.970+0000 7f7882000640 1 --2- 172.21.15.39:0/475601334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c109370 0x7f787c10fba0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7874009900 tx=0x7f787402a410 comp rx=0 tx=0).stop 2024-07-04T08:29:20.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.971+0000 7f7882000640 1 -- 172.21.15.39:0/475601334 shutdown_connections 2024-07-04T08:29:20.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.971+0000 7f7882000640 1 --2- 172.21.15.39:0/475601334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c109370 0x7f787c10fba0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:20.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.971+0000 7f7882000640 1 --2- 172.21.15.39:0/475601334 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c108a20 0x7f787c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:20.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.971+0000 7f7882000640 1 --2- 172.21.15.39:0/475601334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f787c1030e0 0x7f787c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:20.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.971+0000 7f7882000640 1 -- 172.21.15.39:0/475601334 >> 172.21.15.39:0/475601334 conn(0x7f787c0feb50 msgr2=0x7f787c100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:20.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.971+0000 7f7882000640 1 -- 172.21.15.39:0/475601334 shutdown_connections 2024-07-04T08:29:20.960 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.971+0000 7f7882000640 1 -- 172.21.15.39:0/475601334 wait complete. 2024-07-04T08:29:20.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.972+0000 7f7882000640 1 Processor -- start 2024-07-04T08:29:20.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.972+0000 7f7882000640 1 -- start start 2024-07-04T08:29:20.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f7882000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c1030e0 0x7f787c19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:20.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f7882000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c108a20 0x7f787c19c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:20.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f7882000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f787c109370 0x7f787c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:20.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f7882000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f787c113070 con 0x7f787c1030e0 2024-07-04T08:29:20.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f7882000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f787c112ef0 con 0x7f787c108a20 2024-07-04T08:29:20.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f7882000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f787c1131f0 con 0x7f787c109370 2024-07-04T08:29:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f787b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c1030e0 0x7f787c19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f787aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c108a20 0x7f787c19c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f787b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c1030e0 0x7f787c19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51500/0 (socket says 172.21.15.39:51500) 2024-07-04T08:29:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f787b400640 1 -- 172.21.15.39:0/3082564601 learned_addr learned my addr 172.21.15.39:0/3082564601 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.973+0000 7f787be00640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f787c109370 0x7f787c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f787b400640 1 -- 172.21.15.39:0/3082564601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f787c109370 msgr2=0x7f787c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f787b400640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f787c109370 0x7f787c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f787b400640 1 -- 172.21.15.39:0/3082564601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c108a20 msgr2=0x7f787c19c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:20.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f787b400640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c108a20 0x7f787c19c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f787b400640 1 -- 172.21.15.39:0/3082564601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c1a1af0 con 0x7f787c1030e0 2024-07-04T08:29:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f787aa00640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c108a20 0x7f787c19c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f787b400640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c1030e0 0x7f787c19bb80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f786c00d950 tx=0x7f786c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f7873e00640 1 -- 172.21.15.39:0/3082564601 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c017070 con 0x7f787c1030e0 2024-07-04T08:29:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f7873e00640 1 -- 172.21.15.39:0/3082564601 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f786c009c30 con 0x7f787c1030e0 2024-07-04T08:29:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.974+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787c1a1de0 con 0x7f787c1030e0 2024-07-04T08:29:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.975+0000 7f7873e00640 1 -- 172.21.15.39:0/3082564601 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c00f620 con 0x7f787c1030e0 2024-07-04T08:29:20.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.975+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787c1a8540 con 0x7f787c1030e0 2024-07-04T08:29:20.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.976+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787c1034f0 con 0x7f787c1030e0 2024-07-04T08:29:20.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.978+0000 7f7873e00640 1 -- 172.21.15.39:0/3082564601 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f786c010460 con 0x7f787c1030e0 2024-07-04T08:29:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.980+0000 7f7873e00640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b2960 0x7f785c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:20.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.980+0000 7f7873e00640 1 -- 172.21.15.39:0/3082564601 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f786c0e2520 con 0x7f787c1030e0 2024-07-04T08:29:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.980+0000 7f787aa00640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b2960 0x7f785c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:20.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.981+0000 7f787aa00640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b2960 0x7f785c0b4e70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f787c105030 tx=0x7f7864009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:20.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:20.985+0000 7f7873e00640 1 -- 172.21.15.39:0/3082564601 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f786c009de0 con 0x7f787c1030e0 2024-07-04T08:29:21.026 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id b0a3c823 --release --await' 2024-07-04T08:29:21.026 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id b0a3c823 --release --await 2024-07-04T08:29:21.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.246+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f787c1a88b0 con 0x7f787c1030e0 2024-07-04T08:29:21.236 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.247+0000 7f7873e00640 1 -- 172.21.15.39:0/3082564601 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f786c0a9530 con 0x7f787c1030e0 2024-07-04T08:29:21.236 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:29:21.237 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:29:21.237 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:29:21.238 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.250+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b2960 msgr2=0x7f785c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.250+0000 7f7882000640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b2960 0x7f785c0b4e70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f787c105030 tx=0x7f7864009500 comp rx=0 tx=0).stop 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.250+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c1030e0 msgr2=0x7f787c19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.250+0000 7f7882000640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c1030e0 0x7f787c19bb80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f786c00d950 tx=0x7f786c00de10 comp rx=0 tx=0).stop 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.251+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 shutdown_connections 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.251+0000 7f7882000640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b2960 0x7f785c0b4e70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.251+0000 7f7882000640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f787c109370 0x7f787c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.251+0000 7f7882000640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787c108a20 0x7f787c19c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.251+0000 7f7882000640 1 --2- 172.21.15.39:0/3082564601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787c1030e0 0x7f787c19bb80 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.251+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 >> 172.21.15.39:0/3082564601 conn(0x7f787c0feb50 msgr2=0x7f787c107fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.251+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 shutdown_connections 2024-07-04T08:29:21.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:21.251+0000 7f7882000640 1 -- 172.21.15.39:0/3082564601 wait complete. 2024-07-04T08:29:21.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:21 smithi133 ceph-mon[37050]: pgmap v920: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-07-04T08:29:21.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:21 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3082564601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:29:21.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:21 smithi039 ceph-mon[32062]: pgmap v920: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-07-04T08:29:21.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:21 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3082564601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:29:21.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:21 smithi050 ceph-mon[36577]: pgmap v920: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 11 op/s 2024-07-04T08:29:21.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:21 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3082564601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:29:22.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:29:22.201 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:23.375 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:23 smithi039 ceph-mon[32062]: pgmap v921: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-07-04T08:29:23.487 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:23.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:23 smithi133 ceph-mon[37050]: pgmap v921: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-07-04T08:29:23.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:23 smithi050 ceph-mon[36577]: pgmap v921: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-07-04T08:29:24.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.556+0000 7f72b8c00640 1 -- 172.21.15.39:0/3969489858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b410aa50 msgr2=0x7f72b410ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:24.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.556+0000 7f72b8c00640 1 --2- 172.21.15.39:0/3969489858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b410aa50 0x7f72b410ae60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f729c009900 tx=0x7f729c02a410 comp rx=0 tx=0).stop 2024-07-04T08:29:24.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.556+0000 7f72b8c00640 1 -- 172.21.15.39:0/3969489858 shutdown_connections 2024-07-04T08:29:24.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.556+0000 7f72b8c00640 1 --2- 172.21.15.39:0/3969489858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72b410b3a0 0x7f72b4111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.556+0000 7f72b8c00640 1 --2- 172.21.15.39:0/3969489858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b410aa50 0x7f72b410ae60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.556+0000 7f72b8c00640 1 --2- 172.21.15.39:0/3969489858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b4105110 0x7f72b4105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.556+0000 7f72b8c00640 1 -- 172.21.15.39:0/3969489858 >> 172.21.15.39:0/3969489858 conn(0x7f72b4100bc0 msgr2=0x7f72b4103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.556+0000 7f72b8c00640 1 -- 172.21.15.39:0/3969489858 shutdown_connections 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.556+0000 7f72b8c00640 1 -- 172.21.15.39:0/3969489858 wait complete. 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.557+0000 7f72b8c00640 1 Processor -- start 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.558+0000 7f72b8c00640 1 -- start start 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.558+0000 7f72b8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b4105110 0x7f72b419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.558+0000 7f72b8c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b410aa50 0x7f72b419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.558+0000 7f72b8c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72b410b3a0 0x7f72b41a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.558+0000 7f72b8c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4115060 con 0x7f72b410b3a0 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.558+0000 7f72b8c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4114ee0 con 0x7f72b410aa50 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.558+0000 7f72b8c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41151e0 con 0x7f72b4105110 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b4105110 0x7f72b419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b4105110 0x7f72b419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44230/0 (socket says 172.21.15.39:44230) 2024-07-04T08:29:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b3400640 1 -- 172.21.15.39:0/1653405356 learned_addr learned my addr 172.21.15.39:0/1653405356 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b3e00640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72b410b3a0 0x7f72b41a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b2a00640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b410aa50 0x7f72b419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b3400640 1 -- 172.21.15.39:0/1653405356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b410aa50 msgr2=0x7f72b419bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b3400640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b410aa50 0x7f72b419bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b3400640 1 -- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72b410b3a0 msgr2=0x7f72b41a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b3400640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72b410b3a0 0x7f72b41a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.559+0000 7f72b3400640 1 -- 172.21.15.39:0/1653405356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b41a19a0 con 0x7f72b4105110 2024-07-04T08:29:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.560+0000 7f72b3400640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b4105110 0x7f72b419ba30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f72a400ea60 tx=0x7f72a400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.560+0000 7f72abe00640 1 -- 172.21.15.39:0/1653405356 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a400cde0 con 0x7f72b4105110 2024-07-04T08:29:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.560+0000 7f72abe00640 1 -- 172.21.15.39:0/1653405356 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a40035f0 con 0x7f72b4105110 2024-07-04T08:29:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.560+0000 7f72abe00640 1 -- 172.21.15.39:0/1653405356 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4010690 con 0x7f72b4105110 2024-07-04T08:29:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.560+0000 7f72b8c00640 1 -- 172.21.15.39:0/1653405356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b41a1c90 con 0x7f72b4105110 2024-07-04T08:29:24.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.560+0000 7f72b8c00640 1 -- 172.21.15.39:0/1653405356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b41a83f0 con 0x7f72b4105110 2024-07-04T08:29:24.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.564+0000 7f72c08b3740 1 Processor -- start 2024-07-04T08:29:24.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.564+0000 7f72c08b3740 1 -- start start 2024-07-04T08:29:24.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.564+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf28520 0x56243c2cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.564+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.564+0000 7f72abe00640 1 -- 172.21.15.39:0/1653405356 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f72a40037a0 con 0x7f72b4105110 2024-07-04T08:29:24.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.564+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf07840 0x56243c2cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.564+0000 7f72c08b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56243c1e7b20 con 0x56243beff660 2024-07-04T08:29:24.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.564+0000 7f72c08b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56243c2432b0 con 0x56243bf07840 2024-07-04T08:29:24.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.564+0000 7f72c08b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56243c232850 con 0x56243bf28520 2024-07-04T08:29:24.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf28520 0x56243c2cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf07840 0x56243c2cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf28520 0x56243c2cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44234/0 (socket says 172.21.15.39:44234) 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b2a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf07840 0x56243c2cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56040/0 (socket says 172.21.15.39:56040) 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3e00640 1 -- 172.21.15.39:0/1340603635 learned_addr learned my addr 172.21.15.39:0/1340603635 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3e00640 1 -- 172.21.15.39:0/1340603635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf07840 msgr2=0x56243c2cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3e00640 1 --2- 172.21.15.39:0/1340603635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf07840 0x56243c2cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3e00640 1 -- 172.21.15.39:0/1340603635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 msgr2=0x56243c2cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3e00640 1 --2- 172.21.15.39:0/1340603635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3e00640 1 -- 172.21.15.39:0/1340603635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56243c1ec780 con 0x56243bf28520 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.565+0000 7f72b3400640 1 --2- 172.21.15.39:0/1340603635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:29:24.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.566+0000 7f72b3e00640 1 --2- 172.21.15.39:0/1340603635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf28520 0x56243c2cd490 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009fd0 tx=0x7f72ac0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8004631b4035eeb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:24.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.566+0000 7f72a8c00640 1 -- 172.21.15.39:0/1340603635 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac004d20 con 0x56243bf28520 2024-07-04T08:29:24.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.566+0000 7f72a8c00640 1 -- 172.21.15.39:0/1340603635 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ac004050 con 0x56243bf28520 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.566+0000 7f72a8c00640 1 -- 172.21.15.39:0/1340603635 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac005260 con 0x56243bf28520 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.566+0000 7f72abe00640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f72980b29e0 0x7f72980b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.566+0000 7f72abe00640 1 -- 172.21.15.39:0/1653405356 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f72a40e2c20 con 0x7f72b4105110 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72b3400640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f72980b29e0 0x7f72980b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72c08b3740 1 -- 172.21.15.39:0/1340603635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf28520 msgr2=0x56243c2cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72c08b3740 1 --2- 172.21.15.39:0/1340603635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf28520 0x56243c2cd490 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009fd0 tx=0x7f72ac0118d0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72c08b3740 1 -- 172.21.15.39:0/1340603635 shutdown_connections 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72c08b3740 1 --2- 172.21.15.39:0/1340603635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72c08b3740 1 --2- 172.21.15.39:0/1340603635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf07840 0x56243c2cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72c08b3740 1 --2- 172.21.15.39:0/1340603635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf28520 0x56243c2cd490 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72c08b3740 1 -- 172.21.15.39:0/1340603635 >> 172.21.15.39:0/1340603635 conn(0x56243bf1c910 msgr2=0x56243c2cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72c08b3740 1 -- 172.21.15.39:0/1340603635 shutdown_connections 2024-07-04T08:29:24.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.567+0000 7f72b3400640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f72980b29e0 0x7f72980b4ef0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x56243c2cc110 tx=0x7f72a400ef90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:24.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.568+0000 7f72c08b3740 1 -- 172.21.15.39:0/1340603635 wait complete. 2024-07-04T08:29:24.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.568+0000 7f72c08b3740 1 -- start start 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.568+0000 7f72c08b3740 1 Processor -- start 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2d8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 0x56243c2d9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72c08b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56243c1d9d20 con 0x56243bf28520 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72c08b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56243c236890 con 0x56243beff660 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72c08b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56243c096bb0 con 0x56243bf07840 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72c08b3740 10 client.0 _ll_register_callbacks cb 0x56243c1da9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 0x56243c2d9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.569+0000 7f72b2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2d8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.570+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2d8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56056/0 (socket says 172.21.15.39:56056) 2024-07-04T08:29:24.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.570+0000 7f72b3e00640 1 -- 172.21.15.39:0/2204125235 learned_addr learned my addr 172.21.15.39:0/2204125235 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:24.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.570+0000 7f72b2a00640 1 -- 172.21.15.39:0/2204125235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 msgr2=0x56243c2d8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:24.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.570+0000 7f72b2a00640 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2d8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.570+0000 7f72b2a00640 1 -- 172.21.15.39:0/2204125235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 msgr2=0x56243c2d9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:24.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.570+0000 7f72b2a00640 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 0x56243c2d9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.570+0000 7f72b2a00640 1 -- 172.21.15.39:0/2204125235 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56243c1c9660 con 0x56243bf07840 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.570+0000 7f72b2a00640 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2d8fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f729c009230 tx=0x7f729c004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e16852778a77d7f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.570+0000 7f72a1600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.571+0000 7f72a1600640 1 -- 172.21.15.39:0/2204125235 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c035040 con 0x56243bf07840 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.571+0000 7f72a1600640 1 -- 172.21.15.39:0/2204125235 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f729c005450 con 0x56243bf07840 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.571+0000 7f72c08b3740 1 -- 172.21.15.39:0/2204125235 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ac004050 con 0x56243bf07840 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.571+0000 7f72a1600640 1 -- 172.21.15.39:0/2204125235 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f729c04f050 con 0x56243bf07840 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.571+0000 7f72c08b3740 10 client.28447 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.571+0000 7f72c08b3740 1 -- 172.21.15.39:0/2204125235 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56243bf93b00 con 0x56243bf07840 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.571+0000 7f72a1600640 1 -- 172.21.15.39:0/2204125235 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72ac004050 con 0x56243bf07840 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.572+0000 7f72a1600640 1 -- 172.21.15.39:0/2204125235 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f729c0037f0 con 0x56243bf07840 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.572+0000 7f72c08b3740 10 client.28447 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.572+0000 7f72c08b3740 10 client.28447 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:29:24.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.572+0000 7f72c08b3740 20 client.28447 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:24.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.572+0000 7f72c08b3740 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56243bee6ea0 0x56243c2eaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.572+0000 7f72c08b3740 4 client.28447 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:24.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.572+0000 7f72c08b3740 1 -- 172.21.15.39:0/2204125235 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56243c245200 con 0x56243bee6ea0 2024-07-04T08:29:24.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.572+0000 7f72b3e00640 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56243bee6ea0 0x56243c2eaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.573+0000 7f72b3e00640 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56243bee6ea0 0x56243c2eaaa0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28d6822b92d6b95 server_cookie=821f63b5ad1b03fb in_seq=0 out_seq=0 2024-07-04T08:29:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.573+0000 7f72a1600640 10 client.28447 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72a1600640 1 -- 172.21.15.39:0/2204125235 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56243c245200 con 0x56243bee6ea0 2024-07-04T08:29:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72a1600640 10 client.28447 handle_command_reply: tid=0 2024-07-04T08:29:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 client.28447 shutdown 2024-07-04T08:29:24.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 -- 172.21.15.39:0/2204125235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 msgr2=0x56243c2d8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2d8fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f729c009230 tx=0x7f729c004340 comp rx=0 tx=0).stop 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 -- 172.21.15.39:0/2204125235 shutdown_connections 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56243bee6ea0 0x56243c2eaaa0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 0x56243c2d9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2d8fe0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72a1600640 0 client.28447 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 --2- 172.21.15.39:0/2204125235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2d8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 -- 172.21.15.39:0/2204125235 >> 172.21.15.39:0/2204125235 conn(0x56243bf1c910 msgr2=0x56243c2d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 -- 172.21.15.39:0/2204125235 shutdown_connections 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.574+0000 7f72c08b3740 1 -- 172.21.15.39:0/2204125235 wait complete. 2024-07-04T08:29:24.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.575+0000 7f72c08b3740 20 client.28447 trim_cache size 0 max 16384 2024-07-04T08:29:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.595+0000 7f72c08b3740 1 Processor -- start 2024-07-04T08:29:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.595+0000 7f72c08b3740 1 -- start start 2024-07-04T08:29:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.595+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.595+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2f0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.595+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf28520 0x56243c2f0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.595+0000 7f72c08b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b2390 con 0x56243beff660 2024-07-04T08:29:24.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.595+0000 7f72c08b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b3ed0 con 0x56243bf28520 2024-07-04T08:29:24.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.595+0000 7f72c08b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b2a60 con 0x56243bf07840 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48908/0 (socket says 172.21.15.39:48908) 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b3e00640 1 -- 172.21.15.39:0/3853283837 learned_addr learned my addr 172.21.15.39:0/3853283837 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b2a00640 1 --2- 172.21.15.39:0/3853283837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b3e00640 1 --2- 172.21.15.39:0/3853283837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf28520 0x56243c2f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b2a00640 1 -- 172.21.15.39:0/3853283837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf28520 msgr2=0x56243c2f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b2a00640 1 --2- 172.21.15.39:0/3853283837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf28520 0x56243c2f0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b2a00640 1 -- 172.21.15.39:0/3853283837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 msgr2=0x56243c2cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b2a00640 1 --2- 172.21.15.39:0/3853283837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.596+0000 7f72b2a00640 1 -- 172.21.15.39:0/3853283837 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56243c00a960 con 0x56243bf07840 2024-07-04T08:29:24.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.597+0000 7f72b2a00640 1 --2- 172.21.15.39:0/3853283837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2f0240 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f729c005ab0 tx=0x7f729c031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1957a82f2e6747f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:24.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.597+0000 7f72a3e00640 1 -- 172.21.15.39:0/3853283837 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c035030 con 0x56243bf07840 2024-07-04T08:29:24.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.597+0000 7f72a3e00640 1 -- 172.21.15.39:0/3853283837 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f729c041070 con 0x56243bf07840 2024-07-04T08:29:24.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.597+0000 7f72a3e00640 1 -- 172.21.15.39:0/3853283837 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c004900 con 0x56243bf07840 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.598+0000 7f72c08b3740 1 -- 172.21.15.39:0/3853283837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 msgr2=0x56243c2f0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.598+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3853283837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2f0240 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f729c005ab0 tx=0x7f729c031400 comp rx=0 tx=0).stop 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.598+0000 7f72c08b3740 1 -- 172.21.15.39:0/3853283837 shutdown_connections 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.598+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3853283837 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243bf28520 0x56243c2f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.598+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3853283837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2f0240 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.598+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3853283837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243beff660 0x56243c2cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.598+0000 7f72c08b3740 1 -- 172.21.15.39:0/3853283837 >> 172.21.15.39:0/3853283837 conn(0x56243bf1c910 msgr2=0x56243c2c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.598+0000 7f72c08b3740 1 -- 172.21.15.39:0/3853283837 shutdown_connections 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.598+0000 7f72c08b3740 1 -- 172.21.15.39:0/3853283837 wait complete. 2024-07-04T08:29:24.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.599+0000 7f72c08b3740 1 -- start start 2024-07-04T08:29:24.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.599+0000 7f72c08b3740 1 Processor -- start 2024-07-04T08:29:24.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72c08b3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 0x56243c352210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72c08b3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b3800 con 0x56243bf28520 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72c08b3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b3130 con 0x56243beff660 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72c08b3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41b0370 con 0x56243bf07840 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72c08b3740 10 client.0 _ll_register_callbacks cb 0x7f729c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72b2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72b2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2dd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44260/0 (socket says 172.21.15.39:44260) 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 0x56243c352210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72b2a00640 1 -- 172.21.15.39:0/3932512995 learned_addr learned my addr 172.21.15.39:0/3932512995 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.600+0000 7f72b3e00640 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72b2a00640 1 -- 172.21.15.39:0/3932512995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 msgr2=0x56243c2dcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:24.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72b2a00640 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2dcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72b2a00640 1 -- 172.21.15.39:0/3932512995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 msgr2=0x56243c352210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:24.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72b2a00640 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 0x56243c352210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72b2a00640 1 -- 172.21.15.39:0/3932512995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56243c1ff8a0 con 0x56243bf07840 2024-07-04T08:29:24.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72b3e00640 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:29:24.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72b2a00640 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2dd480 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f729c000c00 tx=0x7f729c003880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f246e4d7cf9c7917 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:24.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72a1600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:29:24.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72a1600640 1 -- 172.21.15.39:0/3932512995 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c061060 con 0x56243bf07840 2024-07-04T08:29:24.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72a1600640 1 -- 172.21.15.39:0/3932512995 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f729c005450 con 0x56243bf07840 2024-07-04T08:29:24.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.601+0000 7f72c08b3740 1 -- 172.21.15.39:0/3932512995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f729c041070 con 0x56243bf07840 2024-07-04T08:29:24.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.602+0000 7f72a1600640 1 -- 172.21.15.39:0/3932512995 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f729c068050 con 0x56243bf07840 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.602+0000 7f72c08b3740 10 client.28453 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.602+0000 7f72c08b3740 1 -- 172.21.15.39:0/3932512995 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56243c1fb450 con 0x56243bf07840 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.602+0000 7f72a1600640 1 -- 172.21.15.39:0/3932512995 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f729c041070 con 0x56243bf07840 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.603+0000 7f72a1600640 1 -- 172.21.15.39:0/3932512995 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f729c05b070 con 0x56243bf07840 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.603+0000 7f72c08b3740 10 client.28453 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.603+0000 7f72c08b3740 10 client.28453 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.603+0000 7f72c08b3740 20 client.28453 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.603+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56243bebc250 0x56243c355a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.603+0000 7f72c08b3740 4 client.28453 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b0a3c823", "release": true, "await": true}] 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.603+0000 7f72c08b3740 1 -- 172.21.15.39:0/3932512995 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b0a3c823", "release": true, "await": true}) -- 0x56243c2d8300 con 0x56243bebc250 2024-07-04T08:29:24.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.603+0000 7f72b3e00640 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56243bebc250 0x56243c355a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:24.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.604+0000 7f72b3e00640 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56243bebc250 0x56243c355a90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be1a3b32dfe305ee server_cookie=af2d172eb5e83fd1 in_seq=0 out_seq=0 2024-07-04T08:29:24.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.604+0000 7f72a1600640 10 client.28453 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:24.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.606+0000 7f72a1600640 1 -- 172.21.15.39:0/3932512995 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56243c2d8300 con 0x56243bebc250 2024-07-04T08:29:24.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.606+0000 7f72a1600640 10 client.28453 handle_command_reply: tid=0 2024-07-04T08:29:24.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.606+0000 7f72c08b3740 1 client.28453 shutdown 2024-07-04T08:29:24.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 -- 172.21.15.39:0/3932512995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 msgr2=0x56243c2dd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:24.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2dd480 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f729c000c00 tx=0x7f729c003880 comp rx=0 tx=0).stop 2024-07-04T08:29:24.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 -- 172.21.15.39:0/3932512995 shutdown_connections 2024-07-04T08:29:24.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x56243bebc250 0x56243c355a90 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56243bf28520 0x56243c352210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56243bf07840 0x56243c2dd480 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 --2- 172.21.15.39:0/3932512995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56243beff660 0x56243c2dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 -- 172.21.15.39:0/3932512995 >> 172.21.15.39:0/3932512995 conn(0x56243bf1c910 msgr2=0x56243c2c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:24.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72a1600640 0 client.28453 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:24.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 -- 172.21.15.39:0/3932512995 shutdown_connections 2024-07-04T08:29:24.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 1 -- 172.21.15.39:0/3932512995 wait complete. 2024-07-04T08:29:24.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.607+0000 7f72c08b3740 20 client.28453 trim_cache size 0 max 16384 2024-07-04T08:29:24.596 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 40, 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "b0a3c823": { 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 40, 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:29:24.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 140.2, 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:24.598 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:29:24.599 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:29:24.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 -- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f72980b29e0 msgr2=0x7f72980b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:24.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f72980b29e0 0x7f72980b4ef0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x56243c2cc110 tx=0x7f72a400ef90 comp rx=0 tx=0).stop 2024-07-04T08:29:24.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 -- 172.21.15.39:0/1653405356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b4105110 msgr2=0x7f72b419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:24.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b4105110 0x7f72b419ba30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f72a400ea60 tx=0x7f72a400ef20 comp rx=0 tx=0).stop 2024-07-04T08:29:24.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 -- 172.21.15.39:0/1653405356 shutdown_connections 2024-07-04T08:29:24.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f72980b29e0 0x7f72980b4ef0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72b410b3a0 0x7f72b41a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b410aa50 0x7f72b419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 --2- 172.21.15.39:0/1653405356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72b4105110 0x7f72b419ba30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:24.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.608+0000 7f72b8c00640 1 -- 172.21.15.39:0/1653405356 >> 172.21.15.39:0/1653405356 conn(0x7f72b4100bc0 msgr2=0x7f72b4102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:24.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.609+0000 7f72b8c00640 1 -- 172.21.15.39:0/1653405356 shutdown_connections 2024-07-04T08:29:24.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:24.609+0000 7f72b8c00640 1 -- 172.21.15.39:0/1653405356 wait complete. 2024-07-04T08:29:25.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:25 smithi133 ceph-mon[37050]: pgmap v922: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-07-04T08:29:25.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:25 smithi039 ceph-mon[32062]: pgmap v922: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-07-04T08:29:25.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:25 smithi050 ceph-mon[36577]: pgmap v922: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-07-04T08:29:26.050 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b0a3c823', total seconds elapsed: 30.0 2024-07-04T08:29:26.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.686+0000 7f8bbd600640 1 -- 172.21.15.39:0/3797303071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bb810ee50 msgr2=0x7f8bb810f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.686+0000 7f8bbd600640 1 --2- 172.21.15.39:0/3797303071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bb810ee50 0x7f8bb810f260 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009860 tx=0x7f8ba002a340 comp rx=0 tx=0).stop 2024-07-04T08:29:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.686+0000 7f8bbd600640 1 -- 172.21.15.39:0/3797303071 shutdown_connections 2024-07-04T08:29:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.686+0000 7f8bbd600640 1 --2- 172.21.15.39:0/3797303071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb810f7a0 0x7f8bb8115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.686+0000 7f8bbd600640 1 --2- 172.21.15.39:0/3797303071 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bb810ee50 0x7f8bb810f260 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.686+0000 7f8bbd600640 1 --2- 172.21.15.39:0/3797303071 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bb8109510 0x7f8bb8109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.686+0000 7f8bbd600640 1 -- 172.21.15.39:0/3797303071 >> 172.21.15.39:0/3797303071 conn(0x7f8bb8078b10 msgr2=0x7f8bb8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.686+0000 7f8bbd600640 1 -- 172.21.15.39:0/3797303071 shutdown_connections 2024-07-04T08:29:26.675 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.686+0000 7f8bbd600640 1 -- 172.21.15.39:0/3797303071 wait complete. 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.687+0000 7f8bbd600640 1 Processor -- start 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.687+0000 7f8bbd600640 1 -- start start 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bbd600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bb8109510 0x7f8bb819fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bbd600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bb810ee50 0x7f8bb81a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bbd600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb810f7a0 0x7f8bb81a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bbd600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8119410 con 0x7f8bb8109510 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bbd600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8119290 con 0x7f8bb810ee50 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bbd600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8119590 con 0x7f8bb810f7a0 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bb7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bb8109510 0x7f8bb819fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bb7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bb8109510 0x7f8bb819fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48940/0 (socket says 172.21.15.39:48940) 2024-07-04T08:29:26.677 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bb7e00640 1 -- 172.21.15.39:0/4083466373 learned_addr learned my addr 172.21.15.39:0/4083466373 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:26.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb810f7a0 0x7f8bb81a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.688+0000 7f8baf400640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bb810ee50 0x7f8bb81a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.689+0000 7f8bb7e00640 1 -- 172.21.15.39:0/4083466373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb810f7a0 msgr2=0x7f8bb81a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:26.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.689+0000 7f8bb7e00640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb810f7a0 0x7f8bb81a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.689+0000 7f8bb7e00640 1 -- 172.21.15.39:0/4083466373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bb810ee50 msgr2=0x7f8bb81a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:26.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.689+0000 7f8bb7e00640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bb810ee50 0x7f8bb81a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.689+0000 7f8bb7e00640 1 -- 172.21.15.39:0/4083466373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb81a5e10 con 0x7f8bb8109510 2024-07-04T08:29:26.678 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.689+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb810f7a0 0x7f8bb81a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.689+0000 7f8bb7e00640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bb8109510 0x7f8bb819fea0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800e9c0 tx=0x7f8ba800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.689+0000 7f8bb5600640 1 -- 172.21.15.39:0/4083466373 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba800cde0 con 0x7f8bb8109510 2024-07-04T08:29:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.690+0000 7f8bbd600640 1 -- 172.21.15.39:0/4083466373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb81a6100 con 0x7f8bb8109510 2024-07-04T08:29:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.690+0000 7f8bb5600640 1 -- 172.21.15.39:0/4083466373 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba80035f0 con 0x7f8bb8109510 2024-07-04T08:29:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.690+0000 7f8bb5600640 1 -- 172.21.15.39:0/4083466373 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8010690 con 0x7f8bb8109510 2024-07-04T08:29:26.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.690+0000 7f8bbd600640 1 -- 172.21.15.39:0/4083466373 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb81ac860 con 0x7f8bb8109510 2024-07-04T08:29:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.693+0000 7f8bc5228740 1 Processor -- start 2024-07-04T08:29:26.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.694+0000 7f8bc5228740 1 -- start start 2024-07-04T08:29:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.694+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.682 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.694+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39af1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.694+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39af23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.694+0000 7f8bc5228740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ac39a67ee0 con 0x55ac3972c390 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.694+0000 7f8bc5228740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac39a57480 con 0x55ac397241b0 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.694+0000 7f8bc5228740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac39a5b4c0 con 0x55ac3974d070 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.694+0000 7f8bb5600640 1 -- 172.21.15.39:0/4083466373 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8ba8004000 con 0x7f8bb8109510 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.694+0000 7f8bbcc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bbcc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44288/0 (socket says 172.21.15.39:44288) 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8baf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39af23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bbcc00640 1 -- 172.21.15.39:0/4069259740 learned_addr learned my addr 172.21.15.39:0/4069259740 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:26.683 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bb7e00640 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39af1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bbcc00640 1 -- 172.21.15.39:0/4069259740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 msgr2=0x55ac39af1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39af1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bbcc00640 1 -- 172.21.15.39:0/4069259740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 msgr2=0x55ac39af23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39af23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bbcc00640 1 -- 172.21.15.39:0/4069259740 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac399ef650 con 0x55ac3974d070 2024-07-04T08:29:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8baf400640 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39af23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:29:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bb7e00640 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39af1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:29:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.695+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af1950 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0011790 tx=0x7f8bb0011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61f38b2b709ee825 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.696+0000 7f8bad600640 1 -- 172.21.15.39:0/4069259740 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0003c30 con 0x55ac3974d070 2024-07-04T08:29:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.696+0000 7f8bad600640 1 -- 172.21.15.39:0/4069259740 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb000fe50 con 0x55ac3974d070 2024-07-04T08:29:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.696+0000 7f8bad600640 1 -- 172.21.15.39:0/4069259740 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00193e0 con 0x55ac3974d070 2024-07-04T08:29:26.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.696+0000 7f8bb5600640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8b9c0b2960 0x7f8b9c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bb5600640 1 -- 172.21.15.39:0/4083466373 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f8ba80e2c90 con 0x7f8bb8109510 2024-07-04T08:29:26.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bb7e00640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8b9c0b2960 0x7f8b9c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bc5228740 1 -- 172.21.15.39:0/4069259740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 msgr2=0x55ac39af1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bc5228740 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af1950 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0011790 tx=0x7f8bb0011c50 comp rx=0 tx=0).stop 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bc5228740 1 -- 172.21.15.39:0/4069259740 shutdown_connections 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bc5228740 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39af23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bc5228740 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39af1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bc5228740 1 --2- 172.21.15.39:0/4069259740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af1950 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bc5228740 1 -- 172.21.15.39:0/4069259740 >> 172.21.15.39:0/4069259740 conn(0x55ac39741460 msgr2=0x55ac39aef840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bc5228740 1 -- 172.21.15.39:0/4069259740 shutdown_connections 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.697+0000 7f8bc5228740 1 -- 172.21.15.39:0/4069259740 wait complete. 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.698+0000 7f8bb7e00640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8b9c0b2960 0x7f8b9c0b4e70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55ac39af05d0 tx=0x7f8ba80046c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.698+0000 7f8bc5228740 1 -- start start 2024-07-04T08:29:26.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.698+0000 7f8bc5228740 1 Processor -- start 2024-07-04T08:29:26.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac397241b0 0x55ac39afd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3972c390 0x55ac39afd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 0x55ac39b09480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bc5228740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55ac398bb700 con 0x55ac3974d070 2024-07-04T08:29:26.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bc5228740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac399febb0 con 0x55ac3972c390 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bc5228740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac39b00130 con 0x55ac397241b0 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bc5228740 10 client.0 _ll_register_callbacks cb 0x55ac39a50570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bbcc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 0x55ac39b09480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bbcc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac397241b0 0x55ac39afd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8baf400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3972c390 0x55ac39afd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.699+0000 7f8bbcc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac397241b0 0x55ac39afd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44292/0 (socket says 172.21.15.39:44292) 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.700+0000 7f8bbcc00640 1 -- 172.21.15.39:0/3444343248 learned_addr learned my addr 172.21.15.39:0/3444343248 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.700+0000 7f8baf400640 1 -- 172.21.15.39:0/3444343248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac397241b0 msgr2=0x55ac39afd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.700+0000 7f8baf400640 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac397241b0 0x55ac39afd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.700+0000 7f8baf400640 1 -- 172.21.15.39:0/3444343248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 msgr2=0x55ac39b09480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.700+0000 7f8baf400640 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 0x55ac39b09480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.700+0000 7f8baf400640 1 -- 172.21.15.39:0/3444343248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac397b8650 con 0x55ac3972c390 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.700+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac397241b0 0x55ac39afd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.700+0000 7f8baf400640 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3972c390 0x55ac39afd9a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8ba002a860 tx=0x7f8ba0009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef971683923b38b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.700+0000 7f8ba6000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.701+0000 7f8ba6000640 1 -- 172.21.15.39:0/3444343248 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0042070 con 0x55ac3972c390 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.701+0000 7f8bc5228740 1 -- 172.21.15.39:0/3444343248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb000fe50 con 0x55ac3972c390 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.701+0000 7f8ba6000640 1 -- 172.21.15.39:0/3444343248 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f8ba0030450 con 0x55ac3972c390 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.701+0000 7f8ba6000640 1 -- 172.21.15.39:0/3444343248 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ba004b050 con 0x55ac3972c390 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.701+0000 7f8bc5228740 10 client.28529 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.701+0000 7f8bc5228740 1 -- 172.21.15.39:0/3444343248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac39b0c140 con 0x55ac3972c390 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.702+0000 7f8ba6000640 1 -- 172.21.15.39:0/3444343248 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb000fe50 con 0x55ac3972c390 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.702+0000 7f8ba6000640 1 -- 172.21.15.39:0/3444343248 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ba0045020 con 0x55ac3972c390 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.702+0000 7f8bc5228740 10 client.28529 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:26.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.702+0000 7f8bc5228740 10 client.28529 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:29:26.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.702+0000 7f8bc5228740 20 client.28529 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:26.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.702+0000 7f8bc5228740 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55ac3970b9f0 0x55ac39b0f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.702+0000 7f8bc5228740 4 client.28529 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:26.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.702+0000 7f8bc5228740 1 -- 172.21.15.39:0/3444343248 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac39a69e30 con 0x55ac3970b9f0 2024-07-04T08:29:26.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.703+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55ac3970b9f0 0x55ac39b0f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.703+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55ac3970b9f0 0x55ac39b0f650 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10501705b2efd12 server_cookie=1664e42c99ed12fc in_seq=0 out_seq=0 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.703+0000 7f8ba6000640 10 client.28529 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.704+0000 7f8ba6000640 1 -- 172.21.15.39:0/3444343248 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ac39a69e30 con 0x55ac3970b9f0 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.704+0000 7f8ba6000640 10 client.28529 handle_command_reply: tid=0 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.704+0000 7f8bc5228740 1 client.28529 shutdown 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 -- 172.21.15.39:0/3444343248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3972c390 msgr2=0x55ac39afd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3972c390 0x55ac39afd9a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8ba002a860 tx=0x7f8ba0009510 comp rx=0 tx=0).stop 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 -- 172.21.15.39:0/3444343248 shutdown_connections 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55ac3970b9f0 0x55ac39b0f650 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 0x55ac39b09480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3972c390 0x55ac39afd9a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 --2- 172.21.15.39:0/3444343248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac397241b0 0x55ac39afd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8ba6000640 0 client.28529 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:29:26.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 -- 172.21.15.39:0/3444343248 >> 172.21.15.39:0/3444343248 conn(0x55ac39741460 msgr2=0x55ac39af4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:26.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 -- 172.21.15.39:0/3444343248 shutdown_connections 2024-07-04T08:29:26.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 1 -- 172.21.15.39:0/3444343248 wait complete. 2024-07-04T08:29:26.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.705+0000 7f8bc5228740 20 client.28529 trim_cache size 0 max 16384 2024-07-04T08:29:26.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.720+0000 7f8bc5228740 1 Processor -- start 2024-07-04T08:29:26.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8bc5228740 1 -- start start 2024-07-04T08:29:26.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39aff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39aede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8bc5228740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81b6880 con 0x55ac3972c390 2024-07-04T08:29:26.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8bc5228740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81b83c0 con 0x55ac397241b0 2024-07-04T08:29:26.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8bc5228740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81b6f50 con 0x55ac3974d070 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8baf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39aede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8baf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39aede10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48960/0 (socket says 172.21.15.39:48960) 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8bbcc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39aff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.721+0000 7f8baf400640 1 -- 172.21.15.39:0/2310216602 learned_addr learned my addr 172.21.15.39:0/2310216602 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.722+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/2310216602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.722+0000 7f8bbcc00640 1 -- 172.21.15.39:0/2310216602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 msgr2=0x55ac39af3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.722+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/2310216602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.722+0000 7f8bbcc00640 1 -- 172.21.15.39:0/2310216602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 msgr2=0x55ac39aede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.722+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/2310216602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39aede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.722+0000 7f8bbcc00640 1 -- 172.21.15.39:0/2310216602 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac39af0600 con 0x55ac397241b0 2024-07-04T08:29:26.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.722+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/2310216602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39aff140 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55ac39b0d610 tx=0x7f8bb0009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=190dd4c951e2fa19 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:26.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.722+0000 7f8bacc00640 1 -- 172.21.15.39:0/2310216602 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000fe80 con 0x55ac397241b0 2024-07-04T08:29:26.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.723+0000 7f8bacc00640 1 -- 172.21.15.39:0/2310216602 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0003a30 con 0x55ac397241b0 2024-07-04T08:29:26.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.723+0000 7f8bacc00640 1 -- 172.21.15.39:0/2310216602 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0004c60 con 0x55ac397241b0 2024-07-04T08:29:26.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.723+0000 7f8bc5228740 1 -- 172.21.15.39:0/2310216602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 msgr2=0x55ac39aff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:26.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.723+0000 7f8bc5228740 1 --2- 172.21.15.39:0/2310216602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39aff140 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55ac39b0d610 tx=0x7f8bb0009500 comp rx=0 tx=0).stop 2024-07-04T08:29:26.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.723+0000 7f8bc5228740 1 -- 172.21.15.39:0/2310216602 shutdown_connections 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.723+0000 7f8bc5228740 1 --2- 172.21.15.39:0/2310216602 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3972c390 0x55ac39aede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.723+0000 7f8bc5228740 1 --2- 172.21.15.39:0/2310216602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3974d070 0x55ac39af3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.723+0000 7f8bc5228740 1 --2- 172.21.15.39:0/2310216602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39aff140 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.724+0000 7f8bc5228740 1 -- 172.21.15.39:0/2310216602 >> 172.21.15.39:0/2310216602 conn(0x55ac39741460 msgr2=0x55ac39af2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.724+0000 7f8bc5228740 1 -- 172.21.15.39:0/2310216602 shutdown_connections 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.724+0000 7f8bc5228740 1 -- 172.21.15.39:0/2310216602 wait complete. 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.724+0000 7f8bc5228740 1 -- start start 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.725+0000 7f8bc5228740 1 Processor -- start 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.725+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39b01460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.714 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.725+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3972c390 0x55ac39b019a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.725+0000 7f8bc5228740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 0x55ac39b01ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.725+0000 7f8bc5228740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81b7cf0 con 0x55ac3974d070 2024-07-04T08:29:26.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.725+0000 7f8bc5228740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81b7620 con 0x55ac397241b0 2024-07-04T08:29:26.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.725+0000 7f8bc5228740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81b4860 con 0x55ac3972c390 2024-07-04T08:29:26.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.725+0000 7f8bc5228740 10 client.0 _ll_register_callbacks cb 0x7f8ba8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:26.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8bbcc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39b01460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8baf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3972c390 0x55ac39b019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8bbcc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39b01460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56118/0 (socket says 172.21.15.39:56118) 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8baf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3972c390 0x55ac39b019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44302/0 (socket says 172.21.15.39:44302) 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8bbcc00640 1 -- 172.21.15.39:0/1730301410 learned_addr learned my addr 172.21.15.39:0/1730301410 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 0x55ac39b01ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8baf400640 1 -- 172.21.15.39:0/1730301410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 msgr2=0x55ac39b01460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8baf400640 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39b01460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8baf400640 1 -- 172.21.15.39:0/1730301410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 msgr2=0x55ac39b01ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8baf400640 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 0x55ac39b01ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8baf400640 1 -- 172.21.15.39:0/1730301410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac39b76f40 con 0x55ac3972c390 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8baf400640 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3972c390 0x55ac39b019a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0030ef0 tx=0x7f8ba00045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e97b769aab6519ca server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:26.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.726+0000 7f8ba6000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.727+0000 7f8ba6000640 1 -- 172.21.15.39:0/1730301410 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0042040 con 0x55ac3972c390 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.727+0000 7f8ba6000640 1 -- 172.21.15.39:0/1730301410 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f8ba0030860 con 0x55ac3972c390 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.727+0000 7f8bc5228740 1 -- 172.21.15.39:0/1730301410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb0003a30 con 0x55ac3972c390 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.727+0000 7f8ba6000640 1 -- 172.21.15.39:0/1730301410 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ba0058050 con 0x55ac3972c390 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.727+0000 7f8bc5228740 10 client.28471 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.727+0000 7f8bc5228740 1 -- 172.21.15.39:0/1730301410 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac39b781c0 con 0x55ac3972c390 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.727+0000 7f8ba6000640 1 -- 172.21.15.39:0/1730301410 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0003a30 con 0x55ac3972c390 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.728+0000 7f8ba6000640 1 -- 172.21.15.39:0/1730301410 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ba0045040 con 0x55ac3972c390 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.728+0000 7f8bc5228740 10 client.28471 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.728+0000 7f8bc5228740 10 client.28471 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:29:26.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.728+0000 7f8bc5228740 20 client.28471 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:26.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.728+0000 7f8bc5228740 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55ac396e0da0 0x55ac39b7a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:26.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.728+0000 7f8bc5228740 4 client.28471 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:29:26.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.728+0000 7f8bc5228740 1 -- 172.21.15.39:0/1730301410 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac39a658d0 con 0x55ac396e0da0 2024-07-04T08:29:26.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.728+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55ac396e0da0 0x55ac39b7a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:26.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bbcc00640 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55ac396e0da0 0x55ac39b7a9b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=32ba3e5bf7bfd1ed server_cookie=11ea73c22702101b in_seq=0 out_seq=0 2024-07-04T08:29:26.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8ba6000640 10 client.28471 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:29:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8ba6000640 1 -- 172.21.15.39:0/1730301410 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac39a658d0 con 0x55ac396e0da0 2024-07-04T08:29:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8ba6000640 10 client.28471 handle_command_reply: tid=0 2024-07-04T08:29:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bc5228740 1 client.28471 shutdown 2024-07-04T08:29:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bc5228740 1 -- 172.21.15.39:0/1730301410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3972c390 msgr2=0x55ac39b019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bc5228740 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3972c390 0x55ac39b019a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0030ef0 tx=0x7f8ba00045c0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bc5228740 1 -- 172.21.15.39:0/1730301410 shutdown_connections 2024-07-04T08:29:26.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bc5228740 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x55ac396e0da0 0x55ac39b7a9b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bc5228740 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55ac3974d070 0x55ac39b01ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bc5228740 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac3972c390 0x55ac39b019a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bc5228740 1 --2- 172.21.15.39:0/1730301410 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac397241b0 0x55ac39b01460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8bc5228740 1 -- 172.21.15.39:0/1730301410 >> 172.21.15.39:0/1730301410 conn(0x55ac39741460 msgr2=0x55ac39afefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.729+0000 7f8ba6000640 0 client.28471 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:29:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.730+0000 7f8bc5228740 1 -- 172.21.15.39:0/1730301410 shutdown_connections 2024-07-04T08:29:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.730+0000 7f8bc5228740 1 -- 172.21.15.39:0/1730301410 wait complete. 2024-07-04T08:29:26.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.730+0000 7f8bc5228740 20 client.28471 trim_cache size 0 max 16384 2024-07-04T08:29:26.721 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:29:26.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.730+0000 7f8bbd600640 1 -- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8b9c0b2960 msgr2=0x7f8b9c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:26.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.730+0000 7f8bbd600640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8b9c0b2960 0x7f8b9c0b4e70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55ac39af05d0 tx=0x7f8ba80046c0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.730+0000 7f8bbd600640 1 -- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bb8109510 msgr2=0x7f8bb819fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:26.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.730+0000 7f8bbd600640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bb8109510 0x7f8bb819fea0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800e9c0 tx=0x7f8ba800ee80 comp rx=0 tx=0).stop 2024-07-04T08:29:26.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.731+0000 7f8bbd600640 1 -- 172.21.15.39:0/4083466373 shutdown_connections 2024-07-04T08:29:26.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.731+0000 7f8bbd600640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8b9c0b2960 0x7f8b9c0b4e70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.731+0000 7f8bbd600640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb810f7a0 0x7f8bb81a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.731+0000 7f8bbd600640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8bb810ee50 0x7f8bb81a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.731+0000 7f8bbd600640 1 --2- 172.21.15.39:0/4083466373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bb8109510 0x7f8bb819fea0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:26.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.731+0000 7f8bbd600640 1 -- 172.21.15.39:0/4083466373 >> 172.21.15.39:0/4083466373 conn(0x7f8bb8078b10 msgr2=0x7f8bb8107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:26.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.731+0000 7f8bbd600640 1 -- 172.21.15.39:0/4083466373 shutdown_connections 2024-07-04T08:29:26.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:26.731+0000 7f8bbd600640 1 -- 172.21.15.39:0/4083466373 wait complete. 2024-07-04T08:29:27.491 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:27 smithi039 ceph-mon[32062]: pgmap v923: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:29:27.552 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:29:27.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:27 smithi133 ceph-mon[37050]: pgmap v923: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:29:27.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:27 smithi050 ceph-mon[36577]: pgmap v923: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-07-04T08:29:28.036 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:29.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:29 smithi039 ceph-mon[32062]: pgmap v924: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-07-04T08:29:29.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:29 smithi050 ceph-mon[36577]: pgmap v924: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-07-04T08:29:29.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:29 smithi133 ceph-mon[37050]: pgmap v924: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 10 op/s 2024-07-04T08:29:30.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.280+0000 7f108ae00640 1 -- 172.21.15.39:0/3728710780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1084109340 msgr2=0x7f108410fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:30.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.280+0000 7f108ae00640 1 --2- 172.21.15.39:0/3728710780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1084109340 0x7f108410fb70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f107c009ca0 tx=0x7f107c02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.268 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.280+0000 7f108ae00640 1 -- 172.21.15.39:0/3728710780 shutdown_connections 2024-07-04T08:29:30.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.280+0000 7f108ae00640 1 --2- 172.21.15.39:0/3728710780 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1084109340 0x7f108410fb70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.280+0000 7f108ae00640 1 --2- 172.21.15.39:0/3728710780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10841089f0 0x7f1084108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.280+0000 7f108ae00640 1 --2- 172.21.15.39:0/3728710780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10841030b0 0x7f10841034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.280+0000 7f108ae00640 1 -- 172.21.15.39:0/3728710780 >> 172.21.15.39:0/3728710780 conn(0x7f108407ada0 msgr2=0x7f108407b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:30.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.280+0000 7f108ae00640 1 -- 172.21.15.39:0/3728710780 shutdown_connections 2024-07-04T08:29:30.269 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.280+0000 7f108ae00640 1 -- 172.21.15.39:0/3728710780 wait complete. 2024-07-04T08:29:30.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.282+0000 7f108ae00640 1 Processor -- start 2024-07-04T08:29:30.270 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.282+0000 7f108ae00640 1 -- start start 2024-07-04T08:29:30.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.282+0000 7f108ae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10841030b0 0x7f1084074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f108ae00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10841089f0 0x7f10840751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f108ae00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1084109340 0x7f10840756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f108ae00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1084112ff0 con 0x7f10841089f0 2024-07-04T08:29:30.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f108ae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1084112e70 con 0x7f10841030b0 2024-07-04T08:29:30.271 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f108ae00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1084113170 con 0x7f1084109340 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f1083e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10841030b0 0x7f1084074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f107be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10841089f0 0x7f10840751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f1083e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10841030b0 0x7f1084074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56138/0 (socket says 172.21.15.39:56138) 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f1083e00640 1 -- 172.21.15.39:0/4257032245 learned_addr learned my addr 172.21.15.39:0/4257032245 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.283+0000 7f1088e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1084109340 0x7f10840756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.284+0000 7f107be00640 1 -- 172.21.15.39:0/4257032245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1084109340 msgr2=0x7f10840756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.284+0000 7f107be00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1084109340 0x7f10840756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.284+0000 7f107be00640 1 -- 172.21.15.39:0/4257032245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10841030b0 msgr2=0x7f1084074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.284+0000 7f107be00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10841030b0 0x7f1084074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.284+0000 7f107be00640 1 -- 172.21.15.39:0/4257032245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10841a7c60 con 0x7f10841089f0 2024-07-04T08:29:30.273 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.284+0000 7f1083e00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10841030b0 0x7f1084074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:30.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.284+0000 7f107be00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10841089f0 0x7f10840751a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f108406bf40 tx=0x7f106c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:30.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.284+0000 7f1081600640 1 -- 172.21.15.39:0/4257032245 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c00cc40 con 0x7f10841089f0 2024-07-04T08:29:30.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.284+0000 7f1081600640 1 -- 172.21.15.39:0/4257032245 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f106c00cde0 con 0x7f10841089f0 2024-07-04T08:29:30.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.285+0000 7f1081600640 1 -- 172.21.15.39:0/4257032245 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c018780 con 0x7f10841089f0 2024-07-04T08:29:30.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.285+0000 7f108ae00640 1 -- 172.21.15.39:0/4257032245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10841a7f50 con 0x7f10841089f0 2024-07-04T08:29:30.274 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.285+0000 7f108ae00640 1 -- 172.21.15.39:0/4257032245 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10841a8350 con 0x7f10841089f0 2024-07-04T08:29:30.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.288+0000 7f1091349740 1 Processor -- start 2024-07-04T08:29:30.276 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.288+0000 7f1091349740 1 -- start start 2024-07-04T08:29:30.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1081600640 1 -- 172.21.15.39:0/4257032245 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f106c010a90 con 0x7f10841089f0 2024-07-04T08:29:30.277 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1091349740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 0x5595cdabc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1091349740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdabcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1091349740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdabd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1091349740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5595cda32f00 con 0x5595cd6ef1b0 2024-07-04T08:29:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1091349740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5595cda224a0 con 0x5595cd6f7390 2024-07-04T08:29:30.278 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1091349740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5595cda264e0 con 0x5595cd718070 2024-07-04T08:29:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1083e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdabcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1083e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdabcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49006/0 (socket says 172.21.15.39:49006) 2024-07-04T08:29:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1083e00640 1 -- 172.21.15.39:0/2139491468 learned_addr learned my addr 172.21.15.39:0/2139491468 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.289+0000 7f1088e00640 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 0x5595cdabc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.290+0000 7f107be00640 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdabd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.290+0000 7f1083e00640 1 -- 172.21.15.39:0/2139491468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 msgr2=0x5595cdabc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:30.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.290+0000 7f1083e00640 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 0x5595cdabc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.290+0000 7f1083e00640 1 -- 172.21.15.39:0/2139491468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 msgr2=0x5595cdabd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.290+0000 7f1083e00640 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdabd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.290+0000 7f1083e00640 1 -- 172.21.15.39:0/2139491468 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cd9ba670 con 0x5595cd6ef1b0 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.290+0000 7f107be00640 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdabd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.290+0000 7f1088e00640 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 0x5595cdabc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.290+0000 7f1083e00640 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdabcd10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f10740115f0 tx=0x7f1074011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157c5bec58fcef8e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.291+0000 7f1079600640 1 -- 172.21.15.39:0/2139491468 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107401e070 con 0x5595cd6ef1b0 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.291+0000 7f1079600640 1 -- 172.21.15.39:0/2139491468 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f107400f660 con 0x5595cd6ef1b0 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.291+0000 7f1079600640 1 -- 172.21.15.39:0/2139491468 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074003830 con 0x5595cd6ef1b0 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.291+0000 7f1081600640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10680b2960 0x7f10680b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.291+0000 7f1081600640 1 -- 172.21.15.39:0/4257032245 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f106c0e2c30 con 0x7f10841089f0 2024-07-04T08:29:30.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.291+0000 7f1088e00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10680b2960 0x7f10680b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.292+0000 7f1091349740 1 -- 172.21.15.39:0/2139491468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 msgr2=0x5595cdabcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.292+0000 7f1091349740 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdabcd10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f10740115f0 tx=0x7f1074011ab0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.292+0000 7f1091349740 1 -- 172.21.15.39:0/2139491468 shutdown_connections 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.292+0000 7f1091349740 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdabcd10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.292+0000 7f1091349740 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdabd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.292+0000 7f1091349740 1 --2- 172.21.15.39:0/2139491468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 0x5595cdabc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.292+0000 7f1091349740 1 -- 172.21.15.39:0/2139491468 >> 172.21.15.39:0/2139491468 conn(0x5595cd70c460 msgr2=0x5595cdaba6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.292+0000 7f1091349740 1 -- 172.21.15.39:0/2139491468 shutdown_connections 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.292+0000 7f1091349740 1 -- 172.21.15.39:0/2139491468 wait complete. 2024-07-04T08:29:30.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.293+0000 7f1088e00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10680b2960 0x7f10680b4e70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f107c02f500 tx=0x7f107c02f490 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:30.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.293+0000 7f1091349740 1 -- start start 2024-07-04T08:29:30.287 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.293+0000 7f1091349740 1 Processor -- start 2024-07-04T08:29:30.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1091349740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdac8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1091349740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdac8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1091349740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 0x5595cdad42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1091349740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5595cd886700 con 0x5595cd718070 2024-07-04T08:29:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1091349740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5595cd9c9bd0 con 0x5595cd6ef1b0 2024-07-04T08:29:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1091349740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5595cdacafb0 con 0x5595cd6f7390 2024-07-04T08:29:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1091349740 10 client.0 _ll_register_callbacks cb 0x5595cda1b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1083e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdac8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1083e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdac8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56158/0 (socket says 172.21.15.39:56158) 2024-07-04T08:29:30.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1083e00640 1 -- 172.21.15.39:0/590482286 learned_addr learned my addr 172.21.15.39:0/590482286 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f1088e00640 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 0x5595cdad42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.294+0000 7f107be00640 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdac8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.295+0000 7f1083e00640 1 -- 172.21.15.39:0/590482286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 msgr2=0x5595cdac8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.295+0000 7f1083e00640 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdac8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.295+0000 7f1083e00640 1 -- 172.21.15.39:0/590482286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 msgr2=0x5595cdad42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.295+0000 7f1083e00640 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 0x5595cdad42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.295+0000 7f1083e00640 1 -- 172.21.15.39:0/590482286 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cd783650 con 0x5595cd6ef1b0 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.295+0000 7f1083e00640 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdac8350 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f107400ffc0 tx=0x7f1074005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b1fdcb4e559b01 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.295+0000 7f1072000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:29:30.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.295+0000 7f1072000640 1 -- 172.21.15.39:0/590482286 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107401e070 con 0x5595cd6ef1b0 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.296+0000 7f1091349740 1 -- 172.21.15.39:0/590482286 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f107400f660 con 0x5595cd6ef1b0 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.296+0000 7f1072000640 1 -- 172.21.15.39:0/590482286 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f10740046c0 con 0x5595cd6ef1b0 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.296+0000 7f1072000640 1 -- 172.21.15.39:0/590482286 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1074031050 con 0x5595cd6ef1b0 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.296+0000 7f1091349740 10 client.28544 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.296+0000 7f1091349740 1 -- 172.21.15.39:0/590482286 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595cdad6f00 con 0x5595cd6ef1b0 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.296+0000 7f1072000640 1 -- 172.21.15.39:0/590482286 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f107400f660 con 0x5595cd6ef1b0 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.296+0000 7f1072000640 1 -- 172.21.15.39:0/590482286 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1074019590 con 0x5595cd6ef1b0 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.297+0000 7f1091349740 10 client.28544 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.297+0000 7f1091349740 10 client.28544 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.297+0000 7f1091349740 20 client.28544 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:30.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.297+0000 7f1091349740 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5595cd6d69f0 0x5595cdada2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.297+0000 7f1091349740 4 client.28544 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.297+0000 7f1091349740 1 -- 172.21.15.39:0/590482286 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595cda34e50 con 0x5595cd6d69f0 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.297+0000 7f107be00640 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5595cd6d69f0 0x5595cdada2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.297+0000 7f107be00640 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5595cd6d69f0 0x5595cdada2c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4aedb90e4e6afb80 server_cookie=21b704a26649734 in_seq=0 out_seq=0 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.297+0000 7f1072000640 10 client.28544 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.298+0000 7f1072000640 1 -- 172.21.15.39:0/590482286 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5595cda34e50 con 0x5595cd6d69f0 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.298+0000 7f1072000640 10 client.28544 handle_command_reply: tid=0 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.298+0000 7f1091349740 1 client.28544 shutdown 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 -- 172.21.15.39:0/590482286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 msgr2=0x5595cdac8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdac8350 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f107400ffc0 tx=0x7f1074005270 comp rx=0 tx=0).stop 2024-07-04T08:29:30.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 -- 172.21.15.39:0/590482286 shutdown_connections 2024-07-04T08:29:30.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5595cd6d69f0 0x5595cdada2c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 0x5595cdad42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdac8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 --2- 172.21.15.39:0/590482286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdac8350 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1072000640 0 client.28544 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:29:30.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 -- 172.21.15.39:0/590482286 >> 172.21.15.39:0/590482286 conn(0x5595cd70c460 msgr2=0x5595cdabf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:30.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 -- 172.21.15.39:0/590482286 shutdown_connections 2024-07-04T08:29:30.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 1 -- 172.21.15.39:0/590482286 wait complete. 2024-07-04T08:29:30.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.299+0000 7f1091349740 20 client.28544 trim_cache size 0 max 16384 2024-07-04T08:29:30.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.314+0000 7f1091349740 1 Processor -- start 2024-07-04T08:29:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.315+0000 7f1091349740 1 -- start start 2024-07-04T08:29:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.315+0000 7f1091349740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdaca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.315+0000 7f1091349740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdabdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.315+0000 7f1091349740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 0x5595cdac1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.315+0000 7f1091349740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f10841b2610 con 0x5595cd718070 2024-07-04T08:29:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.315+0000 7f1091349740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f10841b4150 con 0x5595cd6ef1b0 2024-07-04T08:29:30.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.315+0000 7f1091349740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f10841b2ce0 con 0x5595cd6f7390 2024-07-04T08:29:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f107be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdabdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f1083e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdaca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f107be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdabdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44350/0 (socket says 172.21.15.39:44350) 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f1083e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdaca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56168/0 (socket says 172.21.15.39:56168) 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f107be00640 1 -- 172.21.15.39:0/1351647535 learned_addr learned my addr 172.21.15.39:0/1351647535 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f1088e00640 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 0x5595cdac1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f107be00640 1 -- 172.21.15.39:0/1351647535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 msgr2=0x5595cdaca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f107be00640 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdaca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f107be00640 1 -- 172.21.15.39:0/1351647535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 msgr2=0x5595cdac1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f107be00640 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 0x5595cdac1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f107be00640 1 -- 172.21.15.39:0/1351647535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cdabf060 con 0x5595cd6f7390 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f1088e00640 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 0x5595cdac1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.316+0000 7f1083e00640 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdaca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:30.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.317+0000 7f107be00640 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdabdf10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f106c0e0020 tx=0x7f106c0cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6018382f2bc1f566 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.317+0000 7f1078c00640 1 -- 172.21.15.39:0/1351647535 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c08c070 con 0x5595cd6f7390 2024-07-04T08:29:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.317+0000 7f1078c00640 1 -- 172.21.15.39:0/1351647535 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f106c0b29b0 con 0x5595cd6f7390 2024-07-04T08:29:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.317+0000 7f1078c00640 1 -- 172.21.15.39:0/1351647535 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0a5570 con 0x5595cd6f7390 2024-07-04T08:29:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.318+0000 7f1091349740 1 -- 172.21.15.39:0/1351647535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 msgr2=0x5595cdabdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.318+0000 7f1091349740 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdabdf10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f106c0e0020 tx=0x7f106c0cc040 comp rx=0 tx=0).stop 2024-07-04T08:29:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.318+0000 7f1091349740 1 -- 172.21.15.39:0/1351647535 shutdown_connections 2024-07-04T08:29:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.318+0000 7f1091349740 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd718070 0x5595cdac1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.318+0000 7f1091349740 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd6f7390 0x5595cdabdf10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.318+0000 7f1091349740 1 --2- 172.21.15.39:0/1351647535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6ef1b0 0x5595cdaca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.318+0000 7f1091349740 1 -- 172.21.15.39:0/1351647535 >> 172.21.15.39:0/1351647535 conn(0x5595cd70c460 msgr2=0x5595cdabd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.318+0000 7f1091349740 1 -- 172.21.15.39:0/1351647535 shutdown_connections 2024-07-04T08:29:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.318+0000 7f1091349740 1 -- 172.21.15.39:0/1351647535 wait complete. 2024-07-04T08:29:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.319+0000 7f1091349740 1 -- start start 2024-07-04T08:29:30.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.319+0000 7f1091349740 1 Processor -- start 2024-07-04T08:29:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1091349740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdad0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1091349740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdad3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1091349740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 0x5595cdad4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1091349740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f10841b3a80 con 0x5595cd6ef1b0 2024-07-04T08:29:30.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1091349740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f10841b33b0 con 0x5595cd6f7390 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1091349740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f10841b05f0 con 0x5595cd718070 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1091349740 10 client.0 _ll_register_callbacks cb 0x7f107401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1083e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdad0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f107be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdad3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1083e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdad0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49028/0 (socket says 172.21.15.39:49028) 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1083e00640 1 -- 172.21.15.39:0/1287036215 learned_addr learned my addr 172.21.15.39:0/1287036215 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f1088e00640 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 0x5595cdad4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.320+0000 7f107be00640 1 -- 172.21.15.39:0/1287036215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 msgr2=0x5595cdad4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.321+0000 7f107be00640 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 0x5595cdad4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.321+0000 7f107be00640 1 -- 172.21.15.39:0/1287036215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 msgr2=0x5595cdad0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:30.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.321+0000 7f107be00640 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdad0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.321+0000 7f107be00640 1 -- 172.21.15.39:0/1287036215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cdad4920 con 0x5595cd6f7390 2024-07-04T08:29:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.321+0000 7f107be00640 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdad3c30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f106c0e0020 tx=0x7f106c010e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18853732c1c1688c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.321+0000 7f1072000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:29:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.323+0000 7f1072000640 1 -- 172.21.15.39:0/1287036215 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c08c070 con 0x5595cd6f7390 2024-07-04T08:29:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.323+0000 7f1072000640 1 -- 172.21.15.39:0/1287036215 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f106c0a61f0 con 0x5595cd6f7390 2024-07-04T08:29:30.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.323+0000 7f1091349740 1 -- 172.21.15.39:0/1287036215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f106c0b29b0 con 0x5595cd6f7390 2024-07-04T08:29:30.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1072000640 1 -- 172.21.15.39:0/1287036215 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f106c0e0450 con 0x5595cd6f7390 2024-07-04T08:29:30.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1091349740 10 client.28550 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:30.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1091349740 1 -- 172.21.15.39:0/1287036215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595cdb428f0 con 0x5595cd6f7390 2024-07-04T08:29:30.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1072000640 1 -- 172.21.15.39:0/1287036215 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f106c0b29b0 con 0x5595cd6f7390 2024-07-04T08:29:30.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1072000640 1 -- 172.21.15.39:0/1287036215 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f106c0a8a60 con 0x5595cd6f7390 2024-07-04T08:29:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1091349740 10 client.28550 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1091349740 10 client.28550 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:29:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1091349740 20 client.28550 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1091349740 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5595cd6abda0 0x5595cdb44ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1091349740 4 client.28550 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:29:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.327+0000 7f1091349740 1 -- 172.21.15.39:0/1287036215 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595cda308f0 con 0x5595cd6abda0 2024-07-04T08:29:30.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.328+0000 7f1083e00640 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5595cd6abda0 0x5595cdb44ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.328+0000 7f1072000640 10 client.28550 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:29:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.328+0000 7f1083e00640 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5595cd6abda0 0x5595cdb44ff0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4545999a23b9822b server_cookie=f8962c7b342fdf53 in_seq=0 out_seq=0 2024-07-04T08:29:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1072000640 1 -- 172.21.15.39:0/1287036215 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595cda308f0 con 0x5595cd6abda0 2024-07-04T08:29:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1072000640 10 client.28550 handle_command_reply: tid=0 2024-07-04T08:29:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1091349740 1 client.28550 shutdown 2024-07-04T08:29:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1091349740 1 -- 172.21.15.39:0/1287036215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 msgr2=0x5595cdad3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:30.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1091349740 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdad3c30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f106c0e0020 tx=0x7f106c010e30 comp rx=0 tx=0).stop 2024-07-04T08:29:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1091349740 1 -- 172.21.15.39:0/1287036215 shutdown_connections 2024-07-04T08:29:30.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1091349740 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5595cd6abda0 0x5595cdb44ff0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1091349740 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595cd718070 0x5595cdad4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1091349740 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5595cd6f7390 0x5595cdad3c30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1091349740 1 --2- 172.21.15.39:0/1287036215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5595cd6ef1b0 0x5595cdad0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1091349740 1 -- 172.21.15.39:0/1287036215 >> 172.21.15.39:0/1287036215 conn(0x5595cd70c460 msgr2=0x5595cdac9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.329+0000 7f1072000640 0 client.28550 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:29:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.330+0000 7f1091349740 1 -- 172.21.15.39:0/1287036215 shutdown_connections 2024-07-04T08:29:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.330+0000 7f1091349740 1 -- 172.21.15.39:0/1287036215 wait complete. 2024-07-04T08:29:30.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.330+0000 7f1091349740 20 client.28550 trim_cache size 0 max 16384 2024-07-04T08:29:30.320 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:29:30.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 -- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10680b2960 msgr2=0x7f10680b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:30.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10680b2960 0x7f10680b4e70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f107c02f500 tx=0x7f107c02f490 comp rx=0 tx=0).stop 2024-07-04T08:29:30.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 -- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10841089f0 msgr2=0x7f10840751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:30.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10841089f0 0x7f10840751a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f108406bf40 tx=0x7f106c00ee50 comp rx=0 tx=0).stop 2024-07-04T08:29:30.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 -- 172.21.15.39:0/4257032245 shutdown_connections 2024-07-04T08:29:30.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f10680b2960 0x7f10680b4e70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10841089f0 0x7f10840751a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1084109340 0x7f10840756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 --2- 172.21.15.39:0/4257032245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f10841030b0 0x7f1084074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:30.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 -- 172.21.15.39:0/4257032245 >> 172.21.15.39:0/4257032245 conn(0x7f108407ada0 msgr2=0x7f1084106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:30.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 -- 172.21.15.39:0/4257032245 shutdown_connections 2024-07-04T08:29:30.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:30.331+0000 7f108ae00640 1 -- 172.21.15.39:0/4257032245 wait complete. 2024-07-04T08:29:30.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:30 smithi039 ceph-mon[32062]: pgmap v925: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-07-04T08:29:30.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:30 smithi050 ceph-mon[36577]: pgmap v925: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-07-04T08:29:30.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:30 smithi133 ceph-mon[37050]: pgmap v925: 129 pgs: 129 active+clean; 25 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-07-04T08:29:31.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:29:31.647 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:33.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:33 smithi039 ceph-mon[32062]: pgmap v926: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-07-04T08:29:33.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:33 smithi133 ceph-mon[37050]: pgmap v926: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-07-04T08:29:33.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:33 smithi050 ceph-mon[36577]: pgmap v926: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-07-04T08:29:33.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.996+0000 7f8212600640 1 -- 172.21.15.39:0/1020012681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820c109510 msgr2=0x7f820c109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:33.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.996+0000 7f8212600640 1 --2- 172.21.15.39:0/1020012681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820c109510 0x7f820c109920 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009900 tx=0x7f81fc02a410 comp rx=0 tx=0).stop 2024-07-04T08:29:33.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.996+0000 7f8212600640 1 -- 172.21.15.39:0/1020012681 shutdown_connections 2024-07-04T08:29:33.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.996+0000 7f8212600640 1 --2- 172.21.15.39:0/1020012681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f820c10f7a0 0x7f820c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.996+0000 7f8212600640 1 --2- 172.21.15.39:0/1020012681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f820c10ee50 0x7f820c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.996+0000 7f8212600640 1 --2- 172.21.15.39:0/1020012681 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820c109510 0x7f820c109920 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.996+0000 7f8212600640 1 -- 172.21.15.39:0/1020012681 >> 172.21.15.39:0/1020012681 conn(0x7f820c078b10 msgr2=0x7f820c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:33.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.996+0000 7f8212600640 1 -- 172.21.15.39:0/1020012681 shutdown_connections 2024-07-04T08:29:33.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.996+0000 7f8212600640 1 -- 172.21.15.39:0/1020012681 wait complete. 2024-07-04T08:29:33.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.997+0000 7f8212600640 1 Processor -- start 2024-07-04T08:29:33.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8212600640 1 -- start start 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8212600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820c109510 0x7f820c19fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8212600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f820c10ee50 0x7f820c1a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8212600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f820c10f7a0 0x7f820c1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8212600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f820c119450 con 0x7f820c109510 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8212600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1192d0 con 0x7f820c10ee50 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8212600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1195d0 con 0x7f820c10f7a0 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8203e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f820c10ee50 0x7f820c1a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8203e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f820c10ee50 0x7f820c1a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:42250/0 (socket says 172.21.15.39:42250) 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.998+0000 7f8203e00640 1 -- 172.21.15.39:0/2439422927 learned_addr learned my addr 172.21.15.39:0/2439422927 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:33.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.999+0000 7f8211c00640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f820c10f7a0 0x7f820c1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:33.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.999+0000 7f8203e00640 1 -- 172.21.15.39:0/2439422927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f820c10f7a0 msgr2=0x7f820c1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.999+0000 7f8203e00640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f820c10f7a0 0x7f820c1a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.999+0000 7f8203e00640 1 -- 172.21.15.39:0/2439422927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820c109510 msgr2=0x7f820c19fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.999+0000 7f8203e00640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820c109510 0x7f820c19fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.999+0000 7f8203e00640 1 -- 172.21.15.39:0/2439422927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820c1a5df0 con 0x7f820c10ee50 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:33.999+0000 7f8203e00640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f820c10ee50 0x7f820c1a0420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f81f400d740 tx=0x7f81f400dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.000+0000 7f820a000640 1 -- 172.21.15.39:0/2439422927 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4017070 con 0x7f820c10ee50 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.000+0000 7f8212600640 1 -- 172.21.15.39:0/2439422927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820c1a60e0 con 0x7f820c10ee50 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.000+0000 7f820a000640 1 -- 172.21.15.39:0/2439422927 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f400de50 con 0x7f820c10ee50 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.000+0000 7f820a000640 1 -- 172.21.15.39:0/2439422927 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f400f620 con 0x7f820c10ee50 2024-07-04T08:29:33.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.000+0000 7f8212600640 1 -- 172.21.15.39:0/2439422927 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820c1ac840 con 0x7f820c10ee50 2024-07-04T08:29:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.003+0000 7f821a310740 1 Processor -- start 2024-07-04T08:29:33.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.004+0000 7f821a310740 1 -- start start 2024-07-04T08:29:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.004+0000 7f821a310740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a22687950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.004+0000 7f821a310740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22687e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.004+0000 7f821a310740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222c2390 0x555a226883d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.004+0000 7f821a310740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555a225fdee0 con 0x555a222ba1b0 2024-07-04T08:29:33.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.004+0000 7f821a310740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555a225ed480 con 0x555a222e3070 2024-07-04T08:29:33.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.004+0000 7f821a310740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555a225f14c0 con 0x555a222c2390 2024-07-04T08:29:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.004+0000 7f820a000640 1 -- 172.21.15.39:0/2439422927 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f81f401a070 con 0x7f820c10ee50 2024-07-04T08:29:33.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.004+0000 7f8211c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a22687950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8211200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22687e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8211c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a22687950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:42258/0 (socket says 172.21.15.39:42258) 2024-07-04T08:29:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8211c00640 1 -- 172.21.15.39:0/2869241639 learned_addr learned my addr 172.21.15.39:0/2869241639 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8203e00640 1 --2- 172.21.15.39:0/2869241639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222c2390 0x555a226883d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8211c00640 1 -- 172.21.15.39:0/2869241639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222c2390 msgr2=0x555a226883d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8211c00640 1 --2- 172.21.15.39:0/2869241639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222c2390 0x555a226883d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8211c00640 1 -- 172.21.15.39:0/2869241639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 msgr2=0x555a22687e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8211c00640 1 --2- 172.21.15.39:0/2869241639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22687e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8211c00640 1 -- 172.21.15.39:0/2869241639 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a22585650 con 0x555a222e3070 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.005+0000 7f8211c00640 1 --2- 172.21.15.39:0/2869241639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a22687950 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f82040039f0 tx=0x7f8204003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5098574f40eddd5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.006+0000 7f8202000640 1 -- 172.21.15.39:0/2869241639 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820401a470 con 0x555a222e3070 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.006+0000 7f8202000640 1 -- 172.21.15.39:0/2869241639 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204015070 con 0x555a222e3070 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.006+0000 7f8202000640 1 -- 172.21.15.39:0/2869241639 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204010800 con 0x555a222e3070 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.006+0000 7f820a000640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81f00b2960 0x7f81f00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f820a000640 1 -- 172.21.15.39:0/2439422927 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f81f40e23f0 con 0x7f820c10ee50 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f821a310740 1 -- 172.21.15.39:0/2869241639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 msgr2=0x555a22687950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:33.996 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f821a310740 1 --2- 172.21.15.39:0/2869241639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a22687950 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f82040039f0 tx=0x7f8204003eb0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f8211200640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81f00b2960 0x7f81f00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f821a310740 1 -- 172.21.15.39:0/2869241639 shutdown_connections 2024-07-04T08:29:33.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f821a310740 1 --2- 172.21.15.39:0/2869241639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22687e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f821a310740 1 --2- 172.21.15.39:0/2869241639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222c2390 0x555a226883d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f821a310740 1 --2- 172.21.15.39:0/2869241639 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a22687950 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f821a310740 1 -- 172.21.15.39:0/2869241639 >> 172.21.15.39:0/2869241639 conn(0x555a222d7460 msgr2=0x555a22685840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f821a310740 1 -- 172.21.15.39:0/2869241639 shutdown_connections 2024-07-04T08:29:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.007+0000 7f821a310740 1 -- 172.21.15.39:0/2869241639 wait complete. 2024-07-04T08:29:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.008+0000 7f8211200640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81f00b2960 0x7f81f00b4e70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x555a226865d0 tx=0x7f81fc031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.008+0000 7f821a310740 1 -- start start 2024-07-04T08:29:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.008+0000 7f821a310740 1 Processor -- start 2024-07-04T08:29:33.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f821a310740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22693590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f821a310740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222c2390 0x555a22693af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f821a310740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 0x555a2269f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f821a310740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555a22451700 con 0x555a222ba1b0 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f821a310740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555a22594bb0 con 0x555a222c2390 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f821a310740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555a22696130 con 0x555a222e3070 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f821a310740 10 client.0 _ll_register_callbacks cb 0x555a225e6570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f8211200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22693590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f8203e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222c2390 0x555a22693af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f8211200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22693590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56030/0 (socket says 172.21.15.39:56030) 2024-07-04T08:29:34.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f8211200640 1 -- 172.21.15.39:0/3919553329 learned_addr learned my addr 172.21.15.39:0/3919553329 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.009+0000 7f8211c00640 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 0x555a2269f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f8203e00640 1 -- 172.21.15.39:0/3919553329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 msgr2=0x555a2269f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f8203e00640 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 0x555a2269f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f8203e00640 1 -- 172.21.15.39:0/3919553329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 msgr2=0x555a22693590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f8203e00640 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22693590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f8203e00640 1 -- 172.21.15.39:0/3919553329 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a2234e650 con 0x555a222c2390 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f8211200640 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22693590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f8203e00640 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222c2390 0x555a22693af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f81f40df780 tx=0x7f81f40a91a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be1bd85ab3c5eb80 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f81faa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f81faa00640 1 -- 172.21.15.39:0/3919553329 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4021050 con 0x555a222c2390 2024-07-04T08:29:34.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.010+0000 7f821a310740 1 -- 172.21.15.39:0/3919553329 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8204015070 con 0x555a222c2390 2024-07-04T08:29:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f81faa00640 1 -- 172.21.15.39:0/3919553329 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f81f40e2170 con 0x555a222c2390 2024-07-04T08:29:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f81faa00640 1 -- 172.21.15.39:0/3919553329 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81f40dfc10 con 0x555a222c2390 2024-07-04T08:29:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f821a310740 10 client.28562 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:34.002 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f821a310740 1 -- 172.21.15.39:0/3919553329 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a226a2220 con 0x555a222c2390 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f81faa00640 1 -- 172.21.15.39:0/3919553329 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204015070 con 0x555a222c2390 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f81faa00640 1 -- 172.21.15.39:0/3919553329 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f81f40b1ce0 con 0x555a222c2390 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f821a310740 10 client.28562 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f821a310740 10 client.28562 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f821a310740 20 client.28562 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f821a310740 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555a222a19f0 0x555a226a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f821a310740 4 client.28562 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.011+0000 7f821a310740 1 -- 172.21.15.39:0/3919553329 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a225ffe30 con 0x555a222a19f0 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.012+0000 7f8211200640 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555a222a19f0 0x555a226a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.012+0000 7f8211200640 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555a222a19f0 0x555a226a56a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=177bd6342869226b server_cookie=f77d6943b72209a8 in_seq=0 out_seq=0 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.012+0000 7f81faa00640 10 client.28562 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.013+0000 7f81faa00640 1 -- 172.21.15.39:0/3919553329 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a225ffe30 con 0x555a222a19f0 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.013+0000 7f81faa00640 10 client.28562 handle_command_reply: tid=0 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.013+0000 7f821a310740 1 client.28562 shutdown 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 -- 172.21.15.39:0/3919553329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222c2390 msgr2=0x555a22693af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222c2390 0x555a22693af0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f81f40df780 tx=0x7f81f40a91a0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 -- 172.21.15.39:0/3919553329 shutdown_connections 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555a222a19f0 0x555a226a56a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 0x555a2269f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222c2390 0x555a22693af0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 --2- 172.21.15.39:0/3919553329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222ba1b0 0x555a22693590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.005 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f81faa00640 0 client.28562 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:29:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 -- 172.21.15.39:0/3919553329 >> 172.21.15.39:0/3919553329 conn(0x555a222d7460 msgr2=0x555a2268aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 -- 172.21.15.39:0/3919553329 shutdown_connections 2024-07-04T08:29:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 1 -- 172.21.15.39:0/3919553329 wait complete. 2024-07-04T08:29:34.006 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.014+0000 7f821a310740 20 client.28562 trim_cache size 0 max 16384 2024-07-04T08:29:34.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.029+0000 7f821a310740 1 Processor -- start 2024-07-04T08:29:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.030+0000 7f821a310740 1 -- start start 2024-07-04T08:29:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.030+0000 7f821a310740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222ba1b0 0x555a22689340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.030+0000 7f821a310740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a22694fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.030+0000 7f821a310740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 0x555a2268cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.030+0000 7f821a310740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b6930 con 0x555a222c2390 2024-07-04T08:29:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.030+0000 7f821a310740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b8470 con 0x555a222ba1b0 2024-07-04T08:29:34.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.030+0000 7f821a310740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b7000 con 0x555a222e3070 2024-07-04T08:29:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.030+0000 7f8203e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a22694fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8203e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a22694fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56044/0 (socket says 172.21.15.39:56044) 2024-07-04T08:29:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8203e00640 1 -- 172.21.15.39:0/4289037640 learned_addr learned my addr 172.21.15.39:0/4289037640 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8211200640 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222ba1b0 0x555a22689340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8211c00640 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 0x555a2268cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8211200640 1 -- 172.21.15.39:0/4289037640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 msgr2=0x555a2268cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8211200640 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 0x555a2268cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8211200640 1 -- 172.21.15.39:0/4289037640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 msgr2=0x555a22694fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:34.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8211200640 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a22694fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8211200640 1 -- 172.21.15.39:0/4289037640 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a22694200 con 0x555a222ba1b0 2024-07-04T08:29:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8203e00640 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a22694fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.031+0000 7f8211200640 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222ba1b0 0x555a22689340 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x555a226a5de0 tx=0x7f81fc03d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638ad43250b89acd server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.032+0000 7f8201600640 1 -- 172.21.15.39:0/4289037640 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc056070 con 0x555a222ba1b0 2024-07-04T08:29:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.032+0000 7f8201600640 1 -- 172.21.15.39:0/4289037640 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81fc031910 con 0x555a222ba1b0 2024-07-04T08:29:34.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.032+0000 7f8201600640 1 -- 172.21.15.39:0/4289037640 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc035040 con 0x555a222ba1b0 2024-07-04T08:29:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 -- 172.21.15.39:0/4289037640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222ba1b0 msgr2=0x555a22689340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222ba1b0 0x555a22689340 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x555a226a5de0 tx=0x7f81fc03d000 comp rx=0 tx=0).stop 2024-07-04T08:29:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 -- 172.21.15.39:0/4289037640 shutdown_connections 2024-07-04T08:29:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a22694fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222e3070 0x555a2268cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 --2- 172.21.15.39:0/4289037640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222ba1b0 0x555a22689340 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 -- 172.21.15.39:0/4289037640 >> 172.21.15.39:0/4289037640 conn(0x555a222d7460 msgr2=0x555a226948b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:34.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 -- 172.21.15.39:0/4289037640 shutdown_connections 2024-07-04T08:29:34.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 -- 172.21.15.39:0/4289037640 wait complete. 2024-07-04T08:29:34.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.033+0000 7f821a310740 1 -- start start 2024-07-04T08:29:34.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.034+0000 7f821a310740 1 Processor -- start 2024-07-04T08:29:34.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.034+0000 7f821a310740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222ba1b0 0x555a2269b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.034+0000 7f821a310740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a2269b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.034+0000 7f821a310740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a226963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.034+0000 7f821a310740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b7da0 con 0x555a222c2390 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.034+0000 7f821a310740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b76d0 con 0x555a222e3070 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.034+0000 7f821a310740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b4910 con 0x555a222ba1b0 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.034+0000 7f821a310740 10 client.0 _ll_register_callbacks cb 0x7f81f4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8203e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a2269b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8211200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222ba1b0 0x555a2269b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8211200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222ba1b0 0x555a2269b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:52008/0 (socket says 172.21.15.39:52008) 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8211200640 1 -- 172.21.15.39:0/486197034 learned_addr learned my addr 172.21.15.39:0/486197034 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8211c00640 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a226963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8211200640 1 -- 172.21.15.39:0/486197034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 msgr2=0x555a226963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:34.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8211200640 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a226963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8211200640 1 -- 172.21.15.39:0/486197034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 msgr2=0x555a2269b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8211200640 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a2269b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8211200640 1 -- 172.21.15.39:0/486197034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a226968e0 con 0x555a222ba1b0 2024-07-04T08:29:34.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.035+0000 7f8203e00640 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a2269b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.036+0000 7f8211200640 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222ba1b0 0x555a2269b3f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f81fc006010 tx=0x7f81fc031590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3131131e563b31 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.036+0000 7f81faa00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:29:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.036+0000 7f81faa00640 1 -- 172.21.15.39:0/486197034 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc056070 con 0x555a222ba1b0 2024-07-04T08:29:34.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.036+0000 7f821a310740 1 -- 172.21.15.39:0/486197034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81fc031910 con 0x555a222ba1b0 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.036+0000 7f81faa00640 1 -- 172.21.15.39:0/486197034 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f81fc003c10 con 0x555a222ba1b0 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.036+0000 7f81faa00640 1 -- 172.21.15.39:0/486197034 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f81fc064050 con 0x555a222ba1b0 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.036+0000 7f821a310740 10 client.28507 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.036+0000 7f821a310740 1 -- 172.21.15.39:0/486197034 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a2270e530 con 0x555a222ba1b0 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.037+0000 7f81faa00640 1 -- 172.21.15.39:0/486197034 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81fc031910 con 0x555a222ba1b0 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.037+0000 7f81faa00640 1 -- 172.21.15.39:0/486197034 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f81fc05b960 con 0x555a222ba1b0 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.037+0000 7f821a310740 10 client.28507 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.037+0000 7f821a310740 10 client.28507 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.037+0000 7f821a310740 20 client.28507 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.037+0000 7f821a310740 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555a22276da0 0x555a22710c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:34.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.037+0000 7f821a310740 4 client.28507 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.037+0000 7f821a310740 1 -- 172.21.15.39:0/486197034 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a225fb8d0 con 0x555a22276da0 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.037+0000 7f8203e00640 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555a22276da0 0x555a22710c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.038+0000 7f8203e00640 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555a22276da0 0x555a22710c10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=791318a5452394c7 server_cookie=9a9753c19f9ce632 in_seq=0 out_seq=0 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.038+0000 7f81faa00640 10 client.28507 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.038+0000 7f81faa00640 1 -- 172.21.15.39:0/486197034 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a225fb8d0 con 0x555a22276da0 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.038+0000 7f81faa00640 10 client.28507 handle_command_reply: tid=0 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.038+0000 7f821a310740 1 client.28507 shutdown 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.038+0000 7f821a310740 1 -- 172.21.15.39:0/486197034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222ba1b0 msgr2=0x555a2269b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.038+0000 7f821a310740 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222ba1b0 0x555a2269b3f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f81fc006010 tx=0x7f81fc031590 comp rx=0 tx=0).stop 2024-07-04T08:29:34.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.038+0000 7f821a310740 1 -- 172.21.15.39:0/486197034 shutdown_connections 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.038+0000 7f821a310740 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555a22276da0 0x555a22710c10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.039+0000 7f821a310740 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555a222c2390 0x555a2269b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.039+0000 7f821a310740 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555a222e3070 0x555a226963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.039+0000 7f821a310740 1 --2- 172.21.15.39:0/486197034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555a222ba1b0 0x555a2269b3f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.039+0000 7f81faa00640 0 client.28507 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.039+0000 7f821a310740 1 -- 172.21.15.39:0/486197034 >> 172.21.15.39:0/486197034 conn(0x555a222d7460 msgr2=0x555a22689110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.039+0000 7f821a310740 1 -- 172.21.15.39:0/486197034 shutdown_connections 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.039+0000 7f821a310740 1 -- 172.21.15.39:0/486197034 wait complete. 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.039+0000 7f821a310740 20 client.28507 trim_cache size 0 max 16384 2024-07-04T08:29:34.031 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:29:34.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 -- 172.21.15.39:0/2439422927 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81f00b2960 msgr2=0x7f81f00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:34.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81f00b2960 0x7f81f00b4e70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x555a226865d0 tx=0x7f81fc031400 comp rx=0 tx=0).stop 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 -- 172.21.15.39:0/2439422927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f820c10ee50 msgr2=0x7f820c1a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f820c10ee50 0x7f820c1a0420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f81f400d740 tx=0x7f81f400dc00 comp rx=0 tx=0).stop 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 -- 172.21.15.39:0/2439422927 shutdown_connections 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f81f00b2960 0x7f81f00b4e70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f820c10f7a0 0x7f820c1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f820c10ee50 0x7f820c1a0420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 --2- 172.21.15.39:0/2439422927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f820c109510 0x7f820c19fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 -- 172.21.15.39:0/2439422927 >> 172.21.15.39:0/2439422927 conn(0x7f820c078b10 msgr2=0x7f820c0793c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 -- 172.21.15.39:0/2439422927 shutdown_connections 2024-07-04T08:29:34.033 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:34.040+0000 7f8212600640 1 -- 172.21.15.39:0/2439422927 wait complete. 2024-07-04T08:29:34.913 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:29:35.424 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:35.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:35 smithi133 ceph-mon[37050]: pgmap v927: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-07-04T08:29:35.573 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:35 smithi039 ceph-mon[32062]: pgmap v927: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-07-04T08:29:35.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:35 smithi050 ceph-mon[36577]: pgmap v927: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 12 op/s 2024-07-04T08:29:37.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:37 smithi133 ceph-mon[37050]: pgmap v928: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-07-04T08:29:37.558 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:37 smithi039 ceph-mon[32062]: pgmap v928: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-07-04T08:29:37.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:37 smithi050 ceph-mon[36577]: pgmap v928: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-07-04T08:29:37.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.786+0000 7f55a2200640 1 -- 172.21.15.39:0/667020548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10b590 msgr2=0x7f559c111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:37.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.786+0000 7f55a2200640 1 --2- 172.21.15.39:0/667020548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10b590 0x7f559c111dc0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5594009ca0 tx=0x7f559402a8b0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.786+0000 7f55a2200640 1 -- 172.21.15.39:0/667020548 shutdown_connections 2024-07-04T08:29:37.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.786+0000 7f55a2200640 1 --2- 172.21.15.39:0/667020548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10b590 0x7f559c111dc0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.786+0000 7f55a2200640 1 --2- 172.21.15.39:0/667020548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c10ac40 0x7f559c10b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.786+0000 7f55a2200640 1 --2- 172.21.15.39:0/667020548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f559c105300 0x7f559c105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.786+0000 7f55a2200640 1 -- 172.21.15.39:0/667020548 >> 172.21.15.39:0/667020548 conn(0x7f559c100d10 msgr2=0x7f559c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:37.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.786+0000 7f55a2200640 1 -- 172.21.15.39:0/667020548 shutdown_connections 2024-07-04T08:29:37.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.786+0000 7f55a2200640 1 -- 172.21.15.39:0/667020548 wait complete. 2024-07-04T08:29:37.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.787+0000 7f55a2200640 1 Processor -- start 2024-07-04T08:29:37.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.788+0000 7f55a2200640 1 -- start start 2024-07-04T08:29:37.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.788+0000 7f55a2200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c105300 0x7f559c19bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.788+0000 7f55a2200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10ac40 0x7f559c19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.788+0000 7f55a2200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f559c10b590 0x7f559c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.788+0000 7f55a2200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1152b0 con 0x7f559c10ac40 2024-07-04T08:29:37.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.788+0000 7f55a2200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f559c115130 con 0x7f559c10b590 2024-07-04T08:29:37.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.788+0000 7f55a2200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f559c115430 con 0x7f559c105300 2024-07-04T08:29:37.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10ac40 0x7f559c19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10ac40 0x7f559c19c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56076/0 (socket says 172.21.15.39:56076) 2024-07-04T08:29:37.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559aa00640 1 -- 172.21.15.39:0/701459011 learned_addr learned my addr 172.21.15.39:0/701459011 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:37.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559b400640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c105300 0x7f559c19bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559be00640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f559c10b590 0x7f559c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559aa00640 1 -- 172.21.15.39:0/701459011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c105300 msgr2=0x7f559c19bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559aa00640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c105300 0x7f559c19bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559aa00640 1 -- 172.21.15.39:0/701459011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f559c10b590 msgr2=0x7f559c1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559aa00640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f559c10b590 0x7f559c1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559aa00640 1 -- 172.21.15.39:0/701459011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c1a1c30 con 0x7f559c10ac40 2024-07-04T08:29:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.789+0000 7f559b400640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c105300 0x7f559c19bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.790+0000 7f559aa00640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10ac40 0x7f559c19c260 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f558400e9c0 tx=0x7f558400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.790+0000 7f5593e00640 1 -- 172.21.15.39:0/701459011 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558400ccc0 con 0x7f559c10ac40 2024-07-04T08:29:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.790+0000 7f5593e00640 1 -- 172.21.15.39:0/701459011 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55840054d0 con 0x7f559c10ac40 2024-07-04T08:29:37.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.790+0000 7f55a2200640 1 -- 172.21.15.39:0/701459011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c1a1f20 con 0x7f559c10ac40 2024-07-04T08:29:37.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.790+0000 7f5593e00640 1 -- 172.21.15.39:0/701459011 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584010430 con 0x7f559c10ac40 2024-07-04T08:29:37.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.790+0000 7f55a2200640 1 -- 172.21.15.39:0/701459011 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c1a8680 con 0x7f559c10ac40 2024-07-04T08:29:37.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.793+0000 7f55a8895740 1 Processor -- start 2024-07-04T08:29:37.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.794+0000 7f55a8895740 1 -- start start 2024-07-04T08:29:37.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.794+0000 7f5593e00640 1 -- 172.21.15.39:0/701459011 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5584005020 con 0x7f559c10ac40 2024-07-04T08:29:37.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.794+0000 7f55a8895740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d22eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.794+0000 7f55a8895740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 0x5592d22ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.794+0000 7f55a8895740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f26390 0x5592d22ec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.794+0000 7f55a8895740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5592d2261f00 con 0x5592d1f47070 2024-07-04T08:29:37.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.794+0000 7f55a8895740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5592d22514a0 con 0x5592d1f26390 2024-07-04T08:29:37.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.794+0000 7f55a8895740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592d22554e0 con 0x5592d1f1e1b0 2024-07-04T08:29:37.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d22eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 0x5592d22ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d22eb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56084/0 (socket says 172.21.15.39:56084) 2024-07-04T08:29:37.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f26390 0x5592d22ec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559be00640 1 -- 172.21.15.39:0/2736808581 learned_addr learned my addr 172.21.15.39:0/2736808581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:37.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559b400640 1 -- 172.21.15.39:0/2736808581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f26390 msgr2=0x5592d22ec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:37.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559b400640 1 --2- 172.21.15.39:0/2736808581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f26390 0x5592d22ec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559b400640 1 -- 172.21.15.39:0/2736808581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 msgr2=0x5592d22eb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:37.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559b400640 1 --2- 172.21.15.39:0/2736808581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d22eb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.795+0000 7f559b400640 1 -- 172.21.15.39:0/2736808581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592d21e9670 con 0x5592d1f1e1b0 2024-07-04T08:29:37.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.796+0000 7f559b400640 1 --2- 172.21.15.39:0/2736808581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 0x5592d22ebd10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f558c009fd0 tx=0x7f558c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b73a1a29b5d748 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:37.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.796+0000 7f5590c00640 1 -- 172.21.15.39:0/2736808581 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c004020 con 0x5592d1f1e1b0 2024-07-04T08:29:37.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.796+0000 7f5590c00640 1 -- 172.21.15.39:0/2736808581 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f558c0054d0 con 0x5592d1f1e1b0 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.796+0000 7f5590c00640 1 -- 172.21.15.39:0/2736808581 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c004c10 con 0x5592d1f1e1b0 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.796+0000 7f5593e00640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55800b2960 0x7f55800b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.796+0000 7f5593e00640 1 -- 172.21.15.39:0/701459011 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f55840e2c30 con 0x7f559c10ac40 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f559be00640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55800b2960 0x7f55800b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f55a8895740 1 -- 172.21.15.39:0/2736808581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 msgr2=0x5592d22ebd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f55a8895740 1 --2- 172.21.15.39:0/2736808581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 0x5592d22ebd10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f558c009fd0 tx=0x7f558c0118d0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f55a8895740 1 -- 172.21.15.39:0/2736808581 shutdown_connections 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f55a8895740 1 --2- 172.21.15.39:0/2736808581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f26390 0x5592d22ec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f55a8895740 1 --2- 172.21.15.39:0/2736808581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 0x5592d22ebd10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f55a8895740 1 --2- 172.21.15.39:0/2736808581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d22eb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f55a8895740 1 -- 172.21.15.39:0/2736808581 >> 172.21.15.39:0/2736808581 conn(0x5592d1f3b460 msgr2=0x5592d22e96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:37.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f55a8895740 1 -- 172.21.15.39:0/2736808581 shutdown_connections 2024-07-04T08:29:37.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.797+0000 7f559be00640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55800b2960 0x7f55800b4e70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f55940094e0 tx=0x7f5594005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:37.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.798+0000 7f55a8895740 1 -- 172.21.15.39:0/2736808581 wait complete. 2024-07-04T08:29:37.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.798+0000 7f55a8895740 1 -- start start 2024-07-04T08:29:37.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.798+0000 7f55a8895740 1 Processor -- start 2024-07-04T08:29:37.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f55a8895740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f1e1b0 0x5592d22f7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f55a8895740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 0x5592d22f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f55a8895740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d23032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f55a8895740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5592d20b5700 con 0x5592d1f47070 2024-07-04T08:29:37.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f55a8895740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5592d21f8bd0 con 0x5592d1f1e1b0 2024-07-04T08:29:37.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f55a8895740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592d22f9fb0 con 0x5592d1f26390 2024-07-04T08:29:37.792 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f55a8895740 10 client.0 _ll_register_callbacks cb 0x5592d224a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:37.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f559be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d23032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f559be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d23032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56096/0 (socket says 172.21.15.39:56096) 2024-07-04T08:29:37.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f559be00640 1 -- 172.21.15.39:0/737444683 learned_addr learned my addr 172.21.15.39:0/737444683 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:37.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f559aa00640 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 0x5592d22f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.799+0000 7f559b400640 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f1e1b0 0x5592d22f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.800+0000 7f559be00640 1 -- 172.21.15.39:0/737444683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 msgr2=0x5592d22f78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:37.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.800+0000 7f559be00640 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 0x5592d22f78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.800+0000 7f559be00640 1 -- 172.21.15.39:0/737444683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f1e1b0 msgr2=0x5592d22f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:37.793 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.800+0000 7f559be00640 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f1e1b0 0x5592d22f7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.800+0000 7f559be00640 1 -- 172.21.15.39:0/737444683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592d1fb2650 con 0x5592d1f47070 2024-07-04T08:29:37.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.800+0000 7f559b400640 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f1e1b0 0x5592d22f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:37.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.800+0000 7f559be00640 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d23032d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f559402add0 tx=0x7f559402a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff031dfccf42f9d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:37.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.800+0000 7f5589600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:29:37.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.801+0000 7f5589600640 1 -- 172.21.15.39:0/737444683 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594053440 con 0x5592d1f47070 2024-07-04T08:29:37.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.801+0000 7f5589600640 1 -- 172.21.15.39:0/737444683 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f5594049420 con 0x5592d1f47070 2024-07-04T08:29:37.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.801+0000 7f55a8895740 1 -- 172.21.15.39:0/737444683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558c0054d0 con 0x5592d1f47070 2024-07-04T08:29:37.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.801+0000 7f5589600640 1 -- 172.21.15.39:0/737444683 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5594049620 con 0x5592d1f47070 2024-07-04T08:29:37.794 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.801+0000 7f55a8895740 10 client.18459 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.801+0000 7f55a8895740 1 -- 172.21.15.39:0/737444683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5592d2305f70 con 0x5592d1f47070 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.801+0000 7f5589600640 1 -- 172.21.15.39:0/737444683 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f558c0054d0 con 0x5592d1f47070 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.802+0000 7f5589600640 1 -- 172.21.15.39:0/737444683 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5594051070 con 0x5592d1f47070 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.802+0000 7f55a8895740 10 client.18459 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.802+0000 7f55a8895740 10 client.18459 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.802+0000 7f55a8895740 20 client.18459 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.802+0000 7f55a8895740 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5592d1f059f0 0x5592d23094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.802+0000 7f55a8895740 4 client.18459 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.802+0000 7f55a8895740 1 -- 172.21.15.39:0/737444683 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592d2263e50 con 0x5592d1f059f0 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.802+0000 7f559b400640 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5592d1f059f0 0x5592d23094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.803+0000 7f559b400640 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5592d1f059f0 0x5592d23094b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfad9d873fceb179 server_cookie=6a9707a198918ec3 in_seq=0 out_seq=0 2024-07-04T08:29:37.795 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.803+0000 7f5589600640 10 client.18459 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.803+0000 7f5589600640 1 -- 172.21.15.39:0/737444683 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5592d2263e50 con 0x5592d1f059f0 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.803+0000 7f5589600640 10 client.18459 handle_command_reply: tid=0 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 client.18459 shutdown 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 -- 172.21.15.39:0/737444683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 msgr2=0x5592d23032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d23032d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f559402add0 tx=0x7f559402a8b0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 -- 172.21.15.39:0/737444683 shutdown_connections 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5592d1f059f0 0x5592d23094b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f47070 0x5592d23032d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 0x5592d22f78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.797 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 --2- 172.21.15.39:0/737444683 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f1e1b0 0x5592d22f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 -- 172.21.15.39:0/737444683 >> 172.21.15.39:0/737444683 conn(0x5592d1f3b460 msgr2=0x5592d22ee9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:37.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f5589600640 0 client.18459 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:29:37.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 -- 172.21.15.39:0/737444683 shutdown_connections 2024-07-04T08:29:37.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 1 -- 172.21.15.39:0/737444683 wait complete. 2024-07-04T08:29:37.798 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.804+0000 7f55a8895740 20 client.18459 trim_cache size 0 max 16384 2024-07-04T08:29:37.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.820+0000 7f55a8895740 1 Processor -- start 2024-07-04T08:29:37.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.820+0000 7f55a8895740 1 -- start start 2024-07-04T08:29:37.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.820+0000 7f55a8895740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 0x5592d22e8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.820+0000 7f55a8895740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f26390 0x5592d22ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.820+0000 7f55a8895740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 0x5592d22f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.820+0000 7f55a8895740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b2660 con 0x5592d1f26390 2024-07-04T08:29:37.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.820+0000 7f55a8895740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b41a0 con 0x5592d1f47070 2024-07-04T08:29:37.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.820+0000 7f55a8895740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b2d30 con 0x5592d1f1e1b0 2024-07-04T08:29:37.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f26390 0x5592d22ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f26390 0x5592d22ecf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56108/0 (socket says 172.21.15.39:56108) 2024-07-04T08:29:37.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559aa00640 1 -- 172.21.15.39:0/3218608212 learned_addr learned my addr 172.21.15.39:0/3218608212 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:37.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559be00640 1 --2- 172.21.15.39:0/3218608212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 0x5592d22f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559b400640 1 --2- 172.21.15.39:0/3218608212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 0x5592d22e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559aa00640 1 -- 172.21.15.39:0/3218608212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 msgr2=0x5592d22e8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:37.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559aa00640 1 --2- 172.21.15.39:0/3218608212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 0x5592d22e8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559aa00640 1 -- 172.21.15.39:0/3218608212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 msgr2=0x5592d22f0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:37.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559aa00640 1 --2- 172.21.15.39:0/3218608212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 0x5592d22f0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.821+0000 7f559aa00640 1 -- 172.21.15.39:0/3218608212 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592d22ee630 con 0x5592d1f26390 2024-07-04T08:29:37.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.822+0000 7f559aa00640 1 --2- 172.21.15.39:0/3218608212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f26390 0x5592d22ecf10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f55840e6480 tx=0x7f55840e62d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f37e5fa20d9122e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:37.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.822+0000 7f558be00640 1 -- 172.21.15.39:0/3218608212 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55840a94a0 con 0x5592d1f26390 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.822+0000 7f558be00640 1 -- 172.21.15.39:0/3218608212 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5584005310 con 0x5592d1f26390 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.822+0000 7f558be00640 1 -- 172.21.15.39:0/3218608212 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55840a52e0 con 0x5592d1f26390 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.823+0000 7f55a8895740 1 -- 172.21.15.39:0/3218608212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f26390 msgr2=0x5592d22ecf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.823+0000 7f55a8895740 1 --2- 172.21.15.39:0/3218608212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f26390 0x5592d22ecf10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f55840e6480 tx=0x7f55840e62d0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.823+0000 7f55a8895740 1 -- 172.21.15.39:0/3218608212 shutdown_connections 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.823+0000 7f55a8895740 1 --2- 172.21.15.39:0/3218608212 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f26390 0x5592d22ecf10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.823+0000 7f55a8895740 1 --2- 172.21.15.39:0/3218608212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 0x5592d22f0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.823+0000 7f55a8895740 1 --2- 172.21.15.39:0/3218608212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f1e1b0 0x5592d22e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.823+0000 7f55a8895740 1 -- 172.21.15.39:0/3218608212 >> 172.21.15.39:0/3218608212 conn(0x5592d1f3b460 msgr2=0x5592d22ec800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.823+0000 7f55a8895740 1 -- 172.21.15.39:0/3218608212 shutdown_connections 2024-07-04T08:29:37.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.823+0000 7f55a8895740 1 -- 172.21.15.39:0/3218608212 wait complete. 2024-07-04T08:29:37.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.824+0000 7f55a8895740 1 -- start start 2024-07-04T08:29:37.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.824+0000 7f55a8895740 1 Processor -- start 2024-07-04T08:29:37.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f55a8895740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f1e1b0 0x5592d22ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f55a8895740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 0x5592d23029c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f55a8895740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 0x5592d2302f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f55a8895740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b3ad0 con 0x5592d1f1e1b0 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f55a8895740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b3400 con 0x5592d1f47070 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f55a8895740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b0640 con 0x5592d1f26390 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f55a8895740 10 client.0 _ll_register_callbacks cb 0x7f5594009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f559b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f1e1b0 0x5592d22ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f559aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 0x5592d23029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f559b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f1e1b0 0x5592d22ff7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56122/0 (socket says 172.21.15.39:56122) 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.825+0000 7f559b400640 1 -- 172.21.15.39:0/1677235936 learned_addr learned my addr 172.21.15.39:0/1677235936 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.826+0000 7f559be00640 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 0x5592d2302f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.826+0000 7f559b400640 1 -- 172.21.15.39:0/1677235936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 msgr2=0x5592d23029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:37.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.826+0000 7f559b400640 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 0x5592d23029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.826+0000 7f559b400640 1 -- 172.21.15.39:0/1677235936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 msgr2=0x5592d2302f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.826+0000 7f559b400640 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 0x5592d2302f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.826+0000 7f559b400640 1 -- 172.21.15.39:0/1677235936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592d2303540 con 0x5592d1f1e1b0 2024-07-04T08:29:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.826+0000 7f559aa00640 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 0x5592d23029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.826+0000 7f559b400640 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f1e1b0 0x5592d22ff7c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f558c011dd0 tx=0x7f558c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fc2838d97a8bb8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:37.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.826+0000 7f5589600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:29:37.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.827+0000 7f5589600640 1 -- 172.21.15.39:0/1677235936 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c01d040 con 0x5592d1f1e1b0 2024-07-04T08:29:37.819 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.827+0000 7f5589600640 1 -- 172.21.15.39:0/1677235936 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f558c00f8b0 con 0x5592d1f1e1b0 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.827+0000 7f55a8895740 1 -- 172.21.15.39:0/1677235936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5592d225f8f0 con 0x5592d1f1e1b0 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.827+0000 7f5589600640 1 -- 172.21.15.39:0/1677235936 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f558c03a050 con 0x5592d1f1e1b0 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.827+0000 7f55a8895740 10 client.18465 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.827+0000 7f55a8895740 1 -- 172.21.15.39:0/1677235936 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5592d22fd000 con 0x5592d1f1e1b0 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.827+0000 7f5589600640 1 -- 172.21.15.39:0/1677235936 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5592d225f8f0 con 0x5592d1f1e1b0 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.828+0000 7f5589600640 1 -- 172.21.15.39:0/1677235936 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f558c004740 con 0x5592d1f1e1b0 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.828+0000 7f55a8895740 10 client.18465 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.828+0000 7f55a8895740 10 client.18465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.828+0000 7f55a8895740 20 client.18465 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:37.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.828+0000 7f55a8895740 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5592d1edada0 0x5592d2373ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.828+0000 7f55a8895740 4 client.18465 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:29:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.828+0000 7f55a8895740 1 -- 172.21.15.39:0/1677235936 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592d225d030 con 0x5592d1edada0 2024-07-04T08:29:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.828+0000 7f559aa00640 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5592d1edada0 0x5592d2373ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.829+0000 7f559aa00640 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5592d1edada0 0x5592d2373ff0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=944905ed3b26a2cd server_cookie=19ede39b3a3cb9b3 in_seq=0 out_seq=0 2024-07-04T08:29:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.829+0000 7f5589600640 10 client.18465 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:29:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.829+0000 7f5589600640 1 -- 172.21.15.39:0/1677235936 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592d225d030 con 0x5592d1edada0 2024-07-04T08:29:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.829+0000 7f5589600640 10 client.18465 handle_command_reply: tid=0 2024-07-04T08:29:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.829+0000 7f55a8895740 1 client.18465 shutdown 2024-07-04T08:29:37.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 -- 172.21.15.39:0/1677235936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f1e1b0 msgr2=0x5592d22ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f1e1b0 0x5592d22ff7c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f558c011dd0 tx=0x7f558c005d30 comp rx=0 tx=0).stop 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 -- 172.21.15.39:0/1677235936 shutdown_connections 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5592d1edada0 0x5592d2373ff0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592d1f47070 0x5592d2302f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d1f26390 0x5592d23029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 --2- 172.21.15.39:0/1677235936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592d1f1e1b0 0x5592d22ff7c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 -- 172.21.15.39:0/1677235936 >> 172.21.15.39:0/1677235936 conn(0x5592d1f3b460 msgr2=0x5592d22ed1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f5589600640 0 client.18465 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 -- 172.21.15.39:0/1677235936 shutdown_connections 2024-07-04T08:29:37.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 1 -- 172.21.15.39:0/1677235936 wait complete. 2024-07-04T08:29:37.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.830+0000 7f55a8895740 20 client.18465 trim_cache size 0 max 16384 2024-07-04T08:29:37.823 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.831+0000 7f55a2200640 1 -- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55800b2960 msgr2=0x7f55800b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.831+0000 7f55a2200640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55800b2960 0x7f55800b4e70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f55940094e0 tx=0x7f5594005d80 comp rx=0 tx=0).stop 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.831+0000 7f55a2200640 1 -- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10ac40 msgr2=0x7f559c19c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.831+0000 7f55a2200640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10ac40 0x7f559c19c260 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f558400e9c0 tx=0x7f558400ee80 comp rx=0 tx=0).stop 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.832+0000 7f55a2200640 1 -- 172.21.15.39:0/701459011 shutdown_connections 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.832+0000 7f55a2200640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f55800b2960 0x7f55800b4e70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.832+0000 7f55a2200640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f559c10ac40 0x7f559c19c260 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.832+0000 7f55a2200640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f559c10b590 0x7f559c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.832+0000 7f55a2200640 1 --2- 172.21.15.39:0/701459011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c105300 0x7f559c19bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:37.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.832+0000 7f55a2200640 1 -- 172.21.15.39:0/701459011 >> 172.21.15.39:0/701459011 conn(0x7f559c100d10 msgr2=0x7f559c10a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:37.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.832+0000 7f55a2200640 1 -- 172.21.15.39:0/701459011 shutdown_connections 2024-07-04T08:29:37.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:37.832+0000 7f55a2200640 1 -- 172.21.15.39:0/701459011 wait complete. 2024-07-04T08:29:38.700 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:29:39.210 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:39.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:39 smithi133 ceph-mon[37050]: pgmap v929: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-07-04T08:29:39.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:39 smithi039 ceph-mon[32062]: pgmap v929: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-07-04T08:29:39.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:39 smithi050 ceph-mon[36577]: pgmap v929: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-07-04T08:29:41.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:41 smithi039 ceph-mon[32062]: pgmap v930: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-07-04T08:29:41.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:41 smithi133 ceph-mon[37050]: pgmap v930: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-07-04T08:29:41.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:41 smithi050 ceph-mon[36577]: pgmap v930: 129 pgs: 129 active+clean; 26 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-07-04T08:29:41.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.717+0000 7f96eb200640 1 -- 172.21.15.39:0/2299573536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96ec10f7a0 msgr2=0x7f96ec115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:41.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.717+0000 7f96eb200640 1 --2- 172.21.15.39:0/2299573536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96ec10f7a0 0x7f96ec115fd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009be0 tx=0x7f96e402a6c0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.717+0000 7f96eb200640 1 -- 172.21.15.39:0/2299573536 shutdown_connections 2024-07-04T08:29:41.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.717+0000 7f96eb200640 1 --2- 172.21.15.39:0/2299573536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96ec10f7a0 0x7f96ec115fd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.717+0000 7f96eb200640 1 --2- 172.21.15.39:0/2299573536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96ec10ee50 0x7f96ec10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.717+0000 7f96eb200640 1 --2- 172.21.15.39:0/2299573536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96ec109510 0x7f96ec109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.717+0000 7f96eb200640 1 -- 172.21.15.39:0/2299573536 >> 172.21.15.39:0/2299573536 conn(0x7f96ec078b10 msgr2=0x7f96ec107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:41.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.717+0000 7f96eb200640 1 -- 172.21.15.39:0/2299573536 shutdown_connections 2024-07-04T08:29:41.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.717+0000 7f96eb200640 1 -- 172.21.15.39:0/2299573536 wait complete. 2024-07-04T08:29:41.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.719+0000 7f96eb200640 1 Processor -- start 2024-07-04T08:29:41.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.719+0000 7f96eb200640 1 -- start start 2024-07-04T08:29:41.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.719+0000 7f96eb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96ec109510 0x7f96ec19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.719+0000 7f96eb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96ec10ee50 0x7f96ec1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.720+0000 7f96eb200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96ec10f7a0 0x7f96ec1a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.720+0000 7f96eb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec119450 con 0x7f96ec109510 2024-07-04T08:29:41.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.720+0000 7f96eb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1192d0 con 0x7f96ec10ee50 2024-07-04T08:29:41.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.720+0000 7f96eb200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1195d0 con 0x7f96ec10f7a0 2024-07-04T08:29:41.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.720+0000 7f96e9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96ec109510 0x7f96ec19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.720+0000 7f96e9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96ec109510 0x7f96ec19fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58402/0 (socket says 172.21.15.39:58402) 2024-07-04T08:29:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.720+0000 7f96e9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96ec10ee50 0x7f96ec1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.720+0000 7f96e9e00640 1 -- 172.21.15.39:0/2676563506 learned_addr learned my addr 172.21.15.39:0/2676563506 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.720+0000 7f96ea800640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96ec10f7a0 0x7f96ec1a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.721+0000 7f96e9400640 1 -- 172.21.15.39:0/2676563506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96ec10f7a0 msgr2=0x7f96ec1a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.721+0000 7f96e9400640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96ec10f7a0 0x7f96ec1a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.721+0000 7f96e9400640 1 -- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96ec109510 msgr2=0x7f96ec19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.721+0000 7f96e9400640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96ec109510 0x7f96ec19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.721+0000 7f96e9400640 1 -- 172.21.15.39:0/2676563506 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec1a5d90 con 0x7f96ec10ee50 2024-07-04T08:29:41.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.721+0000 7f96e9e00640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96ec109510 0x7f96ec19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:41.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.721+0000 7f96e9400640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96ec10ee50 0x7f96ec1a03c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f96ec06bd90 tx=0x7f96d800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:41.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.721+0000 7f96e2a00640 1 -- 172.21.15.39:0/2676563506 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d800cd00 con 0x7f96ec10ee50 2024-07-04T08:29:41.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.721+0000 7f96e2a00640 1 -- 172.21.15.39:0/2676563506 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d80035f0 con 0x7f96ec10ee50 2024-07-04T08:29:41.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.722+0000 7f96eb200640 1 -- 172.21.15.39:0/2676563506 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec1a6080 con 0x7f96ec10ee50 2024-07-04T08:29:41.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.722+0000 7f96e2a00640 1 -- 172.21.15.39:0/2676563506 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8010690 con 0x7f96ec10ee50 2024-07-04T08:29:41.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.722+0000 7f96eb200640 1 -- 172.21.15.39:0/2676563506 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec1ac7e0 con 0x7f96ec10ee50 2024-07-04T08:29:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.725+0000 7f96f73c3740 1 Processor -- start 2024-07-04T08:29:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.726+0000 7f96e2a00640 1 -- 172.21.15.39:0/2676563506 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f96d8004020 con 0x7f96ec10ee50 2024-07-04T08:29:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.726+0000 7f96f73c3740 1 -- start start 2024-07-04T08:29:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.726+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad594a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.726+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad594ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.726+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad594b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.726+0000 7f96f73c3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556ad58c0ee0 con 0x556ad557d1b0 2024-07-04T08:29:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.726+0000 7f96f73c3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556ad58b0480 con 0x556ad5585390 2024-07-04T08:29:41.715 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.726+0000 7f96f73c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ad58b44c0 con 0x556ad55a6070 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96ea800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad594a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96e9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad594ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96ea800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad594a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58302/0 (socket says 172.21.15.39:58302) 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96ea800640 1 -- 172.21.15.39:0/4175920048 learned_addr learned my addr 172.21.15.39:0/4175920048 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96e9400640 1 --2- 172.21.15.39:0/4175920048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad594b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96ea800640 1 -- 172.21.15.39:0/4175920048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 msgr2=0x556ad594b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96ea800640 1 --2- 172.21.15.39:0/4175920048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad594b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96ea800640 1 -- 172.21.15.39:0/4175920048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 msgr2=0x556ad594ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96ea800640 1 --2- 172.21.15.39:0/4175920048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad594ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96ea800640 1 -- 172.21.15.39:0/4175920048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad5848650 con 0x556ad55a6070 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.727+0000 7f96e9e00640 1 --2- 172.21.15.39:0/4175920048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad594ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:41.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.728+0000 7f96ea800640 1 --2- 172.21.15.39:0/4175920048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad594a950 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f96e402acb0 tx=0x7f96e4030420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6dd2505e5e284ae server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.728+0000 7f96d7400640 1 -- 172.21.15.39:0/4175920048 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4041030 con 0x556ad55a6070 2024-07-04T08:29:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.728+0000 7f96d7400640 1 -- 172.21.15.39:0/4175920048 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e4003e20 con 0x556ad55a6070 2024-07-04T08:29:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.728+0000 7f96d7400640 1 -- 172.21.15.39:0/4175920048 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4036040 con 0x556ad55a6070 2024-07-04T08:29:41.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.728+0000 7f96e2a00640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f96d00b2960 0x7f96d00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.728+0000 7f96e2a00640 1 -- 172.21.15.39:0/2676563506 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f96d80e2b70 con 0x7f96ec10ee50 2024-07-04T08:29:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.728+0000 7f96e9e00640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f96d00b2960 0x7f96d00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.729+0000 7f96f73c3740 1 -- 172.21.15.39:0/4175920048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 msgr2=0x556ad594a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.729+0000 7f96f73c3740 1 --2- 172.21.15.39:0/4175920048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad594a950 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f96e402acb0 tx=0x7f96e4030420 comp rx=0 tx=0).stop 2024-07-04T08:29:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.729+0000 7f96f73c3740 1 -- 172.21.15.39:0/4175920048 shutdown_connections 2024-07-04T08:29:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.729+0000 7f96f73c3740 1 --2- 172.21.15.39:0/4175920048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad594ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.729+0000 7f96f73c3740 1 --2- 172.21.15.39:0/4175920048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad594b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.729+0000 7f96f73c3740 1 --2- 172.21.15.39:0/4175920048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad594a950 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.729+0000 7f96f73c3740 1 -- 172.21.15.39:0/4175920048 >> 172.21.15.39:0/4175920048 conn(0x556ad559a460 msgr2=0x556ad5948840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.729+0000 7f96e9e00640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f96d00b2960 0x7f96d00b4e70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x556ad59495d0 tx=0x7f96dc009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.729+0000 7f96f73c3740 1 -- 172.21.15.39:0/4175920048 shutdown_connections 2024-07-04T08:29:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.730+0000 7f96f73c3740 1 -- 172.21.15.39:0/4175920048 wait complete. 2024-07-04T08:29:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.730+0000 7f96f73c3740 1 -- start start 2024-07-04T08:29:41.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.730+0000 7f96f73c3740 1 Processor -- start 2024-07-04T08:29:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.731+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad5956590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.731+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad5956b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.731+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad59623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.731+0000 7f96f73c3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556ad5714700 con 0x556ad557d1b0 2024-07-04T08:29:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.731+0000 7f96f73c3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556ad5857bb0 con 0x556ad5585390 2024-07-04T08:29:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.731+0000 7f96f73c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ad59590d0 con 0x556ad55a6070 2024-07-04T08:29:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.731+0000 7f96f73c3740 10 client.0 _ll_register_callbacks cb 0x556ad58a9570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:41.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.731+0000 7f96e9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad5956590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96e9e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad5956590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58432/0 (socket says 172.21.15.39:58432) 2024-07-04T08:29:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96e9e00640 1 -- 172.21.15.39:0/135125382 learned_addr learned my addr 172.21.15.39:0/135125382 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96e9400640 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad5956b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96ea800640 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad59623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96e9e00640 1 -- 172.21.15.39:0/135125382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 msgr2=0x556ad59623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96e9e00640 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad59623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96e9e00640 1 -- 172.21.15.39:0/135125382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 msgr2=0x556ad5956b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96e9e00640 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad5956b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96e9e00640 1 -- 172.21.15.39:0/135125382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad5611650 con 0x556ad557d1b0 2024-07-04T08:29:41.723 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96e9e00640 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad5956590 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00ff70 tx=0x7f96dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817dbdba46715622 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.732+0000 7f96cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.733+0000 7f96cbe00640 1 -- 172.21.15.39:0/135125382 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc02d060 con 0x556ad557d1b0 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.733+0000 7f96f73c3740 1 -- 172.21.15.39:0/135125382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e4003e20 con 0x556ad557d1b0 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.733+0000 7f96cbe00640 1 -- 172.21.15.39:0/135125382 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f96dc018030 con 0x556ad557d1b0 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.733+0000 7f96cbe00640 1 -- 172.21.15.39:0/135125382 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96dc034050 con 0x556ad557d1b0 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.733+0000 7f96f73c3740 10 client.18477 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.733+0000 7f96f73c3740 1 -- 172.21.15.39:0/135125382 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ad5965220 con 0x556ad557d1b0 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.733+0000 7f96cbe00640 1 -- 172.21.15.39:0/135125382 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e4003e20 con 0x556ad557d1b0 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.734+0000 7f96cbe00640 1 -- 172.21.15.39:0/135125382 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f96dc028980 con 0x556ad557d1b0 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.734+0000 7f96f73c3740 10 client.18477 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.734+0000 7f96f73c3740 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:29:41.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.734+0000 7f96f73c3740 20 client.18477 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.734+0000 7f96f73c3740 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ad55649f0 0x556ad59686d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.734+0000 7f96f73c3740 4 client.18477 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.734+0000 7f96f73c3740 1 -- 172.21.15.39:0/135125382 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad58c2e30 con 0x556ad55649f0 2024-07-04T08:29:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.734+0000 7f96e9400640 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ad55649f0 0x556ad59686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.735+0000 7f96e9400640 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ad55649f0 0x556ad59686d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05ba67380ae8db0 server_cookie=124548393bfca184 in_seq=0 out_seq=0 2024-07-04T08:29:41.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.735+0000 7f96cbe00640 10 client.18477 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96cbe00640 1 -- 172.21.15.39:0/135125382 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556ad58c2e30 con 0x556ad55649f0 2024-07-04T08:29:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96cbe00640 10 client.18477 handle_command_reply: tid=0 2024-07-04T08:29:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 client.18477 shutdown 2024-07-04T08:29:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 -- 172.21.15.39:0/135125382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 msgr2=0x556ad5956590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad5956590 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00ff70 tx=0x7f96dc004730 comp rx=0 tx=0).stop 2024-07-04T08:29:41.726 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 -- 172.21.15.39:0/135125382 shutdown_connections 2024-07-04T08:29:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ad55649f0 0x556ad59686d0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad55a6070 0x556ad59623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad5956b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 --2- 172.21.15.39:0/135125382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad5956590 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96cbe00640 0 client.18477 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 -- 172.21.15.39:0/135125382 >> 172.21.15.39:0/135125382 conn(0x556ad559a460 msgr2=0x556ad594dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.736+0000 7f96f73c3740 1 -- 172.21.15.39:0/135125382 shutdown_connections 2024-07-04T08:29:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.737+0000 7f96f73c3740 1 -- 172.21.15.39:0/135125382 wait complete. 2024-07-04T08:29:41.727 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.737+0000 7f96f73c3740 20 client.18477 trim_cache size 0 max 16384 2024-07-04T08:29:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.752+0000 7f96f73c3740 1 Processor -- start 2024-07-04T08:29:41.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.752+0000 7f96f73c3740 1 -- start start 2024-07-04T08:29:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.752+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad557d1b0 0x556ad5950170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad59621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad55a6070 0x556ad596ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96f73c3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1b67d0 con 0x556ad55a6070 2024-07-04T08:29:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96f73c3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1b8310 con 0x556ad5585390 2024-07-04T08:29:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96f73c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1b6ea0 con 0x556ad557d1b0 2024-07-04T08:29:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96ea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad55a6070 0x556ad596ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96e9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad59621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96e9e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad557d1b0 0x556ad5950170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96ea800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad55a6070 0x556ad596ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58444/0 (socket says 172.21.15.39:58444) 2024-07-04T08:29:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96e9400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad59621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33844/0 (socket says 172.21.15.39:33844) 2024-07-04T08:29:41.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96ea800640 1 -- 172.21.15.39:0/2629298317 learned_addr learned my addr 172.21.15.39:0/2629298317 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96e9400640 1 -- 172.21.15.39:0/2629298317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad557d1b0 msgr2=0x556ad5950170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96e9400640 1 --2- 172.21.15.39:0/2629298317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad557d1b0 0x556ad5950170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96e9400640 1 -- 172.21.15.39:0/2629298317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad55a6070 msgr2=0x556ad596ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96e9400640 1 --2- 172.21.15.39:0/2629298317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad55a6070 0x556ad596ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.753+0000 7f96e9400640 1 -- 172.21.15.39:0/2629298317 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad594ce70 con 0x556ad5585390 2024-07-04T08:29:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.754+0000 7f96e9400640 1 --2- 172.21.15.39:0/2629298317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad59621b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f96d80dff00 tx=0x7f96d80039a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b29aee36e8bb83f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.754+0000 7f96d6a00640 1 -- 172.21.15.39:0/2629298317 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d80b2230 con 0x556ad5585390 2024-07-04T08:29:41.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.754+0000 7f96d6a00640 1 -- 172.21.15.39:0/2629298317 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d80a95a0 con 0x556ad5585390 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.754+0000 7f96d6a00640 1 -- 172.21.15.39:0/2629298317 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8004310 con 0x556ad5585390 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.755+0000 7f96f73c3740 1 -- 172.21.15.39:0/2629298317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 msgr2=0x556ad59621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.755+0000 7f96f73c3740 1 --2- 172.21.15.39:0/2629298317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad59621b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f96d80dff00 tx=0x7f96d80039a0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.755+0000 7f96f73c3740 1 -- 172.21.15.39:0/2629298317 shutdown_connections 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.755+0000 7f96f73c3740 1 --2- 172.21.15.39:0/2629298317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad55a6070 0x556ad596ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.755+0000 7f96f73c3740 1 --2- 172.21.15.39:0/2629298317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad5585390 0x556ad59621b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.755+0000 7f96f73c3740 1 --2- 172.21.15.39:0/2629298317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad557d1b0 0x556ad5950170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.755+0000 7f96f73c3740 1 -- 172.21.15.39:0/2629298317 >> 172.21.15.39:0/2629298317 conn(0x556ad559a460 msgr2=0x556ad594fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.755+0000 7f96f73c3740 1 -- 172.21.15.39:0/2629298317 shutdown_connections 2024-07-04T08:29:41.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.755+0000 7f96f73c3740 1 -- 172.21.15.39:0/2629298317 wait complete. 2024-07-04T08:29:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.756+0000 7f96f73c3740 1 -- start start 2024-07-04T08:29:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.756+0000 7f96f73c3740 1 Processor -- start 2024-07-04T08:29:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.756+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad595a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.756+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad5585390 0x556ad595ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.756+0000 7f96f73c3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad55a6070 0x556ad59cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96f73c3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1b7c40 con 0x556ad557d1b0 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96f73c3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1b7570 con 0x556ad55a6070 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96f73c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1b47b0 con 0x556ad5585390 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96f73c3740 10 client.0 _ll_register_callbacks cb 0x7f96dc012f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96e9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad5585390 0x556ad595ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96e9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad5585390 0x556ad595ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58338/0 (socket says 172.21.15.39:58338) 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96e9400640 1 -- 172.21.15.39:0/3499461573 learned_addr learned my addr 172.21.15.39:0/3499461573 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96ea800640 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad55a6070 0x556ad59cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96e9e00640 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad595a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96e9400640 1 -- 172.21.15.39:0/3499461573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad55a6070 msgr2=0x556ad59cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:41.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96e9400640 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad55a6070 0x556ad59cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96e9400640 1 -- 172.21.15.39:0/3499461573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 msgr2=0x556ad595a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96e9400640 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad595a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.757+0000 7f96e9400640 1 -- 172.21.15.39:0/3499461573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad59d03e0 con 0x556ad5585390 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.758+0000 7f96e9400640 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad5585390 0x556ad595ae00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f96d80dfed0 tx=0x7f96d80b2a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bce80d4b5a0a7ef server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.758+0000 7f96cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.758+0000 7f96cbe00640 1 -- 172.21.15.39:0/3499461573 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d80a9a90 con 0x556ad5585390 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.758+0000 7f96cbe00640 1 -- 172.21.15.39:0/3499461573 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f96d8004310 con 0x556ad5585390 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.758+0000 7f96f73c3740 1 -- 172.21.15.39:0/3499461573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d80a95a0 con 0x556ad5585390 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.758+0000 7f96cbe00640 1 -- 172.21.15.39:0/3499461573 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96d80aa190 con 0x556ad5585390 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.758+0000 7f96f73c3740 10 client.28543 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.758+0000 7f96f73c3740 1 -- 172.21.15.39:0/3499461573 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556ad59d15a0 con 0x556ad5585390 2024-07-04T08:29:41.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.759+0000 7f96cbe00640 1 -- 172.21.15.39:0/3499461573 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d80a95a0 con 0x556ad5585390 2024-07-04T08:29:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.759+0000 7f96cbe00640 1 -- 172.21.15.39:0/3499461573 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f96d80e0180 con 0x556ad5585390 2024-07-04T08:29:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.759+0000 7f96f73c3740 10 client.28543 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.759+0000 7f96f73c3740 10 client.28543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:29:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.759+0000 7f96f73c3740 20 client.28543 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.759+0000 7f96f73c3740 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ad5539da0 0x556ad59d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.759+0000 7f96f73c3740 4 client.28543 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:29:41.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.759+0000 7f96f73c3740 1 -- 172.21.15.39:0/3499461573 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ad58be8d0 con 0x556ad5539da0 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.759+0000 7f96e9e00640 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ad5539da0 0x556ad59d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.760+0000 7f96e9e00640 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ad5539da0 0x556ad59d3ca0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95957315b719eb57 server_cookie=f8855f0796321fb9 in_seq=0 out_seq=0 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.760+0000 7f96cbe00640 10 client.28543 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96cbe00640 1 -- 172.21.15.39:0/3499461573 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ad58be8d0 con 0x556ad5539da0 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96cbe00640 10 client.28543 handle_command_reply: tid=0 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 client.28543 shutdown 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 -- 172.21.15.39:0/3499461573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad5585390 msgr2=0x556ad595ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad5585390 0x556ad595ae00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f96d80dfed0 tx=0x7f96d80b2a50 comp rx=0 tx=0).stop 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 -- 172.21.15.39:0/3499461573 shutdown_connections 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556ad5539da0 0x556ad59d3ca0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556ad55a6070 0x556ad59cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad5585390 0x556ad595ae00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 --2- 172.21.15.39:0/3499461573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556ad557d1b0 0x556ad595a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96cbe00640 0 client.28543 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:41.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 -- 172.21.15.39:0/3499461573 >> 172.21.15.39:0/3499461573 conn(0x556ad559a460 msgr2=0x556ad594b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:41.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 -- 172.21.15.39:0/3499461573 shutdown_connections 2024-07-04T08:29:41.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 1 -- 172.21.15.39:0/3499461573 wait complete. 2024-07-04T08:29:41.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.762+0000 7f96f73c3740 20 client.28543 trim_cache size 0 max 16384 2024-07-04T08:29:41.753 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:29:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 -- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f96d00b2960 msgr2=0x7f96d00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f96d00b2960 0x7f96d00b4e70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x556ad59495d0 tx=0x7f96dc009260 comp rx=0 tx=0).stop 2024-07-04T08:29:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 -- 172.21.15.39:0/2676563506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96ec10ee50 msgr2=0x7f96ec1a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96ec10ee50 0x7f96ec1a03c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f96ec06bd90 tx=0x7f96d800ee50 comp rx=0 tx=0).stop 2024-07-04T08:29:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 -- 172.21.15.39:0/2676563506 shutdown_connections 2024-07-04T08:29:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f96d00b2960 0x7f96d00b4e70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96ec10f7a0 0x7f96ec1a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96ec10ee50 0x7f96ec1a03c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 --2- 172.21.15.39:0/2676563506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96ec109510 0x7f96ec19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:41.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 -- 172.21.15.39:0/2676563506 >> 172.21.15.39:0/2676563506 conn(0x7f96ec078b10 msgr2=0x7f96ec1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:41.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 -- 172.21.15.39:0/2676563506 shutdown_connections 2024-07-04T08:29:41.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:41.763+0000 7f96eb200640 1 -- 172.21.15.39:0/2676563506 wait complete. 2024-07-04T08:29:43.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:43 smithi133 ceph-mon[37050]: pgmap v931: 129 pgs: 129 active+clean; 26 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 17 op/s 2024-07-04T08:29:43.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:43 smithi039 ceph-mon[32062]: pgmap v931: 129 pgs: 129 active+clean; 26 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 17 op/s 2024-07-04T08:29:43.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:43 smithi050 ceph-mon[36577]: pgmap v931: 129 pgs: 129 active+clean; 26 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 87 KiB/s wr, 17 op/s 2024-07-04T08:29:43.695 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:29:43.696 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:29:44.191 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:45.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:45 smithi133 ceph-mon[37050]: pgmap v932: 129 pgs: 129 active+clean; 26 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 13 op/s 2024-07-04T08:29:45.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:45 smithi039 ceph-mon[32062]: pgmap v932: 129 pgs: 129 active+clean; 26 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 13 op/s 2024-07-04T08:29:45.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:45 smithi050 ceph-mon[36577]: pgmap v932: 129 pgs: 129 active+clean; 26 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 13 op/s 2024-07-04T08:29:46.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.489+0000 7f787d400640 1 -- 172.21.15.39:0/1930117002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7878105300 msgr2=0x7f7878105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:46.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.489+0000 7f787d400640 1 --2- 172.21.15.39:0/1930117002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7878105300 0x7f7878105710 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7868009900 tx=0x7f786802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.489+0000 7f787d400640 1 -- 172.21.15.39:0/1930117002 shutdown_connections 2024-07-04T08:29:46.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.489+0000 7f787d400640 1 --2- 172.21.15.39:0/1930117002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787810b590 0x7f7878111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.489+0000 7f787d400640 1 --2- 172.21.15.39:0/1930117002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f787810ac40 0x7f787810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.489+0000 7f787d400640 1 --2- 172.21.15.39:0/1930117002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7878105300 0x7f7878105710 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.489+0000 7f787d400640 1 -- 172.21.15.39:0/1930117002 >> 172.21.15.39:0/1930117002 conn(0x7f7878100d10 msgr2=0x7f7878103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:46.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.489+0000 7f787d400640 1 -- 172.21.15.39:0/1930117002 shutdown_connections 2024-07-04T08:29:46.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.489+0000 7f787d400640 1 -- 172.21.15.39:0/1930117002 wait complete. 2024-07-04T08:29:46.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.491+0000 7f787d400640 1 Processor -- start 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.491+0000 7f787d400640 1 -- start start 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f787d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7878105300 0x7f787819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f787d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787810ac40 0x7f787819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f787d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787810b590 0x7f78781a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f787d400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7878115240 con 0x7f787810b590 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f787d400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78781150c0 con 0x7f787810ac40 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f787d400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78781153c0 con 0x7f7878105300 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f7876800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7878105300 0x7f787819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f7876800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7878105300 0x7f787819bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58368/0 (socket says 172.21.15.39:58368) 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f7876800640 1 -- 172.21.15.39:0/1367280641 learned_addr learned my addr 172.21.15.39:0/1367280641 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:46.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f786de00640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787810ac40 0x7f787819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.492+0000 7f7877200640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787810b590 0x7f78781a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.493+0000 7f7876800640 1 -- 172.21.15.39:0/1367280641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787810ac40 msgr2=0x7f787819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.493+0000 7f7876800640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787810ac40 0x7f787819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.493+0000 7f7876800640 1 -- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787810b590 msgr2=0x7f78781a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.493+0000 7f7876800640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787810b590 0x7f78781a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.493+0000 7f7876800640 1 -- 172.21.15.39:0/1367280641 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78781a1be0 con 0x7f7878105300 2024-07-04T08:29:46.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.493+0000 7f7876800640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7878105300 0x7f787819bc70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7868005940 tx=0x7f78680040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.494+0000 7f786fe00640 1 -- 172.21.15.39:0/1367280641 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786803f030 con 0x7f7878105300 2024-07-04T08:29:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.494+0000 7f787d400640 1 -- 172.21.15.39:0/1367280641 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78781a1e70 con 0x7f7878105300 2024-07-04T08:29:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.494+0000 7f786fe00640 1 -- 172.21.15.39:0/1367280641 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78680044c0 con 0x7f7878105300 2024-07-04T08:29:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.494+0000 7f786fe00640 1 -- 172.21.15.39:0/1367280641 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868035070 con 0x7f7878105300 2024-07-04T08:29:46.484 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.494+0000 7f787d400640 1 -- 172.21.15.39:0/1367280641 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78781a8630 con 0x7f7878105300 2024-07-04T08:29:46.486 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.498+0000 7f7883986740 1 Processor -- start 2024-07-04T08:29:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.498+0000 7f7883986740 1 -- start start 2024-07-04T08:29:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.498+0000 7f786fe00640 1 -- 172.21.15.39:0/1367280641 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f7868042030 con 0x7f7878105300 2024-07-04T08:29:46.487 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.498+0000 7f7883986740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.498+0000 7f7883986740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.498+0000 7f7883986740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.498+0000 7f7883986740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a4701ec8f0 con 0x55a46ff1e350 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.498+0000 7f7883986740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a470238ed0 con 0x55a46fefd670 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.498+0000 7f7883986740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a470228470 con 0x55a46fef5490 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f7876800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f7877200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f7876800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58384/0 (socket says 172.21.15.39:58384) 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f7876800640 1 -- 172.21.15.39:0/3404463408 learned_addr learned my addr 172.21.15.39:0/3404463408 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:46.488 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f786de00640 1 --2- 172.21.15.39:0/3404463408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f7876800640 1 -- 172.21.15.39:0/3404463408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 msgr2=0x55a4702c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f7876800640 1 --2- 172.21.15.39:0/3404463408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702c30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f7876800640 1 -- 172.21.15.39:0/3404463408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 msgr2=0x55a4702c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f7876800640 1 --2- 172.21.15.39:0/3404463408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f7876800640 1 -- 172.21.15.39:0/3404463408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4701c0290 con 0x55a46fef5490 2024-07-04T08:29:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.499+0000 7f786de00640 1 --2- 172.21.15.39:0/3404463408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.500+0000 7f7876800640 1 --2- 172.21.15.39:0/3404463408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702c2b90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f78680ffbf0 tx=0x7f7868108cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b6d72cd6f63bc48 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.500+0000 7f786fe00640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b29e0 0x7f785c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.489 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.500+0000 7f7867e00640 1 -- 172.21.15.39:0/3404463408 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868111070 con 0x55a46fef5490 2024-07-04T08:29:46.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.500+0000 7f786fe00640 1 -- 172.21.15.39:0/1367280641 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f7868102920 con 0x7f7878105300 2024-07-04T08:29:46.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.500+0000 7f7867e00640 1 -- 172.21.15.39:0/3404463408 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78680c9400 con 0x55a46fef5490 2024-07-04T08:29:46.490 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.500+0000 7f7867e00640 1 -- 172.21.15.39:0/3404463408 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78680c96c0 con 0x55a46fef5490 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.500+0000 7f7877200640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b29e0 0x7f785c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.501+0000 7f7883986740 1 -- 172.21.15.39:0/3404463408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 msgr2=0x55a4702c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.501+0000 7f7877200640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b29e0 0x7f785c0b4ef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7870005fd0 tx=0x7f7870009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.501+0000 7f7883986740 1 --2- 172.21.15.39:0/3404463408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702c2b90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f78680ffbf0 tx=0x7f7868108cb0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.501+0000 7f7883986740 1 -- 172.21.15.39:0/3404463408 shutdown_connections 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.501+0000 7f7883986740 1 --2- 172.21.15.39:0/3404463408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.501+0000 7f7883986740 1 --2- 172.21.15.39:0/3404463408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702c2b90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.501+0000 7f7883986740 1 --2- 172.21.15.39:0/3404463408 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.501+0000 7f7883986740 1 -- 172.21.15.39:0/3404463408 >> 172.21.15.39:0/3404463408 conn(0x55a46ff12740 msgr2=0x55a4702c0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.502+0000 7f7883986740 1 -- 172.21.15.39:0/3404463408 shutdown_connections 2024-07-04T08:29:46.491 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.502+0000 7f7883986740 1 -- 172.21.15.39:0/3404463408 wait complete. 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.502+0000 7f7883986740 1 -- start start 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.503+0000 7f7883986740 1 Processor -- start 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.503+0000 7f7883986740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.503+0000 7f7883986740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.503+0000 7f7883986740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.503+0000 7f7883986740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55a4701cf770 con 0x55a46ff1e350 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.503+0000 7f7883986740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a47022c4b0 con 0x55a46fefd670 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.503+0000 7f7883986740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a47008c890 con 0x55a46fef5490 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.503+0000 7f7883986740 10 client.0 _ll_register_callbacks cb 0x55a4701d02c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:46.494 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.503+0000 7f7877200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7877200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702d9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58508/0 (socket says 172.21.15.39:58508) 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7877200640 1 -- 172.21.15.39:0/602625949 learned_addr learned my addr 172.21.15.39:0/602625949 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7876800640 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f786de00640 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7877200640 1 -- 172.21.15.39:0/602625949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 msgr2=0x55a4702cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7877200640 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7877200640 1 -- 172.21.15.39:0/602625949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 msgr2=0x55a4702ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7877200640 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7877200640 1 -- 172.21.15.39:0/602625949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46ff89930 con 0x55a46ff1e350 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7877200640 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702d9d20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7870003fd0 tx=0x7f7870004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d4c1831c894f27 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.504+0000 7f7864c00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.505+0000 7f7864c00640 1 -- 172.21.15.39:0/602625949 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787000f8b0 con 0x55a46ff1e350 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.505+0000 7f7864c00640 1 -- 172.21.15.39:0/602625949 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f787000fa50 con 0x55a46ff1e350 2024-07-04T08:29:46.495 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.505+0000 7f7883986740 1 -- 172.21.15.39:0/602625949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78680c9400 con 0x55a46ff1e350 2024-07-04T08:29:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.505+0000 7f7864c00640 1 -- 172.21.15.39:0/602625949 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f787000fc50 con 0x55a46ff1e350 2024-07-04T08:29:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.505+0000 7f7883986740 10 client.18495 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:46.496 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.505+0000 7f7883986740 1 -- 172.21.15.39:0/602625949 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4702dc8e0 con 0x55a46ff1e350 2024-07-04T08:29:46.497 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.506+0000 7f7864c00640 1 -- 172.21.15.39:0/602625949 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78680c9400 con 0x55a46ff1e350 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.506+0000 7f7864c00640 1 -- 172.21.15.39:0/602625949 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7870026070 con 0x55a46ff1e350 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.506+0000 7f7883986740 10 client.18495 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.506+0000 7f7883986740 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.506+0000 7f7883986740 20 client.18495 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.506+0000 7f7883986740 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a46fedccd0 0x55a4702dfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.506+0000 7f7883986740 4 client.18495 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.506+0000 7f7883986740 1 -- 172.21.15.39:0/602625949 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a47023ae20 con 0x55a46fedccd0 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.507+0000 7f7876800640 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a46fedccd0 0x55a4702dfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.507+0000 7f7876800640 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a46fedccd0 0x55a4702dfe60 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8283b9992243515 server_cookie=52fdf393f10e8082 in_seq=0 out_seq=0 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.507+0000 7f7864c00640 10 client.18495 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:46.498 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.508+0000 7f7864c00640 1 -- 172.21.15.39:0/602625949 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a47023ae20 con 0x55a46fedccd0 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.508+0000 7f7864c00640 10 client.18495 handle_command_reply: tid=0 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.508+0000 7f7883986740 1 client.18495 shutdown 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.508+0000 7f7883986740 1 -- 172.21.15.39:0/602625949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 msgr2=0x55a4702d9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.508+0000 7f7883986740 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702d9d20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7870003fd0 tx=0x7f7870004ab0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.508+0000 7f7883986740 1 -- 172.21.15.39:0/602625949 shutdown_connections 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.508+0000 7f7883986740 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a46fedccd0 0x55a4702dfe60 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.509+0000 7f7883986740 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46ff1e350 0x55a4702d9d20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.509+0000 7f7883986740 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.509+0000 7f7883986740 1 --2- 172.21.15.39:0/602625949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fef5490 0x55a4702cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.509+0000 7f7864c00640 0 client.18495 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.509+0000 7f7883986740 1 -- 172.21.15.39:0/602625949 >> 172.21.15.39:0/602625949 conn(0x55a46ff12740 msgr2=0x55a4702c5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.509+0000 7f7883986740 1 -- 172.21.15.39:0/602625949 shutdown_connections 2024-07-04T08:29:46.499 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.509+0000 7f7883986740 1 -- 172.21.15.39:0/602625949 wait complete. 2024-07-04T08:29:46.500 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.509+0000 7f7883986740 20 client.18495 trim_cache size 0 max 16384 2024-07-04T08:29:46.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f7883986740 1 Processor -- start 2024-07-04T08:29:46.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f7883986740 1 -- start start 2024-07-04T08:29:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f7883986740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fef5490 0x55a4702c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.518 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f7883986740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fefd670 0x55a4702cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f7883986740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 0x55a4702d9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f7883986740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b25e0 con 0x55a46fefd670 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f7883986740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b4120 con 0x55a46fef5490 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f7883986740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b2cb0 con 0x55a46ff1e350 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f786de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fefd670 0x55a4702cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f786de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fefd670 0x55a4702cfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58514/0 (socket says 172.21.15.39:58514) 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.529+0000 7f786de00640 1 -- 172.21.15.39:0/3916152747 learned_addr learned my addr 172.21.15.39:0/3916152747 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.530+0000 7f7877200640 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 0x55a4702d9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.530+0000 7f7876800640 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fef5490 0x55a4702c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.530+0000 7f786de00640 1 -- 172.21.15.39:0/3916152747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 msgr2=0x55a4702d9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.530+0000 7f786de00640 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 0x55a4702d9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.530+0000 7f786de00640 1 -- 172.21.15.39:0/3916152747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fef5490 msgr2=0x55a4702c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:46.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.530+0000 7f786de00640 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fef5490 0x55a4702c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.530+0000 7f786de00640 1 -- 172.21.15.39:0/3916152747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4702c1300 con 0x55a46fefd670 2024-07-04T08:29:46.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.530+0000 7f7876800640 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fef5490 0x55a4702c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.530+0000 7f786de00640 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fefd670 0x55a4702cfa90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7860009e20 tx=0x7f7860005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70405a4260cbda1c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7867400640 1 -- 172.21.15.39:0/3916152747 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860010430 con 0x55a46fefd670 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7867400640 1 -- 172.21.15.39:0/3916152747 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7860021070 con 0x55a46fefd670 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7867400640 1 -- 172.21.15.39:0/3916152747 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786001c780 con 0x55a46fefd670 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7883986740 1 -- 172.21.15.39:0/3916152747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fefd670 msgr2=0x55a4702cfa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7883986740 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fefd670 0x55a4702cfa90 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7860009e20 tx=0x7f7860005f90 comp rx=0 tx=0).stop 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7883986740 1 -- 172.21.15.39:0/3916152747 shutdown_connections 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7883986740 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fefd670 0x55a4702cfa90 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7883986740 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 0x55a4702d9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7883986740 1 --2- 172.21.15.39:0/3916152747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fef5490 0x55a4702c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.531+0000 7f7883986740 1 -- 172.21.15.39:0/3916152747 >> 172.21.15.39:0/3916152747 conn(0x55a46ff12740 msgr2=0x55a4702cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:46.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.532+0000 7f7883986740 1 -- 172.21.15.39:0/3916152747 shutdown_connections 2024-07-04T08:29:46.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.532+0000 7f7883986740 1 -- 172.21.15.39:0/3916152747 wait complete. 2024-07-04T08:29:46.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.532+0000 7f7883986740 1 -- start start 2024-07-04T08:29:46.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.532+0000 7f7883986740 1 Processor -- start 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7883986740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fef5490 0x55a4702d2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7883986740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7883986740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 0x55a470341600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7883986740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b3a50 con 0x55a46fef5490 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7883986740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b3380 con 0x55a46fefd670 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7883986740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78781b05c0 con 0x55a46ff1e350 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7883986740 10 client.0 _ll_register_callbacks cb 0x7f7870003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7876800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fef5490 0x55a4702d2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f786de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7876800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fef5490 0x55a4702d2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58528/0 (socket says 172.21.15.39:58528) 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7876800640 1 -- 172.21.15.39:0/299527065 learned_addr learned my addr 172.21.15.39:0/299527065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.533+0000 7f7877200640 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 0x55a470341600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f786de00640 1 -- 172.21.15.39:0/299527065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 msgr2=0x55a470341600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f786de00640 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 0x55a470341600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f786de00640 1 -- 172.21.15.39:0/299527065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fef5490 msgr2=0x55a4702d2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f786de00640 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fef5490 0x55a4702d2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f786de00640 1 -- 172.21.15.39:0/299527065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a470341c70 con 0x55a46fefd670 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f7876800640 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fef5490 0x55a4702d2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f7877200640 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 0x55a470341600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f786de00640 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702d2a40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7860009e20 tx=0x7f78600053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41575e8618a12e5b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f7864c00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f7864c00640 1 -- 172.21.15.39:0/299527065 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78600048c0 con 0x55a46fefd670 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.534+0000 7f7883986740 1 -- 172.21.15.39:0/299527065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7860021070 con 0x55a46fefd670 2024-07-04T08:29:46.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.535+0000 7f7864c00640 1 -- 172.21.15.39:0/299527065 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f7860004a60 con 0x55a46fefd670 2024-07-04T08:29:46.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.535+0000 7f7864c00640 1 -- 172.21.15.39:0/299527065 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7860004c80 con 0x55a46fefd670 2024-07-04T08:29:46.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.535+0000 7f7883986740 10 client.28622 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:46.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.535+0000 7f7883986740 1 -- 172.21.15.39:0/299527065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a470348fd0 con 0x55a46fefd670 2024-07-04T08:29:46.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.535+0000 7f7864c00640 1 -- 172.21.15.39:0/299527065 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7860021070 con 0x55a46fefd670 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.536+0000 7f7864c00640 1 -- 172.21.15.39:0/299527065 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7860027530 con 0x55a46fefd670 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.536+0000 7f7883986740 10 client.28622 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.536+0000 7f7883986740 10 client.28622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.536+0000 7f7883986740 20 client.28622 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.536+0000 7f7883986740 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a46feb2080 0x55a47034b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.536+0000 7f7883986740 4 client.28622 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.536+0000 7f7883986740 1 -- 172.21.15.39:0/299527065 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a4702368c0 con 0x55a46feb2080 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.536+0000 7f7876800640 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a46feb2080 0x55a47034b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.537+0000 7f7876800640 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a46feb2080 0x55a47034b600 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888a0aa89b5da5ce server_cookie=920311c1c6fbc965 in_seq=0 out_seq=0 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.537+0000 7f7864c00640 10 client.28622 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.538+0000 7f7864c00640 1 -- 172.21.15.39:0/299527065 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a4702368c0 con 0x55a46feb2080 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.538+0000 7f7864c00640 10 client.28622 handle_command_reply: tid=0 2024-07-04T08:29:46.527 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.538+0000 7f7883986740 1 client.28622 shutdown 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.538+0000 7f7883986740 1 -- 172.21.15.39:0/299527065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 msgr2=0x55a4702d2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.538+0000 7f7883986740 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702d2a40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7860009e20 tx=0x7f78600053b0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7883986740 1 -- 172.21.15.39:0/299527065 shutdown_connections 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7883986740 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55a46feb2080 0x55a47034b600 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7883986740 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46ff1e350 0x55a470341600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7883986740 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a46fefd670 0x55a4702d2a40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7883986740 1 --2- 172.21.15.39:0/299527065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a46fef5490 0x55a4702d2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7883986740 1 -- 172.21.15.39:0/299527065 >> 172.21.15.39:0/299527065 conn(0x55a46ff12740 msgr2=0x55a4702c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7864c00640 0 client.28622 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7883986740 1 -- 172.21.15.39:0/299527065 shutdown_connections 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7883986740 1 -- 172.21.15.39:0/299527065 wait complete. 2024-07-04T08:29:46.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.539+0000 7f7883986740 20 client.28622 trim_cache size 0 max 16384 2024-07-04T08:29:46.530 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:29:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:29:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "f966e3bf-74f0-4a87-b47f-1b1a25fbbbfd", 2024-07-04T08:29:46.530 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:29:46.530 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:29:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 -- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b29e0 msgr2=0x7f785c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b29e0 0x7f785c0b4ef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7870005fd0 tx=0x7f7870009500 comp rx=0 tx=0).stop 2024-07-04T08:29:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 -- 172.21.15.39:0/1367280641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7878105300 msgr2=0x7f787819bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7878105300 0x7f787819bc70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7868005940 tx=0x7f78680040c0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 -- 172.21.15.39:0/1367280641 shutdown_connections 2024-07-04T08:29:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f785c0b29e0 0x7f785c0b4ef0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f787810b590 0x7f78781a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f787810ac40 0x7f787819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 --2- 172.21.15.39:0/1367280641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7878105300 0x7f787819bc70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 -- 172.21.15.39:0/1367280641 >> 172.21.15.39:0/1367280641 conn(0x7f7878100d10 msgr2=0x7f7878104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 -- 172.21.15.39:0/1367280641 shutdown_connections 2024-07-04T08:29:46.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:46.540+0000 7f787d400640 1 -- 172.21.15.39:0/1367280641 wait complete. 2024-07-04T08:29:47.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:29:47.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:47 smithi133 ceph-mon[37050]: pgmap v933: 129 pgs: 129 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 17 op/s 2024-07-04T08:29:47.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:47 smithi039 ceph-mon[32062]: pgmap v933: 129 pgs: 129 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 17 op/s 2024-07-04T08:29:47.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:47 smithi050 ceph-mon[36577]: pgmap v933: 129 pgs: 129 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 91 KiB/s wr, 17 op/s 2024-07-04T08:29:47.930 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:48.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:48 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:29:48.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:48 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:29:48.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:48 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:29:48.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:48 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:29:48.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:48 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:29:48.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:48 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:29:48.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:48 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:29:48.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:48 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:29:48.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:48 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:29:48.554 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:48 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:29:48.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:48 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:29:48.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:48 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:29:48.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:48 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:29:48.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:48 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:29:48.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:48 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:29:49.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:49 smithi133 ceph-mon[37050]: pgmap v934: 129 pgs: 129 active+clean; 27 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-07-04T08:29:49.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:49 smithi039 ceph-mon[32062]: pgmap v934: 129 pgs: 129 active+clean; 27 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-07-04T08:29:49.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:49 smithi050 ceph-mon[36577]: pgmap v934: 129 pgs: 129 active+clean; 27 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-07-04T08:29:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.302+0000 7f8efec00640 1 -- 172.21.15.39:0/3391699302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef80768d0 msgr2=0x7f8ef8074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.302+0000 7f8efec00640 1 --2- 172.21.15.39:0/3391699302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef80768d0 0x7f8ef8074d00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009900 tx=0x7f8eec02a410 comp rx=0 tx=0).stop 2024-07-04T08:29:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.302+0000 7f8efec00640 1 -- 172.21.15.39:0/3391699302 shutdown_connections 2024-07-04T08:29:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.302+0000 7f8efec00640 1 --2- 172.21.15.39:0/3391699302 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef81137b0 0x7f8ef8115c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.302+0000 7f8efec00640 1 --2- 172.21.15.39:0/3391699302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef8075320 0x7f8ef8075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.302+0000 7f8efec00640 1 --2- 172.21.15.39:0/3391699302 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef80768d0 0x7f8ef8074d00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.302+0000 7f8efec00640 1 -- 172.21.15.39:0/3391699302 >> 172.21.15.39:0/3391699302 conn(0x7f8ef80fe930 msgr2=0x7f8ef8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:50.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.302+0000 7f8efec00640 1 -- 172.21.15.39:0/3391699302 shutdown_connections 2024-07-04T08:29:50.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.302+0000 7f8efec00640 1 -- 172.21.15.39:0/3391699302 wait complete. 2024-07-04T08:29:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.303+0000 7f8efec00640 1 Processor -- start 2024-07-04T08:29:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.303+0000 7f8efec00640 1 -- start start 2024-07-04T08:29:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efec00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef8075320 0x7f8ef819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef80768d0 0x7f8ef81a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efec00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef81137b0 0x7f8ef81a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efec00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef81193e0 con 0x7f8ef8075320 2024-07-04T08:29:50.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efec00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef8119260 con 0x7f8ef81137b0 2024-07-04T08:29:50.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef8119560 con 0x7f8ef80768d0 2024-07-04T08:29:50.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef8075320 0x7f8ef819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef8075320 0x7f8ef819fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58540/0 (socket says 172.21.15.39:58540) 2024-07-04T08:29:50.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efd800640 1 -- 172.21.15.39:0/3933195186 learned_addr learned my addr 172.21.15.39:0/3933195186 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:50.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efce00640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef80768d0 0x7f8ef81a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.304+0000 7f8efe200640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef81137b0 0x7f8ef81a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.305+0000 7f8efd800640 1 -- 172.21.15.39:0/3933195186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef80768d0 msgr2=0x7f8ef81a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:50.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.305+0000 7f8efd800640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef80768d0 0x7f8ef81a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.305+0000 7f8efd800640 1 -- 172.21.15.39:0/3933195186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef81137b0 msgr2=0x7f8ef81a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.305+0000 7f8efd800640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef81137b0 0x7f8ef81a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.305+0000 7f8efd800640 1 -- 172.21.15.39:0/3933195186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef81a5dc0 con 0x7f8ef8075320 2024-07-04T08:29:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.305+0000 7f8efce00640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef80768d0 0x7f8ef81a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.306+0000 7f8efd800640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef8075320 0x7f8ef819fe50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8eec005940 tx=0x7f8eec0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.306+0000 7f8ef6000640 1 -- 172.21.15.39:0/3933195186 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec03f030 con 0x7f8ef8075320 2024-07-04T08:29:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.306+0000 7f8ef6000640 1 -- 172.21.15.39:0/3933195186 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec0044c0 con 0x7f8ef8075320 2024-07-04T08:29:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.306+0000 7f8ef6000640 1 -- 172.21.15.39:0/3933195186 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec035070 con 0x7f8ef8075320 2024-07-04T08:29:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.306+0000 7f8efec00640 1 -- 172.21.15.39:0/3933195186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef81a6050 con 0x7f8ef8075320 2024-07-04T08:29:50.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.306+0000 7f8efec00640 1 -- 172.21.15.39:0/3933195186 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef81ac810 con 0x7f8ef8075320 2024-07-04T08:29:50.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.310+0000 7f8f06849740 1 Processor -- start 2024-07-04T08:29:50.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.310+0000 7f8ef6000640 1 -- 172.21.15.39:0/3933195186 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8eec042030 con 0x7f8ef8075320 2024-07-04T08:29:50.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.310+0000 7f8f06849740 1 -- start start 2024-07-04T08:29:50.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8f06849740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3c69910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8f06849740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c69e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8f06849740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c6a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.299 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8f06849740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563dc3bdfea0 con 0x563dc38a4390 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8f06849740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563dc3bcf440 con 0x563dc38c5070 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8f06849740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563dc3bd3480 con 0x563dc389c1b0 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8efd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c69e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8efd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c69e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58454/0 (socket says 172.21.15.39:58454) 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8efd800640 1 -- 172.21.15.39:0/1661235181 learned_addr learned my addr 172.21.15.39:0/1661235181 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8efe200640 1 --2- 172.21.15.39:0/1661235181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3c69910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.311+0000 7f8efce00640 1 --2- 172.21.15.39:0/1661235181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c6a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.312+0000 7f8efd800640 1 -- 172.21.15.39:0/1661235181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 msgr2=0x563dc3c69910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.312+0000 7f8efd800640 1 --2- 172.21.15.39:0/1661235181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3c69910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.312+0000 7f8efd800640 1 -- 172.21.15.39:0/1661235181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 msgr2=0x563dc3c6a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.312+0000 7f8efd800640 1 --2- 172.21.15.39:0/1661235181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c6a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.312+0000 7f8efd800640 1 -- 172.21.15.39:0/1661235181 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dc3b67610 con 0x563dc389c1b0 2024-07-04T08:29:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.312+0000 7f8efd800640 1 --2- 172.21.15.39:0/1661235181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c69e50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0fffe0 tx=0x7f8eec106f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eff44ccfdff06c65 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.313+0000 7f8ef6000640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ee00b2960 0x7f8ee00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.313+0000 7f8efe200640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ee00b2960 0x7f8ee00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.313+0000 7f8ef6000640 1 -- 172.21.15.39:0/3933195186 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f8eec102cd0 con 0x7f8ef8075320 2024-07-04T08:29:50.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.314+0000 7f8efe200640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ee00b2960 0x7f8ee00b4e70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0005fd0 tx=0x7f8ef0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:50.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.315+0000 7f8eeaa00640 1 -- 172.21.15.39:0/1661235181 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec112070 con 0x563dc389c1b0 2024-07-04T08:29:50.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.315+0000 7f8eeaa00640 1 -- 172.21.15.39:0/1661235181 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec0c97f0 con 0x563dc389c1b0 2024-07-04T08:29:50.303 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.315+0000 7f8eeaa00640 1 -- 172.21.15.39:0/1661235181 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0c9450 con 0x563dc389c1b0 2024-07-04T08:29:50.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.316+0000 7f8f06849740 1 -- 172.21.15.39:0/1661235181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 msgr2=0x563dc3c69e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:50.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.316+0000 7f8f06849740 1 --2- 172.21.15.39:0/1661235181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c69e50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0fffe0 tx=0x7f8eec106f40 comp rx=0 tx=0).stop 2024-07-04T08:29:50.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.316+0000 7f8f06849740 1 -- 172.21.15.39:0/1661235181 shutdown_connections 2024-07-04T08:29:50.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.316+0000 7f8f06849740 1 --2- 172.21.15.39:0/1661235181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c6a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.316+0000 7f8f06849740 1 --2- 172.21.15.39:0/1661235181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c69e50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.316+0000 7f8f06849740 1 --2- 172.21.15.39:0/1661235181 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3c69910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.316+0000 7f8f06849740 1 -- 172.21.15.39:0/1661235181 >> 172.21.15.39:0/1661235181 conn(0x563dc38b9460 msgr2=0x563dc3c67800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:50.304 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.316+0000 7f8f06849740 1 -- 172.21.15.39:0/1661235181 shutdown_connections 2024-07-04T08:29:50.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.316+0000 7f8f06849740 1 -- 172.21.15.39:0/1661235181 wait complete. 2024-07-04T08:29:50.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.317+0000 7f8f06849740 1 -- start start 2024-07-04T08:29:50.305 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.317+0000 7f8f06849740 1 Processor -- start 2024-07-04T08:29:50.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8f06849740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c754b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8f06849740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8f06849740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3c81110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.306 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8f06849740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563dc3a33700 con 0x563dc38a4390 2024-07-04T08:29:50.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8f06849740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563dc3b76b70 con 0x563dc38c5070 2024-07-04T08:29:50.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8f06849740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563dc3c77eb0 con 0x563dc389c1b0 2024-07-04T08:29:50.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8f06849740 10 client.0 _ll_register_callbacks cb 0x563dc3bc8530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:50.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8efce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8efd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8efd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58466/0 (socket says 172.21.15.39:58466) 2024-07-04T08:29:50.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8efce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58566/0 (socket says 172.21.15.39:58566) 2024-07-04T08:29:50.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8efd800640 1 -- 172.21.15.39:0/3467184474 learned_addr learned my addr 172.21.15.39:0/3467184474 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:50.307 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.318+0000 7f8efe200640 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3c81110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.319+0000 7f8efce00640 1 -- 172.21.15.39:0/3467184474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 msgr2=0x563dc3c754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.319+0000 7f8efce00640 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c754b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.319+0000 7f8efce00640 1 -- 172.21.15.39:0/3467184474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 msgr2=0x563dc3c81110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.319+0000 7f8efce00640 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3c81110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.319+0000 7f8efce00640 1 -- 172.21.15.39:0/3467184474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dc3930650 con 0x563dc38a4390 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.319+0000 7f8efce00640 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c759f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4012cd0 tx=0x7f8ee400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6a7836064c2b86 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.319+0000 7f8edf400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.320+0000 7f8edf400640 1 -- 172.21.15.39:0/3467184474 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4012110 con 0x563dc38a4390 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.320+0000 7f8edf400640 1 -- 172.21.15.39:0/3467184474 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f8ee4004050 con 0x563dc38a4390 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.320+0000 7f8f06849740 1 -- 172.21.15.39:0/3467184474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eec0c97f0 con 0x563dc38a4390 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.320+0000 7f8edf400640 1 -- 172.21.15.39:0/3467184474 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ee4025050 con 0x563dc38a4390 2024-07-04T08:29:50.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.320+0000 7f8f06849740 10 client.18513 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.320+0000 7f8f06849740 1 -- 172.21.15.39:0/3467184474 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563dc3c83e90 con 0x563dc38a4390 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.320+0000 7f8edf400640 1 -- 172.21.15.39:0/3467184474 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec0c97f0 con 0x563dc38a4390 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.321+0000 7f8edf400640 1 -- 172.21.15.39:0/3467184474 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ee4010430 con 0x563dc38a4390 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.321+0000 7f8f06849740 10 client.18513 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.321+0000 7f8f06849740 10 client.18513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.321+0000 7f8f06849740 20 client.18513 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.321+0000 7f8f06849740 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563dc38839f0 0x563dc3c87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.321+0000 7f8f06849740 4 client.18513 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.321+0000 7f8f06849740 1 -- 172.21.15.39:0/3467184474 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dc3be1df0 con 0x563dc38839f0 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.321+0000 7f8efd800640 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563dc38839f0 0x563dc3c87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8efd800640 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563dc38839f0 0x563dc3c87430 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd5202a9b14bf6a server_cookie=c70d15473baed830 in_seq=0 out_seq=0 2024-07-04T08:29:50.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8edf400640 10 client.18513 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:50.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8edf400640 1 -- 172.21.15.39:0/3467184474 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563dc3be1df0 con 0x563dc38839f0 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8edf400640 10 client.18513 handle_command_reply: tid=0 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 client.18513 shutdown 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 -- 172.21.15.39:0/3467184474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 msgr2=0x563dc3c759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c759f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4012cd0 tx=0x7f8ee400f790 comp rx=0 tx=0).stop 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 -- 172.21.15.39:0/3467184474 shutdown_connections 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563dc38839f0 0x563dc3c87430 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c759f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3c81110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 --2- 172.21.15.39:0/3467184474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 -- 172.21.15.39:0/3467184474 >> 172.21.15.39:0/3467184474 conn(0x563dc38b9460 msgr2=0x563dc3c6caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8edf400640 0 client.18513 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:50.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.322+0000 7f8f06849740 1 -- 172.21.15.39:0/3467184474 shutdown_connections 2024-07-04T08:29:50.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.324+0000 7f8f06849740 1 -- 172.21.15.39:0/3467184474 wait complete. 2024-07-04T08:29:50.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.324+0000 7f8f06849740 20 client.18513 trim_cache size 0 max 16384 2024-07-04T08:29:50.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.345+0000 7f8f06849740 1 Processor -- start 2024-07-04T08:29:50.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.345+0000 7f8f06849740 1 -- start start 2024-07-04T08:29:50.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.345+0000 7f8f06849740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc389c1b0 0x563dc3c76ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.345+0000 7f8f06849740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38a4390 0x563dc3c80f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.345+0000 7f8f06849740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc38c5070 0x563dc3c81440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.345+0000 7f8f06849740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef81b6810 con 0x563dc389c1b0 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.345+0000 7f8f06849740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef81b8350 con 0x563dc38a4390 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.345+0000 7f8f06849740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef81b6ee0 con 0x563dc38c5070 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc389c1b0 0x563dc3c76ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efd800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc389c1b0 0x563dc3c76ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58578/0 (socket says 172.21.15.39:58578) 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efd800640 1 -- 172.21.15.39:0/2785607456 learned_addr learned my addr 172.21.15.39:0/2785607456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efce00640 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38a4390 0x563dc3c80f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efe200640 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc38c5070 0x563dc3c81440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efd800640 1 -- 172.21.15.39:0/2785607456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc38c5070 msgr2=0x563dc3c81440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efd800640 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc38c5070 0x563dc3c81440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efd800640 1 -- 172.21.15.39:0/2785607456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38a4390 msgr2=0x563dc3c80f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efd800640 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38a4390 0x563dc3c80f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efd800640 1 -- 172.21.15.39:0/2785607456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dc3c6c770 con 0x563dc389c1b0 2024-07-04T08:29:50.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efce00640 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38a4390 0x563dc3c80f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:29:50.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.346+0000 7f8efd800640 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc389c1b0 0x563dc3c76ee0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0c9c70 tx=0x7f8eec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2c6a41c938fa41 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:50.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.347+0000 7f8eea000640 1 -- 172.21.15.39:0/2785607456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec049040 con 0x563dc389c1b0 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.347+0000 7f8eea000640 1 -- 172.21.15.39:0/2785607456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec0d23d0 con 0x563dc389c1b0 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.347+0000 7f8eea000640 1 -- 172.21.15.39:0/2785607456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec112040 con 0x563dc389c1b0 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.348+0000 7f8f06849740 1 -- 172.21.15.39:0/2785607456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc389c1b0 msgr2=0x563dc3c76ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.348+0000 7f8f06849740 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc389c1b0 0x563dc3c76ee0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0c9c70 tx=0x7f8eec02a410 comp rx=0 tx=0).stop 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.348+0000 7f8f06849740 1 -- 172.21.15.39:0/2785607456 shutdown_connections 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.348+0000 7f8f06849740 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc38c5070 0x563dc3c81440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.348+0000 7f8f06849740 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38a4390 0x563dc3c80f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.348+0000 7f8f06849740 1 --2- 172.21.15.39:0/2785607456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc389c1b0 0x563dc3c76ee0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.348+0000 7f8f06849740 1 -- 172.21.15.39:0/2785607456 >> 172.21.15.39:0/2785607456 conn(0x563dc38b9460 msgr2=0x563dc3c767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.348+0000 7f8f06849740 1 -- 172.21.15.39:0/2785607456 shutdown_connections 2024-07-04T08:29:50.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.348+0000 7f8f06849740 1 -- 172.21.15.39:0/2785607456 wait complete. 2024-07-04T08:29:50.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.349+0000 7f8f06849740 1 -- start start 2024-07-04T08:29:50.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.349+0000 7f8f06849740 1 Processor -- start 2024-07-04T08:29:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8f06849740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c79700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8f06849740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c79c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8f06849740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3ceebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8f06849740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef81b7c80 con 0x563dc38a4390 2024-07-04T08:29:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8f06849740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef81b75b0 con 0x563dc38c5070 2024-07-04T08:29:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8f06849740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef81b47f0 con 0x563dc389c1b0 2024-07-04T08:29:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8f06849740 10 client.0 _ll_register_callbacks cb 0x7f8ee4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8efce00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c79c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8efd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c79700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8efe200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3ceebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8efe200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3ceebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33946/0 (socket says 172.21.15.39:33946) 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8efd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c79700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:58492/0 (socket says 172.21.15.39:58492) 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.350+0000 7f8efe200640 1 -- 172.21.15.39:0/3264579637 learned_addr learned my addr 172.21.15.39:0/3264579637 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.351+0000 7f8efe200640 1 -- 172.21.15.39:0/3264579637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 msgr2=0x563dc3c79700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.351+0000 7f8efe200640 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c79700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.351+0000 7f8efe200640 1 -- 172.21.15.39:0/3264579637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 msgr2=0x563dc3c79c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.351+0000 7f8efe200640 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c79c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.351+0000 7f8efe200640 1 -- 172.21.15.39:0/3264579637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dc3cef1d0 con 0x563dc38c5070 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.351+0000 7f8efe200640 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3ceebe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0004270 tx=0x7f8ef0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21835467e4280217 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:50.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.351+0000 7f8edf400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:29:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.352+0000 7f8edf400640 1 -- 172.21.15.39:0/3264579637 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000fbf0 con 0x563dc38c5070 2024-07-04T08:29:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.352+0000 7f8f06849740 1 -- 172.21.15.39:0/3264579637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563dc3bdd890 con 0x563dc38c5070 2024-07-04T08:29:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.352+0000 7f8edf400640 1 -- 172.21.15.39:0/3264579637 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f8ef000fd90 con 0x563dc38c5070 2024-07-04T08:29:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.352+0000 7f8edf400640 1 -- 172.21.15.39:0/3264579637 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ef0033050 con 0x563dc38c5070 2024-07-04T08:29:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.352+0000 7f8f06849740 10 client.28640 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.352+0000 7f8f06849740 1 -- 172.21.15.39:0/3264579637 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563dc3cf04f0 con 0x563dc38c5070 2024-07-04T08:29:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.353+0000 7f8edf400640 1 -- 172.21.15.39:0/3264579637 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563dc3bdd890 con 0x563dc38c5070 2024-07-04T08:29:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.353+0000 7f8edf400640 1 -- 172.21.15.39:0/3264579637 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ef001e400 con 0x563dc38c5070 2024-07-04T08:29:50.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.353+0000 7f8f06849740 10 client.28640 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.353+0000 7f8f06849740 10 client.28640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.353+0000 7f8f06849740 20 client.28640 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.353+0000 7f8f06849740 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563dc3858da0 0x563dc3cf2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.353+0000 7f8f06849740 4 client.28640 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.353+0000 7f8f06849740 1 -- 172.21.15.39:0/3264579637 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563dc3bdafd0 con 0x563dc3858da0 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.353+0000 7f8efd800640 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563dc3858da0 0x563dc3cf2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.354+0000 7f8efd800640 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563dc3858da0 0x563dc3cf2d10 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9abe329a0ee1d8c9 server_cookie=14218ee4fdd2d37f in_seq=0 out_seq=0 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.354+0000 7f8edf400640 10 client.28640 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.354+0000 7f8edf400640 1 -- 172.21.15.39:0/3264579637 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563dc3bdafd0 con 0x563dc3858da0 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.354+0000 7f8edf400640 10 client.28640 handle_command_reply: tid=0 2024-07-04T08:29:50.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.354+0000 7f8f06849740 1 client.28640 shutdown 2024-07-04T08:29:50.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 -- 172.21.15.39:0/3264579637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 msgr2=0x563dc3ceebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:50.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3ceebe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0004270 tx=0x7f8ef0004a40 comp rx=0 tx=0).stop 2024-07-04T08:29:50.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 -- 172.21.15.39:0/3264579637 shutdown_connections 2024-07-04T08:29:50.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563dc3858da0 0x563dc3cf2d10 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563dc38a4390 0x563dc3c79c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563dc38c5070 0x563dc3ceebe0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8edf400640 0 client.28640 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 --2- 172.21.15.39:0/3264579637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563dc389c1b0 0x563dc3c79700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 -- 172.21.15.39:0/3264579637 >> 172.21.15.39:0/3264579637 conn(0x563dc38b9460 msgr2=0x563dc3c6b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 -- 172.21.15.39:0/3264579637 shutdown_connections 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 1 -- 172.21.15.39:0/3264579637 wait complete. 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.355+0000 7f8f06849740 20 client.28640 trim_cache size 0 max 16384 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:29:50.345 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:29:50.346 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:29:50.346 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:29:50.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.356+0000 7f8efec00640 1 -- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ee00b2960 msgr2=0x7f8ee00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:50.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.356+0000 7f8efec00640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ee00b2960 0x7f8ee00b4e70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0005fd0 tx=0x7f8ef0009500 comp rx=0 tx=0).stop 2024-07-04T08:29:50.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 -- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef8075320 msgr2=0x7f8ef819fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:50.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef8075320 0x7f8ef819fe50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8eec005940 tx=0x7f8eec0040c0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 -- 172.21.15.39:0/3933195186 shutdown_connections 2024-07-04T08:29:50.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f8ee00b2960 0x7f8ee00b4e70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8ef81137b0 0x7f8ef81a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ef80768d0 0x7f8ef81a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 --2- 172.21.15.39:0/3933195186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ef8075320 0x7f8ef819fe50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:50.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 -- 172.21.15.39:0/3933195186 >> 172.21.15.39:0/3933195186 conn(0x7f8ef80fe930 msgr2=0x7f8ef810f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:50.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 -- 172.21.15.39:0/3933195186 shutdown_connections 2024-07-04T08:29:50.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:50.357+0000 7f8efec00640 1 -- 172.21.15.39:0/3933195186 wait complete. 2024-07-04T08:29:51.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:29:51.125 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:29:51.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:51 smithi039 ceph-mon[32062]: pgmap v935: 129 pgs: 129 active+clean; 27 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-07-04T08:29:51.616 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:51.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:51 smithi050 ceph-mon[36577]: pgmap v935: 129 pgs: 129 active+clean; 27 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-07-04T08:29:51.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:51 smithi133 ceph-mon[37050]: pgmap v935: 129 pgs: 129 active+clean; 27 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 15 op/s 2024-07-04T08:29:53.481 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:53 smithi039 ceph-mon[32062]: pgmap v936: 129 pgs: 129 active+clean; 27 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 18 op/s 2024-07-04T08:29:53.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:53 smithi050 ceph-mon[36577]: pgmap v936: 129 pgs: 129 active+clean; 27 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 18 op/s 2024-07-04T08:29:53.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:53 smithi133 ceph-mon[37050]: pgmap v936: 129 pgs: 129 active+clean; 27 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 93 KiB/s wr, 18 op/s 2024-07-04T08:29:54.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.064+0000 7ff4e1800640 1 -- 172.21.15.39:0/416187828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4dc102ee0 msgr2=0x7ff4dc1032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:54.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.064+0000 7ff4e1800640 1 --2- 172.21.15.39:0/416187828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4dc102ee0 0x7ff4dc1032f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009860 tx=0x7ff4cc02a340 comp rx=0 tx=0).stop 2024-07-04T08:29:54.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.064+0000 7ff4e1800640 1 -- 172.21.15.39:0/416187828 shutdown_connections 2024-07-04T08:29:54.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.064+0000 7ff4e1800640 1 --2- 172.21.15.39:0/416187828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4dc108820 0x7ff4dc108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:54.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.064+0000 7ff4e1800640 1 --2- 172.21.15.39:0/416187828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4dc109170 0x7ff4dc10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:54.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.064+0000 7ff4e1800640 1 --2- 172.21.15.39:0/416187828 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4dc102ee0 0x7ff4dc1032f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:54.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.064+0000 7ff4e1800640 1 -- 172.21.15.39:0/416187828 >> 172.21.15.39:0/416187828 conn(0x7ff4dc0fe930 msgr2=0x7ff4dc100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:54.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.065+0000 7ff4e1800640 1 -- 172.21.15.39:0/416187828 shutdown_connections 2024-07-04T08:29:54.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.065+0000 7ff4e1800640 1 -- 172.21.15.39:0/416187828 wait complete. 2024-07-04T08:29:54.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.066+0000 7ff4e1800640 1 Processor -- start 2024-07-04T08:29:54.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.066+0000 7ff4e1800640 1 -- start start 2024-07-04T08:29:54.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.066+0000 7ff4e1800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4dc102ee0 0x7ff4dc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:54.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.066+0000 7ff4e1800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4dc108820 0x7ff4dc19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:54.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.066+0000 7ff4e1800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4dc109170 0x7ff4dc1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.066+0000 7ff4e1800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112e30 con 0x7ff4dc108820 2024-07-04T08:29:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4e1800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112cb0 con 0x7ff4dc109170 2024-07-04T08:29:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4e1800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112fb0 con 0x7ff4dc102ee0 2024-07-04T08:29:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4dc102ee0 0x7ff4dc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4dc108820 0x7ff4dc19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:54.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4dbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4dc102ee0 0x7ff4dc19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:48662/0 (socket says 172.21.15.39:48662) 2024-07-04T08:29:54.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4e0e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4dc109170 0x7ff4dc1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:54.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4dbe00640 1 -- 172.21.15.39:0/148483839 learned_addr learned my addr 172.21.15.39:0/148483839 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:54.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4dbe00640 1 -- 172.21.15.39:0/148483839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4dc109170 msgr2=0x7ff4dc1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:54.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4dbe00640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4dc109170 0x7ff4dc1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:54.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4dbe00640 1 -- 172.21.15.39:0/148483839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4dc108820 msgr2=0x7ff4dc19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:54.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4dbe00640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4dc108820 0x7ff4dc19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:54.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.067+0000 7ff4dbe00640 1 -- 172.21.15.39:0/148483839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc1a19d0 con 0x7ff4dc102ee0 2024-07-04T08:29:54.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.068+0000 7ff4dbe00640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4dc102ee0 0x7ff4dc19ba60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc035040 tx=0x7ff4cc032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:54.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.068+0000 7ff4d8c00640 1 -- 172.21.15.39:0/148483839 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc037040 con 0x7ff4dc102ee0 2024-07-04T08:29:54.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.068+0000 7ff4d8c00640 1 -- 172.21.15.39:0/148483839 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4cc041070 con 0x7ff4dc102ee0 2024-07-04T08:29:54.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.068+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4dc1a1c60 con 0x7ff4dc102ee0 2024-07-04T08:29:54.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.068+0000 7ff4d8c00640 1 -- 172.21.15.39:0/148483839 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0035f0 con 0x7ff4dc102ee0 2024-07-04T08:29:54.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.068+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4dc1a8420 con 0x7ff4dc102ee0 2024-07-04T08:29:54.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.069+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4dc1032f0 con 0x7ff4dc102ee0 2024-07-04T08:29:54.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.072+0000 7ff4d8c00640 1 -- 172.21.15.39:0/148483839 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff4cc004050 con 0x7ff4dc102ee0 2024-07-04T08:29:54.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.074+0000 7ff4d8c00640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4c00b29e0 0x7ff4c00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.074+0000 7ff4d8c00640 1 -- 172.21.15.39:0/148483839 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7ff4cc102db0 con 0x7ff4dc102ee0 2024-07-04T08:29:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.074+0000 7ff4db400640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4c00b29e0 0x7ff4c00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:54.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.075+0000 7ff4db400640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4c00b29e0 0x7ff4c00b4ef0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc104f70 tx=0x7ff4c8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:54.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.076+0000 7ff4d8c00640 1 -- 172.21.15.39:0/148483839 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7ff4cc04b020 con 0x7ff4dc102ee0 2024-07-04T08:29:54.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.328+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4dc1a8650 con 0x7ff4dc102ee0 2024-07-04T08:29:54.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.337+0000 7ff4d8c00640 1 -- 172.21.15.39:0/148483839 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7ff4cc0ca4e0 con 0x7ff4dc102ee0 2024-07-04T08:29:54.326 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:29:54.326 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:29:54.326 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:29:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4c00b29e0 msgr2=0x7ff4c00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4c00b29e0 0x7ff4c00b4ef0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc104f70 tx=0x7ff4c8009260 comp rx=0 tx=0).stop 2024-07-04T08:29:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4dc102ee0 msgr2=0x7ff4dc19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:54.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4dc102ee0 0x7ff4dc19ba60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc035040 tx=0x7ff4cc032d50 comp rx=0 tx=0).stop 2024-07-04T08:29:54.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 shutdown_connections 2024-07-04T08:29:54.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff4c00b29e0 0x7ff4c00b4ef0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:54.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4dc108820 0x7ff4dc19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:54.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff4dc109170 0x7ff4dc1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:54.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 --2- 172.21.15.39:0/148483839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4dc102ee0 0x7ff4dc19ba60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:54.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.340+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 >> 172.21.15.39:0/148483839 conn(0x7ff4dc0fe930 msgr2=0x7ff4dc107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:54.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.341+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 shutdown_connections 2024-07-04T08:29:54.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:54.341+0000 7ff4e1800640 1 -- 172.21.15.39:0/148483839 wait complete. 2024-07-04T08:29:54.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:54 smithi039 ceph-mon[32062]: pgmap v937: 129 pgs: 129 active+clean; 27 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-07-04T08:29:54.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:54 smithi050 ceph-mon[36577]: pgmap v937: 129 pgs: 129 active+clean; 27 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-07-04T08:29:54.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:54 smithi133 ceph-mon[37050]: pgmap v937: 129 pgs: 129 active+clean; 27 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 14 op/s 2024-07-04T08:29:55.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:29:55.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:55 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/148483839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:29:55.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:55 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/148483839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:29:55.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:55 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/148483839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:29:55.803 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:56.435 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:56 smithi039 ceph-mon[32062]: pgmap v938: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 98 KiB/s wr, 17 op/s 2024-07-04T08:29:56.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:56 smithi050 ceph-mon[36577]: pgmap v938: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 98 KiB/s wr, 17 op/s 2024-07-04T08:29:56.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:56 smithi133 ceph-mon[37050]: pgmap v938: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 98 KiB/s wr, 17 op/s 2024-07-04T08:29:58.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.293+0000 7fe841c00640 1 -- 172.21.15.39:0/417352666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 msgr2=0x7fe83c104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:58.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.293+0000 7fe841c00640 1 --2- 172.21.15.39:0/417352666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 0x7fe83c104120 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009900 tx=0x7fe82c02a410 comp rx=0 tx=0).stop 2024-07-04T08:29:58.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.293+0000 7fe841c00640 1 -- 172.21.15.39:0/417352666 shutdown_connections 2024-07-04T08:29:58.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.293+0000 7fe841c00640 1 --2- 172.21.15.39:0/417352666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c10d170 0x7fe83c10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.293+0000 7fe841c00640 1 --2- 172.21.15.39:0/417352666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c104660 0x7fe83c10cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.293+0000 7fe841c00640 1 --2- 172.21.15.39:0/417352666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 0x7fe83c104120 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.293+0000 7fe841c00640 1 -- 172.21.15.39:0/417352666 >> 172.21.15.39:0/417352666 conn(0x7fe83c0fd810 msgr2=0x7fe83c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:58.281 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.293+0000 7fe841c00640 1 -- 172.21.15.39:0/417352666 shutdown_connections 2024-07-04T08:29:58.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.293+0000 7fe841c00640 1 -- 172.21.15.39:0/417352666 wait complete. 2024-07-04T08:29:58.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.294+0000 7fe841c00640 1 Processor -- start 2024-07-04T08:29:58.282 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.294+0000 7fe841c00640 1 -- start start 2024-07-04T08:29:58.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe841c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 0x7fe83c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe841c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c104660 0x7fe83c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe841c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c10d170 0x7fe83c1a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe841c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c112dd0 con 0x7fe83c103d10 2024-07-04T08:29:58.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe841c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c112c50 con 0x7fe83c10d170 2024-07-04T08:29:58.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe841c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c112f50 con 0x7fe83c104660 2024-07-04T08:29:58.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe83be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 0x7fe83c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.283 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe83be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 0x7fe83c19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60920/0 (socket says 172.21.15.39:60920) 2024-07-04T08:29:58.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe83be00640 1 -- 172.21.15.39:0/3997408080 learned_addr learned my addr 172.21.15.39:0/3997408080 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:58.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.295+0000 7fe841200640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c10d170 0x7fe83c1a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.284 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.296+0000 7fe83b400640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c104660 0x7fe83c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.296+0000 7fe841200640 1 -- 172.21.15.39:0/3997408080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c104660 msgr2=0x7fe83c19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.296+0000 7fe841200640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c104660 0x7fe83c19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.296+0000 7fe841200640 1 -- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 msgr2=0x7fe83c19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.296+0000 7fe841200640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 0x7fe83c19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.296+0000 7fe841200640 1 -- 172.21.15.39:0/3997408080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c1a19b0 con 0x7fe83c10d170 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.296+0000 7fe83be00640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 0x7fe83c19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.296+0000 7fe841200640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c10d170 0x7fe83c1a1280 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe83400e970 tx=0x7fe83400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.297+0000 7fe838c00640 1 -- 172.21.15.39:0/3997408080 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83400ccc0 con 0x7fe83c10d170 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.297+0000 7fe841c00640 1 -- 172.21.15.39:0/3997408080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c1a1ca0 con 0x7fe83c10d170 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.297+0000 7fe838c00640 1 -- 172.21.15.39:0/3997408080 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe834005020 con 0x7fe83c10d170 2024-07-04T08:29:58.285 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.297+0000 7fe838c00640 1 -- 172.21.15.39:0/3997408080 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834004a00 con 0x7fe83c10d170 2024-07-04T08:29:58.286 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.297+0000 7fe841c00640 1 -- 172.21.15.39:0/3997408080 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c1a8400 con 0x7fe83c10d170 2024-07-04T08:29:58.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.301+0000 7fe84989f740 1 Processor -- start 2024-07-04T08:29:58.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.301+0000 7fe838c00640 1 -- 172.21.15.39:0/3997408080 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe834004ba0 con 0x7fe83c10d170 2024-07-04T08:29:58.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.301+0000 7fe84989f740 1 -- start start 2024-07-04T08:29:58.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe84989f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce34f070 0x556dce6f3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe84989f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe84989f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 0x556dce6f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe84989f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556dce669f00 con 0x556dce32e390 2024-07-04T08:29:58.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe84989f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556dce6594a0 con 0x556dce3261b0 2024-07-04T08:29:58.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe84989f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556dce65d4e0 con 0x556dce34f070 2024-07-04T08:29:58.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe83be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe83be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:39366/0 (socket says 172.21.15.39:39366) 2024-07-04T08:29:58.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe83be00640 1 -- 172.21.15.39:0/909476289 learned_addr learned my addr 172.21.15.39:0/909476289 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:58.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe83b400640 1 --2- 172.21.15.39:0/909476289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce34f070 0x556dce6f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.302+0000 7fe83b400640 1 --2- 172.21.15.39:0/909476289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 0x556dce6f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.303+0000 7fe83be00640 1 -- 172.21.15.39:0/909476289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce34f070 msgr2=0x556dce6f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:58.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.303+0000 7fe83be00640 1 --2- 172.21.15.39:0/909476289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce34f070 0x556dce6f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.303+0000 7fe83be00640 1 -- 172.21.15.39:0/909476289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 msgr2=0x556dce6f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:58.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.303+0000 7fe83be00640 1 --2- 172.21.15.39:0/909476289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 0x556dce6f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.303+0000 7fe83be00640 1 -- 172.21.15.39:0/909476289 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dce5f1670 con 0x556dce3261b0 2024-07-04T08:29:58.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.303+0000 7fe838c00640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe81c0b2960 0x7fe81c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.303+0000 7fe83be00640 1 --2- 172.21.15.39:0/909476289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6f3eb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe82c000c00 tx=0x7fe82c005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f41080a3fe9ce7e4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:58.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.303+0000 7fe838c00640 1 -- 172.21.15.39:0/3997408080 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7fe8340e2940 con 0x7fe83c10d170 2024-07-04T08:29:58.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.303+0000 7fe83b400640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe81c0b2960 0x7fe81c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.304+0000 7fe831600640 1 -- 172.21.15.39:0/909476289 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c035040 con 0x556dce3261b0 2024-07-04T08:29:58.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.304+0000 7fe831600640 1 -- 172.21.15.39:0/909476289 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c031820 con 0x556dce3261b0 2024-07-04T08:29:58.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.304+0000 7fe831600640 1 -- 172.21.15.39:0/909476289 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c004a00 con 0x556dce3261b0 2024-07-04T08:29:58.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe84989f740 1 -- 172.21.15.39:0/909476289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 msgr2=0x556dce6f3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:58.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe83b400640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe81c0b2960 0x7fe81c0b4e70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe824009a00 tx=0x7fe824009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:58.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe84989f740 1 --2- 172.21.15.39:0/909476289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6f3eb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe82c000c00 tx=0x7fe82c005ed0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe84989f740 1 -- 172.21.15.39:0/909476289 shutdown_connections 2024-07-04T08:29:58.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe84989f740 1 --2- 172.21.15.39:0/909476289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 0x556dce6f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe84989f740 1 --2- 172.21.15.39:0/909476289 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6f3eb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe84989f740 1 --2- 172.21.15.39:0/909476289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce34f070 0x556dce6f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe84989f740 1 -- 172.21.15.39:0/909476289 >> 172.21.15.39:0/909476289 conn(0x556dce343460 msgr2=0x556dce6f1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:58.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe84989f740 1 -- 172.21.15.39:0/909476289 shutdown_connections 2024-07-04T08:29:58.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.305+0000 7fe84989f740 1 -- 172.21.15.39:0/909476289 wait complete. 2024-07-04T08:29:58.294 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.306+0000 7fe84989f740 1 -- start start 2024-07-04T08:29:58.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.306+0000 7fe84989f740 1 Processor -- start 2024-07-04T08:29:58.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe84989f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6ff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe84989f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce32e390 0x556dce6ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe84989f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 0x556dce70b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe84989f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556dce4bd700 con 0x556dce34f070 2024-07-04T08:29:58.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe84989f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556dce600bd0 con 0x556dce3261b0 2024-07-04T08:29:58.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe84989f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556dce7020f0 con 0x556dce32e390 2024-07-04T08:29:58.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe84989f740 10 client.0 _ll_register_callbacks cb 0x556dce652590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:58.296 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe841200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 0x556dce70b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe83be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe83b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce32e390 0x556dce6ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe83be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6ff490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:39374/0 (socket says 172.21.15.39:39374) 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe83be00640 1 -- 172.21.15.39:0/1289696093 learned_addr learned my addr 172.21.15.39:0/1289696093 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe841200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 0x556dce70b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60944/0 (socket says 172.21.15.39:60944) 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.307+0000 7fe83be00640 1 -- 172.21.15.39:0/1289696093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce32e390 msgr2=0x556dce6ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe83be00640 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce32e390 0x556dce6ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe83be00640 1 -- 172.21.15.39:0/1289696093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 msgr2=0x556dce70b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe83be00640 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 0x556dce70b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe83be00640 1 -- 172.21.15.39:0/1289696093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dce3ba650 con 0x556dce3261b0 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe83b400640 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce32e390 0x556dce6ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:58.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe83be00640 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6ff490 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe82c004320 tx=0x7fe82c0054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdde897b42b0c026 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:58.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe82a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:29:58.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe82a000640 1 -- 172.21.15.39:0/1289696093 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c04f060 con 0x556dce3261b0 2024-07-04T08:29:58.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe82a000640 1 -- 172.21.15.39:0/1289696093 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7fe82c003800 con 0x556dce3261b0 2024-07-04T08:29:58.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.308+0000 7fe84989f740 1 -- 172.21.15.39:0/1289696093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe82c031820 con 0x556dce3261b0 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe82a000640 1 -- 172.21.15.39:0/1289696093 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe82c056050 con 0x556dce3261b0 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe84989f740 10 client.28658 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe84989f740 1 -- 172.21.15.39:0/1289696093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556dce70e090 con 0x556dce3261b0 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe82a000640 1 -- 172.21.15.39:0/1289696093 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c031820 con 0x556dce3261b0 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe82a000640 1 -- 172.21.15.39:0/1289696093 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe82c046830 con 0x556dce3261b0 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe84989f740 10 client.28658 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe84989f740 10 client.28658 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe84989f740 20 client.28658 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe84989f740 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x556dce30d9f0 0x556dce7113f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.300 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe84989f740 4 client.28658 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.309+0000 7fe84989f740 1 -- 172.21.15.39:0/1289696093 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dce66be50 con 0x556dce30d9f0 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.310+0000 7fe83b400640 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x556dce30d9f0 0x556dce7113f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.310+0000 7fe83b400640 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x556dce30d9f0 0x556dce7113f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9330918866854d7b server_cookie=f315da0bf2e8f6f3 in_seq=0 out_seq=0 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.310+0000 7fe82a000640 10 client.28658 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.311+0000 7fe82a000640 1 -- 172.21.15.39:0/1289696093 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556dce66be50 con 0x556dce30d9f0 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.311+0000 7fe82a000640 10 client.28658 handle_command_reply: tid=0 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.311+0000 7fe84989f740 1 client.28658 shutdown 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 -- 172.21.15.39:0/1289696093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 msgr2=0x556dce6ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6ff490 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe82c004320 tx=0x7fe82c0054e0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 -- 172.21.15.39:0/1289696093 shutdown_connections 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x556dce30d9f0 0x556dce7113f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 0x556dce70b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.301 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce32e390 0x556dce6ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe82a000640 0 client.28658 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:29:58.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 --2- 172.21.15.39:0/1289696093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce3261b0 0x556dce6ff490 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 -- 172.21.15.39:0/1289696093 >> 172.21.15.39:0/1289696093 conn(0x556dce343460 msgr2=0x556dce6f6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:58.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 -- 172.21.15.39:0/1289696093 shutdown_connections 2024-07-04T08:29:58.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 1 -- 172.21.15.39:0/1289696093 wait complete. 2024-07-04T08:29:58.302 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.312+0000 7fe84989f740 20 client.28658 trim_cache size 0 max 16384 2024-07-04T08:29:58.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.328+0000 7fe84989f740 1 Processor -- start 2024-07-04T08:29:58.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.328+0000 7fe84989f740 1 -- start start 2024-07-04T08:29:58.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.328+0000 7fe84989f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce6f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.328+0000 7fe84989f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 0x556dce700ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.328+0000 7fe84989f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce34f070 0x556dce70b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.328+0000 7fe84989f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1b2410 con 0x556dce32e390 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.328+0000 7fe84989f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1b3f50 con 0x556dce34f070 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.328+0000 7fe84989f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1b2ae0 con 0x556dce3261b0 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe83b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 0x556dce700ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe83be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce6f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe83b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 0x556dce700ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:60950/0 (socket says 172.21.15.39:60950) 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe83b400640 1 -- 172.21.15.39:0/2225031090 learned_addr learned my addr 172.21.15.39:0/2225031090 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:58.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe841200640 1 --2- 172.21.15.39:0/2225031090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce34f070 0x556dce70b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe83be00640 1 -- 172.21.15.39:0/2225031090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce34f070 msgr2=0x556dce70b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:58.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe83be00640 1 --2- 172.21.15.39:0/2225031090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce34f070 0x556dce70b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe83be00640 1 -- 172.21.15.39:0/2225031090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 msgr2=0x556dce700ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:58.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe83be00640 1 --2- 172.21.15.39:0/2225031090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 0x556dce700ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.329+0000 7fe83be00640 1 -- 172.21.15.39:0/2225031090 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dce6f2150 con 0x556dce3261b0 2024-07-04T08:29:58.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.330+0000 7fe83be00640 1 --2- 172.21.15.39:0/2225031090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce6f9a90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe82c046da0 tx=0x7fe82c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b81d7a304e274a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:58.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.330+0000 7fe830c00640 1 -- 172.21.15.39:0/2225031090 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c05f070 con 0x556dce3261b0 2024-07-04T08:29:58.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.330+0000 7fe830c00640 1 -- 172.21.15.39:0/2225031090 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c046450 con 0x556dce3261b0 2024-07-04T08:29:58.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.330+0000 7fe830c00640 1 -- 172.21.15.39:0/2225031090 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c04f070 con 0x556dce3261b0 2024-07-04T08:29:58.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.331+0000 7fe84989f740 1 -- 172.21.15.39:0/2225031090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 msgr2=0x556dce6f9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:58.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.331+0000 7fe84989f740 1 --2- 172.21.15.39:0/2225031090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce6f9a90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe82c046da0 tx=0x7fe82c001e70 comp rx=0 tx=0).stop 2024-07-04T08:29:58.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.331+0000 7fe84989f740 1 -- 172.21.15.39:0/2225031090 shutdown_connections 2024-07-04T08:29:58.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.331+0000 7fe84989f740 1 --2- 172.21.15.39:0/2225031090 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce32e390 0x556dce700ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.331+0000 7fe84989f740 1 --2- 172.21.15.39:0/2225031090 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce34f070 0x556dce70b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.331+0000 7fe84989f740 1 --2- 172.21.15.39:0/2225031090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce6f9a90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.331+0000 7fe84989f740 1 -- 172.21.15.39:0/2225031090 >> 172.21.15.39:0/2225031090 conn(0x556dce343460 msgr2=0x556dce7007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:58.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.332+0000 7fe84989f740 1 -- 172.21.15.39:0/2225031090 shutdown_connections 2024-07-04T08:29:58.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.332+0000 7fe84989f740 1 -- 172.21.15.39:0/2225031090 wait complete. 2024-07-04T08:29:58.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.332+0000 7fe84989f740 1 -- start start 2024-07-04T08:29:58.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.333+0000 7fe84989f740 1 Processor -- start 2024-07-04T08:29:58.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.333+0000 7fe84989f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce711a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.333+0000 7fe84989f740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce32e390 0x556dce711f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.333+0000 7fe84989f740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 0x556dce777930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.333+0000 7fe84989f740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1b3880 con 0x556dce34f070 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.333+0000 7fe84989f740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1b31b0 con 0x556dce32e390 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.333+0000 7fe84989f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c1b03f0 con 0x556dce3261b0 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.333+0000 7fe84989f740 10 client.0 _ll_register_callbacks cb 0x7fe82c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce711a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce711a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:48724/0 (socket says 172.21.15.39:48724) 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83be00640 1 -- 172.21.15.39:0/2881402141 learned_addr learned my addr 172.21.15.39:0/2881402141 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83b400640 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce32e390 0x556dce711f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe841200640 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 0x556dce777930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83b400640 1 -- 172.21.15.39:0/2881402141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 msgr2=0x556dce711a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:58.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83b400640 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce711a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83b400640 1 -- 172.21.15.39:0/2881402141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 msgr2=0x556dce777930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:29:58.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83b400640 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 0x556dce777930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83b400640 1 -- 172.21.15.39:0/2881402141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dce777e70 con 0x556dce32e390 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.334+0000 7fe83be00640 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce711a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.335+0000 7fe83b400640 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce32e390 0x556dce711f80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x556dce701b90 tx=0x7fe824009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=844f11f7b50f2b4d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.335+0000 7fe82a000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.335+0000 7fe82a000640 1 -- 172.21.15.39:0/2881402141 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82400fac0 con 0x556dce32e390 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.335+0000 7fe84989f740 1 -- 172.21.15.39:0/2881402141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe82c046450 con 0x556dce32e390 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.335+0000 7fe82a000640 1 -- 172.21.15.39:0/2881402141 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7fe82400fc60 con 0x556dce32e390 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.335+0000 7fe82a000640 1 -- 172.21.15.39:0/2881402141 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe82403e050 con 0x556dce32e390 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.335+0000 7fe84989f740 10 client.28664 fetch_fsmap learned FSMap version 30 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.335+0000 7fe84989f740 1 -- 172.21.15.39:0/2881402141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556dce779150 con 0x556dce32e390 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.336+0000 7fe82a000640 1 -- 172.21.15.39:0/2881402141 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82c046450 con 0x556dce32e390 2024-07-04T08:29:58.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.336+0000 7fe82a000640 1 -- 172.21.15.39:0/2881402141 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe8240354f0 con 0x556dce32e390 2024-07-04T08:29:58.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.336+0000 7fe84989f740 10 client.28664 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:29:58.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.336+0000 7fe84989f740 10 client.28664 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:29:58.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.336+0000 7fe84989f740 20 client.28664 populate_metadata read hostname 'smithi039' 2024-07-04T08:29:58.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.336+0000 7fe84989f740 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x556dce2e2da0 0x556dce77b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:29:58.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.336+0000 7fe84989f740 4 client.28664 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:29:58.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.336+0000 7fe84989f740 1 -- 172.21.15.39:0/2881402141 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556dce6678f0 con 0x556dce2e2da0 2024-07-04T08:29:58.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.337+0000 7fe83be00640 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x556dce2e2da0 0x556dce77b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:29:58.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.337+0000 7fe82a000640 10 client.28664 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:29:58.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.337+0000 7fe83be00640 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x556dce2e2da0 0x556dce77b8c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e7e914af4f1be63 server_cookie=66394fe0607267d9 in_seq=0 out_seq=0 2024-07-04T08:29:58.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe82a000640 1 -- 172.21.15.39:0/2881402141 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556dce6678f0 con 0x556dce2e2da0 2024-07-04T08:29:58.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe82a000640 10 client.28664 handle_command_reply: tid=0 2024-07-04T08:29:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 client.28664 shutdown 2024-07-04T08:29:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 -- 172.21.15.39:0/2881402141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce32e390 msgr2=0x556dce711f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce32e390 0x556dce711f80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x556dce701b90 tx=0x7fe824009990 comp rx=0 tx=0).stop 2024-07-04T08:29:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 -- 172.21.15.39:0/2881402141 shutdown_connections 2024-07-04T08:29:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x556dce2e2da0 0x556dce77b8c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556dce34f070 0x556dce777930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556dce32e390 0x556dce711f80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 --2- 172.21.15.39:0/2881402141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556dce3261b0 0x556dce711a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 -- 172.21.15.39:0/2881402141 >> 172.21.15.39:0/2881402141 conn(0x556dce343460 msgr2=0x556dce6efe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:58.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe82a000640 0 client.28664 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:29:58.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 -- 172.21.15.39:0/2881402141 shutdown_connections 2024-07-04T08:29:58.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 1 -- 172.21.15.39:0/2881402141 wait complete. 2024-07-04T08:29:58.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.338+0000 7fe84989f740 20 client.28664 trim_cache size 0 max 16384 2024-07-04T08:29:58.328 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:29:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 -- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe81c0b2960 msgr2=0x7fe81c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe81c0b2960 0x7fe81c0b4e70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe824009a00 tx=0x7fe824009500 comp rx=0 tx=0).stop 2024-07-04T08:29:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 -- 172.21.15.39:0/3997408080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c10d170 msgr2=0x7fe83c1a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:29:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c10d170 0x7fe83c1a1280 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe83400e970 tx=0x7fe83400ee30 comp rx=0 tx=0).stop 2024-07-04T08:29:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 -- 172.21.15.39:0/3997408080 shutdown_connections 2024-07-04T08:29:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe81c0b2960 0x7fe81c0b4e70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe83c10d170 0x7fe83c1a1280 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c104660 0x7fe83c19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 --2- 172.21.15.39:0/3997408080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe83c103d10 0x7fe83c19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:29:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.339+0000 7fe841c00640 1 -- 172.21.15.39:0/3997408080 >> 172.21.15.39:0/3997408080 conn(0x7fe83c0fd810 msgr2=0x7fe83c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:29:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.340+0000 7fe841c00640 1 -- 172.21.15.39:0/3997408080 shutdown_connections 2024-07-04T08:29:58.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:29:58.340+0000 7fe841c00640 1 -- 172.21.15.39:0/3997408080 wait complete. 2024-07-04T08:29:59.144 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:29:59.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:29:59 smithi039 ceph-mon[32062]: pgmap v939: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-07-04T08:29:59.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:29:59 smithi133 ceph-mon[37050]: pgmap v939: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-07-04T08:29:59.629 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:29:59.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:29:59 smithi050 ceph-mon[36577]: pgmap v939: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-07-04T08:30:00.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:00 smithi039 ceph-mon[32062]: overall HEALTH_OK 2024-07-04T08:30:00.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:00 smithi133 ceph-mon[37050]: overall HEALTH_OK 2024-07-04T08:30:00.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:00 smithi050 ceph-mon[36577]: overall HEALTH_OK 2024-07-04T08:30:01.338 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:01 smithi039 ceph-mon[32062]: pgmap v940: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-07-04T08:30:01.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:01 smithi133 ceph-mon[37050]: pgmap v940: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-07-04T08:30:01.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:01 smithi050 ceph-mon[36577]: pgmap v940: 129 pgs: 129 active+clean; 28 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-07-04T08:30:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.954+0000 7f4f6ae00640 1 -- 172.21.15.39:0/468851386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f64103ba0 msgr2=0x7f4f64103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.954+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/468851386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f64103ba0 0x7f4f64103fb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009ca0 tx=0x7f4f5c02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.954+0000 7f4f6ae00640 1 -- 172.21.15.39:0/468851386 shutdown_connections 2024-07-04T08:30:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.954+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/468851386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6410d150 0x7f4f6410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.954+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/468851386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f641045d0 0x7f4f6410cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.954+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/468851386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f64103ba0 0x7f4f64103fb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.954+0000 7f4f6ae00640 1 -- 172.21.15.39:0/468851386 >> 172.21.15.39:0/468851386 conn(0x7f4f640fd810 msgr2=0x7f4f640ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.955+0000 7f4f6ae00640 1 -- 172.21.15.39:0/468851386 shutdown_connections 2024-07-04T08:30:01.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.955+0000 7f4f6ae00640 1 -- 172.21.15.39:0/468851386 wait complete. 2024-07-04T08:30:01.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.956+0000 7f4f6ae00640 1 Processor -- start 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.956+0000 7f4f6ae00640 1 -- start start 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f6ae00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f64103ba0 0x7f4f6419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f6ae00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f641045d0 0x7f4f6419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f6ae00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6410d150 0x7f4f641a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f6ae00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64112e40 con 0x7f4f641045d0 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f6ae00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64112cc0 con 0x7f4f64103ba0 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f6ae00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64112fc0 con 0x7f4f6410d150 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f69a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f64103ba0 0x7f4f6419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f69a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f64103ba0 0x7f4f6419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54450/0 (socket says 172.21.15.39:54450) 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f69a00640 1 -- 172.21.15.39:0/3773662855 learned_addr learned my addr 172.21.15.39:0/3773662855 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:01.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f69000640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f641045d0 0x7f4f6419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6410d150 0x7f4f641a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f69a00640 1 -- 172.21.15.39:0/3773662855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6410d150 msgr2=0x7f4f641a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f69a00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6410d150 0x7f4f641a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f69a00640 1 -- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f641045d0 msgr2=0x7f4f6419bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.957+0000 7f4f69a00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f641045d0 0x7f4f6419bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.958+0000 7f4f69a00640 1 -- 172.21.15.39:0/3773662855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f641a19a0 con 0x7f4f64103ba0 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.958+0000 7f4f69a00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f64103ba0 0x7f4f6419ba30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c005e30 tx=0x7f4f5c031840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.958+0000 7f4f62000640 1 -- 172.21.15.39:0/3773662855 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c003b70 con 0x7f4f64103ba0 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.958+0000 7f4f6ae00640 1 -- 172.21.15.39:0/3773662855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f641a1c30 con 0x7f4f64103ba0 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.958+0000 7f4f62000640 1 -- 172.21.15.39:0/3773662855 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f5c003d10 con 0x7f4f64103ba0 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.958+0000 7f4f62000640 1 -- 172.21.15.39:0/3773662855 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c02fa30 con 0x7f4f64103ba0 2024-07-04T08:30:01.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.958+0000 7f4f6ae00640 1 -- 172.21.15.39:0/3773662855 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f641a83f0 con 0x7f4f64103ba0 2024-07-04T08:30:01.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.962+0000 7f4f729cd740 1 Processor -- start 2024-07-04T08:30:01.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.962+0000 7f4f729cd740 1 -- start start 2024-07-04T08:30:01.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f62000640 1 -- 172.21.15.39:0/3773662855 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4f5c02fbd0 con 0x7f4f64103ba0 2024-07-04T08:30:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9ad0070 0x55bfa9e74950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e74e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9aaf390 0x55bfa9e753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f729cd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bfa9deaee0 con 0x55bfa9aa71b0 2024-07-04T08:30:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f729cd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bfa9dda480 con 0x55bfa9ad0070 2024-07-04T08:30:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f729cd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bfa9dde4c0 con 0x55bfa9aaf390 2024-07-04T08:30:01.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f6a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9ad0070 0x55bfa9e74950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f6a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9ad0070 0x55bfa9e74950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54460/0 (socket says 172.21.15.39:54460) 2024-07-04T08:30:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f6a400640 1 -- 172.21.15.39:0/3353415437 learned_addr learned my addr 172.21.15.39:0/3353415437 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f69000640 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9aaf390 0x55bfa9e753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.963+0000 7f4f69a00640 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e74e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.964+0000 7f4f6a400640 1 -- 172.21.15.39:0/3353415437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9aaf390 msgr2=0x55bfa9e753d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.964+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9aaf390 0x55bfa9e753d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.964+0000 7f4f6a400640 1 -- 172.21.15.39:0/3353415437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aa71b0 msgr2=0x55bfa9e74e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.964+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e74e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.964+0000 7f4f6a400640 1 -- 172.21.15.39:0/3353415437 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa9d72650 con 0x55bfa9ad0070 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.964+0000 7f4f69a00640 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e74e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.964+0000 7f4f69000640 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9aaf390 0x55bfa9e753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.964+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9ad0070 0x55bfa9e74950 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009ca0 tx=0x7f4f54011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9368505b7d486fb2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.965+0000 7f4f5aa00640 1 -- 172.21.15.39:0/3353415437 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5401e070 con 0x55bfa9ad0070 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.965+0000 7f4f5aa00640 1 -- 172.21.15.39:0/3353415437 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f54009e50 con 0x55bfa9ad0070 2024-07-04T08:30:01.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.965+0000 7f4f5aa00640 1 -- 172.21.15.39:0/3353415437 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400e9e0 con 0x55bfa9ad0070 2024-07-04T08:30:01.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.965+0000 7f4f62000640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f480b2960 0x7f4f480b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.965+0000 7f4f62000640 1 -- 172.21.15.39:0/3773662855 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f4f5c103bd0 con 0x7f4f64103ba0 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.965+0000 7f4f729cd740 1 -- 172.21.15.39:0/3353415437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9ad0070 msgr2=0x55bfa9e74950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9ad0070 0x55bfa9e74950 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009ca0 tx=0x7f4f54011810 comp rx=0 tx=0).stop 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f69a00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f480b2960 0x7f4f480b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f729cd740 1 -- 172.21.15.39:0/3353415437 shutdown_connections 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e74e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9aaf390 0x55bfa9e753d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3353415437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9ad0070 0x55bfa9e74950 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f729cd740 1 -- 172.21.15.39:0/3353415437 >> 172.21.15.39:0/3353415437 conn(0x55bfa9ac4460 msgr2=0x55bfa9e72840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f729cd740 1 -- 172.21.15.39:0/3353415437 shutdown_connections 2024-07-04T08:30:01.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f729cd740 1 -- 172.21.15.39:0/3353415437 wait complete. 2024-07-04T08:30:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.966+0000 7f4f69a00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f480b2960 0x7f4f480b4e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55bfa9e735d0 tx=0x7f4f5c02a8c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.967+0000 7f4f729cd740 1 -- start start 2024-07-04T08:30:01.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.967+0000 7f4f729cd740 1 Processor -- start 2024-07-04T08:30:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e80bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9e8c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f729cd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55bfa9c3e700 con 0x55bfa9aaf390 2024-07-04T08:30:01.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f729cd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bfa9d81bb0 con 0x55bfa9aa71b0 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f729cd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55bfa9e83130 con 0x55bfa9ad0070 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f729cd740 10 client.0 _ll_register_callbacks cb 0x55bfa9dd3570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f6a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f69000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e80bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f6a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e80600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54464/0 (socket says 172.21.15.39:54464) 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f6a400640 1 -- 172.21.15.39:0/3614946352 learned_addr learned my addr 172.21.15.39:0/3614946352 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.968+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9e8c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f69000640 1 -- 172.21.15.39:0/3614946352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 msgr2=0x55bfa9e8c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f69000640 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9e8c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f69000640 1 -- 172.21.15.39:0/3614946352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 msgr2=0x55bfa9e80600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:01.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f69000640 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e80600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f69000640 1 -- 172.21.15.39:0/3614946352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa9b3b650 con 0x55bfa9aaf390 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e80600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f69000640 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e80bc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4f64069d50 tx=0x7f4f50012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb23ead619c491c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f4f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f4f400640 1 -- 172.21.15.39:0/3614946352 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50004030 con 0x55bfa9aaf390 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f4f400640 1 -- 172.21.15.39:0/3614946352 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f4f500041d0 con 0x55bfa9aaf390 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.969+0000 7f4f729cd740 1 -- 172.21.15.39:0/3614946352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f54009e50 con 0x55bfa9aaf390 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.970+0000 7f4f4f400640 1 -- 172.21.15.39:0/3614946352 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f50025050 con 0x55bfa9aaf390 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.970+0000 7f4f729cd740 10 client.18555 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:01.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.970+0000 7f4f729cd740 1 -- 172.21.15.39:0/3614946352 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfa9e8f280 con 0x55bfa9aaf390 2024-07-04T08:30:01.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.970+0000 7f4f4f400640 1 -- 172.21.15.39:0/3614946352 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f54009e50 con 0x55bfa9aaf390 2024-07-04T08:30:01.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.970+0000 7f4f4f400640 1 -- 172.21.15.39:0/3614946352 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f50003900 con 0x55bfa9aaf390 2024-07-04T08:30:01.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.970+0000 7f4f729cd740 10 client.18555 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.970+0000 7f4f729cd740 10 client.18555 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.971+0000 7f4f729cd740 20 client.18555 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.971+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bfa9a8e9f0 0x55bfa9e92770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.971+0000 7f4f729cd740 4 client.18555 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.971+0000 7f4f729cd740 1 -- 172.21.15.39:0/3614946352 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfa9dece30 con 0x55bfa9a8e9f0 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.971+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bfa9a8e9f0 0x55bfa9e92770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.971+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bfa9a8e9f0 0x55bfa9e92770 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=935d9e4f0114928a server_cookie=c93ebd17c6e90e35 in_seq=0 out_seq=0 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.971+0000 7f4f4f400640 10 client.18555 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.972+0000 7f4f4f400640 1 -- 172.21.15.39:0/3614946352 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bfa9dece30 con 0x55bfa9a8e9f0 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.972+0000 7f4f4f400640 10 client.18555 handle_command_reply: tid=0 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.972+0000 7f4f729cd740 1 client.18555 shutdown 2024-07-04T08:30:01.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.972+0000 7f4f729cd740 1 -- 172.21.15.39:0/3614946352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 msgr2=0x55bfa9e80bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.972+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e80bc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4f64069d50 tx=0x7f4f50012b60 comp rx=0 tx=0).stop 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f729cd740 1 -- 172.21.15.39:0/3614946352 shutdown_connections 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bfa9a8e9f0 0x55bfa9e92770 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e80bc0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9e8c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3614946352 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e80600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f4f400640 0 client.18555 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f729cd740 1 -- 172.21.15.39:0/3614946352 >> 172.21.15.39:0/3614946352 conn(0x55bfa9ac4460 msgr2=0x55bfa9e77b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f729cd740 1 -- 172.21.15.39:0/3614946352 shutdown_connections 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f729cd740 1 -- 172.21.15.39:0/3614946352 wait complete. 2024-07-04T08:30:01.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.973+0000 7f4f729cd740 20 client.18555 trim_cache size 0 max 16384 2024-07-04T08:30:01.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.988+0000 7f4f729cd740 1 Processor -- start 2024-07-04T08:30:01.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.988+0000 7f4f729cd740 1 -- start start 2024-07-04T08:30:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e7a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e8c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9e8c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f729cd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b2310 con 0x55bfa9aaf390 2024-07-04T08:30:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f729cd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b3e50 con 0x55bfa9aa71b0 2024-07-04T08:30:01.977 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f729cd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b29e0 con 0x55bfa9ad0070 2024-07-04T08:30:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f69000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e8c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f6a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e7a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f6a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e7a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54466/0 (socket says 172.21.15.39:54466) 2024-07-04T08:30:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f6a400640 1 -- 172.21.15.39:0/3757961816 learned_addr learned my addr 172.21.15.39:0/3757961816 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.989+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3757961816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9e8c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.978 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.990+0000 7f4f6a400640 1 -- 172.21.15.39:0/3757961816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 msgr2=0x55bfa9e8c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.990+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3757961816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9e8c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.990+0000 7f4f6a400640 1 -- 172.21.15.39:0/3757961816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 msgr2=0x55bfa9e8c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.990+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3757961816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e8c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.990+0000 7f4f6a400640 1 -- 172.21.15.39:0/3757961816 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfa9e82b40 con 0x55bfa9aa71b0 2024-07-04T08:30:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.990+0000 7f4f6a400640 1 --2- 172.21.15.39:0/3757961816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e7a170 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55bfa9e76df0 tx=0x7f4f54005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36295059cc1002df server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.990+0000 7f4f5a000640 1 -- 172.21.15.39:0/3757961816 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5401e040 con 0x55bfa9aa71b0 2024-07-04T08:30:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.990+0000 7f4f5a000640 1 -- 172.21.15.39:0/3757961816 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f5400ed60 con 0x55bfa9aa71b0 2024-07-04T08:30:01.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.990+0000 7f4f5a000640 1 -- 172.21.15.39:0/3757961816 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54003c60 con 0x55bfa9aa71b0 2024-07-04T08:30:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 -- 172.21.15.39:0/3757961816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 msgr2=0x55bfa9e7a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3757961816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e7a170 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55bfa9e76df0 tx=0x7f4f54005f90 comp rx=0 tx=0).stop 2024-07-04T08:30:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 -- 172.21.15.39:0/3757961816 shutdown_connections 2024-07-04T08:30:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3757961816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e8c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3757961816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9e8c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 --2- 172.21.15.39:0/3757961816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e7a170 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 -- 172.21.15.39:0/3757961816 >> 172.21.15.39:0/3757961816 conn(0x55bfa9ac4460 msgr2=0x55bfa9e79a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 -- 172.21.15.39:0/3757961816 shutdown_connections 2024-07-04T08:30:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 -- 172.21.15.39:0/3757961816 wait complete. 2024-07-04T08:30:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.992+0000 7f4f729cd740 1 -- start start 2024-07-04T08:30:01.981 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.993+0000 7f4f729cd740 1 Processor -- start 2024-07-04T08:30:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.993+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.993+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e84a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.993+0000 7f4f729cd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9ef9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.993+0000 7f4f729cd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b3780 con 0x55bfa9aaf390 2024-07-04T08:30:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.993+0000 7f4f729cd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b30b0 con 0x55bfa9aa71b0 2024-07-04T08:30:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.993+0000 7f4f729cd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b02f0 con 0x55bfa9ad0070 2024-07-04T08:30:01.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.993+0000 7f4f729cd740 10 client.0 _ll_register_callbacks cb 0x7f4f50009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f69000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e84a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f6a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f69000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e84a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43248/0 (socket says 172.21.15.39:43248) 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f69000640 1 -- 172.21.15.39:0/1520522169 learned_addr learned my addr 172.21.15.39:0/1520522169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f6a400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e844f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54480/0 (socket says 172.21.15.39:54480) 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f6a400640 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9ef9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f69000640 1 -- 172.21.15.39:0/1520522169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 msgr2=0x55bfa9ef9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f69000640 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9ef9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f69000640 1 -- 172.21.15.39:0/1520522169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 msgr2=0x55bfa9e844f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f69000640 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e844f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.994+0000 7f4f69000640 1 -- 172.21.15.39:0/1520522169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfa9e85260 con 0x55bfa9aaf390 2024-07-04T08:30:01.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.995+0000 7f4f69000640 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e84a30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4f50003d50 tx=0x7f4f50003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d3c9b51b70434f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.995+0000 7f4f4f400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:30:01.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.995+0000 7f4f4f400640 1 -- 172.21.15.39:0/1520522169 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5002d030 con 0x55bfa9aaf390 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.995+0000 7f4f4f400640 1 -- 172.21.15.39:0/1520522169 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9424+0+0 (secure 0 0 0) 0x7f4f50018040 con 0x55bfa9aaf390 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.995+0000 7f4f729cd740 1 -- 172.21.15.39:0/1520522169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f5400ed60 con 0x55bfa9aaf390 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f4f400640 1 -- 172.21.15.39:0/1520522169 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f50022040 con 0x55bfa9aaf390 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f729cd740 10 client.18561 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f729cd740 1 -- 172.21.15.39:0/1520522169 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bfa9efb490 con 0x55bfa9aaf390 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f4f400640 1 -- 172.21.15.39:0/1520522169 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f5400ed60 con 0x55bfa9aaf390 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f4f400640 1 -- 172.21.15.39:0/1520522169 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f500055c0 con 0x55bfa9aaf390 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f729cd740 10 client.18561 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f729cd740 10 client.18561 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f729cd740 20 client.18561 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:01.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f729cd740 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bfa9a63da0 0x55bfa9efdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:01.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f729cd740 4 client.18561 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:30:01.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.996+0000 7f4f729cd740 1 -- 172.21.15.39:0/1520522169 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfa9de88d0 con 0x55bfa9a63da0 2024-07-04T08:30:01.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.997+0000 7f4f6a400640 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bfa9a63da0 0x55bfa9efdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.997+0000 7f4f6a400640 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bfa9a63da0 0x55bfa9efdd00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=48fc99695b8664d0 server_cookie=b4acc86da484beab in_seq=0 out_seq=0 2024-07-04T08:30:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.997+0000 7f4f4f400640 10 client.18561 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:30:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f4f400640 1 -- 172.21.15.39:0/1520522169 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfa9de88d0 con 0x55bfa9a63da0 2024-07-04T08:30:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f4f400640 10 client.18561 handle_command_reply: tid=0 2024-07-04T08:30:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 client.18561 shutdown 2024-07-04T08:30:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 -- 172.21.15.39:0/1520522169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 msgr2=0x55bfa9e84a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:01.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e84a30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4f50003d50 tx=0x7f4f50003820 comp rx=0 tx=0).stop 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 -- 172.21.15.39:0/1520522169 shutdown_connections 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x55bfa9a63da0 0x55bfa9efdd00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55bfa9aaf390 0x55bfa9e84a30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55bfa9ad0070 0x55bfa9ef9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 --2- 172.21.15.39:0/1520522169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bfa9aa71b0 0x55bfa9e844f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 -- 172.21.15.39:0/1520522169 >> 172.21.15.39:0/1520522169 conn(0x55bfa9ac4460 msgr2=0x55bfa9e75980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f4f400640 0 client.18561 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 -- 172.21.15.39:0/1520522169 shutdown_connections 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.998+0000 7f4f729cd740 1 -- 172.21.15.39:0/1520522169 wait complete. 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.999+0000 7f4f729cd740 20 client.18561 trim_cache size 0 max 16384 2024-07-04T08:30:01.988 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:30:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.999+0000 7f4f6ae00640 1 -- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f480b2960 msgr2=0x7f4f480b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.999+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f480b2960 0x7f4f480b4e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55bfa9e735d0 tx=0x7f4f5c02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.999+0000 7f4f6ae00640 1 -- 172.21.15.39:0/3773662855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f64103ba0 msgr2=0x7f4f6419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:01.999+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f64103ba0 0x7f4f6419ba30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c005e30 tx=0x7f4f5c031840 comp rx=0 tx=0).stop 2024-07-04T08:30:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:02.000+0000 7f4f6ae00640 1 -- 172.21.15.39:0/3773662855 shutdown_connections 2024-07-04T08:30:01.989 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:02.000+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f480b2960 0x7f4f480b4e70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:02.000+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f641045d0 0x7f4f6419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:02.000+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f6410d150 0x7f4f641a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:02.000+0000 7f4f6ae00640 1 --2- 172.21.15.39:0/3773662855 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f64103ba0 0x7f4f6419ba30 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:02.000+0000 7f4f6ae00640 1 -- 172.21.15.39:0/3773662855 >> 172.21.15.39:0/3773662855 conn(0x7f4f640fd810 msgr2=0x7f4f6410c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:02.000+0000 7f4f6ae00640 1 -- 172.21.15.39:0/3773662855 shutdown_connections 2024-07-04T08:30:01.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:02.000+0000 7f4f6ae00640 1 -- 172.21.15.39:0/3773662855 wait complete. 2024-07-04T08:30:02.859 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:30:03.359 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:03.519 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:03 smithi039 ceph-mon[32062]: pgmap v941: 129 pgs: 129 active+clean; 28 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 18 op/s 2024-07-04T08:30:03.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:03 smithi133 ceph-mon[37050]: pgmap v941: 129 pgs: 129 active+clean; 28 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 18 op/s 2024-07-04T08:30:03.565 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-04T08:30:03.566 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.7 2024-07-04T08:30:03.566 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:30:03.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:03 smithi050 ceph-mon[36577]: pgmap v941: 129 pgs: 129 active+clean; 28 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 94 KiB/s wr, 18 op/s 2024-07-04T08:30:04.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:04 smithi133 ceph-mon[37050]: osdmap e109: 12 total, 12 up, 12 in 2024-07-04T08:30:04.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:04 smithi039 ceph-mon[32062]: osdmap e109: 12 total, 12 up, 12 in 2024-07-04T08:30:04.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:04 smithi050 ceph-mon[36577]: osdmap e109: 12 total, 12 up, 12 in 2024-07-04T08:30:05.002 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:05.366 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:05 smithi039 ceph-mon[32062]: pgmap v943: 129 pgs: 129 active+clean; 28 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-07-04T08:30:05.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:05 smithi050 ceph-mon[36577]: pgmap v943: 129 pgs: 129 active+clean; 28 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-07-04T08:30:05.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:05 smithi133 ceph-mon[37050]: pgmap v943: 129 pgs: 129 active+clean; 28 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 78 KiB/s wr, 16 op/s 2024-07-04T08:30:05.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.995+0000 7fc57a800640 1 -- 172.21.15.39:0/2968375513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc574109510 msgr2=0x7fc574109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:05.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.995+0000 7fc57a800640 1 --2- 172.21.15.39:0/2968375513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc574109510 0x7fc574109920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc568009be0 tx=0x7fc56802a6d0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.995+0000 7fc57a800640 1 -- 172.21.15.39:0/2968375513 shutdown_connections 2024-07-04T08:30:05.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.995+0000 7fc57a800640 1 --2- 172.21.15.39:0/2968375513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc57410ee50 0x7fc57410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.995+0000 7fc57a800640 1 --2- 172.21.15.39:0/2968375513 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57410f7a0 0x7fc574115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.995+0000 7fc57a800640 1 --2- 172.21.15.39:0/2968375513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc574109510 0x7fc574109920 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.995+0000 7fc57a800640 1 -- 172.21.15.39:0/2968375513 >> 172.21.15.39:0/2968375513 conn(0x7fc574078b10 msgr2=0x7fc574107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:05.983 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.995+0000 7fc57a800640 1 -- 172.21.15.39:0/2968375513 shutdown_connections 2024-07-04T08:30:05.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.995+0000 7fc57a800640 1 -- 172.21.15.39:0/2968375513 wait complete. 2024-07-04T08:30:05.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.997+0000 7fc57a800640 1 Processor -- start 2024-07-04T08:30:05.984 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.997+0000 7fc57a800640 1 -- start start 2024-07-04T08:30:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.997+0000 7fc57a800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc574109510 0x7fc57419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.997+0000 7fc57a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc57410ee50 0x7fc5741a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.997+0000 7fc57a800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57410f7a0 0x7fc5741a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.997+0000 7fc57a800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc574119450 con 0x7fc574109510 2024-07-04T08:30:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.997+0000 7fc57a800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741192d0 con 0x7fc57410f7a0 2024-07-04T08:30:05.985 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.997+0000 7fc57a800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741195d0 con 0x7fc57410ee50 2024-07-04T08:30:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc579400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc574109510 0x7fc57419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc579e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57410f7a0 0x7fc5741a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc578a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc57410ee50 0x7fc5741a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc579400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc574109510 0x7fc57419fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43258/0 (socket says 172.21.15.39:43258) 2024-07-04T08:30:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc579400640 1 -- 172.21.15.39:0/3354653389 learned_addr learned my addr 172.21.15.39:0/3354653389 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc579400640 1 -- 172.21.15.39:0/3354653389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc57410ee50 msgr2=0x7fc5741a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc579400640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc57410ee50 0x7fc5741a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc579400640 1 -- 172.21.15.39:0/3354653389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57410f7a0 msgr2=0x7fc5741a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc579400640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57410f7a0 0x7fc5741a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.998+0000 7fc579400640 1 -- 172.21.15.39:0/3354653389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5741a5df0 con 0x7fc574109510 2024-07-04T08:30:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.999+0000 7fc579400640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc574109510 0x7fc57419fe80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc568004af0 tx=0x7fc568003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.999+0000 7fc572000640 1 -- 172.21.15.39:0/3354653389 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568040020 con 0x7fc574109510 2024-07-04T08:30:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.999+0000 7fc57a800640 1 -- 172.21.15.39:0/3354653389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5741a6080 con 0x7fc574109510 2024-07-04T08:30:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.999+0000 7fc572000640 1 -- 172.21.15.39:0/3354653389 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc568030970 con 0x7fc574109510 2024-07-04T08:30:05.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.999+0000 7fc572000640 1 -- 172.21.15.39:0/3354653389 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568036040 con 0x7fc574109510 2024-07-04T08:30:05.988 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:05.999+0000 7fc57a800640 1 -- 172.21.15.39:0/3354653389 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741ac840 con 0x7fc574109510 2024-07-04T08:30:05.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.003+0000 7fc582502740 1 Processor -- start 2024-07-04T08:30:05.990 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.003+0000 7fc582502740 1 -- start start 2024-07-04T08:30:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.003+0000 7fc582502740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae540950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.003+0000 7fc582502740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 0x5602ae540e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.003+0000 7fc572000640 1 -- 172.21.15.39:0/3354653389 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fc568043020 con 0x7fc574109510 2024-07-04T08:30:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.003+0000 7fc582502740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 0x5602ae5413d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.003+0000 7fc582502740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5602ae4b6ee0 con 0x5602ae17b390 2024-07-04T08:30:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.003+0000 7fc582502740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5602ae4a6480 con 0x5602ae1731b0 2024-07-04T08:30:05.991 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.003+0000 7fc582502740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5602ae4aa4c0 con 0x5602ae19c070 2024-07-04T08:30:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc579e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae540950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc579e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae540950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36052/0 (socket says 172.21.15.39:36052) 2024-07-04T08:30:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc579e00640 1 -- 172.21.15.39:0/2812821612 learned_addr learned my addr 172.21.15.39:0/2812821612 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc579400640 1 --2- 172.21.15.39:0/2812821612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 0x5602ae540e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc578a00640 1 --2- 172.21.15.39:0/2812821612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 0x5602ae5413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.992 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc579e00640 1 -- 172.21.15.39:0/2812821612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 msgr2=0x5602ae540e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:05.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc579e00640 1 --2- 172.21.15.39:0/2812821612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 0x5602ae540e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc579e00640 1 -- 172.21.15.39:0/2812821612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 msgr2=0x5602ae5413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:05.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc579e00640 1 --2- 172.21.15.39:0/2812821612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 0x5602ae5413d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.004+0000 7fc579e00640 1 -- 172.21.15.39:0/2812821612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ae43e650 con 0x5602ae19c070 2024-07-04T08:30:05.993 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.005+0000 7fc579e00640 1 --2- 172.21.15.39:0/2812821612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae540950 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009ca0 tx=0x7fc56c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caa9e0d4032ad9a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:05.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.005+0000 7fc566a00640 1 -- 172.21.15.39:0/2812821612 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c01e070 con 0x5602ae19c070 2024-07-04T08:30:05.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.005+0000 7fc566a00640 1 -- 172.21.15.39:0/2812821612 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56c009e50 con 0x5602ae19c070 2024-07-04T08:30:05.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.005+0000 7fc566a00640 1 -- 172.21.15.39:0/2812821612 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00e9e0 con 0x5602ae19c070 2024-07-04T08:30:05.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.005+0000 7fc572000640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc55c0b2960 0x7fc55c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.994 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.006+0000 7fc572000640 1 -- 172.21.15.39:0/3354653389 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9512+0+0 (secure 0 0 0) 0x7fc568103550 con 0x7fc574109510 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.006+0000 7fc579400640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc55c0b2960 0x7fc55c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.006+0000 7fc582502740 1 -- 172.21.15.39:0/2812821612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 msgr2=0x5602ae540950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.006+0000 7fc582502740 1 --2- 172.21.15.39:0/2812821612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae540950 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009ca0 tx=0x7fc56c011810 comp rx=0 tx=0).stop 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.006+0000 7fc582502740 1 -- 172.21.15.39:0/2812821612 shutdown_connections 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.006+0000 7fc582502740 1 --2- 172.21.15.39:0/2812821612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 0x5602ae5413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.006+0000 7fc582502740 1 --2- 172.21.15.39:0/2812821612 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 0x5602ae540e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.006+0000 7fc582502740 1 --2- 172.21.15.39:0/2812821612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae540950 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.006+0000 7fc582502740 1 -- 172.21.15.39:0/2812821612 >> 172.21.15.39:0/2812821612 conn(0x5602ae190460 msgr2=0x5602ae53e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.007+0000 7fc582502740 1 -- 172.21.15.39:0/2812821612 shutdown_connections 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.007+0000 7fc579400640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc55c0b2960 0x7fc55c0b4e70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc568100a80 tx=0x7fc568109110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:05.995 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.007+0000 7fc582502740 1 -- 172.21.15.39:0/2812821612 wait complete. 2024-07-04T08:30:05.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.007+0000 7fc582502740 1 -- start start 2024-07-04T08:30:05.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.007+0000 7fc582502740 1 Processor -- start 2024-07-04T08:30:05.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc582502740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 0x5602ae54c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc582502740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 0x5602ae54c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc582502740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae558450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:05.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc582502740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5602ae30a700 con 0x5602ae17b390 2024-07-04T08:30:05.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc582502740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5602ae44dbb0 con 0x5602ae1731b0 2024-07-04T08:30:05.997 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc582502740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5602ae54f130 con 0x5602ae19c070 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc582502740 10 client.0 _ll_register_callbacks cb 0x5602ae49f570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc579e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 0x5602ae54c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc578a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 0x5602ae54c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc579e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 0x5602ae54c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54516/0 (socket says 172.21.15.39:54516) 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc579e00640 1 -- 172.21.15.39:0/2924617429 learned_addr learned my addr 172.21.15.39:0/2924617429 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.008+0000 7fc579e00640 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae558450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.009+0000 7fc578a00640 1 -- 172.21.15.39:0/2924617429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 msgr2=0x5602ae558450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.009+0000 7fc578a00640 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae558450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.009+0000 7fc578a00640 1 -- 172.21.15.39:0/2924617429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 msgr2=0x5602ae54c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.009+0000 7fc578a00640 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 0x5602ae54c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:05.998 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.009+0000 7fc578a00640 1 -- 172.21.15.39:0/2924617429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ae207650 con 0x5602ae17b390 2024-07-04T08:30:05.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.009+0000 7fc578a00640 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 0x5602ae54c9b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc560009530 tx=0x7fc560011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f131e52c40b9e85 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:05.999 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.009+0000 7fc55b400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:30:06.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.010+0000 7fc55b400640 1 -- 172.21.15.39:0/2924617429 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56001e060 con 0x5602ae17b390 2024-07-04T08:30:06.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.010+0000 7fc55b400640 1 -- 172.21.15.39:0/2924617429 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9512+0+0 (secure 0 0 0) 0x7fc56000fb00 con 0x5602ae17b390 2024-07-04T08:30:06.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.010+0000 7fc582502740 1 -- 172.21.15.39:0/2924617429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc56c009e50 con 0x5602ae17b390 2024-07-04T08:30:06.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc55b400640 1 -- 172.21.15.39:0/2924617429 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc560025050 con 0x5602ae17b390 2024-07-04T08:30:06.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc582502740 10 client.18573 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:06.000 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc582502740 1 -- 172.21.15.39:0/2924617429 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5602ae55b130 con 0x5602ae17b390 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc55b400640 1 -- 172.21.15.39:0/2924617429 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56c009e50 con 0x5602ae17b390 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc55b400640 1 -- 172.21.15.39:0/2924617429 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc560019070 con 0x5602ae17b390 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc582502740 10 client.18573 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc582502740 10 client.18573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc582502740 20 client.18573 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc582502740 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5602ae15a9f0 0x5602ae55e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc582502740 4 client.18573 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.011+0000 7fc582502740 1 -- 172.21.15.39:0/2924617429 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602ae4b8e30 con 0x5602ae15a9f0 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.012+0000 7fc579e00640 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5602ae15a9f0 0x5602ae55e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.012+0000 7fc579e00640 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5602ae15a9f0 0x5602ae55e670 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7f343a627f28b82 server_cookie=1d8c8858f46554b0 in_seq=0 out_seq=0 2024-07-04T08:30:06.001 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.012+0000 7fc55b400640 10 client.18573 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.013+0000 7fc55b400640 1 -- 172.21.15.39:0/2924617429 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5602ae4b8e30 con 0x5602ae15a9f0 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.013+0000 7fc55b400640 10 client.18573 handle_command_reply: tid=0 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 client.18573 shutdown 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 -- 172.21.15.39:0/2924617429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 msgr2=0x5602ae54c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 0x5602ae54c9b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc560009530 tx=0x7fc560011b50 comp rx=0 tx=0).stop 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 -- 172.21.15.39:0/2924617429 shutdown_connections 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5602ae15a9f0 0x5602ae55e670 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae17b390 0x5602ae54c9b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae558450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 --2- 172.21.15.39:0/2924617429 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae1731b0 0x5602ae54c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 -- 172.21.15.39:0/2924617429 >> 172.21.15.39:0/2924617429 conn(0x5602ae190460 msgr2=0x5602ae543ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:06.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc55b400640 0 client.18573 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:30:06.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 -- 172.21.15.39:0/2924617429 shutdown_connections 2024-07-04T08:30:06.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.014+0000 7fc582502740 1 -- 172.21.15.39:0/2924617429 wait complete. 2024-07-04T08:30:06.004 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.015+0000 7fc582502740 20 client.18573 trim_cache size 0 max 16384 2024-07-04T08:30:06.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.030+0000 7fc582502740 1 Processor -- start 2024-07-04T08:30:06.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.030+0000 7fc582502740 1 -- start start 2024-07-04T08:30:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.030+0000 7fc582502740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae1731b0 0x5602ae546a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.030+0000 7fc582502740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae54dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.030+0000 7fc582502740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae558240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.030+0000 7fc582502740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b6970 con 0x5602ae1731b0 2024-07-04T08:30:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.030+0000 7fc582502740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b84b0 con 0x5602ae17b390 2024-07-04T08:30:06.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.030+0000 7fc582502740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b7040 con 0x5602ae19c070 2024-07-04T08:30:06.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.030+0000 7fc579e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae1731b0 0x5602ae546a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:06.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc578a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae54dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:06.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc579e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae1731b0 0x5602ae546a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43292/0 (socket says 172.21.15.39:43292) 2024-07-04T08:30:06.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc579e00640 1 -- 172.21.15.39:0/2487387801 learned_addr learned my addr 172.21.15.39:0/2487387801 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc579e00640 1 --2- 172.21.15.39:0/2487387801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae558240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc578a00640 1 -- 172.21.15.39:0/2487387801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 msgr2=0x5602ae558240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc578a00640 1 --2- 172.21.15.39:0/2487387801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae558240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.020 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc578a00640 1 -- 172.21.15.39:0/2487387801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae1731b0 msgr2=0x5602ae546a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc578a00640 1 --2- 172.21.15.39:0/2487387801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae1731b0 0x5602ae546a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc578a00640 1 -- 172.21.15.39:0/2487387801 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602ae543190 con 0x5602ae17b390 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.031+0000 7fc578a00640 1 --2- 172.21.15.39:0/2487387801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae54dea0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc560005750 tx=0x7fc560004660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d2f9561095679f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.032+0000 7fc566000640 1 -- 172.21.15.39:0/2487387801 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56002d070 con 0x5602ae17b390 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.032+0000 7fc566000640 1 -- 172.21.15.39:0/2487387801 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5600293f0 con 0x5602ae17b390 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.032+0000 7fc566000640 1 -- 172.21.15.39:0/2487387801 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56001a650 con 0x5602ae17b390 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 -- 172.21.15.39:0/2487387801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 msgr2=0x5602ae54dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 --2- 172.21.15.39:0/2487387801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae54dea0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc560005750 tx=0x7fc560004660 comp rx=0 tx=0).stop 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 -- 172.21.15.39:0/2487387801 shutdown_connections 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 --2- 172.21.15.39:0/2487387801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae19c070 0x5602ae558240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 --2- 172.21.15.39:0/2487387801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae54dea0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 --2- 172.21.15.39:0/2487387801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae1731b0 0x5602ae546a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 -- 172.21.15.39:0/2487387801 >> 172.21.15.39:0/2487387801 conn(0x5602ae190460 msgr2=0x5602ae54d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:06.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 -- 172.21.15.39:0/2487387801 shutdown_connections 2024-07-04T08:30:06.022 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 -- 172.21.15.39:0/2487387801 wait complete. 2024-07-04T08:30:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.033+0000 7fc582502740 1 -- start start 2024-07-04T08:30:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc582502740 1 Processor -- start 2024-07-04T08:30:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc582502740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae1731b0 0x5602ae5501f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc582502740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae550730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:06.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc582502740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae19c070 0x5602ae550c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc582502740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b7de0 con 0x5602ae19c070 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc582502740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b7710 con 0x5602ae17b390 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc582502740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741b4970 con 0x5602ae1731b0 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc582502740 10 client.0 _ll_register_callbacks cb 0x7fc560009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc578a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae550730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc578a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae550730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54522/0 (socket says 172.21.15.39:54522) 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.034+0000 7fc578a00640 1 -- 172.21.15.39:0/1093848588 learned_addr learned my addr 172.21.15.39:0/1093848588 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc579e00640 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae1731b0 0x5602ae5501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc579e00640 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae19c070 0x5602ae550c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc578a00640 1 -- 172.21.15.39:0/1093848588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae1731b0 msgr2=0x5602ae5501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc578a00640 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae1731b0 0x5602ae5501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc578a00640 1 -- 172.21.15.39:0/1093848588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae19c070 msgr2=0x5602ae550c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc578a00640 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae19c070 0x5602ae550c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc578a00640 1 -- 172.21.15.39:0/1093848588 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602ae5c5e40 con 0x5602ae17b390 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc579e00640 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae19c070 0x5602ae550c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc578a00640 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae550730 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc560011690 tx=0x7fc56000fd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=241f3aabec0d606c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.035+0000 7fc55b400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.036+0000 7fc55b400640 1 -- 172.21.15.39:0/1093848588 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560038070 con 0x5602ae17b390 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.036+0000 7fc582502740 1 -- 172.21.15.39:0/1093848588 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5600293f0 con 0x5602ae17b390 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.036+0000 7fc55b400640 1 -- 172.21.15.39:0/1093848588 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9512+0+0 (secure 0 0 0) 0x7fc5600297b0 con 0x5602ae17b390 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.036+0000 7fc55b400640 1 -- 172.21.15.39:0/1093848588 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc560043050 con 0x5602ae17b390 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.036+0000 7fc582502740 10 client.28700 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.036+0000 7fc582502740 1 -- 172.21.15.39:0/1093848588 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5602ae5c7030 con 0x5602ae17b390 2024-07-04T08:30:06.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.036+0000 7fc55b400640 1 -- 172.21.15.39:0/1093848588 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5600293f0 con 0x5602ae17b390 2024-07-04T08:30:06.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.037+0000 7fc55b400640 1 -- 172.21.15.39:0/1093848588 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc560019070 con 0x5602ae17b390 2024-07-04T08:30:06.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.037+0000 7fc582502740 10 client.28700 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:06.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.037+0000 7fc582502740 10 client.28700 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:30:06.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.037+0000 7fc582502740 20 client.28700 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:06.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.037+0000 7fc582502740 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5602ae12fda0 0x5602ae5c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:06.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.037+0000 7fc582502740 4 client.28700 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:30:06.027 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.037+0000 7fc582502740 1 -- 172.21.15.39:0/1093848588 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602ae4b48d0 con 0x5602ae12fda0 2024-07-04T08:30:06.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.037+0000 7fc579e00640 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5602ae12fda0 0x5602ae5c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:06.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc579e00640 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5602ae12fda0 0x5602ae5c96a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ce370509c3a04d4 server_cookie=e5b4da84721cb58f in_seq=0 out_seq=0 2024-07-04T08:30:06.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc55b400640 10 client.28700 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc55b400640 1 -- 172.21.15.39:0/1093848588 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602ae4b48d0 con 0x5602ae12fda0 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc55b400640 10 client.28700 handle_command_reply: tid=0 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 client.28700 shutdown 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 -- 172.21.15.39:0/1093848588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 msgr2=0x5602ae550730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae550730 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc560011690 tx=0x7fc56000fd30 comp rx=0 tx=0).stop 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 -- 172.21.15.39:0/1093848588 shutdown_connections 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5602ae12fda0 0x5602ae5c96a0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5602ae19c070 0x5602ae550c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5602ae17b390 0x5602ae550730 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 --2- 172.21.15.39:0/1093848588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5602ae1731b0 0x5602ae5501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc55b400640 0 client.28700 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 -- 172.21.15.39:0/1093848588 >> 172.21.15.39:0/1093848588 conn(0x5602ae190460 msgr2=0x5602ae53e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:06.029 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.038+0000 7fc582502740 1 -- 172.21.15.39:0/1093848588 shutdown_connections 2024-07-04T08:30:06.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc582502740 1 -- 172.21.15.39:0/1093848588 wait complete. 2024-07-04T08:30:06.030 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc582502740 20 client.28700 trim_cache size 0 max 16384 2024-07-04T08:30:06.030 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 -- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc55c0b2960 msgr2=0x7fc55c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc55c0b2960 0x7fc55c0b4e70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc568100a80 tx=0x7fc568109110 comp rx=0 tx=0).stop 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 -- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc574109510 msgr2=0x7fc57419fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc574109510 0x7fc57419fe80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc568004af0 tx=0x7fc568003c20 comp rx=0 tx=0).stop 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 -- 172.21.15.39:0/3354653389 shutdown_connections 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fc55c0b2960 0x7fc55c0b4e70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc57410f7a0 0x7fc5741a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc57410ee50 0x7fc5741a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 --2- 172.21.15.39:0/3354653389 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc574109510 0x7fc57419fe80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.039+0000 7fc57a800640 1 -- 172.21.15.39:0/3354653389 >> 172.21.15.39:0/3354653389 conn(0x7fc574078b10 msgr2=0x7fc5740794e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:06.031 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.040+0000 7fc57a800640 1 -- 172.21.15.39:0/3354653389 shutdown_connections 2024-07-04T08:30:06.032 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:06.040+0000 7fc57a800640 1 -- 172.21.15.39:0/3354653389 wait complete. 2024-07-04T08:30:07.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:07 smithi039 ceph-mon[32062]: pgmap v944: 129 pgs: 129 active+clean; 28 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 36 op/s 2024-07-04T08:30:07.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:07 smithi050 ceph-mon[36577]: pgmap v944: 129 pgs: 129 active+clean; 28 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 36 op/s 2024-07-04T08:30:07.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:30:07.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:07 smithi133 ceph-mon[37050]: pgmap v944: 129 pgs: 129 active+clean; 28 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 36 op/s 2024-07-04T08:30:08.006 INFO:tasks.workunit.client.0.smithi039.stdout:499/600: open fd = 3 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:500/600: open fd = 3 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:501/600: open fd = 3 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:502/600: open fd = 3 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.007 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:503/600: open fd = 3 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:504/600: open fd = 3 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:505/600: open fd = 3 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:506/600: open fd = 3 2024-07-04T08:30:08.008 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:507/600: open fd = 3 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:508/600: open fd = 3 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:509/600: open fd = 3 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.009 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:510/600: open fd = 3 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:511/600: open fd = 3 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:512/600: open fd = 3 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.010 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:513/600: open fd = 3 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:514/600: open fd = 3 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:515/600: open fd = 3 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:516/600: open fd = 3 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.011 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:517/600: open fd = 3 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:518/600: open fd = 3 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:519/600: open fd = 3 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:520/600: open fd = 3 2024-07-04T08:30:08.012 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:521/600: open fd = 3 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:522/600: open fd = 3 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:523/600: open fd = 3 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.013 INFO:tasks.workunit.client.0.smithi039.stdout:524/600: open fd = 3 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:525/600: open fd = 3 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:526/600: open fd = 3 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.014 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:527/600: open fd = 3 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:528/600: open fd = 3 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:529/600: open fd = 3 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.015 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:530/600: open fd = 3 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:531/600: open fd = 3 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:532/600: open fd = 3 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.016 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:533/600: open fd = 3 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:534/600: open fd = 3 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:535/600: open fd = 3 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.017 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:536/600: open fd = 3 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:537/600: open fd = 3 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:538/600: open fd = 3 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.018 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:539/600: open fd = 3 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:540/600: open fd = 3 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:541/600: open fd = 3 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.019 INFO:tasks.workunit.client.0.smithi039.stdout:542/600: open fd = 3 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:543/600: open fd = 3 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:544/600: open fd = 3 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:545/600: open fd = 3 2024-07-04T08:30:08.020 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:546/600: open fd = 3 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:547/600: open fd = 3 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:548/600: open fd = 3 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.021 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:549/600: open fd = 3 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:550/600: open fd = 3 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:551/600: open fd = 3 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.022 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:552/600: open fd = 3 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:553/600: open fd = 3 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:554/600: open fd = 3 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.023 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:555/600: open fd = 3 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:556/600: open fd = 3 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:557/600: open fd = 3 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:558/600: open fd = 3 2024-07-04T08:30:08.024 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:559/600: open fd = 3 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:560/600: open fd = 3 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:30:08.025 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:30:08.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.320+0000 7fbd7d800640 1 -- 172.21.15.39:0/2502817155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd781030c0 msgr2=0x7fbd781034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:08.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.320+0000 7fbd7d800640 1 --2- 172.21.15.39:0/2502817155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd781030c0 0x7fbd781034d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009900 tx=0x7fbd6002a410 comp rx=0 tx=0).stop 2024-07-04T08:30:08.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.321+0000 7fbd7d800640 1 -- 172.21.15.39:0/2502817155 shutdown_connections 2024-07-04T08:30:08.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.321+0000 7fbd7d800640 1 --2- 172.21.15.39:0/2502817155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd78109350 0x7fbd7810fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:08.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.321+0000 7fbd7d800640 1 --2- 172.21.15.39:0/2502817155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd78108a00 0x7fbd78108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:08.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.321+0000 7fbd7d800640 1 --2- 172.21.15.39:0/2502817155 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd781030c0 0x7fbd781034d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:08.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.321+0000 7fbd7d800640 1 -- 172.21.15.39:0/2502817155 >> 172.21.15.39:0/2502817155 conn(0x7fbd780feb30 msgr2=0x7fbd78100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:08.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.321+0000 7fbd7d800640 1 -- 172.21.15.39:0/2502817155 shutdown_connections 2024-07-04T08:30:08.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.321+0000 7fbd7d800640 1 -- 172.21.15.39:0/2502817155 wait complete. 2024-07-04T08:30:08.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.322+0000 7fbd7d800640 1 Processor -- start 2024-07-04T08:30:08.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.322+0000 7fbd7d800640 1 -- start start 2024-07-04T08:30:08.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd7d800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd781030c0 0x7fbd7819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:08.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd7d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd78108a00 0x7fbd7819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:08.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd7d800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd78109350 0x7fbd781a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:08.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd7d800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78113020 con 0x7fbd781030c0 2024-07-04T08:30:08.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd7d800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78112ea0 con 0x7fbd78109350 2024-07-04T08:30:08.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd7d800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbd781131a0 con 0x7fbd78108a00 2024-07-04T08:30:08.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd76a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd781030c0 0x7fbd7819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:08.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd6e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd78108a00 0x7fbd7819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:08.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd6e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd78108a00 0x7fbd7819c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36096/0 (socket says 172.21.15.39:36096) 2024-07-04T08:30:08.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd6e000640 1 -- 172.21.15.39:0/1758430954 learned_addr learned my addr 172.21.15.39:0/1758430954 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:08.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.323+0000 7fbd77400640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd78109350 0x7fbd781a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.324+0000 7fbd76a00640 1 -- 172.21.15.39:0/1758430954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd78108a00 msgr2=0x7fbd7819c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.324+0000 7fbd76a00640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd78108a00 0x7fbd7819c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.324+0000 7fbd76a00640 1 -- 172.21.15.39:0/1758430954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd78109350 msgr2=0x7fbd781a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.324+0000 7fbd76a00640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd78109350 0x7fbd781a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.324+0000 7fbd76a00640 1 -- 172.21.15.39:0/1758430954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd781a1ba0 con 0x7fbd781030c0 2024-07-04T08:30:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.324+0000 7fbd6e000640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd78108a00 0x7fbd7819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.324+0000 7fbd76a00640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd781030c0 0x7fbd7819bc30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002a930 tx=0x7fbd60031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:08.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.325+0000 7fbd6fe00640 1 -- 172.21.15.39:0/1758430954 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6003f030 con 0x7fbd781030c0 2024-07-04T08:30:08.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.325+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd781a1e30 con 0x7fbd781030c0 2024-07-04T08:30:08.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.325+0000 7fbd6fe00640 1 -- 172.21.15.39:0/1758430954 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd60004490 con 0x7fbd781030c0 2024-07-04T08:30:08.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.325+0000 7fbd6fe00640 1 -- 172.21.15.39:0/1758430954 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60035070 con 0x7fbd781030c0 2024-07-04T08:30:08.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.325+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd781a85f0 con 0x7fbd781030c0 2024-07-04T08:30:08.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.326+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd781034d0 con 0x7fbd781030c0 2024-07-04T08:30:08.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.329+0000 7fbd6fe00640 1 -- 172.21.15.39:0/1758430954 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fbd60042030 con 0x7fbd781030c0 2024-07-04T08:30:08.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.331+0000 7fbd6fe00640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd5c0b2960 0x7fbd5c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.331+0000 7fbd6fe00640 1 -- 172.21.15.39:0/1758430954 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9512+0+0 (secure 0 0 0) 0x7fbd60102eb0 con 0x7fbd781030c0 2024-07-04T08:30:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.331+0000 7fbd6e000640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd5c0b2960 0x7fbd5c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:08.319 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.332+0000 7fbd6e000640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd5c0b2960 0x7fbd5c0b4e70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbd78108580 tx=0x7fbd68009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:08.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.334+0000 7fbd6fe00640 1 -- 172.21.15.39:0/1758430954 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fbd600c9e00 con 0x7fbd781030c0 2024-07-04T08:30:08.465 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:08 smithi039 ceph-mon[32062]: pgmap v945: 129 pgs: 129 active+clean; 28 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 36 op/s 2024-07-04T08:30:08.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.597+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd7819cd60 con 0x7fbd781030c0 2024-07-04T08:30:08.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.599+0000 7fbd6fe00640 1 -- 172.21.15.39:0/1758430954 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fbd600ce100 con 0x7fbd781030c0 2024-07-04T08:30:08.587 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:30:08.587 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:30:08.587 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:30:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.601+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd5c0b2960 msgr2=0x7fbd5c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:08.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.601+0000 7fbd7d800640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd5c0b2960 0x7fbd5c0b4e70 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbd78108580 tx=0x7fbd68009500 comp rx=0 tx=0).stop 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.601+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd781030c0 msgr2=0x7fbd7819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.601+0000 7fbd7d800640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd781030c0 0x7fbd7819bc30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002a930 tx=0x7fbd60031a20 comp rx=0 tx=0).stop 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.602+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 shutdown_connections 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.602+0000 7fbd7d800640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fbd5c0b2960 0x7fbd5c0b4e70 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.602+0000 7fbd7d800640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbd78109350 0x7fbd781a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.602+0000 7fbd7d800640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd78108a00 0x7fbd7819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.602+0000 7fbd7d800640 1 --2- 172.21.15.39:0/1758430954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd781030c0 0x7fbd7819bc30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.602+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 >> 172.21.15.39:0/1758430954 conn(0x7fbd780feb30 msgr2=0x7fbd7810ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.602+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 shutdown_connections 2024-07-04T08:30:08.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:08.602+0000 7fbd7d800640 1 -- 172.21.15.39:0/1758430954 wait complete. 2024-07-04T08:30:08.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:08 smithi050 ceph-mon[36577]: pgmap v945: 129 pgs: 129 active+clean; 28 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 36 op/s 2024-07-04T08:30:08.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:08 smithi133 ceph-mon[37050]: pgmap v945: 129 pgs: 129 active+clean; 28 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 36 op/s 2024-07-04T08:30:09.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:09 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1758430954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:09.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:09 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1758430954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:09.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:09 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1758430954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:09.953 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 141.7 --await-for 45.0' 2024-07-04T08:30:09.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 141.7 --await-for 45.0 2024-07-04T08:30:10.577 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:10.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:10 smithi050 ceph-mon[36577]: pgmap v946: 129 pgs: 129 active+clean; 28 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 36 op/s 2024-07-04T08:30:10.724 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:10 smithi039 ceph-mon[32062]: pgmap v946: 129 pgs: 129 active+clean; 28 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 36 op/s 2024-07-04T08:30:10.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:10 smithi133 ceph-mon[37050]: pgmap v946: 129 pgs: 129 active+clean; 28 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 134 KiB/s wr, 36 op/s 2024-07-04T08:30:11.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:11 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:30:11.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:11 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:30:11.881 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:11 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:30:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.157+0000 7f7687200640 1 -- 172.21.15.39:0/3125073177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f768810b3a0 msgr2=0x7f7688111bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.157+0000 7f7687200640 1 --2- 172.21.15.39:0/3125073177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f768810b3a0 0x7f7688111bd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7680009ca0 tx=0x7f768002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.157+0000 7f7687200640 1 -- 172.21.15.39:0/3125073177 shutdown_connections 2024-07-04T08:30:12.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.157+0000 7f7687200640 1 --2- 172.21.15.39:0/3125073177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f768810b3a0 0x7f7688111bd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.157+0000 7f7687200640 1 --2- 172.21.15.39:0/3125073177 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f768810aa50 0x7f768810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.157+0000 7f7687200640 1 --2- 172.21.15.39:0/3125073177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7688105110 0x7f7688105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.157+0000 7f7687200640 1 -- 172.21.15.39:0/3125073177 >> 172.21.15.39:0/3125073177 conn(0x7f7688100b60 msgr2=0x7f7688102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.157+0000 7f7687200640 1 -- 172.21.15.39:0/3125073177 shutdown_connections 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.157+0000 7f7687200640 1 -- 172.21.15.39:0/3125073177 wait complete. 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.159+0000 7f7687200640 1 Processor -- start 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.159+0000 7f7687200640 1 -- start start 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7687200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7688105110 0x7f768819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7687200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f768810aa50 0x7f768819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7687200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f768810b3a0 0x7f76881a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7687200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76881150a0 con 0x7f768810aa50 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7687200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7688114f20 con 0x7f7688105110 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7687200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7688115220 con 0x7f768810b3a0 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7685e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7688105110 0x7f768819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7685e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7688105110 0x7f768819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54270/0 (socket says 172.21.15.39:54270) 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7685e00640 1 -- 172.21.15.39:0/454550546 learned_addr learned my addr 172.21.15.39:0/454550546 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7685400640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f768810aa50 0x7f768819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7686800640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f768810b3a0 0x7f76881a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7685e00640 1 -- 172.21.15.39:0/454550546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f768810b3a0 msgr2=0x7f76881a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7685e00640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f768810b3a0 0x7f76881a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7685e00640 1 -- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f768810aa50 msgr2=0x7f768819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7685e00640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f768810aa50 0x7f768819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.160+0000 7f7685e00640 1 -- 172.21.15.39:0/454550546 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76881a19a0 con 0x7f7688105110 2024-07-04T08:30:12.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.161+0000 7f7685e00640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7688105110 0x7f768819ba30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f767800ea60 tx=0x7f767800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.161+0000 7f767ea00640 1 -- 172.21.15.39:0/454550546 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767800cde0 con 0x7f7688105110 2024-07-04T08:30:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.161+0000 7f7687200640 1 -- 172.21.15.39:0/454550546 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76881a1c90 con 0x7f7688105110 2024-07-04T08:30:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.161+0000 7f767ea00640 1 -- 172.21.15.39:0/454550546 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76780035f0 con 0x7f7688105110 2024-07-04T08:30:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.161+0000 7f767ea00640 1 -- 172.21.15.39:0/454550546 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678010690 con 0x7f7688105110 2024-07-04T08:30:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.161+0000 7f7687200640 1 -- 172.21.15.39:0/454550546 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76881a83f0 con 0x7f7688105110 2024-07-04T08:30:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.165+0000 7f76932dd740 1 Processor -- start 2024-07-04T08:30:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.165+0000 7f76932dd740 1 -- start start 2024-07-04T08:30:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.165+0000 7f76932dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 0x559eb255a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.165+0000 7f76932dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb255ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.165+0000 7f767ea00640 1 -- 172.21.15.39:0/454550546 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f76780037a0 con 0x7f7688105110 2024-07-04T08:30:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.165+0000 7f76932dd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb255b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.165+0000 7f76932dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559eb24d0ee0 con 0x559eb21b6070 2024-07-04T08:30:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.165+0000 7f76932dd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559eb24c0480 con 0x559eb2195390 2024-07-04T08:30:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.165+0000 7f76932dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559eb24c44c0 con 0x559eb218d1b0 2024-07-04T08:30:12.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7686800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 0x559eb255a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb255b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb255ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb255b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54272/0 (socket says 172.21.15.39:54272) 2024-07-04T08:30:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685400640 1 -- 172.21.15.39:0/3578775526 learned_addr learned my addr 172.21.15.39:0/3578775526 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685400640 1 -- 172.21.15.39:0/3578775526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 msgr2=0x559eb255ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685400640 1 --2- 172.21.15.39:0/3578775526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb255ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685400640 1 -- 172.21.15.39:0/3578775526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 msgr2=0x559eb255a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685400640 1 --2- 172.21.15.39:0/3578775526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 0x559eb255a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685400640 1 -- 172.21.15.39:0/3578775526 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb2458650 con 0x559eb2195390 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.166+0000 7f7685e00640 1 --2- 172.21.15.39:0/3578775526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb255ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.167+0000 7f7685400640 1 --2- 172.21.15.39:0/3578775526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb255b3d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7674009ba0 tx=0x7f7674011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32adef1627a015a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.167+0000 7f7673400640 1 -- 172.21.15.39:0/3578775526 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674004230 con 0x559eb2195390 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.167+0000 7f7673400640 1 -- 172.21.15.39:0/3578775526 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7674009e10 con 0x559eb2195390 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.167+0000 7f7673400640 1 -- 172.21.15.39:0/3578775526 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674004ca0 con 0x559eb2195390 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.167+0000 7f767ea00640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f766c0b2960 0x7f766c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.167+0000 7f767ea00640 1 -- 172.21.15.39:0/454550546 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9512+0+0 (secure 0 0 0) 0x7f76780e2cf0 con 0x7f7688105110 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.167+0000 7f7686800640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f766c0b2960 0x7f766c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f76932dd740 1 -- 172.21.15.39:0/3578775526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 msgr2=0x559eb255b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f76932dd740 1 --2- 172.21.15.39:0/3578775526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb255b3d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7674009ba0 tx=0x7f7674011810 comp rx=0 tx=0).stop 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f76932dd740 1 -- 172.21.15.39:0/3578775526 shutdown_connections 2024-07-04T08:30:12.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f76932dd740 1 --2- 172.21.15.39:0/3578775526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb255b3d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f76932dd740 1 --2- 172.21.15.39:0/3578775526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb255ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f76932dd740 1 --2- 172.21.15.39:0/3578775526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 0x559eb255a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f76932dd740 1 -- 172.21.15.39:0/3578775526 >> 172.21.15.39:0/3578775526 conn(0x559eb21aa460 msgr2=0x559eb2558840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f7686800640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f766c0b2960 0x7f766c0b4e70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7680006010 tx=0x7f7680005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f76932dd740 1 -- 172.21.15.39:0/3578775526 shutdown_connections 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.168+0000 7f76932dd740 1 -- 172.21.15.39:0/3578775526 wait complete. 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.169+0000 7f76932dd740 1 -- start start 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.169+0000 7f76932dd740 1 Processor -- start 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f76932dd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb218d1b0 0x559eb2566500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f76932dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb2195390 0x559eb2566a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f76932dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 0x559eb2572450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f76932dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559eb2324700 con 0x559eb21b6070 2024-07-04T08:30:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f76932dd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559eb2467bb0 con 0x559eb218d1b0 2024-07-04T08:30:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f76932dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559eb2569130 con 0x559eb2195390 2024-07-04T08:30:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f76932dd740 10 client.0 _ll_register_callbacks cb 0x559eb24b9570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f7685e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb218d1b0 0x559eb2566500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f7685e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb218d1b0 0x559eb2566500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54276/0 (socket says 172.21.15.39:54276) 2024-07-04T08:30:12.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f7686800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 0x559eb2572450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f7685400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb2195390 0x559eb2566a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.170+0000 7f7685e00640 1 -- 172.21.15.39:0/2260977471 learned_addr learned my addr 172.21.15.39:0/2260977471 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f7685400640 1 -- 172.21.15.39:0/2260977471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb218d1b0 msgr2=0x559eb2566500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f7685400640 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb218d1b0 0x559eb2566500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f7685400640 1 -- 172.21.15.39:0/2260977471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 msgr2=0x559eb2572450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f7685400640 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 0x559eb2572450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f7685400640 1 -- 172.21.15.39:0/2260977471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb2221650 con 0x559eb2195390 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f7685400640 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb2195390 0x559eb2566a40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7674000c00 tx=0x7f7674009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f1f230aedef23c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f7667e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f7667e00640 1 -- 172.21.15.39:0/2260977471 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674027020 con 0x559eb2195390 2024-07-04T08:30:12.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f76932dd740 1 -- 172.21.15.39:0/2260977471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7674009e10 con 0x559eb2195390 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.171+0000 7f7667e00640 1 -- 172.21.15.39:0/2260977471 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9512+0+0 (secure 0 0 0) 0x7f76740037f0 con 0x559eb2195390 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.172+0000 7f7667e00640 1 -- 172.21.15.39:0/2260977471 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7674032050 con 0x559eb2195390 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.172+0000 7f76932dd740 10 client.28657 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.172+0000 7f76932dd740 1 -- 172.21.15.39:0/2260977471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559eb2575120 con 0x559eb2195390 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.172+0000 7f7667e00640 1 -- 172.21.15.39:0/2260977471 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7674009e10 con 0x559eb2195390 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.172+0000 7f7667e00640 1 -- 172.21.15.39:0/2260977471 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7674018070 con 0x559eb2195390 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.172+0000 7f76932dd740 10 client.28657 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.172+0000 7f76932dd740 10 client.28657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.172+0000 7f76932dd740 20 client.28657 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.172+0000 7f76932dd740 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559eb21749f0 0x559eb2578480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.173+0000 7f76932dd740 4 client.28657 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:12.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.173+0000 7f76932dd740 1 -- 172.21.15.39:0/2260977471 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eb24d2e30 con 0x559eb21749f0 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.173+0000 7f7685e00640 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559eb21749f0 0x559eb2578480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.173+0000 7f7685e00640 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559eb21749f0 0x559eb2578480 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a56732441e900a82 server_cookie=57ae5f0d930cc2f0 in_seq=0 out_seq=0 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.173+0000 7f7667e00640 10 client.28657 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.174+0000 7f7667e00640 1 -- 172.21.15.39:0/2260977471 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559eb24d2e30 con 0x559eb21749f0 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.174+0000 7f7667e00640 10 client.28657 handle_command_reply: tid=0 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.174+0000 7f76932dd740 1 client.28657 shutdown 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.174+0000 7f76932dd740 1 -- 172.21.15.39:0/2260977471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb2195390 msgr2=0x559eb2566a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb2195390 0x559eb2566a40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7674000c00 tx=0x7f7674009680 comp rx=0 tx=0).stop 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 1 -- 172.21.15.39:0/2260977471 shutdown_connections 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559eb21749f0 0x559eb2578480 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb21b6070 0x559eb2572450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb2195390 0x559eb2566a40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f7667e00640 0 client.28657 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 1 --2- 172.21.15.39:0/2260977471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb218d1b0 0x559eb2566500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 1 -- 172.21.15.39:0/2260977471 >> 172.21.15.39:0/2260977471 conn(0x559eb21aa460 msgr2=0x559eb255db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 1 -- 172.21.15.39:0/2260977471 shutdown_connections 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 1 -- 172.21.15.39:0/2260977471 wait complete. 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.175+0000 7f76932dd740 20 client.28657 trim_cache size 0 max 16384 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.190+0000 7f76932dd740 1 Processor -- start 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f76932dd740 1 -- start start 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f76932dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb25681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f76932dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb2195390 0x559eb255c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f76932dd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb21b6070 0x559eb255fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f76932dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76881b2530 con 0x559eb2195390 2024-07-04T08:30:12.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f76932dd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76881b4070 con 0x559eb21b6070 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f76932dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76881b2c00 con 0x559eb218d1b0 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f7685400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb2195390 0x559eb255c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f7685400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb2195390 0x559eb255c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33678/0 (socket says 172.21.15.39:33678) 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f7685400640 1 -- 172.21.15.39:0/4164949516 learned_addr learned my addr 172.21.15.39:0/4164949516 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f7685e00640 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb25681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.191+0000 7f7686800640 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb21b6070 0x559eb255fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.192+0000 7f7685400640 1 -- 172.21.15.39:0/4164949516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 msgr2=0x559eb25681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.192+0000 7f7685400640 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb25681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.192+0000 7f7685400640 1 -- 172.21.15.39:0/4164949516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb21b6070 msgr2=0x559eb255fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.192+0000 7f7685400640 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb21b6070 0x559eb255fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.192+0000 7f7685400640 1 -- 172.21.15.39:0/4164949516 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb2558a30 con 0x559eb2195390 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.192+0000 7f7685e00640 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb25681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.192+0000 7f7685400640 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb2195390 0x559eb255c090 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7674004440 tx=0x7f7674005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af40a648bdc9eb69 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.192+0000 7f7672a00640 1 -- 172.21.15.39:0/4164949516 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674027040 con 0x559eb2195390 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.193+0000 7f7672a00640 1 -- 172.21.15.39:0/4164949516 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76740097a0 con 0x559eb2195390 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.193+0000 7f7672a00640 1 -- 172.21.15.39:0/4164949516 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767401e040 con 0x559eb2195390 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.193+0000 7f76932dd740 1 -- 172.21.15.39:0/4164949516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb2195390 msgr2=0x559eb255c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.193+0000 7f76932dd740 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb2195390 0x559eb255c090 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7674004440 tx=0x7f7674005f90 comp rx=0 tx=0).stop 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.193+0000 7f76932dd740 1 -- 172.21.15.39:0/4164949516 shutdown_connections 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.193+0000 7f76932dd740 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb2195390 0x559eb255c090 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.193+0000 7f76932dd740 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb21b6070 0x559eb255fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.193+0000 7f76932dd740 1 --2- 172.21.15.39:0/4164949516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb218d1b0 0x559eb25681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.193+0000 7f76932dd740 1 -- 172.21.15.39:0/4164949516 >> 172.21.15.39:0/4164949516 conn(0x559eb21aa460 msgr2=0x559eb255b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:12.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.194+0000 7f76932dd740 1 -- 172.21.15.39:0/4164949516 shutdown_connections 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.194+0000 7f76932dd740 1 -- 172.21.15.39:0/4164949516 wait complete. 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.194+0000 7f76932dd740 1 -- start start 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.194+0000 7f76932dd740 1 Processor -- start 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f76932dd740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb218d1b0 0x559eb256e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f76932dd740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb2571b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f76932dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb21b6070 0x559eb25720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f76932dd740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f76881b39a0 con 0x559eb218d1b0 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f76932dd740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f76881b32d0 con 0x559eb2195390 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f76932dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76881b0530 con 0x559eb21b6070 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f76932dd740 10 client.0 _ll_register_callbacks cb 0x7f7674005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f7685e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb218d1b0 0x559eb256e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f7686800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb21b6070 0x559eb25720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f7685e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb218d1b0 0x559eb256e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33686/0 (socket says 172.21.15.39:33686) 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f7685e00640 1 -- 172.21.15.39:0/4119700688 learned_addr learned my addr 172.21.15.39:0/4119700688 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.195+0000 7f7685400640 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb2571b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7685e00640 1 -- 172.21.15.39:0/4119700688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb21b6070 msgr2=0x559eb25720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7685e00640 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb21b6070 0x559eb25720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7685e00640 1 -- 172.21.15.39:0/4119700688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 msgr2=0x559eb2571b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7685e00640 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb2571b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7685e00640 1 -- 172.21.15.39:0/4119700688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb2572620 con 0x559eb218d1b0 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7685400640 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb2571b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7686800640 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb21b6070 0x559eb25720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7685e00640 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb218d1b0 0x559eb256e870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f767800c760 tx=0x7f7678003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bf4ee6e4af67e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7667e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:30:12.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7667e00640 1 -- 172.21.15.39:0/4119700688 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76780a9300 con 0x559eb218d1b0 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f7667e00640 1 -- 172.21.15.39:0/4119700688 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9512+0+0 (secure 0 0 0) 0x7f76780e0280 con 0x559eb218d1b0 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.196+0000 7f76932dd740 1 -- 172.21.15.39:0/4119700688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76740097a0 con 0x559eb218d1b0 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f7667e00640 1 -- 172.21.15.39:0/4119700688 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76780a5320 con 0x559eb218d1b0 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f76932dd740 10 client.18603 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f76932dd740 1 -- 172.21.15.39:0/4119700688 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559eb256d110 con 0x559eb218d1b0 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f7667e00640 1 -- 172.21.15.39:0/4119700688 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76740097a0 con 0x559eb218d1b0 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f7667e00640 1 -- 172.21.15.39:0/4119700688 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7678004000 con 0x559eb218d1b0 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f76932dd740 10 client.18603 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f76932dd740 10 client.18603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f76932dd740 20 client.18603 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:12.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f76932dd740 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559eb2149da0 0x559eb25e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f76932dd740 4 client.18603 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:30:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.197+0000 7f76932dd740 1 -- 172.21.15.39:0/4119700688 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559eb24ce8d0 con 0x559eb2149da0 2024-07-04T08:30:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.198+0000 7f7685400640 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559eb2149da0 0x559eb25e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.198+0000 7f7685400640 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559eb2149da0 0x559eb25e2fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5465377830a60f80 server_cookie=10217225c715883 in_seq=0 out_seq=0 2024-07-04T08:30:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.198+0000 7f7667e00640 10 client.18603 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:30:12.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f7667e00640 1 -- 172.21.15.39:0/4119700688 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559eb24ce8d0 con 0x559eb2149da0 2024-07-04T08:30:12.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f7667e00640 10 client.18603 handle_command_reply: tid=0 2024-07-04T08:30:12.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 client.18603 shutdown 2024-07-04T08:30:12.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 -- 172.21.15.39:0/4119700688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb218d1b0 msgr2=0x559eb256e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:12.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb218d1b0 0x559eb256e870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f767800c760 tx=0x7f7678003cf0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 -- 172.21.15.39:0/4119700688 shutdown_connections 2024-07-04T08:30:12.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x559eb2149da0 0x559eb25e2fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559eb21b6070 0x559eb25720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559eb2195390 0x559eb2571b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 --2- 172.21.15.39:0/4119700688 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559eb218d1b0 0x559eb256e870 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f7667e00640 0 client.18603 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:30:12.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 -- 172.21.15.39:0/4119700688 >> 172.21.15.39:0/4119700688 conn(0x559eb21aa460 msgr2=0x559eb255c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:12.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 -- 172.21.15.39:0/4119700688 shutdown_connections 2024-07-04T08:30:12.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 1 -- 172.21.15.39:0/4119700688 wait complete. 2024-07-04T08:30:12.606 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:30:12.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.199+0000 7f76932dd740 20 client.18603 trim_cache size 0 max 16384 2024-07-04T08:30:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 -- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f766c0b2960 msgr2=0x7f766c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f766c0b2960 0x7f766c0b4e70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7680006010 tx=0x7f7680005d80 comp rx=0 tx=0).stop 2024-07-04T08:30:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 -- 172.21.15.39:0/454550546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7688105110 msgr2=0x7f768819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7688105110 0x7f768819ba30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f767800ea60 tx=0x7f767800ef20 comp rx=0 tx=0).stop 2024-07-04T08:30:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 -- 172.21.15.39:0/454550546 shutdown_connections 2024-07-04T08:30:12.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f766c0b2960 0x7f766c0b4e70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f768810aa50 0x7f768819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f768810b3a0 0x7f76881a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 --2- 172.21.15.39:0/454550546 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7688105110 0x7f768819ba30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 -- 172.21.15.39:0/454550546 >> 172.21.15.39:0/454550546 conn(0x7f7688100b60 msgr2=0x7f768810a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 -- 172.21.15.39:0/454550546 shutdown_connections 2024-07-04T08:30:12.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:12.200+0000 7f7687200640 1 -- 172.21.15.39:0/454550546 wait complete. 2024-07-04T08:30:12.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:12 smithi133 ceph-mon[37050]: pgmap v947: 129 pgs: 129 active+clean; 29 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 135 KiB/s wr, 36 op/s 2024-07-04T08:30:12.837 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:12 smithi039 ceph-mon[32062]: pgmap v947: 129 pgs: 129 active+clean; 29 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 135 KiB/s wr, 36 op/s 2024-07-04T08:30:12.881 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:12 smithi050 ceph-mon[36577]: pgmap v947: 129 pgs: 129 active+clean; 29 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 135 KiB/s wr, 36 op/s 2024-07-04T08:30:13.460 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:30:13.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:13 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:30:13.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:13 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:30:13.768 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:13 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:30:13.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:13 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:30:13.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:13 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:30:13.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:13 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:30:13.881 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:13 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:30:13.882 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:13 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:30:13.882 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:13 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:30:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.160+0000 7fcd39800640 1 -- 172.21.15.39:0/640269897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd341030e0 msgr2=0x7fcd341034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.160+0000 7fcd39800640 1 --2- 172.21.15.39:0/640269897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd341030e0 0x7fcd341034f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009900 tx=0x7fcd2402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.160+0000 7fcd39800640 1 -- 172.21.15.39:0/640269897 shutdown_connections 2024-07-04T08:30:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.160+0000 7fcd39800640 1 --2- 172.21.15.39:0/640269897 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd34109370 0x7fcd3410fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.160+0000 7fcd39800640 1 --2- 172.21.15.39:0/640269897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd34108a20 0x7fcd34108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.160+0000 7fcd39800640 1 --2- 172.21.15.39:0/640269897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd341030e0 0x7fcd341034f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.160+0000 7fcd39800640 1 -- 172.21.15.39:0/640269897 >> 172.21.15.39:0/640269897 conn(0x7fcd340feb70 msgr2=0x7fcd34100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.160+0000 7fcd39800640 1 -- 172.21.15.39:0/640269897 shutdown_connections 2024-07-04T08:30:14.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.160+0000 7fcd39800640 1 -- 172.21.15.39:0/640269897 wait complete. 2024-07-04T08:30:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.162+0000 7fcd39800640 1 Processor -- start 2024-07-04T08:30:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.162+0000 7fcd39800640 1 -- start start 2024-07-04T08:30:14.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.162+0000 7fcd39800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd341030e0 0x7fcd3419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.162+0000 7fcd39800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd34108a20 0x7fcd3419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.162+0000 7fcd39800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd34109370 0x7fcd341a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.162+0000 7fcd39800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34113040 con 0x7fcd341030e0 2024-07-04T08:30:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.162+0000 7fcd39800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34112ec0 con 0x7fcd34108a20 2024-07-04T08:30:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.162+0000 7fcd39800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341131c0 con 0x7fcd34109370 2024-07-04T08:30:14.151 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.163+0000 7fcd33400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd34109370 0x7fcd341a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.163+0000 7fcd32000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd34108a20 0x7fcd3419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.163+0000 7fcd32000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd34108a20 0x7fcd3419c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54328/0 (socket says 172.21.15.39:54328) 2024-07-04T08:30:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.163+0000 7fcd33400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd34109370 0x7fcd341a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49304/0 (socket says 172.21.15.39:49304) 2024-07-04T08:30:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.163+0000 7fcd32000640 1 -- 172.21.15.39:0/126401787 learned_addr learned my addr 172.21.15.39:0/126401787 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.163+0000 7fcd32000640 1 -- 172.21.15.39:0/126401787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd34109370 msgr2=0x7fcd341a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.163+0000 7fcd32000640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd34109370 0x7fcd341a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.163+0000 7fcd32000640 1 -- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd341030e0 msgr2=0x7fcd3419bc10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-04T08:30:14.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.164+0000 7fcd32a00640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd341030e0 0x7fcd3419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.164+0000 7fcd32000640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd341030e0 0x7fcd3419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.164+0000 7fcd32000640 1 -- 172.21.15.39:0/126401787 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd341a1b80 con 0x7fcd34108a20 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.164+0000 7fcd33400640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd34109370 0x7fcd341a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.164+0000 7fcd32a00640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd341030e0 0x7fcd3419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.164+0000 7fcd32000640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd34108a20 0x7fcd3419c1b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000e9c0 tx=0x7fcd2000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.165+0000 7fcd2b400640 1 -- 172.21.15.39:0/126401787 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2000ccc0 con 0x7fcd34108a20 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.165+0000 7fcd39800640 1 -- 172.21.15.39:0/126401787 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd341a1e70 con 0x7fcd34108a20 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.165+0000 7fcd2b400640 1 -- 172.21.15.39:0/126401787 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd200054d0 con 0x7fcd34108a20 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.165+0000 7fcd2b400640 1 -- 172.21.15.39:0/126401787 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20010430 con 0x7fcd34108a20 2024-07-04T08:30:14.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.165+0000 7fcd39800640 1 -- 172.21.15.39:0/126401787 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd341a85d0 con 0x7fcd34108a20 2024-07-04T08:30:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.169+0000 7fcd3feeb740 1 Processor -- start 2024-07-04T08:30:14.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.169+0000 7fcd3feeb740 1 -- start start 2024-07-04T08:30:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.169+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f343c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.169+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f34900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.169+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 0x563248f34e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.169+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563248ea9ce0 con 0x563248b8f840 2024-07-04T08:30:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.169+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563248e99280 con 0x563248b66980 2024-07-04T08:30:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.169+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563248e9d2c0 con 0x563248b6eb60 2024-07-04T08:30:14.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.170+0000 7fcd33400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.170+0000 7fcd33400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f343c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33710/0 (socket says 172.21.15.39:33710) 2024-07-04T08:30:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.170+0000 7fcd33400640 1 -- 172.21.15.39:0/4251992064 learned_addr learned my addr 172.21.15.39:0/4251992064 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.170+0000 7fcd32a00640 1 --2- 172.21.15.39:0/4251992064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f34900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.170+0000 7fcd32a00640 1 -- 172.21.15.39:0/4251992064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 msgr2=0x563248f34e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:30:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.170+0000 7fcd32a00640 1 --2- 172.21.15.39:0/4251992064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 0x563248f34e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.170+0000 7fcd32a00640 1 -- 172.21.15.39:0/4251992064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 msgr2=0x563248f343c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.170+0000 7fcd32a00640 1 --2- 172.21.15.39:0/4251992064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f343c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.170+0000 7fcd32a00640 1 -- 172.21.15.39:0/4251992064 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563248e306f0 con 0x563248b66980 2024-07-04T08:30:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.171+0000 7fcd2b400640 1 -- 172.21.15.39:0/126401787 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fcd20005000 con 0x7fcd34108a20 2024-07-04T08:30:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.171+0000 7fcd32a00640 1 --2- 172.21.15.39:0/4251992064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f34900 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcd24000c00 tx=0x7fcd24032c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4859806736952395 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.172+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4251992064 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2403f070 con 0x563248b66980 2024-07-04T08:30:14.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.172+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4251992064 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd240316a0 con 0x563248b66980 2024-07-04T08:30:14.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.172+0000 7fcd1fe00640 1 -- 172.21.15.39:0/4251992064 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24035070 con 0x563248b66980 2024-07-04T08:30:14.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.173+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4251992064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 msgr2=0x563248f34900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:14.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.173+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4251992064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f34900 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcd24000c00 tx=0x7fcd24032c30 comp rx=0 tx=0).stop 2024-07-04T08:30:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.173+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4251992064 shutdown_connections 2024-07-04T08:30:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.173+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4251992064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 0x563248f34e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.173+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4251992064 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f34900 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.173+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4251992064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f343c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.173+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4251992064 >> 172.21.15.39:0/4251992064 conn(0x563248b83990 msgr2=0x563248f322b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.173+0000 7fcd2b400640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcd180b2960 0x7fcd180b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.173+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4251992064 shutdown_connections 2024-07-04T08:30:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.174+0000 7fcd2b400640 1 -- 172.21.15.39:0/126401787 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fcd200e2d50 con 0x7fcd34108a20 2024-07-04T08:30:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.174+0000 7fcd32a00640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcd180b2960 0x7fcd180b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.174+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4251992064 wait complete. 2024-07-04T08:30:14.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.175+0000 7fcd3feeb740 1 -- start start 2024-07-04T08:30:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.175+0000 7fcd32a00640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcd180b2960 0x7fcd180b4e70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005f70 tx=0x7fcd24031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.175+0000 7fcd3feeb740 1 Processor -- start 2024-07-04T08:30:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b66980 0x563248f48e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f4e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563248cfd7a0 con 0x563248b8f840 2024-07-04T08:30:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563248e406d0 con 0x563248b6eb60 2024-07-04T08:30:14.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563248f42b40 con 0x563248b66980 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd3feeb740 10 client.0 _ll_register_callbacks cb 0x563248ba7090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd32000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd32000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54356/0 (socket says 172.21.15.39:54356) 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd32000640 1 -- 172.21.15.39:0/2577425102 learned_addr learned my addr 172.21.15.39:0/2577425102 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd32a00640 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b66980 0x563248f48e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.176+0000 7fcd33400640 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f4e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd32000640 1 -- 172.21.15.39:0/2577425102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b66980 msgr2=0x563248f48e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd32000640 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b66980 0x563248f48e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd32000640 1 -- 172.21.15.39:0/2577425102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 msgr2=0x563248f4e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd32000640 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f4e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd32000640 1 -- 172.21.15.39:0/2577425102 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563248e311d0 con 0x563248b6eb60 2024-07-04T08:30:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd32a00640 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b66980 0x563248f48e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd33400640 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f4e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:30:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd32000640 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f493f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c6a0 tx=0x7fcd2000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7704b22fcec7110 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:14.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd1cc00640 1 -- 172.21.15.39:0/2577425102 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200a97c0 con 0x563248b6eb60 2024-07-04T08:30:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2577425102 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd240316a0 con 0x563248b6eb60 2024-07-04T08:30:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.177+0000 7fcd1cc00640 1 -- 172.21.15.39:0/2577425102 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fcd200a62a0 con 0x563248b6eb60 2024-07-04T08:30:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.178+0000 7fcd1cc00640 1 -- 172.21.15.39:0/2577425102 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd200052f0 con 0x563248b6eb60 2024-07-04T08:30:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.178+0000 7fcd3feeb740 10 client.28736 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:14.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.178+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2577425102 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563248c09b90 con 0x563248b6eb60 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.178+0000 7fcd1cc00640 1 -- 172.21.15.39:0/2577425102 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd240316a0 con 0x563248b6eb60 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.178+0000 7fcd1cc00640 1 -- 172.21.15.39:0/2577425102 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcd200c4020 con 0x563248b6eb60 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.179+0000 7fcd3feeb740 10 client.28736 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.179+0000 7fcd3feeb740 10 client.28736 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.179+0000 7fcd3feeb740 20 client.28736 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.179+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563248b4e1c0 0x563248f5a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.179+0000 7fcd3feeb740 4 client.28736 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.179+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2577425102 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563248eabc30 con 0x563248b4e1c0 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.179+0000 7fcd32a00640 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563248b4e1c0 0x563248f5a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.180+0000 7fcd32a00640 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563248b4e1c0 0x563248f5a780 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57de56f5acc665d7 server_cookie=a1b8e0da5a834017 in_seq=0 out_seq=0 2024-07-04T08:30:14.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.180+0000 7fcd1cc00640 10 client.28736 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:14.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd1cc00640 1 -- 172.21.15.39:0/2577425102 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563248eabc30 con 0x563248b4e1c0 2024-07-04T08:30:14.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd1cc00640 10 client.28736 handle_command_reply: tid=0 2024-07-04T08:30:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd3feeb740 1 client.28736 shutdown 2024-07-04T08:30:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2577425102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 msgr2=0x563248f493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f493f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c6a0 tx=0x7fcd2000cf20 comp rx=0 tx=0).stop 2024-07-04T08:30:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2577425102 shutdown_connections 2024-07-04T08:30:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563248b4e1c0 0x563248f5a780 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd1cc00640 0 client.28736 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b8f840 0x563248f4e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f493f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2577425102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b66980 0x563248f48e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.181+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2577425102 >> 172.21.15.39:0/2577425102 conn(0x563248b83990 msgr2=0x563248f375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:14.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.182+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2577425102 shutdown_connections 2024-07-04T08:30:14.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.182+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2577425102 wait complete. 2024-07-04T08:30:14.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.182+0000 7fcd3feeb740 20 client.28736 trim_cache size 0 max 16384 2024-07-04T08:30:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.204+0000 7fcd3feeb740 1 Processor -- start 2024-07-04T08:30:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.204+0000 7fcd3feeb740 1 -- start start 2024-07-04T08:30:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.204+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b4e1c0 0x563248f32480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.204+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f41ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.204+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 0x563248f30870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.204+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341ae150 con 0x563248b4e1c0 2024-07-04T08:30:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.204+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341afc90 con 0x563248b66980 2024-07-04T08:30:14.192 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.204+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341ae820 con 0x563248b6eb60 2024-07-04T08:30:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.205+0000 7fcd32000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f41ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.205+0000 7fcd32000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f41ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54362/0 (socket says 172.21.15.39:54362) 2024-07-04T08:30:14.193 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.205+0000 7fcd32000640 1 -- 172.21.15.39:0/2306876670 learned_addr learned my addr 172.21.15.39:0/2306876670 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.205+0000 7fcd32a00640 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b4e1c0 0x563248f32480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.205+0000 7fcd32000640 1 -- 172.21.15.39:0/2306876670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 msgr2=0x563248f30870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.205+0000 7fcd33400640 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 0x563248f30870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.205+0000 7fcd32000640 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 0x563248f30870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.206+0000 7fcd32000640 1 -- 172.21.15.39:0/2306876670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b4e1c0 msgr2=0x563248f32480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.206+0000 7fcd32000640 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b4e1c0 0x563248f32480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.206+0000 7fcd32000640 1 -- 172.21.15.39:0/2306876670 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563248f384d0 con 0x563248b66980 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.206+0000 7fcd32a00640 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b4e1c0 0x563248f32480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.206+0000 7fcd33400640 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 0x563248f30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:30:14.194 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.206+0000 7fcd32000640 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f41ec0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcd20004650 tx=0x7fcd200a5620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56dc0543b24cb15a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.206+0000 7fcd1f400640 1 -- 172.21.15.39:0/2306876670 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200a97c0 con 0x563248b66980 2024-07-04T08:30:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.206+0000 7fcd1f400640 1 -- 172.21.15.39:0/2306876670 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd200aa3b0 con 0x563248b66980 2024-07-04T08:30:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.207+0000 7fcd1f400640 1 -- 172.21.15.39:0/2306876670 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd200b2b10 con 0x563248b66980 2024-07-04T08:30:14.195 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.208+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2306876670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 msgr2=0x563248f41ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.208+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f41ec0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcd20004650 tx=0x7fcd200a5620 comp rx=0 tx=0).stop 2024-07-04T08:30:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.208+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2306876670 shutdown_connections 2024-07-04T08:30:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.208+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b6eb60 0x563248f30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.208+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b66980 0x563248f41ec0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.208+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/2306876670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b4e1c0 0x563248f32480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.208+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2306876670 >> 172.21.15.39:0/2306876670 conn(0x563248b83990 msgr2=0x563248f42d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:14.196 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.208+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2306876670 shutdown_connections 2024-07-04T08:30:14.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.208+0000 7fcd3feeb740 1 -- 172.21.15.39:0/2306876670 wait complete. 2024-07-04T08:30:14.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.209+0000 7fcd3feeb740 1 -- start start 2024-07-04T08:30:14.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.209+0000 7fcd3feeb740 1 Processor -- start 2024-07-04T08:30:14.197 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b4e1c0 0x563248f54440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b66980 0x563248f549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd3feeb740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f4f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341af5c0 con 0x563248b66980 2024-07-04T08:30:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341aeef0 con 0x563248b6eb60 2024-07-04T08:30:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd3feeb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341ac170 con 0x563248b4e1c0 2024-07-04T08:30:14.198 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd3feeb740 10 client.0 _ll_register_callbacks cb 0x7fcd200a5d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd33400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f4f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd32a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b4e1c0 0x563248f54440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd32000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b66980 0x563248f549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd33400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f4f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54376/0 (socket says 172.21.15.39:54376) 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd33400640 1 -- 172.21.15.39:0/4221788383 learned_addr learned my addr 172.21.15.39:0/4221788383 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd32000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b66980 0x563248f549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33740/0 (socket says 172.21.15.39:33740) 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.210+0000 7fcd32a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b4e1c0 0x563248f54440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49334/0 (socket says 172.21.15.39:49334) 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.211+0000 7fcd33400640 1 -- 172.21.15.39:0/4221788383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b4e1c0 msgr2=0x563248f54440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.211+0000 7fcd33400640 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b4e1c0 0x563248f54440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.211+0000 7fcd33400640 1 -- 172.21.15.39:0/4221788383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b66980 msgr2=0x563248f549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:14.199 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.211+0000 7fcd33400640 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b66980 0x563248f549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.211+0000 7fcd33400640 1 -- 172.21.15.39:0/4221788383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563248f4f960 con 0x563248b6eb60 2024-07-04T08:30:14.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.211+0000 7fcd32a00640 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b4e1c0 0x563248f54440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:14.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.212+0000 7fcd33400640 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f4f3f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c010e00 tx=0x7fcd2c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131b11555fb21877 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:14.200 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.212+0000 7fcd1cc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.212+0000 7fcd1cc00640 1 -- 172.21.15.39:0/4221788383 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c020060 con 0x563248b6eb60 2024-07-04T08:30:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.212+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4221788383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd200aa3b0 con 0x563248b6eb60 2024-07-04T08:30:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.213+0000 7fcd1cc00640 1 -- 172.21.15.39:0/4221788383 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fcd2c003d80 con 0x563248b6eb60 2024-07-04T08:30:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.213+0000 7fcd1cc00640 1 -- 172.21.15.39:0/4221788383 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd2c027050 con 0x563248b6eb60 2024-07-04T08:30:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.213+0000 7fcd3feeb740 10 client.28742 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:14.201 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.213+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4221788383 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563248fc36b0 con 0x563248b6eb60 2024-07-04T08:30:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.213+0000 7fcd1cc00640 1 -- 172.21.15.39:0/4221788383 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd200aa3b0 con 0x563248b6eb60 2024-07-04T08:30:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.213+0000 7fcd1cc00640 1 -- 172.21.15.39:0/4221788383 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcd2c01b660 con 0x563248b6eb60 2024-07-04T08:30:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.214+0000 7fcd3feeb740 10 client.28742 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.214+0000 7fcd3feeb740 10 client.28742 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:30:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.214+0000 7fcd3feeb740 20 client.28742 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.214+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563248b8f840 0x563248fc5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.214+0000 7fcd3feeb740 4 client.28742 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.7, "await_for": 45.0}] 2024-07-04T08:30:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.214+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4221788383 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.7, "await_for": 45.0}) -- 0x563248f48470 con 0x563248b8f840 2024-07-04T08:30:14.202 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.214+0000 7fcd32a00640 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563248b8f840 0x563248fc5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:14.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.215+0000 7fcd32a00640 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563248b8f840 0x563248fc5e30 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a409ddec778330c server_cookie=45b07d86f541ac1b in_seq=0 out_seq=0 2024-07-04T08:30:14.203 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.215+0000 7fcd1cc00640 10 client.28742 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:14.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.229+0000 7fcd1cc00640 1 -- 172.21.15.39:0/4221788383 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563248f48470 con 0x563248b8f840 2024-07-04T08:30:14.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.229+0000 7fcd1cc00640 10 client.28742 handle_command_reply: tid=0 2024-07-04T08:30:14.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.229+0000 7fcd3feeb740 1 client.28742 shutdown 2024-07-04T08:30:14.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.229+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4221788383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 msgr2=0x563248f4f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:14.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.229+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f4f3f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c010e00 tx=0x7fcd2c00f790 comp rx=0 tx=0).stop 2024-07-04T08:30:14.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4221788383 shutdown_connections 2024-07-04T08:30:14.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563248b8f840 0x563248fc5e30 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563248b66980 0x563248f549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563248b6eb60 0x563248f4f3f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd3feeb740 1 --2- 172.21.15.39:0/4221788383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563248b4e1c0 0x563248f54440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4221788383 >> 172.21.15.39:0/4221788383 conn(0x563248b83990 msgr2=0x563248f42d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:14.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd1cc00640 0 client.28742 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:14.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4221788383 shutdown_connections 2024-07-04T08:30:14.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd3feeb740 1 -- 172.21.15.39:0/4221788383 wait complete. 2024-07-04T08:30:14.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.231+0000 7fcd3feeb740 20 client.28742 trim_cache size 0 max 16384 2024-07-04T08:30:14.219 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:30:14.219 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 42, 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "3d6f3bcd": { 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 42, 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 141.7, 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:30:14.220 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:14.221 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:30:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 -- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcd180b2960 msgr2=0x7fcd180b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcd180b2960 0x7fcd180b4e70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcd24005f70 tx=0x7fcd24031400 comp rx=0 tx=0).stop 2024-07-04T08:30:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 -- 172.21.15.39:0/126401787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd34108a20 msgr2=0x7fcd3419c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd34108a20 0x7fcd3419c1b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000e9c0 tx=0x7fcd2000ee80 comp rx=0 tx=0).stop 2024-07-04T08:30:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 -- 172.21.15.39:0/126401787 shutdown_connections 2024-07-04T08:30:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fcd180b2960 0x7fcd180b4e70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.222 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd34109370 0x7fcd341a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcd34108a20 0x7fcd3419c1b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 --2- 172.21.15.39:0/126401787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd341030e0 0x7fcd3419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.232+0000 7fcd39800640 1 -- 172.21.15.39:0/126401787 >> 172.21.15.39:0/126401787 conn(0x7fcd340feb70 msgr2=0x7fcd3410ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.233+0000 7fcd39800640 1 -- 172.21.15.39:0/126401787 shutdown_connections 2024-07-04T08:30:14.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:14.233+0000 7fcd39800640 1 -- 172.21.15.39:0/126401787 wait complete. 2024-07-04T08:30:15.035 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:14 smithi039 ceph-mon[32062]: osdmap e110: 12 total, 12 up, 12 in 2024-07-04T08:30:15.035 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:14 smithi039 ceph-mon[32062]: pgmap v949: 129 pgs: 129 active+clean; 29 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 135 KiB/s wr, 36 op/s 2024-07-04T08:30:15.053 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:14 smithi133 ceph-mon[37050]: osdmap e110: 12 total, 12 up, 12 in 2024-07-04T08:30:15.053 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:14 smithi133 ceph-mon[37050]: pgmap v949: 129 pgs: 129 active+clean; 29 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 135 KiB/s wr, 36 op/s 2024-07-04T08:30:15.131 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:14 smithi050 ceph-mon[36577]: osdmap e110: 12 total, 12 up, 12 in 2024-07-04T08:30:15.131 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:14 smithi050 ceph-mon[36577]: pgmap v949: 129 pgs: 129 active+clean; 29 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 135 KiB/s wr, 36 op/s 2024-07-04T08:30:16.565 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3d6f3bcd', quiesce took 13.0 seconds. Will release after: 8.7 2024-07-04T08:30:17.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.320+0000 7fe791a00640 1 -- 172.21.15.39:0/2068063684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c10f7a0 msgr2=0x7fe78c115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:17.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.320+0000 7fe791a00640 1 --2- 172.21.15.39:0/2068063684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c10f7a0 0x7fe78c115fd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe784009be0 tx=0x7fe78402a6c0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.320+0000 7fe791a00640 1 -- 172.21.15.39:0/2068063684 shutdown_connections 2024-07-04T08:30:17.308 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.320+0000 7fe791a00640 1 --2- 172.21.15.39:0/2068063684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c10f7a0 0x7fe78c115fd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.320+0000 7fe791a00640 1 --2- 172.21.15.39:0/2068063684 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c10ee50 0x7fe78c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.320+0000 7fe791a00640 1 --2- 172.21.15.39:0/2068063684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe78c109510 0x7fe78c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.320+0000 7fe791a00640 1 -- 172.21.15.39:0/2068063684 >> 172.21.15.39:0/2068063684 conn(0x7fe78c078b10 msgr2=0x7fe78c107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:17.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.320+0000 7fe791a00640 1 -- 172.21.15.39:0/2068063684 shutdown_connections 2024-07-04T08:30:17.309 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.320+0000 7fe791a00640 1 -- 172.21.15.39:0/2068063684 wait complete. 2024-07-04T08:30:17.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.321+0000 7fe791a00640 1 Processor -- start 2024-07-04T08:30:17.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.321+0000 7fe791a00640 1 -- start start 2024-07-04T08:30:17.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe791a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c109510 0x7fe78c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe791a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c10ee50 0x7fe78c1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe791a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe78c10f7a0 0x7fe78c1a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe791a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c119450 con 0x7fe78c10f7a0 2024-07-04T08:30:17.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe791a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1192d0 con 0x7fe78c109510 2024-07-04T08:30:17.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe791a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1195d0 con 0x7fe78c10ee50 2024-07-04T08:30:17.310 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe791000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe78c10f7a0 0x7fe78c1a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe78be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c109510 0x7fe78c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe791000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe78c10f7a0 0x7fe78c1a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33762/0 (socket says 172.21.15.39:33762) 2024-07-04T08:30:17.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe78b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c10ee50 0x7fe78c1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.311 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.322+0000 7fe791000640 1 -- 172.21.15.39:0/4154495138 learned_addr learned my addr 172.21.15.39:0/4154495138 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.323+0000 7fe791000640 1 -- 172.21.15.39:0/4154495138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c10ee50 msgr2=0x7fe78c1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.323+0000 7fe791000640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c10ee50 0x7fe78c1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.323+0000 7fe791000640 1 -- 172.21.15.39:0/4154495138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c109510 msgr2=0x7fe78c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.323+0000 7fe791000640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c109510 0x7fe78c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.323+0000 7fe791000640 1 -- 172.21.15.39:0/4154495138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c1a5d90 con 0x7fe78c10f7a0 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.323+0000 7fe78be00640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c109510 0x7fe78c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.323+0000 7fe791000640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe78c10f7a0 0x7fe78c1a5660 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe784009b30 tx=0x7fe784003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.324+0000 7fe788c00640 1 -- 172.21.15.39:0/4154495138 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784040020 con 0x7fe78c10f7a0 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.324+0000 7fe788c00640 1 -- 172.21.15.39:0/4154495138 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7840318f0 con 0x7fe78c10f7a0 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.324+0000 7fe788c00640 1 -- 172.21.15.39:0/4154495138 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784036040 con 0x7fe78c10f7a0 2024-07-04T08:30:17.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.324+0000 7fe791a00640 1 -- 172.21.15.39:0/4154495138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c1a6020 con 0x7fe78c10f7a0 2024-07-04T08:30:17.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.324+0000 7fe791a00640 1 -- 172.21.15.39:0/4154495138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c1ac7e0 con 0x7fe78c10f7a0 2024-07-04T08:30:17.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.327+0000 7fe799620740 1 Processor -- start 2024-07-04T08:30:17.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.328+0000 7fe788c00640 1 -- 172.21.15.39:0/4154495138 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe784043020 con 0x7fe78c10f7a0 2024-07-04T08:30:17.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.328+0000 7fe799620740 1 -- start start 2024-07-04T08:30:17.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.328+0000 7fe799620740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fae3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.328+0000 7fe799620740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f7161b0 0x55941fae3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.328+0000 7fe799620740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f71e390 0x55941fae43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.328+0000 7fe799620740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55941fa59ee0 con 0x55941f73f070 2024-07-04T08:30:17.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.328+0000 7fe799620740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55941fa49480 con 0x55941f71e390 2024-07-04T08:30:17.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.328+0000 7fe799620740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55941fa4d4c0 con 0x55941f7161b0 2024-07-04T08:30:17.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f7161b0 0x55941fae3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fae3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f7161b0 0x55941fae3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:49360/0 (socket says 172.21.15.39:49360) 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78be00640 1 -- 172.21.15.39:0/924276366 learned_addr learned my addr 172.21.15.39:0/924276366 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78b400640 1 --2- 172.21.15.39:0/924276366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f71e390 0x55941fae43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78be00640 1 -- 172.21.15.39:0/924276366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f71e390 msgr2=0x55941fae43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78be00640 1 --2- 172.21.15.39:0/924276366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f71e390 0x55941fae43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78be00640 1 -- 172.21.15.39:0/924276366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 msgr2=0x55941fae3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78be00640 1 --2- 172.21.15.39:0/924276366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fae3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.329+0000 7fe78be00640 1 -- 172.21.15.39:0/924276366 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55941f9e1650 con 0x55941f7161b0 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.330+0000 7fe788c00640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7700b2960 0x7fe7700b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.330+0000 7fe78be00640 1 --2- 172.21.15.39:0/924276366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f7161b0 0x55941fae3e90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe77c011510 tx=0x7fe77c0119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c33196b4cdab43 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.330+0000 7fe781600640 1 -- 172.21.15.39:0/924276366 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c004020 con 0x55941f7161b0 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.330+0000 7fe788c00640 1 -- 172.21.15.39:0/4154495138 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fe784103590 con 0x7fe78c10f7a0 2024-07-04T08:30:17.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.330+0000 7fe781600640 1 -- 172.21.15.39:0/924276366 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77c0041c0 con 0x55941f7161b0 2024-07-04T08:30:17.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.330+0000 7fe781600640 1 -- 172.21.15.39:0/924276366 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00f820 con 0x55941f7161b0 2024-07-04T08:30:17.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.330+0000 7fe78b400640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7700b2960 0x7fe7700b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe799620740 1 -- 172.21.15.39:0/924276366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f7161b0 msgr2=0x55941fae3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe799620740 1 --2- 172.21.15.39:0/924276366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f7161b0 0x55941fae3e90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe77c011510 tx=0x7fe77c0119d0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe78b400640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7700b2960 0x7fe7700b4e70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe778009730 tx=0x7fe7780096c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe799620740 1 -- 172.21.15.39:0/924276366 shutdown_connections 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe799620740 1 --2- 172.21.15.39:0/924276366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f71e390 0x55941fae43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe799620740 1 --2- 172.21.15.39:0/924276366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f7161b0 0x55941fae3e90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe799620740 1 --2- 172.21.15.39:0/924276366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fae3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe799620740 1 -- 172.21.15.39:0/924276366 >> 172.21.15.39:0/924276366 conn(0x55941f733460 msgr2=0x55941fae1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe799620740 1 -- 172.21.15.39:0/924276366 shutdown_connections 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.331+0000 7fe799620740 1 -- 172.21.15.39:0/924276366 wait complete. 2024-07-04T08:30:17.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.332+0000 7fe799620740 1 -- start start 2024-07-04T08:30:17.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.332+0000 7fe799620740 1 Processor -- start 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe799620740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe799620740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe799620740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fafb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe799620740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55941f8ad700 con 0x55941f73f070 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe799620740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55941f9f0bb0 con 0x55941f7161b0 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe799620740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55941faf2130 con 0x55941f71e390 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe799620740 10 client.0 _ll_register_callbacks cb 0x55941fa42570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe791000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fafb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe78be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe78b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.333+0000 7fe791000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fafb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33776/0 (socket says 172.21.15.39:33776) 2024-07-04T08:30:17.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.334+0000 7fe791000640 1 -- 172.21.15.39:0/3806696590 learned_addr learned my addr 172.21.15.39:0/3806696590 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:17.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.334+0000 7fe791000640 1 -- 172.21.15.39:0/3806696590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 msgr2=0x55941faef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:17.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.334+0000 7fe791000640 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.334+0000 7fe791000640 1 -- 172.21.15.39:0/3806696590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 msgr2=0x55941faef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:17.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.334+0000 7fe791000640 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.334+0000 7fe791000640 1 -- 172.21.15.39:0/3806696590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55941f7aa650 con 0x55941f73f070 2024-07-04T08:30:17.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.334+0000 7fe78b400640 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:17.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.334+0000 7fe791000640 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fafb450 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe784003fd0 tx=0x7fe7840ca080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ff0ecfe04f1057 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:17.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.334+0000 7fe776000640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:30:17.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.335+0000 7fe776000640 1 -- 172.21.15.39:0/3806696590 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7840ce860 con 0x55941f73f070 2024-07-04T08:30:17.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.335+0000 7fe776000640 1 -- 172.21.15.39:0/3806696590 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fe7840c6aa0 con 0x55941f73f070 2024-07-04T08:30:17.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.335+0000 7fe799620740 1 -- 172.21.15.39:0/3806696590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe77c0041c0 con 0x55941f73f070 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.335+0000 7fe776000640 1 -- 172.21.15.39:0/3806696590 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe7840d3310 con 0x55941f73f070 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.335+0000 7fe799620740 10 client.18627 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.335+0000 7fe799620740 1 -- 172.21.15.39:0/3806696590 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55941fafe080 con 0x55941f73f070 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.335+0000 7fe776000640 1 -- 172.21.15.39:0/3806696590 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77c0041c0 con 0x55941f73f070 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.336+0000 7fe776000640 1 -- 172.21.15.39:0/3806696590 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe7840ca7a0 con 0x55941f73f070 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.336+0000 7fe799620740 10 client.18627 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.336+0000 7fe799620740 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.336+0000 7fe799620740 20 client.18627 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.336+0000 7fe799620740 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55941f6fd9f0 0x55941fb01520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.336+0000 7fe799620740 4 client.18627 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.336+0000 7fe799620740 1 -- 172.21.15.39:0/3806696590 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55941fa5be30 con 0x55941f6fd9f0 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.336+0000 7fe78be00640 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55941f6fd9f0 0x55941fb01520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.337+0000 7fe78be00640 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55941f6fd9f0 0x55941fb01520 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=807442331f7d8b0c server_cookie=b889d3b15f2dd04a in_seq=0 out_seq=0 2024-07-04T08:30:17.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.337+0000 7fe776000640 10 client.18627 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe776000640 1 -- 172.21.15.39:0/3806696590 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55941fa5be30 con 0x55941f6fd9f0 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe776000640 10 client.18627 handle_command_reply: tid=0 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 client.18627 shutdown 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 -- 172.21.15.39:0/3806696590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 msgr2=0x55941fafb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fafb450 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe784003fd0 tx=0x7fe7840ca080 comp rx=0 tx=0).stop 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 -- 172.21.15.39:0/3806696590 shutdown_connections 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55941f6fd9f0 0x55941fb01520 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941fafb450 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 --2- 172.21.15.39:0/3806696590 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe776000640 0 client.18627 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:17.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 -- 172.21.15.39:0/3806696590 >> 172.21.15.39:0/3806696590 conn(0x55941f733460 msgr2=0x55941fae6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:17.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 -- 172.21.15.39:0/3806696590 shutdown_connections 2024-07-04T08:30:17.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.338+0000 7fe799620740 1 -- 172.21.15.39:0/3806696590 wait complete. 2024-07-04T08:30:17.328 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.339+0000 7fe799620740 20 client.18627 trim_cache size 0 max 16384 2024-07-04T08:30:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.354+0000 7fe799620740 1 Processor -- start 2024-07-04T08:30:17.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.354+0000 7fe799620740 1 -- start start 2024-07-04T08:30:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe799620740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe799620740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f71e390 0x55941fafb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe799620740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f73f070 0x55941fafb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe799620740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b67f0 con 0x55941f71e390 2024-07-04T08:30:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe799620740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b8330 con 0x55941f7161b0 2024-07-04T08:30:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe799620740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b6ec0 con 0x55941f73f070 2024-07-04T08:30:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f71e390 0x55941fafb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f71e390 0x55941fafb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33790/0 (socket says 172.21.15.39:33790) 2024-07-04T08:30:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78b400640 1 -- 172.21.15.39:0/3977252180 learned_addr learned my addr 172.21.15.39:0/3977252180 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78be00640 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe791000640 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f73f070 0x55941fafb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78b400640 1 -- 172.21.15.39:0/3977252180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f73f070 msgr2=0x55941fafb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:17.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78b400640 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f73f070 0x55941fafb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78b400640 1 -- 172.21.15.39:0/3977252180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 msgr2=0x55941faf0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:17.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78b400640 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.344 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78b400640 1 -- 172.21.15.39:0/3977252180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55941faed780 con 0x55941f71e390 2024-07-04T08:30:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.355+0000 7fe78be00640 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.356+0000 7fe78b400640 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f71e390 0x55941fafb240 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe778004640 tx=0x7fe778018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a685e7c4a85c4b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.356+0000 7fe780c00640 1 -- 172.21.15.39:0/3977252180 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780204f0 con 0x55941f71e390 2024-07-04T08:30:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.356+0000 7fe780c00640 1 -- 172.21.15.39:0/3977252180 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe778020b40 con 0x55941f71e390 2024-07-04T08:30:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.356+0000 7fe780c00640 1 -- 172.21.15.39:0/3977252180 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77801fa20 con 0x55941f71e390 2024-07-04T08:30:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.357+0000 7fe799620740 1 -- 172.21.15.39:0/3977252180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f71e390 msgr2=0x55941fafb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:17.345 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.357+0000 7fe799620740 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f71e390 0x55941fafb240 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe778004640 tx=0x7fe778018000 comp rx=0 tx=0).stop 2024-07-04T08:30:17.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.357+0000 7fe799620740 1 -- 172.21.15.39:0/3977252180 shutdown_connections 2024-07-04T08:30:17.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.357+0000 7fe799620740 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f71e390 0x55941fafb240 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.357+0000 7fe799620740 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f73f070 0x55941fafb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.357+0000 7fe799620740 1 --2- 172.21.15.39:0/3977252180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.357+0000 7fe799620740 1 -- 172.21.15.39:0/3977252180 >> 172.21.15.39:0/3977252180 conn(0x55941f733460 msgr2=0x55941faf07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:17.346 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.357+0000 7fe799620740 1 -- 172.21.15.39:0/3977252180 shutdown_connections 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.357+0000 7fe799620740 1 -- 172.21.15.39:0/3977252180 wait complete. 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.358+0000 7fe799620740 1 -- start start 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.358+0000 7fe799620740 1 Processor -- start 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.358+0000 7fe799620740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.358+0000 7fe799620740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faf37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.358+0000 7fe799620740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941faf3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.358+0000 7fe799620740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b7c60 con 0x55941f73f070 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.358+0000 7fe799620740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b7590 con 0x55941f7161b0 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.358+0000 7fe799620740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c1b47d0 con 0x55941f71e390 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe799620740 10 client.0 _ll_register_callbacks cb 0x7fe784009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe791000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941faf3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe78be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe791000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941faf3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33804/0 (socket says 172.21.15.39:33804) 2024-07-04T08:30:17.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe78b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faf37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe78be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:54448/0 (socket says 172.21.15.39:54448) 2024-07-04T08:30:17.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe791000640 1 -- 172.21.15.39:0/3501603157 learned_addr learned my addr 172.21.15.39:0/3501603157 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:17.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe78b400640 1 -- 172.21.15.39:0/3501603157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 msgr2=0x55941faf3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:17.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe78b400640 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe78b400640 1 -- 172.21.15.39:0/3501603157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 msgr2=0x55941faf3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:17.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe78b400640 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941faf3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe78b400640 1 -- 172.21.15.39:0/3501603157 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55941fb68cb0 con 0x55941f71e390 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.359+0000 7fe78be00640 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.360+0000 7fe78b400640 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faf37a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55941fae4930 tx=0x7fe778020840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98ed4b02a1fca047 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.360+0000 7fe776000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.360+0000 7fe776000640 1 -- 172.21.15.39:0/3501603157 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77801f4d0 con 0x55941f71e390 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.360+0000 7fe799620740 1 -- 172.21.15.39:0/3501603157 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe778020b40 con 0x55941f71e390 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.360+0000 7fe776000640 1 -- 172.21.15.39:0/3501603157 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fe77801f920 con 0x55941f71e390 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.360+0000 7fe776000640 1 -- 172.21.15.39:0/3501603157 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe77803f050 con 0x55941f71e390 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.360+0000 7fe799620740 10 client.28693 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.360+0000 7fe799620740 1 -- 172.21.15.39:0/3501603157 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55941fb69f10 con 0x55941f71e390 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.361+0000 7fe776000640 1 -- 172.21.15.39:0/3501603157 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe778020b40 con 0x55941f71e390 2024-07-04T08:30:17.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.361+0000 7fe776000640 1 -- 172.21.15.39:0/3501603157 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe77803a030 con 0x55941f71e390 2024-07-04T08:30:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.361+0000 7fe799620740 10 client.28693 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.361+0000 7fe799620740 10 client.28693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.361+0000 7fe799620740 20 client.28693 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.361+0000 7fe799620740 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55941f6d2da0 0x55941fb6c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.361+0000 7fe799620740 4 client.28693 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:30:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.361+0000 7fe799620740 1 -- 172.21.15.39:0/3501603157 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55941fa578d0 con 0x55941f6d2da0 2024-07-04T08:30:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.361+0000 7fe78be00640 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55941f6d2da0 0x55941fb6c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.362+0000 7fe78be00640 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55941f6d2da0 0x55941fb6c6c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f692fe5cfeaaecf1 server_cookie=b94113dbd9fc1ccf in_seq=0 out_seq=0 2024-07-04T08:30:17.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.362+0000 7fe776000640 10 client.28693 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:17.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.362+0000 7fe776000640 1 -- 172.21.15.39:0/3501603157 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55941fa578d0 con 0x55941f6d2da0 2024-07-04T08:30:17.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.362+0000 7fe776000640 10 client.28693 handle_command_reply: tid=0 2024-07-04T08:30:17.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.362+0000 7fe799620740 1 client.28693 shutdown 2024-07-04T08:30:17.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 -- 172.21.15.39:0/3501603157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 msgr2=0x55941faf37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:17.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faf37a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55941fae4930 tx=0x7fe778020840 comp rx=0 tx=0).stop 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 -- 172.21.15.39:0/3501603157 shutdown_connections 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55941f6d2da0 0x55941fb6c6c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55941f73f070 0x55941faf3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55941f71e390 0x55941faf37a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe776000640 0 client.28693 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 --2- 172.21.15.39:0/3501603157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55941f7161b0 0x55941faf3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 -- 172.21.15.39:0/3501603157 >> 172.21.15.39:0/3501603157 conn(0x55941f733460 msgr2=0x55941fadfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 -- 172.21.15.39:0/3501603157 shutdown_connections 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 1 -- 172.21.15.39:0/3501603157 wait complete. 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.363+0000 7fe799620740 20 client.28693 trim_cache size 0 max 16384 2024-07-04T08:30:17.353 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:30:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 -- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7700b2960 msgr2=0x7fe7700b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7700b2960 0x7fe7700b4e70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe778009730 tx=0x7fe7780096c0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 -- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe78c10f7a0 msgr2=0x7fe78c1a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:17.354 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe78c10f7a0 0x7fe78c1a5660 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe784009b30 tx=0x7fe784003c10 comp rx=0 tx=0).stop 2024-07-04T08:30:17.354 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:17 smithi039 ceph-mon[32062]: pgmap v950: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 15 op/s 2024-07-04T08:30:17.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 -- 172.21.15.39:0/4154495138 shutdown_connections 2024-07-04T08:30:17.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe7700b2960 0x7fe7700b4e70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe78c10f7a0 0x7fe78c1a5660 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe78c10ee50 0x7fe78c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 --2- 172.21.15.39:0/4154495138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe78c109510 0x7fe78c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:17.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 -- 172.21.15.39:0/4154495138 >> 172.21.15.39:0/4154495138 conn(0x7fe78c078b10 msgr2=0x7fe78c1078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:17.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 -- 172.21.15.39:0/4154495138 shutdown_connections 2024-07-04T08:30:17.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:17.364+0000 7fe791a00640 1 -- 172.21.15.39:0/4154495138 wait complete. 2024-07-04T08:30:17.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:17 smithi133 ceph-mon[37050]: pgmap v950: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 15 op/s 2024-07-04T08:30:17.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:17 smithi050 ceph-mon[36577]: pgmap v950: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 15 op/s 2024-07-04T08:30:19.190 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:30:19.190 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:30:19.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:19 smithi133 ceph-mon[37050]: pgmap v951: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 15 op/s 2024-07-04T08:30:19.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:19 smithi039 ceph-mon[32062]: pgmap v951: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 15 op/s 2024-07-04T08:30:19.634 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:19 smithi050 ceph-mon[36577]: pgmap v951: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 15 op/s 2024-07-04T08:30:19.684 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:21.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:21 smithi039 ceph-mon[32062]: pgmap v952: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 15 op/s 2024-07-04T08:30:21.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:21 smithi133 ceph-mon[37050]: pgmap v952: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 15 op/s 2024-07-04T08:30:21.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:21 smithi050 ceph-mon[36577]: pgmap v952: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 74 KiB/s wr, 15 op/s 2024-07-04T08:30:22.003 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.015+0000 7fb7cb000640 1 -- 172.21.15.39:0/371245147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7c4109170 msgr2=0x7fb7c410f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:22.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.015+0000 7fb7cb000640 1 --2- 172.21.15.39:0/371245147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7c4109170 0x7fb7c410f9a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009ca0 tx=0x7fb7bc02a8c0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.015+0000 7fb7cb000640 1 -- 172.21.15.39:0/371245147 shutdown_connections 2024-07-04T08:30:22.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.015+0000 7fb7cb000640 1 --2- 172.21.15.39:0/371245147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c4108820 0x7fb7c4108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.015+0000 7fb7cb000640 1 --2- 172.21.15.39:0/371245147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7c4109170 0x7fb7c410f9a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.015+0000 7fb7cb000640 1 --2- 172.21.15.39:0/371245147 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7c4102ee0 0x7fb7c41032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.015+0000 7fb7cb000640 1 -- 172.21.15.39:0/371245147 >> 172.21.15.39:0/371245147 conn(0x7fb7c40fe930 msgr2=0x7fb7c4100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:22.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.015+0000 7fb7cb000640 1 -- 172.21.15.39:0/371245147 shutdown_connections 2024-07-04T08:30:22.028 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.015+0000 7fb7cb000640 1 -- 172.21.15.39:0/371245147 wait complete. 2024-07-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.017+0000 7fb7cb000640 1 Processor -- start 2024-07-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.017+0000 7fb7cb000640 1 -- start start 2024-07-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7cb000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7c4102ee0 0x7fb7c419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7cb000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7c4108820 0x7fb7c419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7cb000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c4109170 0x7fb7c41a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7cb000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4112e30 con 0x7fb7c4109170 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7cb000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4112cb0 con 0x7fb7c4102ee0 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7cb000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4112fb0 con 0x7fb7c4108820 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7c9c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7c4102ee0 0x7fb7c419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7c9c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7c4102ee0 0x7fb7c419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38564/0 (socket says 172.21.15.39:38564) 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7c9c00640 1 -- 172.21.15.39:0/240152189 learned_addr learned my addr 172.21.15.39:0/240152189 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7c9200640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7c4108820 0x7fb7c419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7ca600640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c4109170 0x7fb7c41a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7c9c00640 1 -- 172.21.15.39:0/240152189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7c4108820 msgr2=0x7fb7c419bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:22.036 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7c9c00640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7c4108820 0x7fb7c419bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7c9c00640 1 -- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c4109170 msgr2=0x7fb7c41a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7c9c00640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c4109170 0x7fb7c41a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.018+0000 7fb7c9c00640 1 -- 172.21.15.39:0/240152189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c41a1970 con 0x7fb7c4102ee0 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.019+0000 7fb7c9c00640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7c4102ee0 0x7fb7c419ba00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800e9f0 tx=0x7fb7b800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.019+0000 7fb7c2000640 1 -- 172.21.15.39:0/240152189 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b800cd80 con 0x7fb7c4102ee0 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.019+0000 7fb7cb000640 1 -- 172.21.15.39:0/240152189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c41a1c60 con 0x7fb7c4102ee0 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.019+0000 7fb7c2000640 1 -- 172.21.15.39:0/240152189 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b80054d0 con 0x7fb7c4102ee0 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.019+0000 7fb7c2000640 1 -- 172.21.15.39:0/240152189 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8010690 con 0x7fb7c4102ee0 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.019+0000 7fb7cb000640 1 -- 172.21.15.39:0/240152189 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c41a83c0 con 0x7fb7c4102ee0 2024-07-04T08:30:22.037 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.023+0000 7fb7d2c40740 1 Processor -- start 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.023+0000 7fb7d2c40740 1 -- start start 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.023+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c630800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.023+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 0x55748c630d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.023+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c631280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.023+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55748c55a8f0 con 0x55748c28c350 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.023+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55748c5a6ed0 con 0x55748c263490 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.023+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55748c596470 con 0x55748c26b670 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7ca600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c630800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7ca600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c630800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48814/0 (socket says 172.21.15.39:48814) 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7ca600640 1 -- 172.21.15.39:0/2509573578 learned_addr learned my addr 172.21.15.39:0/2509573578 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7c9200640 1 --2- 172.21.15.39:0/2509573578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c631280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7c2000640 1 -- 172.21.15.39:0/240152189 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb7b8004e90 con 0x7fb7c4102ee0 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7ca600640 1 -- 172.21.15.39:0/2509573578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 msgr2=0x55748c631280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:22.038 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7ca600640 1 --2- 172.21.15.39:0/2509573578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c631280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7ca600640 1 -- 172.21.15.39:0/2509573578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 msgr2=0x55748c630d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7ca600640 1 --2- 172.21.15.39:0/2509573578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 0x55748c630d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.024+0000 7fb7ca600640 1 -- 172.21.15.39:0/2509573578 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748c52e290 con 0x55748c28c350 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.025+0000 7fb7ca600640 1 --2- 172.21.15.39:0/2509573578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c630800 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005cc0 tx=0x7fb7bc02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d961556f24de3033 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.025+0000 7fb7b6a00640 1 -- 172.21.15.39:0/2509573578 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc031cf0 con 0x55748c28c350 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.025+0000 7fb7b6a00640 1 -- 172.21.15.39:0/2509573578 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7bc036040 con 0x55748c28c350 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.025+0000 7fb7b6a00640 1 -- 172.21.15.39:0/2509573578 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc004040 con 0x55748c28c350 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.026+0000 7fb7c2000640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7ac0b2960 0x7fb7ac0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.026+0000 7fb7d2c40740 1 -- 172.21.15.39:0/2509573578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 msgr2=0x55748c630800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.026+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/2509573578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c630800 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc005cc0 tx=0x7fb7bc02f420 comp rx=0 tx=0).stop 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.026+0000 7fb7d2c40740 1 -- 172.21.15.39:0/2509573578 shutdown_connections 2024-07-04T08:30:22.039 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.026+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/2509573578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c631280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.026+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/2509573578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 0x55748c630d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.026+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/2509573578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c630800 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.026+0000 7fb7d2c40740 1 -- 172.21.15.39:0/2509573578 >> 172.21.15.39:0/2509573578 conn(0x55748c280740 msgr2=0x55748c62e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:22.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.026+0000 7fb7c2000640 1 -- 172.21.15.39:0/240152189 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fb7b80e2e80 con 0x7fb7c4102ee0 2024-07-04T08:30:22.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.027+0000 7fb7c9c00640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7ac0b2960 0x7fb7ac0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.027+0000 7fb7d2c40740 1 -- 172.21.15.39:0/2509573578 shutdown_connections 2024-07-04T08:30:22.040 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.027+0000 7fb7d2c40740 1 -- 172.21.15.39:0/2509573578 wait complete. 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.027+0000 7fb7c9c00640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7ac0b2960 0x7fb7ac0b4e70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55748c62f480 tx=0x7fb7b8009720 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.027+0000 7fb7d2c40740 1 -- start start 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.028+0000 7fb7d2c40740 1 Processor -- start 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.028+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c263490 0x55748c63c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.028+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c63c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.028+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c28c350 0x55748c6480f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.028+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55748c53d770 con 0x55748c263490 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.028+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55748c59a4b0 con 0x55748c28c350 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.028+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55748c3fa890 con 0x55748c26b670 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.028+0000 7fb7d2c40740 10 client.0 _ll_register_callbacks cb 0x55748c53e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7ca600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c263490 0x55748c63c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.041 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7ca600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c263490 0x55748c63c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48828/0 (socket says 172.21.15.39:48828) 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7ca600640 1 -- 172.21.15.39:0/3242082803 learned_addr learned my addr 172.21.15.39:0/3242082803 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7c9200640 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c63c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7ca600640 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c28c350 0x55748c6480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7c9200640 1 -- 172.21.15.39:0/3242082803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c28c350 msgr2=0x55748c6480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7c9200640 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c28c350 0x55748c6480f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7c9200640 1 -- 172.21.15.39:0/3242082803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c263490 msgr2=0x55748c63c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7c9200640 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c263490 0x55748c63c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7c9200640 1 -- 172.21.15.39:0/3242082803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748c2f7930 con 0x55748c26b670 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.029+0000 7fb7ca600640 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c28c350 0x55748c6480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:22.042 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.030+0000 7fb7c9200640 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c63c670 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb7c406bad0 tx=0x7fb7b0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f603037ea95744 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.030+0000 7fb7ab400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.030+0000 7fb7ab400640 1 -- 172.21.15.39:0/3242082803 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000f890 con 0x55748c26b670 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.030+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3242082803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7bc036040 con 0x55748c26b670 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.030+0000 7fb7ab400640 1 -- 172.21.15.39:0/3242082803 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fb7b000fa30 con 0x55748c26b670 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.030+0000 7fb7ab400640 1 -- 172.21.15.39:0/3242082803 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb7b000fc30 con 0x55748c26b670 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.030+0000 7fb7d2c40740 10 client.28705 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.030+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3242082803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55748c64ad50 con 0x55748c26b670 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.031+0000 7fb7ab400640 1 -- 172.21.15.39:0/3242082803 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7bc036040 con 0x55748c26b670 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.031+0000 7fb7ab400640 1 -- 172.21.15.39:0/3242082803 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb7b001c420 con 0x55748c26b670 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.031+0000 7fb7d2c40740 10 client.28705 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:22.043 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.031+0000 7fb7d2c40740 10 client.28705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:22.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.031+0000 7fb7d2c40740 20 client.28705 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:22.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.031+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55748c24acd0 0x55748c64e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.031+0000 7fb7d2c40740 4 client.28705 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:22.044 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.031+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3242082803 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55748c5a8e20 con 0x55748c24acd0 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.032+0000 7fb7ca600640 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55748c24acd0 0x55748c64e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.032+0000 7fb7ca600640 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55748c24acd0 0x55748c64e290 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=425cc09b25604879 server_cookie=829efcf89d6a9adc in_seq=0 out_seq=0 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.032+0000 7fb7ab400640 10 client.28705 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7ab400640 1 -- 172.21.15.39:0/3242082803 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55748c5a8e20 con 0x55748c24acd0 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7ab400640 10 client.28705 handle_command_reply: tid=0 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7d2c40740 1 client.28705 shutdown 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3242082803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 msgr2=0x55748c63c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c63c670 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb7c406bad0 tx=0x7fb7b0012b60 comp rx=0 tx=0).stop 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3242082803 shutdown_connections 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55748c24acd0 0x55748c64e290 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.045 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c28c350 0x55748c6480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c63c670 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3242082803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c263490 0x55748c63c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3242082803 >> 172.21.15.39:0/3242082803 conn(0x55748c280740 msgr2=0x55748c6339e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:22.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.033+0000 7fb7ab400640 0 client.28705 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:22.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.034+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3242082803 shutdown_connections 2024-07-04T08:30:22.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.034+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3242082803 wait complete. 2024-07-04T08:30:22.046 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.034+0000 7fb7d2c40740 20 client.28705 trim_cache size 0 max 16384 2024-07-04T08:30:22.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.055+0000 7fb7d2c40740 1 Processor -- start 2024-07-04T08:30:22.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.055+0000 7fb7d2c40740 1 -- start start 2024-07-04T08:30:22.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.055+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c263490 0x55748c63dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.055+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c26b670 0x55748c647ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.055+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c648420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.051 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.055+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41b2420 con 0x55748c28c350 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.055+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41b3f60 con 0x55748c26b670 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.055+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41b2af0 con 0x55748c263490 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7c9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c26b670 0x55748c647ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7c9200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c26b670 0x55748c647ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38600/0 (socket says 172.21.15.39:38600) 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7c9200640 1 -- 172.21.15.39:0/3553128065 learned_addr learned my addr 172.21.15.39:0/3553128065 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7ca600640 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c263490 0x55748c63dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7ca600640 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c648420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7c9200640 1 -- 172.21.15.39:0/3553128065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c263490 msgr2=0x55748c63dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7c9200640 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c263490 0x55748c63dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7c9200640 1 -- 172.21.15.39:0/3553128065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 msgr2=0x55748c648420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:22.052 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7c9200640 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c648420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7c9200640 1 -- 172.21.15.39:0/3553128065 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748c629e90 con 0x55748c26b670 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.056+0000 7fb7ca600640 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c648420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.057+0000 7fb7c9200640 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c26b670 0x55748c647ee0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00059f0 tx=0x7fb7b00096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1be93d126be150c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.057+0000 7fb7b6000640 1 -- 172.21.15.39:0/3553128065 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b001cb00 con 0x55748c26b670 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.057+0000 7fb7b6000640 1 -- 172.21.15.39:0/3553128065 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b001cca0 con 0x55748c26b670 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.057+0000 7fb7b6000640 1 -- 172.21.15.39:0/3553128065 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00035f0 con 0x55748c26b670 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.058+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3553128065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c26b670 msgr2=0x55748c647ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.058+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c26b670 0x55748c647ee0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00059f0 tx=0x7fb7b00096f0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.058+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3553128065 shutdown_connections 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.058+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c648420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.058+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c26b670 0x55748c647ee0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.053 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.058+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/3553128065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c263490 0x55748c63dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.058+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3553128065 >> 172.21.15.39:0/3553128065 conn(0x55748c280740 msgr2=0x55748c63d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.058+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3553128065 shutdown_connections 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.058+0000 7fb7d2c40740 1 -- 172.21.15.39:0/3553128065 wait complete. 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.059+0000 7fb7d2c40740 1 -- start start 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.059+0000 7fb7d2c40740 1 Processor -- start 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.059+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 0x55748c63fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c6403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7d2c40740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c6408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41b3890 con 0x55748c28c350 2024-07-04T08:30:22.054 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41b31c0 con 0x55748c263490 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7d2c40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c41b0400 con 0x55748c26b670 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7d2c40740 10 client.0 _ll_register_callbacks cb 0x7fb7b0009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7ca600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 0x55748c63fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7ca600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 0x55748c63fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38604/0 (socket says 172.21.15.39:38604) 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7ca600640 1 -- 172.21.15.39:0/43937800 learned_addr learned my addr 172.21.15.39:0/43937800 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7c9200640 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c6403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7ca600640 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c6408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7ca600640 1 -- 172.21.15.39:0/43937800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 msgr2=0x55748c6403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7ca600640 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c6403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7ca600640 1 -- 172.21.15.39:0/43937800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 msgr2=0x55748c6408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:22.055 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7ca600640 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c6408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.060+0000 7fb7ca600640 1 -- 172.21.15.39:0/43937800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748c648960 con 0x55748c263490 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.061+0000 7fb7ca600640 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 0x55748c63fe70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc003ae0 tx=0x7fb7bc02f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=272fafe03ac9223e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.061+0000 7fb7ab400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.061+0000 7fb7ab400640 1 -- 172.21.15.39:0/43937800 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc031450 con 0x55748c263490 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.061+0000 7fb7d2c40740 1 -- 172.21.15.39:0/43937800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b001cca0 con 0x55748c263490 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.061+0000 7fb7ab400640 1 -- 172.21.15.39:0/43937800 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fb7bc04a040 con 0x55748c263490 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7ab400640 1 -- 172.21.15.39:0/43937800 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb7bc0315f0 con 0x55748c263490 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7d2c40740 10 client.28772 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7d2c40740 1 -- 172.21.15.39:0/43937800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55748c6b6b10 con 0x55748c263490 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7ab400640 1 -- 172.21.15.39:0/43937800 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b001cca0 con 0x55748c263490 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7ab400640 1 -- 172.21.15.39:0/43937800 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb7bc0305c0 con 0x55748c263490 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7d2c40740 10 client.28772 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7d2c40740 10 client.28772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:22.056 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7d2c40740 20 client.28772 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55748c220080 0x55748c6b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7d2c40740 4 client.28772 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:30:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.062+0000 7fb7d2c40740 1 -- 172.21.15.39:0/43937800 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55748c5a48c0 con 0x55748c220080 2024-07-04T08:30:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.063+0000 7fb7c9200640 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55748c220080 0x55748c6b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.063+0000 7fb7c9200640 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55748c220080 0x55748c6b8cf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b8a6f2f0d05a30 server_cookie=5946f0a11ba7e44 in_seq=0 out_seq=0 2024-07-04T08:30:22.057 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.063+0000 7fb7ab400640 10 client.28772 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:22.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7ab400640 1 -- 172.21.15.39:0/43937800 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55748c5a48c0 con 0x55748c220080 2024-07-04T08:30:22.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7ab400640 10 client.28772 handle_command_reply: tid=0 2024-07-04T08:30:22.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 client.28772 shutdown 2024-07-04T08:30:22.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 -- 172.21.15.39:0/43937800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 msgr2=0x55748c63fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:22.058 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 0x55748c63fe70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc003ae0 tx=0x7fb7bc02f420 comp rx=0 tx=0).stop 2024-07-04T08:30:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 -- 172.21.15.39:0/43937800 shutdown_connections 2024-07-04T08:30:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55748c220080 0x55748c6b8cf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55748c28c350 0x55748c6408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55748c26b670 0x55748c6403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 --2- 172.21.15.39:0/43937800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55748c263490 0x55748c63fe70 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7ab400640 0 client.28772 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 -- 172.21.15.39:0/43937800 >> 172.21.15.39:0/43937800 conn(0x55748c280740 msgr2=0x55748c6342c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 -- 172.21.15.39:0/43937800 shutdown_connections 2024-07-04T08:30:22.059 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.065+0000 7fb7d2c40740 1 -- 172.21.15.39:0/43937800 wait complete. 2024-07-04T08:30:22.060 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.066+0000 7fb7d2c40740 20 client.28772 trim_cache size 0 max 16384 2024-07-04T08:30:22.060 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:30:22.060 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:30:22.060 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "d4ff3010-0e0f-4e89-85f6-3203c51a000b", 2024-07-04T08:30:22.060 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:30:22.060 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:30:22.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.066+0000 7fb7cb000640 1 -- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7ac0b2960 msgr2=0x7fb7ac0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.066+0000 7fb7cb000640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7ac0b2960 0x7fb7ac0b4e70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55748c62f480 tx=0x7fb7b8009720 comp rx=0 tx=0).stop 2024-07-04T08:30:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.066+0000 7fb7cb000640 1 -- 172.21.15.39:0/240152189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7c4102ee0 msgr2=0x7fb7c419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.066+0000 7fb7cb000640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7c4102ee0 0x7fb7c419ba00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800e9f0 tx=0x7fb7b800edf0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.062 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.067+0000 7fb7cb000640 1 -- 172.21.15.39:0/240152189 shutdown_connections 2024-07-04T08:30:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.067+0000 7fb7cb000640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb7ac0b2960 0x7fb7ac0b4e70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.067+0000 7fb7cb000640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c4109170 0x7fb7c41a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.067+0000 7fb7cb000640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7c4108820 0x7fb7c419bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.067+0000 7fb7cb000640 1 --2- 172.21.15.39:0/240152189 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7c4102ee0 0x7fb7c419ba00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.067+0000 7fb7cb000640 1 -- 172.21.15.39:0/240152189 >> 172.21.15.39:0/240152189 conn(0x7fb7c40fe930 msgr2=0x7fb7c4105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.067+0000 7fb7cb000640 1 -- 172.21.15.39:0/240152189 shutdown_connections 2024-07-04T08:30:22.063 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:22.067+0000 7fb7cb000640 1 -- 172.21.15.39:0/240152189 wait complete. 2024-07-04T08:30:22.893 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:30:23.369 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:23.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:23 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:30:23.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:23 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:30:23.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:23 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:30:23.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:23 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:30:23.549 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:23 smithi039 ceph-mon[32062]: pgmap v953: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-07-04T08:30:23.550 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:23 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:30:23.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:23 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:30:23.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:23 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:30:23.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:23 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:30:23.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:23 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:30:23.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:23 smithi133 ceph-mon[37050]: pgmap v953: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-07-04T08:30:23.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:23 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:30:23.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:23 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:30:23.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:23 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:30:23.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:23 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:30:23.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:23 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:30:23.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:23 smithi050 ceph-mon[36577]: pgmap v953: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 12 op/s 2024-07-04T08:30:23.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:23 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:30:25.267 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:30:25.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:25 smithi133 ceph-mon[37050]: pgmap v954: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-07-04T08:30:25.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.598+0000 7f1c98e00640 1 -- 172.21.15.39:0/3740728760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c9410f690 msgr2=0x7f1c94111ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:25.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.598+0000 7f1c98e00640 1 --2- 172.21.15.39:0/3740728760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c9410f690 0x7f1c94111ab0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009900 tx=0x7f1c8c02a410 comp rx=0 tx=0).stop 2024-07-04T08:30:25.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:25 smithi039 ceph-mon[32062]: pgmap v954: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-07-04T08:30:25.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.598+0000 7f1c98e00640 1 -- 172.21.15.39:0/3740728760 shutdown_connections 2024-07-04T08:30:25.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.598+0000 7f1c98e00640 1 --2- 172.21.15.39:0/3740728760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c940755f0 0x7f1c9410f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.598+0000 7f1c98e00640 1 --2- 172.21.15.39:0/3740728760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c9410f690 0x7f1c94111ab0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.598+0000 7f1c98e00640 1 --2- 172.21.15.39:0/3740728760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c94074ca0 0x7f1c940750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.598+0000 7f1c98e00640 1 -- 172.21.15.39:0/3740728760 >> 172.21.15.39:0/3740728760 conn(0x7f1c940feb20 msgr2=0x7f1c94100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:25.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.598+0000 7f1c98e00640 1 -- 172.21.15.39:0/3740728760 shutdown_connections 2024-07-04T08:30:25.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.599+0000 7f1c98e00640 1 -- 172.21.15.39:0/3740728760 wait complete. 2024-07-04T08:30:25.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.600+0000 7f1c98e00640 1 Processor -- start 2024-07-04T08:30:25.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.601+0000 7f1c98e00640 1 -- start start 2024-07-04T08:30:25.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.601+0000 7f1c98e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c94074ca0 0x7f1c9419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.601+0000 7f1c98e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c940755f0 0x7f1c9419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.601+0000 7f1c98e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c9410f690 0x7f1c941a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.601+0000 7f1c98e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1c94115240 con 0x7f1c9410f690 2024-07-04T08:30:25.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.601+0000 7f1c98e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941150c0 con 0x7f1c940755f0 2024-07-04T08:30:25.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.601+0000 7f1c98e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941153c0 con 0x7f1c94074ca0 2024-07-04T08:30:25.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c92800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c9410f690 0x7f1c941a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c94074ca0 0x7f1c9419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c940755f0 0x7f1c9419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c92800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c9410f690 0x7f1c941a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48866/0 (socket says 172.21.15.39:48866) 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c940755f0 0x7f1c9419c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38624/0 (socket says 172.21.15.39:38624) 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c94074ca0 0x7f1c9419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53076/0 (socket says 172.21.15.39:53076) 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91400640 1 -- 172.21.15.39:0/4030318413 learned_addr learned my addr 172.21.15.39:0/4030318413 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91400640 1 -- 172.21.15.39:0/4030318413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c94074ca0 msgr2=0x7f1c9419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91400640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c94074ca0 0x7f1c9419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91400640 1 -- 172.21.15.39:0/4030318413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c9410f690 msgr2=0x7f1c941a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91400640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c9410f690 0x7f1c941a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91400640 1 -- 172.21.15.39:0/4030318413 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c941a1b00 con 0x7f1c940755f0 2024-07-04T08:30:25.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.602+0000 7f1c91e00640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c94074ca0 0x7f1c9419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:25.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.603+0000 7f1c91400640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c940755f0 0x7f1c9419c130 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400ebc0 tx=0x7f1c8400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:25.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.603+0000 7f1c8aa00640 1 -- 172.21.15.39:0/4030318413 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84010630 con 0x7f1c940755f0 2024-07-04T08:30:25.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.603+0000 7f1c98e00640 1 -- 172.21.15.39:0/4030318413 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c941a1df0 con 0x7f1c940755f0 2024-07-04T08:30:25.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.603+0000 7f1c8aa00640 1 -- 172.21.15.39:0/4030318413 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c84010c80 con 0x7f1c940755f0 2024-07-04T08:30:25.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.603+0000 7f1c8aa00640 1 -- 172.21.15.39:0/4030318413 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84017ba0 con 0x7f1c940755f0 2024-07-04T08:30:25.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.603+0000 7f1c98e00640 1 -- 172.21.15.39:0/4030318413 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c941a8550 con 0x7f1c940755f0 2024-07-04T08:30:25.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.607+0000 7f1c9f4f0740 1 Processor -- start 2024-07-04T08:30:25.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.607+0000 7f1c9f4f0740 1 -- start start 2024-07-04T08:30:25.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 0x561b185bb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c8aa00640 1 -- 172.21.15.39:0/4030318413 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1c840107d0 con 0x7f1c940755f0 2024-07-04T08:30:25.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b18531ed0 con 0x561b18217070 2024-07-04T08:30:25.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561b18521470 con 0x561b181f6390 2024-07-04T08:30:25.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561b185254b0 con 0x561b181ee1b0 2024-07-04T08:30:25.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c91e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c92800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 0x561b185bb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c91400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c91e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185bbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53090/0 (socket says 172.21.15.39:53090) 2024-07-04T08:30:25.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.608+0000 7f1c91e00640 1 -- 172.21.15.39:0/2769543812 learned_addr learned my addr 172.21.15.39:0/2769543812 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:25.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.609+0000 7f1c91400640 1 -- 172.21.15.39:0/2769543812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 msgr2=0x561b185bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:25.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.609+0000 7f1c91400640 1 --2- 172.21.15.39:0/2769543812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.609+0000 7f1c91400640 1 -- 172.21.15.39:0/2769543812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 msgr2=0x561b185bb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:25.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.609+0000 7f1c91400640 1 --2- 172.21.15.39:0/2769543812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 0x561b185bb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.609+0000 7f1c91400640 1 -- 172.21.15.39:0/2769543812 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b184b9640 con 0x561b181f6390 2024-07-04T08:30:25.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.609+0000 7f1c91400640 1 --2- 172.21.15.39:0/2769543812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185bc220 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1c840e0600 tx=0x7f1c840eb360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7077dcdcd8acaa7e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:25.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.609+0000 7f1c83400640 1 -- 172.21.15.39:0/2769543812 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c840f6070 con 0x561b181f6390 2024-07-04T08:30:25.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.610+0000 7f1c83400640 1 -- 172.21.15.39:0/2769543812 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c840e3660 con 0x561b181f6390 2024-07-04T08:30:25.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.610+0000 7f1c83400640 1 -- 172.21.15.39:0/2769543812 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c840f1660 con 0x561b181f6390 2024-07-04T08:30:25.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.610+0000 7f1c8aa00640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c780b2960 0x7f1c780b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.611+0000 7f1c8aa00640 1 -- 172.21.15.39:0/4030318413 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f1c840e3270 con 0x7f1c940755f0 2024-07-04T08:30:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.611+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2769543812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 msgr2=0x561b185bc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.611+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/2769543812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185bc220 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1c840e0600 tx=0x7f1c840eb360 comp rx=0 tx=0).stop 2024-07-04T08:30:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.611+0000 7f1c92800640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c780b2960 0x7f1c780b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.611+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2769543812 shutdown_connections 2024-07-04T08:30:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.611+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/2769543812 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185bc220 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.611+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/2769543812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.611+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/2769543812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 0x561b185bb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.599 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.611+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2769543812 >> 172.21.15.39:0/2769543812 conn(0x561b1820b460 msgr2=0x561b185b9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.612+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2769543812 shutdown_connections 2024-07-04T08:30:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.612+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2769543812 wait complete. 2024-07-04T08:30:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.612+0000 7f1c9f4f0740 1 -- start start 2024-07-04T08:30:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.612+0000 7f1c92800640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c780b2960 0x7f1c780b4e70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c006010 tx=0x7f1c8c005a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:25.600 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.613+0000 7f1c9f4f0740 1 Processor -- start 2024-07-04T08:30:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.613+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.613+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.613+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 0x561b185d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561b18385700 con 0x561b18217070 2024-07-04T08:30:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561b184c8ba0 con 0x561b181f6390 2024-07-04T08:30:25.601 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561b185c9f80 con 0x561b181ee1b0 2024-07-04T08:30:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c9f4f0740 10 client.0 _ll_register_callbacks cb 0x561b1851a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c91400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.602 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c91400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:38644/0 (socket says 172.21.15.39:38644) 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c91400640 1 -- 172.21.15.39:0/188682183 learned_addr learned my addr 172.21.15.39:0/188682183 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c91e00640 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c92800640 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 0x561b185d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c91400640 1 -- 172.21.15.39:0/188682183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 msgr2=0x561b185d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c91400640 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c91400640 1 -- 172.21.15.39:0/188682183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 msgr2=0x561b185d58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c91400640 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 0x561b185d58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c91400640 1 -- 172.21.15.39:0/188682183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b18282650 con 0x561b181f6390 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.614+0000 7f1c92800640 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 0x561b185d58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.615+0000 7f1c91400640 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185d0880 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1c840aa120 tx=0x7f1c840175b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7981fc04a3c2d31 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:25.603 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.615+0000 7f1c77e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:25.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.615+0000 7f1c77e00640 1 -- 172.21.15.39:0/188682183 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c840a6050 con 0x561b181f6390 2024-07-04T08:30:25.604 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.615+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/188682183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c840e3660 con 0x561b181f6390 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.615+0000 7f1c77e00640 1 -- 172.21.15.39:0/188682183 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f1c840ebd70 con 0x561b181f6390 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.616+0000 7f1c77e00640 1 -- 172.21.15.39:0/188682183 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1c840a9900 con 0x561b181f6390 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.616+0000 7f1c9f4f0740 10 client.28784 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.616+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/188682183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b185de7f0 con 0x561b181f6390 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.617+0000 7f1c77e00640 1 -- 172.21.15.39:0/188682183 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c840e3660 con 0x561b181f6390 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.617+0000 7f1c77e00640 1 -- 172.21.15.39:0/188682183 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1c840ae150 con 0x561b181f6390 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.617+0000 7f1c9f4f0740 10 client.28784 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.617+0000 7f1c9f4f0740 10 client.28784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.617+0000 7f1c9f4f0740 20 client.28784 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:25.605 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.617+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561b181d59f0 0x561b185e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.617+0000 7f1c9f4f0740 4 client.28784 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:25.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.617+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/188682183 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b18533e20 con 0x561b181d59f0 2024-07-04T08:30:25.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.618+0000 7f1c91e00640 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561b181d59f0 0x561b185e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.618+0000 7f1c91e00640 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561b181d59f0 0x561b185e1b50 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=549cce3432d9d611 server_cookie=8a2602e9627cad3f in_seq=0 out_seq=0 2024-07-04T08:30:25.606 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.618+0000 7f1c77e00640 10 client.28784 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:25.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.619+0000 7f1c77e00640 1 -- 172.21.15.39:0/188682183 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561b18533e20 con 0x561b181d59f0 2024-07-04T08:30:25.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.619+0000 7f1c77e00640 10 client.28784 handle_command_reply: tid=0 2024-07-04T08:30:25.607 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.619+0000 7f1c9f4f0740 1 client.28784 shutdown 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/188682183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 msgr2=0x561b185d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185d0880 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1c840aa120 tx=0x7f1c840175b0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/188682183 shutdown_connections 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561b181d59f0 0x561b185e1b50 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b18217070 0x561b185d58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181f6390 0x561b185d0880 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/188682183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/188682183 >> 172.21.15.39:0/188682183 conn(0x561b1820b460 msgr2=0x561b185be980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c77e00640 0 client.28784 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/188682183 shutdown_connections 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/188682183 wait complete. 2024-07-04T08:30:25.608 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.620+0000 7f1c9f4f0740 20 client.28784 trim_cache size 0 max 16384 2024-07-04T08:30:25.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.641+0000 7f1c9f4f0740 1 Processor -- start 2024-07-04T08:30:25.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.641+0000 7f1c9f4f0740 1 -- start start 2024-07-04T08:30:25.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.641+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181ee1b0 0x561b185c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.641+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.641+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b18217070 0x561b185d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.641+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941ae210 con 0x561b181f6390 2024-07-04T08:30:25.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.641+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941afd50 con 0x561b181ee1b0 2024-07-04T08:30:25.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.641+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941ae8e0 con 0x561b18217070 2024-07-04T08:30:25.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185b8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48886/0 (socket says 172.21.15.39:48886) 2024-07-04T08:30:25.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91400640 1 -- 172.21.15.39:0/2835264436 learned_addr learned my addr 172.21.15.39:0/2835264436 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:25.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181ee1b0 0x561b185c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c92800640 1 --2- 172.21.15.39:0/2835264436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b18217070 0x561b185d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91e00640 1 -- 172.21.15.39:0/2835264436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b18217070 msgr2=0x561b185d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91e00640 1 --2- 172.21.15.39:0/2835264436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b18217070 0x561b185d5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91e00640 1 -- 172.21.15.39:0/2835264436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 msgr2=0x561b185b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91e00640 1 --2- 172.21.15.39:0/2835264436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91e00640 1 -- 172.21.15.39:0/2835264436 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b185b9bd0 con 0x561b181ee1b0 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.642+0000 7f1c91400640 1 --2- 172.21.15.39:0/2835264436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.643+0000 7f1c91e00640 1 --2- 172.21.15.39:0/2835264436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181ee1b0 0x561b185c92a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1c840a6730 tx=0x7f1c7c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe958a161d21bd2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.643+0000 7f1c82a00640 1 -- 172.21.15.39:0/2835264436 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c010ae0 con 0x561b181ee1b0 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.643+0000 7f1c82a00640 1 -- 172.21.15.39:0/2835264436 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c012500 con 0x561b181ee1b0 2024-07-04T08:30:25.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.643+0000 7f1c82a00640 1 -- 172.21.15.39:0/2835264436 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c004c30 con 0x561b181ee1b0 2024-07-04T08:30:25.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:25 smithi050 ceph-mon[36577]: pgmap v954: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-07-04T08:30:25.632 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.644+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2835264436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181ee1b0 msgr2=0x561b185c92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.644+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/2835264436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181ee1b0 0x561b185c92a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1c840a6730 tx=0x7f1c7c005f90 comp rx=0 tx=0).stop 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.644+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2835264436 shutdown_connections 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.644+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/2835264436 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.644+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/2835264436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b18217070 0x561b185d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.644+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/2835264436 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b181ee1b0 0x561b185c92a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.644+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2835264436 >> 172.21.15.39:0/2835264436 conn(0x561b1820b460 msgr2=0x561b185ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.644+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2835264436 shutdown_connections 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.644+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/2835264436 wait complete. 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.645+0000 7f1c9f4f0740 1 -- start start 2024-07-04T08:30:25.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.645+0000 7f1c9f4f0740 1 Processor -- start 2024-07-04T08:30:25.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c9f4f0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b18217070 0x561b18648080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941af680 con 0x561b181f6390 2024-07-04T08:30:25.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941aefb0 con 0x561b18217070 2024-07-04T08:30:25.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c9f4f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941ac1f0 con 0x561b181ee1b0 2024-07-04T08:30:25.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c9f4f0740 10 client.0 _ll_register_callbacks cb 0x7f1c840f4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:25.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c91e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c91400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c91e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53110/0 (socket says 172.21.15.39:53110) 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c91e00640 1 -- 172.21.15.39:0/821760204 learned_addr learned my addr 172.21.15.39:0/821760204 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.646+0000 7f1c92800640 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b18217070 0x561b18648080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.647+0000 7f1c91e00640 1 -- 172.21.15.39:0/821760204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b18217070 msgr2=0x561b18648080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.647+0000 7f1c91e00640 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b18217070 0x561b18648080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.647+0000 7f1c91e00640 1 -- 172.21.15.39:0/821760204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 msgr2=0x561b185d7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.647+0000 7f1c91e00640 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185d7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.647+0000 7f1c91e00640 1 -- 172.21.15.39:0/821760204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b186485c0 con 0x561b181ee1b0 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.647+0000 7f1c91400640 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185d7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:25.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.647+0000 7f1c91e00640 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d7190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009380 tx=0x7f1c7c0108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c187b3346390b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:25.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.647+0000 7f1c77e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:30:25.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.648+0000 7f1c77e00640 1 -- 172.21.15.39:0/821760204 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c010cd0 con 0x561b181ee1b0 2024-07-04T08:30:25.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.648+0000 7f1c77e00640 1 -- 172.21.15.39:0/821760204 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f1c7c01f430 con 0x561b181ee1b0 2024-07-04T08:30:25.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.648+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/821760204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7c012500 con 0x561b181ee1b0 2024-07-04T08:30:25.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.648+0000 7f1c77e00640 1 -- 172.21.15.39:0/821760204 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1c7c03b050 con 0x561b181ee1b0 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.648+0000 7f1c9f4f0740 10 client.28729 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.648+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/821760204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561b186497e0 con 0x561b181ee1b0 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.649+0000 7f1c77e00640 1 -- 172.21.15.39:0/821760204 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c012500 con 0x561b181ee1b0 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.649+0000 7f1c77e00640 1 -- 172.21.15.39:0/821760204 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1c7c032820 con 0x561b181ee1b0 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.649+0000 7f1c9f4f0740 10 client.28729 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.649+0000 7f1c9f4f0740 10 client.28729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.649+0000 7f1c9f4f0740 20 client.28729 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.649+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561b181aada0 0x561b1864bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.649+0000 7f1c9f4f0740 4 client.28729 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.649+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/821760204 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b1852f8c0 con 0x561b181aada0 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.650+0000 7f1c91400640 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561b181aada0 0x561b1864bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:25.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.650+0000 7f1c91400640 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561b181aada0 0x561b1864bf30 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297048d86fbc85e3 server_cookie=c8d2b5bbdfeb1ad4 in_seq=0 out_seq=0 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.650+0000 7f1c77e00640 10 client.28729 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.650+0000 7f1c77e00640 1 -- 172.21.15.39:0/821760204 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561b1852f8c0 con 0x561b181aada0 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.650+0000 7f1c77e00640 10 client.28729 handle_command_reply: tid=0 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.650+0000 7f1c9f4f0740 1 client.28729 shutdown 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/821760204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 msgr2=0x561b185d7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d7190 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009380 tx=0x7f1c7c0108e0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/821760204 shutdown_connections 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561b181aada0 0x561b1864bf30 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561b181f6390 0x561b185d7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561b18217070 0x561b18648080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c77e00640 0 client.28729 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:25.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 --2- 172.21.15.39:0/821760204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561b181ee1b0 0x561b185d7190 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/821760204 >> 172.21.15.39:0/821760204 conn(0x561b1820b460 msgr2=0x561b185bc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:25.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/821760204 shutdown_connections 2024-07-04T08:30:25.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 1 -- 172.21.15.39:0/821760204 wait complete. 2024-07-04T08:30:25.641 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.651+0000 7f1c9f4f0740 20 client.28729 trim_cache size 0 max 16384 2024-07-04T08:30:25.641 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:30:25.641 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:30:25.641 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:30:25.641 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:30:25.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 -- 172.21.15.39:0/4030318413 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c780b2960 msgr2=0x7f1c780b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:25.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c780b2960 0x7f1c780b4e70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c006010 tx=0x7f1c8c005a40 comp rx=0 tx=0).stop 2024-07-04T08:30:25.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 -- 172.21.15.39:0/4030318413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c940755f0 msgr2=0x7f1c9419c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:25.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c940755f0 0x7f1c9419c130 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400ebc0 tx=0x7f1c8400c6a0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.642 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 -- 172.21.15.39:0/4030318413 shutdown_connections 2024-07-04T08:30:25.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1c780b2960 0x7f1c780b4e70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c9410f690 0x7f1c941a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c940755f0 0x7f1c9419c130 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 --2- 172.21.15.39:0/4030318413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c94074ca0 0x7f1c9419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:25.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 -- 172.21.15.39:0/4030318413 >> 172.21.15.39:0/4030318413 conn(0x7f1c940feb20 msgr2=0x7f1c941091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:25.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 -- 172.21.15.39:0/4030318413 shutdown_connections 2024-07-04T08:30:25.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:25.652+0000 7f1c98e00640 1 -- 172.21.15.39:0/4030318413 wait complete. 2024-07-04T08:30:26.414 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:26.457 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:30:26.458 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:30:27.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:27 smithi133 ceph-mon[37050]: pgmap v955: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-07-04T08:30:27.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:27 smithi039 ceph-mon[32062]: pgmap v955: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-07-04T08:30:27.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:27 smithi050 ceph-mon[36577]: pgmap v955: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 13 op/s 2024-07-04T08:30:27.813 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:29.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:29 smithi039 ceph-mon[32062]: pgmap v956: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2024-07-04T08:30:29.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:29 smithi050 ceph-mon[36577]: pgmap v956: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2024-07-04T08:30:29.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.490+0000 7f22b3e00640 1 -- 172.21.15.39:0/2802808781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b410f7a0 msgr2=0x7f22b4115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:29.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.490+0000 7f22b3e00640 1 --2- 172.21.15.39:0/2802808781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b410f7a0 0x7f22b4115fd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009be0 tx=0x7f22ac02a6d0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.490+0000 7f22b3e00640 1 -- 172.21.15.39:0/2802808781 shutdown_connections 2024-07-04T08:30:29.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.490+0000 7f22b3e00640 1 --2- 172.21.15.39:0/2802808781 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22b410ee50 0x7f22b410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.490+0000 7f22b3e00640 1 --2- 172.21.15.39:0/2802808781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b410f7a0 0x7f22b4115fd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.490+0000 7f22b3e00640 1 --2- 172.21.15.39:0/2802808781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b4109510 0x7f22b4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.490+0000 7f22b3e00640 1 -- 172.21.15.39:0/2802808781 >> 172.21.15.39:0/2802808781 conn(0x7f22b4078b10 msgr2=0x7f22b4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:29.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.490+0000 7f22b3e00640 1 -- 172.21.15.39:0/2802808781 shutdown_connections 2024-07-04T08:30:29.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.490+0000 7f22b3e00640 1 -- 172.21.15.39:0/2802808781 wait complete. 2024-07-04T08:30:29.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.491+0000 7f22b3e00640 1 Processor -- start 2024-07-04T08:30:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.492+0000 7f22b3e00640 1 -- start start 2024-07-04T08:30:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.492+0000 7f22b3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22b4109510 0x7f22b419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.492+0000 7f22b3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b410ee50 0x7f22b41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.492+0000 7f22b3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b410f7a0 0x7f22b41a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.492+0000 7f22b3e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f22b4119470 con 0x7f22b4109510 2024-07-04T08:30:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.492+0000 7f22b3e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41192f0 con 0x7f22b410f7a0 2024-07-04T08:30:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.492+0000 7f22b3e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22b41195f0 con 0x7f22b410ee50 2024-07-04T08:30:29.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.492+0000 7f22b2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22b4109510 0x7f22b419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22b2a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22b4109510 0x7f22b419fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48900/0 (socket says 172.21.15.39:48900) 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22abe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b410ee50 0x7f22b41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22b2a00640 1 -- 172.21.15.39:0/3140738018 learned_addr learned my addr 172.21.15.39:0/3140738018 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22b3400640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b410f7a0 0x7f22b41a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22abe00640 1 -- 172.21.15.39:0/3140738018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b410f7a0 msgr2=0x7f22b41a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22abe00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b410f7a0 0x7f22b41a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22abe00640 1 -- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22b4109510 msgr2=0x7f22b419fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22abe00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22b4109510 0x7f22b419fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22abe00640 1 -- 172.21.15.39:0/3140738018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b41a5dd0 con 0x7f22b410ee50 2024-07-04T08:30:29.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.493+0000 7f22b2a00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22b4109510 0x7f22b419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.494+0000 7f22abe00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b410ee50 0x7f22b41a0400 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f22b4069b60 tx=0x7f229c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.494+0000 7f22ab400640 1 -- 172.21.15.39:0/3140738018 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c00ccd0 con 0x7f22b410ee50 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.494+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b41a60c0 con 0x7f22b410ee50 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.494+0000 7f22ab400640 1 -- 172.21.15.39:0/3140738018 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f229c0035f0 con 0x7f22b410ee50 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.494+0000 7f22ab400640 1 -- 172.21.15.39:0/3140738018 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c010690 con 0x7f22b410ee50 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.494+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b41ac820 con 0x7f22b410ee50 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.495+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b4109920 con 0x7f22b410ee50 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.498+0000 7f22ab400640 1 -- 172.21.15.39:0/3140738018 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f229c003a80 con 0x7f22b410ee50 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.500+0000 7f22ab400640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22980b2960 0x7f22980b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.500+0000 7f22ab400640 1 -- 172.21.15.39:0/3140738018 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f229c0e2a90 con 0x7f22b410ee50 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.500+0000 7f22b2a00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22980b2960 0x7f22980b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:29.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.501+0000 7f22b2a00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22980b2960 0x7f22980b4e70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f22a4011500 tx=0x7f22a4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:29.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.501+0000 7f22ab400640 1 -- 172.21.15.39:0/3140738018 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f229c009510 con 0x7f22b410ee50 2024-07-04T08:30:29.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.759+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22b41a0f90 con 0x7f22b410ee50 2024-07-04T08:30:29.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.760+0000 7f22ab400640 1 -- 172.21.15.39:0/3140738018 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f229c0aa0b0 con 0x7f22b410ee50 2024-07-04T08:30:29.749 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:30:29.749 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:30:29.750 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22980b2960 msgr2=0x7f22980b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22980b2960 0x7f22980b4e70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f22a4011500 tx=0x7f22a4005f90 comp rx=0 tx=0).stop 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b410ee50 msgr2=0x7f22b41a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b410ee50 0x7f22b41a0400 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f22b4069b60 tx=0x7f229c00ee50 comp rx=0 tx=0).stop 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 shutdown_connections 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f22980b2960 0x7f22980b4e70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b410f7a0 0x7f22b41a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b410ee50 0x7f22b41a0400 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 --2- 172.21.15.39:0/3140738018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22b4109510 0x7f22b419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 >> 172.21.15.39:0/3140738018 conn(0x7f22b4078b10 msgr2=0x7f22b4079210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:29.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.763+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 shutdown_connections 2024-07-04T08:30:29.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:29.764+0000 7f22b3e00640 1 -- 172.21.15.39:0/3140738018 wait complete. 2024-07-04T08:30:29.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:29 smithi133 ceph-mon[37050]: pgmap v956: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2024-07-04T08:30:30.324 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:30 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3140738018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:30.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:30 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3140738018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:30.693 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.705+0000 7fd5e4c00640 1 -- 172.21.15.39:0/4158809632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e010ee50 msgr2=0x7fd5e010f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:30.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.705+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/4158809632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e010ee50 0x7fd5e010f260 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009900 tx=0x7fd5c802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.705+0000 7fd5e4c00640 1 -- 172.21.15.39:0/4158809632 shutdown_connections 2024-07-04T08:30:30.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.705+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/4158809632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5e010f7a0 0x7fd5e0115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.705+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/4158809632 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e010ee50 0x7fd5e010f260 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.705+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/4158809632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e0109510 0x7fd5e0109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.705+0000 7fd5e4c00640 1 -- 172.21.15.39:0/4158809632 >> 172.21.15.39:0/4158809632 conn(0x7fd5e0078b10 msgr2=0x7fd5e0107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:30.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.705+0000 7fd5e4c00640 1 -- 172.21.15.39:0/4158809632 shutdown_connections 2024-07-04T08:30:30.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.705+0000 7fd5e4c00640 1 -- 172.21.15.39:0/4158809632 wait complete. 2024-07-04T08:30:30.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.706+0000 7fd5e4c00640 1 Processor -- start 2024-07-04T08:30:30.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.707+0000 7fd5e4c00640 1 -- start start 2024-07-04T08:30:30.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.707+0000 7fd5e4c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e0109510 0x7fd5e019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.707+0000 7fd5e4c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e010ee50 0x7fd5e01a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.707+0000 7fd5e4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5e010f7a0 0x7fd5e01a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.707+0000 7fd5e4c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e0119430 con 0x7fd5e010ee50 2024-07-04T08:30:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.707+0000 7fd5e4c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01192b0 con 0x7fd5e0109510 2024-07-04T08:30:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.707+0000 7fd5e4c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01195b0 con 0x7fd5e010f7a0 2024-07-04T08:30:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5d6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e010ee50 0x7fd5e01a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:30.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5d6a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e010ee50 0x7fd5e01a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:48926/0 (socket says 172.21.15.39:48926) 2024-07-04T08:30:30.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5d6a00640 1 -- 172.21.15.39:0/1338975438 learned_addr learned my addr 172.21.15.39:0/1338975438 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:30.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5df400640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e0109510 0x7fd5e019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5dfe00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5e010f7a0 0x7fd5e01a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5df400640 1 -- 172.21.15.39:0/1338975438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5e010f7a0 msgr2=0x7fd5e01a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5df400640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5e010f7a0 0x7fd5e01a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5df400640 1 -- 172.21.15.39:0/1338975438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e010ee50 msgr2=0x7fd5e01a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5df400640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e010ee50 0x7fd5e01a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.708+0000 7fd5df400640 1 -- 172.21.15.39:0/1338975438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e01a5db0 con 0x7fd5e0109510 2024-07-04T08:30:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.709+0000 7fd5df400640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e0109510 0x7fd5e019fe40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000e9f0 tx=0x7fd5d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.709+0000 7fd5dcc00640 1 -- 172.21.15.39:0/1338975438 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000cd80 con 0x7fd5e0109510 2024-07-04T08:30:30.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.709+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e01a60a0 con 0x7fd5e0109510 2024-07-04T08:30:30.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.709+0000 7fd5dcc00640 1 -- 172.21.15.39:0/1338975438 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d00035f0 con 0x7fd5e0109510 2024-07-04T08:30:30.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.709+0000 7fd5dcc00640 1 -- 172.21.15.39:0/1338975438 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0010690 con 0x7fd5e0109510 2024-07-04T08:30:30.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.709+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e01ac800 con 0x7fd5e0109510 2024-07-04T08:30:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.710+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e0109920 con 0x7fd5e0109510 2024-07-04T08:30:30.702 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.713+0000 7fd5dcc00640 1 -- 172.21.15.39:0/1338975438 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd5d00037a0 con 0x7fd5e0109510 2024-07-04T08:30:30.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.715+0000 7fd5dcc00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5c40b29e0 0x7fd5c40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:30.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.715+0000 7fd5dcc00640 1 -- 172.21.15.39:0/1338975438 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7fd5d00e2db0 con 0x7fd5e0109510 2024-07-04T08:30:30.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.715+0000 7fd5d6a00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5c40b29e0 0x7fd5c40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:30.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.716+0000 7fd5d6a00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5c40b29e0 0x7fd5c40b4ef0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd5e010f5a0 tx=0x7fd5c8009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:30.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.716+0000 7fd5dcc00640 1 -- 172.21.15.39:0/1338975438 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fd5d0055110 con 0x7fd5e0109510 2024-07-04T08:30:30.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:30 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3140738018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.973+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5e01a0f70 con 0x7fd5e0109510 2024-07-04T08:30:30.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.974+0000 7fd5dcc00640 1 -- 172.21.15.39:0/1338975438 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fd5d00aa3d0 con 0x7fd5e0109510 2024-07-04T08:30:30.963 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:30:30.963 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:30:30.963 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:30:30.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.977+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5c40b29e0 msgr2=0x7fd5c40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:30.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.977+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5c40b29e0 0x7fd5c40b4ef0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd5e010f5a0 tx=0x7fd5c8009560 comp rx=0 tx=0).stop 2024-07-04T08:30:30.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.977+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e0109510 msgr2=0x7fd5e019fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:30.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.977+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e0109510 0x7fd5e019fe40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000e9f0 tx=0x7fd5d000eeb0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.977+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 shutdown_connections 2024-07-04T08:30:30.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.978+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd5c40b29e0 0x7fd5c40b4ef0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.978+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e010ee50 0x7fd5e01a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.978+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5e010f7a0 0x7fd5e01a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.978+0000 7fd5e4c00640 1 --2- 172.21.15.39:0/1338975438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5e0109510 0x7fd5e019fe40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:30.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.978+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 >> 172.21.15.39:0/1338975438 conn(0x7fd5e0078b10 msgr2=0x7fd5e0079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:30.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.978+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 shutdown_connections 2024-07-04T08:30:30.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:30.978+0000 7fd5e4c00640 1 -- 172.21.15.39:0/1338975438 wait complete. 2024-07-04T08:30:31.049 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id 3d6f3bcd --release --await' 2024-07-04T08:30:31.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id 3d6f3bcd --release --await 2024-07-04T08:30:31.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:31 smithi039 ceph-mon[32062]: pgmap v957: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 KiB/s wr, 10 op/s 2024-07-04T08:30:31.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:31 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1338975438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:31.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:31 smithi050 ceph-mon[36577]: pgmap v957: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 KiB/s wr, 10 op/s 2024-07-04T08:30:31.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:31 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1338975438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:31.759 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:30:31.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:31 smithi133 ceph-mon[37050]: pgmap v957: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 KiB/s wr, 10 op/s 2024-07-04T08:30:31.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:31 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1338975438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:31.859 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:32.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:32 smithi039 ceph-mon[32062]: pgmap v958: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-07-04T08:30:32.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:32 smithi050 ceph-mon[36577]: pgmap v958: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-07-04T08:30:32.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:32 smithi133 ceph-mon[37050]: pgmap v958: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-07-04T08:30:33.156 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.539+0000 7f4413e00640 1 -- 172.21.15.39:0/2976718790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4414108840 msgr2=0x7f4414108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.539+0000 7f4413e00640 1 --2- 172.21.15.39:0/2976718790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4414108840 0x7f4414108c50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009900 tx=0x7f43fc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.539+0000 7f4413e00640 1 -- 172.21.15.39:0/2976718790 shutdown_connections 2024-07-04T08:30:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.539+0000 7f4413e00640 1 --2- 172.21.15.39:0/2976718790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4414108840 0x7f4414108c50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.539+0000 7f4413e00640 1 --2- 172.21.15.39:0/2976718790 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4414109190 0x7f441410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.528 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.539+0000 7f4413e00640 1 --2- 172.21.15.39:0/2976718790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414102f00 0x7f4414103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.539+0000 7f4413e00640 1 -- 172.21.15.39:0/2976718790 >> 172.21.15.39:0/2976718790 conn(0x7f44140fe990 msgr2=0x7f4414100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.539+0000 7f4413e00640 1 -- 172.21.15.39:0/2976718790 shutdown_connections 2024-07-04T08:30:34.529 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.539+0000 7f4413e00640 1 -- 172.21.15.39:0/2976718790 wait complete. 2024-07-04T08:30:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.540+0000 7f4413e00640 1 Processor -- start 2024-07-04T08:30:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.541+0000 7f4413e00640 1 -- start start 2024-07-04T08:30:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.541+0000 7f4413e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4414102f00 0x7f441419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.541+0000 7f4413e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4414108840 0x7f441419c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.541+0000 7f4413e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414109190 0x7f44141a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.541+0000 7f4413e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4414112e50 con 0x7f4414108840 2024-07-04T08:30:34.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.541+0000 7f4413e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4414112cd0 con 0x7f4414102f00 2024-07-04T08:30:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.541+0000 7f4413e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4414112fd0 con 0x7f4414109190 2024-07-04T08:30:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4412a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4414102f00 0x7f441419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4412a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4414102f00 0x7f441419baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44626/0 (socket says 172.21.15.39:44626) 2024-07-04T08:30:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4412a00640 1 -- 172.21.15.39:0/193919724 learned_addr learned my addr 172.21.15.39:0/193919724 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4413400640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414109190 0x7f44141a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.531 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4412000640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4414108840 0x7f441419c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4412a00640 1 -- 172.21.15.39:0/193919724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414109190 msgr2=0x7f44141a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4412a00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414109190 0x7f44141a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4412a00640 1 -- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4414108840 msgr2=0x7f441419c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4412a00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4414108840 0x7f441419c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.532 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.542+0000 7f4412a00640 1 -- 172.21.15.39:0/193919724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44141a1a10 con 0x7f4414102f00 2024-07-04T08:30:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.543+0000 7f4412a00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4414102f00 0x7f441419baa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f440400e9f0 tx=0x7f440400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.543+0000 7f4418a00640 1 -- 172.21.15.39:0/193919724 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440400ccc0 con 0x7f4414102f00 2024-07-04T08:30:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.543+0000 7f4413e00640 1 -- 172.21.15.39:0/193919724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44141a1d00 con 0x7f4414102f00 2024-07-04T08:30:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.543+0000 7f4418a00640 1 -- 172.21.15.39:0/193919724 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440400ce60 con 0x7f4414102f00 2024-07-04T08:30:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.543+0000 7f4418a00640 1 -- 172.21.15.39:0/193919724 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404010430 con 0x7f4414102f00 2024-07-04T08:30:34.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.543+0000 7f4413e00640 1 -- 172.21.15.39:0/193919724 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141a8460 con 0x7f4414102f00 2024-07-04T08:30:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.547+0000 7f4420427740 1 Processor -- start 2024-07-04T08:30:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.547+0000 7f4420427740 1 -- start start 2024-07-04T08:30:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.547+0000 7f4418a00640 1 -- 172.21.15.39:0/193919724 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f44040046c0 con 0x7f4414102f00 2024-07-04T08:30:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4420427740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93dbc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4420427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939ee660 0x561f93dbc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4420427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 0x561f93dbcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4420427740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561f93cd6b20 con 0x561f93a17520 2024-07-04T08:30:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4420427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561f93d322b0 con 0x561f939ee660 2024-07-04T08:30:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4420427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561f93d21850 con 0x561f939f6840 2024-07-04T08:30:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4412a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939ee660 0x561f93dbc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4412000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 0x561f93dbcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4413400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93dbc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4412000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 0x561f93dbcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50504/0 (socket says 172.21.15.39:50504) 2024-07-04T08:30:34.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4413400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93dbc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45558/0 (socket says 172.21.15.39:45558) 2024-07-04T08:30:34.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.548+0000 7f4412000640 1 -- 172.21.15.39:0/2944230217 learned_addr learned my addr 172.21.15.39:0/2944230217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:34.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.549+0000 7f4412a00640 1 -- 172.21.15.39:0/2944230217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 msgr2=0x561f93dbcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:34.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.549+0000 7f4412a00640 1 --2- 172.21.15.39:0/2944230217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 0x561f93dbcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.549+0000 7f4412a00640 1 -- 172.21.15.39:0/2944230217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 msgr2=0x561f93dbc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:34.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.549+0000 7f4412a00640 1 --2- 172.21.15.39:0/2944230217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93dbc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.538 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.549+0000 7f4412a00640 1 -- 172.21.15.39:0/2944230217 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f93cdb780 con 0x561f939ee660 2024-07-04T08:30:34.539 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.549+0000 7f4412a00640 1 --2- 172.21.15.39:0/2944230217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939ee660 0x561f93dbc9d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f44040e00d0 tx=0x7f44040e6de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4406f7b7bac7cdeb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.549+0000 7f4408c00640 1 -- 172.21.15.39:0/2944230217 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040f4440 con 0x561f939ee660 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.550+0000 7f4408c00640 1 -- 172.21.15.39:0/2944230217 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44040e88c0 con 0x561f939ee660 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.550+0000 7f4408c00640 1 -- 172.21.15.39:0/2944230217 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040f36f0 con 0x561f939ee660 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.550+0000 7f4418a00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43f80b29e0 0x7f43f80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.550+0000 7f4418a00640 1 -- 172.21.15.39:0/193919724 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f44040e2d40 con 0x7f4414102f00 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.550+0000 7f4413400640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43f80b29e0 0x7f43f80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4420427740 1 -- 172.21.15.39:0/2944230217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939ee660 msgr2=0x561f93dbc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4420427740 1 --2- 172.21.15.39:0/2944230217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939ee660 0x561f93dbc9d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f44040e00d0 tx=0x7f44040e6de0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4420427740 1 -- 172.21.15.39:0/2944230217 shutdown_connections 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4420427740 1 --2- 172.21.15.39:0/2944230217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 0x561f93dbcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.540 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4420427740 1 --2- 172.21.15.39:0/2944230217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939ee660 0x561f93dbc9d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4420427740 1 --2- 172.21.15.39:0/2944230217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93dbc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4420427740 1 -- 172.21.15.39:0/2944230217 >> 172.21.15.39:0/2944230217 conn(0x561f93a0b910 msgr2=0x561f93dba380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4413400640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43f80b29e0 0x7f43f80b4ef0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f440c005fd0 tx=0x7f440c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4420427740 1 -- 172.21.15.39:0/2944230217 shutdown_connections 2024-07-04T08:30:34.541 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.551+0000 7f4420427740 1 -- 172.21.15.39:0/2944230217 wait complete. 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.552+0000 7f4420427740 1 -- start start 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.552+0000 7f4420427740 1 Processor -- start 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4420427740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939ee660 0x561f93dc7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4420427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 0x561f93dc80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4420427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 0x561f93dc8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4420427740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x561f93cc8d20 con 0x561f939ee660 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4420427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561f93d25890 con 0x561f93a17520 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4420427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561f93b85bb0 con 0x561f939f6840 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4420427740 10 client.0 _ll_register_callbacks cb 0x561f93cc99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:34.544 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4412a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939ee660 0x561f93dc7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4412a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939ee660 0x561f93dc7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45572/0 (socket says 172.21.15.39:45572) 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4412a00640 1 -- 172.21.15.39:0/3003379934 learned_addr learned my addr 172.21.15.39:0/3003379934 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4412000640 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 0x561f93dc80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.553+0000 7f4413400640 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 0x561f93dc8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4412a00640 1 -- 172.21.15.39:0/3003379934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 msgr2=0x561f93dc80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4412a00640 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 0x561f93dc80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4412a00640 1 -- 172.21.15.39:0/3003379934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 msgr2=0x561f93dc8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4412a00640 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 0x561f93dc8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4412a00640 1 -- 172.21.15.39:0/3003379934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f93cb8660 con 0x561f939ee660 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4412a00640 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939ee660 0x561f93dc7bb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f44040a9bf0 tx=0x7f4404009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba3aab0fdaab7d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:34.545 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4401600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4401600640 1 -- 172.21.15.39:0/3003379934 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040a5b20 con 0x561f939ee660 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4401600640 1 -- 172.21.15.39:0/3003379934 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f44040a9ec0 con 0x561f939ee660 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.554+0000 7f4420427740 1 -- 172.21.15.39:0/3003379934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44040e88c0 con 0x561f939ee660 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.555+0000 7f4401600640 1 -- 172.21.15.39:0/3003379934 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f44040e0560 con 0x561f939ee660 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.555+0000 7f4420427740 10 client.18693 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.555+0000 7f4420427740 1 -- 172.21.15.39:0/3003379934 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561f93a82b00 con 0x561f939ee660 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.555+0000 7f4401600640 1 -- 172.21.15.39:0/3003379934 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44040e88c0 con 0x561f939ee660 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.555+0000 7f4401600640 1 -- 172.21.15.39:0/3003379934 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f44040f2070 con 0x561f939ee660 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.555+0000 7f4420427740 10 client.18693 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.555+0000 7f4420427740 10 client.18693 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.556+0000 7f4420427740 20 client.18693 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:34.546 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.556+0000 7f4420427740 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561f939d5ea0 0x561f93dd9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.556+0000 7f4420427740 4 client.18693 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:34.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.556+0000 7f4420427740 1 -- 172.21.15.39:0/3003379934 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f93d34200 con 0x561f939d5ea0 2024-07-04T08:30:34.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.556+0000 7f4412000640 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561f939d5ea0 0x561f93dd9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.556+0000 7f4412000640 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561f939d5ea0 0x561f93dd9980 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f037fb7295de0f23 server_cookie=b16ef1550f9304c3 in_seq=0 out_seq=0 2024-07-04T08:30:34.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.556+0000 7f4401600640 10 client.18693 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:34.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.557+0000 7f4401600640 1 -- 172.21.15.39:0/3003379934 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561f93d34200 con 0x561f939d5ea0 2024-07-04T08:30:34.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.557+0000 7f4401600640 10 client.18693 handle_command_reply: tid=0 2024-07-04T08:30:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 client.18693 shutdown 2024-07-04T08:30:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 -- 172.21.15.39:0/3003379934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939ee660 msgr2=0x561f93dc7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939ee660 0x561f93dc7bb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f44040a9bf0 tx=0x7f4404009790 comp rx=0 tx=0).stop 2024-07-04T08:30:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 -- 172.21.15.39:0/3003379934 shutdown_connections 2024-07-04T08:30:34.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561f939d5ea0 0x561f93dd9980 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 0x561f93dc8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939f6840 0x561f93dc80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 --2- 172.21.15.39:0/3003379934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939ee660 0x561f93dc7bb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4401600640 0 client.18693 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 -- 172.21.15.39:0/3003379934 >> 172.21.15.39:0/3003379934 conn(0x561f93a0b910 msgr2=0x561f93dbf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 -- 172.21.15.39:0/3003379934 shutdown_connections 2024-07-04T08:30:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.558+0000 7f4420427740 1 -- 172.21.15.39:0/3003379934 wait complete. 2024-07-04T08:30:34.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.559+0000 7f4420427740 20 client.18693 trim_cache size 0 max 16384 2024-07-04T08:30:34.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.586+0000 7f4420427740 1 Processor -- start 2024-07-04T08:30:34.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.586+0000 7f4420427740 1 -- start start 2024-07-04T08:30:34.574 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.586+0000 7f4420427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 0x561f93dc32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.586+0000 7f4420427740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939f6840 0x561f93dbde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.586+0000 7f4420427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 0x561f93ddc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.586+0000 7f4420427740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44141b2390 con 0x561f939f6840 2024-07-04T08:30:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.586+0000 7f4420427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44141b3ed0 con 0x561f93a17520 2024-07-04T08:30:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.586+0000 7f4420427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44141b2a60 con 0x561f939ee660 2024-07-04T08:30:34.575 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.586+0000 7f4412000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939f6840 0x561f93dbde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.587+0000 7f4412a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 0x561f93dc32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.587+0000 7f4412000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939f6840 0x561f93dbde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45574/0 (socket says 172.21.15.39:45574) 2024-07-04T08:30:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.587+0000 7f4412000640 1 -- 172.21.15.39:0/2990390498 learned_addr learned my addr 172.21.15.39:0/2990390498 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.587+0000 7f4413400640 1 --2- 172.21.15.39:0/2990390498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 0x561f93ddc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.587+0000 7f4412a00640 1 -- 172.21.15.39:0/2990390498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 msgr2=0x561f93ddc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.587+0000 7f4412a00640 1 --2- 172.21.15.39:0/2990390498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 0x561f93ddc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.587+0000 7f4412a00640 1 -- 172.21.15.39:0/2990390498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939f6840 msgr2=0x561f93dbde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.587+0000 7f4412a00640 1 --2- 172.21.15.39:0/2990390498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939f6840 0x561f93dbde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.576 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.587+0000 7f4412a00640 1 -- 172.21.15.39:0/2990390498 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f93dbed20 con 0x561f939ee660 2024-07-04T08:30:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.589+0000 7f4412a00640 1 --2- 172.21.15.39:0/2990390498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 0x561f93dc32d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f44040a5ce0 tx=0x7f4404009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec8f591cbc0d1c86 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:34.577 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.589+0000 7f4403e00640 1 -- 172.21.15.39:0/2990390498 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040a53f0 con 0x561f939ee660 2024-07-04T08:30:34.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.589+0000 7f4403e00640 1 -- 172.21.15.39:0/2990390498 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4404091040 con 0x561f939ee660 2024-07-04T08:30:34.578 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.589+0000 7f4403e00640 1 -- 172.21.15.39:0/2990390498 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040a93d0 con 0x561f939ee660 2024-07-04T08:30:34.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.590+0000 7f4420427740 1 -- 172.21.15.39:0/2990390498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 msgr2=0x561f93dc32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:34.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.590+0000 7f4420427740 1 --2- 172.21.15.39:0/2990390498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 0x561f93dc32d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f44040a5ce0 tx=0x7f4404009720 comp rx=0 tx=0).stop 2024-07-04T08:30:34.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.590+0000 7f4420427740 1 -- 172.21.15.39:0/2990390498 shutdown_connections 2024-07-04T08:30:34.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.590+0000 7f4420427740 1 --2- 172.21.15.39:0/2990390498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f939f6840 0x561f93dbde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.590+0000 7f4420427740 1 --2- 172.21.15.39:0/2990390498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f93a17520 0x561f93ddc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.590+0000 7f4420427740 1 --2- 172.21.15.39:0/2990390498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 0x561f93dc32d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.590+0000 7f4420427740 1 -- 172.21.15.39:0/2990390498 >> 172.21.15.39:0/2990390498 conn(0x561f93a0b910 msgr2=0x561f93db8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:34.579 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.591+0000 7f4420427740 1 -- 172.21.15.39:0/2990390498 shutdown_connections 2024-07-04T08:30:34.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.591+0000 7f4420427740 1 -- 172.21.15.39:0/2990390498 wait complete. 2024-07-04T08:30:34.580 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.591+0000 7f4420427740 1 -- start start 2024-07-04T08:30:34.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.591+0000 7f4420427740 1 Processor -- start 2024-07-04T08:30:34.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4420427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 0x561f93ddafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4420427740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939f6840 0x561f93ddb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4420427740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93e40ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4420427740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f44141b3800 con 0x561f93a17520 2024-07-04T08:30:34.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4420427740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f44141b3130 con 0x561f939f6840 2024-07-04T08:30:34.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4420427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44141b0370 con 0x561f939ee660 2024-07-04T08:30:34.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4420427740 10 client.0 _ll_register_callbacks cb 0x7f44040e6ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:34.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4412000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939f6840 0x561f93ddb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4412000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939f6840 0x561f93ddb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44660/0 (socket says 172.21.15.39:44660) 2024-07-04T08:30:34.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.592+0000 7f4412000640 1 -- 172.21.15.39:0/374097333 learned_addr learned my addr 172.21.15.39:0/374097333 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:34.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4413400640 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93e40ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4412000640 1 -- 172.21.15.39:0/374097333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 msgr2=0x561f93ddafc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4412000640 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 0x561f93ddafc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4412000640 1 -- 172.21.15.39:0/374097333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 msgr2=0x561f93e40ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4412000640 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93e40ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4412000640 1 -- 172.21.15.39:0/374097333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f93af9960 con 0x561f939f6840 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4413400640 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93e40ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4412000640 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939f6840 0x561f93ddb500 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009230 tx=0x7f43fc032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8efc657efce7b1ec server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4401600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.593+0000 7f4401600640 1 -- 172.21.15.39:0/374097333 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc04d060 con 0x561f939f6840 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.594+0000 7f4420427740 1 -- 172.21.15.39:0/374097333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4404091040 con 0x561f939f6840 2024-07-04T08:30:34.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.594+0000 7f4401600640 1 -- 172.21.15.39:0/374097333 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f43fc005040 con 0x561f939f6840 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.594+0000 7f4401600640 1 -- 172.21.15.39:0/374097333 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f43fc054050 con 0x561f939f6840 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.594+0000 7f4420427740 10 client.28820 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.594+0000 7f4420427740 1 -- 172.21.15.39:0/374097333 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561f93cee8a0 con 0x561f939f6840 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.594+0000 7f4401600640 1 -- 172.21.15.39:0/374097333 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4404091040 con 0x561f939f6840 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.595+0000 7f4401600640 1 -- 172.21.15.39:0/374097333 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f43fc0035f0 con 0x561f939f6840 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.595+0000 7f4420427740 10 client.28820 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.595+0000 7f4420427740 10 client.28820 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.595+0000 7f4420427740 20 client.28820 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.595+0000 7f4420427740 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561f939ab250 0x561f93e44b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.595+0000 7f4420427740 4 client.28820 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3d6f3bcd", "release": true, "await": true}] 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.595+0000 7f4420427740 1 -- 172.21.15.39:0/374097333 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3d6f3bcd", "release": true, "await": true}) -- 0x561f93d2fca0 con 0x561f939ab250 2024-07-04T08:30:34.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.595+0000 7f4412a00640 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561f939ab250 0x561f93e44b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:34.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.596+0000 7f4412a00640 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561f939ab250 0x561f93e44b40 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff725d085cae161e server_cookie=5de331a8e6173cc1 in_seq=0 out_seq=0 2024-07-04T08:30:34.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.596+0000 7f4401600640 10 client.28820 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.597+0000 7f4401600640 1 -- 172.21.15.39:0/374097333 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561f93d2fca0 con 0x561f939ab250 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.597+0000 7f4401600640 10 client.28820 handle_command_reply: tid=0 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.597+0000 7f4420427740 1 client.28820 shutdown 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 -- 172.21.15.39:0/374097333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939f6840 msgr2=0x561f93ddb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939f6840 0x561f93ddb500 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009230 tx=0x7f43fc032f90 comp rx=0 tx=0).stop 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 -- 172.21.15.39:0/374097333 shutdown_connections 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x561f939ab250 0x561f93e44b40 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x561f93a17520 0x561f93e40ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561f939f6840 0x561f93ddb500 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 --2- 172.21.15.39:0/374097333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561f939ee660 0x561f93ddafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 -- 172.21.15.39:0/374097333 >> 172.21.15.39:0/374097333 conn(0x561f93a0b910 msgr2=0x561f93dbd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:34.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4401600640 0 client.28820 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:34.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 -- 172.21.15.39:0/374097333 shutdown_connections 2024-07-04T08:30:34.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 1 -- 172.21.15.39:0/374097333 wait complete. 2024-07-04T08:30:34.588 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.598+0000 7f4420427740 20 client.28820 trim_cache size 0 max 16384 2024-07-04T08:30:34.588 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:30:34.588 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:30:34.588 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:30:34.588 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 44, 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "3d6f3bcd": { 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 44, 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 141.7, 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:30:34.589 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:30:34.590 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:30:34.590 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:30:34.590 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:30:34.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:34.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:34.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:34.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:34.590 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:34.590 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:30:34.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 -- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43f80b29e0 msgr2=0x7f43f80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:34.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43f80b29e0 0x7f43f80b4ef0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f440c005fd0 tx=0x7f440c009500 comp rx=0 tx=0).stop 2024-07-04T08:30:34.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 -- 172.21.15.39:0/193919724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4414102f00 msgr2=0x7f441419baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:34.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4414102f00 0x7f441419baa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f440400e9f0 tx=0x7f440400edf0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 -- 172.21.15.39:0/193919724 shutdown_connections 2024-07-04T08:30:34.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f43f80b29e0 0x7f43f80b4ef0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4414108840 0x7f441419c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4414109190 0x7f44141a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 --2- 172.21.15.39:0/193919724 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4414102f00 0x7f441419baa0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:34.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 -- 172.21.15.39:0/193919724 >> 172.21.15.39:0/193919724 conn(0x7f44140fe990 msgr2=0x7f4414105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:34.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.599+0000 7f4413e00640 1 -- 172.21.15.39:0/193919724 shutdown_connections 2024-07-04T08:30:34.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:34.600+0000 7f4413e00640 1 -- 172.21.15.39:0/193919724 wait complete. 2024-07-04T08:30:35.520 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:35 smithi039 ceph-mon[32062]: pgmap v959: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-07-04T08:30:35.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:35 smithi133 ceph-mon[37050]: pgmap v959: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-07-04T08:30:35.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:35 smithi050 ceph-mon[36577]: pgmap v959: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-07-04T08:30:36.074 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3d6f3bcd', total seconds elapsed: 32.5 2024-07-04T08:30:36.075 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.7 2024-07-04T08:30:36.075 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.840+0000 7f5bdb800640 1 -- 172.21.15.39:0/3886643375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4109360 msgr2=0x7f5bd410fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.840+0000 7f5bdb800640 1 --2- 172.21.15.39:0/3886643375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4109360 0x7f5bd410fb90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009900 tx=0x7f5bcc02a400 comp rx=0 tx=0).stop 2024-07-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.840+0000 7f5bdb800640 1 -- 172.21.15.39:0/3886643375 shutdown_connections 2024-07-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.840+0000 7f5bdb800640 1 --2- 172.21.15.39:0/3886643375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4109360 0x7f5bd410fb90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.840+0000 7f5bdb800640 1 --2- 172.21.15.39:0/3886643375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bd4108a10 0x7f5bd4108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.840+0000 7f5bdb800640 1 --2- 172.21.15.39:0/3886643375 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bd41030d0 0x7f5bd41034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.840+0000 7f5bdb800640 1 -- 172.21.15.39:0/3886643375 >> 172.21.15.39:0/3886643375 conn(0x7f5bd40feae0 msgr2=0x7f5bd4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.840+0000 7f5bdb800640 1 -- 172.21.15.39:0/3886643375 shutdown_connections 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.840+0000 7f5bdb800640 1 -- 172.21.15.39:0/3886643375 wait complete. 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.841+0000 7f5bdb800640 1 Processor -- start 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.841+0000 7f5bdb800640 1 -- start start 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bdb800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bd41030d0 0x7f5bd4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bdb800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bd4108a10 0x7f5bd419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bdb800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4109360 0x7f5bd419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bdb800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd4113010 con 0x7f5bd4109360 2024-07-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bdb800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd4112e90 con 0x7f5bd4108a10 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bdb800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd4113190 con 0x7f5bd41030d0 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bd4108a10 0x7f5bd419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bd9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4109360 0x7f5bd419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bd4108a10 0x7f5bd419a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44674/0 (socket says 172.21.15.39:44674) 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bd3e00640 1 -- 172.21.15.39:0/670890665 learned_addr learned my addr 172.21.15.39:0/670890665 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.842+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bd41030d0 0x7f5bd4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.843+0000 7f5bd3e00640 1 -- 172.21.15.39:0/670890665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bd41030d0 msgr2=0x7f5bd4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.843+0000 7f5bd3e00640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bd41030d0 0x7f5bd4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.843+0000 7f5bd3e00640 1 -- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4109360 msgr2=0x7f5bd419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.843+0000 7f5bd3e00640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4109360 0x7f5bd419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.843+0000 7f5bd3e00640 1 -- 172.21.15.39:0/670890665 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd419f9f0 con 0x7f5bd4108a10 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.843+0000 7f5bd9800640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4109360 0x7f5bd419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.843+0000 7f5bd3e00640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bd4108a10 0x7f5bd419a020 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00e9f0 tx=0x7f5bbc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.843+0000 7f5bd1600640 1 -- 172.21.15.39:0/670890665 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00ccc0 con 0x7f5bd4108a10 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.844+0000 7f5bd1600640 1 -- 172.21.15.39:0/670890665 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bbc0108e0 con 0x7f5bd4108a10 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.844+0000 7f5bdb800640 1 -- 172.21.15.39:0/670890665 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd419fce0 con 0x7f5bd4108a10 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.844+0000 7f5bd1600640 1 -- 172.21.15.39:0/670890665 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc018860 con 0x7f5bd4108a10 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.844+0000 7f5bdb800640 1 -- 172.21.15.39:0/670890665 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd41a6440 con 0x7f5bd4108a10 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.847+0000 7f5be1e46740 1 Processor -- start 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.847+0000 7f5be1e46740 1 -- start start 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a8f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a8f6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 0x56015a8f6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5be1e46740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56015a867f00 con 0x56015a54d070 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5be1e46740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56015a8574a0 con 0x56015a5241b0 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5be1e46740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56015a85b4e0 con 0x56015a52c390 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a8f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a8f6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:44688/0 (socket says 172.21.15.39:44688) 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd8e00640 1 -- 172.21.15.39:0/753152892 learned_addr learned my addr 172.21.15.39:0/753152892 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd9800640 1 --2- 172.21.15.39:0/753152892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a8f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd1600640 1 -- 172.21.15.39:0/670890665 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5bbc010430 con 0x7f5bd4108a10 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd8e00640 1 -- 172.21.15.39:0/753152892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 msgr2=0x56015a8f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/753152892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 0x56015a8f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd8e00640 1 -- 172.21.15.39:0/753152892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 msgr2=0x56015a8f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/753152892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a8f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.848+0000 7f5bd8e00640 1 -- 172.21.15.39:0/753152892 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015a7ef670 con 0x56015a5241b0 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.849+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/753152892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a8f6120 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40114f0 tx=0x7f5bc40119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f6dddeece8074d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.849+0000 7f5bca000640 1 -- 172.21.15.39:0/753152892 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc401e070 con 0x56015a5241b0 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.850+0000 7f5bca000640 1 -- 172.21.15.39:0/753152892 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bc400fad0 con 0x56015a5241b0 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.850+0000 7f5bca000640 1 -- 172.21.15.39:0/753152892 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4019b10 con 0x56015a5241b0 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.850+0000 7f5bd1600640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5bb80b29e0 0x7f5bb80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.850+0000 7f5bd1600640 1 -- 172.21.15.39:0/670890665 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f5bbc0e2b90 con 0x7f5bd4108a10 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.850+0000 7f5bd9800640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5bb80b29e0 0x7f5bb80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5be1e46740 1 -- 172.21.15.39:0/753152892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 msgr2=0x56015a8f6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5be1e46740 1 --2- 172.21.15.39:0/753152892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a8f6120 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40114f0 tx=0x7f5bc40119b0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5be1e46740 1 -- 172.21.15.39:0/753152892 shutdown_connections 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5be1e46740 1 --2- 172.21.15.39:0/753152892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 0x56015a8f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5be1e46740 1 --2- 172.21.15.39:0/753152892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a8f6120 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5be1e46740 1 --2- 172.21.15.39:0/753152892 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a8f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5be1e46740 1 -- 172.21.15.39:0/753152892 >> 172.21.15.39:0/753152892 conn(0x56015a541460 msgr2=0x56015a8f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5be1e46740 1 -- 172.21.15.39:0/753152892 shutdown_connections 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5be1e46740 1 -- 172.21.15.39:0/753152892 wait complete. 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.851+0000 7f5bd9800640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5bb80b29e0 0x7f5bb80b4ef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc006010 tx=0x7f5bcc0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.852+0000 7f5be1e46740 1 -- start start 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.852+0000 7f5be1e46740 1 Processor -- start 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a901800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a52c390 0x56015a901d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a54d070 0x56015a90d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5be1e46740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x56015a6bb700 con 0x56015a52c390 2024-07-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5be1e46740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56015a7febd0 con 0x56015a54d070 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5be1e46740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56015a9043c0 con 0x56015a5241b0 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5be1e46740 10 client.0 _ll_register_callbacks cb 0x56015a850590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a52c390 0x56015a901d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5bd8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a901800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5bd9800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a54d070 0x56015a90d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a52c390 0x56015a901d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45616/0 (socket says 172.21.15.39:45616) 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5bd3e00640 1 -- 172.21.15.39:0/2813466695 learned_addr learned my addr 172.21.15.39:0/2813466695 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.853+0000 7f5bd3e00640 1 -- 172.21.15.39:0/2813466695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 msgr2=0x56015a901800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bd3e00640 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a901800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bd3e00640 1 -- 172.21.15.39:0/2813466695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a54d070 msgr2=0x56015a90d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bd3e00640 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a54d070 0x56015a90d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bd3e00640 1 -- 172.21.15.39:0/2813466695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015a5b8650 con 0x56015a52c390 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a901800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bd9800640 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a54d070 0x56015a90d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bd3e00640 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a52c390 0x56015a901d40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0a9af0 tx=0x7f5bbc0107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ce6435def2aecf server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2813466695 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0a9b60 con 0x56015a52c390 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2813466695 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f5bbc0e01a0 con 0x56015a52c390 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.854+0000 7f5be1e46740 1 -- 172.21.15.39:0/2813466695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc400fad0 con 0x56015a52c390 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2813466695 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5bbc0ade60 con 0x56015a52c390 2024-07-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5be1e46740 10 client.18711 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5be1e46740 1 -- 172.21.15.39:0/2813466695 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56015a910470 con 0x56015a52c390 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2813466695 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bc400fad0 con 0x56015a52c390 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2813466695 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5bbc0a54d0 con 0x56015a52c390 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5be1e46740 10 client.18711 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5be1e46740 10 client.18711 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5be1e46740 20 client.18711 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56015a50b9f0 0x56015a9138e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5be1e46740 4 client.18711 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.855+0000 7f5be1e46740 1 -- 172.21.15.39:0/2813466695 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56015a869e50 con 0x56015a50b9f0 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.856+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56015a50b9f0 0x56015a9138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.856+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56015a50b9f0 0x56015a9138e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f2539925538c400b server_cookie=cc06216d9ce413eb in_seq=0 out_seq=0 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.856+0000 7f5bc2a00640 10 client.18711 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2813466695 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56015a869e50 con 0x56015a50b9f0 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5bc2a00640 10 client.18711 handle_command_reply: tid=0 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5be1e46740 1 client.18711 shutdown 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5be1e46740 1 -- 172.21.15.39:0/2813466695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a52c390 msgr2=0x56015a901d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a52c390 0x56015a901d40 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0a9af0 tx=0x7f5bbc0107e0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5be1e46740 1 -- 172.21.15.39:0/2813466695 shutdown_connections 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56015a50b9f0 0x56015a9138e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a52c390 0x56015a901d40 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a54d070 0x56015a90d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2813466695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a901800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5be1e46740 1 -- 172.21.15.39:0/2813466695 >> 172.21.15.39:0/2813466695 conn(0x56015a541460 msgr2=0x56015a8f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.857+0000 7f5bc2a00640 0 client.18711 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.858+0000 7f5be1e46740 1 -- 172.21.15.39:0/2813466695 shutdown_connections 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.858+0000 7f5be1e46740 1 -- 172.21.15.39:0/2813466695 wait complete. 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.858+0000 7f5be1e46740 20 client.18711 trim_cache size 0 max 16384 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.873+0000 7f5be1e46740 1 Processor -- start 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.873+0000 7f5be1e46740 1 -- start start 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a8ee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a52c390 0x56015a90d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a903230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5be1e46740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41ae240 con 0x56015a54d070 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5be1e46740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41afd80 con 0x56015a52c390 2024-07-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5be1e46740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41ae910 con 0x56015a5241b0 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5bd8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a8ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5bd8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a8ee960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50586/0 (socket says 172.21.15.39:50586) 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5bd8e00640 1 -- 172.21.15.39:0/9485564 learned_addr learned my addr 172.21.15.39:0/9485564 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5bd9800640 1 --2- 172.21.15.39:0/9485564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a903230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5bd3e00640 1 --2- 172.21.15.39:0/9485564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a52c390 0x56015a90d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5bd8e00640 1 -- 172.21.15.39:0/9485564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a52c390 msgr2=0x56015a90d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/9485564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a52c390 0x56015a90d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.874+0000 7f5bd8e00640 1 -- 172.21.15.39:0/9485564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 msgr2=0x56015a903230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.875+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/9485564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a903230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.875+0000 7f5bd8e00640 1 -- 172.21.15.39:0/9485564 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56015a9020b0 con 0x56015a5241b0 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.875+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/9485564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a8ee960 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4005c40 tx=0x7f5bc4019670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41cf8b33ba3ae17d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.875+0000 7f5bc9600640 1 -- 172.21.15.39:0/9485564 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc401e040 con 0x56015a5241b0 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.875+0000 7f5bc9600640 1 -- 172.21.15.39:0/9485564 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bc402a070 con 0x56015a5241b0 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.875+0000 7f5bc9600640 1 -- 172.21.15.39:0/9485564 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4003bd0 con 0x56015a5241b0 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.876+0000 7f5be1e46740 1 -- 172.21.15.39:0/9485564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 msgr2=0x56015a8ee960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.876+0000 7f5be1e46740 1 --2- 172.21.15.39:0/9485564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a8ee960 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4005c40 tx=0x7f5bc4019670 comp rx=0 tx=0).stop 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.876+0000 7f5be1e46740 1 -- 172.21.15.39:0/9485564 shutdown_connections 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.876+0000 7f5be1e46740 1 --2- 172.21.15.39:0/9485564 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a903230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.876+0000 7f5be1e46740 1 --2- 172.21.15.39:0/9485564 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a52c390 0x56015a90d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.876+0000 7f5be1e46740 1 --2- 172.21.15.39:0/9485564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a5241b0 0x56015a8ee960 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.876+0000 7f5be1e46740 1 -- 172.21.15.39:0/9485564 >> 172.21.15.39:0/9485564 conn(0x56015a541460 msgr2=0x56015a902b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.876+0000 7f5be1e46740 1 -- 172.21.15.39:0/9485564 shutdown_connections 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.876+0000 7f5be1e46740 1 -- 172.21.15.39:0/9485564 wait complete. 2024-07-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.877+0000 7f5be1e46740 1 -- start start 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.877+0000 7f5be1e46740 1 Processor -- start 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a918460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 0x56015a9189a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5be1e46740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a908640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5be1e46740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41af6b0 con 0x56015a54d070 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5be1e46740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41aefe0 con 0x56015a5241b0 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5be1e46740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41ac220 con 0x56015a52c390 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5be1e46740 10 client.0 _ll_register_callbacks cb 0x7f5bbc0a5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 0x56015a9189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5bd8e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a918460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5bd9800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a908640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5bd3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 0x56015a9189a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:50600/0 (socket says 172.21.15.39:50600) 2024-07-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5bd3e00640 1 -- 172.21.15.39:0/2381107705 learned_addr learned my addr 172.21.15.39:0/2381107705 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5bd8e00640 1 -- 172.21.15.39:0/2381107705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 msgr2=0x56015a9189a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.878+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 0x56015a9189a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.879+0000 7f5bd8e00640 1 -- 172.21.15.39:0/2381107705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 msgr2=0x56015a908640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.879+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a908640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.879+0000 7f5bd8e00640 1 -- 172.21.15.39:0/2381107705 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015a908b80 con 0x56015a5241b0 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.879+0000 7f5bd9800640 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a908640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.879+0000 7f5bd8e00640 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a918460 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009370 tx=0x7f5bc40195b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8459b48b2953a47b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.879+0000 7f5bc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.879+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2381107705 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4036070 con 0x56015a5241b0 2024-07-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.879+0000 7f5be1e46740 1 -- 172.21.15.39:0/2381107705 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc402a070 con 0x56015a5241b0 2024-07-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.879+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2381107705 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f5bc400f840 con 0x56015a5241b0 2024-07-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2381107705 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5bc401e070 con 0x56015a5241b0 2024-07-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5be1e46740 10 client.28838 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5be1e46740 1 -- 172.21.15.39:0/2381107705 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56015a909e00 con 0x56015a5241b0 2024-07-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2381107705 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bc402a070 con 0x56015a5241b0 2024-07-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2381107705 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5bc4004040 con 0x56015a5241b0 2024-07-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5be1e46740 10 client.28838 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5be1e46740 10 client.28838 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5be1e46740 20 client.28838 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56015a4e0da0 0x56015a97e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5be1e46740 4 client.28838 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.880+0000 7f5be1e46740 1 -- 172.21.15.39:0/2381107705 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56015a8658f0 con 0x56015a4e0da0 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.881+0000 7f5bd3e00640 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56015a4e0da0 0x56015a97e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.881+0000 7f5bd3e00640 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56015a4e0da0 0x56015a97e040 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d3c661323b1859d1 server_cookie=c2ae2565637cd660 in_seq=0 out_seq=0 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.881+0000 7f5bc2a00640 10 client.28838 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5bc2a00640 1 -- 172.21.15.39:0/2381107705 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56015a8658f0 con 0x56015a4e0da0 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5bc2a00640 10 client.28838 handle_command_reply: tid=0 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 client.28838 shutdown 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 -- 172.21.15.39:0/2381107705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 msgr2=0x56015a918460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a918460 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009370 tx=0x7f5bc40195b0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 -- 172.21.15.39:0/2381107705 shutdown_connections 2024-07-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x56015a4e0da0 0x56015a97e040 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56015a54d070 0x56015a908640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56015a52c390 0x56015a9189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 --2- 172.21.15.39:0/2381107705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56015a5241b0 0x56015a918460 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5bc2a00640 0 client.28838 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 -- 172.21.15.39:0/2381107705 >> 172.21.15.39:0/2381107705 conn(0x56015a541460 msgr2=0x56015a8f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 -- 172.21.15.39:0/2381107705 shutdown_connections 2024-07-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 1 -- 172.21.15.39:0/2381107705 wait complete. 2024-07-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.882+0000 7f5be1e46740 20 client.28838 trim_cache size 0 max 16384 2024-07-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 -- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5bb80b29e0 msgr2=0x7f5bb80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5bb80b29e0 0x7f5bb80b4ef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc006010 tx=0x7f5bcc0058d0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 -- 172.21.15.39:0/670890665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bd4108a10 msgr2=0x7f5bd419a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bd4108a10 0x7f5bd419a020 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00e9f0 tx=0x7f5bbc00eeb0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 -- 172.21.15.39:0/670890665 shutdown_connections 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5bb80b29e0 0x7f5bb80b4ef0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bd4109360 0x7f5bd419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5bd4108a10 0x7f5bd419a020 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 --2- 172.21.15.39:0/670890665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bd41030d0 0x7f5bd4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 -- 172.21.15.39:0/670890665 >> 172.21.15.39:0/670890665 conn(0x7f5bd40feae0 msgr2=0x7f5bd4106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 -- 172.21.15.39:0/670890665 shutdown_connections 2024-07-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:35.883+0000 7f5bdb800640 1 -- 172.21.15.39:0/670890665 wait complete. 2024-07-04T08:30:36.880 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:37.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:37 smithi039 ceph-mon[32062]: pgmap v960: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-07-04T08:30:37.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:37 smithi133 ceph-mon[37050]: pgmap v960: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-07-04T08:30:37.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:37 smithi050 ceph-mon[36577]: pgmap v960: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-07-04T08:30:38.250 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.247+0000 7f78d8400640 1 -- 172.21.15.39:0/1002689162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d010ac40 msgr2=0x7f78d010b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.247+0000 7f78d8400640 1 --2- 172.21.15.39:0/1002689162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d010ac40 0x7f78d010b050 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009900 tx=0x7f78bc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.248+0000 7f78d8400640 1 -- 172.21.15.39:0/1002689162 shutdown_connections 2024-07-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.248+0000 7f78d8400640 1 --2- 172.21.15.39:0/1002689162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d010ac40 0x7f78d010b050 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.248+0000 7f78d8400640 1 --2- 172.21.15.39:0/1002689162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d010b590 0x7f78d0111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.248+0000 7f78d8400640 1 --2- 172.21.15.39:0/1002689162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78d0105300 0x7f78d0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.248+0000 7f78d8400640 1 -- 172.21.15.39:0/1002689162 >> 172.21.15.39:0/1002689162 conn(0x7f78d0100d10 msgr2=0x7f78d0103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.248+0000 7f78d8400640 1 -- 172.21.15.39:0/1002689162 shutdown_connections 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.248+0000 7f78d8400640 1 -- 172.21.15.39:0/1002689162 wait complete. 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.249+0000 7f78d8400640 1 Processor -- start 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.249+0000 7f78d8400640 1 -- start start 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d8400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d0105300 0x7f78d019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d8400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d010ac40 0x7f78d019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d8400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78d010b590 0x7f78d01a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d8400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0115220 con 0x7f78d0105300 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d8400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f78d01150a0 con 0x7f78d010b590 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d8400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f78d01153a0 con 0x7f78d010ac40 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d0105300 0x7f78d019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d5000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d010ac40 0x7f78d019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d5a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d0105300 0x7f78d019bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45668/0 (socket says 172.21.15.39:45668) 2024-07-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d5a00640 1 -- 172.21.15.39:0/3776188180 learned_addr learned my addr 172.21.15.39:0/3776188180 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.250+0000 7f78d6400640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78d010b590 0x7f78d01a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78d5a00640 1 -- 172.21.15.39:0/3776188180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d010ac40 msgr2=0x7f78d019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78d5a00640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d010ac40 0x7f78d019c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78d5a00640 1 -- 172.21.15.39:0/3776188180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78d010b590 msgr2=0x7f78d01a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78d5a00640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78d010b590 0x7f78d01a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78d5a00640 1 -- 172.21.15.39:0/3776188180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d01a1be0 con 0x7f78d0105300 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78d6400640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78d010b590 0x7f78d01a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78d5a00640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d0105300 0x7f78d019bc70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f78c400ea60 tx=0x7f78c400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78ce000640 1 -- 172.21.15.39:0/3776188180 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c400cde0 con 0x7f78d0105300 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78ce000640 1 -- 172.21.15.39:0/3776188180 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c40035f0 con 0x7f78d0105300 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d01a1ed0 con 0x7f78d0105300 2024-07-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78ce000640 1 -- 172.21.15.39:0/3776188180 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4010690 con 0x7f78d0105300 2024-07-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.251+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d01a8630 con 0x7f78d0105300 2024-07-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.253+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d0105710 con 0x7f78d0105300 2024-07-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.255+0000 7f78ce000640 1 -- 172.21.15.39:0/3776188180 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f78c40037a0 con 0x7f78d0105300 2024-07-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.257+0000 7f78ce000640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78b80b29e0 0x7f78b80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.257+0000 7f78ce000640 1 -- 172.21.15.39:0/3776188180 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f78c40e2cd0 con 0x7f78d0105300 2024-07-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.257+0000 7f78d5000640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78b80b29e0 0x7f78b80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.258+0000 7f78d5000640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78b80b29e0 0x7f78b80b4ef0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009230 tx=0x7f78bc031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.261+0000 7f78ce000640 1 -- 172.21.15.39:0/3776188180 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f78c40aa2f0 con 0x7f78d0105300 2024-07-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.523+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78d01a8890 con 0x7f78d0105300 2024-07-04T08:30:39.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:39 smithi039 ceph-mon[32062]: pgmap v961: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-07-04T08:30:39.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.524+0000 7f78ce000640 1 -- 172.21.15.39:0/3776188180 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f78c40a9bf0 con 0x7f78d0105300 2024-07-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78b80b29e0 msgr2=0x7f78b80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78b80b29e0 0x7f78b80b4ef0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009230 tx=0x7f78bc031400 comp rx=0 tx=0).stop 2024-07-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d0105300 msgr2=0x7f78d019bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d0105300 0x7f78d019bc70 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f78c400ea60 tx=0x7f78c400ef20 comp rx=0 tx=0).stop 2024-07-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 shutdown_connections 2024-07-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f78b80b29e0 0x7f78b80b4ef0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f78d010b590 0x7f78d01a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78d010ac40 0x7f78d019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 --2- 172.21.15.39:0/3776188180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78d0105300 0x7f78d019bc70 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.527+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 >> 172.21.15.39:0/3776188180 conn(0x7f78d0100d10 msgr2=0x7f78d0109410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.528+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 shutdown_connections 2024-07-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:39.528+0000 7f78d8400640 1 -- 172.21.15.39:0/3776188180 wait complete. 2024-07-04T08:30:39.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:39 smithi133 ceph-mon[37050]: pgmap v961: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-07-04T08:30:39.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:39 smithi050 ceph-mon[36577]: pgmap v961: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-07-04T08:30:40.553 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:40 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3776188180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:40.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:40 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3776188180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:40.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:40 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3776188180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:30:40.649 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db / --timeout 90 --expiration 141.7 --await-for 45.0' 2024-07-04T08:30:40.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db / --timeout 90 --expiration 141.7 --await-for 45.0 2024-07-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.444+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2472134548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c14075650 msgr2=0x7f0c1410f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.444+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2472134548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c14075650 0x7f0c1410f0f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009900 tx=0x7f0c0002a410 comp rx=0 tx=0).stop 2024-07-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.444+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2472134548 shutdown_connections 2024-07-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.444+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2472134548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1410f630 0x7f0c14111a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.444+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2472134548 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c14075650 0x7f0c1410f0f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.444+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2472134548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c14074d00 0x7f0c14075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.444+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2472134548 >> 172.21.15.39:0/2472134548 conn(0x7f0c140feae0 msgr2=0x7f0c14100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.444+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2472134548 shutdown_connections 2024-07-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.444+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2472134548 wait complete. 2024-07-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.445+0000 7f0c1bc00640 1 Processor -- start 2024-07-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.445+0000 7f0c1bc00640 1 -- start start 2024-07-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c1bc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c14074d00 0x7f0c1419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c1bc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c14075650 0x7f0c1419c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c1bc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1410f630 0x7f0c141a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c1bc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141151b0 con 0x7f0c1410f630 2024-07-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c1bc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14115030 con 0x7f0c14074d00 2024-07-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c1bc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14115330 con 0x7f0c14075650 2024-07-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c19c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1410f630 0x7f0c141a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c14075650 0x7f0c1419c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c19c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1410f630 0x7f0c141a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33636/0 (socket says 172.21.15.39:33636) 2024-07-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c19200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c14074d00 0x7f0c1419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.446+0000 7f0c19c00640 1 -- 172.21.15.39:0/2351407870 learned_addr learned my addr 172.21.15.39:0/2351407870 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.447+0000 7f0c19200640 1 -- 172.21.15.39:0/2351407870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c14075650 msgr2=0x7f0c1419c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.447+0000 7f0c19200640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c14075650 0x7f0c1419c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.447+0000 7f0c19200640 1 -- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1410f630 msgr2=0x7f0c141a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.447+0000 7f0c19200640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1410f630 0x7f0c141a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.447+0000 7f0c19200640 1 -- 172.21.15.39:0/2351407870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c141a1b30 con 0x7f0c14074d00 2024-07-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.447+0000 7f0c13e00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c14075650 0x7f0c1419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.447+0000 7f0c19c00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1410f630 0x7f0c141a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.447+0000 7f0c19200640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c14074d00 0x7f0c1419bbc0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800e9f0 tx=0x7f0c0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.448+0000 7f0c11600640 1 -- 172.21.15.39:0/2351407870 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0800ccc0 con 0x7f0c14074d00 2024-07-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.448+0000 7f0c11600640 1 -- 172.21.15.39:0/2351407870 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c080108e0 con 0x7f0c14074d00 2024-07-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.448+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2351407870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c141a1e20 con 0x7f0c14074d00 2024-07-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.448+0000 7f0c11600640 1 -- 172.21.15.39:0/2351407870 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08018880 con 0x7f0c14074d00 2024-07-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.448+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2351407870 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c141a8580 con 0x7f0c14074d00 2024-07-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.452+0000 7f0c22176740 1 Processor -- start 2024-07-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.452+0000 7f0c22176740 1 -- start start 2024-07-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.452+0000 7f0c22176740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb241070 0x5566bb5e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.452+0000 7f0c22176740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb2181b0 0x5566bb5e5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.452+0000 7f0c22176740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.452+0000 7f0c22176740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5566bb55bf00 con 0x5566bb2181b0 2024-07-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.452+0000 7f0c22176740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5566bb54b4a0 con 0x5566bb241070 2024-07-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.452+0000 7f0c22176740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5566bb54f4e0 con 0x5566bb220390 2024-07-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c11600640 1 -- 172.21.15.39:0/2351407870 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0c08018a20 con 0x7f0c14074d00 2024-07-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44634/0 (socket says 172.21.15.39:44634) 2024-07-04T08:30:41.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c13e00640 1 -- 172.21.15.39:0/2941920595 learned_addr learned my addr 172.21.15.39:0/2941920595 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:41.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c19c00640 1 --2- 172.21.15.39:0/2941920595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb241070 0x5566bb5e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c13e00640 1 -- 172.21.15.39:0/2941920595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb241070 msgr2=0x5566bb5e57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:41.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c13e00640 1 --2- 172.21.15.39:0/2941920595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb241070 0x5566bb5e57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c13e00640 1 -- 172.21.15.39:0/2941920595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb2181b0 msgr2=0x5566bb5e5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:41.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c13e00640 1 --2- 172.21.15.39:0/2941920595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb2181b0 0x5566bb5e5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.453+0000 7f0c13e00640 1 -- 172.21.15.39:0/2941920595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bb4e3670 con 0x5566bb220390 2024-07-04T08:30:41.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.454+0000 7f0c13e00640 1 --2- 172.21.15.39:0/2941920595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e6250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0c14069d50 tx=0x7f0c00005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47e1cef2542cf848 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:41.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.454+0000 7f0c06000640 1 -- 172.21.15.39:0/2941920595 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00040070 con 0x5566bb220390 2024-07-04T08:30:41.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.454+0000 7f0c06000640 1 -- 172.21.15.39:0/2941920595 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c00031720 con 0x5566bb220390 2024-07-04T08:30:41.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.454+0000 7f0c06000640 1 -- 172.21.15.39:0/2941920595 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00035040 con 0x5566bb220390 2024-07-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.455+0000 7f0c11600640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0bfc0b29e0 0x7f0bfc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.455+0000 7f0c11600640 1 -- 172.21.15.39:0/2351407870 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f0c080e2d20 con 0x7f0c14074d00 2024-07-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.456+0000 7f0c22176740 1 -- 172.21.15.39:0/2941920595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 msgr2=0x5566bb5e6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.456+0000 7f0c22176740 1 --2- 172.21.15.39:0/2941920595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e6250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0c14069d50 tx=0x7f0c00005e20 comp rx=0 tx=0).stop 2024-07-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.456+0000 7f0c19c00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0bfc0b29e0 0x7f0bfc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.456+0000 7f0c22176740 1 -- 172.21.15.39:0/2941920595 shutdown_connections 2024-07-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.456+0000 7f0c22176740 1 --2- 172.21.15.39:0/2941920595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb2181b0 0x5566bb5e5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.456+0000 7f0c22176740 1 --2- 172.21.15.39:0/2941920595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e6250 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.456+0000 7f0c22176740 1 --2- 172.21.15.39:0/2941920595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb241070 0x5566bb5e57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.456+0000 7f0c22176740 1 -- 172.21.15.39:0/2941920595 >> 172.21.15.39:0/2941920595 conn(0x5566bb235460 msgr2=0x5566bb5e36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.456+0000 7f0c22176740 1 -- 172.21.15.39:0/2941920595 shutdown_connections 2024-07-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.457+0000 7f0c22176740 1 -- 172.21.15.39:0/2941920595 wait complete. 2024-07-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.457+0000 7f0c22176740 1 -- start start 2024-07-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.457+0000 7f0c19c00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0bfc0b29e0 0x7f0bfc0b4ef0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0049d0 tx=0x7f0c0c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.457+0000 7f0c22176740 1 Processor -- start 2024-07-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.458+0000 7f0c22176740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb2181b0 0x5566bb5fa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.458+0000 7f0c22176740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb220390 0x5566bb5fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.458+0000 7f0c22176740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 0x5566bb5ff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.458+0000 7f0c22176740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5566bb3af700 con 0x5566bb2181b0 2024-07-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.458+0000 7f0c22176740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5566bb4f2bd0 con 0x5566bb220390 2024-07-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.458+0000 7f0c22176740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5566bb5f3f50 con 0x5566bb241070 2024-07-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.458+0000 7f0c22176740 10 client.0 _ll_register_callbacks cb 0x5566bb544590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.458+0000 7f0c13e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb220390 0x5566bb5fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.459+0000 7f0c19c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 0x5566bb5ff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.459+0000 7f0c19200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb2181b0 0x5566bb5fa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.459+0000 7f0c13e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb220390 0x5566bb5fa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50512/0 (socket says 172.21.15.39:50512) 2024-07-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.459+0000 7f0c13e00640 1 -- 172.21.15.39:0/3319941468 learned_addr learned my addr 172.21.15.39:0/3319941468 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.460+0000 7f0c13e00640 1 -- 172.21.15.39:0/3319941468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 msgr2=0x5566bb5ff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.460+0000 7f0c13e00640 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 0x5566bb5ff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.460+0000 7f0c13e00640 1 -- 172.21.15.39:0/3319941468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb2181b0 msgr2=0x5566bb5fa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.460+0000 7f0c13e00640 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb2181b0 0x5566bb5fa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.460+0000 7f0c13e00640 1 -- 172.21.15.39:0/3319941468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bb2ac650 con 0x5566bb220390 2024-07-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.460+0000 7f0c13e00640 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb220390 0x5566bb5fa810 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0c000095a0 tx=0x7f0c00005380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb6240b69187795d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.460+0000 7f0bfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.461+0000 7f0bfaa00640 1 -- 172.21.15.39:0/3319941468 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0004b070 con 0x5566bb220390 2024-07-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.461+0000 7f0c22176740 1 -- 172.21.15.39:0/3319941468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c00031720 con 0x5566bb220390 2024-07-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.461+0000 7f0bfaa00640 1 -- 172.21.15.39:0/3319941468 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f0c00004450 con 0x5566bb220390 2024-07-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.461+0000 7f0bfaa00640 1 -- 172.21.15.39:0/3319941468 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c00057050 con 0x5566bb220390 2024-07-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.461+0000 7f0c22176740 10 client.28856 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.461+0000 7f0c22176740 1 -- 172.21.15.39:0/3319941468 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566bb608780 con 0x5566bb220390 2024-07-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.462+0000 7f0bfaa00640 1 -- 172.21.15.39:0/3319941468 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c00031720 con 0x5566bb220390 2024-07-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.462+0000 7f0bfaa00640 1 -- 172.21.15.39:0/3319941468 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0c0003c560 con 0x5566bb220390 2024-07-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.462+0000 7f0c22176740 10 client.28856 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.462+0000 7f0c22176740 10 client.28856 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.462+0000 7f0c22176740 20 client.28856 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.462+0000 7f0c22176740 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5566bb1ff9f0 0x5566bb60bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.462+0000 7f0c22176740 4 client.28856 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.462+0000 7f0c22176740 1 -- 172.21.15.39:0/3319941468 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566bb55de50 con 0x5566bb1ff9f0 2024-07-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.463+0000 7f0c19200640 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5566bb1ff9f0 0x5566bb60bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.463+0000 7f0c19200640 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5566bb1ff9f0 0x5566bb60bae0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af3f73f220af1ae7 server_cookie=64af8686ec9565e6 in_seq=0 out_seq=0 2024-07-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.463+0000 7f0bfaa00640 10 client.28856 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.464+0000 7f0bfaa00640 1 -- 172.21.15.39:0/3319941468 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5566bb55de50 con 0x5566bb1ff9f0 2024-07-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.464+0000 7f0bfaa00640 10 client.28856 handle_command_reply: tid=0 2024-07-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.464+0000 7f0c22176740 1 client.28856 shutdown 2024-07-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.464+0000 7f0c22176740 1 -- 172.21.15.39:0/3319941468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb220390 msgr2=0x5566bb5fa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.464+0000 7f0c22176740 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb220390 0x5566bb5fa810 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0c000095a0 tx=0x7f0c00005380 comp rx=0 tx=0).stop 2024-07-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0c22176740 1 -- 172.21.15.39:0/3319941468 shutdown_connections 2024-07-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0c22176740 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5566bb1ff9f0 0x5566bb60bae0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0c22176740 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 0x5566bb5ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0c22176740 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb220390 0x5566bb5fa810 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0c22176740 1 --2- 172.21.15.39:0/3319941468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb2181b0 0x5566bb5fa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0c22176740 1 -- 172.21.15.39:0/3319941468 >> 172.21.15.39:0/3319941468 conn(0x5566bb235460 msgr2=0x5566bb5e89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0bfaa00640 0 client.28856 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0c22176740 1 -- 172.21.15.39:0/3319941468 shutdown_connections 2024-07-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0c22176740 1 -- 172.21.15.39:0/3319941468 wait complete. 2024-07-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.465+0000 7f0c22176740 20 client.28856 trim_cache size 0 max 16384 2024-07-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.481+0000 7f0c22176740 1 Processor -- start 2024-07-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.481+0000 7f0c22176740 1 -- start start 2024-07-04T08:30:41.469 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:41 smithi039 ceph-mon[32062]: pgmap v962: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-07-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c22176740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c22176740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c22176740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb241070 0x5566bb5fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c22176740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141ae160 con 0x5566bb241070 2024-07-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c22176740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141afca0 con 0x5566bb2181b0 2024-07-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c22176740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141ae830 con 0x5566bb220390 2024-07-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44652/0 (socket says 172.21.15.39:44652) 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c13e00640 1 -- 172.21.15.39:0/2759802217 learned_addr learned my addr 172.21.15.39:0/2759802217 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c19200640 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.482+0000 7f0c19c00640 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb241070 0x5566bb5fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.483+0000 7f0c13e00640 1 -- 172.21.15.39:0/2759802217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 msgr2=0x5566bb5f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.483+0000 7f0c13e00640 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5f32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.483+0000 7f0c13e00640 1 -- 172.21.15.39:0/2759802217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb241070 msgr2=0x5566bb5fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.483+0000 7f0c13e00640 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb241070 0x5566bb5fbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.483+0000 7f0c13e00640 1 -- 172.21.15.39:0/2759802217 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bb5f3ae0 con 0x5566bb220390 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.483+0000 7f0c19200640 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.483+0000 7f0c13e00640 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e2960 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0c0002a930 tx=0x7f0c00031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82cf9fdd0ed49e9b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.483+0000 7f0c05600640 1 -- 172.21.15.39:0/2759802217 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0004b030 con 0x5566bb220390 2024-07-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.484+0000 7f0c05600640 1 -- 172.21.15.39:0/2759802217 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c0003cd50 con 0x5566bb220390 2024-07-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.484+0000 7f0c05600640 1 -- 172.21.15.39:0/2759802217 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00000e60 con 0x5566bb220390 2024-07-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.485+0000 7f0c22176740 1 -- 172.21.15.39:0/2759802217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 msgr2=0x5566bb5e2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.485+0000 7f0c22176740 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e2960 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0c0002a930 tx=0x7f0c00031400 comp rx=0 tx=0).stop 2024-07-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.485+0000 7f0c22176740 1 -- 172.21.15.39:0/2759802217 shutdown_connections 2024-07-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.485+0000 7f0c22176740 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb241070 0x5566bb5fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.485+0000 7f0c22176740 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb220390 0x5566bb5e2960 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.485+0000 7f0c22176740 1 --2- 172.21.15.39:0/2759802217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.485+0000 7f0c22176740 1 -- 172.21.15.39:0/2759802217 >> 172.21.15.39:0/2759802217 conn(0x5566bb235460 msgr2=0x5566bb5fb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.485+0000 7f0c22176740 1 -- 172.21.15.39:0/2759802217 shutdown_connections 2024-07-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.485+0000 7f0c22176740 1 -- 172.21.15.39:0/2759802217 wait complete. 2024-07-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.486+0000 7f0c22176740 1 -- start start 2024-07-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.486+0000 7f0c22176740 1 Processor -- start 2024-07-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.486+0000 7f0c22176740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c22176740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb220390 0x5566bb600860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c22176740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 0x5566bb600da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c22176740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141af5d0 con 0x5566bb220390 2024-07-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c22176740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141aef00 con 0x5566bb2181b0 2024-07-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c22176740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c141ac1c0 con 0x5566bb241070 2024-07-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c22176740 10 client.0 _ll_register_callbacks cb 0x7f0c00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c19200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c19200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50522/0 (socket says 172.21.15.39:50522) 2024-07-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c19200640 1 -- 172.21.15.39:0/175266349 learned_addr learned my addr 172.21.15.39:0/175266349 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c19c00640 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 0x5566bb600da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c13e00640 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb220390 0x5566bb600860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c19200640 1 -- 172.21.15.39:0/175266349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 msgr2=0x5566bb600da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c19200640 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 0x5566bb600da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c19200640 1 -- 172.21.15.39:0/175266349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb220390 msgr2=0x5566bb600860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c19200640 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb220390 0x5566bb600860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.487+0000 7f0c19200640 1 -- 172.21.15.39:0/175266349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bb5e98e0 con 0x5566bb2181b0 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.488+0000 7f0c19200640 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5fcf80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0c080a5d70 tx=0x7f0c080cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f173187030ca2a2d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.488+0000 7f0bfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.488+0000 7f0bfaa00640 1 -- 172.21.15.39:0/175266349 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080adff0 con 0x5566bb2181b0 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.488+0000 7f0c22176740 1 -- 172.21.15.39:0/175266349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c0003cd50 con 0x5566bb2181b0 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.488+0000 7f0bfaa00640 1 -- 172.21.15.39:0/175266349 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f0c080a62e0 con 0x5566bb2181b0 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0bfaa00640 1 -- 172.21.15.39:0/175266349 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c080b2aa0 con 0x5566bb2181b0 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0c22176740 10 client.28862 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0c22176740 1 -- 172.21.15.39:0/175266349 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566bb606640 con 0x5566bb2181b0 2024-07-04T08:30:41.478 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0bfaa00640 1 -- 172.21.15.39:0/175266349 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c0003cd50 con 0x5566bb2181b0 2024-07-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0bfaa00640 1 -- 172.21.15.39:0/175266349 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0c08091040 con 0x5566bb2181b0 2024-07-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0c22176740 10 client.28862 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0c22176740 10 client.28862 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0c22176740 20 client.28862 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0c22176740 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5566bb1d4da0 0x5566bb676510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0c22176740 4 client.28862 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.489+0000 7f0c22176740 1 -- 172.21.15.39:0/175266349 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566bb5598f0 con 0x5566bb1d4da0 2024-07-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.490+0000 7f0c13e00640 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5566bb1d4da0 0x5566bb676510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.490+0000 7f0bfaa00640 10 client.28862 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.490+0000 7f0c13e00640 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5566bb1d4da0 0x5566bb676510 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fdbc48a9319f93e7 server_cookie=9f3035ec2f8a0ab2 in_seq=0 out_seq=0 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0bfaa00640 1 -- 172.21.15.39:0/175266349 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566bb5598f0 con 0x5566bb1d4da0 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0bfaa00640 10 client.28862 handle_command_reply: tid=0 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 client.28862 shutdown 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 -- 172.21.15.39:0/175266349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 msgr2=0x5566bb5fcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5fcf80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0c080a5d70 tx=0x7f0c080cc040 comp rx=0 tx=0).stop 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 -- 172.21.15.39:0/175266349 shutdown_connections 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x5566bb1d4da0 0x5566bb676510 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5566bb220390 0x5566bb600860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5566bb241070 0x5566bb600da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 --2- 172.21.15.39:0/175266349 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5566bb2181b0 0x5566bb5fcf80 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0bfaa00640 0 client.28862 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 -- 172.21.15.39:0/175266349 >> 172.21.15.39:0/175266349 conn(0x5566bb235460 msgr2=0x5566bb5e6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 -- 172.21.15.39:0/175266349 shutdown_connections 2024-07-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.491+0000 7f0c22176740 1 -- 172.21.15.39:0/175266349 wait complete. 2024-07-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.492+0000 7f0c22176740 20 client.28862 trim_cache size 0 max 16384 2024-07-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0bfc0b29e0 msgr2=0x7f0bfc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0bfc0b29e0 0x7f0bfc0b4ef0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0049d0 tx=0x7f0c0c009500 comp rx=0 tx=0).stop 2024-07-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2351407870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c14074d00 msgr2=0x7f0c1419bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c14074d00 0x7f0c1419bbc0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800e9f0 tx=0x7f0c0800eeb0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2351407870 shutdown_connections 2024-07-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0bfc0b29e0 0x7f0bfc0b4ef0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1410f630 0x7f0c141a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c14075650 0x7f0c1419c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 --2- 172.21.15.39:0/2351407870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c14074d00 0x7f0c1419bbc0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2351407870 >> 172.21.15.39:0/2351407870 conn(0x7f0c140feae0 msgr2=0x7f0c14109160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2351407870 shutdown_connections 2024-07-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:41.493+0000 7f0c1bc00640 1 -- 172.21.15.39:0/2351407870 wait complete. 2024-07-04T08:30:41.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:41 smithi133 ceph-mon[37050]: pgmap v962: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-07-04T08:30:41.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:41 smithi050 ceph-mon[36577]: pgmap v962: 129 pgs: 129 active+clean; 29 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-07-04T08:30:42.283 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:30:43.017 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:43.504 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:43 smithi039 ceph-mon[32062]: pgmap v963: 129 pgs: 129 active+clean; 30 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-07-04T08:30:43.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:43 smithi050 ceph-mon[36577]: pgmap v963: 129 pgs: 129 active+clean; 30 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-07-04T08:30:43.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:43 smithi133 ceph-mon[37050]: pgmap v963: 129 pgs: 129 active+clean; 30 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-07-04T08:30:44.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.463+0000 7f0eafa00640 1 -- 172.21.15.39:0/3624219047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea81096e0 msgr2=0x7f0ea8109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.463+0000 7f0eafa00640 1 --2- 172.21.15.39:0/3624219047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea81096e0 0x7f0ea8109af0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009860 tx=0x7f0e9c02a340 comp rx=0 tx=0).stop 2024-07-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.463+0000 7f0eafa00640 1 -- 172.21.15.39:0/3624219047 shutdown_connections 2024-07-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.463+0000 7f0eafa00640 1 --2- 172.21.15.39:0/3624219047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ea810f970 0x7f0ea81161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.463+0000 7f0eafa00640 1 --2- 172.21.15.39:0/3624219047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea810f020 0x7f0ea810f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.463+0000 7f0eafa00640 1 --2- 172.21.15.39:0/3624219047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea81096e0 0x7f0ea8109af0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.463+0000 7f0eafa00640 1 -- 172.21.15.39:0/3624219047 >> 172.21.15.39:0/3624219047 conn(0x7f0ea8077710 msgr2=0x7f0ea8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.463+0000 7f0eafa00640 1 -- 172.21.15.39:0/3624219047 shutdown_connections 2024-07-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.463+0000 7f0eafa00640 1 -- 172.21.15.39:0/3624219047 wait complete. 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.465+0000 7f0eafa00640 1 Processor -- start 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.465+0000 7f0eafa00640 1 -- start start 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.466+0000 7f0eafa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea81096e0 0x7f0ea819fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.466+0000 7f0eafa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ea810f020 0x7f0ea81a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.466+0000 7f0eafa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea810f970 0x7f0ea81a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.466+0000 7f0eafa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8119620 con 0x7f0ea810f020 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.466+0000 7f0eafa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81194a0 con 0x7f0ea810f970 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.466+0000 7f0eafa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81197a0 con 0x7f0ea81096e0 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.466+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ea810f020 0x7f0ea81a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.466+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ea810f020 0x7f0ea81a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33698/0 (socket says 172.21.15.39:33698) 2024-07-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.466+0000 7f0ea7e00640 1 -- 172.21.15.39:0/1436599339 learned_addr learned my addr 172.21.15.39:0/1436599339 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.467+0000 7f0ead000640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea81096e0 0x7f0ea819fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.467+0000 7f0eada00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea810f970 0x7f0ea81a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.467+0000 7f0ea7e00640 1 -- 172.21.15.39:0/1436599339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea81096e0 msgr2=0x7f0ea819fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.467+0000 7f0ea7e00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea81096e0 0x7f0ea819fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.467+0000 7f0ea7e00640 1 -- 172.21.15.39:0/1436599339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea810f970 msgr2=0x7f0ea81a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.467+0000 7f0ea7e00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea810f970 0x7f0ea81a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.467+0000 7f0ea7e00640 1 -- 172.21.15.39:0/1436599339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea81a5f60 con 0x7f0ea810f020 2024-07-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.467+0000 7f0ead000640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea81096e0 0x7f0ea819fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.467+0000 7f0ea7e00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ea810f020 0x7f0ea81a0590 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e940058c0 tx=0x7f0e9400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.468+0000 7f0ea5600640 1 -- 172.21.15.39:0/1436599339 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9400cd30 con 0x7f0ea810f020 2024-07-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.468+0000 7f0ea5600640 1 -- 172.21.15.39:0/1436599339 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e940035f0 con 0x7f0ea810f020 2024-07-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.468+0000 7f0eafa00640 1 -- 172.21.15.39:0/1436599339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea81a6250 con 0x7f0ea810f020 2024-07-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.468+0000 7f0ea5600640 1 -- 172.21.15.39:0/1436599339 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940044e0 con 0x7f0ea810f020 2024-07-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.468+0000 7f0eafa00640 1 -- 172.21.15.39:0/1436599339 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea81ac9b0 con 0x7f0ea810f020 2024-07-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.472+0000 7f0ea5600640 1 -- 172.21.15.39:0/1436599339 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0e94003aa0 con 0x7f0ea810f020 2024-07-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.473+0000 7f0eb5f65740 1 Processor -- start 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.473+0000 7f0eb5f65740 1 -- start start 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.474+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ff983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.474+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff98900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.474+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 0x55b15ff98e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.474+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b15ff0dce0 con 0x55b15fbd2b60 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.474+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b15fefd280 con 0x55b15fbf3840 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.474+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b15ff012c0 con 0x55b15fbca980 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.474+0000 7f0ea5600640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e900b2960 0x7f0e900b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.474+0000 7f0ea5600640 1 -- 172.21.15.39:0/1436599339 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f0e940e2bf0 con 0x7f0ea810f020 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0ead000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff98900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0eada00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e900b2960 0x7f0e900b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0ead000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff98900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44688/0 (socket says 172.21.15.39:44688) 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0ead000640 1 -- 172.21.15.39:0/1029505272 learned_addr learned my addr 172.21.15.39:0/1029505272 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0ea7e00640 1 --2- 172.21.15.39:0/1029505272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 0x55b15ff98e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0eada00640 1 --2- 172.21.15.39:0/1029505272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ff983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0ead000640 1 -- 172.21.15.39:0/1029505272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 msgr2=0x55b15ff983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0ead000640 1 --2- 172.21.15.39:0/1029505272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ff983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0ead000640 1 -- 172.21.15.39:0/1029505272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 msgr2=0x55b15ff98e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0ead000640 1 --2- 172.21.15.39:0/1029505272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 0x55b15ff98e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.475+0000 7f0ead000640 1 -- 172.21.15.39:0/1029505272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b15fe946f0 con 0x55b15fbca980 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.476+0000 7f0eada00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e900b2960 0x7f0e900b4e70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005fd0 tx=0x7f0ea0009f00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.476+0000 7f0ead000640 1 --2- 172.21.15.39:0/1029505272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff98900 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0057c0 tx=0x7f0e9c005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a03326687ce657 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.476+0000 7f0e9a000640 1 -- 172.21.15.39:0/1029505272 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c037040 con 0x55b15fbca980 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.476+0000 7f0e9a000640 1 -- 172.21.15.39:0/1029505272 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e9c030450 con 0x55b15fbca980 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.476+0000 7f0e9a000640 1 -- 172.21.15.39:0/1029505272 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c032af0 con 0x55b15fbca980 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.477+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1029505272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 msgr2=0x55b15ff98900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.477+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1029505272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff98900 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0057c0 tx=0x7f0e9c005d90 comp rx=0 tx=0).stop 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.477+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1029505272 shutdown_connections 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.477+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1029505272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 0x55b15ff98e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.477+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1029505272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff98900 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.477+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1029505272 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ff983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.477+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1029505272 >> 172.21.15.39:0/1029505272 conn(0x55b15fbe7990 msgr2=0x55b15ff962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.477+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1029505272 shutdown_connections 2024-07-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.478+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1029505272 wait complete. 2024-07-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.478+0000 7f0eb5f65740 1 -- start start 2024-07-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.478+0000 7f0eb5f65740 1 Processor -- start 2024-07-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 0x55b15ffa3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbd2b60 0x55b15ffa4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffafe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55b15fd617a0 con 0x55b15fbca980 2024-07-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b15fea46d0 con 0x55b15fbf3840 2024-07-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b15ffa6b40 con 0x55b15fbd2b60 2024-07-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0eb5f65740 10 client.0 _ll_register_callbacks cb 0x55b15fc0b090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0ead000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 0x55b15ffa3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0eada00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffafe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbd2b60 0x55b15ffa4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.479+0000 7f0eada00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffafe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50554/0 (socket says 172.21.15.39:50554) 2024-07-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.480+0000 7f0eada00640 1 -- 172.21.15.39:0/895426571 learned_addr learned my addr 172.21.15.39:0/895426571 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.480+0000 7f0eada00640 1 -- 172.21.15.39:0/895426571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbd2b60 msgr2=0x55b15ffa4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.480+0000 7f0eada00640 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbd2b60 0x55b15ffa4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.480+0000 7f0eada00640 1 -- 172.21.15.39:0/895426571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 msgr2=0x55b15ffa3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.480+0000 7f0eada00640 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 0x55b15ffa3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.480+0000 7f0eada00640 1 -- 172.21.15.39:0/895426571 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b15fe951d0 con 0x55b15fbf3840 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.480+0000 7f0ea7e00640 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbd2b60 0x55b15ffa4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.480+0000 7f0eada00640 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffafe90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0003b60 tx=0x7f0ea001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56bb3fa7e3b4be20 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.480+0000 7f0e8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.481+0000 7f0e8ea00640 1 -- 172.21.15.39:0/895426571 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0024070 con 0x55b15fbf3840 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.481+0000 7f0e8ea00640 1 -- 172.21.15.39:0/895426571 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f0ea001f450 con 0x55b15fbf3840 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.481+0000 7f0eb5f65740 1 -- 172.21.15.39:0/895426571 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c030450 con 0x55b15fbf3840 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.481+0000 7f0e8ea00640 1 -- 172.21.15.39:0/895426571 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ea0033050 con 0x55b15fbf3840 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.481+0000 7f0eb5f65740 10 client.28874 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.481+0000 7f0eb5f65740 1 -- 172.21.15.39:0/895426571 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b15fc6db90 con 0x55b15fbf3840 2024-07-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.482+0000 7f0e8ea00640 1 -- 172.21.15.39:0/895426571 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e9c030450 con 0x55b15fbf3840 2024-07-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.482+0000 7f0e8ea00640 1 -- 172.21.15.39:0/895426571 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0ea0028400 con 0x55b15fbf3840 2024-07-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.482+0000 7f0eb5f65740 10 client.28874 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.482+0000 7f0eb5f65740 10 client.28874 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.482+0000 7f0eb5f65740 20 client.28874 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.482+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b15fbb21c0 0x55b15ffb5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 46, 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "9072c93": { 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 46, 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 141.7, 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "QUIESCED", 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.482+0000 7f0eb5f65740 4 client.28874 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.482+0000 7f0eb5f65740 1 -- 172.21.15.39:0/895426571 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b15ff0fc30 con 0x55b15fbb21c0 2024-07-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.483+0000 7f0ead000640 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b15fbb21c0 0x55b15ffb5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.483+0000 7f0e8ea00640 10 client.28874 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.483+0000 7f0ead000640 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b15fbb21c0 0x55b15ffb5f20 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed3ac7c3a74f95b server_cookie=a3221cc81c995e56 in_seq=0 out_seq=0 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0e8ea00640 1 -- 172.21.15.39:0/895426571 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b15ff0fc30 con 0x55b15fbb21c0 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0e8ea00640 10 client.28874 handle_command_reply: tid=0 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0eb5f65740 1 client.28874 shutdown 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0eb5f65740 1 -- 172.21.15.39:0/895426571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 msgr2=0x55b15ffafe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffafe90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0003b60 tx=0x7f0ea001d000 comp rx=0 tx=0).stop 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0eb5f65740 1 -- 172.21.15.39:0/895426571 shutdown_connections 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b15fbb21c0 0x55b15ffb5f20 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffafe90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbd2b60 0x55b15ffa4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/895426571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 0x55b15ffa3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0e8ea00640 0 client.28874 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.484+0000 7f0eb5f65740 1 -- 172.21.15.39:0/895426571 >> 172.21.15.39:0/895426571 conn(0x55b15fbe7990 msgr2=0x55b15ff9b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.485+0000 7f0eb5f65740 1 -- 172.21.15.39:0/895426571 shutdown_connections 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.485+0000 7f0eb5f65740 1 -- 172.21.15.39:0/895426571 wait complete. 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.485+0000 7f0eb5f65740 20 client.28874 trim_cache size 0 max 16384 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.505+0000 7f0eb5f65740 1 Processor -- start 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.505+0000 7f0eb5f65740 1 -- start start 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.505+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff95230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.505+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 0x55b15ffafc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.505+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffb01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.505+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81b6940 con 0x55b15fbd2b60 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.505+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81b8480 con 0x55b15fbf3840 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.505+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81b7010 con 0x55b15fbca980 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0eada00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffb01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0eada00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffb01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50562/0 (socket says 172.21.15.39:50562) 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0eada00640 1 -- 172.21.15.39:0/844415366 learned_addr learned my addr 172.21.15.39:0/844415366 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0ead000640 1 --2- 172.21.15.39:0/844415366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff95230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0ea7e00640 1 --2- 172.21.15.39:0/844415366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 0x55b15ffafc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0eada00640 1 -- 172.21.15.39:0/844415366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 msgr2=0x55b15ff95230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0eada00640 1 --2- 172.21.15.39:0/844415366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff95230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0eada00640 1 -- 172.21.15.39:0/844415366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 msgr2=0x55b15ffafc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0eada00640 1 --2- 172.21.15.39:0/844415366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 0x55b15ffafc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0eada00640 1 -- 172.21.15.39:0/844415366 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b15ffa6420 con 0x55b15fbf3840 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.506+0000 7f0eada00640 1 --2- 172.21.15.39:0/844415366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffb01c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0028820 tx=0x7f0ea0011560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf925367e1056af server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.507+0000 7f0e99600640 1 -- 172.21.15.39:0/844415366 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0028e60 con 0x55b15fbf3840 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.507+0000 7f0e99600640 1 -- 172.21.15.39:0/844415366 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea003c070 con 0x55b15fbf3840 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.507+0000 7f0e99600640 1 -- 172.21.15.39:0/844415366 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0024040 con 0x55b15fbf3840 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 -- 172.21.15.39:0/844415366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 msgr2=0x55b15ffb01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/844415366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffb01c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0028820 tx=0x7f0ea0011560 comp rx=0 tx=0).stop 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 -- 172.21.15.39:0/844415366 shutdown_connections 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/844415366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbd2b60 0x55b15ffafc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/844415366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbf3840 0x55b15ffb01c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/844415366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbca980 0x55b15ff95230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 -- 172.21.15.39:0/844415366 >> 172.21.15.39:0/844415366 conn(0x55b15fbe7990 msgr2=0x55b15ff96240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 -- 172.21.15.39:0/844415366 shutdown_connections 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 -- 172.21.15.39:0/844415366 wait complete. 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.508+0000 7f0eb5f65740 1 -- start start 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.509+0000 7f0eb5f65740 1 Processor -- start 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.509+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 0x55b15ffa8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.509+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbd2b60 0x55b15ffa8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.509+0000 7f0eb5f65740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbf3840 0x55b16001d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.509+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81b7db0 con 0x55b15fbca980 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.509+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81b76e0 con 0x55b15fbd2b60 2024-07-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.509+0000 7f0eb5f65740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81b4920 con 0x55b15fbf3840 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.509+0000 7f0eb5f65740 10 client.0 _ll_register_callbacks cb 0x7f0ea0009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbd2b60 0x55b15ffa8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ead000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 0x55b15ffa8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbd2b60 0x55b15ffa8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50570/0 (socket says 172.21.15.39:50570) 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ead000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 0x55b15ffa8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33740/0 (socket says 172.21.15.39:33740) 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ea7e00640 1 -- 172.21.15.39:0/1868305885 learned_addr learned my addr 172.21.15.39:0/1868305885 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0eada00640 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbf3840 0x55b16001d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ea7e00640 1 -- 172.21.15.39:0/1868305885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbf3840 msgr2=0x55b16001d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ea7e00640 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbf3840 0x55b16001d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ea7e00640 1 -- 172.21.15.39:0/1868305885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 msgr2=0x55b15ffa8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ea7e00640 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 0x55b15ffa8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.510+0000 7f0ea7e00640 1 -- 172.21.15.39:0/1868305885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b16001dbf0 con 0x55b15fbd2b60 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.511+0000 7f0eada00640 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbf3840 0x55b16001d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.511+0000 7f0ea7e00640 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbd2b60 0x55b15ffa8940 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0e940a5b40 tx=0x7f0e940aa0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1403b23aad45ca97 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.511+0000 7f0e8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.511+0000 7f0e8ea00640 1 -- 172.21.15.39:0/1868305885 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940b22b0 con 0x55b15fbd2b60 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.511+0000 7f0e8ea00640 1 -- 172.21.15.39:0/1868305885 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f0e940a5480 con 0x55b15fbd2b60 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.511+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1868305885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea003c070 con 0x55b15fbd2b60 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.511+0000 7f0e8ea00640 1 -- 172.21.15.39:0/1868305885 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e940a9620 con 0x55b15fbd2b60 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.511+0000 7f0eb5f65740 10 client.28880 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.511+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1868305885 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b16001ee50 con 0x55b15fbd2b60 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.512+0000 7f0e8ea00640 1 -- 172.21.15.39:0/1868305885 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea003c070 con 0x55b15fbd2b60 2024-07-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.512+0000 7f0e8ea00640 1 -- 172.21.15.39:0/1868305885 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0e940a5680 con 0x55b15fbd2b60 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.512+0000 7f0eb5f65740 10 client.28880 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.512+0000 7f0eb5f65740 10 client.28880 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.512+0000 7f0eb5f65740 20 client.28880 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.512+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b15fb87570 0x55b160021680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.512+0000 7f0eb5f65740 4 client.28880 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.7, "await_for": 45.0}] 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.512+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1868305885 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.7, "await_for": 45.0}) -- 0x55b15ffa3540 con 0x55b15fb87570 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.513+0000 7f0ead000640 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b15fb87570 0x55b160021680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.513+0000 7f0ead000640 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b15fb87570 0x55b160021680 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698f3f5ad0d5542e server_cookie=b06a22d64e8497a7 in_seq=0 out_seq=0 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.513+0000 7f0e8ea00640 10 client.28880 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0e8ea00640 1 -- 172.21.15.39:0/1868305885 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b15ffa3540 con 0x55b15fb87570 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0e8ea00640 10 client.28880 handle_command_reply: tid=0 2024-07-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0eb5f65740 1 client.28880 shutdown 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1868305885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbd2b60 msgr2=0x55b15ffa8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbd2b60 0x55b15ffa8940 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0e940a5b40 tx=0x7f0e940aa0b0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1868305885 shutdown_connections 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55b15fb87570 0x55b160021680 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b15fbf3840 0x55b16001d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b15fbd2b60 0x55b15ffa8940 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0eb5f65740 1 --2- 172.21.15.39:0/1868305885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b15fbca980 0x55b15ffa8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0e8ea00640 0 client.28880 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.523+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1868305885 >> 172.21.15.39:0/1868305885 conn(0x55b15fbe7990 msgr2=0x55b15ff96240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.524+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1868305885 shutdown_connections 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.524+0000 7f0eb5f65740 1 -- 172.21.15.39:0/1868305885 wait complete. 2024-07-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.524+0000 7f0eb5f65740 20 client.28880 trim_cache size 0 max 16384 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 -- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e900b2960 msgr2=0x7f0e900b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e900b2960 0x7f0e900b4e70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005fd0 tx=0x7f0ea0009f00 comp rx=0 tx=0).stop 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 -- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ea810f020 msgr2=0x7f0ea81a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ea810f020 0x7f0ea81a0590 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e940058c0 tx=0x7f0e9400ee50 comp rx=0 tx=0).stop 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 -- 172.21.15.39:0/1436599339 shutdown_connections 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e900b2960 0x7f0e900b4e70 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ea810f020 0x7f0ea81a0590 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ea810f970 0x7f0ea81a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 --2- 172.21.15.39:0/1436599339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea81096e0 0x7f0ea819fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 -- 172.21.15.39:0/1436599339 >> 172.21.15.39:0/1436599339 conn(0x7f0ea8077710 msgr2=0x7f0ea8107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 -- 172.21.15.39:0/1436599339 shutdown_connections 2024-07-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:44.525+0000 7f0eafa00640 1 -- 172.21.15.39:0/1436599339 wait complete. 2024-07-04T08:30:45.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:45 smithi039 ceph-mon[32062]: pgmap v964: 129 pgs: 129 active+clean; 30 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-07-04T08:30:45.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:45 smithi050 ceph-mon[36577]: pgmap v964: 129 pgs: 129 active+clean; 30 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-07-04T08:30:45.680 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9072c93', quiesce took 9.6 seconds. Will release after: 12.1 2024-07-04T08:30:45.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:45 smithi133 ceph-mon[37050]: pgmap v964: 129 pgs: 129 active+clean; 30 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-07-04T08:30:46.515 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:46 smithi039 ceph-mon[32062]: pgmap v965: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-07-04T08:30:46.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:46 smithi050 ceph-mon[36577]: pgmap v965: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-07-04T08:30:46.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:46 smithi133 ceph-mon[37050]: pgmap v965: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 16 op/s 2024-07-04T08:30:46.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.866+0000 7f4f7f200640 1 -- 172.21.15.39:0/3805176397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 msgr2=0x7f4f80105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.866+0000 7f4f7f200640 1 --2- 172.21.15.39:0/3805176397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 0x7f4f80105500 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009900 tx=0x7f4f7002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.866+0000 7f4f7f200640 1 -- 172.21.15.39:0/3805176397 shutdown_connections 2024-07-04T08:30:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.866+0000 7f4f7f200640 1 --2- 172.21.15.39:0/3805176397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f8010b380 0x7f4f80111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.866+0000 7f4f7f200640 1 --2- 172.21.15.39:0/3805176397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f8010aa30 0x7f4f8010ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.866+0000 7f4f7f200640 1 --2- 172.21.15.39:0/3805176397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 0x7f4f80105500 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.866+0000 7f4f7f200640 1 -- 172.21.15.39:0/3805176397 >> 172.21.15.39:0/3805176397 conn(0x7f4f80100b60 msgr2=0x7f4f80102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.866+0000 7f4f7f200640 1 -- 172.21.15.39:0/3805176397 shutdown_connections 2024-07-04T08:30:46.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.866+0000 7f4f7f200640 1 -- 172.21.15.39:0/3805176397 wait complete. 2024-07-04T08:30:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.867+0000 7f4f7f200640 1 Processor -- start 2024-07-04T08:30:46.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.868+0000 7f4f7f200640 1 -- start start 2024-07-04T08:30:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.868+0000 7f4f7f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 0x7f4f8019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.868+0000 7f4f7f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f8010aa30 0x7f4f8019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.868+0000 7f4f7f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f8010b380 0x7f4f801a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.868+0000 7f4f7f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80115080 con 0x7f4f8010aa30 2024-07-04T08:30:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.868+0000 7f4f7f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80114f00 con 0x7f4f8010b380 2024-07-04T08:30:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.868+0000 7f4f7f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80115200 con 0x7f4f801050f0 2024-07-04T08:30:46.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f7de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 0x7f4f8019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f8010aa30 0x7f4f8019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f7de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 0x7f4f8019ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44720/0 (socket says 172.21.15.39:44720) 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f7de00640 1 -- 172.21.15.39:0/4089180643 learned_addr learned my addr 172.21.15.39:0/4089180643 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f7e800640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f8010b380 0x7f4f801a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f7de00640 1 -- 172.21.15.39:0/4089180643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f8010b380 msgr2=0x7f4f801a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f7de00640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f8010b380 0x7f4f801a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f7de00640 1 -- 172.21.15.39:0/4089180643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f8010aa30 msgr2=0x7f4f8019bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f7de00640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f8010aa30 0x7f4f8019bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.869+0000 7f4f7de00640 1 -- 172.21.15.39:0/4089180643 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f801a19c0 con 0x7f4f801050f0 2024-07-04T08:30:46.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.870+0000 7f4f7de00640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 0x7f4f8019ba50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4f70005ab0 tx=0x7f4f70003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.870+0000 7f4f76a00640 1 -- 172.21.15.39:0/4089180643 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7003f070 con 0x7f4f801050f0 2024-07-04T08:30:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.870+0000 7f4f7f200640 1 -- 172.21.15.39:0/4089180643 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f801a1c50 con 0x7f4f801050f0 2024-07-04T08:30:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.870+0000 7f4f76a00640 1 -- 172.21.15.39:0/4089180643 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f70031b60 con 0x7f4f801050f0 2024-07-04T08:30:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.870+0000 7f4f76a00640 1 -- 172.21.15.39:0/4089180643 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70035030 con 0x7f4f801050f0 2024-07-04T08:30:46.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.870+0000 7f4f7f200640 1 -- 172.21.15.39:0/4089180643 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f801a8410 con 0x7f4f801050f0 2024-07-04T08:30:46.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.874+0000 7f4f8b261740 1 Processor -- start 2024-07-04T08:30:46.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.874+0000 7f4f8b261740 1 -- start start 2024-07-04T08:30:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.874+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577d12070 0x55c5780b6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.874+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577ce91b0 0x55c5780b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.874+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577cf1390 0x55c5780b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.874+0000 7f4f8b261740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c57802cee0 con 0x55c577cf1390 2024-07-04T08:30:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.874+0000 7f4f8b261740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c57801c480 con 0x55c577d12070 2024-07-04T08:30:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.874+0000 7f4f8b261740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5780204c0 con 0x55c577ce91b0 2024-07-04T08:30:46.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577cf1390 0x55c5780b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f7e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577d12070 0x55c5780b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f7e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577d12070 0x55c5780b6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50600/0 (socket says 172.21.15.39:50600) 2024-07-04T08:30:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f7e800640 1 -- 172.21.15.39:0/2071963040 learned_addr learned my addr 172.21.15.39:0/2071963040 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f77e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577cf1390 0x55c5780b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33764/0 (socket says 172.21.15.39:33764) 2024-07-04T08:30:46.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f76a00640 1 -- 172.21.15.39:0/4089180643 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4f70042070 con 0x7f4f801050f0 2024-07-04T08:30:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f7e800640 1 -- 172.21.15.39:0/2071963040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577ce91b0 msgr2=0x55c5780b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f7e800640 1 --2- 172.21.15.39:0/2071963040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577ce91b0 0x55c5780b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f7e800640 1 -- 172.21.15.39:0/2071963040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577cf1390 msgr2=0x55c5780b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f7e800640 1 --2- 172.21.15.39:0/2071963040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577cf1390 0x55c5780b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f7e800640 1 -- 172.21.15.39:0/2071963040 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c577fb4650 con 0x55c577d12070 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.875+0000 7f4f77e00640 1 --2- 172.21.15.39:0/2071963040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577cf1390 0x55c5780b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.876+0000 7f4f7e800640 1 --2- 172.21.15.39:0/2071963040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577d12070 0x55c5780b6950 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4f78011440 tx=0x7f4f78011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d090350214d969e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.876+0000 7f4f6f400640 1 -- 172.21.15.39:0/2071963040 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78003c30 con 0x55c577d12070 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.876+0000 7f4f6f400640 1 -- 172.21.15.39:0/2071963040 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f78003dd0 con 0x55c577d12070 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.876+0000 7f4f6f400640 1 -- 172.21.15.39:0/2071963040 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800fa40 con 0x55c577d12070 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f76a00640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f640b2960 0x7f4f640b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f8b261740 1 -- 172.21.15.39:0/2071963040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577d12070 msgr2=0x55c5780b6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f8b261740 1 --2- 172.21.15.39:0/2071963040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577d12070 0x55c5780b6950 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4f78011440 tx=0x7f4f78011900 comp rx=0 tx=0).stop 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f76a00640 1 -- 172.21.15.39:0/4089180643 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f4f70102d50 con 0x7f4f801050f0 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f8b261740 1 -- 172.21.15.39:0/2071963040 shutdown_connections 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f8b261740 1 --2- 172.21.15.39:0/2071963040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577cf1390 0x55c5780b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f8b261740 1 --2- 172.21.15.39:0/2071963040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577ce91b0 0x55c5780b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f8b261740 1 --2- 172.21.15.39:0/2071963040 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577d12070 0x55c5780b6950 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f8b261740 1 -- 172.21.15.39:0/2071963040 >> 172.21.15.39:0/2071963040 conn(0x55c577d06460 msgr2=0x55c5780b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:46.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f7de00640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f640b2960 0x7f4f640b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f8b261740 1 -- 172.21.15.39:0/2071963040 shutdown_connections 2024-07-04T08:30:46.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.877+0000 7f4f8b261740 1 -- 172.21.15.39:0/2071963040 wait complete. 2024-07-04T08:30:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.878+0000 7f4f8b261740 1 -- start start 2024-07-04T08:30:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.878+0000 7f4f7de00640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f640b2960 0x7f4f640b4e70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55c5780b55d0 tx=0x7f4f70009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.878+0000 7f4f8b261740 1 Processor -- start 2024-07-04T08:30:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577ce91b0 0x55c5780c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577cf1390 0x55c5780c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 0x55c5780ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f8b261740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c577e80700 con 0x55c577d12070 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f8b261740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c577fc3bb0 con 0x55c577ce91b0 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f8b261740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5780c5130 con 0x55c577cf1390 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f8b261740 10 client.0 _ll_register_callbacks cb 0x55c578015570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f77e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577cf1390 0x55c5780c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 0x55c5780ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f77e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577cf1390 0x55c5780c2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44728/0 (socket says 172.21.15.39:44728) 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.879+0000 7f4f77e00640 1 -- 172.21.15.39:0/174634851 learned_addr learned my addr 172.21.15.39:0/174634851 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.880+0000 7f4f7e800640 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577ce91b0 0x55c5780c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.880+0000 7f4f77e00640 1 -- 172.21.15.39:0/174634851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577ce91b0 msgr2=0x55c5780c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.880+0000 7f4f77e00640 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577ce91b0 0x55c5780c2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.880+0000 7f4f77e00640 1 -- 172.21.15.39:0/174634851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 msgr2=0x55c5780ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.880+0000 7f4f77e00640 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 0x55c5780ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.880+0000 7f4f77e00640 1 -- 172.21.15.39:0/174634851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c577d7d650 con 0x55c577cf1390 2024-07-04T08:30:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.880+0000 7f4f7e800640 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577ce91b0 0x55c5780c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.880+0000 7f4f77e00640 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577cf1390 0x55c5780c2ad0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4f680124d0 tx=0x7f4f68012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e986491d386366d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.880+0000 7f4f63e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:30:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.881+0000 7f4f63e00640 1 -- 172.21.15.39:0/174634851 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68004050 con 0x55c577cf1390 2024-07-04T08:30:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.881+0000 7f4f63e00640 1 -- 172.21.15.39:0/174634851 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f4f680041f0 con 0x55c577cf1390 2024-07-04T08:30:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.881+0000 7f4f8b261740 1 -- 172.21.15.39:0/174634851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f78003dd0 con 0x55c577cf1390 2024-07-04T08:30:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.881+0000 7f4f63e00640 1 -- 172.21.15.39:0/174634851 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f680043f0 con 0x55c577cf1390 2024-07-04T08:30:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.881+0000 7f4f8b261740 10 client.28822 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:46.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.881+0000 7f4f8b261740 1 -- 172.21.15.39:0/174634851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c5780d1220 con 0x55c577cf1390 2024-07-04T08:30:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.881+0000 7f4f63e00640 1 -- 172.21.15.39:0/174634851 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f78003dd0 con 0x55c577cf1390 2024-07-04T08:30:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.882+0000 7f4f63e00640 1 -- 172.21.15.39:0/174634851 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f6800f7a0 con 0x55c577cf1390 2024-07-04T08:30:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.882+0000 7f4f8b261740 10 client.28822 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.882+0000 7f4f8b261740 10 client.28822 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:30:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.882+0000 7f4f8b261740 20 client.28822 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.882+0000 7f4f8b261740 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c577cd09f0 0x55c5780d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.882+0000 7f4f8b261740 4 client.28822 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.882+0000 7f4f8b261740 1 -- 172.21.15.39:0/174634851 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c57802ee30 con 0x55c577cd09f0 2024-07-04T08:30:46.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.883+0000 7f4f7e800640 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c577cd09f0 0x55c5780d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.883+0000 7f4f7e800640 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c577cd09f0 0x55c5780d4790 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=559ff36519ca689f server_cookie=9f59399eb84973f5 in_seq=0 out_seq=0 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.883+0000 7f4f63e00640 10 client.28822 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f63e00640 1 -- 172.21.15.39:0/174634851 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c57802ee30 con 0x55c577cd09f0 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f63e00640 10 client.28822 handle_command_reply: tid=0 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f8b261740 1 client.28822 shutdown 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f8b261740 1 -- 172.21.15.39:0/174634851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577cf1390 msgr2=0x55c5780c2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f8b261740 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577cf1390 0x55c5780c2ad0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4f680124d0 tx=0x7f4f68012990 comp rx=0 tx=0).stop 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f8b261740 1 -- 172.21.15.39:0/174634851 shutdown_connections 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f8b261740 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c577cd09f0 0x55c5780d4790 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f8b261740 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 0x55c5780ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f8b261740 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577cf1390 0x55c5780c2ad0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f8b261740 1 --2- 172.21.15.39:0/174634851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577ce91b0 0x55c5780c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f63e00640 0 client.28822 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:30:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.884+0000 7f4f8b261740 1 -- 172.21.15.39:0/174634851 >> 172.21.15.39:0/174634851 conn(0x55c577d06460 msgr2=0x55c5780b9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.885+0000 7f4f8b261740 1 -- 172.21.15.39:0/174634851 shutdown_connections 2024-07-04T08:30:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.885+0000 7f4f8b261740 1 -- 172.21.15.39:0/174634851 wait complete. 2024-07-04T08:30:46.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.885+0000 7f4f8b261740 20 client.28822 trim_cache size 0 max 16384 2024-07-04T08:30:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.900+0000 7f4f8b261740 1 Processor -- start 2024-07-04T08:30:46.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.900+0000 7f4f8b261740 1 -- start start 2024-07-04T08:30:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577ce91b0 0x55c5780ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780bbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577d12070 0x55c5780bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f8b261740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b2410 con 0x55c577ce91b0 2024-07-04T08:30:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f8b261740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b3f50 con 0x55c577cf1390 2024-07-04T08:30:46.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f8b261740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b2ae0 con 0x55c577d12070 2024-07-04T08:30:46.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577ce91b0 0x55c5780ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f77e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780bbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f77e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780bbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50612/0 (socket says 172.21.15.39:50612) 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577ce91b0 0x55c5780ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33780/0 (socket says 172.21.15.39:33780) 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.901+0000 7f4f77e00640 1 -- 172.21.15.39:0/120020115 learned_addr learned my addr 172.21.15.39:0/120020115 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.902+0000 7f4f7e800640 1 --2- 172.21.15.39:0/120020115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577d12070 0x55c5780bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.902+0000 7f4f77e00640 1 -- 172.21.15.39:0/120020115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577d12070 msgr2=0x55c5780bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.902+0000 7f4f77e00640 1 --2- 172.21.15.39:0/120020115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577d12070 0x55c5780bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.902+0000 7f4f77e00640 1 -- 172.21.15.39:0/120020115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577ce91b0 msgr2=0x55c5780ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.902+0000 7f4f77e00640 1 --2- 172.21.15.39:0/120020115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577ce91b0 0x55c5780ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.902+0000 7f4f77e00640 1 -- 172.21.15.39:0/120020115 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5780b90a0 con 0x55c577cf1390 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.902+0000 7f4f7e800640 1 --2- 172.21.15.39:0/120020115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577ce91b0 0x55c5780ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:46.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.902+0000 7f4f77e00640 1 --2- 172.21.15.39:0/120020115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780bbbd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4f680124a0 tx=0x7f4f68009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0b8049bbfc585 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:46.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.902+0000 7f4f6ea00640 1 -- 172.21.15.39:0/120020115 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800f5c0 con 0x55c577cf1390 2024-07-04T08:30:46.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.903+0000 7f4f6ea00640 1 -- 172.21.15.39:0/120020115 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f6802e070 con 0x55c577cf1390 2024-07-04T08:30:46.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.903+0000 7f4f6ea00640 1 -- 172.21.15.39:0/120020115 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68010890 con 0x55c577cf1390 2024-07-04T08:30:46.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.903+0000 7f4f8b261740 1 -- 172.21.15.39:0/120020115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 msgr2=0x55c5780bbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.903+0000 7f4f8b261740 1 --2- 172.21.15.39:0/120020115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780bbbd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4f680124a0 tx=0x7f4f68009b80 comp rx=0 tx=0).stop 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.903+0000 7f4f8b261740 1 -- 172.21.15.39:0/120020115 shutdown_connections 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.903+0000 7f4f8b261740 1 --2- 172.21.15.39:0/120020115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577d12070 0x55c5780bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.903+0000 7f4f8b261740 1 --2- 172.21.15.39:0/120020115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780bbbd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.903+0000 7f4f8b261740 1 --2- 172.21.15.39:0/120020115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577ce91b0 0x55c5780ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.904+0000 7f4f8b261740 1 -- 172.21.15.39:0/120020115 >> 172.21.15.39:0/120020115 conn(0x55c577d06460 msgr2=0x55c5780c0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.904+0000 7f4f8b261740 1 -- 172.21.15.39:0/120020115 shutdown_connections 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.904+0000 7f4f8b261740 1 -- 172.21.15.39:0/120020115 wait complete. 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.904+0000 7f4f8b261740 1 -- start start 2024-07-04T08:30:46.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.904+0000 7f4f8b261740 1 Processor -- start 2024-07-04T08:30:46.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.905+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577ce91b0 0x55c5780ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.905+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.905+0000 7f4f8b261740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 0x55c5780c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.905+0000 7f4f8b261740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b3880 con 0x55c577d12070 2024-07-04T08:30:46.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.905+0000 7f4f8b261740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b31b0 con 0x55c577cf1390 2024-07-04T08:30:46.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.905+0000 7f4f8b261740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f801b03f0 con 0x55c577ce91b0 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.905+0000 7f4f8b261740 10 client.0 _ll_register_callbacks cb 0x7f4f68009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.905+0000 7f4f7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 0x55c5780c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f77e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f7e800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 0x55c5780c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33784/0 (socket says 172.21.15.39:33784) 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f7e800640 1 -- 172.21.15.39:0/1951750445 learned_addr learned my addr 172.21.15.39:0/1951750445 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f7e800640 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577ce91b0 0x55c5780ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f77e00640 1 -- 172.21.15.39:0/1951750445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577ce91b0 msgr2=0x55c5780ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f77e00640 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577ce91b0 0x55c5780ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f77e00640 1 -- 172.21.15.39:0/1951750445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 msgr2=0x55c5780c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f77e00640 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 0x55c5780c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.894 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f77e00640 1 -- 172.21.15.39:0/1951750445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5780caff0 con 0x55c577cf1390 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f77e00640 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780ca900 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55c5780c48c0 tx=0x7f4f680052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f6f1ddf7294c03 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.906+0000 7f4f63e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.907+0000 7f4f63e00640 1 -- 172.21.15.39:0/1951750445 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68004590 con 0x55c577cf1390 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.907+0000 7f4f63e00640 1 -- 172.21.15.39:0/1951750445 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f4f68018070 con 0x55c577cf1390 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.907+0000 7f4f8b261740 1 -- 172.21.15.39:0/1951750445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f6802e070 con 0x55c577cf1390 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.907+0000 7f4f63e00640 1 -- 172.21.15.39:0/1951750445 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f68004730 con 0x55c577cf1390 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.907+0000 7f4f8b261740 10 client.28898 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.907+0000 7f4f8b261740 1 -- 172.21.15.39:0/1951750445 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c57813d4d0 con 0x55c577cf1390 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.908+0000 7f4f63e00640 1 -- 172.21.15.39:0/1951750445 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f6802e070 con 0x55c577cf1390 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.908+0000 7f4f63e00640 1 -- 172.21.15.39:0/1951750445 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f680109b0 con 0x55c577cf1390 2024-07-04T08:30:46.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.908+0000 7f4f8b261740 10 client.28898 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:46.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.908+0000 7f4f8b261740 10 client.28898 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:30:46.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.908+0000 7f4f8b261740 20 client.28898 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:46.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.908+0000 7f4f8b261740 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c577ca5da0 0x55c57813fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:46.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.908+0000 7f4f8b261740 4 client.28898 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:30:46.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.908+0000 7f4f8b261740 1 -- 172.21.15.39:0/1951750445 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c57802a8d0 con 0x55c577ca5da0 2024-07-04T08:30:46.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.909+0000 7f4f7e800640 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c577ca5da0 0x55c57813fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.909+0000 7f4f7e800640 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c577ca5da0 0x55c57813fb20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a57bc1bf29ed7525 server_cookie=a0ec1cb2d6f2555a in_seq=0 out_seq=0 2024-07-04T08:30:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.909+0000 7f4f63e00640 10 client.28898 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:30:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f63e00640 1 -- 172.21.15.39:0/1951750445 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c57802a8d0 con 0x55c577ca5da0 2024-07-04T08:30:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f63e00640 10 client.28898 handle_command_reply: tid=0 2024-07-04T08:30:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 client.28898 shutdown 2024-07-04T08:30:46.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 -- 172.21.15.39:0/1951750445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 msgr2=0x55c5780ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780ca900 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x55c5780c48c0 tx=0x7f4f680052d0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 -- 172.21.15.39:0/1951750445 shutdown_connections 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c577ca5da0 0x55c57813fb20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c577d12070 0x55c5780c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c577cf1390 0x55c5780ca900 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 --2- 172.21.15.39:0/1951750445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c577ce91b0 0x55c5780ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f63e00640 0 client.28898 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 -- 172.21.15.39:0/1951750445 >> 172.21.15.39:0/1951750445 conn(0x55c577d06460 msgr2=0x55c5780b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 -- 172.21.15.39:0/1951750445 shutdown_connections 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 1 -- 172.21.15.39:0/1951750445 wait complete. 2024-07-04T08:30:46.899 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.910+0000 7f4f8b261740 20 client.28898 trim_cache size 0 max 16384 2024-07-04T08:30:46.900 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:30:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 -- 172.21.15.39:0/4089180643 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f640b2960 msgr2=0x7f4f640b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f640b2960 0x7f4f640b4e70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55c5780b55d0 tx=0x7f4f70009260 comp rx=0 tx=0).stop 2024-07-04T08:30:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 -- 172.21.15.39:0/4089180643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 msgr2=0x7f4f8019ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:46.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 0x7f4f8019ba50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4f70005ab0 tx=0x7f4f70003c10 comp rx=0 tx=0).stop 2024-07-04T08:30:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 -- 172.21.15.39:0/4089180643 shutdown_connections 2024-07-04T08:30:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f640b2960 0x7f4f640b4e70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f8010aa30 0x7f4f8019bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f8010b380 0x7f4f801a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 --2- 172.21.15.39:0/4089180643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f801050f0 0x7f4f8019ba50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.911+0000 7f4f7f200640 1 -- 172.21.15.39:0/4089180643 >> 172.21.15.39:0/4089180643 conn(0x7f4f80100b60 msgr2=0x7f4f80102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.912+0000 7f4f7f200640 1 -- 172.21.15.39:0/4089180643 shutdown_connections 2024-07-04T08:30:46.901 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:46.912+0000 7f4f7f200640 1 -- 172.21.15.39:0/4089180643 wait complete. 2024-07-04T08:30:47.777 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:30:48.278 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:49.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:49 smithi133 ceph-mon[37050]: pgmap v966: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-07-04T08:30:49.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:49 smithi039 ceph-mon[32062]: pgmap v966: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-07-04T08:30:49.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:49 smithi050 ceph-mon[36577]: pgmap v966: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-07-04T08:30:50.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.663+0000 7f8a07200640 1 -- 172.21.15.39:0/3099002433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0810b370 msgr2=0x7f8a08111ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:50.650 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.663+0000 7f8a07200640 1 --2- 172.21.15.39:0/3099002433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0810b370 0x7f8a08111ba0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009be0 tx=0x7f8a0002a6d0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.663+0000 7f8a07200640 1 -- 172.21.15.39:0/3099002433 shutdown_connections 2024-07-04T08:30:50.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.663+0000 7f8a07200640 1 --2- 172.21.15.39:0/3099002433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a0810aa20 0x7f8a0810ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.663+0000 7f8a07200640 1 --2- 172.21.15.39:0/3099002433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0810b370 0x7f8a08111ba0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.663+0000 7f8a07200640 1 --2- 172.21.15.39:0/3099002433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a081050e0 0x7f8a081054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.663+0000 7f8a07200640 1 -- 172.21.15.39:0/3099002433 >> 172.21.15.39:0/3099002433 conn(0x7f8a08100b30 msgr2=0x7f8a08102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:50.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.663+0000 7f8a07200640 1 -- 172.21.15.39:0/3099002433 shutdown_connections 2024-07-04T08:30:50.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.663+0000 7f8a07200640 1 -- 172.21.15.39:0/3099002433 wait complete. 2024-07-04T08:30:50.651 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.664+0000 7f8a07200640 1 Processor -- start 2024-07-04T08:30:50.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a07200640 1 -- start start 2024-07-04T08:30:50.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a07200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a081050e0 0x7f8a0819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a07200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0810aa20 0x7f8a0819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.652 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a07200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0810b370 0x7f8a081a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a07200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08115020 con 0x7f8a081050e0 2024-07-04T08:30:50.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a07200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08114ea0 con 0x7f8a0810b370 2024-07-04T08:30:50.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a07200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081151a0 con 0x7f8a0810aa20 2024-07-04T08:30:50.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a05e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a081050e0 0x7f8a0819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a05e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a081050e0 0x7f8a0819b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33800/0 (socket says 172.21.15.39:33800) 2024-07-04T08:30:50.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a05e00640 1 -- 172.21.15.39:0/1927294775 learned_addr learned my addr 172.21.15.39:0/1927294775 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:50.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.665+0000 7f8a05400640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0810aa20 0x7f8a0819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.653 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.666+0000 7f8a06800640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0810b370 0x7f8a081a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.666+0000 7f8a05e00640 1 -- 172.21.15.39:0/1927294775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0810aa20 msgr2=0x7f8a0819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:50.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.666+0000 7f8a05e00640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0810aa20 0x7f8a0819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.654 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.666+0000 7f8a05e00640 1 -- 172.21.15.39:0/1927294775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0810b370 msgr2=0x7f8a081a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:50.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.666+0000 7f8a05e00640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0810b370 0x7f8a081a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.666+0000 7f8a05e00640 1 -- 172.21.15.39:0/1927294775 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a081a1960 con 0x7f8a081050e0 2024-07-04T08:30:50.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.666+0000 7f8a05400640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0810aa20 0x7f8a0819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:50.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.666+0000 7f8a05e00640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a081050e0 0x7f8a0819b9f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e9f0 tx=0x7f89f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:50.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.667+0000 7f89fea00640 1 -- 172.21.15.39:0/1927294775 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f800ccc0 con 0x7f8a081050e0 2024-07-04T08:30:50.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.667+0000 7f89fea00640 1 -- 172.21.15.39:0/1927294775 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f80108e0 con 0x7f8a081050e0 2024-07-04T08:30:50.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.667+0000 7f8a07200640 1 -- 172.21.15.39:0/1927294775 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a081a1c50 con 0x7f8a081050e0 2024-07-04T08:30:50.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.667+0000 7f89fea00640 1 -- 172.21.15.39:0/1927294775 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8018880 con 0x7f8a081050e0 2024-07-04T08:30:50.655 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.667+0000 7f8a07200640 1 -- 172.21.15.39:0/1927294775 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081a83b0 con 0x7f8a081050e0 2024-07-04T08:30:50.657 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.670+0000 7f8a13154740 1 Processor -- start 2024-07-04T08:30:50.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.671+0000 7f8a13154740 1 -- start start 2024-07-04T08:30:50.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.671+0000 7f89fea00640 1 -- 172.21.15.39:0/1927294775 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f89f8010430 con 0x7f8a081050e0 2024-07-04T08:30:50.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.671+0000 7f8a13154740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a342970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.671+0000 7f8a13154740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a342eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.671+0000 7f8a13154740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3433f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.671+0000 7f8a13154740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563f6a2b8f00 con 0x563f69f7d390 2024-07-04T08:30:50.658 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.671+0000 7f8a13154740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f6a2a84a0 con 0x563f69f9e070 2024-07-04T08:30:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.671+0000 7f8a13154740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563f6a2ac4e0 con 0x563f69f751b0 2024-07-04T08:30:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a05e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a342eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a05400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a06800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a342970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a05e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a342eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:44782/0 (socket says 172.21.15.39:44782) 2024-07-04T08:30:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a05e00640 1 -- 172.21.15.39:0/1025309502 learned_addr learned my addr 172.21.15.39:0/1025309502 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a05400640 1 -- 172.21.15.39:0/1025309502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 msgr2=0x563f6a342eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a05400640 1 --2- 172.21.15.39:0/1025309502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a342eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a05400640 1 -- 172.21.15.39:0/1025309502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 msgr2=0x563f6a342970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a05400640 1 --2- 172.21.15.39:0/1025309502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a342970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.672+0000 7f8a05400640 1 -- 172.21.15.39:0/1025309502 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f6a240670 con 0x563f69f7d390 2024-07-04T08:30:50.660 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.673+0000 7f8a05400640 1 --2- 172.21.15.39:0/1025309502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3433f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8a0806bba0 tx=0x7f89f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3ad993a45c0fa8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:50.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.673+0000 7f89f3400640 1 -- 172.21.15.39:0/1025309502 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4004030 con 0x563f69f7d390 2024-07-04T08:30:50.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.673+0000 7f89f3400640 1 -- 172.21.15.39:0/1025309502 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f40041d0 con 0x563f69f7d390 2024-07-04T08:30:50.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.673+0000 7f89f3400640 1 -- 172.21.15.39:0/1025309502 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f400f800 con 0x563f69f7d390 2024-07-04T08:30:50.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.673+0000 7f89fea00640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89ec0b29e0 0x7f89ec0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.673+0000 7f89fea00640 1 -- 172.21.15.39:0/1927294775 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f89f80e2b70 con 0x7f8a081050e0 2024-07-04T08:30:50.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a06800640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89ec0b29e0 0x7f89ec0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.661 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a13154740 1 -- 172.21.15.39:0/1025309502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 msgr2=0x563f6a3433f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:50.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a13154740 1 --2- 172.21.15.39:0/1025309502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3433f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8a0806bba0 tx=0x7f89f40118d0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a13154740 1 -- 172.21.15.39:0/1025309502 shutdown_connections 2024-07-04T08:30:50.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a13154740 1 --2- 172.21.15.39:0/1025309502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3433f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a13154740 1 --2- 172.21.15.39:0/1025309502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a342eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.662 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a13154740 1 --2- 172.21.15.39:0/1025309502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a342970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a13154740 1 -- 172.21.15.39:0/1025309502 >> 172.21.15.39:0/1025309502 conn(0x563f69f92460 msgr2=0x563f6a340860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:50.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a13154740 1 -- 172.21.15.39:0/1025309502 shutdown_connections 2024-07-04T08:30:50.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.674+0000 7f8a13154740 1 -- 172.21.15.39:0/1025309502 wait complete. 2024-07-04T08:30:50.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.675+0000 7f8a06800640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89ec0b29e0 0x7f89ec0b4ef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009370 tx=0x7f8a00004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:50.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.675+0000 7f8a13154740 1 -- start start 2024-07-04T08:30:50.663 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.675+0000 7f8a13154740 1 Processor -- start 2024-07-04T08:30:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a13154740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a34e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a13154740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a34ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a13154740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a35a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a13154740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563f6a10c700 con 0x563f69f7d390 2024-07-04T08:30:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a13154740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f6a24fbd0 con 0x563f69f9e070 2024-07-04T08:30:50.664 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a13154740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563f6a351150 con 0x563f69f751b0 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a13154740 10 client.0 _ll_register_callbacks cb 0x563f6a2a1590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a05400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a34ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a05e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a34e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a05400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a34ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33826/0 (socket says 172.21.15.39:33826) 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.676+0000 7f8a05400640 1 -- 172.21.15.39:0/588967230 learned_addr learned my addr 172.21.15.39:0/588967230 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.677+0000 7f8a06800640 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a35a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.677+0000 7f8a05e00640 1 -- 172.21.15.39:0/588967230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 msgr2=0x563f6a35a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.677+0000 7f8a05e00640 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a35a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.677+0000 7f8a05e00640 1 -- 172.21.15.39:0/588967230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 msgr2=0x563f6a34ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.677+0000 7f8a05e00640 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a34ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.677+0000 7f8a05e00640 1 -- 172.21.15.39:0/588967230 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6a009650 con 0x563f69f751b0 2024-07-04T08:30:50.665 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.677+0000 7f8a05e00640 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a34e520 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f89f80a6d00 tx=0x7f89f80a9860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faa297e1a6bc608e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:50.666 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.677+0000 7f89e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:30:50.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.678+0000 7f89e7e00640 1 -- 172.21.15.39:0/588967230 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f800ce60 con 0x563f69f751b0 2024-07-04T08:30:50.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.678+0000 7f8a13154740 1 -- 172.21.15.39:0/588967230 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f40041d0 con 0x563f69f751b0 2024-07-04T08:30:50.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.678+0000 7f89e7e00640 1 -- 172.21.15.39:0/588967230 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f89f80a8cf0 con 0x563f69f751b0 2024-07-04T08:30:50.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.678+0000 7f89e7e00640 1 -- 172.21.15.39:0/588967230 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f89f80e0390 con 0x563f69f751b0 2024-07-04T08:30:50.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.678+0000 7f8a13154740 10 client.28840 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:50.667 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.678+0000 7f8a13154740 1 -- 172.21.15.39:0/588967230 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563f6a35d1f0 con 0x563f69f751b0 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.678+0000 7f89e7e00640 1 -- 172.21.15.39:0/588967230 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f40041d0 con 0x563f69f751b0 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.678+0000 7f89e7e00640 1 -- 172.21.15.39:0/588967230 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f89f80a5400 con 0x563f69f751b0 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.679+0000 7f8a13154740 10 client.28840 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.679+0000 7f8a13154740 10 client.28840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.679+0000 7f8a13154740 20 client.28840 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.679+0000 7f8a13154740 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563f69f5c9f0 0x563f6a3605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.679+0000 7f8a13154740 4 client.28840 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.679+0000 7f8a13154740 1 -- 172.21.15.39:0/588967230 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f6a2bae50 con 0x563f69f5c9f0 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.679+0000 7f8a05400640 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563f69f5c9f0 0x563f6a3605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.680+0000 7f8a05400640 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563f69f5c9f0 0x563f6a3605d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91af67dbdedbfa93 server_cookie=fcbd4dea6d9b5aab in_seq=0 out_seq=0 2024-07-04T08:30:50.668 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.680+0000 7f89e7e00640 10 client.28840 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:30:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.680+0000 7f89e7e00640 1 -- 172.21.15.39:0/588967230 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563f6a2bae50 con 0x563f69f5c9f0 2024-07-04T08:30:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f89e7e00640 10 client.28840 handle_command_reply: tid=0 2024-07-04T08:30:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 client.28840 shutdown 2024-07-04T08:30:50.669 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 -- 172.21.15.39:0/588967230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 msgr2=0x563f6a34e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a34e520 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f89f80a6d00 tx=0x7f89f80a9860 comp rx=0 tx=0).stop 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 -- 172.21.15.39:0/588967230 shutdown_connections 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563f69f5c9f0 0x563f6a3605d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a34ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a35a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 --2- 172.21.15.39:0/588967230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a34e520 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f89e7e00640 0 client.28840 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 -- 172.21.15.39:0/588967230 >> 172.21.15.39:0/588967230 conn(0x563f69f92460 msgr2=0x563f6a345b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 -- 172.21.15.39:0/588967230 shutdown_connections 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 1 -- 172.21.15.39:0/588967230 wait complete. 2024-07-04T08:30:50.670 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.681+0000 7f8a13154740 20 client.28840 trim_cache size 0 max 16384 2024-07-04T08:30:50.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a13154740 1 Processor -- start 2024-07-04T08:30:50.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a13154740 1 -- start start 2024-07-04T08:30:50.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a13154740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a346430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a13154740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a13154740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a33fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.684 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a13154740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89e800fe70 con 0x563f69f7d390 2024-07-04T08:30:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a13154740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89e80119b0 con 0x563f69f9e070 2024-07-04T08:30:50.685 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a13154740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8010540 con 0x563f69f751b0 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a05400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.697+0000 7f8a05400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:33830/0 (socket says 172.21.15.39:33830) 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.698+0000 7f8a05400640 1 -- 172.21.15.39:0/2164645768 learned_addr learned my addr 172.21.15.39:0/2164645768 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.698+0000 7f8a05e00640 1 --2- 172.21.15.39:0/2164645768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a346430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.698+0000 7f8a06800640 1 --2- 172.21.15.39:0/2164645768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a33fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.698+0000 7f8a05400640 1 -- 172.21.15.39:0/2164645768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 msgr2=0x563f6a346430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.698+0000 7f8a05400640 1 --2- 172.21.15.39:0/2164645768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a346430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.698+0000 7f8a05400640 1 -- 172.21.15.39:0/2164645768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 msgr2=0x563f6a33fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.698+0000 7f8a05400640 1 --2- 172.21.15.39:0/2164645768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a33fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.686 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.698+0000 7f8a05400640 1 -- 172.21.15.39:0/2164645768 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f6a344c60 con 0x563f69f7d390 2024-07-04T08:30:50.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.698+0000 7f8a05400640 1 --2- 172.21.15.39:0/2164645768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3440b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009690 tx=0x7f89f4011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a0a588c328bd90 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:50.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.699+0000 7f89f2a00640 1 -- 172.21.15.39:0/2164645768 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4009b90 con 0x563f69f7d390 2024-07-04T08:30:50.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.699+0000 7f89f2a00640 1 -- 172.21.15.39:0/2164645768 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f40054f0 con 0x563f69f7d390 2024-07-04T08:30:50.687 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.699+0000 7f89f2a00640 1 -- 172.21.15.39:0/2164645768 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4003b70 con 0x563f69f7d390 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 -- 172.21.15.39:0/2164645768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 msgr2=0x563f6a3440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 --2- 172.21.15.39:0/2164645768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3440b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009690 tx=0x7f89f4011e50 comp rx=0 tx=0).stop 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 -- 172.21.15.39:0/2164645768 shutdown_connections 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 --2- 172.21.15.39:0/2164645768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f7d390 0x563f6a3440b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 --2- 172.21.15.39:0/2164645768 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f9e070 0x563f6a33fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 --2- 172.21.15.39:0/2164645768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f751b0 0x563f6a346430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 -- 172.21.15.39:0/2164645768 >> 172.21.15.39:0/2164645768 conn(0x563f69f92460 msgr2=0x563f6a3439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 -- 172.21.15.39:0/2164645768 shutdown_connections 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 -- 172.21.15.39:0/2164645768 wait complete. 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.700+0000 7f8a13154740 1 -- start start 2024-07-04T08:30:50.688 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.701+0000 7f8a13154740 1 Processor -- start 2024-07-04T08:30:50.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.701+0000 7f8a13154740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f751b0 0x563f6a356770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.701+0000 7f8a13154740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f7d390 0x563f6a356d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.701+0000 7f8a13154740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f9e070 0x563f6a351dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.701+0000 7f8a13154740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f89e80112e0 con 0x563f69f9e070 2024-07-04T08:30:50.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.701+0000 7f8a13154740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8010c10 con 0x563f69f751b0 2024-07-04T08:30:50.690 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.701+0000 7f8a13154740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89e800de50 con 0x563f69f7d390 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.701+0000 7f8a13154740 10 client.0 _ll_register_callbacks cb 0x7f89f80e6310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a05e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f751b0 0x563f6a356770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a05e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f751b0 0x563f6a356770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:50684/0 (socket says 172.21.15.39:50684) 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a05e00640 1 -- 172.21.15.39:0/835973800 learned_addr learned my addr 172.21.15.39:0/835973800 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a06800640 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f9e070 0x563f6a351dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a05400640 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f7d390 0x563f6a356d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a05e00640 1 -- 172.21.15.39:0/835973800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f7d390 msgr2=0x563f6a356d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a05e00640 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f7d390 0x563f6a356d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a05e00640 1 -- 172.21.15.39:0/835973800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f9e070 msgr2=0x563f6a351dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a05e00640 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f9e070 0x563f6a351dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.702+0000 7f8a05e00640 1 -- 172.21.15.39:0/835973800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6a352560 con 0x563f69f751b0 2024-07-04T08:30:50.691 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.703+0000 7f8a05e00640 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f751b0 0x563f6a356770 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f89f8000c00 tx=0x7f89f80b2a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183f225750d30e51 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:50.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.703+0000 7f89e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:50.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.703+0000 7f89e7e00640 1 -- 172.21.15.39:0/835973800 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f80a5220 con 0x563f69f751b0 2024-07-04T08:30:50.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.703+0000 7f8a13154740 1 -- 172.21.15.39:0/835973800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f40054f0 con 0x563f69f751b0 2024-07-04T08:30:50.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.703+0000 7f89e7e00640 1 -- 172.21.15.39:0/835973800 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f89f80cc2c0 con 0x563f69f751b0 2024-07-04T08:30:50.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.703+0000 7f89e7e00640 1 -- 172.21.15.39:0/835973800 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f89f80a93e0 con 0x563f69f751b0 2024-07-04T08:30:50.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.703+0000 7f8a13154740 10 client.28916 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:50.692 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.703+0000 7f8a13154740 1 -- 172.21.15.39:0/835973800 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563f6a3c8960 con 0x563f69f751b0 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.703+0000 7f89e7e00640 1 -- 172.21.15.39:0/835973800 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f40054f0 con 0x563f69f751b0 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.704+0000 7f89e7e00640 1 -- 172.21.15.39:0/835973800 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f89f80a8760 con 0x563f69f751b0 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.704+0000 7f8a13154740 10 client.28916 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.704+0000 7f8a13154740 10 client.28916 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.704+0000 7f8a13154740 20 client.28916 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.704+0000 7f8a13154740 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563f69f31da0 0x563f6a3cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.704+0000 7f8a13154740 4 client.28916 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.704+0000 7f8a13154740 1 -- 172.21.15.39:0/835973800 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f6a2b68f0 con 0x563f69f31da0 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.704+0000 7f8a05400640 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563f69f31da0 0x563f6a3cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.705+0000 7f8a05400640 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563f69f31da0 0x563f6a3cb0f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7683b75928e78e2f server_cookie=a25cf6d0812ebd81 in_seq=0 out_seq=0 2024-07-04T08:30:50.694 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.705+0000 7f89e7e00640 10 client.28916 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.705+0000 7f89e7e00640 1 -- 172.21.15.39:0/835973800 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f6a2b68f0 con 0x563f69f31da0 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.705+0000 7f89e7e00640 10 client.28916 handle_command_reply: tid=0 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.705+0000 7f8a13154740 1 client.28916 shutdown 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.705+0000 7f8a13154740 1 -- 172.21.15.39:0/835973800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f751b0 msgr2=0x563f6a356770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.705+0000 7f8a13154740 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f751b0 0x563f6a356770 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f89f8000c00 tx=0x7f89f80b2a00 comp rx=0 tx=0).stop 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.705+0000 7f8a13154740 1 -- 172.21.15.39:0/835973800 shutdown_connections 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.706+0000 7f8a13154740 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x563f69f31da0 0x563f6a3cb0f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.706+0000 7f8a13154740 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563f69f9e070 0x563f6a351dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.706+0000 7f8a13154740 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f69f7d390 0x563f6a356d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.706+0000 7f89e7e00640 0 client.28916 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.706+0000 7f8a13154740 1 --2- 172.21.15.39:0/835973800 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f69f751b0 0x563f6a356770 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.706+0000 7f8a13154740 1 -- 172.21.15.39:0/835973800 >> 172.21.15.39:0/835973800 conn(0x563f69f92460 msgr2=0x563f6a343850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:50.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.706+0000 7f8a13154740 1 -- 172.21.15.39:0/835973800 shutdown_connections 2024-07-04T08:30:50.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.706+0000 7f8a13154740 1 -- 172.21.15.39:0/835973800 wait complete. 2024-07-04T08:30:50.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.706+0000 7f8a13154740 20 client.28916 trim_cache size 0 max 16384 2024-07-04T08:30:50.696 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 -- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89ec0b29e0 msgr2=0x7f89ec0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89ec0b29e0 0x7f89ec0b4ef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009370 tx=0x7f8a00004a40 comp rx=0 tx=0).stop 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 -- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a081050e0 msgr2=0x7f8a0819b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a081050e0 0x7f8a0819b9f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e9f0 tx=0x7f89f800edf0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 -- 172.21.15.39:0/1927294775 shutdown_connections 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f89ec0b29e0 0x7f89ec0b4ef0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8a0810b370 0x7f8a081a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0810aa20 0x7f8a0819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 --2- 172.21.15.39:0/1927294775 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a081050e0 0x7f8a0819b9f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 -- 172.21.15.39:0/1927294775 >> 172.21.15.39:0/1927294775 conn(0x7f8a08100b30 msgr2=0x7f8a08102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 -- 172.21.15.39:0/1927294775 shutdown_connections 2024-07-04T08:30:50.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:50.707+0000 7f8a07200640 1 -- 172.21.15.39:0/1927294775 wait complete. 2024-07-04T08:30:51.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:51 smithi039 ceph-mon[32062]: pgmap v967: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-07-04T08:30:51.555 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:51 smithi133 ceph-mon[37050]: pgmap v967: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-07-04T08:30:51.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:51 smithi050 ceph-mon[36577]: pgmap v967: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-07-04T08:30:51.695 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:30:52.180 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:53.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:53 smithi039 ceph-mon[32062]: pgmap v968: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-07-04T08:30:53.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:53 smithi133 ceph-mon[37050]: pgmap v968: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-07-04T08:30:53.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:53 smithi050 ceph-mon[36577]: pgmap v968: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 16 op/s 2024-07-04T08:30:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.439+0000 7f832aa00640 1 -- 172.21.15.39:0/3168633156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83241030b0 msgr2=0x7f83241034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.439+0000 7f832aa00640 1 --2- 172.21.15.39:0/3168633156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83241030b0 0x7f83241034c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8314009900 tx=0x7f831402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.439+0000 7f832aa00640 1 -- 172.21.15.39:0/3168633156 shutdown_connections 2024-07-04T08:30:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.439+0000 7f832aa00640 1 --2- 172.21.15.39:0/3168633156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8324109340 0x7f832410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.427 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.439+0000 7f832aa00640 1 --2- 172.21.15.39:0/3168633156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83241089f0 0x7f8324108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.439+0000 7f832aa00640 1 --2- 172.21.15.39:0/3168633156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f83241030b0 0x7f83241034c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.439+0000 7f832aa00640 1 -- 172.21.15.39:0/3168633156 >> 172.21.15.39:0/3168633156 conn(0x7f832407ada0 msgr2=0x7f832407b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:54.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.439+0000 7f832aa00640 1 -- 172.21.15.39:0/3168633156 shutdown_connections 2024-07-04T08:30:54.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.439+0000 7f832aa00640 1 -- 172.21.15.39:0/3168633156 wait complete. 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.440+0000 7f832aa00640 1 Processor -- start 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.440+0000 7f832aa00640 1 -- start start 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f832aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83241030b0 0x7f832419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f832aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83241089f0 0x7f832419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f832aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8324109340 0x7f83241a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f832aa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8324113060 con 0x7f83241089f0 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f832aa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8324112ee0 con 0x7f8324109340 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f832aa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83241131e0 con 0x7f83241030b0 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f8323e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83241030b0 0x7f832419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.430 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f831b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83241089f0 0x7f832419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f8323e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83241030b0 0x7f832419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53460/0 (socket says 172.21.15.39:53460) 2024-07-04T08:30:54.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.441+0000 7f8323e00640 1 -- 172.21.15.39:0/92012596 learned_addr learned my addr 172.21.15.39:0/92012596 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:54.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.442+0000 7f8328a00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8324109340 0x7f83241a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.442+0000 7f831b400640 1 -- 172.21.15.39:0/92012596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83241030b0 msgr2=0x7f832419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:54.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.442+0000 7f831b400640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83241030b0 0x7f832419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.442+0000 7f831b400640 1 -- 172.21.15.39:0/92012596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8324109340 msgr2=0x7f83241a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:54.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.442+0000 7f831b400640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8324109340 0x7f83241a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.442+0000 7f831b400640 1 -- 172.21.15.39:0/92012596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83241a1be0 con 0x7f83241089f0 2024-07-04T08:30:54.431 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.442+0000 7f8323e00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83241030b0 0x7f832419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.442+0000 7f831b400640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83241089f0 0x7f832419c210 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f832406bf40 tx=0x7f830c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.443+0000 7f8321600640 1 -- 172.21.15.39:0/92012596 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c00ccc0 con 0x7f83241089f0 2024-07-04T08:30:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.443+0000 7f832aa00640 1 -- 172.21.15.39:0/92012596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83241a1ed0 con 0x7f83241089f0 2024-07-04T08:30:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.443+0000 7f8321600640 1 -- 172.21.15.39:0/92012596 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830c009e40 con 0x7f83241089f0 2024-07-04T08:30:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.443+0000 7f8321600640 1 -- 172.21.15.39:0/92012596 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c004a00 con 0x7f83241089f0 2024-07-04T08:30:54.432 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.443+0000 7f832aa00640 1 -- 172.21.15.39:0/92012596 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241a8630 con 0x7f83241089f0 2024-07-04T08:30:54.433 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.446+0000 7f8330ffa740 1 Processor -- start 2024-07-04T08:30:54.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8330ffa740 1 -- start start 2024-07-04T08:30:54.434 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8321600640 1 -- 172.21.15.39:0/92012596 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f830c005020 con 0x7f83241089f0 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 0x5586122b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8330ffa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558612229ee0 con 0x558611f0f070 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8330ffa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558612219480 con 0x558611ee61b0 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8330ffa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55861221d4c0 con 0x558611eee390 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8328a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.447+0000 7f8328a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47390/0 (socket says 172.21.15.39:47390) 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.448+0000 7f8328a00640 1 -- 172.21.15.39:0/2386059578 learned_addr learned my addr 172.21.15.39:0/2386059578 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:54.435 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.448+0000 7f8323e00640 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 0x5586122b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.436 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.448+0000 7f831b400640 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.448+0000 7f8323e00640 1 -- 172.21.15.39:0/2386059578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 msgr2=0x5586122b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.448+0000 7f8323e00640 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.448+0000 7f8323e00640 1 -- 172.21.15.39:0/2386059578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 msgr2=0x5586122b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.448+0000 7f8323e00640 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.448+0000 7f8323e00640 1 -- 172.21.15.39:0/2386059578 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586121b1650 con 0x558611ee61b0 2024-07-04T08:30:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.448+0000 7f8328a00640 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.449+0000 7f8323e00640 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 0x5586122b3cf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8314000c00 tx=0x7f83140044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa9bea98ba21ee6d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:54.437 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.449+0000 7f8321600640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83080b2960 0x7f83080b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.449+0000 7f8319600640 1 -- 172.21.15.39:0/2386059578 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314040070 con 0x558611ee61b0 2024-07-04T08:30:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.449+0000 7f8319600640 1 -- 172.21.15.39:0/2386059578 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83140047a0 con 0x558611ee61b0 2024-07-04T08:30:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.449+0000 7f8319600640 1 -- 172.21.15.39:0/2386059578 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314035040 con 0x558611ee61b0 2024-07-04T08:30:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.449+0000 7f8321600640 1 -- 172.21.15.39:0/92012596 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f830c0e2990 con 0x7f83241089f0 2024-07-04T08:30:54.438 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.449+0000 7f8328a00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83080b2960 0x7f83080b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.450+0000 7f8330ffa740 1 -- 172.21.15.39:0/2386059578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 msgr2=0x5586122b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.450+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 0x5586122b3cf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8314000c00 tx=0x7f83140044b0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.450+0000 7f8328a00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83080b2960 0x7f83080b4e70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f831c012840 tx=0x7f831c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.450+0000 7f8330ffa740 1 -- 172.21.15.39:0/2386059578 shutdown_connections 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.450+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.450+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 0x5586122b3cf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.450+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2386059578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.450+0000 7f8330ffa740 1 -- 172.21.15.39:0/2386059578 >> 172.21.15.39:0/2386059578 conn(0x558611f03460 msgr2=0x5586122b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.451+0000 7f8330ffa740 1 -- 172.21.15.39:0/2386059578 shutdown_connections 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.451+0000 7f8330ffa740 1 -- 172.21.15.39:0/2386059578 wait complete. 2024-07-04T08:30:54.441 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.451+0000 7f8330ffa740 1 -- start start 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.451+0000 7f8330ffa740 1 Processor -- start 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 0x5586122bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8330ffa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55861207d700 con 0x558611f0f070 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8330ffa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5586121c0bb0 con 0x558611ee61b0 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8330ffa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5586122c1f30 con 0x558611eee390 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8330ffa740 10 client.0 _ll_register_callbacks cb 0x558612212570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8328a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f831b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.442 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f831b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122bf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53478/0 (socket says 172.21.15.39:53478) 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8323e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 0x5586122bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f831b400640 1 -- 172.21.15.39:0/3506326487 learned_addr learned my addr 172.21.15.39:0/3506326487 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.452+0000 7f8328a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47394/0 (socket says 172.21.15.39:47394) 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f831b400640 1 -- 172.21.15.39:0/3506326487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 msgr2=0x5586122bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f831b400640 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 0x5586122bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f831b400640 1 -- 172.21.15.39:0/3506326487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 msgr2=0x5586122cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f831b400640 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f831b400640 1 -- 172.21.15.39:0/3506326487 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558611f7a650 con 0x558611eee390 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f831b400640 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122bf880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c6a0 tx=0x7f830c0a9e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=195633ef6f7eed5f server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f8312000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:30:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f8312000640 1 -- 172.21.15.39:0/3506326487 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c01c070 con 0x558611eee390 2024-07-04T08:30:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f8330ffa740 1 -- 172.21.15.39:0/3506326487 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83140047a0 con 0x558611eee390 2024-07-04T08:30:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.453+0000 7f8312000640 1 -- 172.21.15.39:0/3506326487 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f830c004ba0 con 0x558611eee390 2024-07-04T08:30:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.454+0000 7f8312000640 1 -- 172.21.15.39:0/3506326487 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f830c0a9680 con 0x558611eee390 2024-07-04T08:30:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.454+0000 7f8330ffa740 10 client.28858 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:54.444 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.454+0000 7f8330ffa740 1 -- 172.21.15.39:0/3506326487 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5586122cdfa0 con 0x558611eee390 2024-07-04T08:30:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.454+0000 7f8312000640 1 -- 172.21.15.39:0/3506326487 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83140047a0 con 0x558611eee390 2024-07-04T08:30:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.454+0000 7f8312000640 1 -- 172.21.15.39:0/3506326487 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f830c0aabe0 con 0x558611eee390 2024-07-04T08:30:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.455+0000 7f8330ffa740 10 client.28858 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.455+0000 7f8330ffa740 10 client.28858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.455+0000 7f8330ffa740 20 client.28858 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.455+0000 7f8330ffa740 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558611ecd9f0 0x5586122d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.455+0000 7f8330ffa740 4 client.28858 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.455+0000 7f8330ffa740 1 -- 172.21.15.39:0/3506326487 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55861222be30 con 0x558611ecd9f0 2024-07-04T08:30:54.446 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.455+0000 7f8323e00640 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558611ecd9f0 0x5586122d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.456+0000 7f8323e00640 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558611ecd9f0 0x5586122d13f0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adc0be77db396ae server_cookie=38e099515f61c3b3 in_seq=0 out_seq=0 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.456+0000 7f8312000640 10 client.28858 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.456+0000 7f8312000640 1 -- 172.21.15.39:0/3506326487 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55861222be30 con 0x558611ecd9f0 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.456+0000 7f8312000640 10 client.28858 handle_command_reply: tid=0 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 client.28858 shutdown 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 -- 172.21.15.39:0/3506326487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 msgr2=0x5586122bf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122bf880 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c6a0 tx=0x7f830c0a9e00 comp rx=0 tx=0).stop 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 -- 172.21.15.39:0/3506326487 shutdown_connections 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558611ecd9f0 0x5586122d13f0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.447 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122bf880 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 --2- 172.21.15.39:0/3506326487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611ee61b0 0x5586122bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 -- 172.21.15.39:0/3506326487 >> 172.21.15.39:0/3506326487 conn(0x558611f03460 msgr2=0x5586122b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8312000640 0 client.28858 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 -- 172.21.15.39:0/3506326487 shutdown_connections 2024-07-04T08:30:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 1 -- 172.21.15.39:0/3506326487 wait complete. 2024-07-04T08:30:54.448 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.457+0000 7f8330ffa740 20 client.28858 trim_cache size 0 max 16384 2024-07-04T08:30:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.473+0000 7f8330ffa740 1 Processor -- start 2024-07-04T08:30:54.460 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.473+0000 7f8330ffa740 1 -- start start 2024-07-04T08:30:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.473+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611ee61b0 0x5586122b0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.473+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.473+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611f0f070 0x5586122b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.473+0000 7f8330ffa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b2560 con 0x558611ee61b0 2024-07-04T08:30:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.473+0000 7f8330ffa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b40a0 con 0x558611f0f070 2024-07-04T08:30:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.473+0000 7f8330ffa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b2c30 con 0x558611eee390 2024-07-04T08:30:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f8323e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611ee61b0 0x5586122b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f8323e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611ee61b0 0x5586122b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47406/0 (socket says 172.21.15.39:47406) 2024-07-04T08:30:54.461 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f8323e00640 1 -- 172.21.15.39:0/1425495397 learned_addr learned my addr 172.21.15.39:0/1425495397 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:54.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f831b400640 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.462 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f8328a00640 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611f0f070 0x5586122b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f831b400640 1 -- 172.21.15.39:0/1425495397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611f0f070 msgr2=0x5586122b8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f831b400640 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611f0f070 0x5586122b8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f831b400640 1 -- 172.21.15.39:0/1425495397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611ee61b0 msgr2=0x5586122b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f831b400640 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611ee61b0 0x5586122b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f831b400640 1 -- 172.21.15.39:0/1425495397 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586122b6040 con 0x558611eee390 2024-07-04T08:30:54.463 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.474+0000 7f8323e00640 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611ee61b0 0x5586122b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:54.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.475+0000 7f831b400640 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122b16d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f830c0e8190 tx=0x7f830c0a5040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523e39361d448691 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:54.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.475+0000 7f8318c00640 1 -- 172.21.15.39:0/1425495397 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c01c070 con 0x558611eee390 2024-07-04T08:30:54.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.475+0000 7f8318c00640 1 -- 172.21.15.39:0/1425495397 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830c005690 con 0x558611eee390 2024-07-04T08:30:54.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.475+0000 7f8318c00640 1 -- 172.21.15.39:0/1425495397 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c0a9020 con 0x558611eee390 2024-07-04T08:30:54.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.476+0000 7f8330ffa740 1 -- 172.21.15.39:0/1425495397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 msgr2=0x5586122b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:54.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.476+0000 7f8330ffa740 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122b16d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f830c0e8190 tx=0x7f830c0a5040 comp rx=0 tx=0).stop 2024-07-04T08:30:54.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.476+0000 7f8330ffa740 1 -- 172.21.15.39:0/1425495397 shutdown_connections 2024-07-04T08:30:54.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.476+0000 7f8330ffa740 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611f0f070 0x5586122b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.464 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.476+0000 7f8330ffa740 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611eee390 0x5586122b16d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.476+0000 7f8330ffa740 1 --2- 172.21.15.39:0/1425495397 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611ee61b0 0x5586122b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.476+0000 7f8330ffa740 1 -- 172.21.15.39:0/1425495397 >> 172.21.15.39:0/1425495397 conn(0x558611f03460 msgr2=0x5586122b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:54.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.476+0000 7f8330ffa740 1 -- 172.21.15.39:0/1425495397 shutdown_connections 2024-07-04T08:30:54.465 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.476+0000 7f8330ffa740 1 -- 172.21.15.39:0/1425495397 wait complete. 2024-07-04T08:30:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.477+0000 7f8330ffa740 1 -- start start 2024-07-04T08:30:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.477+0000 7f8330ffa740 1 Processor -- start 2024-07-04T08:30:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.477+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611ee61b0 0x5586122c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.477+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611eee390 0x5586122ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.467 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.477+0000 7f8330ffa740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122caee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f8330ffa740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b39d0 con 0x558611f0f070 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f8330ffa740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b3300 con 0x558611eee390 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f8330ffa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b0540 con 0x558611ee61b0 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f8330ffa740 10 client.0 _ll_register_callbacks cb 0x7f830c0e8220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f8323e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611ee61b0 0x5586122c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f8323e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611ee61b0 0x5586122c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:53504/0 (socket says 172.21.15.39:53504) 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f8323e00640 1 -- 172.21.15.39:0/2012780382 learned_addr learned my addr 172.21.15.39:0/2012780382 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f831b400640 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611eee390 0x5586122ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f8328a00640 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f831b400640 1 -- 172.21.15.39:0/2012780382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611ee61b0 msgr2=0x5586122c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:54.468 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f831b400640 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611ee61b0 0x5586122c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f831b400640 1 -- 172.21.15.39:0/2012780382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 msgr2=0x5586122caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f831b400640 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f831b400640 1 -- 172.21.15.39:0/2012780382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586122cb580 con 0x558611eee390 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.478+0000 7f8328a00640 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.479+0000 7f831b400640 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611eee390 0x5586122ca9a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f830c0e8190 tx=0x7f830c0e2cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80acb5bb92edca3b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.479+0000 7f8312000640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.479+0000 7f8312000640 1 -- 172.21.15.39:0/2012780382 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c01c070 con 0x558611eee390 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.479+0000 7f8312000640 1 -- 172.21.15.39:0/2012780382 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f830c004ba0 con 0x558611eee390 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.479+0000 7f8330ffa740 1 -- 172.21.15.39:0/2012780382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586122278d0 con 0x558611eee390 2024-07-04T08:30:54.469 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.479+0000 7f8312000640 1 -- 172.21.15.39:0/2012780382 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f830c0a9200 con 0x558611eee390 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.479+0000 7f8330ffa740 10 client.28934 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.479+0000 7f8330ffa740 1 -- 172.21.15.39:0/2012780382 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558612339b80 con 0x558611eee390 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.480+0000 7f8312000640 1 -- 172.21.15.39:0/2012780382 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5586122278d0 con 0x558611eee390 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.480+0000 7f8312000640 1 -- 172.21.15.39:0/2012780382 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f830c06f080 con 0x558611eee390 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.480+0000 7f8330ffa740 10 client.28934 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.480+0000 7f8330ffa740 10 client.28934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.480+0000 7f8330ffa740 20 client.28934 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.480+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558611ea2da0 0x55861233bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.480+0000 7f8330ffa740 4 client.28934 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:30:54.470 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.480+0000 7f8330ffa740 1 -- 172.21.15.39:0/2012780382 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558612225010 con 0x558611ea2da0 2024-07-04T08:30:54.471 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.480+0000 7f8323e00640 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558611ea2da0 0x55861233bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.481+0000 7f8323e00640 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558611ea2da0 0x55861233bd90 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a3eb7df0e75565 server_cookie=1258f5d55383a4b0 in_seq=0 out_seq=0 2024-07-04T08:30:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.481+0000 7f8312000640 10 client.28934 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.481+0000 7f8312000640 1 -- 172.21.15.39:0/2012780382 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558612225010 con 0x558611ea2da0 2024-07-04T08:30:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.481+0000 7f8312000640 10 client.28934 handle_command_reply: tid=0 2024-07-04T08:30:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.481+0000 7f8330ffa740 1 client.28934 shutdown 2024-07-04T08:30:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 -- 172.21.15.39:0/2012780382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611eee390 msgr2=0x5586122ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611eee390 0x5586122ca9a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f830c0e8190 tx=0x7f830c0e2cd0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.472 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 -- 172.21.15.39:0/2012780382 shutdown_connections 2024-07-04T08:30:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x558611ea2da0 0x55861233bd90 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558611f0f070 0x5586122caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558611eee390 0x5586122ca9a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 --2- 172.21.15.39:0/2012780382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558611ee61b0 0x5586122c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8312000640 0 client.28934 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 -- 172.21.15.39:0/2012780382 >> 172.21.15.39:0/2012780382 conn(0x558611f03460 msgr2=0x5586122c05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 -- 172.21.15.39:0/2012780382 shutdown_connections 2024-07-04T08:30:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 1 -- 172.21.15.39:0/2012780382 wait complete. 2024-07-04T08:30:54.473 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.482+0000 7f8330ffa740 20 client.28934 trim_cache size 0 max 16384 2024-07-04T08:30:54.474 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:30:54.474 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 -- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83080b2960 msgr2=0x7f83080b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83080b2960 0x7f83080b4e70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f831c012840 tx=0x7f831c009500 comp rx=0 tx=0).stop 2024-07-04T08:30:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 -- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83241089f0 msgr2=0x7f832419c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83241089f0 0x7f832419c210 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f832406bf40 tx=0x7f830c00ed90 comp rx=0 tx=0).stop 2024-07-04T08:30:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 -- 172.21.15.39:0/92012596 shutdown_connections 2024-07-04T08:30:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f83080b2960 0x7f83080b4e70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83241089f0 0x7f832419c210 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8324109340 0x7f83241a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 --2- 172.21.15.39:0/92012596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83241030b0 0x7f832419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:54.475 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 -- 172.21.15.39:0/92012596 >> 172.21.15.39:0/92012596 conn(0x7f832407ada0 msgr2=0x7f8324106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 -- 172.21.15.39:0/92012596 shutdown_connections 2024-07-04T08:30:54.476 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:54.483+0000 7f832aa00640 1 -- 172.21.15.39:0/92012596 wait complete. 2024-07-04T08:30:55.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:55 smithi039 ceph-mon[32062]: pgmap v969: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-07-04T08:30:55.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:55 smithi050 ceph-mon[36577]: pgmap v969: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-07-04T08:30:55.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:55 smithi133 ceph-mon[37050]: pgmap v969: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-07-04T08:30:56.415 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:30:56.415 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:30:56.913 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:57.333 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:57 smithi039 ceph-mon[32062]: pgmap v970: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-07-04T08:30:57.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:57 smithi050 ceph-mon[36577]: pgmap v970: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-07-04T08:30:57.780 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:30:57.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:57 smithi133 ceph-mon[37050]: pgmap v970: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-07-04T08:30:58.820 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:30:59.484 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:30:59 smithi039 ceph-mon[32062]: pgmap v971: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-07-04T08:30:59.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:30:59 smithi050 ceph-mon[36577]: pgmap v971: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-07-04T08:30:59.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:30:59 smithi133 ceph-mon[37050]: pgmap v971: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 KiB/s wr, 11 op/s 2024-07-04T08:30:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.455+0000 7ff645800640 1 -- 172.21.15.39:0/1964866476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6401030b0 msgr2=0x7ff6401034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.455+0000 7ff645800640 1 --2- 172.21.15.39:0/1964866476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6401030b0 0x7ff6401034c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff630009ca0 tx=0x7ff63002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.455+0000 7ff645800640 1 -- 172.21.15.39:0/1964866476 shutdown_connections 2024-07-04T08:30:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.455+0000 7ff645800640 1 --2- 172.21.15.39:0/1964866476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff640109340 0x7ff64010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.455+0000 7ff645800640 1 --2- 172.21.15.39:0/1964866476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6401089f0 0x7ff640108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.455+0000 7ff645800640 1 --2- 172.21.15.39:0/1964866476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6401030b0 0x7ff6401034c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.455+0000 7ff645800640 1 -- 172.21.15.39:0/1964866476 >> 172.21.15.39:0/1964866476 conn(0x7ff64007ada0 msgr2=0x7ff64007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.456+0000 7ff645800640 1 -- 172.21.15.39:0/1964866476 shutdown_connections 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.456+0000 7ff645800640 1 -- 172.21.15.39:0/1964866476 wait complete. 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.457+0000 7ff645800640 1 Processor -- start 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.457+0000 7ff645800640 1 -- start start 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.457+0000 7ff645800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6401030b0 0x7ff640199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.457+0000 7ff645800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6401089f0 0x7ff640199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.457+0000 7ff645800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff640109340 0x7ff64019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.457+0000 7ff645800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff640112ff0 con 0x7ff640109340 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.457+0000 7ff645800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff640112e70 con 0x7ff6401089f0 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.457+0000 7ff645800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff640113170 con 0x7ff6401030b0 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff640109340 0x7ff64019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff640109340 0x7ff64019f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47418/0 (socket says 172.21.15.39:47418) 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63f400640 1 -- 172.21.15.39:0/2436465346 learned_addr learned my addr 172.21.15.39:0/2436465346 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63ea00640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6401030b0 0x7ff640199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63e000640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6401089f0 0x7ff640199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63f400640 1 -- 172.21.15.39:0/2436465346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6401030b0 msgr2=0x7ff640199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63f400640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6401030b0 0x7ff640199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63f400640 1 -- 172.21.15.39:0/2436465346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6401089f0 msgr2=0x7ff640199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63f400640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6401089f0 0x7ff640199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.458+0000 7ff63f400640 1 -- 172.21.15.39:0/2436465346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64019f970 con 0x7ff640109340 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.459+0000 7ff63f400640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff640109340 0x7ff64019f240 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff63800e970 tx=0x7ff63800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.459+0000 7ff637400640 1 -- 172.21.15.39:0/2436465346 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63800ccc0 con 0x7ff640109340 2024-07-04T08:30:59.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.459+0000 7ff637400640 1 -- 172.21.15.39:0/2436465346 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff63800ce60 con 0x7ff640109340 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.459+0000 7ff645800640 1 -- 172.21.15.39:0/2436465346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64019fc60 con 0x7ff640109340 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.459+0000 7ff637400640 1 -- 172.21.15.39:0/2436465346 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638010430 con 0x7ff640109340 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.459+0000 7ff645800640 1 -- 172.21.15.39:0/2436465346 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff640074b40 con 0x7ff640109340 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.462+0000 7ff64bdd3740 1 Processor -- start 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.463+0000 7ff637400640 1 -- 172.21.15.39:0/2436465346 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff638004ff0 con 0x7ff640109340 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.463+0000 7ff64bdd3740 1 -- start start 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6287350 0x5576a662b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a662bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a662c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5576a65558e0 con 0x5576a6287350 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5576a65a1ec0 con 0x5576a6266670 2024-07-04T08:30:59.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5576a6591460 con 0x5576a625e490 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6287350 0x5576a662b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a662bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6287350 0x5576a662b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47424/0 (socket says 172.21.15.39:47424) 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63e000640 1 -- 172.21.15.39:0/1331296221 learned_addr learned my addr 172.21.15.39:0/1331296221 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63e000640 1 --2- 172.21.15.39:0/1331296221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a662c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63ea00640 1 -- 172.21.15.39:0/1331296221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 msgr2=0x5576a662c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63ea00640 1 --2- 172.21.15.39:0/1331296221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a662c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63ea00640 1 -- 172.21.15.39:0/1331296221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6287350 msgr2=0x5576a662b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63ea00640 1 --2- 172.21.15.39:0/1331296221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6287350 0x5576a662b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63ea00640 1 -- 172.21.15.39:0/1331296221 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a6529280 con 0x5576a625e490 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.464+0000 7ff63e000640 1 --2- 172.21.15.39:0/1331296221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a662c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.465+0000 7ff637400640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff6240b29e0 0x7ff6240b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.465+0000 7ff637400640 1 -- 172.21.15.39:0/2436465346 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7ff6380e2d60 con 0x7ff640109340 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.465+0000 7ff63ea00640 1 --2- 172.21.15.39:0/1331296221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a662bb80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f80 tx=0x7ff6300093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85587192cdb1202b server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.465+0000 7ff63e000640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff6240b29e0 0x7ff6240b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.465+0000 7ff62fe00640 1 -- 172.21.15.39:0/1331296221 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63004c070 con 0x5576a625e490 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.465+0000 7ff62fe00640 1 -- 172.21.15.39:0/1331296221 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff630036040 con 0x5576a625e490 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.465+0000 7ff62fe00640 1 -- 172.21.15.39:0/1331296221 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63003c720 con 0x5576a625e490 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.466+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1331296221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 msgr2=0x5576a662bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.466+0000 7ff63e000640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff6240b29e0 0x7ff6240b4ef0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff6280058c0 tx=0x7ff628015040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.466+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/1331296221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a662bb80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f80 tx=0x7ff6300093f0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.466+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1331296221 shutdown_connections 2024-07-04T08:30:59.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.466+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/1331296221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a662c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.466+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/1331296221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a662bb80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.467+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/1331296221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6287350 0x5576a662b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.467+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1331296221 >> 172.21.15.39:0/1331296221 conn(0x5576a627b740 msgr2=0x5576a6629530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.467+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1331296221 shutdown_connections 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.467+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1331296221 wait complete. 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.467+0000 7ff64bdd3740 1 -- start start 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.467+0000 7ff64bdd3740 1 Processor -- start 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.468+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a66371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.468+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a66376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.468+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6287350 0x5576a6642f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.468+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5576a6538760 con 0x5576a625e490 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.468+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5576a65954a0 con 0x5576a6266670 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.468+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5576a63f5890 con 0x5576a6287350 2024-07-04T08:30:59.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.468+0000 7ff64bdd3740 10 client.0 _ll_register_callbacks cb 0x5576a65392b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.468+0000 7ff63e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a66376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a66371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63e000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a66376e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35284/0 (socket says 172.21.15.39:35284) 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63e000640 1 -- 172.21.15.39:0/3527944719 learned_addr learned my addr 172.21.15.39:0/3527944719 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63f400640 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6287350 0x5576a6642f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63e000640 1 -- 172.21.15.39:0/3527944719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6287350 msgr2=0x5576a6642f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63e000640 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6287350 0x5576a6642f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63e000640 1 -- 172.21.15.39:0/3527944719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 msgr2=0x5576a66371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63e000640 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a66371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63e000640 1 -- 172.21.15.39:0/3527944719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a62f2930 con 0x5576a6266670 2024-07-04T08:30:59.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff63e000640 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a66376e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff62800faa0 tx=0x7ff628005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f538d02574f32903 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.469+0000 7ff62cc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.470+0000 7ff62cc00640 1 -- 172.21.15.39:0/3527944719 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628004220 con 0x5576a6266670 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.470+0000 7ff64bdd3740 1 -- 172.21.15.39:0/3527944719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff630036040 con 0x5576a6266670 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.470+0000 7ff62cc00640 1 -- 172.21.15.39:0/3527944719 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7ff6280043c0 con 0x5576a6266670 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.470+0000 7ff62cc00640 1 -- 172.21.15.39:0/3527944719 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff628021990 con 0x5576a6266670 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.470+0000 7ff64bdd3740 10 client.28946 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.470+0000 7ff64bdd3740 1 -- 172.21.15.39:0/3527944719 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5576a6645c80 con 0x5576a6266670 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.470+0000 7ff62cc00640 1 -- 172.21.15.39:0/3527944719 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff630036040 con 0x5576a6266670 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.471+0000 7ff62cc00640 1 -- 172.21.15.39:0/3527944719 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff62802b420 con 0x5576a6266670 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.471+0000 7ff64bdd3740 10 client.28946 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.471+0000 7ff64bdd3740 10 client.28946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.471+0000 7ff64bdd3740 20 client.28946 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.471+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5576a6245cd0 0x5576a6649320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.471+0000 7ff64bdd3740 4 client.28946 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.471+0000 7ff64bdd3740 1 -- 172.21.15.39:0/3527944719 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576a65a3e10 con 0x5576a6245cd0 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.471+0000 7ff63ea00640 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5576a6245cd0 0x5576a6649320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.472+0000 7ff63ea00640 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5576a6245cd0 0x5576a6649320 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4372ffc75822104e server_cookie=5c6b36b99400652a in_seq=0 out_seq=0 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.472+0000 7ff62cc00640 10 client.28946 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff62cc00640 1 -- 172.21.15.39:0/3527944719 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5576a65a3e10 con 0x5576a6245cd0 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff62cc00640 10 client.28946 handle_command_reply: tid=0 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 client.28946 shutdown 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 -- 172.21.15.39:0/3527944719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 msgr2=0x5576a66376e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a66376e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff62800faa0 tx=0x7ff628005f90 comp rx=0 tx=0).stop 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 -- 172.21.15.39:0/3527944719 shutdown_connections 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5576a6245cd0 0x5576a6649320 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6287350 0x5576a6642f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6266670 0x5576a66376e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff62cc00640 0 client.28946 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/3527944719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a66371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 -- 172.21.15.39:0/3527944719 >> 172.21.15.39:0/3527944719 conn(0x5576a627b740 msgr2=0x5576a662e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 -- 172.21.15.39:0/3527944719 shutdown_connections 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 1 -- 172.21.15.39:0/3527944719 wait complete. 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.473+0000 7ff64bdd3740 20 client.28946 trim_cache size 0 max 16384 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.494+0000 7ff64bdd3740 1 Processor -- start 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.494+0000 7ff64bdd3740 1 -- start start 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.494+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a6629560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.494+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6266670 0x5576a6642d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.494+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6643290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.494+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401ae0a0 con 0x5576a625e490 2024-07-04T08:30:59.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.494+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401afbe0 con 0x5576a6287350 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.494+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401ae770 con 0x5576a6266670 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a6629560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63ea00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a6629560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:47434/0 (socket says 172.21.15.39:47434) 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63ea00640 1 -- 172.21.15.39:0/1188444691 learned_addr learned my addr 172.21.15.39:0/1188444691 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63e000640 1 --2- 172.21.15.39:0/1188444691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6266670 0x5576a6642d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63f400640 1 --2- 172.21.15.39:0/1188444691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6643290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63ea00640 1 -- 172.21.15.39:0/1188444691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6266670 msgr2=0x5576a6642d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63ea00640 1 --2- 172.21.15.39:0/1188444691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6266670 0x5576a6642d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63ea00640 1 -- 172.21.15.39:0/1188444691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 msgr2=0x5576a6643290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:59.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63ea00640 1 --2- 172.21.15.39:0/1188444691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6643290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63ea00640 1 -- 172.21.15.39:0/1188444691 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a6637a70 con 0x5576a625e490 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.495+0000 7ff63ea00640 1 --2- 172.21.15.39:0/1188444691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a6629560 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x5576a662a2c0 tx=0x7ff63002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20656add43009f69 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.496+0000 7ff62f400640 1 -- 172.21.15.39:0/1188444691 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63004c040 con 0x5576a625e490 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.496+0000 7ff62f400640 1 -- 172.21.15.39:0/1188444691 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff630059070 con 0x5576a625e490 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.496+0000 7ff62f400640 1 -- 172.21.15.39:0/1188444691 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63003caa0 con 0x5576a625e490 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.497+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1188444691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 msgr2=0x5576a6629560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.497+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/1188444691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a6629560 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x5576a662a2c0 tx=0x7ff63002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.497+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1188444691 shutdown_connections 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.497+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/1188444691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6643290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.497+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/1188444691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a6266670 0x5576a6642d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.497+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/1188444691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a625e490 0x5576a6629560 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.497+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1188444691 >> 172.21.15.39:0/1188444691 conn(0x5576a627b740 msgr2=0x5576a6628e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:59.871 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.497+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1188444691 shutdown_connections 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.497+0000 7ff64bdd3740 1 -- 172.21.15.39:0/1188444691 wait complete. 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.498+0000 7ff64bdd3740 1 -- start start 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.498+0000 7ff64bdd3740 1 Processor -- start 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a663f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6266670 0x5576a6633db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff64bdd3740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6634330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401af510 con 0x5576a6266670 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401aee40 con 0x5576a6287350 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff64bdd3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6401ac080 con 0x5576a625e490 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff64bdd3740 10 client.0 _ll_register_callbacks cb 0x7ff62800f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff63e000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6266670 0x5576a6633db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff63f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6634330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff63ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a663f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff63f400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6634330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:35310/0 (socket says 172.21.15.39:35310) 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.499+0000 7ff63f400640 1 -- 172.21.15.39:0/821386234 learned_addr learned my addr 172.21.15.39:0/821386234 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff63f400640 1 -- 172.21.15.39:0/821386234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 msgr2=0x5576a663f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff63f400640 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a663f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff63f400640 1 -- 172.21.15.39:0/821386234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6266670 msgr2=0x5576a6633db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff63f400640 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6266670 0x5576a6633db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff63f400640 1 -- 172.21.15.39:0/821386234 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a6634ac0 con 0x5576a6287350 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff63ea00640 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a663f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff63f400640 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6634330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff6380e0150 tx=0x7ff63800c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338daec3e76388d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff62cc00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff62cc00640 1 -- 172.21.15.39:0/821386234 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6380b2420 con 0x5576a6287350 2024-07-04T08:30:59.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff62cc00640 1 -- 172.21.15.39:0/821386234 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7ff6380c4270 con 0x5576a6287350 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff64bdd3740 1 -- 172.21.15.39:0/821386234 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff630059070 con 0x5576a6287350 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.500+0000 7ff62cc00640 1 -- 172.21.15.39:0/821386234 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6380a9790 con 0x5576a6287350 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff64bdd3740 10 client.28952 fetch_fsmap learned FSMap version 30 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff64bdd3740 1 -- 172.21.15.39:0/821386234 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5576a66b1b50 con 0x5576a6287350 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff62cc00640 1 -- 172.21.15.39:0/821386234 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff630059070 con 0x5576a6287350 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff62cc00640 1 -- 172.21.15.39:0/821386234 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff6380a9ee0 con 0x5576a6287350 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff64bdd3740 10 client.28952 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff64bdd3740 10 client.28952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff64bdd3740 20 client.28952 populate_metadata read hostname 'smithi039' 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5576a621b080 0x5576a66b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff64bdd3740 4 client.28952 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.501+0000 7ff64bdd3740 1 -- 172.21.15.39:0/821386234 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5576a659f8b0 con 0x5576a621b080 2024-07-04T08:30:59.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.502+0000 7ff63ea00640 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5576a621b080 0x5576a66b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.502+0000 7ff63ea00640 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5576a621b080 0x5576a66b4270 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eabafa848841fab6 server_cookie=45e0bc7c336b3e1e in_seq=0 out_seq=0 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.502+0000 7ff62cc00640 10 client.28952 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.503+0000 7ff62cc00640 1 -- 172.21.15.39:0/821386234 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5576a659f8b0 con 0x5576a621b080 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.503+0000 7ff62cc00640 10 client.28952 handle_command_reply: tid=0 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.503+0000 7ff64bdd3740 1 client.28952 shutdown 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 -- 172.21.15.39:0/821386234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 msgr2=0x5576a6634330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6634330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff6380e0150 tx=0x7ff63800c7d0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 -- 172.21.15.39:0/821386234 shutdown_connections 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x5576a621b080 0x5576a66b4270 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5576a6266670 0x5576a6633db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff62cc00640 0 client.28952 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:30:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a6287350 0x5576a6634330 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 --2- 172.21.15.39:0/821386234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576a625e490 0x5576a663f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 -- 172.21.15.39:0/821386234 >> 172.21.15.39:0/821386234 conn(0x5576a627b740 msgr2=0x5576a6638450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 -- 172.21.15.39:0/821386234 shutdown_connections 2024-07-04T08:30:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 1 -- 172.21.15.39:0/821386234 wait complete. 2024-07-04T08:30:59.876 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.504+0000 7ff64bdd3740 20 client.28952 trim_cache size 0 max 16384 2024-07-04T08:30:59.876 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:30:59.876 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:30:59.877 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "cd846242-8e4c-43c6-8e7b-36b984ad8bb0", 2024-07-04T08:30:59.877 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:30:59.877 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:30:59.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 -- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff6240b29e0 msgr2=0x7ff6240b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:59.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff6240b29e0 0x7ff6240b4ef0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff6280058c0 tx=0x7ff628015040 comp rx=0 tx=0).stop 2024-07-04T08:30:59.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 -- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff640109340 msgr2=0x7ff64019f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:30:59.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff640109340 0x7ff64019f240 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff63800e970 tx=0x7ff63800ed70 comp rx=0 tx=0).stop 2024-07-04T08:30:59.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 -- 172.21.15.39:0/2436465346 shutdown_connections 2024-07-04T08:30:59.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff6240b29e0 0x7ff6240b4ef0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff640109340 0x7ff64019f240 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff6401089f0 0x7ff640199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 --2- 172.21.15.39:0/2436465346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6401030b0 0x7ff640199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:30:59.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 -- 172.21.15.39:0/2436465346 >> 172.21.15.39:0/2436465346 conn(0x7ff64007ada0 msgr2=0x7ff640106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:30:59.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 -- 172.21.15.39:0/2436465346 shutdown_connections 2024-07-04T08:30:59.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:30:59.505+0000 7ff645800640 1 -- 172.21.15.39:0/2436465346 wait complete. 2024-07-04T08:31:00.638 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:31:01.221 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:01.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:01 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:31:01.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:01 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:01.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:01 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:31:01.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:01 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:01.372 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:01 smithi039 ceph-mon[32062]: pgmap v972: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-07-04T08:31:01.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:01 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:31:01.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:01 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:01.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:01 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:31:01.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:01 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:01.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:01 smithi050 ceph-mon[36577]: pgmap v972: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-07-04T08:31:01.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:01 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:31:01.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:01 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:01.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:01 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:31:01.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:01 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:01.803 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:01 smithi133 ceph-mon[37050]: pgmap v972: 129 pgs: 129 active+clean; 30 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 KiB/s wr, 11 op/s 2024-07-04T08:31:02.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:02 smithi050 ceph-mon[36577]: pgmap v973: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-07-04T08:31:02.742 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:02 smithi039 ceph-mon[32062]: pgmap v973: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-07-04T08:31:02.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:02 smithi133 ceph-mon[37050]: pgmap v973: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-07-04T08:31:03.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.755+0000 7f829a600640 1 -- 172.21.15.39:0/3979286795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294103dd0 msgr2=0x7f82941041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:03.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.755+0000 7f829a600640 1 --2- 172.21.15.39:0/3979286795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294103dd0 0x7f82941041e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f827c009860 tx=0x7f827c02a340 comp rx=0 tx=0).stop 2024-07-04T08:31:03.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.755+0000 7f829a600640 1 -- 172.21.15.39:0/3979286795 shutdown_connections 2024-07-04T08:31:03.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.755+0000 7f829a600640 1 --2- 172.21.15.39:0/3979286795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8294104720 0x7f829410cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:03.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.755+0000 7f829a600640 1 --2- 172.21.15.39:0/3979286795 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f829410d230 0x7f829410f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:03.086 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.755+0000 7f829a600640 1 --2- 172.21.15.39:0/3979286795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294103dd0 0x7f82941041e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.755+0000 7f829a600640 1 -- 172.21.15.39:0/3979286795 >> 172.21.15.39:0/3979286795 conn(0x7f82940fd8b0 msgr2=0x7f82940ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.755+0000 7f829a600640 1 -- 172.21.15.39:0/3979286795 shutdown_connections 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.755+0000 7f829a600640 1 -- 172.21.15.39:0/3979286795 wait complete. 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.757+0000 7f829a600640 1 Processor -- start 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.757+0000 7f829a600640 1 -- start start 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f829a600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8294103dd0 0x7f829419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f829a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294104720 0x7f829419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f829a600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f829410d230 0x7f82941a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f829a600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8294112eb0 con 0x7f829410d230 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f829a600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8294112d30 con 0x7f8294103dd0 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f829a600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8294113030 con 0x7f8294104720 2024-07-04T08:31:03.087 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f8299c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f829410d230 0x7f82941a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f8299200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8294103dd0 0x7f829419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f828be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294104720 0x7f829419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f8299200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8294103dd0 0x7f829419bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41706/0 (socket says 172.21.15.39:41706) 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f8299c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f829410d230 0x7f82941a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58852/0 (socket says 172.21.15.39:58852) 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f8299200640 1 -- 172.21.15.39:0/1236091951 learned_addr learned my addr 172.21.15.39:0/1236091951 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.758+0000 7f828be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294104720 0x7f829419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:46618/0 (socket says 172.21.15.39:46618) 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.759+0000 7f828be00640 1 -- 172.21.15.39:0/1236091951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8294103dd0 msgr2=0x7f829419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.759+0000 7f828be00640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8294103dd0 0x7f829419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.759+0000 7f828be00640 1 -- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f829410d230 msgr2=0x7f82941a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.759+0000 7f828be00640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f829410d230 0x7f82941a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:03.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.759+0000 7f828be00640 1 -- 172.21.15.39:0/1236091951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82941a1a50 con 0x7f8294104720 2024-07-04T08:31:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.759+0000 7f8299c00640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f829410d230 0x7f82941a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.759+0000 7f828be00640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294104720 0x7f829419c080 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f829406aa60 tx=0x7f828400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.760+0000 7f8292000640 1 -- 172.21.15.39:0/1236091951 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828400cc90 con 0x7f8294104720 2024-07-04T08:31:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.760+0000 7f8292000640 1 -- 172.21.15.39:0/1236091951 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f828400ee70 con 0x7f8294104720 2024-07-04T08:31:03.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.760+0000 7f8292000640 1 -- 172.21.15.39:0/1236091951 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8284018660 con 0x7f8294104720 2024-07-04T08:31:03.089 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:31:03.089 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.760+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82941a1d40 con 0x7f8294104720 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.760+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82941a84a0 con 0x7f8294104720 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.761+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82941041e0 con 0x7f8294104720 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.764+0000 7f8292000640 1 -- 172.21.15.39:0/1236091951 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8284010430 con 0x7f8294104720 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.765+0000 7f8292000640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82740b29e0 0x7f82740b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.766+0000 7f8292000640 1 -- 172.21.15.39:0/1236091951 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f82840e2fd0 con 0x7f8294104720 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.766+0000 7f8299200640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82740b29e0 0x7f82740b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.766+0000 7f8299200640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82740b29e0 0x7f82740b4ef0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f827c0095f0 tx=0x7f827c030340 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:02.767+0000 7f8292000640 1 -- 172.21.15.39:0/1236091951 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f82840a9fa0 con 0x7f8294104720 2024-07-04T08:31:03.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.024+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82941a8670 con 0x7f8294104720 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.025+0000 7f8292000640 1 -- 172.21.15.39:0/1236091951 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f82840ae2a0 con 0x7f8294104720 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82740b29e0 msgr2=0x7f82740b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82740b29e0 0x7f82740b4ef0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f827c0095f0 tx=0x7f827c030340 comp rx=0 tx=0).stop 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294104720 msgr2=0x7f829419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294104720 0x7f829419c080 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f829406aa60 tx=0x7f828400ec20 comp rx=0 tx=0).stop 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 shutdown_connections 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f82740b29e0 0x7f82740b4ef0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f829410d230 0x7f82941a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8294104720 0x7f829419c080 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 --2- 172.21.15.39:0/1236091951 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8294103dd0 0x7f829419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 >> 172.21.15.39:0/1236091951 conn(0x7f82940fd8b0 msgr2=0x7f82941019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:03.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.028+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 shutdown_connections 2024-07-04T08:31:03.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:03.029+0000 7f829a600640 1 -- 172.21.15.39:0/1236091951 wait complete. 2024-07-04T08:31:03.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:03 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:31:03.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:03 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1236091951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:03.693 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:03 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:31:03.694 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:03 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1236091951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:03.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:03 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:31:03.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:03 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1236091951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:04.035 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --set-id 9072c93 --release --await' 2024-07-04T08:31:04.035 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --set-id 9072c93 --release --await 2024-07-04T08:31:04.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.334+0000 7f0d78c00640 1 -- 172.21.15.39:0/2406277638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d74102f00 msgr2=0x7f0d74103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:04.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.334+0000 7f0d78c00640 1 --2- 172.21.15.39:0/2406277638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d74102f00 0x7f0d74103310 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009900 tx=0x7f0d6402a410 comp rx=0 tx=0).stop 2024-07-04T08:31:04.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.334+0000 7f0d78c00640 1 -- 172.21.15.39:0/2406277638 shutdown_connections 2024-07-04T08:31:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.334+0000 7f0d78c00640 1 --2- 172.21.15.39:0/2406277638 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d74109190 0x7f0d7410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.334+0000 7f0d78c00640 1 --2- 172.21.15.39:0/2406277638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d74108840 0x7f0d74108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.334+0000 7f0d78c00640 1 --2- 172.21.15.39:0/2406277638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d74102f00 0x7f0d74103310 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.334+0000 7f0d78c00640 1 -- 172.21.15.39:0/2406277638 >> 172.21.15.39:0/2406277638 conn(0x7f0d740fe990 msgr2=0x7f0d74100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.334+0000 7f0d78c00640 1 -- 172.21.15.39:0/2406277638 shutdown_connections 2024-07-04T08:31:04.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.334+0000 7f0d78c00640 1 -- 172.21.15.39:0/2406277638 wait complete. 2024-07-04T08:31:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.336+0000 7f0d78c00640 1 Processor -- start 2024-07-04T08:31:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.336+0000 7f0d78c00640 1 -- start start 2024-07-04T08:31:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.336+0000 7f0d78c00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d74102f00 0x7f0d7419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.323 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.336+0000 7f0d78c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d74108840 0x7f0d7419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.324 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.336+0000 7f0d78c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d74109190 0x7f0d741a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.336+0000 7f0d78c00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74112e50 con 0x7f0d74102f00 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.336+0000 7f0d78c00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74112cd0 con 0x7f0d74109190 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.336+0000 7f0d78c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d74112fd0 con 0x7f0d74108840 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d73400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d74102f00 0x7f0d7419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d73400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d74102f00 0x7f0d7419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58864/0 (socket says 172.21.15.39:58864) 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d73400640 1 -- 172.21.15.39:0/3473798967 learned_addr learned my addr 172.21.15.39:0/3473798967 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d74108840 0x7f0d7419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d73e00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d74109190 0x7f0d741a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d6aa00640 1 -- 172.21.15.39:0/3473798967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d74109190 msgr2=0x7f0d741a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d74109190 0x7f0d741a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d6aa00640 1 -- 172.21.15.39:0/3473798967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d74102f00 msgr2=0x7f0d7419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:04.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d74102f00 0x7f0d7419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.337+0000 7f0d6aa00640 1 -- 172.21.15.39:0/3473798967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d741a1ad0 con 0x7f0d74108840 2024-07-04T08:31:04.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.338+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d74108840 0x7f0d7419c100 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0d7406aee0 tx=0x7f0d5c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.338+0000 7f0d70c00640 1 -- 172.21.15.39:0/3473798967 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c00cc40 con 0x7f0d74108840 2024-07-04T08:31:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.338+0000 7f0d78c00640 1 -- 172.21.15.39:0/3473798967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d741a1dc0 con 0x7f0d74108840 2024-07-04T08:31:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.338+0000 7f0d70c00640 1 -- 172.21.15.39:0/3473798967 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d5c00cde0 con 0x7f0d74108840 2024-07-04T08:31:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.338+0000 7f0d70c00640 1 -- 172.21.15.39:0/3473798967 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c004a00 con 0x7f0d74108840 2024-07-04T08:31:04.327 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.338+0000 7f0d78c00640 1 -- 172.21.15.39:0/3473798967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d741a8520 con 0x7f0d74108840 2024-07-04T08:31:04.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.342+0000 7f0d80755740 1 Processor -- start 2024-07-04T08:31:04.329 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.342+0000 7f0d80755740 1 -- start start 2024-07-04T08:31:04.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.342+0000 7f0d80755740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8d76960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.342+0000 7f0d80755740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d76ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d80755740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d80755740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562fb8ceced0 con 0x562fb89b1390 2024-07-04T08:31:04.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d80755740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562fb8cdc470 con 0x562fb89a91b0 2024-07-04T08:31:04.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d80755740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562fb8ce04b0 con 0x562fb89d2070 2024-07-04T08:31:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d70c00640 1 -- 172.21.15.39:0/3473798967 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0d5c004ba0 con 0x7f0d74108840 2024-07-04T08:31:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d73400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d76ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d73400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d76ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41744/0 (socket says 172.21.15.39:41744) 2024-07-04T08:31:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d73400640 1 -- 172.21.15.39:0/764448996 learned_addr learned my addr 172.21.15.39:0/764448996 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d73e00640 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8d76960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d73400640 1 -- 172.21.15.39:0/764448996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 msgr2=0x562fb8d76960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d73400640 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8d76960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.343+0000 7f0d73400640 1 -- 172.21.15.39:0/764448996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 msgr2=0x562fb8d773e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.344+0000 7f0d73400640 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d773e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.344+0000 7f0d73400640 1 -- 172.21.15.39:0/764448996 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb8c74640 con 0x562fb89a91b0 2024-07-04T08:31:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.344+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.344+0000 7f0d73400640 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d76ea0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0d64000c00 tx=0x7f0d64031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d369ea0fcf6741f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.344+0000 7f0d68c00640 1 -- 172.21.15.39:0/764448996 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64040070 con 0x562fb89a91b0 2024-07-04T08:31:04.333 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.345+0000 7f0d68c00640 1 -- 172.21.15.39:0/764448996 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d64004030 con 0x562fb89a91b0 2024-07-04T08:31:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.345+0000 7f0d68c00640 1 -- 172.21.15.39:0/764448996 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64035040 con 0x562fb89a91b0 2024-07-04T08:31:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.345+0000 7f0d70c00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d580b29e0 0x7f0d580b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.346+0000 7f0d70c00640 1 -- 172.21.15.39:0/3473798967 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f0d5c0e2900 con 0x7f0d74108840 2024-07-04T08:31:04.334 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.346+0000 7f0d73e00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d580b29e0 0x7f0d580b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.346+0000 7f0d80755740 1 -- 172.21.15.39:0/764448996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 msgr2=0x562fb8d76ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.346+0000 7f0d80755740 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d76ea0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0d64000c00 tx=0x7f0d64031b10 comp rx=0 tx=0).stop 2024-07-04T08:31:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.347+0000 7f0d73e00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d580b29e0 0x7f0d580b4ef0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005fd0 tx=0x7f0d6c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.347+0000 7f0d80755740 1 -- 172.21.15.39:0/764448996 shutdown_connections 2024-07-04T08:31:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.347+0000 7f0d80755740 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.347+0000 7f0d80755740 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d76ea0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.347+0000 7f0d80755740 1 --2- 172.21.15.39:0/764448996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8d76960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.335 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.347+0000 7f0d80755740 1 -- 172.21.15.39:0/764448996 >> 172.21.15.39:0/764448996 conn(0x562fb89c6460 msgr2=0x562fb8d74850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:04.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.347+0000 7f0d80755740 1 -- 172.21.15.39:0/764448996 shutdown_connections 2024-07-04T08:31:04.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.347+0000 7f0d80755740 1 -- 172.21.15.39:0/764448996 wait complete. 2024-07-04T08:31:04.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.348+0000 7f0d80755740 1 -- start start 2024-07-04T08:31:04.336 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.348+0000 7f0d80755740 1 Processor -- start 2024-07-04T08:31:04.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d80755740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89a91b0 0x562fb8d824e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.337 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d80755740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d82a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d80755740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89d2070 0x562fb8d8e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d80755740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562fb8b40700 con 0x562fb89b1390 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d80755740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562fb8c83ba0 con 0x562fb89d2070 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d80755740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562fb8d850e0 con 0x562fb89a91b0 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d80755740 10 client.0 _ll_register_callbacks cb 0x562fb8cd5560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d6aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d82a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d73400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89a91b0 0x562fb8d824e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d6aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d82a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58886/0 (socket says 172.21.15.39:58886) 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d6aa00640 1 -- 172.21.15.39:0/1411828131 learned_addr learned my addr 172.21.15.39:0/1411828131 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.349+0000 7f0d73e00640 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89d2070 0x562fb8d8e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d73400640 1 -- 172.21.15.39:0/1411828131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89d2070 msgr2=0x562fb8d8e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:04.338 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d73400640 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89d2070 0x562fb8d8e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d73400640 1 -- 172.21.15.39:0/1411828131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 msgr2=0x562fb8d82a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:04.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d73400640 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d82a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d73400640 1 -- 172.21.15.39:0/1411828131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb8a3d650 con 0x562fb89a91b0 2024-07-04T08:31:04.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d82a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:04.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d73400640 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89a91b0 0x562fb8d824e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0d64005940 tx=0x7f0d640050c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92dd3fea2bab99d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:04.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d61600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:31:04.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d61600640 1 -- 172.21.15.39:0/1411828131 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64040070 con 0x562fb89a91b0 2024-07-04T08:31:04.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d80755740 1 -- 172.21.15.39:0/1411828131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d64004030 con 0x562fb89a91b0 2024-07-04T08:31:04.339 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.350+0000 7f0d61600640 1 -- 172.21.15.39:0/1411828131 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f0d640314a0 con 0x562fb89a91b0 2024-07-04T08:31:04.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.351+0000 7f0d61600640 1 -- 172.21.15.39:0/1411828131 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d64056050 con 0x562fb89a91b0 2024-07-04T08:31:04.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.351+0000 7f0d80755740 10 client.28900 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:04.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.351+0000 7f0d80755740 1 -- 172.21.15.39:0/1411828131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562fb8d910c0 con 0x562fb89a91b0 2024-07-04T08:31:04.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.351+0000 7f0d61600640 1 -- 172.21.15.39:0/1411828131 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d64004030 con 0x562fb89a91b0 2024-07-04T08:31:04.340 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.352+0000 7f0d61600640 1 -- 172.21.15.39:0/1411828131 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0d640035f0 con 0x562fb89a91b0 2024-07-04T08:31:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.352+0000 7f0d80755740 10 client.28900 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.352+0000 7f0d80755740 10 client.28900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:31:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.352+0000 7f0d80755740 20 client.28900 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.352+0000 7f0d80755740 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562fb89909f0 0x562fb8d94420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.352+0000 7f0d80755740 4 client.28900 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.352+0000 7f0d80755740 1 -- 172.21.15.39:0/1411828131 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fb8ceee20 con 0x562fb89909f0 2024-07-04T08:31:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.352+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562fb89909f0 0x562fb8d94420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.353+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562fb89909f0 0x562fb8d94420 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83539f31c0a4ab0f server_cookie=ed0341cfada1f9bb in_seq=0 out_seq=0 2024-07-04T08:31:04.341 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.353+0000 7f0d61600640 10 client.28900 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d61600640 1 -- 172.21.15.39:0/1411828131 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562fb8ceee20 con 0x562fb89909f0 2024-07-04T08:31:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d61600640 10 client.28900 handle_command_reply: tid=0 2024-07-04T08:31:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d80755740 1 client.28900 shutdown 2024-07-04T08:31:04.342 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d80755740 1 -- 172.21.15.39:0/1411828131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89a91b0 msgr2=0x562fb8d824e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d80755740 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89a91b0 0x562fb8d824e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0d64005940 tx=0x7f0d640050c0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d80755740 1 -- 172.21.15.39:0/1411828131 shutdown_connections 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d80755740 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562fb89909f0 0x562fb8d94420 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d80755740 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d82a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d80755740 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89d2070 0x562fb8d8e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d80755740 1 --2- 172.21.15.39:0/1411828131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89a91b0 0x562fb8d824e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.354+0000 7f0d80755740 1 -- 172.21.15.39:0/1411828131 >> 172.21.15.39:0/1411828131 conn(0x562fb89c6460 msgr2=0x562fb8d79af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.355+0000 7f0d61600640 0 client.28900 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.355+0000 7f0d80755740 1 -- 172.21.15.39:0/1411828131 shutdown_connections 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.355+0000 7f0d80755740 1 -- 172.21.15.39:0/1411828131 wait complete. 2024-07-04T08:31:04.343 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.355+0000 7f0d80755740 20 client.28900 trim_cache size 0 max 16384 2024-07-04T08:31:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.375+0000 7f0d80755740 1 Processor -- start 2024-07-04T08:31:04.362 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.375+0000 7f0d80755740 1 -- start start 2024-07-04T08:31:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d80755740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d7ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d80755740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d80755740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8d8e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d80755740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741b24b0 con 0x562fb89b1390 2024-07-04T08:31:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d80755740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741b3ff0 con 0x562fb89a91b0 2024-07-04T08:31:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d80755740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741b2b80 con 0x562fb89d2070 2024-07-04T08:31:04.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d73400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d7ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d73400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d7ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41760/0 (socket says 172.21.15.39:41760) 2024-07-04T08:31:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d73400640 1 -- 172.21.15.39:0/1066457635 learned_addr learned my addr 172.21.15.39:0/1066457635 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/1066457635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.376+0000 7f0d73e00640 1 --2- 172.21.15.39:0/1066457635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8d8e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.377+0000 7f0d73400640 1 -- 172.21.15.39:0/1066457635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 msgr2=0x562fb8d8e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.377+0000 7f0d73400640 1 --2- 172.21.15.39:0/1066457635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8d8e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.377+0000 7f0d73400640 1 -- 172.21.15.39:0/1066457635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 msgr2=0x562fb8d780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.377+0000 7f0d73400640 1 --2- 172.21.15.39:0/1066457635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d780a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.377+0000 7f0d73400640 1 -- 172.21.15.39:0/1066457635 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb8d737e0 con 0x562fb89a91b0 2024-07-04T08:31:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.377+0000 7f0d73400640 1 --2- 172.21.15.39:0/1066457635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d7ca80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0d64000c00 tx=0x7f0d640058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0486d1b64c812ff server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.377+0000 7f0d63e00640 1 -- 172.21.15.39:0/1066457635 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6405f070 con 0x562fb89a91b0 2024-07-04T08:31:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.378+0000 7f0d63e00640 1 -- 172.21.15.39:0/1066457635 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6404d070 con 0x562fb89a91b0 2024-07-04T08:31:04.366 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.378+0000 7f0d63e00640 1 -- 172.21.15.39:0/1066457635 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64040040 con 0x562fb89a91b0 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.378+0000 7f0d80755740 1 -- 172.21.15.39:0/1066457635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 msgr2=0x562fb8d7ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.378+0000 7f0d80755740 1 --2- 172.21.15.39:0/1066457635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d7ca80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0d64000c00 tx=0x7f0d640058d0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.379+0000 7f0d80755740 1 -- 172.21.15.39:0/1066457635 shutdown_connections 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.379+0000 7f0d80755740 1 --2- 172.21.15.39:0/1066457635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.379+0000 7f0d80755740 1 --2- 172.21.15.39:0/1066457635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8d8e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.379+0000 7f0d80755740 1 --2- 172.21.15.39:0/1066457635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d7ca80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.379+0000 7f0d80755740 1 -- 172.21.15.39:0/1066457635 >> 172.21.15.39:0/1066457635 conn(0x562fb89c6460 msgr2=0x562fb8d77990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.379+0000 7f0d80755740 1 -- 172.21.15.39:0/1066457635 shutdown_connections 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.379+0000 7f0d80755740 1 -- 172.21.15.39:0/1066457635 wait complete. 2024-07-04T08:31:04.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.379+0000 7f0d80755740 1 -- start start 2024-07-04T08:31:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.380+0000 7f0d80755740 1 Processor -- start 2024-07-04T08:31:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.380+0000 7f0d80755740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d94b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.380+0000 7f0d80755740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d950f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.380+0000 7f0d80755740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8dfa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.380+0000 7f0d80755740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741b3920 con 0x562fb89b1390 2024-07-04T08:31:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.380+0000 7f0d80755740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741b3250 con 0x562fb89a91b0 2024-07-04T08:31:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.380+0000 7f0d80755740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d741b0490 con 0x562fb89d2070 2024-07-04T08:31:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.380+0000 7f0d80755740 10 client.0 _ll_register_callbacks cb 0x7f0d6c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:04.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.380+0000 7f0d73400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d94b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d73400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d94b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41764/0 (socket says 172.21.15.39:41764) 2024-07-04T08:31:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d73400640 1 -- 172.21.15.39:0/4188733510 learned_addr learned my addr 172.21.15.39:0/4188733510 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d73e00640 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8dfa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d73400640 1 -- 172.21.15.39:0/4188733510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 msgr2=0x562fb8dfa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d73400640 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8dfa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d73400640 1 -- 172.21.15.39:0/4188733510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 msgr2=0x562fb8d950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:04.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d73400640 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d950f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d73400640 1 -- 172.21.15.39:0/4188733510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb8dfafa0 con 0x562fb89a91b0 2024-07-04T08:31:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d73400640 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d94b50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009a20 tx=0x7f0d64005200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f361c8fae4205557 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.381+0000 7f0d61600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.382+0000 7f0d61600640 1 -- 172.21.15.39:0/4188733510 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d64069060 con 0x562fb89a91b0 2024-07-04T08:31:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.382+0000 7f0d80755740 1 -- 172.21.15.39:0/4188733510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6404d070 con 0x562fb89a91b0 2024-07-04T08:31:04.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.382+0000 7f0d61600640 1 -- 172.21.15.39:0/4188733510 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9424+0+0 (secure 0 0 0) 0x7f0d64003b60 con 0x562fb89a91b0 2024-07-04T08:31:04.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.382+0000 7f0d61600640 1 -- 172.21.15.39:0/4188733510 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0d64070050 con 0x562fb89a91b0 2024-07-04T08:31:04.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.382+0000 7f0d80755740 10 client.28976 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:04.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.382+0000 7f0d80755740 1 -- 172.21.15.39:0/4188733510 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562fb8dfc230 con 0x562fb89a91b0 2024-07-04T08:31:04.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.383+0000 7f0d61600640 1 -- 172.21.15.39:0/4188733510 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6404d070 con 0x562fb89a91b0 2024-07-04T08:31:04.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.383+0000 7f0d61600640 1 -- 172.21.15.39:0/4188733510 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0d64004ae0 con 0x562fb89a91b0 2024-07-04T08:31:04.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.383+0000 7f0d80755740 10 client.28976 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:04.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.383+0000 7f0d80755740 10 client.28976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:31:04.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.383+0000 7f0d80755740 20 client.28976 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.383+0000 7f0d80755740 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562fb8965da0 0x562fb8dfe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.383+0000 7f0d80755740 4 client.28976 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.383+0000 7f0d80755740 1 -- 172.21.15.39:0/4188733510 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562fb8cea8c0 con 0x562fb8965da0 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.384+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562fb8965da0 0x562fb8dfe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.384+0000 7f0d6aa00640 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562fb8965da0 0x562fb8dfe930 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e6d69d568d598a server_cookie=7a5775a09e7b2947 in_seq=0 out_seq=0 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.384+0000 7f0d61600640 10 client.28976 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d61600640 1 -- 172.21.15.39:0/4188733510 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562fb8cea8c0 con 0x562fb8965da0 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d61600640 10 client.28976 handle_command_reply: tid=0 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 client.28976 shutdown 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 -- 172.21.15.39:0/4188733510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 msgr2=0x562fb8d94b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d94b50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0d64009a20 tx=0x7f0d64005200 comp rx=0 tx=0).stop 2024-07-04T08:31:04.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 -- 172.21.15.39:0/4188733510 shutdown_connections 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562fb8965da0 0x562fb8dfe930 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562fb89b1390 0x562fb8d950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562fb89d2070 0x562fb8dfa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 --2- 172.21.15.39:0/4188733510 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562fb89a91b0 0x562fb8d94b50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 -- 172.21.15.39:0/4188733510 >> 172.21.15.39:0/4188733510 conn(0x562fb89c6460 msgr2=0x562fb8d835a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d61600640 0 client.28976 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 -- 172.21.15.39:0/4188733510 shutdown_connections 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.385+0000 7f0d80755740 1 -- 172.21.15.39:0/4188733510 wait complete. 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.386+0000 7f0d80755740 20 client.28976 trim_cache size 0 max 16384 2024-07-04T08:31:04.375 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:31:04.376 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:31:04.376 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:31:04.376 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:31:04.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.386+0000 7f0d78c00640 1 -- 172.21.15.39:0/3473798967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d580b29e0 msgr2=0x7f0d580b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:04.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.386+0000 7f0d78c00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d580b29e0 0x7f0d580b4ef0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c005fd0 tx=0x7f0d6c009500 comp rx=0 tx=0).stop 2024-07-04T08:31:04.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.386+0000 7f0d78c00640 1 -- 172.21.15.39:0/3473798967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d74108840 msgr2=0x7f0d7419c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:04.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.386+0000 7f0d78c00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d74108840 0x7f0d7419c100 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0d7406aee0 tx=0x7f0d5c00ee50 comp rx=0 tx=0).stop 2024-07-04T08:31:04.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.387+0000 7f0d78c00640 1 -- 172.21.15.39:0/3473798967 shutdown_connections 2024-07-04T08:31:04.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.387+0000 7f0d78c00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0d580b29e0 0x7f0d580b4ef0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.387+0000 7f0d78c00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d74109190 0x7f0d741a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.387+0000 7f0d78c00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d74108840 0x7f0d7419c100 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.387+0000 7f0d78c00640 1 --2- 172.21.15.39:0/3473798967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d74102f00 0x7f0d7419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:04.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.387+0000 7f0d78c00640 1 -- 172.21.15.39:0/3473798967 >> 172.21.15.39:0/3473798967 conn(0x7f0d740fe990 msgr2=0x7f0d74107f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:04.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.387+0000 7f0d78c00640 1 -- 172.21.15.39:0/3473798967 shutdown_connections 2024-07-04T08:31:04.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:04.387+0000 7f0d78c00640 1 -- 172.21.15.39:0/3473798967 wait complete. 2024-07-04T08:31:04.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:04 smithi039 ceph-mon[32062]: pgmap v974: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-07-04T08:31:04.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:04 smithi133 ceph-mon[37050]: pgmap v974: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-07-04T08:31:04.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:04 smithi050 ceph-mon[36577]: pgmap v974: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-07-04T08:31:05.396 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:05.431 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:31:05.432 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:31:06.840 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:07.515 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:07 smithi039 ceph-mon[32062]: pgmap v975: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-07-04T08:31:07.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:07 smithi133 ceph-mon[37050]: pgmap v975: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-07-04T08:31:07.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:07 smithi050 ceph-mon[36577]: pgmap v975: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-07-04T08:31:08.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.936+0000 7fe9bf200640 1 -- 172.21.15.39:0/872278412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0109190 msgr2=0x7fe9c010f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:08.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.936+0000 7fe9bf200640 1 --2- 172.21.15.39:0/872278412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0109190 0x7fe9c010f9c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009ca0 tx=0x7fe9b802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.936+0000 7fe9bf200640 1 -- 172.21.15.39:0/872278412 shutdown_connections 2024-07-04T08:31:08.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.936+0000 7fe9bf200640 1 --2- 172.21.15.39:0/872278412 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0109190 0x7fe9c010f9c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.936+0000 7fe9bf200640 1 --2- 172.21.15.39:0/872278412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c0108840 0x7fe9c0108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.936+0000 7fe9bf200640 1 --2- 172.21.15.39:0/872278412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c0102f00 0x7fe9c0103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.936+0000 7fe9bf200640 1 -- 172.21.15.39:0/872278412 >> 172.21.15.39:0/872278412 conn(0x7fe9c00fe950 msgr2=0x7fe9c0100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:08.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.936+0000 7fe9bf200640 1 -- 172.21.15.39:0/872278412 shutdown_connections 2024-07-04T08:31:08.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.936+0000 7fe9bf200640 1 -- 172.21.15.39:0/872278412 wait complete. 2024-07-04T08:31:08.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.938+0000 7fe9bf200640 1 Processor -- start 2024-07-04T08:31:08.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.938+0000 7fe9bf200640 1 -- start start 2024-07-04T08:31:08.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.938+0000 7fe9bf200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0102f00 0x7fe9c019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.938+0000 7fe9bf200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c0108840 0x7fe9c019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9bf200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c0109190 0x7fe9c01a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9bf200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0112e50 con 0x7fe9c0102f00 2024-07-04T08:31:08.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9bf200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0112cd0 con 0x7fe9c0108840 2024-07-04T08:31:08.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9bf200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c0112fd0 con 0x7fe9c0109190 2024-07-04T08:31:08.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0102f00 0x7fe9c019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c0108840 0x7fe9c019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c0108840 0x7fe9c019bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41784/0 (socket says 172.21.15.39:41784) 2024-07-04T08:31:08.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9bd400640 1 -- 172.21.15.39:0/2119520662 learned_addr learned my addr 172.21.15.39:0/2119520662 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:08.927 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0102f00 0x7fe9c019b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58932/0 (socket says 172.21.15.39:58932) 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.939+0000 7fe9be800640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c0109190 0x7fe9c01a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9bd400640 1 -- 172.21.15.39:0/2119520662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c0109190 msgr2=0x7fe9c01a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9bd400640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c0109190 0x7fe9c01a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9bd400640 1 -- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0102f00 msgr2=0x7fe9c019b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9bd400640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0102f00 0x7fe9c019b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9bd400640 1 -- 172.21.15.39:0/2119520662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c01a1930 con 0x7fe9c0108840 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9bde00640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0102f00 0x7fe9c019b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9bd400640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c0108840 0x7fe9c019bf60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e9c0 tx=0x7fe9a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9b6a00640 1 -- 172.21.15.39:0/2119520662 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a800ccc0 con 0x7fe9c0108840 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9bf200640 1 -- 172.21.15.39:0/2119520662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c01a1c20 con 0x7fe9c0108840 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9b6a00640 1 -- 172.21.15.39:0/2119520662 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a80054d0 con 0x7fe9c0108840 2024-07-04T08:31:08.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9b6a00640 1 -- 172.21.15.39:0/2119520662 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8010430 con 0x7fe9c0108840 2024-07-04T08:31:08.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.940+0000 7fe9bf200640 1 -- 172.21.15.39:0/2119520662 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c01a8380 con 0x7fe9c0108840 2024-07-04T08:31:08.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.944+0000 7fe9cb76c740 1 Processor -- start 2024-07-04T08:31:08.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.944+0000 7fe9cb76c740 1 -- start start 2024-07-04T08:31:08.931 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.944+0000 7fe9b6a00640 1 -- 172.21.15.39:0/2119520662 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fe9a8005020 con 0x7fe9c0108840 2024-07-04T08:31:08.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca0660 0x55f1cd06e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccf88b00 con 0x55f1cccc9520 2024-07-04T08:31:08.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccfe4290 con 0x55f1ccca8840 2024-07-04T08:31:08.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccfd3830 con 0x55f1ccca0660 2024-07-04T08:31:08.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9be800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9be800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd06e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58948/0 (socket says 172.21.15.39:58948) 2024-07-04T08:31:08.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca0660 0x55f1cd06e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9be800640 1 -- 172.21.15.39:0/4135041758 learned_addr learned my addr 172.21.15.39:0/4135041758 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:08.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.945+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.946+0000 7fe9bde00640 1 -- 172.21.15.39:0/4135041758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 msgr2=0x55f1cd06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:08.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.946+0000 7fe9bde00640 1 --2- 172.21.15.39:0/4135041758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.946+0000 7fe9bde00640 1 -- 172.21.15.39:0/4135041758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 msgr2=0x55f1cd06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:08.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.946+0000 7fe9bde00640 1 --2- 172.21.15.39:0/4135041758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd06e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.946+0000 7fe9bde00640 1 -- 172.21.15.39:0/4135041758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ccf8d760 con 0x55f1ccca0660 2024-07-04T08:31:08.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.946+0000 7fe9b6a00640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9a40b2960 0x7fe9a40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.946+0000 7fe9bde00640 1 --2- 172.21.15.39:0/4135041758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca0660 0x55f1cd06e9b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009fd0 tx=0x7fe9b00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bddc2d2c8cd0ab3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:08.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.947+0000 7fe9b6a00640 1 -- 172.21.15.39:0/2119520662 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7fe9a80e2e00 con 0x7fe9c0108840 2024-07-04T08:31:08.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.947+0000 7fe9af400640 1 -- 172.21.15.39:0/4135041758 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0004030 con 0x55f1ccca0660 2024-07-04T08:31:08.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.947+0000 7fe9af400640 1 -- 172.21.15.39:0/4135041758 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b00054d0 con 0x55f1ccca0660 2024-07-04T08:31:08.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.947+0000 7fe9be800640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9a40b2960 0x7fe9a40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.937 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.947+0000 7fe9af400640 1 -- 172.21.15.39:0/4135041758 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0004ce0 con 0x55f1ccca0660 2024-07-04T08:31:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4135041758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca0660 msgr2=0x55f1cd06e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9be800640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9a40b2960 0x7fe9a40b4e70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8005e30 tx=0x7fe9b8005c30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4135041758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca0660 0x55f1cd06e9b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009fd0 tx=0x7fe9b00118d0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4135041758 shutdown_connections 2024-07-04T08:31:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4135041758 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4135041758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca0660 0x55f1cd06e9b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4135041758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4135041758 >> 172.21.15.39:0/4135041758 conn(0x55f1cccbd910 msgr2=0x55f1cd06c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4135041758 shutdown_connections 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.948+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4135041758 wait complete. 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.949+0000 7fe9cb76c740 1 -- start start 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.949+0000 7fe9cb76c740 1 Processor -- start 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.949+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd079b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.949+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca8840 0x55f1cd07a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd085e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccf7ad00 con 0x55f1ccca8840 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccfd7870 con 0x55f1ccca0660 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f1cce37bb0 con 0x55f1cccc9520 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9cb76c740 10 client.0 _ll_register_callbacks cb 0x55f1ccf7b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:08.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd079b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd079b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41792/0 (socket says 172.21.15.39:41792) 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9bde00640 1 -- 172.21.15.39:0/4533393 learned_addr learned my addr 172.21.15.39:0/4533393 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9bd400640 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca8840 0x55f1cd07a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9be800640 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd085e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9bde00640 1 -- 172.21.15.39:0/4533393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 msgr2=0x55f1cd085e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9bde00640 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd085e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9bde00640 1 -- 172.21.15.39:0/4533393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca8840 msgr2=0x55f1cd07a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9bde00640 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca8840 0x55f1cd07a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9bde00640 1 -- 172.21.15.39:0/4533393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ccf6a640 con 0x55f1ccca0660 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.950+0000 7fe9be800640 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd085e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.951+0000 7fe9bde00640 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd079b70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009350 tx=0x7fe9b000fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a28e0a6fac7c85a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.951+0000 7fe9a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:08.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.951+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4533393 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0029020 con 0x55f1ccca0660 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.951+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4533393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b00054d0 con 0x55f1ccca0660 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.951+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4533393 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7fe9b00049e0 con 0x55f1ccca0660 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.951+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4533393 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9b0034050 con 0x55f1ccca0660 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.951+0000 7fe9cb76c740 10 client.28988 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.951+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4533393 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f1ccd34b00 con 0x55f1ccca0660 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.952+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4533393 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b00054d0 con 0x55f1ccca0660 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.952+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4533393 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe9b0025470 con 0x55f1ccca0660 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.952+0000 7fe9cb76c740 10 client.28988 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.952+0000 7fe9cb76c740 10 client.28988 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.952+0000 7fe9cb76c740 20 client.28988 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.952+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f1ccc87ea0 0x55f1cd08bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.952+0000 7fe9cb76c740 4 client.28988 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:08.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.952+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4533393 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1ccfe61e0 con 0x55f1ccc87ea0 2024-07-04T08:31:08.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.953+0000 7fe9bd400640 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f1ccc87ea0 0x55f1cd08bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.953+0000 7fe9bd400640 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f1ccc87ea0 0x55f1cd08bac0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e1293981301741 server_cookie=bb7e380c492d66a0 in_seq=0 out_seq=0 2024-07-04T08:31:08.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.953+0000 7fe9a3e00640 10 client.28988 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4533393 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f1ccfe61e0 con 0x55f1ccc87ea0 2024-07-04T08:31:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9a3e00640 10 client.28988 handle_command_reply: tid=0 2024-07-04T08:31:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9cb76c740 1 client.28988 shutdown 2024-07-04T08:31:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4533393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 msgr2=0x55f1cd079b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd079b70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009350 tx=0x7fe9b000fc00 comp rx=0 tx=0).stop 2024-07-04T08:31:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4533393 shutdown_connections 2024-07-04T08:31:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f1ccc87ea0 0x55f1cd08bac0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca8840 0x55f1cd07a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd085e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4533393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd079b70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9a3e00640 0 client.28988 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:08.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.954+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4533393 >> 172.21.15.39:0/4533393 conn(0x55f1cccbd910 msgr2=0x55f1cd071650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:08.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.955+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4533393 shutdown_connections 2024-07-04T08:31:08.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.955+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4533393 wait complete. 2024-07-04T08:31:08.946 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.955+0000 7fe9cb76c740 20 client.28988 trim_cache size 0 max 16384 2024-07-04T08:31:08.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.976+0000 7fe9cb76c740 1 Processor -- start 2024-07-04T08:31:08.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.976+0000 7fe9cb76c740 1 -- start start 2024-07-04T08:31:08.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca0660 0x55f1cd06c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd073580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.964 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01b2460 con 0x55f1ccca0660 2024-07-04T08:31:08.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01b3fa0 con 0x55f1ccca8840 2024-07-04T08:31:08.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01b2b30 con 0x55f1cccc9520 2024-07-04T08:31:08.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca0660 0x55f1cd06c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9bd400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41804/0 (socket says 172.21.15.39:41804) 2024-07-04T08:31:08.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9bd400640 1 -- 172.21.15.39:0/215553886 learned_addr learned my addr 172.21.15.39:0/215553886 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:08.965 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.977+0000 7fe9be800640 1 --2- 172.21.15.39:0/215553886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.978+0000 7fe9bd400640 1 -- 172.21.15.39:0/215553886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 msgr2=0x55f1cd073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:08.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.978+0000 7fe9bd400640 1 --2- 172.21.15.39:0/215553886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.978+0000 7fe9bd400640 1 -- 172.21.15.39:0/215553886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca0660 msgr2=0x55f1cd06c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:08.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.978+0000 7fe9bd400640 1 --2- 172.21.15.39:0/215553886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca0660 0x55f1cd06c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.978+0000 7fe9bd400640 1 -- 172.21.15.39:0/215553886 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ccdab960 con 0x55f1ccca8840 2024-07-04T08:31:08.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.978+0000 7fe9be800640 1 --2- 172.21.15.39:0/215553886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:31:08.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.978+0000 7fe9bd400640 1 --2- 172.21.15.39:0/215553886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06b2d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8004650 tx=0x7fe9a80e5250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae04d23444f81eb8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:08.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.978+0000 7fe9aea00640 1 -- 172.21.15.39:0/215553886 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80a95f0 con 0x55f1ccca8840 2024-07-04T08:31:08.966 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.979+0000 7fe9aea00640 1 -- 172.21.15.39:0/215553886 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a8005310 con 0x55f1ccca8840 2024-07-04T08:31:08.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.979+0000 7fe9aea00640 1 -- 172.21.15.39:0/215553886 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80a5430 con 0x55f1ccca8840 2024-07-04T08:31:08.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 -- 172.21.15.39:0/215553886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 msgr2=0x55f1cd06b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:08.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/215553886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06b2d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8004650 tx=0x7fe9a80e5250 comp rx=0 tx=0).stop 2024-07-04T08:31:08.967 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 -- 172.21.15.39:0/215553886 shutdown_connections 2024-07-04T08:31:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/215553886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1cccc9520 0x55f1cd073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/215553886 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca8840 0x55f1cd06b2d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/215553886 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1ccca0660 0x55f1cd06c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 -- 172.21.15.39:0/215553886 >> 172.21.15.39:0/215553886 conn(0x55f1cccbd910 msgr2=0x55f1cd06f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 -- 172.21.15.39:0/215553886 shutdown_connections 2024-07-04T08:31:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 -- 172.21.15.39:0/215553886 wait complete. 2024-07-04T08:31:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.980+0000 7fe9cb76c740 1 -- start start 2024-07-04T08:31:08.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.981+0000 7fe9cb76c740 1 Processor -- start 2024-07-04T08:31:08.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.981+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd082500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.981+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca8840 0x55f1cd07d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.981+0000 7fe9cb76c740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd07df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.981+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01b38d0 con 0x55f1cccc9520 2024-07-04T08:31:08.969 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.981+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01b3200 con 0x55f1ccca0660 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.981+0000 7fe9cb76c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c01b0460 con 0x55f1ccca8840 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.981+0000 7fe9cb76c740 10 client.0 _ll_register_callbacks cb 0x7fe9b0009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd082500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9be800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd07df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9be800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd07df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:58972/0 (socket says 172.21.15.39:58972) 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9be800640 1 -- 172.21.15.39:0/4182846165 learned_addr learned my addr 172.21.15.39:0/4182846165 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9bde00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd082500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:41814/0 (socket says 172.21.15.39:41814) 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9bd400640 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca8840 0x55f1cd07d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9bde00640 1 -- 172.21.15.39:0/4182846165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca8840 msgr2=0x55f1cd07d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9bde00640 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca8840 0x55f1cd07d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9bde00640 1 -- 172.21.15.39:0/4182846165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 msgr2=0x55f1cd07df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:08.970 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9bde00640 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd07df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9bde00640 1 -- 172.21.15.39:0/4182846165 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ccfa0880 con 0x55f1ccca0660 2024-07-04T08:31:08.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9bd400640 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca8840 0x55f1cd07d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:08.971 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.982+0000 7fe9be800640 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd07df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.983+0000 7fe9bde00640 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd082500 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00048d0 tx=0x7fe9b000fd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52f3332f6d7f75ea server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.983+0000 7fe9a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.983+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4182846165 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0029040 con 0x55f1ccca0660 2024-07-04T08:31:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.983+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4182846165 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7fe9b0025c80 con 0x55f1ccca0660 2024-07-04T08:31:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.983+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4182846165 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f1ccfe1c80 con 0x55f1ccca0660 2024-07-04T08:31:08.972 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.983+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4182846165 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9b0040050 con 0x55f1ccca0660 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9cb76c740 10 client.28994 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4182846165 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f1ccf9c430 con 0x55f1ccca0660 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4182846165 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f1ccfe1c80 con 0x55f1ccca0660 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4182846165 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe9b00037f0 con 0x55f1ccca0660 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9cb76c740 10 client.28994 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9cb76c740 10 client.28994 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9cb76c740 20 client.28994 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f1ccc5d250 0x55f1cd0f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9cb76c740 4 client.28994 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9072c93", "release": true, "await": true}] 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.984+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4182846165 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9072c93", "release": true, "await": true}) -- 0x55f1ccfdf3c0 con 0x55f1ccc5d250 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.985+0000 7fe9bd400640 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f1ccc5d250 0x55f1cd0f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.985+0000 7fe9bd400640 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f1ccc5d250 0x55f1cd0f64d0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95bba2fdcca77c8e server_cookie=4d1fd1ab137e7223 in_seq=0 out_seq=0 2024-07-04T08:31:08.973 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.985+0000 7fe9a3e00640 10 client.28994 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:08.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9a3e00640 1 -- 172.21.15.39:0/4182846165 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f1ccfdf3c0 con 0x55f1ccc5d250 2024-07-04T08:31:08.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9a3e00640 10 client.28994 handle_command_reply: tid=0 2024-07-04T08:31:08.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 client.28994 shutdown 2024-07-04T08:31:08.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4182846165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 msgr2=0x55f1cd082500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:08.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd082500 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00048d0 tx=0x7fe9b000fd60 comp rx=0 tx=0).stop 2024-07-04T08:31:08.975 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4182846165 shutdown_connections 2024-07-04T08:31:08.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x55f1ccc5d250 0x55f1cd0f64d0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55f1cccc9520 0x55f1cd07df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f1ccca8840 0x55f1cd07d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 --2- 172.21.15.39:0/4182846165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f1ccca0660 0x55f1cd082500 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9a3e00640 0 client.28994 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:08.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4182846165 >> 172.21.15.39:0/4182846165 conn(0x55f1cccbd910 msgr2=0x55f1cd06fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:08.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4182846165 shutdown_connections 2024-07-04T08:31:08.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 1 -- 172.21.15.39:0/4182846165 wait complete. 2024-07-04T08:31:08.976 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.988+0000 7fe9cb76c740 20 client.28994 trim_cache size 0 max 16384 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 48, 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": { 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "9072c93": { 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "version": 48, 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "age_ref": 0.0, 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:31:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "timeout": 90.0, 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "expiration": 141.7, 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "members": { 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "file:/": { 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "excluded": false, 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "state": { 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RELEASED", 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:31:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:31:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:31:08.979 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:31:08.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 -- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9a40b2960 msgr2=0x7fe9a40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:08.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9a40b2960 0x7fe9a40b4e70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8005e30 tx=0x7fe9b8005c30 comp rx=0 tx=0).stop 2024-07-04T08:31:08.979 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 -- 172.21.15.39:0/2119520662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c0108840 msgr2=0x7fe9c019bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:08.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c0108840 0x7fe9c019bf60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e9c0 tx=0x7fe9a800ee80 comp rx=0 tx=0).stop 2024-07-04T08:31:08.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 -- 172.21.15.39:0/2119520662 shutdown_connections 2024-07-04T08:31:08.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fe9a40b2960 0x7fe9a40b4e70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9c0109190 0x7fe9c01a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe9c0108840 0x7fe9c019bf60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 --2- 172.21.15.39:0/2119520662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c0102f00 0x7fe9c019b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:08.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 -- 172.21.15.39:0/2119520662 >> 172.21.15.39:0/2119520662 conn(0x7fe9c00fe950 msgr2=0x7fe9c0107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:08.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 -- 172.21.15.39:0/2119520662 shutdown_connections 2024-07-04T08:31:08.980 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:08.989+0000 7fe9bf200640 1 -- 172.21.15.39:0/2119520662 wait complete. 2024-07-04T08:31:09.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:09 smithi039 ceph-mon[32062]: pgmap v976: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-07-04T08:31:09.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:09 smithi039 ceph-mon[32062]: osdmap e111: 12 total, 12 up, 12 in 2024-07-04T08:31:09.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:09 smithi050 ceph-mon[36577]: pgmap v976: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-07-04T08:31:09.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:09 smithi050 ceph-mon[36577]: osdmap e111: 12 total, 12 up, 12 in 2024-07-04T08:31:09.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.813+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1367630771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ff0101fb0 msgr2=0x7f9ff01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:09.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.813+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1367630771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ff0101fb0 0x7f9ff01023c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009900 tx=0x7f9fe002a400 comp rx=0 tx=0).stop 2024-07-04T08:31:09.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.813+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1367630771 shutdown_connections 2024-07-04T08:31:09.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.813+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1367630771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ff0108240 0x7f9ff010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:09.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.813+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1367630771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff01078f0 0x7f9ff0107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:09.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.813+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1367630771 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ff0101fb0 0x7f9ff01023c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:09.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.813+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1367630771 >> 172.21.15.39:0/1367630771 conn(0x7f9ff0078c80 msgr2=0x7f9ff0079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:09.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.813+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1367630771 shutdown_connections 2024-07-04T08:31:09.801 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.813+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1367630771 wait complete. 2024-07-04T08:31:09.802 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.815+0000 7f9ff5e00640 1 Processor -- start 2024-07-04T08:31:09.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:09 smithi133 ceph-mon[37050]: pgmap v976: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-07-04T08:31:09.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:09 smithi133 ceph-mon[37050]: osdmap e111: 12 total, 12 up, 12 in 2024-07-04T08:31:09.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.816+0000 7f9ff5e00640 1 -- start start 2024-07-04T08:31:09.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.816+0000 7f9ff5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff0101fb0 0x7f9ff0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:09.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.816+0000 7f9ff5e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ff01078f0 0x7f9ff0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:09.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.816+0000 7f9ff5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ff0108240 0x7f9ff019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:09.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.816+0000 7f9ff5e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0111f00 con 0x7f9ff01078f0 2024-07-04T08:31:09.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.816+0000 7f9ff5e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0111d80 con 0x7f9ff0108240 2024-07-04T08:31:09.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.816+0000 7f9ff5e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0112080 con 0x7f9ff0101fb0 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9feea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff0101fb0 0x7f9ff0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9fef400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ff0108240 0x7f9ff019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9fee000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ff01078f0 0x7f9ff0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9feea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff0101fb0 0x7f9ff0199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:46736/0 (socket says 172.21.15.39:46736) 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9feea00640 1 -- 172.21.15.39:0/1213003582 learned_addr learned my addr 172.21.15.39:0/1213003582 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9fef400640 1 -- 172.21.15.39:0/1213003582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff0101fb0 msgr2=0x7f9ff0199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9fef400640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff0101fb0 0x7f9ff0199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9fef400640 1 -- 172.21.15.39:0/1213003582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ff01078f0 msgr2=0x7f9ff0199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9fef400640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ff01078f0 0x7f9ff0199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9fef400640 1 -- 172.21.15.39:0/1213003582 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff019f960 con 0x7f9ff0108240 2024-07-04T08:31:09.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.817+0000 7f9feea00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff0101fb0 0x7f9ff0199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:09.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.818+0000 7f9fef400640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ff0108240 0x7f9ff019f230 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009fd0 tx=0x7f9fe800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:09.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.818+0000 7f9ff4a00640 1 -- 172.21.15.39:0/1213003582 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800ce20 con 0x7f9ff0108240 2024-07-04T08:31:09.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.818+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff019fc50 con 0x7f9ff0108240 2024-07-04T08:31:09.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.818+0000 7f9ff4a00640 1 -- 172.21.15.39:0/1213003582 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe8005020 con 0x7f9ff0108240 2024-07-04T08:31:09.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.818+0000 7f9ff4a00640 1 -- 172.21.15.39:0/1213003582 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8010690 con 0x7f9ff0108240 2024-07-04T08:31:09.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.818+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff01a6400 con 0x7f9ff0108240 2024-07-04T08:31:09.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.820+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff01023c0 con 0x7f9ff0108240 2024-07-04T08:31:09.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.822+0000 7f9ff4a00640 1 -- 172.21.15.39:0/1213003582 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9fe80054d0 con 0x7f9ff0108240 2024-07-04T08:31:09.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.824+0000 7f9ff4a00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9fd40b2960 0x7f9fd40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:09.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.824+0000 7f9ff4a00640 1 -- 172.21.15.39:0/1213003582 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7f9fe80e3090 con 0x7f9ff0108240 2024-07-04T08:31:09.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.824+0000 7f9feea00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9fd40b2960 0x7f9fd40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:09.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.825+0000 7f9feea00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9fd40b2960 0x7f9fd40b4e70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00096c0 tx=0x7f9fe0005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:09.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:09.825+0000 7f9ff4a00640 1 -- 172.21.15.39:0/1213003582 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f9fe80aa630 con 0x7f9ff0108240 2024-07-04T08:31:10.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.080+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ff019ab70 con 0x7f9ff0108240 2024-07-04T08:31:10.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.081+0000 7f9ff4a00640 1 -- 172.21.15.39:0/1213003582 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f9fe80a9f30 con 0x7f9ff0108240 2024-07-04T08:31:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:31:10.069 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:31:10.070 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:31:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9fd40b2960 msgr2=0x7f9fd40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9fd40b2960 0x7f9fd40b4e70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00096c0 tx=0x7f9fe0005a20 comp rx=0 tx=0).stop 2024-07-04T08:31:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ff0108240 msgr2=0x7f9ff019f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:10.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ff0108240 0x7f9ff019f230 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009fd0 tx=0x7f9fe800ef10 comp rx=0 tx=0).stop 2024-07-04T08:31:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 shutdown_connections 2024-07-04T08:31:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f9fd40b2960 0x7f9fd40b4e70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ff01078f0 0x7f9ff0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ff0108240 0x7f9ff019f230 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 --2- 172.21.15.39:0/1213003582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ff0101fb0 0x7f9ff0199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.084+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 >> 172.21.15.39:0/1213003582 conn(0x7f9ff0078c80 msgr2=0x7f9ff0105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:10.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.085+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 shutdown_connections 2024-07-04T08:31:10.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:10.085+0000 7f9ff5e00640 1 -- 172.21.15.39:0/1213003582 wait complete. 2024-07-04T08:31:10.185 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9072c93', total seconds elapsed: 34.1 2024-07-04T08:31:10.337 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:10 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1213003582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:10.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:10 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1213003582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:10.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:10 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1213003582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:10.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:31:11.351 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:11.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:11 smithi039 ceph-mon[32062]: pgmap v978: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-07-04T08:31:11.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:11 smithi050 ceph-mon[36577]: pgmap v978: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-07-04T08:31:11.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:11 smithi133 ceph-mon[37050]: pgmap v978: 129 pgs: 129 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 59 KiB/s wr, 31 op/s 2024-07-04T08:31:12.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:12 smithi039 ceph-mon[32062]: pgmap v979: 129 pgs: 129 active+clean; 30 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 KiB/s wr, 32 op/s 2024-07-04T08:31:12.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:12 smithi050 ceph-mon[36577]: pgmap v979: 129 pgs: 129 active+clean; 30 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 KiB/s wr, 32 op/s 2024-07-04T08:31:12.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:12 smithi133 ceph-mon[37050]: pgmap v979: 129 pgs: 129 active+clean; 30 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 KiB/s wr, 32 op/s 2024-07-04T08:31:13.551 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:13 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:31:13.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:13 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:31:13.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.779+0000 7f4f22400640 1 -- 172.21.15.39:0/3246297716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c102470 msgr2=0x7f4f1c10f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:13.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.779+0000 7f4f22400640 1 --2- 172.21.15.39:0/3246297716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c102470 0x7f4f1c10f190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009900 tx=0x7f4f0802a480 comp rx=0 tx=0).stop 2024-07-04T08:31:13.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.779+0000 7f4f22400640 1 -- 172.21.15.39:0/3246297716 shutdown_connections 2024-07-04T08:31:13.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.779+0000 7f4f22400640 1 --2- 172.21.15.39:0/3246297716 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f1c10f6d0 0x7f4f1c111af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.779+0000 7f4f22400640 1 --2- 172.21.15.39:0/3246297716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c102470 0x7f4f1c10f190 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.779+0000 7f4f22400640 1 --2- 172.21.15.39:0/3246297716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1c101b20 0x7f4f1c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.779+0000 7f4f22400640 1 -- 172.21.15.39:0/3246297716 >> 172.21.15.39:0/3246297716 conn(0x7f4f1c078c80 msgr2=0x7f4f1c079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:13.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.779+0000 7f4f22400640 1 -- 172.21.15.39:0/3246297716 shutdown_connections 2024-07-04T08:31:13.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.779+0000 7f4f22400640 1 -- 172.21.15.39:0/3246297716 wait complete. 2024-07-04T08:31:13.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.781+0000 7f4f22400640 1 Processor -- start 2024-07-04T08:31:13.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.781+0000 7f4f22400640 1 -- start start 2024-07-04T08:31:13.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f22400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c101b20 0x7f4f1c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f22400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1c102470 0x7f4f1c19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f22400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f1c10f6d0 0x7f4f1c1a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f22400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1151b0 con 0x7f4f1c102470 2024-07-04T08:31:13.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f22400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c115030 con 0x7f4f1c10f6d0 2024-07-04T08:31:13.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f22400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c115330 con 0x7f4f1c101b20 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1c102470 0x7f4f1c19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f1be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f1c10f6d0 0x7f4f1c1a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1c102470 0x7f4f1c19c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36624/0 (socket says 172.21.15.39:36624) 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f1aa00640 1 -- 172.21.15.39:0/655198642 learned_addr learned my addr 172.21.15.39:0/655198642 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f1be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f1c10f6d0 0x7f4f1c1a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56238/0 (socket says 172.21.15.39:56238) 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.782+0000 7f4f1b400640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c101b20 0x7f4f1c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.783+0000 7f4f1be00640 1 -- 172.21.15.39:0/655198642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c101b20 msgr2=0x7f4f1c19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.783+0000 7f4f1be00640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c101b20 0x7f4f1c19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.783+0000 7f4f1be00640 1 -- 172.21.15.39:0/655198642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1c102470 msgr2=0x7f4f1c19c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:13.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.783+0000 7f4f1be00640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1c102470 0x7f4f1c19c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.783+0000 7f4f1be00640 1 -- 172.21.15.39:0/655198642 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c1a1b70 con 0x7f4f1c10f6d0 2024-07-04T08:31:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.783+0000 7f4f1be00640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f1c10f6d0 0x7f4f1c1a1440 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009fd0 tx=0x7f4f1400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.784+0000 7f4f13e00640 1 -- 172.21.15.39:0/655198642 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1400ce20 con 0x7f4f1c10f6d0 2024-07-04T08:31:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.784+0000 7f4f22400640 1 -- 172.21.15.39:0/655198642 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c1a1e60 con 0x7f4f1c10f6d0 2024-07-04T08:31:13.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.784+0000 7f4f13e00640 1 -- 172.21.15.39:0/655198642 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f14005020 con 0x7f4f1c10f6d0 2024-07-04T08:31:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.784+0000 7f4f13e00640 1 -- 172.21.15.39:0/655198642 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14010690 con 0x7f4f1c10f6d0 2024-07-04T08:31:13.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.784+0000 7f4f22400640 1 -- 172.21.15.39:0/655198642 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1a85c0 con 0x7f4f1c10f6d0 2024-07-04T08:31:13.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.789+0000 7f4f28aab740 1 Processor -- start 2024-07-04T08:31:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.789+0000 7f4f13e00640 1 -- 172.21.15.39:0/655198642 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4f140054d0 con 0x7f4f1c10f6d0 2024-07-04T08:31:13.776 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.789+0000 7f4f28aab740 1 -- start start 2024-07-04T08:31:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592504b4070 0x5592508587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 0x559250858d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x559250859250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f28aab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5592507ceee0 con 0x5592504b4070 2024-07-04T08:31:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f28aab740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5592507be480 con 0x55925048b1b0 2024-07-04T08:31:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f28aab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592507c24c0 con 0x559250493390 2024-07-04T08:31:13.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592504b4070 0x5592508587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592504b4070 0x5592508587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36628/0 (socket says 172.21.15.39:36628) 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f1aa00640 1 -- 172.21.15.39:0/2434859706 learned_addr learned my addr 172.21.15.39:0/2434859706 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.790+0000 7f4f1b400640 1 --2- 172.21.15.39:0/2434859706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 0x559250858d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.791+0000 7f4f1aa00640 1 --2- 172.21.15.39:0/2434859706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x559250859250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.791+0000 7f4f1b400640 1 -- 172.21.15.39:0/2434859706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 msgr2=0x559250859250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.791+0000 7f4f1b400640 1 --2- 172.21.15.39:0/2434859706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x559250859250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.791+0000 7f4f1b400640 1 -- 172.21.15.39:0/2434859706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592504b4070 msgr2=0x5592508587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.791+0000 7f4f1b400640 1 --2- 172.21.15.39:0/2434859706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592504b4070 0x5592508587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.791+0000 7f4f1b400640 1 -- 172.21.15.39:0/2434859706 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559250756650 con 0x55925048b1b0 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.791+0000 7f4f13e00640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f000b2960 0x7f4f000b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.791+0000 7f4f1b400640 1 --2- 172.21.15.39:0/2434859706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 0x559250858d10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009c20 tx=0x7f4f0c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b481a64105901822 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.792+0000 7f4f13e00640 1 -- 172.21.15.39:0/655198642 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7f4f140e30c0 con 0x7f4f1c10f6d0 2024-07-04T08:31:13.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.792+0000 7f4f10c00640 1 -- 172.21.15.39:0/2434859706 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c004050 con 0x55925048b1b0 2024-07-04T08:31:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.792+0000 7f4f1aa00640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f000b2960 0x7f4f000b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.792+0000 7f4f10c00640 1 -- 172.21.15.39:0/2434859706 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c0054d0 con 0x55925048b1b0 2024-07-04T08:31:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.792+0000 7f4f10c00640 1 -- 172.21.15.39:0/2434859706 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c004c80 con 0x55925048b1b0 2024-07-04T08:31:13.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f28aab740 1 -- 172.21.15.39:0/2434859706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 msgr2=0x559250858d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f1aa00640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f000b2960 0x7f4f000b4e70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4f08005f70 tx=0x7f4f08005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2434859706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 0x559250858d10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009c20 tx=0x7f4f0c011810 comp rx=0 tx=0).stop 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f28aab740 1 -- 172.21.15.39:0/2434859706 shutdown_connections 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2434859706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x559250859250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2434859706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 0x559250858d10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2434859706 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5592504b4070 0x5592508587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f28aab740 1 -- 172.21.15.39:0/2434859706 >> 172.21.15.39:0/2434859706 conn(0x5592504a8460 msgr2=0x5592508566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f28aab740 1 -- 172.21.15.39:0/2434859706 shutdown_connections 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.793+0000 7f4f28aab740 1 -- 172.21.15.39:0/2434859706 wait complete. 2024-07-04T08:31:13.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.794+0000 7f4f28aab740 1 -- start start 2024-07-04T08:31:13.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.794+0000 7f4f28aab740 1 Processor -- start 2024-07-04T08:31:13.782 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.795+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 0x559250864360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.795+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559250493390 0x5592508648a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.795+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x5592508702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.795+0000 7f4f28aab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x559250622700 con 0x559250493390 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.795+0000 7f4f28aab740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559250765bb0 con 0x55925048b1b0 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.795+0000 7f4f28aab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559250866fb0 con 0x5592504b4070 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.795+0000 7f4f28aab740 10 client.0 _ll_register_callbacks cb 0x5592507b7570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559250493390 0x5592508648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559250493390 0x5592508648a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36634/0 (socket says 172.21.15.39:36634) 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1aa00640 1 -- 172.21.15.39:0/2449469757 learned_addr learned my addr 172.21.15.39:0/2449469757 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x5592508702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.783 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1b400640 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 0x559250864360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1be00640 1 -- 172.21.15.39:0/2449469757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 msgr2=0x559250864360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:13.784 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1be00640 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 0x559250864360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1be00640 1 -- 172.21.15.39:0/2449469757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559250493390 msgr2=0x5592508648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1be00640 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559250493390 0x5592508648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1be00640 1 -- 172.21.15.39:0/2449469757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55925051f650 con 0x5592504b4070 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.796+0000 7f4f1aa00640 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559250493390 0x5592508648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.797+0000 7f4f1be00640 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x5592508702d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4f140e03d0 tx=0x7f4f140aa660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131eaa9eb82d043a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.797+0000 7f4f05600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.797+0000 7f4f05600640 1 -- 172.21.15.39:0/2449469757 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140b2dc0 con 0x5592504b4070 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.797+0000 7f4f05600640 1 -- 172.21.15.39:0/2449469757 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7f4f140b2930 con 0x5592504b4070 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.797+0000 7f4f28aab740 1 -- 172.21.15.39:0/2449469757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0c0054d0 con 0x5592504b4070 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.797+0000 7f4f05600640 1 -- 172.21.15.39:0/2449469757 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f140b2b30 con 0x5592504b4070 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.797+0000 7f4f28aab740 10 client.28939 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.797+0000 7f4f28aab740 1 -- 172.21.15.39:0/2449469757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559250872fd0 con 0x5592504b4070 2024-07-04T08:31:13.785 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.798+0000 7f4f05600640 1 -- 172.21.15.39:0/2449469757 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c0054d0 con 0x5592504b4070 2024-07-04T08:31:13.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.798+0000 7f4f05600640 1 -- 172.21.15.39:0/2449469757 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f140ad070 con 0x5592504b4070 2024-07-04T08:31:13.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.798+0000 7f4f28aab740 10 client.28939 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:13.786 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.798+0000 7f4f28aab740 10 client.28939 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:31:13.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.798+0000 7f4f28aab740 20 client.28939 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:13.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.798+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592504729f0 0x559250876470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.798+0000 7f4f28aab740 4 client.28939 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:13.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.798+0000 7f4f28aab740 1 -- 172.21.15.39:0/2449469757 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592507d0e30 con 0x5592504729f0 2024-07-04T08:31:13.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.799+0000 7f4f1b400640 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592504729f0 0x559250876470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.799+0000 7f4f1b400640 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592504729f0 0x559250876470 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8604aedda2d8398a server_cookie=d22fe773454c69fa in_seq=0 out_seq=0 2024-07-04T08:31:13.787 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.799+0000 7f4f05600640 10 client.28939 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:31:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f05600640 1 -- 172.21.15.39:0/2449469757 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5592507d0e30 con 0x5592504729f0 2024-07-04T08:31:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f05600640 10 client.28939 handle_command_reply: tid=0 2024-07-04T08:31:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f28aab740 1 client.28939 shutdown 2024-07-04T08:31:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f28aab740 1 -- 172.21.15.39:0/2449469757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 msgr2=0x5592508702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x5592508702d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4f140e03d0 tx=0x7f4f140aa660 comp rx=0 tx=0).stop 2024-07-04T08:31:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f28aab740 1 -- 172.21.15.39:0/2449469757 shutdown_connections 2024-07-04T08:31:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5592504729f0 0x559250876470 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559250493390 0x5592508648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.788 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x5592508702d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f28aab740 1 --2- 172.21.15.39:0/2449469757 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55925048b1b0 0x559250864360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f05600640 0 client.28939 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:31:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.800+0000 7f4f28aab740 1 -- 172.21.15.39:0/2449469757 >> 172.21.15.39:0/2449469757 conn(0x5592504a8460 msgr2=0x55925085b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.801+0000 7f4f28aab740 1 -- 172.21.15.39:0/2449469757 shutdown_connections 2024-07-04T08:31:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.801+0000 7f4f28aab740 1 -- 172.21.15.39:0/2449469757 wait complete. 2024-07-04T08:31:13.789 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.801+0000 7f4f28aab740 20 client.28939 trim_cache size 0 max 16384 2024-07-04T08:31:13.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:13 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:31:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.816+0000 7f4f28aab740 1 Processor -- start 2024-07-04T08:31:13.803 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.817+0000 7f4f28aab740 1 -- start start 2024-07-04T08:31:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.817+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 0x5592508559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.817+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559250493390 0x5592508566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.817+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x55925085d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.817+0000 7f4f28aab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b2650 con 0x55925048b1b0 2024-07-04T08:31:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.817+0000 7f4f28aab740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b4190 con 0x559250493390 2024-07-04T08:31:13.804 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.817+0000 7f4f28aab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b2d20 con 0x5592504b4070 2024-07-04T08:31:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1b400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 0x5592508559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x55925085d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559250493390 0x5592508566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x55925085d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56224/0 (socket says 172.21.15.39:56224) 2024-07-04T08:31:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559250493390 0x5592508566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56276/0 (socket says 172.21.15.39:56276) 2024-07-04T08:31:13.805 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1be00640 1 -- 172.21.15.39:0/3459370494 learned_addr learned my addr 172.21.15.39:0/3459370494 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1b400640 1 -- 172.21.15.39:0/3459370494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 msgr2=0x55925085d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1b400640 1 --2- 172.21.15.39:0/3459370494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x55925085d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1b400640 1 -- 172.21.15.39:0/3459370494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559250493390 msgr2=0x5592508566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1b400640 1 --2- 172.21.15.39:0/3459370494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559250493390 0x5592508566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1b400640 1 -- 172.21.15.39:0/3459370494 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55925085aac0 con 0x55925048b1b0 2024-07-04T08:31:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1be00640 1 --2- 172.21.15.39:0/3459370494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x55925085d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.818+0000 7f4f1aa00640 1 --2- 172.21.15.39:0/3459370494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559250493390 0x5592508566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:13.806 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.819+0000 7f4f1b400640 1 --2- 172.21.15.39:0/3459370494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 0x5592508559d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c000c00 tx=0x7f4f0c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3326d4cba851653 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.819+0000 7f4f07e00640 1 -- 172.21.15.39:0/3459370494 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c02e3f0 con 0x55925048b1b0 2024-07-04T08:31:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.819+0000 7f4f07e00640 1 -- 172.21.15.39:0/3459370494 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c02ea40 con 0x55925048b1b0 2024-07-04T08:31:13.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.819+0000 7f4f07e00640 1 -- 172.21.15.39:0/3459370494 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c019a60 con 0x55925048b1b0 2024-07-04T08:31:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.820+0000 7f4f28aab740 1 -- 172.21.15.39:0/3459370494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 msgr2=0x5592508559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.820+0000 7f4f28aab740 1 --2- 172.21.15.39:0/3459370494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 0x5592508559d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c000c00 tx=0x7f4f0c009b70 comp rx=0 tx=0).stop 2024-07-04T08:31:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.821+0000 7f4f28aab740 1 -- 172.21.15.39:0/3459370494 shutdown_connections 2024-07-04T08:31:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.821+0000 7f4f28aab740 1 --2- 172.21.15.39:0/3459370494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592504b4070 0x55925085d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.821+0000 7f4f28aab740 1 --2- 172.21.15.39:0/3459370494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559250493390 0x5592508566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.821+0000 7f4f28aab740 1 --2- 172.21.15.39:0/3459370494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 0x5592508559d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.821+0000 7f4f28aab740 1 -- 172.21.15.39:0/3459370494 >> 172.21.15.39:0/3459370494 conn(0x5592504a8460 msgr2=0x559250855fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:13.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.821+0000 7f4f28aab740 1 -- 172.21.15.39:0/3459370494 shutdown_connections 2024-07-04T08:31:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.821+0000 7f4f28aab740 1 -- 172.21.15.39:0/3459370494 wait complete. 2024-07-04T08:31:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.822+0000 7f4f28aab740 1 -- start start 2024-07-04T08:31:13.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.822+0000 7f4f28aab740 1 Processor -- start 2024-07-04T08:31:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 0x55925086ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x55925086fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f28aab740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592504b4070 0x559250870350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f28aab740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b3ac0 con 0x55925048b1b0 2024-07-04T08:31:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f28aab740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b33f0 con 0x5592504b4070 2024-07-04T08:31:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f28aab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1b0650 con 0x559250493390 2024-07-04T08:31:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f28aab740 10 client.0 _ll_register_callbacks cb 0x7f4f140a5e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:13.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x55925086fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f1aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x55925086fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56226/0 (socket says 172.21.15.39:56226) 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f1aa00640 1 -- 172.21.15.39:0/1382114883 learned_addr learned my addr 172.21.15.39:0/1382114883 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.823+0000 7f4f1b400640 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 0x55925086ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f1aa00640 1 -- 172.21.15.39:0/1382114883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592504b4070 msgr2=0x559250870350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f1aa00640 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592504b4070 0x559250870350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f1aa00640 1 -- 172.21.15.39:0/1382114883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 msgr2=0x55925086ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f1aa00640 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 0x55925086ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f1aa00640 1 -- 172.21.15.39:0/1382114883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559250870ae0 con 0x559250493390 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f1aa00640 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x55925086fcb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4f08000c00 tx=0x7f4f0802a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f84b102a2f12cdb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f05600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f05600640 1 -- 172.21.15.39:0/1382114883 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08035040 con 0x559250493390 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f28aab740 1 -- 172.21.15.39:0/1382114883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0c02ea40 con 0x559250493390 2024-07-04T08:31:13.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.824+0000 7f4f05600640 1 -- 172.21.15.39:0/1382114883 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7f4f08031af0 con 0x559250493390 2024-07-04T08:31:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.825+0000 7f4f05600640 1 -- 172.21.15.39:0/1382114883 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f08031cf0 con 0x559250493390 2024-07-04T08:31:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.825+0000 7f4f28aab740 10 client.28945 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.825+0000 7f4f28aab740 1 -- 172.21.15.39:0/1382114883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5592508e7520 con 0x559250493390 2024-07-04T08:31:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.825+0000 7f4f05600640 1 -- 172.21.15.39:0/1382114883 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f0c02ea40 con 0x559250493390 2024-07-04T08:31:13.813 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.826+0000 7f4f05600640 1 -- 172.21.15.39:0/1382114883 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4f08055020 con 0x559250493390 2024-07-04T08:31:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.826+0000 7f4f28aab740 10 client.28945 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.826+0000 7f4f28aab740 10 client.28945 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:31:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.826+0000 7f4f28aab740 20 client.28945 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.826+0000 7f4f28aab740 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x559250447da0 0x5592508e9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.826+0000 7f4f28aab740 4 client.28945 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:31:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.826+0000 7f4f28aab740 1 -- 172.21.15.39:0/1382114883 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592507cc8d0 con 0x559250447da0 2024-07-04T08:31:13.814 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.826+0000 7f4f1b400640 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x559250447da0 0x5592508e9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:13.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.827+0000 7f4f1b400640 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x559250447da0 0x5592508e9c90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ff1d7c1fd51067f server_cookie=a015350ca0496c82 in_seq=0 out_seq=0 2024-07-04T08:31:13.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.827+0000 7f4f05600640 10 client.28945 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:31:13.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.827+0000 7f4f05600640 1 -- 172.21.15.39:0/1382114883 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592507cc8d0 con 0x559250447da0 2024-07-04T08:31:13.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.827+0000 7f4f05600640 10 client.28945 handle_command_reply: tid=0 2024-07-04T08:31:13.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.827+0000 7f4f28aab740 1 client.28945 shutdown 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 -- 172.21.15.39:0/1382114883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 msgr2=0x55925086fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x55925086fcb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4f08000c00 tx=0x7f4f0802a480 comp rx=0 tx=0).stop 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 -- 172.21.15.39:0/1382114883 shutdown_connections 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x559250447da0 0x5592508e9c90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5592504b4070 0x559250870350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559250493390 0x55925086fcb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f05600640 0 client.28945 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 --2- 172.21.15.39:0/1382114883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55925048b1b0 0x55925086ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 -- 172.21.15.39:0/1382114883 >> 172.21.15.39:0/1382114883 conn(0x5592504a8460 msgr2=0x559250859800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 -- 172.21.15.39:0/1382114883 shutdown_connections 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.828+0000 7f4f28aab740 1 -- 172.21.15.39:0/1382114883 wait complete. 2024-07-04T08:31:13.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.829+0000 7f4f28aab740 20 client.28945 trim_cache size 0 max 16384 2024-07-04T08:31:13.817 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:31:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.829+0000 7f4f22400640 1 -- 172.21.15.39:0/655198642 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f000b2960 msgr2=0x7f4f000b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:13.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.829+0000 7f4f22400640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f000b2960 0x7f4f000b4e70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4f08005f70 tx=0x7f4f08005a20 comp rx=0 tx=0).stop 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.829+0000 7f4f22400640 1 -- 172.21.15.39:0/655198642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f1c10f6d0 msgr2=0x7f4f1c1a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.829+0000 7f4f22400640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f1c10f6d0 0x7f4f1c1a1440 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009fd0 tx=0x7f4f1400ef10 comp rx=0 tx=0).stop 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.830+0000 7f4f22400640 1 -- 172.21.15.39:0/655198642 shutdown_connections 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.830+0000 7f4f22400640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4f000b2960 0x7f4f000b4e70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.830+0000 7f4f22400640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f1c102470 0x7f4f1c19c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.830+0000 7f4f22400640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4f1c10f6d0 0x7f4f1c1a1440 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.830+0000 7f4f22400640 1 --2- 172.21.15.39:0/655198642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f1c101b20 0x7f4f1c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.830+0000 7f4f22400640 1 -- 172.21.15.39:0/655198642 >> 172.21.15.39:0/655198642 conn(0x7f4f1c078c80 msgr2=0x7f4f1c109270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.830+0000 7f4f22400640 1 -- 172.21.15.39:0/655198642 shutdown_connections 2024-07-04T08:31:13.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:13.830+0000 7f4f22400640 1 -- 172.21.15.39:0/655198642 wait complete. 2024-07-04T08:31:14.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:31:15.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:31:15.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:14 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:31:15.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:14 smithi039 ceph-mon[32062]: pgmap v980: 129 pgs: 129 active+clean; 30 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 KiB/s wr, 32 op/s 2024-07-04T08:31:15.131 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:31:15.131 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:14 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:31:15.131 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:14 smithi050 ceph-mon[36577]: pgmap v980: 129 pgs: 129 active+clean; 30 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 KiB/s wr, 32 op/s 2024-07-04T08:31:15.204 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:15.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:31:15.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:14 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:31:15.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:14 smithi133 ceph-mon[37050]: pgmap v980: 129 pgs: 129 active+clean; 30 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 75 KiB/s wr, 32 op/s 2024-07-04T08:31:16.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:31:16.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:31:16.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:31:16.305 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:31:16.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:31:16.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:31:16.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:31:16.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:31:16.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:31:17.159 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:16 smithi039 ceph-mon[32062]: pgmap v981: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 13 op/s 2024-07-04T08:31:17.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:16 smithi133 ceph-mon[37050]: pgmap v981: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 13 op/s 2024-07-04T08:31:17.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:16 smithi050 ceph-mon[36577]: pgmap v981: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 13 op/s 2024-07-04T08:31:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.524+0000 7f9042800640 1 -- 172.21.15.39:0/936830473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f903c1030b0 msgr2=0x7f903c1034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:17.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.524+0000 7f9042800640 1 --2- 172.21.15.39:0/936830473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f903c1030b0 0x7f903c1034c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f902c009900 tx=0x7f902c02a400 comp rx=0 tx=0).stop 2024-07-04T08:31:17.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.524+0000 7f9042800640 1 -- 172.21.15.39:0/936830473 shutdown_connections 2024-07-04T08:31:17.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.525+0000 7f9042800640 1 --2- 172.21.15.39:0/936830473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c1089f0 0x7f903c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.525+0000 7f9042800640 1 --2- 172.21.15.39:0/936830473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c109340 0x7f903c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.525+0000 7f9042800640 1 --2- 172.21.15.39:0/936830473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f903c1030b0 0x7f903c1034c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.525+0000 7f9042800640 1 -- 172.21.15.39:0/936830473 >> 172.21.15.39:0/936830473 conn(0x7f903c07ada0 msgr2=0x7f903c07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:17.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.525+0000 7f9042800640 1 -- 172.21.15.39:0/936830473 shutdown_connections 2024-07-04T08:31:17.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.525+0000 7f9042800640 1 -- 172.21.15.39:0/936830473 wait complete. 2024-07-04T08:31:17.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.526+0000 7f9042800640 1 Processor -- start 2024-07-04T08:31:17.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.527+0000 7f9042800640 1 -- start start 2024-07-04T08:31:17.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.527+0000 7f9042800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f903c1030b0 0x7f903c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.527+0000 7f9042800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c1089f0 0x7f903c0751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.527+0000 7f9042800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c109340 0x7f903c0756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.527+0000 7f9042800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f903c112ff0 con 0x7f903c1089f0 2024-07-04T08:31:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.527+0000 7f9042800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f903c112e70 con 0x7f903c1030b0 2024-07-04T08:31:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f9042800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f903c113170 con 0x7f903c109340 2024-07-04T08:31:17.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f903c1030b0 0x7f903c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f903c1030b0 0x7f903c074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:56290/0 (socket says 172.21.15.39:56290) 2024-07-04T08:31:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903b400640 1 -- 172.21.15.39:0/1224970175 learned_addr learned my addr 172.21.15.39:0/1224970175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903be00640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c109340 0x7f903c0756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903aa00640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c1089f0 0x7f903c0751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.516 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903be00640 1 -- 172.21.15.39:0/1224970175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f903c1030b0 msgr2=0x7f903c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903be00640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f903c1030b0 0x7f903c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903be00640 1 -- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c1089f0 msgr2=0x7f903c0751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903be00640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c1089f0 0x7f903c0751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903be00640 1 -- 172.21.15.39:0/1224970175 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c1a7d60 con 0x7f903c109340 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.528+0000 7f903aa00640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c1089f0 0x7f903c0751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.529+0000 7f903be00640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c109340 0x7f903c0756e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f903400ec50 tx=0x7f903400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.529+0000 7f9033e00640 1 -- 172.21.15.39:0/1224970175 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034003b70 con 0x7f903c109340 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.529+0000 7f9042800640 1 -- 172.21.15.39:0/1224970175 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c1a7ff0 con 0x7f903c109340 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.529+0000 7f9033e00640 1 -- 172.21.15.39:0/1224970175 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9034003d10 con 0x7f903c109340 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.529+0000 7f9033e00640 1 -- 172.21.15.39:0/1224970175 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034010830 con 0x7f903c109340 2024-07-04T08:31:17.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.529+0000 7f9042800640 1 -- 172.21.15.39:0/1224970175 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c1a8500 con 0x7f903c109340 2024-07-04T08:31:17.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.533+0000 7f9048e19740 1 Processor -- start 2024-07-04T08:31:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.534+0000 7f9033e00640 1 -- 172.21.15.39:0/1224970175 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f9034003ec0 con 0x7f903c109340 2024-07-04T08:31:17.521 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f9048e19740 1 -- start start 2024-07-04T08:31:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f9048e19740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406167c070 0x564061a207b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f9048e19740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 0x564061a20cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f9048e19740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a21230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f9048e19740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564061996ee0 con 0x56406165b390 2024-07-04T08:31:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f9048e19740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564061986480 con 0x5640616531b0 2024-07-04T08:31:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f9048e19740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56406198a4c0 con 0x56406167c070 2024-07-04T08:31:17.522 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f903aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a21230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.523 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f903aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a21230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36696/0 (socket says 172.21.15.39:36696) 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.535+0000 7f903aa00640 1 -- 172.21.15.39:0/2800223456 learned_addr learned my addr 172.21.15.39:0/2800223456 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.536+0000 7f903b400640 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 0x564061a20cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.536+0000 7f903aa00640 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406167c070 0x564061a207b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.536+0000 7f903aa00640 1 -- 172.21.15.39:0/2800223456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406167c070 msgr2=0x564061a207b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.536+0000 7f903aa00640 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406167c070 0x564061a207b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.536+0000 7f903aa00640 1 -- 172.21.15.39:0/2800223456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 msgr2=0x564061a20cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.536+0000 7f903aa00640 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 0x564061a20cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.536+0000 7f903aa00640 1 -- 172.21.15.39:0/2800223456 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56406191e650 con 0x56406165b390 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.536+0000 7f903b400640 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 0x564061a20cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.536+0000 7f903aa00640 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a21230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f90280127c0 tx=0x7f90280096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a84c508fb3e591 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:17.524 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.537+0000 7f9033e00640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90200b29e0 0x7f90200b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.537+0000 7f9033e00640 1 -- 172.21.15.39:0/1224970175 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7f90340e2f20 con 0x7f903c109340 2024-07-04T08:31:17.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.537+0000 7f903b400640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90200b29e0 0x7f90200b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.537+0000 7f9030c00640 1 -- 172.21.15.39:0/2800223456 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028012d40 con 0x56406165b390 2024-07-04T08:31:17.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.537+0000 7f9030c00640 1 -- 172.21.15.39:0/2800223456 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90280107d0 con 0x56406165b390 2024-07-04T08:31:17.525 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.537+0000 7f9030c00640 1 -- 172.21.15.39:0/2800223456 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028003830 con 0x56406165b390 2024-07-04T08:31:17.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.538+0000 7f9048e19740 1 -- 172.21.15.39:0/2800223456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 msgr2=0x564061a21230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:17.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.538+0000 7f9048e19740 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a21230 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f90280127c0 tx=0x7f90280096c0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.526 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.538+0000 7f903b400640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90200b29e0 0x7f90200b4ef0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f902c009230 tx=0x7f902c0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.538+0000 7f9048e19740 1 -- 172.21.15.39:0/2800223456 shutdown_connections 2024-07-04T08:31:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.538+0000 7f9048e19740 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a21230 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.538+0000 7f9048e19740 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 0x564061a20cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.538+0000 7f9048e19740 1 --2- 172.21.15.39:0/2800223456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406167c070 0x564061a207b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.538+0000 7f9048e19740 1 -- 172.21.15.39:0/2800223456 >> 172.21.15.39:0/2800223456 conn(0x564061670460 msgr2=0x564061a1e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.538+0000 7f9048e19740 1 -- 172.21.15.39:0/2800223456 shutdown_connections 2024-07-04T08:31:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.539+0000 7f9048e19740 1 -- 172.21.15.39:0/2800223456 wait complete. 2024-07-04T08:31:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.539+0000 7f9048e19740 1 -- start start 2024-07-04T08:31:17.530 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.539+0000 7f9048e19740 1 Processor -- start 2024-07-04T08:31:17.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f9048e19740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f9048e19740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a2c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f9048e19740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406167c070 0x564061a38280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f9048e19740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5640617ea700 con 0x56406165b390 2024-07-04T08:31:17.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f9048e19740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56406192dbb0 con 0x56406167c070 2024-07-04T08:31:17.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f9048e19740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564061a2ef90 con 0x5640616531b0 2024-07-04T08:31:17.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f9048e19740 10 client.0 _ll_register_callbacks cb 0x56406197f570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f903aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a2c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f903b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f903b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56266/0 (socket says 172.21.15.39:56266) 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f903b400640 1 -- 172.21.15.39:0/3005902003 learned_addr learned my addr 172.21.15.39:0/3005902003 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.540+0000 7f903be00640 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406167c070 0x564061a38280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.541+0000 7f903aa00640 1 -- 172.21.15.39:0/3005902003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 msgr2=0x564061a2c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.541+0000 7f903aa00640 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.541+0000 7f903aa00640 1 -- 172.21.15.39:0/3005902003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406167c070 msgr2=0x564061a38280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.541+0000 7f903aa00640 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406167c070 0x564061a38280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.541+0000 7f903aa00640 1 -- 172.21.15.39:0/3005902003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640616e7650 con 0x56406165b390 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.541+0000 7f903b400640 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:17.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.541+0000 7f903aa00640 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a2c8b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9028012790 tx=0x7f9028004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbd99467e3eb838 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.541+0000 7f9025600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.541+0000 7f9025600640 1 -- 172.21.15.39:0/3005902003 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028012d40 con 0x56406165b390 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9025600640 1 -- 172.21.15.39:0/3005902003 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7f9028004440 con 0x56406165b390 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9048e19740 1 -- 172.21.15.39:0/3005902003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90280107d0 con 0x56406165b390 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9025600640 1 -- 172.21.15.39:0/3005902003 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9028030050 con 0x56406165b390 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9048e19740 10 client.18906 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9048e19740 1 -- 172.21.15.39:0/3005902003 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564061a3afb0 con 0x56406165b390 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9025600640 1 -- 172.21.15.39:0/3005902003 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90280107d0 con 0x56406165b390 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9025600640 1 -- 172.21.15.39:0/3005902003 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f90280035f0 con 0x56406165b390 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9048e19740 10 client.18906 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9048e19740 10 client.18906 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:31:17.535 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.542+0000 7f9048e19740 20 client.18906 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.543+0000 7f9048e19740 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56406163a9f0 0x564061a3e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.543+0000 7f9048e19740 4 client.18906 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.543+0000 7f9048e19740 1 -- 172.21.15.39:0/3005902003 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564061998e30 con 0x56406163a9f0 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.543+0000 7f903b400640 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56406163a9f0 0x564061a3e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.543+0000 7f903b400640 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56406163a9f0 0x564061a3e310 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94806bf9246204d8 server_cookie=f4c9652dd1019686 in_seq=0 out_seq=0 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.543+0000 7f9025600640 10 client.18906 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.544+0000 7f9025600640 1 -- 172.21.15.39:0/3005902003 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564061998e30 con 0x56406163a9f0 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.544+0000 7f9025600640 10 client.18906 handle_command_reply: tid=0 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.544+0000 7f9048e19740 1 client.18906 shutdown 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.544+0000 7f9048e19740 1 -- 172.21.15.39:0/3005902003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 msgr2=0x564061a2c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.544+0000 7f9048e19740 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a2c8b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9028012790 tx=0x7f9028004250 comp rx=0 tx=0).stop 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9048e19740 1 -- 172.21.15.39:0/3005902003 shutdown_connections 2024-07-04T08:31:17.536 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9048e19740 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56406163a9f0 0x564061a3e310 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9048e19740 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406165b390 0x564061a2c8b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9048e19740 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406167c070 0x564061a38280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9048e19740 1 --2- 172.21.15.39:0/3005902003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9025600640 0 client.18906 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:31:17.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9048e19740 1 -- 172.21.15.39:0/3005902003 >> 172.21.15.39:0/3005902003 conn(0x564061670460 msgr2=0x564061a23990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:17.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9048e19740 1 -- 172.21.15.39:0/3005902003 shutdown_connections 2024-07-04T08:31:17.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9048e19740 1 -- 172.21.15.39:0/3005902003 wait complete. 2024-07-04T08:31:17.537 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.545+0000 7f9048e19740 20 client.18906 trim_cache size 0 max 16384 2024-07-04T08:31:17.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.560+0000 7f9048e19740 1 Processor -- start 2024-07-04T08:31:17.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.560+0000 7f9048e19740 1 -- start start 2024-07-04T08:31:17.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.560+0000 7f9048e19740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.560+0000 7f9048e19740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406165b390 0x564061a21ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.560+0000 7f9048e19740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a38070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.560+0000 7f9048e19740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b2630 con 0x56406167c070 2024-07-04T08:31:17.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.560+0000 7f9048e19740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b4170 con 0x56406165b390 2024-07-04T08:31:17.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f9048e19740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b2d00 con 0x5640616531b0 2024-07-04T08:31:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a38070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:56272/0 (socket says 172.21.15.39:56272) 2024-07-04T08:31:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903b400640 1 -- 172.21.15.39:0/4144383169 learned_addr learned my addr 172.21.15.39:0/4144383169 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903aa00640 1 --2- 172.21.15.39:0/4144383169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406165b390 0x564061a21ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903b400640 1 -- 172.21.15.39:0/4144383169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406165b390 msgr2=0x564061a21ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903b400640 1 --2- 172.21.15.39:0/4144383169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406165b390 0x564061a21ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903b400640 1 -- 172.21.15.39:0/4144383169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 msgr2=0x564061a38070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903b400640 1 --2- 172.21.15.39:0/4144383169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a38070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.561+0000 7f903b400640 1 -- 172.21.15.39:0/4144383169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564061a2d430 con 0x5640616531b0 2024-07-04T08:31:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.562+0000 7f903b400640 1 --2- 172.21.15.39:0/4144383169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2e050 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f902c004770 tx=0x7f902c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b86dba117690c8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.562+0000 7f9027e00640 1 -- 172.21.15.39:0/4144383169 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c035040 con 0x5640616531b0 2024-07-04T08:31:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.562+0000 7f9027e00640 1 -- 172.21.15.39:0/4144383169 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f902c059070 con 0x5640616531b0 2024-07-04T08:31:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.562+0000 7f9027e00640 1 -- 172.21.15.39:0/4144383169 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c053610 con 0x5640616531b0 2024-07-04T08:31:17.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 -- 172.21.15.39:0/4144383169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 msgr2=0x564061a2e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:17.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 --2- 172.21.15.39:0/4144383169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2e050 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f902c004770 tx=0x7f902c03f000 comp rx=0 tx=0).stop 2024-07-04T08:31:17.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 -- 172.21.15.39:0/4144383169 shutdown_connections 2024-07-04T08:31:17.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 --2- 172.21.15.39:0/4144383169 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a38070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 --2- 172.21.15.39:0/4144383169 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56406165b390 0x564061a21ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 --2- 172.21.15.39:0/4144383169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5640616531b0 0x564061a2e050 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 -- 172.21.15.39:0/4144383169 >> 172.21.15.39:0/4144383169 conn(0x564061670460 msgr2=0x564061a217e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:17.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 -- 172.21.15.39:0/4144383169 shutdown_connections 2024-07-04T08:31:17.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 -- 172.21.15.39:0/4144383169 wait complete. 2024-07-04T08:31:17.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.563+0000 7f9048e19740 1 -- start start 2024-07-04T08:31:17.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.564+0000 7f9048e19740 1 Processor -- start 2024-07-04T08:31:17.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.564+0000 7f9048e19740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 0x564061a345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.564+0000 7f9048e19740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406165b390 0x564061a2faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.564+0000 7f9048e19740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a30030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.564+0000 7f9048e19740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b3aa0 con 0x56406167c070 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.564+0000 7f9048e19740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b33d0 con 0x5640616531b0 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.564+0000 7f9048e19740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b0630 con 0x56406165b390 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.564+0000 7f9048e19740 10 client.0 _ll_register_callbacks cb 0x7f9028012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.564+0000 7f903be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a30030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a30030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36714/0 (socket says 172.21.15.39:36714) 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903be00640 1 -- 172.21.15.39:0/3200280958 learned_addr learned my addr 172.21.15.39:0/3200280958 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903aa00640 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406165b390 0x564061a2faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903b400640 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 0x564061a345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903aa00640 1 -- 172.21.15.39:0/3200280958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 msgr2=0x564061a345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903aa00640 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 0x564061a345f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903aa00640 1 -- 172.21.15.39:0/3200280958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 msgr2=0x564061a30030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903aa00640 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a30030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903aa00640 1 -- 172.21.15.39:0/3200280958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564061a30570 con 0x56406165b390 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903be00640 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a30030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f903aa00640 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406165b390 0x564061a2faf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9028012760 tx=0x7f9028003fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26215cfc983ac0e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.565+0000 7f9025600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.566+0000 7f9025600640 1 -- 172.21.15.39:0/3200280958 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902803a060 con 0x56406165b390 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.566+0000 7f9025600640 1 -- 172.21.15.39:0/3200280958 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9512+0+0 (secure 0 0 0) 0x7f90280092e0 con 0x56406165b390 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.566+0000 7f9048e19740 1 -- 172.21.15.39:0/3200280958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f902c059070 con 0x56406165b390 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.566+0000 7f9025600640 1 -- 172.21.15.39:0/3200280958 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f90280107a0 con 0x56406165b390 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.566+0000 7f9048e19740 10 client.28963 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.566+0000 7f9048e19740 1 -- 172.21.15.39:0/3200280958 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564061aa6580 con 0x56406165b390 2024-07-04T08:31:17.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.566+0000 7f9025600640 1 -- 172.21.15.39:0/3200280958 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f902c059070 con 0x56406165b390 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.567+0000 7f9025600640 1 -- 172.21.15.39:0/3200280958 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9028003b60 con 0x56406165b390 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.567+0000 7f9048e19740 10 client.28963 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.567+0000 7f9048e19740 10 client.28963 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.567+0000 7f9048e19740 20 client.28963 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.567+0000 7f9048e19740 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56406160fda0 0x564061aa8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.567+0000 7f9048e19740 4 client.28963 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.567+0000 7f9048e19740 1 -- 172.21.15.39:0/3200280958 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640619948d0 con 0x56406160fda0 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.567+0000 7f903b400640 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56406160fda0 0x564061aa8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.568+0000 7f903b400640 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56406160fda0 0x564061aa8ce0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c6362eb9059bafe server_cookie=2f7c310a46ff408b in_seq=0 out_seq=0 2024-07-04T08:31:17.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.568+0000 7f9025600640 10 client.28963 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:31:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.568+0000 7f9025600640 1 -- 172.21.15.39:0/3200280958 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640619948d0 con 0x56406160fda0 2024-07-04T08:31:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.568+0000 7f9025600640 10 client.28963 handle_command_reply: tid=0 2024-07-04T08:31:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.568+0000 7f9048e19740 1 client.28963 shutdown 2024-07-04T08:31:17.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 -- 172.21.15.39:0/3200280958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406165b390 msgr2=0x564061a2faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406165b390 0x564061a2faf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9028012760 tx=0x7f9028003fa0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 -- 172.21.15.39:0/3200280958 shutdown_connections 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x56406160fda0 0x564061aa8ce0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56406167c070 0x564061a30030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56406165b390 0x564061a2faf0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 --2- 172.21.15.39:0/3200280958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5640616531b0 0x564061a345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 -- 172.21.15.39:0/3200280958 >> 172.21.15.39:0/3200280958 conn(0x564061670460 msgr2=0x564061a2dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9025600640 0 client.28963 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 -- 172.21.15.39:0/3200280958 shutdown_connections 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 1 -- 172.21.15.39:0/3200280958 wait complete. 2024-07-04T08:31:17.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.569+0000 7f9048e19740 20 client.28963 trim_cache size 0 max 16384 2024-07-04T08:31:17.560 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:31:17.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 -- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90200b29e0 msgr2=0x7f90200b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:17.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90200b29e0 0x7f90200b4ef0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f902c009230 tx=0x7f902c0058d0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 -- 172.21.15.39:0/1224970175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c109340 msgr2=0x7f903c0756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:17.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c109340 0x7f903c0756e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f903400ec50 tx=0x7f903400c6a0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 -- 172.21.15.39:0/1224970175 shutdown_connections 2024-07-04T08:31:17.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f90200b29e0 0x7f90200b4ef0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c1089f0 0x7f903c0751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f903c109340 0x7f903c0756e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 --2- 172.21.15.39:0/1224970175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f903c1030b0 0x7f903c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:17.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 -- 172.21.15.39:0/1224970175 >> 172.21.15.39:0/1224970175 conn(0x7f903c07ada0 msgr2=0x7f903c106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:17.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 -- 172.21.15.39:0/1224970175 shutdown_connections 2024-07-04T08:31:17.561 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:17.570+0000 7f9042800640 1 -- 172.21.15.39:0/1224970175 wait complete. 2024-07-04T08:31:18.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:31:18.977 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:19.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:19 smithi039 ceph-mon[32062]: pgmap v982: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:19.445 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:19 smithi039 ceph-mon[32062]: osdmap e112: 12 total, 12 up, 12 in 2024-07-04T08:31:19.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:19 smithi133 ceph-mon[37050]: pgmap v982: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:19.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:19 smithi133 ceph-mon[37050]: osdmap e112: 12 total, 12 up, 12 in 2024-07-04T08:31:19.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:19 smithi050 ceph-mon[36577]: pgmap v982: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:19.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:19 smithi050 ceph-mon[36577]: osdmap e112: 12 total, 12 up, 12 in 2024-07-04T08:31:21.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.152+0000 7f57e1000640 1 -- 172.21.15.39:0/843325210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc102f00 msgr2=0x7f57dc103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:21.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.152+0000 7f57e1000640 1 --2- 172.21.15.39:0/843325210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc102f00 0x7f57dc103310 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009900 tx=0x7f57cc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.153+0000 7f57e1000640 1 -- 172.21.15.39:0/843325210 shutdown_connections 2024-07-04T08:31:21.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.153+0000 7f57e1000640 1 --2- 172.21.15.39:0/843325210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57dc109190 0x7f57dc10f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.153+0000 7f57e1000640 1 --2- 172.21.15.39:0/843325210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc108840 0x7f57dc108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.153+0000 7f57e1000640 1 --2- 172.21.15.39:0/843325210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc102f00 0x7f57dc103310 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.153+0000 7f57e1000640 1 -- 172.21.15.39:0/843325210 >> 172.21.15.39:0/843325210 conn(0x7f57dc0fe990 msgr2=0x7f57dc100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:21.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.153+0000 7f57e1000640 1 -- 172.21.15.39:0/843325210 shutdown_connections 2024-07-04T08:31:21.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.153+0000 7f57e1000640 1 -- 172.21.15.39:0/843325210 wait complete. 2024-07-04T08:31:21.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.154+0000 7f57e1000640 1 Processor -- start 2024-07-04T08:31:21.143 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.154+0000 7f57e1000640 1 -- start start 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.155+0000 7f57e1000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc102f00 0x7f57dc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.155+0000 7f57e1000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc108840 0x7f57dc19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.155+0000 7f57e1000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57dc109190 0x7f57dc1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.155+0000 7f57e1000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc112e50 con 0x7f57dc109190 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.155+0000 7f57e1000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc112cd0 con 0x7f57dc108840 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.155+0000 7f57e1000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc112fd0 con 0x7f57dc102f00 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.155+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57dc109190 0x7f57dc1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.155+0000 7f57daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc108840 0x7f57dc19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.155+0000 7f57db400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc102f00 0x7f57dc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.144 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57daa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc108840 0x7f57dc19c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33300/0 (socket says 172.21.15.39:33300) 2024-07-04T08:31:21.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57db400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc102f00 0x7f57dc19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:38560/0 (socket says 172.21.15.39:38560) 2024-07-04T08:31:21.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57daa00640 1 -- 172.21.15.39:0/2242112710 learned_addr learned my addr 172.21.15.39:0/2242112710 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:21.145 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57dc109190 0x7f57dc1a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56206/0 (socket says 172.21.15.39:56206) 2024-07-04T08:31:21.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57daa00640 1 -- 172.21.15.39:0/2242112710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc102f00 msgr2=0x7f57dc19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:21.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57daa00640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc102f00 0x7f57dc19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57daa00640 1 -- 172.21.15.39:0/2242112710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57dc109190 msgr2=0x7f57dc1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:21.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57daa00640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57dc109190 0x7f57dc1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57daa00640 1 -- 172.21.15.39:0/2242112710 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc1a19d0 con 0x7f57dc108840 2024-07-04T08:31:21.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.156+0000 7f57daa00640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc108840 0x7f57dc19c000 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f57c800d950 tx=0x7f57c800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:21.146 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.157+0000 7f57d3e00640 1 -- 172.21.15.39:0/2242112710 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8017070 con 0x7f57dc108840 2024-07-04T08:31:21.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.157+0000 7f57d3e00640 1 -- 172.21.15.39:0/2242112710 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57c8009c30 con 0x7f57dc108840 2024-07-04T08:31:21.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.157+0000 7f57e1000640 1 -- 172.21.15.39:0/2242112710 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc1a1cc0 con 0x7f57dc108840 2024-07-04T08:31:21.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.157+0000 7f57d3e00640 1 -- 172.21.15.39:0/2242112710 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c80047a0 con 0x7f57dc108840 2024-07-04T08:31:21.147 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.157+0000 7f57e1000640 1 -- 172.21.15.39:0/2242112710 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc1a8420 con 0x7f57dc108840 2024-07-04T08:31:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.160+0000 7f57e8ba8740 1 Processor -- start 2024-07-04T08:31:21.148 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.160+0000 7f57e8ba8740 1 -- start start 2024-07-04T08:31:21.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568a8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x5557568a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55575681eee0 con 0x5557564e3390 2024-07-04T08:31:21.149 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55575680e480 con 0x555756504070 2024-07-04T08:31:21.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5557568124c0 con 0x5557564db1b0 2024-07-04T08:31:21.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33304/0 (socket says 172.21.15.39:33304) 2024-07-04T08:31:21.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57dbe00640 1 -- 172.21.15.39:0/3365253698 learned_addr learned my addr 172.21.15.39:0/3365253698 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:21.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57d3e00640 1 -- 172.21.15.39:0/2242112710 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f57c8004020 con 0x7f57dc108840 2024-07-04T08:31:21.150 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.161+0000 7f57db400640 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.151 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57daa00640 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x5557568a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57dbe00640 1 -- 172.21.15.39:0/3365253698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 msgr2=0x5557568a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57dbe00640 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57dbe00640 1 -- 172.21.15.39:0/3365253698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 msgr2=0x5557568a93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57dbe00640 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x5557568a93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57dbe00640 1 -- 172.21.15.39:0/3365253698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557567a6650 con 0x555756504070 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57daa00640 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x5557568a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57dbe00640 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568a8950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009f50 tx=0x7f57d40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb046a6f71c13377 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57d0c00640 1 -- 172.21.15.39:0/3365253698 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4004d20 con 0x555756504070 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57d0c00640 1 -- 172.21.15.39:0/3365253698 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d4004050 con 0x555756504070 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.162+0000 7f57d0c00640 1 -- 172.21.15.39:0/3365253698 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4005260 con 0x555756504070 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.163+0000 7f57d3e00640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c00b29e0 0x7f57c00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.152 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.163+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3365253698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 msgr2=0x5557568a8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.163+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568a8950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009f50 tx=0x7f57d40118d0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.163+0000 7f57d3e00640 1 -- 172.21.15.39:0/2242112710 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f57c80e25a0 con 0x7f57dc108840 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.163+0000 7f57db400640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c00b29e0 0x7f57c00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.163+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3365253698 shutdown_connections 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.163+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x5557568a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.163+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.164+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/3365253698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568a8950 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.164+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3365253698 >> 172.21.15.39:0/3365253698 conn(0x5557564f8460 msgr2=0x5557568a6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.164+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3365253698 shutdown_connections 2024-07-04T08:31:21.153 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.164+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3365253698 wait complete. 2024-07-04T08:31:21.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.164+0000 7f57e8ba8740 1 -- start start 2024-07-04T08:31:21.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.164+0000 7f57db400640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c00b29e0 0x7f57c00b4ef0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f57cc031440 tx=0x7f57cc009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:21.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.165+0000 7f57e8ba8740 1 Processor -- start 2024-07-04T08:31:21.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.165+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.156 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.165+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x5557568b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.165+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568c0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.165+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555756672700 con 0x5557564e3390 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.165+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5557567b5bb0 con 0x555756504070 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.165+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5557568b7130 con 0x5557564db1b0 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.165+0000 7f57e8ba8740 10 client.0 _ll_register_callbacks cb 0x555756807570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57db400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x5557568b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57db400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:38580/0 (socket says 172.21.15.39:38580) 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57db400640 1 -- 172.21.15.39:0/2177777794 learned_addr learned my addr 172.21.15.39:0/2177777794 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57dbe00640 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57db400640 1 -- 172.21.15.39:0/2177777794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 msgr2=0x5557568c0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57db400640 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568c0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57db400640 1 -- 172.21.15.39:0/2177777794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 msgr2=0x5557568b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57db400640 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x5557568b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57db400640 1 -- 172.21.15.39:0/2177777794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575656f650 con 0x5557564db1b0 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.166+0000 7f57dbe00640 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568c0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.167+0000 7f57db400640 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568b4460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0057c0 tx=0x7f57cc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f754d8a6e967e51c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.167+0000 7f57c5600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.167+0000 7f57c5600640 1 -- 172.21.15.39:0/2177777794 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc04d070 con 0x5557564db1b0 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.167+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2177777794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d4004050 con 0x5557564db1b0 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.167+0000 7f57c5600640 1 -- 172.21.15.39:0/2177777794 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f57cc005230 con 0x5557564db1b0 2024-07-04T08:31:21.158 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.167+0000 7f57c5600640 1 -- 172.21.15.39:0/2177777794 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57cc059050 con 0x5557564db1b0 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.167+0000 7f57e8ba8740 10 client.28975 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.167+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2177777794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5557568c3050 con 0x5557564db1b0 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.168+0000 7f57c5600640 1 -- 172.21.15.39:0/2177777794 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d4004050 con 0x5557564db1b0 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.168+0000 7f57c5600640 1 -- 172.21.15.39:0/2177777794 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f57cc003960 con 0x5557564db1b0 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.168+0000 7f57e8ba8740 10 client.28975 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.168+0000 7f57e8ba8740 10 client.28975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.168+0000 7f57e8ba8740 20 client.28975 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.168+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5557564c29f0 0x5557568c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.168+0000 7f57e8ba8740 4 client.28975 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.168+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2177777794 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555756820e30 con 0x5557564c29f0 2024-07-04T08:31:21.159 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.168+0000 7f57daa00640 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5557564c29f0 0x5557568c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.169+0000 7f57daa00640 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5557564c29f0 0x5557568c64b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c582260cb12f85c server_cookie=cf177a7fe07c4dea in_seq=0 out_seq=0 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.169+0000 7f57c5600640 10 client.28975 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.170+0000 7f57c5600640 1 -- 172.21.15.39:0/2177777794 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555756820e30 con 0x5557564c29f0 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.170+0000 7f57c5600640 10 client.28975 handle_command_reply: tid=0 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.170+0000 7f57e8ba8740 1 client.28975 shutdown 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.170+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2177777794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 msgr2=0x5557568b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.170+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568b4460 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f57cc0057c0 tx=0x7f57cc009260 comp rx=0 tx=0).stop 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2177777794 shutdown_connections 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x5557564c29f0 0x5557568c64b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x5557568b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555756504070 0x5557568c0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2177777794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557564db1b0 0x5557568b4460 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57c5600640 0 client.28975 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:31:21.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2177777794 >> 172.21.15.39:0/2177777794 conn(0x5557564f8460 msgr2=0x5557568abb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:21.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2177777794 shutdown_connections 2024-07-04T08:31:21.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2177777794 wait complete. 2024-07-04T08:31:21.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.171+0000 7f57e8ba8740 20 client.28975 trim_cache size 0 max 16384 2024-07-04T08:31:21.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.186+0000 7f57e8ba8740 1 Processor -- start 2024-07-04T08:31:21.174 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.187+0000 7f57e8ba8740 1 -- start start 2024-07-04T08:31:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.187+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564db1b0 0x5557568aea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.187+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564e3390 0x5557568b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.187+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 0x5557568c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.187+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1b2450 con 0x5557564db1b0 2024-07-04T08:31:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.187+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1b3f90 con 0x5557564e3390 2024-07-04T08:31:21.175 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.187+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1b2b20 con 0x555756504070 2024-07-04T08:31:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.187+0000 7f57db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564db1b0 0x5557568aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564db1b0 0x5557568aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56224/0 (socket says 172.21.15.39:56224) 2024-07-04T08:31:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57db400640 1 -- 172.21.15.39:0/3039379671 learned_addr learned my addr 172.21.15.39:0/3039379671 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57dbe00640 1 --2- 172.21.15.39:0/3039379671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 0x5557568c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57daa00640 1 --2- 172.21.15.39:0/3039379671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564e3390 0x5557568b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57db400640 1 -- 172.21.15.39:0/3039379671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 msgr2=0x5557568c0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57db400640 1 --2- 172.21.15.39:0/3039379671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 0x5557568c0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57db400640 1 -- 172.21.15.39:0/3039379671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564e3390 msgr2=0x5557568b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57db400640 1 --2- 172.21.15.39:0/3039379671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564e3390 0x5557568b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57db400640 1 -- 172.21.15.39:0/3039379671 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557568b6be0 con 0x5557564db1b0 2024-07-04T08:31:21.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.188+0000 7f57db400640 1 --2- 172.21.15.39:0/3039379671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564db1b0 0x5557568aea70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f57cc003db0 tx=0x7f57cc009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fef9e01793888a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.189+0000 7f57c7e00640 1 -- 172.21.15.39:0/3039379671 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc04d070 con 0x5557564db1b0 2024-07-04T08:31:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.189+0000 7f57c7e00640 1 -- 172.21.15.39:0/3039379671 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57cc005200 con 0x5557564db1b0 2024-07-04T08:31:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.189+0000 7f57c7e00640 1 -- 172.21.15.39:0/3039379671 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc035040 con 0x5557564db1b0 2024-07-04T08:31:21.178 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.190+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3039379671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564db1b0 msgr2=0x5557568aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.190+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/3039379671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564db1b0 0x5557568aea70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f57cc003db0 tx=0x7f57cc009880 comp rx=0 tx=0).stop 2024-07-04T08:31:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.190+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3039379671 shutdown_connections 2024-07-04T08:31:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.190+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/3039379671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 0x5557568c0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.190+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/3039379671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564e3390 0x5557568b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.190+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/3039379671 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564db1b0 0x5557568aea70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.190+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3039379671 >> 172.21.15.39:0/3039379671 conn(0x5557564f8460 msgr2=0x5557568b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.190+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3039379671 shutdown_connections 2024-07-04T08:31:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.190+0000 7f57e8ba8740 1 -- 172.21.15.39:0/3039379671 wait complete. 2024-07-04T08:31:21.179 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.191+0000 7f57e8ba8740 1 -- start start 2024-07-04T08:31:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.191+0000 7f57e8ba8740 1 Processor -- start 2024-07-04T08:31:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.191+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564db1b0 0x55575692f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x55575692fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57e8ba8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 0x5557568b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1b38c0 con 0x5557564e3390 2024-07-04T08:31:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1b31f0 con 0x5557564db1b0 2024-07-04T08:31:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57e8ba8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc1b0430 con 0x555756504070 2024-07-04T08:31:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57e8ba8740 10 client.0 _ll_register_callbacks cb 0x7f57cc0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:21.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57db400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564db1b0 0x55575692f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57daa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x55575692fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57db400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564db1b0 0x55575692f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33330/0 (socket says 172.21.15.39:33330) 2024-07-04T08:31:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57db400640 1 -- 172.21.15.39:0/2673422270 learned_addr learned my addr 172.21.15.39:0/2673422270 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57dbe00640 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 0x5557568b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57db400640 1 -- 172.21.15.39:0/2673422270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 msgr2=0x5557568b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.192+0000 7f57db400640 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 0x5557568b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57db400640 1 -- 172.21.15.39:0/2673422270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 msgr2=0x55575692fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57db400640 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x55575692fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.182 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57db400640 1 -- 172.21.15.39:0/2673422270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557568b81e0 con 0x5557564db1b0 2024-07-04T08:31:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57dbe00640 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 0x5557568b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57daa00640 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x55575692fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57db400640 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564db1b0 0x55575692f850 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f57cc005930 tx=0x7f57cc004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d195d8fe435e5c8a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:21.183 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57c5600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:21.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57c5600640 1 -- 172.21.15.39:0/2673422270 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc06a070 con 0x5557564db1b0 2024-07-04T08:31:21.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57c5600640 1 -- 172.21.15.39:0/2673422270 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f57cc004960 con 0x5557564db1b0 2024-07-04T08:31:21.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.193+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2673422270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57cc005200 con 0x5557564db1b0 2024-07-04T08:31:21.185 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.194+0000 7f57c5600640 1 -- 172.21.15.39:0/2673422270 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57cc04d040 con 0x5557564db1b0 2024-07-04T08:31:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.194+0000 7f57e8ba8740 10 client.29051 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.194+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2673422270 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555756937000 con 0x5557564db1b0 2024-07-04T08:31:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.194+0000 7f57c5600640 1 -- 172.21.15.39:0/2673422270 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57cc076070 con 0x5557564db1b0 2024-07-04T08:31:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.195+0000 7f57c5600640 1 -- 172.21.15.39:0/2673422270 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f57cc066950 con 0x5557564db1b0 2024-07-04T08:31:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.195+0000 7f57e8ba8740 10 client.29051 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.195+0000 7f57e8ba8740 10 client.29051 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:31:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.195+0000 7f57e8ba8740 20 client.29051 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.195+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555756497da0 0x555756939720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:21.186 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.195+0000 7f57e8ba8740 4 client.29051 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:31:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.195+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2673422270 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55575681c8d0 con 0x555756497da0 2024-07-04T08:31:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.195+0000 7f57daa00640 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555756497da0 0x555756939720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57daa00640 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555756497da0 0x555756939720 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33ab3d2d0fce6e4e server_cookie=a2c5fdd4de21d114 in_seq=0 out_seq=0 2024-07-04T08:31:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57c5600640 10 client.29051 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:31:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57c5600640 1 -- 172.21.15.39:0/2673422270 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55575681c8d0 con 0x555756497da0 2024-07-04T08:31:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57c5600640 10 client.29051 handle_command_reply: tid=0 2024-07-04T08:31:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57e8ba8740 1 client.29051 shutdown 2024-07-04T08:31:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2673422270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564db1b0 msgr2=0x55575692f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:21.187 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564db1b0 0x55575692f850 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f57cc005930 tx=0x7f57cc004c70 comp rx=0 tx=0).stop 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2673422270 shutdown_connections 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x555756497da0 0x555756939720 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5557564e3390 0x55575692fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555756504070 0x5557568b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57e8ba8740 1 --2- 172.21.15.39:0/2673422270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5557564db1b0 0x55575692f850 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2673422270 >> 172.21.15.39:0/2673422270 conn(0x5557564f8460 msgr2=0x5557568a67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.196+0000 7f57c5600640 0 client.29051 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.197+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2673422270 shutdown_connections 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.197+0000 7f57e8ba8740 1 -- 172.21.15.39:0/2673422270 wait complete. 2024-07-04T08:31:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.197+0000 7f57e8ba8740 20 client.29051 trim_cache size 0 max 16384 2024-07-04T08:31:21.189 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:31:21.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 -- 172.21.15.39:0/2242112710 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c00b29e0 msgr2=0x7f57c00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:21.189 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c00b29e0 0x7f57c00b4ef0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f57cc031440 tx=0x7f57cc009560 comp rx=0 tx=0).stop 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 -- 172.21.15.39:0/2242112710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc108840 msgr2=0x7f57dc19c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc108840 0x7f57dc19c000 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f57c800d950 tx=0x7f57c800de10 comp rx=0 tx=0).stop 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 -- 172.21.15.39:0/2242112710 shutdown_connections 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f57c00b29e0 0x7f57c00b4ef0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57dc109190 0x7f57dc1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f57dc108840 0x7f57dc19c000 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 --2- 172.21.15.39:0/2242112710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc102f00 0x7f57dc19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 -- 172.21.15.39:0/2242112710 >> 172.21.15.39:0/2242112710 conn(0x7f57dc0fe990 msgr2=0x7f57dc10dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 -- 172.21.15.39:0/2242112710 shutdown_connections 2024-07-04T08:31:21.190 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:21.198+0000 7f57e1000640 1 -- 172.21.15.39:0/2242112710 wait complete. 2024-07-04T08:31:21.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:21 smithi133 ceph-mon[37050]: pgmap v984: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:21.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:21 smithi039 ceph-mon[32062]: pgmap v984: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:21.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:21 smithi050 ceph-mon[36577]: pgmap v984: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:22.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:31:22.605 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:23.579 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:23 smithi039 ceph-mon[32062]: pgmap v985: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 15 op/s 2024-07-04T08:31:23.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:23 smithi050 ceph-mon[36577]: pgmap v985: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 15 op/s 2024-07-04T08:31:23.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:23 smithi133 ceph-mon[37050]: pgmap v985: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 15 op/s 2024-07-04T08:31:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.862+0000 7f519e600640 1 -- 172.21.15.39:0/1489991332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f519810b3a0 msgr2=0x7f5198111bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.862+0000 7f519e600640 1 --2- 172.21.15.39:0/1489991332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f519810b3a0 0x7f5198111bd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5190009be0 tx=0x7f519002a6c0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.862+0000 7f519e600640 1 -- 172.21.15.39:0/1489991332 shutdown_connections 2024-07-04T08:31:24.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.862+0000 7f519e600640 1 --2- 172.21.15.39:0/1489991332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519810aa50 0x7f519810ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.862+0000 7f519e600640 1 --2- 172.21.15.39:0/1489991332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f519810b3a0 0x7f5198111bd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.862+0000 7f519e600640 1 --2- 172.21.15.39:0/1489991332 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5198105110 0x7f5198105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.862+0000 7f519e600640 1 -- 172.21.15.39:0/1489991332 >> 172.21.15.39:0/1489991332 conn(0x7f5198100b60 msgr2=0x7f5198102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.862+0000 7f519e600640 1 -- 172.21.15.39:0/1489991332 shutdown_connections 2024-07-04T08:31:24.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.862+0000 7f519e600640 1 -- 172.21.15.39:0/1489991332 wait complete. 2024-07-04T08:31:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.863+0000 7f519e600640 1 Processor -- start 2024-07-04T08:31:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.863+0000 7f519e600640 1 -- start start 2024-07-04T08:31:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519e600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5198105110 0x7f519819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f519810aa50 0x7f519819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519e600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519810b3a0 0x7f51981a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519e600640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5198115070 con 0x7f519810b3a0 2024-07-04T08:31:24.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519e600640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5198114ef0 con 0x7f5198105110 2024-07-04T08:31:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51981151f0 con 0x7f519810aa50 2024-07-04T08:31:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5198105110 0x7f519819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5198105110 0x7f519819b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33352/0 (socket says 172.21.15.39:33352) 2024-07-04T08:31:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519810b3a0 0x7f51981a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f519d200640 1 -- 172.21.15.39:0/4139518476 learned_addr learned my addr 172.21.15.39:0/4139518476 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:24.853 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.864+0000 7f5197e00640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f519810aa50 0x7f519819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.865+0000 7f519d200640 1 -- 172.21.15.39:0/4139518476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f519810aa50 msgr2=0x7f519819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.865+0000 7f519d200640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f519810aa50 0x7f519819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.865+0000 7f519d200640 1 -- 172.21.15.39:0/4139518476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519810b3a0 msgr2=0x7f51981a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.865+0000 7f519d200640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519810b3a0 0x7f51981a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.865+0000 7f519d200640 1 -- 172.21.15.39:0/4139518476 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51981a1950 con 0x7f5198105110 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.865+0000 7f519d200640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5198105110 0x7f519819b9e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e9f0 tx=0x7f518c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.865+0000 7f5195600640 1 -- 172.21.15.39:0/4139518476 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c00cd80 con 0x7f5198105110 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.866+0000 7f5195600640 1 -- 172.21.15.39:0/4139518476 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518c0054d0 con 0x7f5198105110 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.866+0000 7f5195600640 1 -- 172.21.15.39:0/4139518476 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c010690 con 0x7f5198105110 2024-07-04T08:31:24.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.866+0000 7f519e600640 1 -- 172.21.15.39:0/4139518476 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51981a1c40 con 0x7f5198105110 2024-07-04T08:31:24.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.866+0000 7f519e600640 1 -- 172.21.15.39:0/4139518476 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51981a83a0 con 0x7f5198105110 2024-07-04T08:31:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.869+0000 7f51a625e740 1 Processor -- start 2024-07-04T08:31:24.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.869+0000 7f51a625e740 1 -- start start 2024-07-04T08:31:24.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f51a625e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bf3070 0x5627c2f97950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f51a625e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2f97e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f51a625e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2f983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f51a625e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5627c2f0dee0 con 0x5627c2bf3070 2024-07-04T08:31:24.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f51a625e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5627c2efd480 con 0x5627c2bca1b0 2024-07-04T08:31:24.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f51a625e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5627c2f014c0 con 0x5627c2bd2390 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f5195600640 1 -- 172.21.15.39:0/4139518476 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f518c004e90 con 0x7f5198105110 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f5197e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2f983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f519d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2f97e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f5197e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2f983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:38616/0 (socket says 172.21.15.39:38616) 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f5197e00640 1 -- 172.21.15.39:0/2585030149 learned_addr learned my addr 172.21.15.39:0/2585030149 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.870+0000 7f519dc00640 1 --2- 172.21.15.39:0/2585030149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bf3070 0x5627c2f97950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.871+0000 7f5197e00640 1 -- 172.21.15.39:0/2585030149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 msgr2=0x5627c2f97e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.871+0000 7f5197e00640 1 --2- 172.21.15.39:0/2585030149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2f97e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.871+0000 7f5197e00640 1 -- 172.21.15.39:0/2585030149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bf3070 msgr2=0x5627c2f97950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:24.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.871+0000 7f5197e00640 1 --2- 172.21.15.39:0/2585030149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bf3070 0x5627c2f97950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.871+0000 7f5197e00640 1 -- 172.21.15.39:0/2585030149 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c2e95650 con 0x5627c2bd2390 2024-07-04T08:31:24.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.871+0000 7f5197e00640 1 --2- 172.21.15.39:0/2585030149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2f983d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5184009c30 tx=0x7f5184011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e64a6ab8a62f505 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:24.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.872+0000 7f518a000640 1 -- 172.21.15.39:0/2585030149 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184004020 con 0x5627c2bd2390 2024-07-04T08:31:24.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.872+0000 7f518a000640 1 -- 172.21.15.39:0/2585030149 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51840041c0 con 0x5627c2bd2390 2024-07-04T08:31:24.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.872+0000 7f518a000640 1 -- 172.21.15.39:0/2585030149 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184019740 con 0x5627c2bd2390 2024-07-04T08:31:24.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.872+0000 7f5195600640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51800b29e0 0x7f51800b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.872+0000 7f5195600640 1 -- 172.21.15.39:0/4139518476 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f518c0e2dd0 con 0x7f5198105110 2024-07-04T08:31:24.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f519dc00640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51800b29e0 0x7f51800b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f51a625e740 1 -- 172.21.15.39:0/2585030149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 msgr2=0x5627c2f983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:24.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f51a625e740 1 --2- 172.21.15.39:0/2585030149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2f983d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5184009c30 tx=0x7f5184011810 comp rx=0 tx=0).stop 2024-07-04T08:31:24.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f51a625e740 1 -- 172.21.15.39:0/2585030149 shutdown_connections 2024-07-04T08:31:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f51a625e740 1 --2- 172.21.15.39:0/2585030149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2f983d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f51a625e740 1 --2- 172.21.15.39:0/2585030149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2f97e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f51a625e740 1 --2- 172.21.15.39:0/2585030149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bf3070 0x5627c2f97950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f51a625e740 1 -- 172.21.15.39:0/2585030149 >> 172.21.15.39:0/2585030149 conn(0x5627c2be7460 msgr2=0x5627c2f95840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f51a625e740 1 -- 172.21.15.39:0/2585030149 shutdown_connections 2024-07-04T08:31:24.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.873+0000 7f51a625e740 1 -- 172.21.15.39:0/2585030149 wait complete. 2024-07-04T08:31:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.874+0000 7f519dc00640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51800b29e0 0x7f51800b4ef0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5190004af0 tx=0x7f5190005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.874+0000 7f51a625e740 1 -- start start 2024-07-04T08:31:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.874+0000 7f51a625e740 1 Processor -- start 2024-07-04T08:31:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f51a625e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2fa3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f51a625e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2fa39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f51a625e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bf3070 0x5627c2faf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f51a625e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5627c2d61700 con 0x5627c2bca1b0 2024-07-04T08:31:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f51a625e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5627c2ea4bb0 con 0x5627c2bf3070 2024-07-04T08:31:24.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f51a625e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5627c2fa6130 con 0x5627c2bd2390 2024-07-04T08:31:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f51a625e740 10 client.0 _ll_register_callbacks cb 0x5627c2ef6570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f519d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2fa3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f519dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bf3070 0x5627c2faf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f519d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2fa3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56256/0 (socket says 172.21.15.39:56256) 2024-07-04T08:31:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f5197e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2fa39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f519dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bf3070 0x5627c2faf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33368/0 (socket says 172.21.15.39:33368) 2024-07-04T08:31:24.864 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.875+0000 7f519d200640 1 -- 172.21.15.39:0/1233036098 learned_addr learned my addr 172.21.15.39:0/1233036098 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.876+0000 7f519d200640 1 -- 172.21.15.39:0/1233036098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 msgr2=0x5627c2fa39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.876+0000 7f519d200640 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2fa39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.876+0000 7f519d200640 1 -- 172.21.15.39:0/1233036098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bf3070 msgr2=0x5627c2faf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.876+0000 7f519d200640 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bf3070 0x5627c2faf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.876+0000 7f519d200640 1 -- 172.21.15.39:0/1233036098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c2c5e650 con 0x5627c2bca1b0 2024-07-04T08:31:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.876+0000 7f519d200640 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2fa3490 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e9c0 tx=0x7f518c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34437ea9a58e48a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.876+0000 7f517ea00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:31:24.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.876+0000 7f517ea00640 1 -- 172.21.15.39:0/1233036098 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c0a9640 con 0x5627c2bca1b0 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.876+0000 7f517ea00640 1 -- 172.21.15.39:0/1233036098 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f518c0b28d0 con 0x5627c2bca1b0 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.877+0000 7f51a625e740 1 -- 172.21.15.39:0/1233036098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51840041c0 con 0x5627c2bca1b0 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.877+0000 7f517ea00640 1 -- 172.21.15.39:0/1233036098 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f518c005180 con 0x5627c2bca1b0 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.877+0000 7f51a625e740 10 client.18939 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.877+0000 7f51a625e740 1 -- 172.21.15.39:0/1233036098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627c2fb20d0 con 0x5627c2bca1b0 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.877+0000 7f517ea00640 1 -- 172.21.15.39:0/1233036098 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51840041c0 con 0x5627c2bca1b0 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.877+0000 7f517ea00640 1 -- 172.21.15.39:0/1233036098 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f518c0a5660 con 0x5627c2bca1b0 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.878+0000 7f51a625e740 10 client.18939 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.878+0000 7f51a625e740 10 client.18939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.878+0000 7f51a625e740 20 client.18939 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.878+0000 7f51a625e740 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5627c2bb19f0 0x5627c2fb5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.878+0000 7f51a625e740 4 client.18939 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.878+0000 7f51a625e740 1 -- 172.21.15.39:0/1233036098 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627c2f0fe30 con 0x5627c2bb19f0 2024-07-04T08:31:24.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.878+0000 7f5197e00640 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5627c2bb19f0 0x5627c2fb5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.879+0000 7f5197e00640 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5627c2bb19f0 0x5627c2fb5600 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12d5b992cfbe3277 server_cookie=b33a35649828a195 in_seq=0 out_seq=0 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.879+0000 7f517ea00640 10 client.18939 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f517ea00640 1 -- 172.21.15.39:0/1233036098 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5627c2f0fe30 con 0x5627c2bb19f0 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f517ea00640 10 client.18939 handle_command_reply: tid=0 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 client.18939 shutdown 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 -- 172.21.15.39:0/1233036098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 msgr2=0x5627c2fa3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2fa3490 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e9c0 tx=0x7f518c00cf20 comp rx=0 tx=0).stop 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 -- 172.21.15.39:0/1233036098 shutdown_connections 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5627c2bb19f0 0x5627c2fb5600 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bf3070 0x5627c2faf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bd2390 0x5627c2fa39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 --2- 172.21.15.39:0/1233036098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2fa3490 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 -- 172.21.15.39:0/1233036098 >> 172.21.15.39:0/1233036098 conn(0x5627c2be7460 msgr2=0x5627c2f9ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f517ea00640 0 client.18939 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:31:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 -- 172.21.15.39:0/1233036098 shutdown_connections 2024-07-04T08:31:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.880+0000 7f51a625e740 1 -- 172.21.15.39:0/1233036098 wait complete. 2024-07-04T08:31:24.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.881+0000 7f51a625e740 20 client.18939 trim_cache size 0 max 16384 2024-07-04T08:31:24.883 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.896+0000 7f51a625e740 1 Processor -- start 2024-07-04T08:31:24.884 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:24 smithi039 ceph-mon[32062]: pgmap v986: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 15 op/s 2024-07-04T08:31:24.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.896+0000 7f51a625e740 1 -- start start 2024-07-04T08:31:24.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.896+0000 7f51a625e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2f9d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.896+0000 7f51a625e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bd2390 0x5627c2faf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f51a625e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 0x5627c2faf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f51a625e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b2380 con 0x5627c2bca1b0 2024-07-04T08:31:24.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f51a625e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b3ec0 con 0x5627c2bd2390 2024-07-04T08:31:24.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f51a625e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b2a50 con 0x5627c2bf3070 2024-07-04T08:31:24.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f519d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2f9d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f5197e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bd2390 0x5627c2faf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f5197e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bd2390 0x5627c2faf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33372/0 (socket says 172.21.15.39:33372) 2024-07-04T08:31:24.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f519d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2f9d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56264/0 (socket says 172.21.15.39:56264) 2024-07-04T08:31:24.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f5197e00640 1 -- 172.21.15.39:0/3405084351 learned_addr learned my addr 172.21.15.39:0/3405084351 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:24.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.897+0000 7f519dc00640 1 --2- 172.21.15.39:0/3405084351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 0x5627c2faf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.898+0000 7f5197e00640 1 -- 172.21.15.39:0/3405084351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 msgr2=0x5627c2faf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:24.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.898+0000 7f5197e00640 1 --2- 172.21.15.39:0/3405084351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 0x5627c2faf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.898+0000 7f5197e00640 1 -- 172.21.15.39:0/3405084351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 msgr2=0x5627c2f9d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:24.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.898+0000 7f5197e00640 1 --2- 172.21.15.39:0/3405084351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2f9d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.886 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.898+0000 7f5197e00640 1 -- 172.21.15.39:0/3405084351 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c2f90850 con 0x5627c2bd2390 2024-07-04T08:31:24.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.898+0000 7f5197e00640 1 --2- 172.21.15.39:0/3405084351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bd2390 0x5627c2faf210 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5184004110 tx=0x7f5184005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76955906b3bc12e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:24.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.899+0000 7f5189600640 1 -- 172.21.15.39:0/3405084351 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184009e60 con 0x5627c2bd2390 2024-07-04T08:31:24.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.899+0000 7f5189600640 1 -- 172.21.15.39:0/3405084351 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51840054f0 con 0x5627c2bd2390 2024-07-04T08:31:24.887 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.899+0000 7f5189600640 1 -- 172.21.15.39:0/3405084351 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51840037b0 con 0x5627c2bd2390 2024-07-04T08:31:24.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.900+0000 7f51a625e740 1 -- 172.21.15.39:0/3405084351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bd2390 msgr2=0x5627c2faf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:24.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.900+0000 7f51a625e740 1 --2- 172.21.15.39:0/3405084351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bd2390 0x5627c2faf210 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5184004110 tx=0x7f5184005e80 comp rx=0 tx=0).stop 2024-07-04T08:31:24.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.900+0000 7f51a625e740 1 -- 172.21.15.39:0/3405084351 shutdown_connections 2024-07-04T08:31:24.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.900+0000 7f51a625e740 1 --2- 172.21.15.39:0/3405084351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 0x5627c2faf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.900+0000 7f51a625e740 1 --2- 172.21.15.39:0/3405084351 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bd2390 0x5627c2faf210 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.900+0000 7f51a625e740 1 --2- 172.21.15.39:0/3405084351 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bca1b0 0x5627c2f9d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.900+0000 7f51a625e740 1 -- 172.21.15.39:0/3405084351 >> 172.21.15.39:0/3405084351 conn(0x5627c2be7460 msgr2=0x5627c2f9ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:24.888 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.900+0000 7f51a625e740 1 -- 172.21.15.39:0/3405084351 shutdown_connections 2024-07-04T08:31:24.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.900+0000 7f51a625e740 1 -- 172.21.15.39:0/3405084351 wait complete. 2024-07-04T08:31:24.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.901+0000 7f51a625e740 1 -- start start 2024-07-04T08:31:24.889 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.901+0000 7f51a625e740 1 Processor -- start 2024-07-04T08:31:24.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f51a625e740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2fab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f51a625e740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bd2390 0x5627c2fa6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f51a625e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 0x5627c2fa7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f51a625e740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b37f0 con 0x5627c2bd2390 2024-07-04T08:31:24.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f51a625e740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b3120 con 0x5627c2bca1b0 2024-07-04T08:31:24.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f51a625e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b0360 con 0x5627c2bf3070 2024-07-04T08:31:24.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f51a625e740 10 client.0 _ll_register_callbacks cb 0x7f518c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:24.890 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f519d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2fab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f519d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2fab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33382/0 (socket says 172.21.15.39:33382) 2024-07-04T08:31:24.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f519d200640 1 -- 172.21.15.39:0/3923371172 learned_addr learned my addr 172.21.15.39:0/3923371172 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:24.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f5197e00640 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bd2390 0x5627c2fa6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f519dc00640 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 0x5627c2fa7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f519d200640 1 -- 172.21.15.39:0/3923371172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 msgr2=0x5627c2fa7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:24.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f519d200640 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 0x5627c2fa7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.891 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f519d200640 1 -- 172.21.15.39:0/3923371172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bd2390 msgr2=0x5627c2fa6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.902+0000 7f519d200640 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bd2390 0x5627c2fa6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.903+0000 7f519d200640 1 -- 172.21.15.39:0/3923371172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c2fa7940 con 0x5627c2bca1b0 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.903+0000 7f519d200640 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2fab910 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e9c0 tx=0x7f518c0b2ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45150cbfbdd293e server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.903+0000 7f517ea00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.903+0000 7f517ea00640 1 -- 172.21.15.39:0/3923371172 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c0a9460 con 0x5627c2bca1b0 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.903+0000 7f51a625e740 1 -- 172.21.15.39:0/3923371172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51840054f0 con 0x5627c2bca1b0 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.903+0000 7f517ea00640 1 -- 172.21.15.39:0/3923371172 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f518c0a62e0 con 0x5627c2bca1b0 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f517ea00640 1 -- 172.21.15.39:0/3923371172 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f518c0c43d0 con 0x5627c2bca1b0 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f51a625e740 10 client.29069 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:24.892 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f51a625e740 1 -- 172.21.15.39:0/3923371172 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627c3026390 con 0x5627c2bca1b0 2024-07-04T08:31:24.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f517ea00640 1 -- 172.21.15.39:0/3923371172 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51840054f0 con 0x5627c2bca1b0 2024-07-04T08:31:24.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f517ea00640 1 -- 172.21.15.39:0/3923371172 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f518c0af230 con 0x5627c2bca1b0 2024-07-04T08:31:24.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f51a625e740 10 client.29069 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:24.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f51a625e740 10 client.29069 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:31:24.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f51a625e740 20 client.29069 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:24.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f51a625e740 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5627c2b86da0 0x5627c30289c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:24.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f51a625e740 4 client.29069 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:31:24.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.904+0000 7f51a625e740 1 -- 172.21.15.39:0/3923371172 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627c2f0b8d0 con 0x5627c2b86da0 2024-07-04T08:31:24.893 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.905+0000 7f5197e00640 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5627c2b86da0 0x5627c30289c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.905+0000 7f5197e00640 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5627c2b86da0 0x5627c30289c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bcf8778dc51cacf server_cookie=ed59d2db12e0c9e7 in_seq=0 out_seq=0 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.905+0000 7f517ea00640 10 client.29069 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f517ea00640 1 -- 172.21.15.39:0/3923371172 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627c2f0b8d0 con 0x5627c2b86da0 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f517ea00640 10 client.29069 handle_command_reply: tid=0 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f51a625e740 1 client.29069 shutdown 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f51a625e740 1 -- 172.21.15.39:0/3923371172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 msgr2=0x5627c2fab910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f51a625e740 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2fab910 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e9c0 tx=0x7f518c0b2ba0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f51a625e740 1 -- 172.21.15.39:0/3923371172 shutdown_connections 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f51a625e740 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x5627c2b86da0 0x5627c30289c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f51a625e740 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5627c2bd2390 0x5627c2fa6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f51a625e740 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5627c2bf3070 0x5627c2fa7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.895 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f517ea00640 0 client.29069 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:31:24.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f51a625e740 1 --2- 172.21.15.39:0/3923371172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5627c2bca1b0 0x5627c2fab910 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.906+0000 7f51a625e740 1 -- 172.21.15.39:0/3923371172 >> 172.21.15.39:0/3923371172 conn(0x5627c2be7460 msgr2=0x5627c2fa4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:24.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.907+0000 7f51a625e740 1 -- 172.21.15.39:0/3923371172 shutdown_connections 2024-07-04T08:31:24.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.907+0000 7f51a625e740 1 -- 172.21.15.39:0/3923371172 wait complete. 2024-07-04T08:31:24.896 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.907+0000 7f51a625e740 20 client.29069 trim_cache size 0 max 16384 2024-07-04T08:31:24.896 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.907+0000 7f519e600640 1 -- 172.21.15.39:0/4139518476 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51800b29e0 msgr2=0x7f51800b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.907+0000 7f519e600640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51800b29e0 0x7f51800b4ef0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5190004af0 tx=0x7f5190005fd0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.907+0000 7f519e600640 1 -- 172.21.15.39:0/4139518476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5198105110 msgr2=0x7f519819b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.907+0000 7f519e600640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5198105110 0x7f519819b9e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f518c00e9f0 tx=0x7f518c00edf0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.908+0000 7f519e600640 1 -- 172.21.15.39:0/4139518476 shutdown_connections 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.908+0000 7f519e600640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51800b29e0 0x7f51800b4ef0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.908+0000 7f519e600640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519810b3a0 0x7f51981a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.908+0000 7f519e600640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f519810aa50 0x7f519819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.908+0000 7f519e600640 1 --2- 172.21.15.39:0/4139518476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5198105110 0x7f519819b9e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:24.897 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.908+0000 7f519e600640 1 -- 172.21.15.39:0/4139518476 >> 172.21.15.39:0/4139518476 conn(0x7f5198100b60 msgr2=0x7f5198102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:24.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.908+0000 7f519e600640 1 -- 172.21.15.39:0/4139518476 shutdown_connections 2024-07-04T08:31:24.898 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:24.908+0000 7f519e600640 1 -- 172.21.15.39:0/4139518476 wait complete. 2024-07-04T08:31:25.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:24 smithi050 ceph-mon[36577]: pgmap v986: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 15 op/s 2024-07-04T08:31:25.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:24 smithi133 ceph-mon[37050]: pgmap v986: 129 pgs: 129 active+clean; 31 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 76 KiB/s wr, 15 op/s 2024-07-04T08:31:25.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:31:26.318 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:27.518 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:27 smithi039 ceph-mon[32062]: pgmap v987: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-07-04T08:31:27.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:27 smithi133 ceph-mon[37050]: pgmap v987: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-07-04T08:31:27.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:27 smithi050 ceph-mon[36577]: pgmap v987: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-07-04T08:31:28.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.843+0000 7f9282800640 1 -- 172.21.15.39:0/2999306149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f927c1030b0 msgr2=0x7f927c1034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:28.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.843+0000 7f9282800640 1 --2- 172.21.15.39:0/2999306149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f927c1030b0 0x7f927c1034c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f926c009900 tx=0x7f926c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.843+0000 7f9282800640 1 -- 172.21.15.39:0/2999306149 shutdown_connections 2024-07-04T08:31:28.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.843+0000 7f9282800640 1 --2- 172.21.15.39:0/2999306149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c109340 0x7f927c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.843+0000 7f9282800640 1 --2- 172.21.15.39:0/2999306149 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f927c1089f0 0x7f927c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.843+0000 7f9282800640 1 --2- 172.21.15.39:0/2999306149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f927c1030b0 0x7f927c1034c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.844+0000 7f9282800640 1 -- 172.21.15.39:0/2999306149 >> 172.21.15.39:0/2999306149 conn(0x7f927c07ada0 msgr2=0x7f927c07b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:28.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.844+0000 7f9282800640 1 -- 172.21.15.39:0/2999306149 shutdown_connections 2024-07-04T08:31:28.833 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.844+0000 7f9282800640 1 -- 172.21.15.39:0/2999306149 wait complete. 2024-07-04T08:31:28.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.845+0000 7f9282800640 1 Processor -- start 2024-07-04T08:31:28.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.845+0000 7f9282800640 1 -- start start 2024-07-04T08:31:28.834 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f9282800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1030b0 0x7f927c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f9282800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f927c1089f0 0x7f927c19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f9282800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f927c109340 0x7f927c1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f9282800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f927c113040 con 0x7f927c109340 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f9282800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f927c112ec0 con 0x7f927c1030b0 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f9282800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1131c0 con 0x7f927c1089f0 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f927b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1030b0 0x7f927c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f927b400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1030b0 0x7f927c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33406/0 (socket says 172.21.15.39:33406) 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f927b400640 1 -- 172.21.15.39:0/1739189831 learned_addr learned my addr 172.21.15.39:0/1739189831 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f9273e00640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f927c1089f0 0x7f927c19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f927be00640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f927c109340 0x7f927c1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f927b400640 1 -- 172.21.15.39:0/1739189831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f927c1089f0 msgr2=0x7f927c19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:28.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.846+0000 7f927b400640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f927c1089f0 0x7f927c19c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f927b400640 1 -- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f927c109340 msgr2=0x7f927c1a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:28.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f927b400640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f927c109340 0x7f927c1a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f927b400640 1 -- 172.21.15.39:0/1739189831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c1a1c20 con 0x7f927c1030b0 2024-07-04T08:31:28.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f9273e00640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f927c1089f0 0x7f927c19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:28.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f927b400640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1030b0 0x7f927c19bcb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f926c005ab0 tx=0x7f926c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:28.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f9278c00640 1 -- 172.21.15.39:0/1739189831 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c03f070 con 0x7f927c1030b0 2024-07-04T08:31:28.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f9278c00640 1 -- 172.21.15.39:0/1739189831 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c031b60 con 0x7f927c1030b0 2024-07-04T08:31:28.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f9278c00640 1 -- 172.21.15.39:0/1739189831 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c035030 con 0x7f927c1030b0 2024-07-04T08:31:28.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f9282800640 1 -- 172.21.15.39:0/1739189831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c1a1eb0 con 0x7f927c1030b0 2024-07-04T08:31:28.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.847+0000 7f9282800640 1 -- 172.21.15.39:0/1739189831 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c1a8670 con 0x7f927c1030b0 2024-07-04T08:31:28.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.851+0000 7f9288d86740 1 Processor -- start 2024-07-04T08:31:28.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.851+0000 7f9288d86740 1 -- start start 2024-07-04T08:31:28.839 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.851+0000 7f9278c00640 1 -- 172.21.15.39:0/1739189831 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f926c042070 con 0x7f927c1030b0 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.851+0000 7f9288d86740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98df7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.851+0000 7f9288d86740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 0x562df98dfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f9288d86740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df951a390 0x562df98e0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f9288d86740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562df9855ee0 con 0x562df951a390 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f9288d86740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562df9845480 con 0x562df95121b0 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f9288d86740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562df98494c0 con 0x562df953b070 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f927be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98df7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f927be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98df7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:38694/0 (socket says 172.21.15.39:38694) 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f927be00640 1 -- 172.21.15.39:0/944900231 learned_addr learned my addr 172.21.15.39:0/944900231 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:28.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f927b400640 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 0x562df98dfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f9273e00640 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df951a390 0x562df98e0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f927be00640 1 -- 172.21.15.39:0/944900231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 msgr2=0x562df98dfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f927be00640 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 0x562df98dfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f927be00640 1 -- 172.21.15.39:0/944900231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df951a390 msgr2=0x562df98e0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f927be00640 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df951a390 0x562df98e0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.852+0000 7f927be00640 1 -- 172.21.15.39:0/944900231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df97dd650 con 0x562df953b070 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.853+0000 7f9273e00640 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df951a390 0x562df98e0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.853+0000 7f927be00640 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98df7b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9274009f50 tx=0x7f92740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d3915b1c8a50d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.853+0000 7f9270c00640 1 -- 172.21.15.39:0/944900231 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274004d20 con 0x562df953b070 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.853+0000 7f9270c00640 1 -- 172.21.15.39:0/944900231 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9274004050 con 0x562df953b070 2024-07-04T08:31:28.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.853+0000 7f9270c00640 1 -- 172.21.15.39:0/944900231 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274005280 con 0x562df953b070 2024-07-04T08:31:28.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f9278c00640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92600b29e0 0x7f92600b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f9288d86740 1 -- 172.21.15.39:0/944900231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 msgr2=0x562df98df7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:28.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f9288d86740 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98df7b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9274009f50 tx=0x7f92740118d0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f9288d86740 1 -- 172.21.15.39:0/944900231 shutdown_connections 2024-07-04T08:31:28.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f9288d86740 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df951a390 0x562df98e0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f9288d86740 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 0x562df98dfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f9288d86740 1 --2- 172.21.15.39:0/944900231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98df7b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f9288d86740 1 -- 172.21.15.39:0/944900231 >> 172.21.15.39:0/944900231 conn(0x562df952f460 msgr2=0x562df98dd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:28.845 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f9278c00640 1 -- 172.21.15.39:0/1739189831 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f926c103070 con 0x7f927c1030b0 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.854+0000 7f927b400640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92600b29e0 0x7f92600b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.855+0000 7f9288d86740 1 -- 172.21.15.39:0/944900231 shutdown_connections 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.855+0000 7f9288d86740 1 -- 172.21.15.39:0/944900231 wait complete. 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.855+0000 7f927b400640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92600b29e0 0x7f92600b4ef0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x562df98de430 tx=0x7f926c106b20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.855+0000 7f9288d86740 1 -- start start 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.856+0000 7f9288d86740 1 Processor -- start 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.856+0000 7f9288d86740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 0x562df98eb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.856+0000 7f9288d86740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df951a390 0x562df98eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.856+0000 7f9288d86740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.856+0000 7f9288d86740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x562df96a9700 con 0x562df95121b0 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.856+0000 7f9288d86740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x562df97ecbb0 con 0x562df951a390 2024-07-04T08:31:28.846 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.856+0000 7f9288d86740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562df98edf90 con 0x562df953b070 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.856+0000 7f9288d86740 10 client.0 _ll_register_callbacks cb 0x562df983e570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.856+0000 7f927be00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 0x562df98eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9273e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df951a390 0x562df98eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9273e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df951a390 0x562df98eb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33414/0 (socket says 172.21.15.39:33414) 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9273e00640 1 -- 172.21.15.39:0/2047162653 learned_addr learned my addr 172.21.15.39:0/2047162653 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f927be00640 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9273e00640 1 -- 172.21.15.39:0/2047162653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 msgr2=0x562df98f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9273e00640 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9273e00640 1 -- 172.21.15.39:0/2047162653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 msgr2=0x562df98eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9273e00640 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 0x562df98eb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9273e00640 1 -- 172.21.15.39:0/2047162653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df95a6650 con 0x562df951a390 2024-07-04T08:31:28.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f927be00640 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9273e00640 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df951a390 0x562df98eb890 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9264013790 tx=0x7f9264013c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72d4e4ada290a0af server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.857+0000 7f9269600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.858+0000 7f9269600640 1 -- 172.21.15.39:0/2047162653 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264020060 con 0x562df951a390 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.858+0000 7f9288d86740 1 -- 172.21.15.39:0/2047162653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9274004050 con 0x562df951a390 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.858+0000 7f9269600640 1 -- 172.21.15.39:0/2047162653 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f9264011dd0 con 0x562df951a390 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.858+0000 7f9269600640 1 -- 172.21.15.39:0/2047162653 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9264027050 con 0x562df951a390 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.858+0000 7f9288d86740 10 client.29081 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.858+0000 7f9288d86740 1 -- 172.21.15.39:0/2047162653 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562df98fa0b0 con 0x562df951a390 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.859+0000 7f9269600640 1 -- 172.21.15.39:0/2047162653 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9274004050 con 0x562df951a390 2024-07-04T08:31:28.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.859+0000 7f9269600640 1 -- 172.21.15.39:0/2047162653 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f926401a030 con 0x562df951a390 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.859+0000 7f9288d86740 10 client.29081 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.859+0000 7f9288d86740 10 client.29081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.859+0000 7f9288d86740 20 client.29081 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.859+0000 7f9288d86740 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562df94f99f0 0x562df98fd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.859+0000 7f9288d86740 4 client.29081 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.859+0000 7f9288d86740 1 -- 172.21.15.39:0/2047162653 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562df9857e30 con 0x562df94f99f0 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.860+0000 7f927be00640 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562df94f99f0 0x562df98fd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.860+0000 7f927be00640 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562df94f99f0 0x562df98fd540 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa8ab5b6c717a660 server_cookie=fdca8082fc500554 in_seq=0 out_seq=0 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.860+0000 7f9269600640 10 client.29081 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9269600640 1 -- 172.21.15.39:0/2047162653 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562df9857e30 con 0x562df94f99f0 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9269600640 10 client.29081 handle_command_reply: tid=0 2024-07-04T08:31:28.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9288d86740 1 client.29081 shutdown 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9288d86740 1 -- 172.21.15.39:0/2047162653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df951a390 msgr2=0x562df98eb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9288d86740 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df951a390 0x562df98eb890 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9264013790 tx=0x7f9264013c50 comp rx=0 tx=0).stop 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9288d86740 1 -- 172.21.15.39:0/2047162653 shutdown_connections 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9288d86740 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562df94f99f0 0x562df98fd540 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9288d86740 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df953b070 0x562df98f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9288d86740 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df951a390 0x562df98eb890 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9288d86740 1 --2- 172.21.15.39:0/2047162653 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 0x562df98eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9269600640 0 client.29081 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.861+0000 7f9288d86740 1 -- 172.21.15.39:0/2047162653 >> 172.21.15.39:0/2047162653 conn(0x562df952f460 msgr2=0x562df98e2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.862+0000 7f9288d86740 1 -- 172.21.15.39:0/2047162653 shutdown_connections 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.862+0000 7f9288d86740 1 -- 172.21.15.39:0/2047162653 wait complete. 2024-07-04T08:31:28.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.862+0000 7f9288d86740 20 client.29081 trim_cache size 0 max 16384 2024-07-04T08:31:28.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.877+0000 7f9288d86740 1 Processor -- start 2024-07-04T08:31:28.865 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.878+0000 7f9288d86740 1 -- start start 2024-07-04T08:31:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.878+0000 7f9288d86740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 0x562df98e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.878+0000 7f9288d86740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df98dc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.878+0000 7f9288d86740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df953b070 0x562df98f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.878+0000 7f9288d86740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1b26c0 con 0x562df953b070 2024-07-04T08:31:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.878+0000 7f9288d86740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1b4200 con 0x562df95121b0 2024-07-04T08:31:28.866 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.878+0000 7f9288d86740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1b2d90 con 0x562df951a390 2024-07-04T08:31:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.878+0000 7f927be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 0x562df98e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.879+0000 7f9273e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df98dc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.879+0000 7f927be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 0x562df98e2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:33428/0 (socket says 172.21.15.39:33428) 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.879+0000 7f927be00640 1 -- 172.21.15.39:0/3064952010 learned_addr learned my addr 172.21.15.39:0/3064952010 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.879+0000 7f927be00640 1 --2- 172.21.15.39:0/3064952010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df953b070 0x562df98f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.879+0000 7f9273e00640 1 -- 172.21.15.39:0/3064952010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 msgr2=0x562df98e2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.879+0000 7f9273e00640 1 --2- 172.21.15.39:0/3064952010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 0x562df98e2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.879+0000 7f9273e00640 1 -- 172.21.15.39:0/3064952010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df953b070 msgr2=0x562df98f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.879+0000 7f9273e00640 1 --2- 172.21.15.39:0/3064952010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df953b070 0x562df98f70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.879+0000 7f9273e00640 1 -- 172.21.15.39:0/3064952010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df98e2610 con 0x562df951a390 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.880+0000 7f9273e00640 1 --2- 172.21.15.39:0/3064952010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df98dc940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9264000c00 tx=0x7f9264009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365541c5d54e1044 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.880+0000 7f926be00640 1 -- 172.21.15.39:0/3064952010 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264000e60 con 0x562df951a390 2024-07-04T08:31:28.868 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.880+0000 7f926be00640 1 -- 172.21.15.39:0/3064952010 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9264011dd0 con 0x562df951a390 2024-07-04T08:31:28.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.880+0000 7f926be00640 1 -- 172.21.15.39:0/3064952010 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92640035f0 con 0x562df951a390 2024-07-04T08:31:28.869 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 -- 172.21.15.39:0/3064952010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 msgr2=0x562df98dc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 --2- 172.21.15.39:0/3064952010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df98dc940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9264000c00 tx=0x7f9264009d60 comp rx=0 tx=0).stop 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 -- 172.21.15.39:0/3064952010 shutdown_connections 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 --2- 172.21.15.39:0/3064952010 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df953b070 0x562df98f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 --2- 172.21.15.39:0/3064952010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df98dc940 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 --2- 172.21.15.39:0/3064952010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df95121b0 0x562df98e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 -- 172.21.15.39:0/3064952010 >> 172.21.15.39:0/3064952010 conn(0x562df952f460 msgr2=0x562df98e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 -- 172.21.15.39:0/3064952010 shutdown_connections 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 -- 172.21.15.39:0/3064952010 wait complete. 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.881+0000 7f9288d86740 1 -- start start 2024-07-04T08:31:28.870 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.882+0000 7f9288d86740 1 Processor -- start 2024-07-04T08:31:28.872 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.882+0000 7f9288d86740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 0x562df9902520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.882+0000 7f9288d86740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df9902a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.882+0000 7f9288d86740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df953b070 0x562df98f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.882+0000 7f9288d86740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1b3b30 con 0x562df95121b0 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.882+0000 7f9288d86740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1b3460 con 0x562df953b070 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.882+0000 7f9288d86740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1b06a0 con 0x562df951a390 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.882+0000 7f9288d86740 10 client.0 _ll_register_callbacks cb 0x7f9264009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9273e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df9902a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9273e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df9902a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:38734/0 (socket says 172.21.15.39:38734) 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9273e00640 1 -- 172.21.15.39:0/1539114759 learned_addr learned my addr 172.21.15.39:0/1539114759 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f927be00640 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 0x562df9902520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f927be00640 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df953b070 0x562df98f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9273e00640 1 -- 172.21.15.39:0/1539114759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df953b070 msgr2=0x562df98f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9273e00640 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df953b070 0x562df98f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9273e00640 1 -- 172.21.15.39:0/1539114759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 msgr2=0x562df9902520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9273e00640 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 0x562df9902520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9273e00640 1 -- 172.21.15.39:0/1539114759 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df98f2800 con 0x562df951a390 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9273e00640 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df9902a60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f926401b670 tx=0x7f9264011820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=672e68b59300d70c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.883+0000 7f9269600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.884+0000 7f9269600640 1 -- 172.21.15.39:0/1539114759 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264020070 con 0x562df951a390 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.884+0000 7f9288d86740 1 -- 172.21.15.39:0/1539114759 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9264011dd0 con 0x562df951a390 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.884+0000 7f9269600640 1 -- 172.21.15.39:0/1539114759 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f92640050d0 con 0x562df951a390 2024-07-04T08:31:28.874 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.884+0000 7f9269600640 1 -- 172.21.15.39:0/1539114759 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f926403a050 con 0x562df951a390 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.884+0000 7f9288d86740 10 client.29014 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.884+0000 7f9288d86740 1 -- 172.21.15.39:0/1539114759 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562df98f3a80 con 0x562df951a390 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.885+0000 7f9269600640 1 -- 172.21.15.39:0/1539114759 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9264011dd0 con 0x562df951a390 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.885+0000 7f9269600640 1 -- 172.21.15.39:0/1539114759 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f926401a070 con 0x562df951a390 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.885+0000 7f9288d86740 10 client.29014 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.885+0000 7f9288d86740 10 client.29014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.885+0000 7f9288d86740 20 client.29014 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.885+0000 7f9288d86740 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562df94ceda0 0x562df99705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.885+0000 7f9288d86740 4 client.29014 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:31:28.875 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.885+0000 7f9288d86740 1 -- 172.21.15.39:0/1539114759 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562df98538d0 con 0x562df94ceda0 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.885+0000 7f927be00640 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562df94ceda0 0x562df99705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.886+0000 7f927be00640 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562df94ceda0 0x562df99705f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d691195cc5ef5877 server_cookie=c05a385044e88ceb in_seq=0 out_seq=0 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.886+0000 7f9269600640 10 client.29014 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.886+0000 7f9269600640 1 -- 172.21.15.39:0/1539114759 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562df98538d0 con 0x562df94ceda0 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.886+0000 7f9269600640 10 client.29014 handle_command_reply: tid=0 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.886+0000 7f9288d86740 1 client.29014 shutdown 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 -- 172.21.15.39:0/1539114759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 msgr2=0x562df9902a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df9902a60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f926401b670 tx=0x7f9264011820 comp rx=0 tx=0).stop 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 -- 172.21.15.39:0/1539114759 shutdown_connections 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x562df94ceda0 0x562df99705f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x562df953b070 0x562df98f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.877 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562df951a390 0x562df9902a60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 --2- 172.21.15.39:0/1539114759 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562df95121b0 0x562df9902520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 -- 172.21.15.39:0/1539114759 >> 172.21.15.39:0/1539114759 conn(0x562df952f460 msgr2=0x562df98dcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:28.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9269600640 0 client.29014 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:28.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 -- 172.21.15.39:0/1539114759 shutdown_connections 2024-07-04T08:31:28.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 1 -- 172.21.15.39:0/1539114759 wait complete. 2024-07-04T08:31:28.878 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.887+0000 7f9288d86740 20 client.29014 trim_cache size 0 max 16384 2024-07-04T08:31:28.878 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:31:28.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 -- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92600b29e0 msgr2=0x7f92600b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:28.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92600b29e0 0x7f92600b4ef0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x562df98de430 tx=0x7f926c106b20 comp rx=0 tx=0).stop 2024-07-04T08:31:28.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 -- 172.21.15.39:0/1739189831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1030b0 msgr2=0x7f927c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:28.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1030b0 0x7f927c19bcb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f926c005ab0 tx=0x7f926c003c10 comp rx=0 tx=0).stop 2024-07-04T08:31:28.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 -- 172.21.15.39:0/1739189831 shutdown_connections 2024-07-04T08:31:28.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92600b29e0 0x7f92600b4ef0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.879 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f927c109340 0x7f927c1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f927c1089f0 0x7f927c19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 --2- 172.21.15.39:0/1739189831 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f927c1030b0 0x7f927c19bcb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:28.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 -- 172.21.15.39:0/1739189831 >> 172.21.15.39:0/1739189831 conn(0x7f927c07ada0 msgr2=0x7f927c106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:28.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 -- 172.21.15.39:0/1739189831 shutdown_connections 2024-07-04T08:31:28.880 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:28.888+0000 7f9282800640 1 -- 172.21.15.39:0/1739189831 wait complete. 2024-07-04T08:31:29.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:29 smithi133 ceph-mon[37050]: pgmap v988: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-07-04T08:31:29.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:29 smithi039 ceph-mon[32062]: pgmap v988: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-07-04T08:31:29.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:29 smithi050 ceph-mon[36577]: pgmap v988: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 75 KiB/s wr, 15 op/s 2024-07-04T08:31:30.776 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-04T08:31:30.777 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-04T08:31:31.274 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:31.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:31 smithi039 ceph-mon[32062]: pgmap v989: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 14 op/s 2024-07-04T08:31:31.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:31 smithi050 ceph-mon[36577]: pgmap v989: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 14 op/s 2024-07-04T08:31:31.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:31 smithi133 ceph-mon[37050]: pgmap v989: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 14 op/s 2024-07-04T08:31:32.199 INFO:tasks.workunit.client.0.smithi039.stdout:561/600: open fd = 3 2024-07-04T08:31:32.199 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.199 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.199 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.199 INFO:tasks.workunit.client.0.smithi039.stdout:562/600: open fd = 3 2024-07-04T08:31:32.199 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:563/600: open fd = 3 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:564/600: open fd = 3 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:565/600: open fd = 3 2024-07-04T08:31:32.200 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:566/600: open fd = 3 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:567/600: open fd = 3 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:568/600: open fd = 3 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.201 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:569/600: open fd = 3 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:570/600: open fd = 3 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:571/600: open fd = 3 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.202 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:572/600: open fd = 3 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:573/600: open fd = 3 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:574/600: open fd = 3 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.203 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:575/600: open fd = 3 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:576/600: open fd = 3 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:577/600: open fd = 3 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.204 INFO:tasks.workunit.client.0.smithi039.stdout:578/600: open fd = 3 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:579/600: open fd = 3 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:580/600: open fd = 3 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:581/600: open fd = 3 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.205 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:582/600: open fd = 3 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:583/600: open fd = 3 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:584/600: open fd = 3 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.206 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:585/600: open fd = 3 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:586/600: open fd = 3 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:587/600: open fd = 3 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:588/600: open fd = 3 2024-07-04T08:31:32.207 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:589/600: open fd = 3 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:590/600: open fd = 3 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:591/600: open fd = 3 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.208 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:592/600: open fd = 3 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:593/600: open fd = 3 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:594/600: open fd = 3 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.209 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:595/600: open fd = 3 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:596/600: open fd = 3 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:597/600: open fd = 3 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.210 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:598/600: open fd = 3 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:599/600: open fd = 3 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:600/600: open fd = 3 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:write ret = 32 2024-07-04T08:31:32.211 INFO:tasks.workunit.client.0.smithi039.stdout:pread ret = 64 2024-07-04T08:31:32.212 INFO:tasks.workunit.client.0.smithi039.stdout:OK 2024-07-04T08:31:32.213 INFO:tasks.workunit.client.0.smithi039.stderr:+ echo OK 2024-07-04T08:31:32.213 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-07-04T08:31:32.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-07-04T08:31:32.345 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-07-04T08:31:32.345 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-07-04T08:31:33.553 DEBUG:teuthology.parallel:result is None 2024-07-04T08:31:33.554 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-07-04T08:31:33.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:33 smithi039 ceph-mon[32062]: pgmap v990: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 17 op/s 2024-07-04T08:31:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.628+0000 7fb2e7e00640 1 -- 172.21.15.39:0/321038057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e01089f0 msgr2=0x7fb2e0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.628+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/321038057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e01089f0 0x7fb2e0108e00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009900 tx=0x7fb2cc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.629+0000 7fb2e7e00640 1 -- 172.21.15.39:0/321038057 shutdown_connections 2024-07-04T08:31:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.629+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/321038057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e01089f0 0x7fb2e0108e00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.629+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/321038057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e0109340 0x7fb2e010fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.629+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/321038057 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e01030b0 0x7fb2e01034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.629+0000 7fb2e7e00640 1 -- 172.21.15.39:0/321038057 >> 172.21.15.39:0/321038057 conn(0x7fb2e007ada0 msgr2=0x7fb2e007b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.629+0000 7fb2e7e00640 1 -- 172.21.15.39:0/321038057 shutdown_connections 2024-07-04T08:31:33.616 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.629+0000 7fb2e7e00640 1 -- 172.21.15.39:0/321038057 wait complete. 2024-07-04T08:31:33.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.630+0000 7fb2e7e00640 1 Processor -- start 2024-07-04T08:31:33.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.630+0000 7fb2e7e00640 1 -- start start 2024-07-04T08:31:33.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e7e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e01030b0 0x7fb2e0074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e7e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e01089f0 0x7fb2e00751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e0109340 0x7fb2e00756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.617 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e7e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0112fb0 con 0x7fb2e01030b0 2024-07-04T08:31:33.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e7e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0112e30 con 0x7fb2e01089f0 2024-07-04T08:31:33.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e0113130 con 0x7fb2e0109340 2024-07-04T08:31:33.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e01089f0 0x7fb2e00751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e01089f0 0x7fb2e00751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47174/0 (socket says 172.21.15.39:47174) 2024-07-04T08:31:33.618 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e4a00640 1 -- 172.21.15.39:0/1494443636 learned_addr learned my addr 172.21.15.39:0/1494443636 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:33.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e5e00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e0109340 0x7fb2e00756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.631+0000 7fb2e5400640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e01030b0 0x7fb2e0074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.632+0000 7fb2e4a00640 1 -- 172.21.15.39:0/1494443636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e0109340 msgr2=0x7fb2e00756e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:33.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.632+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e0109340 0x7fb2e00756e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.632+0000 7fb2e4a00640 1 -- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e01030b0 msgr2=0x7fb2e0074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:33.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.632+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e01030b0 0x7fb2e0074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.619 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.632+0000 7fb2e4a00640 1 -- 172.21.15.39:0/1494443636 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e01a7cc0 con 0x7fb2e01089f0 2024-07-04T08:31:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.632+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e01089f0 0x7fb2e00751a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc005ab0 tx=0x7fb2cc031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.633+0000 7fb2de000640 1 -- 172.21.15.39:0/1494443636 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc03f030 con 0x7fb2e01089f0 2024-07-04T08:31:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.633+0000 7fb2de000640 1 -- 172.21.15.39:0/1494443636 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2cc0043d0 con 0x7fb2e01089f0 2024-07-04T08:31:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.633+0000 7fb2e7e00640 1 -- 172.21.15.39:0/1494443636 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e01a7f50 con 0x7fb2e01089f0 2024-07-04T08:31:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.633+0000 7fb2de000640 1 -- 172.21.15.39:0/1494443636 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc035070 con 0x7fb2e01089f0 2024-07-04T08:31:33.620 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.633+0000 7fb2e7e00640 1 -- 172.21.15.39:0/1494443636 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e01a8350 con 0x7fb2e01089f0 2024-07-04T08:31:33.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.637+0000 7fb2ee4b6740 1 Processor -- start 2024-07-04T08:31:33.623 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.637+0000 7fb2ee4b6740 1 -- start start 2024-07-04T08:31:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.637+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140e8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.637+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.638+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d23670 0x5558140e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.638+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5558140128f0 con 0x555813d1b490 2024-07-04T08:31:33.624 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.638+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55581405eed0 con 0x555813d44350 2024-07-04T08:31:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.638+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55581404e470 con 0x555813d23670 2024-07-04T08:31:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.638+0000 7fb2e5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.638+0000 7fb2e5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140e8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.638+0000 7fb2e5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140e8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39640/0 (socket says 172.21.15.39:39640) 2024-07-04T08:31:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.638+0000 7fb2e5400640 1 -- 172.21.15.39:0/1923152024 learned_addr learned my addr 172.21.15.39:0/1923152024 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:33.625 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.638+0000 7fb2e5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140e8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47178/0 (socket says 172.21.15.39:47178) 2024-07-04T08:31:33.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.639+0000 7fb2de000640 1 -- 172.21.15.39:0/1494443636 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb2cc042030 con 0x7fb2e01089f0 2024-07-04T08:31:33.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.639+0000 7fb2e5e00640 1 -- 172.21.15.39:0/1923152024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d23670 msgr2=0x5558140e90d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:31:33.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.639+0000 7fb2e5e00640 1 --2- 172.21.15.39:0/1923152024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d23670 0x5558140e90d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.639+0000 7fb2e5e00640 1 -- 172.21.15.39:0/1923152024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 msgr2=0x5558140e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:33.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.639+0000 7fb2e5e00640 1 --2- 172.21.15.39:0/1923152024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.626 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.639+0000 7fb2e5e00640 1 -- 172.21.15.39:0/1923152024 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555813fe6290 con 0x555813d44350 2024-07-04T08:31:33.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.639+0000 7fb2e5e00640 1 --2- 172.21.15.39:0/1923152024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140e8650 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009f00 tx=0x7fb2d8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cffb032f913c99c server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:33.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.639+0000 7fb2d2a00640 1 -- 172.21.15.39:0/1923152024 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8004d20 con 0x555813d44350 2024-07-04T08:31:33.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.639+0000 7fb2d2a00640 1 -- 172.21.15.39:0/1923152024 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d8004030 con 0x555813d44350 2024-07-04T08:31:33.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.640+0000 7fb2d2a00640 1 -- 172.21.15.39:0/1923152024 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8005220 con 0x555813d44350 2024-07-04T08:31:33.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.640+0000 7fb2de000640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c80b2960 0x7fb2c80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2de000640 1 -- 172.21.15.39:0/1494443636 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fb2cc102cc0 con 0x7fb2e01089f0 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/1923152024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 msgr2=0x5558140e8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/1923152024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140e8650 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009f00 tx=0x7fb2d8011810 comp rx=0 tx=0).stop 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2e5400640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c80b2960 0x7fb2c80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/1923152024 shutdown_connections 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/1923152024 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/1923152024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d23670 0x5558140e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/1923152024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140e8650 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/1923152024 >> 172.21.15.39:0/1923152024 conn(0x555813d38740 msgr2=0x5558140e6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/1923152024 shutdown_connections 2024-07-04T08:31:33.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.641+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/1923152024 wait complete. 2024-07-04T08:31:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.642+0000 7fb2e5400640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c80b2960 0x7fb2c80b4e70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5558140e72d0 tx=0x7fb2d4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.642+0000 7fb2ee4b6740 1 -- start start 2024-07-04T08:31:33.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.642+0000 7fb2ee4b6740 1 Processor -- start 2024-07-04T08:31:33.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d1b490 0x5558140f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140f4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d44350 0x555814100000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x555813ff5770 con 0x555813d44350 2024-07-04T08:31:33.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5558140524b0 con 0x555813d23670 2024-07-04T08:31:33.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555813eb2890 con 0x555813d1b490 2024-07-04T08:31:33.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2ee4b6740 10 client.0 _ll_register_callbacks cb 0x555813ff62c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:33.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2e4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2e5400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d1b490 0x5558140f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2e4a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140f4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47182/0 (socket says 172.21.15.39:47182) 2024-07-04T08:31:33.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.643+0000 7fb2e4a00640 1 -- 172.21.15.39:0/361844672 learned_addr learned my addr 172.21.15.39:0/361844672 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:33.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.644+0000 7fb2e5e00640 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d44350 0x555814100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:33 smithi050 ceph-mon[36577]: pgmap v990: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 17 op/s 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.644+0000 7fb2e4a00640 1 -- 172.21.15.39:0/361844672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d1b490 msgr2=0x5558140f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.644+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d1b490 0x5558140f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.644+0000 7fb2e4a00640 1 -- 172.21.15.39:0/361844672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d44350 msgr2=0x555814100000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.644+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d44350 0x555814100000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.644+0000 7fb2e4a00640 1 -- 172.21.15.39:0/361844672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555813daf930 con 0x555813d23670 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.644+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140f4720 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc000c00 tx=0x7fb2cc0c9630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c3471b4fd5ae917 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.644+0000 7fb2c7400640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.645+0000 7fb2c7400640 1 -- 172.21.15.39:0/361844672 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc100520 con 0x555813d23670 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.645+0000 7fb2c7400640 1 -- 172.21.15.39:0/361844672 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fb2cc0d25f0 con 0x555813d23670 2024-07-04T08:31:33.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.645+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/361844672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d8004030 con 0x555813d23670 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.645+0000 7fb2c7400640 1 -- 172.21.15.39:0/361844672 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2cc0ca320 con 0x555813d23670 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.645+0000 7fb2ee4b6740 10 client.29099 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.645+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/361844672 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555814102c10 con 0x555813d23670 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.645+0000 7fb2c7400640 1 -- 172.21.15.39:0/361844672 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d8004030 con 0x555813d23670 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.646+0000 7fb2c7400640 1 -- 172.21.15.39:0/361844672 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb2cc0e40d0 con 0x555813d23670 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.646+0000 7fb2ee4b6740 10 client.29099 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.646+0000 7fb2ee4b6740 10 client.29099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.646+0000 7fb2ee4b6740 20 client.29099 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.646+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555813d02cd0 0x555814105fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.646+0000 7fb2ee4b6740 4 client.29099 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.646+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/361844672 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555814060e20 con 0x555813d02cd0 2024-07-04T08:31:33.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.646+0000 7fb2e5400640 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555813d02cd0 0x555814105fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.647+0000 7fb2e5400640 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555813d02cd0 0x555814105fe0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f080a032d72c3093 server_cookie=206d8b08edd02e84 in_seq=0 out_seq=0 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.647+0000 7fb2c7400640 10 client.29099 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.648+0000 7fb2c7400640 1 -- 172.21.15.39:0/361844672 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555814060e20 con 0x555813d02cd0 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.648+0000 7fb2c7400640 10 client.29099 handle_command_reply: tid=0 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.648+0000 7fb2ee4b6740 1 client.29099 shutdown 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/361844672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 msgr2=0x5558140f4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140f4720 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc000c00 tx=0x7fb2cc0c9630 comp rx=0 tx=0).stop 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/361844672 shutdown_connections 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555813d02cd0 0x555814105fe0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d44350 0x555814100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140f4720 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/361844672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d1b490 0x5558140f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2c7400640 0 client.29099 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:33.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/361844672 >> 172.21.15.39:0/361844672 conn(0x555813d38740 msgr2=0x5558140eb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:33.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/361844672 shutdown_connections 2024-07-04T08:31:33.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/361844672 wait complete. 2024-07-04T08:31:33.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.649+0000 7fb2ee4b6740 20 client.29099 trim_cache size 0 max 16384 2024-07-04T08:31:33.642 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-07-04T08:31:33.642 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-07-04T08:31:33.711 INFO:tasks.quiescer:joining Quiescers 2024-07-04T08:31:33.711 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-07-04T08:31:33.711 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:31:33.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.670+0000 7fb2ee4b6740 1 Processor -- start 2024-07-04T08:31:33.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.670+0000 7fb2ee4b6740 1 -- start start 2024-07-04T08:31:33.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.670+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.670+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.670+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d44350 0x555814100330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.670+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e01b2590 con 0x555813d1b490 2024-07-04T08:31:33.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.670+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e01b40d0 con 0x555813d23670 2024-07-04T08:31:33.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.670+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e01b2c60 con 0x555813d44350 2024-07-04T08:31:33.716 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.670+0000 7fb2e5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39656/0 (socket says 172.21.15.39:39656) 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e5400640 1 -- 172.21.15.39:0/2513990871 learned_addr learned my addr 172.21.15.39:0/2513990871 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e5e00640 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d44350 0x555814100330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e5400640 1 -- 172.21.15.39:0/2513990871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d44350 msgr2=0x555814100330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e5400640 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d44350 0x555814100330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e5400640 1 -- 172.21.15.39:0/2513990871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 msgr2=0x5558140ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e5400640 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e5400640 1 -- 172.21.15.39:0/2513990871 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558140e7300 con 0x555813d1b490 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.671+0000 7fb2e5400640 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f5c10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4013010 tx=0x7fb2d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e94d4a45fdca09 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.672+0000 7fb2d2000640 1 -- 172.21.15.39:0/2513990871 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4003ac0 con 0x555813d1b490 2024-07-04T08:31:33.717 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.672+0000 7fb2d2000640 1 -- 172.21.15.39:0/2513990871 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d4003c60 con 0x555813d1b490 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.672+0000 7fb2d2000640 1 -- 172.21.15.39:0/2513990871 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4021670 con 0x555813d1b490 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/2513990871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 msgr2=0x5558140f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f5c10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4013010 tx=0x7fb2d4017000 comp rx=0 tx=0).stop 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/2513990871 shutdown_connections 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d44350 0x555814100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d23670 0x5558140ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/2513990871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f5c10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/2513990871 >> 172.21.15.39:0/2513990871 conn(0x555813d38740 msgr2=0x5558140f5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/2513990871 shutdown_connections 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/2513990871 wait complete. 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.673+0000 7fb2ee4b6740 1 -- start start 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.674+0000 7fb2ee4b6740 1 Processor -- start 2024-07-04T08:31:33.718 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.674+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.674+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d23670 0x5558140f8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.674+0000 7fb2ee4b6740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.674+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e01b3a00 con 0x555813d1b490 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.674+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e01b3330 con 0x555813d44350 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.674+0000 7fb2ee4b6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e01b0570 con 0x555813d23670 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.674+0000 7fb2ee4b6740 10 client.0 _ll_register_callbacks cb 0x7fb2cc0c99f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39658/0 (socket says 172.21.15.39:39658) 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e5400640 1 -- 172.21.15.39:0/4268428835 learned_addr learned my addr 172.21.15.39:0/4268428835 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e5e00640 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d23670 0x5558140f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e5400640 1 -- 172.21.15.39:0/4268428835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d23670 msgr2=0x5558140f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:33.719 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e5400640 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d23670 0x5558140f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e5400640 1 -- 172.21.15.39:0/4268428835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 msgr2=0x5558140f8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e5400640 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140f8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.675+0000 7fb2e5400640 1 -- 172.21.15.39:0/4268428835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581416d970 con 0x555813d1b490 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.676+0000 7fb2e5400640 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f8030 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4003b30 tx=0x7fb2d4003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98c3152bfc63473 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.676+0000 7fb2c7400640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.676+0000 7fb2c7400640 1 -- 172.21.15.39:0/4268428835 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d403b030 con 0x555813d1b490 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.676+0000 7fb2c7400640 1 -- 172.21.15.39:0/4268428835 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fb2d4018040 con 0x555813d1b490 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.676+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/4268428835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d4003c60 con 0x555813d1b490 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.676+0000 7fb2c7400640 1 -- 172.21.15.39:0/4268428835 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb2d4045050 con 0x555813d1b490 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.676+0000 7fb2ee4b6740 10 client.18981 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.676+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/4268428835 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55581416ec30 con 0x555813d1b490 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.677+0000 7fb2c7400640 1 -- 172.21.15.39:0/4268428835 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d4003c60 con 0x555813d1b490 2024-07-04T08:31:33.720 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.677+0000 7fb2c7400640 1 -- 172.21.15.39:0/4268428835 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb2d402c430 con 0x555813d1b490 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.677+0000 7fb2ee4b6740 10 client.18981 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.677+0000 7fb2ee4b6740 10 client.18981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.677+0000 7fb2ee4b6740 20 client.18981 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.677+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555813cd8080 0x555814171410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.677+0000 7fb2ee4b6740 4 client.18981 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.677+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/4268428835 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55581405c8c0 con 0x555813cd8080 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.677+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555813cd8080 0x555814171410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.678+0000 7fb2e4a00640 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555813cd8080 0x555814171410 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d82468931f6fa6 server_cookie=e2906db4e5e6ce35 in_seq=0 out_seq=0 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.678+0000 7fb2c7400640 10 client.18981 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.679+0000 7fb2c7400640 1 -- 172.21.15.39:0/4268428835 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55581405c8c0 con 0x555813cd8080 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.679+0000 7fb2c7400640 10 client.18981 handle_command_reply: tid=0 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.679+0000 7fb2ee4b6740 1 client.18981 shutdown 2024-07-04T08:31:33.721 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/4268428835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 msgr2=0x5558140f8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f8030 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4003b30 tx=0x7fb2d4003b60 comp rx=0 tx=0).stop 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/4268428835 shutdown_connections 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x555813cd8080 0x555814171410 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555813d44350 0x5558140f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555813d23670 0x5558140f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 --2- 172.21.15.39:0/4268428835 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x555813d1b490 0x5558140f8030 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/4268428835 >> 172.21.15.39:0/4268428835 conn(0x555813d38740 msgr2=0x5558140f2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2c7400640 0 client.18981 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/4268428835 shutdown_connections 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 1 -- 172.21.15.39:0/4268428835 wait complete. 2024-07-04T08:31:33.722 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.680+0000 7fb2ee4b6740 20 client.18981 trim_cache size 0 max 16384 2024-07-04T08:31:33.723 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:31:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "return_code": 0, 2024-07-04T08:31:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_tag": "aeab0478-7204-44cd-ac8b-8b5ea4bb4b4c", 2024-07-04T08:31:33.723 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": "asynchronous" 2024-07-04T08:31:33.723 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 -- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c80b2960 msgr2=0x7fb2c80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c80b2960 0x7fb2c80b4e70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x5558140e72d0 tx=0x7fb2d4005f90 comp rx=0 tx=0).stop 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 -- 172.21.15.39:0/1494443636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e01089f0 msgr2=0x7fb2e00751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e01089f0 0x7fb2e00751a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc005ab0 tx=0x7fb2cc031a20 comp rx=0 tx=0).stop 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 -- 172.21.15.39:0/1494443636 shutdown_connections 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb2c80b2960 0x7fb2c80b4e70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e0109340 0x7fb2e00756e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb2e01089f0 0x7fb2e00751a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 --2- 172.21.15.39:0/1494443636 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2e01030b0 0x7fb2e0074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 -- 172.21.15.39:0/1494443636 >> 172.21.15.39:0/1494443636 conn(0x7fb2e007ada0 msgr2=0x7fb2e0106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:33.724 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 -- 172.21.15.39:0/1494443636 shutdown_connections 2024-07-04T08:31:33.725 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:33.681+0000 7fb2e7e00640 1 -- 172.21.15.39:0/1494443636 wait complete. 2024-07-04T08:31:33.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:33 smithi133 ceph-mon[37050]: pgmap v990: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 100 KiB/s wr, 17 op/s 2024-07-04T08:31:34.515 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-04T08:31:34.566 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:35.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:35 smithi039 ceph-mon[32062]: scrub queued for path: / 2024-07-04T08:31:35.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:35 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:35.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:35 smithi039 ceph-mon[32062]: scrub summary: active paths [/] 2024-07-04T08:31:35.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:35 smithi039 ceph-mon[32062]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:35.591 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:35 smithi039 ceph-mon[32062]: pgmap v991: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 13 op/s 2024-07-04T08:31:35.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:35 smithi050 ceph-mon[36577]: scrub queued for path: / 2024-07-04T08:31:35.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:35 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:35.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:35 smithi050 ceph-mon[36577]: scrub summary: active paths [/] 2024-07-04T08:31:35.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:35 smithi050 ceph-mon[36577]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:35.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:35 smithi050 ceph-mon[36577]: pgmap v991: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 13 op/s 2024-07-04T08:31:35.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:35 smithi133 ceph-mon[37050]: scrub queued for path: / 2024-07-04T08:31:35.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:35 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:35.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:35 smithi133 ceph-mon[37050]: scrub summary: active paths [/] 2024-07-04T08:31:35.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:35 smithi133 ceph-mon[37050]: scrub summary: idle+waiting paths [/] 2024-07-04T08:31:35.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:35 smithi133 ceph-mon[37050]: pgmap v991: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 69 KiB/s wr, 13 op/s 2024-07-04T08:31:35.970 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:37.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:37 smithi039 ceph-mon[32062]: pgmap v992: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 87 KiB/s wr, 16 op/s 2024-07-04T08:31:37.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:37 smithi050 ceph-mon[36577]: pgmap v992: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 87 KiB/s wr, 16 op/s 2024-07-04T08:31:37.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:37 smithi133 ceph-mon[37050]: pgmap v992: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 87 KiB/s wr, 16 op/s 2024-07-04T08:31:37.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.541+0000 7f550dc00640 1 -- 172.21.15.39:0/41833323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508109700 msgr2=0x7f5508109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:37.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.541+0000 7f550dc00640 1 --2- 172.21.15.39:0/41833323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508109700 0x7f5508109b10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009900 tx=0x7f54f802a410 comp rx=0 tx=0).stop 2024-07-04T08:31:37.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.541+0000 7f550dc00640 1 -- 172.21.15.39:0/41833323 shutdown_connections 2024-07-04T08:31:37.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.541+0000 7f550dc00640 1 --2- 172.21.15.39:0/41833323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f550810f990 0x7f55081161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:37.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.541+0000 7f550dc00640 1 --2- 172.21.15.39:0/41833323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f550810f040 0x7f550810f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:37.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.541+0000 7f550dc00640 1 --2- 172.21.15.39:0/41833323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5508109700 0x7f5508109b10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:37.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.541+0000 7f550dc00640 1 -- 172.21.15.39:0/41833323 >> 172.21.15.39:0/41833323 conn(0x7f5508077630 msgr2=0x7f5508107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:37.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.541+0000 7f550dc00640 1 -- 172.21.15.39:0/41833323 shutdown_connections 2024-07-04T08:31:37.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.541+0000 7f550dc00640 1 -- 172.21.15.39:0/41833323 wait complete. 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.543+0000 7f550dc00640 1 Processor -- start 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.543+0000 7f550dc00640 1 -- start start 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.543+0000 7f550dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5508109700 0x7f55081a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.543+0000 7f550dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550810f040 0x7f55081a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.543+0000 7f550dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f550810f990 0x7f55081a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f550dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5508119640 con 0x7f550810f040 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f550dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55081194c0 con 0x7f5508109700 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f550dc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55081197c0 con 0x7f550810f990 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f54ffe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550810f040 0x7f55081a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f5506a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5508109700 0x7f55081a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:37.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f54ffe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550810f040 0x7f55081a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39686/0 (socket says 172.21.15.39:39686) 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f5506a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5508109700 0x7f55081a0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:47232/0 (socket says 172.21.15.39:47232) 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f54ffe00640 1 -- 172.21.15.39:0/2430040581 learned_addr learned my addr 172.21.15.39:0/2430040581 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f5507400640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f550810f990 0x7f55081a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f5506a00640 1 -- 172.21.15.39:0/2430040581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f550810f990 msgr2=0x7f55081a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f5506a00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f550810f990 0x7f55081a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f5506a00640 1 -- 172.21.15.39:0/2430040581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550810f040 msgr2=0x7f55081a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f5506a00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550810f040 0x7f55081a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.544+0000 7f5506a00640 1 -- 172.21.15.39:0/2430040581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55081a5f80 con 0x7f5508109700 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.545+0000 7f5506a00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5508109700 0x7f55081a0010 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f54f802a930 tx=0x7f54f8031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.545+0000 7f54ff400640 1 -- 172.21.15.39:0/2430040581 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f803f030 con 0x7f5508109700 2024-07-04T08:31:37.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.545+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55081a6210 con 0x7f5508109700 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.546+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55081ac9d0 con 0x7f5508109700 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.546+0000 7f54ff400640 1 -- 172.21.15.39:0/2430040581 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54f8004490 con 0x7f5508109700 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.546+0000 7f54ff400640 1 -- 172.21.15.39:0/2430040581 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8049060 con 0x7f5508109700 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.547+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e40052c0 con 0x7f5508109700 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.549+0000 7f54ff400640 1 -- 172.21.15.39:0/2430040581 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f54f8042030 con 0x7f5508109700 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.551+0000 7f54ff400640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54ec0b2960 0x7f54ec0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.551+0000 7f54ff400640 1 -- 172.21.15.39:0/2430040581 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f54f8102bb0 con 0x7f5508109700 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.551+0000 7f54ffe00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54ec0b2960 0x7f54ec0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.552+0000 7f54ffe00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54ec0b2960 0x7f54ec0b4e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f550806a6d0 tx=0x7f54f0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:37.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.555+0000 7f54ff400640 1 -- 172.21.15.39:0/2430040581 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f54f80c9b80 con 0x7f5508109700 2024-07-04T08:31:37.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.871+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54e4005880 con 0x7f5508109700 2024-07-04T08:31:37.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.872+0000 7f54ff400640 1 -- 172.21.15.39:0/2430040581 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f54f80cde80 con 0x7f5508109700 2024-07-04T08:31:37.859 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:31:37.859 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:31:37.860 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:31:37.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54ec0b2960 msgr2=0x7f54ec0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:37.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54ec0b2960 0x7f54ec0b4e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f550806a6d0 tx=0x7f54f0005f90 comp rx=0 tx=0).stop 2024-07-04T08:31:37.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5508109700 msgr2=0x7f55081a0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:37.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5508109700 0x7f55081a0010 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f54f802a930 tx=0x7f54f8031a20 comp rx=0 tx=0).stop 2024-07-04T08:31:37.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 shutdown_connections 2024-07-04T08:31:37.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f54ec0b2960 0x7f54ec0b4e70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:37.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f550810f040 0x7f55081a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:37.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f550810f990 0x7f55081a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:37.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 --2- 172.21.15.39:0/2430040581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5508109700 0x7f55081a0010 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:37.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 >> 172.21.15.39:0/2430040581 conn(0x7f5508077630 msgr2=0x7f5508077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:37.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 shutdown_connections 2024-07-04T08:31:37.863 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:37.875+0000 7f550dc00640 1 -- 172.21.15.39:0/2430040581 wait complete. 2024-07-04T08:31:38.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:38 smithi039 ceph-mon[32062]: scrub summary: idle 2024-07-04T08:31:38.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:38 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2430040581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:38.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:38 smithi050 ceph-mon[36577]: scrub summary: idle 2024-07-04T08:31:38.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:38 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2430040581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:38.755 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24707 quiesce db --cancel --all' 2024-07-04T08:31:38.755 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- timeout 105 ceph tell mds.24707 quiesce db --cancel --all 2024-07-04T08:31:38.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:38 smithi133 ceph-mon[37050]: scrub summary: idle 2024-07-04T08:31:38.805 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:38 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2430040581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:39.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.360+0000 7f8759200640 1 -- 172.21.15.39:0/577646039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8754104660 msgr2=0x7f875410cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:39.347 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.360+0000 7f8759200640 1 --2- 172.21.15.39:0/577646039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8754104660 0x7f875410cbb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8740009900 tx=0x7f874002a3f0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.360+0000 7f8759200640 1 -- 172.21.15.39:0/577646039 shutdown_connections 2024-07-04T08:31:39.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.360+0000 7f8759200640 1 --2- 172.21.15.39:0/577646039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f875410d170 0x7f875410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.360+0000 7f8759200640 1 --2- 172.21.15.39:0/577646039 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8754104660 0x7f875410cbb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.360+0000 7f8759200640 1 --2- 172.21.15.39:0/577646039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8754103d10 0x7f8754104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.360+0000 7f8759200640 1 -- 172.21.15.39:0/577646039 >> 172.21.15.39:0/577646039 conn(0x7f87540fd810 msgr2=0x7f87540ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:39.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.360+0000 7f8759200640 1 -- 172.21.15.39:0/577646039 shutdown_connections 2024-07-04T08:31:39.348 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.361+0000 7f8759200640 1 -- 172.21.15.39:0/577646039 wait complete. 2024-07-04T08:31:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.362+0000 7f8759200640 1 Processor -- start 2024-07-04T08:31:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.362+0000 7f8759200640 1 -- start start 2024-07-04T08:31:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8759200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8754103d10 0x7f8754199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.349 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8759200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8754104660 0x7f8754199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8759200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f875410d170 0x7f875419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8759200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f8754112e20 con 0x7f8754103d10 2024-07-04T08:31:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8759200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8754112ca0 con 0x7f8754104660 2024-07-04T08:31:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8759200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8754112fa0 con 0x7f875410d170 2024-07-04T08:31:39.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8753400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8754103d10 0x7f8754199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8753400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8754103d10 0x7f8754199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39700/0 (socket says 172.21.15.39:39700) 2024-07-04T08:31:39.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8753400640 1 -- 172.21.15.39:0/3728624276 learned_addr learned my addr 172.21.15.39:0/3728624276 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:39.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8752a00640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8754104660 0x7f8754199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.363+0000 7f8753e00640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f875410d170 0x7f875419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.364+0000 7f8753400640 1 -- 172.21.15.39:0/3728624276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f875410d170 msgr2=0x7f875419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:39.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.364+0000 7f8753400640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f875410d170 0x7f875419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.364+0000 7f8753400640 1 -- 172.21.15.39:0/3728624276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8754104660 msgr2=0x7f8754199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:39.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.364+0000 7f8753400640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8754104660 0x7f8754199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.364+0000 7f8753400640 1 -- 172.21.15.39:0/3728624276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875419f7e0 con 0x7f8754103d10 2024-07-04T08:31:39.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.364+0000 7f8753400640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8754103d10 0x7f8754199870 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f874400e9f0 tx=0x7f874400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:39.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.365+0000 7f874be00640 1 -- 172.21.15.39:0/3728624276 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874400cd80 con 0x7f8754103d10 2024-07-04T08:31:39.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.365+0000 7f874be00640 1 -- 172.21.15.39:0/3728624276 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87440054d0 con 0x7f8754103d10 2024-07-04T08:31:39.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.365+0000 7f8759200640 1 -- 172.21.15.39:0/3728624276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875419fad0 con 0x7f8754103d10 2024-07-04T08:31:39.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.365+0000 7f8759200640 1 -- 172.21.15.39:0/3728624276 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87541a6230 con 0x7f8754103d10 2024-07-04T08:31:39.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.365+0000 7f874be00640 1 -- 172.21.15.39:0/3728624276 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744010690 con 0x7f8754103d10 2024-07-04T08:31:39.355 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.369+0000 7f8760de0740 1 Processor -- start 2024-07-04T08:31:39.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.369+0000 7f8760de0740 1 -- start start 2024-07-04T08:31:39.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8760de0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7ba070 0x556d9ab5e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8760de0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7911b0 0x556d9ab5ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8760de0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab5f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8760de0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556d9aad4ea0 con 0x556d9a799390 2024-07-04T08:31:39.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8760de0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556d9aac4440 con 0x556d9a7911b0 2024-07-04T08:31:39.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8760de0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556d9aac8480 con 0x556d9a7ba070 2024-07-04T08:31:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8752a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab5f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8752a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab5f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:39714/0 (socket says 172.21.15.39:39714) 2024-07-04T08:31:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8752a00640 1 -- 172.21.15.39:0/4196692259 learned_addr learned my addr 172.21.15.39:0/4196692259 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f8753e00640 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7ba070 0x556d9ab5e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.370+0000 7f874be00640 1 -- 172.21.15.39:0/3728624276 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f8744004e90 con 0x7f8754103d10 2024-07-04T08:31:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.371+0000 7f8752a00640 1 -- 172.21.15.39:0/4196692259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7ba070 msgr2=0x556d9ab5e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:39.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.371+0000 7f8753400640 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7911b0 0x556d9ab5ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.371+0000 7f8752a00640 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7ba070 0x556d9ab5e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.371+0000 7f8752a00640 1 -- 172.21.15.39:0/4196692259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7911b0 msgr2=0x556d9ab5ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.371+0000 7f8752a00640 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7911b0 0x556d9ab5ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.371+0000 7f8752a00640 1 -- 172.21.15.39:0/4196692259 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9aa5c610 con 0x556d9a799390 2024-07-04T08:31:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.371+0000 7f8753400640 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7911b0 0x556d9ab5ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:31:39.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.371+0000 7f8753e00640 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7ba070 0x556d9ab5e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:39.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.371+0000 7f8752a00640 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab5f390 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8754069fe0 tx=0x7f87400040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8a5eef1a95739d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:39.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.372+0000 7f8748c00640 1 -- 172.21.15.39:0/4196692259 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874003f070 con 0x556d9a799390 2024-07-04T08:31:39.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.372+0000 7f8748c00640 1 -- 172.21.15.39:0/4196692259 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87400044c0 con 0x556d9a799390 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.372+0000 7f8748c00640 1 -- 172.21.15.39:0/4196692259 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740035070 con 0x556d9a799390 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.372+0000 7f874be00640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f87380b29e0 0x7f87380b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.373+0000 7f874be00640 1 -- 172.21.15.39:0/3728624276 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f87440e2eb0 con 0x7f8754103d10 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.373+0000 7f8753e00640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f87380b29e0 0x7f87380b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.373+0000 7f8760de0740 1 -- 172.21.15.39:0/4196692259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 msgr2=0x556d9ab5f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.373+0000 7f8760de0740 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab5f390 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8754069fe0 tx=0x7f87400040d0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.373+0000 7f8760de0740 1 -- 172.21.15.39:0/4196692259 shutdown_connections 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.373+0000 7f8760de0740 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab5f390 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.373+0000 7f8760de0740 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7911b0 0x556d9ab5ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.373+0000 7f8760de0740 1 --2- 172.21.15.39:0/4196692259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7ba070 0x556d9ab5e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.373+0000 7f8760de0740 1 -- 172.21.15.39:0/4196692259 >> 172.21.15.39:0/4196692259 conn(0x556d9a7ae460 msgr2=0x556d9ab5c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.374+0000 7f8760de0740 1 -- 172.21.15.39:0/4196692259 shutdown_connections 2024-07-04T08:31:39.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.374+0000 7f8760de0740 1 -- 172.21.15.39:0/4196692259 wait complete. 2024-07-04T08:31:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.374+0000 7f8753e00640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f87380b29e0 0x7f87380b4ef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f874c005fd0 tx=0x7f874c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.374+0000 7f8760de0740 1 -- start start 2024-07-04T08:31:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.375+0000 7f8760de0740 1 Processor -- start 2024-07-04T08:31:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.375+0000 7f8760de0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab6e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.375+0000 7f8760de0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab6edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.375+0000 7f8760de0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 0x556d9ab7a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.363 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.375+0000 7f8760de0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x556d9a928700 con 0x556d9a799390 2024-07-04T08:31:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.375+0000 7f8760de0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556d9aa6bb70 con 0x556d9a7ba070 2024-07-04T08:31:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.375+0000 7f8760de0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556d9ab713c0 con 0x556d9a7911b0 2024-07-04T08:31:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.375+0000 7f8760de0740 10 client.0 _ll_register_callbacks cb 0x556d9aabd530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8753400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab6e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8753400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab6e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39858/0 (socket says 172.21.15.39:39858) 2024-07-04T08:31:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8753400640 1 -- 172.21.15.39:0/436880411 learned_addr learned my addr 172.21.15.39:0/436880411 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8752a00640 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab6edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8753e00640 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 0x556d9ab7a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.364 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8753400640 1 -- 172.21.15.39:0/436880411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 msgr2=0x556d9ab7a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8753400640 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 0x556d9ab7a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8753400640 1 -- 172.21.15.39:0/436880411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 msgr2=0x556d9ab6edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8753400640 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab6edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.365 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.376+0000 7f8753400640 1 -- 172.21.15.39:0/436880411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9a825650 con 0x556d9a7911b0 2024-07-04T08:31:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.377+0000 7f8753400640 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab6e870 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f87440a7040 tx=0x7f87440048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31060071bdf72c8a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:39.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.377+0000 7f873d600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.377+0000 7f873d600640 1 -- 172.21.15.39:0/436880411 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440a98e0 con 0x556d9a7911b0 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.377+0000 7f873d600640 1 -- 172.21.15.39:0/436880411 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f87440b27a0 con 0x556d9a7911b0 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.377+0000 7f8760de0740 1 -- 172.21.15.39:0/436880411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d9aad6df0 con 0x556d9a7911b0 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.377+0000 7f873d600640 1 -- 172.21.15.39:0/436880411 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87440aa4d0 con 0x556d9a7911b0 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.377+0000 7f8760de0740 10 client.29047 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.377+0000 7f8760de0740 1 -- 172.21.15.39:0/436880411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d9ab7d360 con 0x556d9a7911b0 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.378+0000 7f873d600640 1 -- 172.21.15.39:0/436880411 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556d9aad6df0 con 0x556d9a7911b0 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.378+0000 7f873d600640 1 -- 172.21.15.39:0/436880411 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8744091040 con 0x556d9a7911b0 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.378+0000 7f8760de0740 10 client.29047 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:39.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.378+0000 7f8760de0740 10 client.29047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.378+0000 7f8760de0740 20 client.29047 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.378+0000 7f8760de0740 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556d9a7789f0 0x556d9ab808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.378+0000 7f8760de0740 4 client.29047 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.378+0000 7f8760de0740 1 -- 172.21.15.39:0/436880411 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d9aad2890 con 0x556d9a7789f0 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.379+0000 7f8752a00640 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556d9a7789f0 0x556d9ab808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.379+0000 7f8752a00640 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556d9a7789f0 0x556d9ab808e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf0cd052c8626f0 server_cookie=6d1668d700220b4a in_seq=0 out_seq=0 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.379+0000 7f873d600640 10 client.29047 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.380+0000 7f873d600640 1 -- 172.21.15.39:0/436880411 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556d9aad2890 con 0x556d9a7789f0 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.380+0000 7f873d600640 10 client.29047 handle_command_reply: tid=0 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.380+0000 7f8760de0740 1 client.29047 shutdown 2024-07-04T08:31:39.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.380+0000 7f8760de0740 1 -- 172.21.15.39:0/436880411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 msgr2=0x556d9ab6e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.380+0000 7f8760de0740 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab6e870 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f87440a7040 tx=0x7f87440048f0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f8760de0740 1 -- 172.21.15.39:0/436880411 shutdown_connections 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f8760de0740 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556d9a7789f0 0x556d9ab808e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f8760de0740 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a799390 0x556d9ab6edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f8760de0740 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 0x556d9ab7a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f8760de0740 1 --2- 172.21.15.39:0/436880411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab6e870 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f873d600640 0 client.29047 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f8760de0740 1 -- 172.21.15.39:0/436880411 >> 172.21.15.39:0/436880411 conn(0x556d9a7ae460 msgr2=0x556d9ab61aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f8760de0740 1 -- 172.21.15.39:0/436880411 shutdown_connections 2024-07-04T08:31:39.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f8760de0740 1 -- 172.21.15.39:0/436880411 wait complete. 2024-07-04T08:31:39.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.381+0000 7f8760de0740 20 client.29047 trim_cache size 0 max 16384 2024-07-04T08:31:39.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.401+0000 7f8760de0740 1 Processor -- start 2024-07-04T08:31:39.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.402+0000 7f8760de0740 1 -- start start 2024-07-04T08:31:39.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.402+0000 7f8760de0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7911b0 0x556d9ab62460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.402+0000 7f8760de0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a799390 0x556d9ab7a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.402+0000 7f8760de0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 0x556d9ab60050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.402+0000 7f8760de0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87541aded0 con 0x556d9a7911b0 2024-07-04T08:31:39.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.402+0000 7f8760de0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f87541afa10 con 0x556d9a7ba070 2024-07-04T08:31:39.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.402+0000 7f8760de0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f87541ae5a0 con 0x556d9a799390 2024-07-04T08:31:39.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8752a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a799390 0x556d9ab7a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8752a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a799390 0x556d9ab7a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39872/0 (socket says 172.21.15.39:39872) 2024-07-04T08:31:39.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8752a00640 1 -- 172.21.15.39:0/1550459541 learned_addr learned my addr 172.21.15.39:0/1550459541 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:39.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8753400640 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7911b0 0x556d9ab62460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8753e00640 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 0x556d9ab60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8752a00640 1 -- 172.21.15.39:0/1550459541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 msgr2=0x556d9ab60050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8752a00640 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 0x556d9ab60050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8752a00640 1 -- 172.21.15.39:0/1550459541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7911b0 msgr2=0x556d9ab62460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8752a00640 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7911b0 0x556d9ab62460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8752a00640 1 -- 172.21.15.39:0/1550459541 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9ab6cb70 con 0x556d9a799390 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8753400640 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7911b0 0x556d9ab62460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.403+0000 7f8752a00640 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a799390 0x556d9ab7a510 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8740004bc0 tx=0x7f8740004bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f1586cee04a3d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.404+0000 7f873fe00640 1 -- 172.21.15.39:0/1550459541 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874003f040 con 0x556d9a799390 2024-07-04T08:31:39.391 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.404+0000 7f873fe00640 1 -- 172.21.15.39:0/1550459541 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87400046c0 con 0x556d9a799390 2024-07-04T08:31:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.404+0000 7f873fe00640 1 -- 172.21.15.39:0/1550459541 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740035040 con 0x556d9a799390 2024-07-04T08:31:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.405+0000 7f8760de0740 1 -- 172.21.15.39:0/1550459541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a799390 msgr2=0x556d9ab7a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.405+0000 7f8760de0740 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a799390 0x556d9ab7a510 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8740004bc0 tx=0x7f8740004bf0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.405+0000 7f8760de0740 1 -- 172.21.15.39:0/1550459541 shutdown_connections 2024-07-04T08:31:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.405+0000 7f8760de0740 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a7ba070 0x556d9ab60050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.405+0000 7f8760de0740 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a799390 0x556d9ab7a510 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.405+0000 7f8760de0740 1 --2- 172.21.15.39:0/1550459541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7911b0 0x556d9ab62460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.405+0000 7f8760de0740 1 -- 172.21.15.39:0/1550459541 >> 172.21.15.39:0/1550459541 conn(0x556d9a7ae460 msgr2=0x556d9ab5f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.405+0000 7f8760de0740 1 -- 172.21.15.39:0/1550459541 shutdown_connections 2024-07-04T08:31:39.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.405+0000 7f8760de0740 1 -- 172.21.15.39:0/1550459541 wait complete. 2024-07-04T08:31:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.406+0000 7f8760de0740 1 -- start start 2024-07-04T08:31:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.406+0000 7f8760de0740 1 Processor -- start 2024-07-04T08:31:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.407+0000 7f8760de0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab65800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.407+0000 7f8760de0740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a799390 0x556d9ab65d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.407+0000 7f8760de0740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7ba070 0x556d9ab75670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.407+0000 7f8760de0740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f87541af340 con 0x556d9a7ba070 2024-07-04T08:31:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.407+0000 7f8760de0740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f87541aec70 con 0x556d9a799390 2024-07-04T08:31:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.407+0000 7f8760de0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f87541abeb0 con 0x556d9a7911b0 2024-07-04T08:31:39.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.407+0000 7f8760de0740 10 client.0 _ll_register_callbacks cb 0x7f87440a9ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.407+0000 7f8753400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab65800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.407+0000 7f8753400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab65800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:39888/0 (socket says 172.21.15.39:39888) 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8753400640 1 -- 172.21.15.39:0/3636953616 learned_addr learned my addr 172.21.15.39:0/3636953616 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8752a00640 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a799390 0x556d9ab65d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8753e00640 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7ba070 0x556d9ab75670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8753400640 1 -- 172.21.15.39:0/3636953616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a799390 msgr2=0x556d9ab65d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8753400640 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a799390 0x556d9ab65d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8753400640 1 -- 172.21.15.39:0/3636953616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7ba070 msgr2=0x556d9ab75670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8753400640 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7ba070 0x556d9ab75670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8753400640 1 -- 172.21.15.39:0/3636953616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9ab75bb0 con 0x556d9a7911b0 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8753e00640 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7ba070 0x556d9ab75670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f8753400640 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab65800 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f874400cfd0 tx=0x7f8744009720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca8037fda5ba47fb server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:39.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.408+0000 7f873d600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:31:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.409+0000 7f873d600640 1 -- 172.21.15.39:0/3636953616 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440a9540 con 0x556d9a7911b0 2024-07-04T08:31:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.409+0000 7f873d600640 1 -- 172.21.15.39:0/3636953616 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f87440a63c0 con 0x556d9a7911b0 2024-07-04T08:31:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.409+0000 7f8760de0740 1 -- 172.21.15.39:0/3636953616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87400046c0 con 0x556d9a7911b0 2024-07-04T08:31:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.409+0000 7f873d600640 1 -- 172.21.15.39:0/3636953616 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87440a98e0 con 0x556d9a7911b0 2024-07-04T08:31:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.409+0000 7f8760de0740 10 client.29053 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:39.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.409+0000 7f8760de0740 1 -- 172.21.15.39:0/3636953616 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d9ab76e40 con 0x556d9a7911b0 2024-07-04T08:31:39.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.409+0000 7f873d600640 1 -- 172.21.15.39:0/3636953616 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87400046c0 con 0x556d9a7911b0 2024-07-04T08:31:39.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.410+0000 7f873d600640 1 -- 172.21.15.39:0/3636953616 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8744091040 con 0x556d9a7911b0 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.410+0000 7f8760de0740 10 client.29053 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.410+0000 7f8760de0740 10 client.29053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.410+0000 7f8760de0740 20 client.29053 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.410+0000 7f8760de0740 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556d9a74dda0 0x556d9abef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.410+0000 7f8760de0740 4 client.29053 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.410+0000 7f8760de0740 1 -- 172.21.15.39:0/3636953616 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d9aacffd0 con 0x556d9a74dda0 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.410+0000 7f8752a00640 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556d9a74dda0 0x556d9abef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.411+0000 7f873d600640 10 client.29053 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.411+0000 7f8752a00640 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556d9a74dda0 0x556d9abef870 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7aa29b12a88ab65 server_cookie=5c58da57296b491e in_seq=0 out_seq=0 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.411+0000 7f873d600640 1 -- 172.21.15.39:0/3636953616 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556d9aacffd0 con 0x556d9a74dda0 2024-07-04T08:31:39.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.411+0000 7f873d600640 10 client.29053 handle_command_reply: tid=0 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.411+0000 7f8760de0740 1 client.29053 shutdown 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.411+0000 7f8760de0740 1 -- 172.21.15.39:0/3636953616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 msgr2=0x556d9ab65800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.411+0000 7f8760de0740 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab65800 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f874400cfd0 tx=0x7f8744009720 comp rx=0 tx=0).stop 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f8760de0740 1 -- 172.21.15.39:0/3636953616 shutdown_connections 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f8760de0740 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x556d9a74dda0 0x556d9abef870 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f8760de0740 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x556d9a7ba070 0x556d9ab75670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f8760de0740 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556d9a799390 0x556d9ab65d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f8760de0740 1 --2- 172.21.15.39:0/3636953616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556d9a7911b0 0x556d9ab65800 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f873d600640 0 client.29053 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f8760de0740 1 -- 172.21.15.39:0/3636953616 >> 172.21.15.39:0/3636953616 conn(0x556d9a7ae460 msgr2=0x556d9ab5f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f8760de0740 1 -- 172.21.15.39:0/3636953616 shutdown_connections 2024-07-04T08:31:39.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f8760de0740 1 -- 172.21.15.39:0/3636953616 wait complete. 2024-07-04T08:31:39.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.412+0000 7f8760de0740 20 client.29053 trim_cache size 0 max 16384 2024-07-04T08:31:39.401 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:31:39.401 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "no active scrubs running", 2024-07-04T08:31:39.401 INFO:teuthology.orchestra.run.smithi039.stdout: "scrubs": {} 2024-07-04T08:31:39.401 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:31:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 -- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f87380b29e0 msgr2=0x7f87380b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f87380b29e0 0x7f87380b4ef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f874c005fd0 tx=0x7f874c009500 comp rx=0 tx=0).stop 2024-07-04T08:31:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 -- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8754103d10 msgr2=0x7f8754199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8754103d10 0x7f8754199870 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f874400e9f0 tx=0x7f874400edf0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 -- 172.21.15.39:0/3728624276 shutdown_connections 2024-07-04T08:31:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f87380b29e0 0x7f87380b4ef0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f875410d170 0x7f875419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8754104660 0x7f8754199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 --2- 172.21.15.39:0/3728624276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8754103d10 0x7f8754199870 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.413+0000 7f8759200640 1 -- 172.21.15.39:0/3728624276 >> 172.21.15.39:0/3728624276 conn(0x7f87540fd810 msgr2=0x7f8754106c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.414+0000 7f8759200640 1 -- 172.21.15.39:0/3728624276 shutdown_connections 2024-07-04T08:31:39.403 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:39.414+0000 7f8759200640 1 -- 172.21.15.39:0/3728624276 wait complete. 2024-07-04T08:31:39.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:39 smithi039 ceph-mon[32062]: pgmap v993: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:39.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:39 smithi050 ceph-mon[36577]: pgmap v993: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:39.649 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:39.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:39 smithi133 ceph-mon[37050]: pgmap v993: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:40.326 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-04T08:31:40.326 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:31:40.528 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:40 smithi039 ceph-mon[32062]: pgmap v994: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:40.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:40 smithi050 ceph-mon[36577]: pgmap v994: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:40.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:40 smithi133 ceph-mon[37050]: pgmap v994: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 14 op/s 2024-07-04T08:31:41.667 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:42.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.820+0000 7f40d3400640 1 -- 172.21.15.39:0/1044395871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40cc103db0 msgr2=0x7f40cc1041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:42.807 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.820+0000 7f40d3400640 1 --2- 172.21.15.39:0/1044395871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40cc103db0 0x7f40cc1041c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009860 tx=0x7f40bc02a340 comp rx=0 tx=0).stop 2024-07-04T08:31:42.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.820+0000 7f40d3400640 1 -- 172.21.15.39:0/1044395871 shutdown_connections 2024-07-04T08:31:42.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.820+0000 7f40d3400640 1 --2- 172.21.15.39:0/1044395871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40cc10d360 0x7f40cc10f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.820+0000 7f40d3400640 1 --2- 172.21.15.39:0/1044395871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40cc1047e0 0x7f40cc10ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.820+0000 7f40d3400640 1 --2- 172.21.15.39:0/1044395871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40cc103db0 0x7f40cc1041c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.820+0000 7f40d3400640 1 -- 172.21.15.39:0/1044395871 >> 172.21.15.39:0/1044395871 conn(0x7f40cc078c80 msgr2=0x7f40cc079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:42.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.820+0000 7f40d3400640 1 -- 172.21.15.39:0/1044395871 shutdown_connections 2024-07-04T08:31:42.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.820+0000 7f40d3400640 1 -- 172.21.15.39:0/1044395871 wait complete. 2024-07-04T08:31:42.808 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.822+0000 7f40d3400640 1 Processor -- start 2024-07-04T08:31:42.809 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.822+0000 7f40d3400640 1 -- start start 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.822+0000 7f40d3400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40cc103db0 0x7f40cc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40d3400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40cc1047e0 0x7f40cc199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40d3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40cc10d360 0x7f40cc19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40d3400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc112fb0 con 0x7f40cc103db0 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40d3400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc112e30 con 0x7f40cc1047e0 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40d3400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc113130 con 0x7f40cc10d360 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40d0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40cc103db0 0x7f40cc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40d0a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40cc103db0 0x7f40cc1999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56412/0 (socket says 172.21.15.39:56412) 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40d0a00640 1 -- 172.21.15.39:0/1070745958 learned_addr learned my addr 172.21.15.39:0/1070745958 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:42.810 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40cbe00640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40cc1047e0 0x7f40cc199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.823+0000 7f40d1400640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40cc10d360 0x7f40cc19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.824+0000 7f40d0a00640 1 -- 172.21.15.39:0/1070745958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40cc10d360 msgr2=0x7f40cc19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:42.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.824+0000 7f40d0a00640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40cc10d360 0x7f40cc19f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.811 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.824+0000 7f40d0a00640 1 -- 172.21.15.39:0/1070745958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40cc1047e0 msgr2=0x7f40cc199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:42.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.824+0000 7f40d0a00640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40cc1047e0 0x7f40cc199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.824+0000 7f40d0a00640 1 -- 172.21.15.39:0/1070745958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc19f910 con 0x7f40cc103db0 2024-07-04T08:31:42.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.824+0000 7f40d0a00640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40cc103db0 0x7f40cc1999a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f40bc035040 tx=0x7f40bc032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:42.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.824+0000 7f40c9600640 1 -- 172.21.15.39:0/1070745958 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc037040 con 0x7f40cc103db0 2024-07-04T08:31:42.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.824+0000 7f40c9600640 1 -- 172.21.15.39:0/1070745958 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40bc041070 con 0x7f40cc103db0 2024-07-04T08:31:42.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.824+0000 7f40d3400640 1 -- 172.21.15.39:0/1070745958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc19fba0 con 0x7f40cc103db0 2024-07-04T08:31:42.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.825+0000 7f40c9600640 1 -- 172.21.15.39:0/1070745958 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40bc003850 con 0x7f40cc103db0 2024-07-04T08:31:42.812 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.825+0000 7f40d3400640 1 -- 172.21.15.39:0/1070745958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc1a0020 con 0x7f40cc103db0 2024-07-04T08:31:42.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.828+0000 7f40d99fc740 1 Processor -- start 2024-07-04T08:31:42.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.829+0000 7f40d99fc740 1 -- start start 2024-07-04T08:31:42.815 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.829+0000 7f40c9600640 1 -- 172.21.15.39:0/1070745958 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f40bc004050 con 0x7f40cc103db0 2024-07-04T08:31:42.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.829+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392b9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.829+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38eec410 0x563c392ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.829+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.829+0000 7f40d99fc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563c3922fea0 con 0x563c38ef45f0 2024-07-04T08:31:42.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.829+0000 7f40d99fc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563c3921f440 con 0x563c38f152d0 2024-07-04T08:31:42.816 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.829+0000 7f40d99fc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c39223480 con 0x563c38eec410 2024-07-04T08:31:42.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40d0a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38eec410 0x563c392ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ba5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56420/0 (socket says 172.21.15.39:56420) 2024-07-04T08:31:42.817 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40cbe00640 1 -- 172.21.15.39:0/2593632862 learned_addr learned my addr 172.21.15.39:0/2593632862 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40d1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392b9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40d1400640 1 -- 172.21.15.39:0/2593632862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38eec410 msgr2=0x563c392ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40d1400640 1 --2- 172.21.15.39:0/2593632862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38eec410 0x563c392ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40d1400640 1 -- 172.21.15.39:0/2593632862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 msgr2=0x563c392ba5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40d1400640 1 --2- 172.21.15.39:0/2593632862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ba5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40d1400640 1 -- 172.21.15.39:0/2593632862 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c38f808b0 con 0x563c38f152d0 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.830+0000 7f40d1400640 1 --2- 172.21.15.39:0/2593632862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392b9b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009b80 tx=0x7f40c4009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaae466a046c5b80 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.831+0000 7f40c2000640 1 -- 172.21.15.39:0/2593632862 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4004310 con 0x563c38f152d0 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.831+0000 7f40c2000640 1 -- 172.21.15.39:0/2593632862 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c40044b0 con 0x563c38f152d0 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.831+0000 7f40c2000640 1 -- 172.21.15.39:0/2593632862 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c40035f0 con 0x563c38f152d0 2024-07-04T08:31:42.818 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.831+0000 7f40c9600640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40ac0b2960 0x7f40ac0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.831+0000 7f40d0a00640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40ac0b2960 0x7f40ac0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.831+0000 7f40c9600640 1 -- 172.21.15.39:0/1070745958 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f40bc102460 con 0x7f40cc103db0 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d99fc740 1 -- 172.21.15.39:0/2593632862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 msgr2=0x563c392b9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d99fc740 1 --2- 172.21.15.39:0/2593632862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392b9b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009b80 tx=0x7f40c4009f80 comp rx=0 tx=0).stop 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d0a00640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40ac0b2960 0x7f40ac0b4e70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0ff960 tx=0x7f40bc105f10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d99fc740 1 -- 172.21.15.39:0/2593632862 shutdown_connections 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d99fc740 1 --2- 172.21.15.39:0/2593632862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d99fc740 1 --2- 172.21.15.39:0/2593632862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38eec410 0x563c392ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d99fc740 1 --2- 172.21.15.39:0/2593632862 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392b9b50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d99fc740 1 -- 172.21.15.39:0/2593632862 >> 172.21.15.39:0/2593632862 conn(0x563c38f096c0 msgr2=0x563c392b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d99fc740 1 -- 172.21.15.39:0/2593632862 shutdown_connections 2024-07-04T08:31:42.820 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.832+0000 7f40d99fc740 1 -- 172.21.15.39:0/2593632862 wait complete. 2024-07-04T08:31:42.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.833+0000 7f40d99fc740 1 -- start start 2024-07-04T08:31:42.821 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.833+0000 7f40d99fc740 1 Processor -- start 2024-07-04T08:31:42.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 0x563c392d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d99fc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x563c39083810 con 0x563c38ef45f0 2024-07-04T08:31:42.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d99fc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563c391c6900 con 0x563c38eec410 2024-07-04T08:31:42.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d99fc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563c392cc660 con 0x563c38f152d0 2024-07-04T08:31:42.822 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d99fc740 10 client.0 _ll_register_callbacks cb 0x563c391c72f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56426/0 (socket says 172.21.15.39:56426) 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40cbe00640 1 -- 172.21.15.39:0/1698837665 learned_addr learned my addr 172.21.15.39:0/1698837665 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d1400640 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45832/0 (socket says 172.21.15.39:45832) 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.834+0000 7f40d1400640 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 0x563c392d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.835+0000 7f40cbe00640 1 -- 172.21.15.39:0/1698837665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 msgr2=0x563c392d59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.835+0000 7f40cbe00640 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 0x563c392d59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.835+0000 7f40cbe00640 1 -- 172.21.15.39:0/1698837665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 msgr2=0x563c392c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.835+0000 7f40cbe00640 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.835+0000 7f40cbe00640 1 -- 172.21.15.39:0/1698837665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c391b6b60 con 0x563c38ef45f0 2024-07-04T08:31:42.823 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.835+0000 7f40d1400640 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:42.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.835+0000 7f40cbe00640 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ca030 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009c70 tx=0x7f40b400d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373f3a9dd34687a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:42.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.835+0000 7f40baa00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:31:42.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.835+0000 7f40baa00640 1 -- 172.21.15.39:0/1698837665 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b401c030 con 0x563c38ef45f0 2024-07-04T08:31:42.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.836+0000 7f40baa00640 1 -- 172.21.15.39:0/1698837665 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f40b4018460 con 0x563c38ef45f0 2024-07-04T08:31:42.824 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.836+0000 7f40d99fc740 1 -- 172.21.15.39:0/1698837665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c40044b0 con 0x563c38ef45f0 2024-07-04T08:31:42.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.836+0000 7f40baa00640 1 -- 172.21.15.39:0/1698837665 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40b4026050 con 0x563c38ef45f0 2024-07-04T08:31:42.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.836+0000 7f40d99fc740 10 client.19017 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:42.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.836+0000 7f40d99fc740 1 -- 172.21.15.39:0/1698837665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c392d8740 con 0x563c38ef45f0 2024-07-04T08:31:42.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.836+0000 7f40baa00640 1 -- 172.21.15.39:0/1698837665 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c40044b0 con 0x563c38ef45f0 2024-07-04T08:31:42.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.837+0000 7f40baa00640 1 -- 172.21.15.39:0/1698837665 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f40b400b040 con 0x563c38ef45f0 2024-07-04T08:31:42.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.837+0000 7f40d99fc740 10 client.19017 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:42.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.837+0000 7f40d99fc740 10 client.19017 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:31:42.825 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.837+0000 7f40d99fc740 20 client.19017 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:42.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.837+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563c38ed3c50 0x563c392dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.837+0000 7f40d99fc740 4 client.19017 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:42.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.837+0000 7f40d99fc740 1 -- 172.21.15.39:0/1698837665 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c39231df0 con 0x563c38ed3c50 2024-07-04T08:31:42.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.837+0000 7f40d1400640 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563c38ed3c50 0x563c392dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.838+0000 7f40d1400640 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563c38ed3c50 0x563c392dbc90 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5acc5c805269e261 server_cookie=ab896e55e932a401 in_seq=0 out_seq=0 2024-07-04T08:31:42.826 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.838+0000 7f40baa00640 10 client.19017 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40baa00640 1 -- 172.21.15.39:0/1698837665 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563c39231df0 con 0x563c38ed3c50 2024-07-04T08:31:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40baa00640 10 client.19017 handle_command_reply: tid=0 2024-07-04T08:31:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40d99fc740 1 client.19017 shutdown 2024-07-04T08:31:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40d99fc740 1 -- 172.21.15.39:0/1698837665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 msgr2=0x563c392ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ca030 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009c70 tx=0x7f40b400d7b0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.827 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40d99fc740 1 -- 172.21.15.39:0/1698837665 shutdown_connections 2024-07-04T08:31:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563c38ed3c50 0x563c392dbc90 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392ca030 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 0x563c392d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1698837665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40d99fc740 1 -- 172.21.15.39:0/1698837665 >> 172.21.15.39:0/1698837665 conn(0x563c38f096c0 msgr2=0x563c392bcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.839+0000 7f40baa00640 0 client.19017 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.840+0000 7f40d99fc740 1 -- 172.21.15.39:0/1698837665 shutdown_connections 2024-07-04T08:31:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.840+0000 7f40d99fc740 1 -- 172.21.15.39:0/1698837665 wait complete. 2024-07-04T08:31:42.828 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.840+0000 7f40d99fc740 20 client.19017 trim_cache size 0 max 16384 2024-07-04T08:31:42.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.860+0000 7f40d99fc740 1 Processor -- start 2024-07-04T08:31:42.847 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.861+0000 7f40d99fc740 1 -- start start 2024-07-04T08:31:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.861+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.861+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.861+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 0x563c392d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.861+0000 7f40d99fc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1ae110 con 0x563c38ef45f0 2024-07-04T08:31:42.848 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.861+0000 7f40d99fc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1afc50 con 0x563c38eec410 2024-07-04T08:31:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.861+0000 7f40d99fc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1ae7e0 con 0x563c38f152d0 2024-07-04T08:31:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.861+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40d1400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392d57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56442/0 (socket says 172.21.15.39:56442) 2024-07-04T08:31:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40cbe00640 1 -- 172.21.15.39:0/1949323261 learned_addr learned my addr 172.21.15.39:0/1949323261 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:42.849 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40d1400640 1 --2- 172.21.15.39:0/1949323261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 0x563c392d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40cbe00640 1 -- 172.21.15.39:0/1949323261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 msgr2=0x563c392d5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40cbe00640 1 --2- 172.21.15.39:0/1949323261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 0x563c392d5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40cbe00640 1 -- 172.21.15.39:0/1949323261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 msgr2=0x563c392b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40cbe00640 1 --2- 172.21.15.39:0/1949323261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40cbe00640 1 -- 172.21.15.39:0/1949323261 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c392b8330 con 0x563c38ef45f0 2024-07-04T08:31:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.862+0000 7f40d1400640 1 --2- 172.21.15.39:0/1949323261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.863+0000 7f40cbe00640 1 --2- 172.21.15.39:0/1949323261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392d57e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f40b40099b0 tx=0x7f40b4009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5123fc6cde941856 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.863+0000 7f40c1600640 1 -- 172.21.15.39:0/1949323261 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b402e070 con 0x563c38ef45f0 2024-07-04T08:31:42.850 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.863+0000 7f40c1600640 1 -- 172.21.15.39:0/1949323261 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b4021c20 con 0x563c38ef45f0 2024-07-04T08:31:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.863+0000 7f40c1600640 1 -- 172.21.15.39:0/1949323261 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b401c040 con 0x563c38ef45f0 2024-07-04T08:31:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.864+0000 7f40d99fc740 1 -- 172.21.15.39:0/1949323261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 msgr2=0x563c392d57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.864+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1949323261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392d57e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f40b40099b0 tx=0x7f40b4009630 comp rx=0 tx=0).stop 2024-07-04T08:31:42.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.864+0000 7f40d99fc740 1 -- 172.21.15.39:0/1949323261 shutdown_connections 2024-07-04T08:31:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.864+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1949323261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38ef45f0 0x563c392d57e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.864+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1949323261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38f152d0 0x563c392d5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.864+0000 7f40d99fc740 1 --2- 172.21.15.39:0/1949323261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38eec410 0x563c392b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.864+0000 7f40d99fc740 1 -- 172.21.15.39:0/1949323261 >> 172.21.15.39:0/1949323261 conn(0x563c38f096c0 msgr2=0x563c392bcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.864+0000 7f40d99fc740 1 -- 172.21.15.39:0/1949323261 shutdown_connections 2024-07-04T08:31:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.864+0000 7f40d99fc740 1 -- 172.21.15.39:0/1949323261 wait complete. 2024-07-04T08:31:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.865+0000 7f40d99fc740 1 -- start start 2024-07-04T08:31:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.865+0000 7f40d99fc740 1 Processor -- start 2024-07-04T08:31:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.865+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38eec410 0x563c392d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.865+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d99fc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38ea9000 0x563c392cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d99fc740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1af580 con 0x563c38eec410 2024-07-04T08:31:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d99fc740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1aeeb0 con 0x563c38f152d0 2024-07-04T08:31:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d99fc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1ac0f0 con 0x563c38ea9000 2024-07-04T08:31:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d99fc740 10 client.0 _ll_register_callbacks cb 0x7f40b400ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38eec410 0x563c392d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.854 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d1400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38eec410 0x563c392d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56444/0 (socket says 172.21.15.39:56444) 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d1400640 1 -- 172.21.15.39:0/909162721 learned_addr learned my addr 172.21.15.39:0/909162721 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40cbe00640 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d1400640 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38ea9000 0x563c392cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40cbe00640 1 -- 172.21.15.39:0/909162721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38ea9000 msgr2=0x563c392cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40cbe00640 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38ea9000 0x563c392cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40cbe00640 1 -- 172.21.15.39:0/909162721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38eec410 msgr2=0x563c392d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40cbe00640 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38eec410 0x563c392d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40cbe00640 1 -- 172.21.15.39:0/909162721 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c392ce120 con 0x563c38f152d0 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.866+0000 7f40d1400640 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38ea9000 0x563c392cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.867+0000 7f40cbe00640 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392cd4a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x563c392d8630 tx=0x7f40b4021400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a971f44b4af5b9ad server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.867+0000 7f40baa00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:42.855 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.867+0000 7f40baa00640 1 -- 172.21.15.39:0/909162721 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4038020 con 0x563c38f152d0 2024-07-04T08:31:42.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.867+0000 7f40baa00640 1 -- 172.21.15.39:0/909162721 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f40b4003b80 con 0x563c38f152d0 2024-07-04T08:31:42.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.867+0000 7f40d99fc740 1 -- 172.21.15.39:0/909162721 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b4021c20 con 0x563c38f152d0 2024-07-04T08:31:42.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.867+0000 7f40baa00640 1 -- 172.21.15.39:0/909162721 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40b4043050 con 0x563c38f152d0 2024-07-04T08:31:42.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.867+0000 7f40d99fc740 10 client.29144 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:42.856 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.867+0000 7f40d99fc740 1 -- 172.21.15.39:0/909162721 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c393442b0 con 0x563c38f152d0 2024-07-04T08:31:42.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.868+0000 7f40baa00640 1 -- 172.21.15.39:0/909162721 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b4021c20 con 0x563c38f152d0 2024-07-04T08:31:42.857 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.868+0000 7f40baa00640 1 -- 172.21.15.39:0/909162721 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f40b400b040 con 0x563c38f152d0 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.868+0000 7f40d99fc740 10 client.29144 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.868+0000 7f40d99fc740 10 client.29144 resolve_mds: validated gid 24707 aka daemon mds.b 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.868+0000 7f40d99fc740 20 client.29144 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.868+0000 7f40d99fc740 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563c393448c0 0x563c39346ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.868+0000 7f40d99fc740 4 client.29144 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.868+0000 7f40d99fc740 1 -- 172.21.15.39:0/909162721 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x563c3922d890 con 0x563c393448c0 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.869+0000 7f40d1400640 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563c393448c0 0x563c39346ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.869+0000 7f40d1400640 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563c393448c0 0x563c39346ce0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee8cdcab09bd60b server_cookie=61963b2a95b58621 in_seq=0 out_seq=0 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.869+0000 7f40baa00640 10 client.29144 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40baa00640 1 -- 172.21.15.39:0/909162721 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x563c3922d890 con 0x563c393448c0 2024-07-04T08:31:42.858 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40baa00640 10 client.29144 handle_command_reply: tid=0 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 client.29144 shutdown 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 -- 172.21.15.39:0/909162721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 msgr2=0x563c392cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392cd4a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x563c392d8630 tx=0x7f40b4021400 comp rx=0 tx=0).stop 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 -- 172.21.15.39:0/909162721 shutdown_connections 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x563c393448c0 0x563c39346ce0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563c38ea9000 0x563c392cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563c38f152d0 0x563c392cd4a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 --2- 172.21.15.39:0/909162721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563c38eec410 0x563c392d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40baa00640 0 client.29144 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 -- 172.21.15.39:0/909162721 >> 172.21.15.39:0/909162721 conn(0x563c38f096c0 msgr2=0x563c392bab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 -- 172.21.15.39:0/909162721 shutdown_connections 2024-07-04T08:31:42.859 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.870+0000 7f40d99fc740 1 -- 172.21.15.39:0/909162721 wait complete. 2024-07-04T08:31:42.860 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.871+0000 7f40d99fc740 20 client.29144 trim_cache size 0 max 16384 2024-07-04T08:31:42.860 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:31:42.860 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 30, 2024-07-04T08:31:42.860 INFO:teuthology.orchestra.run.smithi039.stdout: "leader": 24707, 2024-07-04T08:31:42.860 INFO:teuthology.orchestra.run.smithi039.stdout: "set_version": 48, 2024-07-04T08:31:42.860 INFO:teuthology.orchestra.run.smithi039.stdout: "sets": {} 2024-07-04T08:31:42.860 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.871+0000 7f40d3400640 1 -- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40ac0b2960 msgr2=0x7f40ac0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.871+0000 7f40d3400640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40ac0b2960 0x7f40ac0b4e70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0ff960 tx=0x7f40bc105f10 comp rx=0 tx=0).stop 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.871+0000 7f40d3400640 1 -- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40cc103db0 msgr2=0x7f40cc1999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.871+0000 7f40d3400640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40cc103db0 0x7f40cc1999a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f40bc035040 tx=0x7f40bc032d50 comp rx=0 tx=0).stop 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.872+0000 7f40d3400640 1 -- 172.21.15.39:0/1070745958 shutdown_connections 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.872+0000 7f40d3400640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40ac0b2960 0x7f40ac0b4e70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.872+0000 7f40d3400640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40cc10d360 0x7f40cc19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.872+0000 7f40d3400640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f40cc1047e0 0x7f40cc199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.872+0000 7f40d3400640 1 --2- 172.21.15.39:0/1070745958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40cc103db0 0x7f40cc1999a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:42.861 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.872+0000 7f40d3400640 1 -- 172.21.15.39:0/1070745958 >> 172.21.15.39:0/1070745958 conn(0x7f40cc078c80 msgr2=0x7f40cc0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.872+0000 7f40d3400640 1 -- 172.21.15.39:0/1070745958 shutdown_connections 2024-07-04T08:31:42.862 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:42.872+0000 7f40d3400640 1 -- 172.21.15.39:0/1070745958 wait complete. 2024-07-04T08:31:43.473 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:43 smithi039 ceph-mon[32062]: pgmap v995: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-07-04T08:31:43.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:43 smithi050 ceph-mon[36577]: pgmap v995: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-07-04T08:31:43.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:43 smithi133 ceph-mon[37050]: pgmap v995: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 61 KiB/s wr, 16 op/s 2024-07-04T08:31:43.945 INFO:tasks.quiescer:done joining Quiescers 2024-07-04T08:31:43.961 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-07-04T08:31:44.271 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-07-04T08:31:44.272 INFO:teuthology.task.exec:Executing custom commands... 2024-07-04T08:31:44.272 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T08:31:44.272 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-07-04T08:31:44.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-07-04T08:31:44.470 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-04T08:31:44.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.756+0000 7ff94fe00640 1 -- 172.21.15.39:0/2940830901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff950108790 msgr2=0x7ff950108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:44.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.756+0000 7ff94fe00640 1 --2- 172.21.15.39:0/2940830901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff950108790 0x7ff950108ba0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff938009900 tx=0x7ff93802a410 comp rx=0 tx=0).stop 2024-07-04T08:31:44.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.756+0000 7ff94fe00640 1 -- 172.21.15.39:0/2940830901 shutdown_connections 2024-07-04T08:31:44.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.756+0000 7ff94fe00640 1 --2- 172.21.15.39:0/2940830901 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9501090e0 0x7ff95010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:44.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.756+0000 7ff94fe00640 1 --2- 172.21.15.39:0/2940830901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff950108790 0x7ff950108ba0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:44.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.756+0000 7ff94fe00640 1 --2- 172.21.15.39:0/2940830901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950102e60 0x7ff950103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:44.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.756+0000 7ff94fe00640 1 -- 172.21.15.39:0/2940830901 >> 172.21.15.39:0/2940830901 conn(0x7ff9500fe880 msgr2=0x7ff950100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:44.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.756+0000 7ff94fe00640 1 -- 172.21.15.39:0/2940830901 shutdown_connections 2024-07-04T08:31:44.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.756+0000 7ff94fe00640 1 -- 172.21.15.39:0/2940830901 wait complete. 2024-07-04T08:31:44.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.757+0000 7ff94fe00640 1 Processor -- start 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.757+0000 7ff94fe00640 1 -- start start 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff94fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff950102e60 0x7ff95019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff94fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108790 0x7ff95019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff94fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9501090e0 0x7ff9501a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff94fe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff950112df0 con 0x7ff950108790 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff94fe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff950112c70 con 0x7ff9501090e0 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff94fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff950112f70 con 0x7ff950102e60 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff947e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108790 0x7ff95019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff947e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108790 0x7ff95019bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56464/0 (socket says 172.21.15.39:56464) 2024-07-04T08:31:44.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff947e00640 1 -- 172.21.15.39:0/1917145318 learned_addr learned my addr 172.21.15.39:0/1917145318 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff94f400640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9501090e0 0x7ff9501a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.758+0000 7ff94ea00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff950102e60 0x7ff95019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.759+0000 7ff947e00640 1 -- 172.21.15.39:0/1917145318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff950102e60 msgr2=0x7ff95019b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.759+0000 7ff947e00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff950102e60 0x7ff95019b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.759+0000 7ff947e00640 1 -- 172.21.15.39:0/1917145318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9501090e0 msgr2=0x7ff9501a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.759+0000 7ff947e00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9501090e0 0x7ff9501a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.759+0000 7ff947e00640 1 -- 172.21.15.39:0/1917145318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9501a1900 con 0x7ff950108790 2024-07-04T08:31:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.759+0000 7ff94f400640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9501090e0 0x7ff9501a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:44.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.759+0000 7ff947e00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108790 0x7ff95019bf70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff938005ab0 tx=0x7ff938003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:44.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.760+0000 7ff954a00640 1 -- 172.21.15.39:0/1917145318 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93803f070 con 0x7ff950108790 2024-07-04T08:31:44.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.760+0000 7ff954a00640 1 -- 172.21.15.39:0/1917145318 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff938031b50 con 0x7ff950108790 2024-07-04T08:31:44.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.760+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9501a1b90 con 0x7ff950108790 2024-07-04T08:31:44.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.760+0000 7ff954a00640 1 -- 172.21.15.39:0/1917145318 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938035030 con 0x7ff950108790 2024-07-04T08:31:44.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.760+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501a8380 con 0x7ff950108790 2024-07-04T08:31:44.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.761+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff950103270 con 0x7ff950108790 2024-07-04T08:31:44.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.765+0000 7ff954a00640 1 -- 172.21.15.39:0/1917145318 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff938042070 con 0x7ff950108790 2024-07-04T08:31:44.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.767+0000 7ff954a00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9340b29e0 0x7ff9340b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:44.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.768+0000 7ff954a00640 1 -- 172.21.15.39:0/1917145318 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7ff938103360 con 0x7ff950108790 2024-07-04T08:31:44.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.768+0000 7ff94ea00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9340b29e0 0x7ff9340b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:44.755 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.769+0000 7ff94ea00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9340b29e0 0x7ff9340b4ef0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff9400095f0 tx=0x7ff940005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:44.758 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:44.771+0000 7ff954a00640 1 -- 172.21.15.39:0/1917145318 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7ff9380ca330 con 0x7ff950108790 2024-07-04T08:31:45.010 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.023+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff95010ebe0 con 0x7ff9340b29e0 2024-07-04T08:31:45.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.036+0000 7ff954a00640 1 -- 172.21.15.39:0/1917145318 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7ff95010ebe0 con 0x7ff9340b29e0 2024-07-04T08:31:45.023 INFO:teuthology.orchestra.run.smithi039.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-04T00:00:00", "created": "2024-07-04T08:15:11", "first": "2024-07-04T08:16:00", "last": "2024-07-04T08:31:00", "last_pruned": "2024-07-04T08:31:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9340b29e0 msgr2=0x7ff9340b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9340b29e0 0x7ff9340b4ef0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff9400095f0 tx=0x7ff940005f90 comp rx=0 tx=0).stop 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108790 msgr2=0x7ff95019bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108790 0x7ff95019bf70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff938005ab0 tx=0x7ff938003b90 comp rx=0 tx=0).stop 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 shutdown_connections 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff9340b29e0 0x7ff9340b4ef0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff950108790 0x7ff95019bf70 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff9501090e0 0x7ff9501a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 --2- 172.21.15.39:0/1917145318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff950102e60 0x7ff95019b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 >> 172.21.15.39:0/1917145318 conn(0x7ff9500fe880 msgr2=0x7ff950107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 shutdown_connections 2024-07-04T08:31:45.026 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.039+0000 7ff94fe00640 1 -- 172.21.15.39:0/1917145318 wait complete. 2024-07-04T08:31:45.053 DEBUG:teuthology.orchestra.run.smithi039:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-04T08:31:45.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.300+0000 7f770d200640 1 -- 172.21.15.39:0/2055891858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708104610 msgr2=0x7f770810cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.300+0000 7f770d200640 1 --2- 172.21.15.39:0/2055891858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708104610 0x7f770810cc50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009860 tx=0x7f76f402a340 comp rx=0 tx=0).stop 2024-07-04T08:31:45.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.300+0000 7f770d200640 1 -- 172.21.15.39:0/2055891858 shutdown_connections 2024-07-04T08:31:45.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.300+0000 7f770d200640 1 --2- 172.21.15.39:0/2055891858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770810d190 0x7f770810f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.288 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.300+0000 7f770d200640 1 --2- 172.21.15.39:0/2055891858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708104610 0x7f770810cc50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.300+0000 7f770d200640 1 --2- 172.21.15.39:0/2055891858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7708103be0 0x7f7708103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.300+0000 7f770d200640 1 -- 172.21.15.39:0/2055891858 >> 172.21.15.39:0/2055891858 conn(0x7f77080fd810 msgr2=0x7f77080ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:45.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.300+0000 7f770d200640 1 -- 172.21.15.39:0/2055891858 shutdown_connections 2024-07-04T08:31:45.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.300+0000 7f770d200640 1 -- 172.21.15.39:0/2055891858 wait complete. 2024-07-04T08:31:45.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.301+0000 7f770d200640 1 Processor -- start 2024-07-04T08:31:45.289 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.302+0000 7f770d200640 1 -- start start 2024-07-04T08:31:45.290 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.302+0000 7f770d200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708103be0 0x7f7708101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.302+0000 7f770d200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7708104610 0x7f7708102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.302+0000 7f770d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770810d190 0x7f7708102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.302+0000 7f770d200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f7708112dd0 con 0x7f7708104610 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.302+0000 7f770d200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7708112c50 con 0x7f7708103be0 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.302+0000 7f770d200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7708112f50 con 0x7f770810d190 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7707400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708103be0 0x7f7708101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7707e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770810d190 0x7f7708102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7707400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708103be0 0x7f7708101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45910/0 (socket says 172.21.15.39:45910) 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7707400640 1 -- 172.21.15.39:0/490922435 learned_addr learned my addr 172.21.15.39:0/490922435 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:45.291 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7706a00640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7708104610 0x7f7708102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7707400640 1 -- 172.21.15.39:0/490922435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770810d190 msgr2=0x7f7708102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7707400640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770810d190 0x7f7708102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7707400640 1 -- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7708104610 msgr2=0x7f7708102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7707400640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7708104610 0x7f7708102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.303+0000 7f7707400640 1 -- 172.21.15.39:0/490922435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77081a7a80 con 0x7f7708103be0 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.304+0000 7f7707400640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708103be0 0x7f7708101af0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f76f800e9c0 tx=0x7f76f800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.304+0000 7f76ffe00640 1 -- 172.21.15.39:0/490922435 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f800ccc0 con 0x7f7708103be0 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.304+0000 7f76ffe00640 1 -- 172.21.15.39:0/490922435 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76f80054d0 con 0x7f7708103be0 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.304+0000 7f76ffe00640 1 -- 172.21.15.39:0/490922435 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8010430 con 0x7f7708103be0 2024-07-04T08:31:45.292 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.304+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77081a7d70 con 0x7f7708103be0 2024-07-04T08:31:45.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.304+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77081a8270 con 0x7f7708103be0 2024-07-04T08:31:45.293 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.305+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7708103ff0 con 0x7f7708103be0 2024-07-04T08:31:45.295 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.308+0000 7f76ffe00640 1 -- 172.21.15.39:0/490922435 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f76f8004f80 con 0x7f7708103be0 2024-07-04T08:31:45.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.310+0000 7f76ffe00640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76ec0b29e0 0x7f76ec0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:45.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.310+0000 7f76ffe00640 1 -- 172.21.15.39:0/490922435 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f76f80e3380 con 0x7f7708103be0 2024-07-04T08:31:45.297 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.310+0000 7f7706a00640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76ec0b29e0 0x7f76ec0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:45.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.311+0000 7f7706a00640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76ec0b29e0 0x7f76ec0b4ef0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009830 tx=0x7f76f40058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:45.298 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.311+0000 7f76ffe00640 1 -- 172.21.15.39:0/490922435 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f76f80aa9a0 con 0x7f7708103be0 2024-07-04T08:31:45.312 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.326+0000 7f1ed6400640 1 -- 172.21.15.39:0/1058071197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed010f580 msgr2=0x7f1ed01119e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.326+0000 7f1ed6400640 1 --2- 172.21.15.39:0/1058071197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed010f580 0x7f1ed01119e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009be0 tx=0x7f1ec802a6c0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.326+0000 7f1ed6400640 1 -- 172.21.15.39:0/1058071197 shutdown_connections 2024-07-04T08:31:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.326+0000 7f1ed6400640 1 --2- 172.21.15.39:0/1058071197 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed010f580 0x7f1ed01119e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.326+0000 7f1ed6400640 1 --2- 172.21.15.39:0/1058071197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed0075650 0x7f1ed010f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.326+0000 7f1ed6400640 1 --2- 172.21.15.39:0/1058071197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed0074d00 0x7f1ed0075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.326+0000 7f1ed6400640 1 -- 172.21.15.39:0/1058071197 >> 172.21.15.39:0/1058071197 conn(0x7f1ed00fea70 msgr2=0x7f1ed0100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.326+0000 7f1ed6400640 1 -- 172.21.15.39:0/1058071197 shutdown_connections 2024-07-04T08:31:45.313 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.326+0000 7f1ed6400640 1 -- 172.21.15.39:0/1058071197 wait complete. 2024-07-04T08:31:45.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.327+0000 7f1ed6400640 1 Processor -- start 2024-07-04T08:31:45.314 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.327+0000 7f1ed6400640 1 -- start start 2024-07-04T08:31:45.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ed6400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed0074d00 0x7f1ed019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:45.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ed6400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed0075650 0x7f1ed019c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:45.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ed6400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed010f580 0x7f1ed01a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:45.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ed6400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed0115160 con 0x7f1ed0074d00 2024-07-04T08:31:45.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ed6400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed0114fe0 con 0x7f1ed0075650 2024-07-04T08:31:45.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ed6400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed01152e0 con 0x7f1ed010f580 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ecea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed0075650 0x7f1ed019c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ecf400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed0074d00 0x7f1ed019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ecea00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed0075650 0x7f1ed019c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45936/0 (socket says 172.21.15.39:45936) 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ecea00640 1 -- 172.21.15.39:0/2059145093 learned_addr learned my addr 172.21.15.39:0/2059145093 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.328+0000 7f1ecfe00640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed010f580 0x7f1ed01a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ecea00640 1 -- 172.21.15.39:0/2059145093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed010f580 msgr2=0x7f1ed01a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ecea00640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed010f580 0x7f1ed01a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ecea00640 1 -- 172.21.15.39:0/2059145093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed0074d00 msgr2=0x7f1ed019bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ecea00640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed0074d00 0x7f1ed019bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ecea00640 1 -- 172.21.15.39:0/2059145093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed01a1b10 con 0x7f1ed0075650 2024-07-04T08:31:45.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ecea00640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed0075650 0x7f1ed019c180 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e970 tx=0x7f1ebc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:45.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ec7e00640 1 -- 172.21.15.39:0/2059145093 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00cd00 con 0x7f1ed0075650 2024-07-04T08:31:45.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ec7e00640 1 -- 172.21.15.39:0/2059145093 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc0035f0 con 0x7f1ed0075650 2024-07-04T08:31:45.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed01a1e00 con 0x7f1ed0075650 2024-07-04T08:31:45.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.329+0000 7f1ec7e00640 1 -- 172.21.15.39:0/2059145093 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc010690 con 0x7f1ed0075650 2024-07-04T08:31:45.317 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.330+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed01a8590 con 0x7f1ed0075650 2024-07-04T08:31:45.318 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.331+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed0075110 con 0x7f1ed0075650 2024-07-04T08:31:45.319 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.333+0000 7f1ec7e00640 1 -- 172.21.15.39:0/2059145093 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f1ebc004020 con 0x7f1ed0075650 2024-07-04T08:31:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.335+0000 7f1ec7e00640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1eb40b2a10 0x7f1eb40b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.335+0000 7f1ec7e00640 1 -- 172.21.15.39:0/2059145093 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f1ebc0e2b40 con 0x7f1ed0075650 2024-07-04T08:31:45.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.335+0000 7f1ecf400640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1eb40b2a10 0x7f1eb40b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:45.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.336+0000 7f1ecf400640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1eb40b2a10 0x7f1eb40b4f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009530 tx=0x7f1ec0009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:45.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.338+0000 7f1ec7e00640 1 -- 172.21.15.39:0/2059145093 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f1ebc0aa160 con 0x7f1ed0075650 2024-07-04T08:31:45.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.565+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7708102df0 con 0x7f7708103be0 2024-07-04T08:31:45.553 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.566+0000 7f76ffe00640 1 -- 172.21.15.39:0/490922435 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f76f80aa2a0 con 0x7f7708103be0 2024-07-04T08:31:45.553 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:31:45.553 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:31:45.554 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76ec0b29e0 msgr2=0x7f76ec0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76ec0b29e0 0x7f76ec0b4ef0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009830 tx=0x7f76f40058d0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708103be0 msgr2=0x7f7708101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708103be0 0x7f7708101af0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f76f800e9c0 tx=0x7f76f800ee80 comp rx=0 tx=0).stop 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 shutdown_connections 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f76ec0b29e0 0x7f76ec0b4ef0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7708104610 0x7f7708102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f770810d190 0x7f7708102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 --2- 172.21.15.39:0/490922435 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7708103be0 0x7f7708101af0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 >> 172.21.15.39:0/490922435 conn(0x7f77080fd810 msgr2=0x7f7708106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:45.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 shutdown_connections 2024-07-04T08:31:45.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.569+0000 7f770d200640 1 -- 172.21.15.39:0/490922435 wait complete. 2024-07-04T08:31:45.572 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.586+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 --> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1ed0109830 con 0x7f1eb40b2a10 2024-07-04T08:31:45.582 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:45 smithi039 ceph-mon[32062]: pgmap v996: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-07-04T08:31:45.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.597+0000 7f1ec7e00640 1 -- 172.21.15.39:0/2059145093 <== mgr.14214 v2:172.21.15.39:6800/3120140278 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1ed0109830 con 0x7f1eb40b2a10 2024-07-04T08:31:45.584 INFO:teuthology.orchestra.run.smithi039.stdout:/ 1m 6m3h 2024-07-04T08:31:45.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1eb40b2a10 msgr2=0x7f1eb40b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1eb40b2a10 0x7f1eb40b4f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009530 tx=0x7f1ec0009260 comp rx=0 tx=0).stop 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed0075650 msgr2=0x7f1ed019c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed0075650 0x7f1ed019c180 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e970 tx=0x7f1ebc00ee30 comp rx=0 tx=0).stop 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 shutdown_connections 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f1eb40b2a10 0x7f1eb40b4f20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed010f580 0x7f1ed01a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ed0075650 0x7f1ed019c180 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 --2- 172.21.15.39:0/2059145093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ed0074d00 0x7f1ed019bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 >> 172.21.15.39:0/2059145093 conn(0x7f1ed00fea70 msgr2=0x7f1ed0109120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 shutdown_connections 2024-07-04T08:31:45.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:45.600+0000 7f1ed6400640 1 -- 172.21.15.39:0/2059145093 wait complete. 2024-07-04T08:31:45.621 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-07-04T08:31:45.631 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-07-04T08:31:45.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:45 smithi050 ceph-mon[36577]: pgmap v996: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-07-04T08:31:45.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:45 smithi133 ceph-mon[37050]: pgmap v996: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-07-04T08:31:46.496 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-04T08:31:46.574 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:46 smithi039 ceph-mon[32062]: from='client.19029 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:31:46.575 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:46 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/490922435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:46.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:46 smithi050 ceph-mon[36577]: from='client.19029 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:31:46.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:46 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/490922435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:46.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:46 smithi133 ceph-mon[37050]: from='client.19029 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:31:46.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:46 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/490922435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:31:46.971 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:47.380 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:47 smithi039 ceph-mon[32062]: from='client.29162 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:31:47.380 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:47 smithi039 ceph-mon[32062]: pgmap v997: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-07-04T08:31:47.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:47 smithi050 ceph-mon[36577]: from='client.29162 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:31:47.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:47 smithi050 ceph-mon[36577]: pgmap v997: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-07-04T08:31:47.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:47 smithi133 ceph-mon[37050]: from='client.29162 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-04T08:31:47.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:47 smithi133 ceph-mon[37050]: pgmap v997: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-07-04T08:31:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.361+0000 7f686f200640 1 -- 172.21.15.39:0/3342751997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68701050f0 msgr2=0x7f6870105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:49.350 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.361+0000 7f686f200640 1 --2- 172.21.15.39:0/3342751997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68701050f0 0x7f6870105500 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6868009900 tx=0x7f686802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.361+0000 7f686f200640 1 -- 172.21.15.39:0/3342751997 shutdown_connections 2024-07-04T08:31:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.361+0000 7f686f200640 1 --2- 172.21.15.39:0/3342751997 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687010b380 0x7f6870111bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.361+0000 7f686f200640 1 --2- 172.21.15.39:0/3342751997 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687010aa30 0x7f687010ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.361+0000 7f686f200640 1 --2- 172.21.15.39:0/3342751997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68701050f0 0x7f6870105500 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.351 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.361+0000 7f686f200640 1 -- 172.21.15.39:0/3342751997 >> 172.21.15.39:0/3342751997 conn(0x7f6870100b60 msgr2=0x7f6870102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:49.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.362+0000 7f686f200640 1 -- 172.21.15.39:0/3342751997 shutdown_connections 2024-07-04T08:31:49.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.362+0000 7f686f200640 1 -- 172.21.15.39:0/3342751997 wait complete. 2024-07-04T08:31:49.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.363+0000 7f686f200640 1 Processor -- start 2024-07-04T08:31:49.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.363+0000 7f686f200640 1 -- start start 2024-07-04T08:31:49.352 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.363+0000 7f686f200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68701050f0 0x7f687019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.363+0000 7f686f200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687010aa30 0x7f687019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.363+0000 7f686f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f687010b380 0x7f68701a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.363+0000 7f686f200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6870115080 con 0x7f68701050f0 2024-07-04T08:31:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.363+0000 7f686f200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6870114f00 con 0x7f687010aa30 2024-07-04T08:31:49.353 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.363+0000 7f686f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6870115200 con 0x7f687010b380 2024-07-04T08:31:49.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68701050f0 0x7f687019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.356 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68701050f0 0x7f687019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:56534/0 (socket says 172.21.15.39:56534) 2024-07-04T08:31:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686de00640 1 -- 172.21.15.39:0/429636534 learned_addr learned my addr 172.21.15.39:0/429636534 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686d400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687010aa30 0x7f687019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686e800640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f687010b380 0x7f68701a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686d400640 1 -- 172.21.15.39:0/429636534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f687010b380 msgr2=0x7f68701a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:49.357 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686d400640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f687010b380 0x7f68701a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686d400640 1 -- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68701050f0 msgr2=0x7f687019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686d400640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68701050f0 0x7f687019ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686d400640 1 -- 172.21.15.39:0/429636534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68701a19d0 con 0x7f687010aa30 2024-07-04T08:31:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.364+0000 7f686de00640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68701050f0 0x7f687019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:49.358 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.365+0000 7f686d400640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687010aa30 0x7f687019c000 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f687006bd90 tx=0x7f685800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.365+0000 7f6866a00640 1 -- 172.21.15.39:0/429636534 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685800cd50 con 0x7f687010aa30 2024-07-04T08:31:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.365+0000 7f686f200640 1 -- 172.21.15.39:0/429636534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68701a1cc0 con 0x7f687010aa30 2024-07-04T08:31:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.365+0000 7f6866a00640 1 -- 172.21.15.39:0/429636534 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6858011920 con 0x7f687010aa30 2024-07-04T08:31:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.365+0000 7f6866a00640 1 -- 172.21.15.39:0/429636534 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685801a9a0 con 0x7f687010aa30 2024-07-04T08:31:49.359 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.365+0000 7f686f200640 1 -- 172.21.15.39:0/429636534 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68701a8420 con 0x7f687010aa30 2024-07-04T08:31:49.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.369+0000 7f687b129740 1 Processor -- start 2024-07-04T08:31:49.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.369+0000 7f6866a00640 1 -- 172.21.15.39:0/429636534 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6858011470 con 0x7f687010aa30 2024-07-04T08:31:49.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.369+0000 7f687b129740 1 -- start start 2024-07-04T08:31:49.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f687b129740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a5814950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.360 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f687b129740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a54471b0 0x5639a5814e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f687b129740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a544f390 0x5639a58153d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f687b129740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5639a578aee0 con 0x5639a54471b0 2024-07-04T08:31:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f687b129740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5639a577a480 con 0x5639a544f390 2024-07-04T08:31:49.361 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f687b129740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5639a577e4c0 con 0x5639a5470070 2024-07-04T08:31:49.369 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f686e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a5814950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f686de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a54471b0 0x5639a5814e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f686e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a5814950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:47598/0 (socket says 172.21.15.39:47598) 2024-07-04T08:31:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f686e800640 1 -- 172.21.15.39:0/557528175 learned_addr learned my addr 172.21.15.39:0/557528175 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:49.370 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.370+0000 7f686d400640 1 --2- 172.21.15.39:0/557528175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a544f390 0x5639a58153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f686de00640 1 -- 172.21.15.39:0/557528175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 msgr2=0x5639a5814950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:49.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f686de00640 1 --2- 172.21.15.39:0/557528175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a5814950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f686de00640 1 -- 172.21.15.39:0/557528175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a544f390 msgr2=0x5639a58153d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:49.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f686de00640 1 --2- 172.21.15.39:0/557528175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a544f390 0x5639a58153d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.371 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f686de00640 1 -- 172.21.15.39:0/557528175 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a5712650 con 0x5639a54471b0 2024-07-04T08:31:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f686e800640 1 --2- 172.21.15.39:0/557528175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a5814950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f6866a00640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68540b2960 0x7f68540b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f686de00640 1 --2- 172.21.15.39:0/557528175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a54471b0 0x5639a5814e90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6868000c00 tx=0x7f6868004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d6d169cb432029 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f6866a00640 1 -- 172.21.15.39:0/429636534 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f68580e2d60 con 0x7f687010aa30 2024-07-04T08:31:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.371+0000 7f686e800640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68540b2960 0x7f68540b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.372+0000 7f685f400640 1 -- 172.21.15.39:0/557528175 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868040070 con 0x5639a54471b0 2024-07-04T08:31:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.372+0000 7f685f400640 1 -- 172.21.15.39:0/557528175 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68680047a0 con 0x5639a54471b0 2024-07-04T08:31:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.372+0000 7f685f400640 1 -- 172.21.15.39:0/557528175 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868035040 con 0x5639a54471b0 2024-07-04T08:31:49.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f687b129740 1 -- 172.21.15.39:0/557528175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a54471b0 msgr2=0x5639a5814e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f686e800640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68540b2960 0x7f68540b4e70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6860012560 tx=0x7f6860005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f687b129740 1 --2- 172.21.15.39:0/557528175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a54471b0 0x5639a5814e90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6868000c00 tx=0x7f6868004560 comp rx=0 tx=0).stop 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f687b129740 1 -- 172.21.15.39:0/557528175 shutdown_connections 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f687b129740 1 --2- 172.21.15.39:0/557528175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a54471b0 0x5639a5814e90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f687b129740 1 --2- 172.21.15.39:0/557528175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a544f390 0x5639a58153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f687b129740 1 --2- 172.21.15.39:0/557528175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a5814950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f687b129740 1 -- 172.21.15.39:0/557528175 >> 172.21.15.39:0/557528175 conn(0x5639a5464460 msgr2=0x5639a5812840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f687b129740 1 -- 172.21.15.39:0/557528175 shutdown_connections 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.373+0000 7f687b129740 1 -- 172.21.15.39:0/557528175 wait complete. 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.374+0000 7f687b129740 1 -- start start 2024-07-04T08:31:49.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.374+0000 7f687b129740 1 Processor -- start 2024-07-04T08:31:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.374+0000 7f687b129740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a54471b0 0x5639a5820520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f687b129740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a5820a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f687b129740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a582c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f687b129740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x5639a55de700 con 0x5639a544f390 2024-07-04T08:31:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f687b129740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5639a5721bb0 con 0x5639a54471b0 2024-07-04T08:31:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f687b129740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5639a58230d0 con 0x5639a5470070 2024-07-04T08:31:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f687b129740 10 client.0 _ll_register_callbacks cb 0x5639a5773570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a54471b0 0x5639a5820520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.374 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a5820a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a54471b0 0x5639a5820520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45984/0 (socket says 172.21.15.39:45984) 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686de00640 1 -- 172.21.15.39:0/3180025327 learned_addr learned my addr 172.21.15.39:0/3180025327 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686e800640 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a582c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686de00640 1 -- 172.21.15.39:0/3180025327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 msgr2=0x5639a582c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686de00640 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a582c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686de00640 1 -- 172.21.15.39:0/3180025327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 msgr2=0x5639a5820a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686de00640 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a5820a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.375+0000 7f686de00640 1 -- 172.21.15.39:0/3180025327 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a54db650 con 0x5639a54471b0 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.376+0000 7f686de00640 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a54471b0 0x5639a5820520 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6868009a20 tx=0x7f6868003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6064e66a7f14c568 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.376+0000 7f6853e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.376+0000 7f6853e00640 1 -- 172.21.15.39:0/3180025327 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868035070 con 0x5639a54471b0 2024-07-04T08:31:49.375 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.376+0000 7f6853e00640 1 -- 172.21.15.39:0/3180025327 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f68680316a0 con 0x5639a54471b0 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.376+0000 7f687b129740 1 -- 172.21.15.39:0/3180025327 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68680047a0 con 0x5639a54471b0 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f6853e00640 1 -- 172.21.15.39:0/3180025327 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6868040040 con 0x5639a54471b0 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f687b129740 10 client.29174 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f687b129740 1 -- 172.21.15.39:0/3180025327 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5639a582f120 con 0x5639a54471b0 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f6853e00640 1 -- 172.21.15.39:0/3180025327 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68680047a0 con 0x5639a54471b0 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f6853e00640 1 -- 172.21.15.39:0/3180025327 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f686803c800 con 0x5639a54471b0 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f687b129740 10 client.29174 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f687b129740 10 client.29174 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f687b129740 20 client.29174 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f687b129740 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5639a542e9f0 0x5639a5832480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f687b129740 4 client.29174 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:49.376 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.377+0000 7f687b129740 1 -- 172.21.15.39:0/3180025327 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639a578ce30 con 0x5639a542e9f0 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.378+0000 7f686d400640 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5639a542e9f0 0x5639a5832480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.378+0000 7f686d400640 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5639a542e9f0 0x5639a5832480 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7975ff3bfdb9402d server_cookie=5325afaed1ccb179 in_seq=0 out_seq=0 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.378+0000 7f6853e00640 10 client.29174 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.379+0000 7f6853e00640 1 -- 172.21.15.39:0/3180025327 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5639a578ce30 con 0x5639a542e9f0 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.379+0000 7f6853e00640 10 client.29174 handle_command_reply: tid=0 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.379+0000 7f687b129740 1 client.29174 shutdown 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 -- 172.21.15.39:0/3180025327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a54471b0 msgr2=0x5639a5820520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a54471b0 0x5639a5820520 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6868009a20 tx=0x7f6868003a50 comp rx=0 tx=0).stop 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 -- 172.21.15.39:0/3180025327 shutdown_connections 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5639a542e9f0 0x5639a5832480 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a5820a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.377 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a5470070 0x5639a582c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 --2- 172.21.15.39:0/3180025327 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a54471b0 0x5639a5820520 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 -- 172.21.15.39:0/3180025327 >> 172.21.15.39:0/3180025327 conn(0x5639a5464460 msgr2=0x5639a5817ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f6853e00640 0 client.29174 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:31:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 -- 172.21.15.39:0/3180025327 shutdown_connections 2024-07-04T08:31:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 1 -- 172.21.15.39:0/3180025327 wait complete. 2024-07-04T08:31:49.378 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.380+0000 7f687b129740 20 client.29174 trim_cache size 0 max 16384 2024-07-04T08:31:49.382 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.395+0000 7f687b129740 1 Processor -- start 2024-07-04T08:31:49.382 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:49 smithi039 ceph-mon[32062]: pgmap v998: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-07-04T08:31:49.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.396+0000 7f687b129740 1 -- start start 2024-07-04T08:31:49.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.396+0000 7f687b129740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a581a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.396+0000 7f687b129740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a582c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.396+0000 7f687b129740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a58354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.383 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.396+0000 7f687b129740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b22f0 con 0x5639a544f390 2024-07-04T08:31:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.396+0000 7f687b129740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b3e30 con 0x5639a5470070 2024-07-04T08:31:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.396+0000 7f687b129740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b29c0 con 0x5639a54471b0 2024-07-04T08:31:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.396+0000 7f686de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a581a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a581a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:47620/0 (socket says 172.21.15.39:47620) 2024-07-04T08:31:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686de00640 1 -- 172.21.15.39:0/3664714651 learned_addr learned my addr 172.21.15.39:0/3664714651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686e800640 1 --2- 172.21.15.39:0/3664714651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a58354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.384 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686d400640 1 --2- 172.21.15.39:0/3664714651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a582c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686de00640 1 -- 172.21.15.39:0/3664714651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 msgr2=0x5639a58354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686de00640 1 --2- 172.21.15.39:0/3664714651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a58354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686de00640 1 -- 172.21.15.39:0/3664714651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 msgr2=0x5639a582c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686de00640 1 --2- 172.21.15.39:0/3664714651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a582c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686de00640 1 -- 172.21.15.39:0/3664714651 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a58215e0 con 0x5639a54471b0 2024-07-04T08:31:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.397+0000 7f686de00640 1 --2- 172.21.15.39:0/3664714651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a581a170 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f686803cc50 tx=0x7f6868005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b6130a34614c63 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:49.385 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.398+0000 7f685ea00640 1 -- 172.21.15.39:0/3664714651 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868060070 con 0x5639a54471b0 2024-07-04T08:31:49.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.398+0000 7f685ea00640 1 -- 172.21.15.39:0/3664714651 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6868005850 con 0x5639a54471b0 2024-07-04T08:31:49.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.398+0000 7f685ea00640 1 -- 172.21.15.39:0/3664714651 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868035040 con 0x5639a54471b0 2024-07-04T08:31:49.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.399+0000 7f687b129740 1 -- 172.21.15.39:0/3664714651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 msgr2=0x5639a581a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:49.386 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.399+0000 7f687b129740 1 --2- 172.21.15.39:0/3664714651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a581a170 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f686803cc50 tx=0x7f6868005a20 comp rx=0 tx=0).stop 2024-07-04T08:31:49.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.399+0000 7f687b129740 1 -- 172.21.15.39:0/3664714651 shutdown_connections 2024-07-04T08:31:49.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.399+0000 7f687b129740 1 --2- 172.21.15.39:0/3664714651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a582c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.399+0000 7f687b129740 1 --2- 172.21.15.39:0/3664714651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a58354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.399+0000 7f687b129740 1 --2- 172.21.15.39:0/3664714651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a581a170 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.399+0000 7f687b129740 1 -- 172.21.15.39:0/3664714651 >> 172.21.15.39:0/3664714651 conn(0x5639a5464460 msgr2=0x5639a5819a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:49.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.399+0000 7f687b129740 1 -- 172.21.15.39:0/3664714651 shutdown_connections 2024-07-04T08:31:49.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.399+0000 7f687b129740 1 -- 172.21.15.39:0/3664714651 wait complete. 2024-07-04T08:31:49.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.400+0000 7f687b129740 1 -- start start 2024-07-04T08:31:49.387 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.400+0000 7f687b129740 1 Processor -- start 2024-07-04T08:31:49.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.400+0000 7f687b129740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a58334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.388 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.400+0000 7f687b129740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a58339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.400+0000 7f687b129740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a5833f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f687b129740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b3760 con 0x5639a544f390 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f687b129740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b3090 con 0x5639a5470070 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f687b129740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b02d0 con 0x5639a54471b0 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f687b129740 10 client.0 _ll_register_callbacks cb 0x7f68580e66e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a58334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a5833f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a58334b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:47632/0 (socket says 172.21.15.39:47632) 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686e800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a5833f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45998/0 (socket says 172.21.15.39:45998) 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686de00640 1 -- 172.21.15.39:0/1286970760 learned_addr learned my addr 172.21.15.39:0/1286970760 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686d400640 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a58339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.389 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686e800640 1 -- 172.21.15.39:0/1286970760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 msgr2=0x5639a58334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686e800640 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a58334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686e800640 1 -- 172.21.15.39:0/1286970760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 msgr2=0x5639a58339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686e800640 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a58339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.401+0000 7f686e800640 1 -- 172.21.15.39:0/1286970760 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a5899b00 con 0x5639a5470070 2024-07-04T08:31:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.402+0000 7f686e800640 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a5833f30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6860009860 tx=0x7f6860004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9291cb45792bb47a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.402+0000 7f6853e00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.402+0000 7f6853e00640 1 -- 172.21.15.39:0/1286970760 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860004020 con 0x5639a5470070 2024-07-04T08:31:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.402+0000 7f687b129740 1 -- 172.21.15.39:0/1286970760 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6868005850 con 0x5639a5470070 2024-07-04T08:31:49.390 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.402+0000 7f6853e00640 1 -- 172.21.15.39:0/1286970760 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f68600041c0 con 0x5639a5470070 2024-07-04T08:31:49.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f6853e00640 1 -- 172.21.15.39:0/1286970760 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6860037050 con 0x5639a5470070 2024-07-04T08:31:49.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f687b129740 10 client.29180 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:49.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f687b129740 1 -- 172.21.15.39:0/1286970760 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5639a589adf0 con 0x5639a5470070 2024-07-04T08:31:49.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f6853e00640 1 -- 172.21.15.39:0/1286970760 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6868005850 con 0x5639a5470070 2024-07-04T08:31:49.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f6853e00640 1 -- 172.21.15.39:0/1286970760 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6860003760 con 0x5639a5470070 2024-07-04T08:31:49.392 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f687b129740 10 client.29180 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f687b129740 10 client.29180 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f687b129740 20 client.29180 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f687b129740 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5639a5403da0 0x5639a589d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f687b129740 4 client.29180 mds_command: new command op to 24673 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.403+0000 7f687b129740 1 -- 172.21.15.39:0/1286970760 --> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639a57888d0 con 0x5639a5403da0 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.404+0000 7f686de00640 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5639a5403da0 0x5639a589d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.404+0000 7f686de00640 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5639a5403da0 0x5639a589d630 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=68d5af0f265afb1d server_cookie=3576cf6b407e7d5 in_seq=0 out_seq=0 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.404+0000 7f6853e00640 10 client.29180 ms_handle_connect on v2:172.21.15.133:6836/431595690 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f6853e00640 1 -- 172.21.15.39:0/1286970760 <== mds.4 v2:172.21.15.133:6836/431595690 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639a57888d0 con 0x5639a5403da0 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f6853e00640 10 client.29180 handle_command_reply: tid=0 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 client.29180 shutdown 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 -- 172.21.15.39:0/1286970760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 msgr2=0x5639a5833f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:49.393 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a5833f30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6860009860 tx=0x7f6860004730 comp rx=0 tx=0).stop 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 -- 172.21.15.39:0/1286970760 shutdown_connections 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.133:6836/431595690,v1:172.21.15.133:6837/431595690] conn(0x5639a5403da0 0x5639a589d630 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5639a544f390 0x5639a58339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5639a5470070 0x5639a5833f30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 --2- 172.21.15.39:0/1286970760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5639a54471b0 0x5639a58334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 -- 172.21.15.39:0/1286970760 >> 172.21.15.39:0/1286970760 conn(0x5639a5464460 msgr2=0x5639a5815980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f6853e00640 0 client.29180 ms_handle_reset on v2:172.21.15.133:6836/431595690 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 -- 172.21.15.39:0/1286970760 shutdown_connections 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 1 -- 172.21.15.39:0/1286970760 wait complete. 2024-07-04T08:31:49.394 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.405+0000 7f687b129740 20 client.29180 trim_cache size 0 max 16384 2024-07-04T08:31:49.395 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:31:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 -- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68540b2960 msgr2=0x7f68540b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68540b2960 0x7f68540b4e70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6860012560 tx=0x7f6860005f90 comp rx=0 tx=0).stop 2024-07-04T08:31:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 -- 172.21.15.39:0/429636534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687010aa30 msgr2=0x7f687019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687010aa30 0x7f687019c000 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f687006bd90 tx=0x7f685800ee80 comp rx=0 tx=0).stop 2024-07-04T08:31:49.395 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 -- 172.21.15.39:0/429636534 shutdown_connections 2024-07-04T08:31:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f68540b2960 0x7f68540b4e70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f687010b380 0x7f68701a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f687010aa30 0x7f687019c000 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 --2- 172.21.15.39:0/429636534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68701050f0 0x7f687019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.406+0000 7f686f200640 1 -- 172.21.15.39:0/429636534 >> 172.21.15.39:0/429636534 conn(0x7f6870100b60 msgr2=0x7f6870102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.407+0000 7f686f200640 1 -- 172.21.15.39:0/429636534 shutdown_connections 2024-07-04T08:31:49.396 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:49.407+0000 7f686f200640 1 -- 172.21.15.39:0/429636534 wait complete. 2024-07-04T08:31:49.604 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:49 smithi133 ceph-mon[37050]: pgmap v998: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-07-04T08:31:49.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:49 smithi050 ceph-mon[36577]: pgmap v998: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.6 KiB/s wr, 11 op/s 2024-07-04T08:31:50.361 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-04T08:31:50.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:50 smithi039 ceph-mon[32062]: pgmap v999: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 14 op/s 2024-07-04T08:31:50.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:50 smithi050 ceph-mon[36577]: pgmap v999: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 14 op/s 2024-07-04T08:31:50.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:50 smithi133 ceph-mon[37050]: pgmap v999: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 14 op/s 2024-07-04T08:31:50.857 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.218+0000 7fee57200640 1 -- 172.21.15.39:0/1773754063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 msgr2=0x7fee58075730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.218+0000 7fee57200640 1 --2- 172.21.15.39:0/1773754063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 0x7fee58075730 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fee40009860 tx=0x7fee4002a340 comp rx=0 tx=0).stop 2024-07-04T08:31:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.219+0000 7fee57200640 1 -- 172.21.15.39:0/1773754063 shutdown_connections 2024-07-04T08:31:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.219+0000 7fee57200640 1 --2- 172.21.15.39:0/1773754063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 0x7fee58075730 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.219+0000 7fee57200640 1 --2- 172.21.15.39:0/1773754063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee581137d0 0x7fee58115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.219+0000 7fee57200640 1 --2- 172.21.15.39:0/1773754063 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee580768d0 0x7fee58074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.219+0000 7fee57200640 1 -- 172.21.15.39:0/1773754063 >> 172.21.15.39:0/1773754063 conn(0x7fee580fe990 msgr2=0x7fee58100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:53.206 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.219+0000 7fee57200640 1 -- 172.21.15.39:0/1773754063 shutdown_connections 2024-07-04T08:31:53.207 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.219+0000 7fee57200640 1 -- 172.21.15.39:0/1773754063 wait complete. 2024-07-04T08:31:53.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.220+0000 7fee57200640 1 Processor -- start 2024-07-04T08:31:53.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.220+0000 7fee57200640 1 -- start start 2024-07-04T08:31:53.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.220+0000 7fee57200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 0x7fee5819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.220+0000 7fee57200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee580768d0 0x7fee581a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.220+0000 7fee57200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee581137d0 0x7fee581a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.208 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee57200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fee58119400 con 0x7fee58075320 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee57200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fee58119280 con 0x7fee581137d0 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee57200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee58119580 con 0x7fee580768d0 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 0x7fee5819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee580768d0 0x7fee581a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 0x7fee5819fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43368/0 (socket says 172.21.15.39:43368) 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55e00640 1 -- 172.21.15.39:0/4200919803 learned_addr learned my addr 172.21.15.39:0/4200919803 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee56800640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee581137d0 0x7fee581a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55400640 1 -- 172.21.15.39:0/4200919803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee581137d0 msgr2=0x7fee581a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55400640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee581137d0 0x7fee581a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55400640 1 -- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 msgr2=0x7fee5819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55400640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 0x7fee5819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55400640 1 -- 172.21.15.39:0/4200919803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee581a5da0 con 0x7fee580768d0 2024-07-04T08:31:53.209 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee55e00640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 0x7fee5819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:53.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.221+0000 7fee56800640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee581137d0 0x7fee581a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:31:53.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.222+0000 7fee55400640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee580768d0 0x7fee581a03d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fee40035040 tx=0x7fee40032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:53.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.222+0000 7fee4ea00640 1 -- 172.21.15.39:0/4200919803 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40040020 con 0x7fee580768d0 2024-07-04T08:31:53.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.222+0000 7fee57200640 1 -- 172.21.15.39:0/4200919803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee581a6030 con 0x7fee580768d0 2024-07-04T08:31:53.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.222+0000 7fee4ea00640 1 -- 172.21.15.39:0/4200919803 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee40030e10 con 0x7fee580768d0 2024-07-04T08:31:53.210 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.222+0000 7fee4ea00640 1 -- 172.21.15.39:0/4200919803 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40037040 con 0x7fee580768d0 2024-07-04T08:31:53.211 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.222+0000 7fee57200640 1 -- 172.21.15.39:0/4200919803 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee581ac7f0 con 0x7fee580768d0 2024-07-04T08:31:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.226+0000 7fee63220740 1 Processor -- start 2024-07-04T08:31:53.212 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.226+0000 7fee63220740 1 -- start start 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.226+0000 7fee63220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea176070 0x558dea51a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.226+0000 7fee63220740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea14d1b0 0x558dea51aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.226+0000 7fee63220740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea155390 0x558dea51b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.226+0000 7fee63220740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558dea490f00 con 0x558dea155390 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.226+0000 7fee63220740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558dea4804a0 con 0x558dea14d1b0 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.226+0000 7fee63220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558dea4844e0 con 0x558dea176070 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee56800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea176070 0x558dea51a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee55e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea14d1b0 0x558dea51aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee56800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea176070 0x558dea51a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36142/0 (socket says 172.21.15.39:36142) 2024-07-04T08:31:53.214 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee55e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea14d1b0 0x558dea51aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:55586/0 (socket says 172.21.15.39:55586) 2024-07-04T08:31:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee56800640 1 -- 172.21.15.39:0/338740120 learned_addr learned my addr 172.21.15.39:0/338740120 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee4ea00640 1 -- 172.21.15.39:0/4200919803 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fee40043020 con 0x7fee580768d0 2024-07-04T08:31:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee56800640 1 -- 172.21.15.39:0/338740120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea14d1b0 msgr2=0x558dea51aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee56800640 1 --2- 172.21.15.39:0/338740120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea14d1b0 0x558dea51aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee56800640 1 -- 172.21.15.39:0/338740120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea155390 msgr2=0x558dea51b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee56800640 1 --2- 172.21.15.39:0/338740120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea155390 0x558dea51b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee56800640 1 -- 172.21.15.39:0/338740120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dea418670 con 0x558dea176070 2024-07-04T08:31:53.215 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.227+0000 7fee55e00640 1 --2- 172.21.15.39:0/338740120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea14d1b0 0x558dea51aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:53.216 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.228+0000 7fee56800640 1 --2- 172.21.15.39:0/338740120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea176070 0x558dea51a970 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fee50010460 tx=0x7fee50010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7887cb005423b386 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.228+0000 7fee47400640 1 -- 172.21.15.39:0/338740120 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5001d070 con 0x558dea176070 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.228+0000 7fee47400640 1 -- 172.21.15.39:0/338740120 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee5000eb10 con 0x558dea176070 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.228+0000 7fee47400640 1 -- 172.21.15.39:0/338740120 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50018b80 con 0x558dea176070 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee63220740 1 -- 172.21.15.39:0/338740120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea176070 msgr2=0x558dea51a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee63220740 1 --2- 172.21.15.39:0/338740120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea176070 0x558dea51a970 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fee50010460 tx=0x7fee50010860 comp rx=0 tx=0).stop 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee4ea00640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee3c0b29e0 0x7fee3c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee63220740 1 -- 172.21.15.39:0/338740120 shutdown_connections 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee63220740 1 --2- 172.21.15.39:0/338740120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea155390 0x558dea51b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee63220740 1 --2- 172.21.15.39:0/338740120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea14d1b0 0x558dea51aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee63220740 1 --2- 172.21.15.39:0/338740120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea176070 0x558dea51a970 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee63220740 1 -- 172.21.15.39:0/338740120 >> 172.21.15.39:0/338740120 conn(0x558dea16a460 msgr2=0x558dea518860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:53.217 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee4ea00640 1 -- 172.21.15.39:0/4200919803 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fee401024d0 con 0x7fee580768d0 2024-07-04T08:31:53.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee63220740 1 -- 172.21.15.39:0/338740120 shutdown_connections 2024-07-04T08:31:53.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.229+0000 7fee55e00640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee3c0b29e0 0x7fee3c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.218 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.230+0000 7fee63220740 1 -- 172.21.15.39:0/338740120 wait complete. 2024-07-04T08:31:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.230+0000 7fee63220740 1 -- start start 2024-07-04T08:31:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.230+0000 7fee55e00640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee3c0b29e0 0x7fee3c0b4ef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x558dea5195f0 tx=0x7fee48005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.231+0000 7fee63220740 1 Processor -- start 2024-07-04T08:31:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.231+0000 7fee63220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea526540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.231+0000 7fee63220740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea526a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.231+0000 7fee63220740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea532470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.231+0000 7fee63220740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x558dea2e4700 con 0x558dea176070 2024-07-04T08:31:53.219 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.231+0000 7fee63220740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558dea427bd0 con 0x558dea155390 2024-07-04T08:31:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.231+0000 7fee63220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558dea529150 con 0x558dea14d1b0 2024-07-04T08:31:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.231+0000 7fee63220740 10 client.0 _ll_register_callbacks cb 0x558dea479590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee55e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea526540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee55e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea526540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36146/0 (socket says 172.21.15.39:36146) 2024-07-04T08:31:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee55e00640 1 -- 172.21.15.39:0/4250626668 learned_addr learned my addr 172.21.15.39:0/4250626668 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee55400640 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea526a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee56800640 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea532470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.220 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee55e00640 1 -- 172.21.15.39:0/4250626668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 msgr2=0x558dea526a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee55e00640 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea526a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee55e00640 1 -- 172.21.15.39:0/4250626668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 msgr2=0x558dea532470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee55e00640 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea532470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.232+0000 7fee55e00640 1 -- 172.21.15.39:0/4250626668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dea1e1650 con 0x558dea14d1b0 2024-07-04T08:31:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.233+0000 7fee55e00640 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea526540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fee48003760 tx=0x7fee48004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5eaaa95eb237d46 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:53.221 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.233+0000 7fee3be00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.233+0000 7fee3be00640 1 -- 172.21.15.39:0/4250626668 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4800f930 con 0x558dea14d1b0 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.233+0000 7fee3be00640 1 -- 172.21.15.39:0/4250626668 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fee4800fad0 con 0x558dea14d1b0 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.233+0000 7fee63220740 1 -- 172.21.15.39:0/4250626668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5000eb10 con 0x558dea14d1b0 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.233+0000 7fee3be00640 1 -- 172.21.15.39:0/4250626668 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee4800fcd0 con 0x558dea14d1b0 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.233+0000 7fee63220740 10 client.29116 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.233+0000 7fee63220740 1 -- 172.21.15.39:0/4250626668 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558dea535200 con 0x558dea14d1b0 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.234+0000 7fee3be00640 1 -- 172.21.15.39:0/4250626668 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee5000eb10 con 0x558dea14d1b0 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.234+0000 7fee3be00640 1 -- 172.21.15.39:0/4250626668 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fee48027070 con 0x558dea14d1b0 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.234+0000 7fee63220740 10 client.29116 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.234+0000 7fee63220740 10 client.29116 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.234+0000 7fee63220740 20 client.29116 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:53.223 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.234+0000 7fee63220740 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x558dea1349f0 0x558dea538760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.234+0000 7fee63220740 4 client.29116 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.234+0000 7fee63220740 1 -- 172.21.15.39:0/4250626668 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dea492e50 con 0x558dea1349f0 2024-07-04T08:31:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.235+0000 7fee55400640 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x558dea1349f0 0x558dea538760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.235+0000 7fee55400640 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x558dea1349f0 0x558dea538760 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fba08d4be4da9e5 server_cookie=74531532a3deb7a9 in_seq=0 out_seq=0 2024-07-04T08:31:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.235+0000 7fee3be00640 10 client.29116 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:31:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.236+0000 7fee3be00640 1 -- 172.21.15.39:0/4250626668 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558dea492e50 con 0x558dea1349f0 2024-07-04T08:31:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.236+0000 7fee3be00640 10 client.29116 handle_command_reply: tid=0 2024-07-04T08:31:53.224 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.236+0000 7fee63220740 1 client.29116 shutdown 2024-07-04T08:31:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 -- 172.21.15.39:0/4250626668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 msgr2=0x558dea526540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea526540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fee48003760 tx=0x7fee48004730 comp rx=0 tx=0).stop 2024-07-04T08:31:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 -- 172.21.15.39:0/4250626668 shutdown_connections 2024-07-04T08:31:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x558dea1349f0 0x558dea538760 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea532470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea526a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.225 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 --2- 172.21.15.39:0/4250626668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea526540 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 -- 172.21.15.39:0/4250626668 >> 172.21.15.39:0/4250626668 conn(0x558dea16a460 msgr2=0x558dea51db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee3be00640 0 client.29116 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:31:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 -- 172.21.15.39:0/4250626668 shutdown_connections 2024-07-04T08:31:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 1 -- 172.21.15.39:0/4250626668 wait complete. 2024-07-04T08:31:53.226 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.237+0000 7fee63220740 20 client.29116 trim_cache size 0 max 16384 2024-07-04T08:31:53.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.252+0000 7fee63220740 1 Processor -- start 2024-07-04T08:31:53.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.253+0000 7fee63220740 1 -- start start 2024-07-04T08:31:53.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.253+0000 7fee63220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea528220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.253+0000 7fee63220740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea51e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.253+0000 7fee63220740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea51e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.253+0000 7fee63220740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b6800 con 0x558dea176070 2024-07-04T08:31:53.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.253+0000 7fee63220740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b8340 con 0x558dea155390 2024-07-04T08:31:53.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.253+0000 7fee63220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b6ed0 con 0x558dea14d1b0 2024-07-04T08:31:53.240 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea528220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.253+0000 7fee56800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea51e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea528220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36156/0 (socket says 172.21.15.39:36156) 2024-07-04T08:31:53.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55e00640 1 -- 172.21.15.39:0/1585110449 learned_addr learned my addr 172.21.15.39:0/1585110449 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:53.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55400640 1 --2- 172.21.15.39:0/1585110449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea51e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.241 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55e00640 1 -- 172.21.15.39:0/1585110449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 msgr2=0x558dea51e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:53.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55e00640 1 --2- 172.21.15.39:0/1585110449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea51e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55e00640 1 -- 172.21.15.39:0/1585110449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 msgr2=0x558dea51e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:53.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55e00640 1 --2- 172.21.15.39:0/1585110449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea51e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55e00640 1 -- 172.21.15.39:0/1585110449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dea51d200 con 0x558dea14d1b0 2024-07-04T08:31:53.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.254+0000 7fee55e00640 1 --2- 172.21.15.39:0/1585110449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea528220 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fee48012f40 tx=0x7fee4801f4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a27e03127401cd11 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:53.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.255+0000 7fee46a00640 1 -- 172.21.15.39:0/1585110449 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4800fb30 con 0x558dea14d1b0 2024-07-04T08:31:53.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.255+0000 7fee46a00640 1 -- 172.21.15.39:0/1585110449 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee480054d0 con 0x558dea14d1b0 2024-07-04T08:31:53.242 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.255+0000 7fee46a00640 1 -- 172.21.15.39:0/1585110449 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee48029440 con 0x558dea14d1b0 2024-07-04T08:31:53.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 -- 172.21.15.39:0/1585110449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 msgr2=0x558dea528220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:53.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 --2- 172.21.15.39:0/1585110449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea528220 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fee48012f40 tx=0x7fee4801f4b0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 -- 172.21.15.39:0/1585110449 shutdown_connections 2024-07-04T08:31:53.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 --2- 172.21.15.39:0/1585110449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea51e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 --2- 172.21.15.39:0/1585110449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea51e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 --2- 172.21.15.39:0/1585110449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea528220 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.243 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 -- 172.21.15.39:0/1585110449 >> 172.21.15.39:0/1585110449 conn(0x558dea16a460 msgr2=0x558dea51da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 -- 172.21.15.39:0/1585110449 shutdown_connections 2024-07-04T08:31:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 -- 172.21.15.39:0/1585110449 wait complete. 2024-07-04T08:31:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.256+0000 7fee63220740 1 -- start start 2024-07-04T08:31:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.257+0000 7fee63220740 1 Processor -- start 2024-07-04T08:31:53.244 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee63220740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea52e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee63220740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea52e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee63220740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea529390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee63220740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b7c70 con 0x558dea176070 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee63220740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b75a0 con 0x558dea155390 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee63220740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee581b47e0 con 0x558dea14d1b0 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee63220740 10 client.0 _ll_register_callbacks cb 0x7fee480035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee55e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea52e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee55e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea52e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36170/0 (socket says 172.21.15.39:36170) 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee55e00640 1 -- 172.21.15.39:0/3109882380 learned_addr learned my addr 172.21.15.39:0/3109882380 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee55400640 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea52e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.245 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.258+0000 7fee56800640 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea529390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee55400640 1 -- 172.21.15.39:0/3109882380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 msgr2=0x558dea52e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:53.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee55400640 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea52e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee55400640 1 -- 172.21.15.39:0/3109882380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 msgr2=0x558dea529390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:53.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee55400640 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea529390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee55400640 1 -- 172.21.15.39:0/3109882380 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dea52f010 con 0x558dea155390 2024-07-04T08:31:53.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee55e00640 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea52e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:53.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee55400640 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea52e920 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fee400ff970 tx=0x7fee40005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c42976f1ebee7e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:53.247 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee3be00640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee3be00640 1 -- 172.21.15.39:0/3109882380 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400c9500 con 0x558dea155390 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee3be00640 1 -- 172.21.15.39:0/3109882380 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fee40102120 con 0x558dea155390 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.259+0000 7fee63220740 1 -- 172.21.15.39:0/3109882380 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee480054d0 con 0x558dea155390 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.260+0000 7fee3be00640 1 -- 172.21.15.39:0/3109882380 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fee400cd8b0 con 0x558dea155390 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.260+0000 7fee63220740 10 client.29195 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.260+0000 7fee63220740 1 -- 172.21.15.39:0/3109882380 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558dea5a14e0 con 0x558dea155390 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.260+0000 7fee3be00640 1 -- 172.21.15.39:0/3109882380 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee480054d0 con 0x558dea155390 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.260+0000 7fee3be00640 1 -- 172.21.15.39:0/3109882380 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fee400c9760 con 0x558dea155390 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.260+0000 7fee63220740 10 client.29195 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.260+0000 7fee63220740 10 client.29195 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.260+0000 7fee63220740 20 client.29195 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:53.248 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.260+0000 7fee63220740 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x558dea109da0 0x558dea5a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:53.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.261+0000 7fee63220740 4 client.29195 mds_command: new command op to 24674 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:31:53.249 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.261+0000 7fee63220740 1 -- 172.21.15.39:0/3109882380 --> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dea48e8f0 con 0x558dea109da0 2024-07-04T08:31:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.261+0000 7fee55e00640 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x558dea109da0 0x558dea5a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.261+0000 7fee55e00640 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x558dea109da0 0x558dea5a3c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9f2b697cb989e3f2 server_cookie=cd74d1ac2b1302f4 in_seq=0 out_seq=0 2024-07-04T08:31:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.261+0000 7fee3be00640 10 client.29195 ms_handle_connect on v2:172.21.15.50:6836/4268659351 2024-07-04T08:31:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee3be00640 1 -- 172.21.15.39:0/3109882380 <== mds.3 v2:172.21.15.50:6836/4268659351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dea48e8f0 con 0x558dea109da0 2024-07-04T08:31:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee3be00640 10 client.29195 handle_command_reply: tid=0 2024-07-04T08:31:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 client.29195 shutdown 2024-07-04T08:31:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 -- 172.21.15.39:0/3109882380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 msgr2=0x558dea52e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea52e920 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fee400ff970 tx=0x7fee40005c90 comp rx=0 tx=0).stop 2024-07-04T08:31:53.250 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 -- 172.21.15.39:0/3109882380 shutdown_connections 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.50:6836/4268659351,v1:172.21.15.50:6837/4268659351] conn(0x558dea109da0 0x558dea5a3c00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558dea176070 0x558dea529390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558dea155390 0x558dea52e920 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 --2- 172.21.15.39:0/3109882380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558dea14d1b0 0x558dea52e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 -- 172.21.15.39:0/3109882380 >> 172.21.15.39:0/3109882380 conn(0x558dea16a460 msgr2=0x558dea51e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee3be00640 0 client.29195 ms_handle_reset on v2:172.21.15.50:6836/4268659351 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 -- 172.21.15.39:0/3109882380 shutdown_connections 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.262+0000 7fee63220740 1 -- 172.21.15.39:0/3109882380 wait complete. 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.263+0000 7fee63220740 20 client.29195 trim_cache size 0 max 16384 2024-07-04T08:31:53.251 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:31:53.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.263+0000 7fee57200640 1 -- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee3c0b29e0 msgr2=0x7fee3c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:53.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.263+0000 7fee57200640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee3c0b29e0 0x7fee3c0b4ef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x558dea5195f0 tx=0x7fee48005f90 comp rx=0 tx=0).stop 2024-07-04T08:31:53.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.263+0000 7fee57200640 1 -- 172.21.15.39:0/4200919803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee580768d0 msgr2=0x7fee581a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:53.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.263+0000 7fee57200640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee580768d0 0x7fee581a03d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fee40035040 tx=0x7fee40032d50 comp rx=0 tx=0).stop 2024-07-04T08:31:53.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.264+0000 7fee57200640 1 -- 172.21.15.39:0/4200919803 shutdown_connections 2024-07-04T08:31:53.252 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.264+0000 7fee57200640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fee3c0b29e0 0x7fee3c0b4ef0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.264+0000 7fee57200640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fee581137d0 0x7fee581a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.264+0000 7fee57200640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee580768d0 0x7fee581a03d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.264+0000 7fee57200640 1 --2- 172.21.15.39:0/4200919803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee58075320 0x7fee5819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:53.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.264+0000 7fee57200640 1 -- 172.21.15.39:0/4200919803 >> 172.21.15.39:0/4200919803 conn(0x7fee580fe990 msgr2=0x7fee5810f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:53.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.264+0000 7fee57200640 1 -- 172.21.15.39:0/4200919803 shutdown_connections 2024-07-04T08:31:53.253 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:53.264+0000 7fee57200640 1 -- 172.21.15.39:0/4200919803 wait complete. 2024-07-04T08:31:53.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:53 smithi039 ceph-mon[32062]: pgmap v1000: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-07-04T08:31:53.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:53 smithi050 ceph-mon[36577]: pgmap v1000: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-07-04T08:31:53.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:53 smithi133 ceph-mon[37050]: pgmap v1000: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-07-04T08:31:54.168 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-04T08:31:54.659 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:55.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:55 smithi039 ceph-mon[32062]: pgmap v1001: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-07-04T08:31:55.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:55 smithi050 ceph-mon[36577]: pgmap v1001: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-07-04T08:31:55.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:55 smithi133 ceph-mon[37050]: pgmap v1001: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-07-04T08:31:56.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.917+0000 7f0e76400640 1 -- 172.21.15.39:0/183122116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7010f040 msgr2=0x7f0e7010f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:56.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.917+0000 7f0e76400640 1 --2- 172.21.15.39:0/183122116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7010f040 0x7f0e7010f450 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009900 tx=0x7f0e5802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.917+0000 7f0e76400640 1 -- 172.21.15.39:0/183122116 shutdown_connections 2024-07-04T08:31:56.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.917+0000 7f0e76400640 1 --2- 172.21.15.39:0/183122116 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7010f040 0x7f0e7010f450 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.917+0000 7f0e76400640 1 --2- 172.21.15.39:0/183122116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e7010f990 0x7f0e701161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.917+0000 7f0e76400640 1 --2- 172.21.15.39:0/183122116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e70109700 0x7f0e70109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.917+0000 7f0e76400640 1 -- 172.21.15.39:0/183122116 >> 172.21.15.39:0/183122116 conn(0x7f0e70077630 msgr2=0x7f0e70107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:56.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.917+0000 7f0e76400640 1 -- 172.21.15.39:0/183122116 shutdown_connections 2024-07-04T08:31:56.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.917+0000 7f0e76400640 1 -- 172.21.15.39:0/183122116 wait complete. 2024-07-04T08:31:56.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.918+0000 7f0e76400640 1 Processor -- start 2024-07-04T08:31:56.905 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e76400640 1 -- start start 2024-07-04T08:31:56.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e76400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70109700 0x7f0e701a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e76400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e7010f040 0x7f0e701a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e76400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e7010f990 0x7f0e701a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e76400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e70119620 con 0x7f0e70109700 2024-07-04T08:31:56.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e76400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701194a0 con 0x7f0e7010f990 2024-07-04T08:31:56.906 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e76400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701197a0 con 0x7f0e7010f040 2024-07-04T08:31:56.907 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e6f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70109700 0x7f0e701a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e6f400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70109700 0x7f0e701a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43436/0 (socket says 172.21.15.39:43436) 2024-07-04T08:31:56.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.919+0000 7f0e6f400640 1 -- 172.21.15.39:0/1418989228 learned_addr learned my addr 172.21.15.39:0/1418989228 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:56.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.920+0000 7f0e67e00640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e7010f040 0x7f0e701a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.920+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e7010f990 0x7f0e701a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.920+0000 7f0e6f400640 1 -- 172.21.15.39:0/1418989228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e7010f040 msgr2=0x7f0e701a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:56.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.920+0000 7f0e6f400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e7010f040 0x7f0e701a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.920+0000 7f0e6f400640 1 -- 172.21.15.39:0/1418989228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e7010f990 msgr2=0x7f0e701a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:56.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.920+0000 7f0e6f400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e7010f990 0x7f0e701a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.920+0000 7f0e6f400640 1 -- 172.21.15.39:0/1418989228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e701a6040 con 0x7f0e70109700 2024-07-04T08:31:56.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.920+0000 7f0e6f400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70109700 0x7f0e701a00d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000e9f0 tx=0x7f0e6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:56.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.921+0000 7f0e6cc00640 1 -- 172.21.15.39:0/1418989228 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000cd80 con 0x7f0e70109700 2024-07-04T08:31:56.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.921+0000 7f0e6cc00640 1 -- 172.21.15.39:0/1418989228 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e600054d0 con 0x7f0e70109700 2024-07-04T08:31:56.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.921+0000 7f0e6cc00640 1 -- 172.21.15.39:0/1418989228 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60010690 con 0x7f0e70109700 2024-07-04T08:31:56.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.921+0000 7f0e76400640 1 -- 172.21.15.39:0/1418989228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e701a6330 con 0x7f0e70109700 2024-07-04T08:31:56.909 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.921+0000 7f0e76400640 1 -- 172.21.15.39:0/1418989228 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e701aca90 con 0x7f0e70109700 2024-07-04T08:31:56.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.924+0000 7f0e7ca67740 1 Processor -- start 2024-07-04T08:31:56.911 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.925+0000 7f0e7ca67740 1 -- start start 2024-07-04T08:31:56.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.925+0000 7f0e6cc00640 1 -- 172.21.15.39:0/1418989228 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f0e60004e90 con 0x7f0e70109700 2024-07-04T08:31:56.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.925+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.925+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad49cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.925+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca984390 0x55c2cad4a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.912 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.925+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c2cacbfee0 con 0x55c2ca9a5070 2024-07-04T08:31:56.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.925+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c2cacaf480 con 0x55c2ca97c1b0 2024-07-04T08:31:56.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.925+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c2cacb34c0 con 0x55c2ca984390 2024-07-04T08:31:56.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad497b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43450/0 (socket says 172.21.15.39:43450) 2024-07-04T08:31:56.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6fe00640 1 -- 172.21.15.39:0/379908791 learned_addr learned my addr 172.21.15.39:0/379908791 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:56.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6f400640 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad49cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.913 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e67e00640 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca984390 0x55c2cad4a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6fe00640 1 -- 172.21.15.39:0/379908791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca984390 msgr2=0x55c2cad4a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:56.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca984390 0x55c2cad4a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6fe00640 1 -- 172.21.15.39:0/379908791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 msgr2=0x55c2cad49cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:56.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad49cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6fe00640 1 -- 172.21.15.39:0/379908791 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2cac47650 con 0x55c2ca9a5070 2024-07-04T08:31:56.916 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.926+0000 7f0e6f400640 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad49cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:31:56.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.927+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad497b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009f50 tx=0x7f0e680118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32369a98f4e5b208 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:56.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.927+0000 7f0e64c00640 1 -- 172.21.15.39:0/379908791 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004d20 con 0x55c2ca9a5070 2024-07-04T08:31:56.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.927+0000 7f0e64c00640 1 -- 172.21.15.39:0/379908791 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e68004050 con 0x55c2ca9a5070 2024-07-04T08:31:56.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.927+0000 7f0e64c00640 1 -- 172.21.15.39:0/379908791 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68005220 con 0x55c2ca9a5070 2024-07-04T08:31:56.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.927+0000 7f0e6cc00640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b29e0 0x7f0e540b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.928+0000 7f0e6cc00640 1 -- 172.21.15.39:0/1418989228 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f0e600e2d50 con 0x7f0e70109700 2024-07-04T08:31:56.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.928+0000 7f0e6f400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b29e0 0x7f0e540b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.917 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.928+0000 7f0e7ca67740 1 -- 172.21.15.39:0/379908791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 msgr2=0x55c2cad497b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:56.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.928+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad497b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009f50 tx=0x7f0e680118d0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.929+0000 7f0e7ca67740 1 -- 172.21.15.39:0/379908791 shutdown_connections 2024-07-04T08:31:56.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.929+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca984390 0x55c2cad4a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.929+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad49cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.929+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/379908791 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad497b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.929+0000 7f0e7ca67740 1 -- 172.21.15.39:0/379908791 >> 172.21.15.39:0/379908791 conn(0x55c2ca999460 msgr2=0x55c2cad476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:56.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.929+0000 7f0e6f400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b29e0 0x7f0e540b4ef0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55c2cad48430 tx=0x7f0e60004a90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:56.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.929+0000 7f0e7ca67740 1 -- 172.21.15.39:0/379908791 shutdown_connections 2024-07-04T08:31:56.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.929+0000 7f0e7ca67740 1 -- 172.21.15.39:0/379908791 wait complete. 2024-07-04T08:31:56.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.929+0000 7f0e7ca67740 1 -- start start 2024-07-04T08:31:56.919 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.930+0000 7f0e7ca67740 1 Processor -- start 2024-07-04T08:31:56.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.930+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca97c1b0 0x55c2cad55360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.930+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 0x55c2cad558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.930+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 0x55c2cad612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.930+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55c2cab13700 con 0x55c2ca97c1b0 2024-07-04T08:31:56.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.930+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c2cac56bb0 con 0x55c2ca984390 2024-07-04T08:31:56.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.930+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c2cad57f90 con 0x55c2ca9a5070 2024-07-04T08:31:56.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.930+0000 7f0e7ca67740 10 client.0 _ll_register_callbacks cb 0x55c2caca8570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:56.921 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca97c1b0 0x55c2cad55360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e6fe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca97c1b0 0x55c2cad55360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43454/0 (socket says 172.21.15.39:43454) 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e6fe00640 1 -- 172.21.15.39:0/2413112547 learned_addr learned my addr 172.21.15.39:0/2413112547 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e67e00640 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 0x55c2cad558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 0x55c2cad612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e67e00640 1 -- 172.21.15.39:0/2413112547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 msgr2=0x55c2cad612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e67e00640 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 0x55c2cad612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e67e00640 1 -- 172.21.15.39:0/2413112547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca97c1b0 msgr2=0x55c2cad55360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e67e00640 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca97c1b0 0x55c2cad55360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e67e00640 1 -- 172.21.15.39:0/2413112547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2caa10650 con 0x55c2ca984390 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.931+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca97c1b0 0x55c2cad55360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:56.922 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.932+0000 7f0e67e00640 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 0x55c2cad558a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0e5802a910 tx=0x7f0e58032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362a2288b7dad176 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.932+0000 7f0e5d600640 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.932+0000 7f0e5d600640 1 -- 172.21.15.39:0/2413112547 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58035040 con 0x55c2ca984390 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.932+0000 7f0e5d600640 1 -- 172.21.15.39:0/2413112547 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f0e58031760 con 0x55c2ca984390 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.932+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2413112547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e68004050 con 0x55c2ca984390 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.932+0000 7f0e5d600640 1 -- 172.21.15.39:0/2413112547 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e5804f050 con 0x55c2ca984390 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.932+0000 7f0e7ca67740 10 client.29207 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.932+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2413112547 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2cad63fb0 con 0x55c2ca984390 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.933+0000 7f0e5d600640 1 -- 172.21.15.39:0/2413112547 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e68004050 con 0x55c2ca984390 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.933+0000 7f0e5d600640 1 -- 172.21.15.39:0/2413112547 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0e58005480 con 0x55c2ca984390 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.933+0000 7f0e7ca67740 10 client.29207 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.933+0000 7f0e7ca67740 10 client.29207 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.933+0000 7f0e7ca67740 20 client.29207 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:56.923 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.933+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c2ca9639f0 0x55c2cad67510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.933+0000 7f0e7ca67740 4 client.29207 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:31:56.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.933+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2413112547 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2cacc1e30 con 0x55c2ca9639f0 2024-07-04T08:31:56.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.934+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c2ca9639f0 0x55c2cad67510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.934+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c2ca9639f0 0x55c2cad67510 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8f6d25ae500bdef server_cookie=e12333bf97abaced in_seq=0 out_seq=0 2024-07-04T08:31:56.924 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.934+0000 7f0e5d600640 10 client.29207 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:31:56.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.935+0000 7f0e5d600640 1 -- 172.21.15.39:0/2413112547 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c2cacc1e30 con 0x55c2ca9639f0 2024-07-04T08:31:56.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.935+0000 7f0e5d600640 10 client.29207 handle_command_reply: tid=0 2024-07-04T08:31:56.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.935+0000 7f0e7ca67740 1 client.29207 shutdown 2024-07-04T08:31:56.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.935+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2413112547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 msgr2=0x55c2cad558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:56.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.935+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 0x55c2cad558a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0e5802a910 tx=0x7f0e58032e90 comp rx=0 tx=0).stop 2024-07-04T08:31:56.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2413112547 shutdown_connections 2024-07-04T08:31:56.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c2ca9639f0 0x55c2cad67510 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 0x55c2cad612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.925 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 0x55c2cad558a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2413112547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca97c1b0 0x55c2cad55360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e5d600640 0 client.29207 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:31:56.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2413112547 >> 172.21.15.39:0/2413112547 conn(0x55c2ca999460 msgr2=0x55c2cad4c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:56.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2413112547 shutdown_connections 2024-07-04T08:31:56.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2413112547 wait complete. 2024-07-04T08:31:56.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.936+0000 7f0e7ca67740 20 client.29207 trim_cache size 0 max 16384 2024-07-04T08:31:56.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.951+0000 7f0e7ca67740 1 Processor -- start 2024-07-04T08:31:56.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.952+0000 7f0e7ca67740 1 -- start start 2024-07-04T08:31:56.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.952+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca97c1b0 0x55c2cad46940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.952+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 0x55c2cad610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.952+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad56d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.952+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701b6a00 con 0x55c2ca9a5070 2024-07-04T08:31:56.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.952+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701b8540 con 0x55c2ca984390 2024-07-04T08:31:56.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.952+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701b70d0 con 0x55c2ca97c1b0 2024-07-04T08:31:56.940 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e6fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca97c1b0 0x55c2cad46940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e6fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca97c1b0 0x55c2cad46940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36216/0 (socket says 172.21.15.39:36216) 2024-07-04T08:31:56.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e6fe00640 1 -- 172.21.15.39:0/2174883487 learned_addr learned my addr 172.21.15.39:0/2174883487 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:56.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e67e00640 1 --2- 172.21.15.39:0/2174883487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 0x55c2cad610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/2174883487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad56d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e6fe00640 1 -- 172.21.15.39:0/2174883487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 msgr2=0x55c2cad610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:56.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/2174883487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 0x55c2cad610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e6fe00640 1 -- 172.21.15.39:0/2174883487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 msgr2=0x55c2cad56d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:56.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/2174883487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad56d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.941 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.953+0000 7f0e6fe00640 1 -- 172.21.15.39:0/2174883487 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2cad4baa0 con 0x55c2ca97c1b0 2024-07-04T08:31:56.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.954+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/2174883487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca97c1b0 0x55c2cad46940 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0e680042c0 tx=0x7f0e680096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22dc533fd24a35e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:56.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.954+0000 7f0e5fe00640 1 -- 172.21.15.39:0/2174883487 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004370 con 0x55c2ca97c1b0 2024-07-04T08:31:56.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.954+0000 7f0e5fe00640 1 -- 172.21.15.39:0/2174883487 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e68004510 con 0x55c2ca97c1b0 2024-07-04T08:31:56.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.954+0000 7f0e5fe00640 1 -- 172.21.15.39:0/2174883487 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e680055a0 con 0x55c2ca97c1b0 2024-07-04T08:31:56.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.955+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2174883487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca97c1b0 msgr2=0x55c2cad46940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.955+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2174883487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca97c1b0 0x55c2cad46940 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0e680042c0 tx=0x7f0e680096c0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.955+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2174883487 shutdown_connections 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.955+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2174883487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca9a5070 0x55c2cad56d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.955+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2174883487 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca984390 0x55c2cad610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.955+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/2174883487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca97c1b0 0x55c2cad46940 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.955+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2174883487 >> 172.21.15.39:0/2174883487 conn(0x55c2ca999460 msgr2=0x55c2cad56680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.955+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2174883487 shutdown_connections 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.955+0000 7f0e7ca67740 1 -- 172.21.15.39:0/2174883487 wait complete. 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.956+0000 7f0e7ca67740 1 -- start start 2024-07-04T08:31:56.944 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.956+0000 7f0e7ca67740 1 Processor -- start 2024-07-04T08:31:56.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad6c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca984390 0x55c2cad6ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e7ca67740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 0x55c2cad5c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701b7e70 con 0x55c2ca984390 2024-07-04T08:31:56.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701b77a0 con 0x55c2ca97c1b0 2024-07-04T08:31:56.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e7ca67740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701b49e0 con 0x55c2ca9a5070 2024-07-04T08:31:56.945 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e7ca67740 10 client.0 _ll_register_callbacks cb 0x7f0e58009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:31:56.947 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e67e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca984390 0x55c2cad6ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e6fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad6c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e6fe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad6c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:55650/0 (socket says 172.21.15.39:55650) 2024-07-04T08:31:56.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e6fe00640 1 -- 172.21.15.39:0/1330653488 learned_addr learned my addr 172.21.15.39:0/1330653488 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:31:56.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.957+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 0x55c2cad5c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.958+0000 7f0e67e00640 1 -- 172.21.15.39:0/1330653488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 msgr2=0x55c2cad5c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:56.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.958+0000 7f0e67e00640 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 0x55c2cad5c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.948 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.958+0000 7f0e67e00640 1 -- 172.21.15.39:0/1330653488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 msgr2=0x55c2cad6c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.958+0000 7f0e67e00640 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad6c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.958+0000 7f0e67e00640 1 -- 172.21.15.39:0/1330653488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2cad5c8b0 con 0x55c2ca984390 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.958+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 0x55c2cad5c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.958+0000 7f0e67e00640 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca984390 0x55c2cad6ca40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0e580096c0 tx=0x7f0e58004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b327974b0bdfe1a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.958+0000 7f0e5d600640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.959+0000 7f0e5d600640 1 -- 172.21.15.39:0/1330653488 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58035070 con 0x55c2ca984390 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.959+0000 7f0e5d600640 1 -- 172.21.15.39:0/1330653488 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f0e58031d20 con 0x55c2ca984390 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.959+0000 7f0e7ca67740 1 -- 172.21.15.39:0/1330653488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e68004510 con 0x55c2ca984390 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.959+0000 7f0e5d600640 1 -- 172.21.15.39:0/1330653488 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e58031560 con 0x55c2ca984390 2024-07-04T08:31:56.949 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.959+0000 7f0e7ca67740 10 client.19092 fetch_fsmap learned FSMap version 30 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.959+0000 7f0e7ca67740 1 -- 172.21.15.39:0/1330653488 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2cad5db30 con 0x55c2ca984390 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.959+0000 7f0e5d600640 1 -- 172.21.15.39:0/1330653488 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e68004510 con 0x55c2ca984390 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.960+0000 7f0e5d600640 1 -- 172.21.15.39:0/1330653488 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0e580035f0 con 0x55c2ca984390 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.960+0000 7f0e7ca67740 10 client.19092 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.960+0000 7f0e7ca67740 10 client.19092 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.960+0000 7f0e7ca67740 20 client.19092 populate_metadata read hostname 'smithi039' 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.960+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c2ca938da0 0x55c2cadd1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.960+0000 7f0e7ca67740 4 client.19092 mds_command: new command op to 24679 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.960+0000 7f0e7ca67740 1 -- 172.21.15.39:0/1330653488 --> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2cacbd8d0 con 0x55c2ca938da0 2024-07-04T08:31:56.950 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.960+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c2ca938da0 0x55c2cadd1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:31:56.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.961+0000 7f0e6fe00640 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c2ca938da0 0x55c2cadd1e90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5bda64734e8988b4 server_cookie=898fb526cd3f6946 in_seq=0 out_seq=0 2024-07-04T08:31:56.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.961+0000 7f0e5d600640 10 client.19092 ms_handle_connect on v2:172.21.15.133:6838/1264801316 2024-07-04T08:31:56.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.961+0000 7f0e5d600640 1 -- 172.21.15.39:0/1330653488 <== mds.2 v2:172.21.15.133:6838/1264801316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2cacbd8d0 con 0x55c2ca938da0 2024-07-04T08:31:56.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.961+0000 7f0e5d600640 10 client.19092 handle_command_reply: tid=0 2024-07-04T08:31:56.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.961+0000 7f0e7ca67740 1 client.19092 shutdown 2024-07-04T08:31:56.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.961+0000 7f0e7ca67740 1 -- 172.21.15.39:0/1330653488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca984390 msgr2=0x55c2cad6ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:56.951 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.961+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca984390 0x55c2cad6ca40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0e580096c0 tx=0x7f0e58004bf0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.961+0000 7f0e7ca67740 1 -- 172.21.15.39:0/1330653488 shutdown_connections 2024-07-04T08:31:56.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.961+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.133:6838/1264801316,v1:172.21.15.133:6839/1264801316] conn(0x55c2ca938da0 0x55c2cadd1e90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.962+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c2ca984390 0x55c2cad6ca40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.962+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c2ca9a5070 0x55c2cad5c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.962+0000 7f0e7ca67740 1 --2- 172.21.15.39:0/1330653488 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c2ca97c1b0 0x55c2cad6c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.962+0000 7f0e7ca67740 1 -- 172.21.15.39:0/1330653488 >> 172.21.15.39:0/1330653488 conn(0x55c2ca999460 msgr2=0x55c2cad56610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:56.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.962+0000 7f0e5d600640 0 client.19092 ms_handle_reset on v2:172.21.15.133:6838/1264801316 2024-07-04T08:31:56.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.962+0000 7f0e7ca67740 1 -- 172.21.15.39:0/1330653488 shutdown_connections 2024-07-04T08:31:56.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.962+0000 7f0e7ca67740 1 -- 172.21.15.39:0/1330653488 wait complete. 2024-07-04T08:31:56.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.962+0000 7f0e7ca67740 20 client.19092 trim_cache size 0 max 16384 2024-07-04T08:31:56.953 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:31:56.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 -- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b29e0 msgr2=0x7f0e540b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:56.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b29e0 0x7f0e540b4ef0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55c2cad48430 tx=0x7f0e60004a90 comp rx=0 tx=0).stop 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 -- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70109700 msgr2=0x7f0e701a00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70109700 0x7f0e701a00d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000e9f0 tx=0x7f0e6000edf0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 -- 172.21.15.39:0/1418989228 shutdown_connections 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f0e540b29e0 0x7f0e540b4ef0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e7010f990 0x7f0e701a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e7010f040 0x7f0e701a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 --2- 172.21.15.39:0/1418989228 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e70109700 0x7f0e701a00d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 -- 172.21.15.39:0/1418989228 >> 172.21.15.39:0/1418989228 conn(0x7f0e70077630 msgr2=0x7f0e70107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 -- 172.21.15.39:0/1418989228 shutdown_connections 2024-07-04T08:31:56.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:31:56.963+0000 7f0e76400640 1 -- 172.21.15.39:0/1418989228 wait complete. 2024-07-04T08:31:57.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:57 smithi039 ceph-mon[32062]: pgmap v1002: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-07-04T08:31:57.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:57 smithi050 ceph-mon[36577]: pgmap v1002: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-07-04T08:31:57.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:57 smithi133 ceph-mon[37050]: pgmap v1002: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-07-04T08:31:57.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-04T08:31:58.445 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:31:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:31:59 smithi039 ceph-mon[32062]: pgmap v1003: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-07-04T08:31:59.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:31:59 smithi050 ceph-mon[36577]: pgmap v1003: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-07-04T08:31:59.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:31:59 smithi133 ceph-mon[37050]: pgmap v1003: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-07-04T08:32:00.513 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:00 smithi039 ceph-mon[32062]: pgmap v1004: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-07-04T08:32:00.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.746+0000 7f4afd000640 1 -- 172.21.15.39:0/3550157005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af8101fb0 msgr2=0x7f4af81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:00.733 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.746+0000 7f4afd000640 1 --2- 172.21.15.39:0/3550157005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af8101fb0 0x7f4af81023c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009900 tx=0x7f4ae802a410 comp rx=0 tx=0).stop 2024-07-04T08:32:00.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.746+0000 7f4afd000640 1 -- 172.21.15.39:0/3550157005 shutdown_connections 2024-07-04T08:32:00.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.746+0000 7f4afd000640 1 --2- 172.21.15.39:0/3550157005 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4af8108240 0x7f4af810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.746+0000 7f4afd000640 1 --2- 172.21.15.39:0/3550157005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81078f0 0x7f4af8107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.746+0000 7f4afd000640 1 --2- 172.21.15.39:0/3550157005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af8101fb0 0x7f4af81023c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.746+0000 7f4afd000640 1 -- 172.21.15.39:0/3550157005 >> 172.21.15.39:0/3550157005 conn(0x7f4af8078c80 msgr2=0x7f4af8079090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:00.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.746+0000 7f4afd000640 1 -- 172.21.15.39:0/3550157005 shutdown_connections 2024-07-04T08:32:00.734 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.746+0000 7f4afd000640 1 -- 172.21.15.39:0/3550157005 wait complete. 2024-07-04T08:32:00.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.747+0000 7f4afd000640 1 Processor -- start 2024-07-04T08:32:00.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.748+0000 7f4afd000640 1 -- start start 2024-07-04T08:32:00.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.748+0000 7f4afd000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4af8101fb0 0x7f4af81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.748+0000 7f4afd000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81078f0 0x7f4af8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.748+0000 7f4afd000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af8108240 0x7f4af819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.748+0000 7f4afd000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8111eb0 con 0x7f4af8108240 2024-07-04T08:32:00.735 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.748+0000 7f4afd000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8111d30 con 0x7f4af8101fb0 2024-07-04T08:32:00.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.748+0000 7f4afd000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8112030 con 0x7f4af81078f0 2024-07-04T08:32:00.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4af8101fb0 0x7f4af81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af8108240 0x7f4af819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4af8101fb0 0x7f4af81999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:55668/0 (socket says 172.21.15.39:55668) 2024-07-04T08:32:00.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5e00640 1 -- 172.21.15.39:0/583899420 learned_addr learned my addr 172.21.15.39:0/583899420 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:00.736 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81078f0 0x7f4af8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5e00640 1 -- 172.21.15.39:0/583899420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81078f0 msgr2=0x7f4af8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5e00640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81078f0 0x7f4af8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5e00640 1 -- 172.21.15.39:0/583899420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af8108240 msgr2=0x7f4af819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5e00640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af8108240 0x7f4af819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5e00640 1 -- 172.21.15.39:0/583899420 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af819f940 con 0x7f4af8101fb0 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.749+0000 7f4af5400640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81078f0 0x7f4af8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.750+0000 7f4af5e00640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4af8101fb0 0x7f4af81999d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8005940 tx=0x7f4ae80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.750+0000 7f4aeea00640 1 -- 172.21.15.39:0/583899420 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae803f030 con 0x7f4af8101fb0 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.750+0000 7f4afd000640 1 -- 172.21.15.39:0/583899420 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af819fbd0 con 0x7f4af8101fb0 2024-07-04T08:32:00.737 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.750+0000 7f4aeea00640 1 -- 172.21.15.39:0/583899420 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae80044c0 con 0x7f4af8101fb0 2024-07-04T08:32:00.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.750+0000 7f4aeea00640 1 -- 172.21.15.39:0/583899420 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8035070 con 0x7f4af8101fb0 2024-07-04T08:32:00.738 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.750+0000 7f4afd000640 1 -- 172.21.15.39:0/583899420 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af81a6390 con 0x7f4af8101fb0 2024-07-04T08:32:00.739 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.753+0000 7f4b03714740 1 Processor -- start 2024-07-04T08:32:00.740 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4b03714740 1 -- start start 2024-07-04T08:32:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4b03714740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b62f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4b03714740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b2621b0 0x55d65b62fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4b03714740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b26a390 0x55d65b630250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4b03714740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d65b5a5f00 con 0x55d65b28b070 2024-07-04T08:32:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4b03714740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d65b5954a0 con 0x55d65b2621b0 2024-07-04T08:32:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4b03714740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d65b5994e0 con 0x55d65b26a390 2024-07-04T08:32:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4af6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b62f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4af6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b62f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43520/0 (socket says 172.21.15.39:43520) 2024-07-04T08:32:00.741 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.754+0000 7f4af6800640 1 -- 172.21.15.39:0/3067736517 learned_addr learned my addr 172.21.15.39:0/3067736517 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.755+0000 7f4af5400640 1 --2- 172.21.15.39:0/3067736517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b26a390 0x55d65b630250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.755+0000 7f4af6800640 1 -- 172.21.15.39:0/3067736517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b26a390 msgr2=0x55d65b630250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:00.742 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.755+0000 7f4af6800640 1 --2- 172.21.15.39:0/3067736517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b26a390 0x55d65b630250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.755+0000 7f4af6800640 1 -- 172.21.15.39:0/3067736517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b2621b0 msgr2=0x55d65b62fd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-04T08:32:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.755+0000 7f4af6800640 1 --2- 172.21.15.39:0/3067736517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b2621b0 0x55d65b62fd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.755+0000 7f4af6800640 1 -- 172.21.15.39:0/3067736517 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65b52d670 con 0x55d65b28b070 2024-07-04T08:32:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.755+0000 7f4af5400640 1 --2- 172.21.15.39:0/3067736517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b26a390 0x55d65b630250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:32:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.755+0000 7f4aeea00640 1 -- 172.21.15.39:0/583899420 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f4ae8042030 con 0x7f4af8101fb0 2024-07-04T08:32:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.755+0000 7f4af6800640 1 --2- 172.21.15.39:0/3067736517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b62f7d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4af0011440 tx=0x7f4af0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a628fcaf812990dc server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.756+0000 7f4ae3400640 1 -- 172.21.15.39:0/3067736517 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0003c30 con 0x55d65b28b070 2024-07-04T08:32:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.756+0000 7f4ae3400640 1 -- 172.21.15.39:0/3067736517 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4af0003dd0 con 0x55d65b28b070 2024-07-04T08:32:00.743 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.756+0000 7f4ae3400640 1 -- 172.21.15.39:0/3067736517 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af000fa80 con 0x55d65b28b070 2024-07-04T08:32:00.744 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4b03714740 1 -- 172.21.15.39:0/3067736517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 msgr2=0x55d65b62f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4b03714740 1 --2- 172.21.15.39:0/3067736517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b62f7d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4af0011440 tx=0x7f4af0011900 comp rx=0 tx=0).stop 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4aeea00640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4adc0b29e0 0x7f4adc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4b03714740 1 -- 172.21.15.39:0/3067736517 shutdown_connections 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4b03714740 1 --2- 172.21.15.39:0/3067736517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b26a390 0x55d65b630250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4b03714740 1 --2- 172.21.15.39:0/3067736517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b2621b0 0x55d65b62fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4b03714740 1 --2- 172.21.15.39:0/3067736517 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b62f7d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4aeea00640 1 -- 172.21.15.39:0/583899420 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f4ae8102c50 con 0x7f4af8101fb0 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4b03714740 1 -- 172.21.15.39:0/3067736517 >> 172.21.15.39:0/3067736517 conn(0x55d65b27f460 msgr2=0x55d65b62d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4af6800640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4adc0b29e0 0x7f4adc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4b03714740 1 -- 172.21.15.39:0/3067736517 shutdown_connections 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.757+0000 7f4b03714740 1 -- 172.21.15.39:0/3067736517 wait complete. 2024-07-04T08:32:00.745 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.758+0000 7f4b03714740 1 -- start start 2024-07-04T08:32:00.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.758+0000 7f4af6800640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4adc0b29e0 0x7f4adc0b4ef0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4af0005fd0 tx=0x7f4af0000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:00.746 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.758+0000 7f4b03714740 1 Processor -- start 2024-07-04T08:32:00.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4b03714740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b63f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4b03714740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b26a390 0x55d65b63fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.747 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4b03714740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b28b070 0x55d65b64b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4b03714740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x55d65b3f9700 con 0x55d65b26a390 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4b03714740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d65b53cbd0 con 0x55d65b28b070 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4b03714740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d65b642310 con 0x55d65b2621b0 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4b03714740 10 client.0 _ll_register_callbacks cb 0x55d65b58e590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4af5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b26a390 0x55d65b63fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4af5400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b26a390 0x55d65b63fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43534/0 (socket says 172.21.15.39:43534) 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4af5400640 1 -- 172.21.15.39:0/3776950253 learned_addr learned my addr 172.21.15.39:0/3776950253 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4af5e00640 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b63f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.759+0000 7f4af6800640 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b28b070 0x55d65b64b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.760+0000 7f4af5400640 1 -- 172.21.15.39:0/3776950253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 msgr2=0x55d65b63f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.760+0000 7f4af5400640 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b63f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.760+0000 7f4af5400640 1 -- 172.21.15.39:0/3776950253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b28b070 msgr2=0x55d65b64b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:00.748 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.760+0000 7f4af5400640 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b28b070 0x55d65b64b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.760+0000 7f4af5400640 1 -- 172.21.15.39:0/3776950253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65b2f6650 con 0x55d65b26a390 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.760+0000 7f4af5e00640 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b63f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.760+0000 7f4af6800640 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b28b070 0x55d65b64b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.760+0000 7f4af5400640 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b26a390 0x55d65b63fcd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4012980 tx=0x7f4ae4012e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb491bd3cc9efa7 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.760+0000 7f4ad7e00640 10 client.0 ms_handle_connect on v2:172.21.15.39:3300/0 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.761+0000 7f4ad7e00640 1 -- 172.21.15.39:0/3776950253 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400fa00 con 0x55d65b26a390 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.761+0000 7f4ad7e00640 1 -- 172.21.15.39:0/3776950253 <== mon.0 v2:172.21.15.39:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f4ae400fba0 con 0x55d65b26a390 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.761+0000 7f4b03714740 1 -- 172.21.15.39:0/3776950253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4af0003dd0 con 0x55d65b26a390 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.761+0000 7f4ad7e00640 1 -- 172.21.15.39:0/3776950253 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4ae400fda0 con 0x55d65b26a390 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.761+0000 7f4b03714740 10 client.19104 fetch_fsmap learned FSMap version 30 2024-07-04T08:32:00.749 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.761+0000 7f4b03714740 1 -- 172.21.15.39:0/3776950253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d65b64e3c0 con 0x55d65b26a390 2024-07-04T08:32:00.751 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.761+0000 7f4ad7e00640 1 -- 172.21.15.39:0/3776950253 <== mon.0 v2:172.21.15.39:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4af0003dd0 con 0x55d65b26a390 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.762+0000 7f4ad7e00640 1 -- 172.21.15.39:0/3776950253 <== mon.0 v2:172.21.15.39:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4ae401c420 con 0x55d65b26a390 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.762+0000 7f4b03714740 10 client.19104 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.762+0000 7f4b03714740 10 client.19104 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.762+0000 7f4b03714740 20 client.19104 populate_metadata read hostname 'smithi039' 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.762+0000 7f4b03714740 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d65b2499f0 0x55d65b651930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.762+0000 7f4b03714740 4 client.19104 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.762+0000 7f4b03714740 1 -- 172.21.15.39:0/3776950253 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d65b5a7e50 con 0x55d65b2499f0 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.762+0000 7f4af5e00640 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d65b2499f0 0x55d65b651930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.763+0000 7f4af5e00640 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d65b2499f0 0x55d65b651930 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6850c163fa4fc627 server_cookie=3ac7fa31d2d6a80c in_seq=0 out_seq=0 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.763+0000 7f4ad7e00640 10 client.19104 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4ad7e00640 1 -- 172.21.15.39:0/3776950253 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d65b5a7e50 con 0x55d65b2499f0 2024-07-04T08:32:00.752 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4ad7e00640 10 client.19104 handle_command_reply: tid=0 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 client.19104 shutdown 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 -- 172.21.15.39:0/3776950253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b26a390 msgr2=0x55d65b63fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b26a390 0x55d65b63fcd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4012980 tx=0x7f4ae4012e40 comp rx=0 tx=0).stop 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 -- 172.21.15.39:0/3776950253 shutdown_connections 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d65b2499f0 0x55d65b651930 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b26a390 0x55d65b63fcd0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b28b070 0x55d65b64b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 --2- 172.21.15.39:0/3776950253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b63f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4ad7e00640 0 client.19104 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 -- 172.21.15.39:0/3776950253 >> 172.21.15.39:0/3776950253 conn(0x55d65b27f460 msgr2=0x55d65b62ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 -- 172.21.15.39:0/3776950253 shutdown_connections 2024-07-04T08:32:00.753 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.764+0000 7f4b03714740 1 -- 172.21.15.39:0/3776950253 wait complete. 2024-07-04T08:32:00.754 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.765+0000 7f4b03714740 20 client.19104 trim_cache size 0 max 16384 2024-07-04T08:32:00.766 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.780+0000 7f4b03714740 1 Processor -- start 2024-07-04T08:32:00.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.780+0000 7f4b03714740 1 -- start start 2024-07-04T08:32:00.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4b03714740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b641470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4b03714740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 0x55d65b630f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4b03714740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b639960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4b03714740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81ae1a0 con 0x55d65b28b070 2024-07-04T08:32:00.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4b03714740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81afce0 con 0x55d65b26a390 2024-07-04T08:32:00.767 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4b03714740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81ae870 con 0x55d65b2621b0 2024-07-04T08:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4af5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b641470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4af5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b641470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36260/0 (socket says 172.21.15.39:36260) 2024-07-04T08:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4af5e00640 1 -- 172.21.15.39:0/3362661539 learned_addr learned my addr 172.21.15.39:0/3362661539 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4af6800640 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b639960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.781+0000 7f4af5400640 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 0x55d65b630f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4af5e00640 1 -- 172.21.15.39:0/3362661539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 msgr2=0x55d65b630f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4af5e00640 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 0x55d65b630f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4af5e00640 1 -- 172.21.15.39:0/3362661539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 msgr2=0x55d65b639960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4af5e00640 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b639960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4af5e00640 1 -- 172.21.15.39:0/3362661539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65b640630 con 0x55d65b2621b0 2024-07-04T08:32:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4af5400640 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 0x55d65b630f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:32:00.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4af5e00640 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b641470 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80c9870 tx=0x7f4ae802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d906a171208b2e6a server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4ae2a00640 1 -- 172.21.15.39:0/3362661539 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8049040 con 0x55d65b2621b0 2024-07-04T08:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4ae2a00640 1 -- 172.21.15.39:0/3362661539 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae80ad020 con 0x55d65b2621b0 2024-07-04T08:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.782+0000 7f4ae2a00640 1 -- 172.21.15.39:0/3362661539 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8100260 con 0x55d65b2621b0 2024-07-04T08:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.783+0000 7f4b03714740 1 -- 172.21.15.39:0/3362661539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 msgr2=0x55d65b641470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.783+0000 7f4b03714740 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b641470 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80c9870 tx=0x7f4ae802a410 comp rx=0 tx=0).stop 2024-07-04T08:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.783+0000 7f4b03714740 1 -- 172.21.15.39:0/3362661539 shutdown_connections 2024-07-04T08:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.783+0000 7f4b03714740 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b639960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.770 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.783+0000 7f4b03714740 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 0x55d65b630f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.783+0000 7f4b03714740 1 --2- 172.21.15.39:0/3362661539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b641470 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.783+0000 7f4b03714740 1 -- 172.21.15.39:0/3362661539 >> 172.21.15.39:0/3362661539 conn(0x55d65b27f460 msgr2=0x55d65b630800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:00.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.784+0000 7f4b03714740 1 -- 172.21.15.39:0/3362661539 shutdown_connections 2024-07-04T08:32:00.771 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.784+0000 7f4b03714740 1 -- 172.21.15.39:0/3362661539 wait complete. 2024-07-04T08:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.784+0000 7f4b03714740 1 -- start start 2024-07-04T08:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.784+0000 7f4b03714740 1 Processor -- start 2024-07-04T08:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.785+0000 7f4b03714740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b636130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.785+0000 7f4b03714740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 0x55d65b636670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.785+0000 7f4b03714740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b636bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.785+0000 7f4b03714740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81af610 con 0x55d65b28b070 2024-07-04T08:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.785+0000 7f4b03714740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81aef40 con 0x55d65b26a390 2024-07-04T08:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.785+0000 7f4b03714740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81ac180 con 0x55d65b2621b0 2024-07-04T08:32:00.772 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.785+0000 7f4b03714740 10 client.0 _ll_register_callbacks cb 0x7f4ae4009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:32:00.773 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b636bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b636130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af6800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b636bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:43550/0 (socket says 172.21.15.39:43550) 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af6800640 1 -- 172.21.15.39:0/197517987 learned_addr learned my addr 172.21.15.39:0/197517987 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b636130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:36262/0 (socket says 172.21.15.39:36262) 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af5400640 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 0x55d65b636670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af5e00640 1 -- 172.21.15.39:0/197517987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 msgr2=0x55d65b636670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af5e00640 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 0x55d65b636670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af5e00640 1 -- 172.21.15.39:0/197517987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 msgr2=0x55d65b636bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af5e00640 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b636bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.786+0000 7f4af5e00640 1 -- 172.21.15.39:0/197517987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65b646770 con 0x55d65b2621b0 2024-07-04T08:32:00.774 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.787+0000 7f4af5e00640 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b636130 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80c9870 tx=0x7f4ae80c6090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f920270377e565 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.787+0000 7f4ad7e00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:32:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.787+0000 7f4ad7e00640 1 -- 172.21.15.39:0/197517987 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8049070 con 0x55d65b2621b0 2024-07-04T08:32:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.787+0000 7f4ad7e00640 1 -- 172.21.15.39:0/197517987 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f4ae80c6160 con 0x55d65b2621b0 2024-07-04T08:32:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.787+0000 7f4b03714740 1 -- 172.21.15.39:0/197517987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae80ad020 con 0x55d65b2621b0 2024-07-04T08:32:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.787+0000 7f4ad7e00640 1 -- 172.21.15.39:0/197517987 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4ae80c9b70 con 0x55d65b2621b0 2024-07-04T08:32:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.787+0000 7f4b03714740 10 client.29158 fetch_fsmap learned FSMap version 30 2024-07-04T08:32:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.787+0000 7f4b03714740 1 -- 172.21.15.39:0/197517987 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d65b6479f0 con 0x55d65b2621b0 2024-07-04T08:32:00.775 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.787+0000 7f4ad7e00640 1 -- 172.21.15.39:0/197517987 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae80ad020 con 0x55d65b2621b0 2024-07-04T08:32:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.788+0000 7f4ad7e00640 1 -- 172.21.15.39:0/197517987 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4ae80d2540 con 0x55d65b2621b0 2024-07-04T08:32:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.788+0000 7f4b03714740 10 client.29158 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:32:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.788+0000 7f4b03714740 10 client.29158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-07-04T08:32:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.788+0000 7f4b03714740 20 client.29158 populate_metadata read hostname 'smithi039' 2024-07-04T08:32:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.788+0000 7f4b03714740 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d65b21eda0 0x55d65b6c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.788+0000 7f4b03714740 4 client.29158 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:32:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.788+0000 7f4b03714740 1 -- 172.21.15.39:0/197517987 --> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d65b5a38f0 con 0x55d65b21eda0 2024-07-04T08:32:00.777 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.788+0000 7f4af5400640 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d65b21eda0 0x55d65b6c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.789+0000 7f4af5400640 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d65b21eda0 0x55d65b6c0870 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9775476e4b769858 server_cookie=167cae28cd29ab92 in_seq=0 out_seq=0 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.789+0000 7f4ad7e00640 10 client.29158 ms_handle_connect on v2:172.21.15.39:6840/2753029904 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.789+0000 7f4ad7e00640 1 -- 172.21.15.39:0/197517987 <== mds.1 v2:172.21.15.39:6840/2753029904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d65b5a38f0 con 0x55d65b21eda0 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.789+0000 7f4ad7e00640 10 client.29158 handle_command_reply: tid=0 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.789+0000 7f4b03714740 1 client.29158 shutdown 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.789+0000 7f4b03714740 1 -- 172.21.15.39:0/197517987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 msgr2=0x55d65b636130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.789+0000 7f4b03714740 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b636130 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80c9870 tx=0x7f4ae80c6090 comp rx=0 tx=0).stop 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4b03714740 1 -- 172.21.15.39:0/197517987 shutdown_connections 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4b03714740 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.39:6840/2753029904,v1:172.21.15.39:6841/2753029904] conn(0x55d65b21eda0 0x55d65b6c0870 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4b03714740 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d65b28b070 0x55d65b636bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4b03714740 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d65b26a390 0x55d65b636670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4b03714740 1 --2- 172.21.15.39:0/197517987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d65b2621b0 0x55d65b636130 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.778 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4b03714740 1 -- 172.21.15.39:0/197517987 >> 172.21.15.39:0/197517987 conn(0x55d65b27f460 msgr2=0x55d65b62c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4ad7e00640 0 client.29158 ms_handle_reset on v2:172.21.15.39:6840/2753029904 2024-07-04T08:32:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4b03714740 1 -- 172.21.15.39:0/197517987 shutdown_connections 2024-07-04T08:32:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4b03714740 1 -- 172.21.15.39:0/197517987 wait complete. 2024-07-04T08:32:00.779 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.790+0000 7f4b03714740 20 client.29158 trim_cache size 0 max 16384 2024-07-04T08:32:00.779 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:32:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 -- 172.21.15.39:0/583899420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4adc0b29e0 msgr2=0x7f4adc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4adc0b29e0 0x7f4adc0b4ef0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4af0005fd0 tx=0x7f4af0000f40 comp rx=0 tx=0).stop 2024-07-04T08:32:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 -- 172.21.15.39:0/583899420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4af8101fb0 msgr2=0x7f4af81999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4af8101fb0 0x7f4af81999d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8005940 tx=0x7f4ae80040c0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 -- 172.21.15.39:0/583899420 shutdown_connections 2024-07-04T08:32:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f4adc0b29e0 0x7f4adc0b4ef0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af8108240 0x7f4af819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81078f0 0x7f4af8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.780 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 --2- 172.21.15.39:0/583899420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4af8101fb0 0x7f4af81999d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:00.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.791+0000 7f4afd000640 1 -- 172.21.15.39:0/583899420 >> 172.21.15.39:0/583899420 conn(0x7f4af8078c80 msgr2=0x7f4af8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:00.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.792+0000 7f4afd000640 1 -- 172.21.15.39:0/583899420 shutdown_connections 2024-07-04T08:32:00.781 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:00.792+0000 7f4afd000640 1 -- 172.21.15.39:0/583899420 wait complete. 2024-07-04T08:32:00.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:00 smithi133 ceph-mon[37050]: pgmap v1004: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-07-04T08:32:00.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:00 smithi050 ceph-mon[36577]: pgmap v1004: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-07-04T08:32:01.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-04T08:32:02.191 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:32:03.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:03 smithi039 ceph-mon[32062]: pgmap v1005: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-07-04T08:32:03.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:03 smithi050 ceph-mon[36577]: pgmap v1005: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-07-04T08:32:03.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:03 smithi133 ceph-mon[37050]: pgmap v1005: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-07-04T08:32:04.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.560+0000 7fd0f2000640 1 -- 172.21.15.39:0/941515811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec105170 msgr2=0x7fd0ec105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:04.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.560+0000 7fd0f2000640 1 --2- 172.21.15.39:0/941515811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec105170 0x7fd0ec105580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009860 tx=0x7fd0d402a340 comp rx=0 tx=0).stop 2024-07-04T08:32:04.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.560+0000 7fd0f2000640 1 -- 172.21.15.39:0/941515811 shutdown_connections 2024-07-04T08:32:04.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.560+0000 7fd0f2000640 1 --2- 172.21.15.39:0/941515811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ec10b370 0x7fd0ec111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.560+0000 7fd0f2000640 1 --2- 172.21.15.39:0/941515811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0ec10aa20 0x7fd0ec10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.560+0000 7fd0f2000640 1 --2- 172.21.15.39:0/941515811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec105170 0x7fd0ec105580 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.560+0000 7fd0f2000640 1 -- 172.21.15.39:0/941515811 >> 172.21.15.39:0/941515811 conn(0x7fd0ec100be0 msgr2=0x7fd0ec103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:04.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.560+0000 7fd0f2000640 1 -- 172.21.15.39:0/941515811 shutdown_connections 2024-07-04T08:32:04.547 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.560+0000 7fd0f2000640 1 -- 172.21.15.39:0/941515811 wait complete. 2024-07-04T08:32:04.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.562+0000 7fd0f2000640 1 Processor -- start 2024-07-04T08:32:04.548 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.562+0000 7fd0f2000640 1 -- start start 2024-07-04T08:32:04.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.562+0000 7fd0f2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0ec105170 0x7fd0ec19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.562+0000 7fd0f2000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ec10aa20 0x7fd0ec19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.549 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.562+0000 7fd0f2000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec10b370 0x7fd0ec1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.562+0000 7fd0f2000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec115020 con 0x7fd0ec10aa20 2024-07-04T08:32:04.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.562+0000 7fd0f2000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec114ea0 con 0x7fd0ec10b370 2024-07-04T08:32:04.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.562+0000 7fd0f2000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1151a0 con 0x7fd0ec105170 2024-07-04T08:32:04.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0f0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0ec105170 0x7fd0ec19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.550 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0f0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0ec105170 0x7fd0ec19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41156/0 (socket says 172.21.15.39:41156) 2024-07-04T08:32:04.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0f0c00640 1 -- 172.21.15.39:0/357169028 learned_addr learned my addr 172.21.15.39:0/357169028 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:04.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ec10aa20 0x7fd0ec19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0f1600640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec10b370 0x7fd0ec1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.551 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0f0c00640 1 -- 172.21.15.39:0/357169028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec10b370 msgr2=0x7fd0ec1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0f0c00640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec10b370 0x7fd0ec1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0f0c00640 1 -- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ec10aa20 msgr2=0x7fd0ec19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0f0c00640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ec10aa20 0x7fd0ec19bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.563+0000 7fd0f0c00640 1 -- 172.21.15.39:0/357169028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec1a19c0 con 0x7fd0ec105170 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.564+0000 7fd0f0c00640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0ec105170 0x7fd0ec19ba50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40303f0 tx=0x7fd0d4032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.564+0000 7fd0e9600640 1 -- 172.21.15.39:0/357169028 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4040020 con 0x7fd0ec105170 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.564+0000 7fd0e9600640 1 -- 172.21.15.39:0/357169028 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0d4030dd0 con 0x7fd0ec105170 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.564+0000 7fd0e9600640 1 -- 172.21.15.39:0/357169028 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4037040 con 0x7fd0ec105170 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.564+0000 7fd0f2000640 1 -- 172.21.15.39:0/357169028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ec1a1c50 con 0x7fd0ec105170 2024-07-04T08:32:04.552 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.564+0000 7fd0f2000640 1 -- 172.21.15.39:0/357169028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ec1a8410 con 0x7fd0ec105170 2024-07-04T08:32:04.554 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.568+0000 7fd0f9d2d740 1 Processor -- start 2024-07-04T08:32:04.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.568+0000 7fd0e9600640 1 -- 172.21.15.39:0/357169028 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fd0d4043020 con 0x7fd0ec105170 2024-07-04T08:32:04.555 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.568+0000 7fd0f9d2d740 1 -- start start 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564b0b847ee0 con 0x564b0b52d070 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564b0b837480 con 0x564b0b5041b0 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b0b83b4c0 con 0x564b0b50c390 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f0c00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8d1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:42118/0 (socket says 172.21.15.39:42118) 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f0c00640 1 -- 172.21.15.39:0/1704604253 learned_addr learned my addr 172.21.15.39:0/1704604253 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:04.556 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0f1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.569+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/1704604253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.570+0000 7fd0f1600640 1 -- 172.21.15.39:0/1704604253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 msgr2=0x564b0b8d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:04.557 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.570+0000 7fd0f1600640 1 --2- 172.21.15.39:0/1704604253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.570+0000 7fd0f1600640 1 -- 172.21.15.39:0/1704604253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 msgr2=0x564b0b8d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.570+0000 7fd0f1600640 1 --2- 172.21.15.39:0/1704604253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8d1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.570+0000 7fd0f1600640 1 -- 172.21.15.39:0/1704604253 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0b7cf650 con 0x564b0b52d070 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.570+0000 7fd0f0c00640 1 --2- 172.21.15.39:0/1704604253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.570+0000 7fd0f1600640 1 --2- 172.21.15.39:0/1704604253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d1950 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009b90 tx=0x7fd0e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443d066ef3f38cd server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.571+0000 7fd0e9600640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd0d00b29e0 0x7fd0d00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.571+0000 7fd0e2000640 1 -- 172.21.15.39:0/1704604253 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e401e070 con 0x564b0b52d070 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.571+0000 7fd0e9600640 1 -- 172.21.15.39:0/357169028 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fd0d4102460 con 0x7fd0ec105170 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.571+0000 7fd0e2000640 1 -- 172.21.15.39:0/1704604253 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e40035f0 con 0x564b0b52d070 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.571+0000 7fd0e2000640 1 -- 172.21.15.39:0/1704604253 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4019670 con 0x564b0b52d070 2024-07-04T08:32:04.558 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.571+0000 7fd0f0c00640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd0d00b29e0 0x7fd0d00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.559 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/1704604253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 msgr2=0x564b0b8d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/1704604253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d1950 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009b90 tx=0x7fd0e4011810 comp rx=0 tx=0).stop 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f0c00640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd0d00b29e0 0x7fd0d00b4ef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4102700 tx=0x7fd0d4105f10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/1704604253 shutdown_connections 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/1704604253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/1704604253 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/1704604253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d1950 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/1704604253 >> 172.21.15.39:0/1704604253 conn(0x564b0b521460 msgr2=0x564b0b8cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/1704604253 shutdown_connections 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.572+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/1704604253 wait complete. 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.573+0000 7fd0f9d2d740 1 -- start start 2024-07-04T08:32:04.560 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.573+0000 7fd0f9d2d740 1 Processor -- start 2024-07-04T08:32:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8dd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8ddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.562 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x564b0b69b700 con 0x564b0b52d070 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564b0b7debb0 con 0x564b0b5041b0 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b0b8e0130 con 0x564b0b50c390 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f9d2d740 10 client.0 _ll_register_callbacks cb 0x564b0b830570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49434/0 (socket says 172.21.15.39:49434) 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f1600640 1 -- 172.21.15.39:0/2629839464 learned_addr learned my addr 172.21.15.39:0/2629839464 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0f1600640 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8dd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.574+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8ddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.575+0000 7fd0ebe00640 1 -- 172.21.15.39:0/2629839464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 msgr2=0x564b0b8dd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.575+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8dd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.575+0000 7fd0ebe00640 1 -- 172.21.15.39:0/2629839464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 msgr2=0x564b0b8e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:04.563 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.575+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.575+0000 7fd0ebe00640 1 -- 172.21.15.39:0/2629839464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0b598650 con 0x564b0b50c390 2024-07-04T08:32:04.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.575+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8ddbb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec06ac50 tx=0x7fd0dc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2cfb30578a854b2 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:04.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.575+0000 7fd0daa00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:32:04.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.576+0000 7fd0daa00640 1 -- 172.21.15.39:0/2629839464 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc009e50 con 0x564b0b50c390 2024-07-04T08:32:04.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.576+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/2629839464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0e40035f0 con 0x564b0b50c390 2024-07-04T08:32:04.564 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.576+0000 7fd0daa00640 1 -- 172.21.15.39:0/2629839464 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fd0dc004b20 con 0x564b0b50c390 2024-07-04T08:32:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.576+0000 7fd0daa00640 1 -- 172.21.15.39:0/2629839464 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd0dc025050 con 0x564b0b50c390 2024-07-04T08:32:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.576+0000 7fd0f9d2d740 10 client.29170 fetch_fsmap learned FSMap version 30 2024-07-04T08:32:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.576+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/2629839464 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b0b8ec2b0 con 0x564b0b50c390 2024-07-04T08:32:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.576+0000 7fd0daa00640 1 -- 172.21.15.39:0/2629839464 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e40035f0 con 0x564b0b50c390 2024-07-04T08:32:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.577+0000 7fd0daa00640 1 -- 172.21.15.39:0/2629839464 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd0dc019030 con 0x564b0b50c390 2024-07-04T08:32:04.566 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.577+0000 7fd0f9d2d740 10 client.29170 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.577+0000 7fd0f9d2d740 10 client.29170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.577+0000 7fd0f9d2d740 20 client.29170 populate_metadata read hostname 'smithi039' 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.577+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b0b4eb9f0 0x564b0b8ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.577+0000 7fd0f9d2d740 4 client.29170 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.577+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/2629839464 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b0b849e30 con 0x564b0b4eb9f0 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.577+0000 7fd0f1600640 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b0b4eb9f0 0x564b0b8ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.578+0000 7fd0f1600640 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b0b4eb9f0 0x564b0b8ef790 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b26b5e9c848c826 server_cookie=2e31eeb367958d12 in_seq=0 out_seq=0 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.578+0000 7fd0daa00640 10 client.29170 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0daa00640 1 -- 172.21.15.39:0/2629839464 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564b0b849e30 con 0x564b0b4eb9f0 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0daa00640 10 client.29170 handle_command_reply: tid=0 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0f9d2d740 1 client.29170 shutdown 2024-07-04T08:32:04.567 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/2629839464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 msgr2=0x564b0b8ddbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8ddbb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec06ac50 tx=0x7fd0dc00f790 comp rx=0 tx=0).stop 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/2629839464 shutdown_connections 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b0b4eb9f0 0x564b0b8ef790 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8ddbb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/2629839464 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8dd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/2629839464 >> 172.21.15.39:0/2629839464 conn(0x564b0b521460 msgr2=0x564b0b8d4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.579+0000 7fd0daa00640 0 client.29170 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.580+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/2629839464 shutdown_connections 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.580+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/2629839464 wait complete. 2024-07-04T08:32:04.568 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.580+0000 7fd0f9d2d740 20 client.29170 trim_cache size 0 max 16384 2024-07-04T08:32:04.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.595+0000 7fd0f9d2d740 1 Processor -- start 2024-07-04T08:32:04.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.595+0000 7fd0f9d2d740 1 -- start start 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b5041b0 0x564b0b8ddc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b50c390 0x564b0b8d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b23f0 con 0x564b0b52d070 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b3f30 con 0x564b0b50c390 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b2ac0 con 0x564b0b5041b0 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0ebe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b50c390 0x564b0b8d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f1600640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49436/0 (socket says 172.21.15.39:49436) 2024-07-04T08:32:04.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f1600640 1 -- 172.21.15.39:0/392675243 learned_addr learned my addr 172.21.15.39:0/392675243 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:04.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.596+0000 7fd0f1600640 1 --2- 172.21.15.39:0/392675243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b5041b0 0x564b0b8ddc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.597+0000 7fd0ebe00640 1 -- 172.21.15.39:0/392675243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b5041b0 msgr2=0x564b0b8ddc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:04.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.597+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/392675243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b5041b0 0x564b0b8ddc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.597+0000 7fd0ebe00640 1 -- 172.21.15.39:0/392675243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 msgr2=0x564b0b8d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:04.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.597+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/392675243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.597+0000 7fd0ebe00640 1 -- 172.21.15.39:0/392675243 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0b8d41e0 con 0x564b0b50c390 2024-07-04T08:32:04.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.597+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/392675243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b50c390 0x564b0b8d6a60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009a00 tx=0x7fd0dc005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4941fac17064d22d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:04.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.597+0000 7fd0e1600640 1 -- 172.21.15.39:0/392675243 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc01a5e0 con 0x564b0b50c390 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.597+0000 7fd0e1600640 1 -- 172.21.15.39:0/392675243 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0dc01a780 con 0x564b0b50c390 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.597+0000 7fd0e1600640 1 -- 172.21.15.39:0/392675243 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc005310 con 0x564b0b50c390 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.598+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/392675243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b50c390 msgr2=0x564b0b8d6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.598+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/392675243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b50c390 0x564b0b8d6a60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009a00 tx=0x7fd0dc005d10 comp rx=0 tx=0).stop 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.598+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/392675243 shutdown_connections 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.598+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/392675243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.598+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/392675243 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b50c390 0x564b0b8d6a60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.598+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/392675243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b5041b0 0x564b0b8ddc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.598+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/392675243 >> 172.21.15.39:0/392675243 conn(0x564b0b521460 msgr2=0x564b0b8dbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.598+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/392675243 shutdown_connections 2024-07-04T08:32:04.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.598+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/392675243 wait complete. 2024-07-04T08:32:04.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.599+0000 7fd0f9d2d740 1 -- start start 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.599+0000 7fd0f9d2d740 1 Processor -- start 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8e0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f9d2d740 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b3860 con 0x564b0b52d070 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b3190 con 0x564b0b5041b0 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f9d2d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec1b03d0 con 0x564b0b50c390 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f9d2d740 10 client.0 _ll_register_callbacks cb 0x7fd0dc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0ebe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.589 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0ebe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8e09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41200/0 (socket says 172.21.15.39:41200) 2024-07-04T08:32:04.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0ebe00640 1 -- 172.21.15.39:0/4137242048 learned_addr learned my addr 172.21.15.39:0/4137242048 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:04.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f1600640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8e0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f1600640 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0ebe00640 1 -- 172.21.15.39:0/4137242048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 msgr2=0x564b0b8e0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:04.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8e0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0ebe00640 1 -- 172.21.15.39:0/4137242048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 msgr2=0x564b0b8e0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-04T08:32:04.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0ebe00640 1 -- 172.21.15.39:0/4137242048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0b956130 con 0x564b0b50c390 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.600+0000 7fd0f1600640 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.601+0000 7fd0ebe00640 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8e09c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009500 tx=0x7fd0dc009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb5beddd7389c8d server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.601+0000 7fd0daa00640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.601+0000 7fd0daa00640 1 -- 172.21.15.39:0/4137242048 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc02e040 con 0x564b0b50c390 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.601+0000 7fd0daa00640 1 -- 172.21.15.39:0/4137242048 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fd0dc003b70 con 0x564b0b50c390 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.601+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/4137242048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0dc01a780 con 0x564b0b50c390 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0daa00640 1 -- 172.21.15.39:0/4137242048 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd0dc040050 con 0x564b0b50c390 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0f9d2d740 10 client.29176 fetch_fsmap learned FSMap version 30 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/4137242048 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564b0b9573b0 con 0x564b0b50c390 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0daa00640 1 -- 172.21.15.39:0/4137242048 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0dc01a780 con 0x564b0b50c390 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0daa00640 1 -- 172.21.15.39:0/4137242048 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd0dc031070 con 0x564b0b50c390 2024-07-04T08:32:04.591 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0f9d2d740 10 client.29176 fetch_fsmap finished waiting for FSMap version 30 2024-07-04T08:32:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0f9d2d740 10 client.29176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-04T08:32:04.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0f9d2d740 20 client.29176 populate_metadata read hostname 'smithi039' 2024-07-04T08:32:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b0b4c0da0 0x564b0b9599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0f9d2d740 4 client.29176 mds_command: new command op to 24707 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-04T08:32:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.602+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/4137242048 --> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b0b8458d0 con 0x564b0b4c0da0 2024-07-04T08:32:04.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.603+0000 7fd0f1600640 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b0b4c0da0 0x564b0b9599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:04.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.603+0000 7fd0f1600640 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b0b4c0da0 0x564b0b9599c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739035efb34b2629 server_cookie=aaa3d9c7abd0c664 in_seq=0 out_seq=0 2024-07-04T08:32:04.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.603+0000 7fd0daa00640 10 client.29176 ms_handle_connect on v2:172.21.15.50:6838/2852157181 2024-07-04T08:32:04.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.603+0000 7fd0daa00640 1 -- 172.21.15.39:0/4137242048 <== mds.0 v2:172.21.15.50:6838/2852157181 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b0b8458d0 con 0x564b0b4c0da0 2024-07-04T08:32:04.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.603+0000 7fd0daa00640 10 client.29176 handle_command_reply: tid=0 2024-07-04T08:32:04.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.603+0000 7fd0f9d2d740 1 client.29176 shutdown 2024-07-04T08:32:04.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/4137242048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 msgr2=0x564b0b8e09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:04.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8e09c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009500 tx=0x7fd0dc009e50 comp rx=0 tx=0).stop 2024-07-04T08:32:04.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/4137242048 shutdown_connections 2024-07-04T08:32:04.595 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.50:6838/2852157181,v1:172.21.15.50:6839/2852157181] conn(0x564b0b4c0da0 0x564b0b9599c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564b0b52d070 0x564b0b8e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b0b50c390 0x564b0b8e09c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 --2- 172.21.15.39:0/4137242048 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0b5041b0 0x564b0b8e0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0daa00640 0 client.29176 ms_handle_reset on v2:172.21.15.50:6838/2852157181 2024-07-04T08:32:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/4137242048 >> 172.21.15.39:0/4137242048 conn(0x564b0b521460 msgr2=0x564b0b8d6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/4137242048 shutdown_connections 2024-07-04T08:32:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 1 -- 172.21.15.39:0/4137242048 wait complete. 2024-07-04T08:32:04.596 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.604+0000 7fd0f9d2d740 20 client.29176 trim_cache size 0 max 16384 2024-07-04T08:32:04.596 INFO:teuthology.orchestra.run.smithi039.stdout:[] 2024-07-04T08:32:04.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 -- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd0d00b29e0 msgr2=0x7fd0d00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:04.597 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd0d00b29e0 0x7fd0d00b4ef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4102700 tx=0x7fd0d4105f10 comp rx=0 tx=0).stop 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 -- 172.21.15.39:0/357169028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0ec105170 msgr2=0x7fd0ec19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0ec105170 0x7fd0ec19ba50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd0d40303f0 tx=0x7fd0d4032da0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 -- 172.21.15.39:0/357169028 shutdown_connections 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fd0d00b29e0 0x7fd0d00b4ef0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0ec10aa20 0x7fd0ec19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0ec10b370 0x7fd0ec1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 --2- 172.21.15.39:0/357169028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0ec105170 0x7fd0ec19ba50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 -- 172.21.15.39:0/357169028 >> 172.21.15.39:0/357169028 conn(0x7fd0ec100be0 msgr2=0x7fd0ec103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 -- 172.21.15.39:0/357169028 shutdown_connections 2024-07-04T08:32:04.598 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:04.605+0000 7fd0f2000640 1 -- 172.21.15.39:0/357169028 wait complete. 2024-07-04T08:32:05.523 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_2e9f68b5506c607b2f7d69d380cb7887cf00dc84/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_2e9f68b5506c607b2f7d69d380cb7887cf00dc84/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_2e9f68b5506c607b2f7d69d380cb7887cf00dc84/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-07-04T08:32:05.523 INFO:tasks.fwd_scrub:done joining 2024-07-04T08:32:05.524 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-07-04T08:32:05.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-04T08:32:05.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:05 smithi039 ceph-mon[32062]: pgmap v1006: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:32:05.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:05 smithi050 ceph-mon[36577]: pgmap v1006: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:32:05.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:05 smithi133 ceph-mon[37050]: pgmap v1006: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:32:05.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.844+0000 7f5191e00640 1 -- 172.21.15.39:0/624498308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 msgr2=0x7f518c105630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:05.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.844+0000 7f5191e00640 1 --2- 172.21.15.39:0/624498308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 0x7f518c105630 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f517c009900 tx=0x7f517c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:32:05.831 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.844+0000 7f5191e00640 1 -- 172.21.15.39:0/624498308 shutdown_connections 2024-07-04T08:32:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.844+0000 7f5191e00640 1 --2- 172.21.15.39:0/624498308 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f518c10b4a0 0x7f518c111cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.844+0000 7f5191e00640 1 --2- 172.21.15.39:0/624498308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c10ab50 0x7f518c10af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.844+0000 7f5191e00640 1 --2- 172.21.15.39:0/624498308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 0x7f518c105630 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.844+0000 7f5191e00640 1 -- 172.21.15.39:0/624498308 >> 172.21.15.39:0/624498308 conn(0x7f518c100c20 msgr2=0x7f518c103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.844+0000 7f5191e00640 1 -- 172.21.15.39:0/624498308 shutdown_connections 2024-07-04T08:32:05.832 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.844+0000 7f5191e00640 1 -- 172.21.15.39:0/624498308 wait complete. 2024-07-04T08:32:05.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f5191e00640 1 Processor -- start 2024-07-04T08:32:05.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f5191e00640 1 -- start start 2024-07-04T08:32:05.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f5191e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 0x7f518c19bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:05.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f5191e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c10ab50 0x7f518c19c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:05.835 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f5191e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f518c10b4a0 0x7f518c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f5191e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1151a0 con 0x7f518c105220 2024-07-04T08:32:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f5191e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f518c115020 con 0x7f518c10b4a0 2024-07-04T08:32:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f5191e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f518c115320 con 0x7f518c10ab50 2024-07-04T08:32:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f518aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 0x7f518c19bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.846+0000 7f518aa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 0x7f518c19bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49470/0 (socket says 172.21.15.39:49470) 2024-07-04T08:32:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.847+0000 7f518aa00640 1 -- 172.21.15.39:0/990622121 learned_addr learned my addr 172.21.15.39:0/990622121 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.847+0000 7f518a000640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c10ab50 0x7f518c19c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.847+0000 7f518b400640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f518c10b4a0 0x7f518c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:05.836 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.847+0000 7f518aa00640 1 -- 172.21.15.39:0/990622121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c10ab50 msgr2=0x7f518c19c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.847+0000 7f518aa00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c10ab50 0x7f518c19c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.847+0000 7f518aa00640 1 -- 172.21.15.39:0/990622121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f518c10b4a0 msgr2=0x7f518c1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.847+0000 7f518aa00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f518c10b4a0 0x7f518c1a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.847+0000 7f518aa00640 1 -- 172.21.15.39:0/990622121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1a1b10 con 0x7f518c105220 2024-07-04T08:32:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.847+0000 7f518aa00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 0x7f518c19bbe0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f517c005ab0 tx=0x7f517c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.848+0000 7f5190a00640 1 -- 172.21.15.39:0/990622121 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c03f070 con 0x7f518c105220 2024-07-04T08:32:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.848+0000 7f5190a00640 1 -- 172.21.15.39:0/990622121 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517c031b60 con 0x7f518c105220 2024-07-04T08:32:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.848+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1a1da0 con 0x7f518c105220 2024-07-04T08:32:05.837 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.848+0000 7f5190a00640 1 -- 172.21.15.39:0/990622121 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c035030 con 0x7f518c105220 2024-07-04T08:32:05.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.848+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1a8590 con 0x7f518c105220 2024-07-04T08:32:05.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.849+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518c105630 con 0x7f518c105220 2024-07-04T08:32:05.838 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.852+0000 7f5190a00640 1 -- 172.21.15.39:0/990622121 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f517c042070 con 0x7f518c105220 2024-07-04T08:32:05.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.854+0000 7f5190a00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51700b2960 0x7f51700b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:05.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.854+0000 7f5190a00640 1 -- 172.21.15.39:0/990622121 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f517c1038b0 con 0x7f518c105220 2024-07-04T08:32:05.840 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.854+0000 7f518a000640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51700b2960 0x7f51700b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:05.841 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.855+0000 7f518a000640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51700b2960 0x7f51700b4e70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f518c1044d0 tx=0x7f5178005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:05.842 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:05.855+0000 7f5190a00640 1 -- 172.21.15.39:0/990622121 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f517c0caed0 con 0x7f518c105220 2024-07-04T08:32:06.061 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.074+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f518c10d280 con 0x7f518c105220 2024-07-04T08:32:06.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.078+0000 7f5190a00640 1 -- 172.21.15.39:0/990622121 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+25145 (secure 0 0 0) 0x7f517c0ca7d0 con 0x7f518c105220 2024-07-04T08:32:06.066 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:32:06.066 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":112,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:31:18.809110+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:32:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.081+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51700b2960 msgr2=0x7f51700b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.081+0000 7f5191e00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51700b2960 0x7f51700b4e70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f518c1044d0 tx=0x7f5178005f90 comp rx=0 tx=0).stop 2024-07-04T08:32:06.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.081+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 msgr2=0x7f518c19bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.081+0000 7f5191e00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 0x7f518c19bbe0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f517c005ab0 tx=0x7f517c003c10 comp rx=0 tx=0).stop 2024-07-04T08:32:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.082+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 shutdown_connections 2024-07-04T08:32:06.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.082+0000 7f5191e00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f51700b2960 0x7f51700b4e70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.082+0000 7f5191e00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f518c10b4a0 0x7f518c1a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.082+0000 7f5191e00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c10ab50 0x7f518c19c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.082+0000 7f5191e00640 1 --2- 172.21.15.39:0/990622121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f518c105220 0x7f518c19bbe0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.082+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 >> 172.21.15.39:0/990622121 conn(0x7f518c100c20 msgr2=0x7f518c10a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:06.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.082+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 shutdown_connections 2024-07-04T08:32:06.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.082+0000 7f5191e00640 1 -- 172.21.15.39:0/990622121 wait complete. 2024-07-04T08:32:06.098 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-04T08:04:17.509668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-04T08:09:32.643834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-04T08:09:37.246918+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-07-04T08:09:49.616671+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-07-04T08:32:06.098 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-04T08:32:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.410+0000 7f6d7dc00640 1 -- 172.21.15.39:0/1417389211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78102e10 msgr2=0x7f6d78103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.410+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/1417389211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78102e10 0x7f6d78103220 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009900 tx=0x7f6d6802a3f0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.410+0000 7f6d7dc00640 1 -- 172.21.15.39:0/1417389211 shutdown_connections 2024-07-04T08:32:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.410+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/1417389211 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78109090 0x7f6d7810f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.410+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/1417389211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d78108740 0x7f6d78108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.410+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/1417389211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78102e10 0x7f6d78103220 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.410+0000 7f6d7dc00640 1 -- 172.21.15.39:0/1417389211 >> 172.21.15.39:0/1417389211 conn(0x7f6d780fe850 msgr2=0x7f6d78100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.410+0000 7f6d7dc00640 1 -- 172.21.15.39:0/1417389211 shutdown_connections 2024-07-04T08:32:06.397 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.410+0000 7f6d7dc00640 1 -- 172.21.15.39:0/1417389211 wait complete. 2024-07-04T08:32:06.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.412+0000 7f6d7dc00640 1 Processor -- start 2024-07-04T08:32:06.398 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.412+0000 7f6d7dc00640 1 -- start start 2024-07-04T08:32:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.412+0000 7f6d7dc00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d78102e10 0x7f6d78074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7dc00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78108740 0x7f6d780751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78109090 0x7f6d780756e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7dc00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78112d50 con 0x7f6d78108740 2024-07-04T08:32:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7dc00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78112bd0 con 0x7f6d78102e10 2024-07-04T08:32:06.399 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7dc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78112ed0 con 0x7f6d78109090 2024-07-04T08:32:06.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78109090 0x7f6d780756e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:06.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d77400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78108740 0x7f6d780751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:06.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d77e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d78102e10 0x7f6d78074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:06.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78109090 0x7f6d780756e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41224/0 (socket says 172.21.15.39:41224) 2024-07-04T08:32:06.400 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7d200640 1 -- 172.21.15.39:0/4122711201 learned_addr learned my addr 172.21.15.39:0/4122711201 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7d200640 1 -- 172.21.15.39:0/4122711201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d78102e10 msgr2=0x7f6d78074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7d200640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d78102e10 0x7f6d78074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7d200640 1 -- 172.21.15.39:0/4122711201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78108740 msgr2=0x7f6d780751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7d200640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78108740 0x7f6d780751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.413+0000 7f6d7d200640 1 -- 172.21.15.39:0/4122711201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d781a7a60 con 0x7f6d78109090 2024-07-04T08:32:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.414+0000 7f6d7d200640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78109090 0x7f6d780756e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000efd0 tx=0x7f6d7000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.414+0000 7f6d74c00640 1 -- 172.21.15.39:0/4122711201 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d70003b70 con 0x7f6d78109090 2024-07-04T08:32:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.414+0000 7f6d74c00640 1 -- 172.21.15.39:0/4122711201 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d70003d10 con 0x7f6d78109090 2024-07-04T08:32:06.401 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.414+0000 7f6d74c00640 1 -- 172.21.15.39:0/4122711201 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d700108a0 con 0x7f6d78109090 2024-07-04T08:32:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.414+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d781a7d50 con 0x7f6d78109090 2024-07-04T08:32:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.414+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d781a81f0 con 0x7f6d78109090 2024-07-04T08:32:06.402 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.415+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d78104080 con 0x7f6d78109090 2024-07-04T08:32:06.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.418+0000 7f6d74c00640 1 -- 172.21.15.39:0/4122711201 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6d70004b40 con 0x7f6d78109090 2024-07-04T08:32:06.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.419+0000 7f6d74c00640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6d5c0b2990 0x7f6d5c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:06.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.420+0000 7f6d74c00640 1 -- 172.21.15.39:0/4122711201 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7f6d700e2fd0 con 0x7f6d78109090 2024-07-04T08:32:06.406 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.420+0000 7f6d77e00640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6d5c0b2990 0x7f6d5c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:06.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.421+0000 7f6d77e00640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6d5c0b2990 0x7f6d5c0b4ea0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6d68000c00 tx=0x7f6d68031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:06.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.421+0000 7f6d74c00640 1 -- 172.21.15.39:0/4122711201 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f6d700aa5f0 con 0x7f6d78109090 2024-07-04T08:32:06.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:06 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/990622121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:32:06.621 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.635+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d78075f60 con 0x7f6d78109090 2024-07-04T08:32:06.622 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.636+0000 7f6d74c00640 1 -- 172.21.15.39:0/4122711201 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7f6d78075f60 con 0x7f6d78109090 2024-07-04T08:32:06.622 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 1 2024-07-04T08:32:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6d5c0b2990 msgr2=0x7f6d5c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.627 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6d5c0b2990 0x7f6d5c0b4ea0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6d68000c00 tx=0x7f6d68031400 comp rx=0 tx=0).stop 2024-07-04T08:32:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78109090 msgr2=0x7f6d780756e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78109090 0x7f6d780756e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6d7000efd0 tx=0x7f6d7000c6a0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 shutdown_connections 2024-07-04T08:32:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6d5c0b2990 0x7f6d5c0b4ea0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.628 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d78108740 0x7f6d780751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d78109090 0x7f6d780756e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 --2- 172.21.15.39:0/4122711201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6d78102e10 0x7f6d78074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 >> 172.21.15.39:0/4122711201 conn(0x7f6d780fe850 msgr2=0x7f6d78100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.640+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 shutdown_connections 2024-07-04T08:32:06.629 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.641+0000 7f6d7dc00640 1 -- 172.21.15.39:0/4122711201 wait complete. 2024-07-04T08:32:06.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:06 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/990622121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:32:06.658 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-04T08:32:06.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:06 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/990622121' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:32:06.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.965+0000 7fb9dbe00640 1 -- 172.21.15.39:0/2526541765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9dc105050 msgr2=0x7fb9dc105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.952 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.965+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/2526541765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9dc105050 0x7fb9dc105460 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009900 tx=0x7fb9c402a3f0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.966+0000 7fb9dbe00640 1 -- 172.21.15.39:0/2526541765 shutdown_connections 2024-07-04T08:32:06.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.966+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/2526541765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9dc10b2d0 0x7fb9dc111b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.966+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/2526541765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9dc10a980 0x7fb9dc10ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.966+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/2526541765 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9dc105050 0x7fb9dc105460 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.966+0000 7fb9dbe00640 1 -- 172.21.15.39:0/2526541765 >> 172.21.15.39:0/2526541765 conn(0x7fb9dc100ab0 msgr2=0x7fb9dc102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:06.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.966+0000 7fb9dbe00640 1 -- 172.21.15.39:0/2526541765 shutdown_connections 2024-07-04T08:32:06.953 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.966+0000 7fb9dbe00640 1 -- 172.21.15.39:0/2526541765 wait complete. 2024-07-04T08:32:06.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.967+0000 7fb9dbe00640 1 Processor -- start 2024-07-04T08:32:06.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.967+0000 7fb9dbe00640 1 -- start start 2024-07-04T08:32:06.954 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9dbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9dc105050 0x7fb9dc19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9dbe00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9dc10a980 0x7fb9dc19bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9dbe00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9dc10b2d0 0x7fb9dc1a1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9dbe00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc114ff0 con 0x7fb9dc10b2d0 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9dbe00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc114e70 con 0x7fb9dc10a980 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9dbe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc115170 con 0x7fb9dc105050 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9dc105050 0x7fb9dc19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9dc10b2d0 0x7fb9dc1a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9da000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9dc10a980 0x7fb9dc19bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9daa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9dc105050 0x7fb9dc19b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41242/0 (socket says 172.21.15.39:41242) 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9db400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9dc10b2d0 0x7fb9dc1a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49510/0 (socket says 172.21.15.39:49510) 2024-07-04T08:32:06.955 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.968+0000 7fb9daa00640 1 -- 172.21.15.39:0/1041693744 learned_addr learned my addr 172.21.15.39:0/1041693744 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:06.956 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9da000640 1 -- 172.21.15.39:0/1041693744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9dc105050 msgr2=0x7fb9dc19b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9da000640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9dc105050 0x7fb9dc19b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9da000640 1 -- 172.21.15.39:0/1041693744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9dc10b2d0 msgr2=0x7fb9dc1a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9da000640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9dc10b2d0 0x7fb9dc1a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9da000640 1 -- 172.21.15.39:0/1041693744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc1a1900 con 0x7fb9dc10a980 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9da000640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9dc10a980 0x7fb9dc19bf70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc06bd80 tx=0x7fb9cc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9d3400640 1 -- 172.21.15.39:0/1041693744 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc00cc40 con 0x7fb9dc10a980 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9d3400640 1 -- 172.21.15.39:0/1041693744 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9cc00cde0 con 0x7fb9dc10a980 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc1a1bf0 con 0x7fb9dc10a980 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.969+0000 7fb9d3400640 1 -- 172.21.15.39:0/1041693744 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc004a00 con 0x7fb9dc10a980 2024-07-04T08:32:06.957 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.970+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc1a8380 con 0x7fb9dc10a980 2024-07-04T08:32:06.958 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.971+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc105460 con 0x7fb9dc10a980 2024-07-04T08:32:06.959 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.973+0000 7fb9d3400640 1 -- 172.21.15.39:0/1041693744 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb9cc004ba0 con 0x7fb9dc10a980 2024-07-04T08:32:06.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.975+0000 7fb9d3400640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9c00b2960 0x7fb9c00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:06.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.975+0000 7fb9d3400640 1 -- 172.21.15.39:0/1041693744 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9424+0+0 (secure 0 0 0) 0x7fb9cc0e29b0 con 0x7fb9dc10a980 2024-07-04T08:32:06.961 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.975+0000 7fb9daa00640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9c00b2960 0x7fb9c00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:06.962 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.976+0000 7fb9daa00640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9c00b2960 0x7fb9c00b4e70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40096c0 tx=0x7fb9c4005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:06.963 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:06.977+0000 7fb9d3400640 1 -- 172.21.15.39:0/1041693744 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb9cc0aa060 con 0x7fb9dc10a980 2024-07-04T08:32:07.176 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.190+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb9dc02de10 con 0x7fb9dc10a980 2024-07-04T08:32:07.177 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.191+0000 7fb9d3400640 1 -- 172.21.15.39:0/1041693744 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7fb9cc0a9960 con 0x7fb9dc10a980 2024-07-04T08:32:07.177 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-07-04T08:32:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.193+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9c00b2960 msgr2=0x7fb9c00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.193+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9c00b2960 0x7fb9c00b4e70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40096c0 tx=0x7fb9c4005a20 comp rx=0 tx=0).stop 2024-07-04T08:32:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.193+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9dc10a980 msgr2=0x7fb9dc19bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.193+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9dc10a980 0x7fb9dc19bf70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc06bd80 tx=0x7fb9cc00ee50 comp rx=0 tx=0).stop 2024-07-04T08:32:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.194+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 shutdown_connections 2024-07-04T08:32:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.194+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb9c00b2960 0x7fb9c00b4e70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.180 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.194+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9dc10b2d0 0x7fb9dc1a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.194+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb9dc10a980 0x7fb9dc19bf70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.194+0000 7fb9dbe00640 1 --2- 172.21.15.39:0/1041693744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9dc105050 0x7fb9dc19b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.194+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 >> 172.21.15.39:0/1041693744 conn(0x7fb9dc100ab0 msgr2=0x7fb9dc102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.194+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 shutdown_connections 2024-07-04T08:32:07.181 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.194+0000 7fb9dbe00640 1 -- 172.21.15.39:0/1041693744 wait complete. 2024-07-04T08:32:07.207 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-04T08:32:07.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.523+0000 7f282de00640 1 -- 172.21.15.39:0/2758306820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2828101d40 msgr2=0x7f2828102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:07.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.523+0000 7f282de00640 1 --2- 172.21.15.39:0/2758306820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2828101d40 0x7f2828102150 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f281c009900 tx=0x7f281c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.523+0000 7f282de00640 1 -- 172.21.15.39:0/2758306820 shutdown_connections 2024-07-04T08:32:07.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.523+0000 7f282de00640 1 --2- 172.21.15.39:0/2758306820 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2828107670 0x7f2828107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.523+0000 7f282de00640 1 --2- 172.21.15.39:0/2758306820 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2828107fc0 0x7f282810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.523+0000 7f282de00640 1 --2- 172.21.15.39:0/2758306820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2828101d40 0x7f2828102150 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.523+0000 7f282de00640 1 -- 172.21.15.39:0/2758306820 >> 172.21.15.39:0/2758306820 conn(0x7f2828079a30 msgr2=0x7f2828079e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:07.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.523+0000 7f282de00640 1 -- 172.21.15.39:0/2758306820 shutdown_connections 2024-07-04T08:32:07.511 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.523+0000 7f282de00640 1 -- 172.21.15.39:0/2758306820 wait complete. 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.525+0000 7f282de00640 1 Processor -- start 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.525+0000 7f282de00640 1 -- start start 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.525+0000 7f282de00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2828101d40 0x7f2828199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.525+0000 7f282de00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2828107670 0x7f2828199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.525+0000 7f282de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2828107fc0 0x7f282819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.525+0000 7f282de00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2828111c80 con 0x7f2828107670 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.525+0000 7f282de00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2828111b00 con 0x7f2828101d40 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.525+0000 7f282de00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2828111e00 con 0x7f2828107fc0 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f2827e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2828107670 0x7f2828199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:07.512 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f2827e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2828107670 0x7f2828199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49534/0 (socket says 172.21.15.39:49534) 2024-07-04T08:32:07.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f2827e00640 1 -- 172.21.15.39:0/1943957156 learned_addr learned my addr 172.21.15.39:0/1943957156 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:07.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f282ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2828101d40 0x7f2828199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:07.513 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f282d400640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2828107fc0 0x7f282819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f2827e00640 1 -- 172.21.15.39:0/1943957156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2828107fc0 msgr2=0x7f282819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f2827e00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2828107fc0 0x7f282819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f2827e00640 1 -- 172.21.15.39:0/1943957156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2828101d40 msgr2=0x7f2828199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f2827e00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2828101d40 0x7f2828199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f2827e00640 1 -- 172.21.15.39:0/1943957156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282819f740 con 0x7f2828107670 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f282ca00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2828101d40 0x7f2828199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.526+0000 7f2827e00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2828107670 0x7f2828199db0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f281400e970 tx=0x7f281400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.527+0000 7f2825600640 1 -- 172.21.15.39:0/1943957156 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281400cd80 con 0x7f2828107670 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.527+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282819fa30 con 0x7f2828107670 2024-07-04T08:32:07.514 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.527+0000 7f2825600640 1 -- 172.21.15.39:0/1943957156 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28140035f0 con 0x7f2828107670 2024-07-04T08:32:07.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.527+0000 7f2825600640 1 -- 172.21.15.39:0/1943957156 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814010690 con 0x7f2828107670 2024-07-04T08:32:07.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.527+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28281a61c0 con 0x7f2828107670 2024-07-04T08:32:07.515 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.528+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2828102150 con 0x7f2828107670 2024-07-04T08:32:07.517 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.531+0000 7f2825600640 1 -- 172.21.15.39:0/1943957156 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2814003aa0 con 0x7f2828107670 2024-07-04T08:32:07.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.533+0000 7f2825600640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28100b29e0 0x7f28100b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:07.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.533+0000 7f2825600640 1 -- 172.21.15.39:0/1943957156 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9512+0+0 (secure 0 0 0) 0x7f28140e2d40 con 0x7f2828107670 2024-07-04T08:32:07.519 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.533+0000 7f282ca00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28100b29e0 0x7f28100b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.534+0000 7f282ca00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28100b29e0 0x7f28100b4ef0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f281c031440 tx=0x7f281c032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:07.520 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.534+0000 7f2825600640 1 -- 172.21.15.39:0/1943957156 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f28140aa2e0 con 0x7f2828107670 2024-07-04T08:32:07.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:07 smithi039 ceph-mon[32062]: pgmap v1007: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-07-04T08:32:07.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:07 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/4122711201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-04T08:32:07.589 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:07 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1041693744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-04T08:32:07.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:07 smithi050 ceph-mon[36577]: pgmap v1007: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-07-04T08:32:07.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:07 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/4122711201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-04T08:32:07.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:07 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1041693744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-04T08:32:07.763 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.777+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f282802de10 con 0x7f2828107670 2024-07-04T08:32:07.765 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.778+0000 7f2825600640 1 -- 172.21.15.39:0/1943957156 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7f28140a9be0 con 0x7f2828107670 2024-07-04T08:32:07.765 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 64 2024-07-04T08:32:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28100b29e0 msgr2=0x7f28100b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28100b29e0 0x7f28100b4ef0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f281c031440 tx=0x7f281c032ed0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2828107670 msgr2=0x7f2828199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2828107670 0x7f2828199db0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f281400e970 tx=0x7f281400ee30 comp rx=0 tx=0).stop 2024-07-04T08:32:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 shutdown_connections 2024-07-04T08:32:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f28100b29e0 0x7f28100b4ef0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.768 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2828107670 0x7f2828199db0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2828107fc0 0x7f282819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 --2- 172.21.15.39:0/1943957156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2828101d40 0x7f2828199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.781+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 >> 172.21.15.39:0/1943957156 conn(0x7f2828079a30 msgr2=0x7f282806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.782+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 shutdown_connections 2024-07-04T08:32:07.769 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:07.782+0000 7f282de00640 1 -- 172.21.15.39:0/1943957156 wait complete. 2024-07-04T08:32:07.800 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-07-04T08:32:07.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:07 smithi133 ceph-mon[37050]: pgmap v1007: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-07-04T08:32:07.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:07 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/4122711201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-04T08:32:07.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:07 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1041693744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-04T08:32:08.083 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.096+0000 7f5a8ca00640 1 -- 172.21.15.39:0/2413873696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a88109b60 msgr2=0x7f5a88115e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:08.085 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.096+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/2413873696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a88109b60 0x7f5a88115e20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009ca0 tx=0x7f5a8002a8c0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.097+0000 7f5a8ca00640 1 -- 172.21.15.39:0/2413873696 shutdown_connections 2024-07-04T08:32:08.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.097+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/2413873696 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a88109210 0x7f5a88109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.097+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/2413873696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a88109b60 0x7f5a88115e20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.097+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/2413873696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a8810dfc0 0x7f5a8810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.097+0000 7f5a8ca00640 1 -- 172.21.15.39:0/2413873696 >> 172.21.15.39:0/2413873696 conn(0x7f5a88078bd0 msgr2=0x7f5a88107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:08.088 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.097+0000 7f5a8ca00640 1 -- 172.21.15.39:0/2413873696 shutdown_connections 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.097+0000 7f5a8ca00640 1 -- 172.21.15.39:0/2413873696 wait complete. 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.098+0000 7f5a8ca00640 1 Processor -- start 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.098+0000 7f5a8ca00640 1 -- start start 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.098+0000 7f5a8ca00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a88109210 0x7f5a8819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.098+0000 7f5a8ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a88109b60 0x7f5a881a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.098+0000 7f5a8ca00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a8810dfc0 0x7f5a881a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.098+0000 7f5a8ca00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a881193c0 con 0x7f5a88109210 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.098+0000 7f5a8ca00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88119240 con 0x7f5a8810dfc0 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.098+0000 7f5a8ca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88119540 con 0x7f5a88109b60 2024-07-04T08:32:08.089 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a86800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a88109210 0x7f5a8819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a86800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a88109210 0x7f5a8819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:49538/0 (socket says 172.21.15.39:49538) 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a87200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a8810dfc0 0x7f5a881a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a86800640 1 -- 172.21.15.39:0/1893051247 learned_addr learned my addr 172.21.15.39:0/1893051247 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a85e00640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a88109b60 0x7f5a881a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a86800640 1 -- 172.21.15.39:0/1893051247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a88109b60 msgr2=0x7f5a881a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a86800640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a88109b60 0x7f5a881a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a86800640 1 -- 172.21.15.39:0/1893051247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a8810dfc0 msgr2=0x7f5a881a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a86800640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a8810dfc0 0x7f5a881a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.099+0000 7f5a86800640 1 -- 172.21.15.39:0/1893051247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a881a5c40 con 0x7f5a88109210 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.100+0000 7f5a86800640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a88109210 0x7f5a8819fd10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800e9c0 tx=0x7f5a7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.100+0000 7f5a7f400640 1 -- 172.21.15.39:0/1893051247 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7800cd80 con 0x7f5a88109210 2024-07-04T08:32:08.090 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.100+0000 7f5a7f400640 1 -- 172.21.15.39:0/1893051247 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a780035f0 con 0x7f5a88109210 2024-07-04T08:32:08.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.100+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a881a5f30 con 0x7f5a88109210 2024-07-04T08:32:08.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.100+0000 7f5a7f400640 1 -- 172.21.15.39:0/1893051247 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78010690 con 0x7f5a88109210 2024-07-04T08:32:08.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.100+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a881ac6c0 con 0x7f5a88109210 2024-07-04T08:32:08.091 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.102+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a88109620 con 0x7f5a88109210 2024-07-04T08:32:08.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.104+0000 7f5a7f400640 1 -- 172.21.15.39:0/1893051247 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5a780037a0 con 0x7f5a88109210 2024-07-04T08:32:08.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.106+0000 7f5a7f400640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a6c0b2960 0x7f5a6c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:08.092 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.106+0000 7f5a7f400640 1 -- 172.21.15.39:0/1893051247 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9512+0+0 (secure 0 0 0) 0x7f5a780e2e40 con 0x7f5a88109210 2024-07-04T08:32:08.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.106+0000 7f5a85e00640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a6c0b2960 0x7f5a6c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:08.093 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.107+0000 7f5a85e00640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a6c0b2960 0x7f5a6c0b4e70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a740095f0 tx=0x7f5a74005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:08.096 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.110+0000 7f5a7f400640 1 -- 172.21.15.39:0/1893051247 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5a780aa3e0 con 0x7f5a88109210 2024-07-04T08:32:08.315 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.329+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5a8802de10 con 0x7f5a88109210 2024-07-04T08:32:08.316 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.330+0000 7f5a7f400640 1 -- 172.21.15.39:0/1893051247 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v113) ==== 103+0+11 (secure 0 0 0) 0x7f5a780a9ce0 con 0x7f5a88109210 2024-07-04T08:32:08.316 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 32 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.333+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a6c0b2960 msgr2=0x7f5a6c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.333+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a6c0b2960 0x7f5a6c0b4e70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a740095f0 tx=0x7f5a74005f90 comp rx=0 tx=0).stop 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.333+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a88109210 msgr2=0x7f5a8819fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.333+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a88109210 0x7f5a8819fd10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5a7800e9c0 tx=0x7f5a7800ee80 comp rx=0 tx=0).stop 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.334+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 shutdown_connections 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.334+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a6c0b2960 0x7f5a6c0b4e70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.334+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a8810dfc0 0x7f5a881a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.334+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a88109b60 0x7f5a881a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.334+0000 7f5a8ca00640 1 --2- 172.21.15.39:0/1893051247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a88109210 0x7f5a8819fd10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.320 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.334+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 >> 172.21.15.39:0/1893051247 conn(0x7f5a88078bd0 msgr2=0x7f5a8810c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:08.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.334+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 shutdown_connections 2024-07-04T08:32:08.321 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.334+0000 7f5a8ca00640 1 -- 172.21.15.39:0/1893051247 wait complete. 2024-07-04T08:32:08.348 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-07-04T08:32:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.643+0000 7fccf6000640 1 -- 172.21.15.39:0/1335766762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccf0108910 msgr2=0x7fccf0108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.643+0000 7fccf6000640 1 --2- 172.21.15.39:0/1335766762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccf0108910 0x7fccf0108d20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009900 tx=0x7fccd802a410 comp rx=0 tx=0).stop 2024-07-04T08:32:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.643+0000 7fccf6000640 1 -- 172.21.15.39:0/1335766762 shutdown_connections 2024-07-04T08:32:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.643+0000 7fccf6000640 1 --2- 172.21.15.39:0/1335766762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf0109260 0x7fccf010fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.643+0000 7fccf6000640 1 --2- 172.21.15.39:0/1335766762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccf0108910 0x7fccf0108d20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.643+0000 7fccf6000640 1 --2- 172.21.15.39:0/1335766762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccf0102fe0 0x7fccf01033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.643+0000 7fccf6000640 1 -- 172.21.15.39:0/1335766762 >> 172.21.15.39:0/1335766762 conn(0x7fccf00fea00 msgr2=0x7fccf0100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:08.630 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.643+0000 7fccf6000640 1 -- 172.21.15.39:0/1335766762 shutdown_connections 2024-07-04T08:32:08.631 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.643+0000 7fccf6000640 1 -- 172.21.15.39:0/1335766762 wait complete. 2024-07-04T08:32:08.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:08 smithi050 ceph-mon[36577]: osdmap e113: 12 total, 12 up, 12 in 2024-07-04T08:32:08.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:08 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1943957156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-04T08:32:08.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:08 smithi050 ceph-mon[36577]: pgmap v1009: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-04T08:32:08.632 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:08 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1893051247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-04T08:32:08.633 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.645+0000 7fccf6000640 1 Processor -- start 2024-07-04T08:32:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.645+0000 7fccf6000640 1 -- start start 2024-07-04T08:32:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.645+0000 7fccf6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccf0102fe0 0x7fccf019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.645+0000 7fccf6000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccf0108910 0x7fccf019c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fccf6000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf0109260 0x7fccf01a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fccf6000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0112f20 con 0x7fccf0109260 2024-07-04T08:32:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fccf6000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0112da0 con 0x7fccf0108910 2024-07-04T08:32:08.634 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fccf6000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01130a0 con 0x7fccf0102fe0 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fccef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccf0102fe0 0x7fccf019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fccef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccf0102fe0 0x7fccf019bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:41290/0 (socket says 172.21.15.39:41290) 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fccef400640 1 -- 172.21.15.39:0/1344818651 learned_addr learned my addr 172.21.15.39:0/1344818651 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fccefe00640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf0109260 0x7fccf01a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fcceea00640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccf0108910 0x7fccf019c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fcceea00640 1 -- 172.21.15.39:0/1344818651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccf0102fe0 msgr2=0x7fccf019bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fcceea00640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccf0102fe0 0x7fccf019bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fcceea00640 1 -- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf0109260 msgr2=0x7fccf01a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fcceea00640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf0109260 0x7fccf01a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.646+0000 7fcceea00640 1 -- 172.21.15.39:0/1344818651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf01a1a70 con 0x7fccf0108910 2024-07-04T08:32:08.635 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.647+0000 7fcceea00640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccf0108910 0x7fccf019c0e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fccd802a930 tx=0x7fccd8031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.647+0000 7fcce7e00640 1 -- 172.21.15.39:0/1344818651 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd803f070 con 0x7fccf0108910 2024-07-04T08:32:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.647+0000 7fcce7e00640 1 -- 172.21.15.39:0/1344818651 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd8004490 con 0x7fccf0108910 2024-07-04T08:32:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.647+0000 7fcce7e00640 1 -- 172.21.15.39:0/1344818651 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8035070 con 0x7fccf0108910 2024-07-04T08:32:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.647+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf01a1d00 con 0x7fccf0108910 2024-07-04T08:32:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.647+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf01a84f0 con 0x7fccf0108910 2024-07-04T08:32:08.636 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.648+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf0104250 con 0x7fccf0108910 2024-07-04T08:32:08.637 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.651+0000 7fcce7e00640 1 -- 172.21.15.39:0/1344818651 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fccd803b460 con 0x7fccf0108910 2024-07-04T08:32:08.638 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.652+0000 7fcce7e00640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fccd40b2960 0x7fccd40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:08.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.653+0000 7fcce7e00640 1 -- 172.21.15.39:0/1344818651 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9512+0+0 (secure 0 0 0) 0x7fccd8102b80 con 0x7fccf0108910 2024-07-04T08:32:08.639 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.653+0000 7fccef400640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fccd40b2960 0x7fccd40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:08.640 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.654+0000 7fccef400640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fccd40b2960 0x7fccd40b4e70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcce00049d0 tx=0x7fcce0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:08.643 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.657+0000 7fcce7e00640 1 -- 172.21.15.39:0/1344818651 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fccd80c9ad0 con 0x7fccf0108910 2024-07-04T08:32:08.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:08 smithi133 ceph-mon[37050]: osdmap e113: 12 total, 12 up, 12 in 2024-07-04T08:32:08.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:08 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1943957156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-04T08:32:08.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:08 smithi133 ceph-mon[37050]: pgmap v1009: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-04T08:32:08.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:08 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1893051247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-04T08:32:08.837 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:08 smithi039 ceph-mon[32062]: osdmap e113: 12 total, 12 up, 12 in 2024-07-04T08:32:08.837 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:08 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1943957156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-04T08:32:08.837 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:08 smithi039 ceph-mon[32062]: pgmap v1009: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-04T08:32:08.837 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:08 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1893051247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-04T08:32:08.908 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.922+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fccf01a86c0 con 0x7fccf0108910 2024-07-04T08:32:08.918 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.932+0000 7fcce7e00640 1 -- 172.21.15.39:0/1344818651 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+816172 (secure 0 0 0) 0x7fccd80d21a0 con 0x7fccf0108910 2024-07-04T08:32:08.931 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:32:08.931 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:32:08.931 INFO:teuthology.orchestra.run.smithi039.stdout: "epoch": 25, 2024-07-04T08:32:08.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.931 INFO:teuthology.orchestra.run.smithi039.stdout: "active_gid": 14214, 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: "active_name": "x", 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addrs": { 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6800", 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3120140278 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v1", 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:6801", 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3120140278 2024-07-04T08:32:08.932 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: "active_addr": "172.21.15.39:6801/3120140278", 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: "active_change": "2024-07-04T08:01:51.344333+0000", 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: "active_mgr_features": 4540701547738038271, 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: "available": true, 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: "standbys": [ 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 24148, 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "z", 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540701547738038271, 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-07-04T08:32:08.933 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.935 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.936 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.937 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:08.938 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:08.939 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.940 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.942 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-07-04T08:32:08.944 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-07-04T08:32:08.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.946 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:08.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:08.949 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.950 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.951 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.952 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:08.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.955 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-07-04T08:32:08.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-07-04T08:32:08.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.958 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-04T08:32:08.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:08.960 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:08.961 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:08.961 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:08.961 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:08.961 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:08.961 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:08.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.961 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.961 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.962 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:08.963 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.964 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-07-04T08:32:08.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-07-04T08:32:08.966 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-07-04T08:32:08.967 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.968 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.969 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.970 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-04T08:32:08.972 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-04T08:32:08.973 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-07-04T08:32:08.974 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.977 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.979 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:08.980 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-07-04T08:32:08.981 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-04T08:32:08.982 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.983 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.984 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:08.985 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.986 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-07-04T08:32:08.987 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.990 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-04T08:32:08.991 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.992 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.993 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-07-04T08:32:08.994 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.995 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:08.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:08.999 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.000 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba": { 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba", 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB container image", 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.003 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-07-04T08:32:09.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-07-04T08:32:09.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.008 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-07-04T08:32:09.009 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.010 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:09.011 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "grafana_dashboards_path": { 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "grafana_dashboards_path", 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-04T08:32:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-07-04T08:32:09.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-07-04T08:32:09.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.017 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-07-04T08:32:09.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.019 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.020 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.021 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-04T08:32:09.024 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-07-04T08:32:09.025 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.026 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-07-04T08:32:09.029 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-07-04T08:32:09.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-07-04T08:32:09.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-07-04T08:32:09.033 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.035 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-04T08:32:09.037 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.039 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.042 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.044 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.047 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.048 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.054 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.055 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.057 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-04T08:32:09.061 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.064 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.069 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.071 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-04T08:32:09.072 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-04T08:32:09.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.075 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.078 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-04T08:32:09.079 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-04T08:32:09.080 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.081 INFO:teuthology.orchestra.run.smithi039.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "[]", 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.084 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.085 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.086 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.087 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-04T08:32:09.088 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-04T08:32:09.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.092 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.093 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-04T08:32:09.095 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.096 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-04T08:32:09.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-07-04T08:32:09.099 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.101 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.103 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.104 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.109 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.112 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.115 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.117 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.118 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.120 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.121 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-07-04T08:32:09.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.123 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.125 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.126 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.127 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.128 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.130 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.133 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-07-04T08:32:09.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.136 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.137 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.139 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-07-04T08:32:09.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.142 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-04T08:32:09.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.145 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.146 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.147 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-07-04T08:32:09.149 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.150 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.152 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.153 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.156 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.157 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.163 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.165 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.166 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.167 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.168 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.169 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.170 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.172 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.174 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-07-04T08:32:09.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.176 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.177 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.179 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.180 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.181 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.182 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.183 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.184 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.185 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.186 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.187 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.188 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.189 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.190 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.191 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.193 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.194 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.195 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.196 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.197 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.199 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.200 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.201 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.202 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.203 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.204 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.205 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.206 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.207 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.208 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.210 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-07-04T08:32:09.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.212 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.214 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.215 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.216 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.217 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.218 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.219 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.221 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.222 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.223 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.224 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.225 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.226 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.227 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.228 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.234 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.235 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.236 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.239 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.240 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.241 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.243 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.246 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.247 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.248 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.249 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.250 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.251 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.252 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.253 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.255 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.256 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.257 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.258 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.261 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.264 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.265 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.266 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.267 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.268 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.269 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.270 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.271 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-07-04T08:32:09.272 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.273 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.274 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.275 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.276 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.277 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.278 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.279 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.280 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.281 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-07-04T08:32:09.282 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.283 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-07-04T08:32:09.284 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.285 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.286 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.287 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.288 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.288 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.288 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.288 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.288 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.288 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.288 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.289 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.290 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.291 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.293 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.294 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.295 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-07-04T08:32:09.296 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.297 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-07-04T08:32:09.299 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.300 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.301 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.302 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.303 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.304 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.306 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.307 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.308 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.309 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.311 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.313 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.314 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.315 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.316 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.317 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.318 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.319 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.320 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.321 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.322 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.323 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.324 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.325 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.326 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.327 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.328 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.329 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.330 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.331 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.332 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.333 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.334 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.335 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.336 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.337 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.338 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-07-04T08:32:09.339 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.340 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.341 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.342 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.343 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.344 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.345 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.347 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.348 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.349 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.350 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.351 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-07-04T08:32:09.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.353 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.354 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.355 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-07-04T08:32:09.356 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.357 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.358 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-07-04T08:32:09.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.360 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.361 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smb", 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.362 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.363 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.364 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.365 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.366 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.367 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.368 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "update_orchestration": { 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "update_orchestration", 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.369 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.370 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.371 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.372 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.373 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.374 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.375 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.376 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.377 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.379 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.381 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.382 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.384 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.385 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.386 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.387 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.388 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.389 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.390 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.391 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.392 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.393 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.394 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.395 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.397 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.398 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.399 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.400 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.401 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.402 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.403 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.404 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.405 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.407 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.408 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-07-04T08:32:09.409 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-07-04T08:32:09.410 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.411 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.412 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.413 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.414 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.415 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.416 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.417 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.418 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.419 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.420 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.422 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.423 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.424 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.425 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-07-04T08:32:09.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.429 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.430 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.431 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.432 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.434 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.435 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.437 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.438 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.440 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.441 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.442 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.443 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-07-04T08:32:09.444 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.446 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.447 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-07-04T08:32:09.448 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-07-04T08:32:09.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.450 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.452 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.453 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.454 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.455 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.456 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.457 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.459 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.460 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.461 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: "gid": 24161, 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "y", 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: "mgr_features": 4540701547738038271, 2024-07-04T08:32:09.462 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:09.463 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.464 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.465 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.467 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.468 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.469 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.470 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.471 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.472 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.473 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.474 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-07-04T08:32:09.475 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-07-04T08:32:09.476 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.477 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-07-04T08:32:09.478 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.479 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-07-04T08:32:09.480 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.481 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-04T08:32:09.482 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-04T08:32:09.483 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-07-04T08:32:09.484 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.485 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.486 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.487 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-07-04T08:32:09.488 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.489 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.490 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.491 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.492 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.493 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.494 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.495 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.496 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-07-04T08:32:09.497 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.498 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.499 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.500 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.501 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-07-04T08:32:09.502 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-04T08:32:09.504 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-07-04T08:32:09.505 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.506 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.507 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.508 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.509 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.510 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-07-04T08:32:09.511 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.512 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.514 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-04T08:32:09.515 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.516 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-07-04T08:32:09.517 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.518 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-07-04T08:32:09.519 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-07-04T08:32:09.520 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.522 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.523 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.524 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-04T08:32:09.525 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.526 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-07-04T08:32:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-07-04T08:32:09.528 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.530 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.531 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-07-04T08:32:09.532 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba": { 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba", 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.533 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB container image", 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.534 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.535 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.536 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-07-04T08:32:09.537 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.538 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.539 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-07-04T08:32:09.540 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.541 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:09.542 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "grafana_dashboards_path": { 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "grafana_dashboards_path", 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.543 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.544 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-07-04T08:32:09.545 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-07-04T08:32:09.546 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.547 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.548 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.549 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.550 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.551 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.552 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.553 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.554 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.555 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-07-04T08:32:09.556 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-07-04T08:32:09.557 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.558 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.559 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-07-04T08:32:09.560 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-07-04T08:32:09.561 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.562 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.563 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-07-04T08:32:09.564 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.565 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.566 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.567 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-04T08:32:09.568 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.569 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.570 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.571 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.572 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-07-04T08:32:09.573 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.574 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.575 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-04T08:32:09.576 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.577 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.578 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.579 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.580 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.581 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.582 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.583 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.584 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.585 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-07-04T08:32:09.586 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.587 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.588 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.589 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.590 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-07-04T08:32:09.591 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-04T08:32:09.592 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.593 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.594 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.595 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.596 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.597 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.598 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.599 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.600 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.601 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.602 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.603 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.604 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.605 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.606 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-07-04T08:32:09.607 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.608 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.609 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-04T08:32:09.610 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-04T08:32:09.611 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.612 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "[]", 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.613 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.614 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.615 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.616 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.617 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-04T08:32:09.618 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.619 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.620 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.621 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.622 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.623 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.624 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.625 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-04T08:32:09.626 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.627 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.628 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.629 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.630 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.631 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.632 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.633 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.634 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-04T08:32:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-04T08:32:09.636 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.637 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.638 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.639 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.640 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.641 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.642 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-07-04T08:32:09.643 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.644 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.645 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.647 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.648 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.649 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.650 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.651 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.652 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-07-04T08:32:09.653 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.654 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.655 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-07-04T08:32:09.656 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.657 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.658 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.659 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-07-04T08:32:09.660 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.661 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-07-04T08:32:09.662 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.663 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.664 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.665 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.666 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.667 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.668 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.669 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.670 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-04T08:32:09.671 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.672 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.673 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.674 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.675 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:09.676 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.676 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.676 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.676 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-07-04T08:32:09.676 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.676 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.676 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.676 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.676 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.677 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-04T08:32:09.678 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.679 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.680 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.681 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.682 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.683 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.684 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-07-04T08:32:09.685 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.686 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.687 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.688 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.689 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.690 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.691 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-07-04T08:32:09.692 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.693 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.695 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.696 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.697 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.698 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.699 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-07-04T08:32:09.700 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-07-04T08:32:09.701 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.702 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.703 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.704 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.705 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-07-04T08:32:09.706 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.708 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.709 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.710 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.711 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.712 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.713 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.714 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.715 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.716 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.717 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.718 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.719 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.720 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.721 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.722 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.723 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.724 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.725 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.726 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.727 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.728 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.729 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.730 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.731 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.732 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.733 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.734 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.735 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.736 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.737 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.738 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.739 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-04T08:32:09.740 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-07-04T08:32:09.741 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.742 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.744 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-07-04T08:32:09.745 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.746 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.747 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.748 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.749 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.750 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.751 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.752 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.753 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.754 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.755 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.756 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.757 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.758 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.759 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.760 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-07-04T08:32:09.761 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.762 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.763 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.764 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.765 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.766 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.767 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.768 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.769 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.770 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.771 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.772 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.773 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.774 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.776 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.777 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.778 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.779 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.780 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.781 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.782 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.783 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.784 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.785 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.786 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.787 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.788 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.789 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.790 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.791 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.792 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.793 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.794 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.795 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.796 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.797 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.798 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:09.799 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.800 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-07-04T08:32:09.801 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.802 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.803 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.804 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.805 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.806 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.807 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.808 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.809 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.810 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-07-04T08:32:09.811 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.812 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.813 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.814 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.815 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-04T08:32:09.816 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.817 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.818 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.819 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.820 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.821 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.822 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.823 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.824 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.825 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.826 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.827 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.828 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.829 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-07-04T08:32:09.830 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.831 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.832 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.833 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.834 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.835 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.836 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.837 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.838 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:09.839 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.840 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.841 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-07-04T08:32:09.842 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.843 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.844 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.845 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.846 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.847 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.848 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.849 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.850 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.851 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.852 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.853 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.854 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.855 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.856 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.857 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.858 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.859 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.860 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.861 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.862 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.863 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.864 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.865 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.866 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.867 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.868 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.869 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.870 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.872 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.873 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.874 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.875 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.876 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.877 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.878 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.879 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.880 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:09.881 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-07-04T08:32:09.882 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.883 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.884 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.885 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.886 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.887 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.888 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.889 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.890 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smb", 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.891 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.892 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.893 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.894 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.895 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.896 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.897 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "update_orchestration": { 2024-07-04T08:32:09.898 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "update_orchestration", 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.899 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.900 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.901 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.902 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.903 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.904 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.905 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.906 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.907 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.908 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.909 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.910 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.911 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.912 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.913 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.914 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.915 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.917 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.918 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.919 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.920 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.921 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.922 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.923 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-07-04T08:32:09.924 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-07-04T08:32:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-07-04T08:32:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.925 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.926 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.927 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.928 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.929 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.930 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.931 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.932 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.933 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.934 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.935 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.936 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.937 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.938 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.939 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-07-04T08:32:09.940 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-07-04T08:32:09.941 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.942 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.943 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.944 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-07-04T08:32:09.945 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.946 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.947 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-07-04T08:32:09.948 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.949 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.950 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.951 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.952 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.953 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.954 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.955 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.956 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.957 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-07-04T08:32:09.958 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.959 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.960 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.961 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.962 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.963 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.964 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.965 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.966 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.967 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.968 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.969 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.970 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.971 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.972 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.973 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.974 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.975 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.976 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.977 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-07-04T08:32:09.978 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.979 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.980 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.981 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.982 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.983 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.984 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.985 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.986 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.987 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.988 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.989 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.990 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.991 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "modules": [ 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard", 2024-07-04T08:32:09.992 INFO:teuthology.orchestra.run.smithi039.stdout: "iostat", 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "nfs", 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "restful", 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "snap_schedule" 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "available_modules": [ 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "alerts", 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:09.993 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How frequently to reexamine health status", 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.995 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.996 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:09.997 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:09.998 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:09.999 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_destination": { 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_destination", 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email address to send alerts to", 2024-07-04T08:32:10.000 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_from_name": { 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_from_name", 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "Ceph", 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.001 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Email From: name", 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_host": { 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_host", 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.002 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP server", 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_password": { 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_password", 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.003 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Password to authenticate with", 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_port": { 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_port", 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.004 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "465", 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP port", 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_sender": { 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_sender", 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.005 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SMTP envelope sender", 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_ssl": { 2024-07-04T08:32:10.006 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_ssl", 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.007 INFO:teuthology.orchestra.run.smithi039.stdout: "smtp_user": { 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smtp_user", 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "User to authenticate as", 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.008 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.009 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "balancer", 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "active": { 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "active", 2024-07-04T08:32:10.010 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically balance PGs across cluster", 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_time": { 2024-07-04T08:32:10.011 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_time", 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0000", 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "beginning time of day to automatically balance", 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.012 INFO:teuthology.orchestra.run.smithi039.stdout: "begin_weekday": { 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "begin_weekday", 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.013 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_max_iterations": { 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_max_iterations", 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "25", 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "250", 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.014 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_metrics": { 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_metrics", 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "pgs,objects,bytes", 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.015 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "crush_compat_step": { 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crush_compat_step", 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.016 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.5", 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0.001", 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "0.999", 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "aggressiveness of optimization", 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "end_time": { 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_time", 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.017 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2359", 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ending time of day to automatically balance", 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "end_weekday": { 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "end_weekday", 2024-07-04T08:32:10.018 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "0", 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "6", 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.020 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.021 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.022 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.023 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.024 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.025 INFO:teuthology.orchestra.run.smithi039.stdout: "min_score": { 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_score", 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.026 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "upmap", 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "crush-compat", 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "none", 2024-07-04T08:32:10.027 INFO:teuthology.orchestra.run.smithi039.stdout: "read", 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap", 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap-read" 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Balancer mode", 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_ids": { 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_ids", 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.028 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.029 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.030 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_deviation": { 2024-07-04T08:32:10.031 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_deviation", 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "upmap_max_optimizations": { 2024-07-04T08:32:10.032 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "upmap_max_optimizations", 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.033 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm", 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_down_multiplier": { 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_down_multiplier", 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.034 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_refresh_rate": { 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_refresh_rate", 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.035 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "20", 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "agent_starting_port": { 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "agent_starting_port", 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.036 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4721", 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_ptrace": { 2024-07-04T08:32:10.037 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_ptrace", 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.038 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_interval": { 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_interval", 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.039 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "autotune_memory_target_ratio": { 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "autotune_memory_target_ratio", 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0.7", 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-04T08:32:10.040 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm_log_destination": { 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cephadm_log_destination", 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.041 INFO:teuthology.orchestra.run.smithi039.stdout: "file", 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "file,syslog", 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "syslog" 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "cgroups_split": { 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cgroups_split", 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.042 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "config_checks_enabled": { 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_checks_enabled", 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.043 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "config_dashboard": { 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_dashboard", 2024-07-04T08:32:10.044 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_alertmanager": { 2024-07-04T08:32:10.045 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_alertmanager", 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.046 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_base": { 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_base", 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Container image name, without the tag", 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.047 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_elasticsearch": { 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_elasticsearch", 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "elasticsearch container image", 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.048 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_grafana": { 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_grafana", 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:10.049 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_haproxy": { 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_haproxy", 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.050 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "HAproxy container image", 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_agent": { 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_agent", 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.051 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger agent container image", 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_collector": { 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_collector", 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.052 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger collector container image", 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_jaeger_query": { 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_jaeger_query", 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.053 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Jaeger query container image", 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_keepalived": { 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_keepalived", 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Keepalived container image", 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_loki": { 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_loki", 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.055 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Loki container image", 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_node_exporter": { 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_node_exporter", 2024-07-04T08:32:10.056 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_nvmeof": { 2024-07-04T08:32:10.057 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_nvmeof", 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Nvme-of container image", 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.058 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_prometheus": { 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_prometheus", 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Prometheus container image", 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.059 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_promtail": { 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_promtail", 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Promtail container image", 2024-07-04T08:32:10.060 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_samba": { 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_samba", 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Samba/SMB container image", 2024-07-04T08:32:10.061 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "container_image_snmp_gateway": { 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_image_snmp_gateway", 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.062 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "SNMP Gateway container image", 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "container_init": { 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "container_init", 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.063 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Run podman/docker with `--init`", 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "daemon_cache_timeout": { 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "daemon_cache_timeout", 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:10.064 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_cephadm_command_timeout": { 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_cephadm_command_timeout", 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.065 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "900", 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "default_registry": { 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "default_registry", 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.066 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "docker.io", 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "device_cache_timeout": { 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_cache_timeout", 2024-07-04T08:32:10.067 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1800", 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache device inventory", 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.068 INFO:teuthology.orchestra.run.smithi039.stdout: "device_enhanced_scan": { 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_enhanced_scan", 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "facts_cache_timeout": { 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "facts_cache_timeout", 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "seconds to cache host facts data", 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.070 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "grafana_dashboards_path": { 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "grafana_dashboards_path", 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-04T08:32:10.071 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "host_check_interval": { 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "host_check_interval", 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.072 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to perform a host check", 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "hw_monitoring": { 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hw_monitoring", 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.073 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "inventory_list_all": { 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "inventory_list_all", 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.074 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.075 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.076 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "log_refresh_metadata": { 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_refresh_metadata", 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.077 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.078 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.080 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.082 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "*", 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.083 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max_count_per_host": { 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_count_per_host", 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.084 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of daemons per service per host", 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: "max_osd_draining_count": { 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_osd_draining_count", 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.085 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "migration_current": { 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "migration_current", 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.086 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "internal - do not modify", 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "mode": { 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mode", 2024-07-04T08:32:10.087 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "root", 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm-package", 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "root" 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "mode for remote execution of cephadm", 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.088 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "oob_default_addr": { 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "oob_default_addr", 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "169.254.1.1", 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.089 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "prometheus_alerts_path": { 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus_alerts_path", 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.090 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_insecure": { 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_insecure", 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.091 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_password": { 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_password", 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.092 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_url": { 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_url", 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.093 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "registry_username": { 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "registry_username", 2024-07-04T08:32:10.094 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.095 INFO:teuthology.orchestra.run.smithi039.stdout: "secure_monitoring_stack": { 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "secure_monitoring_stack", 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.096 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "service_discovery_port": { 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "service_discovery_port", 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8765", 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "cephadm service discovery port", 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.097 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.098 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "ssh_config_file": { 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssh_config_file", 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-04T08:32:10.099 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "use_agent": { 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_agent", 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.100 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "use_repo_digest": { 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "use_repo_digest", 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.101 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_failed_host_check": { 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_failed_host_check", 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.102 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if the host check fails", 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_daemons": { 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_daemons", 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.103 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_on_stray_hosts": { 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_on_stray_hosts", 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.104 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.105 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crash", 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.106 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.107 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.108 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.109 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.110 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.111 INFO:teuthology.orchestra.run.smithi039.stdout: "retain_interval": { 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retain_interval", 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "31536000", 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.112 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.113 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_recent_interval": { 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_recent_interval", 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1209600", 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.114 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dashboard", 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.115 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_HOST": { 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.116 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-04T08:32:10.117 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_ENABLED": { 2024-07-04T08:32:10.118 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_ENABLED", 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.119 INFO:teuthology.orchestra.run.smithi039.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.120 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "ENABLE_BROWSABLE_API": { 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.121 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.122 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.123 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.124 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.125 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.126 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.127 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.129 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.130 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.131 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_PASSWORD": { 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.132 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.133 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_URL": { 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_URL", 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.134 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_API_USERNAME": { 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.135 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.136 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-04T08:32:10.137 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.138 INFO:teuthology.orchestra.run.smithi039.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.139 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.140 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "[]", 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.141 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "MULTICLUSTER_CONFIG": { 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "{}", 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.142 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_HOST": { 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.143 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.144 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.145 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.146 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-04T08:32:10.148 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.149 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.150 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.151 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.152 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_ENABLED": { 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.153 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.154 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.155 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.156 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8", 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.157 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "45", 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ACCESS_KEY": { 2024-07-04T08:32:10.158 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.159 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "admin", 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.160 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SECRET_KEY": { 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.161 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "RGW_API_SSL_VERIFY": { 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.162 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "UNSAFE_TLS_v1_2": { 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.163 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.164 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:10.165 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.166 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "cross_origin_url": { 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cross_origin_url", 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.167 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "crt_file": { 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "crt_file", 2024-07-04T08:32:10.168 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "debug": { 2024-07-04T08:32:10.169 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "debug", 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Enable/disable debug options", 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.170 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "jwt_token_ttl": { 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "jwt_token_ttl", 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "28800", 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.171 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.172 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.173 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.174 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.175 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.176 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.177 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "motd": { 2024-07-04T08:32:10.178 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "motd", 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "The message of the day", 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.179 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect_resolve_ip_addr": { 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "redirect_resolve_ip_addr", 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.180 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.181 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8080", 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.182 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.183 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.184 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl_server_port": { 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl_server_port", 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8443", 2024-07-04T08:32:10.185 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.186 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "redirect", 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "redirect" 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.187 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.188 INFO:teuthology.orchestra.run.smithi039.stdout: "url_prefix": { 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url_prefix", 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.189 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_monitoring": { 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_monitoring", 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.190 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "monitor device health metrics", 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.191 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.192 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.193 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.194 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.195 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.196 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "mark_out_threshold": { 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mark_out_threshold", 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "2419200", 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.197 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "pool_name": { 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pool_name", 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.198 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "device_health_metrics", 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "retention_period": { 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "retention_period", 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15552000", 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long to retain device health metrics", 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_frequency": { 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_frequency", 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.200 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "self_heal": { 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "self_heal", 2024-07-04T08:32:10.201 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:10.202 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how frequently to wake up and check device health", 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.203 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.204 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "warn_threshold": { 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "warn_threshold", 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7257600", 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.205 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "diskprediction_local", 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.206 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.207 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.208 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.209 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.210 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.211 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "predict_interval": { 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predict_interval", 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "86400", 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.212 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "predictor_model": { 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "predictor_model", 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "prophetstor", 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.213 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "600", 2024-07-04T08:32:10.214 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.215 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "influx", 2024-07-04T08:32:10.216 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": false, 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "influxdb python module not found", 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "batch_size": { 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "batch_size", 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5000", 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.217 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "database": { 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "database", 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "ceph", 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.218 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "hostname": { 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "hostname", 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.219 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server hostname", 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "30", 2024-07-04T08:32:10.220 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "5", 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.221 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.222 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.223 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.224 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.225 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.226 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "password": { 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "password", 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "password of InfluxDB server user", 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.227 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "port": { 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "port", 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "8086", 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "InfluxDB server port", 2024-07-04T08:32:10.228 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.229 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "ssl": { 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ssl", 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "false", 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.230 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "threads": { 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threads", 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.231 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "32", 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "username": { 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "username", 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.232 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "username of InfluxDB server user", 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "verify_ssl": { 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "verify_ssl", 2024-07-04T08:32:10.233 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "true", 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "insights", 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.235 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.236 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.237 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.238 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.239 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.240 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.241 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "iostat", 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.242 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.243 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.244 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.246 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.247 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.248 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "k8sevents", 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_event_retention_days": { 2024-07-04T08:32:10.249 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "ceph_event_retention_days", 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "7", 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.250 INFO:teuthology.orchestra.run.smithi039.stdout: "config_check_secs": { 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "config_check_secs", 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "10", 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.251 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.252 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.253 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.254 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.255 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.256 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.257 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "localpool", 2024-07-04T08:32:10.258 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "failure_domain": { 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "failure_domain", 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "host", 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "failure domain for any created local pool", 2024-07-04T08:32:10.259 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.260 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.261 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.262 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.263 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.264 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "min_size": { 2024-07-04T08:32:10.265 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "min_size", 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default min_size for any created local pool", 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.266 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "num_rep": { 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "num_rep", 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3", 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default replica count for any created local pool", 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.267 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_num": { 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_num", 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "128", 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.268 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "default pg_num for any created local pool", 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "prefix": { 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prefix", 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.269 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "name prefix for any created local pool", 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "subtree": { 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "subtree", 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.271 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "rack", 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.272 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mds_autoscaler", 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.273 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.275 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.276 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.277 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.278 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.279 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirroring", 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.280 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.281 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.282 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.283 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.285 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.286 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "nfs", 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.287 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.288 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.289 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.290 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.291 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.292 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.293 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "fail_fs": { 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "fail_fs", 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.295 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.296 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.297 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.298 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.299 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.300 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator": { 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "orchestrator", 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "cephadm", 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "rook", 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: "test_orchestrator" 2024-07-04T08:32:10.302 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Orchestrator backend", 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.303 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_perf_query", 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.304 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.305 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.306 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.307 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.308 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.309 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.310 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.311 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "osd_support", 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.312 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.313 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.314 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.315 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.316 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.317 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.318 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "pg_autoscaler", 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.319 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.320 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.321 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.322 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.323 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.324 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.325 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.326 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "threshold": { 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "threshold", 2024-07-04T08:32:10.327 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "3.0", 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1.0", 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "scaling threshold", 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.328 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "progress", 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_pg_recovery_event": { 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_pg_recovery_event", 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.329 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.330 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.331 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.332 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.333 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.334 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.335 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.335 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.335 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.335 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.337 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.337 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.337 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.337 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.337 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.337 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.337 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.337 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.338 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "max_completed_events": { 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_completed_events", 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "50", 2024-07-04T08:32:10.339 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "number of past completed events to remember", 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "sleep_interval": { 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sleep_interval", 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.340 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "5", 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "how long the module is going to sleep", 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.341 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.342 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "prometheus", 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "cache": { 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "cache", 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.343 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "exclude_perf_counters": { 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "exclude_perf_counters", 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.344 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.345 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.346 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.347 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.348 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.349 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.350 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools": { 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools", 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.351 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "300", 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.352 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "scrape_interval": { 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "scrape_interval", 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.353 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15.0", 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.354 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "::", 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-07-04T08:32:10.355 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "9283", 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.356 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.357 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "stale_cache_strategy": { 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stale_cache_strategy", 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "log", 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.358 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_behaviour": { 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_behaviour", 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "default", 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.359 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "default", 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "error" 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "standby_error_status_code": { 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "standby_error_status_code", 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.360 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "500", 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "400", 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "599", 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.361 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.362 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.364 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.365 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.366 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.367 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_snap_create": { 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_snap_create", 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10", 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.368 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "mirror_snapshot_schedule": { 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "mirror_snapshot_schedule", 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.369 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.370 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "trash_purge_schedule": { 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "trash_purge_schedule", 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.371 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "restful", 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.372 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "enable_auth": { 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enable_auth", 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.373 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "key_file": { 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "key_file", 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.374 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.375 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.377 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.378 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.379 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "server_addr": { 2024-07-04T08:32:10.380 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_addr", 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.381 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "server_port": { 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "server_port", 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.382 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.383 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rgw", 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.384 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.385 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.386 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.387 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.388 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.389 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.390 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rook", 2024-07-04T08:32:10.391 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.392 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.393 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.394 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.395 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.396 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.397 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "storage_class": { 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "storage_class", 2024-07-04T08:32:10.398 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "local", 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.399 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "selftest", 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.400 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.401 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.402 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.403 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.404 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.405 INFO:teuthology.orchestra.run.smithi039.stdout: "roption1": { 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption1", 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.406 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "roption2": { 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "roption2", 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "xyz", 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption1": { 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption1", 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.408 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption2": { 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption2", 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.409 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption3": { 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption3", 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "float", 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.410 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption4": { 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption4", 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.411 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption5": { 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption5", 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.412 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption6": { 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption6", 2024-07-04T08:32:10.413 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.414 INFO:teuthology.orchestra.run.smithi039.stdout: "rwoption7": { 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rwoption7", 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "1", 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "42", 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.416 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "testkey": { 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testkey", 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.417 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "testlkey": { 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testlkey", 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.418 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "testnewline": { 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "testnewline", 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.419 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "smb", 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.420 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.421 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.422 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.424 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.425 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.426 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "update_orchestration": { 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "update_orchestration", 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.427 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "allow_m_granularity": { 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "allow_m_granularity", 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.429 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "allow minute scheduled snapshots", 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "dump_on_update": { 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "dump_on_update", 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.430 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "dump database to debug log on update", 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.431 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.432 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.433 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.434 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.435 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.436 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.437 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.438 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "stats", 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.439 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.440 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.441 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.442 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.443 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.444 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.445 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.446 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.446 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.446 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "status", 2024-07-04T08:32:10.446 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.446 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.446 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.446 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.446 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.447 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.448 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.449 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.450 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.451 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.452 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telegraf", 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: "address": { 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "address", 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.453 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.454 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "15", 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.455 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.456 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.457 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.458 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.459 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.460 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.461 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "telemetry", 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.462 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_basic": { 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_basic", 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.463 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_crash": { 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_crash", 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.464 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_device": { 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_device", 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.465 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_ident": { 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_ident", 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.466 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "channel_perf": { 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "channel_perf", 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.467 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "contact": { 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "contact", 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.468 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "description": { 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "description", 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.469 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "device_url": { 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "device_url", 2024-07-04T08:32:10.470 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "enabled": { 2024-07-04T08:32:10.471 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "enabled", 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "24", 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "8", 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.473 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "last_opt_revision": { 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "last_opt_revision", 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "1", 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.474 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard": { 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard", 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "leaderboard_description": { 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "leaderboard_description", 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.476 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.477 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.478 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.479 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.480 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.480 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.480 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.480 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.480 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.480 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.480 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.480 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.480 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.481 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.481 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.481 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.481 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.481 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.481 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.481 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.481 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.482 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.482 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.482 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.482 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.482 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.482 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.482 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.483 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.483 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.483 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.483 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.483 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.483 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.483 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "organization": { 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "organization", 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.484 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "proxy": { 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "proxy", 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.485 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.486 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "url": { 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "url", 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.487 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "test_orchestrator", 2024-07-04T08:32:10.488 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.489 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.490 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.491 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.492 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.493 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.494 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-07-04T08:32:10.495 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.496 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.497 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.498 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.499 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.500 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "max_concurrent_clones": { 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "max_concurrent_clones", 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.501 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "4", 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "periodic_async_work": { 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "periodic_async_work", 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.502 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Periodically check for async work", 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_delay": { 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_delay", 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.503 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "snapshot_clone_no_wait": { 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snapshot_clone_no_wait", 2024-07-04T08:32:10.504 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "True", 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.505 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.506 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix", 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "can_run": true, 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "error_string": "", 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "module_options": { 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "discovery_interval": { 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "discovery_interval", 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "uint", 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.507 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "100", 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "identifier": { 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "identifier", 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.508 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "interval": { 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "interval", 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "secs", 2024-07-04T08:32:10.509 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "60", 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "log_level": { 2024-07-04T08:32:10.510 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_level", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.511 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster": { 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster", 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.512 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_cluster_level": { 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_cluster_level", 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "info", 2024-07-04T08:32:10.513 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [ 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "", 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "critical", 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "debug", 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "error", 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "info", 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "warning" 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.514 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "log_to_file": { 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "log_to_file", 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "bool", 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "False", 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.515 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "sqlite3_killpoint": { 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "sqlite3_killpoint", 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "dev", 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 1, 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "0", 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.516 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_host": { 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_host", 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "", 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_port": { 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_port", 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "int", 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "10051", 2024-07-04T08:32:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "zabbix_sender": { 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "zabbix_sender", 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "str", 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "level": "advanced", 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": 0, 2024-07-04T08:32:10.519 INFO:teuthology.orchestra.run.smithi039.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: "min": "", 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: "max": "", 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: "enum_allowed": [], 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: "desc": "", 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: "long_desc": "", 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: "tags": [], 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: "see_also": [] 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: "services": { 2024-07-04T08:32:10.520 INFO:teuthology.orchestra.run.smithi039.stdout: "dashboard": "https://172.21.15.39:8443/" 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "always_on_modules": { 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "octopus": [ 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-07-04T08:32:10.521 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "pacific": [ 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-07-04T08:32:10.522 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "quincy": [ 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.523 INFO:teuthology.orchestra.run.smithi039.stdout: "reef": [ 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "squid": [ 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "balancer", 2024-07-04T08:32:10.524 INFO:teuthology.orchestra.run.smithi039.stdout: "crash", 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "devicehealth", 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "orchestrator", 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "pg_autoscaler", 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "progress", 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "rbd_support", 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "status", 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "telemetry", 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "volumes" 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "last_failure_osd_epoch": 5, 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: "active_clients": [ 2024-07-04T08:32:10.525 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "devicehealth", 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 2623988781 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "libcephsqlite", 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-07-04T08:32:10.526 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 1811411189 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "rbd_support", 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-07-04T08:32:10.527 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 4259770207 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 3657362582 2024-07-04T08:32:10.528 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "snap_schedule", 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 407107249 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:10.529 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: "name": "volumes", 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: "addrvec": [ 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: "type": "v2", 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: "addr": "172.21.15.39:0", 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: "nonce": 757586727 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-07-04T08:32:10.530 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:32:10.531 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:09 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1344818651' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-04T08:32:10.532 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:09 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1344818651' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fccd40b2960 msgr2=0x7fccd40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fccd40b2960 0x7fccd40b4e70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcce00049d0 tx=0x7fcce0009500 comp rx=0 tx=0).stop 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccf0108910 msgr2=0x7fccf019c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccf0108910 0x7fccf019c0e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fccd802a930 tx=0x7fccd8031a20 comp rx=0 tx=0).stop 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 shutdown_connections 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fccd40b2960 0x7fccd40b4e70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccf0109260 0x7fccf01a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fccf0108910 0x7fccf019c0e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 --2- 172.21.15.39:0/1344818651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccf0102fe0 0x7fccf019bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 >> 172.21.15.39:0/1344818651 conn(0x7fccf00fea00 msgr2=0x7fccf0107ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:10.533 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 shutdown_connections 2024-07-04T08:32:10.534 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:08.944+0000 7fccf6000640 1 -- 172.21.15.39:0/1344818651 wait complete. 2024-07-04T08:32:10.540 DEBUG:tasks.check_counter:Getting stats from a 2024-07-04T08:32:10.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.a perf dump 2024-07-04T08:32:10.541 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:09 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1344818651' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-04T08:32:10.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:10 smithi133 ceph-mon[37050]: pgmap v1010: 129 pgs: 129 active+clean; 32 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-04T08:32:10.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:10 smithi039 ceph-mon[32062]: pgmap v1010: 129 pgs: 129 active+clean; 32 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-04T08:32:10.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:10 smithi050 ceph-mon[36577]: pgmap v1010: 129 pgs: 129 active+clean; 32 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-04T08:32:11.025 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:32:13.339 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:32:13.339 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:13.339 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 140, 2024-07-04T08:32:13.339 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 142, 2024-07-04T08:32:13.339 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 39714, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 620600, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 59, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 6, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.081868628, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.018186630, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.049267286, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000336600, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 140, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.069684801, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000497748 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.340 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 208, 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000089772, 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000431 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1292, 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1487, 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1318652, 2024-07-04T08:32:13.341 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 586466, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 8, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 5, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.418014088, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.197721040, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.263935439, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.012400164, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1486, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.169469392, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000114044 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:13.342 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1871, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001008643, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000539 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1084576, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 235376 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 111, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 414, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 38002, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 735119, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 18, 2024-07-04T08:32:13.343 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 4, 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.115139242, 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.074870899, 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.033835415, 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000671686, 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 413, 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.064423826, 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000155989 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 215, 2024-07-04T08:32:13.344 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000148472, 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000690 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 5472, 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 646528 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-07-04T08:32:13.345 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 36, 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.001934906, 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000053747 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2, 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000027417, 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000013708 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:13.346 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:13.347 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 11, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 10, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 10, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 10, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 1, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 24, 2024-07-04T08:32:13.348 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 24, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-07-04T08:32:13.349 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 4, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 234, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 6, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-07-04T08:32:13.350 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-07-04T08:32:13.351 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 1, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:13.352 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 12, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 12, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:13.353 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 3, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 3, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 2, 2024-07-04T08:32:13.354 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 2, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 4195283, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 0, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 2, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.005266606, 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.002633303 2024-07-04T08:32:13.355 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 0 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 13, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 13, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 0, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 11, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 12, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 1, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 10, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 10, 2024-07-04T08:32:13.356 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 0, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 119548, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 223540 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 12, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-07-04T08:32:13.357 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.358 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.359 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.360 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-07-04T08:32:13.361 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.362 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.362 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.362 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.362 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-07-04T08:32:13.362 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.362 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.363 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-07-04T08:32:13.364 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.365 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.366 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.367 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-07-04T08:32:13.368 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.369 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 1, 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 1, 2024-07-04T08:32:13.370 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 0, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 1, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 969, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:13.371 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:13.372 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:13.373 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 281079, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 103, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:13.374 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 9688, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 197, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 50872, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 198, 2024-07-04T08:32:13.375 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 37, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 11191, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 33, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 37, 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-07-04T08:32:13.376 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 37, 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.305993946, 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.008270106 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 37, 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 59 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 37, 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 0, 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 37, 2024-07-04T08:32:13.377 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 0, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 10, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 0, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 2, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 6, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 6, 2024-07-04T08:32:13.378 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 20, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:13.379 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-07-04T08:32:13.380 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 113, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 23, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 10, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 14, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 4, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-07-04T08:32:13.381 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 14, 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 0, 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 1 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 1, 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 24 2024-07-04T08:32:13.382 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-07-04T08:32:13.383 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 1539, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1232065, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 1539, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 1539, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1232065, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.384 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 37, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 11191, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 37, 2024-07-04T08:32:13.385 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 11191, 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-07-04T08:32:13.386 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 37, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 37, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 37, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 37, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.387 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:13.388 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:13.388 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-07-04T08:32:13.388 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:13.388 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-07-04T08:32:13.388 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-07-04T08:32:13.388 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:13.388 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:13.388 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-07-04T08:32:13.388 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x5561814069c0": { 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-07-04T08:32:13.389 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 3, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 979, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 3, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 2, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 979, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:13.390 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:13.391 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:13.391 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:32:13.391 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:13 smithi039 ceph-mon[32062]: pgmap v1011: 129 pgs: 129 active+clean; 32 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-04T08:32:13.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:13 smithi050 ceph-mon[36577]: pgmap v1011: 129 pgs: 129 active+clean; 32 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-04T08:32:13.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:13 smithi133 ceph-mon[37050]: pgmap v1011: 129 pgs: 129 active+clean; 32 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-07-04T08:32:14.392 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-07-04T08:32:14.392 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-07-04T08:32:14.392 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-07-04T08:32:14.392 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-07-04T08:32:14.393 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-07-04T08:32:14.393 DEBUG:tasks.check_counter:Getting stats from d 2024-07-04T08:32:14.393 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.d perf dump 2024-07-04T08:32:14.891 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:32:15.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:14 smithi039 ceph-mon[32062]: osdmap e114: 12 total, 12 up, 12 in 2024-07-04T08:32:15.087 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:14 smithi039 ceph-mon[32062]: pgmap v1013: 129 pgs: 129 active+clean; 32 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 127 B/s wr, 9 op/s 2024-07-04T08:32:15.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:14 smithi050 ceph-mon[36577]: osdmap e114: 12 total, 12 up, 12 in 2024-07-04T08:32:15.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:14 smithi050 ceph-mon[36577]: pgmap v1013: 129 pgs: 129 active+clean; 32 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 127 B/s wr, 9 op/s 2024-07-04T08:32:15.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:14 smithi133 ceph-mon[37050]: osdmap e114: 12 total, 12 up, 12 in 2024-07-04T08:32:15.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:14 smithi133 ceph-mon[37050]: pgmap v1013: 129 pgs: 129 active+clean; 32 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 127 B/s wr, 9 op/s 2024-07-04T08:32:16.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:32:16.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:15 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:32:16.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.253 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.254 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:15 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:32:16.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:16.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:15 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:17.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:16 smithi050 ceph-mon[36577]: pgmap v1014: 129 pgs: 129 active+clean; 32 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 229 B/s wr, 9 op/s 2024-07-04T08:32:17.265 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:16 smithi039 ceph-mon[32062]: pgmap v1014: 129 pgs: 129 active+clean; 32 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 229 B/s wr, 9 op/s 2024-07-04T08:32:17.269 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:32:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1275, 2024-07-04T08:32:17.269 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1277, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 336136, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 405514, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 7, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.318982395, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.144377658, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.069977680, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.072091939, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1275, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.109773167, 2024-07-04T08:32:17.270 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000086096 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1573, 2024-07-04T08:32:17.271 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1565, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1345262, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 605389, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.469397905, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.222925844, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.188196512, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.076360230, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1564, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.140634491, 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000089919 2024-07-04T08:32:17.272 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1058896, 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 235232 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 1273, 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 1554, 2024-07-04T08:32:17.273 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 429152, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 953960, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 3, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.455491215, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.262574822, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.078872836, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.077469111, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 1553, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.167226026, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000107679 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.274 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 550736 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 1, 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-MDSRank": { 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 6143, 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.353113554, 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000057482 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "queue_len": 0, 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "complete_latency": { 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.276 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "mds": { 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "request": 0, 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "reply": 0, 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "reply_latency": { 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_reply": 0, 2024-07-04T08:32:17.277 INFO:teuthology.orchestra.run.smithi039.stdout: "forward": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_commit": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_split": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_merge": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_top": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_bottom": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_pinned": 0, 2024-07-04T08:32:17.278 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_expired": 0, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "inodes_with_caps": 0, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "caps": 0, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "subtrees": 1, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse": 0, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_hit": 0, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_forward": 0, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_discover": 0, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:17.279 INFO:teuthology.orchestra.run.smithi039.stdout: "traverse_lock": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "load_cent": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "q": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "exported": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "exported_inodes": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "imported": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "imported_inodes": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_backtrace_fetch": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "openino_peer_discover": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rfiles": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rbytes": 0, 2024-07-04T08:32:17.280 INFO:teuthology.orchestra.run.smithi039.stdout: "root_rsnaps": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_fetch": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_inodes": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dir_base_inodes": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_dirfrag_rstats": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "scrub_file_inodes": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_revoke": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_grant": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_trunc": 0, 2024-07-04T08:32:17.281 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_caps_dirty": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_cap_release": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "process_request_cap_release": 0 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_cache": { 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_try_discover": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_send_discover": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "dir_handle_discover": 0, 2024-07-04T08:32:17.282 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_created": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_enqueued": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "strays_migrated": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_started": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "recovery_completed": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_path": 0, 2024-07-04T08:32:17.283 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_quiesce_inode": 0, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_enqueue_scrub": 0, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_flush": 0, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "ireq_inodestats": 0 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_log": { 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "evlrg": 0, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "evadd": 0, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "evex": 0, 2024-07-04T08:32:17.284 INFO:teuthology.orchestra.run.smithi039.stdout: "evtrm": 0, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "ev": 5, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "evexg": 0, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "evexd": 0, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "segadd": 0, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "segex": 0, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "segtrm": 0, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "seg": 2, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "segmjr": 2, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "segexg": 0, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "segexd": 0, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "expos": 4194304, 2024-07-04T08:32:17.285 INFO:teuthology.orchestra.run.smithi039.stdout: "wrpos": 4195815, 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "rdpos": 4195424, 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "jlat": { 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "replayed": 5 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_mem": { 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "ino": 3, 2024-07-04T08:32:17.286 INFO:teuthology.orchestra.run.smithi039.stdout: "ino+": 3, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "ino-": 0, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "dir": 1, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "dir+": 1, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "dir-": 0, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "dn": 0, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "dn+": 0, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "dn-": 0, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "cap": 0, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "cap+": 0, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "cap-": 0, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "rss": 115188, 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: "heap": 215348 2024-07-04T08:32:17.287 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_server": { 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_client_request": 0, 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_request": 0, 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_client_session": 0, 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "handle_peer_request": 0, 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "req_create_latency": { 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getattr_latency": { 2024-07-04T08:32:17.288 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.289 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "req_link_latency": { 2024-07-04T08:32:17.291 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookup_latency": { 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.292 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupino_latency": { 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupname_latency": { 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:17.293 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "req_lssnap_latency": { 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.294 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mkdir_latency": { 2024-07-04T08:32:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.295 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mknod_latency": { 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "req_mksnap_latency": { 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.296 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "req_open_latency": { 2024-07-04T08:32:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.297 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "req_readdir_latency": { 2024-07-04T08:32:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.298 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rename_latency": { 2024-07-04T08:32:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.299 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.300 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmdir_latency": { 2024-07-04T08:32:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.301 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.302 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setattr_latency": { 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.303 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setlayout_latency": { 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "req_setxattr_latency": { 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "req_symlink_latency": { 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.304 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "req_unlink_latency": { 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.305 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_sessions": { 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "session_count": 1, 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "session_add": 2, 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "session_remove": 1, 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_open": 1, 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "sessions_stale": 0, 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "total_load": 0, 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "average_load": 0, 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "avg_session_uptime": 2578, 2024-07-04T08:32:17.306 INFO:teuthology.orchestra.run.smithi039.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:17.307 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:17.308 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:17.309 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 191982, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 55, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 9448, 2024-07-04T08:32:17.310 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 191, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 8512, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 13, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "objecter": { 2024-07-04T08:32:17.311 INFO:teuthology.orchestra.run.smithi039.stdout: "op_active": 0, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "op_laggy": 0, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send": 3707, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "op_send_bytes": 21, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "op_resend": 3, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "op_reply": 3707, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "op_latency": { 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3707, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3.823924709, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.001031541 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "op_inflight": 0, 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "oplen_avg": { 2024-07-04T08:32:17.312 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 3707, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 3708 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "op": 3707, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "op_r": 3707, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "op_w": 0, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "op_rmw": 0, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "op_pg": 0, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_stat": 2430, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_create": 0, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_read": 1275, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_write": 0, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writefull": 0, 2024-07-04T08:32:17.313 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_writesame": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_append": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_zero": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_truncate": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_delete": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_mapext": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_getxattr": 1, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_setxattr": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:17.314 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_call": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_watch": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_notify": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "osdop_other": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_active": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_send": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_resend": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "linger_ping": 0, 2024-07-04T08:32:17.315 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_active": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_send": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "poolop_resend": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_active": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_send": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "poolstat_resend": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_active": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_send": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "statfs_resend": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "command_active": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "command_send": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "command_resend": 0, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "map_epoch": 114, 2024-07-04T08:32:17.316 INFO:teuthology.orchestra.run.smithi039.stdout: "map_full": 0, 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "map_inc": 24, 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_sessions": 5, 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_open": 6, 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_session_close": 1, 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_laggy": 0, 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_wr": 0, 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_rd": 2, 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_del": 0 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "oft": { 2024-07-04T08:32:17.317 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_objs": 0, 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_updates": 0, 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "omap_total_removes": 0 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "osd-slow-ops": { 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "slow_ops_count": 0 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "purge_queue": { 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_ops_high_water": 0, 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing": 0, 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executing_high_water": 0, 2024-07-04T08:32:17.318 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed_ops": 0, 2024-07-04T08:32:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_executed": 0, 2024-07-04T08:32:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-07-04T08:32:17.319 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 4117, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 1716112, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 4117, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 4117, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1716112, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.320 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3707, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 1516, 2024-07-04T08:32:17.321 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3707, 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 1516, 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 1024, 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-07-04T08:32:17.322 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 3707, 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 3707, 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 3707, 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 3707, 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:17.323 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-07-04T08:32:17.324 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-write_buf_throttle-0x55e2116c89c0": { 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:17.325 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 3758096384, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 0, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 0, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 0, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 0, 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:17.326 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:17.327 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:17.327 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:32:17.327 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:16 smithi133 ceph-mon[37050]: pgmap v1014: 129 pgs: 129 active+clean; 32 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 229 B/s wr, 9 op/s 2024-07-04T08:32:18.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:18 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.631 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:18 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.653 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-07-04T08:32:18.653 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-07-04T08:32:18.653 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-07-04T08:32:18.653 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-07-04T08:32:18.653 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-07-04T08:32:18.653 DEBUG:tasks.check_counter:Getting stats from g 2024-07-04T08:32:18.654 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.g perf dump 2024-07-04T08:32:18.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:18.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:18 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:19.147 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:32:19.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:19 smithi039 ceph-mon[32062]: pgmap v1015: 129 pgs: 129 active+clean; 32 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:32:19.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:19 smithi050 ceph-mon[36577]: pgmap v1015: 129 pgs: 129 active+clean; 32 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:32:19.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:19 smithi133 ceph-mon[37050]: pgmap v1015: 129 pgs: 129 active+clean; 32 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:32:21.464 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:21 smithi039 ceph-mon[32062]: pgmap v1016: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:32:21.465 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:21 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:21.465 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:21 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 374, 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 368, 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1006736, 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 227616, 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.149998684, 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.076505422, 2024-07-04T08:32:21.467 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.094229271, 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 367, 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.038632243, 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000105264 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1006736, 2024-07-04T08:32:21.468 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 227616 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 6, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 284, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 3264, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 208320, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.066672421, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.053808356, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.006568744, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:21.469 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 283, 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.040703086, 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000143827 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 208320 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 0, 2024-07-04T08:32:21.470 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 0, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 0, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 2, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.001176246, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000000000, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.000881403, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:21.471 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.472 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:21.473 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:21.474 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:21.475 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 149735, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 30, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2024-07-04T08:32:21.476 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 928, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:21.477 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 376, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 953864, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 376, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 376, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 953864, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:21.478 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:21.479 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:21.479 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:21.479 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:21.479 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:32:21.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:21 smithi050 ceph-mon[36577]: pgmap v1016: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:32:21.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:21 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:21.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:21 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:21.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:21 smithi133 ceph-mon[37050]: pgmap v1016: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-07-04T08:32:21.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:21 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:21.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:21 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:22.191 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-07-04T08:32:22.191 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-07-04T08:32:22.191 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-07-04T08:32:22.191 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-07-04T08:32:22.191 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-07-04T08:32:22.192 DEBUG:tasks.check_counter:Getting stats from j 2024-07-04T08:32:22.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.j perf dump 2024-07-04T08:32:22.666 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:32:23.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:23.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:23 smithi133 ceph-mon[37050]: pgmap v1017: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-04T08:32:23.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:23.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:32:23.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:32:23.552 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:23.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:23.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:23 smithi039 ceph-mon[32062]: pgmap v1017: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-04T08:32:23.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:23.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:32:23.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:32:23.588 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:23.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:23.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:23 smithi050 ceph-mon[36577]: pgmap v1017: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-07-04T08:32:23.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:23.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:32:23.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:32:23.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:32:25.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:24 smithi039 ceph-mon[32062]: pgmap v1018: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 10 op/s 2024-07-04T08:32:25.093 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-07-04T08:32:25.093 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:25.093 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 385, 2024-07-04T08:32:25.093 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 377, 2024-07-04T08:32:25.093 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 1011584, 2024-07-04T08:32:25.093 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 232544, 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 0, 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.165454912, 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.078982307, 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.116249819, 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 377, 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.050356068, 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000133570 2024-07-04T08:32:25.094 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 1011584, 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 232544 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 2, 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 287, 2024-07-04T08:32:25.095 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 320, 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 208752, 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 2, 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.064623862, 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.053823076, 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.004191781, 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 285, 2024-07-04T08:32:25.096 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.031148586, 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000109293 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 208752 2024-07-04T08:32:25.097 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.098 INFO:teuthology.orchestra.run.smithi039.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:25.098 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_messages": 0, 2024-07-04T08:32:25.098 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages": 2, 2024-07-04T08:32:25.098 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_bytes": 0, 2024-07-04T08:32:25.098 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_bytes": 0, 2024-07-04T08:32:25.098 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_created_connections": 2, 2024-07-04T08:32:25.098 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_active_connections": 1, 2024-07-04T08:32:25.098 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_total_time": 0.001085270, 2024-07-04T08:32:25.099 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_send_time": 0.000000000, 2024-07-04T08:32:25.099 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_recv_time": 0.000852449, 2024-07-04T08:32:25.099 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-04T08:32:25.099 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:25.099 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:25.099 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:25.099 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:25.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.100 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:25.100 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:25.100 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:25.100 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:25.100 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-04T08:32:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "msgr_send_encrypted_bytes": 0 2024-07-04T08:32:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "cct": { 2024-07-04T08:32:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "total_workers": 0, 2024-07-04T08:32:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: "unhealthy_workers": 0 2024-07-04T08:32:25.101 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "mempool": { 2024-07-04T08:32:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:25.102 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:25.103 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:25.104 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:25.105 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:25.106 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_items": 0, 2024-07-04T08:32:25.107 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_bytes": 148704, 2024-07-04T08:32:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_anon_items": 29, 2024-07-04T08:32:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:25.108 INFO:teuthology.orchestra.run.smithi039.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_bytes": 0, 2024-07-04T08:32:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_items": 0, 2024-07-04T08:32:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:25.109 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_bytes": 0, 2024-07-04T08:32:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_items": 0, 2024-07-04T08:32:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "pgmap_items": 0, 2024-07-04T08:32:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_bytes": 928, 2024-07-04T08:32:25.110 INFO:teuthology.orchestra.run.smithi039.stdout: "mds_co_items": 2, 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_1_items": 0, 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "unittest_2_items": 0 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "val": 0, 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 104857600, 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "get_started": 0, 2024-07-04T08:32:25.111 INFO:teuthology.orchestra.run.smithi039.stdout: "get": 383, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "get_sum": 954774, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "get_or_fail_success": 383, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "take": 0, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "take_sum": 0, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "put": 383, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "put_sum": 954774, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "wait": { 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "avgcount": 0, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "sum": 0.000000000, 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: "avgtime": 0.000000000 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:25.112 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-07-04T08:32:25.113 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-07-04T08:32:25.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:24 smithi050 ceph-mon[36577]: pgmap v1018: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 10 op/s 2024-07-04T08:32:25.301 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:24 smithi133 ceph-mon[37050]: pgmap v1018: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 98 B/s wr, 10 op/s 2024-07-04T08:32:25.971 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-07-04T08:32:25.972 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-07-04T08:32:25.972 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-07-04T08:32:25.972 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-07-04T08:32:25.972 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-07-04T08:32:25.972 DEBUG:tasks.check_counter:Getting stats from b 2024-07-04T08:32:25.972 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.b perf dump 2024-07-04T08:32:26.247 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:32:27.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:27 smithi039 ceph-mon[32062]: pgmap v1019: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:32:27.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:27 smithi050 ceph-mon[36577]: pgmap v1019: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:32:27.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:27 smithi133 ceph-mon[37050]: pgmap v1019: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-07-04T08:32:28.068 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-07-04T08:32:28.068 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:28.068 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 4324, 2024-07-04T08:32:28.068 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 3888, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1248227, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 16132617, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 206, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 4, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.551703601, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.219096252, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.271634273, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.062554453, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3893, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.936180064, 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000240477 2024-07-04T08:32:28.069 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3604, 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000818715, 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000227 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 448 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2242, 2024-07-04T08:32:28.070 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2369, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 648039, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 16659151, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 9, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 6, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.292515489, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.148915168, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.113210209, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.039472681, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2370, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.134137420, 2024-07-04T08:32:28.071 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000478538 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1853, 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000527896, 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000284 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 585296 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 5906, 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 6127, 2024-07-04T08:32:28.072 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 3226300, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 6549444, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 26, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 10, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.865378999, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.364812998, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.399251543, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.151444571, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 6117, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.249835652, 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000040842 2024-07-04T08:32:28.073 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 6522, 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.001979270, 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000303 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1698896, 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 469888 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-07-04T08:32:28.074 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 5569, 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.672151160, 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000300260 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-07-04T08:32:28.075 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 16, 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000433709, 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000027106 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 876, 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 876, 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 876, 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 4.230419917, 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.004829246 2024-07-04T08:32:28.076 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 0, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 12, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 0, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 0, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 45, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 0, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 45, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 44, 2024-07-04T08:32:28.077 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 0, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 3, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 3, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 2, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 985, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 977, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 0, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 0, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 0, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-07-04T08:32:28.078 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 0, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 0, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 0, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 0, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 315, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 4, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 234, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 6, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 315, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:28.079 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 203, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 23, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 180, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 112, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 663, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 656, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 599, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 777, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 718, 2024-07-04T08:32:28.080 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 2, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 628 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 0, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 0, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 0, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 5, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 2, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 3, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 1, 2024-07-04T08:32:28.081 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_path": 12, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_inode": 166, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 23, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-07-04T08:32:28.082 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 0, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 2460, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 0, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 0, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 2460, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 0, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 9, 2024-07-04T08:32:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 0, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 0, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 9, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 2, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 0, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 4194304, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 36420568, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 0, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2373, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 6.764826093, 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002850748 2024-07-04T08:32:28.084 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 0 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 40, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 64, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 24, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 20, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 20, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 0, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 45, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 46, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 1, 2024-07-04T08:32:28.085 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 3, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 19, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 16, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 125188, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 231732 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 2148, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 876, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 36, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 0, 2024-07-04T08:32:28.086 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 16, 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.008793153, 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000549572 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 186, 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.039723022, 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000213564 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.087 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3, 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.001775052, 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000591684 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:28.088 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1, 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000174364, 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000174364 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.089 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-07-04T08:32:28.090 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 17, 2024-07-04T08:32:28.091 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.004068018, 2024-07-04T08:32:28.091 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000239295 2024-07-04T08:32:28.091 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.091 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-07-04T08:32:28.091 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 8, 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.024312091, 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.003039011 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 17, 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.111042334, 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.006531902 2024-07-04T08:32:28.092 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1, 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000222354, 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000222354 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2, 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.004424086, 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.002212043 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2, 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.002213981, 2024-07-04T08:32:28.093 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.001106990 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.094 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2, 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.011596553, 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.005798276 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 11, 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.068127955, 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.006193450 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.095 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 601, 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 3.924567444, 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.006530062 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:28.096 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 8, 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.024392558, 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.003049069 2024-07-04T08:32:28.097 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1, 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.004986952, 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.004986952 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:28.098 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 2, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 4, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 2, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 2, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 72, 2024-07-04T08:32:28.099 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 36, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 1020, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:28.100 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:28.101 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:28.102 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 311716, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 110, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 9448, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 191, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 214859, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 1406, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.104 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 3981, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 32249364, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 31, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 3981, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3981, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 34.444083276, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.008652118 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-07-04T08:32:28.105 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3981, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 4028 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 3981, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 362, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 3619, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 0, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 11, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 24, 2024-07-04T08:32:28.106 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 2374, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 73, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 0, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 521, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 338, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 22, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:28.107 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 599, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-07-04T08:32:28.108 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 114, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-07-04T08:32:28.109 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 33, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 12, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 12, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 0, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 56, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 0, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 10 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 1, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-07-04T08:32:28.110 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 178 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 2, 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 1, 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 2, 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 1, 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:28.111 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 12468, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 3949090, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 12468, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 12468, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 3949090, 2024-07-04T08:32:28.112 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:28.113 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:28.114 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 3981, 2024-07-04T08:32:28.114 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 32249521, 2024-07-04T08:32:28.114 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 3981, 2024-07-04T08:32:28.114 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 32249521, 2024-07-04T08:32:28.114 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:28.118 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.118 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.118 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.118 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:28.118 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.118 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:28.118 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:28.118 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-07-04T08:32:28.118 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 3981, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 3981, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 3981, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 3981, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:28.119 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 1, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 157, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 1, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 1, 2024-07-04T08:32:28.120 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 157, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x556ad2324680": { 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 2460, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 32226264, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:28.121 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 2460, 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 2373, 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 32226264, 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:28.122 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-07-04T08:32:28.876 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-07-04T08:32:28.876 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-07-04T08:32:28.876 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-07-04T08:32:28.876 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=17 2024-07-04T08:32:28.876 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=11 2024-07-04T08:32:28.876 DEBUG:tasks.check_counter:Getting stats from e 2024-07-04T08:32:28.877 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.e perf dump 2024-07-04T08:32:29.146 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:32:29.407 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:29 smithi050 ceph-mon[36577]: pgmap v1020: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-04T08:32:29.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:29 smithi133 ceph-mon[37050]: pgmap v1020: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-04T08:32:29.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:29 smithi039 ceph-mon[32062]: pgmap v1020: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-04T08:32:30.551 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:30 smithi050 ceph-mon[36577]: pgmap v1021: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:32:30.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:30 smithi133 ceph-mon[37050]: pgmap v1021: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:32:30.834 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-07-04T08:32:30.834 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:30.834 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 3559, 2024-07-04T08:32:30.834 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 3561, 2024-07-04T08:32:30.834 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1013114, 2024-07-04T08:32:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 1131769, 2024-07-04T08:32:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 6, 2024-07-04T08:32:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-07-04T08:32:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.364748090, 2024-07-04T08:32:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.165688326, 2024-07-04T08:32:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.075323081, 2024-07-04T08:32:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.098448653, 2024-07-04T08:32:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:30.835 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3559, 2024-07-04T08:32:30.836 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.143270716, 2024-07-04T08:32:30.836 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000040255 2024-07-04T08:32:30.836 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.836 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:30.836 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.836 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.836 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.836 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.836 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-04T08:32:30.837 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-07-04T08:32:30.837 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.837 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:30.837 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 8, 2024-07-04T08:32:30.837 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 291, 2024-07-04T08:32:30.837 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 3820, 2024-07-04T08:32:30.837 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 723066, 2024-07-04T08:32:30.837 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 7, 2024-07-04T08:32:30.837 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 3, 2024-07-04T08:32:30.838 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.026205341, 2024-07-04T08:32:30.838 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.018323829, 2024-07-04T08:32:30.838 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.004536022, 2024-07-04T08:32:30.838 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000064796, 2024-07-04T08:32:30.838 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:30.838 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 290, 2024-07-04T08:32:30.838 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.025335700, 2024-07-04T08:32:30.838 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000087364 2024-07-04T08:32:30.839 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.839 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:30.839 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.839 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.839 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.839 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.839 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-07-04T08:32:30.839 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 722368 2024-07-04T08:32:30.839 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 412, 2024-07-04T08:32:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 404, 2024-07-04T08:32:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1059708, 2024-07-04T08:32:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 236830, 2024-07-04T08:32:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-07-04T08:32:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.076278887, 2024-07-04T08:32:30.840 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.037908145, 2024-07-04T08:32:30.841 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.048490024, 2024-07-04T08:32:30.841 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000029327, 2024-07-04T08:32:30.841 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:30.841 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 403, 2024-07-04T08:32:30.841 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.026693015, 2024-07-04T08:32:30.841 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000066235 2024-07-04T08:32:30.841 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.841 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1059440, 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 236512 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-07-04T08:32:30.842 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-07-04T08:32:30.843 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.843 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-07-04T08:32:30.843 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-07-04T08:32:30.843 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-07-04T08:32:30.843 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 6124, 2024-07-04T08:32:30.843 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.163406574, 2024-07-04T08:32:30.843 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000026682 2024-07-04T08:32:30.843 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.844 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 0, 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 0, 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 0, 2024-07-04T08:32:30.845 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:30.846 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:30.846 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 0, 2024-07-04T08:32:30.846 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 0, 2024-07-04T08:32:30.846 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 0, 2024-07-04T08:32:30.846 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 0, 2024-07-04T08:32:30.846 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 0, 2024-07-04T08:32:30.846 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 0, 2024-07-04T08:32:30.846 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:30.847 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 0, 2024-07-04T08:32:30.847 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 0, 2024-07-04T08:32:30.847 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 0, 2024-07-04T08:32:30.847 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 0, 2024-07-04T08:32:30.847 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 1, 2024-07-04T08:32:30.847 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 0, 2024-07-04T08:32:30.847 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 0, 2024-07-04T08:32:30.847 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 0, 2024-07-04T08:32:30.847 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:30.848 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 0, 2024-07-04T08:32:30.849 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-07-04T08:32:30.849 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 0, 2024-07-04T08:32:30.849 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 0, 2024-07-04T08:32:30.849 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 0, 2024-07-04T08:32:30.849 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 0, 2024-07-04T08:32:30.849 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:30.849 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:30.850 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:30.850 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 0, 2024-07-04T08:32:30.850 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 0, 2024-07-04T08:32:30.850 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 0, 2024-07-04T08:32:30.850 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 0, 2024-07-04T08:32:30.850 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:30.850 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 0, 2024-07-04T08:32:30.850 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 0, 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 0, 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 0, 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 0, 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 0, 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 0 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 0, 2024-07-04T08:32:30.851 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:30.852 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_path": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_inode": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:30.853 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 0, 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 0, 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 0, 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 0, 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 3, 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 0, 2024-07-04T08:32:30.854 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 0, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 0, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 0, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 2, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 2, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 0, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 4194304, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 4195283, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 4194892, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.855 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 3 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 3, 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 3, 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 0, 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 1, 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 1, 2024-07-04T08:32:30.856 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 0, 2024-07-04T08:32:30.857 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 0, 2024-07-04T08:32:30.857 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 0, 2024-07-04T08:32:30.857 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 0, 2024-07-04T08:32:30.857 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 0, 2024-07-04T08:32:30.857 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 0, 2024-07-04T08:32:30.857 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 0, 2024-07-04T08:32:30.857 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 117144, 2024-07-04T08:32:30.857 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 215348 2024-07-04T08:32:30.857 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.858 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-07-04T08:32:30.858 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 0, 2024-07-04T08:32:30.858 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:30.858 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 0, 2024-07-04T08:32:30.858 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 0, 2024-07-04T08:32:30.858 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 0, 2024-07-04T08:32:30.858 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-07-04T08:32:30.858 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:30.859 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.860 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.861 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.862 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.863 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.863 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:30.863 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.863 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.863 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.863 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.863 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-07-04T08:32:30.863 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.864 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.864 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.864 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.864 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-07-04T08:32:30.864 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.865 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:30 smithi039 ceph-mon[32062]: pgmap v1021: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:32:30.865 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.865 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.865 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.865 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-07-04T08:32:30.866 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.867 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.867 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.867 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.867 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-07-04T08:32:30.867 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.867 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.867 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.867 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.868 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.869 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:30.870 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.870 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.870 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.870 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.870 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:30.870 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.870 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.870 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.871 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.871 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-07-04T08:32:30.871 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.871 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.871 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.871 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.871 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.872 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-07-04T08:32:30.873 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.873 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.873 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.873 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.873 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-07-04T08:32:30.873 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.873 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.873 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.873 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.874 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-07-04T08:32:30.875 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 1, 2024-07-04T08:32:30.876 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 1, 2024-07-04T08:32:30.876 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 0, 2024-07-04T08:32:30.876 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 1, 2024-07-04T08:32:30.876 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-07-04T08:32:30.876 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 0, 2024-07-04T08:32:30.876 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 0, 2024-07-04T08:32:30.876 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 2920, 2024-07-04T08:32:30.876 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:30.876 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.877 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-07-04T08:32:30.877 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:30.877 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:30.877 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:30.877 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:30.877 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:30.877 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:30.877 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:30.877 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:30.878 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:30.878 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:30.878 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:30.878 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:30.878 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:30.878 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:30.878 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:30.878 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:30.879 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:30.879 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:30.879 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:30.879 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:30.879 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:30.879 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:30.879 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:30.879 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:30.879 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:30.880 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:30.880 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:30.880 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:30.880 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:30.880 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:30.880 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:30.880 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-07-04T08:32:30.880 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 182043, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 48, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:30.881 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:30.882 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:30.882 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:30.882 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 9448, 2024-07-04T08:32:30.882 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 191, 2024-07-04T08:32:30.882 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:30.882 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:30.882 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:30.882 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-07-04T08:32:30.882 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 8512, 2024-07-04T08:32:30.883 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 13, 2024-07-04T08:32:30.883 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:30.883 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-07-04T08:32:30.883 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:30.883 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-07-04T08:32:30.883 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.883 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-07-04T08:32:30.883 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-07-04T08:32:30.883 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-07-04T08:32:30.884 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 3562, 2024-07-04T08:32:30.884 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 21, 2024-07-04T08:32:30.884 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 3, 2024-07-04T08:32:30.884 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 3562, 2024-07-04T08:32:30.884 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-07-04T08:32:30.884 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3562, 2024-07-04T08:32:30.884 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 2.088944399, 2024-07-04T08:32:30.884 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000586452 2024-07-04T08:32:30.884 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.885 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-07-04T08:32:30.885 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-07-04T08:32:30.885 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3562, 2024-07-04T08:32:30.885 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 3563 2024-07-04T08:32:30.885 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.885 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 3562, 2024-07-04T08:32:30.885 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 3562, 2024-07-04T08:32:30.885 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 0, 2024-07-04T08:32:30.885 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-07-04T08:32:30.886 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-07-04T08:32:30.886 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 2272, 2024-07-04T08:32:30.886 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 0, 2024-07-04T08:32:30.886 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 1288, 2024-07-04T08:32:30.886 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 0, 2024-07-04T08:32:30.886 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 0, 2024-07-04T08:32:30.886 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-07-04T08:32:30.886 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-07-04T08:32:30.886 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 0, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 0, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 1, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 0, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:30.887 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-07-04T08:32:30.888 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-07-04T08:32:30.889 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-07-04T08:32:30.890 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-07-04T08:32:30.890 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 114, 2024-07-04T08:32:30.890 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-07-04T08:32:30.890 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 24, 2024-07-04T08:32:30.890 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 4, 2024-07-04T08:32:30.890 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 6, 2024-07-04T08:32:30.890 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 2, 2024-07-04T08:32:30.890 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-07-04T08:32:30.890 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 0, 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 2, 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 0 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 0, 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.891 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 0 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 0, 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 0, 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 0, 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 0, 2024-07-04T08:32:30.892 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 3975, 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1694837, 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 3975, 2024-07-04T08:32:30.893 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 3975, 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1694837, 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:30.894 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:30.895 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-07-04T08:32:30.895 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:30.895 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:30.895 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:30.895 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:30.895 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:30.895 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 3562, 2024-07-04T08:32:30.895 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 984, 2024-07-04T08:32:30.896 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 3562, 2024-07-04T08:32:30.896 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 984, 2024-07-04T08:32:30.896 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:30.896 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.896 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.896 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.896 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:30.896 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.896 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 3562, 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 3562, 2024-07-04T08:32:30.897 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 3562, 2024-07-04T08:32:30.898 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 3562, 2024-07-04T08:32:30.898 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:30.898 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.898 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.898 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.898 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:30.898 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:30.899 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 0, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 0, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x5642697848f0": { 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:30.900 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 0, 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 0, 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:30.901 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:30.902 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-07-04T08:32:31.661 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-07-04T08:32:31.661 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-07-04T08:32:31.661 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-07-04T08:32:31.662 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-07-04T08:32:31.662 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-07-04T08:32:31.662 DEBUG:tasks.check_counter:Getting stats from h 2024-07-04T08:32:31.662 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.h perf dump 2024-07-04T08:32:31.926 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:32:33.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:33 smithi039 ceph-mon[32062]: pgmap v1022: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:32:33.594 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:33 smithi050 ceph-mon[36577]: pgmap v1022: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:32:33.595 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-07-04T08:32:33.595 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:33.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 902, 2024-07-04T08:32:33.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 1121, 2024-07-04T08:32:33.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 242811, 2024-07-04T08:32:33.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 1085628, 2024-07-04T08:32:33.595 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 56, 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 6, 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.172018877, 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.081664480, 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.078590531, 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.006524103, 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1119, 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.089620830, 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000080090 2024-07-04T08:32:33.596 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1901, 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000442938, 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000233 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 0 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 119, 2024-07-04T08:32:33.597 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 459, 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 39622, 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 823599, 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 17, 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 5, 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.050933636, 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.028850154, 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.018217141, 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000424798, 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:33.598 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 456, 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.044376693, 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000097317 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 208, 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000071280, 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000342 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 480, 2024-07-04T08:32:33.599 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 723360 2024-07-04T08:32:33.600 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.600 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:33.600 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 547, 2024-07-04T08:32:33.600 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 561, 2024-07-04T08:32:33.600 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1114723, 2024-07-04T08:32:33.600 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 314132, 2024-07-04T08:32:33.600 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 12, 2024-07-04T08:32:33.600 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 6, 2024-07-04T08:32:33.600 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.094584415, 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.043466359, 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.065129422, 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000232440, 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 561, 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.062371417, 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000111178 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 192, 2024-07-04T08:32:33.601 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000054609, 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000284 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1082848, 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 250832 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.602 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 54, 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.001763419, 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000032655 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-07-04T08:32:33.603 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2, 2024-07-04T08:32:33.604 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000015974, 2024-07-04T08:32:33.604 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000007987 2024-07-04T08:32:33.604 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:33.604 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.604 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-07-04T08:32:33.604 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 1, 2024-07-04T08:32:33.604 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 1, 2024-07-04T08:32:33.604 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-07-04T08:32:33.605 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1, 2024-07-04T08:32:33.605 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000107749, 2024-07-04T08:32:33.605 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000107749 2024-07-04T08:32:33.605 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.605 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-07-04T08:32:33.605 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 0, 2024-07-04T08:32:33.605 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:33.605 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 11, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 0, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 0, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 10, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 0, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 10, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 10, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 0, 2024-07-04T08:32:33.606 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 1, 2024-07-04T08:32:33.607 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 1, 2024-07-04T08:32:33.607 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 1, 2024-07-04T08:32:33.607 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 24, 2024-07-04T08:32:33.607 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 24, 2024-07-04T08:32:33.607 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 0, 2024-07-04T08:32:33.607 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:33.608 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 0, 2024-07-04T08:32:33.609 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-07-04T08:32:33.609 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 4, 2024-07-04T08:32:33.609 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 234, 2024-07-04T08:32:33.609 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 6, 2024-07-04T08:32:33.609 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 0, 2024-07-04T08:32:33.609 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:33.609 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:33.610 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:33.610 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 0, 2024-07-04T08:32:33.610 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 0, 2024-07-04T08:32:33.610 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 0, 2024-07-04T08:32:33.610 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 0, 2024-07-04T08:32:33.610 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:33.610 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 1, 2024-07-04T08:32:33.610 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 0, 2024-07-04T08:32:33.611 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 0, 2024-07-04T08:32:33.611 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:33.611 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:33.611 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 1, 2024-07-04T08:32:33.611 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 0, 2024-07-04T08:32:33.611 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 0, 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 0 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 0, 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 0, 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 1, 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 0, 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 0, 2024-07-04T08:32:33.612 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:33.613 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:33.613 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 0, 2024-07-04T08:32:33.613 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 0, 2024-07-04T08:32:33.613 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:33.613 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-07-04T08:32:33.613 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:33.613 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:33.614 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:33.614 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-07-04T08:32:33.614 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-07-04T08:32:33.614 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_path": 12, 2024-07-04T08:32:33.614 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_inode": 12, 2024-07-04T08:32:33.614 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 0, 2024-07-04T08:32:33.614 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:33.614 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-07-04T08:32:33.615 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:33.615 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:33.615 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-07-04T08:32:33.615 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.615 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-07-04T08:32:33.615 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 0, 2024-07-04T08:32:33.615 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 4, 2024-07-04T08:32:33.615 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 0, 2024-07-04T08:32:33.616 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 0, 2024-07-04T08:32:33.616 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 4, 2024-07-04T08:32:33.616 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-07-04T08:32:33.616 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 0, 2024-07-04T08:32:33.616 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 2, 2024-07-04T08:32:33.616 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 0, 2024-07-04T08:32:33.616 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 0, 2024-07-04T08:32:33.617 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 2, 2024-07-04T08:32:33.617 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 2, 2024-07-04T08:32:33.617 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-07-04T08:32:33.617 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 0, 2024-07-04T08:32:33.617 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 4194304, 2024-07-04T08:32:33.617 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 4195589, 2024-07-04T08:32:33.617 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 0, 2024-07-04T08:32:33.618 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-07-04T08:32:33.618 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3, 2024-07-04T08:32:33.618 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.009934399, 2024-07-04T08:32:33.618 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.003311466 2024-07-04T08:32:33.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.618 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 0 2024-07-04T08:32:33.618 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.618 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 13, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 13, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 0, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 11, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 12, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 1, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 10, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 10, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 0, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 1, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 1, 2024-07-04T08:32:33.619 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 0, 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 118188, 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 223540 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 1, 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 1, 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 26, 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 0, 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-07-04T08:32:33.620 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.621 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.622 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1, 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000107749, 2024-07-04T08:32:33.623 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000107749 2024-07-04T08:32:33.624 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.624 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-07-04T08:32:33.624 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.624 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.624 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.624 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.624 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:33.624 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.624 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.625 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.626 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.626 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.626 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-07-04T08:32:33.626 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.627 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.627 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.627 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.627 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-07-04T08:32:33.627 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.627 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.627 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.627 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.627 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-07-04T08:32:33.628 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.628 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.628 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.628 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.628 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-07-04T08:32:33.628 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.628 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.628 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.629 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.630 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.631 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.632 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.633 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:33.633 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.633 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.633 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.633 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.633 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:33.633 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.633 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.634 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.635 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.636 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.636 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.636 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:33.636 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:33.636 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:33.636 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.636 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.636 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.636 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:33.637 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.637 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-07-04T08:32:33.637 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 2, 2024-07-04T08:32:33.637 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 2, 2024-07-04T08:32:33.637 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 0, 2024-07-04T08:32:33.637 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 2, 2024-07-04T08:32:33.637 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-07-04T08:32:33.637 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 1, 2024-07-04T08:32:33.637 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 1, 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 1000, 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:33.638 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:33.639 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:33.640 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:33.640 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:33.640 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:33.640 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:33.640 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:33.640 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:33.640 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:33.640 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:33.640 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:33.641 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:33.641 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:33.641 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:33.641 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:33.641 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-07-04T08:32:33.641 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:33.641 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:33.641 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:33.641 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 264485, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 99, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:33.642 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:33.643 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 9448, 2024-07-04T08:32:33.643 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 191, 2024-07-04T08:32:33.643 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:33.643 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:33.643 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:33.643 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-07-04T08:32:33.643 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 51240, 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 200, 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 39, 2024-07-04T08:32:33.644 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 11587, 2024-07-04T08:32:33.645 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 29, 2024-07-04T08:32:33.645 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 39, 2024-07-04T08:32:33.645 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-07-04T08:32:33.645 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 39, 2024-07-04T08:32:33.645 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.461987703, 2024-07-04T08:32:33.645 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.011845838 2024-07-04T08:32:33.645 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.645 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 39, 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 61 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 39, 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 0, 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 39, 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-07-04T08:32:33.646 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 0, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 10, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 0, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 3, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 7, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 0, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 6, 2024-07-04T08:32:33.647 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-07-04T08:32:33.648 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:33.648 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:33.648 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 0, 2024-07-04T08:32:33.648 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 20, 2024-07-04T08:32:33.648 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:33.648 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:33.648 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:33.648 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-07-04T08:32:33.648 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-07-04T08:32:33.649 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-07-04T08:32:33.650 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-07-04T08:32:33.651 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 114, 2024-07-04T08:32:33.651 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-07-04T08:32:33.651 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 24, 2024-07-04T08:32:33.651 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 9, 2024-07-04T08:32:33.651 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 12, 2024-07-04T08:32:33.651 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 3, 2024-07-04T08:32:33.651 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-07-04T08:32:33.651 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 14, 2024-07-04T08:32:33.651 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 0, 2024-07-04T08:32:33.652 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 1 2024-07-04T08:32:33.652 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.652 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-07-04T08:32:33.652 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 1, 2024-07-04T08:32:33.652 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:33.652 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-07-04T08:32:33.652 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-07-04T08:32:33.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.653 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-07-04T08:32:33.653 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 24 2024-07-04T08:32:33.653 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.653 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-07-04T08:32:33.653 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:33.653 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 0, 2024-07-04T08:32:33.653 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-07-04T08:32:33.654 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 0, 2024-07-04T08:32:33.654 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 0, 2024-07-04T08:32:33.654 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 0, 2024-07-04T08:32:33.654 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:33.654 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.654 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:33.654 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:33.655 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-07-04T08:32:33.655 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:33.655 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 1564, 2024-07-04T08:32:33.655 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1230672, 2024-07-04T08:32:33.655 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:33.655 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 1564, 2024-07-04T08:32:33.655 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:33.655 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:33.656 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 1564, 2024-07-04T08:32:33.656 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1230672, 2024-07-04T08:32:33.656 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:33.656 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.656 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.657 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.657 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:33.657 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.657 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:33.657 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:33.658 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-07-04T08:32:33.658 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:33.658 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:33.658 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:33.658 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:33.658 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:33.658 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 39, 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 11587, 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 39, 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 11587, 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.659 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 39, 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 39, 2024-07-04T08:32:33.660 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 39, 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 39, 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:33.661 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 0, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 0, 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:33.662 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x55f78d6e05b0": { 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 4, 2024-07-04T08:32:33.663 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1285, 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 4, 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 3, 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1285, 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:33.664 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:33.665 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:33.665 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:33.665 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-07-04T08:32:33.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:33 smithi133 ceph-mon[37050]: pgmap v1022: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:32:34.389 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-07-04T08:32:34.389 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-07-04T08:32:34.389 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-07-04T08:32:34.389 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-07-04T08:32:34.389 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-07-04T08:32:34.389 DEBUG:tasks.check_counter:Getting stats from k 2024-07-04T08:32:34.390 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.k perf dump 2024-07-04T08:32:34.655 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.b/config 2024-07-04T08:32:35.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:35 smithi050 ceph-mon[36577]: pgmap v1023: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:32:35.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:35 smithi133 ceph-mon[37050]: pgmap v1023: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:32:35.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:35 smithi039 ceph-mon[32062]: pgmap v1023: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:32:36.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:36 smithi050 ceph-mon[36577]: pgmap v1024: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-04T08:32:36.403 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-07-04T08:32:36.403 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 1294, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 1294, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 436329, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 410669, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 7, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 2, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.191003964, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.098994389, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.035316544, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.039453687, 2024-07-04T08:32:36.404 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:36.405 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 1293, 2024-07-04T08:32:36.405 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.087001213, 2024-07-04T08:32:36.405 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000067286 2024-07-04T08:32:36.405 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.405 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:36.405 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.405 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.405 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.405 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.406 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-07-04T08:32:36.406 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 448 2024-07-04T08:32:36.406 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.406 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:36.406 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 2721, 2024-07-04T08:32:36.406 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 2713, 2024-07-04T08:32:36.406 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1649803, 2024-07-04T08:32:36.406 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 972759, 2024-07-04T08:32:36.406 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:36.407 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 3, 2024-07-04T08:32:36.407 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.290380092, 2024-07-04T08:32:36.407 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.132223384, 2024-07-04T08:32:36.407 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.095145835, 2024-07-04T08:32:36.407 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.062433988, 2024-07-04T08:32:36.407 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:36.407 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 2712, 2024-07-04T08:32:36.407 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.126747430, 2024-07-04T08:32:36.408 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000046735 2024-07-04T08:32:36.408 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.408 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:36.408 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.408 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.408 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.408 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.408 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 1060800, 2024-07-04T08:32:36.408 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 239712 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_messages": 5, 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages": 293, 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_bytes": 1668, 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_bytes": 645733, 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_created_connections": 8, 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_active_connections": 4, 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_total_time": 0.022955099, 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_send_time": 0.017119041, 2024-07-04T08:32:36.409 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_recv_time": 0.002676531, 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_running_fast_dispatch_time": 0.000098198, 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 291, 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.018585844, 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000063868 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.410 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.411 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-04T08:32:36.411 INFO:teuthology.orchestra.run.smithi050.stdout: "msgr_send_encrypted_bytes": 644768 2024-07-04T08:32:36.411 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.411 INFO:teuthology.orchestra.run.smithi050.stdout: "cct": { 2024-07-04T08:32:36.411 INFO:teuthology.orchestra.run.smithi050.stdout: "total_workers": 1, 2024-07-04T08:32:36.411 INFO:teuthology.orchestra.run.smithi050.stdout: "unhealthy_workers": 0 2024-07-04T08:32:36.411 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.411 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-MDSRank": { 2024-07-04T08:32:36.411 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-07-04T08:32:36.412 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-07-04T08:32:36.412 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 6177, 2024-07-04T08:32:36.412 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.178264814, 2024-07-04T08:32:36.412 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000028859 2024-07-04T08:32:36.412 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:36.412 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.412 INFO:teuthology.orchestra.run.smithi050.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: "queue_len": 0, 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: "complete_latency": { 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: "mds": { 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: "request": 0, 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: "reply": 0, 2024-07-04T08:32:36.413 INFO:teuthology.orchestra.run.smithi050.stdout: "reply_latency": { 2024-07-04T08:32:36.414 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.414 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.414 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.414 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.414 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_reply": 0, 2024-07-04T08:32:36.414 INFO:teuthology.orchestra.run.smithi050.stdout: "forward": 0, 2024-07-04T08:32:36.414 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:36.414 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:36.414 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_commit": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_split": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_merge": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_top": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_bottom": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_pinned": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_expired": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "inodes_with_caps": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "caps": 0, 2024-07-04T08:32:36.415 INFO:teuthology.orchestra.run.smithi050.stdout: "subtrees": 1, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_hit": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_forward": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_discover": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "traverse_lock": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "load_cent": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "q": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "exported": 0, 2024-07-04T08:32:36.416 INFO:teuthology.orchestra.run.smithi050.stdout: "exported_inodes": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "imported": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "imported_inodes": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_backtrace_fetch": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "openino_peer_discover": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rfiles": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rbytes": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "root_rsnaps": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_fetch": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:36.417 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_inodes": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dir_base_inodes": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_dirfrag_rstats": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "scrub_file_inodes": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_revoke": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_grant": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_trunc": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:36.418 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps": 0, 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_caps_dirty": 0, 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_cap_release": 0, 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "process_request_cap_release": 0 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_cache": { 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update": 0, 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_try_discover": 0, 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_send_discover": 0, 2024-07-04T08:32:36.419 INFO:teuthology.orchestra.run.smithi050.stdout: "dir_handle_discover": 0, 2024-07-04T08:32:36.420 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays": 0, 2024-07-04T08:32:36.420 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:36.420 INFO:teuthology.orchestra.run.smithi050.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:36.420 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_created": 0, 2024-07-04T08:32:36.420 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_enqueued": 0, 2024-07-04T08:32:36.420 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:36.420 INFO:teuthology.orchestra.run.smithi050.stdout: "strays_migrated": 0, 2024-07-04T08:32:36.420 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:36.420 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_started": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "recovery_completed": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_path": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_quiesce_inode": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_enqueue_scrub": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_flush": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:36.421 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: "ireq_inodestats": 0 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_log": { 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: "evlrg": 0, 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: "evadd": 0, 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: "evex": 0, 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: "evtrm": 0, 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: "ev": 4, 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: "evexg": 0, 2024-07-04T08:32:36.422 INFO:teuthology.orchestra.run.smithi050.stdout: "evexd": 0, 2024-07-04T08:32:36.423 INFO:teuthology.orchestra.run.smithi050.stdout: "segadd": 0, 2024-07-04T08:32:36.423 INFO:teuthology.orchestra.run.smithi050.stdout: "segex": 0, 2024-07-04T08:32:36.423 INFO:teuthology.orchestra.run.smithi050.stdout: "segtrm": 0, 2024-07-04T08:32:36.423 INFO:teuthology.orchestra.run.smithi050.stdout: "seg": 2, 2024-07-04T08:32:36.423 INFO:teuthology.orchestra.run.smithi050.stdout: "segmjr": 2, 2024-07-04T08:32:36.423 INFO:teuthology.orchestra.run.smithi050.stdout: "segexg": 0, 2024-07-04T08:32:36.423 INFO:teuthology.orchestra.run.smithi050.stdout: "segexd": 0, 2024-07-04T08:32:36.424 INFO:teuthology.orchestra.run.smithi050.stdout: "expos": 4194304, 2024-07-04T08:32:36.424 INFO:teuthology.orchestra.run.smithi050.stdout: "wrpos": 4195589, 2024-07-04T08:32:36.424 INFO:teuthology.orchestra.run.smithi050.stdout: "rdpos": 4195198, 2024-07-04T08:32:36.424 INFO:teuthology.orchestra.run.smithi050.stdout: "jlat": { 2024-07-04T08:32:36.424 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.424 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.424 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.424 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.425 INFO:teuthology.orchestra.run.smithi050.stdout: "replayed": 4 2024-07-04T08:32:36.425 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.425 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_mem": { 2024-07-04T08:32:36.425 INFO:teuthology.orchestra.run.smithi050.stdout: "ino": 3, 2024-07-04T08:32:36.425 INFO:teuthology.orchestra.run.smithi050.stdout: "ino+": 3, 2024-07-04T08:32:36.425 INFO:teuthology.orchestra.run.smithi050.stdout: "ino-": 0, 2024-07-04T08:32:36.425 INFO:teuthology.orchestra.run.smithi050.stdout: "dir": 1, 2024-07-04T08:32:36.425 INFO:teuthology.orchestra.run.smithi050.stdout: "dir+": 1, 2024-07-04T08:32:36.425 INFO:teuthology.orchestra.run.smithi050.stdout: "dir-": 0, 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "dn": 0, 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "dn+": 0, 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "dn-": 0, 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "cap": 0, 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "cap+": 0, 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "cap-": 0, 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "rss": 117180, 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "heap": 215348 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_server": { 2024-07-04T08:32:36.426 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_client_request": 0, 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_request": 0, 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_client_session": 0, 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "handle_peer_request": 0, 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "req_create_latency": { 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getattr_latency": { 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.427 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: "req_link_latency": { 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.428 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookup_latency": { 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupino_latency": { 2024-07-04T08:32:36.429 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupname_latency": { 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.430 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: "req_lssnap_latency": { 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.431 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.432 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mkdir_latency": { 2024-07-04T08:32:36.432 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mknod_latency": { 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: "req_mksnap_latency": { 2024-07-04T08:32:36.433 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.434 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.434 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.434 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.434 INFO:teuthology.orchestra.run.smithi050.stdout: "req_open_latency": { 2024-07-04T08:32:36.434 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.434 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.434 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.434 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.434 INFO:teuthology.orchestra.run.smithi050.stdout: "req_readdir_latency": { 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rename_latency": { 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.435 INFO:teuthology.orchestra.run.smithi050.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:36.436 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.436 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.436 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.436 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.436 INFO:teuthology.orchestra.run.smithi050.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:36.436 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.436 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.437 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.437 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.437 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmdir_latency": { 2024-07-04T08:32:36.437 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.437 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.437 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.437 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.437 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:36.438 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.438 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.438 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.438 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.438 INFO:teuthology.orchestra.run.smithi050.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:36.438 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.438 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.438 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.439 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.439 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setattr_latency": { 2024-07-04T08:32:36.439 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.439 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.439 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.439 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.439 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:36.439 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.439 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.440 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.440 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.440 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:36.440 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.440 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.440 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.440 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.440 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setlayout_latency": { 2024-07-04T08:32:36.440 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.441 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.441 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.441 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.441 INFO:teuthology.orchestra.run.smithi050.stdout: "req_setxattr_latency": { 2024-07-04T08:32:36.441 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.441 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.441 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.442 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.442 INFO:teuthology.orchestra.run.smithi050.stdout: "req_symlink_latency": { 2024-07-04T08:32:36.442 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.442 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.442 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.442 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.442 INFO:teuthology.orchestra.run.smithi050.stdout: "req_unlink_latency": { 2024-07-04T08:32:36.442 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.443 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.443 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.443 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.443 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:36.443 INFO:teuthology.orchestra.run.smithi050.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:36.443 INFO:teuthology.orchestra.run.smithi050.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:36.443 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.443 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.443 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.444 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:36.444 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.444 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_sessions": { 2024-07-04T08:32:36.444 INFO:teuthology.orchestra.run.smithi050.stdout: "session_count": 2, 2024-07-04T08:32:36.444 INFO:teuthology.orchestra.run.smithi050.stdout: "session_add": 2, 2024-07-04T08:32:36.444 INFO:teuthology.orchestra.run.smithi050.stdout: "session_remove": 0, 2024-07-04T08:32:36.444 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_open": 2, 2024-07-04T08:32:36.444 INFO:teuthology.orchestra.run.smithi050.stdout: "sessions_stale": 0, 2024-07-04T08:32:36.444 INFO:teuthology.orchestra.run.smithi050.stdout: "total_load": 0, 2024-07-04T08:32:36.445 INFO:teuthology.orchestra.run.smithi050.stdout: "average_load": 0, 2024-07-04T08:32:36.445 INFO:teuthology.orchestra.run.smithi050.stdout: "avg_session_uptime": 2923, 2024-07-04T08:32:36.445 INFO:teuthology.orchestra.run.smithi050.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:36.445 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.445 INFO:teuthology.orchestra.run.smithi050.stdout: "mempool": { 2024-07-04T08:32:36.445 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:36.445 INFO:teuthology.orchestra.run.smithi050.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:36.445 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:36.446 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:36.446 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:36.446 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:36.446 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:36.446 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:36.446 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:36.447 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:36.447 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:36.447 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:36.447 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:36.447 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:36.447 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:36.447 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:36.447 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:36.448 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:36.448 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:36.448 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:36.448 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:36.448 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:36.448 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:36.448 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:36.449 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:36.449 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:36.449 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:36.449 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:36.449 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:36.449 INFO:teuthology.orchestra.run.smithi050.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:36.449 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:36.449 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_items": 0, 2024-07-04T08:32:36.450 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:36.450 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:36.450 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:36.450 INFO:teuthology.orchestra.run.smithi050.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:36.450 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_bytes": 198225, 2024-07-04T08:32:36.450 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_anon_items": 56, 2024-07-04T08:32:36.450 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:36.450 INFO:teuthology.orchestra.run.smithi050.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:36.451 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_bytes": 0, 2024-07-04T08:32:36.451 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_items": 0, 2024-07-04T08:32:36.451 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:36.451 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:36.451 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:36.451 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:36.451 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_bytes": 9448, 2024-07-04T08:32:36.451 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_items": 191, 2024-07-04T08:32:36.451 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:36.452 INFO:teuthology.orchestra.run.smithi050.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:36.452 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:36.452 INFO:teuthology.orchestra.run.smithi050.stdout: "pgmap_items": 0, 2024-07-04T08:32:36.452 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_bytes": 8512, 2024-07-04T08:32:36.452 INFO:teuthology.orchestra.run.smithi050.stdout: "mds_co_items": 13, 2024-07-04T08:32:36.452 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:36.452 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_1_items": 0, 2024-07-04T08:32:36.452 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:36.453 INFO:teuthology.orchestra.run.smithi050.stdout: "unittest_2_items": 0 2024-07-04T08:32:36.453 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.453 INFO:teuthology.orchestra.run.smithi050.stdout: "objecter": { 2024-07-04T08:32:36.453 INFO:teuthology.orchestra.run.smithi050.stdout: "op_active": 0, 2024-07-04T08:32:36.453 INFO:teuthology.orchestra.run.smithi050.stdout: "op_laggy": 0, 2024-07-04T08:32:36.453 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send": 3598, 2024-07-04T08:32:36.453 INFO:teuthology.orchestra.run.smithi050.stdout: "op_send_bytes": 21, 2024-07-04T08:32:36.453 INFO:teuthology.orchestra.run.smithi050.stdout: "op_resend": 3, 2024-07-04T08:32:36.453 INFO:teuthology.orchestra.run.smithi050.stdout: "op_reply": 3598, 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: "op_latency": { 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3598, 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 1.724954090, 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000479420 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: "op_inflight": 0, 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: "oplen_avg": { 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 3598, 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 3599 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.454 INFO:teuthology.orchestra.run.smithi050.stdout: "op": 3598, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "op_r": 3598, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "op_w": 0, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "op_rmw": 0, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "op_pg": 0, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_stat": 2301, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_create": 0, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_read": 1295, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_write": 0, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writefull": 0, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_writesame": 0, 2024-07-04T08:32:36.455 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_append": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_zero": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_truncate": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_delete": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_mapext": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_getxattr": 1, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_setxattr": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:36.456 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_call": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_watch": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_notify": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "osdop_other": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_active": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_send": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_resend": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "linger_ping": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_active": 0, 2024-07-04T08:32:36.457 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_send": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "poolop_resend": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_active": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_send": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "poolstat_resend": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_active": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_send": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "statfs_resend": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "command_active": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "command_send": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "command_resend": 0, 2024-07-04T08:32:36.458 INFO:teuthology.orchestra.run.smithi050.stdout: "map_epoch": 114, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "map_full": 0, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "map_inc": 24, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_sessions": 6, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_open": 9, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_session_close": 3, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "osd_laggy": 0, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_wr": 0, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_rd": 2, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_del": 0 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "oft": { 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_objs": 0, 2024-07-04T08:32:36.459 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:36.460 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_updates": 0, 2024-07-04T08:32:36.460 INFO:teuthology.orchestra.run.smithi050.stdout: "omap_total_removes": 0 2024-07-04T08:32:36.460 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.460 INFO:teuthology.orchestra.run.smithi050.stdout: "osd-slow-ops": { 2024-07-04T08:32:36.460 INFO:teuthology.orchestra.run.smithi050.stdout: "slow_ops_count": 0 2024-07-04T08:32:36.460 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.460 INFO:teuthology.orchestra.run.smithi050.stdout: "purge_queue": { 2024-07-04T08:32:36.460 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:36.460 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_ops_high_water": 0, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing": 0, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executing_high_water": 0, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed_ops": 0, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_executed": 0, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 4016, 2024-07-04T08:32:36.461 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 1702045, 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 4016, 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 4016, 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1702045, 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.462 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.463 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:36.463 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.463 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:36.463 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:36.463 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 104857600, 2024-07-04T08:32:36.463 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:36.463 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:36.464 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:36.464 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:36.464 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:36.464 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 3598, 2024-07-04T08:32:36.464 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 1290, 2024-07-04T08:32:36.464 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 3598, 2024-07-04T08:32:36.464 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 1290, 2024-07-04T08:32:36.464 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:36.464 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 1024, 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:36.465 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 3598, 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 3598, 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 3598, 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 3598, 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:36.466 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.467 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:36.467 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:36.467 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-07-04T08:32:36.467 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:36.467 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:36.467 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:36.468 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:36.468 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:36.468 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:36.468 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:36.468 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 0, 2024-07-04T08:32:36.468 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 0, 2024-07-04T08:32:36.468 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:36.468 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.468 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "throttle-write_buf_throttle-0x55bf9f9888f0": { 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "val": 0, 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "max": 3758096384, 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "get_started": 0, 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "get": 0, 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "get_sum": 0, 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:36.469 INFO:teuthology.orchestra.run.smithi050.stdout: "take": 0, 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout: "take_sum": 0, 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout: "put": 0, 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout: "put_sum": 0, 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout: "wait": { 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout: "avgcount": 0, 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout: "sum": 0.000000000, 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout: "avgtime": 0.000000000 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-07-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-07-04T08:32:36.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:36 smithi133 ceph-mon[37050]: pgmap v1024: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-04T08:32:36.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:36 smithi039 ceph-mon[32062]: pgmap v1024: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-04T08:32:37.281 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-07-04T08:32:37.281 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-07-04T08:32:37.281 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-07-04T08:32:37.281 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-07-04T08:32:37.282 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-07-04T08:32:37.282 DEBUG:tasks.check_counter:Getting stats from c 2024-07-04T08:32:37.282 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.c perf dump 2024-07-04T08:32:37.546 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:32:39.453 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-07-04T08:32:39.454 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:39.454 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 862, 2024-07-04T08:32:39.454 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1080, 2024-07-04T08:32:39.454 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 239995, 2024-07-04T08:32:39.454 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 382594, 2024-07-04T08:32:39.454 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 11, 2024-07-04T08:32:39.454 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-07-04T08:32:39.454 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.140511615, 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.065121249, 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.061245831, 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.008597923, 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1079, 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.077053921, 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000071412 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1867, 2024-07-04T08:32:39.455 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000460277, 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000246 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 115, 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 429, 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 38656, 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 733548, 2024-07-04T08:32:39.456 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 17, 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.047263959, 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.027095875, 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.016822514, 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000178809, 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 427, 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.052032935, 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000121856 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:39.457 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 212, 2024-07-04T08:32:39.458 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000100383, 2024-07-04T08:32:39.458 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000473 2024-07-04T08:32:39.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.458 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-07-04T08:32:39.458 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 637312 2024-07-04T08:32:39.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.458 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:39.458 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 590, 2024-07-04T08:32:39.458 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 580, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1113485, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 865232, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 57, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.109683590, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.046569981, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.076315908, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000459397, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 579, 2024-07-04T08:32:39.459 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.048354211, 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000083513 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 195, 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000057905, 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000296 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1072448, 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 242592 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-07-04T08:32:39.460 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 55, 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.001303571, 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000023701 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:39.461 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2, 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000012324, 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000006162 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 1, 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-07-04T08:32:39.462 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 1, 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 11, 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-07-04T08:32:39.463 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 10, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 10, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 10, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 1, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 24, 2024-07-04T08:32:39.464 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 24, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-07-04T08:32:39.465 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 1, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 4, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 234, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:39.466 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:39.467 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-07-04T08:32:39.467 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-07-04T08:32:39.467 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-07-04T08:32:39.467 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-07-04T08:32:39.467 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:39.467 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-07-04T08:32:39.467 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-07-04T08:32:39.467 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-07-04T08:32:39.467 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:39.468 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:39.468 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-07-04T08:32:39.468 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-07-04T08:32:39.468 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-07-04T08:32:39.468 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-07-04T08:32:39.468 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.468 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 1, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-07-04T08:32:39.469 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 12, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 12, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 5, 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 5, 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-07-04T08:32:39.471 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 2, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 2, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 4195730, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-07-04T08:32:39.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.063829338, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.015957334 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 13, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 13, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 0, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 11, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 12, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 1, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 10, 2024-07-04T08:32:39.473 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 10, 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 118908, 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 223540 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 1, 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:39.474 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 1, 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 18, 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.476 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.477 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.477 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:39.477 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.477 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.477 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.478 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.478 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-07-04T08:32:39.478 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.478 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.478 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.479 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:39.480 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.480 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.480 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.480 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.480 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-07-04T08:32:39.481 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.481 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.481 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.481 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-07-04T08:32:39.482 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.482 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.482 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.482 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.482 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.483 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.484 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-07-04T08:32:39.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.486 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.487 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-07-04T08:32:39.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.487 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.488 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-07-04T08:32:39.489 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.489 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.489 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.489 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-07-04T08:32:39.489 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.489 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.489 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.490 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 2, 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 1, 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-07-04T08:32:39.491 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 1011, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:39.492 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:39.493 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:39.494 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 247109, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 84, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:39.495 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 9448, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 191, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 50872, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 198, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:39.496 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 40, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 11728, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 33, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 40, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 40, 2024-07-04T08:32:39.497 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.413992250, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.010349806 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 40, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 62 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 40, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 0, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 40, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-07-04T08:32:39.498 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 0, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 10, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 0, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 4, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 7, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 6, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:39.499 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 20, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-07-04T08:32:39.500 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-07-04T08:32:39.501 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 114, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 24, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 9, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 13, 2024-07-04T08:32:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 4, 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 14, 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 0, 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 1 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-07-04T08:32:39.503 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 24 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-07-04T08:32:39.504 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 1563, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1225946, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 1563, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1563, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1225946, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.505 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:39.506 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:39.507 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 40, 2024-07-04T08:32:39.507 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 11728, 2024-07-04T08:32:39.507 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 40, 2024-07-04T08:32:39.507 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 11728, 2024-07-04T08:32:39.507 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:39.507 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.507 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.507 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.507 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:39.508 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 40, 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 40, 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 40, 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 40, 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.509 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:39.510 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:39.510 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-07-04T08:32:39.510 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:39.510 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:39.510 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.511 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55cbe6baa820": { 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 5, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1426, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 5, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:39.512 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4, 2024-07-04T08:32:39.513 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1426, 2024-07-04T08:32:39.513 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:39.513 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:39.513 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:39.513 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:39.513 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:39.513 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:39.513 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-07-04T08:32:39.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:39 smithi050 ceph-mon[36577]: pgmap v1025: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-04T08:32:39.790 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:39 smithi133 ceph-mon[37050]: pgmap v1025: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-04T08:32:39.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:39 smithi039 ceph-mon[32062]: pgmap v1025: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-04T08:32:40.167 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-07-04T08:32:40.167 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-07-04T08:32:40.167 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-07-04T08:32:40.167 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-07-04T08:32:40.167 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-07-04T08:32:40.167 DEBUG:tasks.check_counter:Getting stats from f 2024-07-04T08:32:40.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.f perf dump 2024-07-04T08:32:40.428 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:32:40.645 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:40 smithi050 ceph-mon[36577]: pgmap v1026: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:32:40.689 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:40 smithi133 ceph-mon[37050]: pgmap v1026: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:32:40.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:40 smithi039 ceph-mon[32062]: pgmap v1026: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:32:42.164 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-07-04T08:32:42.164 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:42.164 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 669, 2024-07-04T08:32:42.164 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 669, 2024-07-04T08:32:42.164 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 11426183, 2024-07-04T08:32:42.164 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 211924, 2024-07-04T08:32:42.164 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 9, 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.081303912, 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.029258272, 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.027671366, 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.018668786, 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 668, 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.038412426, 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000057503 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:42.165 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2755, 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 3042, 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 13389874, 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 1512267, 2024-07-04T08:32:42.166 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.398537054, 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.173438920, 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.094619487, 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.101171199, 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3039, 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.136320697, 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000044857 2024-07-04T08:32:42.167 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 636752 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:42.168 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1560, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1552, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 9736029, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 602390, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.219061206, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.100400372, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.082472274, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.036091319, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1551, 2024-07-04T08:32:42.169 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.076454674, 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000049293 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1060144, 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 237952 2024-07-04T08:32:42.170 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 7727, 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.226279905, 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000029284 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:42.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-07-04T08:32:42.172 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 36, 2024-07-04T08:32:42.173 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-07-04T08:32:42.174 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 36, 2024-07-04T08:32:42.174 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:42.174 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 36, 2024-07-04T08:32:42.174 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-07-04T08:32:42.174 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-07-04T08:32:42.174 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-07-04T08:32:42.174 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 2, 2024-07-04T08:32:42.174 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2024-07-04T08:32:42.174 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-07-04T08:32:42.175 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 4, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 234, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-07-04T08:32:42.176 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-07-04T08:32:42.177 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-07-04T08:32:42.178 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-07-04T08:32:42.179 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2024-07-04T08:32:42.180 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 2460, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 9, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-07-04T08:32:42.181 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 36420568, 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 36420432, 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 2460 2024-07-04T08:32:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 31, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 32, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 1, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 11, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 11, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 36, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 36, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-07-04T08:32:42.183 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-07-04T08:32:42.184 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-07-04T08:32:42.184 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-07-04T08:32:42.184 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 117324, 2024-07-04T08:32:42.184 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215348 2024-07-04T08:32:42.184 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.184 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-07-04T08:32:42.184 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2024-07-04T08:32:42.184 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:42.184 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.185 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.186 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.187 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.187 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-07-04T08:32:42.187 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.187 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.187 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.187 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.187 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:42.187 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.187 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.188 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-07-04T08:32:42.189 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.190 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-07-04T08:32:42.192 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.193 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.194 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-07-04T08:32:42.195 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.197 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.198 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.198 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.198 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-07-04T08:32:42.198 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.198 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.198 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.198 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.198 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:42.198 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 4, 2024-07-04T08:32:42.199 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 2, 2024-07-04T08:32:42.200 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 2, 2024-07-04T08:32:42.200 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-07-04T08:32:42.200 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-07-04T08:32:42.200 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-07-04T08:32:42.200 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 2925, 2024-07-04T08:32:42.200 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:42.200 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.200 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-07-04T08:32:42.201 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:42.201 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:42.201 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:42.201 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:42.201 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:42.201 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:42.201 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:42.201 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:42.201 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:42.202 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:42.202 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:42.202 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:42.202 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:42.202 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:42.202 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:42.202 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:42.202 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:42.203 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:42.203 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:42.203 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:42.203 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:42.203 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:42.203 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:42.203 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:42.203 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:42.204 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:42.205 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:42.205 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 215813, 2024-07-04T08:32:42.205 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 102, 2024-07-04T08:32:42.205 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:42.205 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:42.205 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-07-04T08:32:42.205 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-07-04T08:32:42.205 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:42.205 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:42.206 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:42.206 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:42.206 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 9448, 2024-07-04T08:32:42.206 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 191, 2024-07-04T08:32:42.206 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:42.206 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:42.206 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:42.206 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-07-04T08:32:42.206 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 183650, 2024-07-04T08:32:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 1359, 2024-07-04T08:32:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-07-04T08:32:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-07-04T08:32:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-07-04T08:32:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-07-04T08:32:42.207 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 4564, 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 26, 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 4, 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 4564, 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4564, 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.900945540, 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000635614 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.208 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4564, 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 4565 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 4564, 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 4564, 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-07-04T08:32:42.209 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 2586, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 1975, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2024-07-04T08:32:42.210 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 2, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:42.211 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-07-04T08:32:42.212 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:42.212 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-07-04T08:32:42.214 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-07-04T08:32:42.214 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-07-04T08:32:42.214 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-07-04T08:32:42.214 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-07-04T08:32:42.214 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-07-04T08:32:42.214 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-07-04T08:32:42.215 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-07-04T08:32:42.215 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-07-04T08:32:42.215 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-07-04T08:32:42.215 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-07-04T08:32:42.215 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-07-04T08:32:42.215 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-07-04T08:32:42.215 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-07-04T08:32:42.216 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-07-04T08:32:42.216 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-07-04T08:32:42.216 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-07-04T08:32:42.216 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 114, 2024-07-04T08:32:42.216 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-07-04T08:32:42.216 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 24, 2024-07-04T08:32:42.216 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 8, 2024-07-04T08:32:42.216 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 10, 2024-07-04T08:32:42.217 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 2, 2024-07-04T08:32:42.217 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-07-04T08:32:42.217 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2024-07-04T08:32:42.217 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-07-04T08:32:42.217 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2024-07-04T08:32:42.217 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.217 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-07-04T08:32:42.217 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-07-04T08:32:42.217 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:42.218 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-07-04T08:32:42.218 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-07-04T08:32:42.218 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.218 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-07-04T08:32:42.218 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-07-04T08:32:42.218 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.218 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-07-04T08:32:42.218 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:42.218 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 4980, 2024-07-04T08:32:42.219 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 34079681, 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 4980, 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4980, 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 34079681, 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.220 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:42.221 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 4564, 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 32226274, 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4564, 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 32226274, 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.222 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 4564, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 4564, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4564, 2024-07-04T08:32:42.223 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 4564, 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-07-04T08:32:42.224 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:42.225 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.226 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.226 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.226 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:42.226 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:42.226 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x5588c916c340": { 2024-07-04T08:32:42.226 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:42.226 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-07-04T08:32:42.226 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:42.227 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:42.228 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:42.228 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:42.228 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-07-04T08:32:42.975 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-07-04T08:32:42.976 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-07-04T08:32:42.976 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-07-04T08:32:42.976 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-07-04T08:32:42.976 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-07-04T08:32:42.976 DEBUG:tasks.check_counter:Getting stats from i 2024-07-04T08:32:42.976 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.i perf dump 2024-07-04T08:32:43.226 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:32:43.600 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:43 smithi133 ceph-mon[37050]: pgmap v1027: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:32:43.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:43 smithi050 ceph-mon[36577]: pgmap v1027: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:32:43.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:43 smithi039 ceph-mon[32062]: pgmap v1027: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:32:44.551 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:44 smithi133 ceph-mon[37050]: pgmap v1028: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:32:44.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:44 smithi050 ceph-mon[36577]: pgmap v1028: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:32:44.738 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:44 smithi039 ceph-mon[32062]: pgmap v1028: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-04T08:32:44.896 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-07-04T08:32:44.896 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:44.896 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 568, 2024-07-04T08:32:44.896 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 612, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1122967, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 372308, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 20, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.102465014, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.047529973, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.062015434, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000244021, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 609, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.049986986, 2024-07-04T08:32:44.897 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000082080 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 242, 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000051860, 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000214 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1089536, 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 263504 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:44.898 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 914, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1355, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 237223, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 1428038, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 40, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.169156008, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.084698876, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.072681766, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.008602012, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1353, 2024-07-04T08:32:44.899 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.082374088, 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000060882 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1896, 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000446707, 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000235 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 726944 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.900 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 200, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 198, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 69377, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 286968, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 27, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.028588144, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.009265728, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.019782678, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000402680, 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:44.901 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 199, 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.040989921, 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000205979 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 321, 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000078786, 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000245 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.902 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 51, 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.001887367, 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000037007 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:44.903 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2, 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000026284, 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000013142 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 7, 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 4, 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4, 2024-07-04T08:32:44.904 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.001018135, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000254533 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 3, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 11, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 11, 2024-07-04T08:32:44.905 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 11, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 10, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 1, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 1, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 2, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 35, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 32, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 1, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:44.906 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 4, 2024-07-04T08:32:44.907 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 234, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 1, 2024-07-04T08:32:44.908 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 1, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 1 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-07-04T08:32:44.909 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 2, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:44.910 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 12, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 24, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-07-04T08:32:44.911 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 5, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 5, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 2, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 2, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-07-04T08:32:44.912 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 4195815, 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4, 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.009107432, 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002276858 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 14, 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 14, 2024-07-04T08:32:44.913 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 0, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 12, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 13, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 1, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 11, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 11, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 1, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 2, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 1, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 118216, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 223540 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.914 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 8, 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 7, 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 17, 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-07-04T08:32:44.915 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 4, 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.001018135, 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000254533 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.916 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-07-04T08:32:44.917 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.918 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.918 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.918 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-07-04T08:32:44.918 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.918 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.918 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.919 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-07-04T08:32:44.920 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.920 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.920 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.920 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.920 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-07-04T08:32:44.921 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.921 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.921 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.921 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.921 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-07-04T08:32:44.921 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.921 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.921 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.921 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-07-04T08:32:44.922 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.923 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:44.924 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.925 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.926 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.926 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:44.926 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.926 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.926 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.926 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.926 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.927 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.928 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.928 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.928 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-07-04T08:32:44.928 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.928 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.928 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.928 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-07-04T08:32:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:44.929 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 2, 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 1, 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-07-04T08:32:44.930 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 3, 2024-07-04T08:32:44.931 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 3, 2024-07-04T08:32:44.931 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 999, 2024-07-04T08:32:44.931 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:44.931 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.931 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-07-04T08:32:44.931 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:44.931 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:44.932 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:44.932 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:44.932 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:44.932 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:44.932 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:44.932 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:44.932 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:44.933 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:44.933 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:44.933 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:44.933 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:44.933 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:44.933 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:44.933 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:44.933 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:44.934 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 271767, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 91, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:44.935 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 9448, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 191, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 55440, 2024-07-04T08:32:44.936 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 214, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 40, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 11813, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 30, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 40, 2024-07-04T08:32:44.937 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 40, 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.433991843, 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.010849796 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 40, 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 62 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 40, 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 0, 2024-07-04T08:32:44.938 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 40, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 0, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 10, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 0, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 4, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 7, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 6, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-07-04T08:32:44.939 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 20, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:44.940 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-07-04T08:32:44.941 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 114, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 24, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 9, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 14, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 5, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-07-04T08:32:44.942 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 14, 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 0, 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 1 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 36 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-07-04T08:32:44.943 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 1678, 2024-07-04T08:32:44.944 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1251777, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 1678, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1678, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1251777, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.945 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 40, 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 11813, 2024-07-04T08:32:44.946 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 40, 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 11813, 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:44.947 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 40, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 40, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 40, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 40, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.948 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:44.949 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x559a7d104820": { 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 5, 2024-07-04T08:32:44.950 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1511, 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 5, 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4, 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1511, 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:44.951 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-07-04T08:32:45.688 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-07-04T08:32:45.688 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-07-04T08:32:45.688 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-07-04T08:32:45.688 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-07-04T08:32:45.689 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-07-04T08:32:45.689 DEBUG:tasks.check_counter:Getting stats from l 2024-07-04T08:32:45.689 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph daemon mds.l perf dump 2024-07-04T08:32:45.940 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.c/config 2024-07-04T08:32:47.551 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:47 smithi133 ceph-mon[37050]: pgmap v1029: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-04T08:32:47.556 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-07-04T08:32:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-07-04T08:32:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1304, 2024-07-04T08:32:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1304, 2024-07-04T08:32:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 439689, 2024-07-04T08:32:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 413849, 2024-07-04T08:32:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-07-04T08:32:47.556 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.215979703, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.090364263, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.049962055, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.057830357, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1303, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.091792768, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000070447 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.557 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2917, 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2909, 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1699048, 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 1036071, 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-07-04T08:32:47.558 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.305754210, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.143085470, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.094864930, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.063603548, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2908, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.171840228, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000059092 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.559 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1062512, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 241968 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 6, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 298, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1828, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 730581, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 8, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.022360361, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.016617891, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002499729, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000077281, 2024-07-04T08:32:47.560 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 295, 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.026740609, 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000090646 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 480, 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 729616 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-07-04T08:32:47.561 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 6398, 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.205857697, 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000032175 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-07-04T08:32:47.562 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.563 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-07-04T08:32:47.564 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 1, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-07-04T08:32:47.565 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-07-04T08:32:47.566 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.567 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-07-04T08:32:47.568 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-07-04T08:32:47.569 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 5, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 2, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-07-04T08:32:47.570 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 4195730, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 4195424, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 5 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 3, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 3, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 0, 2024-07-04T08:32:47.571 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 1, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 1, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 0, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 0, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 117176, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215348 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-07-04T08:32:47.572 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.574 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.575 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.576 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-07-04T08:32:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-07-04T08:32:47.577 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.578 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-07-04T08:32:47.579 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.581 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.583 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.584 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 1, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 2, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 1, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 1, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 2933, 2024-07-04T08:32:47.585 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-04T08:32:47.586 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-07-04T08:32:47.587 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-07-04T08:32:47.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 198225, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 56, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 9448, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 191, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-07-04T08:32:47.589 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 8512, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 13, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-07-04T08:32:47.590 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 3800, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 21, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 3, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 3800, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3800, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.738948584, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000720775 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3800, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3801 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.591 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 3800, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 3800, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 2492, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 1306, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-07-04T08:32:47.592 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 1, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-07-04T08:32:47.593 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-07-04T08:32:47.594 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 114, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 24, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 6, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 8, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 2, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-07-04T08:32:47.595 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-07-04T08:32:47.596 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 4223, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 1735942, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 4223, 2024-07-04T08:32:47.597 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 4223, 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1735942, 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-07-04T08:32:47.598 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 3800, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 1431, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3800, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1431, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:47.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 3800, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 3800, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3800, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3800, 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:47.600 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:47.601 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x555e27284c30": { 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-07-04T08:32:47.602 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-07-04T08:32:47.603 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-07-04T08:32:47.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:47 smithi050 ceph-mon[36577]: pgmap v1029: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-04T08:32:47.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:47 smithi039 ceph-mon[32062]: pgmap v1029: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-07-04T08:32:48.295 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-07-04T08:32:48.295 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-07-04T08:32:48.295 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-07-04T08:32:48.295 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-07-04T08:32:48.295 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-07-04T08:32:48.295 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-07-04T08:32:48.306 INFO:tasks.kclient:Unmounting kernel clients... 2024-07-04T08:32:48.306 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:32:48.307 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T08:32:48.372 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-07-04T08:32:48.373 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-04T08:32:48.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd blocklist ls 2024-07-04T08:32:48.551 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:48 smithi133 ceph-mon[37050]: pgmap v1030: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-04T08:32:48.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:48 smithi039 ceph-mon[32062]: pgmap v1030: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-04T08:32:48.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:48 smithi050 ceph-mon[36577]: pgmap v1030: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-07-04T08:32:48.896 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:32:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.144+0000 7f4113e00640 1 -- 172.21.15.39:0/1436985285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4114075650 msgr2=0x7f411410ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.144+0000 7f4113e00640 1 --2- 172.21.15.39:0/1436985285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4114075650 0x7f411410ef20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009860 tx=0x7f40fc02a340 comp rx=0 tx=0).stop 2024-07-04T08:32:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.144+0000 7f4113e00640 1 -- 172.21.15.39:0/1436985285 shutdown_connections 2024-07-04T08:32:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.144+0000 7f4113e00640 1 --2- 172.21.15.39:0/1436985285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f411410f460 0x7f41141118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.144+0000 7f4113e00640 1 --2- 172.21.15.39:0/1436985285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4114075650 0x7f411410ef20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:51.131 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.144+0000 7f4113e00640 1 --2- 172.21.15.39:0/1436985285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4114074d00 0x7f4114075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:51.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.144+0000 7f4113e00640 1 -- 172.21.15.39:0/1436985285 >> 172.21.15.39:0/1436985285 conn(0x7f41140fe990 msgr2=0x7f4114100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:51.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.145+0000 7f4113e00640 1 -- 172.21.15.39:0/1436985285 shutdown_connections 2024-07-04T08:32:51.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.145+0000 7f4113e00640 1 -- 172.21.15.39:0/1436985285 wait complete. 2024-07-04T08:32:51.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.146+0000 7f4113e00640 1 Processor -- start 2024-07-04T08:32:51.132 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.146+0000 7f4113e00640 1 -- start start 2024-07-04T08:32:51.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.147+0000 7f4113e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4114074d00 0x7f411419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:51.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.147+0000 7f4113e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4114075650 0x7f411419c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:51.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.147+0000 7f4113e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411410f460 0x7f41141a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:51.133 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.147+0000 7f4113e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f4114114fe0 con 0x7f4114074d00 2024-07-04T08:32:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.147+0000 7f4113e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4114114e60 con 0x7f411410f460 2024-07-04T08:32:51.134 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.147+0000 7f4113e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4114115160 con 0x7f4114075650 2024-07-04T08:32:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.147+0000 7f4112a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4114074d00 0x7f411419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.147+0000 7f4112000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4114075650 0x7f411419c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.147+0000 7f4112a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4114074d00 0x7f411419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45582/0 (socket says 172.21.15.39:45582) 2024-07-04T08:32:51.135 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.148+0000 7f4112a00640 1 -- 172.21.15.39:0/1087359722 learned_addr learned my addr 172.21.15.39:0/1087359722 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.148+0000 7f4113400640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411410f460 0x7f41141a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.148+0000 7f4112000640 1 -- 172.21.15.39:0/1087359722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411410f460 msgr2=0x7f41141a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.148+0000 7f4112000640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411410f460 0x7f41141a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.148+0000 7f4112000640 1 -- 172.21.15.39:0/1087359722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4114074d00 msgr2=0x7f411419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.148+0000 7f4112000640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4114074d00 0x7f411419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.148+0000 7f4112000640 1 -- 172.21.15.39:0/1087359722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41141a1a40 con 0x7f4114075650 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.148+0000 7f4112000640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4114075650 0x7f411419c070 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f40fc035040 tx=0x7f40fc032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.149+0000 7f410b400640 1 -- 172.21.15.39:0/1087359722 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc040020 con 0x7f4114075650 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.149+0000 7f410b400640 1 -- 172.21.15.39:0/1087359722 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40fc030e10 con 0x7f4114075650 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.149+0000 7f410b400640 1 -- 172.21.15.39:0/1087359722 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40fc037040 con 0x7f4114075650 2024-07-04T08:32:51.136 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.149+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41141a1cd0 con 0x7f4114075650 2024-07-04T08:32:51.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.149+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41141a8490 con 0x7f4114075650 2024-07-04T08:32:51.137 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.150+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4114075110 con 0x7f4114075650 2024-07-04T08:32:51.139 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.153+0000 7f410b400640 1 -- 172.21.15.39:0/1087359722 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f40fc043020 con 0x7f4114075650 2024-07-04T08:32:51.140 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.155+0000 7f410b400640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40f40b29e0 0x7f40f40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:51.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.155+0000 7f410b400640 1 -- 172.21.15.39:0/1087359722 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9424+0+0 (secure 0 0 0) 0x7f40fc102be0 con 0x7f4114075650 2024-07-04T08:32:51.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.155+0000 7f4112a00640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40f40b29e0 0x7f40f40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:51.141 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.156+0000 7f4112a00640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40f40b29e0 0x7f40f40b4ef0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4104009c90 tx=0x7f4104005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:51.142 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.156+0000 7f410b400640 1 -- 172.21.15.39:0/1087359722 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f40fc04a030 con 0x7f4114075650 2024-07-04T08:32:51.367 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.381+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f41141a8660 con 0x7f4114075650 2024-07-04T08:32:51.368 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.382+0000 7f410b400640 1 -- 172.21.15.39:0/1087359722 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v114) ==== 81+0+886 (secure 0 0 0) 0x7f40fc0ca410 con 0x7f4114075650 2024-07-04T08:32:51.368 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/2939871551 2024-07-05T07:59:57.104896+0000 2024-07-04T08:32:51.368 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/3207006577 2024-07-05T08:00:28.376822+0000 2024-07-04T08:32:51.368 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/2939871551 2024-07-05T07:59:57.104896+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1920202184 2024-07-05T07:59:57.104896+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/764812681 2024-07-05T07:59:57.104896+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/3823748890 2024-07-05T08:01:51.343953+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1949300646 2024-07-05T08:00:28.376822+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/2171986895 2024-07-05T07:59:57.104896+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1596793936 2024-07-05T08:00:28.376822+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/3207006577 2024-07-05T08:00:28.376822+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6801/1922588202 2024-07-05T08:01:51.343953+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1763496185 2024-07-05T08:01:51.343953+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/917994444 2024-07-05T08:00:28.376822+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:0/1237664844 2024-07-05T08:01:51.343953+0000 2024-07-04T08:32:51.369 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39:6800/1922588202 2024-07-05T08:01:51.343953+0000 2024-07-04T08:32:51.370 INFO:teuthology.orchestra.run.smithi039.stderr:listed 15 entries 2024-07-04T08:32:51.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40f40b29e0 msgr2=0x7f40f40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:51.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40f40b29e0 0x7f40f40b4ef0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4104009c90 tx=0x7f4104005f90 comp rx=0 tx=0).stop 2024-07-04T08:32:51.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4114075650 msgr2=0x7f411419c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:51.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4114075650 0x7f411419c070 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f40fc035040 tx=0x7f40fc032d50 comp rx=0 tx=0).stop 2024-07-04T08:32:51.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 shutdown_connections 2024-07-04T08:32:51.372 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f40f40b29e0 0x7f40f40b4ef0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:51.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411410f460 0x7f41141a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:51.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4114075650 0x7f411419c070 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:51.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 --2- 172.21.15.39:0/1087359722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4114074d00 0x7f411419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:51.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 >> 172.21.15.39:0/1087359722 conn(0x7f41140fe990 msgr2=0x7f4114109000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:51.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.386+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 shutdown_connections 2024-07-04T08:32:51.373 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:51.387+0000 7f4113e00640 1 -- 172.21.15.39:0/1087359722 wait complete. 2024-07-04T08:32:51.551 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:51 smithi133 ceph-mon[37050]: pgmap v1031: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:32:51.629 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:51 smithi050 ceph-mon[36577]: pgmap v1031: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:32:51.745 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:51 smithi039 ceph-mon[32062]: pgmap v1031: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-07-04T08:32:52.175 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-04T08:32:52.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:32:52.176 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-04T08:32:52.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T08:32:52.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:32:52.342 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-04T08:32:52.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:32:52.383 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2024-07-04T08:32:52.465 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-04T08:32:52.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:32:52.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-04T08:32:52.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:52 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1087359722' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-04T08:32:52.580 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:52 smithi039 ceph-mon[32062]: pgmap v1032: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:32:52.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-04T08:32:52.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2024-07-04T08:32:52.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:52 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1087359722' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-04T08:32:52.630 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:52 smithi050 ceph-mon[36577]: pgmap v1032: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:32:52.695 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-07-04T08:32:52.706 INFO:tasks.cephadm:Teardown begin 2024-07-04T08:32:52.706 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-04T08:32:52.707 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:32:52.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:52 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1087359722' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-04T08:32:52.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:52 smithi133 ceph-mon[37050]: pgmap v1032: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-04T08:32:53.185 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:32:54.510 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:54 smithi039 ceph-mon[32062]: pgmap v1033: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:32:54.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:54 smithi133 ceph-mon[37050]: pgmap v1033: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:32:54.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:54 smithi050 ceph-mon[36577]: pgmap v1033: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-04T08:32:55.404 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.418+0000 7fb3f7a00640 1 -- 172.21.15.39:0/3333908300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0108a10 msgr2=0x7fb3f0108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.418+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/3333908300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0108a10 0x7fb3f0108e20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009900 tx=0x7fb3dc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.418+0000 7fb3f7a00640 1 -- 172.21.15.39:0/3333908300 shutdown_connections 2024-07-04T08:32:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.418+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/3333908300 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0108a10 0x7fb3f0108e20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.418+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/3333908300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f0109360 0x7fb3f010fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.418+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/3333908300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3f0101530 0x7fb3f0101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.418+0000 7fb3f7a00640 1 -- 172.21.15.39:0/3333908300 >> 172.21.15.39:0/3333908300 conn(0x7fb3f0077110 msgr2=0x7fb3f0077550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.418+0000 7fb3f7a00640 1 -- 172.21.15.39:0/3333908300 shutdown_connections 2024-07-04T08:32:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.418+0000 7fb3f7a00640 1 -- 172.21.15.39:0/3333908300 wait complete. 2024-07-04T08:32:55.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.419+0000 7fb3f7a00640 1 Processor -- start 2024-07-04T08:32:55.407 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.419+0000 7fb3f7a00640 1 -- start start 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f7a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0101530 0x7fb3f0199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f7a00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3f0108a10 0x7fb3f019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f7a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f0109360 0x7fb3f019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f7a00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f0112fb0 con 0x7fb3f0101530 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f7a00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f0112e30 con 0x7fb3f0108a10 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f7a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f0113130 con 0x7fb3f0109360 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0101530 0x7fb3f0199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f5000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0101530 0x7fb3f0199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45610/0 (socket says 172.21.15.39:45610) 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f5000640 1 -- 172.21.15.39:0/1934793092 learned_addr learned my addr 172.21.15.39:0/1934793092 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3efe00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3f0108a10 0x7fb3f019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.420+0000 7fb3f5a00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f0109360 0x7fb3f019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:55.408 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.421+0000 7fb3f5000640 1 -- 172.21.15.39:0/1934793092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f0109360 msgr2=0x7fb3f019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.421+0000 7fb3f5000640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f0109360 0x7fb3f019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.421+0000 7fb3f5000640 1 -- 172.21.15.39:0/1934793092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3f0108a10 msgr2=0x7fb3f019a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.421+0000 7fb3f5000640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3f0108a10 0x7fb3f019a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.421+0000 7fb3f5000640 1 -- 172.21.15.39:0/1934793092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f019f9f0 con 0x7fb3f0101530 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.421+0000 7fb3f5000640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0101530 0x7fb3f0199a80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400ea60 tx=0x7fb3e400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.421+0000 7fb3ed600640 1 -- 172.21.15.39:0/1934793092 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e400cde0 con 0x7fb3f0101530 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.422+0000 7fb3ed600640 1 -- 172.21.15.39:0/1934793092 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e40035f0 con 0x7fb3f0101530 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.422+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f019fce0 con 0x7fb3f0101530 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.422+0000 7fb3ed600640 1 -- 172.21.15.39:0/1934793092 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4010690 con 0x7fb3f0101530 2024-07-04T08:32:55.409 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.422+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f01a6440 con 0x7fb3f0101530 2024-07-04T08:32:55.410 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.423+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f0101940 con 0x7fb3f0101530 2024-07-04T08:32:55.412 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.427+0000 7fb3ed600640 1 -- 172.21.15.39:0/1934793092 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7fb3e40037a0 con 0x7fb3f0101530 2024-07-04T08:32:55.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.428+0000 7fb3ed600640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3d80b2960 0x7fb3d80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:55.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.428+0000 7fb3ed600640 1 -- 172.21.15.39:0/1934793092 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9424+0+0 (secure 0 0 0) 0x7fb3e40e2c70 con 0x7fb3f0101530 2024-07-04T08:32:55.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.429+0000 7fb3efe00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3d80b2960 0x7fb3d80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:55.415 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.429+0000 7fb3efe00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3d80b2960 0x7fb3d80b4e70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009230 tx=0x7fb3dc031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:55.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.433+0000 7fb3ed600640 1 -- 172.21.15.39:0/1934793092 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7fb3e40aa290 con 0x7fb3f0101530 2024-07-04T08:32:55.674 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.688+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3f002de10 con 0x7fb3f0101530 2024-07-04T08:32:55.676 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.690+0000 7fb3ed600640 1 -- 172.21.15.39:0/1934793092 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7fb3e40a9b90 con 0x7fb3f0101530 2024-07-04T08:32:55.677 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:32:55.677 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:32:55.677 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:32:55.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3d80b2960 msgr2=0x7fb3d80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:55.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3d80b2960 0x7fb3d80b4e70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009230 tx=0x7fb3dc031400 comp rx=0 tx=0).stop 2024-07-04T08:32:55.679 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0101530 msgr2=0x7fb3f0199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:55.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0101530 0x7fb3f0199a80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400ea60 tx=0x7fb3e400ef20 comp rx=0 tx=0).stop 2024-07-04T08:32:55.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 shutdown_connections 2024-07-04T08:32:55.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7fb3d80b2960 0x7fb3d80b4e70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f0109360 0x7fb3f019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb3f0108a10 0x7fb3f019a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 --2- 172.21.15.39:0/1934793092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3f0101530 0x7fb3f0199a80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:55.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.693+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 >> 172.21.15.39:0/1934793092 conn(0x7fb3f0077110 msgr2=0x7fb3f0106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:55.680 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.694+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 shutdown_connections 2024-07-04T08:32:55.681 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:55.694+0000 7fb3f7a00640 1 -- 172.21.15.39:0/1934793092 wait complete. 2024-07-04T08:32:56.051 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:55 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1934793092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:32:56.064 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:55 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1934793092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:32:56.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:55 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1934793092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:32:56.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs dump --format=json 2024-07-04T08:32:56.900 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:32:57.051 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:56 smithi039 ceph-mon[32062]: pgmap v1034: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-04T08:32:57.051 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:56 smithi133 ceph-mon[37050]: pgmap v1034: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-04T08:32:57.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:56 smithi050 ceph-mon[36577]: pgmap v1034: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-04T08:32:58.688 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:58 smithi039 ceph-mon[32062]: pgmap v1035: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 596 B/s wr, 10 op/s 2024-07-04T08:32:58.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:58 smithi133 ceph-mon[37050]: pgmap v1035: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 596 B/s wr, 10 op/s 2024-07-04T08:32:58.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:58 smithi050 ceph-mon[36577]: pgmap v1035: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 596 B/s wr, 10 op/s 2024-07-04T08:32:59.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.077+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3919793171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a280768d0 msgr2=0x7f5a28074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:59.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.077+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3919793171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a280768d0 0x7f5a28074d00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009ca0 tx=0x7f5a1802a8c0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.077+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3919793171 shutdown_connections 2024-07-04T08:32:59.064 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.077+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3919793171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a28113990 0x7f5a28115e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.077+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3919793171 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a28075320 0x7f5a28075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.077+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3919793171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a280768d0 0x7f5a28074d00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.077+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3919793171 >> 172.21.15.39:0/3919793171 conn(0x7f5a280feb30 msgr2=0x7f5a28100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:59.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.077+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3919793171 shutdown_connections 2024-07-04T08:32:59.065 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.077+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3919793171 wait complete. 2024-07-04T08:32:59.066 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.079+0000 7f5a2fa00640 1 Processor -- start 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.079+0000 7f5a2fa00640 1 -- start start 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.079+0000 7f5a2fa00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a28075320 0x7f5a2819ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.079+0000 7f5a2fa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a280768d0 0x7f5a281a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.079+0000 7f5a2fa00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a28113990 0x7f5a281a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.079+0000 7f5a2fa00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28119630 con 0x7f5a28075320 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.079+0000 7f5a2fa00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281194b0 con 0x7f5a28113990 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.079+0000 7f5a2fa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281197b0 con 0x7f5a280768d0 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a28075320 0x7f5a2819ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a27e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a280768d0 0x7f5a281a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:59.067 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a2da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a28113990 0x7f5a281a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a2da00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a28113990 0x7f5a281a57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.39:45042/0 (socket says 172.21.15.39:45042) 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a27e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a280768d0 0x7f5a281a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:43312/0 (socket says 172.21.15.39:43312) 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a2da00640 1 -- 172.21.15.39:0/3387153257 learned_addr learned my addr 172.21.15.39:0/3387153257 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a2da00640 1 -- 172.21.15.39:0/3387153257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a280768d0 msgr2=0x7f5a281a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a2da00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a280768d0 0x7f5a281a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a2da00640 1 -- 172.21.15.39:0/3387153257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a28075320 msgr2=0x7f5a2819ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a2da00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a28075320 0x7f5a2819ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a2da00640 1 -- 172.21.15.39:0/3387153257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a281a5f10 con 0x7f5a28113990 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.080+0000 7f5a27e00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a280768d0 0x7f5a281a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:32:59.068 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.081+0000 7f5a2da00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a28113990 0x7f5a281a57e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000e970 tx=0x7f5a2000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:59.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.081+0000 7f5a25600640 1 -- 172.21.15.39:0/3387153257 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2000ccc0 con 0x7f5a28113990 2024-07-04T08:32:59.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.081+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a281a6200 con 0x7f5a28113990 2024-07-04T08:32:59.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.081+0000 7f5a25600640 1 -- 172.21.15.39:0/3387153257 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2000ce60 con 0x7f5a28113990 2024-07-04T08:32:59.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.081+0000 7f5a25600640 1 -- 172.21.15.39:0/3387153257 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20010430 con 0x7f5a28113990 2024-07-04T08:32:59.069 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.081+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a281ac960 con 0x7f5a28113990 2024-07-04T08:32:59.070 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.082+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a28075730 con 0x7f5a28113990 2024-07-04T08:32:59.071 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.085+0000 7f5a25600640 1 -- 172.21.15.39:0/3387153257 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5a20004f70 con 0x7f5a28113990 2024-07-04T08:32:59.072 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.087+0000 7f5a25600640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a0c0b2960 0x7f5a0c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:32:59.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.087+0000 7f5a25600640 1 -- 172.21.15.39:0/3387153257 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9424+0+0 (secure 0 0 0) 0x7f5a200e2f80 con 0x7f5a28113990 2024-07-04T08:32:59.073 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.087+0000 7f5a2d000640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a0c0b2960 0x7f5a0c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:32:59.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.088+0000 7f5a2d000640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a0c0b2960 0x7f5a0c0b4e70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5a18006010 tx=0x7f5a18009820 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:32:59.074 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.088+0000 7f5a25600640 1 -- 172.21.15.39:0/3387153257 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5a20055110 con 0x7f5a28113990 2024-07-04T08:32:59.325 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.339+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a281a10d0 con 0x7f5a28113990 2024-07-04T08:32:59.326 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.341+0000 7f5a25600640 1 -- 172.21.15.39:0/3387153257 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11975 (secure 0 0 0) 0x7f5a200aa5a0 con 0x7f5a28113990 2024-07-04T08:32:59.327 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:32:59.328 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":30,"btime":"2024-07-04T08:14:54:107186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14529,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1612340282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1612340282},{"type":"v1","addr":"172.21.15.39:6835","nonce":1612340282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6839/2419136306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6838","nonce":2419136306},{"type":"v1","addr":"172.21.15.39:6839","nonce":2419136306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-04T08:09:41.252924+0000","modified":"2024-07-04T08:14:54.008520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24707,"mds_1":24701,"mds_2":24679,"mds_3":24674,"mds_4":24673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24619":{"gid":24619,"name":"l","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2411107381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2411107381},{"type":"v1","addr":"172.21.15.133:6833","nonce":2411107381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24626":{"gid":24626,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6833/264080488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":264080488},{"type":"v1","addr":"172.21.15.50:6833","nonce":264080488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24644":{"gid":24644,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.39:6837/1240380045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6836","nonce":1240380045},{"type":"v1","addr":"172.21.15.39:6837","nonce":1240380045}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/2710764279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":2710764279},{"type":"v1","addr":"172.21.15.50:6835","nonce":2710764279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2187182943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2187182943},{"type":"v1","addr":"172.21.15.133:6835","nonce":2187182943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24673":{"gid":24673,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":32,"addr":"172.21.15.133:6837/431595690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":431595690},{"type":"v1","addr":"172.21.15.133:6837","nonce":431595690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24674":{"gid":24674,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":31,"addr":"172.21.15.50:6837/4268659351","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":4268659351},{"type":"v1","addr":"172.21.15.50:6837","nonce":4268659351}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":30,"addr":"172.21.15.133:6839/1264801316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":1264801316},{"type":"v1","addr":"172.21.15.133:6839","nonce":1264801316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":27,"addr":"172.21.15.39:6841/2753029904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6840","nonce":2753029904},{"type":"v1","addr":"172.21.15.39:6841","nonce":2753029904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.50:6839/2852157181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":2852157181},{"type":"v1","addr":"172.21.15.50:6839","nonce":2852157181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24707,"qdb_cluster":[24707,24701,24679,24674,24673]},"id":1}]} 2024-07-04T08:32:59.328 INFO:teuthology.orchestra.run.smithi039.stderr:dumped fsmap epoch 30 2024-07-04T08:32:59.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a0c0b2960 msgr2=0x7f5a0c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:59.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a0c0b2960 0x7f5a0c0b4e70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5a18006010 tx=0x7f5a18009820 comp rx=0 tx=0).stop 2024-07-04T08:32:59.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a28113990 msgr2=0x7f5a281a57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:32:59.330 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a28113990 0x7f5a281a57e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000e970 tx=0x7f5a2000ed70 comp rx=0 tx=0).stop 2024-07-04T08:32:59.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 shutdown_connections 2024-07-04T08:32:59.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5a0c0b2960 0x7f5a0c0b4e70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a28113990 0x7f5a281a57e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a280768d0 0x7f5a281a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 --2- 172.21.15.39:0/3387153257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5a28075320 0x7f5a2819ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:32:59.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.344+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 >> 172.21.15.39:0/3387153257 conn(0x7f5a280feb30 msgr2=0x7f5a2810f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:32:59.331 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.345+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 shutdown_connections 2024-07-04T08:32:59.332 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:32:59.345+0000 7f5a2fa00640 1 -- 172.21.15.39:0/3387153257 wait complete. 2024-07-04T08:32:59.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:32:59 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3387153257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:32:59.781 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:32:59 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3387153257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:32:59.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:32:59 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3387153257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-04T08:33:00.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd dump --format=json 2024-07-04T08:33:00.585 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:33:00.739 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:00 smithi039 ceph-mon[32062]: pgmap v1036: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-07-04T08:33:00.799 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:00 smithi050 ceph-mon[36577]: pgmap v1036: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-07-04T08:33:00.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:00 smithi133 ceph-mon[37050]: pgmap v1036: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-07-04T08:33:02.560 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:02 smithi039 ceph-mon[32062]: pgmap v1037: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 596 B/s wr, 11 op/s 2024-07-04T08:33:02.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:02 smithi133 ceph-mon[37050]: pgmap v1037: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 596 B/s wr, 11 op/s 2024-07-04T08:33:02.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:02 smithi050 ceph-mon[36577]: pgmap v1037: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 596 B/s wr, 11 op/s 2024-07-04T08:33:02.928 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.941+0000 7ff914e00640 1 -- 172.21.15.39:0/3775230225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff910108a20 msgr2=0x7ff910108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.941+0000 7ff914e00640 1 --2- 172.21.15.39:0/3775230225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff910108a20 0x7ff910108e30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff900009be0 tx=0x7ff90002a6c0 comp rx=0 tx=0).stop 2024-07-04T08:33:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.941+0000 7ff914e00640 1 -- 172.21.15.39:0/3775230225 shutdown_connections 2024-07-04T08:33:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.941+0000 7ff914e00640 1 --2- 172.21.15.39:0/3775230225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff910109370 0x7ff91010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.941+0000 7ff914e00640 1 --2- 172.21.15.39:0/3775230225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff910108a20 0x7ff910108e30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.941+0000 7ff914e00640 1 --2- 172.21.15.39:0/3775230225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9101030e0 0x7ff9101034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.941+0000 7ff914e00640 1 -- 172.21.15.39:0/3775230225 >> 172.21.15.39:0/3775230225 conn(0x7ff9100feb30 msgr2=0x7ff910100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.941+0000 7ff914e00640 1 -- 172.21.15.39:0/3775230225 shutdown_connections 2024-07-04T08:33:02.929 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.941+0000 7ff914e00640 1 -- 172.21.15.39:0/3775230225 wait complete. 2024-07-04T08:33:02.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.943+0000 7ff914e00640 1 Processor -- start 2024-07-04T08:33:02.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.943+0000 7ff914e00640 1 -- start start 2024-07-04T08:33:02.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.943+0000 7ff914e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9101030e0 0x7ff91019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:02.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.943+0000 7ff914e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff910108a20 0x7ff91019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:02.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.943+0000 7ff914e00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff910109370 0x7ff9101a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:02.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.943+0000 7ff914e00640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7ff910112fe0 con 0x7ff910108a20 2024-07-04T08:33:02.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.943+0000 7ff914e00640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff910112e60 con 0x7ff910109370 2024-07-04T08:33:02.930 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.943+0000 7ff914e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff910113160 con 0x7ff9101030e0 2024-07-04T08:33:02.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff910108a20 0x7ff91019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:02.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90d400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff910108a20 0x7ff91019c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:44924/0 (socket says 172.21.15.39:44924) 2024-07-04T08:33:02.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90d400640 1 -- 172.21.15.39:0/3171211036 learned_addr learned my addr 172.21.15.39:0/3171211036 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:33:02.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90de00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9101030e0 0x7ff91019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:02.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90e800640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff910109370 0x7ff9101a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:02.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90d400640 1 -- 172.21.15.39:0/3171211036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9101030e0 msgr2=0x7ff91019bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:02.932 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90d400640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9101030e0 0x7ff91019bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90d400640 1 -- 172.21.15.39:0/3171211036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff910109370 msgr2=0x7ff9101a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90d400640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff910109370 0x7ff9101a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90d400640 1 -- 172.21.15.39:0/3171211036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9101a1b60 con 0x7ff910108a20 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.944+0000 7ff90e800640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff910109370 0x7ff9101a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.945+0000 7ff90d400640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff910108a20 0x7ff91019c190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff90002acb0 tx=0x7ff900003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.945+0000 7ff906a00640 1 -- 172.21.15.39:0/3171211036 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900041020 con 0x7ff910108a20 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.945+0000 7ff906a00640 1 -- 172.21.15.39:0/3171211036 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff900003e10 con 0x7ff910108a20 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.945+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9101a1df0 con 0x7ff910108a20 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.945+0000 7ff906a00640 1 -- 172.21.15.39:0/3171211036 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900036040 con 0x7ff910108a20 2024-07-04T08:33:02.933 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.945+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101a85b0 con 0x7ff910108a20 2024-07-04T08:33:02.934 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.946+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9101034f0 con 0x7ff910108a20 2024-07-04T08:33:02.936 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.949+0000 7ff906a00640 1 -- 172.21.15.39:0/3171211036 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7ff900044020 con 0x7ff910108a20 2024-07-04T08:33:02.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.951+0000 7ff906a00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8f40b2960 0x7ff8f40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:02.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.951+0000 7ff906a00640 1 -- 172.21.15.39:0/3171211036 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9424+0+0 (secure 0 0 0) 0x7ff900103a90 con 0x7ff910108a20 2024-07-04T08:33:02.938 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.951+0000 7ff90de00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8f40b2960 0x7ff8f40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:02.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.952+0000 7ff90de00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8f40b2960 0x7ff8f40b4e70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8005fd0 tx=0x7ff8f8009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:02.939 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:02.953+0000 7ff906a00640 1 -- 172.21.15.39:0/3171211036 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7ff90004b030 con 0x7ff910108a20 2024-07-04T08:33:03.154 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.167+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9101a8850 con 0x7ff910108a20 2024-07-04T08:33:03.157 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.170+0000 7ff906a00640 1 -- 172.21.15.39:0/3171211036 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+25145 (secure 0 0 0) 0x7ff9000cb0b0 con 0x7ff910108a20 2024-07-04T08:33:03.157 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-07-04T08:33:03.158 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":114,"fsid":"2ee561ca-39db-11ef-bcab-c7b262605968","created":"2024-07-04T07:59:21.010226+0000","modified":"2024-07-04T08:32:13.839822+0000","last_up_change":"2024-07-04T08:08:30.639762+0000","last_in_change":"2024-07-04T08:08:10.861992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-04T08:04:17.509668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-04T08:09:32.643834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-04T08:09:37.246918+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-04T08:09:49.616671+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"cfc7b735-e66c-4ace-958a-f47b6df89f37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6803","nonce":1924112753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6805","nonce":1924112753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6809","nonce":1924112753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":1924112753},{"type":"v1","addr":"172.21.15.39:6807","nonce":1924112753}]},"public_addr":"172.21.15.39:6803/1924112753","cluster_addr":"172.21.15.39:6805/1924112753","heartbeat_back_addr":"172.21.15.39:6809/1924112753","heartbeat_front_addr":"172.21.15.39:6807/1924112753","state":["exists","up"]},{"osd":1,"uuid":"de735138-774d-4816-b14f-4ad189b8b79d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6801","nonce":735117817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6803","nonce":735117817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6807","nonce":735117817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":735117817},{"type":"v1","addr":"172.21.15.50:6805","nonce":735117817}]},"public_addr":"172.21.15.50:6801/735117817","cluster_addr":"172.21.15.50:6803/735117817","heartbeat_back_addr":"172.21.15.50:6807/735117817","heartbeat_front_addr":"172.21.15.50:6805/735117817","state":["exists","up"]},{"osd":2,"uuid":"ad4f9bb6-fc12-476f-a359-b137919ed63d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6801","nonce":3737341609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6803","nonce":3737341609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6807","nonce":3737341609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3737341609},{"type":"v1","addr":"172.21.15.133:6805","nonce":3737341609}]},"public_addr":"172.21.15.133:6801/3737341609","cluster_addr":"172.21.15.133:6803/3737341609","heartbeat_back_addr":"172.21.15.133:6807/3737341609","heartbeat_front_addr":"172.21.15.133:6805/3737341609","state":["exists","up"]},{"osd":3,"uuid":"7accf662-69bc-4e7d-a7bf-c174c683367a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6811","nonce":4257278793}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6813","nonce":4257278793}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6817","nonce":4257278793}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":4257278793},{"type":"v1","addr":"172.21.15.39:6815","nonce":4257278793}]},"public_addr":"172.21.15.39:6811/4257278793","cluster_addr":"172.21.15.39:6813/4257278793","heartbeat_back_addr":"172.21.15.39:6817/4257278793","heartbeat_front_addr":"172.21.15.39:6815/4257278793","state":["exists","up"]},{"osd":4,"uuid":"9db0ba72-e313-4a94-a5d0-ff30e4e8028f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6809","nonce":1784161771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6811","nonce":1784161771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6815","nonce":1784161771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1784161771},{"type":"v1","addr":"172.21.15.50:6813","nonce":1784161771}]},"public_addr":"172.21.15.50:6809/1784161771","cluster_addr":"172.21.15.50:6811/1784161771","heartbeat_back_addr":"172.21.15.50:6815/1784161771","heartbeat_front_addr":"172.21.15.50:6813/1784161771","state":["exists","up"]},{"osd":5,"uuid":"5a0b6a8f-16d6-4018-a67a-c654f6deb5dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6809","nonce":787209680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6811","nonce":787209680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6815","nonce":787209680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":787209680},{"type":"v1","addr":"172.21.15.133:6813","nonce":787209680}]},"public_addr":"172.21.15.133:6809/787209680","cluster_addr":"172.21.15.133:6811/787209680","heartbeat_back_addr":"172.21.15.133:6815/787209680","heartbeat_front_addr":"172.21.15.133:6813/787209680","state":["exists","up"]},{"osd":6,"uuid":"fae5d481-b487-4e26-ae87-395fa655837c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6819","nonce":636897432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6821","nonce":636897432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6825","nonce":636897432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":636897432},{"type":"v1","addr":"172.21.15.39:6823","nonce":636897432}]},"public_addr":"172.21.15.39:6819/636897432","cluster_addr":"172.21.15.39:6821/636897432","heartbeat_back_addr":"172.21.15.39:6825/636897432","heartbeat_front_addr":"172.21.15.39:6823/636897432","state":["exists","up"]},{"osd":7,"uuid":"8ff9a0e2-4610-450d-be51-0726c8bc14ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6817","nonce":1720421182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6819","nonce":1720421182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6823","nonce":1720421182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1720421182},{"type":"v1","addr":"172.21.15.50:6821","nonce":1720421182}]},"public_addr":"172.21.15.50:6817/1720421182","cluster_addr":"172.21.15.50:6819/1720421182","heartbeat_back_addr":"172.21.15.50:6823/1720421182","heartbeat_front_addr":"172.21.15.50:6821/1720421182","state":["exists","up"]},{"osd":8,"uuid":"a1bdae27-2643-4838-ba2b-080777c325e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6817","nonce":341977020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6819","nonce":341977020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6823","nonce":341977020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":341977020},{"type":"v1","addr":"172.21.15.133:6821","nonce":341977020}]},"public_addr":"172.21.15.133:6817/341977020","cluster_addr":"172.21.15.133:6819/341977020","heartbeat_back_addr":"172.21.15.133:6823/341977020","heartbeat_front_addr":"172.21.15.133:6821/341977020","state":["exists","up"]},{"osd":9,"uuid":"005c7632-7a25-4d09-9d4b-59675ba2cf70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6827","nonce":2701785798}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6829","nonce":2701785798}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6833","nonce":2701785798}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":2701785798},{"type":"v1","addr":"172.21.15.39:6831","nonce":2701785798}]},"public_addr":"172.21.15.39:6827/2701785798","cluster_addr":"172.21.15.39:6829/2701785798","heartbeat_back_addr":"172.21.15.39:6833/2701785798","heartbeat_front_addr":"172.21.15.39:6831/2701785798","state":["exists","up"]},{"osd":10,"uuid":"191d4999-048f-4bd9-b484-e20564492846","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6825","nonce":2876845435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6827","nonce":2876845435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6831","nonce":2876845435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":2876845435},{"type":"v1","addr":"172.21.15.50:6829","nonce":2876845435}]},"public_addr":"172.21.15.50:6825/2876845435","cluster_addr":"172.21.15.50:6827/2876845435","heartbeat_back_addr":"172.21.15.50:6831/2876845435","heartbeat_front_addr":"172.21.15.50:6829/2876845435","state":["exists","up"]},{"osd":11,"uuid":"a3dddd70-6e8a-424f-a69d-7c2b41575aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6825","nonce":3104157254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6827","nonce":3104157254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6831","nonce":3104157254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":3104157254},{"type":"v1","addr":"172.21.15.133:6829","nonce":3104157254}]},"public_addr":"172.21.15.133:6825/3104157254","cluster_addr":"172.21.15.133:6827/3104157254","heartbeat_back_addr":"172.21.15.133:6831/3104157254","heartbeat_front_addr":"172.21.15.133:6829/3104157254","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:26.642346+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:03:51.139682+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:13.496856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:04:47.585195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:13.263325+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:05:37.020868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:10.723197+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:06:38.419605+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:02.914510+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:07:37.552443+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:03.911266+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-04T08:08:28.401177+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.15","mappings":[{"from":5,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.39:6800/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:6800/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/2939871551":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1920202184":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/764812681":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/3823748890":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1949300646":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/2171986895":"2024-07-05T07:59:57.104896+0000","172.21.15.39:0/1596793936":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/3207006577":"2024-07-05T08:00:28.376822+0000","172.21.15.39:6801/1922588202":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/1763496185":"2024-07-05T08:01:51.343953+0000","172.21.15.39:0/917994444":"2024-07-05T08:00:28.376822+0000","172.21.15.39:0/1237664844":"2024-07-05T08:01:51.343953+0000","172.21.15.39:6800/1922588202":"2024-07-05T08:01:51.343953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]},{"pool":4,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-04T08:33:03.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8f40b2960 msgr2=0x7ff8f40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:03.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8f40b2960 0x7ff8f40b4e70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8005fd0 tx=0x7ff8f8009260 comp rx=0 tx=0).stop 2024-07-04T08:33:03.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff910108a20 msgr2=0x7ff91019c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:03.160 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff910108a20 0x7ff91019c190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff90002acb0 tx=0x7ff900003c20 comp rx=0 tx=0).stop 2024-07-04T08:33:03.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 shutdown_connections 2024-07-04T08:33:03.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7ff8f40b2960 0x7ff8f40b4e70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:03.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff910108a20 0x7ff91019c190 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:03.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff910109370 0x7ff9101a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:03.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 --2- 172.21.15.39:0/3171211036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9101030e0 0x7ff91019bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:03.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.173+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 >> 172.21.15.39:0/3171211036 conn(0x7ff9100feb30 msgr2=0x7ff9101080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:03.161 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.174+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 shutdown_connections 2024-07-04T08:33:03.162 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:03.174+0000 7ff914e00640 1 -- 172.21.15.39:0/3171211036 wait complete. 2024-07-04T08:33:03.545 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:03 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3171211036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:33:03.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:03 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3171211036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:33:03.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:03 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3171211036' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-04T08:33:03.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs fail cephfs 2024-07-04T08:33:04.491 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:33:04.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:04 smithi133 ceph-mon[37050]: pgmap v1038: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 596 B/s wr, 10 op/s 2024-07-04T08:33:04.811 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:04 smithi039 ceph-mon[32062]: pgmap v1038: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 596 B/s wr, 10 op/s 2024-07-04T08:33:04.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:04 smithi050 ceph-mon[36577]: pgmap v1038: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 596 B/s wr, 10 op/s 2024-07-04T08:33:06.703 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:06 smithi039 ceph-mon[32062]: pgmap v1039: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-07-04T08:33:06.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.715+0000 7f2d19800640 1 -- 172.21.15.39:0/3462072111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d1410ac40 msgr2=0x7f2d1410b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.715+0000 7f2d19800640 1 --2- 172.21.15.39:0/3462072111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d1410ac40 0x7f2d1410b050 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009900 tx=0x7f2cfc02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.715+0000 7f2d19800640 1 -- 172.21.15.39:0/3462072111 shutdown_connections 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.715+0000 7f2d19800640 1 --2- 172.21.15.39:0/3462072111 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d1410b590 0x7f2d14111dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.715+0000 7f2d19800640 1 --2- 172.21.15.39:0/3462072111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d1410ac40 0x7f2d1410b050 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.715+0000 7f2d19800640 1 --2- 172.21.15.39:0/3462072111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d14105300 0x7f2d14105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.715+0000 7f2d19800640 1 -- 172.21.15.39:0/3462072111 >> 172.21.15.39:0/3462072111 conn(0x7f2d14100d10 msgr2=0x7f2d14103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.715+0000 7f2d19800640 1 -- 172.21.15.39:0/3462072111 shutdown_connections 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.715+0000 7f2d19800640 1 -- 172.21.15.39:0/3462072111 wait complete. 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.717+0000 7f2d19800640 1 Processor -- start 2024-07-04T08:33:06.704 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.717+0000 7f2d19800640 1 -- start start 2024-07-04T08:33:06.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d19800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d14105300 0x7f2d1419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d19800640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1410ac40 0x7f2d1419c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d19800640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d1410b590 0x7f2d141a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d19800640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14115220 con 0x7f2d1410ac40 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d19800640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141150a0 con 0x7f2d1410b590 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d19800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141153a0 con 0x7f2d14105300 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d12a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d14105300 0x7f2d1419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d12000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1410ac40 0x7f2d1419c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d13400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d1410b590 0x7f2d141a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d12a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d14105300 0x7f2d1419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.39:42590/0 (socket says 172.21.15.39:42590) 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d12a00640 1 -- 172.21.15.39:0/3399327667 learned_addr learned my addr 172.21.15.39:0/3399327667 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:33:06.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d13400640 1 -- 172.21.15.39:0/3399327667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d14105300 msgr2=0x7f2d1419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:06.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d13400640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d14105300 0x7f2d1419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d13400640 1 -- 172.21.15.39:0/3399327667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1410ac40 msgr2=0x7f2d1419c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:06.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d13400640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1410ac40 0x7f2d1419c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.718+0000 7f2d13400640 1 -- 172.21.15.39:0/3399327667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d141a1ba0 con 0x7f2d1410b590 2024-07-04T08:33:06.707 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.719+0000 7f2d13400640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d1410b590 0x7f2d141a1470 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009fd0 tx=0x7f2d0c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:06.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.719+0000 7f2d0b400640 1 -- 172.21.15.39:0/3399327667 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00ce20 con 0x7f2d1410b590 2024-07-04T08:33:06.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.719+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d141a1e90 con 0x7f2d1410b590 2024-07-04T08:33:06.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.719+0000 7f2d0b400640 1 -- 172.21.15.39:0/3399327667 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c005020 con 0x7f2d1410b590 2024-07-04T08:33:06.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.719+0000 7f2d0b400640 1 -- 172.21.15.39:0/3399327667 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c010690 con 0x7f2d1410b590 2024-07-04T08:33:06.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.719+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d141a85f0 con 0x7f2d1410b590 2024-07-04T08:33:06.708 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.721+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d14105710 con 0x7f2d1410b590 2024-07-04T08:33:06.710 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.723+0000 7f2d0b400640 1 -- 172.21.15.39:0/3399327667 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f2d0c0054d0 con 0x7f2d1410b590 2024-07-04T08:33:06.711 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.725+0000 7f2d0b400640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2cf80b29e0 0x7f2cf80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:06.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.725+0000 7f2d0b400640 1 -- 172.21.15.39:0/3399327667 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9424+0+0 (secure 0 0 0) 0x7f2d0c0e2f60 con 0x7f2d1410b590 2024-07-04T08:33:06.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.725+0000 7f2d12a00640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2cf80b29e0 0x7f2cf80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:06.712 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.726+0000 7f2d12a00640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2cf80b29e0 0x7f2cf80b4ef0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2d04005fd0 tx=0x7f2d0400b4a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:06.713 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.726+0000 7f2d0b400640 1 -- 172.21.15.39:0/3399327667 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f2d0c0aa580 con 0x7f2d1410b590 2024-07-04T08:33:06.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:06 smithi133 ceph-mon[37050]: pgmap v1039: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-07-04T08:33:06.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:06 smithi050 ceph-mon[36577]: pgmap v1039: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-07-04T08:33:06.968 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.981+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2d1419cd60 con 0x7f2d1410b590 2024-07-04T08:33:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.995+0000 7f2d0b400640 1 -- 172.21.15.39:0/3399327667 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7f2d1419cd60 con 0x7f2d1410b590 2024-07-04T08:33:06.982 INFO:teuthology.orchestra.run.smithi039.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-04T08:33:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.998+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2cf80b29e0 msgr2=0x7f2cf80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.998+0000 7f2d19800640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2cf80b29e0 0x7f2cf80b4ef0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2d04005fd0 tx=0x7f2d0400b4a0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d1410b590 msgr2=0x7f2d141a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d1410b590 0x7f2d141a1470 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009fd0 tx=0x7f2d0c00ef10 comp rx=0 tx=0).stop 2024-07-04T08:33:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 shutdown_connections 2024-07-04T08:33:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f2cf80b29e0 0x7f2cf80b4ef0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d1410ac40 0x7f2d1419c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.986 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d1410b590 0x7f2d141a1470 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 --2- 172.21.15.39:0/3399327667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d14105300 0x7f2d1419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:06.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 >> 172.21.15.39:0/3399327667 conn(0x7f2d14100d10 msgr2=0x7f2d14104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:06.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 shutdown_connections 2024-07-04T08:33:06.987 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:06.999+0000 7f2d19800640 1 -- 172.21.15.39:0/3399327667 wait complete. 2024-07-04T08:33:07.301 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l[68060]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.301 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l[68060]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.302 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l[68060]: starting mds.l at 2024-07-04T08:33:07.302 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c[68662]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.302 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c[68662]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.302 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mds[68666]: -- 172.21.15.133:0/2516942361 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585e60ab860 con 0x5585ea25d180 2024-07-04T08:33:07.302 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c[68662]: starting mds.c at 2024-07-04T08:33:07.303 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f[68363]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.303 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f[68363]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.303 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mds[68367]: -- 172.21.15.133:0/1751396500 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56154aa9f520 con 0x56154ec47180 2024-07-04T08:33:07.303 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mds[68367]: -- 172.21.15.133:0/1751396500 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56154aa95860 con 0x56154ec47180 2024-07-04T08:33:07.303 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f[68363]: starting mds.f at 2024-07-04T08:33:07.303 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i[68974]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.303 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i[68974]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.303 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mds[68978]: -- 172.21.15.133:0/2161819629 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562bb20dbc20 con 0x562bb628da80 2024-07-04T08:33:07.303 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i[68974]: starting mds.i at 2024-07-04T08:33:07.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:06 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a[32058]: 2024-07-04T08:33:06.984+0000 7f2da3a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T08:33:07.337 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d[77659]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.337 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d[77659]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.337 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-mds[77663]: -- 172.21.15.39:0/2328202149 <== mon.0 v2:172.21.15.39:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f53d43860 con 0x556f57ef5180 2024-07-04T08:33:07.337 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d[77659]: starting mds.d at 2024-07-04T08:33:07.337 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a[78830]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.337 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a[78830]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.338 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-mds[78834]: -- 172.21.15.39:0/250079297 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff327db860 con 0x55ff3698d600 2024-07-04T08:33:07.338 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a[78830]: starting mds.a at 2024-07-04T08:33:07.380 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h[68472]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.380 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h[68472]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.380 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mds[68476]: -- 172.21.15.50:0/3499565216 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c9b6381860 con 0x55c9ba533600 2024-07-04T08:33:07.380 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h[68472]: starting mds.h at 2024-07-04T08:33:07.380 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b[68787]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.381 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b[68787]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.381 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mds[68791]: -- 172.21.15.50:0/4054723270 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e064c2f860 con 0x55e068de1180 2024-07-04T08:33:07.381 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b[68787]: starting mds.b at 2024-07-04T08:33:07.381 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e[68162]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.381 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e[68162]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.381 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mds[68166]: -- 172.21.15.50:0/2046884078 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b3ac9e5520 con 0x55b3b0b8da80 2024-07-04T08:33:07.381 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mds[68166]: -- 172.21.15.50:0/2046884078 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3ac9db860 con 0x55b3b0b8da80 2024-07-04T08:33:07.381 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e[68162]: starting mds.e at 2024-07-04T08:33:07.382 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k[67846]: ignoring --setuser ceph since I am not root 2024-07-04T08:33:07.382 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k[67846]: ignoring --setgroup ceph since I am not root 2024-07-04T08:33:07.382 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mds[67850]: -- 172.21.15.50:0/499651302 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56201b1db860 con 0x56201f38d180 2024-07-04T08:33:07.382 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k[67846]: starting mds.k at 2024-07-04T08:33:07.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/3399327667' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-04T08:33:07.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-04T08:33:07.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-mon[32062]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-04T08:33:07.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-mon[32062]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T08:33:07.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-04T08:33:07.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-mon[32062]: osdmap e115: 12 total, 12 up, 12 in 2024-07-04T08:33:07.689 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:07 smithi039 ceph-mon[32062]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-04T08:33:07.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-07-04T08:33:07.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/3399327667' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-04T08:33:07.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-04T08:33:07.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mon[37050]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-04T08:33:07.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mon[37050]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T08:33:07.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-04T08:33:07.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mon[37050]: osdmap e115: 12 total, 12 up, 12 in 2024-07-04T08:33:07.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:07 smithi133 ceph-mon[37050]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-04T08:33:07.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/3399327667' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-04T08:33:07.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-04T08:33:07.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mon[36577]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-04T08:33:07.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mon[36577]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-04T08:33:07.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-04T08:33:07.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mon[36577]: osdmap e115: 12 total, 12 up, 12 in 2024-07-04T08:33:07.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:07 smithi050 ceph-mon[36577]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-04T08:33:08.258 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:33:09.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6832/683058915,v1:172.21.15.133:6833/683058915] up:boot 2024-07-04T08:33:09.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6834/3134469084,v1:172.21.15.133:6835/3134469084] up:boot 2024-07-04T08:33:09.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6836/19015856,v1:172.21.15.133:6838/19015856] up:boot 2024-07-04T08:33:09.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.133:6837/4134856797,v1:172.21.15.133:6839/4134856797] up:boot 2024-07-04T08:33:09.088 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6832/1611393454,v1:172.21.15.50:6834/1611393454] up:boot 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6833/3424190372,v1:172.21.15.50:6835/3424190372] up:boot 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6836/2719900797,v1:172.21.15.50:6838/2719900797] up:boot 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.50:6837/2598473821,v1:172.21.15.50:6839/2598473821] up:boot 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.39:6836/2063996395,v1:172.21.15.39:6837/2063996395] up:boot 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: mds.? [v2:172.21.15.39:6840/3047500733,v1:172.21.15.39:6841/3047500733] up:boot 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-04T08:33:09.089 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-04T08:33:09.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-04T08:33:09.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-04T08:33:09.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-04T08:33:09.090 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:09 smithi039 ceph-mon[32062]: pgmap v1041: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-07-04T08:33:09.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6832/683058915,v1:172.21.15.133:6833/683058915] up:boot 2024-07-04T08:33:09.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6834/3134469084,v1:172.21.15.133:6835/3134469084] up:boot 2024-07-04T08:33:09.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6836/19015856,v1:172.21.15.133:6838/19015856] up:boot 2024-07-04T08:33:09.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.133:6837/4134856797,v1:172.21.15.133:6839/4134856797] up:boot 2024-07-04T08:33:09.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6832/1611393454,v1:172.21.15.50:6834/1611393454] up:boot 2024-07-04T08:33:09.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6833/3424190372,v1:172.21.15.50:6835/3424190372] up:boot 2024-07-04T08:33:09.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6836/2719900797,v1:172.21.15.50:6838/2719900797] up:boot 2024-07-04T08:33:09.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.50:6837/2598473821,v1:172.21.15.50:6839/2598473821] up:boot 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.39:6836/2063996395,v1:172.21.15.39:6837/2063996395] up:boot 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: mds.? [v2:172.21.15.39:6840/3047500733,v1:172.21.15.39:6841/3047500733] up:boot 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-04T08:33:09.303 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-04T08:33:09.304 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:09 smithi133 ceph-mon[37050]: pgmap v1041: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-07-04T08:33:09.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6832/683058915,v1:172.21.15.133:6833/683058915] up:boot 2024-07-04T08:33:09.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6834/3134469084,v1:172.21.15.133:6835/3134469084] up:boot 2024-07-04T08:33:09.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6836/19015856,v1:172.21.15.133:6838/19015856] up:boot 2024-07-04T08:33:09.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.133:6837/4134856797,v1:172.21.15.133:6839/4134856797] up:boot 2024-07-04T08:33:09.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6832/1611393454,v1:172.21.15.50:6834/1611393454] up:boot 2024-07-04T08:33:09.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6833/3424190372,v1:172.21.15.50:6835/3424190372] up:boot 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6836/2719900797,v1:172.21.15.50:6838/2719900797] up:boot 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.50:6837/2598473821,v1:172.21.15.50:6839/2598473821] up:boot 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.39:6836/2063996395,v1:172.21.15.39:6837/2063996395] up:boot 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: mds.? [v2:172.21.15.39:6840/3047500733,v1:172.21.15.39:6841/3047500733] up:boot 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-04T08:33:09.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-04T08:33:09.382 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-04T08:33:09.382 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-04T08:33:09.382 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-04T08:33:09.382 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-04T08:33:09.382 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:09 smithi050 ceph-mon[36577]: pgmap v1041: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-07-04T08:33:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.594+0000 7f6f53400640 1 -- 172.21.15.39:0/850532186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c10b570 msgr2=0x7f6f4c111da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.594+0000 7f6f53400640 1 --2- 172.21.15.39:0/850532186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c10b570 0x7f6f4c111da0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009860 tx=0x7f6f3c02a340 comp rx=0 tx=0).stop 2024-07-04T08:33:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.594+0000 7f6f53400640 1 -- 172.21.15.39:0/850532186 shutdown_connections 2024-07-04T08:33:10.581 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.594+0000 7f6f53400640 1 --2- 172.21.15.39:0/850532186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c10b570 0x7f6f4c111da0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.594+0000 7f6f53400640 1 --2- 172.21.15.39:0/850532186 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c10ac20 0x7f6f4c10b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.594+0000 7f6f53400640 1 --2- 172.21.15.39:0/850532186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f4c1052e0 0x7f6f4c1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.594+0000 7f6f53400640 1 -- 172.21.15.39:0/850532186 >> 172.21.15.39:0/850532186 conn(0x7f6f4c100db0 msgr2=0x7f6f4c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.594+0000 7f6f53400640 1 -- 172.21.15.39:0/850532186 shutdown_connections 2024-07-04T08:33:10.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.594+0000 7f6f53400640 1 -- 172.21.15.39:0/850532186 wait complete. 2024-07-04T08:33:10.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.596+0000 7f6f53400640 1 Processor -- start 2024-07-04T08:33:10.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.596+0000 7f6f53400640 1 -- start start 2024-07-04T08:33:10.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f53400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c1052e0 0x7f6f4c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:10.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f53400640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c10ac20 0x7f6f4c19c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:10.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f53400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f4c10b570 0x7f6f4c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:10.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f53400640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c115220 con 0x7f6f4c10b570 2024-07-04T08:33:10.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f53400640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1150a0 con 0x7f6f4c10ac20 2024-07-04T08:33:10.584 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f53400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1153a0 con 0x7f6f4c1052e0 2024-07-04T08:33:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f51400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f4c10b570 0x7f6f4c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f51400640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f4c10b570 0x7f6f4c1a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:45000/0 (socket says 172.21.15.39:45000) 2024-07-04T08:33:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f4be00640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c10ac20 0x7f6f4c19c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.597+0000 7f6f51400640 1 -- 172.21.15.39:0/580782958 learned_addr learned my addr 172.21.15.39:0/580782958 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:33:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.598+0000 7f6f50a00640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c1052e0 0x7f6f4c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.598+0000 7f6f51400640 1 -- 172.21.15.39:0/580782958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c1052e0 msgr2=0x7f6f4c19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:10.585 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.598+0000 7f6f51400640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c1052e0 0x7f6f4c19bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:10.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.598+0000 7f6f51400640 1 -- 172.21.15.39:0/580782958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c10ac20 msgr2=0x7f6f4c19c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:10.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.598+0000 7f6f51400640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c10ac20 0x7f6f4c19c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:10.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.598+0000 7f6f51400640 1 -- 172.21.15.39:0/580782958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c1a1b40 con 0x7f6f4c10b570 2024-07-04T08:33:10.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.598+0000 7f6f51400640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f4c10b570 0x7f6f4c1a1410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c02a930 tx=0x7f6f3c032d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:10.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.599+0000 7f6f49600640 1 -- 172.21.15.39:0/580782958 <== mon.0 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c042070 con 0x7f6f4c10b570 2024-07-04T08:33:10.586 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.599+0000 7f6f49600640 1 -- 172.21.15.39:0/580782958 <== mon.0 v2:172.21.15.39:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f3c030d50 con 0x7f6f4c10b570 2024-07-04T08:33:10.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.599+0000 7f6f49600640 1 -- 172.21.15.39:0/580782958 <== mon.0 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c037040 con 0x7f6f4c10b570 2024-07-04T08:33:10.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.599+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c1a1dd0 con 0x7f6f4c10b570 2024-07-04T08:33:10.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.599+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c1a8590 con 0x7f6f4c10b570 2024-07-04T08:33:10.587 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.600+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c1056f0 con 0x7f6f4c10b570 2024-07-04T08:33:10.590 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.604+0000 7f6f49600640 1 -- 172.21.15.39:0/580782958 <== mon.0 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f6f3c045070 con 0x7f6f4c10b570 2024-07-04T08:33:10.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.605+0000 7f6f49600640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6f300b2960 0x7f6f300b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:10.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.605+0000 7f6f49600640 1 -- 172.21.15.39:0/580782958 <== mon.0 v2:172.21.15.39:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9766+0+0 (secure 0 0 0) 0x7f6f3c1023a0 con 0x7f6f4c10b570 2024-07-04T08:33:10.592 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.606+0000 7f6f50a00640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6f300b2960 0x7f6f300b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:10.593 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.606+0000 7f6f50a00640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6f300b2960 0x7f6f300b4e70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009530 tx=0x7f6f44009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:10.594 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.608+0000 7f6f49600640 1 -- 172.21.15.39:0/580782958 <== mon.0 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f6f3c04c080 con 0x7f6f4c10b570 2024-07-04T08:33:10.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:10 smithi133 ceph-mon[37050]: pgmap v1042: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-07-04T08:33:10.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:10 smithi039 ceph-mon[32062]: pgmap v1042: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-07-04T08:33:10.851 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:10.864+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6f4c10d360 con 0x7f6f4c10b570 2024-07-04T08:33:10.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:10 smithi050 ceph-mon[36577]: pgmap v1042: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 29 op/s 2024-07-04T08:33:11.418 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.430+0000 7f6f49600640 1 -- 172.21.15.39:0/580782958 <== mon.0 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7f6f3c0c94a0 con 0x7f6f4c10b570 2024-07-04T08:33:11.421 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6f300b2960 msgr2=0x7f6f300b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6f300b2960 0x7f6f300b4e70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009530 tx=0x7f6f44009260 comp rx=0 tx=0).stop 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f4c10b570 msgr2=0x7f6f4c1a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f4c10b570 0x7f6f4c1a1410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c02a930 tx=0x7f6f3c032d40 comp rx=0 tx=0).stop 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 shutdown_connections 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f6f300b2960 0x7f6f300b4e70 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f4c10b570 0x7f6f4c1a1410 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c10ac20 0x7f6f4c19c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 --2- 172.21.15.39:0/580782958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f4c1052e0 0x7f6f4c19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.434+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 >> 172.21.15.39:0/580782958 conn(0x7f6f4c100db0 msgr2=0x7f6f4c10a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.435+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 shutdown_connections 2024-07-04T08:33:11.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:11.435+0000 7f6f53400640 1 -- 172.21.15.39:0/580782958 wait complete. 2024-07-04T08:33:11.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:11 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/580782958' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-04T08:33:11.805 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:11 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/580782958' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-04T08:33:11.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:11 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/580782958' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-04T08:33:12.340 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-04T08:33:12.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:12 smithi039 ceph-mon[32062]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-04T08:33:12.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:12 smithi039 ceph-mon[32062]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-04T08:33:12.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:12 smithi039 ceph-mon[32062]: Cluster is now healthy 2024-07-04T08:33:12.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:12 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/580782958' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-04T08:33:12.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:12 smithi039 ceph-mon[32062]: fsmap 12 up:standby 2024-07-04T08:33:12.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:12 smithi039 ceph-mon[32062]: pgmap v1043: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-04T08:33:12.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:12 smithi133 ceph-mon[37050]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-04T08:33:12.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:12 smithi133 ceph-mon[37050]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-04T08:33:12.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:12 smithi133 ceph-mon[37050]: Cluster is now healthy 2024-07-04T08:33:12.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:12 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/580782958' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-04T08:33:12.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:12 smithi133 ceph-mon[37050]: fsmap 12 up:standby 2024-07-04T08:33:12.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:12 smithi133 ceph-mon[37050]: pgmap v1043: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-04T08:33:12.841 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:33:12.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:12 smithi050 ceph-mon[36577]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-04T08:33:12.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:12 smithi050 ceph-mon[36577]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-04T08:33:12.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:12 smithi050 ceph-mon[36577]: Cluster is now healthy 2024-07-04T08:33:12.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:12 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/580782958' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-04T08:33:12.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:12 smithi050 ceph-mon[36577]: fsmap 12 up:standby 2024-07-04T08:33:12.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:12 smithi050 ceph-mon[36577]: pgmap v1043: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-04T08:33:14.682 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:14 smithi039 ceph-mon[32062]: pgmap v1044: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-07-04T08:33:14.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:14 smithi133 ceph-mon[37050]: pgmap v1044: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-07-04T08:33:14.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:14 smithi050 ceph-mon[36577]: pgmap v1044: 129 pgs: 129 active+clean; 32 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-07-04T08:33:15.014 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.028+0000 7f94ab000640 1 -- 172.21.15.39:0/1691629373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a410f990 msgr2=0x7f94a41161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.028+0000 7f94ab000640 1 --2- 172.21.15.39:0/1691629373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a410f990 0x7f94a41161c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f949c009ca0 tx=0x7f949c02a8b0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.028+0000 7f94ab000640 1 -- 172.21.15.39:0/1691629373 shutdown_connections 2024-07-04T08:33:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.028+0000 7f94ab000640 1 --2- 172.21.15.39:0/1691629373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a410f990 0x7f94a41161c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.028+0000 7f94ab000640 1 --2- 172.21.15.39:0/1691629373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a410f040 0x7f94a410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.028+0000 7f94ab000640 1 --2- 172.21.15.39:0/1691629373 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94a4109700 0x7f94a4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.028+0000 7f94ab000640 1 -- 172.21.15.39:0/1691629373 >> 172.21.15.39:0/1691629373 conn(0x7f94a4077630 msgr2=0x7f94a4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.028+0000 7f94ab000640 1 -- 172.21.15.39:0/1691629373 shutdown_connections 2024-07-04T08:33:15.015 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.028+0000 7f94ab000640 1 -- 172.21.15.39:0/1691629373 wait complete. 2024-07-04T08:33:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.030+0000 7f94ab000640 1 Processor -- start 2024-07-04T08:33:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.030+0000 7f94ab000640 1 -- start start 2024-07-04T08:33:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.030+0000 7f94ab000640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a4109700 0x7f94a419ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.030+0000 7f94ab000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a410f040 0x7f94a41a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.030+0000 7f94ab000640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94a410f990 0x7f94a41a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.030+0000 7f94ab000640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41196b0 con 0x7f94a4109700 2024-07-04T08:33:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.030+0000 7f94ab000640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4119530 con 0x7f94a410f990 2024-07-04T08:33:15.016 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.030+0000 7f94ab000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4119830 con 0x7f94a410f040 2024-07-04T08:33:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a4109700 0x7f94a419ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a4109700 0x7f94a419ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32888/0 (socket says 172.21.15.39:32888) 2024-07-04T08:33:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a3e00640 1 -- 172.21.15.39:0/1903991462 learned_addr learned my addr 172.21.15.39:0/1903991462 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:33:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a3400640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a410f040 0x7f94a41a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a9000640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94a410f990 0x7f94a41a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a3400640 1 -- 172.21.15.39:0/1903991462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94a410f990 msgr2=0x7f94a41a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a3400640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94a410f990 0x7f94a41a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.017 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a3400640 1 -- 172.21.15.39:0/1903991462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a4109700 msgr2=0x7f94a419ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:15.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a3400640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a4109700 0x7f94a419ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.031+0000 7f94a3400640 1 -- 172.21.15.39:0/1903991462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a41a5f10 con 0x7f94a410f040 2024-07-04T08:33:15.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.032+0000 7f94a3400640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a410f040 0x7f94a41a0540 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f94a406bf70 tx=0x7f948c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:15.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.032+0000 7f94a0c00640 1 -- 172.21.15.39:0/1903991462 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c00cc40 con 0x7f94a410f040 2024-07-04T08:33:15.018 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.032+0000 7f94a0c00640 1 -- 172.21.15.39:0/1903991462 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f948c00cde0 con 0x7f94a410f040 2024-07-04T08:33:15.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.032+0000 7f94a0c00640 1 -- 172.21.15.39:0/1903991462 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c004a00 con 0x7f94a410f040 2024-07-04T08:33:15.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.032+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a41a6200 con 0x7f94a410f040 2024-07-04T08:33:15.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.032+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a41ac960 con 0x7f94a410f040 2024-07-04T08:33:15.019 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.033+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a4109b10 con 0x7f94a410f040 2024-07-04T08:33:15.021 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.036+0000 7f94a0c00640 1 -- 172.21.15.39:0/1903991462 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f948c004ba0 con 0x7f94a410f040 2024-07-04T08:33:15.023 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.037+0000 7f94a0c00640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94880b2960 0x7f94880b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:15.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.038+0000 7f94a0c00640 1 -- 172.21.15.39:0/1903991462 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9766+0+0 (secure 0 0 0) 0x7f948c0e2ab0 con 0x7f94a410f040 2024-07-04T08:33:15.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.038+0000 7f94a3e00640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94880b2960 0x7f94880b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:15.024 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.039+0000 7f94a3e00640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94880b2960 0x7f94880b4e70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9494005fd0 tx=0x7f949400b4a0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:15.025 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.039+0000 7f94a0c00640 1 -- 172.21.15.39:0/1903991462 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f948c0a9fe0 con 0x7f94a410f040 2024-07-04T08:33:15.239 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.253+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f94a41a10d0 con 0x7f94a410f040 2024-07-04T08:33:15.422 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.436+0000 7f94a0c00640 1 -- 172.21.15.39:0/1903991462 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) ==== 181+0+0 (secure 0 0 0) 0x7f948c0aee20 con 0x7f94a410f040 2024-07-04T08:33:15.423 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_metadata' removed 2024-07-04T08:33:15.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.440+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94880b2960 msgr2=0x7f94880b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:15.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.440+0000 7f94ab000640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94880b2960 0x7f94880b4e70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9494005fd0 tx=0x7f949400b4a0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.440+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a410f040 msgr2=0x7f94a41a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:15.428 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.440+0000 7f94ab000640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a410f040 0x7f94a41a0540 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f94a406bf70 tx=0x7f948c00ee50 comp rx=0 tx=0).stop 2024-07-04T08:33:15.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.441+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 shutdown_connections 2024-07-04T08:33:15.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.441+0000 7f94ab000640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f94880b2960 0x7f94880b4e70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.441+0000 7f94ab000640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f94a410f990 0x7f94a41a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.441+0000 7f94ab000640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94a410f040 0x7f94a41a0540 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.441+0000 7f94ab000640 1 --2- 172.21.15.39:0/1903991462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a4109700 0x7f94a419ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:15.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.441+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 >> 172.21.15.39:0/1903991462 conn(0x7f94a4077630 msgr2=0x7f94a4077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:15.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.442+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 shutdown_connections 2024-07-04T08:33:15.429 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:15.442+0000 7f94ab000640 1 -- 172.21.15.39:0/1903991462 wait complete. 2024-07-04T08:33:15.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:15 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/1903991462' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:15.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:15 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:15.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:15 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/1903991462' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:15.813 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:15 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:15.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:15 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/1903991462' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:15.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:15 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:16.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-04T08:33:16.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:16 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-04T08:33:16.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:16 smithi039 ceph-mon[32062]: osdmap e116: 12 total, 12 up, 12 in 2024-07-04T08:33:16.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:16 smithi039 ceph-mon[32062]: pgmap v1046: 97 pgs: 97 active+clean; 577 KiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:16.772 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:33:16.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:16 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-04T08:33:16.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:16 smithi133 ceph-mon[37050]: osdmap e116: 12 total, 12 up, 12 in 2024-07-04T08:33:16.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:16 smithi133 ceph-mon[37050]: pgmap v1046: 97 pgs: 97 active+clean; 577 KiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:16.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:16 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-04T08:33:16.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:16 smithi050 ceph-mon[36577]: osdmap e116: 12 total, 12 up, 12 in 2024-07-04T08:33:16.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:16 smithi050 ceph-mon[36577]: pgmap v1046: 97 pgs: 97 active+clean; 577 KiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:18.533 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:18 smithi039 ceph-mon[32062]: pgmap v1047: 97 pgs: 97 active+clean; 577 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:18.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:18 smithi133 ceph-mon[37050]: pgmap v1047: 97 pgs: 97 active+clean; 577 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:18.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:18 smithi050 ceph-mon[36577]: pgmap v1047: 97 pgs: 97 active+clean; 577 KiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:19.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.113+0000 7f5feb200640 1 -- 172.21.15.39:0/3023269524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe4113c90 msgr2=0x7f5fe41160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:19.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.113+0000 7f5feb200640 1 --2- 172.21.15.39:0/3023269524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe4113c90 0x7f5fe41160b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009900 tx=0x7f5fdc02a400 comp rx=0 tx=0).stop 2024-07-04T08:33:19.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.113+0000 7f5feb200640 1 -- 172.21.15.39:0/3023269524 shutdown_connections 2024-07-04T08:33:19.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.113+0000 7f5feb200640 1 --2- 172.21.15.39:0/3023269524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe4113c90 0x7f5fe41160b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.113+0000 7f5feb200640 1 --2- 172.21.15.39:0/3023269524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe40777e0 0x7f5fe4113750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.113+0000 7f5feb200640 1 --2- 172.21.15.39:0/3023269524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe4076e90 0x7f5fe40772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.113+0000 7f5feb200640 1 -- 172.21.15.39:0/3023269524 >> 172.21.15.39:0/3023269524 conn(0x7f5fe4100ce0 msgr2=0x7f5fe4103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:19.100 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.113+0000 7f5feb200640 1 -- 172.21.15.39:0/3023269524 shutdown_connections 2024-07-04T08:33:19.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.113+0000 7f5feb200640 1 -- 172.21.15.39:0/3023269524 wait complete. 2024-07-04T08:33:19.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.115+0000 7f5feb200640 1 Processor -- start 2024-07-04T08:33:19.101 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.115+0000 7f5feb200640 1 -- start start 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.115+0000 7f5feb200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe4076e90 0x7f5fe419ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.115+0000 7f5feb200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe40777e0 0x7f5fe41a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.115+0000 7f5feb200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe4113c90 0x7f5fe41a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.115+0000 7f5feb200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4119610 con 0x7f5fe4113c90 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.115+0000 7f5feb200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4119490 con 0x7f5fe4076e90 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.115+0000 7f5feb200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4119790 con 0x7f5fe40777e0 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe4113c90 0x7f5fe41a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe40777e0 0x7f5fe41a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe9200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe4113c90 0x7f5fe41a5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:32896/0 (socket says 172.21.15.39:32896) 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe9200640 1 -- 172.21.15.39:0/447517382 learned_addr learned my addr 172.21.15.39:0/447517382 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:33:19.102 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe3e00640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe4076e90 0x7f5fe419ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:19.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe3400640 1 -- 172.21.15.39:0/447517382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe4076e90 msgr2=0x7f5fe419ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:19.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe3400640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe4076e90 0x7f5fe419ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.103 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe3400640 1 -- 172.21.15.39:0/447517382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe4113c90 msgr2=0x7f5fe41a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:19.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe3400640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe4113c90 0x7f5fe41a5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe3400640 1 -- 172.21.15.39:0/447517382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe41a5f30 con 0x7f5fe40777e0 2024-07-04T08:33:19.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.116+0000 7f5fe3400640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe40777e0 0x7f5fe41a0560 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5fe406bec0 tx=0x7f5fcc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:19.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.117+0000 7f5fe0c00640 1 -- 172.21.15.39:0/447517382 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc003b70 con 0x7f5fe40777e0 2024-07-04T08:33:19.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.117+0000 7f5fe0c00640 1 -- 172.21.15.39:0/447517382 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fcc003d10 con 0x7f5fe40777e0 2024-07-04T08:33:19.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.117+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe41a6220 con 0x7f5fe40777e0 2024-07-04T08:33:19.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.117+0000 7f5fe0c00640 1 -- 172.21.15.39:0/447517382 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc010830 con 0x7f5fe40777e0 2024-07-04T08:33:19.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.117+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe41ac980 con 0x7f5fe40777e0 2024-07-04T08:33:19.104 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.118+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe40772a0 con 0x7f5fe40777e0 2024-07-04T08:33:19.106 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.121+0000 7f5fe0c00640 1 -- 172.21.15.39:0/447517382 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f5fcc004a40 con 0x7f5fe40777e0 2024-07-04T08:33:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.122+0000 7f5fe0c00640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5fc80b2960 0x7f5fc80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.122+0000 7f5fe0c00640 1 -- 172.21.15.39:0/447517382 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9212+0+0 (secure 0 0 0) 0x7f5fcc0e2fd0 con 0x7f5fe40777e0 2024-07-04T08:33:19.108 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.123+0000 7f5fe3e00640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5fc80b2960 0x7f5fc80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.123+0000 7f5fe3e00640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5fc80b2960 0x7f5fc80b4e70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009bd0 tx=0x7f5fd4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:19.109 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.124+0000 7f5fe0c00640 1 -- 172.21.15.39:0/447517382 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f5fcc0aa6a0 con 0x7f5fe40777e0 2024-07-04T08:33:19.322 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.336+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5fe41a10f0 con 0x7f5fe40777e0 2024-07-04T08:33:19.900 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.911+0000 7f5fe0c00640 1 -- 172.21.15.39:0/447517382 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) ==== 169+0+0 (secure 0 0 0) 0x7f5fe41a10f0 con 0x7f5fe40777e0 2024-07-04T08:33:19.900 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data' removed 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.916+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5fc80b2960 msgr2=0x7f5fc80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.916+0000 7f5feb200640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5fc80b2960 0x7f5fc80b4e70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009bd0 tx=0x7f5fd4005f90 comp rx=0 tx=0).stop 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.916+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe40777e0 msgr2=0x7f5fe41a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.916+0000 7f5feb200640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe40777e0 0x7f5fe41a0560 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5fe406bec0 tx=0x7f5fcc00c6a0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.917+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 shutdown_connections 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.917+0000 7f5feb200640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f5fc80b2960 0x7f5fc80b4e70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.917+0000 7f5feb200640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe4113c90 0x7f5fe41a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.917+0000 7f5feb200640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fe40777e0 0x7f5fe41a0560 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.917+0000 7f5feb200640 1 --2- 172.21.15.39:0/447517382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5fe4076e90 0x7f5fe419ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:19.903 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.917+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 >> 172.21.15.39:0/447517382 conn(0x7f5fe4100ce0 msgr2=0x7f5fe4103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:19.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.917+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 shutdown_connections 2024-07-04T08:33:19.904 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:19.918+0000 7f5feb200640 1 -- 172.21.15.39:0/447517382 wait complete. 2024-07-04T08:33:20.129 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:19 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/447517382' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:20.130 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:19 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:20.301 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:19 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/447517382' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:20.301 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:19 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:20.312 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:19 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/447517382' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:20.312 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:19 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:20.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84 shell --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-07-04T08:33:21.203 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:20 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-04T08:33:21.203 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:20 smithi133 ceph-mon[37050]: osdmap e117: 12 total, 12 up, 12 in 2024-07-04T08:33:21.203 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:20 smithi133 ceph-mon[37050]: pgmap v1049: 33 pgs: 33 active+clean; 577 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:21.324 INFO:teuthology.orchestra.run.smithi039.stderr:Inferring config /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/mon.a/config 2024-07-04T08:33:21.335 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:20 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-04T08:33:21.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:20 smithi039 ceph-mon[32062]: osdmap e117: 12 total, 12 up, 12 in 2024-07-04T08:33:21.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:20 smithi039 ceph-mon[32062]: pgmap v1049: 33 pgs: 33 active+clean; 577 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:21.379 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:20 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-04T08:33:21.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:20 smithi050 ceph-mon[36577]: osdmap e117: 12 total, 12 up, 12 in 2024-07-04T08:33:21.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:20 smithi050 ceph-mon[36577]: pgmap v1049: 33 pgs: 33 active+clean; 577 KiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:22.481 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:22 smithi133 ceph-mon[37050]: pgmap v1050: 33 pgs: 33 active+clean; 577 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:22.512 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:22 smithi039 ceph-mon[32062]: pgmap v1050: 33 pgs: 33 active+clean; 577 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:22.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:22 smithi050 ceph-mon[36577]: pgmap v1050: 33 pgs: 33 active+clean; 577 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:23.695 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.709+0000 7f921b200640 1 -- 172.21.15.39:0/1719248026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f921c105110 msgr2=0x7f921c105520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:23.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.709+0000 7f921b200640 1 --2- 172.21.15.39:0/1719248026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f921c105110 0x7f921c105520 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f920c009900 tx=0x7f920c02a3f0 comp rx=0 tx=0).stop 2024-07-04T08:33:23.696 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:23 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:33:23.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.709+0000 7f921b200640 1 -- 172.21.15.39:0/1719248026 shutdown_connections 2024-07-04T08:33:23.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.709+0000 7f921b200640 1 --2- 172.21.15.39:0/1719248026 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f921c10aa50 0x7f921c10ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:23.696 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.709+0000 7f921b200640 1 --2- 172.21.15.39:0/1719248026 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f921c10b3a0 0x7f921c111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:23.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.709+0000 7f921b200640 1 --2- 172.21.15.39:0/1719248026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f921c105110 0x7f921c105520 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:23.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.709+0000 7f921b200640 1 -- 172.21.15.39:0/1719248026 >> 172.21.15.39:0/1719248026 conn(0x7f921c100b60 msgr2=0x7f921c102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:23.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.709+0000 7f921b200640 1 -- 172.21.15.39:0/1719248026 shutdown_connections 2024-07-04T08:33:23.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.709+0000 7f921b200640 1 -- 172.21.15.39:0/1719248026 wait complete. 2024-07-04T08:33:23.697 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.711+0000 7f921b200640 1 Processor -- start 2024-07-04T08:33:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.711+0000 7f921b200640 1 -- start start 2024-07-04T08:33:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f921b200640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f921c105110 0x7f921c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f921b200640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f921c10aa50 0x7f921c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f921b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f921c10b3a0 0x7f921c1a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f921b200640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 -- 0x7f921c1150c0 con 0x7f921c105110 2024-07-04T08:33:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f921b200640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f921c114f40 con 0x7f921c10aa50 2024-07-04T08:33:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f921b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f921c115240 con 0x7f921c10b3a0 2024-07-04T08:33:23.698 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f9219e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f921c105110 0x7f921c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:23.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f9219e00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f921c105110 0x7f921c19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:51846/0 (socket says 172.21.15.39:51846) 2024-07-04T08:33:23.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f9219e00640 1 -- 172.21.15.39:0/2378755222 learned_addr learned my addr 172.21.15.39:0/2378755222 (peer_addr_for_me v2:172.21.15.39:0/0) 2024-07-04T08:33:23.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f921a800640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f921c10b3a0 0x7f921c1a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:23.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.712+0000 7f9219400640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f921c10aa50 0x7f921c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:23.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.713+0000 7f9219400640 1 -- 172.21.15.39:0/2378755222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f921c10b3a0 msgr2=0x7f921c1a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:23.699 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.713+0000 7f9219400640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f921c10b3a0 0x7f921c1a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:23.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.713+0000 7f9219400640 1 -- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f921c105110 msgr2=0x7f921c19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:23.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.713+0000 7f9219400640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f921c105110 0x7f921c19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:23.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.713+0000 7f9219400640 1 -- 172.21.15.39:0/2378755222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c1a19c0 con 0x7f921c10aa50 2024-07-04T08:33:23.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.713+0000 7f9219e00640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f921c105110 0x7f921c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-04T08:33:23.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.713+0000 7f9219400640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f921c10aa50 0x7f921c19bff0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f921c06ae70 tx=0x7f920800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:23.700 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.714+0000 7f9212a00640 1 -- 172.21.15.39:0/2378755222 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920800cd00 con 0x7f921c10aa50 2024-07-04T08:33:23.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.714+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c1a1cb0 con 0x7f921c10aa50 2024-07-04T08:33:23.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.714+0000 7f9212a00640 1 -- 172.21.15.39:0/2378755222 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92080035f0 con 0x7f921c10aa50 2024-07-04T08:33:23.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.714+0000 7f9212a00640 1 -- 172.21.15.39:0/2378755222 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208010690 con 0x7f921c10aa50 2024-07-04T08:33:23.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.714+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c1a8410 con 0x7f921c10aa50 2024-07-04T08:33:23.701 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.715+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f921c105520 con 0x7f921c10aa50 2024-07-04T08:33:23.703 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.718+0000 7f9212a00640 1 -- 172.21.15.39:0/2378755222 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 25) ==== 149783+0+0 (secure 0 0 0) 0x7f92080037a0 con 0x7f921c10aa50 2024-07-04T08:33:23.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.719+0000 7f9212a00640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92000b2960 0x7f92000b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-04T08:33:23.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.719+0000 7f9212a00640 1 -- 172.21.15.39:0/2378755222 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8711+0+0 (secure 0 0 0) 0x7f92080e2980 con 0x7f921c10aa50 2024-07-04T08:33:23.705 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.720+0000 7f9219e00640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92000b2960 0x7f92000b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-04T08:33:23.706 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.720+0000 7f9219e00640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92000b2960 0x7f92000b4e70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f920c0096c0 tx=0x7f920c005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-04T08:33:23.709 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.723+0000 7f9212a00640 1 -- 172.21.15.39:0/2378755222 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+192762 (secure 0 0 0) 0x7f92080aa240 con 0x7f921c10aa50 2024-07-04T08:33:23.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:23 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:33:23.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:23 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-04T08:33:23.926 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:23.938+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f921c02de10 con 0x7f921c10aa50 2024-07-04T08:33:24.504 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.517+0000 7f9212a00640 1 -- 172.21.15.39:0/2378755222 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v118) ==== 178+0+0 (secure 0 0 0) 0x7f92080a9b40 con 0x7f921c10aa50 2024-07-04T08:33:24.504 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'cephfs_data_ec' removed 2024-07-04T08:33:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.522+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92000b2960 msgr2=0x7f92000b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:24.508 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.522+0000 7f921b200640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92000b2960 0x7f92000b4e70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f920c0096c0 tx=0x7f920c005a20 comp rx=0 tx=0).stop 2024-07-04T08:33:24.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.522+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f921c10aa50 msgr2=0x7f921c19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-04T08:33:24.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.522+0000 7f921b200640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f921c10aa50 0x7f921c19bff0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f921c06ae70 tx=0x7f920800ee50 comp rx=0 tx=0).stop 2024-07-04T08:33:24.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.523+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 shutdown_connections 2024-07-04T08:33:24.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.523+0000 7f921b200640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:6800/3120140278,v1:172.21.15.39:6801/3120140278] conn(0x7f92000b2960 0x7f92000b4e70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:24.509 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.523+0000 7f921b200640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f921c10b3a0 0x7f921c1a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:24.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.523+0000 7f921b200640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f921c10aa50 0x7f921c19bff0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:24.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.523+0000 7f921b200640 1 --2- 172.21.15.39:0/2378755222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f921c105110 0x7f921c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-04T08:33:24.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.523+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 >> 172.21.15.39:0/2378755222 conn(0x7f921c100b60 msgr2=0x7f921c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-04T08:33:24.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.524+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 shutdown_connections 2024-07-04T08:33:24.510 INFO:teuthology.orchestra.run.smithi039.stderr:2024-07-04T08:33:24.524+0000 7f921b200640 1 -- 172.21.15.39:0/2378755222 wait complete. 2024-07-04T08:33:24.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:33:24.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:33:24.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:24 smithi133 ceph-mon[37050]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:33:24.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:24 smithi133 ceph-mon[37050]: from='client.? 172.21.15.39:0/2378755222' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:24.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:24 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:24.802 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:24 smithi133 ceph-mon[37050]: pgmap v1051: 33 pgs: 33 active+clean; 577 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:24.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:33:24.836 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:33:24.837 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:24 smithi039 ceph-mon[32062]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:33:24.837 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:24 smithi039 ceph-mon[32062]: from='client.? 172.21.15.39:0/2378755222' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:24.837 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:24 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:24.837 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:24 smithi039 ceph-mon[32062]: pgmap v1051: 33 pgs: 33 active+clean; 577 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:24.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-04T08:33:24.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-04T08:33:24.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:24 smithi050 ceph-mon[36577]: from='mgr.14214 172.21.15.39:0/2853033455' entity='mgr.x' 2024-07-04T08:33:24.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:24 smithi050 ceph-mon[36577]: from='client.? 172.21.15.39:0/2378755222' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:24.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:24 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-04T08:33:24.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:24 smithi050 ceph-mon[36577]: pgmap v1051: 33 pgs: 33 active+clean; 577 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-04T08:33:25.428 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-04T08:33:25.518 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-04T08:33:25.553 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-04T08:33:25.584 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-07-04T08:33:25.585 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-04T08:33:25.643 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-04T08:33:25.661 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-04T08:33:25.679 INFO:tasks.cephadm:Stopping all daemons... 2024-07-04T08:33:25.679 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-07-04T08:33:25.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.a 2024-07-04T08:33:25.758 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:25 smithi039 ceph-mon[32062]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-07-04T08:33:25.758 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:25 smithi039 ceph-mon[32062]: osdmap e118: 12 total, 12 up, 12 in 2024-07-04T08:33:25.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:25 smithi133 ceph-mon[37050]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-07-04T08:33:25.801 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:25 smithi133 ceph-mon[37050]: osdmap e118: 12 total, 12 up, 12 in 2024-07-04T08:33:25.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:25 smithi050 ceph-mon[36577]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-07-04T08:33:25.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:25 smithi050 ceph-mon[36577]: osdmap e118: 12 total, 12 up, 12 in 2024-07-04T08:33:26.086 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:25 smithi039 systemd[1]: Stopping Ceph mon.a for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:33:26.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a[32058]: 2024-07-04T08:33:26.275+0000 7f2daa800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:33:26.586 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a[32058]: 2024-07-04T08:33:26.275+0000 7f2daa800640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-07-04T08:33:26.587 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:26 smithi039 podman[147649]: 2024-07-04 08:33:26.36814413 +0000 UTC m=+0.228793700 container died c524f94894efdc0689a4fab33c574be8acbcf4855f470d6e185b5872f257159a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:33:27.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:26 smithi039 podman[147649]: 2024-07-04 08:33:26.776837078 +0000 UTC m=+0.637486699 container cleanup c524f94894efdc0689a4fab33c574be8acbcf4855f470d6e185b5872f257159a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0) 2024-07-04T08:33:27.006 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:26 smithi039 bash[147649]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a 2024-07-04T08:33:27.336 INFO:journalctl@ceph.mon.a.smithi039.stdout:Jul 04 08:33:27 smithi039 podman[147660]: 2024-07-04 08:33:27.019635018 +0000 UTC m=+0.634147164 container remove c524f94894efdc0689a4fab33c574be8acbcf4855f470d6e185b5872f257159a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux ) 2024-07-04T08:33:27.569 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.a.service' 2024-07-04T08:33:27.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:33:27.715 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-07-04T08:33:27.715 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-07-04T08:33:27.715 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.b 2024-07-04T08:33:28.026 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:27 smithi050 systemd[1]: Stopping Ceph mon.b for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:33:28.379 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:28 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-b[36572]: 2024-07-04T08:33:28.036+0000 7f26e7a53640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:33:28.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:28 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-b[36572]: 2024-07-04T08:33:28.036+0000 7f26e7a53640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-07-04T08:33:28.380 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:28 smithi050 podman[75104]: 2024-07-04 08:33:28.272181604 +0000 UTC m=+0.342629481 container died 5233190bdf2600f51b27c7101a48b8a5e5419955f9ccaab427eae13a12d5e8f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:33:28.879 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:28 smithi050 podman[75104]: 2024-07-04 08:33:28.645366992 +0000 UTC m=+0.715814868 container cleanup 5233190bdf2600f51b27c7101a48b8a5e5419955f9ccaab427eae13a12d5e8f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-b, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, io.buildah.version=1.36.0) 2024-07-04T08:33:28.880 INFO:journalctl@ceph.mon.b.smithi050.stdout:Jul 04 08:33:28 smithi050 bash[75104]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-b 2024-07-04T08:33:29.213 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.b.service' 2024-07-04T08:33:29.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:33:29.260 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-07-04T08:33:29.260 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-07-04T08:33:29.260 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.c 2024-07-04T08:33:29.551 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:29 smithi133 systemd[1]: Stopping Ceph mon.c for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:33:29.970 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:29 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-c[37046]: 2024-07-04T08:33:29.583+0000 7fc7afcbd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:33:29.970 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:29 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-c[37046]: 2024-07-04T08:33:29.583+0000 7fc7afcbd640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-07-04T08:33:29.970 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:29 smithi133 podman[75968]: 2024-07-04 08:33:29.695862927 +0000 UTC m=+0.230175471 container died 42ab45a0f5c93e5b87a2c690c1eb787cc8b84d249295ca162ac41ad62afa2384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-04T08:33:30.301 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:30 smithi133 podman[75968]: 2024-07-04 08:33:30.083945773 +0000 UTC m=+0.618258301 container cleanup 42ab45a0f5c93e5b87a2c690c1eb787cc8b84d249295ca162ac41ad62afa2384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:33:30.301 INFO:journalctl@ceph.mon.c.smithi133.stdout:Jul 04 08:33:30 smithi133 bash[75968]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mon-c 2024-07-04T08:33:30.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mon.c.service' 2024-07-04T08:33:30.554 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:33:30.554 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-07-04T08:33:30.554 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-07-04T08:33:30.555 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.x 2024-07-04T08:33:31.086 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:33:30 smithi039 systemd[1]: Stopping Ceph mgr.x for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:33:31.587 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:33:31 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:33:31.187+0000 7fe04be00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:33:31.587 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:33:31 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x[32287]: 2024-07-04T08:33:31.187+0000 7fe04be00640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-07-04T08:33:31.587 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:33:31 smithi039 podman[147768]: 2024-07-04 08:33:31.293051983 +0000 UTC m=+0.245638583 container died fcb3d6dbfbd2b59ef8f87388db058d22b322d6d38d990c4934d1082a0fde5aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-07-04T08:33:32.336 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:33:31 smithi039 podman[147768]: 2024-07-04 08:33:31.906963937 +0000 UTC m=+0.859550535 container cleanup fcb3d6dbfbd2b59ef8f87388db058d22b322d6d38d990c4934d1082a0fde5aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, ceph=True) 2024-07-04T08:33:32.337 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:33:31 smithi039 bash[147768]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x 2024-07-04T08:33:32.337 INFO:journalctl@ceph.mgr.x.smithi039.stdout:Jul 04 08:33:32 smithi039 podman[147779]: 2024-07-04 08:33:32.108688309 +0000 UTC m=+0.798663870 container remove fcb3d6dbfbd2b59ef8f87388db058d22b322d6d38d990c4934d1082a0fde5aa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2e9f68b5506c607b2f7d69d380cb7887cf00dc84, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=) 2024-07-04T08:33:32.682 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.x.service' 2024-07-04T08:33:32.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:33:32.800 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-07-04T08:33:32.801 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-07-04T08:33:32.801 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.y 2024-07-04T08:33:33.129 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:33:32 smithi050 systemd[1]: Stopping Ceph mgr.y for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:33:33.454 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:33:33 smithi050 podman[75237]: 2024-07-04 08:33:33.162706307 +0000 UTC m=+0.141832550 container died 71760403a058fd641abe0d45df05e78fa9fb7e2b173c6de31c4d8597dae25dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, ceph=True) 2024-07-04T08:33:33.813 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:33:33 smithi050 podman[75237]: 2024-07-04 08:33:33.55944529 +0000 UTC m=+0.538571524 container cleanup 71760403a058fd641abe0d45df05e78fa9fb7e2b173c6de31c4d8597dae25dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-07-04T08:33:33.813 INFO:journalctl@ceph.mgr.y.smithi050.stdout:Jul 04 08:33:33 smithi050 bash[75237]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-y 2024-07-04T08:33:33.949 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.y.service' 2024-07-04T08:33:33.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:33:33.985 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-07-04T08:33:33.986 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-07-04T08:33:33.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.z 2024-07-04T08:33:34.301 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jul 04 08:33:34 smithi133 systemd[1]: Stopping Ceph mgr.z for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:33:34.649 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jul 04 08:33:34 smithi133 podman[76091]: 2024-07-04 08:33:34.384139588 +0000 UTC m=+0.186276231 container died 4bc09aefe4546c971ce5caba6dc794bca233e78aa20175af22d6f9dc66b0146f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-z, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-04T08:33:35.042 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jul 04 08:33:34 smithi133 podman[76091]: 2024-07-04 08:33:34.764213559 +0000 UTC m=+0.566350193 container cleanup 4bc09aefe4546c971ce5caba6dc794bca233e78aa20175af22d6f9dc66b0146f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b) 2024-07-04T08:33:35.042 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Jul 04 08:33:34 smithi133 bash[76091]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mgr-z 2024-07-04T08:33:35.219 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mgr.z.service' 2024-07-04T08:33:35.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:33:35.252 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-07-04T08:33:35.252 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-07-04T08:33:35.252 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.0 2024-07-04T08:33:35.587 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:35 smithi039 systemd[1]: Stopping Ceph osd.0 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:33:36.337 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:35 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0[51606]: 2024-07-04T08:33:35.876+0000 7fb354e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:33:36.337 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:35 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0[51606]: 2024-07-04T08:33:35.876+0000 7fb354e00640 -1 osd.0 118 *** Got signal Terminated *** 2024-07-04T08:33:36.337 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:35 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0[51606]: 2024-07-04T08:33:35.876+0000 7fb354e00640 -1 osd.0 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:33:41.552 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:41 smithi039 podman[147889]: 2024-07-04 08:33:41.262589434 +0000 UTC m=+5.522586634 container died 1be44477711f8a635452e731a7e870c1cc54b11722f3a7c7e88a676a4573e9a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-07-04T08:33:41.835 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:41 smithi039 podman[147889]: 2024-07-04 08:33:41.778139434 +0000 UTC m=+6.038136671 container cleanup 1be44477711f8a635452e731a7e870c1cc54b11722f3a7c7e88a676a4573e9a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240625) 2024-07-04T08:33:41.836 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:41 smithi039 bash[147889]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0 2024-07-04T08:33:42.336 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:42 smithi039 podman[147905]: 2024-07-04 08:33:42.029709829 +0000 UTC m=+0.749425639 container remove 1be44477711f8a635452e731a7e870c1cc54b11722f3a7c7e88a676a4573e9a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-07-04T08:33:43.086 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:42 smithi039 podman[147999]: 2024-07-04 08:33:42.754399709 +0000 UTC m=+0.033864546 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:33:43.087 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:42 smithi039 podman[147999]: 2024-07-04 08:33:42.863871439 +0000 UTC m=+0.143336316 container create 0e6676b51f4d9f83fc56a8decc1a83cee31fcee167d80508bcce2e82f272ac46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-04T08:33:43.424 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:43 smithi039 podman[147999]: 2024-07-04 08:33:43.109032322 +0000 UTC m=+0.388497203 container init 0e6676b51f4d9f83fc56a8decc1a83cee31fcee167d80508bcce2e82f272ac46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-04T08:33:43.424 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:43 smithi039 podman[147999]: 2024-07-04 08:33:43.12249857 +0000 UTC m=+0.401963415 container start 0e6676b51f4d9f83fc56a8decc1a83cee31fcee167d80508bcce2e82f272ac46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-07-04T08:33:43.425 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:43 smithi039 podman[147999]: 2024-07-04 08:33:43.216334915 +0000 UTC m=+0.495799836 container attach 0e6676b51f4d9f83fc56a8decc1a83cee31fcee167d80508bcce2e82f272ac46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-07-04T08:33:43.811 INFO:journalctl@ceph.osd.0.smithi039.stdout:Jul 04 08:33:43 smithi039 podman[147999]: 2024-07-04 08:33:43.439104648 +0000 UTC m=+0.718569492 container died 0e6676b51f4d9f83fc56a8decc1a83cee31fcee167d80508bcce2e82f272ac46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-0-deactivate, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:33:44.229 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.0.service' 2024-07-04T08:33:44.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:33:44.340 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-07-04T08:33:44.341 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-07-04T08:33:44.341 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.1 2024-07-04T08:33:44.629 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:44 smithi050 systemd[1]: Stopping Ceph osd.1 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:33:45.129 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1[42438]: 2024-07-04T08:33:44.671+0000 7f859a847640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:33:45.130 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1[42438]: 2024-07-04T08:33:44.671+0000 7f859a847640 -1 osd.1 118 *** Got signal Terminated *** 2024-07-04T08:33:45.130 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1[42438]: 2024-07-04T08:33:44.671+0000 7f859a847640 -1 osd.1 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:33:50.380 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:50 smithi050 podman[75365]: 2024-07-04 08:33:50.002467445 +0000 UTC m=+5.449886562 container died 587b8a7b2ff179734cbb8a851677f68cf24be0a695049978d3cc785ea127e67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-07-04T08:33:50.717 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:50 smithi050 podman[75365]: 2024-07-04 08:33:50.531393687 +0000 UTC m=+5.978812794 container cleanup 587b8a7b2ff179734cbb8a851677f68cf24be0a695049978d3cc785ea127e67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:33:50.718 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:50 smithi050 bash[75365]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1 2024-07-04T08:33:51.129 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:50 smithi050 podman[75382]: 2024-07-04 08:33:50.732769412 +0000 UTC m=+0.723628992 container remove 587b8a7b2ff179734cbb8a851677f68cf24be0a695049978d3cc785ea127e67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:33:51.429 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:51 smithi050 podman[75474]: 2024-07-04 08:33:51.07435446 +0000 UTC m=+0.017794374 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:33:51.430 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:51 smithi050 podman[75474]: 2024-07-04 08:33:51.183871238 +0000 UTC m=+0.127311179 container create 55c591660945fdd6cfcd74319b43b12591796da69e4393fbc18876337cca08a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-07-04T08:33:51.879 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:51 smithi050 podman[75474]: 2024-07-04 08:33:51.44470111 +0000 UTC m=+0.388141003 container init 55c591660945fdd6cfcd74319b43b12591796da69e4393fbc18876337cca08a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-07-04T08:33:51.880 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:51 smithi050 podman[75474]: 2024-07-04 08:33:51.448851269 +0000 UTC m=+0.392291182 container start 55c591660945fdd6cfcd74319b43b12591796da69e4393fbc18876337cca08a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625) 2024-07-04T08:33:51.880 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:51 smithi050 podman[75474]: 2024-07-04 08:33:51.52802228 +0000 UTC m=+0.471462180 container attach 55c591660945fdd6cfcd74319b43b12591796da69e4393fbc18876337cca08a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625) 2024-07-04T08:33:51.880 INFO:journalctl@ceph.osd.1.smithi050.stdout:Jul 04 08:33:51 smithi050 podman[75474]: 2024-07-04 08:33:51.643817625 +0000 UTC m=+0.587257533 container died 55c591660945fdd6cfcd74319b43b12591796da69e4393fbc18876337cca08a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625) 2024-07-04T08:33:52.375 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.1.service' 2024-07-04T08:33:52.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:33:52.421 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-07-04T08:33:52.421 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-07-04T08:33:52.422 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.2 2024-07-04T08:33:52.769 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:52 smithi133 systemd[1]: Stopping Ceph osd.2 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:33:53.049 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2[43380]: 2024-07-04T08:33:52.784+0000 7f6470443640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:33:53.049 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2[43380]: 2024-07-04T08:33:52.784+0000 7f6470443640 -1 osd.2 118 *** Got signal Terminated *** 2024-07-04T08:33:53.049 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2[43380]: 2024-07-04T08:33:52.784+0000 7f6470443640 -1 osd.2 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:33:58.361 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:58 smithi133 podman[76214]: 2024-07-04 08:33:58.120512372 +0000 UTC m=+5.485398193 container died 87154d8b7e7b6b0c3512bee14c02c9821fcfdf2f49b79a2fdf904358a1e7b1d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240625) 2024-07-04T08:33:58.630 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:58 smithi133 podman[76214]: 2024-07-04 08:33:58.459017765 +0000 UTC m=+5.823903580 container cleanup 87154d8b7e7b6b0c3512bee14c02c9821fcfdf2f49b79a2fdf904358a1e7b1d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-04T08:33:58.631 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:58 smithi133 bash[76214]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2 2024-07-04T08:33:59.051 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:58 smithi133 podman[76240]: 2024-07-04 08:33:58.645781941 +0000 UTC m=+0.519839605 container remove 87154d8b7e7b6b0c3512bee14c02c9821fcfdf2f49b79a2fdf904358a1e7b1d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-07-04T08:33:59.435 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:59 smithi133 podman[76330]: 2024-07-04 08:33:58.981769543 +0000 UTC m=+0.017079635 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:33:59.435 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:59 smithi133 podman[76330]: 2024-07-04 08:33:59.104904976 +0000 UTC m=+0.140215054 container create 31009cea962ce380902457af2516441800f1e94cd90e75bc137310b551580618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux ) 2024-07-04T08:33:59.801 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:59 smithi133 podman[76330]: 2024-07-04 08:33:59.450082269 +0000 UTC m=+0.485392374 container init 31009cea962ce380902457af2516441800f1e94cd90e75bc137310b551580618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-07-04T08:33:59.801 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:59 smithi133 podman[76330]: 2024-07-04 08:33:59.453816137 +0000 UTC m=+0.489126229 container start 31009cea962ce380902457af2516441800f1e94cd90e75bc137310b551580618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:33:59.801 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:59 smithi133 podman[76330]: 2024-07-04 08:33:59.541086074 +0000 UTC m=+0.576396173 container attach 31009cea962ce380902457af2516441800f1e94cd90e75bc137310b551580618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:33:59.801 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:33:59 smithi133 podman[76330]: 2024-07-04 08:33:59.651295435 +0000 UTC m=+0.686605521 container died 31009cea962ce380902457af2516441800f1e94cd90e75bc137310b551580618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:34:00.180 INFO:journalctl@ceph.osd.2.smithi133.stdout:Jul 04 08:34:00 smithi133 podman[76354]: 2024-07-04 08:34:00.178164931 +0000 UTC m=+0.521273831 container remove 31009cea962ce380902457af2516441800f1e94cd90e75bc137310b551580618 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-07-04T08:34:00.186 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.2.service' 2024-07-04T08:34:00.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:34:00.227 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-07-04T08:34:00.227 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-07-04T08:34:00.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.3 2024-07-04T08:34:00.587 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:00 smithi039 systemd[1]: Stopping Ceph osd.3 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:34:01.086 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:00 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3[58520]: 2024-07-04T08:34:00.827+0000 7fa64ca00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:34:01.086 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:00 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3[58520]: 2024-07-04T08:34:00.827+0000 7fa64ca00640 -1 osd.3 118 *** Got signal Terminated *** 2024-07-04T08:34:01.086 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:00 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3[58520]: 2024-07-04T08:34:00.827+0000 7fa64ca00640 -1 osd.3 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:34:02.775 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:02 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3[58520]: 2024-07-04T08:34:02.511+0000 7fa643400640 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.398213+0000 front 2024-07-04T08:33:38.398660+0000 (oldest deadline 2024-07-04T08:34:02.497950+0000) 2024-07-04T08:34:02.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:02 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:02.491+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:03.087 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:02 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:02.789+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:03.757 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3[58520]: 2024-07-04T08:34:03.512+0000 7fa643400640 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.398213+0000 front 2024-07-04T08:33:38.398660+0000 (oldest deadline 2024-07-04T08:34:02.497950+0000) 2024-07-04T08:34:03.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:03.537+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:04.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:03 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:03.770+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:04.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:04 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:04.555+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:04.836 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:04 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3[58520]: 2024-07-04T08:34:04.534+0000 7fa643400640 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.398213+0000 front 2024-07-04T08:33:38.398660+0000 (oldest deadline 2024-07-04T08:34:02.497950+0000) 2024-07-04T08:34:04.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:04 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:04.776+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:05.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:04.751+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:05.300 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:05 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:05.012+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:05.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:05 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:05.561+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:05.836 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:05 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3[58520]: 2024-07-04T08:34:05.540+0000 7fa643400640 -1 osd.3 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.398213+0000 front 2024-07-04T08:33:38.398660+0000 (oldest deadline 2024-07-04T08:34:02.497950+0000) 2024-07-04T08:34:05.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:05 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:05.755+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:06.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:05 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:05.777+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:06.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:06 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:06.032+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:06.639 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:06 smithi039 podman[148113]: 2024-07-04 08:34:06.378339827 +0000 UTC m=+5.678940326 container died f7dc0efe5f5074cd81cb22638546a3379814c725f28892f842c993e0e9a54640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:34:06.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:06 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:06.520+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:06.895 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:06 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:06.712+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:06.895 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:06 smithi039 podman[148113]: 2024-07-04 08:34:06.88000045 +0000 UTC m=+6.180600928 container cleanup f7dc0efe5f5074cd81cb22638546a3379814c725f28892f842c993e0e9a54640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b) 2024-07-04T08:34:06.895 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:06 smithi039 bash[148113]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3 2024-07-04T08:34:07.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:06 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:06.765+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:07.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:07.053+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:07.336 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:07 smithi039 podman[148129]: 2024-07-04 08:34:07.098111668 +0000 UTC m=+0.704661978 container remove f7dc0efe5f5074cd81cb22638546a3379814c725f28892f842c993e0e9a54640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:34:07.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:07.564+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:07.976 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:07 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:07.695+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:08.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:07 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:07.775+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:08.297 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:07 smithi039 podman[148221]: 2024-07-04 08:34:07.990632481 +0000 UTC m=+0.124407785 container create 46c62b53e8dbc3dd368483bebb733e06f10cc06512c4a4fbf11450afc17427d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b) 2024-07-04T08:34:08.297 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:07 smithi039 podman[148221]: 2024-07-04 08:34:07.89992127 +0000 UTC m=+0.033696563 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:34:08.297 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:08 smithi039 podman[148221]: 2024-07-04 08:34:08.311539589 +0000 UTC m=+0.445314938 container init 46c62b53e8dbc3dd368483bebb733e06f10cc06512c4a4fbf11450afc17427d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-04T08:34:08.300 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:08.055+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:08.549 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:08 smithi039 podman[148221]: 2024-07-04 08:34:08.32390641 +0000 UTC m=+0.457681739 container start 46c62b53e8dbc3dd368483bebb733e06f10cc06512c4a4fbf11450afc17427d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True) 2024-07-04T08:34:08.549 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:08 smithi039 podman[148221]: 2024-07-04 08:34:08.435156416 +0000 UTC m=+0.568931792 container attach 46c62b53e8dbc3dd368483bebb733e06f10cc06512c4a4fbf11450afc17427d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3-deactivate, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:34:08.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:08.534+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:08.836 INFO:journalctl@ceph.osd.3.smithi039.stdout:Jul 04 08:34:08 smithi039 podman[148245]: 2024-07-04 08:34:08.695211065 +0000 UTC m=+0.046438574 container died 46c62b53e8dbc3dd368483bebb733e06f10cc06512c4a4fbf11450afc17427d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS) 2024-07-04T08:34:08.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:08 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:08.732+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:09.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:08 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:08.767+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:09.281 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.3.service' 2024-07-04T08:34:09.300 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:09.025+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:09.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:34:09.387 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-07-04T08:34:09.388 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-07-04T08:34:09.388 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.4 2024-07-04T08:34:09.752 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:09 smithi050 systemd[1]: Stopping Ceph osd.4 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:34:09.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:09.485+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:10.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:09 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:09.737+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:10.129 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:09 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4[49345]: 2024-07-04T08:34:09.765+0000 7f1b69536640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:34:10.129 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:09 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4[49345]: 2024-07-04T08:34:09.765+0000 7f1b69536640 -1 osd.4 118 *** Got signal Terminated *** 2024-07-04T08:34:10.129 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:09 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4[49345]: 2024-07-04T08:34:09.765+0000 7f1b69536640 -1 osd.4 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:34:10.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:09 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:09.810+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:10.237 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:09.990+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:10.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:10.522+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:11.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:10 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:10.717+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:11.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:10 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:10.947+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:11.129 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:10 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4[49345]: 2024-07-04T08:34:10.881+0000 7f1b65b22640 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.543399+0000 front 2024-07-04T08:33:47.543432+0000 (oldest deadline 2024-07-04T08:34:10.443628+0000) 2024-07-04T08:34:11.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:10 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:10.850+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:11.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:10 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:10.850+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:11.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:10.985+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:11.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:11.613+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:11.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:11.481+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:11.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:11.481+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:12.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:11 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:11.937+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:12.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:11 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:11.766+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:12.129 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:11 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4[49345]: 2024-07-04T08:34:11.923+0000 7f1b65b22640 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.543399+0000 front 2024-07-04T08:33:47.543432+0000 (oldest deadline 2024-07-04T08:34:10.443628+0000) 2024-07-04T08:34:12.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:11 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:11.871+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:12.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:11 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:11.871+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:12.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:11.999+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:12.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:12.585+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:12.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:12.503+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:12.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:12.503+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:12.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:12 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:12.583+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:13.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:12 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:12.942+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:13.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:12 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:12.768+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:13.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:12 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:12.768+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:13.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:12.989+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:13.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:12.989+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:13.379 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:12 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4[49345]: 2024-07-04T08:34:12.957+0000 7f1b65b22640 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.543399+0000 front 2024-07-04T08:33:47.543432+0000 (oldest deadline 2024-07-04T08:34:10.443628+0000) 2024-07-04T08:34:13.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:12 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:12.910+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:13.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:12 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:12.910+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:13.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:13 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:13.574+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:13.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:13 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:13.523+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:13.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:13 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:13.523+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:13.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:13 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:13.558+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:13.987 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:13 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:13.968+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:13.988 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:13 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:13.729+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:13.988 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:13 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:13.729+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:14.284 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:13 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4[49345]: 2024-07-04T08:34:13.956+0000 7f1b65b22640 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.543399+0000 front 2024-07-04T08:33:47.543432+0000 (oldest deadline 2024-07-04T08:34:10.443628+0000) 2024-07-04T08:34:14.284 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:13 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:13.938+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:14.284 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:13 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:13.938+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:14.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:13 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:13.973+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:14.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:13 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:13.973+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:14.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:14 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:14.580+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:14.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:14 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:14.503+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:14.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:14 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:14.504+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:14.878 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:14 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:14.605+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:15.040 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:14 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:14.941+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:15.041 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:14 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:14.773+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:15.041 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:14 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:14.773+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:15.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:14 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:14.892+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:15.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:14 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:14.892+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:15.300 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:14 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:14.997+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:15.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:14 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:14.997+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:15.587 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:15 smithi050 podman[75594]: 2024-07-04 08:34:15.298254559 +0000 UTC m=+5.695322381 container died effa0a9709ab657f0fc0ccbc19335a675bad0b94bc8c17ee0f0e5f77ad225016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-07-04T08:34:15.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:15 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:15.613+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:15.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:15 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:15.506+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:15.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:15 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:15.506+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:15.859 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:15 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:15.635+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:15.860 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:15 smithi050 podman[75594]: 2024-07-04 08:34:15.701351294 +0000 UTC m=+6.098419110 container cleanup effa0a9709ab657f0fc0ccbc19335a675bad0b94bc8c17ee0f0e5f77ad225016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-04T08:34:15.860 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:15 smithi050 bash[75594]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4 2024-07-04T08:34:16.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:15 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:15.929+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:16.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:15 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:15.782+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:16.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:15 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:15.782+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:16.129 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:15 smithi050 podman[75610]: 2024-07-04 08:34:15.944466041 +0000 UTC m=+0.646449195 container remove effa0a9709ab657f0fc0ccbc19335a675bad0b94bc8c17ee0f0e5f77ad225016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625) 2024-07-04T08:34:16.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:15 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:15.874+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:16.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:15 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:15.874+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:16.300 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:16 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:16.036+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:16.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:16 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:16.036+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:16.575 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:16 smithi050 podman[75697]: 2024-07-04 08:34:16.181240454 +0000 UTC m=+0.017893966 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:34:16.575 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:16 smithi050 podman[75697]: 2024-07-04 08:34:16.304470962 +0000 UTC m=+0.141124507 container create 3d430fa991081c176a04789e0d4c4ede4ebf3d05e30e470174b375615c287c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-04T08:34:16.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:16 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:16.581+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:16.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:16 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:16.528+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:16.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:16 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:16.528+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:16.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:16 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:16.596+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:16.879 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:16 smithi050 podman[75697]: 2024-07-04 08:34:16.590869159 +0000 UTC m=+0.427522666 container init 3d430fa991081c176a04789e0d4c4ede4ebf3d05e30e470174b375615c287c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux ) 2024-07-04T08:34:16.880 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:16 smithi050 podman[75697]: 2024-07-04 08:34:16.594611209 +0000 UTC m=+0.431264729 container start 3d430fa991081c176a04789e0d4c4ede4ebf3d05e30e470174b375615c287c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:34:16.880 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:16 smithi050 podman[75697]: 2024-07-04 08:34:16.690740982 +0000 UTC m=+0.527394493 container attach 3d430fa991081c176a04789e0d4c4ede4ebf3d05e30e470174b375615c287c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:34:16.880 INFO:journalctl@ceph.osd.4.smithi050.stdout:Jul 04 08:34:16 smithi050 podman[75697]: 2024-07-04 08:34:16.790255597 +0000 UTC m=+0.626909100 container died 3d430fa991081c176a04789e0d4c4ede4ebf3d05e30e470174b375615c287c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240625, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:34:17.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:16 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:16.955+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:17.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:16 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:16.786+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:17.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:16 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:16.786+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:17.300 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:17 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:17.024+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:17.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:17 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:17.024+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:17.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:16 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:16.918+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:17.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:16 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:16.918+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:17.456 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.4.service' 2024-07-04T08:34:17.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:34:17.499 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-07-04T08:34:17.499 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-07-04T08:34:17.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.5 2024-07-04T08:34:17.798 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:17 smithi133 systemd[1]: Stopping Ceph osd.5 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:34:17.799 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:17 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:17.569+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:17.799 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:17 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:17.566+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:17.799 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:17 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:17.566+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:17.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:17 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:17.581+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:18.050 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:17 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:17.813+0000 7f6795488640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:34:18.051 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:17 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:17.813+0000 7f6795488640 -1 osd.5 118 *** Got signal Terminated *** 2024-07-04T08:34:18.051 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:17 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:17.813+0000 7f6795488640 -1 osd.5 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:34:18.051 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:18 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:18.044+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:18.051 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:18 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:18.044+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:18.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:17 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:17.954+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:18.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:17 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:17.817+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:18.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:17 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:17.817+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:18.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:17 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:17.918+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:18.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:17 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:17.918+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:18.863 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:18 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:18.571+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:19.020 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:18 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:18.599+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:19.021 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:18 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:18.569+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:19.021 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:18 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:18.569+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:19.021 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:18 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:18.569+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:19.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:18 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:18.935+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:19.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:18 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:18.818+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:19.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:18 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:18.818+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:19.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:18 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:18.818+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:19.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:18 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:18.878+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:19.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:18 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:18.878+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:19.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:19 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:19.036+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:19.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:19 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:19.036+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:19.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:19 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:19.575+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:20.046 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:19 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:19.608+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:20.046 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:19 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:19.617+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:20.047 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:19 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:19.617+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:20.047 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:19 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:19.617+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:20.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:19 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:19.933+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:20.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:19 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:19.933+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:20.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:19 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:19.824+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:20.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:19 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:19.824+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:20.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:19 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:19.824+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:20.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:20 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:20.061+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:20.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:20 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:20.061+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:20.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:20 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:20.061+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.334208+0000 front 2024-07-04T08:33:56.334234+0000 (oldest deadline 2024-07-04T08:34:19.834449+0000) 2024-07-04T08:34:20.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:19 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:19.894+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:20.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:19 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:19.894+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:20.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:20 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:20.544+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:21.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:20 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:20.628+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:21.051 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:20 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:20.578+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:21.051 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:20 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:20.578+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:21.051 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:20 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:20.578+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:21.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:20 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:20.909+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:21.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:20 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:20.909+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:21.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:20 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:20.815+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:21.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:20 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:20.815+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:21.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:20 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:20.815+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:21.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:20 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:20.925+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:21.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:20 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:20.925+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:21.550 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:21 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:21.065+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:21.551 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:21 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:21.065+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:21.551 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:21 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:21.065+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.334208+0000 front 2024-07-04T08:33:56.334234+0000 (oldest deadline 2024-07-04T08:34:19.834449+0000) 2024-07-04T08:34:21.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:21 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:21.562+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:22.026 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:21 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:21.599+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:22.026 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:21 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:21.599+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:22.026 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:21 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:21.599+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:22.026 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:21 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:21.613+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:22.026 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:21 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:21.613+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:22.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:21 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:21.879+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:22.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:21 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:21.879+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:22.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:21 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:21.777+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:22.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:21 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:21.777+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:22.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:21 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:21.777+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:22.296 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:22 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:22.040+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:40.031634+0000 front 2024-07-04T08:33:40.031449+0000 (oldest deadline 2024-07-04T08:34:04.131404+0000) 2024-07-04T08:34:22.296 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:22 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:22.040+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.032899+0000 front 2024-07-04T08:33:47.032626+0000 (oldest deadline 2024-07-04T08:34:12.332619+0000) 2024-07-04T08:34:22.297 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:22 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5[49851]: 2024-07-04T08:34:22.040+0000 7f678da6c640 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.334208+0000 front 2024-07-04T08:33:56.334234+0000 (oldest deadline 2024-07-04T08:34:19.834449+0000) 2024-07-04T08:34:22.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:21 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:21.917+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:22.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:21 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:21.917+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:22.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:21 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:21.917+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:22.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:22 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:22.607+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:22.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:22 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:22.607+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:23.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:22 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:22.628+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:23.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:22 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:22.628+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:23.051 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:22 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:22.619+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:23.051 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:22 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:22.619+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:23.051 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:22 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:22.619+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:23.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:22 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:22.854+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:23.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:22 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:22.854+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:23.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:22 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:22.798+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:23.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:22 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:22.798+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:23.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:22 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:22.798+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:23.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:22 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:22.936+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:23.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:22 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:22.936+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:23.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:22 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:22.936+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:23.551 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:23 smithi133 podman[76434]: 2024-07-04 08:34:23.359547185 +0000 UTC m=+5.651740654 container died 69b99062481086b1f324a4e1e2d3804086f78d0a451d3e6e6ed9257a15baaf1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-07-04T08:34:23.991 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:23 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:23.596+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:23.991 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:23 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:23.596+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:23.991 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:23 smithi133 podman[76434]: 2024-07-04 08:34:23.763448113 +0000 UTC m=+6.055641626 container cleanup 69b99062481086b1f324a4e1e2d3804086f78d0a451d3e6e6ed9257a15baaf1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-07-04T08:34:23.992 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:23 smithi133 bash[76434]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5 2024-07-04T08:34:23.992 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:23 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:23.573+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:23.992 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:23 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:23.573+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:23.992 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:23 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:23.573+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:24.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:23 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:23.899+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:24.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:23 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:23.899+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:24.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:23 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:23.818+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:24.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:23 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:23.818+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:24.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:23 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:23.818+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:24.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:23 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:23.654+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:24.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:23 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:23.654+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:24.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:23 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:23.899+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:24.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:23 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:23.899+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:24.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:23 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:23.899+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:24.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:24 smithi133 podman[76462]: 2024-07-04 08:34:24.006441213 +0000 UTC m=+0.643982562 container remove 69b99062481086b1f324a4e1e2d3804086f78d0a451d3e6e6ed9257a15baaf1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-04T08:34:24.555 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:24 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:24.550+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:24.556 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:24 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:24.550+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:24.556 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:24 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:24.550+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:24.556 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:24 smithi133 podman[76550]: 2024-07-04 08:34:24.221299234 +0000 UTC m=+0.016423935 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:34:24.556 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:24 smithi133 podman[76550]: 2024-07-04 08:34:24.340570427 +0000 UTC m=+0.135695109 container create 62dc54fd9d1d23ebd3de3b808c059abcce8856f2f03b777c4023d56cbaaeb536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0) 2024-07-04T08:34:24.556 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:24 smithi133 podman[76550]: 2024-07-04 08:34:24.568191279 +0000 UTC m=+0.363315967 container init 62dc54fd9d1d23ebd3de3b808c059abcce8856f2f03b777c4023d56cbaaeb536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:34:24.556 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:24 smithi133 podman[76550]: 2024-07-04 08:34:24.571220286 +0000 UTC m=+0.366344977 container start 62dc54fd9d1d23ebd3de3b808c059abcce8856f2f03b777c4023d56cbaaeb536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240625) 2024-07-04T08:34:24.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:24 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:24.627+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:24.879 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:24 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:24.627+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:25.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:24 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:24.571+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:25.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:24 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:24.571+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:25.051 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:24 smithi133 podman[76550]: 2024-07-04 08:34:24.668582724 +0000 UTC m=+0.463707421 container attach 62dc54fd9d1d23ebd3de3b808c059abcce8856f2f03b777c4023d56cbaaeb536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0) 2024-07-04T08:34:25.051 INFO:journalctl@ceph.osd.5.smithi133.stdout:Jul 04 08:34:24 smithi133 podman[76550]: 2024-07-04 08:34:24.763802047 +0000 UTC m=+0.558926746 container died 62dc54fd9d1d23ebd3de3b808c059abcce8856f2f03b777c4023d56cbaaeb536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:34:25.075 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:24 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:24.818+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:25.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:24 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:24.818+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:25.076 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:24 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:24.818+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:25.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:24 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:24.871+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:25.076 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:24 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:24.871+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:25.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:24 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:24.948+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:25.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:24 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:24.948+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:25.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:24 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:24.948+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:25.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.5.service' 2024-07-04T08:34:25.431 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:34:25.431 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-07-04T08:34:25.431 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-07-04T08:34:25.432 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.6 2024-07-04T08:34:25.781 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:25 smithi039 systemd[1]: Stopping Ceph osd.6 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:34:25.800 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:25 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:25.555+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:25.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:25 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:25.555+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:25.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:25 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:25.550+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:25.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:25 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:25.550+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:25.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:25 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:25.550+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:25.927 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:25 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:25.663+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:25.927 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:25 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:25.663+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:26.058 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:25 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:25.896+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:26.058 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:25 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:25.896+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:26.058 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:25 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:25.794+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:26.058 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:25 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:25.794+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:26.058 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:25 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:25.794+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:26.335 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:26.072+0000 7fd9fc800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:34:26.336 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:26.072+0000 7fd9fc800640 -1 osd.6 118 *** Got signal Terminated *** 2024-07-04T08:34:26.336 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:26.072+0000 7fd9fc800640 -1 osd.6 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:34:26.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:25 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:25.942+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:26.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:25 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:25.942+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:26.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:25 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:25.942+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:26.800 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:26 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:26.549+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:26.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:26 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:26.549+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:26.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:26 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:26.582+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:26.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:26 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:26.582+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:26.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:26 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:26.582+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:26.951 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:26 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:26.708+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:26.951 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:26 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:26.708+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:27.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:26.939+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:27.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:26.939+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:27.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:26.777+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:27.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:26.777+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:27.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:26 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:26.777+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:27.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:26 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:26.965+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:27.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:26 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:26.965+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:27.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:26 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:26.965+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:27.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:27 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:27.529+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:27.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:27 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:27.529+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:27.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:27 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:27.534+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:27.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:27 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:27.534+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:27.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:27 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:27.534+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:28.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:27 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:27.919+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:28.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:27 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:27.919+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:28.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:27 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:27.787+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:28.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:27 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:27.787+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:28.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:27 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:27.787+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:28.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:27 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:27.731+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:28.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:27 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:27.731+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:28.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:27 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:27.936+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:28.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:27 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:27.936+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:28.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:27 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:27.936+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:28.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:27 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:27.936+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:28.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:28 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:28.563+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:28.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:28 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:28.563+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:28.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:28 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:28.494+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:28.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:28 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:28.494+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:28.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:28 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:28.494+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:29.085 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:28 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:28.887+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:29.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:28 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:28.887+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:29.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:28 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:28.773+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:29.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:28 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:28.773+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:29.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:28 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:28.773+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:29.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:28 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:28.740+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:29.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:28 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:28.740+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:29.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:28 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:28.923+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:29.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:28 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:28.923+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:29.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:28 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:28.923+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:29.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:28 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:28.923+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:29.800 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:29 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:29.561+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:29.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:29 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:29.561+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:29.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:29 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:29.475+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:29.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:29 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:29.475+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:29.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:29 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:29.475+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:30.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:29 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:29.842+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:30.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:29 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:29.842+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:30.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:29 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:29.736+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:30.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:29 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:29.736+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:30.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:29 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:29.736+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:30.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:29 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:29.736+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:30.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:29 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:29.749+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:30.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:29 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:29.749+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:30.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:29 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:29.749+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:30.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:29 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:29.899+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:30.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:29 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:29.899+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:30.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:29 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:29.899+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:30.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:29 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:29.899+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:30.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:30 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:30.432+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:30.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:30 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:30.432+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:30.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:30 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:30.432+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:30.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:30 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:30.528+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:30.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:30 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:30.528+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:30.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:30 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:30.528+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:31.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:30.874+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:45.490624+0000 front 2024-07-04T08:33:45.490819+0000 (oldest deadline 2024-07-04T08:34:10.190672+0000) 2024-07-04T08:34:31.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:30.874+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.695357+0000 front 2024-07-04T08:33:57.695517+0000 (oldest deadline 2024-07-04T08:34:19.395632+0000) 2024-07-04T08:34:31.086 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6[65290]: 2024-07-04T08:34:30.874+0000 7fd9f8200640 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.698340+0000 front 2024-07-04T08:34:04.698231+0000 (oldest deadline 2024-07-04T08:34:30.597962+0000) 2024-07-04T08:34:31.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:30.739+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:31.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:30.739+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:31.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:30.739+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:31.087 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:30 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:30.739+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:31.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:30 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:30.781+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:31.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:30 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:30.781+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:31.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:30 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:30.781+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:31.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:30 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:30.916+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:31.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:30 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:30.916+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:31.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:30 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:30.916+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:31.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:30 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:30.916+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:31.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:31 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:31.383+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:31.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:31 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:31.383+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:31.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:31 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:31.383+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:31.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:31 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:31.529+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:31.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:31 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:31.529+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:31.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:31 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:31.529+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:31.939 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:31 smithi039 podman[148334]: 2024-07-04 08:34:31.663217659 +0000 UTC m=+5.730794685 container died ba3d86240c225c1541c765456df242c3827d198aa1a2f409f53ec633ee949941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-07-04T08:34:31.939 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:31 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:31.695+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:31.940 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:31 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:31.695+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:31.940 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:31 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:31.695+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:31.940 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:31 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:31.695+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:32.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:31 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:31.746+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:32.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:31 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:31.746+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:32.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:31 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:31.746+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:32.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:31 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:31.913+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:32.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:31 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:31.913+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:32.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:31 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:31.913+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:32.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:31 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:31.913+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:32.190 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:32 smithi039 podman[148334]: 2024-07-04 08:34:32.18221758 +0000 UTC m=+6.249794556 container cleanup ba3d86240c225c1541c765456df242c3827d198aa1a2f409f53ec633ee949941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-04T08:34:32.190 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:32 smithi039 bash[148334]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6 2024-07-04T08:34:32.672 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:32 smithi039 podman[148353]: 2024-07-04 08:34:32.417012346 +0000 UTC m=+0.740442664 container remove ba3d86240c225c1541c765456df242c3827d198aa1a2f409f53ec633ee949941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-07-04T08:34:32.731 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:32 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:32.345+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:32.731 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:32 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:32.345+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:32.731 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:32 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:32.345+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:32.731 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:32 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:32.573+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:32.731 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:32 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:32.573+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:32.731 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:32 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:32.573+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:33.030 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:32 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:32.755+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:33.030 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:32 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:32.755+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:33.031 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:32 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:32.755+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:33.031 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:32 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:32.933+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:33.031 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:32 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:32.933+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:33.031 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:32 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:32.933+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:33.031 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:32 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:32.933+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:33.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:32 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:32.686+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:33.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:32 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:32.686+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:33.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:32 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:32.686+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:33.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:32 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:32.686+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:33.524 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:33 smithi039 podman[148437]: 2024-07-04 08:34:33.234407954 +0000 UTC m=+0.126000507 container create f8f64c059c208da0fc5bea52aeffe0fe8b993a994955ab881f1539f72c66629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240625, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-07-04T08:34:33.524 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:33 smithi039 podman[148437]: 2024-07-04 08:34:33.143858872 +0000 UTC m=+0.035451421 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:34:33.524 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:33 smithi039 podman[148437]: 2024-07-04 08:34:33.538232819 +0000 UTC m=+0.429825419 container init f8f64c059c208da0fc5bea52aeffe0fe8b993a994955ab881f1539f72c66629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:34:33.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:33 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:33.326+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:33.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:33 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:33.326+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:33.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:33 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:33.326+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:33.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:33 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:33.531+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:33.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:33 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:33.531+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:33.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:33 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:33.531+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:33.835 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:33 smithi039 podman[148437]: 2024-07-04 08:34:33.550827148 +0000 UTC m=+0.442419677 container start f8f64c059c208da0fc5bea52aeffe0fe8b993a994955ab881f1539f72c66629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-04T08:34:33.835 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:33 smithi039 podman[148437]: 2024-07-04 08:34:33.645253982 +0000 UTC m=+0.536846595 container attach f8f64c059c208da0fc5bea52aeffe0fe8b993a994955ab881f1539f72c66629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-07-04T08:34:33.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:33 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:33.736+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:33.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:33 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:33.736+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:33.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:33 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:33.736+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:33.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:33 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:33.736+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:34.042 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:33 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:33.798+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:34.043 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:33 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:33.798+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:34.043 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:33 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:33.798+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:34.043 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:33 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:33.893+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:34.044 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:33 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:33.893+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:34.044 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:33 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:33.893+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:34.044 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:33 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:33.893+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:34.156 INFO:journalctl@ceph.osd.6.smithi039.stdout:Jul 04 08:34:33 smithi039 podman[148437]: 2024-07-04 08:34:33.877371739 +0000 UTC m=+0.768964270 container died f8f64c059c208da0fc5bea52aeffe0fe8b993a994955ab881f1539f72c66629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-07-04T08:34:34.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:34 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:34.282+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:34.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:34 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:34.282+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:34.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:34 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:34.282+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:34.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:34 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:34.505+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:34.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:34 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:34.505+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:34.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:34 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:34.505+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:34.595 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.6.service' 2024-07-04T08:34:34.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:34:34.716 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-07-04T08:34:34.716 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-07-04T08:34:34.716 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.7 2024-07-04T08:34:34.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:34 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:34.748+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:34.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:34 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:34.748+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:34.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:34 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:34.748+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:34.836 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:34 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:34.748+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:35.050 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:34 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:34.910+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:35.051 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:34 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:34.910+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:35.051 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:34 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:34.910+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:35.051 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:34 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:34.910+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:35.051 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:34 smithi050 systemd[1]: Stopping Ceph osd.7 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:34:35.051 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:34 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:34.840+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:35.051 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:34 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:34.840+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:35.051 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:34 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:34.840+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:35.051 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:35.065+0000 7fc41a648640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:34:35.051 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:35.065+0000 7fc41a648640 -1 osd.7 118 *** Got signal Terminated *** 2024-07-04T08:34:35.052 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:35.065+0000 7fc41a648640 -1 osd.7 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:34:35.528 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:35 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:35.284+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:35.528 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:35 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:35.284+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:35.528 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:35 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:35.284+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:35.800 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:35 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:35.542+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:35.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:35 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:35.542+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:35.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:35 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:35.542+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:36.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:35 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:35.731+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:36.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:35 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:35.731+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:36.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:35 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:35.731+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:36.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:35 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:35.731+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:36.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:35.953+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:36.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:35.953+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:36.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:35.953+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:36.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:35.953+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:36.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:35.812+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:36.130 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:35.812+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:36.130 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:35 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:35.812+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:36.550 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:36 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:36.496+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:36.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:36 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:36.496+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:36.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:36 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:36.496+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:36.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:36 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:36.253+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:36.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:36 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:36.253+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:36.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:36 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:36.253+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:37.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:36 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:36.772+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:37.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:36 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:36.772+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:37.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:36 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:36.772+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:37.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:36 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:36.772+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:37.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:36 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:36.772+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:37.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:36 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:36.860+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:37.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:36 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:36.860+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:37.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:36 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:36.860+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:37.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:36 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:36.954+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:37.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:36 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:36.954+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:37.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:36 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:36.954+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:37.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:36 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:36.954+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:37.529 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:37 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:37.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:37.529 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:37 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:37.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:37.529 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:37 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:37.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:37.529 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:37 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:37.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:37.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:37 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:37.542+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:37.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:37 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:37.542+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:37.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:37 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:37.542+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:37.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:37 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:37.542+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:38.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:37 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:37.784+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:38.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:37 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:37.784+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:38.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:37 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:37.784+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:38.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:37 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:37.784+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:38.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:37 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:37.784+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:38.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:37 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:37.871+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:38.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:37 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:37.871+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:38.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:37 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:37.871+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:38.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:37 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:37.871+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.400429+0000 front 2024-07-04T08:34:14.400457+0000 (oldest deadline 2024-07-04T08:34:37.300589+0000) 2024-07-04T08:34:38.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:37 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:37.918+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:38.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:37 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:37.918+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:38.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:37 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:37.918+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:38.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:37 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:37.918+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:38.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:37 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:37.918+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:38.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:38.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:38.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:38.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:38.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:38.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:38.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:38.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:39.050 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:38.567+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:39.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:38.567+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:39.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:38.567+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:39.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:38 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:38.567+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:39.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:38 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:38.830+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:39.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:38 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:38.830+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:39.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:38 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:38.830+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:39.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:38 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:38.830+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:39.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:38 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:38.830+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:39.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:38 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:38.857+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:39.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:38 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:38.857+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:39.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:38 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:38.857+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:39.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:38 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:38.857+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.400429+0000 front 2024-07-04T08:34:14.400457+0000 (oldest deadline 2024-07-04T08:34:37.300589+0000) 2024-07-04T08:34:39.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:38 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:38.923+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:39.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:38 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:38.923+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:39.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:38 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:38.923+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:39.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:38 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:38.923+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:39.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:38 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:38.923+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:39.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:39 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:39.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:39.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:39 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:39.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:39.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:39 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:39.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:39.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:39 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:39.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:40.050 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:39 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:39.597+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:40.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:39 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:39.597+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:40.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:39 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:39.597+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:40.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:39 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:39.597+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:40.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:39 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:39.786+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:40.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:39 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:39.786+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:40.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:39 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:39.786+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:40.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:39 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:39.786+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:40.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:39 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:39.786+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:40.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:39.886+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.496670+0000 front 2024-07-04T08:33:47.496813+0000 (oldest deadline 2024-07-04T08:34:12.196938+0000) 2024-07-04T08:34:40.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:39.886+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.798355+0000 front 2024-07-04T08:33:56.798451+0000 (oldest deadline 2024-07-04T08:34:22.098416+0000) 2024-07-04T08:34:40.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:39.886+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:04.399531+0000 front 2024-07-04T08:34:04.399731+0000 (oldest deadline 2024-07-04T08:34:29.699507+0000) 2024-07-04T08:34:40.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7[56163]: 2024-07-04T08:34:39.886+0000 7fc412c2c640 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.400429+0000 front 2024-07-04T08:34:14.400457+0000 (oldest deadline 2024-07-04T08:34:37.300589+0000) 2024-07-04T08:34:40.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:39.922+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:40.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:39.922+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:40.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:39.922+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:40.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:39.922+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:40.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:39 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:39.922+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:40.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:40 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:40.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:40.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:40 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:40.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:40.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:40 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:40.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:40.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:40 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:40.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:40.760 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:40 smithi050 podman[75809]: 2024-07-04 08:34:40.472579179 +0000 UTC m=+5.530868868 container died 6ead830bfc3da8a89e72f0e48e44371085248e9889505044c70e7a946fcc6dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-07-04T08:34:41.036 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:40 smithi050 podman[75809]: 2024-07-04 08:34:40.857436123 +0000 UTC m=+5.915725794 container cleanup 6ead830bfc3da8a89e72f0e48e44371085248e9889505044c70e7a946fcc6dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7, ceph=True, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-07-04T08:34:41.036 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:40 smithi050 bash[75809]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7 2024-07-04T08:34:41.036 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:40 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:40.958+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:41.036 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:40 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:40.958+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:41.036 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:40 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:40.958+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:41.036 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:40 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:40.958+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:41.037 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:40 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:40.958+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:41.050 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:40 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:40.646+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:41.050 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:40 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:40.646+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:41.050 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:40 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:40.646+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:41.050 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:40 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:40.646+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:41.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:40 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:40.799+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:41.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:40 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:40.799+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:41.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:40 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:40.799+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:41.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:40 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:40.799+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:41.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:40 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:40.799+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:41.379 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:41 smithi050 podman[75825]: 2024-07-04 08:34:41.051472239 +0000 UTC m=+0.572270013 container remove 6ead830bfc3da8a89e72f0e48e44371085248e9889505044c70e7a946fcc6dc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0) 2024-07-04T08:34:41.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:41 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:41.257+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:41.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:41 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:41.257+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:41.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:41 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:41.257+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:41.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:41 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:41.257+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:41.781 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:41 smithi050 podman[75917]: 2024-07-04 08:34:41.477172351 +0000 UTC m=+0.105149299 container create f99acdf3c03f3a6271a5d525ea51d38498e5b13548e83496c6fa32894336ab02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-07-04T08:34:41.781 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:41 smithi050 podman[75917]: 2024-07-04 08:34:41.389547426 +0000 UTC m=+0.017524385 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:34:41.781 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:41 smithi050 podman[75917]: 2024-07-04 08:34:41.696674005 +0000 UTC m=+0.324650955 container init f99acdf3c03f3a6271a5d525ea51d38498e5b13548e83496c6fa32894336ab02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7-deactivate, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:34:41.781 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:41 smithi050 podman[75917]: 2024-07-04 08:34:41.6999919 +0000 UTC m=+0.327968851 container start f99acdf3c03f3a6271a5d525ea51d38498e5b13548e83496c6fa32894336ab02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:34:41.781 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:41 smithi050 podman[75917]: 2024-07-04 08:34:41.796502983 +0000 UTC m=+0.424479930 container attach f99acdf3c03f3a6271a5d525ea51d38498e5b13548e83496c6fa32894336ab02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=) 2024-07-04T08:34:42.050 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:41 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:41.610+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:42.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:41 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:41.610+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:42.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:41 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:41.610+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:42.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:41 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:41.610+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:42.085 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:41 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:41.816+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:42.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:41 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:41.816+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:42.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:41 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:41.816+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:42.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:41 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:41.816+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:42.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:41 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:41.816+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:42.129 INFO:journalctl@ceph.osd.7.smithi050.stdout:Jul 04 08:34:41 smithi050 podman[75917]: 2024-07-04 08:34:41.903057795 +0000 UTC m=+0.531034752 container died f99acdf3c03f3a6271a5d525ea51d38498e5b13548e83496c6fa32894336ab02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240625, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-07-04T08:34:42.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:41 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:41.987+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:42.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:41 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:41.987+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:42.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:41 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:41.987+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:42.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:41 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:41.987+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:42.130 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:41 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:41.987+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:42.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:42.268+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:42.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:42.268+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:42.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:42.268+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:42.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:42.268+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:42.556 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.7.service' 2024-07-04T08:34:42.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:34:42.603 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-07-04T08:34:42.604 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-07-04T08:34:42.604 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.8 2024-07-04T08:34:42.908 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:42.577+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:42.908 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:42.577+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:42.909 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:42.577+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:42.909 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:42.577+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:42.909 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:42 smithi133 systemd[1]: Stopping Ceph osd.8 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:34:42.909 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:42.923+0000 7f27e471a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:34:42.909 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:42.923+0000 7f27e471a640 -1 osd.8 118 *** Got signal Terminated *** 2024-07-04T08:34:42.909 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:42 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:42.923+0000 7f27e471a640 -1 osd.8 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:34:43.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:42 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:42.854+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:43.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:42 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:42.854+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:43.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:42 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:42.854+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:43.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:42 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:42.854+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:43.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:42 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:42.854+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:43.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:42 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:42.980+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:43.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:42 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:42.980+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:43.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:42 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:42.980+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:43.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:42 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:42.980+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:43.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:42 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:42.980+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:43.546 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:43 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:43.302+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:43.546 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:43 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:43.302+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:43.546 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:43 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:43.302+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:43.546 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:43 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:43.302+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:43.800 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:43 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:43.561+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:43.800 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:43 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:43.561+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:43.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:43 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:43.561+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:43.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:43 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:43.561+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:44.238 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:43 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:43.876+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:44.238 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:43 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:43.876+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:44.238 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:43 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:43.876+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:44.238 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:43 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:43.876+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:44.238 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:43 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:43.876+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:44.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:43 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:43.947+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:44.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:43 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:43.947+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:44.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:43 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:43.947+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:44.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:43 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:43.947+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:44.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:43 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:43.947+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:44.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:44.305+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:44.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:44.305+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:44.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:44.305+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:44.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:44.305+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:44.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:44.305+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:45.050 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:44.593+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:45.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:44.593+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:45.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:44.593+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:45.051 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:44 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:44.593+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:45.264 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:44 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:44.925+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:45.264 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:44 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:44.925+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:45.264 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:44 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:44.925+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:45.264 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:44 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:44.925+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:45.264 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:44 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:44.925+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:45.264 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:44 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:44.925+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:45.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:44.930+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:45.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:44.930+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:45.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:44.930+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:45.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:44.930+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:45.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:44.930+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:45.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:44.930+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:45.800 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:45 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:45.558+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:45.800 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:45 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:45.558+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:45.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:45 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:45.558+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:45.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:45 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:45.558+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:45.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:45 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:45.335+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:45.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:45 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:45.335+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:45.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:45 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:45.335+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:45.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:45 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:45.335+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:45.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:45 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:45.335+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:46.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:45 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:45.908+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:46.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:45 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:45.908+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:46.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:45 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:45.908+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:46.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:45 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:45.908+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:46.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:45 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:45.908+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:46.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:45 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:45.908+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:46.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:45 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:45.950+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:46.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:45 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:45.950+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:46.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:45 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:45.950+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:46.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:45 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:45.950+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:46.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:45 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:45.950+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:46.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:45 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:45.950+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:46.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:46.533+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:46.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:46.533+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:46.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:46.533+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:46.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:46.533+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:46.801 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:46.533+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:17.599869+0000 front 2024-07-04T08:34:17.599815+0000 (oldest deadline 2024-07-04T08:34:46.400199+0000) 2024-07-04T08:34:46.802 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:46.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:46.802 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:46.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:46.802 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:46.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:46.802 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:46.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:46.803 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:46 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:46.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:47.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:46 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:46.864+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:47.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:46 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:46.864+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:47.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:46 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:46.864+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:47.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:46 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:46.864+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:47.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:46 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:46.864+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:47.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:46 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:46.864+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:47.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:46 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:46.965+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:47.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:46 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:46.965+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:47.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:46 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:46.965+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:47.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:46 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:46.965+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:47.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:46 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:46.965+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:47.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:46 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:46.965+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:47.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:47.313+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:47.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:47.313+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:47.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:47.313+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:47.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:47.313+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:47.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:47.313+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:47.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:47.551+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.095081+0000 front 2024-07-04T08:33:46.095197+0000 (oldest deadline 2024-07-04T08:34:11.395049+0000) 2024-07-04T08:34:47.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:47.551+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.296024+0000 front 2024-07-04T08:33:57.296003+0000 (oldest deadline 2024-07-04T08:34:21.396134+0000) 2024-07-04T08:34:47.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:47.551+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.397072+0000 front 2024-07-04T08:34:01.396972+0000 (oldest deadline 2024-07-04T08:34:29.597216+0000) 2024-07-04T08:34:47.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:47.551+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:14.298300+0000 front 2024-07-04T08:34:14.298181+0000 (oldest deadline 2024-07-04T08:34:37.098886+0000) 2024-07-04T08:34:47.552 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:47 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8[56370]: 2024-07-04T08:34:47.551+0000 7f27dccfe640 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:17.599869+0000 front 2024-07-04T08:34:17.599815+0000 (oldest deadline 2024-07-04T08:34:46.400199+0000) 2024-07-04T08:34:48.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:47 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:47.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:48.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:47 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:47.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:48.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:47 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:47.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:48.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:47 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:47.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:48.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:47 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:47.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:48.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:47 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:47.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:48.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:47 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:47.951+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:48.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:47 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:47.951+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:48.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:47 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:47.951+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:48.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:47 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:47.951+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:48.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:47 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:47.951+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:48.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:47 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:47.951+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:48.551 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:48 smithi133 podman[76651]: 2024-07-04 08:34:48.455303285 +0000 UTC m=+5.637946978 container died 1b4187dc50881484d761bd17594cc048994e4e605706d4b0802eb8f184b0f9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-07-04T08:34:48.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:48 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:48.314+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:48.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:48 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:48.314+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:48.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:48 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:48.314+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:48.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:48 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:48.314+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:48.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:48 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:48.314+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:49.025 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:48 smithi133 podman[76651]: 2024-07-04 08:34:48.856603235 +0000 UTC m=+6.039246919 container cleanup 1b4187dc50881484d761bd17594cc048994e4e605706d4b0802eb8f184b0f9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-04T08:34:49.026 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:48 smithi133 bash[76651]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8 2024-07-04T08:34:49.300 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:49 smithi133 podman[76665]: 2024-07-04 08:34:49.041128038 +0000 UTC m=+0.579754963 container remove 1b4187dc50881484d761bd17594cc048994e4e605706d4b0802eb8f184b0f9af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-07-04T08:34:49.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:48 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:48.896+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:49.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:48 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:48.896+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:49.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:48 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:48.896+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:49.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:48 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:48.896+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:49.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:48 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:48.896+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:49.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:48 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:48.896+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:49.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:48 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:48.967+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:49.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:48 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:48.967+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:49.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:48 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:48.967+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:49.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:48 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:48.967+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:49.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:48 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:48.967+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:49.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:48 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:48.967+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:49.600 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:49 smithi133 podman[76759]: 2024-07-04 08:34:49.382362995 +0000 UTC m=+0.017337833 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:34:49.600 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:49 smithi133 podman[76759]: 2024-07-04 08:34:49.500233127 +0000 UTC m=+0.135207950 container create 1a21a62c47fd457721121c1326d50d882a18c4869ea11f1d85850985a446ebcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-07-04T08:34:49.601 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:49 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:49.343+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:49.601 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:49 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:49.343+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:49.601 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:49 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:49.343+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:49.601 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:49 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:49.343+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:49.601 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:49 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:49.343+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:49.921 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:49 smithi133 podman[76759]: 2024-07-04 08:34:49.73599656 +0000 UTC m=+0.370971400 container init 1a21a62c47fd457721121c1326d50d882a18c4869ea11f1d85850985a446ebcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-07-04T08:34:49.921 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:49 smithi133 podman[76759]: 2024-07-04 08:34:49.738973924 +0000 UTC m=+0.373948751 container start 1a21a62c47fd457721121c1326d50d882a18c4869ea11f1d85850985a446ebcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-07-04T08:34:49.921 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:49 smithi133 podman[76759]: 2024-07-04 08:34:49.835954224 +0000 UTC m=+0.470929061 container attach 1a21a62c47fd457721121c1326d50d882a18c4869ea11f1d85850985a446ebcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux ) 2024-07-04T08:34:50.273 INFO:journalctl@ceph.osd.8.smithi133.stdout:Jul 04 08:34:49 smithi133 podman[76759]: 2024-07-04 08:34:49.936901393 +0000 UTC m=+0.571876234 container died 1a21a62c47fd457721121c1326d50d882a18c4869ea11f1d85850985a446ebcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-8-deactivate, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-07-04T08:34:50.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:49 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:49.850+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:50.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:49 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:49.850+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:50.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:49 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:49.850+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:50.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:49 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:49.850+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:50.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:49 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:49.850+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:50.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:49 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:49.850+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:50.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:49 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:49.938+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:50.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:49 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:49.938+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:50.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:49 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:49.938+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:50.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:49 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:49.938+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:50.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:49 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:49.938+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:50.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:49 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:49.938+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:50.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:50 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:50.381+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:50.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:50 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:50.381+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:50.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:50 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:50.381+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:50.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:50 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:50.381+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:50.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:50 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:50.381+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:50.603 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.8.service' 2024-07-04T08:34:50.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:34:50.644 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-07-04T08:34:50.644 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-07-04T08:34:50.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.9 2024-07-04T08:34:51.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:50 smithi039 systemd[1]: Stopping Ceph osd.9 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:34:51.087 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:50 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:50.813+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:51.087 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:50 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:50.813+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:51.087 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:50 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:50.813+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:51.087 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:50 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:50.813+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:51.087 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:50 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:50.813+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:51.087 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:50 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:50.813+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:51.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:50 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:50.908+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:51.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:50 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:50.908+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:51.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:50 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:50.908+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:51.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:50 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:50.908+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:51.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:50 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:50.908+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:51.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:50 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:50.908+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:51.585 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:51 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:51.258+0000 7f2ca6400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:34:51.586 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:51 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:51.258+0000 7f2ca6400640 -1 osd.9 118 *** Got signal Terminated *** 2024-07-04T08:34:51.586 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:51 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:51.258+0000 7f2ca6400640 -1 osd.9 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:34:51.782 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:51 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:51.339+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:51.782 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:51 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:51.339+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:51.782 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:51 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:51.339+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:51.782 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:51 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:51.339+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:51.782 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:51 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:51.339+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:52.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:51 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:51.820+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:52.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:51 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:51.820+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:52.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:51 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:51.820+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:52.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:51 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:51.820+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:52.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:51 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:51.820+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:52.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:51 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:51.820+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:52.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:51 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:51.935+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:52.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:51 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:51.935+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:52.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:51 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:51.935+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:52.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:51 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:51.935+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:52.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:51 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:51.935+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:52.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:51 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:51.935+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:52.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:52.345+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:52.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:52.345+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:52.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:52.345+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:52.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:52.345+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:52.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:52.345+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:52.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:52.345+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:34:53.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:52 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:52.869+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:53.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:52 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:52.869+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:53.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:52 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:52.869+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:53.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:52 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:52.869+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:53.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:52 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:52.869+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:53.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:52 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:52.869+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:53.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:52 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:52.942+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:53.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:52 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:52.942+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:53.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:52 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:52.942+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:53.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:52 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:52.942+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:53.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:52 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:52.942+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:53.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:52 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:52.942+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:53.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:53 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:53.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:53.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:53 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:53.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:53.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:53 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:53.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:53.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:53 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:53.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:53.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:53 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:53.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:53.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:53 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:53.301+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:34:54.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:53.880+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:54.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:53.880+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:54.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:53.880+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:54.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:53.880+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:54.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:53.880+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:54.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:53 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:53.880+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:54.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:53 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:53.982+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:54.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:53 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:53.982+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:54.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:53 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:53.982+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:54.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:53 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:53.982+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:54.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:53 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:53.982+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:54.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:53 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:53.982+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:54.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:54 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:54.324+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:54.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:54 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:54.324+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:54.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:54 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:54.324+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:54.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:54 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:54.324+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:54.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:54 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:54.324+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:54.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:54 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:54.324+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:34:55.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:54 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:54.897+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:55.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:54 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:54.897+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:55.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:54 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:54.897+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:55.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:54 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:54.897+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:55.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:54 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:54.897+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:55.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:54 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:54.897+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:55.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:55 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:55.027+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:55.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:55 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:55.027+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:55.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:55 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:55.027+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:55.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:55 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:55.027+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:55.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:55 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:55.027+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:55.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:55 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:55.027+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:55.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:55 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:55.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:55.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:55 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:55.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:55.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:55 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:55.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:55.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:55 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:55.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:55.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:55 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:55.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:55.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:55 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:55.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:34:56.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:55 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:55.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:37.524106+0000 front 2024-07-04T08:33:37.524478+0000 (oldest deadline 2024-07-04T08:34:02.224256+0000) 2024-07-04T08:34:56.335 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:55 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:55.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:47.427309+0000 front 2024-07-04T08:33:47.427382+0000 (oldest deadline 2024-07-04T08:34:12.727183+0000) 2024-07-04T08:34:56.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:55 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:55.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:56.828964+0000 front 2024-07-04T08:33:56.829093+0000 (oldest deadline 2024-07-04T08:34:18.529240+0000) 2024-07-04T08:34:56.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:55 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:55.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:05.532167+0000 front 2024-07-04T08:34:05.532011+0000 (oldest deadline 2024-07-04T08:34:29.632022+0000) 2024-07-04T08:34:56.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:55 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:55.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:13.133861+0000 front 2024-07-04T08:34:13.134101+0000 (oldest deadline 2024-07-04T08:34:36.633932+0000) 2024-07-04T08:34:56.336 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:55 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9[72119]: 2024-07-04T08:34:55.912+0000 7f2c9ce00640 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:18.936068+0000 front 2024-07-04T08:34:18.935950+0000 (oldest deadline 2024-07-04T08:34:44.236231+0000) 2024-07-04T08:34:56.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:56 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:56.048+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:56.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:56 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:56.048+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:56.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:56 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:56.048+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:56.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:56 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:56.048+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:56.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:56 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:56.048+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:56.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:56 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:56.048+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:56.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:56 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:56.350+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:56.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:56 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:56.350+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:56.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:56 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:56.350+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:56.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:56 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:56.350+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:56.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:56 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:56.350+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:56.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:56 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:56.350+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:34:57.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:56 smithi039 podman[148549]: 2024-07-04 08:34:56.764213829 +0000 UTC m=+5.641676522 container died e3d35e52eb8ee919e99422928380cc50f4c2196e726bb6a497b0d1ba96a1eb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:34:57.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:57 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:57.092+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:57.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:57 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:57.092+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:57.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:57 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:57.092+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:57.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:57 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:57.092+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:57.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:57 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:57.092+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:57.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:57 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:57.092+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:57.583 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:57 smithi039 podman[148549]: 2024-07-04 08:34:57.352972727 +0000 UTC m=+6.230435457 container cleanup e3d35e52eb8ee919e99422928380cc50f4c2196e726bb6a497b0d1ba96a1eb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:34:57.583 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:57 smithi039 bash[148549]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9 2024-07-04T08:34:57.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:57 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:57.367+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:57.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:57 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:57.367+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:57.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:57 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:57.367+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:57.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:57 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:57.367+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:57.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:57 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:57.367+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:57.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:57 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:57.367+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:34:57.835 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:57 smithi039 podman[148571]: 2024-07-04 08:34:57.595877323 +0000 UTC m=+0.814793815 container remove e3d35e52eb8ee919e99422928380cc50f4c2196e726bb6a497b0d1ba96a1eb9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-04T08:34:58.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:58 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:58.126+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:58.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:58 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:58.126+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:58.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:58 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:58.126+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:58.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:58 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:58.126+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:58.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:58 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:58.126+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:58.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:58 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:58.126+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:58.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:58 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:58.126+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:29.506264+0000 front 2024-07-04T08:34:29.506378+0000 (oldest deadline 2024-07-04T08:34:57.106278+0000) 2024-07-04T08:34:58.511 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:58 smithi039 podman[148658]: 2024-07-04 08:34:58.136763814 +0000 UTC m=+0.037156919 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:34:58.511 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:58 smithi039 podman[148658]: 2024-07-04 08:34:58.247007259 +0000 UTC m=+0.147400384 container create 34407ff871d792674c0b1091a9a7d9b6b8f2fc6db87f7a3cbf7c285cd96a4caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-07-04T08:34:58.511 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:58 smithi039 podman[148658]: 2024-07-04 08:34:58.525537884 +0000 UTC m=+0.425931053 container init 34407ff871d792674c0b1091a9a7d9b6b8f2fc6db87f7a3cbf7c285cd96a4caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-07-04T08:34:58.775 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:58 smithi039 podman[148658]: 2024-07-04 08:34:58.536673442 +0000 UTC m=+0.437066597 container start 34407ff871d792674c0b1091a9a7d9b6b8f2fc6db87f7a3cbf7c285cd96a4caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:34:58.775 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:58 smithi039 podman[148658]: 2024-07-04 08:34:58.599270131 +0000 UTC m=+0.499663234 container attach 34407ff871d792674c0b1091a9a7d9b6b8f2fc6db87f7a3cbf7c285cd96a4caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.schema-version=1.0) 2024-07-04T08:34:58.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:58 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:58.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:58.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:58 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:58.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:58.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:58 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:58.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:58.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:58 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:58.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:58.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:58 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:58.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:58.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:58 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:58.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:34:59.086 INFO:journalctl@ceph.osd.9.smithi039.stdout:Jul 04 08:34:58 smithi039 podman[148658]: 2024-07-04 08:34:58.849067269 +0000 UTC m=+0.749460378 container died 34407ff871d792674c0b1091a9a7d9b6b8f2fc6db87f7a3cbf7c285cd96a4caa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-9-deactivate, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:34:59.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:59 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:59.100+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:34:59.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:59 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:59.100+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:34:59.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:59 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:59.100+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:34:59.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:59 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:59.100+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:34:59.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:59 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:59.100+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:34:59.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:59 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:59.100+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:34:59.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:59 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:34:59.100+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:29.506264+0000 front 2024-07-04T08:34:29.506378+0000 (oldest deadline 2024-07-04T08:34:57.106278+0000) 2024-07-04T08:34:59.622 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.9.service' 2024-07-04T08:34:59.724 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:34:59.724 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-07-04T08:34:59.725 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-07-04T08:34:59.725 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.10 2024-07-04T08:34:59.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:59 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:59.336+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:34:59.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:59 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:59.336+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:34:59.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:59 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:59.336+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:34:59.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:59 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:59.336+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:34:59.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:59 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:59.336+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:34:59.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:34:59 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:34:59.336+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:00.037 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:34:59 smithi050 systemd[1]: Stopping Ceph osd.10 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:00.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.052+0000 7fa20ce58640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:00.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.052+0000 7fa20ce58640 -1 osd.10 118 *** Got signal Terminated *** 2024-07-04T08:35:00.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.052+0000 7fa20ce58640 -1 osd.10 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:35:00.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.130+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:35:00.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.130+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:35:00.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.130+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:35:00.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.130+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:35:00.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.130+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:35:00.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.130+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:35:00.380 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:00 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:00.130+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:29.506264+0000 front 2024-07-04T08:34:29.506378+0000 (oldest deadline 2024-07-04T08:34:57.106278+0000) 2024-07-04T08:35:00.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:00 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:00.299+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:00.566 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:00 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:00.299+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:00.566 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:00 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:00.299+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:00.566 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:00 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:00.299+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:00.567 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:00 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:00.299+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:00.567 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:00 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:00.299+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:01.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:01 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:01.103+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:35:01.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:01 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:01.103+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:35:01.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:01 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:01.103+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:35:01.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:01 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:01.103+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:35:01.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:01 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:01.103+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:35:01.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:01 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:01.103+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:35:01.379 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:01 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:01.103+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:29.506264+0000 front 2024-07-04T08:34:29.506378+0000 (oldest deadline 2024-07-04T08:34:57.106278+0000) 2024-07-04T08:35:01.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:01 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:01.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:01.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:01 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:01.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:01.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:01 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:01.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:01.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:01 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:01.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:01.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:01 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:01.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:01.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:01 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:01.341+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:02.629 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:02 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:02.153+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:35:02.629 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:02 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:02.153+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:35:02.629 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:02 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:02.153+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:35:02.629 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:02 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:02.153+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:35:02.629 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:02 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:02.153+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:35:02.629 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:02 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:02.153+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:35:02.629 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:02 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:02.153+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:29.506264+0000 front 2024-07-04T08:34:29.506378+0000 (oldest deadline 2024-07-04T08:34:57.106278+0000) 2024-07-04T08:35:02.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:02 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:02.380+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:02.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:02 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:02.380+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:02.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:02 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:02.380+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:02.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:02 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:02.380+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:02.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:02 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:02.380+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:02.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:02 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:02.380+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:02.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:02 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:02.380+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:03.530 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:03 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:03.193+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:35:03.530 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:03 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:03.193+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:35:03.531 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:03 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:03.193+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:35:03.531 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:03 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:03.193+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:35:03.531 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:03 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:03.193+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:35:03.531 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:03 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:03.193+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:35:03.531 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:03 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:03.193+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:29.506264+0000 front 2024-07-04T08:34:29.506378+0000 (oldest deadline 2024-07-04T08:34:57.106278+0000) 2024-07-04T08:35:03.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:03.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:03.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:03.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:03.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:03.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:03.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:03.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:03.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:03.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:03.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:03.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:03.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:03 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:03.346+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:04.548 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:04.176+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:39.998429+0000 front 2024-07-04T08:33:39.998534+0000 (oldest deadline 2024-07-04T08:34:04.098466+0000) 2024-07-04T08:35:04.548 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:04.176+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:49.300004+0000 front 2024-07-04T08:33:49.300102+0000 (oldest deadline 2024-07-04T08:34:10.400074+0000) 2024-07-04T08:35:04.548 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:04.176+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:57.301859+0000 front 2024-07-04T08:33:57.301985+0000 (oldest deadline 2024-07-04T08:34:21.401915+0000) 2024-07-04T08:35:04.548 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:04.176+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6814 osd.3 since back 2024-07-04T08:34:01.402542+0000 front 2024-07-04T08:34:01.402695+0000 (oldest deadline 2024-07-04T08:34:27.302476+0000) 2024-07-04T08:35:04.548 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:04.176+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:12.603446+0000 front 2024-07-04T08:34:12.603474+0000 (oldest deadline 2024-07-04T08:34:37.903626+0000) 2024-07-04T08:35:04.549 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:04.176+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:22.004784+0000 front 2024-07-04T08:34:22.004948+0000 (oldest deadline 2024-07-04T08:34:44.304645+0000) 2024-07-04T08:35:04.549 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:04 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10[63146]: 2024-07-04T08:35:04.176+0000 7fa209444640 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:29.506264+0000 front 2024-07-04T08:34:29.506378+0000 (oldest deadline 2024-07-04T08:34:57.106278+0000) 2024-07-04T08:35:04.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:04 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:04.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:04.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:04 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:04.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:04.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:04 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:04.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:04.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:04 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:04.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:04.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:04 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:04.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:04.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:04 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:04.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:04.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:04 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:04.330+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:05.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:05 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:05.333+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:05.800 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:05 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:05.333+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:05.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:05 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:05.333+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:05.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:05 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:05.333+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:05.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:05 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:05.333+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:05.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:05 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:05.333+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:05.801 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:05 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:05.333+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:05.804 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:05 smithi050 podman[76020]: 2024-07-04 08:35:05.475143371 +0000 UTC m=+5.534612390 container died a7adfbf174d74e4943b28e50a86e0e1529b699a4538d2e411797ddc9e2af1815 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux ) 2024-07-04T08:35:06.063 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:05 smithi050 podman[76020]: 2024-07-04 08:35:05.910734411 +0000 UTC m=+5.970203409 container cleanup a7adfbf174d74e4943b28e50a86e0e1529b699a4538d2e411797ddc9e2af1815 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10, ceph=True, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-07-04T08:35:06.063 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:05 smithi050 bash[76020]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10 2024-07-04T08:35:06.378 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:06 smithi050 podman[76048]: 2024-07-04 08:35:06.078589687 +0000 UTC m=+0.601765355 container remove a7adfbf174d74e4943b28e50a86e0e1529b699a4538d2e411797ddc9e2af1815 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-04T08:35:06.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:06 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:06.288+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:06.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:06 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:06.288+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:06.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:06 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:06.288+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:06.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:06 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:06.288+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:06.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:06 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:06.288+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:06.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:06 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:06.288+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:06.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:06 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:06.288+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:06.790 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:06 smithi050 podman[76135]: 2024-07-04 08:35:06.479281022 +0000 UTC m=+0.109144740 container create 31305184d4bb2ffa8e8fe37bf482b5678cb144649c918c7302e6716169e0615a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-07-04T08:35:06.791 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:06 smithi050 podman[76135]: 2024-07-04 08:35:06.38788314 +0000 UTC m=+0.017746859 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:35:06.791 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:06 smithi050 podman[76135]: 2024-07-04 08:35:06.723139495 +0000 UTC m=+0.353003208 container init 31305184d4bb2ffa8e8fe37bf482b5678cb144649c918c7302e6716169e0615a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:35:06.791 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:06 smithi050 podman[76135]: 2024-07-04 08:35:06.72650681 +0000 UTC m=+0.356370528 container start 31305184d4bb2ffa8e8fe37bf482b5678cb144649c918c7302e6716169e0615a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-07-04T08:35:06.791 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:06 smithi050 podman[76135]: 2024-07-04 08:35:06.806311317 +0000 UTC m=+0.436175039 container attach 31305184d4bb2ffa8e8fe37bf482b5678cb144649c918c7302e6716169e0615a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:35:07.129 INFO:journalctl@ceph.osd.10.smithi050.stdout:Jul 04 08:35:06 smithi050 podman[76135]: 2024-07-04 08:35:06.919871612 +0000 UTC m=+0.549735344 container died 31305184d4bb2ffa8e8fe37bf482b5678cb144649c918c7302e6716169e0615a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-07-04T08:35:07.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:07.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:07.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:07.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:07.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:07.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:07.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:07.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:07.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:07.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:07.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:07.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:07.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:07 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:07.259+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:07.669 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.10.service' 2024-07-04T08:35:07.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:07.715 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-07-04T08:35:07.716 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-07-04T08:35:07.716 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.11 2024-07-04T08:35:08.050 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:07 smithi133 systemd[1]: Stopping Ceph osd.11 for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:08.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.066+0000 7f7f53201640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:08.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.066+0000 7f7f53201640 -1 osd.11 118 *** Got signal Terminated *** 2024-07-04T08:35:08.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.066+0000 7f7f53201640 -1 osd.11 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-04T08:35:08.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.231+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:08.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.231+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:08.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.231+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:08.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.231+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:08.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.231+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:08.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.231+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:08.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:08 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:08.231+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:09.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:09.255+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:09.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:09.255+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:09.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:09.255+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:09.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:09.255+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:09.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:09.255+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:09.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:09.255+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:09.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:09 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:09.255+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:10.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:10.237+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:10.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:10.237+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:10.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:10.237+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:10.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:10.237+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:10.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:10.237+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:10.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:10.237+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:10.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:10.237+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:10.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:10 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:10.237+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6820 osd.8 since back 2024-07-04T08:34:47.192900+0000 front 2024-07-04T08:34:47.192932+0000 (oldest deadline 2024-07-04T08:35:09.492975+0000) 2024-07-04T08:35:11.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:11.211+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:11.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:11.211+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:11.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:11.211+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:11.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:11.211+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:11.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:11.211+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:11.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:11.211+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:11.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:11.211+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:11.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:11 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:11.211+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6820 osd.8 since back 2024-07-04T08:34:47.192900+0000 front 2024-07-04T08:34:47.192932+0000 (oldest deadline 2024-07-04T08:35:09.492975+0000) 2024-07-04T08:35:12.550 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:12.251+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6806 osd.0 since back 2024-07-04T08:33:38.080848+0000 front 2024-07-04T08:33:38.080669+0000 (oldest deadline 2024-07-04T08:34:01.580614+0000) 2024-07-04T08:35:12.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:12.251+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-07-04T08:33:46.182195+0000 front 2024-07-04T08:33:46.182048+0000 (oldest deadline 2024-07-04T08:34:11.482111+0000) 2024-07-04T08:35:12.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:12.251+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-07-04T08:33:55.483646+0000 front 2024-07-04T08:33:55.483590+0000 (oldest deadline 2024-07-04T08:34:18.383753+0000) 2024-07-04T08:35:12.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:12.251+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-07-04T08:34:11.186877+0000 front 2024-07-04T08:34:11.186967+0000 (oldest deadline 2024-07-04T08:34:37.086772+0000) 2024-07-04T08:35:12.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:12.251+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-07-04T08:34:21.088439+0000 front 2024-07-04T08:34:21.088578+0000 (oldest deadline 2024-07-04T08:34:43.388632+0000) 2024-07-04T08:35:12.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:12.251+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.39:6822 osd.6 since back 2024-07-04T08:34:26.290082+0000 front 2024-07-04T08:34:26.289981+0000 (oldest deadline 2024-07-04T08:34:52.089964+0000) 2024-07-04T08:35:12.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:12.251+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-07-04T08:34:37.291700+0000 front 2024-07-04T08:34:37.291629+0000 (oldest deadline 2024-07-04T08:35:01.891906+0000) 2024-07-04T08:35:12.551 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:12 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11[63391]: 2024-07-04T08:35:12.251+0000 7f7f4b7e5640 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.133:6820 osd.8 since back 2024-07-04T08:34:47.192900+0000 front 2024-07-04T08:34:47.192932+0000 (oldest deadline 2024-07-04T08:35:09.492975+0000) 2024-07-04T08:35:13.717 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:13 smithi133 podman[76872]: 2024-07-04 08:35:13.483029872 +0000 UTC m=+5.556186464 container died 2d09fe9c181a5e524dfae44015601bb8c0bb9d524d4bf9293ca84513da9746c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:35:14.001 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:13 smithi133 podman[76872]: 2024-07-04 08:35:13.815575531 +0000 UTC m=+5.888732131 container cleanup 2d09fe9c181a5e524dfae44015601bb8c0bb9d524d4bf9293ca84513da9746c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:35:14.001 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:13 smithi133 bash[76872]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11 2024-07-04T08:35:14.300 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:14 smithi133 podman[76887]: 2024-07-04 08:35:14.016906367 +0000 UTC m=+0.533515663 container remove 2d09fe9c181a5e524dfae44015601bb8c0bb9d524d4bf9293ca84513da9746c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0) 2024-07-04T08:35:14.706 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:14 smithi133 podman[76975]: 2024-07-04 08:35:14.341654541 +0000 UTC m=+0.017055517 image pull ef295368c83494faa28834a2410e04f12db7b4bce9484d8893e11e38b9bbbd7f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35 2024-07-04T08:35:14.706 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:14 smithi133 podman[76975]: 2024-07-04 08:35:14.442581972 +0000 UTC m=+0.117982948 container create 034aa9ddb714684d9969f92e0eb9b0ecd8a133e05e4031eb5c585ea040943f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.build-date=20240625) 2024-07-04T08:35:14.706 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:14 smithi133 podman[76975]: 2024-07-04 08:35:14.64679616 +0000 UTC m=+0.322197157 container init 034aa9ddb714684d9969f92e0eb9b0ecd8a133e05e4031eb5c585ea040943f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240625, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS) 2024-07-04T08:35:14.706 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:14 smithi133 podman[76975]: 2024-07-04 08:35:14.649851796 +0000 UTC m=+0.325252778 container start 034aa9ddb714684d9969f92e0eb9b0ecd8a133e05e4031eb5c585ea040943f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS) 2024-07-04T08:35:15.050 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:14 smithi133 podman[76975]: 2024-07-04 08:35:14.721650763 +0000 UTC m=+0.397051740 container attach 034aa9ddb714684d9969f92e0eb9b0ecd8a133e05e4031eb5c585ea040943f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0) 2024-07-04T08:35:15.050 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:14 smithi133 podman[76975]: 2024-07-04 08:35:14.837040388 +0000 UTC m=+0.512441369 container died 034aa9ddb714684d9969f92e0eb9b0ecd8a133e05e4031eb5c585ea040943f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11-deactivate, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:35:15.352 INFO:journalctl@ceph.osd.11.smithi133.stdout:Jul 04 08:35:15 smithi133 podman[76998]: 2024-07-04 08:35:15.366402485 +0000 UTC m=+0.523930772 container remove 034aa9ddb714684d9969f92e0eb9b0ecd8a133e05e4031eb5c585ea040943f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-07-04T08:35:15.368 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@osd.11.service' 2024-07-04T08:35:15.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:15.406 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-07-04T08:35:15.406 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-07-04T08:35:15.406 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.a 2024-07-04T08:35:15.835 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:35:15 smithi039 systemd[1]: Stopping Ceph mds.a for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:16.336 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:35:15 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a[78830]: 2024-07-04T08:35:15.985+0000 7f8e3e400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:16.336 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:35:15 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a[78830]: 2024-07-04T08:35:15.985+0000 7f8e3e400640 -1 mds.a *** got signal Terminated *** 2024-07-04T08:35:19.488 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:35:19 smithi039 podman[148772]: 2024-07-04 08:35:19.220944859 +0000 UTC m=+3.354391895 container died c8d3c9318c52d513d27443997d6dce74ed25659db5df6e5f5f9396a9bee912cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2) 2024-07-04T08:35:19.835 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:35:19 smithi039 podman[148772]: 2024-07-04 08:35:19.695649269 +0000 UTC m=+3.829096298 container cleanup c8d3c9318c52d513d27443997d6dce74ed25659db5df6e5f5f9396a9bee912cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-07-04T08:35:19.835 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:35:19 smithi039 bash[148772]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a 2024-07-04T08:35:20.308 INFO:journalctl@ceph.mds.a.smithi039.stdout:Jul 04 08:35:19 smithi039 podman[148797]: 2024-07-04 08:35:19.880268561 +0000 UTC m=+0.635715562 container remove c8d3c9318c52d513d27443997d6dce74ed25659db5df6e5f5f9396a9bee912cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:35:20.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.a.service' 2024-07-04T08:35:20.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:20.459 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-07-04T08:35:20.459 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-07-04T08:35:20.459 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.d 2024-07-04T08:35:20.585 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:35:20 smithi039 systemd[1]: Stopping Ceph mds.d for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:21.335 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:35:21 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d[77659]: 2024-07-04T08:35:21.074+0000 7f96b7e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:21.335 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:35:21 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d[77659]: 2024-07-04T08:35:21.074+0000 7f96b7e00640 -1 mds.d *** got signal Terminated *** 2024-07-04T08:35:23.506 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:35:23 smithi039 podman[148904]: 2024-07-04 08:35:23.237535136 +0000 UTC m=+2.312889786 container stop 6d577a8c2acde370defa56d20d34a1359c17b9ed991c71c1b48e8a053bddad68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, ceph=True) 2024-07-04T08:35:23.506 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:35:23 smithi039 podman[148904]: 2024-07-04 08:35:23.265599463 +0000 UTC m=+2.340954127 container died 6d577a8c2acde370defa56d20d34a1359c17b9ed991c71c1b48e8a053bddad68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-07-04T08:35:23.835 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:35:23 smithi039 podman[148904]: 2024-07-04 08:35:23.733013112 +0000 UTC m=+2.808367741 container cleanup 6d577a8c2acde370defa56d20d34a1359c17b9ed991c71c1b48e8a053bddad68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625) 2024-07-04T08:35:23.835 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:35:23 smithi039 bash[148904]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d 2024-07-04T08:35:24.335 INFO:journalctl@ceph.mds.d.smithi039.stdout:Jul 04 08:35:23 smithi039 podman[148918]: 2024-07-04 08:35:23.934658884 +0000 UTC m=+0.674218940 container remove 6d577a8c2acde370defa56d20d34a1359c17b9ed991c71c1b48e8a053bddad68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-d, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-07-04T08:35:24.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.d.service' 2024-07-04T08:35:24.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:24.565 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-07-04T08:35:24.565 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-07-04T08:35:24.565 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.g 2024-07-04T08:35:25.085 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:35:24 smithi039 systemd[1]: Stopping Ceph mds.g for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:25.586 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:35:25 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-g[78423]: 2024-07-04T08:35:25.185+0000 7f13f1c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:25.586 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:35:25 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-g[78423]: 2024-07-04T08:35:25.185+0000 7f13f1c00640 -1 mds.g *** got signal Terminated *** 2024-07-04T08:35:26.979 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:35:26 smithi039 podman[149024]: 2024-07-04 08:35:26.68302863 +0000 UTC m=+1.633292812 container died dc3524787faaaa36d46967ad5404726518b3bb604ba9eb0da21c03f1664089d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-07-04T08:35:27.335 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:35:27 smithi039 podman[149024]: 2024-07-04 08:35:27.179136121 +0000 UTC m=+2.129400299 container cleanup dc3524787faaaa36d46967ad5404726518b3bb604ba9eb0da21c03f1664089d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:35:27.335 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:35:27 smithi039 bash[149024]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-g 2024-07-04T08:35:27.668 INFO:journalctl@ceph.mds.g.smithi039.stdout:Jul 04 08:35:27 smithi039 podman[149037]: 2024-07-04 08:35:27.455474058 +0000 UTC m=+0.756741640 container remove dc3524787faaaa36d46967ad5404726518b3bb604ba9eb0da21c03f1664089d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-g, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:35:27.714 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.g.service' 2024-07-04T08:35:27.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:27.824 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-07-04T08:35:27.825 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-07-04T08:35:27.825 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.j 2024-07-04T08:35:27.961 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:35:27 smithi039 systemd[1]: Stopping Ceph mds.j for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:28.664 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:35:28 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j[77262]: 2024-07-04T08:35:28.422+0000 7f27f8600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:28.664 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:35:28 smithi039 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j[77262]: 2024-07-04T08:35:28.422+0000 7f27f8600640 -1 mds.j *** got signal Terminated *** 2024-07-04T08:35:32.585 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:35:32 smithi039 podman[149143]: 2024-07-04 08:35:32.337453142 +0000 UTC m=+4.024576708 container died 1504f2280418f9d2897594d5e6d63ac9efa38f4128e426c10afd4b9a2aad16ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625) 2024-07-04T08:35:32.996 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:35:32 smithi039 podman[149143]: 2024-07-04 08:35:32.799491383 +0000 UTC m=+4.486614955 container cleanup 1504f2280418f9d2897594d5e6d63ac9efa38f4128e426c10afd4b9a2aad16ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:35:32.996 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:35:32 smithi039 bash[149143]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j 2024-07-04T08:35:33.334 INFO:journalctl@ceph.mds.j.smithi039.stdout:Jul 04 08:35:33 smithi039 podman[149169]: 2024-07-04 08:35:33.010891959 +0000 UTC m=+0.650632283 container remove 1504f2280418f9d2897594d5e6d63ac9efa38f4128e426c10afd4b9a2aad16ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b) 2024-07-04T08:35:33.555 DEBUG:teuthology.orchestra.run.smithi039:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.j.service' 2024-07-04T08:35:33.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:33.664 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-07-04T08:35:33.664 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-07-04T08:35:33.664 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.b 2024-07-04T08:35:33.954 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:35:33 smithi050 systemd[1]: Stopping Ceph mds.b for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:34.241 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:35:33 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b[68787]: 2024-07-04T08:35:33.969+0000 7fe12c8bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:34.242 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:35:33 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b[68787]: 2024-07-04T08:35:33.969+0000 7fe12c8bd640 -1 mds.b *** got signal Terminated *** 2024-07-04T08:35:35.354 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:35:35 smithi050 podman[76239]: 2024-07-04 08:35:35.096403594 +0000 UTC m=+1.213113145 container died 4c3b3ec82ea0845436c5c8f3ef55571e78e8a5b16b3148ff3757d0e440a99c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, ceph=True) 2024-07-04T08:35:35.629 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:35:35 smithi050 podman[76239]: 2024-07-04 08:35:35.452755641 +0000 UTC m=+1.569465190 container cleanup 4c3b3ec82ea0845436c5c8f3ef55571e78e8a5b16b3148ff3757d0e440a99c0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True) 2024-07-04T08:35:35.629 INFO:journalctl@ceph.mds.b.smithi050.stdout:Jul 04 08:35:35 smithi050 bash[76239]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-b 2024-07-04T08:35:35.875 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.b.service' 2024-07-04T08:35:35.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:35.911 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-07-04T08:35:35.911 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-07-04T08:35:35.912 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.e 2024-07-04T08:35:36.250 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:35:36 smithi050 systemd[1]: Stopping Ceph mds.e for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:36.629 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:35:36 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e[68162]: 2024-07-04T08:35:36.265+0000 7f074c854640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:36.629 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:35:36 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e[68162]: 2024-07-04T08:35:36.265+0000 7f074c854640 -1 mds.e *** got signal Terminated *** 2024-07-04T08:35:39.378 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:35:39 smithi050 podman[76370]: 2024-07-04 08:35:39.097270232 +0000 UTC m=+2.948700071 container died 9ddac52d00b7eebb49a08361a0ef2d78dd2bde2944ca4bf1fbcdd5b351c309af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:35:39.808 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:35:39 smithi050 podman[76370]: 2024-07-04 08:35:39.573538743 +0000 UTC m=+3.424968579 container cleanup 9ddac52d00b7eebb49a08361a0ef2d78dd2bde2944ca4bf1fbcdd5b351c309af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b) 2024-07-04T08:35:39.809 INFO:journalctl@ceph.mds.e.smithi050.stdout:Jul 04 08:35:39 smithi050 bash[76370]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-e 2024-07-04T08:35:39.956 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.e.service' 2024-07-04T08:35:39.990 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:39.990 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-07-04T08:35:39.990 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-07-04T08:35:39.990 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.h 2024-07-04T08:35:40.353 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:35:40 smithi050 systemd[1]: Stopping Ceph mds.h for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:40.628 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:35:40 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h[68472]: 2024-07-04T08:35:40.368+0000 7f27c36c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:40.628 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:35:40 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h[68472]: 2024-07-04T08:35:40.368+0000 7f27c36c0640 -1 mds.h *** got signal Terminated *** 2024-07-04T08:35:43.378 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:35:43 smithi050 podman[76491]: 2024-07-04 08:35:43.111503656 +0000 UTC m=+2.881482028 container died dab9c6006cf41869ccc7e4c2e0ddf6c2b181a1e2f2796952e565fd27c68712df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-04T08:35:43.712 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:35:43 smithi050 podman[76491]: 2024-07-04 08:35:43.468310551 +0000 UTC m=+3.238288923 container cleanup dab9c6006cf41869ccc7e4c2e0ddf6c2b181a1e2f2796952e565fd27c68712df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240625, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:35:43.712 INFO:journalctl@ceph.mds.h.smithi050.stdout:Jul 04 08:35:43 smithi050 bash[76491]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-h 2024-07-04T08:35:43.880 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.h.service' 2024-07-04T08:35:43.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:43.914 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-07-04T08:35:43.914 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-07-04T08:35:43.914 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.k 2024-07-04T08:35:44.231 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:35:44 smithi050 systemd[1]: Stopping Ceph mds.k for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:44.628 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:35:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k[67846]: 2024-07-04T08:35:44.247+0000 7f5518bee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:44.628 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:35:44 smithi050 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k[67846]: 2024-07-04T08:35:44.247+0000 7f5518bee640 -1 mds.k *** got signal Terminated *** 2024-07-04T08:35:47.366 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:35:47 smithi050 podman[76612]: 2024-07-04 08:35:47.119599978 +0000 UTC m=+2.971843539 container died 9628c14682d4b98bdc5a0f648125f29d8142ef4c3494fd35c3291d8d49861c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b) 2024-07-04T08:35:47.628 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:35:47 smithi050 podman[76612]: 2024-07-04 08:35:47.472586654 +0000 UTC m=+3.324830207 container cleanup 9628c14682d4b98bdc5a0f648125f29d8142ef4c3494fd35c3291d8d49861c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-07-04T08:35:47.628 INFO:journalctl@ceph.mds.k.smithi050.stdout:Jul 04 08:35:47 smithi050 bash[76612]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-k 2024-07-04T08:35:47.895 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.k.service' 2024-07-04T08:35:47.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:47.927 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-07-04T08:35:47.927 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-07-04T08:35:47.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.c 2024-07-04T08:35:48.233 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:35:47 smithi133 systemd[1]: Stopping Ceph mds.c for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:48.550 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:35:48 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c[68662]: 2024-07-04T08:35:48.248+0000 7f2446476640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:48.550 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:35:48 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c[68662]: 2024-07-04T08:35:48.248+0000 7f2446476640 -1 mds.c *** got signal Terminated *** 2024-07-04T08:35:51.357 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:35:51 smithi133 podman[77079]: 2024-07-04 08:35:51.095546218 +0000 UTC m=+2.960590291 container died 1cc5e95b4f354cd6a4e7a30382e2478823e54ae67f0bd0c156513a9909635dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240625, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-07-04T08:35:51.689 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:35:51 smithi133 podman[77079]: 2024-07-04 08:35:51.487875014 +0000 UTC m=+3.352919054 container cleanup 1cc5e95b4f354cd6a4e7a30382e2478823e54ae67f0bd0c156513a9909635dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux ) 2024-07-04T08:35:51.689 INFO:journalctl@ceph.mds.c.smithi133.stdout:Jul 04 08:35:51 smithi133 bash[77079]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-c 2024-07-04T08:35:51.839 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.c.service' 2024-07-04T08:35:51.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:51.874 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-07-04T08:35:51.874 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-07-04T08:35:51.874 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.f 2024-07-04T08:35:52.300 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:35:51 smithi133 systemd[1]: Stopping Ceph mds.f for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:52.300 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:35:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f[68363]: 2024-07-04T08:35:52.207+0000 7f40d0c87640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:52.300 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:35:52 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f[68363]: 2024-07-04T08:35:52.207+0000 7f40d0c87640 -1 mds.f *** got signal Terminated *** 2024-07-04T08:35:55.359 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:35:55 smithi133 podman[77214]: 2024-07-04 08:35:55.095621444 +0000 UTC m=+2.982167734 container died 0422af43ab13d767baa4f15e16c5bc026467bcfccb7d0e52820e6740e28bcfc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-07-04T08:35:55.701 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:35:55 smithi133 podman[77214]: 2024-07-04 08:35:55.466142043 +0000 UTC m=+3.352688324 container cleanup 0422af43ab13d767baa4f15e16c5bc026467bcfccb7d0e52820e6740e28bcfc2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, maintainer=Guillaume Abrioux ) 2024-07-04T08:35:55.701 INFO:journalctl@ceph.mds.f.smithi133.stdout:Jul 04 08:35:55 smithi133 bash[77214]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-f 2024-07-04T08:35:55.870 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.f.service' 2024-07-04T08:35:55.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:55.902 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-07-04T08:35:55.902 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-07-04T08:35:55.902 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.i 2024-07-04T08:35:56.212 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:35:55 smithi133 systemd[1]: Stopping Ceph mds.i for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:35:56.550 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:35:56 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i[68974]: 2024-07-04T08:35:56.227+0000 7f3ced8a0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:35:56.550 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:35:56 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i[68974]: 2024-07-04T08:35:56.227+0000 7f3ced8a0640 -1 mds.i *** got signal Terminated *** 2024-07-04T08:35:59.370 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:35:59 smithi133 podman[77335]: 2024-07-04 08:35:59.09988129 +0000 UTC m=+2.969084592 container died 88c7fd526c4f164fe73202cdaa15c206a020ca7073c2ddc3a6c43814a4cca60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240625, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:35:59.712 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:35:59 smithi133 podman[77335]: 2024-07-04 08:35:59.469660975 +0000 UTC m=+3.338864275 container cleanup 88c7fd526c4f164fe73202cdaa15c206a020ca7073c2ddc3a6c43814a4cca60b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240625, org.label-schema.schema-version=1.0) 2024-07-04T08:35:59.713 INFO:journalctl@ceph.mds.i.smithi133.stdout:Jul 04 08:35:59 smithi133 bash[77335]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-i 2024-07-04T08:35:59.847 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.i.service' 2024-07-04T08:35:59.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:35:59.878 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-07-04T08:35:59.878 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-07-04T08:35:59.878 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.l 2024-07-04T08:36:00.050 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:35:59 smithi133 systemd[1]: Stopping Ceph mds.l for 2ee561ca-39db-11ef-bcab-c7b262605968... 2024-07-04T08:36:00.503 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:36:00 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l[68060]: 2024-07-04T08:36:00.214+0000 7f7542bfa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-04T08:36:00.503 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:36:00 smithi133 ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l[68060]: 2024-07-04T08:36:00.214+0000 7f7542bfa640 -1 mds.l *** got signal Terminated *** 2024-07-04T08:36:03.398 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:36:03 smithi133 podman[77456]: 2024-07-04 08:36:03.093417047 +0000 UTC m=+2.979922267 container died 4a82775ff921d0366e2c4ee6b6ad65bb749baf1ace8ca38ab69a703743f81022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-07-04T08:36:03.399 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:36:03 smithi133 podman[77456]: 2024-07-04 08:36:03.414526 +0000 UTC m=+3.301031218 container cleanup 4a82775ff921d0366e2c4ee6b6ad65bb749baf1ace8ca38ab69a703743f81022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, org.label-schema.build-date=20240625, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-04T08:36:03.756 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:36:03 smithi133 bash[77456]: ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l 2024-07-04T08:36:03.756 INFO:journalctl@ceph.mds.l.smithi133.stdout:Jul 04 08:36:03 smithi133 podman[77471]: 2024-07-04 08:36:03.565448061 +0000 UTC m=+0.465902944 container remove 4a82775ff921d0366e2c4ee6b6ad65bb749baf1ace8ca38ab69a703743f81022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4350f457c5f50f74b06d6c01de8a3d390042fbf9cf1bc0caaabed099f7393f35, name=ceph-2ee561ca-39db-11ef-bcab-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240703.143006-debug-2e9f68b, io.buildah.version=1.36.0, org.label-schema.build-date=20240625, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-07-04T08:36:03.780 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2ee561ca-39db-11ef-bcab-c7b262605968@mds.l.service' 2024-07-04T08:36:03.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-04T08:36:03.812 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-07-04T08:36:03.812 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 --force --keep-logs 2024-07-04T08:36:04.159 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:36:09.618 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 --force --keep-logs 2024-07-04T08:36:09.834 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:36:12.353 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 --force --keep-logs 2024-07-04T08:36:12.554 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:36:14.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-04T08:36:14.952 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-04T08:36:14.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-04T08:36:15.010 INFO:tasks.cephadm:Archiving crash dumps... 2024-07-04T08:36:15.012 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/remote/smithi039/crash 2024-07-04T08:36:15.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/crash -- . 2024-07-04T08:36:15.104 INFO:teuthology.orchestra.run.smithi039.stderr:tar: /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/crash: Cannot open: No such file or directory 2024-07-04T08:36:15.104 INFO:teuthology.orchestra.run.smithi039.stderr:tar: Error is not recoverable: exiting now 2024-07-04T08:36:15.108 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/remote/smithi050/crash 2024-07-04T08:36:15.109 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/crash -- . 2024-07-04T08:36:15.172 INFO:teuthology.orchestra.run.smithi050.stderr:tar: /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/crash: Cannot open: No such file or directory 2024-07-04T08:36:15.172 INFO:teuthology.orchestra.run.smithi050.stderr:tar: Error is not recoverable: exiting now 2024-07-04T08:36:15.174 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/remote/smithi133/crash 2024-07-04T08:36:15.174 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/crash -- . 2024-07-04T08:36:15.214 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/crash: Cannot open: No such file or directory 2024-07-04T08:36:15.214 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2024-07-04T08:36:15.215 INFO:tasks.cephadm:Checking cluster log for badness... 2024-07-04T08:36:15.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-04T08:36:15.329 INFO:teuthology.orchestra.run.smithi039.stdout:2024-07-04T08:10:00.000315+0000 mon.a (mon.0) 1113 : cluster [WRN] application not enabled on pool 'cephfs_data_ec' 2024-07-04T08:36:15.330 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-07-04T08:36:15.330 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[SEC\]' /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-04T08:36:15.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]' /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-04T08:36:15.507 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[WRN\]' /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-04T08:36:15.591 INFO:teuthology.orchestra.run.smithi039.stdout:2024-07-04T08:10:00.000315+0000 mon.a (mon.0) 1113 : cluster [WRN] application not enabled on pool 'cephfs_data_ec' 2024-07-04T08:36:15.593 INFO:tasks.cephadm:Compressing logs... 2024-07-04T08:36:15.593 DEBUG:teuthology.orchestra.run.smithi039:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-04T08:36:15.597 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-04T08:36:15.599 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-04T08:36:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log 2024-07-04T08:36:15.624 INFO:teuthology.orchestra.run.smithi133.stderr:: No such file or directory 2024-07-04T08:36:15.625 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.2.log 2024-07-04T08:36:15.625 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.f.log 2024-07-04T08:36:15.625 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log: /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.audit.log 2024-07-04T08:36:15.625 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.5.log 2024-07-04T08:36:15.627 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.audit.log: -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.c.log 2024-07-04T08:36:15.627 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mon.c.log 2024-07-04T08:36:15.627 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mgr.z.log 2024-07-04T08:36:15.627 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.i.log 2024-07-04T08:36:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mgr.z.log: 87.7% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log.gz 2024-07-04T08:36:15.631 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- 90.3% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.audit.log.gz 2024-07-04T08:36:15.632 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.cephadm.log 2024-07-04T08:36:15.632 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.11.log 2024-07-04T08:36:15.632 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.cephadm.log: 81.7% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.cephadm.log.gz 2024-07-04T08:36:15.633 INFO:teuthology.orchestra.run.smithi050.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr: -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.audit.log 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.7.log 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log: /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.b.log 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.4.log 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.1.log 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mgr.y.log 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.h.log 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mgr.y.log: 90.3% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.audit.log.gz 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.10.log 2024-07-04T08:36:15.634 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.h.log: 87.7% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log.gz 2024-07-04T08:36:15.635 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.cephadm.log 2024-07-04T08:36:15.635 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.e.log 2024-07-04T08:36:15.635 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.cephadm.log: 81.7% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.cephadm.log.gz 2024-07-04T08:36:15.636 INFO:teuthology.orchestra.run.smithi133.stderr: 93.4% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mgr.z.log.gz 2024-07-04T08:36:15.637 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.8.log 2024-07-04T08:36:15.637 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-volume.log 2024-07-04T08:36:15.639 INFO:teuthology.orchestra.run.smithi050.stderr: 93.3% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mgr.y.log.gz 2024-07-04T08:36:15.644 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-volume.log 2024-07-04T08:36:15.645 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mon.b.log 2024-07-04T08:36:15.650 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.11.log: /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.l.log 2024-07-04T08:36:15.651 INFO:teuthology.orchestra.run.smithi039.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-07-04T08:36:15.654 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.k.log 2024-07-04T08:36:15.655 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log 2024-07-04T08:36:15.656 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.audit.log 2024-07-04T08:36:15.656 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.g.log 2024-07-04T08:36:15.657 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.j.log 2024-07-04T08:36:15.657 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-04T08:36:15.658 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.3.log 2024-07-04T08:36:15.660 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-volume.log.gz 2024-07-04T08:36:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.0.log 2024-07-04T08:36:15.661 INFO:teuthology.orchestra.run.smithi039.stderr: 87.7% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.log.gz 2024-07-04T08:36:15.661 INFO:teuthology.orchestra.run.smithi039.stderr: 90.1% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.audit.log.gz 2024-07-04T08:36:15.661 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mgr.x.log 2024-07-04T08:36:15.662 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mon.a.log 2024-07-04T08:36:15.663 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-04T08:36:15.664 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.6.log 2024-07-04T08:36:15.665 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.cephadm.log 2024-07-04T08:36:15.667 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.6.log: /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.a.log 2024-07-04T08:36:15.669 INFO:teuthology.orchestra.run.smithi039.stderr: 94.9% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.j.log.gz 2024-07-04T08:36:15.670 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.a.log: 84.2% 94.9% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.g.log.gz 2024-07-04T08:36:15.670 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph.cephadm.log.gz 2024-07-04T08:36:15.670 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-volume.log 2024-07-04T08:36:15.672 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.d.log 2024-07-04T08:36:15.677 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-04T08:36:15.678 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-volume.log: /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.9.log 2024-07-04T08:36:15.678 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-04T08:36:15.679 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-04T08:36:15.679 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-volume.log.gz 2024-07-04T08:36:15.684 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-07-04T08:36:15.691 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.log: 90.9% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-volume.log.gz 2024-07-04T08:36:15.692 INFO:teuthology.orchestra.run.smithi039.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-07-04T08:36:15.707 INFO:teuthology.orchestra.run.smithi039.stderr: 95.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-04T08:36:15.736 INFO:teuthology.orchestra.run.smithi133.stderr: 92.7% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.c.log.gz 2024-07-04T08:36:15.748 INFO:teuthology.orchestra.run.smithi133.stderr: 92.7% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.i.log.gz 2024-07-04T08:36:15.751 INFO:teuthology.orchestra.run.smithi050.stderr: 92.7% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.h.log.gz 2024-07-04T08:36:15.779 INFO:teuthology.orchestra.run.smithi050.stderr: 94.6% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.e.log.gz 2024-07-04T08:36:15.792 INFO:teuthology.orchestra.run.smithi039.stderr: 92.8% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.a.log.gz 2024-07-04T08:36:15.795 INFO:teuthology.orchestra.run.smithi050.stderr: 94.5% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.k.log.gz 2024-07-04T08:36:15.826 INFO:teuthology.orchestra.run.smithi133.stderr: 94.5% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.l.log.gz 2024-07-04T08:36:15.833 INFO:teuthology.orchestra.run.smithi039.stderr: 94.4% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.d.log.gz 2024-07-04T08:36:15.935 INFO:teuthology.orchestra.run.smithi133.stderr: 93.5% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.f.log.gz 2024-07-04T08:36:16.091 INFO:teuthology.orchestra.run.smithi039.stderr: 90.2% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mgr.x.log.gz 2024-07-04T08:36:16.256 INFO:teuthology.orchestra.run.smithi133.stderr: 91.5% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mon.c.log.gz 2024-07-04T08:36:16.536 INFO:teuthology.orchestra.run.smithi050.stderr: 91.6% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mon.b.log.gz 2024-07-04T08:36:17.051 INFO:teuthology.orchestra.run.smithi050.stderr: 92.4% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.4.log.gz 2024-07-04T08:36:17.058 INFO:teuthology.orchestra.run.smithi039.stderr: 92.6% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.0.log.gz 2024-07-04T08:36:17.182 INFO:teuthology.orchestra.run.smithi039.stderr: 92.1% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.6.log.gz 2024-07-04T08:36:17.389 INFO:teuthology.orchestra.run.smithi050.stderr: 92.0% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.1.log.gz 2024-07-04T08:36:17.466 INFO:teuthology.orchestra.run.smithi039.stderr: 90.2% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mon.a.log.gz 2024-07-04T08:36:17.623 INFO:teuthology.orchestra.run.smithi133.stderr: 92.5% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.11.log.gz 2024-07-04T08:36:17.693 INFO:teuthology.orchestra.run.smithi133.stderr: 92.4% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.8.log.gz 2024-07-04T08:36:17.962 INFO:teuthology.orchestra.run.smithi039.stderr: 92.4% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.9.log.gz 2024-07-04T08:36:18.135 INFO:teuthology.orchestra.run.smithi050.stderr: 92.1% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.7.log.gz 2024-07-04T08:36:18.135 INFO:teuthology.orchestra.run.smithi050.stderr: 92.0% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.10.log.gz 2024-07-04T08:36:18.187 INFO:teuthology.orchestra.run.smithi039.stderr: 92.7% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.3.log.gz 2024-07-04T08:36:18.190 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-07-04T08:36:18.191 INFO:teuthology.orchestra.run.smithi039.stderr:real 0m2.561s 2024-07-04T08:36:18.191 INFO:teuthology.orchestra.run.smithi039.stderr:user 0m9.559s 2024-07-04T08:36:18.191 INFO:teuthology.orchestra.run.smithi039.stderr:sys 0m0.792s 2024-07-04T08:36:18.202 INFO:teuthology.orchestra.run.smithi050.stderr: 92.9% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-mds.b.log.gz 2024-07-04T08:36:18.204 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-07-04T08:36:18.204 INFO:teuthology.orchestra.run.smithi050.stderr:real 0m2.589s 2024-07-04T08:36:18.204 INFO:teuthology.orchestra.run.smithi050.stderr:user 0m11.596s 2024-07-04T08:36:18.204 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m0.385s 2024-07-04T08:36:18.376 INFO:teuthology.orchestra.run.smithi133.stderr: 92.1% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.2.log.gz 2024-07-04T08:36:18.400 INFO:teuthology.orchestra.run.smithi133.stderr: 92.3% -- replaced with /var/log/ceph/2ee561ca-39db-11ef-bcab-c7b262605968/ceph-osd.5.log.gz 2024-07-04T08:36:18.403 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-07-04T08:36:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m2.788s 2024-07-04T08:36:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:user 0m10.499s 2024-07-04T08:36:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m0.353s 2024-07-04T08:36:18.403 INFO:tasks.cephadm:Archiving logs... 2024-07-04T08:36:18.403 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/remote/smithi039/log 2024-07-04T08:36:18.404 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-04T08:36:19.823 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/remote/smithi050/log 2024-07-04T08:36:19.824 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-04T08:36:21.483 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/remote/smithi133/log 2024-07-04T08:36:21.484 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-04T08:36:23.166 INFO:tasks.cephadm:Removing cluster... 2024-07-04T08:36:23.167 DEBUG:teuthology.orchestra.run.smithi039:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 --force 2024-07-04T08:36:23.519 INFO:teuthology.orchestra.run.smithi039.stdout:Deleting cluster with fsid: 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:36:24.147 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 --force 2024-07-04T08:36:24.363 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:36:24.638 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2ee561ca-39db-11ef-bcab-c7b262605968 --force 2024-07-04T08:36:24.842 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: 2ee561ca-39db-11ef-bcab-c7b262605968 2024-07-04T08:36:25.113 INFO:tasks.cephadm:Removing cephadm ... 2024-07-04T08:36:25.113 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-04T08:36:25.173 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-04T08:36:25.191 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-04T08:36:25.206 INFO:tasks.cephadm:Teardown complete 2024-07-04T08:36:25.206 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-07-04T08:36:25.219 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-07-04T08:36:25.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-04T08:36:25.227 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-04T08:36:25.233 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-04T08:36:25.384 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-07-04T08:36:25.395 INFO:teuthology.task.clock:Checking final clock skew... 2024-07-04T08:36:25.396 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-04T08:36:25.403 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-04T08:36:25.406 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-04T08:36:25.419 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-07-04T08:36:25.420 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-07-04T08:36:25.424 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-04T08:36:25.424 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-07-04T08:36:25.424 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +1071us[+1052us] +/- 57ms 2024-07-04T08:36:25.424 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 81 -1148us[-1167us] +/- 76ms 2024-07-04T08:36:25.424 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -56us[ -56us] +/- 79ms 2024-07-04T08:36:25.424 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T08:36:25.425 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-04T08:36:25.425 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-07-04T08:36:25.425 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +970us[ +970us] +/- 57ms 2024-07-04T08:36:25.425 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 19 -1265us[-1265us] +/- 76ms 2024-07-04T08:36:25.425 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -152us[ -152us] +/- 79ms 2024-07-04T08:36:25.425 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T08:36:25.454 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-07-04T08:36:25.464 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-04T08:36:25.464 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-07-04T08:36:25.465 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 +983us[ +986us] +/- 57ms 2024-07-04T08:36:25.465 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 -1286us[-1282us] +/- 76ms 2024-07-04T08:36:25.465 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -164us[ -160us] +/- 79ms 2024-07-04T08:36:25.465 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-04T08:36:25.466 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-07-04T08:36:25.477 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-07-04T08:36:25.478 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-07-04T08:36:25.490 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-04T08:36:25.729 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-04T08:36:25.804 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-04T08:36:25.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-04T08:36:25.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T08:36:25.974 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-04T08:36:26.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T08:36:26.005 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-04T08:36:26.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T08:36:26.033 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-07-04T08:36:26.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-07-04T08:36:26.056 INFO:teuthology.task.internal:Duration was 2805.357530 seconds 2024-07-04T08:36:26.057 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-07-04T08:36:26.074 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-07-04T08:36:26.075 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-04T08:36:26.083 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-04T08:36:26.086 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-04T08:36:26.122 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-04T08:36:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-04T08:36:26.215 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-04T08:36:26.588 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-07-04T08:36:26.588 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-07-04T08:36:26.588 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-04T08:36:26.653 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-07-04T08:36:26.654 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-04T08:36:26.683 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-07-04T08:36:26.683 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-04T08:36:26.706 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-07-04T08:36:26.706 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-04T08:36:26.710 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-04T08:36:26.726 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-04T08:36:26.772 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-07-04T08:36:26.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-04T08:36:26.777 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-04T08:36:26.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-04T08:36:28.769 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-07-04T08:36:28.788 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-07-04T08:36:28.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-04T08:36:28.874 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-04T08:36:28.904 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-04T08:36:28.932 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-07-04T08:36:28.947 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-04T08:36:28.956 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-04T08:36:28.958 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-04T08:36:28.983 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-07-04T08:36:28.997 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-07-04T08:36:29.046 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-07-04T08:36:29.092 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-04T08:36:29.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T08:36:29.143 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-04T08:36:29.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T08:36:29.161 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-04T08:36:29.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-04T08:36:29.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-07-04T08:36:29.191 INFO:teuthology.task.internal:Transferring archived files... 2024-07-04T08:36:29.192 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/remote/smithi039 2024-07-04T08:36:29.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-04T08:36:29.340 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/remote/smithi050 2024-07-04T08:36:29.341 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-04T08:36:29.387 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-07-04_01:31:57-fs-wip-pdonnell-testing-20240703.143006-debug-distro-default-smithi/7786821/remote/smithi133 2024-07-04T08:36:29.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-04T08:36:29.426 INFO:teuthology.task.internal:Removing archive directory... 2024-07-04T08:36:29.426 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-04T08:36:29.434 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-04T08:36:29.436 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-04T08:36:29.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-07-04T08:36:29.498 INFO:teuthology.task.internal:Not uploading archives. 2024-07-04T08:36:29.499 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-07-04T08:36:29.582 INFO:teuthology.task.internal:Tidying up after the test... 2024-07-04T08:36:29.583 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-04T08:36:29.590 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-04T08:36:29.593 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-04T08:36:29.608 INFO:teuthology.orchestra.run.smithi133.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 4 08:36 /home/ubuntu/cephtest 2024-07-04T08:36:29.609 INFO:teuthology.orchestra.run.smithi050.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 4 08:36 /home/ubuntu/cephtest 2024-07-04T08:36:29.641 INFO:teuthology.orchestra.run.smithi039.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 4 08:36 /home/ubuntu/cephtest 2024-07-04T08:36:29.644 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-07-04T08:36:29.657 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-07-04T08:36:29.686 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 0d32a7bba447dd2470f4de43560317375ca9e9c8 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2805.3575296401978 failure_reason: '"2024-07-04T08:10:00.000315+0000 mon.a (mon.0) 1113 : cluster [WRN] application not enabled on pool ''cephfs_data_ec''" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-07-04T08:36:29.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-04T08:36:29.773 INFO:teuthology.run:FAIL